Patchwork [powerpc] Rework VSX scalar floating point support

login
register
mail settings
Submitter Michael Meissner
Date Aug. 22, 2013, 6:56 p.m.
Message ID <20130822185658.GA30430@ibm-tiger.the-meissners.org>
Download mbox | patch
Permalink /patch/269139/
State New
Headers show

Comments

Michael Meissner - Aug. 22, 2013, 6:56 p.m.
I'm working on adding the secondary reload support in the PowerPC so that we
can use the upper registers for scalar floating point support.  For those of
you who do not know the layout of the PowerPC floating point unit, there are 2
sets of registers (traditional floating point scalar registers and altivec
vector registers).  When ISA 2.06 (power7) came out, it added new instructions
(VSX) that could use the combined register set for either scalar double
precision or vector logical/floating point operations.  In ISA 2.07 (power8),
instructions were added so that scalar single precision support could also be
done on the upper registers.

However, to load data in the upper registers that overlay the Altivec register
set, we can only use register + register addressing, while loading up scalar
floating point values in the traditional floating point register set can use
auto-update and offset addressing modes.

These patches reverse a decision that I made back in the initial ISA 2.06 time
frame, where if you did -mvsx, it only used the VSX form of the arithmetic
instruction, even though scalar values were resticted to using the traditional
floating point registers.  The patches now combine both SFmode and DFmode
expanders and insns with mode iterators.  If all of the registers used are in
the traditional floating point register set, it uses the traditional floating
point instruction (i.e. fadd instead of xsadddp).  If any of the registers come
from the upper register set, it will use the ISA 2.06/2.07 VSX instructions.

I have bootstraped the compiler at subversion id 201798 with these patches, and
ran make check with no regressions.  In addition, I've been running the SpecFP
2006 benchmark suite comparing the results of runs before the change was made
and with the changes for a power7 target, and I don't see any significant
changes in runtime behavior.  I am including patches for the tests that need to
be adjusted with these changes.

This patch adds a few new constraints.  For floating point work, the intention
is that the constraints will be used as follows:

    f	traditional SFmode insns (i.e. fadds)
    d	traditional DFmode insns (i.e. fadd)
    wy	VSX SFmode insns (i.e. xsaddsp), could be FLOAT_REGS or VSX_REGS
    ws	VSX DFmode insns (i.e. xsadddp), could be FLOAT_REGS or VSX_REGS
    wu	SFmode load to or store  from Altivec regs (i.e. lxsspx)
    wv	DFmode load to or store  from Altivec regs (i.e. lxsdx)
    ww	VSX instructions used in converting to/from SFmode

Note, that the wv constraint added in previous power8 changes went from being
SFmode to DFmode (nothing used wv in the current patches that are committed).

The patch adds 3 debug switches (-mvsx-scalar-float, -mupper-regs-sf, and
-mupper-regs-df) that I'm using to debug the reload stuff.  It is anticipated
that -mvsx will imply -mupper-regs-df, and -mpower8-vector will imply
-mupper-regs-sf when the reload patches are done.

At the moment, the current trunk (subversion id 201924) does not bootstrap on
the powerpc, and I will be away from the computer starting on August 24th.  I
won't be getting back until September 3rd, and I don't anticipate checking my
mail until I get back.  Are these patches ok to check in?  I can either check
them in now, or I can delay checking them in until we've fixed the boostrap
bug.  Ideally, I would like to get permission to check these in on September
3rd now, but I can resubmit them on the 3rd if desired.

[gcc]
2013-08-22  Michael Meissner  <meissner@linux.vnet.ibm.com>

	* config/rs6000/constraints.md (wa constraint): Add documentation
	to all w* constraints.  Make the documentation agree with
	md.texi.  Sort the w* constraints to be in alphabetical order.
	Add wu, wv, ww, and wy constraints for supporting using the upper
	registers for DFmode under power7 and SFmode under power8.
	(wd constraint): Likewise.
	(wf constraint): Likewise.
	(wg constraint): Likewise.
	(wl constraint): Likewise.
	(wm constraint): Likewise.
	(wn constraint): Likewise.
	(wr constraint): Likewise.
	(ws constraint): Likewise.
	(wt constraint): Likewise.
	(wu constraint): Likewise.
	(wv constraint): Likewise.
	(ww constraint): Likewise.
	(wx constraint): Likewise.
	(wy constraint): Likewise.
	(wz constraint): Likewise.
	* doc/md.texi (PowerPC and IBM RS6000): Likewise.

	* config/rs6000/rs6000-builtin.def (xsrdpim): Use floor, ceil,
	btrunc insns, instead of vsx_<name>.

	* config/rs6000/rs6000.opt (-mvsx-scalar-float): New debug swtich
	to allow/disallow single precision VSX scalar instructions.
	(-mvsx-double-float): Change initial value to 1 from -1.
	(-mvsx-scalar-memory): Make this an alias of -mupper-regs-df.
	(-mupper-regs-df): New debug switches to control whether DFmode
	and SFmode can use the upper registers on power7/power8
	respectively.
	(-mupper-regs-sf): Likewise.

	* config/rs6000/rs6000.c (rs6000_hard_regno_mode_ok): Add support
	for -mupper-regs-sf and -mupper-regs-df.
	(rs6000_init_hard_regno_mode_ok): Likewise.
	(rs6000_opt_masks): Likewise.
	(rs6000_debug_reg_global): Print wu, ww, and wy constraints.
	Print which type of floating point unit and registers are
	available for DFmode/SFmode.

	* config/rs6000/vsx.md (vsx_add<mode>3): Move all scalar DF VSX
	support to rs6000.md.  Make SF/DFmode insns common where
	possible.  Add support for power8 scalar float instructions using
	the upper registers.  Don't use VSv and VStype_simple mode
	attributes on the insns that only handle vectors after moving
	scalar support to rs6000.md.  Merge some expanders into the
	define_insn if there is only one option.
	(vsx_sub<mode>3): Likewise.
	(vsx_mul<mode>3): Likewise.
	(vsx_div<mode>3): Likewise.
	(vsx_fre<mode>2): Likewise.
	(vsx_neg<mode>2): Likewise.
	(vsx_abs<mode>2): Likewise.
	(vsx_nabs<mode>2): Likewise.
	(vsx_smax<mode>3): Likewise.
	(vsx_smaxsf3): Likewise.
	(vsx_smin<mode>3): Likewise.
	(vsx_sminsf3): Likewise.
	(vsx_sqrt<mode>2): Likewise.
	(vsx_rsqrte<mode>2): Likewise.
	(vsx_fmadf4): Likewise.
	(vsx_fmsdf4): Likewise.
	(vsx_fms<mode>4): Likewise.
	(vsx_nfmadf4): Likewise.
	(vsx_nfma<mode>4): Likewise.
	(vsx_nfmadf4): Likewise.
	(vsx_cmpdf_internal1): Likewise.
	(vsx_copysign<mode>3): Likewise.
	(vsx_btrunc<mode>2): Likewise.
	(vsx_floor<mode>2): Likewise.
	(vsx_ceil<mode>2): Likewise.
	* config/rs6000/rs6000.md (Ftrad): Likewise.
	(Fvsx): Likewise.
	(Ff): Likewise.
	(Fv): Likewise.
	(Fs): Likewise.
	(Ffre): Likewise.
	(FFRE): Likewise.
	(abs<mode>2): Likewise.
	(abs<mode>2_fpr): Likewise.
	(nabs<mode>2_fpr): Likewise.
	(neg<mode>2): Likewise.
	(neg<mode>2_fpr): Likewise.
	(smax<mode>3): Likewise.
	(smax<mode>3_vsx): Likewise.
	(smin<mode>3): Likewise.
	(smin<mode>3_fpr): Likewise.
	(smin/smax peephole): Likewise.
	(add<mode>3): Likewise.
	(add<mode>3_fpr): Likewise.
	(sub<mode>3): Likewise.
	(sub<mode>3_fpr): Likewise.
	(mul<mode>3): Likewise.
	(mul<mode>3_fpr): Likewise.
	(div<mode>3): Likewise.
	(div<mode>3_fpr): Likewise.
	(fre<Fs>): Likewise.
	(sqrt<mode>2): Likewise.
	(rsqrt<mode>2): Likewise.
	(cmp<mode>_fpr): Likewise.
	(negsf2): Likewise.
	(abssf2): Likewise.
	(negative abssf2 unnamed pattern): Likewise.
	(addsf3): Likewise.
	(subsf3): Likewise.
	(mulsf3): Likewise.
	(divsf3): Likewise.
	(fres): Likewise.
	(fmasf4_fpr): Likewise.
	(fmssf4_fpr): Likewise.
	(nfmasf4_fpr): Likewise.
	(nfmssf4_fpr): Likewise.
	(sqrtsf2): Likewise.
	(rsqrtsf_internal1): Likewise.
	(copysign<mode>3_fcpsgn): Likewise.
	(smaxsf3): Likewise.
	(sminsf3): Likewise.
	(sminsf3/smaxsf3 splitter): Likewise.
	(negdf2): Likewise.
	(negdf2_fpr): Likewise.
	(absdf2): Likewise.
	(absdf2_fpr): Likewise.
	(nabsdf2_fpr): Likewise.
	(adddf3): Likewise.
	(adddf3_fpr): Likewise.
	(subdf3): Likewise.
	(subdf3_fpr): Likewise.
	(muldf3): Likewise.
	(muldf3_fpr): Likewise.
	(divdf3): Likewise.
	(divdf3_fpr): Likewise.
	(fred_fpr): Likewise.
	(rsqrtdf_internal1): Likewise.
	(fmadf4_fpr): Likewise.
	(fmsdf4_fpr): Likewise.
	(nfmadf4_fpr): Likewise.
	(nfmsdf4_fpr): Likewise.
	(sqrtdf2): Likewise.
	(sqrtdf2_fpr): Likewise.
	(smaxdf3): Likewise.
	(smindf3): Likewise.
	(smaxdf3, smindf3 splitter): Likewise.
	(lrint<mode>di2): Likewise.
	(btrunc<mode>2): Likewise.
	(btranc<mode>2_fpr): Likewise.
	(ceil<mode>2): Likewise.
	(ceil<mode>2_fpr): Likewise.
	(floor<mode>2): Likewise.
	(floor<mode>2_fpr): Likewise.
	(cmpsf_internal1): Likewise.
	(cmpdf_internal1): Likewise.
	(fma<mode>4_fpr): Likewise.
	(fms<mode>4_fpr): Likewise.
	(nfma<mode>4): Likewise.
	(nfma<mode>4_fpr): Likewise.
	(fnms<mode>4): Likewise.
	(nfmssf4_fpr): Likewise.
	(zero_extendsidi2_lfiwzx): Restrict using VSX load/stores to
	Altivec registers.  Add support for power8 32-bit VSX memory
	operations.
	(extendsidi2_lfiwax): Likewise.
	(lfiwax): Likewise.
	(floatsi<mode>2_lfiwax_mem): Likewise.
	(lfiwzx): Likewise.
	(floatunssi<mode>2_lfiwzx_mem): Likewise.
	(mov<mode>_hardfloat, SFmode/SDmode): Likewise.
	(mov<mode>_hardfloat32, DFmode/DDmode): Likewise.
	(mov<mode>_hardfloat64, DFmode/DDmode): Likewise.
	(movdi_internal64): Likewise.
	(mov<mode>cc): Merge SF/DF conditional move patterns.
	(fsel<mode>sf4): Likewise.
	(fsel<mode>df4): Likewise.
	(fselsfsf4): Likewise.
	(fselsfdf4): Likewise.
	(fseldfsf4): Likewise.
	(fseldfdf4): Likewise.

	* config/rs6000/rs6000.h (TARGET_SF_SPE): Define new macros to
	simplify testing what kind of SFmode/DFmode floating point unit we
	have.
	(TARGET_DF_SPE): Likewise.
	(TARGET_SF_FPR): Likewise.
	(TARGET_DF_FPR): Likewise.
	(TARGET_SF_INSN): Likewise.
	(TARGET_DF_INSN): Likewise.
	(res6000_reg_class_enum): Add wu, ww, and wy constraints.  Sort
	elements.

[gcc/testsuite]
2013-08-21  Michael Meissner  <meissner@linux.vnet.ibm.com>

	* gcc.target/powerpc/recip-3.c: Update VSX tests to allow
	generation of traditional floating point ops.
	* gcc.target/powerpc/recip-5.c: Likewise.
	* gcc.target/powerpc/ppc-target-1.c: Likewise.
	* gcc.target/powerpc/ppc-target-2.c: Likewise.
	* gcc.target/powerpc/pr42747.c: Likewise.
	* gcc.target/powerpc/vsx-builtin-3.c: Likewise.
David Edelsohn - Sept. 6, 2013, 3:39 a.m.
Mike,

Why does rs6000_init_hard_regno_mode_ok() include #if 0 code?

Because the patch affects e500, has this patch been tested in an e500
configuration?

The cover message did not make it clear that you are merging SFmode
and DFmode patterns using mode iterators. That really should have been
a separate patch. We'll proceed with the current form.  But, would you
please clarify the ChangeLog entry for rs6000.md to make it clear
exactly which patterns are being deleted instead of listing everything
as "Likewise"?

Thanks, David

Patch

Index: gcc/config/rs6000/constraints.md
===================================================================
--- gcc/config/rs6000/constraints.md	(revision 201924)
+++ gcc/config/rs6000/constraints.md	(working copy)
@@ -52,29 +52,18 @@  (define_register_constraint "z" "CA_REGS
   "@internal")
 
 ;; Use w as a prefix to add VSX modes
-;; vector double (V2DF)
+;; any VSX register
+(define_register_constraint "wa" "rs6000_constraints[RS6000_CONSTRAINT_wa]"
+  "Any VSX register if the -mvsx option was used or NO_REGS.")
+
 (define_register_constraint "wd" "rs6000_constraints[RS6000_CONSTRAINT_wd]"
-  "@internal")
+  "VSX vector register to hold vector double data or NO_REGS.")
 
-;; vector float (V4SF)
 (define_register_constraint "wf" "rs6000_constraints[RS6000_CONSTRAINT_wf]"
-  "@internal")
-
-;; scalar double (DF)
-(define_register_constraint "ws" "rs6000_constraints[RS6000_CONSTRAINT_ws]"
-  "@internal")
-
-;; TImode in VSX registers
-(define_register_constraint "wt" "rs6000_constraints[RS6000_CONSTRAINT_wt]"
-  "@internal")
-
-;; any VSX register
-(define_register_constraint "wa" "rs6000_constraints[RS6000_CONSTRAINT_wa]"
-  "@internal")
+  "VSX vector register to hold vector float data or NO_REGS.")
 
-;; Register constraints to simplify move patterns
 (define_register_constraint "wg" "rs6000_constraints[RS6000_CONSTRAINT_wg]"
-  "Floating point register if -mmfpgpr is used, or NO_REGS.")
+  "If -mmfpgpr was used, a floating point register or NO_REGS.")
 
 (define_register_constraint "wl" "rs6000_constraints[RS6000_CONSTRAINT_wl]"
   "Floating point register if the LFIWAX instruction is enabled or NO_REGS.")
@@ -82,23 +71,38 @@  (define_register_constraint "wl" "rs6000
 (define_register_constraint "wm" "rs6000_constraints[RS6000_CONSTRAINT_wm]"
   "VSX register if direct move instructions are enabled, or NO_REGS.")
 
+;; NO_REGs register constraint, used to merge mov{sd,sf}, since movsd can use
+;; direct move directly, and movsf can't to move between the register sets.
+;; There is a mode_attr that resolves to wm for SDmode and wn for SFmode
+(define_register_constraint "wn" "NO_REGS" "No register (NO_REGS).")
+
 (define_register_constraint "wr" "rs6000_constraints[RS6000_CONSTRAINT_wr]"
   "General purpose register if 64-bit instructions are enabled or NO_REGS.")
 
+(define_register_constraint "ws" "rs6000_constraints[RS6000_CONSTRAINT_ws]"
+  "VSX vector register to hold scalar double values or NO_REGS.")
+
+(define_register_constraint "wt" "rs6000_constraints[RS6000_CONSTRAINT_wt]"
+  "VSX vector register to hold 128 bit integer or NO_REGS.")
+
+(define_register_constraint "wu" "rs6000_constraints[RS6000_CONSTRAINT_wu]"
+  "Altivec register to use for float/32-bit int loads/stores  or NO_REGS.")
+
 (define_register_constraint "wv" "rs6000_constraints[RS6000_CONSTRAINT_wv]"
-  "Altivec register if -mpower8-vector is used or NO_REGS.")
+  "Altivec register to use for double loads/stores  or NO_REGS.")
+
+(define_register_constraint "ww" "rs6000_constraints[RS6000_CONSTRAINT_ww]"
+  "FP or VSX register to perform float operations under -mvsx or NO_REGS.")
 
 (define_register_constraint "wx" "rs6000_constraints[RS6000_CONSTRAINT_wx]"
   "Floating point register if the STFIWX instruction is enabled or NO_REGS.")
 
+(define_register_constraint "wy" "rs6000_constraints[RS6000_CONSTRAINT_wy]"
+  "VSX vector register to hold scalar float values or NO_REGS.")
+
 (define_register_constraint "wz" "rs6000_constraints[RS6000_CONSTRAINT_wz]"
   "Floating point register if the LFIWZX instruction is enabled or NO_REGS.")
 
-;; NO_REGs register constraint, used to merge mov{sd,sf}, since movsd can use
-;; direct move directly, and movsf can't to move between the register sets.
-;; There is a mode_attr that resolves to wm for SDmode and wn for SFmode
-(define_register_constraint "wn" "NO_REGS")
-
 ;; Lq/stq validates the address for load/store quad
 (define_memory_constraint "wQ"
   "Memory operand suitable for the load/store quad instructions"
Index: gcc/config/rs6000/rs6000-builtin.def
===================================================================
--- gcc/config/rs6000/rs6000-builtin.def	(revision 201924)
+++ gcc/config/rs6000/rs6000-builtin.def	(working copy)
@@ -1209,9 +1209,9 @@  BU_VSX_1 (XVRSPIZ,	      "xvrspiz",	CONS
 
 BU_VSX_1 (XSRDPI,	      "xsrdpi",		CONST,	vsx_xsrdpi)
 BU_VSX_1 (XSRDPIC,	      "xsrdpic",	CONST,	vsx_xsrdpic)
-BU_VSX_1 (XSRDPIM,	      "xsrdpim",	CONST,	vsx_floordf2)
-BU_VSX_1 (XSRDPIP,	      "xsrdpip",	CONST,	vsx_ceildf2)
-BU_VSX_1 (XSRDPIZ,	      "xsrdpiz",	CONST,	vsx_btruncdf2)
+BU_VSX_1 (XSRDPIM,	      "xsrdpim",	CONST,	floordf2)
+BU_VSX_1 (XSRDPIP,	      "xsrdpip",	CONST,	ceildf2)
+BU_VSX_1 (XSRDPIZ,	      "xsrdpiz",	CONST,	btruncdf2)
 
 /* VSX predicate functions.  */
 BU_VSX_P (XVCMPEQSP_P,	      "xvcmpeqsp_p",	CONST,	vector_eq_v4sf_p)
Index: gcc/config/rs6000/rs6000.opt
===================================================================
--- gcc/config/rs6000/rs6000.opt	(revision 201924)
+++ gcc/config/rs6000/rs6000.opt	(working copy)
@@ -181,13 +181,16 @@  mvsx
 Target Report Mask(VSX) Var(rs6000_isa_flags)
 Use vector/scalar (VSX) instructions
 
+mvsx-scalar-float
+Target Undocumented Report Var(TARGET_VSX_SCALAR_FLOAT) Init(1)
+; If -mpower8-vector, use VSX arithmetic instructions for SFmode (on by default)
+
 mvsx-scalar-double
-Target Undocumented Report Var(TARGET_VSX_SCALAR_DOUBLE) Init(-1)
-; If -mvsx, use VSX arithmetic instructions for scalar double (on by default)
+Target Undocumented Report Var(TARGET_VSX_SCALAR_DOUBLE) Init(1)
+; If -mvsx, use VSX arithmetic instructions for DFmode (on by default)
 
 mvsx-scalar-memory
-Target Undocumented Report Var(TARGET_VSX_SCALAR_MEMORY)
-; If -mvsx, use VSX scalar memory reference instructions for scalar double (off by default)
+Target Undocumented Report Alias(mupper-regs-df)
 
 mvsx-align-128
 Target Undocumented Report Var(TARGET_VSX_ALIGN_128)
@@ -550,3 +553,11 @@  Generate the quad word memory instructio
 mcompat-align-parm
 Target Report Var(rs6000_compat_align_parm) Init(0) Save
 Generate aggregate parameter passing code with at most 64-bit alignment.
+
+mupper-regs-df
+Target Undocumented Mask(UPPER_REGS_DF) Var(rs6000_isa_flags)
+Allow double variables in upper registers with -mcpu=power7 or -mvsx
+
+mupper-regs-sf
+Target Undocumented Mask(UPPER_REGS_SF) Var(rs6000_isa_flags)
+Allow float variables in upper registers with -mcpu=power8 or -mp8-vector
Index: gcc/config/rs6000/rs6000.c
===================================================================
--- gcc/config/rs6000/rs6000.c	(revision 201924)
+++ gcc/config/rs6000/rs6000.c	(working copy)
@@ -1625,18 +1625,26 @@  rs6000_hard_regno_mode_ok (int regno, en
   /* VSX registers that overlap the FPR registers are larger than for non-VSX
      implementations.  Don't allow an item to be split between a FP register
      and an Altivec register.  */
-  if (VECTOR_MEM_VSX_P (mode))
+  if (VECTOR_UNIT_VSX_OR_P8_VECTOR_P (mode)
+      || VECTOR_MEM_VSX_OR_P8_VECTOR_P (mode))
     {
       if (FP_REGNO_P (regno))
 	return FP_REGNO_P (last_regno);
 
       if (ALTIVEC_REGNO_P (regno))
-	return ALTIVEC_REGNO_P (last_regno);
-    }
+	{
+	  if (!ALTIVEC_REGNO_P (last_regno))
+	    return false;
 
-  /* Allow TImode in all VSX registers if the user asked for it.  */
-  if (mode == TImode && TARGET_VSX_TIMODE && VSX_REGNO_P (regno))
-    return 1;
+	  if (mode == DFmode && !TARGET_UPPER_REGS_DF)
+	    return false;
+
+	  if (mode == SFmode && !TARGET_UPPER_REGS_SF)
+	    return false;
+
+	  return true;
+	}
+    }
 
   /* The GPRs can hold any mode, but values bigger than one register
      cannot go past R31.  */
@@ -1891,8 +1899,11 @@  rs6000_debug_reg_global (void)
 	   "wr reg_class = %s\n"
 	   "ws reg_class = %s\n"
 	   "wt reg_class = %s\n"
+	   "wu reg_class = %s\n"
 	   "wv reg_class = %s\n"
+	   "ww reg_class = %s\n"
 	   "wx reg_class = %s\n"
+	   "wy reg_class = %s\n"
 	   "wz reg_class = %s\n"
 	   "\n",
 	   reg_class_names[rs6000_constraints[RS6000_CONSTRAINT_d]],
@@ -1907,8 +1918,11 @@  rs6000_debug_reg_global (void)
 	   reg_class_names[rs6000_constraints[RS6000_CONSTRAINT_wr]],
 	   reg_class_names[rs6000_constraints[RS6000_CONSTRAINT_ws]],
 	   reg_class_names[rs6000_constraints[RS6000_CONSTRAINT_wt]],
+	   reg_class_names[rs6000_constraints[RS6000_CONSTRAINT_wu]],
 	   reg_class_names[rs6000_constraints[RS6000_CONSTRAINT_wv]],
+	   reg_class_names[rs6000_constraints[RS6000_CONSTRAINT_ww]],
 	   reg_class_names[rs6000_constraints[RS6000_CONSTRAINT_wx]],
+	   reg_class_names[rs6000_constraints[RS6000_CONSTRAINT_wy]],
 	   reg_class_names[rs6000_constraints[RS6000_CONSTRAINT_wz]]);
 
   for (m = 0; m < NUM_MACHINE_MODES; ++m)
@@ -2135,6 +2149,24 @@  rs6000_debug_reg_global (void)
   if (rs6000_float_gprs)
     fprintf (stderr, DEBUG_FMT_S, "float_gprs", "true");
 
+  if (VECTOR_UNIT_VSX_P (DFmode))
+    fprintf (stderr, DEBUG_FMT_S, "vsx double",
+	     (TARGET_UPPER_REGS_DF) ? "fpr + altivec" : "fpr");
+  else if (TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT)
+    fprintf (stderr, DEBUG_FMT_S, "trad. double", "fpr");
+  else if (TARGET_HARD_FLOAT && TARGET_E500_DOUBLE)
+    fprintf (stderr, DEBUG_FMT_S, "E500 double", "gpr");
+  else
+    fprintf (stderr, DEBUG_FMT_S, "software double", "gpr");
+
+  if (VECTOR_UNIT_P8_VECTOR_P (SFmode))
+    fprintf (stderr, DEBUG_FMT_S, "p8 float",
+	     (TARGET_UPPER_REGS_SF) ? "fpr + altivec" : "fpr");
+  else if (TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT)
+    fprintf (stderr, DEBUG_FMT_S, "trad. float", "fpr");
+  else
+    fprintf (stderr, DEBUG_FMT_S, "software float", "gpr");
+
   if (TARGET_LINK_STACK)
     fprintf (stderr, DEBUG_FMT_S, "link_stack", "true");
 
@@ -2315,12 +2347,21 @@  rs6000_init_hard_regno_mode_ok (bool glo
       rs6000_vector_align[V2DImode] = align64;
     }
 
-  /* DFmode, see if we want to use the VSX unit.  */
+  /* SFmode, see if we want to use the VSX unit.  Don't set the VSX memory unit
+     for now, to allow use of the traditional floating point instructions.  */
+  if (TARGET_P8_VECTOR && TARGET_VSX_SCALAR_FLOAT)
+    {
+      rs6000_vector_unit[SFmode] = VECTOR_P8_VECTOR;
+      rs6000_vector_mem[SFmode] = VECTOR_NONE;
+      rs6000_vector_align[SFmode] = align32;
+    }
+
+  /* DFmode, see if we want to use the VSX unit.  Don't set the VSX memory unit
+     for now, to allow use of the traditional floating point instructions.  */
   if (TARGET_VSX && TARGET_VSX_SCALAR_DOUBLE)
     {
       rs6000_vector_unit[DFmode] = VECTOR_VSX;
-      rs6000_vector_mem[DFmode]
-	= (TARGET_VSX_SCALAR_MEMORY ? VECTOR_VSX : VECTOR_NONE);
+      rs6000_vector_mem[DFmode] = VECTOR_NONE;
       rs6000_vector_align[DFmode] = align64;
     }
 
@@ -2349,13 +2390,15 @@  rs6000_init_hard_regno_mode_ok (bool glo
 	 V4SF, wd = register class to use for V2DF, and ws = register classs to
 	 use for DF scalars.  */
       rs6000_constraints[RS6000_CONSTRAINT_wa] = VSX_REGS;
-      rs6000_constraints[RS6000_CONSTRAINT_wf] = VSX_REGS;
       rs6000_constraints[RS6000_CONSTRAINT_wd] = VSX_REGS;
-      rs6000_constraints[RS6000_CONSTRAINT_ws] = (TARGET_VSX_SCALAR_MEMORY
-						  ? VSX_REGS
-						  : FLOAT_REGS);
+      rs6000_constraints[RS6000_CONSTRAINT_wf] = VSX_REGS;
+      rs6000_constraints[RS6000_CONSTRAINT_wu] = ALTIVEC_REGS;
+
       if (TARGET_VSX_TIMODE)
 	rs6000_constraints[RS6000_CONSTRAINT_wt] = VSX_REGS;
+
+      rs6000_constraints[RS6000_CONSTRAINT_ws]
+	= (TARGET_UPPER_REGS_DF) ? VSX_REGS : FLOAT_REGS;
     }
 
   /* Add conditional constraints based on various options, to allow us to
@@ -2376,7 +2419,14 @@  rs6000_init_hard_regno_mode_ok (bool glo
     rs6000_constraints[RS6000_CONSTRAINT_wr] = GENERAL_REGS;
 
   if (TARGET_P8_VECTOR)
-    rs6000_constraints[RS6000_CONSTRAINT_wv] = ALTIVEC_REGS;
+    {
+      rs6000_constraints[RS6000_CONSTRAINT_wv] = ALTIVEC_REGS;
+      rs6000_constraints[RS6000_CONSTRAINT_wy]
+	= rs6000_constraints[RS6000_CONSTRAINT_ww]
+	= (TARGET_UPPER_REGS_SF) ? VSX_REGS : FLOAT_REGS;
+    }
+  else if (TARGET_VSX)
+    rs6000_constraints[RS6000_CONSTRAINT_ww] = FLOAT_REGS;
 
   if (TARGET_STFIWX)
     rs6000_constraints[RS6000_CONSTRAINT_wx] = FLOAT_REGS;
@@ -2409,20 +2459,22 @@  rs6000_init_hard_regno_mode_ok (bool glo
 	  rs6000_vector_reload[V4SFmode][1]  = CODE_FOR_reload_v4sf_di_load;
 	  rs6000_vector_reload[V2DFmode][0]  = CODE_FOR_reload_v2df_di_store;
 	  rs6000_vector_reload[V2DFmode][1]  = CODE_FOR_reload_v2df_di_load;
-	  if (TARGET_VSX && TARGET_VSX_SCALAR_MEMORY)
+#if 0
+	  if (VECTOR_UNIT_VSX_P (DFmode) && TARGET_UPPER_REGS_DF)
 	    {
 	      rs6000_vector_reload[DFmode][0]  = CODE_FOR_reload_df_di_store;
 	      rs6000_vector_reload[DFmode][1]  = CODE_FOR_reload_df_di_load;
 	      rs6000_vector_reload[DDmode][0]  = CODE_FOR_reload_dd_di_store;
 	      rs6000_vector_reload[DDmode][1]  = CODE_FOR_reload_dd_di_load;
 	    }
-	  if (TARGET_P8_VECTOR)
+	  if (VECTOR_UNIT_P8_VECTOR_P (SFmode) && TARGET_UPPER_REGS_SF)
 	    {
 	      rs6000_vector_reload[SFmode][0]  = CODE_FOR_reload_sf_di_store;
 	      rs6000_vector_reload[SFmode][1]  = CODE_FOR_reload_sf_di_load;
 	      rs6000_vector_reload[SDmode][0]  = CODE_FOR_reload_sd_di_store;
 	      rs6000_vector_reload[SDmode][1]  = CODE_FOR_reload_sd_di_load;
 	    }
+#endif
 	  if (TARGET_VSX_TIMODE)
 	    {
 	      rs6000_vector_reload[TImode][0]  = CODE_FOR_reload_ti_di_store;
@@ -2472,20 +2524,22 @@  rs6000_init_hard_regno_mode_ok (bool glo
 	  rs6000_vector_reload[V4SFmode][1]  = CODE_FOR_reload_v4sf_si_load;
 	  rs6000_vector_reload[V2DFmode][0]  = CODE_FOR_reload_v2df_si_store;
 	  rs6000_vector_reload[V2DFmode][1]  = CODE_FOR_reload_v2df_si_load;
-	  if (TARGET_VSX && TARGET_VSX_SCALAR_MEMORY)
+#if 0
+	  if (VECTOR_UNIT_VSX_P (DFmode) && TARGET_UPPER_REGS_DF)
 	    {
 	      rs6000_vector_reload[DFmode][0]  = CODE_FOR_reload_df_si_store;
 	      rs6000_vector_reload[DFmode][1]  = CODE_FOR_reload_df_si_load;
 	      rs6000_vector_reload[DDmode][0]  = CODE_FOR_reload_dd_si_store;
 	      rs6000_vector_reload[DDmode][1]  = CODE_FOR_reload_dd_si_load;
 	    }
-	  if (TARGET_P8_VECTOR)
+	  if (VECTOR_UNIT_P8_VECTOR_P (SFmode) && TARGET_UPPER_REGS_SF)
 	    {
 	      rs6000_vector_reload[SFmode][0]  = CODE_FOR_reload_sf_si_store;
 	      rs6000_vector_reload[SFmode][1]  = CODE_FOR_reload_sf_si_load;
 	      rs6000_vector_reload[SDmode][0]  = CODE_FOR_reload_sd_si_store;
 	      rs6000_vector_reload[SDmode][1]  = CODE_FOR_reload_sd_si_load;
 	    }
+#endif
 	  if (TARGET_VSX_TIMODE)
 	    {
 	      rs6000_vector_reload[TImode][0]  = CODE_FOR_reload_ti_si_store;
@@ -29162,6 +29216,8 @@  static struct rs6000_opt_mask const rs60
   { "recip-precision",		OPTION_MASK_RECIP_PRECISION,	false, true  },
   { "string",			OPTION_MASK_STRING,		false, true  },
   { "update",			OPTION_MASK_NO_UPDATE,		true , true  },
+  { "upper-regs-df",		OPTION_MASK_UPPER_REGS_DF,	false, false },
+  { "upper-regs-sf",		OPTION_MASK_UPPER_REGS_SF,	false, false },
   { "vsx",			OPTION_MASK_VSX,		false, true  },
   { "vsx-timode",		OPTION_MASK_VSX_TIMODE,		false, true  },
 #ifdef OPTION_MASK_64BIT
Index: gcc/config/rs6000/vsx.md
===================================================================
--- gcc/config/rs6000/vsx.md	(revision 201924)
+++ gcc/config/rs6000/vsx.md	(working copy)
@@ -316,40 +316,42 @@  (define_expand "vsx_store_<mode>"
   "")
 
 
-;; VSX scalar and vector floating point arithmetic instructions
+;; VSX vector floating point arithmetic instructions.  The VSX scalar
+;; instructions are now combined with the insn for the traditional floating
+;; point unit.
 (define_insn "*vsx_add<mode>3"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-        (plus:VSX_B (match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")
-		    (match_operand:VSX_B 2 "vsx_register_operand" "<VSr>,wa")))]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+        (plus:VSX_F (match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")
+		    (match_operand:VSX_F 2 "vsx_register_operand" "<VSr>,wa")))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>add<VSs> %x0,%x1,%x2"
-  [(set_attr "type" "<VStype_simple>")
+  "xvadd<VSs> %x0,%x1,%x2"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 (define_insn "*vsx_sub<mode>3"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-        (minus:VSX_B (match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")
-		     (match_operand:VSX_B 2 "vsx_register_operand" "<VSr>,wa")))]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+        (minus:VSX_F (match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")
+		     (match_operand:VSX_F 2 "vsx_register_operand" "<VSr>,wa")))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>sub<VSs> %x0,%x1,%x2"
-  [(set_attr "type" "<VStype_simple>")
+  "xvsub<VSs> %x0,%x1,%x2"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 (define_insn "*vsx_mul<mode>3"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-        (mult:VSX_B (match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")
-		    (match_operand:VSX_B 2 "vsx_register_operand" "<VSr>,wa")))]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+        (mult:VSX_F (match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")
+		    (match_operand:VSX_F 2 "vsx_register_operand" "<VSr>,wa")))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>mul<VSs> %x0,%x1,%x2"
-  [(set_attr "type" "<VStype_mul>")
+  "xvmul<VSs> %x0,%x1,%x2"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_mul>")])
 
 (define_insn "*vsx_div<mode>3"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-        (div:VSX_B (match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")
-		   (match_operand:VSX_B 2 "vsx_register_operand" "<VSr>,wa")))]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+        (div:VSX_F (match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")
+		   (match_operand:VSX_F 2 "vsx_register_operand" "<VSr>,wa")))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>div<VSs> %x0,%x1,%x2"
+  "xvdiv<VSs> %x0,%x1,%x2"
   [(set_attr "type" "<VStype_div>")
    (set_attr "fp_type" "<VSfptype_div>")])
 
@@ -392,94 +394,72 @@  (define_insn "*vsx_tdiv<mode>3_internal"
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 (define_insn "vsx_fre<mode>2"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-	(unspec:VSX_B [(match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+	(unspec:VSX_F [(match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")]
 		      UNSPEC_FRES))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>re<VSs> %x0,%x1"
-  [(set_attr "type" "<VStype_simple>")
+  "xvre<VSs> %x0,%x1"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 (define_insn "*vsx_neg<mode>2"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-        (neg:VSX_B (match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")))]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+        (neg:VSX_F (match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>neg<VSs> %x0,%x1"
-  [(set_attr "type" "<VStype_simple>")
+  "xvneg<VSs> %x0,%x1"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 (define_insn "*vsx_abs<mode>2"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-        (abs:VSX_B (match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")))]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+        (abs:VSX_F (match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>abs<VSs> %x0,%x1"
-  [(set_attr "type" "<VStype_simple>")
+  "xvabs<VSs> %x0,%x1"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 (define_insn "vsx_nabs<mode>2"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-        (neg:VSX_B
-	 (abs:VSX_B
-	  (match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa"))))]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+        (neg:VSX_F
+	 (abs:VSX_F
+	  (match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa"))))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>nabs<VSs> %x0,%x1"
-  [(set_attr "type" "<VStype_simple>")
+  "xvnabs<VSs> %x0,%x1"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 (define_insn "vsx_smax<mode>3"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-        (smax:VSX_B (match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")
-		    (match_operand:VSX_B 2 "vsx_register_operand" "<VSr>,wa")))]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+        (smax:VSX_F (match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")
+		    (match_operand:VSX_F 2 "vsx_register_operand" "<VSr>,wa")))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>max<VSs> %x0,%x1,%x2"
-  [(set_attr "type" "<VStype_simple>")
+  "xvmax<VSs> %x0,%x1,%x2"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 (define_insn "*vsx_smin<mode>3"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-        (smin:VSX_B (match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")
-		    (match_operand:VSX_B 2 "vsx_register_operand" "<VSr>,wa")))]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+        (smin:VSX_F (match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")
+		    (match_operand:VSX_F 2 "vsx_register_operand" "<VSr>,wa")))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>min<VSs> %x0,%x1,%x2"
-  [(set_attr "type" "<VStype_simple>")
+  "xvmin<VSs> %x0,%x1,%x2"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
-;; Special VSX version of smin/smax for single precision floating point.  Since
-;; both numbers are rounded to single precision, we can just use the DP version
-;; of the instruction.
-
-(define_insn "*vsx_smaxsf3"
-  [(set (match_operand:SF 0 "vsx_register_operand" "=f")
-        (smax:SF (match_operand:SF 1 "vsx_register_operand" "f")
-		 (match_operand:SF 2 "vsx_register_operand" "f")))]
-  "VECTOR_UNIT_VSX_P (DFmode)"
-  "xsmaxdp %x0,%x1,%x2"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_addsub_d")])
-
-(define_insn "*vsx_sminsf3"
-  [(set (match_operand:SF 0 "vsx_register_operand" "=f")
-        (smin:SF (match_operand:SF 1 "vsx_register_operand" "f")
-		 (match_operand:SF 2 "vsx_register_operand" "f")))]
-  "VECTOR_UNIT_VSX_P (DFmode)"
-  "xsmindp %x0,%x1,%x2"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_addsub_d")])
-
 (define_insn "*vsx_sqrt<mode>2"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-        (sqrt:VSX_B (match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")))]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+        (sqrt:VSX_F (match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>sqrt<VSs> %x0,%x1"
+  "xvsqrt<VSs> %x0,%x1"
   [(set_attr "type" "<VStype_sqrt>")
    (set_attr "fp_type" "<VSfptype_sqrt>")])
 
 (define_insn "*vsx_rsqrte<mode>2"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-	(unspec:VSX_B [(match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+	(unspec:VSX_F [(match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")]
 		      UNSPEC_RSQRT))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>rsqrte<VSs> %x0,%x1"
+  "xvrsqrte<VSs> %x0,%x1"
   [(set_attr "type" "<VStype_simple>")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
@@ -518,26 +498,9 @@  (define_insn "*vsx_tsqrt<mode>2_internal
   [(set_attr "type" "<VStype_simple>")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
-;; Fused vector multiply/add instructions Support the classical DF versions of
-;; fma, which allows the target to be a separate register from the 3 inputs.
-;; Under VSX, the target must be either the addend or the first multiply.
-;; Where we can, also do the same for the Altivec V4SF fmas.
-
-(define_insn "*vsx_fmadf4"
-  [(set (match_operand:DF 0 "vsx_register_operand" "=ws,ws,?wa,?wa,d")
-	(fma:DF
-	  (match_operand:DF 1 "vsx_register_operand" "%ws,ws,wa,wa,d")
-	  (match_operand:DF 2 "vsx_register_operand" "ws,0,wa,0,d")
-	  (match_operand:DF 3 "vsx_register_operand" "0,ws,0,wa,d")))]
-  "VECTOR_UNIT_VSX_P (DFmode)"
-  "@
-   xsmaddadp %x0,%x1,%x2
-   xsmaddmdp %x0,%x1,%x3
-   xsmaddadp %x0,%x1,%x2
-   xsmaddmdp %x0,%x1,%x3
-   fmadd %0,%1,%2,%3"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_maddsub_d")])
+;; Fused vector multiply/add instructions.  Under VSX, the target must be
+;; either the addend or the first multiply.  Where we can, also do the same for
+;; the Altivec V4SF fmas.
 
 (define_insn "*vsx_fmav4sf4"
   [(set (match_operand:V4SF 0 "vsx_register_operand" "=ws,ws,?wa,?wa,v")
@@ -568,23 +531,6 @@  (define_insn "*vsx_fmav2df4"
    xvmaddmdp %x0,%x1,%x3"
   [(set_attr "type" "vecdouble")])
 
-(define_insn "*vsx_fmsdf4"
-  [(set (match_operand:DF 0 "vsx_register_operand" "=ws,ws,?wa,?wa,d")
-	(fma:DF
-	  (match_operand:DF 1 "vsx_register_operand" "%ws,ws,wa,wa,d")
-	  (match_operand:DF 2 "vsx_register_operand" "ws,0,wa,0,d")
-	  (neg:DF
-	    (match_operand:DF 3 "vsx_register_operand" "0,ws,0,wa,d"))))]
-  "VECTOR_UNIT_VSX_P (DFmode)"
-  "@
-   xsmsubadp %x0,%x1,%x2
-   xsmsubmdp %x0,%x1,%x3
-   xsmsubadp %x0,%x1,%x2
-   xsmsubmdp %x0,%x1,%x3
-   fmsub %0,%1,%2,%3"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_maddsub_d")])
-
 (define_insn "*vsx_fms<mode>4"
   [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,<VSr>,?wa,?wa")
 	(fma:VSX_F
@@ -594,29 +540,12 @@  (define_insn "*vsx_fms<mode>4"
 	    (match_operand:VSX_F 3 "vsx_register_operand" "0,<VSr>,0,wa"))))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
   "@
-   x<VSv>msuba<VSs> %x0,%x1,%x2
-   x<VSv>msubm<VSs> %x0,%x1,%x3
-   x<VSv>msuba<VSs> %x0,%x1,%x2
-   x<VSv>msubm<VSs> %x0,%x1,%x3"
+   xvmsuba<VSs> %x0,%x1,%x2
+   xvmsubm<VSs> %x0,%x1,%x3
+   xvmsuba<VSs> %x0,%x1,%x2
+   xvmsubm<VSs> %x0,%x1,%x3"
   [(set_attr "type" "<VStype_mul>")])
 
-(define_insn "*vsx_nfmadf4"
-  [(set (match_operand:DF 0 "vsx_register_operand" "=ws,ws,?wa,?wa,d")
-	(neg:DF
-	 (fma:DF
-	  (match_operand:DF 1 "vsx_register_operand" "ws,ws,wa,wa,d")
-	  (match_operand:DF 2 "vsx_register_operand" "ws,0,wa,0,d")
-	  (match_operand:DF 3 "vsx_register_operand" "0,ws,0,wa,d"))))]
-  "VECTOR_UNIT_VSX_P (DFmode)"
-  "@
-   xsnmaddadp %x0,%x1,%x2
-   xsnmaddmdp %x0,%x1,%x3
-   xsnmaddadp %x0,%x1,%x2
-   xsnmaddmdp %x0,%x1,%x3
-   fnmadd %0,%1,%2,%3"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_maddsub_d")])
-
 (define_insn "*vsx_nfma<mode>4"
   [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,<VSr>,?wa,?wa")
 	(neg:VSX_F
@@ -626,31 +555,13 @@  (define_insn "*vsx_nfma<mode>4"
 	  (match_operand:VSX_F 3 "vsx_register_operand" "0,<VSr>,0,wa"))))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
   "@
-   x<VSv>nmadda<VSs> %x0,%x1,%x2
-   x<VSv>nmaddm<VSs> %x0,%x1,%x3
-   x<VSv>nmadda<VSs> %x0,%x1,%x2
-   x<VSv>nmaddm<VSs> %x0,%x1,%x3"
+   xvnmadda<VSs> %x0,%x1,%x2
+   xvnmaddm<VSs> %x0,%x1,%x3
+   xvnmadda<VSs> %x0,%x1,%x2
+   xvnmaddm<VSs> %x0,%x1,%x3"
   [(set_attr "type" "<VStype_mul>")
    (set_attr "fp_type" "<VSfptype_mul>")])
 
-(define_insn "*vsx_nfmsdf4"
-  [(set (match_operand:DF 0 "vsx_register_operand" "=ws,ws,?wa,?wa,d")
-	(neg:DF
-	 (fma:DF
-	   (match_operand:DF 1 "vsx_register_operand" "%ws,ws,wa,wa,d")
-	   (match_operand:DF 2 "vsx_register_operand" "ws,0,wa,0,d")
-	   (neg:DF
-	     (match_operand:DF 3 "vsx_register_operand" "0,ws,0,wa,d")))))]
-  "VECTOR_UNIT_VSX_P (DFmode)"
-  "@
-   xsnmsubadp %x0,%x1,%x2
-   xsnmsubmdp %x0,%x1,%x3
-   xsnmsubadp %x0,%x1,%x2
-   xsnmsubmdp %x0,%x1,%x3
-   fnmsub %0,%1,%2,%3"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_maddsub_d")])
-
 (define_insn "*vsx_nfmsv4sf4"
   [(set (match_operand:V4SF 0 "vsx_register_operand" "=wf,wf,?wa,?wa,v")
 	(neg:V4SF
@@ -712,16 +623,6 @@  (define_insn "*vsx_ge<mode>"
   [(set_attr "type" "<VStype_simple>")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
-;; Floating point scalar compare
-(define_insn "*vsx_cmpdf_internal1"
-  [(set (match_operand:CCFP 0 "cc_reg_operand" "=y,?y")
-	(compare:CCFP (match_operand:DF 1 "gpc_reg_operand" "ws,wa")
-		      (match_operand:DF 2 "gpc_reg_operand" "ws,wa")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && VECTOR_UNIT_VSX_P (DFmode)"
-  "xscmpudp %0,%x1,%x2"
-  [(set_attr "type" "fpcompare")])
-
 ;; Compare vectors producing a vector result and a predicate, setting CR6 to
 ;; indicate a combined status
 (define_insn "*vsx_eq_<mode>_p"
@@ -788,14 +689,14 @@  (define_insn "*vsx_xxsel<mode>_uns"
 
 ;; Copy sign
 (define_insn "vsx_copysign<mode>3"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-	(unspec:VSX_B
-	 [(match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")
-	  (match_operand:VSX_B 2 "vsx_register_operand" "<VSr>,wa")]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+	(unspec:VSX_F
+	 [(match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")
+	  (match_operand:VSX_F 2 "vsx_register_operand" "<VSr>,wa")]
 	 UNSPEC_COPYSIGN))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>cpsgn<VSs> %x0,%x2,%x1"
-  [(set_attr "type" "<VStype_simple>")
+  "xvcpsgn<VSs> %x0,%x2,%x1"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 ;; For the conversions, limit the register class for the integer value to be
@@ -855,11 +756,11 @@  (define_insn "vsx_x<VSv>r<VSs>ic"
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 (define_insn "vsx_btrunc<mode>2"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-	(fix:VSX_B (match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")))]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+	(fix:VSX_F (match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>r<VSs>iz %x0,%x1"
-  [(set_attr "type" "<VStype_simple>")
+  "xvr<VSs>iz %x0,%x1"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 (define_insn "*vsx_b2trunc<mode>2"
@@ -872,21 +773,21 @@  (define_insn "*vsx_b2trunc<mode>2"
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 (define_insn "vsx_floor<mode>2"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-	(unspec:VSX_B [(match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+	(unspec:VSX_F [(match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")]
 		      UNSPEC_FRIM))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>r<VSs>im %x0,%x1"
-  [(set_attr "type" "<VStype_simple>")
+  "xvr<VSs>im %x0,%x1"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 (define_insn "vsx_ceil<mode>2"
-  [(set (match_operand:VSX_B 0 "vsx_register_operand" "=<VSr>,?wa")
-	(unspec:VSX_B [(match_operand:VSX_B 1 "vsx_register_operand" "<VSr>,wa")]
+  [(set (match_operand:VSX_F 0 "vsx_register_operand" "=<VSr>,?wa")
+	(unspec:VSX_F [(match_operand:VSX_F 1 "vsx_register_operand" "<VSr>,wa")]
 		      UNSPEC_FRIP))]
   "VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "x<VSv>r<VSs>ip %x0,%x1"
-  [(set_attr "type" "<VStype_simple>")
+  "xvr<VSs>ip %x0,%x1"
+  [(set_attr "type" "vecdouble")
    (set_attr "fp_type" "<VSfptype_simple>")])
 
 
Index: gcc/config/rs6000/rs6000.h
===================================================================
--- gcc/config/rs6000/rs6000.h	(revision 201924)
+++ gcc/config/rs6000/rs6000.h	(working copy)
@@ -617,6 +617,25 @@  extern int rs6000_vector_align[];
 			  || rs6000_cpu == PROCESSOR_PPC8548)
 
 
+/* Whether SF/DF operations are supported on the E500.  */
+#define TARGET_SF_SPE	(TARGET_HARD_FLOAT && TARGET_SINGLE_FLOAT	\
+			 && !TARGET_FPRS)
+
+#define TARGET_DF_SPE	(TARGET_HARD_FLOAT && TARGET_DOUBLE_FLOAT	\
+			 && !TARGET_FPRS && TARGET_E500_DOUBLE)
+
+/* Whether SF/DF operations are supported by by the normal floating point unit
+   (or the vector/scalar unit).  */
+#define TARGET_SF_FPR	(TARGET_HARD_FLOAT && TARGET_FPRS		\
+			 && TARGET_SINGLE_FLOAT)
+
+#define TARGET_DF_FPR	(TARGET_HARD_FLOAT && TARGET_FPRS		\
+			 && TARGET_DOUBLE_FLOAT)
+
+/* Whether SF/DF operations are supported by any hardware.  */
+#define TARGET_SF_INSN	(TARGET_SF_FPR || TARGET_SF_SPE)
+#define TARGET_DF_INSN	(TARGET_DF_FPR || TARGET_DF_SPE)
+
 /* Which machine supports the various reciprocal estimate instructions.  */
 #define TARGET_FRES	(TARGET_HARD_FLOAT && TARGET_PPC_GFXOPT \
 			 && TARGET_FPRS && TARGET_SINGLE_FLOAT)
@@ -1403,15 +1422,18 @@  enum r6000_reg_class_enum {
   RS6000_CONSTRAINT_v,		/* Altivec registers */
   RS6000_CONSTRAINT_wa,		/* Any VSX register */
   RS6000_CONSTRAINT_wd,		/* VSX register for V2DF */
-  RS6000_CONSTRAINT_wg,		/* FPR register for -mmfpgpr */
   RS6000_CONSTRAINT_wf,		/* VSX register for V4SF */
+  RS6000_CONSTRAINT_wg,		/* FPR register for -mmfpgpr */
   RS6000_CONSTRAINT_wl,		/* FPR register for LFIWAX */
   RS6000_CONSTRAINT_wm,		/* VSX register for direct move */
   RS6000_CONSTRAINT_wr,		/* GPR register if 64-bit  */
   RS6000_CONSTRAINT_ws,		/* VSX register for DF */
   RS6000_CONSTRAINT_wt,		/* VSX register for TImode */
-  RS6000_CONSTRAINT_wv,		/* Altivec register for power8 vector */
+  RS6000_CONSTRAINT_wu,		/* Altivec register for float load/stores.  */
+  RS6000_CONSTRAINT_wv,		/* Altivec register for double load/stores.  */
+  RS6000_CONSTRAINT_ww,		/* FP or VSX register for vsx float ops.  */
   RS6000_CONSTRAINT_wx,		/* FPR register for STFIWX */
+  RS6000_CONSTRAINT_wy,		/* VSX register for SF */
   RS6000_CONSTRAINT_wz,		/* FPR register for LFIWZX */
   RS6000_CONSTRAINT_MAX
 };
Index: gcc/config/rs6000/rs6000.md
===================================================================
--- gcc/config/rs6000/rs6000.md	(revision 201924)
+++ gcc/config/rs6000/rs6000.md	(working copy)
@@ -335,6 +335,25 @@  (define_mode_iterator RECIPF [SF DF V4SF
 ; Iterator for just SF/DF
 (define_mode_iterator SFDF [SF DF])
 
+; SF/DF suffix for traditional floating instructions
+(define_mode_attr Ftrad		[(SF "s") (DF "")])
+
+; SF/DF suffix for VSX instructions
+(define_mode_attr Fvsx		[(SF "sp") (DF	"dp")])
+
+; SF/DF constraint for arithmetic on traditional floating point registers
+(define_mode_attr Ff		[(SF "f") (DF "d")])
+
+; SF/DF constraint for arithmetic on VSX registers
+(define_mode_attr Fv		[(SF "wy") (DF "ws")])
+
+; s/d suffix for things like fp_addsub_s/fp_addsub_d
+(define_mode_attr Fs		[(SF "s")  (DF "d")])
+
+; FRE/FRES support
+(define_mode_attr Ffre		[(SF "fres") (DF "fre")])
+(define_mode_attr FFRE		[(SF "FRES") (DF "FRE")])
+
 ; Conditional returns.
 (define_code_iterator any_return [return simple_return])
 (define_code_attr return_pred [(return "direct_return ()")
@@ -541,7 +560,7 @@  (define_split
   "")
 
 (define_insn "*zero_extendsidi2_lfiwzx"
-  [(set (match_operand:DI 0 "gpc_reg_operand" "=r,r,??wm,!wz,!wm")
+  [(set (match_operand:DI 0 "gpc_reg_operand" "=r,r,??wm,!wz,!wu")
 	(zero_extend:DI (match_operand:SI 1 "reg_or_mem_operand" "m,r,r,Z,Z")))]
   "TARGET_POWERPC64 && TARGET_LFIWZX"
   "@
@@ -711,7 +730,7 @@  (define_expand "extendsidi2"
   "")
 
 (define_insn "*extendsidi2_lfiwax"
-  [(set (match_operand:DI 0 "gpc_reg_operand" "=r,r,??wm,!wl,!wm")
+  [(set (match_operand:DI 0 "gpc_reg_operand" "=r,r,??wm,!wl,!wu")
 	(sign_extend:DI (match_operand:SI 1 "lwa_operand" "m,r,r,Z,Z")))]
   "TARGET_POWERPC64 && TARGET_LFIWAX"
   "@
@@ -5042,232 +5061,292 @@  (define_split
   "")
 
 ;; Floating-point insns, excluding normal data motion.
-;;
-;; PowerPC has a full set of single-precision floating point instructions.
-;;
-;; For the POWER architecture, we pretend that we have both SFmode and
-;; DFmode insns, while, in fact, all fp insns are actually done in double.
-;; The only conversions we will do will be when storing to memory.  In that
-;; case, we will use the "frsp" instruction before storing.
-;;
-;; Note that when we store into a single-precision memory location, we need to
-;; use the frsp insn first.  If the register being stored isn't dead, we
-;; need a scratch register for the frsp.  But this is difficult when the store
-;; is done by reload.  It is not incorrect to do the frsp on the register in
-;; this case, we just lose precision that we would have otherwise gotten but
-;; is not guaranteed.  Perhaps this should be tightened up at some point.
 
-(define_expand "extendsfdf2"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	(float_extend:DF (match_operand:SF 1 "reg_or_none500mem_operand" "")))]
-  "TARGET_HARD_FLOAT && ((TARGET_FPRS && TARGET_DOUBLE_FLOAT) || TARGET_E500_DOUBLE)"
+;; This section provides the expanders for single/double precision binary
+;; floating point, but the support the E500 floating point instructions is in
+;; spe.md.
+
+;; Use the double varient of the instruction if we aren't doing calculations
+;; that modify the bottom bits (abs, min, max, copysign, etc.) and there isn't
+;; an explicit single precision version.
+
+;; We use the VSX instructions for accessing the upper 32 registers under VSX
+;; for DFmode and p8-vector for SFmode.  We generate the traditional floating
+;; point instructions if all of the registers come from the traditional float
+;; point register set.  Originally when the first support for VSX went in, we
+;; would use the xs* instruction all of the time, but this simplifies things.
+
+(define_expand "abs<mode>2"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
+	(abs:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "")))]
+  "TARGET_<MODE>_INSN"
   "")
 
-(define_insn_and_split "*extendsfdf2_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=d,?d,d")
-	(float_extend:DF (match_operand:SF 1 "reg_or_mem_operand" "0,f,m")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT"
+(define_insn "*abs<mode>2_fpr"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(abs:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>")))]
+  "TARGET_<MODE>_FPR"
   "@
-   #
-   fmr %0,%1
-   lfs%U1%X1 %0,%1"
-  "&& reload_completed && REG_P (operands[1]) && REGNO (operands[0]) == REGNO (operands[1])"
-  [(const_int 0)]
-{
-  emit_note (NOTE_INSN_DELETED);
-  DONE;
-}
-  [(set_attr_alternative "type"
-      [(const_string "fp")
-       (const_string "fp")
-       (if_then_else
-	 (match_test "update_indexed_address_mem (operands[1], VOIDmode)")
-	 (const_string "fpload_ux")
-	 (if_then_else
-	   (match_test "update_address_mem (operands[1], VOIDmode)")
-	   (const_string "fpload_u")
-	   (const_string "fpload")))])])
-
-(define_expand "truncdfsf2"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	(float_truncate:SF (match_operand:DF 1 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT && ((TARGET_FPRS && TARGET_DOUBLE_FLOAT) || TARGET_E500_DOUBLE)"
-  "")
+   fabs %0,%1
+   xsabsdp %x0,%x1"
+  [(set_attr "type" "fp")])
 
-(define_insn "*truncdfsf2_fpr"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(float_truncate:SF (match_operand:DF 1 "gpc_reg_operand" "d")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT"
-  "frsp %0,%1"
+(define_insn "*nabs<mode>2_fpr"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(neg:SFDF
+	 (abs:SFDF
+	  (match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>"))))]
+  "TARGET_<MODE>_FPR"
+  "@
+   fnabs %0,%1
+   xsnabsdp %x0,%x1"
   [(set_attr "type" "fp")])
 
-(define_expand "negsf2"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	(neg:SF (match_operand:SF 1 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT && TARGET_SINGLE_FLOAT"
+(define_expand "neg<mode>2"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
+	(neg:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "")))]
+  "TARGET_<MODE>_INSN"
   "")
 
-(define_insn "*negsf2"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(neg:SF (match_operand:SF 1 "gpc_reg_operand" "f")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
-  "fneg %0,%1"
+(define_insn "*neg<mode>2_fpr"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(neg:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>")))]
+  "TARGET_<MODE>_FPR"
+  "@
+   fneg %0,%1
+   xsnegdp %x0,%x1"
   [(set_attr "type" "fp")])
 
-(define_expand "abssf2"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	(abs:SF (match_operand:SF 1 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT && TARGET_SINGLE_FLOAT"
-  "")
+;; For MIN, MAX on non-VSX machines, and conditional move all of the time, we
+;; use DEFINE_EXPAND's that involve a fsel instruction and some auxiliary
+;; computations.  Then we just have a single DEFINE_INSN for fsel and the
+;; define_splits to make them if made by combine.  On VSX machines we have the
+;; min/max instructions.
+;;
+;; On VSX, we only check for TARGET_VSX instead of checking for a vsx/p8 vector
+;; to allow either DF/SF to use only traditional registers.
 
-(define_insn "*abssf2"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(abs:SF (match_operand:SF 1 "gpc_reg_operand" "f")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
-  "fabs %0,%1"
-  [(set_attr "type" "fp")])
+(define_expand "smax<mode>3"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
+	(if_then_else:SFDF (ge (match_operand:SFDF 1 "gpc_reg_operand" "")
+			       (match_operand:SFDF 2 "gpc_reg_operand" ""))
+			   (match_dup 1)
+			   (match_dup 2)))]
+  "TARGET_<MODE>_FPR && TARGET_PPC_GFXOPT && !flag_trapping_math"
+{
+  rs6000_emit_minmax (operands[0], SMAX, operands[1], operands[2]);
+  DONE;
+})
 
-(define_insn ""
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(neg:SF (abs:SF (match_operand:SF 1 "gpc_reg_operand" "f"))))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
-  "fnabs %0,%1"
+(define_insn "*smax<mode>3_vsx"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(smax:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "%<Ff>,<Fv>")
+		   (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>,<Fv>")))]
+  "TARGET_<MODE>_FPR && TARGET_VSX"
+  "xsmaxdp %x0,%x1,%x2"
   [(set_attr "type" "fp")])
 
-(define_expand "addsf3"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	(plus:SF (match_operand:SF 1 "gpc_reg_operand" "")
-		 (match_operand:SF 2 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT && TARGET_SINGLE_FLOAT"
-  "")
+(define_expand "smin<mode>3"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
+	(if_then_else:SFDF (ge (match_operand:SFDF 1 "gpc_reg_operand" "")
+			       (match_operand:SFDF 2 "gpc_reg_operand" ""))
+			   (match_dup 2)
+			   (match_dup 1)))]
+  "TARGET_<MODE>_FPR && TARGET_PPC_GFXOPT && !flag_trapping_math"
+{
+  rs6000_emit_minmax (operands[0], SMIN, operands[1], operands[2]);
+  DONE;
+})
 
-(define_insn ""
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(plus:SF (match_operand:SF 1 "gpc_reg_operand" "%f")
-		 (match_operand:SF 2 "gpc_reg_operand" "f")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
-  "fadds %0,%1,%2"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_addsub_s")])
+(define_insn "*smin<mode>3_vsx"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(smin:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "%<Ff>,<Fv>")
+		   (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>,<Fv>")))]
+  "TARGET_<MODE>_FPR && TARGET_VSX"
+  "xsmindp %x0,%x1,%x2"
+  [(set_attr "type" "fp")])
 
-(define_expand "subsf3"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	(minus:SF (match_operand:SF 1 "gpc_reg_operand" "")
-		  (match_operand:SF 2 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT && TARGET_SINGLE_FLOAT"
+(define_split
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
+	(match_operator:SFDF 3 "min_max_operator"
+	 [(match_operand:SFDF 1 "gpc_reg_operand" "")
+	  (match_operand:SFDF 2 "gpc_reg_operand" "")]))]
+  "TARGET_<MODE>_FPR && TARGET_PPC_GFXOPT && !flag_trapping_math
+   && !TARGET_VSX"
+  [(const_int 0)]
+{
+  rs6000_emit_minmax (operands[0], GET_CODE (operands[3]), operands[1],
+		      operands[2]);
+  DONE;
+})
+
+(define_expand "add<mode>3"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
+	(plus:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "")
+		   (match_operand:SFDF 2 "gpc_reg_operand" "")))]
+  "TARGET_<MODE>_INSN"
   "")
 
-(define_insn ""
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(minus:SF (match_operand:SF 1 "gpc_reg_operand" "f")
-		  (match_operand:SF 2 "gpc_reg_operand" "f")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
-  "fsubs %0,%1,%2"
+(define_insn "*add<mode>3_fpr"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(plus:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "%<Ff>,<Fv>")
+		   (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>,<Fv>")))]
+  "TARGET_<MODE>_FPR"
+  "@
+   fadd<Ftrad> %0,%1,%2
+   xsadd<Fvsx> %x0,%x1,%x2"
   [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_addsub_s")])
+   (set_attr "fp_type" "fp_addsub_<Fs>")])
 
-(define_expand "mulsf3"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	(mult:SF (match_operand:SF 1 "gpc_reg_operand" "")
-		 (match_operand:SF 2 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT && TARGET_SINGLE_FLOAT"
+(define_expand "sub<mode>3"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
+	(minus:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "")
+		    (match_operand:SFDF 2 "gpc_reg_operand" "")))]
+  "TARGET_<MODE>_INSN"
   "")
 
-(define_insn ""
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(mult:SF (match_operand:SF 1 "gpc_reg_operand" "%f")
-		 (match_operand:SF 2 "gpc_reg_operand" "f")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
-  "fmuls %0,%1,%2"
+(define_insn "*sub<mode>3_fpr"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(minus:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>")
+		    (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>,<Fv>")))]
+  "TARGET_<MODE>_FPR"
+  "@
+   fsub<Ftrad> %0,%1,%2
+   xssub<Fvsx> %x0,%x1,%x2"
   [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_mul_s")])
+   (set_attr "fp_type" "fp_addsub_<Fs>")])
 
-(define_expand "divsf3"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	(div:SF (match_operand:SF 1 "gpc_reg_operand" "")
-		(match_operand:SF 2 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT && TARGET_SINGLE_FLOAT && !TARGET_SIMPLE_FPU"
+(define_expand "mul<mode>3"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
+	(mult:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "")
+		   (match_operand:SFDF 2 "gpc_reg_operand" "")))]
+  "TARGET_<MODE>_INSN"
   "")
 
-(define_insn ""
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(div:SF (match_operand:SF 1 "gpc_reg_operand" "f")
-		(match_operand:SF 2 "gpc_reg_operand" "f")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS
-   && TARGET_SINGLE_FLOAT && !TARGET_SIMPLE_FPU"
-  "fdivs %0,%1,%2"
-  [(set_attr "type" "sdiv")])
+(define_insn "*mul<mode>3_fpr"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(mult:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "%<Ff>,<Fv>")
+		   (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>,<Fv>")))]
+  "TARGET_<MODE>_FPR"
+  "@
+   fmul<Ftrad> %0,%1,%2
+   xsmul<Fvsx> %x0,%x1,%x2"
+  [(set_attr "type" "fp")
+   (set_attr "fp_type" "fp_mul_<Fs>")])
 
-(define_insn "fres"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(unspec:SF [(match_operand:SF 1 "gpc_reg_operand" "f")] UNSPEC_FRES))]
-  "TARGET_FRES"
-  "fres %0,%1"
+(define_expand "div<mode>3"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
+	(div:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "")
+		  (match_operand:SFDF 2 "gpc_reg_operand" "")))]
+  "TARGET_<MODE>_INSN && !TARGET_SIMPLE_FPU"
+  "")
+
+(define_insn "*div<mode>3_fpr"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(div:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>")
+		  (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>,<Fv>")))]
+  "TARGET_<MODE>_FPR && !TARGET_SIMPLE_FPU"
+  "@
+   fdiv<Ftrad> %0,%1,%2
+   xsdiv<Fvsx> %x0,%x1,%x2"
+  [(set_attr "type" "<Fs>div")])
+
+(define_insn "fre<Fs>"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>")]
+		     UNSPEC_FRES))]
+  "TARGET_<FFRE>"
+  "@
+   fre<Ftrad> %0,%1
+   xsre<Fvsx> %x0,%x1"
   [(set_attr "type" "fp")])
 
-; builtin fmaf support
-(define_insn "*fmasf4_fpr"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(fma:SF (match_operand:SF 1 "gpc_reg_operand" "f")
-		(match_operand:SF 2 "gpc_reg_operand" "f")
-		(match_operand:SF 3 "gpc_reg_operand" "f")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
-  "fmadds %0,%1,%2,%3"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_maddsub_s")])
+(define_insn "sqrt<mode>2"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(sqrt:SFDF (match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>")))]
+  "TARGET_<MODE>_FPR && !TARGET_SIMPLE_FPU
+   && (TARGET_PPC_GPOPT || (<MODE>mode == SFmode && TARGET_XILINX_FPU))"
+  "@
+   fsqrt<Ftrad> %0,%1
+   xssqrt<Fvsx> %x0,%x1"
+  [(set_attr "type" "<Fs>sqrt")])
+
+(define_insn "*rsqrt<mode>2"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>")]
+		     UNSPEC_RSQRT))]
+  "RS6000_RECIP_HAVE_RSQRTE_P (<MODE>mode)"
+  "@
+   frsqrte<Ftrad> %0,%1
+   xsrsqrte<Fvsx> %x0,%x1"
+  [(set_attr "type" "fp")])
 
-(define_insn "*fmssf4_fpr"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(fma:SF (match_operand:SF 1 "gpc_reg_operand" "f")
-		(match_operand:SF 2 "gpc_reg_operand" "f")
-		(neg:SF (match_operand:SF 3 "gpc_reg_operand" "f"))))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
-  "fmsubs %0,%1,%2,%3"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_maddsub_s")])
+(define_insn "*cmp<mode>_fpr"
+  [(set (match_operand:CCFP 0 "cc_reg_operand" "=y,y")
+	(compare:CCFP (match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>")
+		      (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>,<Fv>")))]
+  "TARGET_<MODE>_FPR"
+  "@
+   fcmpu %0,%1,%2
+   xscmpudp %x0,%x1,%x2"
+  [(set_attr "type" "fpcompare")])
 
-(define_insn "*nfmasf4_fpr"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(neg:SF (fma:SF (match_operand:SF 1 "gpc_reg_operand" "f")
-			(match_operand:SF 2 "gpc_reg_operand" "f")
-			(match_operand:SF 3 "gpc_reg_operand" "f"))))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
-  "fnmadds %0,%1,%2,%3"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_maddsub_s")])
+(define_expand "extendsfdf2"
+  [(set (match_operand:DF 0 "gpc_reg_operand" "")
+	(float_extend:DF (match_operand:SF 1 "reg_or_none500mem_operand" "")))]
+  "TARGET_HARD_FLOAT && ((TARGET_FPRS && TARGET_DOUBLE_FLOAT) || TARGET_E500_DOUBLE)"
+  "")
 
-(define_insn "*nfmssf4_fpr"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(neg:SF (fma:SF (match_operand:SF 1 "gpc_reg_operand" "f")
-			(match_operand:SF 2 "gpc_reg_operand" "f")
-			(neg:SF (match_operand:SF 3 "gpc_reg_operand" "f")))))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
-  "fnmsubs %0,%1,%2,%3"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_maddsub_s")])
+(define_insn_and_split "*extendsfdf2_fpr"
+  [(set (match_operand:DF 0 "gpc_reg_operand" "=d,?d,d,wy,?wy,wy")
+	(float_extend:DF (match_operand:SF 1 "reg_or_mem_operand" "0,f,m,0,wy,Z")))]
+  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT"
+  "@
+   #
+   fmr %0,%1
+   lfs%U1%X1 %0,%1
+   #
+   xxlor %x0,%x1,%x1
+   lxsspx %x0,%y1"
+  "&& reload_completed && REG_P (operands[1]) && REGNO (operands[0]) == REGNO (operands[1])"
+  [(const_int 0)]
+{
+  emit_note (NOTE_INSN_DELETED);
+  DONE;
+}
+  [(set_attr_alternative "type"
+      [(const_string "fp")
+       (const_string "fp")
+       (if_then_else
+	 (match_test "update_indexed_address_mem (operands[1], VOIDmode)")
+	 (const_string "fpload_ux")
+	 (if_then_else
+	   (match_test "update_address_mem (operands[1], VOIDmode)")
+	   (const_string "fpload_u")
+	   (const_string "fpload")))
+       (const_string "fp")
+       (const_string "vecsimple")
+       (if_then_else
+	(match_test "update_indexed_address_mem (operands[1], VOIDmode)")
+	(const_string "fpload_ux")
+	(if_then_else
+	 (match_test "update_address_mem (operands[1], VOIDmode)")
+	 (const_string "fpload_u")
+	 (const_string "fpload")))])])
 
-(define_expand "sqrtsf2"
+(define_expand "truncdfsf2"
   [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	(sqrt:SF (match_operand:SF 1 "gpc_reg_operand" "")))]
-  "(TARGET_PPC_GPOPT || TARGET_XILINX_FPU)
-   && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT
-   && !TARGET_SIMPLE_FPU"
+	(float_truncate:SF (match_operand:DF 1 "gpc_reg_operand" "")))]
+  "TARGET_DF_INSN"
   "")
 
-(define_insn ""
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(sqrt:SF (match_operand:SF 1 "gpc_reg_operand" "f")))]
-  "(TARGET_PPC_GPOPT || TARGET_XILINX_FPU) && TARGET_HARD_FLOAT
-   && TARGET_FPRS && TARGET_SINGLE_FLOAT && !TARGET_SIMPLE_FPU"
-  "fsqrts %0,%1"
-  [(set_attr "type" "ssqrt")])
-
-(define_insn "*rsqrtsf_internal1"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(unspec:SF [(match_operand:SF 1 "gpc_reg_operand" "f")]
-		   UNSPEC_RSQRT))]
-  "TARGET_FRSQRTES"
-  "frsqrtes %0,%1"
+(define_insn "*truncdfsf2_fpr"
+  [(set (match_operand:SF 0 "gpc_reg_operand" "=f,wy")
+	(float_truncate:SF (match_operand:DF 1 "gpc_reg_operand" "d,wy")))]
+  "TARGET_DF_FPR"
+  "@
+   frsp %0,%1
+   xscvdpsp %x0,%x1"
   [(set_attr "type" "fp")])
 
 ;; This expander is here to avoid FLOAT_WORDS_BIGENDIAN tests in
@@ -5337,52 +5416,16 @@  (define_expand "copysign<mode>3"
 ;; Use an unspec rather providing an if-then-else in RTL, to prevent the
 ;; compiler from optimizing -0.0
 (define_insn "copysign<mode>3_fcpsgn"
-  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<rreg2>")
-	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "<rreg2>")
-		      (match_operand:SFDF 2 "gpc_reg_operand" "<rreg2>")]
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>")
+		      (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>,<Fv>")]
 		     UNSPEC_COPYSIGN))]
-  "TARGET_CMPB && !VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "fcpsgn %0,%2,%1"
+  "TARGET_<MODE>_FPR && TARGET_CMPB"
+  "@
+   fcpsgn %0,%2,%1
+   xscpsgn<VSs> %x0,%x2,%x1"
   [(set_attr "type" "fp")])
 
-;; For MIN, MAX, and conditional move, we use DEFINE_EXPAND's that involve a
-;; fsel instruction and some auxiliary computations.  Then we just have a
-;; single DEFINE_INSN for fsel and the define_splits to make them if made by
-;; combine.
-(define_expand "smaxsf3"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	(if_then_else:SF (ge (match_operand:SF 1 "gpc_reg_operand" "")
-			     (match_operand:SF 2 "gpc_reg_operand" ""))
-			 (match_dup 1)
-			 (match_dup 2)))]
-  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS 
-   && TARGET_SINGLE_FLOAT && !flag_trapping_math"
-  "{ rs6000_emit_minmax (operands[0], SMAX, operands[1], operands[2]); DONE;}")
-
-(define_expand "sminsf3"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	(if_then_else:SF (ge (match_operand:SF 1 "gpc_reg_operand" "")
-			     (match_operand:SF 2 "gpc_reg_operand" ""))
-			 (match_dup 2)
-			 (match_dup 1)))]
-  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS 
-   && TARGET_SINGLE_FLOAT && !flag_trapping_math"
-  "{ rs6000_emit_minmax (operands[0], SMIN, operands[1], operands[2]); DONE;}")
-
-(define_split
-  [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	(match_operator:SF 3 "min_max_operator"
-	 [(match_operand:SF 1 "gpc_reg_operand" "")
-	  (match_operand:SF 2 "gpc_reg_operand" "")]))]
-  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS 
-   && TARGET_SINGLE_FLOAT && !flag_trapping_math"
-  [(const_int 0)]
-  "
-{ rs6000_emit_minmax (operands[0], GET_CODE (operands[3]),
-		      operands[1], operands[2]);
-  DONE;
-}")
-
 (define_expand "mov<mode>cc"
    [(set (match_operand:GPR 0 "gpc_reg_operand" "")
 	 (if_then_else:GPR (match_operand 1 "comparison_operator" "")
@@ -5465,12 +5508,12 @@  (define_insn "*isel_reversed_unsigned_<m
   [(set_attr "type" "isel")
    (set_attr "length" "4")])
 
-(define_expand "movsfcc"
-   [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	 (if_then_else:SF (match_operand 1 "comparison_operator" "")
-			  (match_operand:SF 2 "gpc_reg_operand" "")
-			  (match_operand:SF 3 "gpc_reg_operand" "")))]
-  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
+(define_expand "mov<mode>cc"
+   [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
+	 (if_then_else:SFDF (match_operand 1 "comparison_operator" "")
+			    (match_operand:SFDF 2 "gpc_reg_operand" "")
+			    (match_operand:SFDF 3 "gpc_reg_operand" "")))]
+  "TARGET_<MODE>_FPR && TARGET_PPC_GFXOPT"
   "
 {
   if (rs6000_emit_cmove (operands[0], operands[1], operands[2], operands[3]))
@@ -5479,282 +5522,37 @@  (define_expand "movsfcc"
     FAIL;
 }")
 
-(define_insn "*fselsfsf4"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(if_then_else:SF (ge (match_operand:SF 1 "gpc_reg_operand" "f")
-			     (match_operand:SF 4 "zero_fp_constant" "F"))
-			 (match_operand:SF 2 "gpc_reg_operand" "f")
-			 (match_operand:SF 3 "gpc_reg_operand" "f")))]
-  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
-  "fsel %0,%1,%2,%3"
-  [(set_attr "type" "fp")])
-
-(define_insn "*fseldfsf4"
-  [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(if_then_else:SF (ge (match_operand:DF 1 "gpc_reg_operand" "d")
-			     (match_operand:DF 4 "zero_fp_constant" "F"))
-			 (match_operand:SF 2 "gpc_reg_operand" "f")
-			 (match_operand:SF 3 "gpc_reg_operand" "f")))]
-  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT && TARGET_SINGLE_FLOAT"
+;; There is no single VSX instruction that does what fsel does, so limit this
+;; to traditional floating point registers.
+(define_insn "*fsel<mode>sf4"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>")
+	(if_then_else:SFDF (ge (match_operand:SF 1 "gpc_reg_operand" "f")
+			       (match_operand:SF 4 "zero_fp_constant" "F"))
+			 (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>")
+			 (match_operand:SFDF 3 "gpc_reg_operand" "<Ff>")))]
+  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT
+   && TARGET_DOUBLE_FLOAT"
   "fsel %0,%1,%2,%3"
   [(set_attr "type" "fp")])
 
-(define_expand "negdf2"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	(neg:DF (match_operand:DF 1 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT && ((TARGET_FPRS && TARGET_DOUBLE_FLOAT) || TARGET_E500_DOUBLE)"
-  "")
-
-(define_insn "*negdf2_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(neg:DF (match_operand:DF 1 "gpc_reg_operand" "d")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && !VECTOR_UNIT_VSX_P (DFmode)"
-  "fneg %0,%1"
-  [(set_attr "type" "fp")])
-
-(define_expand "absdf2"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	(abs:DF (match_operand:DF 1 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT && ((TARGET_FPRS && TARGET_DOUBLE_FLOAT) || TARGET_E500_DOUBLE)"
-  "")
-
-(define_insn "*absdf2_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(abs:DF (match_operand:DF 1 "gpc_reg_operand" "d")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && !VECTOR_UNIT_VSX_P (DFmode)"
-  "fabs %0,%1"
-  [(set_attr "type" "fp")])
-
-(define_insn "*nabsdf2_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(neg:DF (abs:DF (match_operand:DF 1 "gpc_reg_operand" "d"))))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && !VECTOR_UNIT_VSX_P (DFmode)"
-  "fnabs %0,%1"
-  [(set_attr "type" "fp")])
-
-(define_expand "adddf3"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	(plus:DF (match_operand:DF 1 "gpc_reg_operand" "")
-		 (match_operand:DF 2 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT && ((TARGET_FPRS && TARGET_DOUBLE_FLOAT) || TARGET_E500_DOUBLE)"
-  "")
-
-(define_insn "*adddf3_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(plus:DF (match_operand:DF 1 "gpc_reg_operand" "%d")
-		 (match_operand:DF 2 "gpc_reg_operand" "d")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && !VECTOR_UNIT_VSX_P (DFmode)"
-  "fadd %0,%1,%2"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_addsub_d")])
-
-(define_expand "subdf3"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	(minus:DF (match_operand:DF 1 "gpc_reg_operand" "")
-		  (match_operand:DF 2 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT && ((TARGET_FPRS && TARGET_DOUBLE_FLOAT) || TARGET_E500_DOUBLE)"
-  "")
-
-(define_insn "*subdf3_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(minus:DF (match_operand:DF 1 "gpc_reg_operand" "d")
-		  (match_operand:DF 2 "gpc_reg_operand" "d")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && !VECTOR_UNIT_VSX_P (DFmode)"
-  "fsub %0,%1,%2"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_addsub_d")])
-
-(define_expand "muldf3"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	(mult:DF (match_operand:DF 1 "gpc_reg_operand" "")
-		 (match_operand:DF 2 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT && ((TARGET_FPRS && TARGET_DOUBLE_FLOAT) || TARGET_E500_DOUBLE)"
-  "")
-
-(define_insn "*muldf3_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(mult:DF (match_operand:DF 1 "gpc_reg_operand" "%d")
-		 (match_operand:DF 2 "gpc_reg_operand" "d")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && !VECTOR_UNIT_VSX_P (DFmode)"
-  "fmul %0,%1,%2"
-  [(set_attr "type" "dmul")
-   (set_attr "fp_type" "fp_mul_d")])
-
-(define_expand "divdf3"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	(div:DF (match_operand:DF 1 "gpc_reg_operand" "")
-		(match_operand:DF 2 "gpc_reg_operand" "")))]
-  "TARGET_HARD_FLOAT
-   && ((TARGET_FPRS && TARGET_DOUBLE_FLOAT) || TARGET_E500_DOUBLE)
-   && !TARGET_SIMPLE_FPU"
-  "")
-
-(define_insn "*divdf3_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(div:DF (match_operand:DF 1 "gpc_reg_operand" "d")
-		(match_operand:DF 2 "gpc_reg_operand" "d")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT && !TARGET_SIMPLE_FPU
-   && !VECTOR_UNIT_VSX_P (DFmode)"
-  "fdiv %0,%1,%2"
-  [(set_attr "type" "ddiv")])
-
-(define_insn "*fred_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=f")
-	(unspec:DF [(match_operand:DF 1 "gpc_reg_operand" "f")] UNSPEC_FRES))]
-  "TARGET_FRE && !VECTOR_UNIT_VSX_P (DFmode)"
-  "fre %0,%1"
-  [(set_attr "type" "fp")])
-
-(define_insn "*rsqrtdf_internal1"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(unspec:DF [(match_operand:DF 1 "gpc_reg_operand" "d")]
-		   UNSPEC_RSQRT))]
-  "TARGET_FRSQRTE && !VECTOR_UNIT_VSX_P (DFmode)"
-  "frsqrte %0,%1"
-  [(set_attr "type" "fp")])
-
-; builtin fma support
-(define_insn "*fmadf4_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=f")
-	(fma:DF (match_operand:DF 1 "gpc_reg_operand" "f")
-		(match_operand:DF 2 "gpc_reg_operand" "f")
-		(match_operand:DF 3 "gpc_reg_operand" "f")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && VECTOR_UNIT_NONE_P (DFmode)"
-  "fmadd %0,%1,%2,%3"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_maddsub_d")])
-
-(define_insn "*fmsdf4_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=f")
-	(fma:DF (match_operand:DF 1 "gpc_reg_operand" "f")
-		(match_operand:DF 2 "gpc_reg_operand" "f")
-		(neg:DF (match_operand:DF 3 "gpc_reg_operand" "f"))))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && VECTOR_UNIT_NONE_P (DFmode)"
-  "fmsub %0,%1,%2,%3"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_maddsub_d")])
-
-(define_insn "*nfmadf4_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=f")
-	(neg:DF (fma:DF (match_operand:DF 1 "gpc_reg_operand" "f")
-			(match_operand:DF 2 "gpc_reg_operand" "f")
-			(match_operand:DF 3 "gpc_reg_operand" "f"))))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && VECTOR_UNIT_NONE_P (DFmode)"
-  "fnmadd %0,%1,%2,%3"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_maddsub_d")])
-
-(define_insn "*nfmsdf4_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=f")
-	(neg:DF (fma:DF (match_operand:DF 1 "gpc_reg_operand" "f")
-			(match_operand:DF 2 "gpc_reg_operand" "f")
-			(neg:DF (match_operand:DF 3 "gpc_reg_operand" "f")))))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && VECTOR_UNIT_NONE_P (DFmode)"
-  "fnmsub %0,%1,%2,%3"
-  [(set_attr "type" "fp")
-   (set_attr "fp_type" "fp_maddsub_d")])
-
-(define_expand "sqrtdf2"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	(sqrt:DF (match_operand:DF 1 "gpc_reg_operand" "")))]
-  "TARGET_PPC_GPOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT"
-  "")
-
-(define_insn "*sqrtdf2_fpr"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(sqrt:DF (match_operand:DF 1 "gpc_reg_operand" "d")))]
-  "TARGET_PPC_GPOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && !VECTOR_UNIT_VSX_P (DFmode)"
-  "fsqrt %0,%1"
-  [(set_attr "type" "dsqrt")])
-
-;; The conditional move instructions allow us to perform max and min
-;; operations even when
-
-(define_expand "smaxdf3"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	(if_then_else:DF (ge (match_operand:DF 1 "gpc_reg_operand" "")
-			     (match_operand:DF 2 "gpc_reg_operand" ""))
-			 (match_dup 1)
-			 (match_dup 2)))]
-  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT 
-   && !flag_trapping_math"
-  "{ rs6000_emit_minmax (operands[0], SMAX, operands[1], operands[2]); DONE;}")
-
-(define_expand "smindf3"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	(if_then_else:DF (ge (match_operand:DF 1 "gpc_reg_operand" "")
-			     (match_operand:DF 2 "gpc_reg_operand" ""))
-			 (match_dup 2)
-			 (match_dup 1)))]
-  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT 
-   && !flag_trapping_math"
-  "{ rs6000_emit_minmax (operands[0], SMIN, operands[1], operands[2]); DONE;}")
-
-(define_split
-  [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	(match_operator:DF 3 "min_max_operator"
-	 [(match_operand:DF 1 "gpc_reg_operand" "")
-	  (match_operand:DF 2 "gpc_reg_operand" "")]))]
-  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT 
-   && !flag_trapping_math"
-  [(const_int 0)]
-  "
-{ rs6000_emit_minmax (operands[0], GET_CODE (operands[3]),
-		      operands[1], operands[2]);
-  DONE;
-}")
-
-(define_expand "movdfcc"
-   [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	 (if_then_else:DF (match_operand 1 "comparison_operator" "")
-			  (match_operand:DF 2 "gpc_reg_operand" "")
-			  (match_operand:DF 3 "gpc_reg_operand" "")))]
-  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT"
-  "
-{
-  if (rs6000_emit_cmove (operands[0], operands[1], operands[2], operands[3]))
-    DONE;
-  else
-    FAIL;
-}")
-
-(define_insn "*fseldfdf4"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(if_then_else:DF (ge (match_operand:DF 1 "gpc_reg_operand" "d")
-			     (match_operand:DF 4 "zero_fp_constant" "F"))
-			 (match_operand:DF 2 "gpc_reg_operand" "d")
-			 (match_operand:DF 3 "gpc_reg_operand" "d")))]
-  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT"
+(define_insn "*fsel<mode>df4"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>")
+	(if_then_else:SFDF (ge (match_operand:DF 1 "gpc_reg_operand" "d")
+			       (match_operand:DF 4 "zero_fp_constant" "F"))
+			 (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>")
+			 (match_operand:SFDF 3 "gpc_reg_operand" "<Ff>")))]
+  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT
+   && TARGET_DOUBLE_FLOAT"
   "fsel %0,%1,%2,%3"
   [(set_attr "type" "fp")])
 
-(define_insn "*fselsfdf4"
-  [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(if_then_else:DF (ge (match_operand:SF 1 "gpc_reg_operand" "f")
-			     (match_operand:SF 4 "zero_fp_constant" "F"))
-			 (match_operand:DF 2 "gpc_reg_operand" "d")
-			 (match_operand:DF 3 "gpc_reg_operand" "d")))]
-  "TARGET_PPC_GFXOPT && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT && TARGET_SINGLE_FLOAT"
-  "fsel %0,%1,%2,%3"
-  [(set_attr "type" "fp")])
 
 ;; Conversions to and from floating-point.
 
 ; We don't define lfiwax/lfiwzx with the normal definition, because we
 ; don't want to support putting SImode in FPR registers.
 (define_insn "lfiwax"
-  [(set (match_operand:DI 0 "gpc_reg_operand" "=d,wm,!wm")
+  [(set (match_operand:DI 0 "gpc_reg_operand" "=d,wu,wm")
 	(unspec:DI [(match_operand:SI 1 "reg_or_indexed_operand" "Z,Z,r")]
 		   UNSPEC_LFIWAX))]
   "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT && TARGET_LFIWAX"
@@ -5811,11 +5609,11 @@  (define_insn_and_split "floatsi<mode>2_l
    (set_attr "type" "fpload")])
 
 (define_insn_and_split "floatsi<mode>2_lfiwax_mem"
-  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=d,<rreg2>")
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=d,<rreg2>,wu")
 	(float:SFDF
 	 (sign_extend:DI
-	  (match_operand:SI 1 "memory_operand" "Z,Z"))))
-   (clobber (match_scratch:DI 2 "=0,d"))]
+	  (match_operand:SI 1 "memory_operand" "Z,Z,Z"))))
+   (clobber (match_scratch:DI 2 "=0,d,wa"))]
   "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT && TARGET_LFIWAX
    && <SI_CONVERT_FP>"
   "#"
@@ -5834,7 +5632,7 @@  (define_insn_and_split "floatsi<mode>2_l
    (set_attr "type" "fpload")])
 
 (define_insn "lfiwzx"
-  [(set (match_operand:DI 0 "gpc_reg_operand" "=d,wm,!wm")
+  [(set (match_operand:DI 0 "gpc_reg_operand" "=d,wu,wm")
 	(unspec:DI [(match_operand:SI 1 "reg_or_indexed_operand" "Z,Z,r")]
 		   UNSPEC_LFIWZX))]
   "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT && TARGET_LFIWZX"
@@ -5886,11 +5684,11 @@  (define_insn_and_split "floatunssi<mode>
    (set_attr "type" "fpload")])
 
 (define_insn_and_split "floatunssi<mode>2_lfiwzx_mem"
-  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=d,<rreg2>")
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=d,<rreg2>,wu")
 	(unsigned_float:SFDF
 	 (zero_extend:DI
-	  (match_operand:SI 1 "memory_operand" "Z,Z"))))
-   (clobber (match_scratch:DI 2 "=0,d"))]
+	  (match_operand:SI 1 "memory_operand" "Z,Z,Z"))))
+   (clobber (match_scratch:DI 2 "=0,d,wa"))]
   "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT && TARGET_LFIWZX
    && <SI_CONVERT_FP>"
   "#"
@@ -6384,56 +6182,38 @@  (define_insn "lrint<mode>di2"
   [(set (match_operand:DI 0 "gpc_reg_operand" "=d")
 	(unspec:DI [(match_operand:SFDF 1 "gpc_reg_operand" "<rreg2>")]
 		   UNSPEC_FCTID))]
-  "TARGET_FPRND && TARGET_HARD_FLOAT && TARGET_FPRS && <TARGET_FLOAT>"
+  "TARGET_<MODE>_FPR && TARGET_FPRND"
   "fctid %0,%1"
   [(set_attr "type" "fp")])
 
-(define_expand "btrunc<mode>2"
-  [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
-	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "")]
-		     UNSPEC_FRIZ))]
-  "TARGET_FPRND && TARGET_HARD_FLOAT && TARGET_FPRS && <TARGET_FLOAT>"
-  "")
-
-(define_insn "*btrunc<mode>2_fpr"
-  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<rreg2>")
-	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "<rreg2>")]
+(define_insn "btrunc<mode>2"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>")]
 		     UNSPEC_FRIZ))]
-  "TARGET_FPRND && TARGET_HARD_FLOAT && TARGET_FPRS && <TARGET_FLOAT>
-   && !VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "friz %0,%1"
+  "TARGET_<MODE>_FPR && TARGET_FPRND"
+  "@
+   friz %0,%1
+   xsrdpiz %x0,%x1"
   [(set_attr "type" "fp")])
 
-(define_expand "ceil<mode>2"
-  [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
-	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "")]
-		     UNSPEC_FRIP))]
-  "TARGET_FPRND && TARGET_HARD_FLOAT && TARGET_FPRS && <TARGET_FLOAT>"
-  "")
-
-(define_insn "*ceil<mode>2_fpr"
-  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<rreg2>")
-	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "<rreg2>")]
+(define_insn "ceil<mode>2"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>")]
 		     UNSPEC_FRIP))]
-  "TARGET_FPRND && TARGET_HARD_FLOAT && TARGET_FPRS && <TARGET_FLOAT>
-   && !VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "frip %0,%1"
+  "TARGET_<MODE>_FPR && TARGET_FPRND"
+  "@
+   frip %0,%1
+   xsrdpip %x0,%x1"
   [(set_attr "type" "fp")])
 
-(define_expand "floor<mode>2"
-  [(set (match_operand:SFDF 0 "gpc_reg_operand" "")
-	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "")]
-		     UNSPEC_FRIM))]
-  "TARGET_FPRND && TARGET_HARD_FLOAT && TARGET_FPRS && <TARGET_FLOAT>"
-  "")
-
-(define_insn "*floor<mode>2_fpr"
-  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<rreg2>")
-	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "<rreg2>")]
+(define_insn "floor<mode>2"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>")
+	(unspec:SFDF [(match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>")]
 		     UNSPEC_FRIM))]
-  "TARGET_FPRND && TARGET_HARD_FLOAT && TARGET_FPRS && <TARGET_FLOAT>
-   && !VECTOR_UNIT_VSX_P (<MODE>mode)"
-  "frim %0,%1"
+  "TARGET_<MODE>_FPR && TARGET_FPRND"
+  "@
+   frim %0,%1
+   xsrdpim %x0,%x1"
   [(set_attr "type" "fp")])
 
 ;; No VSX equivalent to frin
@@ -9288,8 +9068,8 @@  (define_split
 }")
 
 (define_insn "mov<mode>_hardfloat"
-  [(set (match_operand:FMOVE32 0 "nonimmediate_operand" "=!r,!r,m,f,wa,wa,<f32_lr>,<f32_sm>,wm,Z,?<f32_dm>,?r,*c*l,!r,*h,!r,!r")
-	(match_operand:FMOVE32 1 "input_operand" "r,m,r,f,wa,j,<f32_lm>,<f32_sr>,Z,wm,r,<f32_dm>,r,h,0,G,Fn"))]
+  [(set (match_operand:FMOVE32 0 "nonimmediate_operand" "=!r,!r,m,f,wa,wa,<f32_lr>,<f32_sm>,wu,Z,?<f32_dm>,?r,*c*l,!r,*h,!r,!r")
+	(match_operand:FMOVE32 1 "input_operand" "r,m,r,f,wa,j,<f32_lm>,<f32_sr>,Z,wu,r,<f32_dm>,r,h,0,G,Fn"))]
   "(gpc_reg_operand (operands[0], <MODE>mode)
    || gpc_reg_operand (operands[1], <MODE>mode))
    && (TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT)"
@@ -9490,8 +9270,8 @@  (define_split
 ;; reloading.
 
 (define_insn "*mov<mode>_hardfloat32"
-  [(set (match_operand:FMOVE64 0 "nonimmediate_operand" "=m,d,d,ws,?wa,Z,?Z,ws,?wa,wa,Y,r,!r,!r,!r,!r")
-	(match_operand:FMOVE64 1 "input_operand" "d,m,d,Z,Z,ws,wa,ws,wa,j,r,Y,r,G,H,F"))]
+  [(set (match_operand:FMOVE64 0 "nonimmediate_operand" "=m,d,d,wv,Z,wa,wa,Y,r,!r,!r,!r,!r")
+	(match_operand:FMOVE64 1 "input_operand" "d,m,d,Z,wv,wa,j,r,Y,r,G,H,F"))]
   "! TARGET_POWERPC64 && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT 
    && (gpc_reg_operand (operands[0], <MODE>mode)
        || gpc_reg_operand (operands[1], <MODE>mode))"
@@ -9500,11 +9280,8 @@  (define_insn "*mov<mode>_hardfloat32"
    lfd%U1%X1 %0,%1
    fmr %0,%1
    lxsd%U1x %x0,%y1
-   lxsd%U1x %x0,%y1
-   stxsd%U0x %x1,%y0
    stxsd%U0x %x1,%y0
    xxlor %x0,%x1,%x1
-   xxlor %x0,%x1,%x1
    xxlxor %x0,%x0,%x0
    #
    #
@@ -9533,27 +9310,18 @@  (define_insn "*mov<mode>_hardfloat32"
 	 (const_string "fpload_ux")
 	 (const_string "fpload"))
        (if_then_else
-	 (match_test "update_indexed_address_mem (operands[1], VOIDmode)")
-	 (const_string "fpload_ux")
-	 (const_string "fpload"))
-       (if_then_else
-	 (match_test "update_indexed_address_mem (operands[0], VOIDmode)")
-	 (const_string "fpstore_ux")
-	 (const_string "fpstore"))
-       (if_then_else
 	 (match_test "update_indexed_address_mem (operands[0], VOIDmode)")
 	 (const_string "fpstore_ux")
 	 (const_string "fpstore"))
        (const_string "vecsimple")
        (const_string "vecsimple")
-       (const_string "vecsimple")
        (const_string "store")
        (const_string "load")
        (const_string "two")
        (const_string "fp")
        (const_string "fp")
        (const_string "*")])
-   (set_attr "length" "4,4,4,4,4,4,4,4,4,4,8,8,8,8,12,16")])
+   (set_attr "length" "4,4,4,4,4,4,4,8,8,8,8,12,16")])
 
 (define_insn "*mov<mode>_softfloat32"
   [(set (match_operand:FMOVE64 0 "nonimmediate_operand" "=Y,r,r,r,r,r")
@@ -9570,8 +9338,8 @@  (define_insn "*mov<mode>_softfloat32"
 ; ld/std require word-aligned displacements -> 'Y' constraint.
 ; List Y->r and r->Y before r->r for reload.
 (define_insn "*mov<mode>_hardfloat64"
-  [(set (match_operand:FMOVE64 0 "nonimmediate_operand" "=m,d,d,ws,?wa,Z,?Z,ws,?wa,wa,Y,r,!r,*c*l,!r,*h,!r,!r,!r,r,wg,r,wm")
-	(match_operand:FMOVE64 1 "input_operand" "d,m,d,Z,Z,ws,wa,ws,wa,j,r,Y,r,r,h,0,G,H,F,wg,r,wm,r"))]
+  [(set (match_operand:FMOVE64 0 "nonimmediate_operand" "=m,d,d,wv,Z,wa,wa,Y,r,!r,*c*l,!r,*h,!r,!r,!r,r,wg,r,wm")
+	(match_operand:FMOVE64 1 "input_operand" "d,m,d,Z,wv,wa,j,r,Y,r,r,h,0,G,H,F,wg,r,wm,r"))]
   "TARGET_POWERPC64 && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
    && (gpc_reg_operand (operands[0], <MODE>mode)
        || gpc_reg_operand (operands[1], <MODE>mode))"
@@ -9580,10 +9348,7 @@  (define_insn "*mov<mode>_hardfloat64"
    lfd%U1%X1 %0,%1
    fmr %0,%1
    lxsd%U1x %x0,%y1
-   lxsd%U1x %x0,%y1
    stxsd%U0x %x1,%y0
-   stxsd%U0x %x1,%y0
-   xxlor %x0,%x1,%x1
    xxlor %x0,%x1,%x1
    xxlxor %x0,%x0,%x0
    std%U0%X0 %1,%0
@@ -9620,18 +9385,9 @@  (define_insn "*mov<mode>_hardfloat64"
 	 (const_string "fpload_ux")
 	 (const_string "fpload"))
        (if_then_else
-	 (match_test "update_indexed_address_mem (operands[1], VOIDmode)")
-	 (const_string "fpload_ux")
-	 (const_string "fpload"))
-       (if_then_else
 	 (match_test "update_indexed_address_mem (operands[0], VOIDmode)")
 	 (const_string "fpstore_ux")
 	 (const_string "fpstore"))
-       (if_then_else
-	 (match_test "update_indexed_address_mem (operands[0], VOIDmode)")
-	 (const_string "fpstore_ux")
-	 (const_string "fpstore"))
-       (const_string "vecsimple")
        (const_string "vecsimple")
        (const_string "vecsimple")
        (if_then_else
@@ -9659,7 +9415,7 @@  (define_insn "*mov<mode>_hardfloat64"
        (const_string "mffgpr")
        (const_string "mftgpr")
        (const_string "mffgpr")])
-   (set_attr "length" "4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,8,12,16,4,4,4,4")])
+   (set_attr "length" "4,4,4,4,4,4,4,4,4,4,4,4,4,8,12,16,4,4,4,4")])
 
 (define_insn "*mov<mode>_softfloat64"
   [(set (match_operand:FMOVE64 0 "nonimmediate_operand" "=Y,r,r,cl,r,r,r,r,*h")
@@ -10322,8 +10078,8 @@  (define_split
 { rs6000_split_multireg_move (operands[0], operands[1]); DONE; })
 
 (define_insn "*movdi_internal64"
-  [(set (match_operand:DI 0 "nonimmediate_operand" "=Y,r,r,r,r,r,?m,?*d,?*d,?Z,?wa,?wa,r,*h,*h,?wa,r,?*wg,r,?*wm")
-	(match_operand:DI 1 "input_operand" "r,Y,r,I,L,nF,d,m,d,wa,Z,wa,*h,r,0,O,*wg,r,*wm,r"))]
+  [(set (match_operand:DI 0 "nonimmediate_operand" "=Y,r,r,r,r,r,?m,?*d,?*d,?Z,?wv,?wa,r,*h,*h,?wa,r,?*wg,r,?*wm")
+	(match_operand:DI 1 "input_operand" "r,Y,r,I,L,nF,d,m,d,wv,Z,wa,*h,r,0,O,*wg,r,*wm,r"))]
   "TARGET_POWERPC64
    && (gpc_reg_operand (operands[0], DImode)
        || gpc_reg_operand (operands[1], DImode))"
@@ -13339,23 +13095,6 @@  (define_split
   [(set (match_dup 3) (compare:CCUNS (match_dup 1) (match_dup 2)))
    (set (match_dup 0) (plus:SI (match_dup 1) (match_dup 4)))])
 
-(define_insn "*cmpsf_internal1"
-  [(set (match_operand:CCFP 0 "cc_reg_operand" "=y")
-	(compare:CCFP (match_operand:SF 1 "gpc_reg_operand" "f")
-		      (match_operand:SF 2 "gpc_reg_operand" "f")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT"
-  "fcmpu %0,%1,%2"
-  [(set_attr "type" "fpcompare")])
-
-(define_insn "*cmpdf_internal1"
-  [(set (match_operand:CCFP 0 "cc_reg_operand" "=y")
-	(compare:CCFP (match_operand:DF 1 "gpc_reg_operand" "d")
-		      (match_operand:DF 2 "gpc_reg_operand" "d")))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && !VECTOR_UNIT_VSX_P (DFmode)"
-  "fcmpu %0,%1,%2"
-  [(set_attr "type" "fpcompare")])
-
 ;; Only need to compare second words if first words equal
 (define_insn "*cmptf_internal1"
   [(set (match_operand:CCFP 0 "cc_reg_operand" "=y")
@@ -15638,8 +15377,9 @@  (define_insn "bpermd_<mode>"
   [(set_attr "type" "popcnt")])
 
 
-;; Builtin fma support.  Handle 
-;; Note that the conditions for expansion are in the FMA_F iterator.
+;; Builtin fma support.  Handle VSX instructions needing to overlap the
+;; desination with one of the two inputs.  Note that the conditions for
+;; expansion are in the FMA_F iterator.
 
 (define_expand "fma<mode>4"
   [(set (match_operand:FMA_F 0 "register_operand" "")
@@ -15650,6 +15390,20 @@  (define_expand "fma<mode>4"
   ""
   "")
 
+(define_insn "*fma<mode>4_fpr"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>,<Fv>")
+	(fma:SFDF
+	  (match_operand:SFDF 1 "gpc_reg_operand" "%<Ff>,<Fv>,<Fv>")
+	  (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>,<Fv>,0")
+	  (match_operand:SFDF 3 "gpc_reg_operand" "<Ff>,0,<Fv>")))]
+  "TARGET_<MODE>_FPR"
+  "@
+   fmadd<Ftrad> %0,%1,%2,%3
+   xsmadda<Fvsx> %x0,%x1,%x2
+   xsmaddm<Fvsx> %x0,%x1,%x3"
+  [(set_attr "type" "fp")
+   (set_attr "fp_type" "fp_maddsub_<Fs>")])
+
 ; Altivec only has fma and nfms.
 (define_expand "fms<mode>4"
   [(set (match_operand:FMA_F 0 "register_operand" "")
@@ -15660,6 +15414,20 @@  (define_expand "fms<mode>4"
   "!VECTOR_UNIT_ALTIVEC_P (<MODE>mode)"
   "")
 
+(define_insn "*fms<mode>4_fpr"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>,<Fv>")
+	(fma:SFDF
+	 (match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>,<Fv>")
+	 (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>,<Fv>,0")
+	 (neg:SFDF (match_operand:SFDF 3 "gpc_reg_operand" "<Ff>,0,<Fv>"))))]
+  "TARGET_<MODE>_FPR"
+  "@
+   fmsub<Ftrad> %0,%1,%2,%3
+   xsmsuba<Fvsx> %x0,%x1,%x2
+   xsmsubm<Fvsx> %x0,%x1,%x3"
+  [(set_attr "type" "fp")
+   (set_attr "fp_type" "fp_maddsub_<Fs>")])
+
 ;; If signed zeros are ignored, -(a * b - c) = -a * b + c.
 (define_expand "fnma<mode>4"
   [(set (match_operand:FMA_F 0 "register_operand" "")
@@ -15693,6 +15461,21 @@  (define_expand "nfma<mode>4"
   "!VECTOR_UNIT_ALTIVEC_P (<MODE>mode)"
   "")
 
+(define_insn "*nfma<mode>4_fpr"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>,<Fv>")
+	(neg:SFDF
+	 (fma:SFDF
+	  (match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>,<Fv>")
+	  (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>,<Fv>,0")
+	  (match_operand:SFDF 3 "gpc_reg_operand" "<Ff>,0,<Fv>"))))]
+  "TARGET_<MODE>_FPR"
+  "@
+   fnmadd<Ftrad> %0,%1,%2,%3
+   xsnmadda<Fvsx> %x0,%x1,%x2
+   xsnmaddm<Fvsx> %x0,%x1,%x3"
+  [(set_attr "type" "fp")
+   (set_attr "fp_type" "fp_maddsub_<Fs>")])
+
 ; Not an official optab name, but used from builtins.
 (define_expand "nfms<mode>4"
   [(set (match_operand:FMA_F 0 "register_operand" "")
@@ -15704,6 +15487,24 @@  (define_expand "nfms<mode>4"
   ""
   "")
 
+(define_insn "*nfmssf4_fpr"
+  [(set (match_operand:SFDF 0 "gpc_reg_operand" "=<Ff>,<Fv>,<Fv>")
+	(neg:SFDF
+	 (fma:SFDF
+	  (match_operand:SFDF 1 "gpc_reg_operand" "<Ff>,<Fv>,<Fv>")
+	  (match_operand:SFDF 2 "gpc_reg_operand" "<Ff>,<Fv>,0")
+	  (neg:SFDF
+	   (match_operand:SFDF 3 "gpc_reg_operand" "<Ff>,0,<Fv>")))))]
+  "TARGET_<MODE>_FPR"
+  "@
+   fnmsub<Ftrad> %0,%1,%2,%3
+   xsnmsuba<Fvsx> %x0,%x1,%x2
+   xsnmsubm<Fvsx> %x0,%x1,%x3"
+  [(set_attr "type" "fp")
+   (set_attr "fp_type" "fp_maddsub_<Fs>")])
+
+
+;; Timebase builtins
 (define_expand "rs6000_get_timebase"
   [(use (match_operand:DI 0 "gpc_reg_operand" ""))]
   ""
Index: gcc/doc/md.texi
===================================================================
--- gcc/doc/md.texi	(revision 201924)
+++ gcc/doc/md.texi	(working copy)
@@ -2067,40 +2067,52 @@  Floating point register (containing 32-b
 Altivec vector register
 
 @item wa
-Any VSX register
+Any VSX register if the -mvsx option was used or NO_REGS.
 
 @item wd
-VSX vector register to hold vector double data
+VSX vector register to hold vector double data or NO_REGS.
 
 @item wf
-VSX vector register to hold vector float data
+VSX vector register to hold vector float data or NO_REGS.
 
 @item wg
-If @option{-mmfpgpr} was used, a floating point register
+If @option{-mmfpgpr} was used, a floating point register or NO_REGS.
 
 @item wl
-If the LFIWAX instruction is enabled, a floating point register
+Floating point register if the LFIWAX instruction is enabled or NO_REGS.
 
 @item wm
-If direct moves are enabled, a VSX register.
+VSX register if direct move instructions are enabled, or NO_REGS.
 
 @item wn
-No register.
+No register (NO_REGS).
 
 @item wr
-General purpose register if 64-bit mode is used
+General purpose register if 64-bit instructions are enabled or NO_REGS.
 
 @item ws
-VSX vector register to hold scalar float data
+VSX vector register to hold scalar double values or NO_REGS.
 
 @item wt
-VSX vector register to hold 128 bit integer
+VSX vector register to hold 128 bit integer or NO_REGS.
+
+@item wu
+Altivec register to use for float/32-bit int loads/stores  or NO_REGS.
+
+@item wv
+Altivec register to use for double loads/stores  or NO_REGS.
+
+@item ww
+FP or VSX register to perform float operations under @option{-mvsx} or NO_REGS.
 
 @item wx
-If the STFIWX instruction is enabled, a floating point register
+Floating point register if the STFIWX instruction is enabled or NO_REGS.
+
+@item wy
+VSX vector register to hold scalar float values or NO_REGS.
 
 @item wz
-If the LFIWZX instruction is enabled, a floating point register
+Floating point register if the LFIWZX instruction is enabled or NO_REGS.
 
 @item wQ
 A memory address that will work with the @code{lq} and @code{stq}
Index: gcc/testsuite/gcc.target/powerpc/ppc-target-2.c
===================================================================
--- gcc/testsuite/gcc.target/powerpc/ppc-target-2.c	(revision 201924)
+++ gcc/testsuite/gcc.target/powerpc/ppc-target-2.c	(working copy)
@@ -5,8 +5,7 @@ 
 /* { dg-final { scan-assembler-times "fabs" 3 } } */
 /* { dg-final { scan-assembler-times "fnabs" 3 } } */
 /* { dg-final { scan-assembler-times "fsel" 3 } } */
-/* { dg-final { scan-assembler-times "fcpsgn" 3 } } */
-/* { dg-final { scan-assembler-times "xscpsgndp" 1 } } */
+/* { dg-final { scan-assembler-times "fcpsgn\|xscpsgndp" 4 } } */
 
 /* fabs/fnabs/fsel */
 double normal1 (double a, double b) { return __builtin_copysign (a, b); }
Index: gcc/testsuite/gcc.target/powerpc/recip-3.c
===================================================================
--- gcc/testsuite/gcc.target/powerpc/recip-3.c	(revision 201924)
+++ gcc/testsuite/gcc.target/powerpc/recip-3.c	(working copy)
@@ -1,14 +1,14 @@ 
 /* { dg-do compile { target { { powerpc*-*-* } && { ! powerpc*-apple-darwin* } } } } */
 /* { dg-require-effective-target powerpc_fprs } */
 /* { dg-options "-O2 -mrecip -ffast-math -mcpu=power7" } */
-/* { dg-final { scan-assembler-times "xsrsqrtedp" 1 } } */
+/* { dg-final { scan-assembler-times "xsrsqrtedp\|frsqrte\ " 1 } } */
 /* { dg-final { scan-assembler-times "xsmsub.dp\|fmsub\ " 1 } } */
-/* { dg-final { scan-assembler-times "xsmuldp" 4 } } */
+/* { dg-final { scan-assembler-times "xsmuldp\|fmul\ " 4 } } */
 /* { dg-final { scan-assembler-times "xsnmsub.dp\|fnmsub\ " 2 } } */
-/* { dg-final { scan-assembler-times "frsqrtes" 1 } } */
-/* { dg-final { scan-assembler-times "fmsubs" 1 } } */
-/* { dg-final { scan-assembler-times "fmuls" 2 } } */
-/* { dg-final { scan-assembler-times "fnmsubs" 1 } } */
+/* { dg-final { scan-assembler-times "xsrsqrtesp\|frsqrtes" 1 } } */
+/* { dg-final { scan-assembler-times "xsmsub.sp\|fmsubs" 1 } } */
+/* { dg-final { scan-assembler-times "xsmulsp\|fmuls" 2 } } */
+/* { dg-final { scan-assembler-times "xsnmsub.sp\|fnmsubs" 1 } } */
 
 double
 rsqrt_d (double a)
Index: gcc/testsuite/gcc.target/powerpc/pr42747.c
===================================================================
--- gcc/testsuite/gcc.target/powerpc/pr42747.c	(revision 201924)
+++ gcc/testsuite/gcc.target/powerpc/pr42747.c	(working copy)
@@ -5,4 +5,4 @@ 
 
 double foo (double x) { return __builtin_sqrt (x); }
 
-/* { dg-final { scan-assembler "xssqrtdp" } } */
+/* { dg-final { scan-assembler "xssqrtdp\|fsqrt" } } */
Index: gcc/testsuite/gcc.target/powerpc/recip-5.c
===================================================================
--- gcc/testsuite/gcc.target/powerpc/recip-5.c	(revision 201924)
+++ gcc/testsuite/gcc.target/powerpc/recip-5.c	(working copy)
@@ -4,12 +4,12 @@ 
 /* { dg-options "-O3 -ftree-vectorize -mrecip=all -ffast-math -mcpu=power7 -fno-unroll-loops" } */
 /* { dg-final { scan-assembler-times "xvredp" 4 } } */
 /* { dg-final { scan-assembler-times "xvresp" 5 } } */
-/* { dg-final { scan-assembler-times "xsredp" 2 } } */
-/* { dg-final { scan-assembler-times "fres" 2 } } */
-/* { dg-final { scan-assembler-times "fmuls" 2 } } */
-/* { dg-final { scan-assembler-times "fnmsubs" 2 } } */
-/* { dg-final { scan-assembler-times "xsmuldp" 2 } } */
-/* { dg-final { scan-assembler-times "xsnmsub.dp" 4 } } */
+/* { dg-final { scan-assembler-times "xsredp\|fre\ " 2 } } */
+/* { dg-final { scan-assembler-times "fres\|xsresp" 2 } } */
+/* { dg-final { scan-assembler-times "fmuls\|xsmulsp" 2 } } */
+/* { dg-final { scan-assembler-times "fnmsubs\|xsnmsub.sp" 2 } } */
+/* { dg-final { scan-assembler-times "xsmuldp\|fmul\ " 2 } } */
+/* { dg-final { scan-assembler-times "xsnmsub.dp\|fnmsub\ " 4 } } */
 /* { dg-final { scan-assembler-times "xvmulsp" 7 } } */
 /* { dg-final { scan-assembler-times "xvnmsub.sp" 5 } } */
 /* { dg-final { scan-assembler-times "xvmuldp" 6 } } */
Index: gcc/testsuite/gcc.target/powerpc/vsx-builtin-3.c
===================================================================
--- gcc/testsuite/gcc.target/powerpc/vsx-builtin-3.c	(revision 201924)
+++ gcc/testsuite/gcc.target/powerpc/vsx-builtin-3.c	(working copy)
@@ -16,9 +16,9 @@ 
 /* { dg-final { scan-assembler "xvrspiz" } } */
 /* { dg-final { scan-assembler "xsrdpi" } } */
 /* { dg-final { scan-assembler "xsrdpic" } } */
-/* { dg-final { scan-assembler "xsrdpim" } } */
-/* { dg-final { scan-assembler "xsrdpip" } } */
-/* { dg-final { scan-assembler "xsrdpiz" } } */
+/* { dg-final { scan-assembler "xsrdpim\|frim" } } */
+/* { dg-final { scan-assembler "xsrdpip\|frip" } } */
+/* { dg-final { scan-assembler "xsrdpiz\|friz" } } */
 /* { dg-final { scan-assembler "xsmaxdp" } } */
 /* { dg-final { scan-assembler "xsmindp" } } */
 /* { dg-final { scan-assembler "xxland" } } */
Index: gcc/testsuite/gcc.target/powerpc/ppc-target-1.c
===================================================================
--- gcc/testsuite/gcc.target/powerpc/ppc-target-1.c	(revision 201924)
+++ gcc/testsuite/gcc.target/powerpc/ppc-target-1.c	(working copy)
@@ -5,8 +5,7 @@ 
 /* { dg-final { scan-assembler-times "fabs" 3 } } */
 /* { dg-final { scan-assembler-times "fnabs" 3 } } */
 /* { dg-final { scan-assembler-times "fsel" 3 } } */
-/* { dg-final { scan-assembler-times "fcpsgn" 3 } } */
-/* { dg-final { scan-assembler-times "xscpsgndp" 1 } } */
+/* { dg-final { scan-assembler-times "fcpsgn\|xscpsgndp" 4 } } */
 
 double normal1 (double, double);
 double power5  (double, double) __attribute__((__target__("cpu=power5")));