Patchwork [RFC] KVM: PPC: Book3S HV: Reserve POWER8 space in get/set_one_reg

login
register
mail settings
Submitter Michael Neuling
Date Aug. 30, 2013, 6:09 a.m.
Message ID <8097.1377842940@ale.ozlabs.ibm.com>
Download mbox | patch
Permalink /patch/271151/
State New
Headers show

Comments

Michael Neuling - Aug. 30, 2013, 6:09 a.m.
Alex,

This reserves space in get/set_one_reg ioctl for the extra guest state
needed for POWER8.  It doesn't implement these at all, it just reserves
them so that the ABI is defined now.  

A few things to note here:

- POWER8 has 6 PMCs and an additional 2 SPMCs for the supervisor.  Here
  I'm storing these 2 SPMCs in PMC7/8.

- This add *a lot* state for transactional memory.  TM suspend mode,
  this is unavoidable, you can't simply roll back all transactions and
  store only the checkpointed state.  I've added this all to
  get/set_one_reg (including GPRs) rather than creating a new ioctl
  which returns a struct kvm_regs like KVM_GET_REGS does.  This means we
  if we need to extract the TM state, we are going to need a bucket load
  of IOCTLs.  Hopefully most of the time this will not be needed as we
  can look at the MSR to see if TM is active and only grab them when
  needed.

- The TM state is offset bu 0x1000.  Other than being bigger than the
  SPR space, it's fairly arbitrarily chose. 

- For TM, I've done away with VMX and FP and created a single 64x128 bit
  VSX register space.

Alex: I'll add the documentation Documentation/virtual/kvm/api.txt if
you're happy with all this.

Signed-off-by: Michael Neuling <mikey@neuling.org>


--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Alexander Graf - Aug. 30, 2013, 2:01 p.m.
On 30.08.2013, at 08:09, Michael Neuling wrote:

> Alex,
> 
> This reserves space in get/set_one_reg ioctl for the extra guest state
> needed for POWER8.  It doesn't implement these at all, it just reserves
> them so that the ABI is defined now.  
> 
> A few things to note here:
> 
> - POWER8 has 6 PMCs and an additional 2 SPMCs for the supervisor.  Here
>  I'm storing these 2 SPMCs in PMC7/8.

Is this safe to do? Are we guaranteed that POWER9 or POWER10 doesn't introduce a real PMC7?

> - This add *a lot* state for transactional memory.  TM suspend mode,
>  this is unavoidable, you can't simply roll back all transactions and
>  store only the checkpointed state.  I've added this all to
>  get/set_one_reg (including GPRs) rather than creating a new ioctl
>  which returns a struct kvm_regs like KVM_GET_REGS does.  This means we
>  if we need to extract the TM state, we are going to need a bucket load
>  of IOCTLs.  Hopefully most of the time this will not be needed as we
>  can look at the MSR to see if TM is active and only grab them when
>  needed.

If we find this to be a performance issue, we can always add a new ioctl that allows multiple ONE_REG accesses at a time. The only reason we don't have that yet is that bulk one_reg access hasn't happened in any performance critical path so far.

> 
> - The TM state is offset bu 0x1000.  Other than being bigger than the
>  SPR space, it's fairly arbitrarily chose. 
> 
> - For TM, I've done away with VMX and FP and created a single 64x128 bit
>  VSX register space.
> 
> Alex: I'll add the documentation Documentation/virtual/kvm/api.txt if
> you're happy with all this.

Looks perfectly reasonable to me :).


Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Benjamin Herrenschmidt - Aug. 30, 2013, 10:17 p.m.
On Fri, 2013-08-30 at 16:01 +0200, Alexander Graf wrote:
> > 
> > - The TM state is offset bu 0x1000.  Other than being bigger than
> the
> >  SPR space, it's fairly arbitrarily chose. 

Make it higher, just in case....

Ben.


--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Michael Neuling - Aug. 30, 2013, 10:21 p.m.
On Sat, Aug 31, 2013 at 8:17 AM, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> On Fri, 2013-08-30 at 16:01 +0200, Alexander Graf wrote:
>> >
>> > - The TM state is offset bu 0x1000.  Other than being bigger than
>> the
>> >  SPR space, it's fairly arbitrarily chose.
>
> Make it higher, just in case....

Ok but how high?  KVM_REG_SiZE is set by generic code and it's way up
at 0x0030000000000000ULL

Mikey
--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Michael Neuling - Aug. 30, 2013, 10:22 p.m.
On Sat, Aug 31, 2013 at 12:01 AM, Alexander Graf <agraf@suse.de> wrote:
>
> On 30.08.2013, at 08:09, Michael Neuling wrote:
>
>> Alex,
>>
>> This reserves space in get/set_one_reg ioctl for the extra guest state
>> needed for POWER8.  It doesn't implement these at all, it just reserves
>> them so that the ABI is defined now.
>>
>> A few things to note here:
>>
>> - POWER8 has 6 PMCs and an additional 2 SPMCs for the supervisor.  Here
>>  I'm storing these 2 SPMCs in PMC7/8.
>
> Is this safe to do? Are we guaranteed that POWER9 or POWER10 doesn't introduce a real PMC7?

Good point.  I'll change it.

>
>> - This add *a lot* state for transactional memory.  TM suspend mode,
>>  this is unavoidable, you can't simply roll back all transactions and
>>  store only the checkpointed state.  I've added this all to
>>  get/set_one_reg (including GPRs) rather than creating a new ioctl
>>  which returns a struct kvm_regs like KVM_GET_REGS does.  This means we
>>  if we need to extract the TM state, we are going to need a bucket load
>>  of IOCTLs.  Hopefully most of the time this will not be needed as we
>>  can look at the MSR to see if TM is active and only grab them when
>>  needed.
>
> If we find this to be a performance issue, we can always add a new ioctl that allows multiple ONE_REG accesses at a time. The only reason we don't have that yet is that bulk one_reg access hasn't happened in any performance critical path so far.

Ok.

>
>>
>> - The TM state is offset bu 0x1000.  Other than being bigger than the
>>  SPR space, it's fairly arbitrarily chose.
>>
>> - For TM, I've done away with VMX and FP and created a single 64x128 bit
>>  VSX register space.
>>
>> Alex: I'll add the documentation Documentation/virtual/kvm/api.txt if
>> you're happy with all this.
>
> Looks perfectly reasonable to me :).

Thanks.  I'll repost.

Mikey
--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/arch/powerpc/include/uapi/asm/kvm.h b/arch/powerpc/include/uapi/asm/kvm.h
index 0fb1a6e..33b8007 100644
--- a/arch/powerpc/include/uapi/asm/kvm.h
+++ b/arch/powerpc/include/uapi/asm/kvm.h
@@ -429,6 +429,11 @@  struct kvm_get_htab_header {
 #define KVM_REG_PPC_MMCR0	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0x10)
 #define KVM_REG_PPC_MMCR1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0x11)
 #define KVM_REG_PPC_MMCRA	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0x12)
+#define KVM_REG_PPC_MMCR2	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0x13)
+#define KVM_REG_PPC_MMCRS	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0x14)
+#define KVM_REG_PPC_SIAR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0x15)
+#define KVM_REG_PPC_SDAR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0x16)
+#define KVM_REG_PPC_SIER	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0x17)
 
 #define KVM_REG_PPC_PMC1	(KVM_REG_PPC | KVM_REG_SIZE_U32 | 0x18)
 #define KVM_REG_PPC_PMC2	(KVM_REG_PPC | KVM_REG_SIZE_U32 | 0x19)
@@ -499,6 +504,53 @@  struct kvm_get_htab_header {
 #define KVM_REG_PPC_TLB3PS	(KVM_REG_PPC | KVM_REG_SIZE_U32 | 0x9a)
 #define KVM_REG_PPC_EPTCFG	(KVM_REG_PPC | KVM_REG_SIZE_U32 | 0x9b)
 
+/* POWER8 new SPRs */
+#define KVM_REG_PPC_IAMR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0x9c)
+#define KVM_REG_PPC_TFHAR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0x9d)
+#define KVM_REG_PPC_TFIAR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0x9e)
+#define KVM_REG_PPC_TEXASR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0x9f)
+#define KVM_REG_PPC_FSCR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xa0)
+#define KVM_REG_PPC_PSPB	(KVM_REG_PPC | KVM_REG_SIZE_U32 | 0xa1)
+#define KVM_REG_PPC_EBBHR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xa2)
+#define KVM_REG_PPC_EBBRR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xa3)
+#define KVM_REG_PPC_BESCR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xa4)
+#define KVM_REG_PPC_TAR		(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xa5)
+#define KVM_REG_PPC_DPDES	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xa6)
+#define KVM_REG_PPC_DAWR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xa7)
+#define KVM_REG_PPC_DAWRX	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xa8)
+#define KVM_REG_PPC_CIABR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xa9)
+#define KVM_REG_PPC_IC		(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xaa)
+#define KVM_REG_PPC_VTB		(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xab)
+#define KVM_REG_PPC_CSIGR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xac)
+#define KVM_REG_PPC_TACR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xad)
+#define KVM_REG_PPC_TCSCR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xae)
+#define KVM_REG_PPC_PID		(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xaf)
+#define KVM_REG_PPC_ACOP	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xb0)
+
+/* Transactional Memory checkpointed state:
+ * This is all GPRs, all VSX regs and a subset of SPRs
+ */
+#define KVM_REG_PPC_TM		(KVM_REG_PPC | 0x1000)
+/* TM GPRs */
+#define KVM_REG_PPC_TM_GPR0	(KVM_REG_PPC_TM | KVM_REG_SIZE_U64 | 0)
+#define KVM_REG_PPC_TM_GPR(n)	(KVM_REG_PPC_TM_GPR0 + (n))
+#define KVM_REG_PPC_TM_GPR31	(KVM_REG_PPC_TM | KVM_REG_SIZE_U64 | 0x1f)
+/* TM VSX */
+#define KVM_REG_PPC_TM_VSR0	(KVM_REG_PPC | KVM_REG_SIZE_U128 | 0x20)
+#define KVM_REG_PPC_TM_VSR(n)	(KVM_REG_PPC_VSR0 + (n))
+#define KVM_REG_PPC_VSR63	(KVM_REG_PPC | KVM_REG_SIZE_U128 | 0x5f)
+/* TM SPRS */
+#define KVM_REG_PPC_TM_CR	(KVM_REG_PPC_TM | KVM_REG_SIZE_U64 | 0x60)
+#define KVM_REG_PPC_TM_LR	(KVM_REG_PPC_TM | KVM_REG_SIZE_U64 | 0x61)
+#define KVM_REG_PPC_TM_CTR	(KVM_REG_PPC_TM | KVM_REG_SIZE_U64 | 0x62)
+#define KVM_REG_PPC_TM_FPSCR	(KVM_REG_PPC_TM | KVM_REG_SIZE_U64 | 0x63)
+#define KVM_REG_PPC_TM_AMR	(KVM_REG_PPC_TM | KVM_REG_SIZE_U64 | 0x64)
+#define KVM_REG_PPC_TM_PPR	(KVM_REG_PPC_TM | KVM_REG_SIZE_U64 | 0x65)
+#define KVM_REG_PPC_TM_VRSAVE	(KVM_REG_PPC_TM | KVM_REG_SIZE_U64 | 0x66)
+#define KVM_REG_PPC_TM_VSCR	(KVM_REG_PPC_TM | KVM_REG_SIZE_U32 | 0x67)
+#define KVM_REG_PPC_TM_DSCR	(KVM_REG_PPC_TM | KVM_REG_SIZE_U64 | 0x68)
+#define KVM_REG_PPC_TM_TAR	(KVM_REG_PPC_TM | KVM_REG_SIZE_U64 | 0x69)
+
 /* PPC64 eXternal Interrupt Controller Specification */
 #define KVM_DEV_XICS_GRP_SOURCES	1	/* 64-bit source attributes */