[2/2] powerpc/64s/radix: Explicitly flush ERAT with local LPID invalidation

Local radix TLB flush operations that operate on congruence classes
have explicit ERAT flushes for POWER9. The process scoped LPID flush
did not have a flush, so add it.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 arch/powerpc/mm/tlb-radix.c | 1 +
 1 file changed, 1 insertion(+)

On Mon, 2018-08-27 at 13:03 +1000, Nicholas Piggin wrote:
> Local radix TLB flush operations that operate on congruence classes
> have explicit ERAT flushes for POWER9. The process scoped LPID flush
> did not have a flush, so add it.

Paul, is that an actual bug ? I think the ERAT is flushed on LPID
changes...

> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>  arch/powerpc/mm/tlb-radix.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c
> index fef3e1eb3a19..4e798f33c530 100644
> --- a/arch/powerpc/mm/tlb-radix.c
> +++ b/arch/powerpc/mm/tlb-radix.c
> @@ -366,6 +366,7 @@ static inline void _tlbiel_lpid_guest(unsigned long lpid, unsigned long ric)
>  		__tlbiel_lpid_guest(lpid, set, RIC_FLUSH_TLB);
>  
>  	asm volatile("ptesync": : :"memory");
> +	asm volatile(PPC_INVALIDATE_ERAT : : :"memory");
>  }
>  
>

On Mon, 27 Aug 2018 18:16:05 +1000
Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote:

> On Mon, 2018-08-27 at 13:03 +1000, Nicholas Piggin wrote:
> > Local radix TLB flush operations that operate on congruence classes
> > have explicit ERAT flushes for POWER9. The process scoped LPID flush
> > did not have a flush, so add it.  
> 
> Paul, is that an actual bug ? I think the ERAT is flushed on LPID
> changes...

We also have a PPC_INVALIDATE_ERAT in tlbiel_lpid. I'd like to add some
comments for these things because I'm not entirely clear on them myself.

The P9 UM says, "Additionally, mtpidr and mtlpidr instructions perform
an implicit slbia with IH = x‘3’."

Although slbia IH=3 does not appear to invalidate LPID!=0 && PID==0
entries (guest OS mappigs).

The UM also says that tlbiels will clear the involved ERATs, but this
might not match hardware?

Thanks,
Nick

> 
> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> >  arch/powerpc/mm/tlb-radix.c | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c
> > index fef3e1eb3a19..4e798f33c530 100644
> > --- a/arch/powerpc/mm/tlb-radix.c
> > +++ b/arch/powerpc/mm/tlb-radix.c
> > @@ -366,6 +366,7 @@ static inline void _tlbiel_lpid_guest(unsigned long lpid, unsigned long ric)
> >  		__tlbiel_lpid_guest(lpid, set, RIC_FLUSH_TLB);
> >  
> >  	asm volatile("ptesync": : :"memory");
> > +	asm volatile(PPC_INVALIDATE_ERAT : : :"memory");
> >  }
> >  
> >    
>

On Mon, 27 Aug 2018 18:16:05 +1000
Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote:

> On Mon, 2018-08-27 at 13:03 +1000, Nicholas Piggin wrote:
> > Local radix TLB flush operations that operate on congruence classes
> > have explicit ERAT flushes for POWER9. The process scoped LPID flush
> > did not have a flush, so add it.  
> 
> Paul, is that an actual bug ? I think the ERAT is flushed on LPID
> changes...

FWIW I'd like to add the following patch after these fixes. While I
have your attention...

[PATCH] powerpc/64s/radix: keep kernel ERAT over local process/guest
 invalidates

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 arch/powerpc/include/asm/ppc-opcode.h | 10 +++++++++-
 arch/powerpc/mm/tlb-radix.c           |  6 +++---
 2 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/arch/powerpc/include/asm/ppc-opcode.h b/arch/powerpc/include/asm/ppc-opcode.h
index 665af14850e4..5fe617ab680a 100644
--- a/arch/powerpc/include/asm/ppc-opcode.h
+++ b/arch/powerpc/include/asm/ppc-opcode.h
@@ -568,7 +568,15 @@
 
 #define PPC_SLBIA(IH)	stringify_in_c(.long PPC_INST_SLBIA | \
 				       ((IH & 0x7) << 21))
-#define PPC_INVALIDATE_ERAT	PPC_SLBIA(7)
+
+/*
+ * These may only be used by ARCH_300.
+ * GUEST/USER invalidates should only be used by radix mode, on HPT they also
+ * invalidate SLBs so the SLBIA instruction should be used directly.
+ */
+#define PPC_INVALIDATE_ERAT		PPC_SLBIA(7)
+#define PPC_INVALIDATE_GUEST_ERAT	PPC_SLBIA(6)
+#define PPC_INVALIDATE_USER_ERAT	PPC_SLBIA(3)
 
 #define VCMPEQUD_RC(vrt, vra, vrb)	stringify_in_c(.long PPC_INST_VCMPEQUD | \
 			      ___PPC_RT(vrt) | ___PPC_RA(vra) | \
diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c
index 4e798f33c530..6887e4b2568b 100644
--- a/arch/powerpc/mm/tlb-radix.c
+++ b/arch/powerpc/mm/tlb-radix.c
@@ -262,7 +262,7 @@ static inline void _tlbiel_pid(unsigned long pid, unsigned long ric)
 		__tlbiel_pid(pid, set, RIC_FLUSH_TLB);
 
 	asm volatile("ptesync": : :"memory");
-	asm volatile(PPC_INVALIDATE_ERAT "; isync" : : :"memory");
+	asm volatile(PPC_INVALIDATE_USER_ERAT "; isync" : : :"memory");
 }
 
 static inline void _tlbie_pid(unsigned long pid, unsigned long ric)
@@ -314,7 +314,7 @@ static inline void _tlbiel_lpid(unsigned long lpid, unsigned long ric)
 		__tlbiel_lpid(lpid, set, RIC_FLUSH_TLB);
 
 	asm volatile("ptesync": : :"memory");
-	asm volatile(PPC_INVALIDATE_ERAT "; isync" : : :"memory");
+	asm volatile(PPC_INVALIDATE_GUEST_ERAT "; isync" : : :"memory");
 }
 
 static inline void _tlbie_lpid(unsigned long lpid, unsigned long ric)
@@ -366,7 +366,7 @@ static inline void _tlbiel_lpid_guest(unsigned long lpid, unsigned long ric)
 		__tlbiel_lpid_guest(lpid, set, RIC_FLUSH_TLB);
 
 	asm volatile("ptesync": : :"memory");
-	asm volatile(PPC_INVALIDATE_ERAT : : :"memory");
+	asm volatile(PPC_INVALIDATE_GUEST_ERAT : : :"memory");
 }
 
 
--