From patchwork Tue Apr 10 12:48:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 896671 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40L6b651XMz9s1p for ; Tue, 10 Apr 2018 22:55:14 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="jEwpXU/f"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40L6b63P5JzF24T for ; Tue, 10 Apr 2018 22:55:14 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="jEwpXU/f"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:400e:c00::241; helo=mail-pf0-x241.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="jEwpXU/f"; dkim-atps=neutral Received: from mail-pf0-x241.google.com (mail-pf0-x241.google.com [IPv6:2607:f8b0:400e:c00::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40L6Rs2NcpzF1x1 for ; Tue, 10 Apr 2018 22:48:57 +1000 (AEST) Received: by mail-pf0-x241.google.com with SMTP id o16so8203303pfk.9 for ; Tue, 10 Apr 2018 05:48:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=z27GieaoKGerBPW7txqIgdu7iBhwLiD7kgZkaFrsjWY=; b=jEwpXU/fBX8yepxHJFqklQf+NrBFT0TjmGoy5UBk5SQnh0IxxLXlOmWJbQQauOgLeD 5AXUSo4kL0GhhIrX/9+EiLYdiguCSth+nUjw6tHI2SWQp8qRuJWcBwPz/dZvkfJa2Wy1 pnnqU4pTmcy9tHuqqiZkwdo0azrek6cqfcH9a7Kq/ENBbk/4Q6mV8J7hQLVCpKnbaX2C Sbx/aJqFux8N7oyEh+tIeDtCT6hYxZdv0BI/3+TODCQhJaldzplAkMZq8qQcGy0MXnVG qANhNlOzUC3GHUCt7hdHVU8W3ibDNqYDCD9vULTgk1bTO2q0n1VpZzM2QKEfp4w3gICD +moQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=z27GieaoKGerBPW7txqIgdu7iBhwLiD7kgZkaFrsjWY=; b=NLthYW1JFVgs5R4wOeg1OFQnA9JOkUtz2+oMwYdgorXYlK5JwQE4stnFL/eIMzo6GS 6xVE10pYO7m3H6/qqXaoGRxbvEg9Oo+Wp3ZoHCc1Tof/VhzkSO47GFncxB/1BeyND2Yx 8DTbx/049szywdZSzKcheRS959W0caWfxj0QGKSj/aeCbYhtM1zxXXcixQzxNG3gsdEz L7zpLFkM4+4/4BY9BNpjj2X3hA55+1InQBjifALLSGdf0zxzRmok8XEBRzGE5f55TcaM pivogVr7QUx5S5ED1WHmvSONC/3LlZ+jdsJbByiHHJBQBVxqfsl8yUvb0b6/BLJiBBfD 7ymA== X-Gm-Message-State: ALQs6tDa0BAbjV81awNykaATHLL9bOUa4/cJmKNZWd1bprSvkhK6uUYd asySgzhYwhgq5usAAY9ASyc= X-Google-Smtp-Source: AIpwx48JsJcREUmrovxiMJmNtxyWiLz36qdr98+r4tah2H9NKMtJia7xTMwZuW9ZjOrghxtG5KMf8w== X-Received: by 10.101.97.200 with SMTP id j8mr174773pgv.443.1523364535554; Tue, 10 Apr 2018 05:48:55 -0700 (PDT) Received: from roar.au.ibm.com (59-102-70-78.tpgi.com.au. [59.102.70.78]) by smtp.gmail.com with ESMTPSA id j20sm6037529pfa.149.2018.04.10.05.48.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Apr 2018 05:48:54 -0700 (PDT) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Subject: [RFC PATCH 1/5] powerpc/64s/mm: Implement LPID based TLB flushes to be used by KVM Date: Tue, 10 Apr 2018 22:48:38 +1000 Message-Id: <20180410124842.30184-2-npiggin@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180410124842.30184-1-npiggin@gmail.com> References: <20180410124842.30184-1-npiggin@gmail.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linuxppc-dev@lists.ozlabs.org, Nicholas Piggin Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Implent local TLB flush for entire LPID, for hash and radix, and a global TLB flush for a partition scoped page in an LPID, for radix. These will be used by KVM in subsequent patches. Signed-off-by: Nicholas Piggin --- .../include/asm/book3s/64/tlbflush-hash.h | 2 + .../include/asm/book3s/64/tlbflush-radix.h | 5 ++ arch/powerpc/mm/hash_native_64.c | 8 ++ arch/powerpc/mm/tlb-radix.c | 87 +++++++++++++++++++ 4 files changed, 102 insertions(+) diff --git a/arch/powerpc/include/asm/book3s/64/tlbflush-hash.h b/arch/powerpc/include/asm/book3s/64/tlbflush-hash.h index 64d02a704bcb..8b328fd87722 100644 --- a/arch/powerpc/include/asm/book3s/64/tlbflush-hash.h +++ b/arch/powerpc/include/asm/book3s/64/tlbflush-hash.h @@ -53,6 +53,8 @@ static inline void arch_leave_lazy_mmu_mode(void) extern void hash__tlbiel_all(unsigned int action); +extern void hash__local_flush_tlb_lpid(unsigned int lpid); + extern void flush_hash_page(unsigned long vpn, real_pte_t pte, int psize, int ssize, unsigned long flags); extern void flush_hash_range(unsigned long number, int local); diff --git a/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h b/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h index 19b45ba6caf9..2ddaadf3e9ea 100644 --- a/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h +++ b/arch/powerpc/include/asm/book3s/64/tlbflush-radix.h @@ -51,4 +51,9 @@ extern void radix__flush_tlb_all(void); extern void radix__flush_tlb_pte_p9_dd1(unsigned long old_pte, struct mm_struct *mm, unsigned long address); +extern void radix__flush_tlb_lpid_page(unsigned int lpid, + unsigned long addr, + unsigned long page_size); +extern void radix__local_flush_tlb_lpid(unsigned int lpid); + #endif diff --git a/arch/powerpc/mm/hash_native_64.c b/arch/powerpc/mm/hash_native_64.c index 1d049c78c82a..2f02cd780c19 100644 --- a/arch/powerpc/mm/hash_native_64.c +++ b/arch/powerpc/mm/hash_native_64.c @@ -294,6 +294,14 @@ static inline void tlbie(unsigned long vpn, int psize, int apsize, raw_spin_unlock(&native_tlbie_lock); } +void hash__local_flush_tlb_lpid(unsigned int lpid) +{ + VM_BUG_ON(mfspr(SPRN_LPID) != lpid); + + hash__tlbiel_all(TLB_INVAL_SCOPE_LPID); +} +EXPORT_SYMBOL_GPL(hash__local_flush_tlb_lpid); + static inline void native_lock_hpte(struct hash_pte *hptep) { unsigned long *word = (unsigned long *)&hptep->v; diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c index 2fba6170ab3f..f246fb0ac049 100644 --- a/arch/powerpc/mm/tlb-radix.c +++ b/arch/powerpc/mm/tlb-radix.c @@ -119,6 +119,22 @@ static inline void __tlbie_pid(unsigned long pid, unsigned long ric) trace_tlbie(0, 0, rb, rs, ric, prs, r); } +static inline void __tlbiel_lpid(unsigned long lpid, int set, + unsigned long ric) +{ + unsigned long rb,rs,prs,r; + + rb = PPC_BIT(52); /* IS = 2 */ + rb |= set << PPC_BITLSHIFT(51); + rs = 0; /* LPID comes from LPIDR */ + prs = 0; /* partition scoped */ + r = 1; /* radix format */ + + asm volatile(PPC_TLBIEL(%0, %4, %3, %2, %1) + : : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(rs) : "memory"); + trace_tlbie(lpid, 1, rb, rs, ric, prs, r); +} + static inline void __tlbiel_va(unsigned long va, unsigned long pid, unsigned long ap, unsigned long ric) { @@ -151,6 +167,22 @@ static inline void __tlbie_va(unsigned long va, unsigned long pid, trace_tlbie(0, 0, rb, rs, ric, prs, r); } +static inline void __tlbie_lpid_va(unsigned long va, unsigned long lpid, + unsigned long ap, unsigned long ric) +{ + unsigned long rb,rs,prs,r; + + rb = va & ~(PPC_BITMASK(52, 63)); + rb |= ap << PPC_BITLSHIFT(58); + rs = lpid; + prs = 0; /* partition scoped */ + r = 1; /* radix format */ + + asm volatile(PPC_TLBIE_5(%0, %4, %3, %2, %1) + : : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(rs) : "memory"); + trace_tlbie(lpid, 0, rb, rs, ric, prs, r); +} + static inline void fixup_tlbie(void) { unsigned long pid = 0; @@ -215,6 +247,34 @@ static inline void _tlbie_pid(unsigned long pid, unsigned long ric) asm volatile("eieio; tlbsync; ptesync": : :"memory"); } +static inline void _tlbiel_lpid(unsigned long lpid, unsigned long ric) +{ + int set; + + VM_BUG_ON(mfspr(SPRN_LPID) != lpid); + + asm volatile("ptesync": : :"memory"); + + /* + * Flush the first set of the TLB, and if we're doing a RIC_FLUSH_ALL, + * also flush the entire Page Walk Cache. + */ + __tlbiel_lpid(lpid, 0, ric); + + /* For PWC, only one flush is needed */ + if (ric == RIC_FLUSH_PWC) { + asm volatile("ptesync": : :"memory"); + return; + } + + /* For the remaining sets, just flush the TLB */ + for (set = 1; set < POWER9_TLB_SETS_RADIX ; set++) + __tlbiel_lpid(lpid, set, RIC_FLUSH_TLB); + + asm volatile("ptesync": : :"memory"); + asm volatile(PPC_INVALIDATE_ERAT "; isync" : : :"memory"); +} + static inline void __tlbiel_va_range(unsigned long start, unsigned long end, unsigned long pid, unsigned long page_size, unsigned long psize) @@ -269,6 +329,17 @@ static inline void _tlbie_va(unsigned long va, unsigned long pid, asm volatile("eieio; tlbsync; ptesync": : :"memory"); } +static inline void _tlbie_lpid_va(unsigned long va, unsigned long lpid, + unsigned long psize, unsigned long ric) +{ + unsigned long ap = mmu_get_ap(psize); + + asm volatile("ptesync": : :"memory"); + __tlbie_lpid_va(va, lpid, ap, ric); + fixup_tlbie(); + asm volatile("eieio; tlbsync; ptesync": : :"memory"); +} + static inline void _tlbie_va_range(unsigned long start, unsigned long end, unsigned long pid, unsigned long page_size, unsigned long psize, bool also_pwc) @@ -535,6 +606,22 @@ static int radix_get_mmu_psize(int page_size) return psize; } +void radix__flush_tlb_lpid_page(unsigned int lpid, + unsigned long addr, + unsigned long page_size) +{ + int psize = radix_get_mmu_psize(page_size); + + _tlbie_lpid_va(addr, lpid, psize, RIC_FLUSH_TLB); +} +EXPORT_SYMBOL_GPL(radix__flush_tlb_lpid_page); + +void radix__local_flush_tlb_lpid(unsigned int lpid) +{ + _tlbiel_lpid(lpid, RIC_FLUSH_ALL); +} +EXPORT_SYMBOL_GPL(radix__local_flush_tlb_lpid); + static void radix__flush_tlb_pwc_range_psize(struct mm_struct *mm, unsigned long start, unsigned long end, int psize); From patchwork Tue Apr 10 12:48:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 896680 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40L6h25K3hz9s15 for ; Tue, 10 Apr 2018 22:59:30 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="L7njjOew"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40L6h23p3wzF270 for ; Tue, 10 Apr 2018 22:59:30 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="L7njjOew"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:400e:c01::242; helo=mail-pl0-x242.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="L7njjOew"; dkim-atps=neutral Received: from mail-pl0-x242.google.com (mail-pl0-x242.google.com [IPv6:2607:f8b0:400e:c01::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40L6Rw37n2zF1x1 for ; Tue, 10 Apr 2018 22:49:00 +1000 (AEST) Received: by mail-pl0-x242.google.com with SMTP id a39-v6so7432342pla.10 for ; Tue, 10 Apr 2018 05:49:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=yTMv2pR3P1xIK+dgMkT8NxITMpA6Wa0KHRqxbJBRLbs=; b=L7njjOewtPijvan7Tt7sxRhstxj4UosyFlZfsMoEnsdmxSR5Vy0gu3Pak1yFqIkqu/ ucrItnVBm3lmdRzrqtAaM2uy8F5rHL5P5nqDr4E9GmC0Zg3KEDTb3IBum/0NFxeaX5n8 V8QETtmsMeHtqOPetp0um/BhyyLAvnUBZTnOho0/1Togh3XH8GKuYVp34sBZ581cwd0A ah2hDwpoF75POumG2T/fDGcHlO94zsPB0wzcAZZrRcnOBn92s3jdgKopfrLFU1yoZqMR B3CPn0J7KJRH6eILUTdaxcih28DGGwOsvSOXJ0BpMzh2xlX4AEmw0gdFGJ+QpiIInZCw NsdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=yTMv2pR3P1xIK+dgMkT8NxITMpA6Wa0KHRqxbJBRLbs=; b=PaqKYp10ExfgURXTYUfyQgl5IEO46awfNaxDLZZehd5woAe8TldLw/Vb4+fwnuGB5Z l5g8+K4dxlL1MZw3r2V5n+FC9KKKI7M6c65mN5Zr69MiQN713s3vPR1mGDkutePjdOFu 4m3NF2MvFeGlmU2j0TXAmD05VSb7Qxk2VdEYadgf29xjRJFclV5DyDiauBtnS0eQu8gt 4YbVifBHKPppZ4CijIdCT6D2HYrsX+M2FMdVaPHIwcOAaZWAzn+fFVIUO73VthMGGDeg LqAOZAV+GxOj74woveLb7EZ/HnAN5+JvQxFeDOjK/Fz272OEBGIMt+/2er7hS9OiTLHI hOMw== X-Gm-Message-State: ALQs6tAyNgexI/q78IonB3sHBbHdS4ph0KkN697SjzEoChM60tBbEVzg 6gr8hcQzOKgZgmInUmgsf645GA== X-Google-Smtp-Source: AIpwx4+n0aFypBEPOulsuFc9CjJrzwii7VwauFWmuyVvNOm766GFANbVfTfaokUSmYW1IHrZypBxuA== X-Received: by 2002:a17:902:5a0b:: with SMTP id q11-v6mr282241pli.199.1523364538918; Tue, 10 Apr 2018 05:48:58 -0700 (PDT) Received: from roar.au.ibm.com (59-102-70-78.tpgi.com.au. [59.102.70.78]) by smtp.gmail.com with ESMTPSA id j20sm6037529pfa.149.2018.04.10.05.48.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Apr 2018 05:48:58 -0700 (PDT) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Subject: [RFC PATCH 2/5] KVM: PPC: Book3S HV: kvmppc_radix_tlbie_page use Linux flush function Date: Tue, 10 Apr 2018 22:48:39 +1000 Message-Id: <20180410124842.30184-3-npiggin@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180410124842.30184-1-npiggin@gmail.com> References: <20180410124842.30184-1-npiggin@gmail.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linuxppc-dev@lists.ozlabs.org, Nicholas Piggin Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" This has the advantage of consolidating TLB flush code in fewer places, and it also implements powerpc:tlbie trace events. 1GB pages should be handled without further modification. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_64_mmu_radix.c | 26 +++++++------------------- 1 file changed, 7 insertions(+), 19 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 81d5ad26f9a1..dab6b622011c 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -139,28 +139,16 @@ int kvmppc_mmu_radix_xlate(struct kvm_vcpu *vcpu, gva_t eaddr, return 0; } -#ifdef CONFIG_PPC_64K_PAGES -#define MMU_BASE_PSIZE MMU_PAGE_64K -#else -#define MMU_BASE_PSIZE MMU_PAGE_4K -#endif - static void kvmppc_radix_tlbie_page(struct kvm *kvm, unsigned long addr, unsigned int pshift) { - int psize = MMU_BASE_PSIZE; - - if (pshift >= PMD_SHIFT) - psize = MMU_PAGE_2M; - addr &= ~0xfffUL; - addr |= mmu_psize_defs[psize].ap << 5; - asm volatile("ptesync": : :"memory"); - asm volatile(PPC_TLBIE_5(%0, %1, 0, 0, 1) - : : "r" (addr), "r" (kvm->arch.lpid) : "memory"); - if (cpu_has_feature(CPU_FTR_P9_TLBIE_BUG)) - asm volatile(PPC_TLBIE_5(%0, %1, 0, 0, 1) - : : "r" (addr), "r" (kvm->arch.lpid) : "memory"); - asm volatile("eieio ; tlbsync ; ptesync": : :"memory"); + unsigned long psize = PAGE_SIZE; + + if (pshift) + psize = 1UL << pshift; + + addr &= ~(psize - 1); + radix__flush_tlb_lpid_page(kvm->arch.lpid, addr, psize); } unsigned long kvmppc_radix_update_pte(struct kvm *kvm, pte_t *ptep, From patchwork Tue Apr 10 12:48:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 896693 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40L6mL6zs7z9s15 for ; Tue, 10 Apr 2018 23:03:14 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Gl3XU6KO"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40L6mL52rNzF28S for ; Tue, 10 Apr 2018 23:03:14 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Gl3XU6KO"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:400e:c01::241; helo=mail-pl0-x241.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Gl3XU6KO"; dkim-atps=neutral Received: from mail-pl0-x241.google.com (mail-pl0-x241.google.com [IPv6:2607:f8b0:400e:c01::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40L6Rz43xSzF1x2 for ; Tue, 10 Apr 2018 22:49:03 +1000 (AEST) Received: by mail-pl0-x241.google.com with SMTP id 61-v6so7448281plb.2 for ; Tue, 10 Apr 2018 05:49:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=0bkJfrGvGPz3S8u54aSlAoTLdFnjRWHUVMWzkejBiJs=; b=Gl3XU6KO+Nzzk8PdSfxvUUR6h1iepzJH5YVi9W2FSoPNDDgYTtusJ94V/f5SX0d4Yb afrCCyrsD09tyEGhFivTHDwMnFxGjUqCG5P6Mw0ixEUdmePcwcxrWJiG07mhT5EQ+bBJ IVzs9DSOiSQ+u+Be5Yb7SaWfGyNYn4hZzwoiB9zlDrkZ5Dq1pc+8wVy54W41EHsY53v/ da0SsdoxqQOuiWCtKbMziXSRqbZIt02nad5TUWBkNM/jEVE+yzzfC4e1Fcstsy3uRPGI GaZMJA+hcJUXXZlbrfRWdGowL+qYSEOgaoflnNOHFatsjwD1eXxTgLcR15u2V7vRpBxN hb4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=0bkJfrGvGPz3S8u54aSlAoTLdFnjRWHUVMWzkejBiJs=; b=nAOQYf8GZkGnTOmUlMn0uFRWD86/F9HV7U+PKNLXkeYTre/2zuT62d8qe56sDQqkEm qbL0Cogz9jeA0fxEBPc8bd/ITtYZ2akhUapv4Vqh95ppgwdhMI9H6shTJESmtTAjL+7n ChuyP0JNACZkY4VCaoJnV5hGPXekSz3AdKX6lTVl6fr/uL7VmeRp9xdTxSzJExAF7794 xsjhUj73saNGu3qQGUUTqtdOAG+lyLH7Z6Eof2GRFjBao9xdyHIbZ1UPsX1hAvrmW4ST 4Vt1NQ5iLZlp86CakISZWfTjViXXpHVo3I+L2A0aMzgO+00JmwDp2Ox5RT0KxLAE9CgZ tqEw== X-Gm-Message-State: ALQs6tDJNv9VtWJMD3UAUDEKHQb8CsStF9bxZvpxmz03cWivNk9VlpJl Lh+vPMmmOiqBobZXOSzBRP4= X-Google-Smtp-Source: AIpwx491xtZWuSBHa01qe/UrCFnX/udruK8IknMAAyEIEI6Ey5lI5ubliCyBTNADVuYt5wSElrcGIA== X-Received: by 2002:a17:902:4545:: with SMTP id m63-v6mr278386pld.149.1523364541916; Tue, 10 Apr 2018 05:49:01 -0700 (PDT) Received: from roar.au.ibm.com (59-102-70-78.tpgi.com.au. [59.102.70.78]) by smtp.gmail.com with ESMTPSA id j20sm6037529pfa.149.2018.04.10.05.48.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Apr 2018 05:49:01 -0700 (PDT) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Subject: [RFC PATCH 3/5] KVM: PPC: Book3S HV: kvmhv_p9_set_lpcr use Linux flush function Date: Tue, 10 Apr 2018 22:48:40 +1000 Message-Id: <20180410124842.30184-4-npiggin@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180410124842.30184-1-npiggin@gmail.com> References: <20180410124842.30184-1-npiggin@gmail.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linuxppc-dev@lists.ozlabs.org, Nicholas Piggin Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" The existing flush uses the radix value for sets, and uses R=0 tlbiel instructions. This can't be quite right, but I'm not entirely sure if this is the right way to fix it. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv_builtin.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c index 0b9b8e188bfa..577769fbfae9 100644 --- a/arch/powerpc/kvm/book3s_hv_builtin.c +++ b/arch/powerpc/kvm/book3s_hv_builtin.c @@ -676,7 +676,7 @@ static void wait_for_sync(struct kvm_split_mode *sip, int phase) void kvmhv_p9_set_lpcr(struct kvm_split_mode *sip) { - unsigned long rb, set; + struct kvm *kvm = local_paca->kvm_hstate.kvm_vcpu->kvm; /* wait for every other thread to get to real mode */ wait_for_sync(sip, PHASE_REALMODE); @@ -689,14 +689,10 @@ void kvmhv_p9_set_lpcr(struct kvm_split_mode *sip) /* Invalidate the TLB on thread 0 */ if (local_paca->kvm_hstate.tid == 0) { sip->do_set = 0; - asm volatile("ptesync" : : : "memory"); - for (set = 0; set < POWER9_TLB_SETS_RADIX; ++set) { - rb = TLBIEL_INVAL_SET_LPID + - (set << TLBIEL_INVAL_SET_SHIFT); - asm volatile(PPC_TLBIEL(%0, %1, 0, 0, 0) : : - "r" (rb), "r" (0)); - } - asm volatile("ptesync" : : : "memory"); + if (kvm_is_radix(kvm)) + radix__local_flush_tlb_lpid(kvm->arch.lpid); + else + hash__local_flush_tlb_lpid(kvm->arch.lpid); } /* indicate that we have done so and wait for others */ From patchwork Tue Apr 10 12:48:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 896696 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40L6rH4g0pz9s0n for ; Tue, 10 Apr 2018 23:06:39 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Lv15yWoo"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40L6rH0mcFzF1xc for ; Tue, 10 Apr 2018 23:06:39 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Lv15yWoo"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:400e:c01::243; helo=mail-pl0-x243.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Lv15yWoo"; dkim-atps=neutral Received: from mail-pl0-x243.google.com (mail-pl0-x243.google.com [IPv6:2607:f8b0:400e:c01::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40L6S36yBdzF1y2 for ; Tue, 10 Apr 2018 22:49:07 +1000 (AEST) Received: by mail-pl0-x243.google.com with SMTP id 61-v6so7448396plb.2 for ; Tue, 10 Apr 2018 05:49:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=RNDO28U9y25nuBDXmhiq0uvDQCJSbiEnA/vvjyB7dxc=; b=Lv15yWoocgH6i6kGLCz5fC8FwHabe0PjWlL/c5ORrR6uN51YoUBdA3fGqYAuHCx+aC z+mgivk1uazSNjUr9IzB1W2cHaGFlvD5ODbqzjOFjUZbXAPwctpxf6C29eRwHUwUIzFn Q9EGXkSQljMbdekdeZLy9EvJkBxiQgEHGfurL4x0HxxjjgWtbFxHGDXefyKsFlSVqSXc ZRjU9pFBmbyU4AvXDjBHB/sH+FlhEHeVE8HvQZSW8k4FLnabrRkdZb5xY8VRti3XmRlo QnpxEvROt9QFzMacFQv3VZDhshgwYGIn5at6fPWxAC+/trRVEHaY+sXYaahqJ/HAsDDe JawQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=RNDO28U9y25nuBDXmhiq0uvDQCJSbiEnA/vvjyB7dxc=; b=FLKaCPJqGtWuxBSLl2kkAZdfJKUIWzWytRe5UK8b5BdZRalo0IU9vHh/1IAu5SyVR+ XtO6BXVSx1cGgAZ9itffXO51GntlTuzHh2paQXsEWiYbJ/k7fDh+BpnR5WVTOjLF+H/y 7TZ1atwdEjN67MHuS82igJnCgyxpqMMsgJqsE3+6azWbo1Q4c9LLdSfDHBTuEBQodXHv DlFxnCN44sy53caE1Zj5RSM09fdBqhhE6tKmMvJqOBCXAvZJLrV/k78auc4gNFtd6zyN Kale02ST6xvJ/S4a/qWsX5wWEDX50WQAh6r50MlxmnzMWb21n2HYna18XcnGCNOv8As1 KwgA== X-Gm-Message-State: ALQs6tBw7IRBlZS/lALE8RhbYbmWDi9Hnf3WSjd8pk2/Xz7wxRXFnYGg UVpb7ucfK6kYo2h10WtmhFE= X-Google-Smtp-Source: AIpwx4/M9PwVmp4UL9tnXOTROoq/fuol4W1ihIWXM3czbrGCjbBwQcnTkAn+4AMWABpty2nypnlTpw== X-Received: by 2002:a17:902:6547:: with SMTP id d7-v6mr267968pln.253.1523364546277; Tue, 10 Apr 2018 05:49:06 -0700 (PDT) Received: from roar.au.ibm.com (59-102-70-78.tpgi.com.au. [59.102.70.78]) by smtp.gmail.com with ESMTPSA id j20sm6037529pfa.149.2018.04.10.05.49.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Apr 2018 05:49:05 -0700 (PDT) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Subject: [RFC PATCH 4/5] KVM: PPC: Book3S HV: handle need_tlb_flush in C before low-level guest entry Date: Tue, 10 Apr 2018 22:48:41 +1000 Message-Id: <20180410124842.30184-5-npiggin@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180410124842.30184-1-npiggin@gmail.com> References: <20180410124842.30184-1-npiggin@gmail.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linuxppc-dev@lists.ozlabs.org, Nicholas Piggin Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Move this flushing out of assembly and have it use Linux TLB flush implementations introduced earlier. This allows powerpc:tlbie trace events to be used. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 21 +++++++++++- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 43 +------------------------ 2 files changed, 21 insertions(+), 43 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 81e2ea882d97..5d4783b5b47a 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -2680,7 +2680,7 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc) int sub; bool thr0_done; unsigned long cmd_bit, stat_bit; - int pcpu, thr; + int pcpu, thr, tmp; int target_threads; int controlled_threads; int trap; @@ -2780,6 +2780,25 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc) return; } + /* + * Do we need to flush the TLB for the LPAR? (see TLB comment above) + * On POWER9, individual threads can come in here, but the + * TLB is shared between the 4 threads in a core, hence + * invalidating on one thread invalidates for all. + * Thus we make all 4 threads use the same bit here. + */ + tmp = pcpu; + if (cpu_has_feature(CPU_FTR_ARCH_300)) + tmp &= ~0x3UL; + if (cpumask_test_cpu(tmp, &vc->kvm->arch.need_tlb_flush)) { + if (kvm_is_radix(vc->kvm)) + radix__local_flush_tlb_lpid(vc->kvm->arch.lpid); + else + hash__local_flush_tlb_lpid(vc->kvm->arch.lpid); + /* Clear the bit after the TLB flush */ + cpumask_clear_cpu(tmp, &vc->kvm->arch.need_tlb_flush); + } + kvmppc_clear_host_core(pcpu); /* Decide on micro-threading (split-core) mode */ diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index bd63fa8a08b5..6a23a0f3ceea 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -647,49 +647,8 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) mtspr SPRN_LPID,r7 isync - /* See if we need to flush the TLB */ - lhz r6,PACAPACAINDEX(r13) /* test_bit(cpu, need_tlb_flush) */ -BEGIN_FTR_SECTION - /* - * On POWER9, individual threads can come in here, but the - * TLB is shared between the 4 threads in a core, hence - * invalidating on one thread invalidates for all. - * Thus we make all 4 threads use the same bit here. - */ - clrrdi r6,r6,2 -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) - clrldi r7,r6,64-6 /* extract bit number (6 bits) */ - srdi r6,r6,6 /* doubleword number */ - sldi r6,r6,3 /* address offset */ - add r6,r6,r9 - addi r6,r6,KVM_NEED_FLUSH /* dword in kvm->arch.need_tlb_flush */ - li r8,1 - sld r8,r8,r7 - ld r7,0(r6) - and. r7,r7,r8 - beq 22f - /* Flush the TLB of any entries for this LPID */ - lwz r0,KVM_TLB_SETS(r9) - mtctr r0 - li r7,0x800 /* IS field = 0b10 */ - ptesync - li r0,0 /* RS for P9 version of tlbiel */ - bne cr7, 29f -28: tlbiel r7 /* On P9, rs=0, RIC=0, PRS=0, R=0 */ - addi r7,r7,0x1000 - bdnz 28b - b 30f -29: PPC_TLBIEL(7,0,2,1,1) /* for radix, RIC=2, PRS=1, R=1 */ - addi r7,r7,0x1000 - bdnz 29b -30: ptesync -23: ldarx r7,0,r6 /* clear the bit after TLB flushed */ - andc r7,r7,r8 - stdcx. r7,0,r6 - bne 23b - /* Add timebase offset onto timebase */ -22: ld r8,VCORE_TB_OFFSET(r5) + ld r8,VCORE_TB_OFFSET(r5) cmpdi r8,0 beq 37f mftb r6 /* current host timebase */ From patchwork Tue Apr 10 12:48:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 896697 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40L6vY5zFqz9s0n for ; Tue, 10 Apr 2018 23:09:29 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="EmHbJTQ0"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40L6vY3pZBzF2D4 for ; Tue, 10 Apr 2018 23:09:29 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="EmHbJTQ0"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:400e:c01::241; helo=mail-pl0-x241.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="EmHbJTQ0"; dkim-atps=neutral Received: from mail-pl0-x241.google.com (mail-pl0-x241.google.com [IPv6:2607:f8b0:400e:c01::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40L6S66qcGzF1ys for ; Tue, 10 Apr 2018 22:49:10 +1000 (AEST) Received: by mail-pl0-x241.google.com with SMTP id v5-v6so7431425plo.4 for ; Tue, 10 Apr 2018 05:49:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=kt6s5OIHB5AxXxWa6R0dBucBB3m+NQERaPJJzWIT/h0=; b=EmHbJTQ0K70bPMpDEGhrzgHGUvviU9H2z4J7Hu/pwIriIup809vIS+R+K3F7MrPbws +wYdcWh893qsNzNtUBZOiRfTFcj2fy7oJGF9c9iyR14t598G1RCTDiNu1q+GEdRkmO/B oYQlQKgsuIzCoWNB1bnRrwki9scwNwypCcjXiyndFrYLregYjIRf/wAVE+dmy8ccP2Mj zRay6VnROp9uX/JYL2gAkTSTcXvxtciYMA3O8YzN2Upk2hMLHMmx665aQS+nU4zuIE3m G3dtZsn8QlmbS5C+Wkdi4XrwlZ/PN/IpFUqnDIoDHURaExrkvVxBRp1/SmksgsxOsL2Q zewA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=kt6s5OIHB5AxXxWa6R0dBucBB3m+NQERaPJJzWIT/h0=; b=Kf1pqfXR0OibErToXyVVAavUFYvptFG1OeY3rC9L0vM8HDV1/575gMRlhyScxWMwuo khHT4OkWNo5B/GglW2IN6QHFm6C0vOd0+XzibPaAVTegc7larRCKkaMznxzIsj65tzRi 4r1+p159kSxv433XB7pHlhfWmmkcqDbXWDenf1xkDzXTIKinvPtT7WldlwW9RFEy4aq6 jWOVT5EPaJ/flf38aiMFSNvKEzKkkIQWyL9EdcQhH8MekqdiKHEAOUjaIDr0sAUjsxgK CLdUnnc9fTNPZM9+/HDX87F4PNikE3ksBhIsxjEjt9OjbGKFlgeAWONIuxuNlGxyRHoN xDVA== X-Gm-Message-State: ALQs6tCG6sc+5UJJRo3xu2BBO+e6N9frpxqDMcGFrnPd6XZMa3GaAdCK l2Q6pxVuOON4sXpy2t5P0UUsMw== X-Google-Smtp-Source: AIpwx49wrQ9eN9dKMuAgkVIXquPFzubrbwUg4ll63AMY5/QTmmrtBLzp+ZtaeRi465iNFhE3ZRTKjw== X-Received: by 2002:a17:902:464:: with SMTP id 91-v6mr318126ple.126.1523364549426; Tue, 10 Apr 2018 05:49:09 -0700 (PDT) Received: from roar.au.ibm.com (59-102-70-78.tpgi.com.au. [59.102.70.78]) by smtp.gmail.com with ESMTPSA id j20sm6037529pfa.149.2018.04.10.05.49.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Apr 2018 05:49:08 -0700 (PDT) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Subject: [RFC PATCH 5/5] KVM: PPC: Book3S HV: Radix do not clear partition scoped page table when page fault races with other vCPUs. Date: Tue, 10 Apr 2018 22:48:42 +1000 Message-Id: <20180410124842.30184-6-npiggin@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180410124842.30184-1-npiggin@gmail.com> References: <20180410124842.30184-1-npiggin@gmail.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linuxppc-dev@lists.ozlabs.org, Nicholas Piggin Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" KVM with an SMP radix guest can get into storms of page faults and tlbies due to the partition scopd page tables being invalidated and TLB flushed if they were found to race with another page fault that set them up. This tends to cause vCPUs to pile up if several hit common addresses, then page faults will get serialized on common locks, and then they each invalidate the previous entry and it's long enough before installing the new entry that will cause more CPUs to hit page faults and they will invalidate that new entry. There doesn't seem to be a need to invalidate in the case of an existing entry. This solves the tlbie storms. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_64_mmu_radix.c | 39 +++++++++++++++----------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index dab6b622011c..4af177d24f6c 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -243,6 +243,7 @@ static int kvmppc_create_pte(struct kvm *kvm, pte_t pte, unsigned long gpa, pmd = pmd_offset(pud, gpa); if (pmd_is_leaf(*pmd)) { unsigned long lgpa = gpa & PMD_MASK; + pte_t old_pte = *pmdp_ptep(pmd); /* * If we raced with another CPU which has just put @@ -252,18 +253,17 @@ static int kvmppc_create_pte(struct kvm *kvm, pte_t pte, unsigned long gpa, ret = -EAGAIN; goto out_unlock; } - /* Valid 2MB page here already, remove it */ - old = kvmppc_radix_update_pte(kvm, pmdp_ptep(pmd), - ~0UL, 0, lgpa, PMD_SHIFT); - kvmppc_radix_tlbie_page(kvm, lgpa, PMD_SHIFT); - if (old & _PAGE_DIRTY) { - unsigned long gfn = lgpa >> PAGE_SHIFT; - struct kvm_memory_slot *memslot; - memslot = gfn_to_memslot(kvm, gfn); - if (memslot && memslot->dirty_bitmap) - kvmppc_update_dirty_map(memslot, - gfn, PMD_SIZE); + WARN_ON_ONCE(pte_pfn(old_pte) != pte_pfn(pte)); + if (pte_val(old_pte) == pte_val(pte)) { + ret = -EAGAIN; + goto out_unlock; } + + /* Valid 2MB page here already, remove it */ + kvmppc_radix_update_pte(kvm, pmdp_ptep(pmd), + 0, pte_val(pte), lgpa, PMD_SHIFT); + ret = 0; + goto out_unlock; } else if (level == 1 && !pmd_none(*pmd)) { /* * There's a page table page here, but we wanted @@ -274,6 +274,8 @@ static int kvmppc_create_pte(struct kvm *kvm, pte_t pte, unsigned long gpa, goto out_unlock; } if (level == 0) { + pte_t old_pte; + if (pmd_none(*pmd)) { if (!new_ptep) goto out_unlock; @@ -281,13 +283,16 @@ static int kvmppc_create_pte(struct kvm *kvm, pte_t pte, unsigned long gpa, new_ptep = NULL; } ptep = pte_offset_kernel(pmd, gpa); - if (pte_present(*ptep)) { + old_pte = *ptep; + if (pte_present(old_pte)) { /* PTE was previously valid, so invalidate it */ - old = kvmppc_radix_update_pte(kvm, ptep, _PAGE_PRESENT, - 0, gpa, 0); - kvmppc_radix_tlbie_page(kvm, gpa, 0); - if (old & _PAGE_DIRTY) - mark_page_dirty(kvm, gpa >> PAGE_SHIFT); + WARN_ON_ONCE(pte_pfn(old_pte) != pte_pfn(pte)); + if (pte_val(old_pte) == pte_val(pte)) { + ret = -EAGAIN; + goto out_unlock; + } + kvmppc_radix_update_pte(kvm, ptep, 0, + pte_val(pte), gpa, 0); } kvmppc_radix_set_pte_at(kvm, gpa, ptep, pte); } else {