From patchwork Fri Mar 14 00:00:43 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Scott Wood X-Patchwork-Id: 330153 X-Patchwork-Delegate: scottwood@freescale.com Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from ozlabs.org (localhost [IPv6:::1]) by ozlabs.org (Postfix) with ESMTP id 456352C07E5 for ; Fri, 14 Mar 2014 11:04:39 +1100 (EST) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2lp0206.outbound.protection.outlook.com [207.46.163.206]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id A5E992C0103 for ; Fri, 14 Mar 2014 11:01:17 +1100 (EST) Received: from snotra.am.freescale.net (192.88.168.49) by BY2PR03MB395.namprd03.prod.outlook.com (10.141.141.14) with Microsoft SMTP Server (TLS) id 15.0.898.11; Fri, 14 Mar 2014 00:01:09 +0000 From: Scott Wood To: Benjamin Herrenschmidt Subject: [PATCH 04/10] powerpc/e6500: Make TLB lock recursive Date: Thu, 13 Mar 2014 19:00:43 -0500 Message-ID: <1394755249-8856-5-git-send-email-scottwood@freescale.com> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1394755249-8856-1-git-send-email-scottwood@freescale.com> References: <1394755249-8856-1-git-send-email-scottwood@freescale.com> MIME-Version: 1.0 X-Originating-IP: [192.88.168.49] X-ClientProxiedBy: DM2PR04CA004.namprd04.prod.outlook.com (10.141.96.14) To BY2PR03MB395.namprd03.prod.outlook.com (10.141.141.14) X-Forefront-PRVS: 0150F3F97D X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009001)(6009001)(428001)(199002)(189002)(90146001)(56816005)(92726001)(33646001)(76482001)(42186004)(51856001)(92566001)(53806001)(4396001)(50226001)(83072002)(47976001)(46102001)(49866001)(47736001)(63696002)(85852003)(50986001)(97186001)(47776003)(65816001)(97336001)(20776003)(89996001)(80022001)(66066001)(79102001)(54316002)(56776001)(59766001)(95666003)(77982001)(94316002)(86362001)(93516002)(95416001)(94946001)(36756003)(76786001)(77096001)(77156001)(76796001)(81342001)(69226001)(62966002)(81542001)(93136001)(50466002)(74876001)(74706001)(74502001)(47446002)(31966008)(74662001)(48376002)(74366001)(81686001)(19580405001)(83322001)(19580395003)(80976001)(81816001)(87286001)(87976001)(87266001)(85306002); DIR:OUT; SFP:1101; SCL:1; SRVR:BY2PR03MB395; H:snotra.am.freescale.net; FPR:901AFEFE.2982C788.71F195F7.CA04DACC.203C3; MLV:sfv; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (: freescale.com does not designate permitted sender hosts) X-OriginatorOrg: freescale.com Cc: Scott Wood , Tiejun Chen , linuxppc-dev@lists.ozlabs.org X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Once special level interrupts are supported, we may take nested TLB misses -- so allow the same thread to acquire the lock recursively. The lock will not be effective against the nested TLB miss handler trying to write the same entry as the interrupted TLB miss handler, but that's also a problem on non-threaded CPUs that lack TLB write conditional. This will be addressed in the patch that enables crit/mc support by invalidating the TLB on return from level exceptions. Signed-off-by: Scott Wood --- arch/powerpc/include/asm/mmu-book3e.h | 9 ++++++--- arch/powerpc/kernel/setup_64.c | 2 ++ arch/powerpc/mm/tlb_low_64e.S | 19 ++++++++++++------- 3 files changed, 20 insertions(+), 10 deletions(-) diff --git a/arch/powerpc/include/asm/mmu-book3e.h b/arch/powerpc/include/asm/mmu-book3e.h index 89b785d..901dac6 100644 --- a/arch/powerpc/include/asm/mmu-book3e.h +++ b/arch/powerpc/include/asm/mmu-book3e.h @@ -287,11 +287,14 @@ extern int mmu_linear_psize; extern int mmu_vmemmap_psize; struct tlb_core_data { + /* + * Per-core spinlock for e6500 TLB handlers (no tlbsrx.) + * Must be the first struct element. + */ + u8 lock; + /* For software way selection, as on Freescale TLB1 */ u8 esel_next, esel_max, esel_first; - - /* Per-core spinlock for e6500 TLB handlers (no tlbsrx.) */ - u8 lock; }; #ifdef CONFIG_PPC64 diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c index da9c42f..4933909 100644 --- a/arch/powerpc/kernel/setup_64.c +++ b/arch/powerpc/kernel/setup_64.c @@ -102,6 +102,8 @@ static void setup_tlb_core_data(void) { int cpu; + BUILD_BUG_ON(offsetof(struct tlb_core_data, lock) != 0); + for_each_possible_cpu(cpu) { int first = cpu_first_thread_sibling(cpu); diff --git a/arch/powerpc/mm/tlb_low_64e.S b/arch/powerpc/mm/tlb_low_64e.S index 6bf5050..1e50249 100644 --- a/arch/powerpc/mm/tlb_low_64e.S +++ b/arch/powerpc/mm/tlb_low_64e.S @@ -284,7 +284,7 @@ itlb_miss_fault_bolted: * r14 = page table base * r13 = PACA * r11 = tlb_per_core ptr - * r10 = crap (free to use) + * r10 = cpu number */ tlb_miss_common_e6500: /* @@ -293,15 +293,18 @@ tlb_miss_common_e6500: * * MAS6:IND should be already set based on MAS4 */ - addi r10,r11,TCD_LOCK -1: lbarx r15,0,r10 +1: lbarx r15,0,r11 + lhz r10,PACAPACAINDEX(r13) cmpdi r15,0 + cmpdi cr1,r15,1 /* set cr1.eq = 0 for non-recursive */ bne 2f - li r15,1 - stbcx. r15,0,r10 + stbcx. r10,0,r11 bne 1b +3: .subsection 1 -2: lbz r15,0(r10) +2: cmpd cr1,r15,r10 /* recursive lock due to mcheck/crit/etc? */ + beq cr1,3b /* unlock will happen if cr1.eq = 0 */ + lbz r15,0(r11) cmpdi r15,0 bne 2b b 1b @@ -379,9 +382,11 @@ tlb_miss_common_e6500: tlb_miss_done_e6500: .macro tlb_unlock_e6500 + beq cr1,1f /* no unlock if lock was recursively grabbed */ li r15,0 isync - stb r15,TCD_LOCK(r11) + stb r15,0(r11) +1: .endm tlb_unlock_e6500