From patchwork Wed May 2 20:39:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: dann frazier X-Patchwork-Id: 907704 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) by ozlabs.org (Postfix) with ESMTP id 40bqsB2zVnz9s4s; Thu, 3 May 2018 06:39:58 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1fDyXW-0007VU-Lg; Wed, 02 May 2018 20:39:50 +0000 Received: from complete.lackof.org ([198.49.126.79]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1fDyXS-0007TM-4s for kernel-team@lists.ubuntu.com; Wed, 02 May 2018 20:39:46 +0000 Received: from localhost (c-73-78-137-212.hsd1.co.comcast.net [73.78.137.212]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by complete.lackof.org (Postfix) with ESMTPSA id 9762D33E00F7 for ; Wed, 2 May 2018 14:39:43 -0600 (MDT) From: dann frazier To: kernel-team@lists.ubuntu.com Subject: [PATCH 1/2][SRU Bionic] arm64: mmu: add the entry trampolines start/end section markers into sections.h Date: Wed, 2 May 2018 14:39:17 -0600 Message-Id: <20180502203918.13324-2-dann.frazier@canonical.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180502203918.13324-1-dann.frazier@canonical.com> References: <20180502203918.13324-1-dann.frazier@canonical.com> X-Virus-Scanned: clamav-milter 0.99.4 at complete.lackof.org X-Virus-Status: Clean X-Spam-Status: No, score=0.3 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, UNPARSEABLE_RELAY autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on complete.lackof.org X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: James Morse BugLink: https://bugs.launchpad.net/bugs/1768630 SDEI needs to calculate an offset in the trampoline page too. Move the extern char[] to sections.h. This patch just moves code around. Signed-off-by: James Morse Signed-off-by: Catalin Marinas (cherry picked from commit 83f8ee3a73f551aebb5b0bb029feaf6936615730) Signed-off-by: dann frazier --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/mm/mmu.c | 2 -- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 941267caa39c..caab039d6305 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -28,5 +28,6 @@ extern char __initdata_begin[], __initdata_end[]; extern char __inittext_begin[], __inittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; +extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index f15dc3dfecf8..d11d43d20633 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -532,8 +532,6 @@ early_param("rodata", parse_rodata); #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 static int __init map_entry_trampoline(void) { - extern char __entry_tramp_text_start[]; - pgprot_t prot = rodata_enabled ? PAGE_KERNEL_ROX : PAGE_KERNEL_EXEC; phys_addr_t pa_start = __pa_symbol(__entry_tramp_text_start); From patchwork Wed May 2 20:39:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: dann frazier X-Patchwork-Id: 907706 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) by ozlabs.org (Postfix) with ESMTP id 40bqsC3zWnz9s73; Thu, 3 May 2018 06:39:59 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1fDyXX-0007Vh-QU; Wed, 02 May 2018 20:39:51 +0000 Received: from complete.lackof.org ([198.49.126.79]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1fDyXS-0007TN-4r for kernel-team@lists.ubuntu.com; Wed, 02 May 2018 20:39:46 +0000 Received: from localhost (c-73-78-137-212.hsd1.co.comcast.net [73.78.137.212]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by complete.lackof.org (Postfix) with ESMTPSA id 3D45633E01A2 for ; Wed, 2 May 2018 14:39:44 -0600 (MDT) From: dann frazier To: kernel-team@lists.ubuntu.com Subject: [PATCH 2/2][SRU Bionic] arm64: sdei: Add trampoline code for remapping the kernel Date: Wed, 2 May 2018 14:39:18 -0600 Message-Id: <20180502203918.13324-3-dann.frazier@canonical.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180502203918.13324-1-dann.frazier@canonical.com> References: <20180502203918.13324-1-dann.frazier@canonical.com> X-Virus-Scanned: clamav-milter 0.99.4 at complete.lackof.org X-Virus-Status: Clean X-Spam-Status: No, score=0.3 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, UNPARSEABLE_RELAY autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on complete.lackof.org X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: James Morse BugLink: https://bugs.launchpad.net/bugs/1768630 When CONFIG_UNMAP_KERNEL_AT_EL0 is set the SDEI entry point and the rest of the kernel may be unmapped when we take an event. If this may be the case, use an entry trampoline that can switch to the kernel page tables. We can't use the provided PSTATE to determine whether to switch page tables as we may have interrupted the kernel's entry trampoline, (or a normal-priority event that interrupted the kernel's entry trampoline). Instead test for a user ASID in ttbr1_el1. Save a value in regs->addr_limit to indicate whether we need to restore the original ASID when returning from this event. This value is only used by do_page_fault(), which we don't call with the SDEI regs. Signed-off-by: James Morse Signed-off-by: Catalin Marinas (cherry picked from commit 79e9aa59dc29a995921fb01e64cd36b73cf5abe0) Signed-off-by: dann frazier --- arch/arm64/include/asm/mmu.h | 3 +- arch/arm64/include/asm/sdei.h | 6 +++ arch/arm64/kernel/entry.S | 98 +++++++++++++++++++++++++++++++---- arch/arm64/kernel/sdei.c | 20 ++++++- 4 files changed, 113 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 6dd83d75b82a..a050d4f3615d 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -17,7 +17,8 @@ #define __ASM_MMU_H #define MMCF_AARCH32 0x1 /* mm context flag for AArch32 executables */ -#define USER_ASID_FLAG (UL(1) << 48) +#define USER_ASID_BIT 48 +#define USER_ASID_FLAG (UL(1) << USER_ASID_BIT) #define TTBR_ASID_MASK (UL(0xffff) << 48) #ifndef __ASSEMBLY__ diff --git a/arch/arm64/include/asm/sdei.h b/arch/arm64/include/asm/sdei.h index d58a31ab525a..e073e6886685 100644 --- a/arch/arm64/include/asm/sdei.h +++ b/arch/arm64/include/asm/sdei.h @@ -23,6 +23,12 @@ extern unsigned long sdei_exit_mode; asmlinkage void __sdei_asm_handler(unsigned long event_num, unsigned long arg, unsigned long pc, unsigned long pstate); +/* and its CONFIG_UNMAP_KERNEL_AT_EL0 trampoline */ +asmlinkage void __sdei_asm_entry_trampoline(unsigned long event_num, + unsigned long arg, + unsigned long pc, + unsigned long pstate); + /* * The above entry point does the minimum to call C code. This function does * anything else, before calling the driver. diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index bae2daf5d486..f730baf124f0 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -1162,6 +1162,78 @@ NOKPROBE(ret_from_fork) #include #include +.macro sdei_handler_exit exit_mode + /* On success, this call never returns... */ + cmp \exit_mode, #SDEI_EXIT_SMC + b.ne 99f + smc #0 + b . +99: hvc #0 + b . +.endm + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +/* + * The regular SDEI entry point may have been unmapped along with the rest of + * the kernel. This trampoline restores the kernel mapping to make the x1 memory + * argument accessible. + * + * This clobbers x4, __sdei_handler() will restore this from firmware's + * copy. + */ +.ltorg +.pushsection ".entry.tramp.text", "ax" +ENTRY(__sdei_asm_entry_trampoline) + mrs x4, ttbr1_el1 + tbz x4, #USER_ASID_BIT, 1f + + tramp_map_kernel tmp=x4 + isb + mov x4, xzr + + /* + * Use reg->interrupted_regs.addr_limit to remember whether to unmap + * the kernel on exit. + */ +1: str x4, [x1, #(SDEI_EVENT_INTREGS + S_ORIG_ADDR_LIMIT)] + +#ifdef CONFIG_RANDOMIZE_BASE + adr x4, tramp_vectors + PAGE_SIZE + add x4, x4, #:lo12:__sdei_asm_trampoline_next_handler + ldr x4, [x4] +#else + ldr x4, =__sdei_asm_handler +#endif + br x4 +ENDPROC(__sdei_asm_entry_trampoline) +NOKPROBE(__sdei_asm_entry_trampoline) + +/* + * Make the exit call and restore the original ttbr1_el1 + * + * x0 & x1: setup for the exit API call + * x2: exit_mode + * x4: struct sdei_registered_event argument from registration time. + */ +ENTRY(__sdei_asm_exit_trampoline) + ldr x4, [x4, #(SDEI_EVENT_INTREGS + S_ORIG_ADDR_LIMIT)] + cbnz x4, 1f + + tramp_unmap_kernel tmp=x4 + +1: sdei_handler_exit exit_mode=x2 +ENDPROC(__sdei_asm_exit_trampoline) +NOKPROBE(__sdei_asm_exit_trampoline) + .ltorg +.popsection // .entry.tramp.text +#ifdef CONFIG_RANDOMIZE_BASE +.pushsection ".rodata", "a" +__sdei_asm_trampoline_next_handler: + .quad __sdei_asm_handler +.popsection // .rodata +#endif /* CONFIG_RANDOMIZE_BASE */ +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ + /* * Software Delegated Exception entry point. * @@ -1169,6 +1241,7 @@ NOKPROBE(ret_from_fork) * x1: struct sdei_registered_event argument from registration time. * x2: interrupted PC * x3: interrupted PSTATE + * x4: maybe clobbered by the trampoline * * Firmware has preserved x0->x17 for us, we must save/restore the rest to * follow SMC-CC. We save (or retrieve) all the registers as the handler may @@ -1234,10 +1307,11 @@ ENTRY(__sdei_asm_handler) msr sp_el0, x28 /* restore regs >x17 that we clobbered */ - ldp x28, x29, [x19, #SDEI_EVENT_INTREGS + 16 * 14] - ldp lr, x4, [x19, #SDEI_EVENT_INTREGS + S_LR] - mov sp, x4 - ldp x18, x19, [x19, #SDEI_EVENT_INTREGS + 16 * 9] + mov x4, x19 // keep x4 for __sdei_asm_exit_trampoline + ldp x28, x29, [x4, #SDEI_EVENT_INTREGS + 16 * 14] + ldp x18, x19, [x4, #SDEI_EVENT_INTREGS + 16 * 9] + ldp lr, x1, [x4, #SDEI_EVENT_INTREGS + S_LR] + mov sp, x1 mov x1, x0 // address to complete_and_resume /* x0 = (x0 <= 1) ? EVENT_COMPLETE:EVENT_COMPLETE_AND_RESUME */ @@ -1246,14 +1320,16 @@ ENTRY(__sdei_asm_handler) mov_q x3, SDEI_1_0_FN_SDEI_EVENT_COMPLETE_AND_RESUME csel x0, x2, x3, ls - /* On success, this call never returns... */ ldr_l x2, sdei_exit_mode - cmp x2, #SDEI_EXIT_SMC - b.ne 1f - smc #0 - b . -1: hvc #0 - b . + +alternative_if_not ARM64_UNMAP_KERNEL_AT_EL0 + sdei_handler_exit exit_mode=x2 +alternative_else_nop_endif + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + tramp_alias dst=x5, sym=__sdei_asm_exit_trampoline + br x5 +#endif ENDPROC(__sdei_asm_handler) NOKPROBE(__sdei_asm_handler) #endif /* CONFIG_ARM_SDE_INTERFACE */ diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index f9dffacaa5d6..6b8d90d5ceae 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -10,7 +10,9 @@ #include #include +#include #include +#include #include #include @@ -124,7 +126,18 @@ unsigned long sdei_arch_get_entry_point(int conduit) } sdei_exit_mode = (conduit == CONDUIT_HVC) ? SDEI_EXIT_HVC : SDEI_EXIT_SMC; - return (unsigned long)__sdei_asm_handler; + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + if (arm64_kernel_unmapped_at_el0()) { + unsigned long offset; + + offset = (unsigned long)__sdei_asm_entry_trampoline - + (unsigned long)__entry_tramp_text_start; + return TRAMP_VALIAS + offset; + } else +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ + return (unsigned long)__sdei_asm_handler; + } /* @@ -138,11 +151,14 @@ static __kprobes unsigned long _sdei_handler(struct pt_regs *regs, { u32 mode; int i, err = 0; - const int clobbered_registers = 4; + int clobbered_registers = 4; u64 elr = read_sysreg(elr_el1); u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */ unsigned long vbar = read_sysreg(vbar_el1); + if (arm64_kernel_unmapped_at_el0()) + clobbered_registers++; + /* Retrieve the missing registers values */ for (i = 0; i < clobbered_registers; i++) { /* from within the handler, this call always succeeds */