From patchwork Mon Dec 10 07:00:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Russell Currey X-Patchwork-Id: 1010283 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43CvGT1r12z9s47 for ; Mon, 10 Dec 2018 18:05:05 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=russell.cc Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.b="Q+8qUHAp"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 43CvGT052bzDr0P for ; Mon, 10 Dec 2018 18:05:05 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=russell.cc Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.b="Q+8qUHAp"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (helo) smtp.helo=out1-smtp.messagingengine.com (client-ip=66.111.4.25; helo=out1-smtp.messagingengine.com; envelope-from=ruscur@russell.cc; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=russell.cc Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.b="Q+8qUHAp"; dkim-atps=neutral Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 43Cv9F6VKNzDqsj for ; Mon, 10 Dec 2018 18:00:33 +1100 (AEDT) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 248CE21D0C; Mon, 10 Dec 2018 02:00:28 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Mon, 10 Dec 2018 02:00:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=jXZseDeB0rjmPwzPlsXPjsGfNInIjhjIkcFiU8pGsdg=; b=Q+8qUHAp 0xb4/iPhF8/IDqrnv3dl4nOPD08kyTrKXX9EES3DdeKdAvoi4HvhrU6jwwZAOU9P ZOYJb7iHeRe6tfv0CY9AizZ2scZIATctIxNwWpRMpnsTeezk+0em1QSqRE3ZNMxv sVkBiMk03zkDHp/7rNXkgNMtxWX9Z8DuAcfd21yw9vT7WxqMmuP7G4GLAkBVlZJn lgdDTMI4Rni9olksEX2mhtE/LMFrSaRdRC9F5ptHIbp1rl67bcuVnuv7zwF9zdWF B4QdzuCzAJsQ4CVwAGJtdoKxiBYSz3v2H90c7wSxJ0MrftUma9Yl/E2kjgRKLgSv bNNeNUrhdzSfWA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtkedrudeggedguddtiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfhuthenuceurghilhhouhhtmecu fedttdenucgfrhhlucfvnfffucdlfedtmdenucfjughrpefhvffufffkofgjfhgggfestd ekredtredttdenucfhrhhomheptfhushhsvghllhcuvehurhhrvgihuceorhhushgtuhhr sehruhhsshgvlhhlrdgttgeqnecukfhppeduvddvrdelledrkedvrddutdenucfrrghrrg hmpehmrghilhhfrhhomheprhhushgtuhhrsehruhhsshgvlhhlrdgttgenucevlhhushht vghrufhiiigvpedt X-ME-Proxy: Received: from crackle.ozlabs.ibm.com (unknown [122.99.82.10]) by mail.messagingengine.com (Postfix) with ESMTPA id 39F72E4892; Mon, 10 Dec 2018 02:00:24 -0500 (EST) From: Russell Currey To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 1/3] powerpc/mm/radix: Use KUEP API for Radix MMU Date: Mon, 10 Dec 2018 18:00:42 +1100 Message-Id: <20181210070044.27503-2-ruscur@russell.cc> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181210070044.27503-1-ruscur@russell.cc> References: <20181210070044.27503-1-ruscur@russell.cc> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mikey@neuling.org, kernel-hardening@lists.openwall.com, Russell Currey , npiggin@gmail.com Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Execution protection already exists on radix, this just refactors the radix init to provide the KUEP setup function instead. Thus, the only functional change is that it can now be disabled. Signed-off-by: Russell Currey --- arch/powerpc/mm/pgtable-radix.c | 11 ++++++++--- arch/powerpc/platforms/Kconfig.cputype | 1 + 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/mm/pgtable-radix.c b/arch/powerpc/mm/pgtable-radix.c index 931156069a81..3565e266994b 100644 --- a/arch/powerpc/mm/pgtable-radix.c +++ b/arch/powerpc/mm/pgtable-radix.c @@ -535,8 +535,14 @@ static void radix_init_amor(void) mtspr(SPRN_AMOR, (3ul << 62)); } -static void radix_init_iamr(void) +#ifdef CONFIG_PPC_KUEP +void __init setup_kuep(bool disabled) { + if (disabled || !early_radix_enabled()) + return; + + pr_warn("Activating Kernel Userspace Execution Prevention\n"); + /* * Radix always uses key0 of the IAMR to determine if an access is * allowed. We set bit 0 (IBM bit 1) of key0, to prevent instruction @@ -544,6 +550,7 @@ static void radix_init_iamr(void) */ mtspr(SPRN_IAMR, (1ul << 62)); } +#endif void __init radix__early_init_mmu(void) { @@ -605,7 +612,6 @@ void __init radix__early_init_mmu(void) memblock_set_current_limit(MEMBLOCK_ALLOC_ANYWHERE); - radix_init_iamr(); radix_init_pgtable(); /* Switch to the guard PID before turning on MMU */ radix__switch_mmu_context(NULL, &init_mm); @@ -627,7 +633,6 @@ void radix__early_init_mmu_secondary(void) __pa(partition_tb) | (PATB_SIZE_SHIFT - 12)); radix_init_amor(); } - radix_init_iamr(); radix__switch_mmu_context(NULL, &init_mm); if (cpu_has_feature(CPU_FTR_HVMODE)) diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype index 9997b5ea5693..48cc8df0fdd2 100644 --- a/arch/powerpc/platforms/Kconfig.cputype +++ b/arch/powerpc/platforms/Kconfig.cputype @@ -335,6 +335,7 @@ config PPC_RADIX_MMU bool "Radix MMU Support" depends on PPC_BOOK3S_64 select ARCH_HAS_GIGANTIC_PAGE if (MEMORY_ISOLATION && COMPACTION) || CMA + select PPC_HAVE_KUEP default y help Enable support for the Power ISA 3.0 Radix style MMU. Currently this From patchwork Mon Dec 10 07:00:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Russell Currey X-Patchwork-Id: 1010282 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43CvCj1yM3z9s47 for ; Mon, 10 Dec 2018 18:02:41 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=russell.cc Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.b="QuDg/L9e"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 43CvCj0Bp4zDqSt for ; Mon, 10 Dec 2018 18:02:41 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=russell.cc Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.b="QuDg/L9e"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (helo) smtp.helo=out1-smtp.messagingengine.com (client-ip=66.111.4.25; helo=out1-smtp.messagingengine.com; envelope-from=ruscur@russell.cc; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=russell.cc Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.b="QuDg/L9e"; dkim-atps=neutral Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 43Cv9F6gjPzDqst for ; Mon, 10 Dec 2018 18:00:33 +1100 (AEDT) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 40F7A2073F; Mon, 10 Dec 2018 02:00:31 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Mon, 10 Dec 2018 02:00:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=WuMQsQw9Zm0J7cVhI10l8/CcKE+X5cLFWxbKFpS/UJo=; b=QuDg/L9e Q64EqnGq1IN8xkIO93lt5qVLlsUiTqWcdu+BYqmCzDKkSjLQPlFRMUsKGChi/n+y /J6U2lSdzhT8oJC54vSFHABa8ENq7bqsALGQa78PmBN7UFIZx+t9Xw1unn86nBkR ylLEnQoZzZRQ0b5/JfK/jqiGvEwMtSdo8PK6VjMais9LbBJrUp6x9b6Vn8KPfoU3 JU9vpk5WTyLThE/pFtkOJFCUJZTp2MLDPrYsdDo36AOQ897VWMRo8k51R7jS4Yxp djh7jNAv/INJpBill76EdbO8JMS9kt0tQb7aIDA4hGx2hnBVX5Mk+NtSkvMLy7zQ JMctXFVsKonz+w== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtkedrudeggedguddtiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfhuthenuceurghilhhouhhtmecu fedttdenucgfrhhlucfvnfffucdlfedtmdenucfjughrpefhvffufffkofgjfhgggfestd ekredtredttdenucfhrhhomheptfhushhsvghllhcuvehurhhrvgihuceorhhushgtuhhr sehruhhsshgvlhhlrdgttgeqnecukfhppeduvddvrdelledrkedvrddutdenucfrrghrrg hmpehmrghilhhfrhhomheprhhushgtuhhrsehruhhsshgvlhhlrdgttgenucevlhhushht vghrufhiiigvpedu X-ME-Proxy: Received: from crackle.ozlabs.ibm.com (unknown [122.99.82.10]) by mail.messagingengine.com (Postfix) with ESMTPA id 5267DE4445; Mon, 10 Dec 2018 02:00:28 -0500 (EST) From: Russell Currey To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 2/3] powerpc/lib: Refactor __patch_instruction() to use __put_user_asm() Date: Mon, 10 Dec 2018 18:00:43 +1100 Message-Id: <20181210070044.27503-3-ruscur@russell.cc> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181210070044.27503-1-ruscur@russell.cc> References: <20181210070044.27503-1-ruscur@russell.cc> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mikey@neuling.org, kernel-hardening@lists.openwall.com, Russell Currey , npiggin@gmail.com Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" __patch_instruction() is called in early boot, and uses __put_user_size(), which includes the locks and unlocks for KUAP, which could either be called too early, or in the Radix case, forced to use "early_" versions of functions just to safely handle this one case. __put_user_asm() does not do this, and thus is safe to use both in early boot, and later on since in this case it should only ever be touching kernel memory. __patch_instruction() was previously refactored to use __put_user_size() in order to be able to return -EFAULT, which would allow the kernel to patch instructions in userspace, which should never happen. This has the functional change of causing faults on userspace addresses if KUAP is turned on, which should never happen in practice. A future enhancement could be to double check the patch address is definitely allowed to be tampered with by the kernel. Signed-off-by: Russell Currey --- arch/powerpc/lib/code-patching.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c index 89502cbccb1b..15e8c6339960 100644 --- a/arch/powerpc/lib/code-patching.c +++ b/arch/powerpc/lib/code-patching.c @@ -26,9 +26,9 @@ static int __patch_instruction(unsigned int *exec_addr, unsigned int instr, unsigned int *patch_addr) { - int err; + int err = 0; - __put_user_size(instr, patch_addr, 4, err); + __put_user_asm(instr, patch_addr, err, "stw"); if (err) return err; From patchwork Mon Dec 10 07:00:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Russell Currey X-Patchwork-Id: 1010286 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43CvN70PL7z9s47 for ; Mon, 10 Dec 2018 18:09:59 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=russell.cc Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.b="SjkBGwOD"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 43CvN65mHqzDqxQ for ; Mon, 10 Dec 2018 18:09:58 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=russell.cc Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.b="SjkBGwOD"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (helo) smtp.helo=out1-smtp.messagingengine.com (client-ip=66.111.4.25; helo=out1-smtp.messagingengine.com; envelope-from=ruscur@russell.cc; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=russell.cc Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.b="SjkBGwOD"; dkim-atps=neutral Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 43Cv9K1VqZzDqsQ for ; Mon, 10 Dec 2018 18:00:36 +1100 (AEDT) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 817E221CF3; Mon, 10 Dec 2018 02:00:34 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Mon, 10 Dec 2018 02:00:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=73sbAKTf02ya4avlHYPn17fRJV8GVDmSDzOrPjc9Y3A=; b=SjkBGwOD P0MRR3NztDZB7ajatvLMaLKVMoKdhI7rvngyATGabMRJrY0bwRye24nH+nSYhS3h dsKxzvuKaf5p/CnB3s209XKX8W4trqFRSpHYyFtvySiHfevv6vQD7ebK+CKAsFxP /POuH+K+eUgNrimVHDMoENS09j2PxJ7XFfJ4wcW0O857oT4TAjFi5McX6Pv2PL+u XBSKHV3WqQ+Mq7ioJBBs3Wpsec7QVIZH5EyE5FAqKYg6g73iBJIbhxMBpE+fpQIH 5idOoMf1e3WwXrcaj52bMxhT+bofKYNyKnDau7n3A/Z9D2K+b5KcKcihjPazUYfu Rci4wvH4RVLQYw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtkedrudeggedguddtiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfhuthenuceurghilhhouhhtmecu fedttdenucgfrhhlucfvnfffucdlfedtmdenucfjughrpefhvffufffkofgjfhgggfestd ekredtredttdenucfhrhhomheptfhushhsvghllhcuvehurhhrvgihuceorhhushgtuhhr sehruhhsshgvlhhlrdgttgeqnecukfhppeduvddvrdelledrkedvrddutdenucfrrghrrg hmpehmrghilhhfrhhomheprhhushgtuhhrsehruhhsshgvlhhlrdgttgenucevlhhushht vghrufhiiigvpedu X-ME-Proxy: Received: from crackle.ozlabs.ibm.com (unknown [122.99.82.10]) by mail.messagingengine.com (Postfix) with ESMTPA id 6FEADE4445; Mon, 10 Dec 2018 02:00:31 -0500 (EST) From: Russell Currey To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 3/3] powerpc/64s: Implement KUAP for Radix MMU Date: Mon, 10 Dec 2018 18:00:44 +1100 Message-Id: <20181210070044.27503-4-ruscur@russell.cc> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181210070044.27503-1-ruscur@russell.cc> References: <20181210070044.27503-1-ruscur@russell.cc> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mikey@neuling.org, kernel-hardening@lists.openwall.com, Russell Currey , npiggin@gmail.com Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Kernel Userspace Access Prevention utilises a feature of the Radix MMU which disallows read and write access to userspace addresses. By utilising this, the kernel is prevented from accessing user data from outside of trusted paths that perform proper safety checks, such as copy_{to/from}_user() and friends. Userspace access is disabled from early boot and is only enabled when: - exiting the kernel and entering userspace - performing an operation like copy_{to/from}_user() - context switching to a process that has access enabled and similarly, access is disabled again when exiting userspace and entering the kernel. This feature has a slight performance impact which I roughly measured to be 3% slower in the worst case (performing 1GB of 1 byte read()/write() syscalls), and is gated behind the CONFIG_PPC_KUAP option for performance-critical builds. This feature can be tested by using the lkdtm driver (CONFIG_LKDTM=y) and performing the following: echo ACCESS_USERSPACE > [debugfs]/provoke-crash/DIRECT if enabled, this should send SIGSEGV to the thread. The KUAP state is tracked in the PACA because reading the register that manages these accesses is costly. This Has the unfortunate downside of another layer of abstraction for platforms that implement the locks and unlocks, but this could be useful in future for other things too, like counters for benchmarking or smartly handling lots of small accesses at once. Signed-off-by: Russell Currey --- .../powerpc/include/asm/book3s/64/kup-radix.h | 36 +++++++++++++++++++ arch/powerpc/include/asm/exception-64s.h | 15 ++++++-- arch/powerpc/include/asm/kup.h | 3 ++ arch/powerpc/include/asm/mmu.h | 9 ++++- arch/powerpc/include/asm/reg.h | 1 + arch/powerpc/mm/pgtable-radix.c | 14 ++++++++ arch/powerpc/mm/pkeys.c | 7 ++-- arch/powerpc/platforms/Kconfig.cputype | 1 + 8 files changed, 81 insertions(+), 5 deletions(-) create mode 100644 arch/powerpc/include/asm/book3s/64/kup-radix.h diff --git a/arch/powerpc/include/asm/book3s/64/kup-radix.h b/arch/powerpc/include/asm/book3s/64/kup-radix.h new file mode 100644 index 000000000000..93273ca99310 --- /dev/null +++ b/arch/powerpc/include/asm/book3s/64/kup-radix.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_POWERPC_KUP_RADIX_H +#define _ASM_POWERPC_KUP_RADIX_H + +#ifndef __ASSEMBLY__ +#ifdef CONFIG_PPC_KUAP +#include +/* + * We do have the ability to individually lock/unlock reads and writes rather + * than both at once, however it's a significant performance hit due to needing + * to do a read-modify-write, which adds a mfspr, which is slow. As a result, + * locking/unlocking both at once is preferred. + */ +static inline void unlock_user_access(void __user *to, const void __user *from, + unsigned long size) +{ + if (!mmu_has_feature(MMU_FTR_RADIX_KUAP)) + return; + + mtspr(SPRN_AMR, 0); + isync(); + get_paca()->user_access_allowed = 1; +} + +static inline void lock_user_access(void __user *to, const void __user *from, + unsigned long size) +{ + if (!mmu_has_feature(MMU_FTR_RADIX_KUAP)) + return; + + mtspr(SPRN_AMR, AMR_LOCKED); + get_paca()->user_access_allowed = 0; +} +#endif /* CONFIG_PPC_KUAP */ +#endif /* __ASSEMBLY__ */ +#endif diff --git a/arch/powerpc/include/asm/exception-64s.h b/arch/powerpc/include/asm/exception-64s.h index 4d971ca1e69b..0ed4923c1282 100644 --- a/arch/powerpc/include/asm/exception-64s.h +++ b/arch/powerpc/include/asm/exception-64s.h @@ -264,8 +264,19 @@ BEGIN_FTR_SECTION_NESTED(943) \ std ra,offset(r13); \ END_FTR_SECTION_NESTED(ftr,ftr,943) -#define LOCK_USER_ACCESS(reg) -#define UNLOCK_USER_ACCESS(reg) +#define LOCK_USER_ACCESS(reg) \ +BEGIN_MMU_FTR_SECTION_NESTED(944) \ + lis reg,(AMR_LOCKED)@highest; \ + rldicr reg,reg,32,31; \ + mtspr SPRN_AMR,reg; \ +END_MMU_FTR_SECTION_NESTED(MMU_FTR_RADIX_KUAP,MMU_FTR_RADIX_KUAP,944) + +#define UNLOCK_USER_ACCESS(reg) \ +BEGIN_MMU_FTR_SECTION_NESTED(945) \ + li reg,0; \ + mtspr SPRN_AMR,reg; \ + isync; \ +END_MMU_FTR_SECTION_NESTED(MMU_FTR_RADIX_KUAP,MMU_FTR_RADIX_KUAP,945) #define EXCEPTION_PROLOG_0(area) \ GET_PACA(r13); \ diff --git a/arch/powerpc/include/asm/kup.h b/arch/powerpc/include/asm/kup.h index 16ff3f9da29c..7813e2bcfb7c 100644 --- a/arch/powerpc/include/asm/kup.h +++ b/arch/powerpc/include/asm/kup.h @@ -8,6 +8,9 @@ #ifdef CONFIG_PPC_BOOK3S_32 #include #endif +#ifdef CONFIG_PPC_BOOK3S_64 +#include +#endif #ifndef __ASSEMBLY__ diff --git a/arch/powerpc/include/asm/mmu.h b/arch/powerpc/include/asm/mmu.h index eb20eb3b8fb0..048df188fc10 100644 --- a/arch/powerpc/include/asm/mmu.h +++ b/arch/powerpc/include/asm/mmu.h @@ -107,6 +107,10 @@ */ #define MMU_FTR_1T_SEGMENT ASM_CONST(0x40000000) +/* Supports KUAP (key 0 controlling userspace addresses) on radix + */ +#define MMU_FTR_RADIX_KUAP ASM_CONST(0x80000000) + /* MMU feature bit sets for various CPUs */ #define MMU_FTRS_DEFAULT_HPTE_ARCH_V2 \ MMU_FTR_HPTE_TABLE | MMU_FTR_PPCAS_ARCH_V2 @@ -143,7 +147,10 @@ enum { MMU_FTR_KERNEL_RO | MMU_FTR_68_BIT_VA | #ifdef CONFIG_PPC_RADIX_MMU MMU_FTR_TYPE_RADIX | -#endif +#ifdef CONFIG_PPC_KUAP + MMU_FTR_RADIX_KUAP | +#endif /* CONFIG_PPC_KUAP */ +#endif /* CONFIG_PPC_RADIX_MMU */ 0, }; diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h index c9c382e57017..6b5d2a61af5a 100644 --- a/arch/powerpc/include/asm/reg.h +++ b/arch/powerpc/include/asm/reg.h @@ -246,6 +246,7 @@ #define SPRN_DSCR 0x11 #define SPRN_CFAR 0x1c /* Come From Address Register */ #define SPRN_AMR 0x1d /* Authority Mask Register */ +#define AMR_LOCKED 0xC000000000000000UL /* Read & Write disabled */ #define SPRN_UAMOR 0x9d /* User Authority Mask Override Register */ #define SPRN_AMOR 0x15d /* Authority Mask Override Register */ #define SPRN_ACOP 0x1F /* Available Coprocessor Register */ diff --git a/arch/powerpc/mm/pgtable-radix.c b/arch/powerpc/mm/pgtable-radix.c index 3565e266994b..e248f330d6fd 100644 --- a/arch/powerpc/mm/pgtable-radix.c +++ b/arch/powerpc/mm/pgtable-radix.c @@ -29,6 +29,7 @@ #include #include #include +#include #include @@ -552,6 +553,19 @@ void __init setup_kuep(bool disabled) } #endif +#ifdef CONFIG_PPC_KUAP +void __init setup_kuap(bool disabled) +{ + if (disabled || !early_radix_enabled()) + return; + + pr_info("Activating Kernel Userspace Access Prevention\n"); + + cur_cpu_spec->mmu_features |= MMU_FTR_RADIX_KUAP; + mtspr(SPRN_AMR, AMR_LOCKED); +} +#endif + void __init radix__early_init_mmu(void) { unsigned long lpcr; diff --git a/arch/powerpc/mm/pkeys.c b/arch/powerpc/mm/pkeys.c index b271b283c785..bb3cf915016f 100644 --- a/arch/powerpc/mm/pkeys.c +++ b/arch/powerpc/mm/pkeys.c @@ -7,6 +7,7 @@ #include #include +#include #include #include @@ -266,7 +267,8 @@ int __arch_set_user_pkey_access(struct task_struct *tsk, int pkey, void thread_pkey_regs_save(struct thread_struct *thread) { - if (static_branch_likely(&pkey_disabled)) + if (static_branch_likely(&pkey_disabled) && + !mmu_has_feature(MMU_FTR_RADIX_KUAP)) return; /* @@ -280,7 +282,8 @@ void thread_pkey_regs_save(struct thread_struct *thread) void thread_pkey_regs_restore(struct thread_struct *new_thread, struct thread_struct *old_thread) { - if (static_branch_likely(&pkey_disabled)) + if (static_branch_likely(&pkey_disabled) && + !mmu_has_feature(MMU_FTR_RADIX_KUAP)) return; if (old_thread->amr != new_thread->amr) diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype index 48cc8df0fdd2..0b9a8eda413a 100644 --- a/arch/powerpc/platforms/Kconfig.cputype +++ b/arch/powerpc/platforms/Kconfig.cputype @@ -336,6 +336,7 @@ config PPC_RADIX_MMU depends on PPC_BOOK3S_64 select ARCH_HAS_GIGANTIC_PAGE if (MEMORY_ISOLATION && COMPACTION) || CMA select PPC_HAVE_KUEP + select PPC_HAVE_KUAP default y help Enable support for the Power ISA 3.0 Radix style MMU. Currently this