From patchwork Wed Nov 10 00:37:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Niethe X-Patchwork-Id: 1553199 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=QLV/LF4O; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HpmCW6Nb5z9sRK for ; Wed, 10 Nov 2021 11:37:43 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HpmCT6Y9lz3bXR for ; Wed, 10 Nov 2021 11:37:41 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=QLV/LF4O; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::62c; helo=mail-pl1-x62c.google.com; envelope-from=jniethe5@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=QLV/LF4O; dkim-atps=neutral Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HpmCG2d70z2xrm for ; Wed, 10 Nov 2021 11:37:28 +1100 (AEDT) Received: by mail-pl1-x62c.google.com with SMTP id r5so1610913pls.1 for ; Tue, 09 Nov 2021 16:37:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wHDlWeOgU30NN6ENKp1b8YZhUqZ6c4bprLjEFL13Z18=; b=QLV/LF4OQfkNNEhLqWNZgFkjZvZ0QhFs+a8yH5fBdgzaNbYRoOzofU3U5W0qZhwZ3a McNoi9kXmN9TQgVHLGC+wKnrxJRabZ7iVxz+LxLGjyZ5VePFThyJqw1uoYs7HxwXeiHb 3x3BKPshGwyiSzv4yRRpdHANooghRCTiFllmEG3XphqoMwZ7/TzrNVaA4d4F/a2RdT+m 71QV8AACo+t7QckhGrG7FD3bKrpnc8s6vQVA3FjVVKWa/O2ogFELjexDFZCbocp3omtb iaEo5m2w8vRfQ7RhUsvCmGIHMmE20EBX19m6ySyoLmZpQV+nUDX2QQKeNIW1rT/jFZ0i 7e7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wHDlWeOgU30NN6ENKp1b8YZhUqZ6c4bprLjEFL13Z18=; b=6KOjailELDB8/T26jY4u7ww/mD2CWY2s+8bPdCpbfAvOhL5Uv7Mn6EVhgfoISM8Hyv JnmAsZLB2eLRl47GNahxN7ODwrg0cfmAz4la0WkuBC6vzm0ZaFV/iIR8LNaYB7fKWLp9 VoQqTwDv20rksHAzckMoYrGdZ6B0P3zCmgZGjUD93Hm4CPB9EvIVY6W2Y/Huoe0HfS/E TpEQKQcGsCwwljt/kybXg9/Ma43FqKGK1lej4HNj+5CMlclGRzXm8tk/0RiT53fEFHB6 RZuLdShwlQ0G+HaURlFakXi01BbL8gJ1styot7lKngBIYQ1FQAoc+Z4xOdfymLpqlxNW ihyw== X-Gm-Message-State: AOAM5308I21oVwVntIiLnGdX3PNgbfsweMK2K6+sRCes8ljiCgAIruQJ 9iI27xKe6otKlOB2RmRdtv+lcIQxSXttHg== X-Google-Smtp-Source: ABdhPJyltz02dr6tIZ16ouqfPzpWn7t7yBnEKKHcq2wDt0+darURf0en6SKaqjaVK/KJmoG7J+WrpA== X-Received: by 2002:a17:90b:1d0f:: with SMTP id on15mr12106785pjb.144.1636504644936; Tue, 09 Nov 2021 16:37:24 -0800 (PST) Received: from tee480.ozlabs.ibm.com (159-196-117-139.9fc475.syd.nbn.aussiebb.net. [159.196.117.139]) by smtp.gmail.com with ESMTPSA id t21sm15854074pgo.12.2021.11.09.16.37.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Nov 2021 16:37:24 -0800 (PST) From: Jordan Niethe To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v7 1/5] powerpc: Allow clearing and restoring registers independent of saved breakpoint state Date: Wed, 10 Nov 2021 11:37:13 +1100 Message-Id: <20211110003717.1150965-2-jniethe5@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211110003717.1150965-1-jniethe5@gmail.com> References: <20211110003717.1150965-1-jniethe5@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jordan Niethe , cmr@bluescreens.de Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" For the coming temporary mm used for instruction patching, the breakpoint registers need to be cleared to prevent them from accidentally being triggered. As soon as the patching is done, the breakpoints will be restored. The breakpoint state is stored in the per cpu variable current_brk[]. Add a pause_breakpoints() function which will clear the breakpoint registers without touching the state in current_bkr[]. Add a pair function unpause_breakpoints() which will move the state in current_brk[] back to the registers. Signed-off-by: Jordan Niethe --- v7: New to series --- arch/powerpc/include/asm/debug.h | 2 ++ arch/powerpc/kernel/process.c | 36 +++++++++++++++++++++++++++++--- 2 files changed, 35 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/include/asm/debug.h b/arch/powerpc/include/asm/debug.h index 86a14736c76c..83f2dc3785e8 100644 --- a/arch/powerpc/include/asm/debug.h +++ b/arch/powerpc/include/asm/debug.h @@ -46,6 +46,8 @@ static inline int debugger_fault_handler(struct pt_regs *regs) { return 0; } #endif void __set_breakpoint(int nr, struct arch_hw_breakpoint *brk); +void pause_breakpoints(void); +void unpause_breakpoints(void); bool ppc_breakpoint_available(void); #ifdef CONFIG_PPC_ADV_DEBUG_REGS extern void do_send_trap(struct pt_regs *regs, unsigned long address, diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c index 406d7ee9e322..22ed72430683 100644 --- a/arch/powerpc/kernel/process.c +++ b/arch/powerpc/kernel/process.c @@ -688,6 +688,7 @@ DEFINE_INTERRUPT_HANDLER(do_break) static DEFINE_PER_CPU(struct arch_hw_breakpoint, current_brk[HBP_NUM_MAX]); + #ifdef CONFIG_PPC_ADV_DEBUG_REGS /* * Set the debug registers back to their default "safe" values. @@ -865,10 +866,8 @@ static inline int set_breakpoint_8xx(struct arch_hw_breakpoint *brk) return 0; } -void __set_breakpoint(int nr, struct arch_hw_breakpoint *brk) +static void ____set_breakpoint(int nr, struct arch_hw_breakpoint *brk) { - memcpy(this_cpu_ptr(¤t_brk[nr]), brk, sizeof(*brk)); - if (dawr_enabled()) // Power8 or later set_dawr(nr, brk); @@ -882,6 +881,12 @@ void __set_breakpoint(int nr, struct arch_hw_breakpoint *brk) WARN_ON_ONCE(1); } +void __set_breakpoint(int nr, struct arch_hw_breakpoint *brk) +{ + memcpy(this_cpu_ptr(¤t_brk[nr]), brk, sizeof(*brk)); + ____set_breakpoint(nr, brk); +} + /* Check if we have DAWR or DABR hardware */ bool ppc_breakpoint_available(void) { @@ -894,6 +899,31 @@ bool ppc_breakpoint_available(void) } EXPORT_SYMBOL_GPL(ppc_breakpoint_available); +/* Disable the breakpoint in hardware without touching current_brk[] */ +void pause_breakpoints(void) +{ + struct arch_hw_breakpoint brk = {0}; + int i; + + if (!ppc_breakpoint_available()) + return; + + for (i = 0; i < nr_wp_slots(); i++) + ____set_breakpoint(i, &brk); +} + +/* Renable the breakpoint in hardware from current_brk[] */ +void unpause_breakpoints(void) +{ + int i; + + if (!ppc_breakpoint_available()) + return; + + for (i = 0; i < nr_wp_slots(); i++) + ____set_breakpoint(i, this_cpu_ptr(¤t_brk[i])); +} + #ifdef CONFIG_PPC_TRANSACTIONAL_MEM static inline bool tm_enabled(struct task_struct *tsk) From patchwork Wed Nov 10 00:37:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Niethe X-Patchwork-Id: 1553201 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=JIhKCnzL; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HpmDx58RYz9sRK for ; Wed, 10 Nov 2021 11:38:57 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HpmDw70vmz3cRg for ; Wed, 10 Nov 2021 11:38:56 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=JIhKCnzL; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::62e; helo=mail-pl1-x62e.google.com; envelope-from=jniethe5@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=JIhKCnzL; dkim-atps=neutral Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HpmCG205dz2xXB for ; Wed, 10 Nov 2021 11:37:29 +1100 (AEDT) Received: by mail-pl1-x62e.google.com with SMTP id v20so1547100plo.7 for ; Tue, 09 Nov 2021 16:37:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hI5qO+iLs1Icw0RwXkga/qBssOGyWJOJPoNMfMvfgV8=; b=JIhKCnzL4MmUOWj2PwsRE8g5vStXgFTj+cEIa1QbBwYuW+OSbmzSRLPyIGu/2PX88X CSCIKoSEdFKoH0BEZm54RQ9BlwBXuxTV/fwB5O+/ujXyeUGUvQkjD4BeGxs4OdHC5EHY skq3s3515Enkd7EWRdbDM7xGWX/z4s4r+LLKYYdhtZ5Az8nYiivezQs5zEl1IvPyZqNt fzxVFDwuDUsiCpLlEir5zSlcqzoVX0E0fEfszepZO1JQqxCj133u8/UQwmdZGMYwYK5P Bl7lWP+ejDUXkhySxHqr+gechs84nzauT2pRf57rsguiNUnxOU2H/9s75LfywJ6fSNjA p4YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hI5qO+iLs1Icw0RwXkga/qBssOGyWJOJPoNMfMvfgV8=; b=GOj+YPx5USXm1I8+X/O11vT64iZUWg9yWj75oDL2dYIcVxOv0dwZm2aJgyCoC35Li9 Tmp605WouxhoDp/1VmT0xMMchQlxaszHXqVu3ooaV6q1z2BM19aS6sXzlc0xmApAKARA NSnJuyHf8+5Qefb9mef12vH7NJBgAopqY4Siq0Avvpx+KPO+5VvKCP/kzAsujm5TKhE9 mJIUTb39aKlTgqS4J3USoha7G55Ho/19W8bSCzYTqwceCM4xJI2BVTKllr5GCjnsJ5yk S9K2UhJ1Jpuk5sM+fk0UGIZtX2OJu9jP6wTiJXjBBg0jUPGsP7i+xuwRAxT7757PSa3B 1rtg== X-Gm-Message-State: AOAM530fq4onbYlcXWvAb2WrO1XMK21l2RAuTkA3BEOrPhSE0x+BBJdn ZhZnAWBLDn+ZqhkCHvZcxl89M5gFzCAJiQ== X-Google-Smtp-Source: ABdhPJycfObs+Ufewi2xFUgQRb4XSXUEKoYlD22zjYeoaPA7e9ie8pMui+6bSuheSBjascXdszjFXA== X-Received: by 2002:a17:90b:4b03:: with SMTP id lx3mr11583817pjb.18.1636504647153; Tue, 09 Nov 2021 16:37:27 -0800 (PST) Received: from tee480.ozlabs.ibm.com (159-196-117-139.9fc475.syd.nbn.aussiebb.net. [159.196.117.139]) by smtp.gmail.com with ESMTPSA id t21sm15854074pgo.12.2021.11.09.16.37.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Nov 2021 16:37:26 -0800 (PST) From: Jordan Niethe To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v7 2/5] powerpc/64s: Introduce temporary mm for Radix MMU Date: Wed, 10 Nov 2021 11:37:14 +1100 Message-Id: <20211110003717.1150965-3-jniethe5@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211110003717.1150965-1-jniethe5@gmail.com> References: <20211110003717.1150965-1-jniethe5@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jordan Niethe , cmr@bluescreens.de Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: "Christopher M. Riedl" x86 supports the notion of a temporary mm which restricts access to temporary PTEs to a single CPU. A temporary mm is useful for situations where a CPU needs to perform sensitive operations (such as patching a STRICT_KERNEL_RWX kernel) requiring temporary mappings without exposing said mappings to other CPUs. Another benefit is that other CPU TLBs do not need to be flushed when the temporary mm is torn down. Mappings in the temporary mm can be set in the userspace portion of the address-space. Interrupts must be disabled while the temporary mm is in use. HW breakpoints, which may have been set by userspace as watchpoints on addresses now within the temporary mm, are saved and disabled when loading the temporary mm. The HW breakpoints are restored when unloading the temporary mm. All HW breakpoints are indiscriminately disabled while the temporary mm is in use - this may include breakpoints set by perf. Based on x86 implementation: commit cefa929c034e ("x86/mm: Introduce temporary mm structs") Signed-off-by: Christopher M. Riedl Signed-off-by: Jordan Niethe --- v7: - use breakpoint_pause()/breakpoint_unpause() - simplify the temp mm struct, don't need init_temp_mm() --- arch/powerpc/lib/code-patching.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c index c5ed98823835..29a30c3068ff 100644 --- a/arch/powerpc/lib/code-patching.c +++ b/arch/powerpc/lib/code-patching.c @@ -17,6 +17,9 @@ #include #include #include +#include +#include +#include static int __patch_instruction(u32 *exec_addr, struct ppc_inst instr, u32 *patch_addr) { @@ -45,6 +48,32 @@ int raw_patch_instruction(u32 *addr, struct ppc_inst instr) } #ifdef CONFIG_STRICT_KERNEL_RWX + +struct temp_mm_state { + struct mm_struct *mm; +}; + +static inline struct temp_mm_state start_using_temp_mm(struct mm_struct *mm) +{ + struct temp_mm_state temp_state; + + lockdep_assert_irqs_disabled(); + temp_state.mm = current->active_mm; + switch_mm_irqs_off(current->active_mm, mm, current); + + WARN_ON(!mm_is_thread_local(mm)); + + pause_breakpoints(); + return temp_state; +} + +static inline void stop_using_temp_mm(struct temp_mm_state prev_state) +{ + lockdep_assert_irqs_disabled(); + switch_mm_irqs_off(current->active_mm, prev_state.mm, current); + unpause_breakpoints(); +} + static DEFINE_PER_CPU(struct vm_struct *, text_poke_area); static int text_area_cpu_up(unsigned int cpu) From patchwork Wed Nov 10 00:37:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Niethe X-Patchwork-Id: 1553202 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=H0+ytpgR; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HpmFg06FPz9sRK for ; Wed, 10 Nov 2021 11:39:35 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HpmFf5jlcz3cXw for ; Wed, 10 Nov 2021 11:39:34 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=H0+ytpgR; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::1031; helo=mail-pj1-x1031.google.com; envelope-from=jniethe5@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=H0+ytpgR; dkim-atps=neutral Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HpmCH73DMz2xXB for ; Wed, 10 Nov 2021 11:37:31 +1100 (AEDT) Received: by mail-pj1-x1031.google.com with SMTP id o6-20020a17090a0a0600b001a64b9a11aeso112895pjo.3 for ; Tue, 09 Nov 2021 16:37:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0mYhAY0X+H1yQpitBOKt0IPpPGrvNZzkiJpGGsRAQsk=; b=H0+ytpgRkhdzgY34vEJPzhMYosn3W4l3BpoB9M5CkJ/7U0Lu2QU8LSta86Dhw59wdO Yj4MhNhOlaX1MHAcbZN3T+O20Jhj3RDWcdYY9+2k5+x4paBPNdSYqReWiuxJ/5ZDqL0L FdmenZXsuXyrc9VXikau07HpUwzV1RN2oKasdQPPcM9VPwS33K2n2sev3qK3cNbPAq3L Dmz82QUIs9vvJzjblMuBe8CaRkfNgHuiTpcaUGcIwzodhA1k8rc1vTNv503cRKpeDvgM nNUKKqp4WUN4sbFc+YXfGhjJ1908uv85xcbcKvONxJZYmApEAVQuyI66Wq2qK3htuXmu r/Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0mYhAY0X+H1yQpitBOKt0IPpPGrvNZzkiJpGGsRAQsk=; b=B6PYGq3OsVqWxBKBrJh+/1OiC3eryQ1DZsDfJuK8z4jnKs7bJVCAJey3Rpym/5hpfC UP+Pf8B1yT8l4fynnHuvsMqLrLIe3usLz60xtbtGu7m5OuULK/2Cum2bF6IoZPmp72Jk MwOTuXURUQ/PtDY5eFNrllAcXfXV0VgEUCaKmF3YTKyaI9mnkBJ415tCiToEnIZgn1cx ADrm7dOkmnrmFEJcXonKBiX7H+mrAH1i2rTPCgBT037dVT/gTK9MwfzGaIFi1SPHwvNo xr6mELfElIOqftSjgDG/pc59Z6pde3yGaNKLFnKf7EkeeOKtbmqWU614sXAUilAqN4+9 rtPw== X-Gm-Message-State: AOAM530BXVPVkTcbpL84cGkFAezWqe5hCDtXrAo2cns7VFPRJyv5YVD/ 07t3uS1S69BrHJlVHGifqxi9KBQRUDmd9A== X-Google-Smtp-Source: ABdhPJzy7rV89Q0I093JhJGjKlV9G/Zgjd34HLUED7ecKBnFM/Z2Pz6HNt7hSn7R7h7MRV+SZcuXiA== X-Received: by 2002:a17:90a:c257:: with SMTP id d23mr12330431pjx.42.1636504649426; Tue, 09 Nov 2021 16:37:29 -0800 (PST) Received: from tee480.ozlabs.ibm.com (159-196-117-139.9fc475.syd.nbn.aussiebb.net. [159.196.117.139]) by smtp.gmail.com with ESMTPSA id t21sm15854074pgo.12.2021.11.09.16.37.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Nov 2021 16:37:29 -0800 (PST) From: Jordan Niethe To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v7 3/5] powerpc: Rework and improve STRICT_KERNEL_RWX patching Date: Wed, 10 Nov 2021 11:37:15 +1100 Message-Id: <20211110003717.1150965-4-jniethe5@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211110003717.1150965-1-jniethe5@gmail.com> References: <20211110003717.1150965-1-jniethe5@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jordan Niethe , cmr@bluescreens.de Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: "Christopher M. Riedl" Rework code-patching with STRICT_KERNEL_RWX to prepare for a later patch which uses a temporary mm for patching under the Book3s64 Radix MMU. Make improvements by adding a WARN_ON when the patchsite doesn't match after patching and return the error from __patch_instruction() properly. Signed-off-by: Christopher M. Riedl Signed-off-by: Jordan Niethe --- v7: still pass addr to map_patch_area() --- arch/powerpc/lib/code-patching.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c index 29a30c3068ff..d586bf9c7581 100644 --- a/arch/powerpc/lib/code-patching.c +++ b/arch/powerpc/lib/code-patching.c @@ -75,6 +75,7 @@ static inline void stop_using_temp_mm(struct temp_mm_state prev_state) } static DEFINE_PER_CPU(struct vm_struct *, text_poke_area); +static DEFINE_PER_CPU(unsigned long, cpu_patching_addr); static int text_area_cpu_up(unsigned int cpu) { @@ -87,6 +88,7 @@ static int text_area_cpu_up(unsigned int cpu) return -1; } this_cpu_write(text_poke_area, area); + this_cpu_write(cpu_patching_addr, (unsigned long)area->addr); return 0; } @@ -172,11 +174,10 @@ static inline int unmap_patch_area(unsigned long addr) static int do_patch_instruction(u32 *addr, struct ppc_inst instr) { - int err; + int err, rc = 0; u32 *patch_addr = NULL; unsigned long flags; unsigned long text_poke_addr; - unsigned long kaddr = (unsigned long)addr; /* * During early early boot patch_instruction is called @@ -188,15 +189,13 @@ static int do_patch_instruction(u32 *addr, struct ppc_inst instr) local_irq_save(flags); - text_poke_addr = (unsigned long)__this_cpu_read(text_poke_area)->addr; - if (map_patch_area(addr, text_poke_addr)) { - err = -1; + text_poke_addr = __this_cpu_read(cpu_patching_addr); + err = map_patch_area(addr, text_poke_addr); + if (err) goto out; - } - - patch_addr = (u32 *)(text_poke_addr + (kaddr & ~PAGE_MASK)); - __patch_instruction(addr, instr, patch_addr); + patch_addr = (u32 *)(text_poke_addr | offset_in_page(addr)); + rc = __patch_instruction(addr, instr, patch_addr); err = unmap_patch_area(text_poke_addr); if (err) @@ -204,8 +203,9 @@ static int do_patch_instruction(u32 *addr, struct ppc_inst instr) out: local_irq_restore(flags); + WARN_ON(!ppc_inst_equal(ppc_inst_read(addr), instr)); - return err; + return rc ? rc : err; } #else /* !CONFIG_STRICT_KERNEL_RWX */ From patchwork Wed Nov 10 00:37:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Niethe X-Patchwork-Id: 1553203 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=W8KpR1kT; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HpmGW3jH0z9sRK for ; Wed, 10 Nov 2021 11:40:19 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HpmGW0mvlz3cRH for ; Wed, 10 Nov 2021 11:40:19 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=W8KpR1kT; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::1034; helo=mail-pj1-x1034.google.com; envelope-from=jniethe5@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=W8KpR1kT; dkim-atps=neutral Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HpmCK4H1jz2yb5 for ; Wed, 10 Nov 2021 11:37:33 +1100 (AEDT) Received: by mail-pj1-x1034.google.com with SMTP id gf14-20020a17090ac7ce00b001a7a2a0b5c3so107635pjb.5 for ; Tue, 09 Nov 2021 16:37:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ffiIi7pnxS+eWhKlfZ45oaRwJIGH3NoD9FTVCvyTzjM=; b=W8KpR1kTzufuQk5FPNtU/1E7DR8oTWeiL8aXKWW8M7nzRpD+XVcvT2YRPXSnRM0LYf KIooYf/LbS9hVh7lZpXinxuKjOmEEUr58Rqg8NNW1wfp6p/u/3ReNSdzflVl8pu4vXcc O+P0cRjW13fkw7TXsq2sDlwfvEvI36D1ANiWzO/w+eeseJZCXtxr1oPkxwFpDiT3iy8y U8UlFgU1Ag/LRJkbGd7YCzwZmghBZRX1zfc0hcpmo3tsgV15sifQm/W+NXRjjd4DDR3z q96TAcTszSm1Lv7R8xMo8p5vmmVe4rSyVS+hg/XKlTgnJJmVsJCa41UDmvLl9v7B/OZM ThqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ffiIi7pnxS+eWhKlfZ45oaRwJIGH3NoD9FTVCvyTzjM=; b=vSjJocvo5PWDaijWmi+z9HM9uL51Kw2hvyjUzhYS6CZurwaM/tbnx8EH31wA0hzpsR Bas8d5fkQq+6soDMCBg76+Q+B4RqXAsbOWkFIdluU38DzGhIoFWYoLiGSXGIuV1OHkHb yHPJq0XnVkkA2yE+0w/KIP2KupAQFg09TGfgED9X597lq5kMVBVVFJErzYwgyYnsxoeB OkUNXSrQ/pJ4ifth42fsQC0YN/SQObHgaBlwI++oCzLS2o6On7pqawewk5a8bHyM7TA6 evhCJyutN4shSMXuKWl3ljpFaoz08Hflaw5/scgYA8GluERtXtxGuDOPx/g+vfYNlGnR Yyiw== X-Gm-Message-State: AOAM532LiMgp17wc0HeNE7XsbKEGDA8QMYT8eAq1q0ccffeNvZ6df4Ev 3rK/EBwGUVndBdh3mRzKpt/VBm6LxfWc0w== X-Google-Smtp-Source: ABdhPJzzxgBliXI7ZblPlqkWBXYEEvtNggNiKbMBcxfVo9XniIqc5ZKilIYQNuAFd2xVX9cUMofc1Q== X-Received: by 2002:a17:902:b189:b0:143:8079:3d3b with SMTP id s9-20020a170902b18900b0014380793d3bmr1007956plr.71.1636504651684; Tue, 09 Nov 2021 16:37:31 -0800 (PST) Received: from tee480.ozlabs.ibm.com (159-196-117-139.9fc475.syd.nbn.aussiebb.net. [159.196.117.139]) by smtp.gmail.com with ESMTPSA id t21sm15854074pgo.12.2021.11.09.16.37.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Nov 2021 16:37:31 -0800 (PST) From: Jordan Niethe To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v7 4/5] powerpc: Use WARN_ON and fix check in poking_init Date: Wed, 10 Nov 2021 11:37:16 +1100 Message-Id: <20211110003717.1150965-5-jniethe5@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211110003717.1150965-1-jniethe5@gmail.com> References: <20211110003717.1150965-1-jniethe5@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jordan Niethe , cmr@bluescreens.de Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: "Christopher M. Riedl" The latest kernel docs list BUG_ON() as 'deprecated' and that they should be replaced with WARN_ON() (or pr_warn()) when possible. The BUG_ON() in poking_init() warrants a WARN_ON() rather than a pr_warn() since the error condition is deemed "unreachable". Also take this opportunity to fix the failure check in the WARN_ON(): cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, ...) returns a positive integer on success and a negative integer on failure. Signed-off-by: Christopher M. Riedl Signed-off-by: Jordan Niethe --- v7: no change --- arch/powerpc/lib/code-patching.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c index d586bf9c7581..aa466e4930ec 100644 --- a/arch/powerpc/lib/code-patching.c +++ b/arch/powerpc/lib/code-patching.c @@ -99,16 +99,11 @@ static int text_area_cpu_down(unsigned int cpu) return 0; } -/* - * Although BUG_ON() is rude, in this case it should only happen if ENOMEM, and - * we judge it as being preferable to a kernel that will crash later when - * someone tries to use patch_instruction(). - */ void __init poking_init(void) { - BUG_ON(!cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, + WARN_ON(cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "powerpc/text_poke:online", text_area_cpu_up, - text_area_cpu_down)); + text_area_cpu_down) < 0); } /* From patchwork Wed Nov 10 00:37:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Niethe X-Patchwork-Id: 1553205 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=SIG2w+Vx; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HpmHD6H6cz9sRK for ; Wed, 10 Nov 2021 11:40:56 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HpmHD56wJz3cV7 for ; Wed, 10 Nov 2021 11:40:56 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=SIG2w+Vx; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::102a; helo=mail-pj1-x102a.google.com; envelope-from=jniethe5@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=SIG2w+Vx; dkim-atps=neutral Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HpmCN1Y2Jz2yph for ; Wed, 10 Nov 2021 11:37:36 +1100 (AEDT) Received: by mail-pj1-x102a.google.com with SMTP id o6-20020a17090a0a0600b001a64b9a11aeso113029pjo.3 for ; Tue, 09 Nov 2021 16:37:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PJnjdi2x7mkLhV9OOe/in7fWckZtp/pUjD1KaPWSj3M=; b=SIG2w+VxvgXELvXWmR7WaKGBg0orec5sWesIsjCdJ2F2YP6atY1k5iIjOnoZvAW33I l51roYxQbgdlNiFYGxvBsH64w1CJaB94qcdqpG4I7chAiG/IRhNkSL0FnsV2hlLe9Wqe sDsir1P+57FRwjqm83tX10gtzDmdptz+fQS+S9sM8JUifEKgm5JU9BcHYuXP1azAaZsx Gg5kH99qoAIeZos0NKAJlC/Cd2Ki7KZIDaulO/1LWBMM7aLVRNGLKXI2O0lDBwY60qSM BkVWylUkKkhsOHT6XJY6EyacMqICTyFP8lBeMbtxpdCatQ81Vj1i5K+pNuHFzcsxz08M 2MQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PJnjdi2x7mkLhV9OOe/in7fWckZtp/pUjD1KaPWSj3M=; b=TI6IcdL8Zwyk9n1zG+D+Q0jPK5QVdJ5L32EalH509HviMoslbZOBBoaYLTq4vi3c53 71zxszBqik8cZQFxiIZpzJjKXdhx23880yVr6XUtzfbQCQdktxkIlCIDqjUDDFuP+t/L vtpB8byfR78mcaAmmKGjW4QFZBux0Tgkk5NqfPXhasTmXVt3kjjnyBoHZhs0lUPDrcCL bPlzEjpZPRuC7G8mJmB8xXuLI9nLQj/yQ8rwXdEJrmZj2FfXvtlkArGO7AP9pz4xSugi h1TuPlkkv384XI7zz2KopORwslPTnhMD/+5OlM9oIP9vsD9zd1QCU7uOD4iYzWkWKOqy nJBQ== X-Gm-Message-State: AOAM532UnNXgaHlkWKu2D9iIzc/2MCf4I9AIgffR7b5fXTYoxb5cHwXF dYI6o2xyfGONHXvC2N1WNPpqTvSGtlE+8w== X-Google-Smtp-Source: ABdhPJzTi97Uuj3wqB645/qf49/wzYbyPnRlDXVlTcz2cTk/M/KPP4TgDlRY2LcTlA205FAGzRsMHg== X-Received: by 2002:a17:902:8e85:b0:142:7621:e3b3 with SMTP id bg5-20020a1709028e8500b001427621e3b3mr11150044plb.84.1636504653686; Tue, 09 Nov 2021 16:37:33 -0800 (PST) Received: from tee480.ozlabs.ibm.com (159-196-117-139.9fc475.syd.nbn.aussiebb.net. [159.196.117.139]) by smtp.gmail.com with ESMTPSA id t21sm15854074pgo.12.2021.11.09.16.37.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Nov 2021 16:37:33 -0800 (PST) From: Jordan Niethe To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v7 5/5] powerpc/64s: Initialize and use a temporary mm for patching on Radix Date: Wed, 10 Nov 2021 11:37:17 +1100 Message-Id: <20211110003717.1150965-6-jniethe5@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211110003717.1150965-1-jniethe5@gmail.com> References: <20211110003717.1150965-1-jniethe5@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jordan Niethe , cmr@bluescreens.de Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: "Christopher M. Riedl" When code patching a STRICT_KERNEL_RWX kernel the page containing the address to be patched is temporarily mapped as writeable. Currently, a per-cpu vmalloc patch area is used for this purpose. While the patch area is per-cpu, the temporary page mapping is inserted into the kernel page tables for the duration of patching. The mapping is exposed to CPUs other than the patching CPU - this is undesirable from a hardening perspective. Use a temporary mm instead which keeps the mapping local to the CPU doing the patching. Use the `poking_init` init hook to prepare a temporary mm and patching address. Initialize the temporary mm by copying the init mm. Choose a randomized patching address inside the temporary mm userspace address space. The patching address is randomized between PAGE_SIZE and DEFAULT_MAP_WINDOW-PAGE_SIZE. Bits of entropy with 64K page size on BOOK3S_64: bits of entropy = log2(DEFAULT_MAP_WINDOW_USER64 / PAGE_SIZE) PAGE_SIZE=64K, DEFAULT_MAP_WINDOW_USER64=128TB bits of entropy = log2(128TB / 64K) bits of entropy = 31 The upper limit is DEFAULT_MAP_WINDOW due to how the Book3s64 Hash MMU operates - by default the space above DEFAULT_MAP_WINDOW is not available. Currently the Hash MMU does not use a temporary mm so technically this upper limit isn't necessary; however, a larger randomization range does not further "harden" this overall approach and future work may introduce patching with a temporary mm on Hash as well. Randomization occurs only once during initialization at boot for each possible CPU in the system. Introduce a new function, patch_instruction_mm(), to perform the patching with a temporary mapping with write permissions at patching_addr. Map the page with PAGE_KERNEL to set EAA[0] for the PTE which ignores the AMR (so no need to unlock/lock KUAP) according to PowerISA v3.0b Figure 35 on Radix. Based on x86 implementation: commit 4fc19708b165 ("x86/alternatives: Initialize temporary mm for patching") and: commit b3fd8e83ada0 ("x86/alternatives: Use temporary mm for text poking") Signed-off-by: Christopher M. Riedl Signed-off-by: Jordan Niethe --- v7: - Change to patch_instruction_mm() instead of map_patch_mm() and unmap_patch_mm() - include ptesync --- arch/powerpc/lib/code-patching.c | 106 +++++++++++++++++++++++++++++-- 1 file changed, 101 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c index aa466e4930ec..7722dec4a914 100644 --- a/arch/powerpc/lib/code-patching.c +++ b/arch/powerpc/lib/code-patching.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include @@ -76,6 +77,7 @@ static inline void stop_using_temp_mm(struct temp_mm_state prev_state) static DEFINE_PER_CPU(struct vm_struct *, text_poke_area); static DEFINE_PER_CPU(unsigned long, cpu_patching_addr); +static DEFINE_PER_CPU(struct mm_struct *, cpu_patching_mm); static int text_area_cpu_up(unsigned int cpu) { @@ -99,8 +101,48 @@ static int text_area_cpu_down(unsigned int cpu) return 0; } +static __always_inline void __poking_init_temp_mm(void) +{ + int cpu; + spinlock_t *ptl; + pte_t *ptep; + struct mm_struct *patching_mm; + unsigned long patching_addr; + + for_each_possible_cpu(cpu) { + patching_mm = copy_init_mm(); + WARN_ON(!patching_mm); + per_cpu(cpu_patching_mm, cpu) = patching_mm; + + /* + * Choose a randomized, page-aligned address from the range: + * [PAGE_SIZE, DEFAULT_MAP_WINDOW - PAGE_SIZE] The lower + * address bound is PAGE_SIZE to avoid the zero-page. The + * upper address bound is DEFAULT_MAP_WINDOW - PAGE_SIZE to + * stay under DEFAULT_MAP_WINDOW with the Book3s64 Hash MMU. + */ + patching_addr = PAGE_SIZE + ((get_random_long() & PAGE_MASK) % + (DEFAULT_MAP_WINDOW - 2 * PAGE_SIZE)); + per_cpu(cpu_patching_addr, cpu) = patching_addr; + + /* + * PTE allocation uses GFP_KERNEL which means we need to + * pre-allocate the PTE here because we cannot do the + * allocation during patching when IRQs are disabled. + */ + ptep = get_locked_pte(patching_mm, patching_addr, &ptl); + WARN_ON(!ptep); + pte_unmap_unlock(ptep, ptl); + } +} + void __init poking_init(void) { + if (radix_enabled()) { + __poking_init_temp_mm(); + return; + } + WARN_ON(cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "powerpc/text_poke:online", text_area_cpu_up, text_area_cpu_down) < 0); @@ -167,6 +209,57 @@ static inline int unmap_patch_area(unsigned long addr) return 0; } +/* + * This can be called for kernel text or a module. + */ +static int patch_instruction_mm(u32 *addr, struct ppc_inst instr) +{ + struct mm_struct *patching_mm = __this_cpu_read(cpu_patching_mm); + unsigned long text_poke_addr; + u32 *patch_addr = NULL; + struct temp_mm_state prev; + unsigned long flags; + struct page *page; + spinlock_t *ptl; + int rc; + pte_t *ptep; + + text_poke_addr = __this_cpu_read(cpu_patching_addr); + + local_irq_save(flags); + + if (is_vmalloc_or_module_addr(addr)) + page = vmalloc_to_page(addr); + else + page = virt_to_page(addr); + + ptep = get_locked_pte(patching_mm, text_poke_addr, &ptl); + if (unlikely(!ptep)) { + pr_warn("map patch: failed to allocate pte for patching\n"); + return -1; + } + + set_pte_at(patching_mm, text_poke_addr, ptep, pte_mkdirty(mk_pte(page, PAGE_KERNEL))); + asm volatile("ptesync": : :"memory"); + + prev = start_using_temp_mm(patching_mm); + + patch_addr = (u32 *)(text_poke_addr | offset_in_page(addr)); + rc = __patch_instruction(addr, instr, patch_addr); + + pte_clear(patching_mm, text_poke_addr, ptep); + + local_flush_tlb_mm(patching_mm); + + stop_using_temp_mm(prev); + pte_unmap_unlock(ptep, ptl); + + local_irq_restore(flags); + WARN_ON(!ppc_inst_equal(ppc_inst_read(addr), instr)); + + return rc; +} + static int do_patch_instruction(u32 *addr, struct ppc_inst instr) { int err, rc = 0; @@ -175,16 +268,19 @@ static int do_patch_instruction(u32 *addr, struct ppc_inst instr) unsigned long text_poke_addr; /* - * During early early boot patch_instruction is called - * when text_poke_area is not ready, but we still need - * to allow patching. We just do the plain old patching + * During early boot patch_instruction is called when the + * patching_mm/text_poke_area is not ready, but we still need to allow + * patching. We just do the plain old patching. */ - if (!this_cpu_read(text_poke_area)) + text_poke_addr = __this_cpu_read(cpu_patching_addr); + if (!text_poke_addr) return raw_patch_instruction(addr, instr); + if (radix_enabled()) + return patch_instruction_mm(addr, instr); + local_irq_save(flags); - text_poke_addr = __this_cpu_read(cpu_patching_addr); err = map_patch_area(addr, text_poke_addr); if (err) goto out;