From patchwork Wed Jan 25 14:20:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 1731702 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=tMPq+/0u; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sifive.com header.i=@sifive.com header.a=rsa-sha256 header.s=google header.b=EwaZnhUa; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4P25dM11XSz23gY for ; Thu, 26 Jan 2023 01:22:15 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=QKyaz/98DuDgBwvsautOsE2uKCx7IbaL9/ztZyZ3Puw=; b=tMPq+/0ut5hvyz WN+4EZnTq/cSXZL74gtMr3GsDvZ5ox3Fhr47Psof8hLlFcaaxkohw7Dm0RVfbN0mPN1bekqJG8Hvg MWj2ni2HDda5alvTf7mAWCxcHbaWH4GDzdxXDMfxb5r93wVVMmyh4adRPE/iWmgAEvNEpOh1k7ALG TXGccDYeIpk/BLEm3ISX0djlB6a2Sbq2CWrpgCQU/QqEk93n9KeP9cTXdy3ejIsoXdkShrWBtb18T pd+IGoolsvpJnp7JQwrt8kQI0BM4eT6m8w1iSF8avz6+jLzcH+SdYn8Tznm4ZbW4m6Q08M4ZN9okT GKkj7gk2+eunLg//zKHw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pKgfC-007Vhc-RK; Wed, 25 Jan 2023 14:22:10 +0000 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pKgf9-007Vc8-TX for kvm-riscv@lists.infradead.org; Wed, 25 Jan 2023 14:22:09 +0000 Received: by mail-pl1-x632.google.com with SMTP id be8so5117649plb.7 for ; Wed, 25 Jan 2023 06:22:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=P7edCeAwjfiwkkKg0W4tIhLAZjBnv1bdBFUA+mBx+50=; b=EwaZnhUa+ekhcI4IQm40kDpmqQ91VW57Kg2SGqKJUBq+BXQ4XFzQwgtpna5MYFWjXk m8KS2maRHs4iKz+NOF7lhgcsSHaDNWzuP0c01A8zD2VH0S8PpozRaru+EVokxZBVdEuQ 0ZrBwcJ2ixaPI9EjgjVvSQjNhoJOovxiBN5VHOxemhhasziBW3F5+S/EfrjlbSrCvFji 4WUHa7OhyFYvSRnmZr9PRtqYR6OjPqL+H883VvRa6tsHv7aImWELVe67liCJHJ6/riyj T7dC+WGWz70aTYBOOnVio113pdh8ADdZo2Epci9YgnYgfT86m8lr0iQqoTZHTddY/Yaj cq1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=P7edCeAwjfiwkkKg0W4tIhLAZjBnv1bdBFUA+mBx+50=; b=bXJqmzp+AWw5TQYRHxhjO/2XKpCON69/3VPN74HhaNh6yIhC0OV6dpu3hHHbivCLLf RWuvudKEUJR4h/TkV5kbZyS65qIoWq/umR2LFuN1lGPxq1LaIAVi7NoT6gA1mvvQWF8s N6Dak2fal2V9uE9LfeL/Yrpuq+R2IK7tEfPK1znxhV0HkU06qdOraBleK60qSreGCTNn PalD9JvmTY9RaqeFsTLVGqCMLJnY865OLxGCwik24KDQogoirNPD7fuSxY0PDR6MLSKN qYgfHOup0Ken4PulrkDRalOnfRlcOMlSX96UsunG34hwyTIfYo0snL55SDU2aG5DLeeO wnRw== X-Gm-Message-State: AFqh2kpPA1uuy8xMlXnnsoHjkQh7Wik9+kTGMudopLCOJIaalpfFzyb2 BoICZl8nT0FaJAV2QqXmwZwLmQ== X-Google-Smtp-Source: AMrXdXtG1sIOVUuMs9I/wo6VgU4VFoE992ZnxdCDUMdsDubVbCL/9Smu69AiCEMOx3G13m1aOuX3/w== X-Received: by 2002:a17:90b:4b92:b0:229:f4e1:d4b1 with SMTP id lr18-20020a17090b4b9200b00229f4e1d4b1mr23173072pjb.22.1674656521831; Wed, 25 Jan 2023 06:22:01 -0800 (PST) Received: from hsinchu25.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id bu11-20020a63294b000000b004a3510effa5sm3203520pgb.65.2023.01.25.06.21.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Jan 2023 06:22:01 -0800 (PST) From: Andy Chiu To: linux-riscv@lists.infradead.org, palmer@dabbelt.com, anup@brainfault.org, atishp@atishpatra.org, kvm-riscv@lists.infradead.org, kvm@vger.kernel.org Cc: vineetg@rivosinc.com, greentime.hu@sifive.com, guoren@linux.alibaba.com, Andy Chiu , Paul Walmsley , Albert Ou , Heiko Stuebner , Andrew Jones , Lad Prabhakar , Conor Dooley , Jisheng Zhang , Vincent Chen , Guo Ren , Li Zhengyu , Masahiro Yamada , Changbin Du , Richard Henderson Subject: [PATCH -next v13 10/19] riscv: Allocate user's vector context in the first-use trap Date: Wed, 25 Jan 2023 14:20:47 +0000 Message-Id: <20230125142056.18356-11-andy.chiu@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230125142056.18356-1-andy.chiu@sifive.com> References: <20230125142056.18356-1-andy.chiu@sifive.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230125_062207_979697_72873660 X-CRM114-Status: GOOD ( 21.15 ) X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "bombadil.infradead.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Vector unit is disabled by default for all user processes. Thus, a process will take a trap (illegal instruction) into kernel at the first time when it uses Vector. Only after then, the kernel allocat [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:632 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-BeenThere: kvm-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "kvm-riscv" Errors-To: kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org Vector unit is disabled by default for all user processes. Thus, a process will take a trap (illegal instruction) into kernel at the first time when it uses Vector. Only after then, the kernel allocates V context and starts take care of the context for that user process. Suggested-by: Richard Henderson Link: https://lore.kernel.org/r/3923eeee-e4dc-0911-40bf-84c34aee962d@linaro.org Signed-off-by: Andy Chiu --- arch/riscv/include/asm/insn.h | 24 +++++++++ arch/riscv/include/asm/vector.h | 2 + arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/vector.c | 89 +++++++++++++++++++++++++++++++++ 4 files changed, 116 insertions(+) create mode 100644 arch/riscv/kernel/vector.c diff --git a/arch/riscv/include/asm/insn.h b/arch/riscv/include/asm/insn.h index 25ef9c0b19e7..b1ef3617881f 100644 --- a/arch/riscv/include/asm/insn.h +++ b/arch/riscv/include/asm/insn.h @@ -133,6 +133,24 @@ #define RVG_OPCODE_JALR 0x67 #define RVG_OPCODE_JAL 0x6f #define RVG_OPCODE_SYSTEM 0x73 +#define RVG_SYSTEM_CSR_OFF 20 +#define RVG_SYSTEM_CSR_MASK GENMASK(12, 0) + +/* parts of opcode for RVV */ +#define OPCODE_VECTOR 0x57 +#define LSFP_WIDTH_RVV_8 0 +#define LSFP_WIDTH_RVV_16 5 +#define LSFP_WIDTH_RVV_32 6 +#define LSFP_WIDTH_RVV_64 7 + +/* parts of opcode for RVF, RVD and RVQ */ +#define LSFP_WIDTH_OFF 12 +#define LSFP_WIDTH_MASK GENMASK(3, 0) +#define LSFP_WIDTH_FP_W 2 +#define LSFP_WIDTH_FP_D 3 +#define LSFP_WIDTH_FP_Q 4 +#define OPCODE_LOADFP 0x07 +#define OPCODE_STOREFP 0x27 /* parts of opcode for RVC*/ #define RVC_OPCODE_C0 0x0 @@ -291,6 +309,12 @@ static __always_inline bool riscv_insn_is_branch(u32 code) (RVC_X(x_, RVC_B_IMM_7_6_OPOFF, RVC_B_IMM_7_6_MASK) << RVC_B_IMM_7_6_OFF) | \ (RVC_IMM_SIGN(x_) << RVC_B_IMM_SIGN_OFF); }) +#define EXTRACT_LOAD_STORE_FP_WIDTH(x) \ + ({typeof(x) x_ = (x); RV_X(x_, LSFP_WIDTH_OFF, LSFP_WIDTH_MASK); }) + +#define EXTRACT_SYSTEM_CSR(x) \ + ({typeof(x) x_ = (x); RV_X(x_, RVG_SYSTEM_CSR_OFF, RVG_SYSTEM_CSR_MASK); }) + /* * Get the immediate from a J-type instruction. * diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index f8a9e37c4374..7c77696d704a 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -19,6 +19,7 @@ #define CSR_STR(x) __ASM_STR(x) extern unsigned long riscv_vsize; +bool rvv_first_use_handler(struct pt_regs *regs); static __always_inline bool has_vector(void) { @@ -138,6 +139,7 @@ static inline void vstate_restore(struct task_struct *task, struct pt_regs; static __always_inline bool has_vector(void) { return false; } +static inline bool rvv_first_use_handler(struct pt_regs *regs) { return false; } static inline bool vstate_query(struct pt_regs *regs) { return false; } #define riscv_vsize (0) #define vstate_save(task, regs) do {} while (0) diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 4cf303a779ab..48d345a5f326 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -55,6 +55,7 @@ obj-$(CONFIG_MMU) += vdso.o vdso/ obj-$(CONFIG_RISCV_M_MODE) += traps_misaligned.o obj-$(CONFIG_FPU) += fpu.o +obj-$(CONFIG_RISCV_ISA_V) += vector.o obj-$(CONFIG_SMP) += smpboot.o obj-$(CONFIG_SMP) += smp.o obj-$(CONFIG_SMP) += cpu_ops.o diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c new file mode 100644 index 000000000000..cdd58d1c8b3c --- /dev/null +++ b/arch/riscv/kernel/vector.c @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2023 SiFive + * Author: Andy Chiu + */ +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +static bool insn_is_vector(u32 insn_buf) +{ + u32 opcode = insn_buf & __INSN_OPCODE_MASK; + /* + * All V-related instructions, including CSR operations are 4-Byte. So, + * do not handle if the instruction length is not 4-Byte. + */ + if (unlikely(GET_INSN_LENGTH(insn_buf) != 4)) + return false; + if (opcode == OPCODE_VECTOR) { + return true; + } else if (opcode == OPCODE_LOADFP || opcode == OPCODE_STOREFP) { + u32 width = EXTRACT_LOAD_STORE_FP_WIDTH(insn_buf); + + if (width == LSFP_WIDTH_RVV_8 || width == LSFP_WIDTH_RVV_16 || + width == LSFP_WIDTH_RVV_32 || width == LSFP_WIDTH_RVV_64) + return true; + } else if (opcode == RVG_OPCODE_SYSTEM) { + u32 csr = EXTRACT_SYSTEM_CSR(insn_buf); + + if ((csr >= CSR_VSTART && csr <= CSR_VCSR) || + (csr >= CSR_VL && csr <= CSR_VLENB)) + return true; + } + return false; +} + +int rvv_thread_zalloc(void) +{ + void *datap; + + datap = kzalloc(riscv_vsize, GFP_KERNEL); + if (!datap) + return -ENOMEM; + current->thread.vstate.datap = datap; + memset(¤t->thread.vstate, 0, offsetof(struct __riscv_v_state, + datap)); + return 0; +} + +bool rvv_first_use_handler(struct pt_regs *regs) +{ + __user u32 *epc = (u32 *)regs->epc; + u32 tval = (u32)regs->badaddr; + + /* If V has been enabled then it is not the first-use trap */ + if (vstate_query(regs)) + return false; + /* Get the instruction */ + if (!tval) { + if (__get_user(tval, epc)) + return false; + } + /* Filter out non-V instructions */ + if (!insn_is_vector(tval)) + return false; + /* Sanity check. datap should be null by the time of the first-use trap */ + WARN_ON(current->thread.vstate.datap); + /* + * Now we sure that this is a V instruction. And it executes in the + * context where VS has been off. So, try to allocate the user's V + * context and resume execution. + */ + if (rvv_thread_zalloc()) { + force_sig(SIGKILL); + return true; + } + vstate_on(regs); + return true; +} +