From patchwork Fri Feb 24 17:01:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 1747583 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=rGTCUYdS; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sifive.com header.i=@sifive.com header.a=rsa-sha256 header.s=google header.b=mCv4sAOX; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PNbmP50rmz1yYg for ; Sat, 25 Feb 2023 04:02:29 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=f149XtmFSpRKmmKG9CiB8FC61fRqG9x5CBHMnUNB2eI=; b=rGTCUYdSkFaW6f sbx3pSecnbPGnSQTPwXByDr+cJdYVGewaksJhUPk/3sNEXcsJ7kRurVOJKkTiDd6CzSQF0LoeHsI7 cehKxDoin25xUYHT7CP2G2+y25QMdOoEkN+WU9PPG6W1W8avHr2pMyiyeDmQhjLpt76lsC+Vtwa8D hsHYXrMfKtw2S3vrJWsofFEY45yJOtXr4A51xHuaJAr0krJDFQkG2ci3P+75enD2K2z+TFA/rEZWK Es3pfYRgU4zsRzuHaeQdTzk2eA09z8bCfQD1adXB1lz2m1Ai3+2Nlqfe6doPqXyYW4AVjgWeV5H40 QDNwnmLZLbtzFX/TywMw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pVbSj-003Dej-DW; Fri, 24 Feb 2023 17:02:25 +0000 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pVbSg-003DbG-89 for kvm-riscv@lists.infradead.org; Fri, 24 Feb 2023 17:02:24 +0000 Received: by mail-pj1-x102a.google.com with SMTP id z20-20020a17090a8b9400b002372d7f823eso3462404pjn.4 for ; Fri, 24 Feb 2023 09:02:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=oeU1EzgQZPviUx8x1M7BAweIqWoiI9GPVL7R1I1QzEM=; b=mCv4sAOXa6oisDiWg8GBB0kENtNWNon7JU+fper8v5+3Jca7jjEYPtzqjo+GDcKTRe dKQWOL/12E3JIHbfMwmwsjVqTzcMkbdGG0tU2mJCvYQhmq2qZtul4aVJpiv9vTjB2CEG s2tX/NYrLjyVC7gK5UbMStg2KZWL7aa4Th2HZJgW6NBSord4riefsGbIYHn2TtO11swN gpFObbkXYjqCIwmOjipi67WxH6YeSk5gFZRxq4VLihFHoq1RXGxXpVyTI0MwPSBGb/Wc j16z5G28M6/5z1avj7DaB5K928ZA9BM+KnG83aMj2BQffJAhGslMpHuHFemuhEJgOv/N 5HZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oeU1EzgQZPviUx8x1M7BAweIqWoiI9GPVL7R1I1QzEM=; b=JZcBEC/SFYXCw/8mgdTcrUD9boBPFb04Qetn/D3G2/jmuDIlxpm7H5G+63FLhU2jpJ /DfdpWkbUe0Zd1OQBo58NK8FRecp4D8/kl5Vj/38gayOvPbc94JmcQUYGmP3FHbZtcOX FJRLhftwq7cAwN1VmmAQgdIKlqN+pdg8bTCRPEFT/MHrbAq4pTIfpXj/8X7BW34hNpe6 4TjAehRQgj+dIIu1o9oStySu1/UYS3hMjt/a03diMt+MhgTR+GPTOQQzUryhRZElnBbr 4kZ9pCxw8YV4O8eJWyCWYpkRvCklBv+h1+FU5vs4HJMJtuCzgNCTsa8Ps3ErJH2uVqxm tMTw== X-Gm-Message-State: AO0yUKWsVb3X+fzfhqNoaqiAIOvxrj2WxC4agOY1I5F4/cdKoQldnQO6 i8VauuUAl+++h6J0/KXpIjfCoQ== X-Google-Smtp-Source: AK7set82aDGPKd/oN7NmcHtGanjLX/28ckwBJ2M7mKySEAYrKIk1lgwMUdj//Im6eaBTyENN/Kiv2w== X-Received: by 2002:a17:902:c950:b0:19a:b44b:cca6 with SMTP id i16-20020a170902c95000b0019ab44bcca6mr19680763pla.24.1677258141591; Fri, 24 Feb 2023 09:02:21 -0800 (PST) Received: from hsinchu25.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id b12-20020a170902b60c00b0019472226769sm9234731pls.251.2023.02.24.09.02.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Feb 2023 09:02:20 -0800 (PST) From: Andy Chiu To: linux-riscv@lists.infradead.org, palmer@dabbelt.com, anup@brainfault.org, atishp@atishpatra.org, kvm-riscv@lists.infradead.org, kvm@vger.kernel.org Cc: vineetg@rivosinc.com, greentime.hu@sifive.com, guoren@linux.alibaba.com, Vincent Chen , Andy Chiu , Paul Walmsley , Albert Ou , Guo Ren , Richard Henderson Subject: [PATCH -next v14 08/19] riscv: Introduce struct/helpers to save/restore per-task Vector state Date: Fri, 24 Feb 2023 17:01:07 +0000 Message-Id: <20230224170118.16766-9-andy.chiu@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230224170118.16766-1-andy.chiu@sifive.com> References: <20230224170118.16766-1-andy.chiu@sifive.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230224_090222_322345_0C9506C6 X-CRM114-Status: GOOD ( 11.43 ) X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "bombadil.infradead.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: From: Greentime Hu Add vector state context struct to be added later in thread_struct. And prepare low-level helper functions to save/restore vector contexts. This include Vector Regfile and CSRs holding dynamic configuration state (vstart, vl, vtype, vcsr). The Vec Register width could be implementation defined, but same for all processes, so that is saved [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:102a listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-BeenThere: kvm-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "kvm-riscv" Errors-To: kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org From: Greentime Hu Add vector state context struct to be added later in thread_struct. And prepare low-level helper functions to save/restore vector contexts. This include Vector Regfile and CSRs holding dynamic configuration state (vstart, vl, vtype, vcsr). The Vec Register width could be implementation defined, but same for all processes, so that is saved separately. This is not yet wired into final thread_struct - will be done when __switch_to actually starts doing this in later patches. Given the variable (and potentially large) size of regfile, they are saved in dynamically allocated memory, pointed to by datap pointer in __riscv_v_ext_state. Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu Signed-off-by: Vineet Gupta [vineetg: merged bits from 2 different patches] Signed-off-by: Andy Chiu [andy.chiu: use inline asm to save/restore context, remove asm vaiant] Acked-by: Conor Dooley --- arch/riscv/include/asm/vector.h | 84 ++++++++++++++++++++++++++++ arch/riscv/include/uapi/asm/ptrace.h | 17 ++++++ 2 files changed, 101 insertions(+) diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index 692d3ee2d2d3..9c025f2efdc3 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -12,6 +12,9 @@ #include #include +#include + +#define CSR_STR(x) __ASM_STR(x) extern unsigned long riscv_v_vsize; void riscv_v_setup_vsize(void); @@ -21,6 +24,26 @@ static __always_inline bool has_vector(void) return riscv_has_extension_likely(RISCV_ISA_EXT_v); } +static inline void __riscv_v_vstate_clean(struct pt_regs *regs) +{ + regs->status = (regs->status & ~(SR_VS)) | SR_VS_CLEAN; +} + +static inline void riscv_v_vstate_off(struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_OFF; +} + +static inline void riscv_v_vstate_on(struct pt_regs *regs) +{ + regs->status = (regs->status & ~(SR_VS)) | SR_VS_INITIAL; +} + +static inline bool riscv_v_vstate_query(struct pt_regs *regs) +{ + return (regs->status & SR_VS) != 0; +} + static __always_inline void riscv_v_enable(void) { csr_set(CSR_SSTATUS, SR_VS); @@ -31,11 +54,72 @@ static __always_inline void riscv_v_disable(void) csr_clear(CSR_SSTATUS, SR_VS); } +static __always_inline void __vstate_csr_save(struct __riscv_v_ext_state *dest) +{ + asm volatile ( + "csrr %0, " CSR_STR(CSR_VSTART) "\n\t" + "csrr %1, " CSR_STR(CSR_VTYPE) "\n\t" + "csrr %2, " CSR_STR(CSR_VL) "\n\t" + "csrr %3, " CSR_STR(CSR_VCSR) "\n\t" + : "=r" (dest->vstart), "=r" (dest->vtype), "=r" (dest->vl), + "=r" (dest->vcsr) : :); +} + +static __always_inline void __vstate_csr_restore(struct __riscv_v_ext_state *src) +{ + asm volatile ( + "vsetvl x0, %2, %1\n\t" + "csrw " CSR_STR(CSR_VSTART) ", %0\n\t" + "csrw " CSR_STR(CSR_VCSR) ", %3\n\t" + : : "r" (src->vstart), "r" (src->vtype), "r" (src->vl), + "r" (src->vcsr) :); +} + +static inline void __riscv_v_vstate_save(struct __riscv_v_ext_state *save_to, void *datap) +{ + riscv_v_enable(); + __vstate_csr_save(save_to); + asm volatile ( + "vsetvli t4, x0, e8, m8, ta, ma\n\t" + "vse8.v v0, (%0)\n\t" + "add %0, %0, t4\n\t" + "vse8.v v8, (%0)\n\t" + "add %0, %0, t4\n\t" + "vse8.v v16, (%0)\n\t" + "add %0, %0, t4\n\t" + "vse8.v v24, (%0)\n\t" + : : "r" (datap) : "t4", "memory"); + riscv_v_disable(); +} + +static inline void __riscv_v_vstate_restore(struct __riscv_v_ext_state *restore_from, + void *datap) +{ + riscv_v_enable(); + asm volatile ( + "vsetvli t4, x0, e8, m8, ta, ma\n\t" + "vle8.v v0, (%0)\n\t" + "add %0, %0, t4\n\t" + "vle8.v v8, (%0)\n\t" + "add %0, %0, t4\n\t" + "vle8.v v16, (%0)\n\t" + "add %0, %0, t4\n\t" + "vle8.v v24, (%0)\n\t" + : : "r" (datap) : "t4"); + __vstate_csr_restore(restore_from); + riscv_v_disable(); +} + #else /* ! CONFIG_RISCV_ISA_V */ +struct pt_regs; + static __always_inline bool has_vector(void) { return false; } +static inline bool riscv_v_vstate_query(struct pt_regs *regs) { return false; } #define riscv_v_vsize (0) #define riscv_v_setup_vsize() do {} while (0) +#define riscv_v_vstate_off(regs) do {} while (0) +#define riscv_v_vstate_on(regs) do {} while (0) #endif /* CONFIG_RISCV_ISA_V */ diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h index 882547f6bd5c..586786d023c4 100644 --- a/arch/riscv/include/uapi/asm/ptrace.h +++ b/arch/riscv/include/uapi/asm/ptrace.h @@ -77,6 +77,23 @@ union __riscv_fp_state { struct __riscv_q_ext_state q; }; +struct __riscv_v_ext_state { + unsigned long vstart; + unsigned long vl; + unsigned long vtype; + unsigned long vcsr; + void *datap; + /* + * In signal handler, datap will be set a correct user stack offset + * and vector registers will be copied to the address of datap + * pointer. + * + * In ptrace syscall, datap will be set to zero and the vector + * registers will be copied to the address right after this + * structure. + */ +}; + #endif /* __ASSEMBLY__ */ #endif /* _UAPI_ASM_RISCV_PTRACE_H */