{"id":831224,"url":"http://patchwork.ozlabs.org/api/1.2/patches/831224/?format=json","web_url":"http://patchwork.ozlabs.org/project/glibc/patch/1509101470-7881-19-git-send-email-Dave.Martin@arm.com/","project":{"id":41,"url":"http://patchwork.ozlabs.org/api/1.2/projects/41/?format=json","name":"GNU C Library","link_name":"glibc","list_id":"libc-alpha.sourceware.org","list_email":"libc-alpha@sourceware.org","web_url":"","scm_url":"","webscm_url":"","list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<1509101470-7881-19-git-send-email-Dave.Martin@arm.com>","list_archive_url":null,"date":"2017-10-27T10:51:00","name":"[v4,18/28] arm64/sve: Preserve SVE registers around EFI runtime service calls","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"7961938a815db0857112c0dfd1840fab33630076","submitter":{"id":26612,"url":"http://patchwork.ozlabs.org/api/1.2/people/26612/?format=json","name":"Dave Martin","email":"Dave.Martin@arm.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/glibc/patch/1509101470-7881-19-git-send-email-Dave.Martin@arm.com/mbox/","series":[{"id":10555,"url":"http://patchwork.ozlabs.org/api/1.2/series/10555/?format=json","web_url":"http://patchwork.ozlabs.org/project/glibc/list/?series=10555","date":"2017-10-27T10:50:42","name":"ARM Scalable Vector Extension (SVE)","version":4,"mbox":"http://patchwork.ozlabs.org/series/10555/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/831224/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/831224/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":["patchwork-incoming@bilbo.ozlabs.org","mailing list libc-alpha@sourceware.org"],"Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=sourceware.org\n\t(client-ip=209.132.180.131; helo=sourceware.org;\n\tenvelope-from=libc-alpha-return-86460-incoming=patchwork.ozlabs.org@sourceware.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tsecure) header.d=sourceware.org header.i=@sourceware.org\n\theader.b=\"ve1Wy0PP\"; dkim-atps=neutral","sourceware.org; auth=none"],"Received":["from sourceware.org (server1.sourceware.org [209.132.180.131])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yNgkY1DGyz9rxj\n\tfor ;\n\tFri, 27 Oct 2017 21:55:00 +1100 (AEDT)","(qmail 119636 invoked by alias); 27 Oct 2017 10:51:58 -0000","(qmail 119607 invoked by uid 89); 27 Oct 2017 10:51:57 -0000"],"DomainKey-Signature":"a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id\n\t:list-unsubscribe:list-subscribe:list-archive:list-post\n\t:list-help:sender:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-type:content-transfer-encoding;\n\tq=dns; s=default; b=cqncbRXjmzL+SIuSDJ48Ckpawl5prCfy0Ox2Nvx7Z0X\n\tJZBQ5yoq3quYIdjZV3J7+6sS98f1GJJEd03u06A4pluHtPhTb4gw2JL2mp6zzP1f\n\taFFFsXBJ+9eHb26AWilIj/crjpEZF6EEYlSwUyuNPUvD5QAr82kDg5qcKzPPK4Ms\n\t=","DKIM-Signature":"v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id\n\t:list-unsubscribe:list-subscribe:list-archive:list-post\n\t:list-help:sender:from:to:cc:subject:date:message-id:in-reply-to\n\t:references:mime-version:content-type:content-transfer-encoding;\n\ts=default; bh=xz+uZAqT03HrB/Ct2A8muldehMI=; b=ve1Wy0PPxHT6Zn6Cf\n\tS4bju//vbCSSGPAKVhpAnO2p/enu07/7HQNlssJupvBlYlOoaA8yBzxA+bHuqj3B\n\tEMSMsQ6TFWcq1hweQ5xd89oLUuLBhgloYol6lfILRn2s7YuNVlIzdKjkHEz+J+vo\n\tv5vEvQRozn/qVRDhjSu3icgYCc=","Mailing-List":"contact libc-alpha-help@sourceware.org; run by ezmlm","Precedence":"bulk","List-Id":"","List-Unsubscribe":"","List-Subscribe":"","List-Archive":"","List-Post":"","List-Help":",\n\t","Sender":"libc-alpha-owner@sourceware.org","X-Virus-Found":"No","X-Spam-SWARE-Status":"No, score=-26.9 required=5.0 tests=BAYES_00, GIT_PATCH_0,\n\tGIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RP_MATCHES_RCVD,\n\tSPF_PASS autolearn=ham version=3.3.2 spammy=contemporary,\n\tpanic","X-HELO":"foss.arm.com","From":"Dave Martin ","To":"linux-arm-kernel@lists.infradead.org","Cc":"Catalin Marinas , Will Deacon\n\t, \tArd Biesheuvel ,\n\t=?utf-8?q?Alex_Benn=C3=A9?= =?utf-8?q?e?= ,\n\tSzabolcs Nagy , Okamoto Takayuki\n\t, \tkvmarm@lists.cs.columbia.edu,\n\tlibc-alpha@sourceware.org, \tlinux-arch@vger.kernel.org","Subject":"[PATCH v4 18/28] arm64/sve: Preserve SVE registers around EFI\n\truntime service calls","Date":"Fri, 27 Oct 2017 11:51:00 +0100","Message-Id":"<1509101470-7881-19-git-send-email-Dave.Martin@arm.com>","In-Reply-To":"<1509101470-7881-1-git-send-email-Dave.Martin@arm.com>","References":"<1509101470-7881-1-git-send-email-Dave.Martin@arm.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"8bit"},"content":"The EFI runtime services ABI allows EFI to make free use of the\nFPSIMD registers during EFI runtime service calls, subject to the\ncallee-save requirements of the AArch64 procedure call standard.\n\nHowever, the SVE architecture allows upper bits of the SVE vector\nregisters to be zeroed as a side-effect of FPSIMD V-register\nwrites. This means that the SVE vector registers must be saved in\ntheir entirety in order to avoid data loss: non-SVE-aware EFI\nimplementations cannot restore them correctly.\n\nThe non-IRQ case is already handled gracefully by\nkernel_neon_begin(). For the IRQ case, this patch allocates a\nsuitable per-CPU stash buffer for the full SVE register state and\nuses it to preserve the affected registers around EFI calls. It is\ncurrently unclear how the EFI runtime services ABI will be\nclarified with respect to SVE, so it safest to assume that the\npredicate registers and FFR must be saved and restored too.\n\nNo attempt is made to restore the restore the vector length after\na call, for now. It is deemed rather insane for EFI to change it,\nand contemporary EFI implementations certainly won't.\n\nSigned-off-by: Dave Martin \nReviewed-by: Alex Bennée \nReviewed-by: Catalin Marinas \nCc: Ard Biesheuvel \n---\n arch/arm64/kernel/fpsimd.c | 67 +++++++++++++++++++++++++++++++++++++++++-----\n 1 file changed, 61 insertions(+), 6 deletions(-)","diff":"diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c\nindex d5108e8..edc27d2 100644\n--- a/arch/arm64/kernel/fpsimd.c\n+++ b/arch/arm64/kernel/fpsimd.c\n@@ -121,11 +121,13 @@ static int sve_default_vl = -1;\n int __ro_after_init sve_max_vl = -1;\n /* Set of available vector lengths, as vq_to_bit(vq): */\n static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);\n+static void __percpu *efi_sve_state;\n \n #else /* ! CONFIG_ARM64_SVE */\n \n /* Dummy declaration for code that will be optimised out: */\n extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);\n+extern void __percpu *efi_sve_state;\n \n #endif /* ! CONFIG_ARM64_SVE */\n \n@@ -550,6 +552,30 @@ int sve_verify_vq_map(void)\n \treturn ret;\n }\n \n+static void __init sve_efi_setup(void)\n+{\n+\tif (!IS_ENABLED(CONFIG_EFI))\n+\t\treturn;\n+\n+\t/*\n+\t * alloc_percpu() warns and prints a backtrace if this goes wrong.\n+\t * This is evidence of a crippled system and we are returning void,\n+\t * so no attempt is made to handle this situation here.\n+\t */\n+\tif (!sve_vl_valid(sve_max_vl))\n+\t\tgoto fail;\n+\n+\tefi_sve_state = __alloc_percpu(\n+\t\tSVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);\n+\tif (!efi_sve_state)\n+\t\tgoto fail;\n+\n+\treturn;\n+\n+fail:\n+\tpanic(\"Cannot allocate percpu memory for EFI SVE save/restore\");\n+}\n+\n /*\n * Enable SVE for EL1.\n * Intended for use by the cpufeatures code during CPU boot.\n@@ -597,6 +623,8 @@ void __init sve_setup(void)\n \t\tsve_max_vl);\n \tpr_info(\"SVE: default vector length %u bytes per vector\\n\",\n \t\tsve_default_vl);\n+\n+\tsve_efi_setup();\n }\n \n /*\n@@ -925,6 +953,7 @@ EXPORT_SYMBOL(kernel_neon_end);\n \n static DEFINE_PER_CPU(struct fpsimd_state, efi_fpsimd_state);\n static DEFINE_PER_CPU(bool, efi_fpsimd_state_used);\n+static DEFINE_PER_CPU(bool, efi_sve_state_used);\n \n /*\n * EFI runtime services support functions\n@@ -950,10 +979,24 @@ void __efi_fpsimd_begin(void)\n \n \tWARN_ON(preemptible());\n \n-\tif (may_use_simd())\n+\tif (may_use_simd()) {\n \t\tkernel_neon_begin();\n-\telse {\n-\t\tfpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));\n+\t} else {\n+\t\t/*\n+\t\t * If !efi_sve_state, SVE can't be in use yet and doesn't need\n+\t\t * preserving:\n+\t\t */\n+\t\tif (system_supports_sve() && likely(efi_sve_state)) {\n+\t\t\tchar *sve_state = this_cpu_ptr(efi_sve_state);\n+\n+\t\t\t__this_cpu_write(efi_sve_state_used, true);\n+\n+\t\t\tsve_save_state(sve_state + sve_ffr_offset(sve_max_vl),\n+\t\t\t\t &this_cpu_ptr(&efi_fpsimd_state)->fpsr);\n+\t\t} else {\n+\t\t\tfpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));\n+\t\t}\n+\n \t\t__this_cpu_write(efi_fpsimd_state_used, true);\n \t}\n }\n@@ -966,10 +1009,22 @@ void __efi_fpsimd_end(void)\n \tif (!system_supports_fpsimd())\n \t\treturn;\n \n-\tif (__this_cpu_xchg(efi_fpsimd_state_used, false))\n-\t\tfpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));\n-\telse\n+\tif (!__this_cpu_xchg(efi_fpsimd_state_used, false)) {\n \t\tkernel_neon_end();\n+\t} else {\n+\t\tif (system_supports_sve() &&\n+\t\t likely(__this_cpu_read(efi_sve_state_used))) {\n+\t\t\tchar const *sve_state = this_cpu_ptr(efi_sve_state);\n+\n+\t\t\tsve_load_state(sve_state + sve_ffr_offset(sve_max_vl),\n+\t\t\t\t &this_cpu_ptr(&efi_fpsimd_state)->fpsr,\n+\t\t\t\t sve_vq_from_vl(sve_get_vl()) - 1);\n+\n+\t\t\t__this_cpu_write(efi_sve_state_used, false);\n+\t\t} else {\n+\t\t\tfpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));\n+\t\t}\n+\t}\n }\n \n #endif /* CONFIG_EFI */\n","prefixes":["v4","18/28"]}