{"id":2183226,"url":"http://patchwork.ozlabs.org/api/patches/2183226/?format=json","web_url":"http://patchwork.ozlabs.org/project/linuxppc-dev/patch/20260112192035.10427-15-ebiggers@kernel.org/","project":{"id":2,"url":"http://patchwork.ozlabs.org/api/projects/2/?format=json","name":"Linux PPC development","link_name":"linuxppc-dev","list_id":"linuxppc-dev.lists.ozlabs.org","list_email":"linuxppc-dev@lists.ozlabs.org","web_url":"https://github.com/linuxppc/wiki/wiki","scm_url":"https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git","webscm_url":"https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/","list_archive_url":"https://lore.kernel.org/linuxppc-dev/","list_archive_url_format":"https://lore.kernel.org/linuxppc-dev/{}/","commit_url_format":"https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id={}"},"msgid":"<20260112192035.10427-15-ebiggers@kernel.org>","list_archive_url":"https://lore.kernel.org/linuxppc-dev/20260112192035.10427-15-ebiggers@kernel.org/","date":"2026-01-12T19:20:12","name":"[v2,14/35] lib/crypto: riscv/aes: Migrate optimized code into library","commit_ref":null,"pull_url":null,"state":"handled-elsewhere","archived":false,"hash":"c88402fe337ef61766db839749765540ecc3ccda","submitter":{"id":74690,"url":"http://patchwork.ozlabs.org/api/people/74690/?format=json","name":"Eric Biggers","email":"ebiggers@kernel.org"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/linuxppc-dev/patch/20260112192035.10427-15-ebiggers@kernel.org/mbox/","series":[{"id":488089,"url":"http://patchwork.ozlabs.org/api/series/488089/?format=json","web_url":"http://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=488089","date":"2026-01-12T19:19:58","name":"AES library improvements","version":2,"mbox":"http://patchwork.ozlabs.org/series/488089/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/2183226/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/2183226/checks/","tags":{},"related":[],"headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256\n header.s=k20201202 header.b=pqT16Xv5;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org\n (client-ip=112.213.38.117; helo=lists.ozlabs.org;\n envelope-from=linuxppc-dev+bounces-15583-incoming=patchwork.ozlabs.org@lists.ozlabs.org;\n receiver=patchwork.ozlabs.org)","lists.ozlabs.org;\n arc=none smtp.remote-ip=172.234.252.31","lists.ozlabs.org;\n dmarc=pass (p=quarantine dis=none) header.from=kernel.org","lists.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256\n header.s=k20201202 header.b=pqT16Xv5;\n\tdkim-atps=neutral","lists.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org\n (client-ip=172.234.252.31; helo=sea.source.kernel.org;\n envelope-from=ebiggers@kernel.org; receiver=lists.ozlabs.org)"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4dqj7Y13XDz1xpY\n\tfor ; Tue, 13 Jan 2026 06:27:21 +1100 (AEDT)","from boromir.ozlabs.org (localhost [127.0.0.1])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 4dqj3g02kfz3btn;\n\tTue, 13 Jan 2026 06:23:59 +1100 (AEDT)","from sea.source.kernel.org (sea.source.kernel.org [172.234.252.31])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 4dqj3d3dF5z3c3H\n\tfor ; Tue, 13 Jan 2026 06:23:57 +1100 (AEDT)","from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58])\n\tby sea.source.kernel.org (Postfix) with ESMTP id CD15040630;\n\tMon, 12 Jan 2026 19:23:25 +0000 (UTC)","by smtp.kernel.org (Postfix) with ESMTPSA id 4252DC19422;\n\tMon, 12 Jan 2026 19:23:25 +0000 (UTC)"],"ARC-Seal":"i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1768245838;\n\tcv=none;\n b=A44xJ1iNk5NvgMi6ujpY2nFZdjuS7olkHcnQevqBJElPPaYiLNReevOL011XI1oJSyFue+t8wxQd3ivC9kiBASJjhNc7AZRkEute6NWqRetpw/tjK+Y9Xg+5c2RX54jj8Fat5P7Ql6Z8GbT8eE1kdhulENdX/cnEzdEUg0oQzu5mpkTMe1Zbccs13BtOQaAnB55FT7twdlwufV7zE2eSBU3P90vC9GSzSzDh61nawZ4Bksn5mZs8DG4avBAqGZnLhKN+oK4n+C2/0oDmUC4issf1utLMc46yf+nScFRSLjayADtfiio0CMJ9ABzF06QGI50sSCymcqa1cIB+fh3fKQ==","ARC-Message-Signature":"i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707;\n\tt=1768245838; c=relaxed/relaxed;\n\tbh=SEoENnWUcb8l0Cmsrhzc+Gdxwbk1lpPM3wnRuQBpEso=;\n\th=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References:\n\t MIME-Version:Content-Type;\n b=g3OoyDaggmvkvanVMazb2LzWW6zVF/jY6KA+BSBSF1nxexqBCnX5gBtk4XzMm4b9hP29xa2JHMpHmHIfYN3UguF09IzvAAGbCONG8yHqNcCS6+7xwq+XdbEtxDtMKQXsYCR6Nbh9oIb8+9rnsMiA3hrqO7DLGfexR3pGGXyfgvPlZhaFCoNU64V+cpEfFmHeKm7vC60ttVw6SoPJ+/Gtz8NplZcmwi7mm9JtfCuvui8/v5n3iPtmFaGNLTmGXUE3wSchuDG4RjrOlkqTA5s8tWPEJzPnWWIHhSiBwg8G0VeHOEQgXbeDMQR3e5jE44esZbHvXf5bSJpOg5YDEb6/nQ==","ARC-Authentication-Results":"i=1; lists.ozlabs.org;\n dmarc=pass (p=quarantine dis=none) header.from=kernel.org;\n dkim=pass (2048-bit key;\n unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256\n header.s=k20201202 header.b=pqT16Xv5; dkim-atps=neutral;\n spf=pass (client-ip=172.234.252.31; helo=sea.source.kernel.org;\n envelope-from=ebiggers@kernel.org;\n receiver=lists.ozlabs.org) smtp.mailfrom=kernel.org","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;\n\ts=k20201202; t=1768245805;\n\tbh=cAzlxjnfetAXICKQEDRHR7jpw8oC/uEaOKgrdLNR7pI=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=pqT16Xv5y39AtbmZYAgOkHjMWCOa/h4oZZmyTmq0R4g9MqhPim0TUNaaIMF0+v9ze\n\t GsnV3fuIps6KEtb87qL3FbYr4EoQRM/M3VAguchs2auk6wcNXGZufbX69abPZcPkE8\n\t 63X9xxnUfPinp28JVmVSIvjQwFrH2cjGKSUmGsbqcqb6CB9JzwtJWlifgMRYUPfn9x\n\t 8fadfn9CewGB62+Zhn7oxXxU+VERktjR06ce8npPtrH9id1Uk2Ln4g3wqaDda2OXVU\n\t TaY3zHzfmGReU8xSJEBgbYZxPy9gnv3tR6lhLF7/x0I/eXHUD6QwSY9Ux/LB6ZBgIK\n\t sYxHbcLk1xb5Q==","From":"Eric Biggers ","To":"linux-crypto@vger.kernel.org","Cc":"linux-kernel@vger.kernel.org,\n\tArd Biesheuvel ,\n\t\"Jason A . Donenfeld\" ,\n\tHerbert Xu ,\n\tlinux-arm-kernel@lists.infradead.org,\n\tlinuxppc-dev@lists.ozlabs.org,\n\tlinux-riscv@lists.infradead.org,\n\tlinux-s390@vger.kernel.org,\n\tsparclinux@vger.kernel.org,\n\tx86@kernel.org,\n\tHolger Dengler ,\n\tHarald Freudenberger ,\n\tEric Biggers ","Subject":"[PATCH v2 14/35] lib/crypto: riscv/aes: Migrate optimized code into\n library","Date":"Mon, 12 Jan 2026 11:20:12 -0800","Message-ID":"<20260112192035.10427-15-ebiggers@kernel.org>","X-Mailer":"git-send-email 2.52.0","In-Reply-To":"<20260112192035.10427-1-ebiggers@kernel.org>","References":"<20260112192035.10427-1-ebiggers@kernel.org>","X-Mailing-List":"linuxppc-dev@lists.ozlabs.org","List-Id":"","List-Help":"","List-Owner":"","List-Post":"","List-Archive":",\n ","List-Subscribe":",\n ,\n ","List-Unsubscribe":"","Precedence":"list","MIME-Version":"1.0","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"8bit","X-Spam-Status":"No, score=-0.2 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,\n\tDKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS\n\tautolearn=disabled version=4.0.1 OzLabs 8","X-Spam-Checker-Version":"SpamAssassin 4.0.1 (2024-03-25) on lists.ozlabs.org"},"content":"Move the aes_encrypt_zvkned() and aes_decrypt_zvkned() assembly\nfunctions into lib/crypto/, wire them up to the AES library API, and\nremove the \"aes-riscv64-zvkned\" crypto_cipher algorithm.\n\nTo make this possible, change the prototypes of these functions to\ntake (rndkeys, key_len) instead of a pointer to crypto_aes_ctx, and\nchange the RISC-V AES-XTS code to implement tweak encryption using the\nAES library instead of directly calling aes_encrypt_zvkned().\n\nThe result is that both the AES library and crypto_cipher APIs use\nRISC-V's AES instructions, whereas previously only crypto_cipher did\n(and it wasn't enabled by default, which this commit fixes as well).\n\nAcked-by: Ard Biesheuvel \nSigned-off-by: Eric Biggers \n---\n arch/riscv/crypto/Kconfig | 2 -\n arch/riscv/crypto/aes-macros.S | 12 +++-\n arch/riscv/crypto/aes-riscv64-glue.c | 81 ++-----------------------\n arch/riscv/crypto/aes-riscv64-zvkned.S | 27 ---------\n lib/crypto/Kconfig | 2 +\n lib/crypto/Makefile | 1 +\n lib/crypto/riscv/aes-riscv64-zvkned.S | 84 ++++++++++++++++++++++++++\n lib/crypto/riscv/aes.h | 63 +++++++++++++++++++\n 8 files changed, 166 insertions(+), 106 deletions(-)\n create mode 100644 lib/crypto/riscv/aes-riscv64-zvkned.S\n create mode 100644 lib/crypto/riscv/aes.h","diff":"diff --git a/arch/riscv/crypto/Kconfig b/arch/riscv/crypto/Kconfig\nindex 14c5acb935e9..22d4eaab15f3 100644\n--- a/arch/riscv/crypto/Kconfig\n+++ b/arch/riscv/crypto/Kconfig\n@@ -4,15 +4,13 @@ menu \"Accelerated Cryptographic Algorithms for CPU (riscv)\"\n \n config CRYPTO_AES_RISCV64\n \ttristate \"Ciphers: AES, modes: ECB, CBC, CTS, CTR, XTS\"\n \tdepends on 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \\\n \t\t RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS\n-\tselect CRYPTO_ALGAPI\n \tselect CRYPTO_LIB_AES\n \tselect CRYPTO_SKCIPHER\n \thelp\n-\t Block cipher: AES cipher algorithms\n \t Length-preserving ciphers: AES with ECB, CBC, CTS, CTR, XTS\n \n \t Architecture: riscv64 using:\n \t - Zvkned vector crypto extension\n \t - Zvbb vector extension (XTS)\ndiff --git a/arch/riscv/crypto/aes-macros.S b/arch/riscv/crypto/aes-macros.S\nindex d1a258d04bc7..1384164621a5 100644\n--- a/arch/riscv/crypto/aes-macros.S\n+++ b/arch/riscv/crypto/aes-macros.S\n@@ -49,12 +49,14 @@\n // - If AES-128, loads round keys into v1-v11 and jumps to \\label128.\n // - If AES-192, loads round keys into v1-v13 and jumps to \\label192.\n // - If AES-256, loads round keys into v1-v15 and continues onwards.\n //\n // Also sets vl=4 and vtype=e32,m1,ta,ma. Clobbers t0 and t1.\n-.macro\taes_begin\tkeyp, label128, label192\n+.macro\taes_begin\tkeyp, label128, label192, key_len\n+.ifb \\key_len\n \tlwu\t\tt0, 480(\\keyp)\t// t0 = key length in bytes\n+.endif\n \tli\t\tt1, 24\t\t// t1 = key length for AES-192\n \tvsetivli\tzero, 4, e32, m1, ta, ma\n \tvle32.v\t\tv1, (\\keyp)\n \taddi\t\t\\keyp, \\keyp, 16\n \tvle32.v\t\tv2, (\\keyp)\n@@ -74,16 +76,24 @@\n \tvle32.v\t\tv9, (\\keyp)\n \taddi\t\t\\keyp, \\keyp, 16\n \tvle32.v\t\tv10, (\\keyp)\n \taddi\t\t\\keyp, \\keyp, 16\n \tvle32.v\t\tv11, (\\keyp)\n+.ifb \\key_len\n \tblt\t\tt0, t1, \\label128\t// If AES-128, goto label128.\n+.else\n+\tblt\t\t\\key_len, t1, \\label128\t// If AES-128, goto label128.\n+.endif\n \taddi\t\t\\keyp, \\keyp, 16\n \tvle32.v\t\tv12, (\\keyp)\n \taddi\t\t\\keyp, \\keyp, 16\n \tvle32.v\t\tv13, (\\keyp)\n+.ifb \\key_len\n \tbeq\t\tt0, t1, \\label192\t// If AES-192, goto label192.\n+.else\n+\tbeq\t\t\\key_len, t1, \\label192\t// If AES-192, goto label192.\n+.endif\n \t// Else, it's AES-256.\n \taddi\t\t\\keyp, \\keyp, 16\n \tvle32.v\t\tv14, (\\keyp)\n \taddi\t\t\\keyp, \\keyp, 16\n \tvle32.v\t\tv15, (\\keyp)\ndiff --git a/arch/riscv/crypto/aes-riscv64-glue.c b/arch/riscv/crypto/aes-riscv64-glue.c\nindex f814ee048555..8bbf7f348c23 100644\n--- a/arch/riscv/crypto/aes-riscv64-glue.c\n+++ b/arch/riscv/crypto/aes-riscv64-glue.c\n@@ -1,9 +1,8 @@\n // SPDX-License-Identifier: GPL-2.0-only\n /*\n- * AES using the RISC-V vector crypto extensions. Includes the bare block\n- * cipher and the ECB, CBC, CBC-CTS, CTR, and XTS modes.\n+ * AES modes using the RISC-V vector crypto extensions\n *\n * Copyright (C) 2023 VRULL GmbH\n * Author: Heiko Stuebner \n *\n * Copyright (C) 2023 SiFive, Inc.\n@@ -13,25 +12,17 @@\n */\n \n #include \n #include \n #include \n-#include \n #include \n #include \n #include \n #include \n #include \n #include \n \n-asmlinkage void aes_encrypt_zvkned(const struct crypto_aes_ctx *key,\n-\t\t\t\t const u8 in[AES_BLOCK_SIZE],\n-\t\t\t\t u8 out[AES_BLOCK_SIZE]);\n-asmlinkage void aes_decrypt_zvkned(const struct crypto_aes_ctx *key,\n-\t\t\t\t const u8 in[AES_BLOCK_SIZE],\n-\t\t\t\t u8 out[AES_BLOCK_SIZE]);\n-\n asmlinkage void aes_ecb_encrypt_zvkned(const struct crypto_aes_ctx *key,\n \t\t\t\t const u8 *in, u8 *out, size_t len);\n asmlinkage void aes_ecb_decrypt_zvkned(const struct crypto_aes_ctx *key,\n \t\t\t\t const u8 *in, u8 *out, size_t len);\n \n@@ -84,54 +75,18 @@ static int riscv64_aes_setkey(struct crypto_aes_ctx *ctx,\n \t * struct crypto_aes_ctx and aes_expandkey() everywhere.\n \t */\n \treturn aes_expandkey(ctx, key, keylen);\n }\n \n-static int riscv64_aes_setkey_cipher(struct crypto_tfm *tfm,\n-\t\t\t\t const u8 *key, unsigned int keylen)\n-{\n-\tstruct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm);\n-\n-\treturn riscv64_aes_setkey(ctx, key, keylen);\n-}\n-\n static int riscv64_aes_setkey_skcipher(struct crypto_skcipher *tfm,\n \t\t\t\t const u8 *key, unsigned int keylen)\n {\n \tstruct crypto_aes_ctx *ctx = crypto_skcipher_ctx(tfm);\n \n \treturn riscv64_aes_setkey(ctx, key, keylen);\n }\n \n-/* Bare AES, without a mode of operation */\n-\n-static void riscv64_aes_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src)\n-{\n-\tconst struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm);\n-\n-\tif (crypto_simd_usable()) {\n-\t\tkernel_vector_begin();\n-\t\taes_encrypt_zvkned(ctx, src, dst);\n-\t\tkernel_vector_end();\n-\t} else {\n-\t\taes_encrypt(ctx, dst, src);\n-\t}\n-}\n-\n-static void riscv64_aes_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src)\n-{\n-\tconst struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm);\n-\n-\tif (crypto_simd_usable()) {\n-\t\tkernel_vector_begin();\n-\t\taes_decrypt_zvkned(ctx, src, dst);\n-\t\tkernel_vector_end();\n-\t} else {\n-\t\taes_decrypt(ctx, dst, src);\n-\t}\n-}\n-\n /* AES-ECB */\n \n static inline int riscv64_aes_ecb_crypt(struct skcipher_request *req, bool enc)\n {\n \tstruct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);\n@@ -336,21 +291,21 @@ static int riscv64_aes_ctr_crypt(struct skcipher_request *req)\n \n /* AES-XTS */\n \n struct riscv64_aes_xts_ctx {\n \tstruct crypto_aes_ctx ctx1;\n-\tstruct crypto_aes_ctx ctx2;\n+\tstruct aes_enckey tweak_key;\n };\n \n static int riscv64_aes_xts_setkey(struct crypto_skcipher *tfm, const u8 *key,\n \t\t\t\t unsigned int keylen)\n {\n \tstruct riscv64_aes_xts_ctx *ctx = crypto_skcipher_ctx(tfm);\n \n \treturn xts_verify_key(tfm, key, keylen) ?:\n \t riscv64_aes_setkey(&ctx->ctx1, key, keylen / 2) ?:\n-\t riscv64_aes_setkey(&ctx->ctx2, key + keylen / 2, keylen / 2);\n+\t aes_prepareenckey(&ctx->tweak_key, key + keylen / 2, keylen / 2);\n }\n \n static int riscv64_aes_xts_crypt(struct skcipher_request *req, bool enc)\n {\n \tstruct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);\n@@ -364,13 +319,11 @@ static int riscv64_aes_xts_crypt(struct skcipher_request *req, bool enc)\n \n \tif (req->cryptlen < AES_BLOCK_SIZE)\n \t\treturn -EINVAL;\n \n \t/* Encrypt the IV with the tweak key to get the first tweak. */\n-\tkernel_vector_begin();\n-\taes_encrypt_zvkned(&ctx->ctx2, req->iv, req->iv);\n-\tkernel_vector_end();\n+\taes_encrypt(&ctx->tweak_key, req->iv, req->iv);\n \n \terr = skcipher_walk_virt(&walk, req, false);\n \n \t/*\n \t * If the message length isn't divisible by the AES block size and the\n@@ -454,27 +407,10 @@ static int riscv64_aes_xts_decrypt(struct skcipher_request *req)\n \treturn riscv64_aes_xts_crypt(req, false);\n }\n \n /* Algorithm definitions */\n \n-static struct crypto_alg riscv64_zvkned_aes_cipher_alg = {\n-\t.cra_flags = CRYPTO_ALG_TYPE_CIPHER,\n-\t.cra_blocksize = AES_BLOCK_SIZE,\n-\t.cra_ctxsize = sizeof(struct crypto_aes_ctx),\n-\t.cra_priority = 300,\n-\t.cra_name = \"aes\",\n-\t.cra_driver_name = \"aes-riscv64-zvkned\",\n-\t.cra_cipher = {\n-\t\t.cia_min_keysize = AES_MIN_KEY_SIZE,\n-\t\t.cia_max_keysize = AES_MAX_KEY_SIZE,\n-\t\t.cia_setkey = riscv64_aes_setkey_cipher,\n-\t\t.cia_encrypt = riscv64_aes_encrypt,\n-\t\t.cia_decrypt = riscv64_aes_decrypt,\n-\t},\n-\t.cra_module = THIS_MODULE,\n-};\n-\n static struct skcipher_alg riscv64_zvkned_aes_skcipher_algs[] = {\n \t{\n \t\t.setkey = riscv64_aes_setkey_skcipher,\n \t\t.encrypt = riscv64_aes_ecb_encrypt,\n \t\t.decrypt = riscv64_aes_ecb_decrypt,\n@@ -572,19 +508,15 @@ static int __init riscv64_aes_mod_init(void)\n {\n \tint err = -ENODEV;\n \n \tif (riscv_isa_extension_available(NULL, ZVKNED) &&\n \t riscv_vector_vlen() >= 128) {\n-\t\terr = crypto_register_alg(&riscv64_zvkned_aes_cipher_alg);\n-\t\tif (err)\n-\t\t\treturn err;\n-\n \t\terr = crypto_register_skciphers(\n \t\t\triscv64_zvkned_aes_skcipher_algs,\n \t\t\tARRAY_SIZE(riscv64_zvkned_aes_skcipher_algs));\n \t\tif (err)\n-\t\t\tgoto unregister_zvkned_cipher_alg;\n+\t\t\treturn err;\n \n \t\tif (riscv_isa_extension_available(NULL, ZVKB)) {\n \t\t\terr = crypto_register_skcipher(\n \t\t\t\t&riscv64_zvkned_zvkb_aes_skcipher_alg);\n \t\t\tif (err)\n@@ -605,12 +537,10 @@ static int __init riscv64_aes_mod_init(void)\n \tif (riscv_isa_extension_available(NULL, ZVKB))\n \t\tcrypto_unregister_skcipher(&riscv64_zvkned_zvkb_aes_skcipher_alg);\n unregister_zvkned_skcipher_algs:\n \tcrypto_unregister_skciphers(riscv64_zvkned_aes_skcipher_algs,\n \t\t\t\t ARRAY_SIZE(riscv64_zvkned_aes_skcipher_algs));\n-unregister_zvkned_cipher_alg:\n-\tcrypto_unregister_alg(&riscv64_zvkned_aes_cipher_alg);\n \treturn err;\n }\n \n static void __exit riscv64_aes_mod_exit(void)\n {\n@@ -618,11 +548,10 @@ static void __exit riscv64_aes_mod_exit(void)\n \t\tcrypto_unregister_skcipher(&riscv64_zvkned_zvbb_zvkg_aes_skcipher_alg);\n \tif (riscv_isa_extension_available(NULL, ZVKB))\n \t\tcrypto_unregister_skcipher(&riscv64_zvkned_zvkb_aes_skcipher_alg);\n \tcrypto_unregister_skciphers(riscv64_zvkned_aes_skcipher_algs,\n \t\t\t\t ARRAY_SIZE(riscv64_zvkned_aes_skcipher_algs));\n-\tcrypto_unregister_alg(&riscv64_zvkned_aes_cipher_alg);\n }\n \n module_init(riscv64_aes_mod_init);\n module_exit(riscv64_aes_mod_exit);\n \ndiff --git a/arch/riscv/crypto/aes-riscv64-zvkned.S b/arch/riscv/crypto/aes-riscv64-zvkned.S\nindex 23d063f94ce6..d0fc4581a380 100644\n--- a/arch/riscv/crypto/aes-riscv64-zvkned.S\n+++ b/arch/riscv/crypto/aes-riscv64-zvkned.S\n@@ -54,37 +54,10 @@\n #define INP\t\ta1\n #define OUTP\t\ta2\n #define LEN\t\ta3\n #define IVP\t\ta4\n \n-.macro\t__aes_crypt_zvkned\tenc, keylen\n-\tvle32.v\t\tv16, (INP)\n-\taes_crypt\tv16, \\enc, \\keylen\n-\tvse32.v\t\tv16, (OUTP)\n-\tret\n-.endm\n-\n-.macro\taes_crypt_zvkned\tenc\n-\taes_begin\tKEYP, 128f, 192f\n-\t__aes_crypt_zvkned\t\\enc, 256\n-128:\n-\t__aes_crypt_zvkned\t\\enc, 128\n-192:\n-\t__aes_crypt_zvkned\t\\enc, 192\n-.endm\n-\n-// void aes_encrypt_zvkned(const struct crypto_aes_ctx *key,\n-//\t\t\t const u8 in[16], u8 out[16]);\n-SYM_FUNC_START(aes_encrypt_zvkned)\n-\taes_crypt_zvkned\t1\n-SYM_FUNC_END(aes_encrypt_zvkned)\n-\n-// Same prototype and calling convention as the encryption function\n-SYM_FUNC_START(aes_decrypt_zvkned)\n-\taes_crypt_zvkned\t0\n-SYM_FUNC_END(aes_decrypt_zvkned)\n-\n .macro\t__aes_ecb_crypt\tenc, keylen\n \tsrli\t\tt0, LEN, 2\n \t// t0 is the remaining length in 32-bit words. It's a multiple of 4.\n 1:\n \tvsetvli\t\tt1, t0, e32, m8, ta, ma\ndiff --git a/lib/crypto/Kconfig b/lib/crypto/Kconfig\nindex a0f1c105827e..2690b5ffc5ca 100644\n--- a/lib/crypto/Kconfig\n+++ b/lib/crypto/Kconfig\n@@ -15,10 +15,12 @@ config CRYPTO_LIB_AES_ARCH\n \tbool\n \tdepends on CRYPTO_LIB_AES && !UML && !KMSAN\n \tdefault y if ARM\n \tdefault y if ARM64\n \tdefault y if PPC && (SPE || (PPC64 && VSX))\n+\tdefault y if RISCV && 64BIT && TOOLCHAIN_HAS_VECTOR_CRYPTO && \\\n+\t\t RISCV_EFFICIENT_VECTOR_UNALIGNED_ACCESS\n \n config CRYPTO_LIB_AESCFB\n \ttristate\n \tselect CRYPTO_LIB_AES\n \tselect CRYPTO_LIB_UTILS\ndiff --git a/lib/crypto/Makefile b/lib/crypto/Makefile\nindex 16140616ace8..811b60787dd5 100644\n--- a/lib/crypto/Makefile\n+++ b/lib/crypto/Makefile\n@@ -48,10 +48,11 @@ $(obj)/powerpc/aesp8-ppc.S: $(src)/powerpc/aesp8-ppc.pl FORCE\n targets += powerpc/aesp8-ppc.S\n OBJECT_FILES_NON_STANDARD_powerpc/aesp8-ppc.o := y\n endif # !CONFIG_SPE\n endif # CONFIG_PPC\n \n+libaes-$(CONFIG_RISCV) += riscv/aes-riscv64-zvkned.o\n endif # CONFIG_CRYPTO_LIB_AES_ARCH\n \n ################################################################################\n \n obj-$(CONFIG_CRYPTO_LIB_AESCFB)\t\t\t+= libaescfb.o\ndiff --git a/lib/crypto/riscv/aes-riscv64-zvkned.S b/lib/crypto/riscv/aes-riscv64-zvkned.S\nnew file mode 100644\nindex 000000000000..0d988bc3d37b\n--- /dev/null\n+++ b/lib/crypto/riscv/aes-riscv64-zvkned.S\n@@ -0,0 +1,84 @@\n+/* SPDX-License-Identifier: Apache-2.0 OR BSD-2-Clause */\n+//\n+// This file is dual-licensed, meaning that you can use it under your\n+// choice of either of the following two licenses:\n+//\n+// Copyright 2023 The OpenSSL Project Authors. All Rights Reserved.\n+//\n+// Licensed under the Apache License 2.0 (the \"License\"). You can obtain\n+// a copy in the file LICENSE in the source distribution or at\n+// https://www.openssl.org/source/license.html\n+//\n+// or\n+//\n+// Copyright (c) 2023, Christoph Müllner \n+// Copyright (c) 2023, Phoebe Chen \n+// Copyright (c) 2023, Jerry Shih \n+// Copyright 2024 Google LLC\n+// All rights reserved.\n+//\n+// Redistribution and use in source and binary forms, with or without\n+// modification, are permitted provided that the following conditions\n+// are met:\n+// 1. Redistributions of source code must retain the above copyright\n+// notice, this list of conditions and the following disclaimer.\n+// 2. Redistributions in binary form must reproduce the above copyright\n+// notice, this list of conditions and the following disclaimer in the\n+// documentation and/or other materials provided with the distribution.\n+//\n+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n+// \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\n+// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\n+// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT\n+// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\n+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT\n+// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\n+// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY\n+// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n+\n+// The generated code of this file depends on the following RISC-V extensions:\n+// - RV64I\n+// - RISC-V Vector ('V') with VLEN >= 128\n+// - RISC-V Vector AES block cipher extension ('Zvkned')\n+\n+#include \n+\n+.text\n+.option arch, +zvkned\n+\n+#include \"../../arch/riscv/crypto/aes-macros.S\"\n+\n+#define RNDKEYS\t\ta0\n+#define KEY_LEN\t\ta1\n+#define OUTP\t\ta2\n+#define INP\t\ta3\n+\n+.macro\t__aes_crypt_zvkned\tenc, keybits\n+\tvle32.v\t\tv16, (INP)\n+\taes_crypt\tv16, \\enc, \\keybits\n+\tvse32.v\t\tv16, (OUTP)\n+\tret\n+.endm\n+\n+.macro\taes_crypt_zvkned\tenc\n+\taes_begin\tRNDKEYS, 128f, 192f, KEY_LEN\n+\t__aes_crypt_zvkned\t\\enc, 256\n+128:\n+\t__aes_crypt_zvkned\t\\enc, 128\n+192:\n+\t__aes_crypt_zvkned\t\\enc, 192\n+.endm\n+\n+// void aes_encrypt_zvkned(const u32 rndkeys[], int key_len,\n+//\t\t\t u8 out[AES_BLOCK_SIZE], const u8 in[AES_BLOCK_SIZE]);\n+SYM_FUNC_START(aes_encrypt_zvkned)\n+\taes_crypt_zvkned\t1\n+SYM_FUNC_END(aes_encrypt_zvkned)\n+\n+// void aes_decrypt_zvkned(const u32 rndkeys[], int key_len,\n+//\t\t\t u8 out[AES_BLOCK_SIZE], const u8 in[AES_BLOCK_SIZE]);\n+SYM_FUNC_START(aes_decrypt_zvkned)\n+\taes_crypt_zvkned\t0\n+SYM_FUNC_END(aes_decrypt_zvkned)\ndiff --git a/lib/crypto/riscv/aes.h b/lib/crypto/riscv/aes.h\nnew file mode 100644\nindex 000000000000..0b26f58faf2b\n--- /dev/null\n+++ b/lib/crypto/riscv/aes.h\n@@ -0,0 +1,63 @@\n+/* SPDX-License-Identifier: GPL-2.0-only */\n+/*\n+ * Copyright (C) 2023 VRULL GmbH\n+ * Copyright (C) 2023 SiFive, Inc.\n+ * Copyright 2024 Google LLC\n+ */\n+\n+#include \n+#include \n+\n+static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_zvkned);\n+\n+void aes_encrypt_zvkned(const u32 rndkeys[], int key_len,\n+\t\t\tu8 out[AES_BLOCK_SIZE], const u8 in[AES_BLOCK_SIZE]);\n+void aes_decrypt_zvkned(const u32 rndkeys[], int key_len,\n+\t\t\tu8 out[AES_BLOCK_SIZE], const u8 in[AES_BLOCK_SIZE]);\n+\n+static void aes_preparekey_arch(union aes_enckey_arch *k,\n+\t\t\t\tunion aes_invkey_arch *inv_k,\n+\t\t\t\tconst u8 *in_key, int key_len, int nrounds)\n+{\n+\taes_expandkey_generic(k->rndkeys, inv_k ? inv_k->inv_rndkeys : NULL,\n+\t\t\t in_key, key_len);\n+}\n+\n+static void aes_encrypt_arch(const struct aes_enckey *key,\n+\t\t\t u8 out[AES_BLOCK_SIZE],\n+\t\t\t const u8 in[AES_BLOCK_SIZE])\n+{\n+\tif (static_branch_likely(&have_zvkned) && likely(may_use_simd())) {\n+\t\tkernel_vector_begin();\n+\t\taes_encrypt_zvkned(key->k.rndkeys, key->len, out, in);\n+\t\tkernel_vector_end();\n+\t} else {\n+\t\taes_encrypt_generic(key->k.rndkeys, key->nrounds, out, in);\n+\t}\n+}\n+\n+static void aes_decrypt_arch(const struct aes_key *key,\n+\t\t\t u8 out[AES_BLOCK_SIZE],\n+\t\t\t const u8 in[AES_BLOCK_SIZE])\n+{\n+\t/*\n+\t * Note that the Zvkned code uses the standard round keys, while the\n+\t * fallback uses the inverse round keys. Thus both must be present.\n+\t */\n+\tif (static_branch_likely(&have_zvkned) && likely(may_use_simd())) {\n+\t\tkernel_vector_begin();\n+\t\taes_decrypt_zvkned(key->k.rndkeys, key->len, out, in);\n+\t\tkernel_vector_end();\n+\t} else {\n+\t\taes_decrypt_generic(key->inv_k.inv_rndkeys, key->nrounds,\n+\t\t\t\t out, in);\n+\t}\n+}\n+\n+#define aes_mod_init_arch aes_mod_init_arch\n+static void aes_mod_init_arch(void)\n+{\n+\tif (riscv_isa_extension_available(NULL, ZVKNED) &&\n+\t riscv_vector_vlen() >= 128)\n+\t\tstatic_branch_enable(&have_zvkned);\n+}\n","prefixes":["v2","14/35"]}