Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/2216761/?format=api
{ "id": 2216761, "url": "http://patchwork.ozlabs.org/api/patches/2216761/?format=api", "web_url": "http://patchwork.ozlabs.org/project/sparclinux/patch/20260327061704.3707577-27-hch@lst.de/", "project": { "id": 10, "url": "http://patchwork.ozlabs.org/api/projects/10/?format=api", "name": "Linux SPARC Development ", "link_name": "sparclinux", "list_id": "sparclinux.vger.kernel.org", "list_email": "sparclinux@vger.kernel.org", "web_url": null, "scm_url": null, "webscm_url": null, "list_archive_url": "", "list_archive_url_format": "", "commit_url_format": "" }, "msgid": "<20260327061704.3707577-27-hch@lst.de>", "list_archive_url": null, "date": "2026-03-27T06:16:58", "name": "[26/28] xor: pass the entire operation to the low-level ops", "commit_ref": null, "pull_url": null, "state": "new", "archived": false, "hash": "b87a6892847e22cbe7962e5326377db5e50a6dfd", "submitter": { "id": 82, "url": "http://patchwork.ozlabs.org/api/people/82/?format=api", "name": "Christoph Hellwig", "email": "hch@lst.de" }, "delegate": null, "mbox": "http://patchwork.ozlabs.org/project/sparclinux/patch/20260327061704.3707577-27-hch@lst.de/mbox/", "series": [ { "id": 497694, "url": "http://patchwork.ozlabs.org/api/series/497694/?format=api", "web_url": "http://patchwork.ozlabs.org/project/sparclinux/list/?series=497694", "date": "2026-03-27T06:16:33", "name": "[01/28] xor: assert that xor_blocks is not call from interrupt context", "version": 1, "mbox": "http://patchwork.ozlabs.org/series/497694/mbox/" } ], "comments": "http://patchwork.ozlabs.org/api/patches/2216761/comments/", "check": "pending", "checks": "http://patchwork.ozlabs.org/api/patches/2216761/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "\n <SRS0=VS9p=B3=vger.kernel.org=sparclinux+bounces-6601-patchwork-incoming=ozlabs.org@ozlabs.org>", "X-Original-To": [ "incoming@patchwork.ozlabs.org", "sparclinux@vger.kernel.org" ], "Delivered-To": [ "patchwork-incoming@legolas.ozlabs.org", "patchwork-incoming@ozlabs.org" ], "Authentication-Results": [ "legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n secure) header.d=infradead.org header.i=@infradead.org header.a=rsa-sha256\n header.s=bombadil.20210309 header.b=Kf8TDQtW;\n\tdkim-atps=neutral", "legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org\n (client-ip=2404:9400:2221:ea00::3; helo=mail.ozlabs.org;\n envelope-from=srs0=vs9p=b3=vger.kernel.org=sparclinux+bounces-6601-patchwork-incoming=ozlabs.org@ozlabs.org;\n receiver=patchwork.ozlabs.org)", "gandalf.ozlabs.org;\n arc=pass smtp.remote-ip=172.234.253.10 arc.chain=subspace.kernel.org", "gandalf.ozlabs.org;\n dmarc=fail (p=none dis=none) header.from=lst.de", "gandalf.ozlabs.org;\n\tdkim=pass (2048-bit key;\n secure) header.d=infradead.org header.i=@infradead.org header.a=rsa-sha256\n header.s=bombadil.20210309 header.b=Kf8TDQtW;\n\tdkim-atps=neutral", "gandalf.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=172.234.253.10; helo=sea.lore.kernel.org;\n envelope-from=sparclinux+bounces-6601-patchwork-incoming=ozlabs.org@vger.kernel.org;\n receiver=ozlabs.org)", "smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org\n header.b=\"Kf8TDQtW\"", "smtp.subspace.kernel.org;\n arc=none smtp.client-ip=198.137.202.133", "smtp.subspace.kernel.org;\n dmarc=fail (p=none dis=none) header.from=lst.de", "smtp.subspace.kernel.org;\n spf=none smtp.mailfrom=bombadil.srs.infradead.org" ], "Received": [ "from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fhrQb4qXtz1y1j\n\tfor <incoming@patchwork.ozlabs.org>; Fri, 27 Mar 2026 17:31:51 +1100 (AEDT)", "from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3])\n\tby gandalf.ozlabs.org (Postfix) with ESMTP id 4fhrQb4Jnfz4wSc\n\tfor <incoming@patchwork.ozlabs.org>; Fri, 27 Mar 2026 17:31:51 +1100 (AEDT)", "by gandalf.ozlabs.org (Postfix)\n\tid 4fhrQb4Dlkz4wT1; Fri, 27 Mar 2026 17:31:51 +1100 (AEDT)", "from sea.lore.kernel.org (sea.lore.kernel.org [172.234.253.10])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby gandalf.ozlabs.org (Postfix) with ESMTPS id 4fhrQW1d52z4wSc\n\tfor <patchwork-incoming@ozlabs.org>; Fri, 27 Mar 2026 17:31:47 +1100 (AEDT)", "from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby sea.lore.kernel.org (Postfix) with ESMTP id 5667E3080D7B\n\tfor <patchwork-incoming@ozlabs.org>; Fri, 27 Mar 2026 06:29:22 +0000 (UTC)", "from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id AF275391E50;\n\tFri, 27 Mar 2026 06:23:43 +0000 (UTC)", "from bombadil.infradead.org (bombadil.infradead.org\n [198.137.202.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 211B73A6B97;\n\tFri, 27 Mar 2026 06:23:33 +0000 (UTC)", "from\n 2a02-8389-2341-5b80-d601-7564-c2e0-491c.cable.dynamic.v6.surfer.at\n ([2a02:8389:2341:5b80:d601:7564:c2e0:491c] helo=localhost)\n\tby bombadil.infradead.org with esmtpsa (Exim 4.98.2 #2 (Red Hat Linux))\n\tid 1w60bG-00000006o5n-04FH;\n\tFri, 27 Mar 2026 06:23:18 +0000" ], "ARC-Seal": [ "i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1774593111; cv=pass;\n\tb=Kpx+Ys4/LKd09xa1cffc9Mla2swar/dly6x+JKktH+UBduxwNGA8OK+NgWtzvDw75fWkTO19DKAKO7DGy4A4HAo5BeMRrsskResFzdtFqQAmx0nXn1c+5+7xoii9DOFbMXV1dmVYjfQktcgAnMKm689ZjTB7YtZTb75i+tLVtvce/z+sDPc4ufyYFTteOql9MqUga80cdTPkDlyoNZMl/1f22TVzvmXTSqTnY17crPUe6Ttbik1HKxiDrK84vgKJlvSpHNHEj1iXtPVOwgqQkJ1kmAoRqB7csFFcPScLrAAz/DjNCCzKqDCPTF6lvra7LMB+pIaF3lCDx0bJ0Siqaw==", "i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1774592622; cv=none;\n b=GrXgmPyNWEYOARWStOyYNX51+rvsWwIqPbeAeRv9fqCaP7CBNfXAH1SAQ8ua4MrtTc722qlx9REtg4Wi1AHZV2jdwiFZEN4yNfNQP/IQ3jMWOIGWng8cxWQwYXWy5OyPVoGDl2cRpXzirIOERAgcfVrcg1pz4T2Ytxhbnc8mGPw=" ], "ARC-Message-Signature": [ "i=2; a=rsa-sha256; d=ozlabs.org; s=201707;\n\tt=1774593111; c=relaxed/relaxed;\n\tbh=SaVrBDZB/tG/efqeAlXq+MRamlDub4+veix+YQd6Qck=;\n\th=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References:\n\t MIME-Version;\n b=lXfWbxdl2muBBL9lZw/3uInISrCxTyBu3cVAhZ+XP5ukr0RGAef5fJl9VmS2Oovu8pp4FQborZPM6+1Lj/CkIITeIHmCzjqnsc+qym9iCxd6sPllY9iIovphTVFIZjjE63ju658IR0pIfPz/KnahEi5Fki62FKkepgdQ0eWoTgNVBYjNgFxkNr2wgEzLp/7J/pWJ5/eQkaZIujvQqD60MMccJky1ENUEN88+sP03CHSzZpw0qPultCFVpPOnowJEEOY8lunPH2Ukmsw7I2JDG40+4HeBEAmRIreD69EKfntqwLrFJizSItbQNMhM/IQWFrWnrFhe41m+LxpaslISkg==", "i=1; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1774592622; c=relaxed/simple;\n\tbh=SByx6hc12cGfeFAANGN5R9WWaMV3Qdpq8OIUjPRFbh4=;\n\th=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References:\n\t MIME-Version;\n b=oBmjbf6WDIUbhOnt3FGMVmjw/LE4AUvHDRbRNfeDBdeHRaJX3Oj3WLjyNGRe68X68L5FzAaFyZlcCHqUF//4mRIbK2B5kv+Wm0UA4x173hFSvJIWcS7fMgejdesa/KAmJ7UBf/zgG5zpng6LRf5f67SDrBT8T+ca1BgY9nexAoA=" ], "ARC-Authentication-Results": [ "i=2; gandalf.ozlabs.org;\n dmarc=fail (p=none dis=none) header.from=lst.de; dkim=pass (2048-bit key;\n secure) header.d=infradead.org header.i=@infradead.org header.a=rsa-sha256\n header.s=bombadil.20210309 header.b=Kf8TDQtW; dkim-atps=neutral;\n spf=pass (client-ip=172.234.253.10; helo=sea.lore.kernel.org;\n envelope-from=sparclinux+bounces-6601-patchwork-incoming=ozlabs.org@vger.kernel.org;\n receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org", "i=1; smtp.subspace.kernel.org;\n dmarc=fail (p=none dis=none) header.from=lst.de;\n spf=none smtp.mailfrom=bombadil.srs.infradead.org;\n dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org\n header.b=Kf8TDQtW; arc=none smtp.client-ip=198.137.202.133" ], "DKIM-Signature": "v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding:\n\tMIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender\n\t:Reply-To:Content-Type:Content-ID:Content-Description;\n\tbh=SaVrBDZB/tG/efqeAlXq+MRamlDub4+veix+YQd6Qck=; b=Kf8TDQtWYv+CnvmwRXv7ZZGr2j\n\t6T3qfGfFWayZ1Cs+U+QeQmcGoNbn2i7Lb2sVIYqtzuNhRuckeZnkSR4XX1IJRcobj3Ha1HVW2qQlT\n\tDWYxNGl84l7G/uWfBmdxiF6BL+MXv576iQn1dCcV8tbWDvshowvsYP8sEHXeySaZcOh8gXfez+7kM\n\tYum7wBwVgkxBI48cbe4gHv1cFvPkxJwQNAatz+g/idB1smg/X75vfGlWzSrV7EFZQrnq1BrVALzMR\n\tU0DmWhFwmWGZbSueBy7CKkhvCZQwxIXPm52slHhH3I7uiCQG/02AN36kmiZMrnkFKtGHlePjf7uqe\n\tUvbISUVg==;", "From": "Christoph Hellwig <hch@lst.de>", "To": "Andrew Morton <akpm@linux-foundation.org>", "Cc": "Richard Henderson <richard.henderson@linaro.org>,\n\tMatt Turner <mattst88@gmail.com>,\n\tMagnus Lindholm <linmag7@gmail.com>,\n\tRussell King <linux@armlinux.org.uk>,\n\tCatalin Marinas <catalin.marinas@arm.com>,\n\tWill Deacon <will@kernel.org>,\n\tArd Biesheuvel <ardb@kernel.org>,\n\tHuacai Chen <chenhuacai@kernel.org>,\n\tWANG Xuerui <kernel@xen0n.name>,\n\tMadhavan Srinivasan <maddy@linux.ibm.com>,\n\tMichael Ellerman <mpe@ellerman.id.au>,\n\tNicholas Piggin <npiggin@gmail.com>,\n\t\"Christophe Leroy (CS GROUP)\" <chleroy@kernel.org>,\n\tPaul Walmsley <pjw@kernel.org>,\n\tPalmer Dabbelt <palmer@dabbelt.com>,\n\tAlbert Ou <aou@eecs.berkeley.edu>,\n\tAlexandre Ghiti <alex@ghiti.fr>,\n\tHeiko Carstens <hca@linux.ibm.com>,\n\tVasily Gorbik <gor@linux.ibm.com>,\n\tAlexander Gordeev <agordeev@linux.ibm.com>,\n\tChristian Borntraeger <borntraeger@linux.ibm.com>,\n\tSven Schnelle <svens@linux.ibm.com>,\n\t\"David S. Miller\" <davem@davemloft.net>,\n\tAndreas Larsson <andreas@gaisler.com>,\n\tRichard Weinberger <richard@nod.at>,\n\tAnton Ivanov <anton.ivanov@cambridgegreys.com>,\n\tJohannes Berg <johannes@sipsolutions.net>,\n\tThomas Gleixner <tglx@kernel.org>,\n\tIngo Molnar <mingo@redhat.com>,\n\tBorislav Petkov <bp@alien8.de>,\n\tDave Hansen <dave.hansen@linux.intel.com>,\n\tx86@kernel.org,\n\t\"H. Peter Anvin\" <hpa@zytor.com>,\n\tHerbert Xu <herbert@gondor.apana.org.au>,\n\tDan Williams <dan.j.williams@intel.com>,\n\tChris Mason <clm@fb.com>,\n\tDavid Sterba <dsterba@suse.com>,\n\tArnd Bergmann <arnd@arndb.de>,\n\tSong Liu <song@kernel.org>,\n\tYu Kuai <yukuai@fnnas.com>,\n\tLi Nan <linan122@huawei.com>,\n\t\"Theodore Ts'o\" <tytso@mit.edu>,\n\t\"Jason A. Donenfeld\" <Jason@zx2c4.com>,\n\tlinux-alpha@vger.kernel.org,\n\tlinux-kernel@vger.kernel.org,\n\tlinux-arm-kernel@lists.infradead.org,\n\tloongarch@lists.linux.dev,\n\tlinuxppc-dev@lists.ozlabs.org,\n\tlinux-riscv@lists.infradead.org,\n\tlinux-s390@vger.kernel.org,\n\tsparclinux@vger.kernel.org,\n\tlinux-um@lists.infradead.org,\n\tlinux-crypto@vger.kernel.org,\n\tlinux-btrfs@vger.kernel.org,\n\tlinux-arch@vger.kernel.org,\n\tlinux-raid@vger.kernel.org", "Subject": "[PATCH 26/28] xor: pass the entire operation to the low-level ops", "Date": "Fri, 27 Mar 2026 07:16:58 +0100", "Message-ID": "<20260327061704.3707577-27-hch@lst.de>", "X-Mailer": "git-send-email 2.47.3", "In-Reply-To": "<20260327061704.3707577-1-hch@lst.de>", "References": "<20260327061704.3707577-1-hch@lst.de>", "Precedence": "bulk", "X-Mailing-List": "sparclinux@vger.kernel.org", "List-Id": "<sparclinux.vger.kernel.org>", "List-Subscribe": "<mailto:sparclinux+subscribe@vger.kernel.org>", "List-Unsubscribe": "<mailto:sparclinux+unsubscribe@vger.kernel.org>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-SRS-Rewrite": "SMTP reverse-path rewritten from <hch@infradead.org> by\n bombadil.infradead.org. See http://www.infradead.org/rpr.html", "X-Spam-Status": "No, score=-0.2 required=5.0 tests=ARC_SIGNED,ARC_VALID,\n\tDKIM_SIGNED,DKIM_VALID,DMARC_NONE,HEADER_FROM_DIFFERENT_DOMAINS,\n\tMAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=disabled\n\tversion=4.0.1", "X-Spam-Checker-Version": "SpamAssassin 4.0.1 (2024-03-25) on gandalf.ozlabs.org" }, "content": "Currently the high-level xor code chunks up all operations into small\nunits for only up to 1 + 4 vectors, and passes it to four different\nmethods. This means the FPU/vector context is entered and left a lot\nfor wide stripes, and a lot of indirect expensive indirect calls are\nperformed. Switch to passing the entire gen_xor request to the\nlow-level ops, and provide a macro to dispatch it to the existing\nhelper.\n\nThis reduce the number of indirect calls and FPU/vector context switches\nby a factor approaching nr_stripes / 4, and also reduces source and\nbinary code size.\n\nSigned-off-by: Christoph Hellwig <hch@lst.de>\n---\n include/linux/raid/xor.h | 5 --\n lib/raid/xor/alpha/xor.c | 19 ++++----\n lib/raid/xor/arm/xor-neon-glue.c | 49 ++------------------\n lib/raid/xor/arm/xor-neon.c | 9 +---\n lib/raid/xor/arm/xor.c | 10 ++--\n lib/raid/xor/arm/xor_arch.h | 3 ++\n lib/raid/xor/arm64/xor-neon-glue.c | 44 ++----------------\n lib/raid/xor/arm64/xor-neon.c | 20 +++++---\n lib/raid/xor/arm64/xor-neon.h | 32 ++-----------\n lib/raid/xor/loongarch/xor_simd_glue.c | 62 +++++--------------------\n lib/raid/xor/powerpc/xor_vmx.c | 40 ++++++++--------\n lib/raid/xor/powerpc/xor_vmx.h | 16 +------\n lib/raid/xor/powerpc/xor_vmx_glue.c | 49 ++------------------\n lib/raid/xor/riscv/xor-glue.c | 43 +++--------------\n lib/raid/xor/s390/xor.c | 9 ++--\n lib/raid/xor/sparc/xor-sparc32.c | 9 ++--\n lib/raid/xor/sparc/xor-sparc64-glue.c | 19 ++++----\n lib/raid/xor/x86/xor-avx.c | 29 ++++--------\n lib/raid/xor/x86/xor-mmx.c | 64 ++++++++++----------------\n lib/raid/xor/x86/xor-sse.c | 63 +++++++++----------------\n lib/raid/xor/xor-32regs-prefetch.c | 10 ++--\n lib/raid/xor/xor-32regs.c | 9 ++--\n lib/raid/xor/xor-8regs-prefetch.c | 11 +++--\n lib/raid/xor/xor-8regs.c | 9 ++--\n lib/raid/xor/xor-core.c | 48 ++-----------------\n lib/raid/xor/xor_impl.h | 48 +++++++++++++------\n 26 files changed, 224 insertions(+), 505 deletions(-)", "diff": "diff --git a/include/linux/raid/xor.h b/include/linux/raid/xor.h\nindex 6d9a39fd85dd..870558c9d36e 100644\n--- a/include/linux/raid/xor.h\n+++ b/include/linux/raid/xor.h\n@@ -2,11 +2,6 @@\n #ifndef _XOR_H\n #define _XOR_H\n \n-#define MAX_XOR_BLOCKS 4\n-\n-extern void xor_blocks(unsigned int count, unsigned int bytes,\n-\tvoid *dest, void **srcs);\n-\n void xor_gen(void *dest, void **srcs, unsigned int src_cnt, unsigned int bytes);\n \n #endif /* _XOR_H */\ndiff --git a/lib/raid/xor/alpha/xor.c b/lib/raid/xor/alpha/xor.c\nindex 90694cc47395..a8f72f2dd3a5 100644\n--- a/lib/raid/xor/alpha/xor.c\n+++ b/lib/raid/xor/alpha/xor.c\n@@ -832,18 +832,17 @@ xor_alpha_prefetch_5:\t\t\t\t\t\t\\n\\\n \t.end xor_alpha_prefetch_5\t\t\t\t\\n\\\n \");\n \n+DO_XOR_BLOCKS(alpha, xor_alpha_2, xor_alpha_3, xor_alpha_4, xor_alpha_5);\n+\n struct xor_block_template xor_block_alpha = {\n-\t.name\t= \"alpha\",\n-\t.do_2\t= xor_alpha_2,\n-\t.do_3\t= xor_alpha_3,\n-\t.do_4\t= xor_alpha_4,\n-\t.do_5\t= xor_alpha_5,\n+\t.name\t\t= \"alpha\",\n+\t.xor_gen\t= xor_gen_alpha,\n };\n \n+DO_XOR_BLOCKS(alpha_prefetch, xor_alpha_prefetch_2, xor_alpha_prefetch_3,\n+\t\txor_alpha_prefetch_4, xor_alpha_prefetch_5);\n+\n struct xor_block_template xor_block_alpha_prefetch = {\n-\t.name\t= \"alpha prefetch\",\n-\t.do_2\t= xor_alpha_prefetch_2,\n-\t.do_3\t= xor_alpha_prefetch_3,\n-\t.do_4\t= xor_alpha_prefetch_4,\n-\t.do_5\t= xor_alpha_prefetch_5,\n+\t.name\t\t= \"alpha prefetch\",\n+\t.xor_gen\t= xor_gen_alpha_prefetch,\n };\ndiff --git a/lib/raid/xor/arm/xor-neon-glue.c b/lib/raid/xor/arm/xor-neon-glue.c\nindex 7afd6294464b..cea39e019904 100644\n--- a/lib/raid/xor/arm/xor-neon-glue.c\n+++ b/lib/raid/xor/arm/xor-neon-glue.c\n@@ -5,54 +5,15 @@\n #include \"xor_impl.h\"\n #include \"xor_arch.h\"\n \n-extern struct xor_block_template const xor_block_neon_inner;\n-\n-static void\n-xor_neon_2(unsigned long bytes, unsigned long * __restrict p1,\n-\t const unsigned long * __restrict p2)\n-{\n-\tkernel_neon_begin();\n-\txor_block_neon_inner.do_2(bytes, p1, p2);\n-\tkernel_neon_end();\n-}\n-\n-static void\n-xor_neon_3(unsigned long bytes, unsigned long * __restrict p1,\n-\t const unsigned long * __restrict p2,\n-\t const unsigned long * __restrict p3)\n-{\n-\tkernel_neon_begin();\n-\txor_block_neon_inner.do_3(bytes, p1, p2, p3);\n-\tkernel_neon_end();\n-}\n-\n-static void\n-xor_neon_4(unsigned long bytes, unsigned long * __restrict p1,\n-\t const unsigned long * __restrict p2,\n-\t const unsigned long * __restrict p3,\n-\t const unsigned long * __restrict p4)\n-{\n-\tkernel_neon_begin();\n-\txor_block_neon_inner.do_4(bytes, p1, p2, p3, p4);\n-\tkernel_neon_end();\n-}\n-\n-static void\n-xor_neon_5(unsigned long bytes, unsigned long * __restrict p1,\n-\t const unsigned long * __restrict p2,\n-\t const unsigned long * __restrict p3,\n-\t const unsigned long * __restrict p4,\n-\t const unsigned long * __restrict p5)\n+static void xor_gen_neon(void *dest, void **srcs, unsigned int src_cnt,\n+\t\tunsigned int bytes)\n {\n \tkernel_neon_begin();\n-\txor_block_neon_inner.do_5(bytes, p1, p2, p3, p4, p5);\n+\txor_gen_neon_inner(dest, srcs, src_cnt, bytes);\n \tkernel_neon_end();\n }\n \n struct xor_block_template xor_block_neon = {\n-\t.name\t= \"neon\",\n-\t.do_2\t= xor_neon_2,\n-\t.do_3\t= xor_neon_3,\n-\t.do_4\t= xor_neon_4,\n-\t.do_5\t= xor_neon_5\n+\t.name\t\t= \"neon\",\n+\t.xor_gen\t= xor_gen_neon,\n };\ndiff --git a/lib/raid/xor/arm/xor-neon.c b/lib/raid/xor/arm/xor-neon.c\nindex 806a42c5952c..23147e3a7904 100644\n--- a/lib/raid/xor/arm/xor-neon.c\n+++ b/lib/raid/xor/arm/xor-neon.c\n@@ -4,6 +4,7 @@\n */\n \n #include \"xor_impl.h\"\n+#include \"xor_arch.h\"\n \n #ifndef __ARM_NEON__\n #error You should compile this file with '-march=armv7-a -mfloat-abi=softfp -mfpu=neon'\n@@ -22,10 +23,4 @@\n #define NO_TEMPLATE\n #include \"../xor-8regs.c\"\n \n-struct xor_block_template const xor_block_neon_inner = {\n-\t.name\t= \"__inner_neon__\",\n-\t.do_2\t= xor_8regs_2,\n-\t.do_3\t= xor_8regs_3,\n-\t.do_4\t= xor_8regs_4,\n-\t.do_5\t= xor_8regs_5,\n-};\n+__DO_XOR_BLOCKS(neon_inner, xor_8regs_2, xor_8regs_3, xor_8regs_4, xor_8regs_5);\ndiff --git a/lib/raid/xor/arm/xor.c b/lib/raid/xor/arm/xor.c\nindex 5bd5f048bbe9..45139b6c55ea 100644\n--- a/lib/raid/xor/arm/xor.c\n+++ b/lib/raid/xor/arm/xor.c\n@@ -127,10 +127,10 @@ xor_arm4regs_5(unsigned long bytes, unsigned long * __restrict p1,\n \t} while (--lines);\n }\n \n+DO_XOR_BLOCKS(arm4regs, xor_arm4regs_2, xor_arm4regs_3, xor_arm4regs_4,\n+\t\txor_arm4regs_5);\n+\n struct xor_block_template xor_block_arm4regs = {\n-\t.name\t= \"arm4regs\",\n-\t.do_2\t= xor_arm4regs_2,\n-\t.do_3\t= xor_arm4regs_3,\n-\t.do_4\t= xor_arm4regs_4,\n-\t.do_5\t= xor_arm4regs_5,\n+\t.name\t\t= \"arm4regs\",\n+\t.xor_gen\t= xor_gen_arm4regs,\n };\ndiff --git a/lib/raid/xor/arm/xor_arch.h b/lib/raid/xor/arm/xor_arch.h\nindex 5a7eedb48fbb..775ff835df65 100644\n--- a/lib/raid/xor/arm/xor_arch.h\n+++ b/lib/raid/xor/arm/xor_arch.h\n@@ -7,6 +7,9 @@\n extern struct xor_block_template xor_block_arm4regs;\n extern struct xor_block_template xor_block_neon;\n \n+void xor_gen_neon_inner(void *dest, void **srcs, unsigned int src_cnt,\n+\t\tunsigned int bytes);\n+\n static __always_inline void __init arch_xor_init(void)\n {\n \txor_register(&xor_block_arm4regs);\ndiff --git a/lib/raid/xor/arm64/xor-neon-glue.c b/lib/raid/xor/arm64/xor-neon-glue.c\nindex 3db0a318cf5b..f0284f86feb4 100644\n--- a/lib/raid/xor/arm64/xor-neon-glue.c\n+++ b/lib/raid/xor/arm64/xor-neon-glue.c\n@@ -10,50 +10,16 @@\n #include \"xor-neon.h\"\n \n #define XOR_TEMPLATE(_name)\t\t\t\t\t\t\\\n-static void\t\t\t\t\t\t\t\t\\\n-xor_##_name##_2(unsigned long bytes, unsigned long * __restrict p1,\t\\\n-\t const unsigned long * __restrict p2)\t\t\t\t\\\n+static void xor_gen_##_name(void *dest, void **srcs, unsigned int src_cnt, \\\n+\t\tunsigned int bytes)\t\t\t\t\t\\\n {\t\t\t\t\t\t\t\t\t\\\n \tscoped_ksimd()\t\t\t\t\t\t\t\\\n-\t\t__xor_##_name##_2(bytes, p1, p2);\t\t\t\\\n-}\t\t\t\t\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\t\\\n-static void\t\t\t\t\t\t\t\t\\\n-xor_##_name##_3(unsigned long bytes, unsigned long * __restrict p1,\t\\\n-\t const unsigned long * __restrict p2,\t\t\t\t\\\n-\t const unsigned long * __restrict p3)\t\t\t\t\\\n-{\t\t\t\t\t\t\t\t\t\\\n-\tscoped_ksimd()\t\t\t\t\t\t\t\\\n-\t\t__xor_##_name##_3(bytes, p1, p2, p3);\t\t\t\\\n-}\t\t\t\t\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\t\\\n-static void\t\t\t\t\t\t\t\t\\\n-xor_##_name##_4(unsigned long bytes, unsigned long * __restrict p1,\t\\\n-\t const unsigned long * __restrict p2,\t\t\t\t\\\n-\t const unsigned long * __restrict p3,\t\t\t\t\\\n-\t const unsigned long * __restrict p4)\t\t\t\t\\\n-{\t\t\t\t\t\t\t\t\t\\\n-\tscoped_ksimd()\t\t\t\t\t\t\t\\\n-\t\t__xor_##_name##_4(bytes, p1, p2, p3, p4);\t\t\\\n-}\t\t\t\t\t\t\t\t\t\\\n-\t\t\t\t\t\t\t\t\t\\\n-static void\t\t\t\t\t\t\t\t\\\n-xor_##_name##_5(unsigned long bytes, unsigned long * __restrict p1,\t\\\n-\t const unsigned long * __restrict p2,\t\t\t\t\\\n-\t const unsigned long * __restrict p3,\t\t\t\t\\\n-\t const unsigned long * __restrict p4,\t\t\t\t\\\n-\t const unsigned long * __restrict p5)\t\t\t\t\\\n-{\t\t\t\t\t\t\t\t\t\\\n-\tscoped_ksimd()\t\t\t\t\t\t\t\\\n-\t\t__xor_##_name##_5(bytes, p1, p2, p3, p4, p5);\t\t\\\n+\t\txor_gen_##_name##_inner(dest, srcs, src_cnt, bytes);\t\\\n }\t\t\t\t\t\t\t\t\t\\\n \t\t\t\t\t\t\t\t\t\\\n struct xor_block_template xor_block_##_name = {\t\t\t\t\\\n-\t.name = __stringify(_name),\t\t\t\t\t\\\n-\t.do_2 = xor_##_name##_2,\t\t\t\t\t\\\n-\t.do_3 = xor_##_name##_3,\t\t\t\t\t\\\n-\t.do_4 = xor_##_name##_4,\t\t\t\t\t\\\n-\t.do_5\t= xor_##_name##_5\t\t\t\t\t\\\n+\t.name \t= __stringify(_name),\t\t\t\t\\\n+\t.xor_gen\t= xor_gen_##_name,\t\t\t\t\\\n };\n \n XOR_TEMPLATE(neon);\ndiff --git a/lib/raid/xor/arm64/xor-neon.c b/lib/raid/xor/arm64/xor-neon.c\nindex 61f00c4fee49..97ef3cb92496 100644\n--- a/lib/raid/xor/arm64/xor-neon.c\n+++ b/lib/raid/xor/arm64/xor-neon.c\n@@ -10,7 +10,7 @@\n #include \"xor_arch.h\"\n #include \"xor-neon.h\"\n \n-void __xor_neon_2(unsigned long bytes, unsigned long * __restrict p1,\n+static void __xor_neon_2(unsigned long bytes, unsigned long * __restrict p1,\n \t\tconst unsigned long * __restrict p2)\n {\n \tuint64_t *dp1 = (uint64_t *)p1;\n@@ -37,7 +37,7 @@ void __xor_neon_2(unsigned long bytes, unsigned long * __restrict p1,\n \t} while (--lines > 0);\n }\n \n-void __xor_neon_3(unsigned long bytes, unsigned long * __restrict p1,\n+static void __xor_neon_3(unsigned long bytes, unsigned long * __restrict p1,\n \t\tconst unsigned long * __restrict p2,\n \t\tconst unsigned long * __restrict p3)\n {\n@@ -73,7 +73,7 @@ void __xor_neon_3(unsigned long bytes, unsigned long * __restrict p1,\n \t} while (--lines > 0);\n }\n \n-void __xor_neon_4(unsigned long bytes, unsigned long * __restrict p1,\n+static void __xor_neon_4(unsigned long bytes, unsigned long * __restrict p1,\n \t\tconst unsigned long * __restrict p2,\n \t\tconst unsigned long * __restrict p3,\n \t\tconst unsigned long * __restrict p4)\n@@ -118,7 +118,7 @@ void __xor_neon_4(unsigned long bytes, unsigned long * __restrict p1,\n \t} while (--lines > 0);\n }\n \n-void __xor_neon_5(unsigned long bytes, unsigned long * __restrict p1,\n+static void __xor_neon_5(unsigned long bytes, unsigned long * __restrict p1,\n \t\tconst unsigned long * __restrict p2,\n \t\tconst unsigned long * __restrict p3,\n \t\tconst unsigned long * __restrict p4,\n@@ -172,6 +172,9 @@ void __xor_neon_5(unsigned long bytes, unsigned long * __restrict p1,\n \t} while (--lines > 0);\n }\n \n+__DO_XOR_BLOCKS(neon_inner, __xor_neon_2, __xor_neon_3, __xor_neon_4,\n+\t\t__xor_neon_5);\n+\n static inline uint64x2_t eor3(uint64x2_t p, uint64x2_t q, uint64x2_t r)\n {\n \tuint64x2_t res;\n@@ -182,7 +185,7 @@ static inline uint64x2_t eor3(uint64x2_t p, uint64x2_t q, uint64x2_t r)\n \treturn res;\n }\n \n-void __xor_eor3_3(unsigned long bytes, unsigned long * __restrict p1,\n+static void __xor_eor3_3(unsigned long bytes, unsigned long * __restrict p1,\n \t\tconst unsigned long * __restrict p2,\n \t\tconst unsigned long * __restrict p3)\n {\n@@ -216,7 +219,7 @@ void __xor_eor3_3(unsigned long bytes, unsigned long * __restrict p1,\n \t} while (--lines > 0);\n }\n \n-void __xor_eor3_4(unsigned long bytes, unsigned long * __restrict p1,\n+static void __xor_eor3_4(unsigned long bytes, unsigned long * __restrict p1,\n \t\tconst unsigned long * __restrict p2,\n \t\tconst unsigned long * __restrict p3,\n \t\tconst unsigned long * __restrict p4)\n@@ -259,7 +262,7 @@ void __xor_eor3_4(unsigned long bytes, unsigned long * __restrict p1,\n \t} while (--lines > 0);\n }\n \n-void __xor_eor3_5(unsigned long bytes, unsigned long * __restrict p1,\n+static void __xor_eor3_5(unsigned long bytes, unsigned long * __restrict p1,\n \t\tconst unsigned long * __restrict p2,\n \t\tconst unsigned long * __restrict p3,\n \t\tconst unsigned long * __restrict p4,\n@@ -304,3 +307,6 @@ void __xor_eor3_5(unsigned long bytes, unsigned long * __restrict p1,\n \t\tdp5 += 8;\n \t} while (--lines > 0);\n }\n+\n+__DO_XOR_BLOCKS(eor3_inner, __xor_neon_2, __xor_eor3_3, __xor_eor3_4,\n+\t\t__xor_eor3_5);\ndiff --git a/lib/raid/xor/arm64/xor-neon.h b/lib/raid/xor/arm64/xor-neon.h\nindex cec0ac846fea..514699ba8f5f 100644\n--- a/lib/raid/xor/arm64/xor-neon.h\n+++ b/lib/raid/xor/arm64/xor-neon.h\n@@ -1,30 +1,6 @@\n /* SPDX-License-Identifier: GPL-2.0-only */\n \n-void __xor_neon_2(unsigned long bytes, unsigned long * __restrict p1,\n-\t\tconst unsigned long * __restrict p2);\n-void __xor_neon_3(unsigned long bytes, unsigned long * __restrict p1,\n-\t\tconst unsigned long * __restrict p2,\n-\t\tconst unsigned long * __restrict p3);\n-void __xor_neon_4(unsigned long bytes, unsigned long * __restrict p1,\n-\t\tconst unsigned long * __restrict p2,\n-\t\tconst unsigned long * __restrict p3,\n-\t\tconst unsigned long * __restrict p4);\n-void __xor_neon_5(unsigned long bytes, unsigned long * __restrict p1,\n-\t\tconst unsigned long * __restrict p2,\n-\t\tconst unsigned long * __restrict p3,\n-\t\tconst unsigned long * __restrict p4,\n-\t\tconst unsigned long * __restrict p5);\n-\n-#define __xor_eor3_2\t__xor_neon_2\n-void __xor_eor3_3(unsigned long bytes, unsigned long * __restrict p1,\n-\t\tconst unsigned long * __restrict p2,\n-\t\tconst unsigned long * __restrict p3);\n-void __xor_eor3_4(unsigned long bytes, unsigned long * __restrict p1,\n-\t\tconst unsigned long * __restrict p2,\n-\t\tconst unsigned long * __restrict p3,\n-\t\tconst unsigned long * __restrict p4);\n-void __xor_eor3_5(unsigned long bytes, unsigned long * __restrict p1,\n-\t\tconst unsigned long * __restrict p2,\n-\t\tconst unsigned long * __restrict p3,\n-\t\tconst unsigned long * __restrict p4,\n-\t\tconst unsigned long * __restrict p5);\n+void xor_gen_neon_inner(void *dest, void **srcs, unsigned int src_cnt,\n+\t\tunsigned int bytes);\n+void xor_gen_eor3_inner(void *dest, void **srcs, unsigned int src_cnt,\n+\t\tunsigned int bytes);\ndiff --git a/lib/raid/xor/loongarch/xor_simd_glue.c b/lib/raid/xor/loongarch/xor_simd_glue.c\nindex b387aa0213b4..7f324d924f87 100644\n--- a/lib/raid/xor/loongarch/xor_simd_glue.c\n+++ b/lib/raid/xor/loongarch/xor_simd_glue.c\n@@ -11,63 +11,23 @@\n #include \"xor_arch.h\"\n #include \"xor_simd.h\"\n \n-#define MAKE_XOR_GLUE_2(flavor)\t\t\t\t\t\t\t\\\n-static void xor_##flavor##_2(unsigned long bytes, unsigned long * __restrict p1,\\\n-\t\t const unsigned long * __restrict p2)\t\t\t\\\n+#define MAKE_XOR_GLUES(flavor)\t\t\t\t\t\t\t\\\n+DO_XOR_BLOCKS(flavor##_inner, __xor_##flavor##_2, __xor_##flavor##_3,\t\t\\\n+\t\t__xor_##flavor##_4, __xor_##flavor##_5);\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\t\\\n+static void xor_gen_##flavor(void *dest, void **srcs, unsigned int src_cnt,\t\\\n+\t\tunsigned int bytes)\t\t\t\t\t\t\\\n {\t\t\t\t\t\t\t\t\t\t\\\n \tkernel_fpu_begin();\t\t\t\t\t\t\t\\\n-\t__xor_##flavor##_2(bytes, p1, p2);\t\t\t\t\t\\\n+\txor_gen_##flavor##_inner(dest, srcs, src_cnt, bytes);\t\t\t\\\n \tkernel_fpu_end();\t\t\t\t\t\t\t\\\n }\t\t\t\t\t\t\t\t\t\t\\\n-\n-#define MAKE_XOR_GLUE_3(flavor)\t\t\t\t\t\t\t\\\n-static void xor_##flavor##_3(unsigned long bytes, unsigned long * __restrict p1,\\\n-\t\t const unsigned long * __restrict p2,\t\t\t\\\n-\t\t const unsigned long * __restrict p3)\t\t\t\\\n-{\t\t\t\t\t\t\t\t\t\t\\\n-\tkernel_fpu_begin();\t\t\t\t\t\t\t\\\n-\t__xor_##flavor##_3(bytes, p1, p2, p3);\t\t\t\t\t\\\n-\tkernel_fpu_end();\t\t\t\t\t\t\t\\\n-}\t\t\t\t\t\t\t\t\t\t\\\n-\n-#define MAKE_XOR_GLUE_4(flavor)\t\t\t\t\t\t\t\\\n-static void xor_##flavor##_4(unsigned long bytes, unsigned long * __restrict p1,\\\n-\t\t const unsigned long * __restrict p2,\t\t\t\\\n-\t\t const unsigned long * __restrict p3,\t\t\t\\\n-\t\t const unsigned long * __restrict p4)\t\t\t\\\n-{\t\t\t\t\t\t\t\t\t\t\\\n-\tkernel_fpu_begin();\t\t\t\t\t\t\t\\\n-\t__xor_##flavor##_4(bytes, p1, p2, p3, p4);\t\t\t\t\\\n-\tkernel_fpu_end();\t\t\t\t\t\t\t\\\n-}\t\t\t\t\t\t\t\t\t\t\\\n-\n-#define MAKE_XOR_GLUE_5(flavor)\t\t\t\t\t\t\t\\\n-static void xor_##flavor##_5(unsigned long bytes, unsigned long * __restrict p1,\\\n-\t\t const unsigned long * __restrict p2,\t\t\t\\\n-\t\t const unsigned long * __restrict p3,\t\t\t\\\n-\t\t const unsigned long * __restrict p4,\t\t\t\\\n-\t\t const unsigned long * __restrict p5)\t\t\t\\\n-{\t\t\t\t\t\t\t\t\t\t\\\n-\tkernel_fpu_begin();\t\t\t\t\t\t\t\\\n-\t__xor_##flavor##_5(bytes, p1, p2, p3, p4, p5);\t\t\t\t\\\n-\tkernel_fpu_end();\t\t\t\t\t\t\t\\\n-}\t\t\t\t\t\t\t\t\t\t\\\n-\n-#define MAKE_XOR_GLUES(flavor)\t\t\t\t\\\n-\tMAKE_XOR_GLUE_2(flavor);\t\t\t\\\n-\tMAKE_XOR_GLUE_3(flavor);\t\t\t\\\n-\tMAKE_XOR_GLUE_4(flavor);\t\t\t\\\n-\tMAKE_XOR_GLUE_5(flavor);\t\t\t\\\n-\t\t\t\t\t\t\t\\\n-struct xor_block_template xor_block_##flavor = {\t\\\n-\t.name = __stringify(flavor),\t\t\t\\\n-\t.do_2 = xor_##flavor##_2,\t\t\t\\\n-\t.do_3 = xor_##flavor##_3,\t\t\t\\\n-\t.do_4 = xor_##flavor##_4,\t\t\t\\\n-\t.do_5 = xor_##flavor##_5,\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\t\\\n+struct xor_block_template xor_block_##flavor = {\t\t\t\t\\\n+\t.name\t\t= __stringify(flavor),\t\t\t\t\t\\\n+\t.xor_gen\t= xor_gen_##flavor\t\t\t\t\t\\\n }\n \n-\n #ifdef CONFIG_CPU_HAS_LSX\n MAKE_XOR_GLUES(lsx);\n #endif /* CONFIG_CPU_HAS_LSX */\ndiff --git a/lib/raid/xor/powerpc/xor_vmx.c b/lib/raid/xor/powerpc/xor_vmx.c\nindex aab49d056d18..09bed98c1bc7 100644\n--- a/lib/raid/xor/powerpc/xor_vmx.c\n+++ b/lib/raid/xor/powerpc/xor_vmx.c\n@@ -10,6 +10,7 @@\n * Sparse (as at v0.5.0) gets very, very confused by this file.\n * Make it a bit simpler for it.\n */\n+#include \"xor_impl.h\"\n #if !defined(__CHECKER__)\n #include <altivec.h>\n #else\n@@ -49,9 +50,9 @@ typedef vector signed char unative_t;\n \t\tV1##_3 = vec_xor(V1##_3, V2##_3);\t\\\n \t} while (0)\n \n-void __xor_altivec_2(unsigned long bytes,\n-\t\t unsigned long * __restrict v1_in,\n-\t\t const unsigned long * __restrict v2_in)\n+static void __xor_altivec_2(unsigned long bytes,\n+\t\tunsigned long * __restrict v1_in,\n+\t\tconst unsigned long * __restrict v2_in)\n {\n \tDEFINE(v1);\n \tDEFINE(v2);\n@@ -68,10 +69,10 @@ void __xor_altivec_2(unsigned long bytes,\n \t} while (--lines > 0);\n }\n \n-void __xor_altivec_3(unsigned long bytes,\n-\t\t unsigned long * __restrict v1_in,\n-\t\t const unsigned long * __restrict v2_in,\n-\t\t const unsigned long * __restrict v3_in)\n+static void __xor_altivec_3(unsigned long bytes,\n+\t\tunsigned long * __restrict v1_in,\n+\t\tconst unsigned long * __restrict v2_in,\n+\t\tconst unsigned long * __restrict v3_in)\n {\n \tDEFINE(v1);\n \tDEFINE(v2);\n@@ -92,11 +93,11 @@ void __xor_altivec_3(unsigned long bytes,\n \t} while (--lines > 0);\n }\n \n-void __xor_altivec_4(unsigned long bytes,\n-\t\t unsigned long * __restrict v1_in,\n-\t\t const unsigned long * __restrict v2_in,\n-\t\t const unsigned long * __restrict v3_in,\n-\t\t const unsigned long * __restrict v4_in)\n+static void __xor_altivec_4(unsigned long bytes,\n+\t\tunsigned long * __restrict v1_in,\n+\t\tconst unsigned long * __restrict v2_in,\n+\t\tconst unsigned long * __restrict v3_in,\n+\t\tconst unsigned long * __restrict v4_in)\n {\n \tDEFINE(v1);\n \tDEFINE(v2);\n@@ -121,12 +122,12 @@ void __xor_altivec_4(unsigned long bytes,\n \t} while (--lines > 0);\n }\n \n-void __xor_altivec_5(unsigned long bytes,\n-\t\t unsigned long * __restrict v1_in,\n-\t\t const unsigned long * __restrict v2_in,\n-\t\t const unsigned long * __restrict v3_in,\n-\t\t const unsigned long * __restrict v4_in,\n-\t\t const unsigned long * __restrict v5_in)\n+static void __xor_altivec_5(unsigned long bytes,\n+\t\tunsigned long * __restrict v1_in,\n+\t\tconst unsigned long * __restrict v2_in,\n+\t\tconst unsigned long * __restrict v3_in,\n+\t\tconst unsigned long * __restrict v4_in,\n+\t\tconst unsigned long * __restrict v5_in)\n {\n \tDEFINE(v1);\n \tDEFINE(v2);\n@@ -154,3 +155,6 @@ void __xor_altivec_5(unsigned long bytes,\n \t\tv5 += 4;\n \t} while (--lines > 0);\n }\n+\n+__DO_XOR_BLOCKS(altivec_inner, __xor_altivec_2, __xor_altivec_3,\n+\t\t__xor_altivec_4, __xor_altivec_5);\ndiff --git a/lib/raid/xor/powerpc/xor_vmx.h b/lib/raid/xor/powerpc/xor_vmx.h\nindex 573c41d90dac..1d26c1133a86 100644\n--- a/lib/raid/xor/powerpc/xor_vmx.h\n+++ b/lib/raid/xor/powerpc/xor_vmx.h\n@@ -6,17 +6,5 @@\n * outside of the enable/disable altivec block.\n */\n \n-void __xor_altivec_2(unsigned long bytes, unsigned long * __restrict p1,\n-\t\t const unsigned long * __restrict p2);\n-void __xor_altivec_3(unsigned long bytes, unsigned long * __restrict p1,\n-\t\t const unsigned long * __restrict p2,\n-\t\t const unsigned long * __restrict p3);\n-void __xor_altivec_4(unsigned long bytes, unsigned long * __restrict p1,\n-\t\t const unsigned long * __restrict p2,\n-\t\t const unsigned long * __restrict p3,\n-\t\t const unsigned long * __restrict p4);\n-void __xor_altivec_5(unsigned long bytes, unsigned long * __restrict p1,\n-\t\t const unsigned long * __restrict p2,\n-\t\t const unsigned long * __restrict p3,\n-\t\t const unsigned long * __restrict p4,\n-\t\t const unsigned long * __restrict p5);\n+void xor_gen_altivec_inner(void *dest, void **srcs, unsigned int src_cnt,\n+\t\tunsigned int bytes);\ndiff --git a/lib/raid/xor/powerpc/xor_vmx_glue.c b/lib/raid/xor/powerpc/xor_vmx_glue.c\nindex 56e99ddfb64f..dbfbb5cadc36 100644\n--- a/lib/raid/xor/powerpc/xor_vmx_glue.c\n+++ b/lib/raid/xor/powerpc/xor_vmx_glue.c\n@@ -12,56 +12,17 @@\n #include \"xor_arch.h\"\n #include \"xor_vmx.h\"\n \n-static void xor_altivec_2(unsigned long bytes, unsigned long * __restrict p1,\n-\t\tconst unsigned long * __restrict p2)\n+static void xor_gen_altivec(void *dest, void **srcs, unsigned int src_cnt,\n+\t\tunsigned int bytes)\n {\n \tpreempt_disable();\n \tenable_kernel_altivec();\n-\t__xor_altivec_2(bytes, p1, p2);\n-\tdisable_kernel_altivec();\n-\tpreempt_enable();\n-}\n-\n-static void xor_altivec_3(unsigned long bytes, unsigned long * __restrict p1,\n-\t\tconst unsigned long * __restrict p2,\n-\t\tconst unsigned long * __restrict p3)\n-{\n-\tpreempt_disable();\n-\tenable_kernel_altivec();\n-\t__xor_altivec_3(bytes, p1, p2, p3);\n-\tdisable_kernel_altivec();\n-\tpreempt_enable();\n-}\n-\n-static void xor_altivec_4(unsigned long bytes, unsigned long * __restrict p1,\n-\t\tconst unsigned long * __restrict p2,\n-\t\tconst unsigned long * __restrict p3,\n-\t\tconst unsigned long * __restrict p4)\n-{\n-\tpreempt_disable();\n-\tenable_kernel_altivec();\n-\t__xor_altivec_4(bytes, p1, p2, p3, p4);\n-\tdisable_kernel_altivec();\n-\tpreempt_enable();\n-}\n-\n-static void xor_altivec_5(unsigned long bytes, unsigned long * __restrict p1,\n-\t\tconst unsigned long * __restrict p2,\n-\t\tconst unsigned long * __restrict p3,\n-\t\tconst unsigned long * __restrict p4,\n-\t\tconst unsigned long * __restrict p5)\n-{\n-\tpreempt_disable();\n-\tenable_kernel_altivec();\n-\t__xor_altivec_5(bytes, p1, p2, p3, p4, p5);\n+\txor_gen_altivec_inner(dest, srcs, src_cnt, bytes);\n \tdisable_kernel_altivec();\n \tpreempt_enable();\n }\n \n struct xor_block_template xor_block_altivec = {\n-\t.name = \"altivec\",\n-\t.do_2 = xor_altivec_2,\n-\t.do_3 = xor_altivec_3,\n-\t.do_4 = xor_altivec_4,\n-\t.do_5 = xor_altivec_5,\n+\t.name\t\t= \"altivec\",\n+\t.xor_gen\t= xor_gen_altivec,\n };\ndiff --git a/lib/raid/xor/riscv/xor-glue.c b/lib/raid/xor/riscv/xor-glue.c\nindex 060e5f22ebcc..2e4c1b05d998 100644\n--- a/lib/raid/xor/riscv/xor-glue.c\n+++ b/lib/raid/xor/riscv/xor-glue.c\n@@ -9,48 +9,17 @@\n #include \"xor_impl.h\"\n #include \"xor_arch.h\"\n \n-static void xor_vector_2(unsigned long bytes, unsigned long *__restrict p1,\n-\t\t\t const unsigned long *__restrict p2)\n-{\n-\tkernel_vector_begin();\n-\txor_regs_2_(bytes, p1, p2);\n-\tkernel_vector_end();\n-}\n-\n-static void xor_vector_3(unsigned long bytes, unsigned long *__restrict p1,\n-\t\t\t const unsigned long *__restrict p2,\n-\t\t\t const unsigned long *__restrict p3)\n-{\n-\tkernel_vector_begin();\n-\txor_regs_3_(bytes, p1, p2, p3);\n-\tkernel_vector_end();\n-}\n-\n-static void xor_vector_4(unsigned long bytes, unsigned long *__restrict p1,\n-\t\t\t const unsigned long *__restrict p2,\n-\t\t\t const unsigned long *__restrict p3,\n-\t\t\t const unsigned long *__restrict p4)\n-{\n-\tkernel_vector_begin();\n-\txor_regs_4_(bytes, p1, p2, p3, p4);\n-\tkernel_vector_end();\n-}\n+DO_XOR_BLOCKS(vector_inner, xor_regs_2_, xor_regs_3_, xor_regs_4_, xor_regs_5_);\n \n-static void xor_vector_5(unsigned long bytes, unsigned long *__restrict p1,\n-\t\t\t const unsigned long *__restrict p2,\n-\t\t\t const unsigned long *__restrict p3,\n-\t\t\t const unsigned long *__restrict p4,\n-\t\t\t const unsigned long *__restrict p5)\n+static void xor_gen_vector(void *dest, void **srcs, unsigned int src_cnt,\n+\t\tunsigned int bytes)\n {\n \tkernel_vector_begin();\n-\txor_regs_5_(bytes, p1, p2, p3, p4, p5);\n+\txor_gen_vector_inner(dest, srcs, src_cnt, bytes);\n \tkernel_vector_end();\n }\n \n struct xor_block_template xor_block_rvv = {\n-\t.name = \"rvv\",\n-\t.do_2 = xor_vector_2,\n-\t.do_3 = xor_vector_3,\n-\t.do_4 = xor_vector_4,\n-\t.do_5 = xor_vector_5\n+\t.name\t\t= \"rvv\",\n+\t.xor_gen\t= xor_gen_vector,\n };\ndiff --git a/lib/raid/xor/s390/xor.c b/lib/raid/xor/s390/xor.c\nindex c28cb56fec92..0c478678a129 100644\n--- a/lib/raid/xor/s390/xor.c\n+++ b/lib/raid/xor/s390/xor.c\n@@ -125,10 +125,9 @@ static void xor_xc_5(unsigned long bytes, unsigned long * __restrict p1,\n \t\t: : \"0\", \"cc\", \"memory\");\n }\n \n+DO_XOR_BLOCKS(xc, xor_xc_2, xor_xc_3, xor_xc_4, xor_xc_5);\n+\n struct xor_block_template xor_block_xc = {\n-\t.name = \"xc\",\n-\t.do_2 = xor_xc_2,\n-\t.do_3 = xor_xc_3,\n-\t.do_4 = xor_xc_4,\n-\t.do_5 = xor_xc_5,\n+\t.name\t\t= \"xc\",\n+\t.xor_gen\t= xor_gen_xc,\n };\ndiff --git a/lib/raid/xor/sparc/xor-sparc32.c b/lib/raid/xor/sparc/xor-sparc32.c\nindex 307c4a84f535..fb37631e90e6 100644\n--- a/lib/raid/xor/sparc/xor-sparc32.c\n+++ b/lib/raid/xor/sparc/xor-sparc32.c\n@@ -244,10 +244,9 @@ sparc_5(unsigned long bytes, unsigned long * __restrict p1,\n \t} while (--lines > 0);\n }\n \n+DO_XOR_BLOCKS(sparc32, sparc_2, sparc_3, sparc_4, sparc_5);\n+\n struct xor_block_template xor_block_SPARC = {\n-\t.name\t= \"SPARC\",\n-\t.do_2\t= sparc_2,\n-\t.do_3\t= sparc_3,\n-\t.do_4\t= sparc_4,\n-\t.do_5\t= sparc_5,\n+\t.name\t\t= \"SPARC\",\n+\t.xor_gen\t= xor_gen_sparc32,\n };\ndiff --git a/lib/raid/xor/sparc/xor-sparc64-glue.c b/lib/raid/xor/sparc/xor-sparc64-glue.c\nindex 5f90c2460b54..a8a686e0d258 100644\n--- a/lib/raid/xor/sparc/xor-sparc64-glue.c\n+++ b/lib/raid/xor/sparc/xor-sparc64-glue.c\n@@ -28,12 +28,11 @@ void xor_vis_5(unsigned long bytes, unsigned long * __restrict p1,\n \n /* XXX Ugh, write cheetah versions... -DaveM */\n \n+DO_XOR_BLOCKS(vis, xor_vis_2, xor_vis_3, xor_vis_4, xor_vis_5);\n+\n struct xor_block_template xor_block_VIS = {\n- .name\t= \"VIS\",\n- .do_2\t= xor_vis_2,\n- .do_3\t= xor_vis_3,\n- .do_4\t= xor_vis_4,\n- .do_5\t= xor_vis_5,\n+ .name\t\t= \"VIS\",\n+\t.xor_gen\t= xor_gen_vis,\n };\n \n void xor_niagara_2(unsigned long bytes, unsigned long * __restrict p1,\n@@ -51,10 +50,10 @@ void xor_niagara_5(unsigned long bytes, unsigned long * __restrict p1,\n \t\t const unsigned long * __restrict p4,\n \t\t const unsigned long * __restrict p5);\n \n+DO_XOR_BLOCKS(niagara, xor_niagara_2, xor_niagara_3, xor_niagara_4,\n+\t\txor_niagara_5);\n+\n struct xor_block_template xor_block_niagara = {\n- .name\t= \"Niagara\",\n- .do_2\t= xor_niagara_2,\n- .do_3\t= xor_niagara_3,\n- .do_4\t= xor_niagara_4,\n- .do_5\t= xor_niagara_5,\n+ .name\t\t= \"Niagara\",\n+\t.xor_gen\t= xor_gen_niagara,\n };\ndiff --git a/lib/raid/xor/x86/xor-avx.c b/lib/raid/xor/x86/xor-avx.c\nindex d411efa1ff43..f7777d7aa269 100644\n--- a/lib/raid/xor/x86/xor-avx.c\n+++ b/lib/raid/xor/x86/xor-avx.c\n@@ -29,8 +29,6 @@ static void xor_avx_2(unsigned long bytes, unsigned long * __restrict p0,\n {\n \tunsigned long lines = bytes >> 9;\n \n-\tkernel_fpu_begin();\n-\n \twhile (lines--) {\n #undef BLOCK\n #define BLOCK(i, reg) \\\n@@ -47,8 +45,6 @@ do { \\\n \t\tp0 = (unsigned long *)((uintptr_t)p0 + 512);\n \t\tp1 = (unsigned long *)((uintptr_t)p1 + 512);\n \t}\n-\n-\tkernel_fpu_end();\n }\n \n static void xor_avx_3(unsigned long bytes, unsigned long * __restrict p0,\n@@ -57,8 +53,6 @@ static void xor_avx_3(unsigned long bytes, unsigned long * __restrict p0,\n {\n \tunsigned long lines = bytes >> 9;\n \n-\tkernel_fpu_begin();\n-\n \twhile (lines--) {\n #undef BLOCK\n #define BLOCK(i, reg) \\\n@@ -78,8 +72,6 @@ do { \\\n \t\tp1 = (unsigned long *)((uintptr_t)p1 + 512);\n \t\tp2 = (unsigned long *)((uintptr_t)p2 + 512);\n \t}\n-\n-\tkernel_fpu_end();\n }\n \n static void xor_avx_4(unsigned long bytes, unsigned long * __restrict p0,\n@@ -89,8 +81,6 @@ static void xor_avx_4(unsigned long bytes, unsigned long * __restrict p0,\n {\n \tunsigned long lines = bytes >> 9;\n \n-\tkernel_fpu_begin();\n-\n \twhile (lines--) {\n #undef BLOCK\n #define BLOCK(i, reg) \\\n@@ -113,8 +103,6 @@ do { \\\n \t\tp2 = (unsigned long *)((uintptr_t)p2 + 512);\n \t\tp3 = (unsigned long *)((uintptr_t)p3 + 512);\n \t}\n-\n-\tkernel_fpu_end();\n }\n \n static void xor_avx_5(unsigned long bytes, unsigned long * __restrict p0,\n@@ -125,8 +113,6 @@ static void xor_avx_5(unsigned long bytes, unsigned long * __restrict p0,\n {\n \tunsigned long lines = bytes >> 9;\n \n-\tkernel_fpu_begin();\n-\n \twhile (lines--) {\n #undef BLOCK\n #define BLOCK(i, reg) \\\n@@ -152,14 +138,19 @@ do { \\\n \t\tp3 = (unsigned long *)((uintptr_t)p3 + 512);\n \t\tp4 = (unsigned long *)((uintptr_t)p4 + 512);\n \t}\n+}\n+\n+DO_XOR_BLOCKS(avx_inner, xor_avx_2, xor_avx_3, xor_avx_4, xor_avx_5);\n \n+static void xor_gen_avx(void *dest, void **srcs, unsigned int src_cnt,\n+\t\t\tunsigned int bytes)\n+{\n+\tkernel_fpu_begin();\n+\txor_gen_avx_inner(dest, srcs, src_cnt, bytes);\n \tkernel_fpu_end();\n }\n \n struct xor_block_template xor_block_avx = {\n-\t.name = \"avx\",\n-\t.do_2 = xor_avx_2,\n-\t.do_3 = xor_avx_3,\n-\t.do_4 = xor_avx_4,\n-\t.do_5 = xor_avx_5,\n+\t.name\t\t= \"avx\",\n+\t.xor_gen\t= xor_gen_avx,\n };\ndiff --git a/lib/raid/xor/x86/xor-mmx.c b/lib/raid/xor/x86/xor-mmx.c\nindex e48c58f92874..63a8b0444fce 100644\n--- a/lib/raid/xor/x86/xor-mmx.c\n+++ b/lib/raid/xor/x86/xor-mmx.c\n@@ -21,8 +21,6 @@ xor_pII_mmx_2(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 7;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n #undef BLOCK\n #define BLOCK(i)\t\t\t\t\\\n@@ -55,8 +53,6 @@ xor_pII_mmx_2(unsigned long bytes, unsigned long * __restrict p1,\n \t \"+r\" (p1), \"+r\" (p2)\n \t:\n \t: \"memory\");\n-\n-\tkernel_fpu_end();\n }\n \n static void\n@@ -66,8 +62,6 @@ xor_pII_mmx_3(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 7;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n #undef BLOCK\n #define BLOCK(i)\t\t\t\t\\\n@@ -105,8 +99,6 @@ xor_pII_mmx_3(unsigned long bytes, unsigned long * __restrict p1,\n \t \"+r\" (p1), \"+r\" (p2), \"+r\" (p3)\n \t:\n \t: \"memory\");\n-\n-\tkernel_fpu_end();\n }\n \n static void\n@@ -117,8 +109,6 @@ xor_pII_mmx_4(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 7;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n #undef BLOCK\n #define BLOCK(i)\t\t\t\t\\\n@@ -161,8 +151,6 @@ xor_pII_mmx_4(unsigned long bytes, unsigned long * __restrict p1,\n \t \"+r\" (p1), \"+r\" (p2), \"+r\" (p3), \"+r\" (p4)\n \t:\n \t: \"memory\");\n-\n-\tkernel_fpu_end();\n }\n \n \n@@ -175,8 +163,6 @@ xor_pII_mmx_5(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 7;\n \n-\tkernel_fpu_begin();\n-\n \t/* Make sure GCC forgets anything it knows about p4 or p5,\n \t such that it won't pass to the asm volatile below a\n \t register that is shared with any other variable. That's\n@@ -237,8 +223,6 @@ xor_pII_mmx_5(unsigned long bytes, unsigned long * __restrict p1,\n \t Clobber them just to be sure nobody does something stupid\n \t like assuming they have some legal value. */\n \tasm(\"\" : \"=r\" (p4), \"=r\" (p5));\n-\n-\tkernel_fpu_end();\n }\n \n #undef LD\n@@ -255,8 +239,6 @@ xor_p5_mmx_2(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 6;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n \t\" .align 32\t ;\\n\"\n \t\" 1: ;\\n\"\n@@ -293,8 +275,6 @@ xor_p5_mmx_2(unsigned long bytes, unsigned long * __restrict p1,\n \t \"+r\" (p1), \"+r\" (p2)\n \t:\n \t: \"memory\");\n-\n-\tkernel_fpu_end();\n }\n \n static void\n@@ -304,8 +284,6 @@ xor_p5_mmx_3(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 6;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n \t\" .align 32,0x90 ;\\n\"\n \t\" 1: ;\\n\"\n@@ -351,8 +329,6 @@ xor_p5_mmx_3(unsigned long bytes, unsigned long * __restrict p1,\n \t \"+r\" (p1), \"+r\" (p2), \"+r\" (p3)\n \t:\n \t: \"memory\" );\n-\n-\tkernel_fpu_end();\n }\n \n static void\n@@ -363,8 +339,6 @@ xor_p5_mmx_4(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 6;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n \t\" .align 32,0x90 ;\\n\"\n \t\" 1: ;\\n\"\n@@ -419,8 +393,6 @@ xor_p5_mmx_4(unsigned long bytes, unsigned long * __restrict p1,\n \t \"+r\" (p1), \"+r\" (p2), \"+r\" (p3), \"+r\" (p4)\n \t:\n \t: \"memory\");\n-\n-\tkernel_fpu_end();\n }\n \n static void\n@@ -432,8 +404,6 @@ xor_p5_mmx_5(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 6;\n \n-\tkernel_fpu_begin();\n-\n \t/* Make sure GCC forgets anything it knows about p4 or p5,\n \t such that it won't pass to the asm volatile below a\n \t register that is shared with any other variable. That's\n@@ -510,22 +480,36 @@ xor_p5_mmx_5(unsigned long bytes, unsigned long * __restrict p1,\n \t Clobber them just to be sure nobody does something stupid\n \t like assuming they have some legal value. */\n \tasm(\"\" : \"=r\" (p4), \"=r\" (p5));\n+}\n+\n+DO_XOR_BLOCKS(pII_mmx_inner, xor_pII_mmx_2, xor_pII_mmx_3, xor_pII_mmx_4,\n+\t\txor_pII_mmx_5);\n \n+static void xor_gen_pII_mmx(void *dest, void **srcs, unsigned int src_cnt,\n+\t\tunsigned int bytes)\n+{\n+\tkernel_fpu_begin();\n+\txor_gen_pII_mmx_inner(dest, srcs, src_cnt, bytes);\n \tkernel_fpu_end();\n }\n \n struct xor_block_template xor_block_pII_mmx = {\n-\t.name = \"pII_mmx\",\n-\t.do_2 = xor_pII_mmx_2,\n-\t.do_3 = xor_pII_mmx_3,\n-\t.do_4 = xor_pII_mmx_4,\n-\t.do_5 = xor_pII_mmx_5,\n+\t.name\t\t= \"pII_mmx\",\n+\t.xor_gen\t= xor_gen_pII_mmx,\n };\n \n+DO_XOR_BLOCKS(p5_mmx_inner, xor_p5_mmx_2, xor_p5_mmx_3, xor_p5_mmx_4,\n+\t\txor_p5_mmx_5);\n+\n+static void xor_gen_p5_mmx(void *dest, void **srcs, unsigned int src_cnt,\n+\t\tunsigned int bytes)\n+{\n+\tkernel_fpu_begin();\n+\txor_gen_p5_mmx_inner(dest, srcs, src_cnt, bytes);\n+\tkernel_fpu_end();\n+}\n+\n struct xor_block_template xor_block_p5_mmx = {\n-\t.name = \"p5_mmx\",\n-\t.do_2 = xor_p5_mmx_2,\n-\t.do_3 = xor_p5_mmx_3,\n-\t.do_4 = xor_p5_mmx_4,\n-\t.do_5 = xor_p5_mmx_5,\n+\t.name\t\t= \"p5_mmx\",\n+\t.xor_gen\t= xor_gen_p5_mmx,\n };\ndiff --git a/lib/raid/xor/x86/xor-sse.c b/lib/raid/xor/x86/xor-sse.c\nindex 5993ed688c15..c6626ecae6ba 100644\n--- a/lib/raid/xor/x86/xor-sse.c\n+++ b/lib/raid/xor/x86/xor-sse.c\n@@ -51,8 +51,6 @@ xor_sse_2(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 8;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n #undef BLOCK\n #define BLOCK(i)\t\t\t\t\t\\\n@@ -93,8 +91,6 @@ xor_sse_2(unsigned long bytes, unsigned long * __restrict p1,\n \t [p1] \"+r\" (p1), [p2] \"+r\" (p2)\n \t: [inc] XOR_CONSTANT_CONSTRAINT (256UL)\n \t: \"memory\");\n-\n-\tkernel_fpu_end();\n }\n \n static void\n@@ -103,8 +99,6 @@ xor_sse_2_pf64(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 8;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n #undef BLOCK\n #define BLOCK(i)\t\t\t\\\n@@ -128,8 +122,6 @@ xor_sse_2_pf64(unsigned long bytes, unsigned long * __restrict p1,\n \t [p1] \"+r\" (p1), [p2] \"+r\" (p2)\n \t: [inc] XOR_CONSTANT_CONSTRAINT (256UL)\n \t: \"memory\");\n-\n-\tkernel_fpu_end();\n }\n \n static void\n@@ -139,8 +131,6 @@ xor_sse_3(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 8;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n #undef BLOCK\n #define BLOCK(i) \\\n@@ -188,8 +178,6 @@ xor_sse_3(unsigned long bytes, unsigned long * __restrict p1,\n \t [p1] \"+r\" (p1), [p2] \"+r\" (p2), [p3] \"+r\" (p3)\n \t: [inc] XOR_CONSTANT_CONSTRAINT (256UL)\n \t: \"memory\");\n-\n-\tkernel_fpu_end();\n }\n \n static void\n@@ -199,8 +187,6 @@ xor_sse_3_pf64(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 8;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n #undef BLOCK\n #define BLOCK(i)\t\t\t\\\n@@ -226,8 +212,6 @@ xor_sse_3_pf64(unsigned long bytes, unsigned long * __restrict p1,\n \t [p1] \"+r\" (p1), [p2] \"+r\" (p2), [p3] \"+r\" (p3)\n \t: [inc] XOR_CONSTANT_CONSTRAINT (256UL)\n \t: \"memory\");\n-\n-\tkernel_fpu_end();\n }\n \n static void\n@@ -238,8 +222,6 @@ xor_sse_4(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 8;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n #undef BLOCK\n #define BLOCK(i) \\\n@@ -294,8 +276,6 @@ xor_sse_4(unsigned long bytes, unsigned long * __restrict p1,\n \t [p2] \"+r\" (p2), [p3] \"+r\" (p3), [p4] \"+r\" (p4)\n \t: [inc] XOR_CONSTANT_CONSTRAINT (256UL)\n \t: \"memory\");\n-\n-\tkernel_fpu_end();\n }\n \n static void\n@@ -306,8 +286,6 @@ xor_sse_4_pf64(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 8;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n #undef BLOCK\n #define BLOCK(i)\t\t\t\\\n@@ -335,8 +313,6 @@ xor_sse_4_pf64(unsigned long bytes, unsigned long * __restrict p1,\n \t [p2] \"+r\" (p2), [p3] \"+r\" (p3), [p4] \"+r\" (p4)\n \t: [inc] XOR_CONSTANT_CONSTRAINT (256UL)\n \t: \"memory\");\n-\n-\tkernel_fpu_end();\n }\n \n static void\n@@ -348,8 +324,6 @@ xor_sse_5(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 8;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n #undef BLOCK\n #define BLOCK(i) \\\n@@ -411,8 +385,6 @@ xor_sse_5(unsigned long bytes, unsigned long * __restrict p1,\n \t [p3] \"+r\" (p3), [p4] \"+r\" (p4), [p5] \"+r\" (p5)\n \t: [inc] XOR_CONSTANT_CONSTRAINT (256UL)\n \t: \"memory\");\n-\n-\tkernel_fpu_end();\n }\n \n static void\n@@ -424,8 +396,6 @@ xor_sse_5_pf64(unsigned long bytes, unsigned long * __restrict p1,\n {\n \tunsigned long lines = bytes >> 8;\n \n-\tkernel_fpu_begin();\n-\n \tasm volatile(\n #undef BLOCK\n #define BLOCK(i)\t\t\t\\\n@@ -455,22 +425,35 @@ xor_sse_5_pf64(unsigned long bytes, unsigned long * __restrict p1,\n \t [p3] \"+r\" (p3), [p4] \"+r\" (p4), [p5] \"+r\" (p5)\n \t: [inc] XOR_CONSTANT_CONSTRAINT (256UL)\n \t: \"memory\");\n+}\n+\n+DO_XOR_BLOCKS(sse_inner, xor_sse_2, xor_sse_3, xor_sse_4, xor_sse_5);\n \n+static void xor_gen_sse(void *dest, void **srcs, unsigned int src_cnt,\n+\t\t\tunsigned int bytes)\n+{\n+\tkernel_fpu_begin();\n+\txor_gen_sse_inner(dest, srcs, src_cnt, bytes);\n \tkernel_fpu_end();\n }\n \n struct xor_block_template xor_block_sse = {\n-\t.name = \"sse\",\n-\t.do_2 = xor_sse_2,\n-\t.do_3 = xor_sse_3,\n-\t.do_4 = xor_sse_4,\n-\t.do_5 = xor_sse_5,\n+\t.name\t\t= \"sse\",\n+\t.xor_gen\t= xor_gen_sse,\n };\n \n+DO_XOR_BLOCKS(sse_pf64_inner, xor_sse_2_pf64, xor_sse_3_pf64, xor_sse_4_pf64,\n+\t\txor_sse_5_pf64);\n+\n+static void xor_gen_sse_pf64(void *dest, void **srcs, unsigned int src_cnt,\n+\t\t\tunsigned int bytes)\n+{\n+\tkernel_fpu_begin();\n+\txor_gen_sse_pf64_inner(dest, srcs, src_cnt, bytes);\n+\tkernel_fpu_end();\n+}\n+\n struct xor_block_template xor_block_sse_pf64 = {\n-\t.name = \"prefetch64-sse\",\n-\t.do_2 = xor_sse_2_pf64,\n-\t.do_3 = xor_sse_3_pf64,\n-\t.do_4 = xor_sse_4_pf64,\n-\t.do_5 = xor_sse_5_pf64,\n+\t.name\t\t= \"prefetch64-sse\",\n+\t.xor_gen\t= xor_gen_sse_pf64,\n };\ndiff --git a/lib/raid/xor/xor-32regs-prefetch.c b/lib/raid/xor/xor-32regs-prefetch.c\nindex 2856a8e50cb8..ade2a7d8cbe2 100644\n--- a/lib/raid/xor/xor-32regs-prefetch.c\n+++ b/lib/raid/xor/xor-32regs-prefetch.c\n@@ -258,10 +258,10 @@ xor_32regs_p_5(unsigned long bytes, unsigned long * __restrict p1,\n \t\tgoto once_more;\n }\n \n+DO_XOR_BLOCKS(32regs_p, xor_32regs_p_2, xor_32regs_p_3, xor_32regs_p_4,\n+\t\txor_32regs_p_5);\n+\n struct xor_block_template xor_block_32regs_p = {\n-\t.name = \"32regs_prefetch\",\n-\t.do_2 = xor_32regs_p_2,\n-\t.do_3 = xor_32regs_p_3,\n-\t.do_4 = xor_32regs_p_4,\n-\t.do_5 = xor_32regs_p_5,\n+\t.name\t\t= \"32regs_prefetch\",\n+\t.xor_gen\t= xor_gen_32regs_p,\n };\ndiff --git a/lib/raid/xor/xor-32regs.c b/lib/raid/xor/xor-32regs.c\nindex cc44d64032fa..acb4a10d1e95 100644\n--- a/lib/raid/xor/xor-32regs.c\n+++ b/lib/raid/xor/xor-32regs.c\n@@ -209,10 +209,9 @@ xor_32regs_5(unsigned long bytes, unsigned long * __restrict p1,\n \t} while (--lines > 0);\n }\n \n+DO_XOR_BLOCKS(32regs, xor_32regs_2, xor_32regs_3, xor_32regs_4, xor_32regs_5);\n+\n struct xor_block_template xor_block_32regs = {\n-\t.name = \"32regs\",\n-\t.do_2 = xor_32regs_2,\n-\t.do_3 = xor_32regs_3,\n-\t.do_4 = xor_32regs_4,\n-\t.do_5 = xor_32regs_5,\n+\t.name\t\t= \"32regs\",\n+\t.xor_gen\t= xor_gen_32regs,\n };\ndiff --git a/lib/raid/xor/xor-8regs-prefetch.c b/lib/raid/xor/xor-8regs-prefetch.c\nindex 1d53aec50d27..451527a951b1 100644\n--- a/lib/raid/xor/xor-8regs-prefetch.c\n+++ b/lib/raid/xor/xor-8regs-prefetch.c\n@@ -136,10 +136,11 @@ xor_8regs_p_5(unsigned long bytes, unsigned long * __restrict p1,\n \t\tgoto once_more;\n }\n \n+\n+DO_XOR_BLOCKS(8regs_p, xor_8regs_p_2, xor_8regs_p_3, xor_8regs_p_4,\n+\t\txor_8regs_p_5);\n+\n struct xor_block_template xor_block_8regs_p = {\n-\t.name = \"8regs_prefetch\",\n-\t.do_2 = xor_8regs_p_2,\n-\t.do_3 = xor_8regs_p_3,\n-\t.do_4 = xor_8regs_p_4,\n-\t.do_5 = xor_8regs_p_5,\n+\t.name\t\t= \"8regs_prefetch\",\n+\t.xor_gen\t= xor_gen_8regs_p,\n };\ndiff --git a/lib/raid/xor/xor-8regs.c b/lib/raid/xor/xor-8regs.c\nindex 72a44e898c55..1edaed8acffe 100644\n--- a/lib/raid/xor/xor-8regs.c\n+++ b/lib/raid/xor/xor-8regs.c\n@@ -94,11 +94,10 @@ xor_8regs_5(unsigned long bytes, unsigned long * __restrict p1,\n }\n \n #ifndef NO_TEMPLATE\n+DO_XOR_BLOCKS(8regs, xor_8regs_2, xor_8regs_3, xor_8regs_4, xor_8regs_5);\n+\n struct xor_block_template xor_block_8regs = {\n-\t.name = \"8regs\",\n-\t.do_2 = xor_8regs_2,\n-\t.do_3 = xor_8regs_3,\n-\t.do_4 = xor_8regs_4,\n-\t.do_5 = xor_8regs_5,\n+\t.name\t\t= \"8regs\",\n+\t.xor_gen\t= xor_gen_8regs,\n };\n #endif /* NO_TEMPLATE */\ndiff --git a/lib/raid/xor/xor-core.c b/lib/raid/xor/xor-core.c\nindex 2e46b6b83b0a..9e043d8c3a7a 100644\n--- a/lib/raid/xor/xor-core.c\n+++ b/lib/raid/xor/xor-core.c\n@@ -13,39 +13,9 @@\n #include <linux/preempt.h>\n #include \"xor_impl.h\"\n \n-/* The xor routines to use. */\n+/* The xor routine to use. */\n static struct xor_block_template *active_template;\n \n-void\n-xor_blocks(unsigned int src_count, unsigned int bytes, void *dest, void **srcs)\n-{\n-\tunsigned long *p1, *p2, *p3, *p4;\n-\n-\tWARN_ON_ONCE(!in_task() || irqs_disabled() || softirq_count());\n-\n-\tp1 = (unsigned long *) srcs[0];\n-\tif (src_count == 1) {\n-\t\tactive_template->do_2(bytes, dest, p1);\n-\t\treturn;\n-\t}\n-\n-\tp2 = (unsigned long *) srcs[1];\n-\tif (src_count == 2) {\n-\t\tactive_template->do_3(bytes, dest, p1, p2);\n-\t\treturn;\n-\t}\n-\n-\tp3 = (unsigned long *) srcs[2];\n-\tif (src_count == 3) {\n-\t\tactive_template->do_4(bytes, dest, p1, p2, p3);\n-\t\treturn;\n-\t}\n-\n-\tp4 = (unsigned long *) srcs[3];\n-\tactive_template->do_5(bytes, dest, p1, p2, p3, p4);\n-}\n-EXPORT_SYMBOL(xor_blocks);\n-\n /**\n * xor_gen - generate RAID-style XOR information\n * @dest:\tdestination vector\n@@ -63,20 +33,11 @@ EXPORT_SYMBOL(xor_blocks);\n */\n void xor_gen(void *dest, void **srcs, unsigned int src_cnt, unsigned int bytes)\n {\n-\tunsigned int src_off = 0;\n-\n-\tWARN_ON_ONCE(in_interrupt());\n+\tWARN_ON_ONCE(!in_task() || irqs_disabled() || softirq_count());\n \tWARN_ON_ONCE(bytes == 0);\n \tWARN_ON_ONCE(bytes & 511);\n \n-\twhile (src_cnt > 0) {\n-\t\tunsigned int this_cnt = min(src_cnt, MAX_XOR_BLOCKS);\n-\n-\t\txor_blocks(this_cnt, bytes, dest, srcs + src_off);\n-\n-\t\tsrc_cnt -= this_cnt;\n-\t\tsrc_off += this_cnt;\n-\t}\n+\tactive_template->xor_gen(dest, srcs, src_cnt, bytes);\n }\n EXPORT_SYMBOL(xor_gen);\n \n@@ -120,6 +81,7 @@ do_xor_speed(struct xor_block_template *tmpl, void *b1, void *b2)\n \tint speed;\n \tunsigned long reps;\n \tktime_t min, start, t0;\n+\tvoid *srcs[1] = { b2 };\n \n \tpreempt_disable();\n \n@@ -130,7 +92,7 @@ do_xor_speed(struct xor_block_template *tmpl, void *b1, void *b2)\n \t\tcpu_relax();\n \tdo {\n \t\tmb(); /* prevent loop optimization */\n-\t\ttmpl->do_2(BENCH_SIZE, b1, b2);\n+\t\ttmpl->xor_gen(b1, srcs, 1, BENCH_SIZE);\n \t\tmb();\n \t} while (reps++ < REPS || (t0 = ktime_get()) == start);\n \tmin = ktime_sub(t0, start);\ndiff --git a/lib/raid/xor/xor_impl.h b/lib/raid/xor/xor_impl.h\nindex 44b6c99e2093..09ae2916f71e 100644\n--- a/lib/raid/xor/xor_impl.h\n+++ b/lib/raid/xor/xor_impl.h\n@@ -3,27 +3,47 @@\n #define _XOR_IMPL_H\n \n #include <linux/init.h>\n+#include <linux/minmax.h>\n \n struct xor_block_template {\n \tstruct xor_block_template *next;\n \tconst char *name;\n \tint speed;\n-\tvoid (*do_2)(unsigned long, unsigned long * __restrict,\n-\t\t const unsigned long * __restrict);\n-\tvoid (*do_3)(unsigned long, unsigned long * __restrict,\n-\t\t const unsigned long * __restrict,\n-\t\t const unsigned long * __restrict);\n-\tvoid (*do_4)(unsigned long, unsigned long * __restrict,\n-\t\t const unsigned long * __restrict,\n-\t\t const unsigned long * __restrict,\n-\t\t const unsigned long * __restrict);\n-\tvoid (*do_5)(unsigned long, unsigned long * __restrict,\n-\t\t const unsigned long * __restrict,\n-\t\t const unsigned long * __restrict,\n-\t\t const unsigned long * __restrict,\n-\t\t const unsigned long * __restrict);\n+\tvoid (*xor_gen)(void *dest, void **srcs, unsigned int src_cnt,\n+\t\t\tunsigned int bytes);\n };\n \n+#define __DO_XOR_BLOCKS(_name, _handle1, _handle2, _handle3, _handle4)\t\\\n+void\t\t\t\t\t\t\t\t\\\n+xor_gen_##_name(void *dest, void **srcs, unsigned int src_cnt,\t\t\\\n+\t\tunsigned int bytes)\t\t\t\t\t\\\n+{\t\t\t\t\t\t\t\t\t\\\n+\tunsigned int src_off = 0;\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\twhile (src_cnt > 0) {\t\t\t\t\t\t\\\n+\t\tunsigned int this_cnt = min(src_cnt, 4);\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\t\tif (this_cnt == 1)\t\t\t\t\t\\\n+\t\t\t_handle1(bytes, dest, srcs[src_off]);\t\t\\\n+\t\telse if (this_cnt == 2)\t\t\t\t\t\\\n+\t\t\t_handle2(bytes, dest, srcs[src_off],\t\t\\\n+\t\t\t\tsrcs[src_off + 1]);\t\t\t\\\n+\t\telse if (this_cnt == 3)\t\t\t\t\t\\\n+\t\t\t_handle3(bytes, dest, srcs[src_off],\t\t\\\n+\t\t\t\tsrcs[src_off + 1], srcs[src_off + 2]);\t\\\n+\t\telse\t\t\t\t\t\t\t\\\n+\t\t\t_handle4(bytes, dest, srcs[src_off],\t\t\\\n+\t\t\t\tsrcs[src_off + 1], srcs[src_off + 2],\t\\\n+\t\t\t\tsrcs[src_off + 3]);\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\t\tsrc_cnt -= this_cnt;\t\t\t\t\t\\\n+\t\tsrc_off += this_cnt;\t\t\t\t\t\\\n+\t}\t\t\t\t\t\t\t\t\\\n+}\n+\n+#define DO_XOR_BLOCKS(_name, _handle1, _handle2, _handle3, _handle4)\t\\\n+\tstatic __DO_XOR_BLOCKS(_name, _handle1, _handle2, _handle3, _handle4)\n+\n /* generic implementations */\n extern struct xor_block_template xor_block_8regs;\n extern struct xor_block_template xor_block_32regs;\n", "prefixes": [ "26/28" ] }