From patchwork Mon Feb 26 16:40:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 877984 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zqnfq50Ynz9s1S for ; Tue, 27 Feb 2018 03:42:11 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=c-s.fr Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3zqnfq3p4pzF1TH for ; Tue, 27 Feb 2018 03:42:11 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=c-s.fr X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=c-s.fr (client-ip=93.17.236.30; helo=pegase1.c-s.fr; envelope-from=christophe.leroy@c-s.fr; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=c-s.fr Received: from pegase1.c-s.fr (pegase1.c-s.fr [93.17.236.30]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3zqncT2f9fzF10S for ; Tue, 27 Feb 2018 03:40:08 +1100 (AEDT) Received: from localhost (mailhub1-int [192.168.12.234]) by localhost (Postfix) with ESMTP id 3zqncG6d0Rz9ttrf; Mon, 26 Feb 2018 17:39:58 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [192.168.12.234]) (amavisd-new, port 10024) with ESMTP id XJWwDA_ZJ0p0; Mon, 26 Feb 2018 17:39:58 +0100 (CET) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 3zqncG5vMvz9ttrP; Mon, 26 Feb 2018 17:39:58 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id DBC508B8E1; Mon, 26 Feb 2018 17:40:04 +0100 (CET) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id sSkwdyKV6X9r; Mon, 26 Feb 2018 17:40:04 +0100 (CET) Received: from po15720vm.idsi0.si.c-s.fr (po15451.idsi0.si.c-s.fr [172.25.231.5]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 7A4978B8D9; Mon, 26 Feb 2018 17:40:04 +0100 (CET) Received: by po15720vm.idsi0.si.c-s.fr (Postfix, from userid 0) id 4473F6E8BA; Mon, 26 Feb 2018 17:40:04 +0100 (CET) Message-Id: <7c934f0788d60bd82fb1cb51d712c9ab1ea7fcb4.1519660621.git.christophe.leroy@c-s.fr> From: Christophe Leroy Subject: [PATCH 1/2] crypto: talitos - don't persistently map req_ctx->hw_context and req_ctx->buf To: Herbert Xu , "David S. Miller" , Horia Geanta Date: Mon, 26 Feb 2018 17:40:04 +0100 (CET) X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linuxppc-dev@lists.ozlabs.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Commit 49f9783b0cea ("crypto: talitos - do hw_context DMA mapping outside the requests") introduced a persistent dma mapping of req_ctx->hw_context Commit 37b5e8897eb5 ("crypto: talitos - chain in buffered data for ahash on SEC1") introduced a persistent dma mapping of req_ctx->buf As there is no destructor for req_ctx (the request context), the associated dma handlers where set in ctx (the tfm context). This is wrong as several hash operations can run with the same ctx. This patch removes this persistent mapping. Reported-by: Horia Geanta Fixes: 49f9783b0cea ("crypto: talitos - do hw_context DMA mapping outside the requests") Fixes: 37b5e8897eb5 ("crypto: talitos - chain in buffered data for ahash on SEC1") Signed-off-by: Christophe Leroy Tested-by: Horia Geantă --- drivers/crypto/talitos.c | 132 ++++++++++++++++------------------------------- 1 file changed, 44 insertions(+), 88 deletions(-) diff --git a/drivers/crypto/talitos.c b/drivers/crypto/talitos.c index 6882fa2f8bad..bb44df1c1f2d 100644 --- a/drivers/crypto/talitos.c +++ b/drivers/crypto/talitos.c @@ -832,8 +832,6 @@ struct talitos_ctx { unsigned int keylen; unsigned int enckeylen; unsigned int authkeylen; - dma_addr_t dma_buf; - dma_addr_t dma_hw_context; }; #define HASH_MAX_BLOCK_SIZE SHA512_BLOCK_SIZE @@ -1690,9 +1688,30 @@ static void common_nonsnoop_hash_unmap(struct device *dev, struct ahash_request *areq) { struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq); + struct talitos_private *priv = dev_get_drvdata(dev); + bool is_sec1 = has_ftr_sec1(priv); + struct talitos_desc *desc = &edesc->desc; + struct talitos_desc *desc2 = desc + 1; + + unmap_single_talitos_ptr(dev, &edesc->desc.ptr[5], DMA_FROM_DEVICE); + if (desc->next_desc && + desc->ptr[5].ptr != desc2->ptr[5].ptr) + unmap_single_talitos_ptr(dev, &desc2->ptr[5], DMA_FROM_DEVICE); talitos_sg_unmap(dev, edesc, req_ctx->psrc, NULL, 0, 0); + /* When using hashctx-in, must unmap it. */ + if (from_talitos_ptr_len(&edesc->desc.ptr[1], is_sec1)) + unmap_single_talitos_ptr(dev, &edesc->desc.ptr[1], + DMA_TO_DEVICE); + else if (desc->next_desc) + unmap_single_talitos_ptr(dev, &desc2->ptr[1], + DMA_TO_DEVICE); + + if (is_sec1 && req_ctx->nbuf) + unmap_single_talitos_ptr(dev, &desc->ptr[3], + DMA_TO_DEVICE); + if (edesc->dma_len) dma_unmap_single(dev, edesc->dma_link_tbl, edesc->dma_len, DMA_BIDIRECTIONAL); @@ -1766,8 +1785,10 @@ static int common_nonsnoop_hash(struct talitos_edesc *edesc, /* hash context in */ if (!req_ctx->first || req_ctx->swinit) { - to_talitos_ptr(&desc->ptr[1], ctx->dma_hw_context, - req_ctx->hw_context_size, is_sec1); + map_single_talitos_ptr(dev, &desc->ptr[1], + req_ctx->hw_context_size, + (char *)req_ctx->hw_context, + DMA_TO_DEVICE); req_ctx->swinit = 0; } /* Indicate next op is not the first. */ @@ -1793,10 +1814,9 @@ static int common_nonsnoop_hash(struct talitos_edesc *edesc, * data in */ if (is_sec1 && req_ctx->nbuf) { - dma_addr_t dma_buf = ctx->dma_buf + req_ctx->buf_idx * - HASH_MAX_BLOCK_SIZE; - - to_talitos_ptr(&desc->ptr[3], dma_buf, req_ctx->nbuf, is_sec1); + map_single_talitos_ptr(dev, &desc->ptr[3], req_ctx->nbuf, + req_ctx->buf[req_ctx->buf_idx], + DMA_TO_DEVICE); } else { sg_count = talitos_sg_map(dev, req_ctx->psrc, length, edesc, &desc->ptr[3], sg_count, offset, 0); @@ -1812,8 +1832,9 @@ static int common_nonsnoop_hash(struct talitos_edesc *edesc, crypto_ahash_digestsize(tfm), areq->result, DMA_FROM_DEVICE); else - to_talitos_ptr(&desc->ptr[5], ctx->dma_hw_context, - req_ctx->hw_context_size, is_sec1); + map_single_talitos_ptr(dev, &desc->ptr[5], + req_ctx->hw_context_size, + req_ctx->hw_context, DMA_FROM_DEVICE); /* last DWORD empty */ @@ -1832,9 +1853,14 @@ static int common_nonsnoop_hash(struct talitos_edesc *edesc, desc->hdr |= DESC_HDR_MODE0_MDEU_CONT; desc->hdr &= ~DESC_HDR_DONE_NOTIFY; - to_talitos_ptr(&desc2->ptr[1], ctx->dma_hw_context, - req_ctx->hw_context_size, is_sec1); - + if (desc->ptr[1].ptr) + copy_talitos_ptr(&desc2->ptr[1], &desc->ptr[1], + is_sec1); + else + map_single_talitos_ptr(dev, &desc2->ptr[1], + req_ctx->hw_context_size, + req_ctx->hw_context, + DMA_TO_DEVICE); copy_talitos_ptr(&desc2->ptr[2], &desc->ptr[2], is_sec1); sg_count = talitos_sg_map(dev, req_ctx->psrc, length, edesc, &desc2->ptr[3], sg_count, offset, 0); @@ -1842,8 +1868,10 @@ static int common_nonsnoop_hash(struct talitos_edesc *edesc, sync_needed = true; copy_talitos_ptr(&desc2->ptr[5], &desc->ptr[5], is_sec1); if (req_ctx->last) - to_talitos_ptr(&desc->ptr[5], ctx->dma_hw_context, - req_ctx->hw_context_size, is_sec1); + map_single_talitos_ptr(dev, &desc->ptr[5], + req_ctx->hw_context_size, + req_ctx->hw_context, + DMA_FROM_DEVICE); next_desc = dma_map_single(dev, &desc2->hdr1, TALITOS_DESC_SIZE, DMA_BIDIRECTIONAL); @@ -1881,12 +1909,8 @@ static struct talitos_edesc *ahash_edesc_alloc(struct ahash_request *areq, static int ahash_init(struct ahash_request *areq) { struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq); - struct talitos_ctx *ctx = crypto_ahash_ctx(tfm); - struct device *dev = ctx->dev; struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq); unsigned int size; - struct talitos_private *priv = dev_get_drvdata(dev); - bool is_sec1 = has_ftr_sec1(priv); /* Initialize the context */ req_ctx->buf_idx = 0; @@ -1898,18 +1922,6 @@ static int ahash_init(struct ahash_request *areq) : TALITOS_MDEU_CONTEXT_SIZE_SHA384_SHA512; req_ctx->hw_context_size = size; - if (ctx->dma_hw_context) - dma_unmap_single(dev, ctx->dma_hw_context, size, - DMA_BIDIRECTIONAL); - ctx->dma_hw_context = dma_map_single(dev, req_ctx->hw_context, size, - DMA_BIDIRECTIONAL); - if (ctx->dma_buf) - dma_unmap_single(dev, ctx->dma_buf, sizeof(req_ctx->buf), - DMA_TO_DEVICE); - if (is_sec1) - ctx->dma_buf = dma_map_single(dev, req_ctx->buf, - sizeof(req_ctx->buf), - DMA_TO_DEVICE); return 0; } @@ -1920,9 +1932,6 @@ static int ahash_init(struct ahash_request *areq) static int ahash_init_sha224_swinit(struct ahash_request *areq) { struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq); - struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq); - struct talitos_ctx *ctx = crypto_ahash_ctx(tfm); - struct device *dev = ctx->dev; ahash_init(areq); req_ctx->swinit = 1;/* prevent h/w initting context with sha256 values*/ @@ -1940,9 +1949,6 @@ static int ahash_init_sha224_swinit(struct ahash_request *areq) req_ctx->hw_context[8] = 0; req_ctx->hw_context[9] = 0; - dma_sync_single_for_device(dev, ctx->dma_hw_context, - req_ctx->hw_context_size, DMA_TO_DEVICE); - return 0; } @@ -2046,13 +2052,6 @@ static int ahash_process_req(struct ahash_request *areq, unsigned int nbytes) /* request SEC to INIT hash. */ if (req_ctx->first && !req_ctx->swinit) edesc->desc.hdr |= DESC_HDR_MODE0_MDEU_INIT; - if (is_sec1) { - dma_addr_t dma_buf = ctx->dma_buf + req_ctx->buf_idx * - HASH_MAX_BLOCK_SIZE; - - dma_sync_single_for_device(dev, dma_buf, - req_ctx->nbuf, DMA_TO_DEVICE); - } /* When the tfm context has a keylen, it's an HMAC. * A first or last (ie. not middle) descriptor must request HMAC. @@ -2106,12 +2105,7 @@ static int ahash_export(struct ahash_request *areq, void *out) { struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq); struct talitos_export_state *export = out; - struct crypto_ahash *ahash = crypto_ahash_reqtfm(areq); - struct talitos_ctx *ctx = crypto_ahash_ctx(ahash); - struct device *dev = ctx->dev; - dma_sync_single_for_cpu(dev, ctx->dma_hw_context, - req_ctx->hw_context_size, DMA_FROM_DEVICE); memcpy(export->hw_context, req_ctx->hw_context, req_ctx->hw_context_size); memcpy(export->buf, req_ctx->buf[req_ctx->buf_idx], req_ctx->nbuf); @@ -2130,31 +2124,14 @@ static int ahash_import(struct ahash_request *areq, const void *in) struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq); const struct talitos_export_state *export = in; unsigned int size; - struct talitos_ctx *ctx = crypto_ahash_ctx(tfm); - struct device *dev = ctx->dev; - struct talitos_private *priv = dev_get_drvdata(dev); - bool is_sec1 = has_ftr_sec1(priv); memset(req_ctx, 0, sizeof(*req_ctx)); size = (crypto_ahash_digestsize(tfm) <= SHA256_DIGEST_SIZE) ? TALITOS_MDEU_CONTEXT_SIZE_MD5_SHA1_SHA256 : TALITOS_MDEU_CONTEXT_SIZE_SHA384_SHA512; req_ctx->hw_context_size = size; - if (ctx->dma_hw_context) - dma_unmap_single(dev, ctx->dma_hw_context, size, - DMA_BIDIRECTIONAL); - memcpy(req_ctx->hw_context, export->hw_context, size); - ctx->dma_hw_context = dma_map_single(dev, req_ctx->hw_context, size, - DMA_BIDIRECTIONAL); - if (ctx->dma_buf) - dma_unmap_single(dev, ctx->dma_buf, sizeof(req_ctx->buf), - DMA_TO_DEVICE); memcpy(req_ctx->buf[0], export->buf, export->nbuf); - if (is_sec1) - ctx->dma_buf = dma_map_single(dev, req_ctx->buf, - sizeof(req_ctx->buf), - DMA_TO_DEVICE); req_ctx->swinit = export->swinit; req_ctx->first = export->first; req_ctx->last = export->last; @@ -3064,27 +3041,6 @@ static void talitos_cra_exit(struct crypto_tfm *tfm) dma_unmap_single(dev, ctx->dma_key, ctx->keylen, DMA_TO_DEVICE); } -static void talitos_cra_exit_ahash(struct crypto_tfm *tfm) -{ - struct talitos_ctx *ctx = crypto_tfm_ctx(tfm); - struct device *dev = ctx->dev; - unsigned int size; - - talitos_cra_exit(tfm); - - size = (crypto_ahash_digestsize(__crypto_ahash_cast(tfm)) <= - SHA256_DIGEST_SIZE) - ? TALITOS_MDEU_CONTEXT_SIZE_MD5_SHA1_SHA256 - : TALITOS_MDEU_CONTEXT_SIZE_SHA384_SHA512; - - if (ctx->dma_hw_context) - dma_unmap_single(dev, ctx->dma_hw_context, size, - DMA_BIDIRECTIONAL); - if (ctx->dma_buf) - dma_unmap_single(dev, ctx->dma_buf, HASH_MAX_BLOCK_SIZE * 2, - DMA_TO_DEVICE); -} - /* * given the alg's descriptor header template, determine whether descriptor * type and primary/secondary execution units required match the hw @@ -3183,7 +3139,7 @@ static struct talitos_crypto_alg *talitos_alg_alloc(struct device *dev, case CRYPTO_ALG_TYPE_AHASH: alg = &t_alg->algt.alg.hash.halg.base; alg->cra_init = talitos_cra_init_ahash; - alg->cra_exit = talitos_cra_exit_ahash; + alg->cra_exit = talitos_cra_exit; alg->cra_type = &crypto_ahash_type; t_alg->algt.alg.hash.init = ahash_init; t_alg->algt.alg.hash.update = ahash_update; From patchwork Mon Feb 26 16:40:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 877985 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zqnjG01tSz9s1S for ; Tue, 27 Feb 2018 03:44:18 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=c-s.fr Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3zqnjF5CKWzF1TH for ; Tue, 27 Feb 2018 03:44:17 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=c-s.fr X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=c-s.fr (client-ip=93.17.236.30; helo=pegase1.c-s.fr; envelope-from=christophe.leroy@c-s.fr; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=c-s.fr Received: from pegase1.c-s.fr (pegase1.c-s.fr [93.17.236.30]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3zqncV2DtjzF1C6 for ; Tue, 27 Feb 2018 03:40:10 +1100 (AEDT) Received: from localhost (mailhub1-int [192.168.12.234]) by localhost (Postfix) with ESMTP id 3zqncJ5ZPyz9ttrk; Mon, 26 Feb 2018 17:40:00 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [192.168.12.234]) (amavisd-new, port 10024) with ESMTP id dNNPJowE3N47; Mon, 26 Feb 2018 17:40:00 +0100 (CET) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 3zqncJ51tDz9ttrj; Mon, 26 Feb 2018 17:40:00 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 80D7C8B8E3; Mon, 26 Feb 2018 17:40:06 +0100 (CET) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id DRPBQTPYWsEa; Mon, 26 Feb 2018 17:40:06 +0100 (CET) Received: from po15720vm.idsi0.si.c-s.fr (po15451.idsi0.si.c-s.fr [172.25.231.5]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 421BF8B8E1; Mon, 26 Feb 2018 17:40:06 +0100 (CET) Received: by po15720vm.idsi0.si.c-s.fr (Postfix, from userid 0) id 338D96E8BA; Mon, 26 Feb 2018 17:40:06 +0100 (CET) Message-Id: In-Reply-To: <7c934f0788d60bd82fb1cb51d712c9ab1ea7fcb4.1519660621.git.christophe.leroy@c-s.fr> References: <7c934f0788d60bd82fb1cb51d712c9ab1ea7fcb4.1519660621.git.christophe.leroy@c-s.fr> From: Christophe Leroy Subject: [PATCH 2/2] crypto: talitos - do not perform unnecessary dma synchronisation To: Herbert Xu , "David S. Miller" , Horia Geanta Date: Mon, 26 Feb 2018 17:40:06 +0100 (CET) X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linuxppc-dev@lists.ozlabs.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" req_ctx->hw_context is mainly used only by the HW. So it is not needed to sync the HW and the CPU each time hw_context in DMA mapped. This patch modifies the DMA mapping in order to limit synchronisation to necessary situations. Signed-off-by: Christophe Leroy --- drivers/crypto/talitos.c | 85 +++++++++++++++++++++++++++++++++++------------- 1 file changed, 63 insertions(+), 22 deletions(-) diff --git a/drivers/crypto/talitos.c b/drivers/crypto/talitos.c index bb44df1c1f2d..447cb8b1b16a 100644 --- a/drivers/crypto/talitos.c +++ b/drivers/crypto/talitos.c @@ -104,16 +104,34 @@ static void to_talitos_ptr_ext_or(struct talitos_ptr *ptr, u8 val, bool is_sec1) /* * map virtual single (contiguous) pointer to h/w descriptor pointer */ +static void __map_single_talitos_ptr(struct device *dev, + struct talitos_ptr *ptr, + unsigned int len, void *data, + enum dma_data_direction dir, + unsigned long attrs) +{ + dma_addr_t dma_addr = dma_map_single_attrs(dev, data, len, dir, attrs); + struct talitos_private *priv = dev_get_drvdata(dev); + bool is_sec1 = has_ftr_sec1(priv); + + to_talitos_ptr(ptr, dma_addr, len, is_sec1); +} + static void map_single_talitos_ptr(struct device *dev, struct talitos_ptr *ptr, unsigned int len, void *data, enum dma_data_direction dir) { - dma_addr_t dma_addr = dma_map_single(dev, data, len, dir); - struct talitos_private *priv = dev_get_drvdata(dev); - bool is_sec1 = has_ftr_sec1(priv); + __map_single_talitos_ptr(dev, ptr, len, data, dir, 0); +} - to_talitos_ptr(ptr, dma_addr, len, is_sec1); +static void map_single_talitos_ptr_nosync(struct device *dev, + struct talitos_ptr *ptr, + unsigned int len, void *data, + enum dma_data_direction dir) +{ + __map_single_talitos_ptr(dev, ptr, len, data, dir, + DMA_ATTR_SKIP_CPU_SYNC); } /* @@ -1785,10 +1803,10 @@ static int common_nonsnoop_hash(struct talitos_edesc *edesc, /* hash context in */ if (!req_ctx->first || req_ctx->swinit) { - map_single_talitos_ptr(dev, &desc->ptr[1], - req_ctx->hw_context_size, - (char *)req_ctx->hw_context, - DMA_TO_DEVICE); + map_single_talitos_ptr_nosync(dev, &desc->ptr[1], + req_ctx->hw_context_size, + req_ctx->hw_context, + DMA_TO_DEVICE); req_ctx->swinit = 0; } /* Indicate next op is not the first. */ @@ -1832,9 +1850,10 @@ static int common_nonsnoop_hash(struct talitos_edesc *edesc, crypto_ahash_digestsize(tfm), areq->result, DMA_FROM_DEVICE); else - map_single_talitos_ptr(dev, &desc->ptr[5], - req_ctx->hw_context_size, - req_ctx->hw_context, DMA_FROM_DEVICE); + map_single_talitos_ptr_nosync(dev, &desc->ptr[5], + req_ctx->hw_context_size, + req_ctx->hw_context, + DMA_FROM_DEVICE); /* last DWORD empty */ @@ -1857,10 +1876,10 @@ static int common_nonsnoop_hash(struct talitos_edesc *edesc, copy_talitos_ptr(&desc2->ptr[1], &desc->ptr[1], is_sec1); else - map_single_talitos_ptr(dev, &desc2->ptr[1], - req_ctx->hw_context_size, - req_ctx->hw_context, - DMA_TO_DEVICE); + map_single_talitos_ptr_nosync(dev, &desc2->ptr[1], + req_ctx->hw_context_size, + req_ctx->hw_context, + DMA_TO_DEVICE); copy_talitos_ptr(&desc2->ptr[2], &desc->ptr[2], is_sec1); sg_count = talitos_sg_map(dev, req_ctx->psrc, length, edesc, &desc2->ptr[3], sg_count, offset, 0); @@ -1868,10 +1887,10 @@ static int common_nonsnoop_hash(struct talitos_edesc *edesc, sync_needed = true; copy_talitos_ptr(&desc2->ptr[5], &desc->ptr[5], is_sec1); if (req_ctx->last) - map_single_talitos_ptr(dev, &desc->ptr[5], - req_ctx->hw_context_size, - req_ctx->hw_context, - DMA_FROM_DEVICE); + map_single_talitos_ptr_nosync(dev, &desc->ptr[5], + req_ctx->hw_context_size, + req_ctx->hw_context, + DMA_FROM_DEVICE); next_desc = dma_map_single(dev, &desc2->hdr1, TALITOS_DESC_SIZE, DMA_BIDIRECTIONAL); @@ -1909,8 +1928,11 @@ static struct talitos_edesc *ahash_edesc_alloc(struct ahash_request *areq, static int ahash_init(struct ahash_request *areq) { struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq); + struct talitos_ctx *ctx = crypto_ahash_ctx(tfm); + struct device *dev = ctx->dev; struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq); unsigned int size; + dma_addr_t dma; /* Initialize the context */ req_ctx->buf_idx = 0; @@ -1922,6 +1944,10 @@ static int ahash_init(struct ahash_request *areq) : TALITOS_MDEU_CONTEXT_SIZE_SHA384_SHA512; req_ctx->hw_context_size = size; + dma = dma_map_single(dev, req_ctx->hw_context, req_ctx->hw_context_size, + DMA_TO_DEVICE); + dma_unmap_single(dev, dma, req_ctx->hw_context_size, DMA_TO_DEVICE); + return 0; } @@ -1933,9 +1959,6 @@ static int ahash_init_sha224_swinit(struct ahash_request *areq) { struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq); - ahash_init(areq); - req_ctx->swinit = 1;/* prevent h/w initting context with sha256 values*/ - req_ctx->hw_context[0] = SHA224_H0; req_ctx->hw_context[1] = SHA224_H1; req_ctx->hw_context[2] = SHA224_H2; @@ -1949,6 +1972,9 @@ static int ahash_init_sha224_swinit(struct ahash_request *areq) req_ctx->hw_context[8] = 0; req_ctx->hw_context[9] = 0; + ahash_init(areq); + req_ctx->swinit = 1;/* prevent h/w initting context with sha256 values*/ + return 0; } @@ -2105,6 +2131,14 @@ static int ahash_export(struct ahash_request *areq, void *out) { struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq); struct talitos_export_state *export = out; + struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq); + struct talitos_ctx *ctx = crypto_ahash_ctx(tfm); + struct device *dev = ctx->dev; + dma_addr_t dma; + + dma = dma_map_single(dev, req_ctx->hw_context, req_ctx->hw_context_size, + DMA_FROM_DEVICE); + dma_unmap_single(dev, dma, req_ctx->hw_context_size, DMA_FROM_DEVICE); memcpy(export->hw_context, req_ctx->hw_context, req_ctx->hw_context_size); @@ -2122,8 +2156,11 @@ static int ahash_import(struct ahash_request *areq, const void *in) { struct talitos_ahash_req_ctx *req_ctx = ahash_request_ctx(areq); struct crypto_ahash *tfm = crypto_ahash_reqtfm(areq); + struct talitos_ctx *ctx = crypto_ahash_ctx(tfm); + struct device *dev = ctx->dev; const struct talitos_export_state *export = in; unsigned int size; + dma_addr_t dma; memset(req_ctx, 0, sizeof(*req_ctx)); size = (crypto_ahash_digestsize(tfm) <= SHA256_DIGEST_SIZE) @@ -2138,6 +2175,10 @@ static int ahash_import(struct ahash_request *areq, const void *in) req_ctx->to_hash_later = export->to_hash_later; req_ctx->nbuf = export->nbuf; + dma = dma_map_single(dev, req_ctx->hw_context, req_ctx->hw_context_size, + DMA_TO_DEVICE); + dma_unmap_single(dev, dma, req_ctx->hw_context_size, DMA_TO_DEVICE); + return 0; }