From patchwork Thu Oct 24 15:15:23 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Konstantin Tokarev X-Patchwork-Id: 285935 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from casper.infradead.org (unknown [IPv6:2001:770:15f::2]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id F20182C010A for ; Fri, 25 Oct 2013 02:16:09 +1100 (EST) Received: from merlin.infradead.org ([2001:4978:20e::2]) by casper.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1VZMdg-0001Vv-Ic; Thu, 24 Oct 2013 15:15:56 +0000 Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1VZMda-0003TE-JB; Thu, 24 Oct 2013 15:15:50 +0000 Received: from forward16.mail.yandex.net ([2a02:6b8:0:1402::1]) by merlin.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1VZMdX-0003SL-5O for linux-mtd@lists.infradead.org; Thu, 24 Oct 2013 15:15:49 +0000 Received: from web13g.yandex.ru (web13g.yandex.ru [95.108.252.113]) by forward16.mail.yandex.net (Yandex) with ESMTP id 5F7D5D2240B; Thu, 24 Oct 2013 19:15:24 +0400 (MSK) Received: from 127.0.0.1 (localhost [127.0.0.1]) by web13g.yandex.ru (Yandex) with ESMTP id 788FB620301; Thu, 24 Oct 2013 19:15:23 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1382627724; bh=Yrs00gHqRyaJsAQqaH7TMyANAN/zK6bnvCan9nA8DKE=; h=From:To:In-Reply-To:References:Subject:Date; b=RJd68dPHacYlfb0ETGMtstOlMkBOqUlhLRGMcwB9Bhb4FavtKqgC7U4MAtx2XHkWY kWb1nq/NiXECCKdilgVjQHMmp9cz4PbuICDd7m12H8svwNztQ+Dtv7NNQOl4WI663k FvE4Uew1l/GrB1/LusiA4AMrNkRh7TPoT9LhKgcI= Received: from fw01.smlabs.net (fw01.smlabs.net [185.5.42.3]) by web13g.yandex.ru with HTTP; Thu, 24 Oct 2013 19:15:23 +0400 From: Konstantin Tokarev To: Yann Collet , "linux-mtd@lists.infradead.org" , Brent Taylor , Artem Bityutskiy , linux-kernel@vger.kernel.org, akpm@linux-foundation.org In-Reply-To: <237221382623942@web21m.yandex.ru> References: <55541379679397@web20h.yandex.ru> <237221382623942@web21m.yandex.ru> Subject: Re: lz4hc compression in UBIFS? MIME-Version: 1.0 Message-Id: <106621382627723@web13g.yandex.ru> X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Thu, 24 Oct 2013 19:15:23 +0400 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20131024_111547_653335_56FA6D94 X-CRM114-Status: GOOD ( 12.53 ) X-Spam-Score: 0.4 (/) X-Spam-Report: SpamAssassin version 3.3.2 on merlin.infradead.org summary: Content analysis details: (0.4 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (annulen[at]yandex.ru) -0.0 SPF_PASS SPF: sender matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 2.4 FSL_HELO_BARE_IP_2 FSL_HELO_BARE_IP_2 X-BeenThere: linux-mtd@lists.infradead.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux MTD discussion mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-mtd" Errors-To: linux-mtd-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org 24.10.2013, 18:20, "Konstantin Tokarev" : > 23.10.2013, 22:26, "Yann Collet" : > >>  UBIFS error (pid 4288): ubifs_decompress: cannot decompress 12 bytes, >>  (...) >>          data size      12 >>          data: >>          00000000: 1f 00 01 00 ff e8 50 00 00 00 00 00 >> >>  The compressed format is correct. >>  It describes a flow of 00, of size ~500. >> >>  So the problem seems more likely on the decompression side. >> >>  Are you sure you are providing "12" as the "input size" parameter ? and that >>  the "maximum output size" parameter is correctly provided ? (i.e. >= to >>  original data size) > > Decompression code in kernel[1] is heavily modified. In particular, lz4_uncompress > function (used in this case) does not have input size parameter at all, > while it's present in lz4_uncompress_unknownoutputsize. > > [1] lib/lz4/lz4_decompress.c Attached patch to crypto API wrapper for lz4hc seems to fix the issue. I can save and read files on LZ4HC-compressed volume, and I'm running on LZ4HC-compressed rootfs, flashed from mkfs.ubifs generated image (patch by Elie De Brauwer). My software works correctly. Unfortunately, on my board LZ4HC-compressed UBIFS volume performs noticeable worse than LZO, while still faster than zlib. I guess the reason is that CPU is no longer a bottleneck for my system, but NAND speed. Thank you all for your help! diff --git a/crypto/lz4.c b/crypto/lz4.c index 4586dd1..91a0613 100644 --- a/crypto/lz4.c +++ b/crypto/lz4.c @@ -66,9 +66,8 @@ static int lz4_decompress_crypto(struct crypto_tfm *tfm, const u8 *src, { int err; size_t tmp_len = *dlen; - size_t __slen = slen; - err = lz4_decompress(src, &__slen, dst, tmp_len); + err = lz4_decompress_unknownoutputsize(src, slen, dst, &tmp_len); if (err < 0) return -EINVAL; diff --git a/crypto/lz4hc.c b/crypto/lz4hc.c index 151ba31..9987741 100644 --- a/crypto/lz4hc.c +++ b/crypto/lz4hc.c @@ -66,9 +66,8 @@ static int lz4hc_decompress_crypto(struct crypto_tfm *tfm, const u8 *src, { int err; size_t tmp_len = *dlen; - size_t __slen = slen; - err = lz4_decompress(src, &__slen, dst, tmp_len); + err = lz4_decompress_unknownoutputsize(src, slen, dst, &tmp_len); if (err < 0) return -EINVAL;