Patchwork UBIFS FAQ: document reasons for loss of space efficiency

login
register
mail settings
Submitter Daniel Drake
Date Oct. 10, 2011, 12:07 p.m.
Message ID <20111010120701.C525F9D401E@zog.reactivated.net>
Download mbox | patch
Permalink /patch/118724/
State New
Headers show

Comments

Daniel Drake - Oct. 10, 2011, 12:07 p.m.
OLPC is suffering a bit from the loss of space efficiency of UBIFS,
document reasons for this based on an IRC discussion with Artem.

Also document the difference in default compression schemes between
JFFS2 and UBIFS which bit us at first.

Signed-off-by: Daniel Drake <dsd@laptop.org>
---
 faq/ubifs.xml |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 58 insertions(+), 0 deletions(-)
Ricard Wanderlof - Oct. 10, 2011, 12:31 p.m.
On Mon, 10 Oct 2011, Daniel Drake wrote:

> +this overhead to be approximately 50mb per 1GB of storage (using JFFS2 as a

MB instead of mb (although personally I find the notion of 'millibits' 
rather cute:-)

/Ricard
Jon Povey - Oct. 11, 2011, 3:04 a.m.
linux-mtd-bounces@lists.infradead.org wrote:
> On Mon, 10 Oct 2011, Daniel Drake wrote:
>
>> +this overhead to be approximately 50mb per 1GB of storage (using
>> JFFS2 as a
>
> MB instead of mb (although personally I find the notion of 'millibits'
> rather cute:-)

Should probably be MiB if you want to distinguish 1000^2 from 1024^2

--
Jon Povey
jon.povey@racelogic.co.uk

Racelogic is a limited company registered in England. Registered number 2743719 .
Registered Office Unit 10, Swan Business Centre, Osier Way, Buckingham, Bucks, MK18 1TB .

The information contained in this electronic mail transmission is intended by Racelogic Ltd for the use of the named individual or entity to which it is directed and may contain information that is confidential or privileged. If you have received this electronic mail transmission in error, please delete it from your system without copying or forwarding it, and notify the sender of the error by reply email so that the sender's address records can be corrected. The views expressed by the sender of this communication do not necessarily represent those of Racelogic Ltd. Please note that Racelogic reserves the right to monitor e-mail communications passing through its network
Artem Bityutskiy - Oct. 14, 2011, 10:29 a.m.
On Mon, 2011-10-10 at 13:07 +0100, Daniel Drake wrote:
> OLPC is suffering a bit from the loss of space efficiency of UBIFS,
> document reasons for this based on an IRC discussion with Artem.
> 
> Also document the difference in default compression schemes between
> JFFS2 and UBIFS which bit us at first.
> 
> Signed-off-by: Daniel Drake <dsd@laptop.org>

Changed mb and GB to MiB and GiB and pushed, thanks!

Patch

diff --git a/faq/ubifs.xml b/faq/ubifs.xml
index b6b0b14..a39ef46 100644
--- a/faq/ubifs.xml
+++ b/faq/ubifs.xml
@@ -29,6 +29,7 @@ 
 	<li><a href="ubifs.html#L_nfs">Does UBIFS support NFS?</a></li>
 	<li><a href="ubifs.html#L_slow_when_full">Does UBIFS become slower when it is full?</a></li>
 	<li><a href="ubifs.html#L_df_report">Why <code>df</code> reports too few free space?</a></li>
+	<li><a href="ubifs.html#L_jffs2_space">Why does my UBIFS volume have significantly lower capacity than my equivalent jffs2 volume?</a></li>
 	<li><a href="ubifs.html#L_comproff">How do I disable compression?</a></li>
 	<li><a href="ubifs.html#L_ubifs_nandsim">How do I use UBIFS with nandsim?</a></li>
 	<li><a href="ubifs.html#L_ubifs_extract">How do I extract files from an UBI/UBIFS image?</a></li>
@@ -871,6 +872,63 @@  accurate is free space reporting.</p>
 
 
 
+<h2><a name="L_jffs2_space">Why does my UBIFS volume have significantly lower
+capacity than my equivalent JFFS2 volume?</a></h2>
+
+<p>When migrating a filesystem image from JFFS2 to UBIFS you may notice some
+or all of the following:</p>
+
+<ul>
+	<li>The size of the image file to be flashed is significantly larger
+	than before.</li>
+	<li><code>df</code> reports significantly less available free space
+	than before (while there may be <em>some</em> truth in this, please be
+	aware of the other considerations that must be applied to
+	<code>df</code> values described elsewhere in this FAQ).</li>
+	<li>When measuring the capacity of the disk by seeing how much data
+	can be written, JFFS2 is able to store significantly more data than
+	UBIFS.</li>
+</ul>
+
+<p>There are several reasons for this.</p>
+
+<p>Firstly, did you create your volumes with default arguments to
+<code>mkfs.jffs2</code> and <code>mkfs.ubifs</code>? This alone introduces
+a significant difference: <code>mkfs.jffs2</code> effectively defaults to zlib
+compression and disables lzo, but <code>mkfs.ubifs</code> defaults to LZO
+compression. zlib compresses significantly better, but is noticeably slower to
+compress/decompress at runtime. Using zlib compression with
+<code>mkfs.ubifs</code> will probably reduce the size of your NAND image file
+by 10-15%, put it on-par with an equivalent JFFS2 image, and increase the available capacity, at the price of performance.</p>
+
+<p>Secondly, did you choose a good value for <code>vol_size</code> or enable
+autoresize in your ubinize configuration? It is possible that your UBI volume
+is not utilising the full space of the available flash.</p>
+
+<p>Beyond that, it is unfortunately true that UBI/UBIFS has higher space
+overhead than its predecessors. OLPC has
+<a href="http://lists.laptop.org/pipermail/devel/2011-September/033274.html">measured</a>
+this overhead to be approximately 50mb per 1GB of storage (using JFFS2 as a
+baseline). While UBIFS cannot be made as good as JFFS2 in this respect, work
+could be undertaken to improve space efficiency for current or future
+UBI/UBIFS versions. Reasons for overhead and opportunities for improvement
+include:</p>
+
+<ul>
+	<li>For MTD controllers (such as CaFe) that do not support sub-page
+	writes, 2 KiB is taken from each eraseblock. If such controller drivers
+	were adapted (or hacked) to use software ECC for the first NAND page of
+	every eraseblock, around 8 MiB could be recovered per 1GB of
+	storage.</li>
+	<li>UBIFS on-flash data structures could be reduced in size. These
+	headers currently include many unused fields that are reserved for
+	future use.</li>
+	<li>UBIFS could be improved to reserve less space for internal needs,
+	but touching this part of the code is risky.</li>
+</ul>
+
+
+
 <h2><a name="L_comproff">How do I disable compression?</a></h2>
 
 <p>UBIFS compression may be disabled for whole file system during the image