Patchwork ubiattach fails with "bad image sequence number"

login
register
mail settings
Submitter Artem Bityutskiy
Date Jan. 9, 2010, 11:51 p.m.
Message ID <1263081089.7315.119.camel@localhost.localdomain>
Download mbox | patch
Permalink /patch/42566/
State New, archived
Headers show

Comments

Artem Bityutskiy - Jan. 9, 2010, 11:51 p.m.
Hi,

On Mon, 2009-12-21 at 17:33 -0500, Jeff Angielski wrote:
> If I ubiformat my NAND partition with an UBIFS image I keep on getting 
> "bad image sequence number" when I try to do the ubiattach.  If I just 
> do a plain ubiformat with no image then everything works fine when I 
> ubiattach.
> 
> I was wondering if somebody ran into something similar.  The steps are 
> so trivial it seems like something fundamentally wrong with either the 
> mkfs.ubifs or the ubiformat.
> 
> I did not see anything related to this on the UBFI FAQ or documentation.

I believe this is an ubiformat bug. Thanks for reporting and sorry for
inconvenience.

I attach and inline 2 patches which should fix this. I cannot have a
possibility to test them now, so they are untested. They are against the
latest mtd-utils.git (commit a4e502d99129da8ebba6d40b373a4422a175e9af).

Here we go.


>From 69d3d2a45e58d6f1561d7685008e90dae459cb85 Mon Sep 17 00:00:00 2001
From: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Date: Sun, 10 Jan 2010 01:33:31 +0200
Subject: [PATCH 1/2] ubiformat: always initialize seq number

For some reasons sequence number was set to 0 in some case, which
is wrong.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
---
 ubi-utils/src/ubiformat.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
Artem Bityutskiy - Jan. 9, 2010, 11:53 p.m.
On Sun, 2010-01-10 at 01:51 +0200, Artem Bityutskiy wrote:
> Hi,
> 
> On Mon, 2009-12-21 at 17:33 -0500, Jeff Angielski wrote:
> > If I ubiformat my NAND partition with an UBIFS image I keep on getting 
> > "bad image sequence number" when I try to do the ubiattach.  If I just 
> > do a plain ubiformat with no image then everything works fine when I 
> > ubiattach.
> > 
> > I was wondering if somebody ran into something similar.  The steps are 
> > so trivial it seems like something fundamentally wrong with either the 
> > mkfs.ubifs or the ubiformat.
> > 
> > I did not see anything related to this on the UBFI FAQ or documentation.
> 
> I believe this is an ubiformat bug. Thanks for reporting and sorry for
> inconvenience.
> 
> I attach and inline 2 patches which should fix this. I cannot have a
> possibility to test them now, so they are untested. They are against the
> latest mtd-utils.git (commit a4e502d99129da8ebba6d40b373a4422a175e9af).

Could you please try them and report if they fix your problem or not?

P.S. when we fix this, we'll have to tag a new mtd-utils version, since
this is a very bad bug.
Jeff Angielski - Jan. 13, 2010, 3:26 a.m.
Artem Bityutskiy wrote:
> On Sun, 2010-01-10 at 01:51 +0200, Artem Bityutskiy wrote:
>> Hi,
>>
>> On Mon, 2009-12-21 at 17:33 -0500, Jeff Angielski wrote:
>>> If I ubiformat my NAND partition with an UBIFS image I keep on getting 
>>> "bad image sequence number" when I try to do the ubiattach.  If I just 
>>> do a plain ubiformat with no image then everything works fine when I 
>>> ubiattach.
>>>
>>> I was wondering if somebody ran into something similar.  The steps are 
>>> so trivial it seems like something fundamentally wrong with either the 
>>> mkfs.ubifs or the ubiformat.
>>>
>>> I did not see anything related to this on the UBFI FAQ or documentation.
>> I believe this is an ubiformat bug. Thanks for reporting and sorry for
>> inconvenience.
>>
>> I attach and inline 2 patches which should fix this. I cannot have a
>> possibility to test them now, so they are untested. They are against the
>> latest mtd-utils.git (commit a4e502d99129da8ebba6d40b373a4422a175e9af).
> 
> Could you please try them and report if they fix your problem or not?
> 
> P.S. when we fix this, we'll have to tag a new mtd-utils version, since
> this is a very bad bug.

This patch worked for me.

FWIW, I am using 2.6.31 kernel and the mtd-utils last commit was: 
a4e502d99129da8ebba6d40b373a4422a175e9af.

I did receive a warning "ubi_eba_init_scan: cannot reserve enough PEBs 
for bad PEB handling, reserved 9, need 20" but I believe that is 
operator error related to the large volume size in the mtd partition.

I have one slight change from the original command line commands I 
posted.  For mkfs.ubifs my original -e was incorrectly the PEB instead 
of the LEB.  So for those kids following along, don't use the original 
command sequence.


So the new command line sequence on the x86 host looks like:

$ mkfs.ubifs -r build/targetfs_tiny -m 4096 -e 258048 -c 2047 -o tiny.ubifs

$ ubinize -s 1024 -o ubifs1b.img -m 4096 -p 256KiB ubifs1.cfg

$ cat ubifs1.cfg
[ubifs]
mode=ubi
image=tiny.ubifs
vol_id=0
vol_size=500MiB
vol_type=dynamic
vol_name=fs1
vol_alignment=1
vol_flags=autoresize

And on the PowerPC target:

# ubiformat -s 1024 -f ubifs1b.img /dev/mtd8
ubiformat: mtd8 (nand), size 536870912 bytes (512.0 MiB), 2048 
eraseblocks of 262144 bytes (256.0 KiB), min. I/O size 4096 bytes
libscan: scanning eraseblock 2047 -- 100 % complete 

ubiformat: 2045 eraseblocks have valid erase counter, mean value is 6 

ubiformat: 3 bad eraseblocks found, numbers: 72, 930, 1846 

ubiformat: flashing eraseblock 30 -- 100 % complete 

ubiformat: formatting eraseblock 2047 -- 100 % complete 


# ubiattach /dev/ubi_ctrl -m 8
[ 3149.923007] UBI: attaching mtd8 to ubi0
[ 3149.927224] UBI: physical eraseblock size:   262144 bytes (256 KiB)
[ 3149.934138] UBI: logical eraseblock size:    258048 bytes
[ 3149.939914] UBI: smallest flash I/O unit:    4096
[ 3149.944762] UBI: sub-page size:              1024
[ 3149.949573] UBI: VID header offset:          1024 (aligned 1024)
[ 3149.955688] UBI: data offset:                4096
[ 3151.183443] UBI warning: ubi_eba_init_scan: cannot reserve enough 
PEBs for bad PEB handling, reserved 9, need 20
[ 3151.209875] UBI: volume 0 ("fs1") re-sized from 2032 to 2032 LEBs
[ 3151.224125] UBI: attached mtd8 to ubi0
[ 3151.228345] UBI: MTD device name:            "fs1"
[ 3151.234848] UBI: MTD device size:            512 MiB
[ 3151.239993] UBI: number of good PEBs:        2045
[ 3151.244830] UBI: number of bad PEBs:         3
[ 3151.249382] UBI: max. allowed volumes:       128
[ 3151.254106] UBI: wear-leveling threshold:    4096
[ 3151.258916] UBI: number of internal volumes: 1
[ 3151.263465] UBI: number of user volumes:     1
[ 3151.268013] UBI: available PEBs:             0
[ 3151.272562] UBI: total number of reserved PEBs: 2045
[ 3151.277633] UBI: number of PEBs reserved for bad PEB handling: 9
[ 3151.283748] UBI: max/mean erase counter: 8/7
[ 3151.288124] UBI: image sequence number: 882190247
[ 3151.292941] UBI: background thread "ubi_bgt0d" started, PID 1055
UBI device number 0, total 2045 LEBs (527708160 bytes, 503.3 MiB), 
available 0 LEBs (0 bytes), LEB size 258048 bytes (252.0 KiB)

# mount -t ubifs ubi0:fs1 /mnt/ubifs
[ 3165.535851] UBIFS: mounted UBI device 0, volume 0, name "fs1"
[ 3165.541636] UBIFS: file system size:   521773056 bytes (509544 KiB, 
497 MiB, 2022 LEBs)
[ 3165.549640] UBIFS: journal size:       9420800 bytes (9200 KiB, 8 
MiB, 37 LEBs)
[ 3165.556948] UBIFS: media format:       w4/r0 (latest is w4/r0)
[ 3165.562779] UBIFS: default compressor: lzo
[ 3165.566873] UBIFS: reserved for root:  0 bytes (0 KiB)
Artem Bityutskiy - Jan. 15, 2010, 5:15 p.m.
On Tue, 2010-01-12 at 22:26 -0500, Jeff Angielski wrote:
> Artem Bityutskiy wrote:
> > On Sun, 2010-01-10 at 01:51 +0200, Artem Bityutskiy wrote:
> >> Hi,
> >>
> >> On Mon, 2009-12-21 at 17:33 -0500, Jeff Angielski wrote:
> >>> If I ubiformat my NAND partition with an UBIFS image I keep on getting 
> >>> "bad image sequence number" when I try to do the ubiattach.  If I just 
> >>> do a plain ubiformat with no image then everything works fine when I 
> >>> ubiattach.
> >>>
> >>> I was wondering if somebody ran into something similar.  The steps are 
> >>> so trivial it seems like something fundamentally wrong with either the 
> >>> mkfs.ubifs or the ubiformat.
> >>>
> >>> I did not see anything related to this on the UBFI FAQ or documentation.
> >> I believe this is an ubiformat bug. Thanks for reporting and sorry for
> >> inconvenience.
> >>
> >> I attach and inline 2 patches which should fix this. I cannot have a
> >> possibility to test them now, so they are untested. They are against the
> >> latest mtd-utils.git (commit a4e502d99129da8ebba6d40b373a4422a175e9af).
> > 
> > Could you please try them and report if they fix your problem or not?
> > 
> > P.S. when we fix this, we'll have to tag a new mtd-utils version, since
> > this is a very bad bug.
> 
> This patch worked for me.

Ok, pushed them.

> FWIW, I am using 2.6.31 kernel and the mtd-utils last commit was: 
> a4e502d99129da8ebba6d40b373a4422a175e9af.
> 
> I did receive a warning "ubi_eba_init_scan: cannot reserve enough PEBs 
> for bad PEB handling, reserved 9, need 20" but I believe that is 
> operator error related to the large volume size in the mtd partition.

Yes, create smaller volumes.

> I have one slight change from the original command line commands I 
> posted.  For mkfs.ubifs my original -e was incorrectly the PEB instead 
> of the LEB.  So for those kids following along, don't use the original 
> command sequence.
> 
> 
> So the new command line sequence on the x86 host looks like:
> 
> $ mkfs.ubifs -r build/targetfs_tiny -m 4096 -e 258048 -c 2047 -o tiny.ubifs
> 
> $ ubinize -s 1024 -o ubifs1b.img -m 4096 -p 256KiB ubifs1.cfg
> 
> $ cat ubifs1.cfg
> [ubifs]
> mode=ubi
> image=tiny.ubifs
> vol_id=0
> vol_size=500MiB
> vol_type=dynamic
> vol_name=fs1
> vol_alignment=1
> vol_flags=autoresize

You need to make vol_size=500MiB smaller to get rid of the warning.
Peter Korsgaard - Jan. 16, 2010, 1:22 p.m.
>>>>> "Artem" == Artem Bityutskiy <dedekind1@gmail.com> writes:

Hi,

 >> > P.S. when we fix this, we'll have to tag a new mtd-utils version, since
 >> > this is a very bad bug.
 >> 
 >> This patch worked for me.

 Artem> Ok, pushed them.

David, could we have a 1.3.1 tarball on
ftp://ftp.infradead.org/pub/mtd-utils/ as well?

Thanks.
Peter Korsgaard - Jan. 26, 2010, 9:47 a.m.
>>>>> "Peter" == Peter Korsgaard <jacmet@sunsite.dk> writes:

Hi,

 >>> > P.S. when we fix this, we'll have to tag a new mtd-utils version, since
 >>> > this is a very bad bug.
 >>> 
 >>> This patch worked for me.

 Artem> Ok, pushed them.

 Peter> David, could we have a 1.3.1 tarball on
 Peter> ftp://ftp.infradead.org/pub/mtd-utils/ as well?

 Peter> Thanks.

Ping?
David Woodhouse - Jan. 28, 2010, 6:56 p.m.
On Tue, 2010-01-26 at 10:47 +0100, Peter Korsgaard wrote:
> >>>>> "Peter" == Peter Korsgaard <jacmet@sunsite.dk> writes:
> 
> Hi,
> 
>  >>> > P.S. when we fix this, we'll have to tag a new mtd-utils version, since
>  >>> > this is a very bad bug.
>  >>> 
>  >>> This patch worked for me.
> 
>  Artem> Ok, pushed them.
> 
>  Peter> David, could we have a 1.3.1 tarball on
>  Peter> ftp://ftp.infradead.org/pub/mtd-utils/ as well?
> 
>  Peter> Thanks.
> 
> Ping?

Done.

Patch

From 180c09de57d8e254716c562bd0537df543d12577 Mon Sep 17 00:00:00 2001
From: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Date: Sun, 10 Jan 2010 01:39:24 +0200
Subject: [PATCH 2/2] ubiformat: be consistent with sequence numbers

This commit fixes a stupid an nasty bug. When we flash an UBI image,
we do not change its sequence numbers. But when we format the rest
of the PEBs (beyond the flashed image), we use a random (or specified
via cmdline) sequence number. As a result, we have a broken flash
format and UBI refuses it, because half of it has one sequence number,
another half has a different one.

What we have to do instead, we have to substitute image's sequence
number with ours.

Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
---
 ubi-utils/src/ubiformat.c |   11 +++++++----
 1 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/ubi-utils/src/ubiformat.c b/ubi-utils/src/ubiformat.c
index 80f3a6d..2316d67 100644
--- a/ubi-utils/src/ubiformat.c
+++ b/ubi-utils/src/ubiformat.c
@@ -295,7 +295,8 @@  static void print_bad_eraseblocks(const struct mtd_dev_info *mtd,
 	printf("\n");
 }
 
-static int change_ec(struct ubi_ec_hdr *hdr, long long ec)
+static int change_ech(struct ubi_ec_hdr *hdr, uint32_t image_seq,
+		      long long ec)
 {
 	uint32_t crc;
 
@@ -309,6 +310,7 @@  static int change_ec(struct ubi_ec_hdr *hdr, long long ec)
 		return errmsg("bad CRC %#08x, should be %#08x\n",
 			      crc, be32_to_cpu(hdr->hdr_crc));
 
+	hdr->image_seq = cpu_to_be32(image_seq);
 	hdr->ec = cpu_to_be64(ec);
 	crc = crc32(UBI_CRC32_INIT, hdr, UBI_EC_HDR_SIZE_CRC);
 	hdr->hdr_crc = cpu_to_be32(crc);
@@ -438,7 +440,8 @@  static int mark_bad(const struct mtd_dev_info *mtd, struct ubi_scan_info *si, in
 	return consecutive_bad_check(eb);
 }
 
-static int flash_image(const struct mtd_dev_info *mtd, struct ubi_scan_info *si)
+static int flash_image(const struct mtd_dev_info *mtd, const struct ubigen_info *ui,
+		       struct ubi_scan_info *si)
 {
 	int fd, img_ebs, eb, written_ebs = 0, divisor;
 	off_t st_size;
@@ -518,7 +521,7 @@  static int flash_image(const struct mtd_dev_info *mtd, struct ubi_scan_info *si)
 			fflush(stdout);
 		}
 
-		err = change_ec((struct ubi_ec_hdr *)buf, ec);
+		err = change_ech((struct ubi_ec_hdr *)buf, ui->image_seq, ec);
 		if (err) {
 			errmsg("bad EC header at eraseblock %d of \"%s\"",
 			       written_ebs, args.image);
@@ -918,7 +921,7 @@  int main(int argc, char * const argv[])
 	}
 
 	if (args.image) {
-		err = flash_image(&mtd, si);
+		err = flash_image(&mtd, &ui, si);
 		if (err < 0)
 			goto out_free;
 
-- 
1.6.2.5