diff mbox series

ubifs: read bad node type in ubifs_tnc_read_wbuf

Message ID c710c620d24c4a418e9def23fd0ed2d6@shmbx04.spreadtrum.com
State RFC
Delegated to: Richard Weinberger
Headers show
Series ubifs: read bad node type in ubifs_tnc_read_wbuf | expand

Commit Message

李傲傲 (Carson Li1/9542) Jan. 10, 2020, 9:10 a.m. UTC 
Hi mtd-list:
I have a problem recently on ubifs: kernel reports to read a bad node type
in ubifs_tnc_read_wbuf.

I guess the problem occurs because of tnc_mutex is freed in ubifs_tnc_locate.
Though it marks the journal heads will not be GCed. But if there is a commit
just finished, LEBs in journal heads will not be in bud rbtree anymore. And
it might be GCed. Here is the kernel log:

------------------------------------------------kernel log ------------------------------------
[   30.072901] c0 UBIFS error (ubi0:23 pid 944): ubifs_read_node_wbuf: bad node type (232 but expected 0)
[   30.085890] c1 UBIFS error (ubi0:23 pid 944): ubifs_read_node_wbuf: wbuf=cd008f40,buf=d4cd9000,
wbuf->buf=c90ec000,offs=164320, len=160, rlen=0
[   30.109436] c1 UBIFS error (ubi0:23 pid 944): ubifs_read_node_wbuf: wxf bad node at LEB 139:164320
[   30.121841] c0 Not a node, first 24 bytes:
[   30.125697] c1 00000000: 65 72 2e 78 6d 6c 2e 62 61 6b 00 b0 4d 14 c0 20 31 18 10 06 e8 3a 41 27
[   30.141302] c1 Kernel panic - not syncing: ch address is d4cd9000
[   30.152510] c1 CPU: 1 PID: 944 Comm: http-thread Tainted: G           O    4.4.83 #2
[   30.152522] c1 Hardware name: Generic DT based system
[   30.152557] c1 [<c001624c>] (unwind_backtrace) from [<c0012f8c>] (show_stack+0x10/0x14)
[   30.152574] c1 [<c0012f8c>] (show_stack) from [<c0287188>] (dump_stack+0x80/0xa0)
[   30.152590] c1 [<c0287188>] (dump_stack) from [<c00acff0>] (panic+0x70/0x1e4)
[   30.152609] c1 [<c00acff0>] (panic) from [<c01e3274>] (ubifs_dump_node+0x6c/0x9a0)
[   30.152626] c1 [<c01e3274>] (ubifs_dump_node) from [<c01c3200>] (ubifs_read_node_wbuf+0x350/0x384)
[   30.152641] c1 [<c01c3200>] (ubifs_read_node_wbuf) from [<c01e0d28>] (ubifs_tnc_read_node+0x54/0x214)
[   30.152658] c1 [<c01e0d28>] (ubifs_tnc_read_node) from [<c01c65ac>] (ubifs_tnc_locate+0x118/0x1b4)
[   30.152675] c1 [<c01c65ac>] (ubifs_tnc_locate) from [<c01bd2c8>] (ubifs_iget+0xb8/0x68c)
[   30.152691] c1 [<c01bd2c8>] (ubifs_iget) from [<c01b9b90>] (ubifs_lookup+0x1b4/0x258)
[   30.152706] c1 [<c01b9b90>] (ubifs_lookup) from [<c00f8134>] (lookup_real+0x30/0x4c)
[   30.152720] c1 [<c00f8134>] (lookup_real) from [<c00f8e68>] (__lookup_hash+0x34/0x3c)
[   30.152732] c1 [<c00f8e68>] (__lookup_hash) from [<c00f8f5c>] (walk_component+0xec/0x2a0)
[   30.152745] c1 [<c00f8f5c>] (walk_component) from [<c00fa600>] (path_lookupat+0x80/0xfc)
[   30.152761] c1 [<c00fa600>] (path_lookupat) from [<c00fd358>] (filename_lookup+0x5c/0xfc)
[   30.152776] c1 [<c00fd358>] (filename_lookup) from [<c00ef114>] (SyS_faccessat+0xa0/0x1c0)
[   30.152792] c1 [<c00ef114>] (SyS_faccessat) from [<c000f8c0>] (ret_fast_syscall+0x0/0x34)
------------------------------------------------kernel log ------------------------------------
And here is my analysis:
--------------------------zbranch passed to ubifs_tnc_read_node -------------------
crash_arm> ubifs_zbranch  c612bd90 struct ubifs_zbranch {
  key = {
    u8 = "]\003\000\000\000\000\000",
    u32 = {861, 0},
    u64 = {861},
    j32 = {861, 0}
  },
  {
    znode = 0x0,
    leaf = 0x0
  },
  lnum = 139,                               //jhead[DATAHD]
  offs = 164320,
  len = 160
}
--------------------------zbranch matches the key {861,0} in TNC-------------------
crash_arm> ubifs_zbranch 0xc8fc2230 struct ubifs_zbranch {
  key = {
    u8 = "]\003\000\000\000\000\000",
    u32 = {861, 0},
    u64 = {861},
    j32 = {861, 0}
  },
  {
    znode = 0x0,
    leaf = 0x0
  },
  lnum = 195,                            //jhead[GCHD]
  offs = 246080,
  len = 160
}
The zbr->lnum if different, and I found that the lnum = 139 is gced_lnum and also the
jhead[DATAHD].wbuf.lnum. The lnum = 195 is jhead[GCHD].wbuf.lnum.

crash_arm> ubifs_jhead.wbuf.lnum  0xcd008e00
  wbuf.lnum = 195,               //jhead[GCHD]
crash_arm> ubifs_jhead.wbuf.lnum  0xcd008ea0
  wbuf.lnum = 279,               //jhead[BASEHD]
crash_arm> ubifs_jhead.wbuf.lnum  0xcd008f40
  wbuf.lnum = 139,               //jhead[DATAHD]

so it can somehow prove my guess that when tnc_mutex is dropped, the zbr->lnum = 139
is GCed and taken to be the leb used by jhead[DATAHD]. In the GC progress, the nodes
lay in the GCing leb are moved to jhead[GCHD].wbuf, at the same time the nodes in the
TNC will be uptodated to point to the jhead[GCHD].wbuf.lnum = 195.

All progress above happened while ubifs_get_wbuf returns true in ubifs_tnc_locate.
So I doubt that marks saying “We do not GC journal heads” and delete some codes as a
temporary solution for this problem. This deletion is just take every leb found
as the same to execute fallible_read_node.

------------------------------------temporary solution--------------------------------

after the modification, the LEB contains node will not be GCed since even though
there is a commit, the wbuf.lnum is still on the bud rbtree as a journal head leb.

I am not sure if I have made things right, and it is a problem with low probability.
so I haven’t applied the modification yet. could you please kindly check for me
if I am on the wrong direction? If so, is there any relevant information for me to
fix that?

Thanks
Carson.li
diff mbox series

Patch

--- a/fs/ubifs/tnc.c
+++ b/fs/ubifs/tnc.c
@@ -1482,12 +1482,6 @@  again:
 gc_seq1 = c->gc_seq;
 mutex_unlock(&c->tnc_mutex);

-if (ubifs_get_wbuf(c, zbr.lnum)) {
-/* We do not GC journal heads */
-err = ubifs_tnc_read_node(c, &zbr, node);
-return err;
-}
-
Actually, compared to that solution described above, I more suggest to modify the
ubifs_get_wbuf.ubifs_get_wbuf is to check if the LEB is on the jhead, but
ubifs_tnc_read_wbuf only read node from wbuf when the lnum is equal to wbuf.lnum
and the others still need to read on flash. It seems to be better to just make
ubifs_get_wbuf to check if the LEB is equal to the wbufs.lnum, and then there is
no need to have a double check in ubifs_tnc_read_wbuf.
-----------------------------more suggested but not tested solution---------------------
--- a/fs/ubifs/log.c
+++ b/fs/ubifs/log.c
@@ -70,28 +70,16 @@  struct ubifs_bud *ubifs_search_bud(struct ubifs_info *c, int lnum)
  */
 struct ubifs_wbuf *ubifs_get_wbuf(struct ubifs_info *c, int lnum)  {
-struct rb_node *p;
-struct ubifs_bud *bud;
 int jhead;

 if (!c->jheads)
 return NULL;

-spin_lock(&c->buds_lock);
-p = c->buds.rb_node;
-while (p) {
-bud = rb_entry(p, struct ubifs_bud, rb);
-if (lnum < bud->lnum)
-p = p->rb_left;
-else if (lnum > bud->lnum)
-p = p->rb_right;
-else {
-jhead = bud->jhead;
-spin_unlock(&c->buds_lock);
+for(jhead = 0; jhead < c->jhead_cnt; jhead++){
+if(lnum == c->jheads[jhead].wbuf.lnum)
 return &c->jheads[jhead].wbuf;
-}
 }
-spin_unlock(&c->buds_lock);
+
 return NULL;
 }

--- a/fs/ubifs/io.c
+++ b/fs/ubifs/io.c
@@ -906,9 +906,10 @@  int ubifs_read_node_wbuf(struct ubifs_wbuf *wbuf, void *buf, int type, int len,
 ubifs_assert(wbuf && lnum >= 0 && lnum < c->leb_cnt && offs >= 0);
 ubifs_assert(!(offs & 7) && offs < c->leb_size);
 ubifs_assert(type >= 0 && type < UBIFS_NODE_TYPES_CNT);
+ubifs_assert(wbuf->lnum == lnum);

 spin_lock(&wbuf->lock);
-overlap = (lnum == wbuf->lnum && offs + len > wbuf->offs);
+overlap = (offs + len > wbuf->offs);
 if (!overlap) {
 /* We may safely unlock the write-buffer and read the data */
 spin_unlock(&wbuf->lock);