From patchwork Tue Aug 20 14:19:57 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Mats_K=C3=A4rrman?= X-Patchwork-Id: 268555 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from casper.infradead.org (unknown [IPv6:2001:770:15f::2]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 4FBFD2C0113 for ; Wed, 21 Aug 2013 00:21:08 +1000 (EST) Received: from merlin.infradead.org ([2001:4978:20e::2]) by casper.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1VBmng-0004Am-At; Tue, 20 Aug 2013 14:20:48 +0000 Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1VBmnY-0005o1-Jt; Tue, 20 Aug 2013 14:20:40 +0000 Received: from tritech.se ([46.59.120.190]) by merlin.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1VBmnS-0005mR-Ho for linux-mtd@lists.infradead.org; Tue, 20 Aug 2013 14:20:38 +0000 Received: from post.tritech.se (post.tritech.se [10.75.60.100]) by tritech.se (8.14.3/8.14.3) with ESMTP id r7KEJw2m020022; Tue, 20 Aug 2013 16:20:03 +0200 Received: from POST.tritech.se ([fe80::b4fc:968e:bbef:54cb]) by post.tritech.se ([fe80::b4fc:968e:bbef:54cb%10]) with mapi id 14.02.0342.003; Tue, 20 Aug 2013 16:19:58 +0200 From: =?iso-8859-1?Q?Mats_K=E4rrman?= To: "linux-mtd@lists.infradead.org" Subject: ubifs: Mean crash when dumping orphan node Thread-Topic: ubifs: Mean crash when dumping orphan node Thread-Index: Ac6dsFkkY2UBBOhRT0OmCAz+tH6XwA== Date: Tue, 20 Aug 2013 14:19:57 +0000 Message-ID: Accept-Language: en-US, sv-SE Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [194.132.137.82] MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20130820_102035_148476_BD36F5CB X-CRM114-Status: GOOD ( 16.13 ) X-Spam-Score: -4.7 (----) X-Spam-Report: SpamAssassin version 3.3.2 on merlin.infradead.org summary: Content analysis details: (-4.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record -2.8 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] Cc: Artem Bityutskiy X-BeenThere: linux-mtd@lists.infradead.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux MTD discussion mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-mtd" Errors-To: linux-mtd-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org Problem: Found when running integck with power-cut emulation. An orphan node ended up with a corrupted (huge) "len" field. The error is detected by ubifs_check_node() that calls ubifs_dump_node(). This in turn does not check the len and tries to dump from outside the leb buffer. --> Kernel oops. --> note: flush-ubifs_1_0[2143] exited with preempt_count 1 --> ubifs_bgt1_0 stuck in uninterruptible sleep. Solution: ubifs_dump_node() doesn't have access to leb number or offset, also it is called from many places so adding those as arguments is not pleasing. The following ugly patch limits the buffer len field in ubifs_check_node() in case the len is out of range. Perhaps someone see a better alternative? ----------------------- Console log --------------------------- UBI: attaching mtd1 to ubi1 UBI: physical eraseblock size: 131072 bytes (128 KiB) UBI: logical eraseblock size: 130944 bytes UBI: smallest flash I/O unit: 1 UBI: VID header offset: 64 (aligned 64) UBI: data offset: 128 UBI: max. sequence number: 511497 UBI: attached mtd1 to ubi1 UBI: MTD device name: "integ_chk" UBI: MTD device size: 11 MiB UBI: number of good PEBs: 95 UBI: number of bad PEBs: 0 UBI: number of corrupted PEBs: 0 UBI: max. allowed volumes: 128 UBI: wear-leveling threshold: 4096 UBI: number of internal volumes: 1 UBI: number of user volumes: 1 UBI: available PEBs: 0 UBI: total number of reserved PEBs: 95 UBI: number of PEBs reserved for bad PEB handling: 0 UBI: max/mean erase counter: 30449/29798 UBI: image sequence number: 537117096 UBI: background thread "ubi_bgt1d" started, PID 2141 UBIFS: background thread "ubifs_bgt1_0" started, PID 2142 UBIFS: recovery needed UBIFS: recovery completed UBIFS: mounted UBI device 1, volume 0, name "integ_chk"(null) UBIFS: LEB size: 130944 bytes (127 KiB), min./max. I/O unit sizes: 8 bytes/512 bytes UBIFS: FS size: 10606464 bytes (10 MiB, 81 LEBs), journal size 1047553 bytes (0 MiB, 6 LEBs) UBIFS: reserved for root: 500969 bytes (489 KiB) UBIFS: media format: w4/r0 (latest is w4/r0), UUID 23ADF8B1-A415-4349-9ED5-E8348AFA64BF, small LPT model UBIFS: background thread "ubifs_bgt1_0" stops UBIFS: background thread "ubifs_bgt1_0" started, PID 2148 UBIFS warning (pid 21052): power_cut_emulated: failing after 6062 calls UBIFS warning (pid 2143): power_cut_emulated: failing in GC head LEB 49 UBIFS warning (pid 2143): power_cut_emulated: ========== Power cut emulated ========== Call Trace: [ce54da50] [c000884c] show_stack+0x48/0x168 (unreliable) [ce54da90] [c011927c] power_cut_emulated+0x2d8/0x668 [ce54dab0] [c011980c] dbg_leb_write+0x64/0x1b4 [ce54dae0] [c00f1df8] ubifs_leb_write+0x64/0x124 [ce54db00] [c00f2568] ubifs_wbuf_write_nolock+0x4a0/0xa00 [ce54db50] [c010096c] move_node+0x40/0xb4 [ce54db80] [c0101118] ubifs_garbage_collect_leb+0x608/0x874 [ce54dbe0] [c010150c] ubifs_garbage_collect+0x188/0x704 [ce54dc60] [c00e2c04] make_reservation+0x110/0x64c [ce54dcd0] [c00e59e4] ubifs_jnl_write_data+0x124/0x354 [ce54dd40] [c00e7ae4] do_writepage+0x88/0x1e0 [ce54dd80] [c006226c] __writepage+0x24/0x8c [ce54dd90] [c0062ce4] write_cache_pages+0x200/0x38c [ce54de30] [c00aa380] writeback_single_inode+0xfc/0x364 [ce54de60] [c00aabc0] writeback_sb_inodes+0x160/0x270 [ce54dea0] [c00ab4ec] wb_writeback+0x224/0x2dc [ce54df20] [c00ab648] wb_do_writeback+0xa4/0x1a8 [ce54df70] [c00ab79c] bdi_writeback_task+0x50/0x124 [ce54dfa0] [c0070074] bdi_start_fn+0x8c/0x114 [ce54dfc0] [c003ca3c] kthread+0x78/0x7c [ce54dff0] [c00119a8] kernel_thread+0x4c/0x68 UBIFS warning (pid 2143): corrupt_data: filled bytes 346-511 with random data UBIFS warning (pid 2143): dbg_leb_write: actually write 512 bytes to LEB 49:110592 (the buffer was corrupted) UBIFS error (pid 2143): ubifs_leb_write: writing 512 bytes to LEB 49:110592 failed, error -30 UBIFS warning (pid 2143): ubifs_ro_mode: switched to read-only mode, error -30 Call Trace: [ce54da90] [c000884c] show_stack+0x48/0x168 (unreliable) [ce54dad0] [c00f0ef4] ubifs_ro_mode+0x70/0x74 [ce54dae0] [c00f1e84] ubifs_leb_write+0xf0/0x124 [ce54db00] [c00f2568] ubifs_wbuf_write_nolock+0x4a0/0xa00 [ce54db50] [c010096c] move_node+0x40/0xb4 [ce54db80] [c0101118] ubifs_garbage_collect_leb+0x608/0x874 [ce54dbe0] [c010150c] ubifs_garbage_collect+0x188/0x704 [ce54dc60] [c00e2c04] make_reservation+0x110/0x64c [ce54dcd0] [c00e59e4] ubifs_jnl_write_data+0x124/0x354 [ce54dd40] [c00e7ae4] do_writepage+0x88/0x1e0 [ce54dd80] [c006226c] __writepage+0x24/0x8c [ce54dd90] [c0062ce4] write_cache_pages+0x200/0x38c [ce54de30] [c00aa380] writeback_single_inode+0xfc/0x364 [ce54de60] [c00aabc0] writeback_sb_inodes+0x160/0x270 [ce54dea0] [c00ab4ec] wb_writeback+0x224/0x2dc [ce54df20] [c00ab648] wb_do_writeback+0xa4/0x1a8 [ce54df70] [c00ab79c] bdi_writeback_task+0x50/0x124 [ce54dfa0] [c0070074] bdi_start_fn+0x8c/0x114 [ce54dfc0] [c003ca3c] kthread+0x78/0x7c [ce54dff0] [c00119a8] kernel_thread+0x4c/0x68 Call Trace: [ce54daa0] [c000884c] show_stack+0x48/0x168 (unreliable) [ce54dae0] [c00f1e88] ubifs_leb_write+0xf4/0x124 [ce54db00] [c00f2568] ubifs_wbuf_write_nolock+0x4a0/0xa00 [ce54db50] [c010096c] move_node+0x40/0xb4 [ce54db80] [c0101118] ubifs_garbage_collect_leb+0x608/0x874 [ce54dbe0] [c010150c] ubifs_garbage_collect+0x188/0x704 [ce54dc60] [c00e2c04] make_reservation+0x110/0x64c [ce54dcd0] [c00e59e4] ubifs_jnl_write_data+0x124/0x354 [ce54dd40] [c00e7ae4] do_writepage+0x88/0x1e0 [ce54dd80] [c006226c] __writepage+0x24/0x8c [ce54dd90] [c0062ce4] write_cache_pages+0x200/0x38c [ce54de30] [c00aa380] writeback_single_inode+0xfc/0x364 [ce54de60] [c00aabc0] writeback_sb_inodes+0x160/0x270 [ce54dea0] [c00ab4ec] wb_writeback+0x224/0x2dc [ce54df20] [c00ab648] wb_do_writeback+0xa4/0x1a8 [ce54df70] [c00ab79c] bdi_writeback_task+0x50/0x124 [ce54dfa0] [c0070074] bdi_start_fn+0x8c/0x114 [ce54dfc0] [c003ca3c] kthread+0x78/0x7c [ce54dff0] [c00119a8] kernel_thread+0x4c/0x68 UBIFS error (pid 2143): ubifs_wbuf_write_nolock: cannot write 63 bytes to LEB 49:110592, error -30 magic 0x6101831 crc 0x5e5c8e6c node_type 2 (direntry node) group_type 1 (in node group) sqnum 27165524 len 63 key (1019352, direntry, 0x258e374) inum 1019371 type 1 nlen 6 name 522879 Call Trace: [ce54dac0] [c000884c] show_stack+0x48/0x168 (unreliable) [ce54db00] [c00f24b0] ubifs_wbuf_write_nolock+0x3e8/0xa00 [ce54db50] [c010096c] move_node+0x40/0xb4 [ce54db80] [c0101118] ubifs_garbage_collect_leb+0x608/0x874 [ce54dbe0] [c010150c] ubifs_garbage_collect+0x188/0x704 [ce54dc60] [c00e2c04] make_reservation+0x110/0x64c [ce54dcd0] [c00e59e4] ubifs_jnl_write_data+0x124/0x354 [ce54dd40] [c00e7ae4] do_writepage+0x88/0x1e0 [ce54dd80] [c006226c] __writepage+0x24/0x8c [ce54dd90] [c0062ce4] write_cache_pages+0x200/0x38c [ce54de30] [c00aa380] writeback_single_inode+0xfc/0x364 [ce54de60] [c00aabc0] writeback_sb_inodes+0x160/0x270 [ce54dea0] [c00ab4ec] wb_writeback+0x224/0x2dc [ce54df20] [c00ab648] wb_do_writeback+0xa4/0x1a8 [ce54df70] [c00ab79c] bdi_writeback_task+0x50/0x124 [ce54dfa0] [c0070074] bdi_start_fn+0x8c/0x114 [ce54dfc0] [c003ca3c] kthread+0x78/0x7c [ce54dff0] [c00119a8] kernel_thread+0x4c/0x68 (pid 2143) start dumping LEB 49 UBIFS error (pid 2143): ubifs_check_node: bad node length 673382463 UBIFS error (pid 2143): ubifs_check_node: bad node at LEB 49:110920 magic 0x6101831 crc 0xa416d062 node_type 11 (orphan node) group_type 64 (unknown) sqnum 27165548 len 673382463 commit number 65620158181089514 last node flag 0 84172803 orphan inode numbers: ino 3562129533828147616 ino 8117303588418149870 ino 4640262820834542106 ino 6726456995349585035 ino 14489074808184778633 ino 13946985241486444853 ino 10749874589530534200 ino 6464459056807361480 ino 6626419564033690030 ino 11788878205927043913 ino 6555818361502561268 ino 17704711455940586508 ino 14549808009372943550 ino 8481426162481423107 ino 10339464532359928517 ino 3645254393624201195 ino 14333515036885505844 ino 13565517142754118993 ino 16678727471166591753 ino 18446744073709551615 ino 18446744073709551615 //snip// ino 18446744073709551615 ino 18446744073709551615 ino 18446744073709551615 ino 9821846592259568493 ino 11102932228019499002 ino 12626009155550599837 ino 2895740556906901342 ino 3426259527302190334 ino 4358966078677474170 ino 10542940169215034251 ino 13900150150557332777 ino 2325828422616975107 ino 6561521274669858056 ino 16499483703353557986 ino 8672796104748479605 ino 11865067532203562597 ino 4607242104969698445 ino 17031714597478032265 ino 11075308794197177645 Unable to handle kernel paging request for data at address 0xd6d92000 Faulting instruction address: 0xc011be34 Oops: Kernel access of bad area, sig: 11 [#1] PREEMPT MPC5125 last sysfs file: /sys/devices/virtual/ubi/ubi1/mtd_num Modules linked in: xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntr ack iptable_filter ip_tables x_tables rfcomm l2cap bluetooth ext2 vfat fat sg sd _mod usb_storage scsi_mod usb_libusual ehci_hcd usbcore NIP: c011be34 LR: c011be6c CTR: c01829f4 REGS: ce54d910 TRAP: 0300 Tainted: G W (2.6.35.14-debug-3) MSR: 00009032 CR: 44002024 XER: 20000000 DAR: d6d92000, DSISR: 20000000 TASK = cf30e7f0[2143] 'flush-ubifs_1_0' THREAD: ce54c000 GPR00: c011be6c ce54d9c0 cf30e7f0 00000022 120f6f8a ffffffff c03265e2 00010000 GPR08: c03269c8 00000001 00000004 120f6f8a 44002022 00000000 c029ec2c c02a6350 GPR16: c0262094 c029ec2c 00100100 00200200 c02a499c c0318c80 c03222b0 c03222a8 GPR24: cf325ca0 00000031 00000031 c02ada68 000009d3 d6d91fe0 ce54c000 05046003 NIP [c011be34] ubifs_dump_node+0x310/0x1700 LR [c011be6c] ubifs_dump_node+0x348/0x1700 Call Trace: [ce54d9c0] [c011be6c] ubifs_dump_node+0x348/0x1700 (unreliable) [ce54da40] [c00f171c] ubifs_check_node+0x1a8/0x2b8 [ce54da60] [c00fad7c] ubifs_scan_a_node+0x1b0/0x388 [ce54da90] [c00faff0] ubifs_scan+0x9c/0x394 [ce54dad0] [c011ea9c] ubifs_dump_leb+0x5c/0x158 [ce54db00] [c00f24bc] ubifs_wbuf_write_nolock+0x3f4/0xa00 [ce54db50] [c010096c] move_node+0x40/0xb4 [ce54db80] [c0101118] ubifs_garbage_collect_leb+0x608/0x874 [ce54dbe0] [c010150c] ubifs_garbage_collect+0x188/0x704 [ce54dc60] [c00e2c04] make_reservation+0x110/0x64c [ce54dcd0] [c00e59e4] ubifs_jnl_write_data+0x124/0x354 [ce54dd40] [c00e7ae4] do_writepage+0x88/0x1e0 [ce54dd80] [c006226c] __writepage+0x24/0x8c [ce54dd90] [c0062ce4] write_cache_pages+0x200/0x38c [ce54de30] [c00aa380] writeback_single_inode+0xfc/0x364 [ce54de60] [c00aabc0] writeback_sb_inodes+0x160/0x270 [ce54dea0] [c00ab4ec] wb_writeback+0x224/0x2dc [ce54df20] [c00ab648] wb_do_writeback+0xa4/0x1a8 [ce54df70] [c00ab79c] bdi_writeback_task+0x50/0x124 [ce54dfa0] [c0070074] bdi_start_fn+0x8c/0x114 [ce54dfc0] [c003ca3c] kthread+0x78/0x7c [ce54dff0] [c00119a8] kernel_thread+0x4c/0x68 Instruction dump: 5009421e 5009c00e 3929ffe0 553fe8fe 7fe4fb78 48134761 2f9f0000 419efe94 3d20c02b 7f9de378 3b69da68 3b800000 <813d0020> 3b9c0001 7f63db78 801d0024 ---[ end trace 47de3350cfddc1f3 ]--- note: flush-ubifs_1_0[2143] exited with preempt_count 1 ^Z[1]+ Stopped time ./integck.2 -eP /sys/kernel/debug/ubifs/ubi1_0/tst_recovery -m 1 /home/root/ubifs root@debug:~# ps PID USER VSZ STAT COMMAND 1 root 1884 S init [5] 2 root 0 SW [kthreadd] 3 root 0 SW [ksoftirqd/0] 4 root 0 SW [events/0] 5 root 0 SW [khelper] 6 root 0 SW [async/mgr] 7 root 0 SW [sync_supers] 8 root 0 SW [bdi-default] 9 root 0 SW [kblockd/0] 10 root 0 SW [kswapd0] 11 root 0 SW [aio/0] 18 root 0 SW [mtdblock0] 19 root 0 SW [mtdblock1] 20 root 0 SW [mtdblock2] 21 root 0 SW [mtdblock3] 22 root 0 SW [mtdblock4] 23 root 0 SW [mtdblock5] 24 root 0 SW [mtdblock6] 25 root 0 SW [mtdblock7] 26 root 0 SW [mtdblock8] 27 root 0 SW [80011200.spi] 28 root 0 DW [80011900.spi] 29 root 0 SW [ubi_bgt0d] 30 root 0 SW [ubifs_bgt0_0] 67 root 0 DW [spi_receive_thr] 81 root 0 SW [khubd] 93 root 2468 S < /sbin/udevd -d 671 root 0 SW [l2cap] 672 root 0 SW< [krfcommd] 772 messageb 2912 S /usr/bin/dbus-daemon --system 857 root 3764 S /bin/login -- 959 root 3488 S -sh 1076 root 3292 S /sbin/syslogd -n -C64 -m 20 2141 root 0 SW [ubi_bgt1d] 2148 root 0 DW [ubifs_bgt1_0] 10938 root 2464 S < /sbin/udevd -d 19488 root 2464 S < /sbin/udevd -d 21051 root 3096 T time ./integck.2 -eP /sys/kernel/debug/ubifs/ubi1_0/ 21052 root 2060 D ./integck.2 -eP /sys/kernel/debug/ubifs/ubi1_0/tst_r 29046 root 3488 R ps diff --git a/fs/ubifs/io.c b/fs/ubifs/io.c index e18b988..0e6b1df 100644 --- a/fs/ubifs/io.c +++ b/fs/ubifs/io.c @@ -261,8 +261,11 @@ int ubifs_check_node(const struct ubifs_info *c, const void *buf, int lnum, } node_len = le32_to_cpu(ch->len); - if (node_len + offs > c->leb_size) + if (node_len + offs > c->leb_size) { + /* Prevent crash when dumping node */ + ch->len = cpu_to_le32(c->leb_size - offs); goto out_len; + } if (c->ranges[type].max_len == 0) { if (node_len != c->ranges[type].len)