Message ID | 4990CB6A.1020403@cn.fujitsu.com |
---|---|
State | Rejected, archived |
Headers | show |
On Tue, Feb 10, 2009 at 08:33:46AM +0800, Gui Xiaohua wrote: > When mkfs.ext4 with flex_bg option, resize2fs with -M option will fail. > > Steps to reproduce: > > # mkfs.ext4 -O flex_bg /dev/sda6 > # resize2fs /dev/sda6 -M > > resize2fs: No space left on device while trying to resize /dev/sda6 These problems were fixed in e2fsprogs 1.41.4. See especially commit id's 793a04a0 and 9227c5bb. (There were a number of other resize2fs bugs fixed in 1.41.4, but these two relate to the problems you were trying to solve.) I'm not convinced your attempt at trying to guesstimate how the number of extra blocks needed by counting fragments works for anything other than an empty filesystem. If the filesystem is filled with contents, and then it is shrunk, the resulting number of new extent blocks which are needed is very much dependent on how the blocks end up getting moved, and trying to determine the exact number up front is extremely difficult. Regards, - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Theodore Tso 写道: > On Tue, Feb 10, 2009 at 08:33:46AM +0800, Gui Xiaohua wrote: >> When mkfs.ext4 with flex_bg option, resize2fs with -M option will fail. >> >> Steps to reproduce: >> >> # mkfs.ext4 -O flex_bg /dev/sda6 >> # resize2fs /dev/sda6 -M >> >> resize2fs: No space left on device while trying to resize /dev/sda6 > > These problems were fixed in e2fsprogs 1.41.4. See especially commit > id's 793a04a0 and 9227c5bb. (There were a number of other resize2fs > bugs fixed in 1.41.4, but these two relate to the problems you were > trying to solve.) > > I'm not convinced your attempt at trying to guesstimate how the number > of extra blocks needed by counting fragments works for anything other > than an empty filesystem. If the filesystem is filled with contents, > and then it is shrunk, the resulting number of new extent blocks which > are needed is very much dependent on how the blocks end up getting > moved, and trying to determine the exact number up front is extremely > difficult. > > Regards, > > - Ted Hi Ted I have tested your patch,but occurs some messages like below: resize2fs: Illegal triply indirect block found while trying to resize /dev/sda7 Steps to reproduce: # mkfs.ext4 -O flex_bg /dev/sda7 # mount -t ext4 /dev/sda7 /mnt # dumpe2fs /dev/sda7 dumpe2fs 1.41.4 (27-Jan-2009) Filesystem volume name: <none> Last mounted on: <not available> Filesystem UUID: 56f80bde-6a27-4564-87cf-a7186d05e86b Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent flex_bg sparse_super huge_file uninit_bg dir_nlink extra_isize Filesystem flags: signed_directory_hash Default mount options: (none) Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 4016 Block count: 16032 Reserved block count: 801 Free blocks: 14359 Free inodes: 4005 First block: 1 Block size: 1024 Fragment size: 1024 Reserved GDT blocks: 62 Blocks per group: 8192 Fragments per group: 8192 Inodes per group: 2008 Inode blocks per group: 251 Flex block group size: 16 Filesystem created: Tue Feb 3 22:06:30 2009 Last mount time: n/a Last write time: Tue Feb 3 22:06:30 2009 Mount count: 0 Maximum mount count: 27 Last checked: Tue Feb 3 22:06:30 2009 Check interval: 15552000 (6 months) Next check after: Sun Aug 2 22:06:30 2009 Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 128 Journal inode: 8 Default directory hash: half_md4 Directory Hash Seed: 397bda27-6808-47a9-9dc1-e736908e4a05 Journal backup: inode blocks Journal size: 1024k Group 0: (Blocks 1-8192) [ITABLE_ZEROED] Checksum 0x3698, unused inodes 1997 Primary superblock at 1, Group descriptors at 2-2 Reserved GDT blocks at 3-64 Block bitmap at 65 (+64), Inode bitmap at 81 (+80) Inode table at 97-347 (+96) 7608 free blocks, 1997 free inodes, 2 directories, 1997 unused inodes Free blocks: 80, 83-96, 600-8192 Free inodes: 12-2008 Group 1: (Blocks 8193-16031) [INODE_UNINIT, ITABLE_ZEROED] Checksum 0x88a4, unused inodes 2008 Backup superblock at 8193, Group descriptors at 8194-8194 Reserved GDT blocks at 8195-8256 Block bitmap at 66, Inode bitmap at 82 Inode table at 348-598 6751 free blocks, 2008 free inodes, 0 directories, 2008 unused inodes Free blocks: 9281-16031 Free inodes: # created many files with size 1k <----the sum of files is greater than inode per group, such as 2500 # rm -f files <----the sum is 2000,so the second group will be cut while resize # umount /dev/sda7 # e2fsck -f /dev/sda7 [root@localhost /]# resize2fs /dev/sda7 -M resize2fs 1.41.4 (27-Jan-2009) Resizing the filesystem on /dev/sda7 to 1940 (1k) blocks. resize2fs: Illegal triply indirect block found while trying to resize /dev/sda7 > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Feb 10, 2009 at 11:32:22AM +0800, Gui Xiaohua wrote: > Hi Ted > I have tested your patch,but occurs some messages like below: > resize2fs: Illegal triply indirect block found while trying to resize /dev/sda7 > Hi, Could you be a bit more precise with your reproduction instructions? I've tried this, and it works for me. Stupid question --- you did install e2fsprogs 1.41.4 before you tried this, right? If you built with shared libraries, and ended up using 1.41.3's shared libraries with the 1.41.4 binaries, this might cause this problem, since a number of the bug fixes which I applied were in the libext2fs library. - Ted <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] 513# dd if=/dev/zero of=/var/tmp/resize-test.img bs=4k count=16032 16032+0 records in 16032+0 records out 65667072 bytes (66 MB) copied, 1.08768 s, 60.4 MB/s <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] 514# mke2fs -t ext4 -O flex_bg /var/tmp/resize-test.img mke2fs 1.41.4 (27-Jan-2009) /var/tmp/resize-test.img is not a block special device. Proceed anyway? (y,n) y Filesystem label= OS type: Linux Block size=1024 (log=0) Fragment size=1024 (log=0) 16064 inodes, 64128 blocks 3206 blocks (5.00%) reserved for the super user First data block=1 Maximum filesystem blocks=65798144 8 block groups 8192 blocks per group, 8192 fragments per group 2008 inodes per group Superblock backups stored on blocks: 8193, 24577, 40961, 57345 Writing inode tables: done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 39 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] 515# mount -o loop -t ext4 /var/tmp/resize-test.img /mnt <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] 516# (cd /mnt; mkdir -p foo; cd foo; seq 1 2500 | xargs touch) <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] 517# umount /mnt <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] 518# e2fsck -f /var/tmp/resize-test.img e2fsck 1.41.4 (27-Jan-2009) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information /var/tmp/resize-test.img: 2512/16064 files (0.1% non-contiguous), 7441/64128 blocks <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] 519# resize2fs -M /var/tmp/resize-test.img resize2fs 1.41.4 (27-Jan-2009) Resizing the filesystem on /var/tmp/foo.img to 8857 (1k) blocks. The filesystem on /var/tmp/foo.img is now 8857 blocks long. <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] 520# -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Theodore Tso ??: > On Tue, Feb 10, 2009 at 11:32:22AM +0800, Gui Xiaohua wrote: >> Hi Ted >> I have tested your patch,but occurs some messages like below: >> resize2fs: Illegal triply indirect block found while trying to resize /dev/sda7 >> > > Hi, Could you be a bit more precise with your reproduction > instructions? I've tried this, and it works for me. Stupid question > --- you did install e2fsprogs 1.41.4 before you tried this, right? If Yes,you are right.Then make uninstall e2fsprogs 1.41.3 and recompile e2fsprogs 1.41.4, resize succuess, but some messages occurs after do e2fsck > you built with shared libraries, and ended up using 1.41.3's shared > libraries with the 1.41.4 binaries, this might cause this problem, > since a number of the bug fixes which I applied were in the libext2fs > library. > > - Ted > > <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] > 513# dd if=/dev/zero of=/var/tmp/resize-test.img bs=4k count=16032 > 16032+0 records in > 16032+0 records out > 65667072 bytes (66 MB) copied, 1.08768 s, 60.4 MB/s > <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] > 514# mke2fs -t ext4 -O flex_bg /var/tmp/resize-test.img > mke2fs 1.41.4 (27-Jan-2009) > /var/tmp/resize-test.img is not a block special device. > Proceed anyway? (y,n) y > Filesystem label= > OS type: Linux > Block size=1024 (log=0) > Fragment size=1024 (log=0) > 16064 inodes, 64128 blocks > 3206 blocks (5.00%) reserved for the super user > First data block=1 > Maximum filesystem blocks=65798144 > 8 block groups > 8192 blocks per group, 8192 fragments per group > 2008 inodes per group > Superblock backups stored on blocks: > 8193, 24577, 40961, 57345 > > Writing inode tables: done > Creating journal (4096 blocks): done > Writing superblocks and filesystem accounting information: done > > This filesystem will be automatically checked every 39 mounts or > 180 days, whichever comes first. Use tune2fs -c or -i to override. > <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] > 515# mount -o loop -t ext4 /var/tmp/resize-test.img /mnt > <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] > 516# (cd /mnt; mkdir -p foo; cd foo; seq 1 2500 | xargs touch) > <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] After this step you weren't remove 2000 files > 517# umount /mnt > <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] > 518# e2fsck -f /var/tmp/resize-test.img > e2fsck 1.41.4 (27-Jan-2009) > Pass 1: Checking inodes, blocks, and sizes > Pass 2: Checking directory structure > Pass 3: Checking directory connectivity > Pass 4: Checking reference counts > Pass 5: Checking group summary information > /var/tmp/resize-test.img: 2512/16064 files (0.1% non-contiguous), 7441/64128 blocks > <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] > 519# resize2fs -M /var/tmp/resize-test.img > resize2fs 1.41.4 (27-Jan-2009) > Resizing the filesystem on /var/tmp/foo.img to 8857 (1k) blocks. > The filesystem on /var/tmp/foo.img is now 8857 blocks long. > > <tytso.root@mini-me> {/home/tytso/e2fsprogs/build}, level 2 [master] > 520# Do e2fsck -f /var/tmp/resize-test.img there are many problems > > > -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
--- e2fsprogs-1.41.3/resize/resize2fs.c 2008-10-07 22:22:39.000000000 +0800 +++ e2fsprogs-1.41.3_org/resize/resize2fs.c 2009-02-03 01:56:24.000000000 +0800 @@ -49,6 +49,10 @@ static errcode_t inode_ref_fix(ext2_resi static errcode_t move_itables(ext2_resize_t rfs); static errcode_t fix_resize_inode(ext2_filsys fs); static errcode_t ext2fs_calculate_summary_stats(ext2_filsys fs); +static int calculate_extent(ext2_filsys fs); +static blk_t count_map(ext2fs_block_bitmap bitmap, blk_t begin, blk_t end); +static void adjust_flex_group(ext2_filsys old_fs, ext2_filsys new_fs); + /* * Some helper CPP macros @@ -112,6 +116,12 @@ errcode_t resize_fs(ext2_filsys fs, blk_ *new_size = rfs->new_fs->super->s_blocks_count; + /*umark meta-data of would be removed groups*/ + if (fs->group_desc_count > rfs->new_fs->group_desc_count && + fs->super->s_log_groups_per_flex) { + adjust_flex_group(rfs->old_fs, rfs->new_fs); + } + retval = blocks_to_move(rfs); if (retval) goto errout; @@ -1792,7 +1802,7 @@ blk_t calculate_minimum_resize_size(ext2 { blk_t inode_count, blks_needed, groups, data_blocks; blk_t grp, data_needed, last_start; - int overhead = 0, num_of_superblocks = 0; + int overhead = 0, num_of_superblocks = 0, extent = 0, i = 0; /* * first figure out how many group descriptors we need to @@ -1918,5 +1928,110 @@ blk_t calculate_minimum_resize_size(ext2 blks_needed = (groups-1) * EXT2_BLOCKS_PER_GROUP(fs->super); blks_needed += overhead; + /*figure out how many fragment will be produced*/ + if (fs->group_desc_count > groups && fs->super->s_log_groups_per_flex) { + extent = calculate_extent(fs); + i = (sizeof(__u32) * EXT2_N_BLOCKS - + sizeof(struct ext3_extent_header))/sizeof(struct ext3_extent); + if (extent >= i) + blks_needed += 1; + } + return blks_needed; } + +static int calculate_extent(ext2_filsys fs) +{ + blk_t start_blk, end_blk, range; + dgrp_t group_count; + blk_t map = 0; + int extent = 0; + + int flex_size = 1 << fs->super->s_log_groups_per_flex; + group_count = fs->group_desc_count; + + ext2fs_read_bitmaps(fs); + fix_uninit_block_bitmaps(fs); + + /*block_map*/ + start_blk = fs->group_desc[group_count-1].bg_block_bitmap; + end_blk = fs->group_desc[0].bg_block_bitmap + flex_size; + range = end_blk - start_blk; + map = count_map(fs->block_map, start_blk, end_blk); + if (map == range || map == 1) + extent += 1; + else + extent += 2; + + /*inode_map*/ + start_blk = fs->group_desc[group_count-1].bg_inode_bitmap; + end_blk = fs->group_desc[0].bg_inode_bitmap + flex_size; + range = end_blk - start_blk; + map = count_map(fs->block_map, start_blk, end_blk); + if (map == range || map == 1) + extent += 1; + else + extent += 2; + + /*inode_table*/ + start_blk = fs->group_desc[group_count-1].bg_inode_table + + fs->inode_blocks_per_group; + end_blk = start_blk + 1; + map = count_map(fs->block_map, start_blk, end_blk); + if (map) + extent += 1; + + return extent; +} + +static blk_t count_map(ext2fs_block_bitmap bitmap, blk_t begin, blk_t end) +{ + blk_t map = 0; + blk_t i = 0; + + for (i = begin; i < end; i++) { + if (ext2fs_test_block_bitmap(bitmap, i)) + map++; + } + return map; +} + +static void adjust_flex_group(ext2_filsys old_fs, ext2_filsys new_fs) +{ + blk_t b; + unsigned int j; + unsigned int start, end; + dgrp_t i = new_fs->group_desc_count; + ext2fs_block_bitmap new_bitmap = new_fs->block_map; + ext2fs_block_bitmap old_bitmap = old_fs->block_map; + + start = ext2fs_get_block_bitmap_start(new_bitmap); + end = ext2fs_get_block_bitmap_end(new_bitmap); + + for (; i < old_fs->group_desc_count; i++) { + for (j = 0, b = old_fs->group_desc[i].bg_inode_table; + j < (unsigned int) old_fs->inode_blocks_per_group; + j++, b++) { + ext2fs_unmark_block_bitmap(old_bitmap, b); + + if (start <= b && end >= b) + ext2fs_unmark_block_bitmap(new_bitmap, b); + } + + ext2fs_unmark_block_bitmap(old_bitmap, + old_fs->group_desc[i].bg_block_bitmap); + ext2fs_unmark_block_bitmap(old_bitmap, + old_fs->group_desc[i].bg_inode_bitmap); + if ((start <= old_fs->group_desc[i].bg_inode_bitmap) && + (end >= old_fs->group_desc[i].bg_inode_bitmap)) { + ext2fs_unmark_block_bitmap(new_bitmap, + old_fs->group_desc[i].bg_inode_bitmap); + } + if ((start <= old_fs->group_desc[i].bg_block_bitmap) && + (end >= old_fs->group_desc[i].bg_block_bitmap)) { + ext2fs_unmark_block_bitmap(new_bitmap, + old_fs->group_desc[i].bg_block_bitmap); + } + } +} +
When mkfs.ext4 with flex_bg option, resize2fs with -M option will fail. Steps to reproduce: # mkfs.ext4 -O flex_bg /dev/sda6 # resize2fs /dev/sda6 -M resize2fs: No space left on device while trying to resize /dev/sda6 I found that the group which hand been moved out its meta-data still in the orignally block after resize2fs,but its meta-data was invalidation. I try to fix it and the patch likes below. Signed-off-by: Gui Xiaohua <guixh@cn.fujitsu.com> -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html