e2fsprogs: dumpe2fs manpage updates

I've added documentation for all defined features, including legacy
and work in progress.
* Some pre-existing feature explanations have been expanded based on
ext4.wiki.kernel.org.
* Future features (dirdata, ea_inode and metadata_csum) are commented out

Regards,
Uri Corin
uri.corin@gmail.com
---

+The first e2fsck run after enabling this feature will take the full
time, but subsequent e2fsck runs will take only a fraction of the
original time, depending on how full the file system is.
+SH BUGS
 You need to know the physical filesystem structure to understand the
 output.
 .SH AUTHOR
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 12/17/2013 06:49:49 AM, Uri Corin wrote:

> +.TP
> +.B extra_isize
> +.br
> +This feature enables storage of nanosecond timestamps and creation
> +time, if the inode size is larger than 256 bytes or larger.

this should either say "256 bytes or larger" or "larger than 256 bytes"
(i'm not sure which is true)--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Dec 17, 2013, at 7:49 AM, Uri Corin <uri.corin@gmail.com> wrote:
> I've added documentation for all defined features, including legacy
> and work in progress.
> * Some pre-existing feature explanations have been expanded based on
> ext4.wiki.kernel.org.
> * Future features (dirdata, ea_inode and metadata_csum) are commented out

Thanks for the updates.  It is really helpful for users to have good
descriptions of these features, and man pages are often forgotten when
writing the feature itself.

Is it worthwhile to include a short list after each feature to indicate
which filesystem type (ext2, ext3, ext4) supports each feature?  Would
it be useful to indicate the kernel in which it was added?

> +.B 64bit
> +.br
> +Enables the file system to be larger than 2^32 blocks.  This feature may be set
> +automatically, as needed.

The 64bit feature can also be set on filesystems with < 2^32 blocks,
to prepare the filesystem for resizing beyond 2^32 blocks in the future.
I think Ted updated this description in the mke2fs man page.  (ext4)

> +.B bigalloc
> +.br
> +This feature enables clustered allocation, so that the unit of

I just updated this to be "clustered block allocation" to be more clear.

> +allocation is a power of two number of blocks.  That is, each bit in the
> +what had traditionally been known as the block allocation bitmap now
> +indicates whether a cluster is in use or not, where a cluster is by
> +default composed of 16 blocks.

(ext4)

> +.\" .TP
> +.\" .B dirdata
> +.\" .br
> +.\" Data in directory entry.

FYI, this is to allow 64-bit inode numbers to be stored in the
directory entry, along with Lustre 128-bit File IDentifiers (FID).
(Lustre ldiskfs only)

> +.\" .br
> +.\" Not implemented.
> +.TP
> +.B dir_index
> +.br
> +Filesystem Uses hashed b-trees to speed up lookups in large directories.

(ext3, ext4)

> +.B dir_nlink
> +.br
> +Allows more than 65000 subdirectories per directory.

(ext3, ext4)

> +.TP
> +.B dir_prealloc
> +.br
> +Block pre-allocation for new directories is enabled.

(ext2?)

> +.\" .TP
> +.\" .B ea_inode
> +.\" .br
> +.\" Inodes can be used for large extended attributes.
> +.\" .br
> +.\" Not implemented.

(Lustre ldiskfs only)

> +.TP
> +.B extent
> +.br
> +Instead of using the indirect block scheme for storing the location of
> +data blocks in an inode, use extents instead.  This is a much more
> +efficient encoding which speeds up filesystem access, especially for
> +large files.  (Note: both
> +.B extent
> +and
> +.B extents
> +are accepted as valid names for this feature for
> +historical/backwards compatibility reasons.)

(ext4)

> +.TP
> +.B extra_isize
> +.br
> +This feature enables storage of nanosecond timestamps and creation
> +time, if the inode size is larger than 256 bytes or larger.

Could you please use the updated description I just sent for mke2fs.

> +This feature enables the storage file type information in directory
> +entries.  It is normally enabled by default for all file system types.

(ext4)

> +.TP
> +.B flex_bg
> +.br
> +Allow the per-block group metadata (allocation bitmaps and inode tables)
> +to be placed anywhere on the storage media.  In addition,
> +.B mke2fs
> +will place the per-block group metadata together starting at the first
> +block group of each "flex_bg group".   The size of the flex_bg group

... ?  truncated sentence.

(ext4)

> +.TP
> +.B inline_data
> +.br
> +Allows small files or directories to be stored within the in-inode
> extended attribute area.

(ext4 after 3.13?)

> +.TP
> +.B has_journal
> +.br
> +Uses a journal to ensure filesystem consistency even across unclean shutdowns.

(ext3, ext4)

> +.TP
> +.B huge_file
> +.br
> +Allows files larger than 2TiB in size.

(ext4)

> +.TP
> +.B imagic_inodes
> +.br
> +Legacy feature, not settable by
> +.BR mke2fs (8)
> +and
> +.BR tune2fs (8)
> +.TP
> +.B journal_dev
> +.br
> +This is a journal device
> +(see journal-options in
> +.BR tune2fs (8)
> +or
> +.BR mke2fs (8)).

(ext3, ext4)

> +.TP
> +.B large_dir
> +Supports large directory >2GiB or 3-level htree.

This is for Lustre ldiskfs only so far, so should probably be commented out.

> +.TP
> +.B large_file
> +.br
> +Filesystem can contain files that are greater than 2GiB.  (Modern kernels
> +set this feature automatically when a file > 2GiB is created.)

(ext2, ext3, ext4)

> +.TP
> +.B lazy_bg
> +.br
> +Legacy parameter for testing purposes, no longer supported by kernel.
> +.br
> +Allows an ext2/ext3 developer to create very large filesystems using
> sparse files where most of the block groups are not initialized and so
> do not require much disk space.

Not sure if it is worthwhile to document this in the man page?

> +.TP
> +.B meta_bg
> +.br
> +This feature allows file systems to be resized on-line without explicitly
> +needing to reserve space for growth in the size of the block group
> +descriptors.  This scheme is also used to resize file systems which are
> +larger than 2^32 blocks.  It is not recommended that this feature be set
> +when a file system is created, since this alternate method of storing
> +the block group descriptor will slow down the time needed to mount the
> +file system, and newer kernels can automatically set this feature as
> +necessary when doing an online resize and no more reserved space is
> +available in the resize inode.

(ext4, some support in ext3?)

> +.\" .TP
> +.\" .B metadata_csum
> +.\" .br
> +.\" Filesystem supports metadata checksumming.  This feature enables a
> +.\" superset of the functionality of the
> +.\" .B uninit_bg
> +.\" feature.

Please see my recent patch for mke2fs for an updated description of
this feature.

> +.\" .br
> +.\" .B Future feature, available in e2fsprogs 1.43-WIP

More importantly, only ext4 beyond 3.xx?

> +.TP
> +.B mmp
> +.br
> +Multiple mount protection (MMP) helps to protect the filesystem from
> being multiply mounted and is useful in shared storage environments.
> +.TP
> +.B needs_recovery
> +.br
> +This flag is set by the kernel to indicate either a mounted filesystem with
> +.B has_journal
> +is write enabled or, an unmounted filesystem is in an unclean state.
> +@QUOTA_MAN_COMMENT@.TP
> +@QUOTA_MAN_COMMENT@.B quota
> +@QUOTA_MAN_COMMENT@.br
> +@QUOTA_MAN_COMMENT@Create quota inodes (inode #3 for userquota and inode
> +@QUOTA_MAN_COMMENT@#4 for group quota) and set them in the superblock.
> +@QUOTA_MAN_COMMENT@With this feature, the quotas will be enabled
> +@QUOTA_MAN_COMMENT@automatically when the filesystem is mounted.
> +@QUOTA_MAN_COMMENT@.IP
> +@QUOTA_MAN_COMMENT@Causes the quota files (i.e., user.quota and
> +@QUOTA_MAN_COMMENT@group.quota which existed
> +@QUOTA_MAN_COMMENT@in the older quota design) to be hidden inodes.
> +@QUOTA_MAN_COMMENT@.IP
> +@QUOTA_MAN_COMMENT@.B Not defined by default,
> +@QUOTA_MAN_COMMENT@see https://ext4.wiki.kernel.org/index.php/Quota
> +.TP
> +.B replica
> +.br
> +Non-upstream feature.

I think this should be commented out like the other features not in
the upstream kernel.

> +.TP
> +.B resize_inode
> +.br
> +Reserves space so the block group descriptor table may grow in the future.
> +Useful for online resizing using
> +.BR resize2fs .
> +.TP
> +.B snapshot_bitmap
> +.br
> +Legacy parameter, no longer supported by kernel.

Likewise, I'm not sure if it is useful to document this one, since
it was never supported by any mainline kernel.

> +.TP
> +
> +.B sparse_super
> +.br
> +Filesystem has fewer superblock backup copies
> +(saves space on large filesystems).  This feature is normally enabled by
> +default for all file systems.
> +.TP
> +.B uninit_bg
> +.br
> +Not all of the block groups are initialized.
> +This feature allows the kernel to initialize bitmaps and inode tables
> +and keep a high watermark for the unused inodes in a filesystem, to reduce
> +.BR e2fsck (8)
> +time.  The result is that it can speed up filesystem creation time noticeably
> +(if lazy_itable_init is enabled).
> +.IP
> +This first e2fsck run after enabling this feature will take the
> +full time, but subsequent e2fsck runs will take only a fraction of the
> +original time, depending on how full the file system is.
> +It is only supported by the ext4 filesystem in recent Linux kernels.

This last hunk is not accurate - the slow e2fsck is only needed if the
feature is enabled on an existing filesystem, not if it is enabled by
e2fsck.  Also, the uninit_bg feature has been available for all ext4
versions.  Maybe you mix this up with metadata_csum?

> +.B sparse_super
> +.br
> +Limits the number of backup superblocks to save space on large filesystems.

"sparse_super" is already listed above.

> +.TP
> +.B uninit_bg, uninit_groups
> +.br
> +Allows the kernel to initialize bitmaps and inode tables and keep a high
> +watermark for the unused inodes in a filesystem, to reduce
> +.BR e2fsck (8)
> +time.
> +.br
> +The first e2fsck run after enabling this feature will take the full
> time, but subsequent e2fsck runs will take only a fraction of the
> original time, depending on how full the file system is.

"uninit_bg" is also listed twice.

> +SH BUGS
> You need to know the physical filesystem structure to understand the
> output.
> .SH AUTHOR
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


Cheers, Andreas

On Tue, Dec 17, 2013 at 04:49:49PM +0200, Uri Corin wrote:
> I've added documentation for all defined features, including legacy
> and work in progress.
> * Some pre-existing feature explanations have been expanded based on
> ext4.wiki.kernel.org.
> * Future features (dirdata, ea_inode and metadata_csum) are commented out

It doesn't make sense to try to document the file system features in
both the man pags for mke2fs and dumpe2fs.  So what I've done is to
create a man page, ext4.5, which we can use to document file system
features.

						- Ted

.\" -*- nroff -*-
.\" Copyright 1993, 1994, 1995 by Theodore Ts'o.  All Rights Reserved.
.\" This file may be copied under the terms of the GNU Public License.
.\"
.TH EXT4 5 "@E2FSPROGS_MONTH@ @E2FSPROGS_YEAR@" "E2fsprogs version @E2FSPROGS_VERSION@"
.SH NAME
ext2 \- the second extended file system
.br
ext2 \- the third extended file system
.br
ext4 \- the fourth extended file system
.SH DESCRIPTION
The second, third, and fourth extended file systems, or ext2, ext3, and
ext4 as they are commonly known, are Linux file systems that have
historically been the default file system for many Linux distributions.
They are general purpose file systems that have been designed for
extensibility and backwards compatibility.  In particular, file systems
previously intended for use with the ext2 and ext3 file systems can be
mounted using the ext4 file system driver, and indeed in many modern
Linux distributions, the ext4 file system driver has been configured
handle mount requests for ext2 and ext3 file systems.
.SH FILE SYSTEM FEATURES
A file system formated for ext2, ext3, or ext4 can be have some
collection of the follow file system feature flags enabled.  Some of
these features are not supported by all implementations of the ext2,
ext3, and ext4 file system drivers, depending on Linux kernel version in
use.  On other operating systems, such as the GNU/HURD or FreeBSD, only
a very restrictive set of file system features may be supported in their
implementations of ext2.
.RS 1.2i
.TP
.B 64bit
.br
Enables the file system to be larger than 2^32 blocks.  This feature is set
automatically, as needed, but it can be useful to specify this feature
explicitly if the file system might need to be resized larger than 2^32
blocks, even if it was smaller than that threshold when it was
originally created.  Note that some older kernels and older versions
of e2fsprogs will not support file systems with this ext4 feature enabled.
.TP
.B bigalloc
.br
This ext4 feature enables clustered block allocation, so that the unit of
allocation is a power of two number of blocks.  That is, each bit in the
what had traditionally been known as the block allocation bitmap now
indicates whether a cluster is in use or not, where a cluster is by
default composed of 16 blocks.  This feature can decrease the time
spent on doing block allocation and brings smaller fragmentation, especially
for large files.  The size can be specified using the
.B \-C option.
.IP
.B Warning:
The bigalloc feature is still under development, and may not be fully
supported with your kernel or may have various bugs.  Please see the web
page http://ext4.wiki.kernel.org/index.php/Bigalloc for details.
May clash with delayed allocation (see
.BR nodelalloc mount option).
.IP
This feature requires that the
.B extent
features be enabled.
.TP
.B dir_index
.br
Use hashed b-trees to speed up name lookups in large directories.  This
feature is supported by ext3 and ext4 file systems, and is ignored by
ext2 file systems.
.TP
.B dir_nlink
.br
This ext4 feature allows more than 65000 subdirectories per directory.
.TP
.B extent
.br
This ext4 feature allows the mapping of logical block numbers for a
particular inode to physical blocks on the storage device to be stored
using an extent tree, which is a more efficient data structure than the
traditional indirect block scheme used by the ext2 and ext3 file
systems.  The use of the extent tree decreases metadata block overhead,
improves file system performance, and decreases the needed to run
.BR e2fsck (8)
on the file system.
(Note: both
.B extent
and
.B extents
are accepted as valid names for this feature for
historical/backwards compatibility reasons.)
.TP
.B extra_isize
.br
This ext4 feature reserves a specific amount of space in each inode for
extended metadata such as nanosecond timestamps and file creation time,
even if the current kernel does not current need to reserve this much
space.  Without this feature, the kernel will reserve the amount of
space for features currently it currently needs, and the rest may be
consumed by extended attributes.

For this feature to be useful the inode size must be 256 bytes in size
or larger.
.TP
.B ext_attr
.br
This feature enables the use of extended attributes.  This feature is
supported by ext2, ext3, and ext4.
.TP
.B filetype
.br
This feature enables the storage file type information in directory
entries.  This feature is supported by ext2, ext3, and ext4.
.TP
.TP
.B flex_bg
.br
This ext4 feature allows the per-block group metadata (allocation
bitmaps
and inode tables)
to be placed anywhere on the storage media.  In addition,
.B mke2fs
will place the per-block group metadata together starting at the first
block group of each "flex_bg group".   The size of the flex_bg group
can be specified using the 
.B \-G
option.
.TP
.B has_journal
.br
Create a journal to ensure filesystem consistency even across unclean
shutdowns.  Setting the filesystem feature is equivalent to using the
.B \-j
option.  This feature is supported by ext3 and ext4, and ignored by the
ext2 file system driver.
.TP
.B huge_file
.br
This ext4 feature allows files to be larger than 2 terabytes in size.
.TP
.B journal_dev
.br
This feature is enabled on the superblock found on an external journal
device.  The block size for the external journal must be the same as the
file system which uses it.
.IP
The external journal device can be used by a file system by specifying
the
.B \-J
.BR device= <external-device>
option to
.BR mke2fs (8)
or
.BR tune2fs(8).
.TP
.B large_file
.br
This feature flag is set automatically by modern kernels when a file
larger than 2 gigabytes is created.  Very old kernels could not
handle large files, so this feature flag was used to prohibit those
kernels from mounting file systems that they could not understand.
.\" .TP
.\" .B metadata_csum
.\" .br
.\" This ext4 feature enables metadata checksumming.  This feature stores
.\" checksums for all of the filesystem metadata (superblock, group
.\" descriptor blocks, inode and block bitmaps, directories, and
.\" extent tree blocks).  The checksum algorithm used for the metadata
.\" blocks is different than the one used for group descriptors with the
.\" .B uninit_bg
.\" feature, these two features are incompatible and
.\" .B metadata_csum
.\" will be used preferentially instead of
.\" .BR uninit_bg .
.\" .br
.\" .B Future feature, available in e2fsprogs 1.43-WIP
.TP
.B meta_bg
.br
This ext4 feature allows file systems to be resized on-line without explicitly
needing to reserve space for growth in the size of the block group
descriptors.  This scheme is also used to resize file systems which are
larger than 2^32 blocks.  It is not recommended that this feature be set
when a file system is created, since this alternate method of storing
the block group descriptor will slow down the time needed to mount the
file system, and newer kernels can automatically set this feature as
necessary when doing an online resize and no more reserved space is
available in the resize inode.
.TP
.B mmp
.br
This ext4 feature provides multiple mount protection (MMP).  MMP helps to
protect the filesystem from being multiply mounted and is useful in
shared storage environments.
@QUOTA_MAN_COMMENT@.TP
@QUOTA_MAN_COMMENT@.B quota
@QUOTA_MAN_COMMENT@.br
@QUOTA_MAN_COMMENT@Create quota inodes (inode #3 for userquota and inode
@QUOTA_MAN_COMMENT@#4 for group quota) and set them in the superblock.
@QUOTA_MAN_COMMENT@With this feature, the quotas will be enabled
@QUOTA_MAN_COMMENT@automatically when the filesystem is mounted.
@QUOTA_MAN_COMMENT@.IP
@QUOTA_MAN_COMMENT@Causes the quota files (i.e., user.quota and
@QUOTA_MAN_COMMENT@group.quota which existed
@QUOTA_MAN_COMMENT@in the older quota design) to be hidden inodes.
@QUOTA_MAN_COMMENT@.IP
@QUOTA_MAN_COMMENT@.B Warning: 
@QUOTA_MAN_COMMENT@The quota feature is still under development, 
@QUOTA_MAN_COMMENT@and may not be fully supported with your kernel
@QUOTA_MAN_COMMENT@or may have various bugs.  
@QUOTA_MAN_COMMENT@Please see https://ext4.wiki.kernel.org/index.php/Quota
@QUOTA_MAN_COMMENT@for more details.
.TP
.B resize_inode
.br
This file system feature indicates that space has been reserved so
the block group descriptor table can be extended by the file system is
resized while the file system is mounted.  The online resize operation
is carried out by the kernel, triggered, by
.BR resize2fs (8).
By default
.B mke2fs
will attempt to reserve enough space so that the
filesystem may grow to 1024 times its initial size.  This can be changed
using the
.B resize
extended option.
.IP
This feature requires that the
.B sparse_super
feature be enabled.
.TP
.B sparse_super
.br
This file system feature is set on all modern ext2, ext3, and ext4 file
system.  It indicates that backup copies of the superblock and block
group descriptors be present only on a few block groups, and not all of
them.
.TP
.B uninit_bg
.br
This ext4 file system feature indicates that the block group descriptors
will be protected using checksums, making it safe for
.BR mke2fs (8)
to create a file system without initializing all of the block groups.
The kernel will keep a high watermark of unused inodes, and initialize
inode tables and block lazily.  This feature speeds up the time to check
the file system using
.BR e2fsck (8),
and it also speeds up the time required for
.BR mke2fs (8)
to create the file system.
.RE
.SH SEE ALSO
.BR mke2fs (8),
.BR mke2fs.conf (5),
.BR e2fsck (8),
.BR dumpe2fs (8),
.BR tune2fs (8),
.BR debugfs (8)
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html