From patchwork Sat Mar 24 01:57:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Darrick Wong X-Patchwork-Id: 890365 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.b="mJ2WU6p5"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 407Nnk2Tlfz9s08 for ; Sat, 24 Mar 2018 12:57:14 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751732AbeCXB5N (ORCPT ); Fri, 23 Mar 2018 21:57:13 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:47626 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751661AbeCXB5M (ORCPT ); Fri, 23 Mar 2018 21:57:12 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w2O1uB77189464; Sat, 24 Mar 2018 01:57:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2017-10-26; bh=j0jezBtVX/p7PfoX3DcnMV/x8YxxlhdqkmAB2nEfzfA=; b=mJ2WU6p5jcbz6L/8EHosmfwEQrj6jxi/Z30wy6t4/Foxd16AmLcQzx3qdvrIgQN3Ubju 3zYTvBBoGW+LxynMmyez9rN9MUdUeKPCZf3Ur2o0h/b1wBTC60aEInG3oo0WSc7+gVUq l0Yj1smc2xxklcVqBzKuUdgQVAcfrlhK/jMy16bB1qLJa6KBVWdzRn8ZOQAw0hYUU/ij OHyvDYl3VxLIIj39cKSBVbFQ1QDhyye10deGVE8sWtLkuufuuk32YR6l6q7E90TVLxig qChTEQDR65L+lEiai9uB28shoBWwEv20Ovk0bYdD62i6LvXhk8um8nrkIWJxqNC40uOu wg== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2130.oracle.com with ESMTP id 2gwded0020-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:06 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1v4c3005182 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:05 GMT Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1v4c5020106; Sat, 24 Mar 2018 01:57:04 GMT Received: from localhost (/10.159.132.121) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 23 Mar 2018 18:57:03 -0700 Subject: [PATCH 1/4] tune2fs: allow setting the filesystem error bit From: "Darrick J. Wong" To: tytso@mit.edu, darrick.wong@oracle.com Cc: Andreas Dilger , Lukas Czerner , linux-ext4@vger.kernel.org Date: Fri, 23 Mar 2018 18:57:02 -0700 Message-ID: <152185662255.10434.15640980790894254157.stgit@magnolia> In-Reply-To: <152185661631.10434.3295057734679624788.stgit@magnolia> References: <152185661631.10434.3295057734679624788.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8841 signatures=668695 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=29 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=937 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1803240010 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Darrick J. Wong Allow the administrator to mark the filesystem's error bit to force a fsck at the next mount. Signed-off-by: Darrick J. Wong Reviewed-by: Andreas Dilger Reviewed-by: Lukas Czerner --- misc/tune2fs.8.in | 4 ++++ misc/tune2fs.c | 5 +++++ 2 files changed, 9 insertions(+) diff --git a/misc/tune2fs.8.in b/misc/tune2fs.8.in index 740d166..f1def2b 100644 --- a/misc/tune2fs.8.in +++ b/misc/tune2fs.8.in @@ -238,6 +238,10 @@ program. This superblock setting is only honored in 2.6.35+ kernels; and not at all by the ext2 and ext3 file system drivers. .TP +.B force_fsck +Set a flag in the filesystem superblock indicating that errors have been found. +This will force fsck to run at the next mount. +.TP .B test_fs Set a flag in the filesystem superblock indicating that it may be mounted using experimental kernel code, such as the ext4dev filesystem. diff --git a/misc/tune2fs.c b/misc/tune2fs.c index 93160c9..ea84e1e 100644 --- a/misc/tune2fs.c +++ b/misc/tune2fs.c @@ -2117,6 +2117,10 @@ static int parse_extended_opts(ext2_filsys fs, const char *opts) intv); fs->super->s_mmp_update_interval = intv; ext2fs_mark_super_dirty(fs); + } else if (!strcmp(token, "force_fsck")) { + fs->super->s_state |= EXT2_ERROR_FS; + printf(_("Setting filesystem error flag to force fsck.\n")); + ext2fs_mark_super_dirty(fs); } else if (!strcmp(token, "test_fs")) { fs->super->s_flags |= EXT2_FLAGS_TEST_FILESYS; printf("Setting test filesystem flag\n"); @@ -2199,6 +2203,7 @@ static int parse_extended_opts(ext2_filsys fs, const char *opts) "\tmmp_update_interval=\n" "\tstride=\n" "\tstripe_width=\n" + "\tforce_fsck\n" "\ttest_fs\n" "\t^test_fs\n")); free(buf); From patchwork Sat Mar 24 01:57:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Darrick Wong X-Patchwork-Id: 890366 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.b="ulO3OWt6"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 407Nnm3V14z9s08 for ; Sat, 24 Mar 2018 12:57:16 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751777AbeCXB5P (ORCPT ); Fri, 23 Mar 2018 21:57:15 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:33354 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751675AbeCXB5O (ORCPT ); Fri, 23 Mar 2018 21:57:14 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w2O1hBPK040216; Sat, 24 Mar 2018 01:57:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2017-10-26; bh=FVBH6H+RshogUyXua8bM2fZRWIalyHoeMQ9aTE7udCk=; b=ulO3OWt6/+8P5R0HGEgPHQ+fMo5mMbEJ8/wKTKGyTFjgDxbcRg8RPwwA2hIX0nA9dA7v s4aMB0lqkBea0tacc8kcmTBh7wKLzYeUZOQ2hg8cGSoZ994S+pAB7heROHoubDZ+ZHz/ KooxALx0rBhUMxaDkmWjo96QHP9OPsFcEYnSmVCQau9zEfSkKjUHFQ6JR23pF05gT9d0 1MBCwlJVmTxsqiy2fdoSwOqA5uyN7Azj8UT9YO2W+bjZeaQiTS9sU0hcFfSAZKPOici8 3perLWIfzDmYjA2kF2z34vP54bn9afCafocPpMosKcwkaFOKCb6UIdBhDVgY8zaBwkTp yg== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2120.oracle.com with ESMTP id 2gwd8rr0fg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:12 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1vBbc004890 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:11 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1vAK6020192; Sat, 24 Mar 2018 01:57:10 GMT Received: from localhost (/10.159.132.121) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 23 Mar 2018 18:57:10 -0700 Subject: [PATCH 2/4] e2scrub: create online fsck tool of sorts From: "Darrick J. Wong" To: tytso@mit.edu, darrick.wong@oracle.com Cc: linux-ext4@vger.kernel.org Date: Fri, 23 Mar 2018 18:57:09 -0700 Message-ID: <152185662916.10434.10223789589439750106.stgit@magnolia> In-Reply-To: <152185661631.10434.3295057734679624788.stgit@magnolia> References: <152185661631.10434.3295057734679624788.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8841 signatures=668695 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=29 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1803240010 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Darrick J. Wong Implement online fsck for ext* filesystems which live on LVM-managed logical volumes. The basic strategy mirrors that of e2croncheck -- create a snapshot, fsck the snapshot, report whatever errors appear, remove snapshot. Unlike e2croncheck, this utility accepts any LVM device path, knows about snapshots running out of space, and can call fstrim having validated that the fs metadata is ok. Signed-off-by: Darrick J. Wong --- MCONFIG.in | 3 + Makefile.in | 3 - configure | 149 ++++++++++++++++++++++++++++-- configure.ac | 56 ++++++++++- debian/control | 2 debian/e2fsprogs.files | 1 scrub/Makefile.in | 97 +++++++++++++++++++ scrub/e2scrub.8.in | 60 ++++++++++++ scrub/e2scrub.conf.in | 10 ++ scrub/e2scrub.in | 239 ++++++++++++++++++++++++++++++++++++++++++++++++ scrub/e2scrub.rules.in | 2 util/subst.conf.in | 3 + 12 files changed, 608 insertions(+), 17 deletions(-) create mode 100644 scrub/Makefile.in create mode 100644 scrub/e2scrub.8.in create mode 100644 scrub/e2scrub.conf.in create mode 100644 scrub/e2scrub.in create mode 100644 scrub/e2scrub.rules.in diff --git a/MCONFIG.in b/MCONFIG.in index 22b74eb..adeb5bd 100644 --- a/MCONFIG.in +++ b/MCONFIG.in @@ -33,6 +33,9 @@ infodir = @infodir@ datadir = @datadir@ pkgconfigdir = $(libdir)/pkgconfig +HAVE_UDEV = @have_udev@ +UDEV_RULES_DIR = @pkg_udev_rules_dir@ + @SET_MAKE@ @ifGNUmake@ V = diff --git a/Makefile.in b/Makefile.in index 37b6069..ddd94ec 100644 --- a/Makefile.in +++ b/Makefile.in @@ -13,10 +13,11 @@ INSTALL = @INSTALL@ @DEBUGFS_CMT@DEBUGFS_DIR= debugfs @UUID_CMT@UUID_LIB_SUBDIR= lib/uuid @BLKID_CMT@BLKID_LIB_SUBDIR= lib/blkid +@E2SCRUB_CMT@E2SCRUB_DIR= scrub SUPPORT_LIB_SUBDIR= lib/support LIB_SUBDIRS=lib/et lib/ss lib/e2p $(UUID_LIB_SUBDIR) $(BLKID_LIB_SUBDIR) $(SUPPORT_LIB_SUBDIR) lib/ext2fs intl -PROG_SUBDIRS=e2fsck $(DEBUGFS_DIR) misc $(RESIZE_DIR) tests/progs po +PROG_SUBDIRS=e2fsck $(DEBUGFS_DIR) misc $(RESIZE_DIR) tests/progs po $(E2SCRUB_DIR) SUBDIRS=util $(LIB_SUBDIRS) $(PROG_SUBDIRS) tests SUBS= util/subst.conf lib/config.h $(top_builddir)/lib/dirpaths.h \ diff --git a/configure b/configure index b2701d2..986e057 100755 --- a/configure +++ b/configure @@ -625,6 +625,10 @@ gl_use_threads_default= ac_func_list= ac_subst_vars='LTLIBOBJS LIBOBJS +pkg_udev_rules_dir +have_udev +udev_LIBS +udev_CFLAGS LDFLAGS_SHLIB CFLAGS_STLIB CFLAGS_SHLIB @@ -639,6 +643,7 @@ root_libdir root_sbindir root_bindir root_prefix +E2SCRUB_CMT UNIX_CMT CYGWIN_CMT LINUX_CMT @@ -795,6 +800,7 @@ build_os build_vendor build_cpu build +E2FSPROGS_DATE E2FSPROGS_PKGVER E2FSPROGS_VERSION E2FSPROGS_DAY @@ -892,6 +898,7 @@ with_included_gettext with_libintl_prefix enable_fuse2fs with_multiarch +with_udev_rules_dir ' ac_precious_vars='build_alias host_alias @@ -904,7 +911,9 @@ CPPFLAGS CPP PKG_CONFIG PKG_CONFIG_PATH -PKG_CONFIG_LIBDIR' +PKG_CONFIG_LIBDIR +udev_CFLAGS +udev_LIBS' # Initialize some variables set by options. @@ -1580,6 +1589,8 @@ Optional Packages: --with-libintl-prefix[=DIR] search for libintl in DIR/include and DIR/lib --without-libintl-prefix don't search for libintl in includedir and libdir --with-multiarch=ARCH specify the multiarch triplet + --with-udev-rules-dir[=DIR] + Install udev rules into DIR. Some influential environment variables: CC C compiler command @@ -1595,6 +1606,8 @@ Some influential environment variables: directories to add to pkg-config's search path PKG_CONFIG_LIBDIR path overriding pkg-config's built-in search path + udev_CFLAGS C compiler flags for udev, overriding pkg-config + udev_LIBS linker flags for udev, overriding pkg-config Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. @@ -2758,11 +2771,11 @@ MCONFIG=./MCONFIG BINARY_TYPE=bin E2FSPROGS_VERSION=`grep E2FSPROGS_VERSION ${srcdir}/version.h \ | awk '{print $3}' | tr \" " " | awk '{print $1}'` -DATE=`grep E2FSPROGS_DATE ${srcdir}/version.h | awk '{print $3}' \ - | tr \" " "` -E2FSPROGS_DAY=$(echo $DATE | awk -F- '{print $1}' | sed -e '/^[1-9]$/s/^/0/') -MONTH=`echo $DATE | awk -F- '{print $2}'` -YEAR=`echo $DATE | awk -F- '{print $3}'` +E2FSPROGS_DATE=`grep E2FSPROGS_DATE ${srcdir}/version.h | awk '{print $3}' \ + | tr \" " " | awk '{print $1}'` +E2FSPROGS_DAY=$(echo $E2FSPROGS_DATE | awk -F- '{print $1}' | sed -e '/^[1-9]$/s/^/0/') +MONTH=`echo $E2FSPROGS_DATE | awk -F- '{print $2}'` +YEAR=`echo $E2FSPROGS_DATE | awk -F- '{print $3}'` if expr $YEAR ">" 1900 > /dev/null ; then E2FSPROGS_YEAR=$YEAR @@ -2813,6 +2826,7 @@ $as_echo "Release date is ${E2FSPROGS_MONTH}, ${E2FSPROGS_YEAR}" >&6; } + WITH_DIET_LIBC= # Check whether --with-diet-libc was given. @@ -7236,8 +7250,6 @@ main () if (*(data + i) != *(data3 + i)) return 14; close (fd); - free (data); - free (data3); return 0; } _ACEOF @@ -13713,6 +13725,8 @@ esac +E2SCRUB_CMT="$LINUX_CMT" + case "$host_os" in linux* | gnu* | k*bsd*-gnu) if test "$prefix" = NONE -a "$root_prefix" = NONE ; then @@ -13878,6 +13892,123 @@ LDFLAGS_SHLIB=${LDFLAGS_SHLIB:-$LDFLAGS} + + +# Check whether --with-udev_rules_dir was given. +if test "${with_udev_rules_dir+set}" = set; then : + withval=$with_udev_rules_dir; +else + with_udev_rules_dir=yes +fi + +if test "x${with_udev_rules_dir}" != "xno"; then : + + if test "x${with_udev_rules_dir}" = "xyes"; then : + + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for udev" >&5 +$as_echo_n "checking for udev... " >&6; } + +if test -n "$udev_CFLAGS"; then + pkg_cv_udev_CFLAGS="$udev_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"udev\""; } >&5 + ($PKG_CONFIG --exists --print-errors "udev") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_udev_CFLAGS=`$PKG_CONFIG --cflags "udev" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$udev_LIBS"; then + pkg_cv_udev_LIBS="$udev_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"udev\""; } >&5 + ($PKG_CONFIG --exists --print-errors "udev") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_udev_LIBS=`$PKG_CONFIG --libs "udev" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + udev_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "udev" 2>&1` + else + udev_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "udev" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$udev_PKG_ERRORS" >&5 + + + with_udev_rules_dir="" + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + with_udev_rules_dir="" + +else + udev_CFLAGS=$pkg_cv_udev_CFLAGS + udev_LIBS=$pkg_cv_udev_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + with_udev_rules_dir="$($PKG_CONFIG --variable=udevdir udev)/rules.d" + +fi + +fi + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for udev rules dir" >&5 +$as_echo_n "checking for udev rules dir... " >&6; } + pkg_udev_rules_dir="${with_udev_rules_dir}" + if test -n "${pkg_udev_rules_dir}"; then : + + { $as_echo "$as_me:${as_lineno-$LINENO}: result: ${pkg_udev_rules_dir}" >&5 +$as_echo "${pkg_udev_rules_dir}" >&6; } + have_udev="yes" + +else + + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + have_udev="no" + +fi + +else + + have_udev="disabled" + +fi + + + test -d lib || mkdir lib test -d include || mkdir include test -d include/linux || mkdir include/linux @@ -13899,7 +14030,7 @@ for i in MCONFIG Makefile e2fsprogs.spec \ misc/Makefile ext2ed/Makefile e2fsck/Makefile \ debugfs/Makefile tests/Makefile tests/progs/Makefile \ resize/Makefile doc/Makefile intl/Makefile \ - intl/libgnuintl.h po/Makefile.in ; do + intl/libgnuintl.h po/Makefile.in scrub/Makefile; do if test -d `dirname ${srcdir}/$i` ; then outlist="$outlist $i" fi diff --git a/configure.ac b/configure.ac index 7392959..6e549c5 100644 --- a/configure.ac +++ b/configure.ac @@ -11,11 +11,11 @@ dnl This is to figure out the version number and the date.... dnl E2FSPROGS_VERSION=`grep E2FSPROGS_VERSION ${srcdir}/version.h \ | awk '{print $3}' | tr \" " " | awk '{print $1}'` -DATE=`grep E2FSPROGS_DATE ${srcdir}/version.h | awk '{print $3}' \ - | tr \" " "` -E2FSPROGS_DAY=$(echo $DATE | awk -F- '{print $1}' | sed -e '/^[[1-9]]$/s/^/0/') -MONTH=`echo $DATE | awk -F- '{print $2}'` -YEAR=`echo $DATE | awk -F- '{print $3}'` +E2FSPROGS_DATE=`grep E2FSPROGS_DATE ${srcdir}/version.h | awk '{print $3}' \ + | tr \" " " | awk '{print $1}'` +E2FSPROGS_DAY=$(echo $E2FSPROGS_DATE | awk -F- '{print $1}' | sed -e '/^[[1-9]]$/s/^/0/') +MONTH=`echo $E2FSPROGS_DATE | awk -F- '{print $2}'` +YEAR=`echo $E2FSPROGS_DATE | awk -F- '{print $3}'` if expr $YEAR ">" 1900 > /dev/null ; then E2FSPROGS_YEAR=$YEAR @@ -63,6 +63,7 @@ AC_SUBST(E2FSPROGS_MONTH) AC_SUBST(E2FSPROGS_DAY) AC_SUBST(E2FSPROGS_VERSION) AC_SUBST(E2FSPROGS_PKGVER) +AC_SUBST(E2FSPROGS_DATE) dnl dnl Use diet libc dnl @@ -1312,6 +1313,11 @@ AC_SUBST(LINUX_CMT) AC_SUBST(CYGWIN_CMT) AC_SUBST(UNIX_CMT) dnl +dnl e2scrub only builds on linux +dnl +E2SCRUB_CMT="$LINUX_CMT" +AC_SUBST(E2SCRUB_CMT) +dnl dnl Linux and Hurd places root files in the / by default dnl case "$host_os" in @@ -1469,6 +1475,44 @@ LDFLAGS_SHLIB=${LDFLAGS_SHLIB:-$LDFLAGS} AC_SUBST(CFLAGS_SHLIB) AC_SUBST(CFLAGS_STLIB) AC_SUBST(LDFLAGS_SHLIB) + +dnl +dnl Where do udev rules go? +dnl +AC_ARG_WITH([udev_rules_dir], + [AS_HELP_STRING([--with-udev-rules-dir@<:@=DIR@:>@], + [Install udev rules into DIR.])], + [], + [with_udev_rules_dir=yes]) +AS_IF([test "x${with_udev_rules_dir}" != "xno"], + [ + AS_IF([test "x${with_udev_rules_dir}" = "xyes"], + [ + PKG_CHECK_MODULES([udev], [udev], + [ + with_udev_rules_dir="$($PKG_CONFIG --variable=udevdir udev)/rules.d" + ], [ + with_udev_rules_dir="" + ]) + ]) + AC_MSG_CHECKING([for udev rules dir]) + pkg_udev_rules_dir="${with_udev_rules_dir}" + AS_IF([test -n "${pkg_udev_rules_dir}"], + [ + AC_MSG_RESULT(${pkg_udev_rules_dir}) + have_udev="yes" + ], + [ + AC_MSG_RESULT(no) + have_udev="no" + ]) + ], + [ + have_udev="disabled" + ]) +AC_SUBST(have_udev) +AC_SUBST(pkg_udev_rules_dir) + dnl dnl Make our output files, being sure that we create the some miscellaneous dnl directories @@ -1494,7 +1538,7 @@ for i in MCONFIG Makefile e2fsprogs.spec \ misc/Makefile ext2ed/Makefile e2fsck/Makefile \ debugfs/Makefile tests/Makefile tests/progs/Makefile \ resize/Makefile doc/Makefile intl/Makefile \ - intl/libgnuintl.h po/Makefile.in ; do + intl/libgnuintl.h po/Makefile.in scrub/Makefile; do if test -d `dirname ${srcdir}/$i` ; then outlist="$outlist $i" fi diff --git a/debian/control b/debian/control index ee0f66b..3515d3d 100644 --- a/debian/control +++ b/debian/control @@ -189,7 +189,7 @@ Essential: yes Pre-Depends: ${shlibs:Depends}, ${misc:Depends}, libblkid1, libuuid1 Multi-Arch: foreign Suggests: gpart, parted, fuse2fs, e2fsck-static -Recommends: e2fsprogs-l10n +Recommends: e2fsprogs-l10n, lvm2, util-linux, coreutils Architecture: any Description: ext2/ext3/ext4 file system utilities The ext2, ext3 and ext4 file systems are successors of the original ext diff --git a/debian/e2fsprogs.files b/debian/e2fsprogs.files index 0a22f31..78720fe 100644 --- a/debian/e2fsprogs.files +++ b/debian/e2fsprogs.files @@ -3,3 +3,4 @@ usr/bin usr/sbin usr/share/man etc +lib/udev/rules.d diff --git a/scrub/Makefile.in b/scrub/Makefile.in new file mode 100644 index 0000000..a8bb06b --- /dev/null +++ b/scrub/Makefile.in @@ -0,0 +1,97 @@ +# +# Makefile for e2scrub +# + +srcdir = @srcdir@ +top_srcdir = @top_srcdir@ +VPATH = @srcdir@ +top_builddir = .. +my_dir = scrub +INSTALL = @INSTALL@ + +@MCONFIG@ + +PROGS= e2scrub +MANPAGES= e2scrub.8 +CONFFILES= e2scrub.conf + +ifeq ($(HAVE_UDEV),yes) +UDEV_RULES = e2scrub.rules +INSTALLDIRS_TGT += installdirs-udev +INSTALL_TGT += install-udev +UNINSTALL_TGT += uninstall-udev +endif + +all:: $(PROGS) $(MANPAGES) $(CONFFILES) $(UDEV_RULES) + +e2scrub: $(DEP_SUBSTITUTE) e2scrub.in + $(E) " SUBST $@" + $(Q) $(SUBSTITUTE_UPTIME) $(srcdir)/e2scrub.in $@ + $(Q) chmod a+x $@ + +%.8: %.8.in $(DEP_SUBSTITUTE) + $(E) " SUBST $@" + $(Q) $(SUBSTITUTE_UPTIME) $< $@ + +%.conf: %.conf.in $(DEP_SUBSTITUTE) + $(E) " SUBST $@" + $(Q) $(SUBSTITUTE_UPTIME) $< $@ + +%.rules: %.rules.in $(DEP_SUBSTITUTE) + $(E) " SUBST $@" + $(Q) $(SUBSTITUTE_UPTIME) $< $@ + +installdirs-udev: + $(E) " MKINSTALLDIRS $(UDEV_RULES_DIR)" + $(Q) $(MKINSTALLDIRS) $(DESTDIR)$(UDEV_RULES_DIR) + +installdirs: $(INSTALLDIRS_TGT) + $(E) " MKINSTALLDIRS $(root_sbindir) $(man8dir) $(root_sysconfdir)" + $(Q) $(MKINSTALLDIRS) $(DESTDIR)$(root_sbindir) \ + $(DESTDIR)$(man8dir) $(DESTDIR)$(root_sysconfdir) + +install-udev: + $(Q) for i in $(UDEV_RULES); do \ + $(ES) " INSTALL $(UDEV_RULES_DIR)/$$i"; \ + $(INSTALL_PROGRAM) $$i $(DESTDIR)$(UDEV_RULES_DIR)/96-$$i; \ + done + +install: $(PROGS) $(MANPAGES) $(FMANPAGES) installdirs $(INSTALL_TGT) + $(Q) for i in $(PROGS); do \ + $(ES) " INSTALL $(root_sbindir)/$$i"; \ + $(INSTALL_PROGRAM) $$i $(DESTDIR)$(root_sbindir)/$$i; \ + done + $(Q) for i in $(MANPAGES); do \ + for j in $(COMPRESS_EXT); do \ + $(RM) -f $(DESTDIR)$(man8dir)/$$i.$$j; \ + done; \ + $(ES) " INSTALL_DATA $(man8dir)/$$i"; \ + $(INSTALL_DATA) $$i $(DESTDIR)$(man8dir)/$$i; \ + done + $(Q) for i in $(CONFFILES); do \ + $(ES) " INSTALL_DATA $(root_sysconfdir)/$$i"; \ + $(INSTALL_DATA) $$i $(DESTDIR)$(root_sysconfdir)/$$i; \ + done + +uninstall-udev: + for i in $(UDEV_RULES); do \ + $(RM) -f $(DESTDIR)$(UDEV_RULES_DIR)/96-$$i; \ + done + +uninstall: $(UNINSTALL_TGT) + for i in $(PROGS); do \ + $(RM) -f $(DESTDIR)$(root_sbindir)/$$i; \ + done + for i in $(MANPAGES); do \ + $(RM) -f $(DESTDIR)$(man8dir)/$$i; \ + done + for i in $(CONFFILES); do \ + $(RM) -f $(DESTDIR)$(root_sysconfdir)/$$i; \ + done + +clean:: + $(RM) -f $(PROGS) + +mostlyclean: clean +distclean: clean + $(RM) -f .depend Makefile $(srcdir)/TAGS $(srcdir)/Makefile.in.old diff --git a/scrub/e2scrub.8.in b/scrub/e2scrub.8.in new file mode 100644 index 0000000..ff03523 --- /dev/null +++ b/scrub/e2scrub.8.in @@ -0,0 +1,60 @@ +.TH E2SCRUB 8 "@E2FSPROGS_MONTH@ @E2FSPROGS_YEAR@" "E2fsprogs version @E2FSPROGS_VERSION@" +.SH NAME +e2scrub - check the contents of a mounted ext[234] filesystem +.SH SYNOPSYS +.B +e2scrub [OPTION] MOUNTPOINT | DEVICE +.SH DESCRIPTION +.B e2scrub +attempts to check (but not repair) all metadata in a mounted ext[234] +filesystem if the filesystem resides on a LVM logical volume. +The block device of the LVM logical volume can also be passed in. + +This program snapshots the volume and runs a file system check on the snapshot +to look for corruption errors. +The LVM volume group must have at least 256MiB of unallocated space to +dedicate to the snapshot or the logical volume will be skipped. +The snapshot will be named +.IR lvname ".e2scrub" +and +.B udev +will not create symbolic links to it under +.IR /dev/disk . +Every attempt will be made to remove the snapshots prior to running +.BR e2scrub , +but in a dire situation it may be necessary to remove the snapshot manually. + +If no errors are found, +.B fstrim +can be called on the file system if it is mounted. +If errors are found, the file system will be marked as having errors. +The filesystem should be taken offline and +.B e2fsck +run as soon as possible, because +.B e2scrub +does not fix corruptions. +If the filesystem is not repaired, +.B e2fsck +will be run before the next mount. +.SH OPTIONS +.TP +\fB-r\fR +Remove the e2scrub snapshot and exit without checking anything. +.TP +\fB-t\fR +Run +.B +fstrim(1) +on the mounted filesystem if no errors are found. +.TP +\fB-V\fR +Print version information and exit. +.SH EXIT CODE +The exit codes are the same as in +.BR e2fsck (8) +.SH SEE ALSO +.BR e2fsck (8) +.SH AUTHOR +Darrick J. Wong +.SH COPYRIGHT +Copyright ©2018 Oracle. License is GPLv2+. diff --git a/scrub/e2scrub.conf.in b/scrub/e2scrub.conf.in new file mode 100644 index 0000000..d578cc7 --- /dev/null +++ b/scrub/e2scrub.conf.in @@ -0,0 +1,10 @@ +# e2scrub configuration file + +# Snapshots will be created to run fsck; the snapshot will be of this size. +# snap_size_mb=256 + +# Set this to 1 to enable fstrim for everyone. +# fstrim=0 + +# Arguments passed into e2fsck. +# e2fsck_opts="-vtt" diff --git a/scrub/e2scrub.in b/scrub/e2scrub.in new file mode 100644 index 0000000..08b3600 --- /dev/null +++ b/scrub/e2scrub.in @@ -0,0 +1,239 @@ +#!/bin/bash + +# Copyright (C) 2018 Oracle. All Rights Reserved. +# +# Author: Darrick J. Wong +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA. + +# Automatically check a LVM-managed filesystem online. +# We use lvm snapshots to do this, which means that we can only +# check filesystems in VGs that have at least 256MB (or so) of +# free space. + +snap_size_mb=256 +fstrim=0 +reap=0 +e2fsck_opts="" +conffile="@root_sysconfdir@/e2scrub.conf" + +test -f "${conffile}" && . "${conffile}" + +print_help() { + echo "Usage: $0 [OPTIONS] mountpoint | device" + echo + echo "mountpoint must be on a LVM-managed block device" + echo "-r: Remove e2scrub snapshot and exit, do not check anything." + echo "-t: Run fstrim if successful." + echo "-V: Print version information and exit." +} + +print_version() { + echo "e2scrub @E2FSPROGS_VERSION@ (@E2FSPROGS_DATE@)" +} + +while getopts "rtV" opt; do + case "${opt}" in + "r") reap=1;; + "t") fstrim=1;; + "V") print_version; exit 0;; + *) print_help; exit 2;; + esac +done +shift "$((OPTIND - 1))" + +arg="$1" +if [ -z "${arg}" ]; then + print_help + exit 1 +fi + +# Find the device for a given mountpoint +dev_from_mount() { + local mountpt="$(realpath "$1")" + + lsblk -o NAME,FSTYPE,MOUNTPOINT -p -P -n 2> /dev/null | while read vars; do + eval "${vars}" + if [ "${mountpt}" != "${MOUNTPOINT}" ]; then + continue + fi + case "${FSTYPE}" in + ext[234]) + echo "${NAME}" + return 0 + ;; + esac + done + return 1 +} + +# Check a device argument +dev_from_arg() { + local dev="$1" + local fstype="$(lsblk -o FSTYPE -n "${dev}" 2> /dev/null)" + + case "${fstype}" in + ext[234]) + echo "${dev}" + return 0 + ;; + esac + return 1 +} + +mnt_from_dev() { + local dev="$1" + + if [ -n "${dev}" ]; then + lsblk -o MOUNTPOINT -n "${dev}" + fi +} + +# Construct block device path and mountpoint from argument +if [ -b "${arg}" ]; then + dev="$(dev_from_arg "${arg}")" + mnt="$(mnt_from_dev "${dev}")" +else + dev="$(dev_from_mount "${arg}")" + mnt="${arg}" +fi +if [ ! -e "${dev}" ]; then + echo "${arg}: Not an ext[234] filesystem." + print_help + exit 16 +fi + +# Make sure this is an LVM device we can snapshot +lvm_vars="$(lvs --nameprefixes -o name,vgname,lv_role --noheadings "${dev}" 2> /dev/null)" +eval "${lvm_vars}" +if [ -z "${LVM2_VG_NAME}" ] || [ -z "${LVM2_LV_NAME}" ] || + echo "${LVM2_LV_ROLE}" | grep -q "snapshot"; then + echo "${arg}: Not connnected to a LVM logical volume." + print_help + exit 16 +fi +start_time="$(date +'%Y%m%d%H%M%S')" +snap="${LVM2_LV_NAME}.e2scrub" +snap_dev="/dev/${LVM2_VG_NAME}/${snap}" + +teardown() { + # Remove and wait for removal to succeed. + ${DBG} lvremove -f "${LVM2_VG_NAME}/${snap}" 3>&- + while [ -e "${snap_dev}" ] && [ "$?" -eq "5" ]; do + sleep 0.5 + ${DBG} lvremove -f "${LVM2_VG_NAME}/${snap}" 3>&- + done +} + +check() { + # First we recover the journal, then we see if e2fsck tries any + # non-optimization repairs. If either of these two returns a + # non-zero status (errors fixed or remaining) then this fs is bad. + E2FSCK_FIXES_ONLY=1 + export E2FSCK_FIXES_ONLY + ${DBG} "@root_sbindir@/e2fsck" -E journal_only -p ${e2fsck_opts} "${snap_dev}" || return $? + ${DBG} "@root_sbindir@/e2fsck" -f -y ${e2fsck_opts} "${snap_dev}" +} + +mark_clean() { + ${DBG} "@root_sbindir@/tune2fs" -C 0 -T "${start_time}" "${dev}" +} + +mark_corrupt() { + ${DBG} "@root_sbindir@/tune2fs" -E force_fsck "${dev}" +} + +setup() { + # Try to remove snapshot for 30s, bail out if we can't remove it. + lveremove_deadline="$(( $(date "+%s") + 30))" + ${DBG} lvremove -f "${LVM2_VG_NAME}/${snap}" 3>&- 2>/dev/null + while [ -e "${snap_dev}" ] && [ "$?" -eq "5" ] && + [ "$(date "+%s")" -lt "${lvremove_deadline}" ]; do + sleep 0.5 + ${DBG} lvremove -f "${LVM2_VG_NAME}/${snap}" 3>&- + done + if [ -e "${snap_dev}" ]; then + echo "${arg}: e2scrub snapshot is in use, cannot check!" + return 1 + fi + # Create the snapshot, wait for device to appear. + ${DBG} lvcreate -s -L "${snap_size_mb}m" -n "${snap}" "${LVM2_VG_NAME}/${LVM2_LV_NAME}" 3>&- + if [ $? -ne 0 ]; then + echo "${arg}: e2scrub snapshot FAILED, will not check!" + return 1 + fi + ${DBG} udevadm settle 2> /dev/null + return 0 +} + +if [ "${reap}" -gt 0 ]; then + if [ -e "${snap_dev}" ]; then + teardown 2> /dev/null + fi + exit 0 +fi +if ! setup; then + exit 8 +fi +trap "teardown; exit 1" EXIT INT QUIT TERM + +# Check and react +check +case "$?" in +"0") + # Clean check! + echo "${arg}: Scrub succeeded." + mark_clean + teardown + trap '' EXIT + + # Trim the free space, which requires the snapshot be deleted. + if [ "${fstrim}" -eq 1 ] && [ -d "${mnt}" ] && type fstrim > /dev/null 2>&1; then + echo "${arg}: Trimming free space." + fstrim -v "${mnt}" + fi + + ret=0 + ;; +"8") + # Operational error, what now? + echo "${arg}: e2fsck operational error." + teardown + trap '' EXIT + ret=8 + ;; +*) + # fsck failed. Check if the snapshot is invalid; if so, make a + # note of that at the end of the log. This isn't necessarily a + # failure because the mounted fs could have overflowed the + # snapshot with regular disk writes /or/ our repair process + # could have done it by repairing too much. + # + # If it's really corrupt we ought to fsck at next boot. + is_invalid="$(lvs -o lv_snapshot_invalid --noheadings "${snap_dev}" | awk '{print $1}')" + if [ -n "${is_invalid}" ]; then + echo "${arg}: Scrub FAILED due to invalid snapshot." + ret=8 + else + echo "${arg}: Scrub FAILED due to corruption! Unmount and run e2fsck -y." + mark_corrupt + ret=6 + fi + teardown + trap '' EXIT + ;; +esac + +exit "${ret}" diff --git a/scrub/e2scrub.rules.in b/scrub/e2scrub.rules.in new file mode 100644 index 0000000..5e1b35b --- /dev/null +++ b/scrub/e2scrub.rules.in @@ -0,0 +1,2 @@ +# Try to hide our fsck snapshots from udev's /dev/disk linking... +ACTION=="add|change", ENV{DM_LV_NAME}=="*.e2scrub", OPTIONS="link_priority=-100" diff --git a/util/subst.conf.in b/util/subst.conf.in index fbc044d..6bf658d 100644 --- a/util/subst.conf.in +++ b/util/subst.conf.in @@ -2,6 +2,7 @@ AWK @AWK@ SED @SED@ E2FSPROGS_MONTH @E2FSPROGS_MONTH@ E2FSPROGS_YEAR @E2FSPROGS_YEAR@ +E2FSPROGS_DATE @E2FSPROGS_DATE@ E2FSPROGS_VERSION @E2FSPROGS_VERSION@ SIZEOF_LONG_LONG @SIZEOF_LONG_LONG@ SIZEOF_LONG @SIZEOF_LONG@ @@ -18,3 +19,5 @@ $prefix @prefix@ JDEV # Enable the documentation for the tdb profile in e2fsck.conf's man page TDB_MAN_COMMENT @TDB_MAN_COMMENT@ +root_sbindir @root_sbindir@ +root_bindir @root_bindir@ From patchwork Sat Mar 24 01:57:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Darrick Wong X-Patchwork-Id: 890367 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.b="WtdvWBzl"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 407Nnv2nXPz9s08 for ; Sat, 24 Mar 2018 12:57:23 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751811AbeCXB5W (ORCPT ); Fri, 23 Mar 2018 21:57:22 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:51320 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751675AbeCXB5W (ORCPT ); Fri, 23 Mar 2018 21:57:22 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w2O1sPW2110663; Sat, 24 Mar 2018 01:57:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2017-10-26; bh=/lkzVX7hks2IUKIqlyNBCpvFJul7LuqneZzRjNrNNyk=; b=WtdvWBzlOs0KXWFrqKVEmuCnMzJrGVSJFySybgrl7VS1lynbdXGowGE4LWzdPXH7SI4/ dNScg0NaMPIeczK3N5LMMZRt39y21ZID9SgKVFb7oYafHTbcnKxyIXcMOXrMDsh167Pz znerL/Q7D+6BKrO6Yg8q34p9doc99Pu3OHznwZUeuUa8kWl0PmlQl+3yhO+QkjpOJAEA 5BQNffREBPpPd3hELhaxlS6RGTDhPk2VpmLsUqVyhdbJN64LS0XZj/g2C8l7MR0U8PUd cmliuA+3LNNMFW0XcfEvMiXh0BH0ioUAEK4I/VZw6DrVtdt201zDoERa61HCXpYU3u4i fQ== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2gwdder046-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:17 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1vHnZ005491 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:17 GMT Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w2O1vHHW023874; Sat, 24 Mar 2018 01:57:17 GMT Received: from localhost (/10.159.132.121) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 23 Mar 2018 18:57:16 -0700 Subject: [PATCH 3/4] e2scrub: create a script to scrub all ext* filesystems From: "Darrick J. Wong" To: tytso@mit.edu, darrick.wong@oracle.com Cc: Andreas Dilger , linux-ext4@vger.kernel.org Date: Fri, 23 Mar 2018 18:57:15 -0700 Message-ID: <152185663566.10434.10842263720369617262.stgit@magnolia> In-Reply-To: <152185661631.10434.3295057734679624788.stgit@magnolia> References: <152185661631.10434.3295057734679624788.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8841 signatures=668695 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=9 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1803240010 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Darrick J. Wong Create an e2scrub_all command to find all ext* filesystems and run an online scrub against them all. Signed-off-by: Darrick J. Wong Reviewed-by: Andreas Dilger --- scrub/Makefile.in | 9 ++++- scrub/e2scrub.conf.in | 3 ++ scrub/e2scrub_all.8.in | 36 ++++++++++++++++++++ scrub/e2scrub_all.in | 85 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 131 insertions(+), 2 deletions(-) create mode 100644 scrub/e2scrub_all.8.in create mode 100644 scrub/e2scrub_all.in diff --git a/scrub/Makefile.in b/scrub/Makefile.in index a8bb06b..1744941 100644 --- a/scrub/Makefile.in +++ b/scrub/Makefile.in @@ -11,8 +11,8 @@ INSTALL = @INSTALL@ @MCONFIG@ -PROGS= e2scrub -MANPAGES= e2scrub.8 +PROGS= e2scrub e2scrub_all +MANPAGES= e2scrub.8 e2scrub_all.8 CONFFILES= e2scrub.conf ifeq ($(HAVE_UDEV),yes) @@ -29,6 +29,11 @@ e2scrub: $(DEP_SUBSTITUTE) e2scrub.in $(Q) $(SUBSTITUTE_UPTIME) $(srcdir)/e2scrub.in $@ $(Q) chmod a+x $@ +e2scrub_all: e2scrub_all.in + $(E) " SUBST $@" + $(Q) $(SUBSTITUTE_UPTIME) $(srcdir)/e2scrub_all.in $@ + $(Q) chmod a+x $@ + %.8: %.8.in $(DEP_SUBSTITUTE) $(E) " SUBST $@" $(Q) $(SUBSTITUTE_UPTIME) $< $@ diff --git a/scrub/e2scrub.conf.in b/scrub/e2scrub.conf.in index d578cc7..69f997f 100644 --- a/scrub/e2scrub.conf.in +++ b/scrub/e2scrub.conf.in @@ -8,3 +8,6 @@ # Arguments passed into e2fsck. # e2fsck_opts="-vtt" + +# Set this to 1 to have e2scrub_all scrub all LVs, not just the mounted ones. +# scrub_all=0 diff --git a/scrub/e2scrub_all.8.in b/scrub/e2scrub_all.8.in new file mode 100644 index 0000000..ba3b873 --- /dev/null +++ b/scrub/e2scrub_all.8.in @@ -0,0 +1,36 @@ +.TH E2SCRUB 8 "@E2FSPROGS_MONTH@ @E2FSPROGS_YEAR@" "E2fsprogs version @E2FSPROGS_VERSION@" +.SH NAME +e2scrub_all - check all mounted ext[234] filesystems for errors. +.SH SYNOPSYS +.B +e2scrub_all [OPTION] +.SH DESCRIPTION +Searches the system for all LVM logical volumes containing an ext2, ext3, or +ext4 file system, and checks them for problems. +The checking is performed by invoking the +.B e2scrub +tool, which will look for corruptions. +Corrupt filesystems will be tagged as having errors so that fsck will be +invoked before the next mount. +If no errors are encountered, +.B fstrim +will be called on the filesystem if it is mounted. +See the +.B e2scrub +manual page for more information about how the checking is performed. +.SH OPTIONS +.TP +\fB-A\fR +Scrub all ext[234] filesystems even if they are not mounted. +.TP +\fB-r\fR +Remove e2scrub snapshots but do not check anything. +.TP +\fB-V\fR +Print version information and exit. +.SH SEE ALSO +.BR e2scrub "(8)" +.SH AUTHOR +Darrick J. Wong +.SH COPYRIGHT +Copyright ©2018 Oracle. License is GPLv2+. diff --git a/scrub/e2scrub_all.in b/scrub/e2scrub_all.in new file mode 100644 index 0000000..b9e5dea --- /dev/null +++ b/scrub/e2scrub_all.in @@ -0,0 +1,85 @@ +#!/bin/bash + +# Copyright (C) 2018 Oracle. All Rights Reserved. +# +# Author: Darrick J. Wong +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA. + +scrub_all=0 +conffile="@root_sysconfdir@/e2scrub.conf" + +test -f "${conffile}" && . "${conffile}" + +scrub_args="" + +print_help() { + echo "Usage: $0 [OPTIONS]" + echo " -A: Scrub all ext[234] filesystems even if not mounted." + echo " -r: Remove e2scrub snapshots." + echo " -V: Print version information and exit." +} + +print_version() { + echo "e2scrub_all @E2FSPROGS_VERSION@ (@E2FSPROGS_DATE@)" +} + +while getopts "ArV" opt; do + case "${opt}" in + "A") scrub_all=1;; + "r") scrub_args="${scrub_args} -r";; + "V") print_version; exit 0;; + *) print_help; exit 2;; + esac +done +shift "$((OPTIND - 1))" + +# Find scrub targets, make sure we only do this once. +ls_scrub_targets() { + lsblk -o NAME,FSTYPE,MOUNTPOINT -p -P -n | while read vars; do + eval "${vars}" + + # Skip non-ext[234] + case "${FSTYPE}" in + ext[234]) ;; + *) continue;; + esac + + # Skip unmounted filesystems unless -A + if [ "${scrub_all}" -eq 0 ] && [ -z "${MOUNTPOINT}" ]; then + continue; + fi + + # Skip non-lvm devices and lvm snapshots + lvm_vars="$(lvs --nameprefixes -o vg_name,lv_name,lv_role --noheadings "${NAME}" 2> /dev/null)" + test $? -ne 0 && continue + eval "${lvm_vars}" + echo "${LVM2_LV_ROLE}" | grep -q "snapshot" && continue + + if [ -n "${MOUNTPOINT}" ]; then + echo "${MOUNTPOINT}" + else + echo "${NAME}" + fi + done | sort | uniq +} + +# Scrub any mounted fs on lvm by creating a snapshot and fscking that. +stdin="$(realpath /dev/stdin)" +ls_scrub_targets | while read tgt; do + ${DBG} "@root_sbindir@/e2scrub" ${scrub_args} "${tgt}" < "${stdin}" +done + +exit 0 From patchwork Sat Mar 24 01:57:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Darrick Wong X-Patchwork-Id: 890368 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.b="dbFsJkzg"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 407Np15Pf0z9s12 for ; Sat, 24 Mar 2018 12:57:29 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751821AbeCXB53 (ORCPT ); Fri, 23 Mar 2018 21:57:29 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:47804 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751675AbeCXB51 (ORCPT ); Fri, 23 Mar 2018 21:57:27 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w2O1tupT189391; Sat, 24 Mar 2018 01:57:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2017-10-26; bh=1Bbha+jRyAJKb91IiALvhRCY/2AN7AYUC5l3xCAG+5g=; b=dbFsJkzg2Dr4wL6uP0lEL2OMcVYkBjg2+qJjThNLmbDgLAf/x6McwLxsX2W5N3lY9R8J gmDIj6dJZI9Th5LIs7NDFcYFMGk9wH4KJcUUneNOZQRHgw87wzmWs9JrVQ8vpej0ltNy smku0yUhXhiMaAglCcQx0Bqyl+sVz7mFoG7CKLjBPx1oq1G0/QaJ1Zeeewacl8iEAdOG uQNSLK3aWJNL8yUsPp8K3HvWoxH5bbwDi4nW871uatuWGPuPyKJrHVByVz7aA5wP0U2g 3XkzyXTtqnMGb5xgQNW6zxWFCCNoGG55orsBA3aF9pJOU4zrINDIeOkaGPQb11/miARe 9A== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp2130.oracle.com with ESMTP id 2gwded002p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:24 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1vO31024869 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:24 GMT Received: from abhmp0016.oracle.com (abhmp0016.oracle.com [141.146.116.22]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1vNva023660; Sat, 24 Mar 2018 01:57:23 GMT Received: from localhost (/10.159.132.121) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 23 Mar 2018 18:57:23 -0700 Subject: [PATCH 4/4] e2scrub: add service (cron, systemd) support From: "Darrick J. Wong" To: tytso@mit.edu, darrick.wong@oracle.com Cc: linux-ext4@vger.kernel.org Date: Fri, 23 Mar 2018 18:57:22 -0700 Message-ID: <152185664204.10434.16107479107033301787.stgit@magnolia> In-Reply-To: <152185661631.10434.3295057734679624788.stgit@magnolia> References: <152185661631.10434.3295057734679624788.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8841 signatures=668695 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=9 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1803240010 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Darrick J. Wong Add the ability to run the e2scrub utilities as a periodically scheduled system service. Signed-off-by: Darrick J. Wong --- MCONFIG.in | 5 + configure | 179 ++++++++++++++++++++++++++++++++++++++++ configure.ac | 74 ++++++++++++++++- debian/e2fsprogs.files | 3 + debian/e2fsprogs.postinst | 19 ++++ debian/e2fsprogs.postrm | 35 ++++++++ scrub/Makefile.in | 85 +++++++++++++++++++ scrub/e2scrub.in | 36 ++++++-- scrub/e2scrub@.service.in | 20 ++++ scrub/e2scrub_all.cron.in | 2 scrub/e2scrub_all.in | 53 +++++++++++- scrub/e2scrub_all.service.in | 10 ++ scrub/e2scrub_all.timer.in | 11 ++ scrub/e2scrub_all_cron.in | 68 +++++++++++++++ scrub/e2scrub_fail.in | 25 ++++++ scrub/e2scrub_fail@.service.in | 10 ++ scrub/e2scrub_reap.service.in | 21 +++++ util/subst.conf.in | 3 + 18 files changed, 646 insertions(+), 13 deletions(-) create mode 100644 debian/e2fsprogs.postrm create mode 100644 scrub/e2scrub@.service.in create mode 100644 scrub/e2scrub_all.cron.in create mode 100644 scrub/e2scrub_all.service.in create mode 100644 scrub/e2scrub_all.timer.in create mode 100644 scrub/e2scrub_all_cron.in create mode 100644 scrub/e2scrub_fail.in create mode 100644 scrub/e2scrub_fail@.service.in create mode 100644 scrub/e2scrub_reap.service.in diff --git a/MCONFIG.in b/MCONFIG.in index adeb5bd..ee83554 100644 --- a/MCONFIG.in +++ b/MCONFIG.in @@ -32,9 +32,14 @@ man8dir = $(mandir)/man8 infodir = @infodir@ datadir = @datadir@ pkgconfigdir = $(libdir)/pkgconfig +pkglibdir = $(libdir)/e2fsprogs HAVE_UDEV = @have_udev@ UDEV_RULES_DIR = @pkg_udev_rules_dir@ +HAVE_CROND = @have_crond@ +CROND_DIR = @crond_dir@ +HAVE_SYSTEMD = @have_systemd@ +SYSTEMD_SYSTEM_UNIT_DIR = @systemd_system_unit_dir@ @SET_MAKE@ diff --git a/configure b/configure index 986e057..f6a0dea 100755 --- a/configure +++ b/configure @@ -625,6 +625,12 @@ gl_use_threads_default= ac_func_list= ac_subst_vars='LTLIBOBJS LIBOBJS +systemd_system_unit_dir +have_systemd +systemd_LIBS +systemd_CFLAGS +crond_dir +have_crond pkg_udev_rules_dir have_udev udev_LIBS @@ -899,6 +905,8 @@ with_libintl_prefix enable_fuse2fs with_multiarch with_udev_rules_dir +with_crond_dir +with_systemd_unit_dir ' ac_precious_vars='build_alias host_alias @@ -913,7 +921,9 @@ PKG_CONFIG PKG_CONFIG_PATH PKG_CONFIG_LIBDIR udev_CFLAGS -udev_LIBS' +udev_LIBS +systemd_CFLAGS +systemd_LIBS' # Initialize some variables set by options. @@ -1591,6 +1601,9 @@ Optional Packages: --with-multiarch=ARCH specify the multiarch triplet --with-udev-rules-dir[=DIR] Install udev rules into DIR. + --with-crond-dir[=DIR] Install system crontabs into DIR. + --with-systemd-unit-dir[=DIR] + Install systemd system units into DIR. Some influential environment variables: CC C compiler command @@ -1608,6 +1621,10 @@ Some influential environment variables: path overriding pkg-config's built-in search path udev_CFLAGS C compiler flags for udev, overriding pkg-config udev_LIBS linker flags for udev, overriding pkg-config + systemd_CFLAGS + C compiler flags for systemd, overriding pkg-config + systemd_LIBS + linker flags for systemd, overriding pkg-config Use these variables to override the choices made by `configure' or to help it to find libraries and programs with nonstandard names/locations. @@ -13803,6 +13820,7 @@ else fi fi + { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether we can link with -static" >&5 $as_echo_n "checking whether we can link with -static... " >&6; } if ${ac_cv_e2fsprogs_use_static+:} false; then : @@ -14009,6 +14027,165 @@ fi + +# Check whether --with-crond_dir was given. +if test "${with_crond_dir+set}" = set; then : + withval=$with_crond_dir; +else + with_crond_dir=yes +fi + +if test "x${with_crond_dir}" != "xno"; then : + + if test "x${with_crond_dir}" = "xyes"; then : + + if test -d "/etc/cron.d"; then : + with_crond_dir="/etc/cron.d" +fi + +fi + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for system crontab dir" >&5 +$as_echo_n "checking for system crontab dir... " >&6; } + crond_dir="${with_crond_dir}" + if test -n "${crond_dir}"; then : + + { $as_echo "$as_me:${as_lineno-$LINENO}: result: ${crond_dir}" >&5 +$as_echo "${crond_dir}" >&6; } + have_crond="yes" + +else + + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + have_crond="no" + +fi + +else + + have_crond="disabled" + +fi + + + + +# Check whether --with-systemd_unit_dir was given. +if test "${with_systemd_unit_dir+set}" = set; then : + withval=$with_systemd_unit_dir; +else + with_systemd_unit_dir=yes +fi + +if test "x${with_systemd_unit_dir}" != "xno"; then : + + if test "x${with_systemd_unit_dir}" = "xyes"; then : + + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for systemd" >&5 +$as_echo_n "checking for systemd... " >&6; } + +if test -n "$systemd_CFLAGS"; then + pkg_cv_systemd_CFLAGS="$systemd_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"systemd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "systemd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_systemd_CFLAGS=`$PKG_CONFIG --cflags "systemd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$systemd_LIBS"; then + pkg_cv_systemd_LIBS="$systemd_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"systemd\""; } >&5 + ($PKG_CONFIG --exists --print-errors "systemd") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_systemd_LIBS=`$PKG_CONFIG --libs "systemd" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + systemd_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "systemd" 2>&1` + else + systemd_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "systemd" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$systemd_PKG_ERRORS" >&5 + + + with_systemd_unit_dir="" + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + with_systemd_unit_dir="" + +else + systemd_CFLAGS=$pkg_cv_systemd_CFLAGS + systemd_LIBS=$pkg_cv_systemd_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + + with_systemd_unit_dir="$($PKG_CONFIG --variable=systemdsystemunitdir systemd)" + +fi + + +fi + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for systemd system unit dir" >&5 +$as_echo_n "checking for systemd system unit dir... " >&6; } + systemd_system_unit_dir="${with_systemd_unit_dir}" + if test -n "${systemd_system_unit_dir}"; then : + + { $as_echo "$as_me:${as_lineno-$LINENO}: result: ${systemd_system_unit_dir}" >&5 +$as_echo "${systemd_system_unit_dir}" >&6; } + have_systemd="yes" + +else + + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + have_systemd="no" + +fi + +else + + have_systemd="disabled" + +fi + + + test -d lib || mkdir lib test -d include || mkdir include test -d include/linux || mkdir include/linux diff --git a/configure.ac b/configure.ac index 6e549c5..5a2c8be 100644 --- a/configure.ac +++ b/configure.ac @@ -1392,7 +1392,8 @@ else libdir=$libdir/$withval root_libdir=$root_libdir/$withval fi -)dnl +) +dnl dnl dnl See if -static works. This could fail if the linker does not dnl support -static, or if required external libraries are not available @@ -1514,6 +1515,77 @@ AC_SUBST(have_udev) AC_SUBST(pkg_udev_rules_dir) dnl +dnl Where do cron jobs go? +dnl +AC_ARG_WITH([crond_dir], + [AS_HELP_STRING([--with-crond-dir@<:@=DIR@:>@], + [Install system crontabs into DIR.])], + [], + [with_crond_dir=yes]) +AS_IF([test "x${with_crond_dir}" != "xno"], + [ + AS_IF([test "x${with_crond_dir}" = "xyes"], + [ + AS_IF([test -d "/etc/cron.d"], + [with_crond_dir="/etc/cron.d"]) + ]) + AC_MSG_CHECKING([for system crontab dir]) + crond_dir="${with_crond_dir}" + AS_IF([test -n "${crond_dir}"], + [ + AC_MSG_RESULT(${crond_dir}) + have_crond="yes" + ], + [ + AC_MSG_RESULT(no) + have_crond="no" + ]) + ], + [ + have_crond="disabled" + ]) +AC_SUBST(have_crond) +AC_SUBST(crond_dir) + +dnl +dnl Where do systemd services go? +dnl +AC_ARG_WITH([systemd_unit_dir], + [AS_HELP_STRING([--with-systemd-unit-dir@<:@=DIR@:>@], + [Install systemd system units into DIR.])], + [], + [with_systemd_unit_dir=yes]) +AS_IF([test "x${with_systemd_unit_dir}" != "xno"], + [ + AS_IF([test "x${with_systemd_unit_dir}" = "xyes"], + [ + PKG_CHECK_MODULES([systemd], [systemd], + [ + with_systemd_unit_dir="$($PKG_CONFIG --variable=systemdsystemunitdir systemd)" + ], [ + with_systemd_unit_dir="" + ]) + m4_pattern_allow([^PKG_(MAJOR|MINOR|BUILD|REVISION)$]) + ]) + AC_MSG_CHECKING([for systemd system unit dir]) + systemd_system_unit_dir="${with_systemd_unit_dir}" + AS_IF([test -n "${systemd_system_unit_dir}"], + [ + AC_MSG_RESULT(${systemd_system_unit_dir}) + have_systemd="yes" + ], + [ + AC_MSG_RESULT(no) + have_systemd="no" + ]) + ], + [ + have_systemd="disabled" + ]) +AC_SUBST(have_systemd) +AC_SUBST(systemd_system_unit_dir) + +dnl dnl Make our output files, being sure that we create the some miscellaneous dnl directories dnl diff --git a/debian/e2fsprogs.files b/debian/e2fsprogs.files index 78720fe..e0e49ce 100644 --- a/debian/e2fsprogs.files +++ b/debian/e2fsprogs.files @@ -1,6 +1,9 @@ sbin usr/bin +usr/lib usr/sbin usr/share/man etc lib/udev/rules.d +lib/systemd/system +usr/lib diff --git a/debian/e2fsprogs.postinst b/debian/e2fsprogs.postinst index 00ac363..e7acb0e 100644 --- a/debian/e2fsprogs.postinst +++ b/debian/e2fsprogs.postinst @@ -10,4 +10,23 @@ fi #DEBHELPER# +# debhelper doesn't know what timers are... +update_svc() { + deb-systemd-helper unmask "$1" >/dev/null || true + + if deb-systemd-helper --quiet was-enabled "$1"; then + deb-systemd-helper enable "$1" >/dev/null || true + else + deb-systemd-helper update-state "$1" >/dev/null || true + fi +} +update_svc e2scrub_all.timer +update_svc e2scrub_reap.service + +# Start our new services +if [ -d /run/systemd/system ]; then + systemctl --system daemon-reload >/dev/null || true + deb-systemd-invoke start e2scrub_all.timer >/dev/null || true +fi + exit 0 diff --git a/debian/e2fsprogs.postrm b/debian/e2fsprogs.postrm new file mode 100644 index 0000000..32cb642 --- /dev/null +++ b/debian/e2fsprogs.postrm @@ -0,0 +1,35 @@ +#!/bin/sh + +update_svc() { + deb-systemd-helper mask "$1" >/dev/null || true + + if deb-systemd-helper --quiet was-enabled "$1"; then + # Enables the unit on first installation, creates new + # symlinks on upgrades if the unit file has changed. + deb-systemd-helper disable "$1" >/dev/null || true + fi +} + +if [ "$1" != "upgrade" ]; then + # Abort on error. + set -e + + if [ -x /usr/sbin/update-initramfs -a \ + -e /etc/initramfs-tools/initramfs.conf ]; then + update-initramfs -u + fi + + #DEBHELPER# + + # debhelper doesn't know what timers are... + update_svc e2scrub_all.timer + update_svc e2scrub_reap.service + + # Start our new services + if [ -d /run/systemd/system ]; then + deb-systemd-invoke stop e2scrub_all.timer >/dev/null || true + fi +fi + +exit 0 + diff --git a/scrub/Makefile.in b/scrub/Makefile.in index 1744941..f58331d 100644 --- a/scrub/Makefile.in +++ b/scrub/Makefile.in @@ -22,7 +22,23 @@ INSTALL_TGT += install-udev UNINSTALL_TGT += uninstall-udev endif -all:: $(PROGS) $(MANPAGES) $(CONFFILES) $(UDEV_RULES) +ifeq ($(HAVE_CROND),yes) +CRONTABS = e2scrub_all.cron +LIBPROGS += e2scrub_all_cron +INSTALLDIRS_TGT += installdirs-crond installdirs-libprogs +INSTALL_TGT += install-crond install-libprogs +UNINSTALL_TGT += uninstall-crond uninstall-libprogs +endif + +ifeq ($(HAVE_SYSTEMD),yes) +SERVICE_FILES = e2scrub@.service e2scrub_all.service e2scrub_all.timer e2scrub_fail@.service e2scrub_reap.service +LIBPROGS += e2scrub_fail +INSTALLDIRS_TGT += installdirs-systemd installdirs-libprogs +INSTALL_TGT += install-systemd install-libprogs +UNINSTALL_TGT += uninstall-systemd uninstall-libprogs +endif + +all:: $(PROGS) $(MANPAGES) $(CONFFILES) $(UDEV_RULES) $(SERVICE_FILES) $(CRONTABS) $(LIBPROGS) e2scrub: $(DEP_SUBSTITUTE) e2scrub.in $(E) " SUBST $@" @@ -34,6 +50,16 @@ e2scrub_all: e2scrub_all.in $(Q) $(SUBSTITUTE_UPTIME) $(srcdir)/e2scrub_all.in $@ $(Q) chmod a+x $@ +e2scrub_fail: e2scrub_fail.in + $(E) " SUBST $@" + $(Q) $(SUBSTITUTE_UPTIME) $(srcdir)/e2scrub_fail.in $@ + $(Q) chmod a+x $@ + +e2scrub_all_cron: e2scrub_all_cron.in + $(E) " SUBST $@" + $(Q) $(SUBSTITUTE_UPTIME) $(srcdir)/e2scrub_all_cron.in $@ + $(Q) chmod a+x $@ + %.8: %.8.in $(DEP_SUBSTITUTE) $(E) " SUBST $@" $(Q) $(SUBSTITUTE_UPTIME) $< $@ @@ -46,10 +72,34 @@ e2scrub_all: e2scrub_all.in $(E) " SUBST $@" $(Q) $(SUBSTITUTE_UPTIME) $< $@ +%.service: %.service.in $(DEP_SUBSTITUTE) + $(E) " SUBST $@" + $(Q) $(SUBSTITUTE_UPTIME) $< $@ + +%.cron: %.cron.in $(DEP_SUBSTITUTE) + $(E) " SUBST $@" + $(Q) $(SUBSTITUTE_UPTIME) $< $@ + +%.timer: %.timer.in $(DEP_SUBSTITUTE) + $(E) " SUBST $@" + $(Q) $(SUBSTITUTE_UPTIME) $< $@ + installdirs-udev: $(E) " MKINSTALLDIRS $(UDEV_RULES_DIR)" $(Q) $(MKINSTALLDIRS) $(DESTDIR)$(UDEV_RULES_DIR) +installdirs-crond: + $(E) " MKINSTALLDIRS $(CROND_DIR)" + $(Q) $(MKINSTALLDIRS) $(DESTDIR)$(CROND_DIR) + +installdirs-libprogs: + $(E) " MKINSTALLDIRS $(pkglibdir)" + $(Q) $(MKINSTALLDIRS) $(DESTDIR)$(pkglibdir) + +installdirs-systemd: + $(E) " MKINSTALLDIRS $(SYSTEMD_SYSTEM_UNIT_DIR)" + $(Q) $(MKINSTALLDIRS) $(DESTDIR)$(SYSTEMD_SYSTEM_UNIT_DIR) + installdirs: $(INSTALLDIRS_TGT) $(E) " MKINSTALLDIRS $(root_sbindir) $(man8dir) $(root_sysconfdir)" $(Q) $(MKINSTALLDIRS) $(DESTDIR)$(root_sbindir) \ @@ -61,6 +111,24 @@ install-udev: $(INSTALL_PROGRAM) $$i $(DESTDIR)$(UDEV_RULES_DIR)/96-$$i; \ done +install-crond: + $(Q) for i in $(CRONTABS); do \ + $(ES) " INSTALL $(CROND_DIR)/$$i"; \ + $(INSTALL_PROGRAM) $$i $(DESTDIR)$(CROND_DIR)/$$i; \ + done + +install-libprogs: $(LIBPROGS) + $(Q) for i in $(LIBPROGS); do \ + $(ES) " INSTALL $(pkglibdir)/$$i"; \ + $(INSTALL_PROGRAM) $$i $(DESTDIR)$(pkglibdir)/$$i; \ + done + +install-systemd: $(SERVICE_FILES) + $(Q) for i in $(SERVICE_FILES); do \ + $(ES) " INSTALL_DATA $(SYSTEMD_SYSTEM_UNIT_DIR)/$$i"; \ + $(INSTALL_DATA) $$i $(DESTDIR)$(SYSTEMD_SYSTEM_UNIT_DIR)/$$i; \ + done + install: $(PROGS) $(MANPAGES) $(FMANPAGES) installdirs $(INSTALL_TGT) $(Q) for i in $(PROGS); do \ $(ES) " INSTALL $(root_sbindir)/$$i"; \ @@ -83,6 +151,21 @@ uninstall-udev: $(RM) -f $(DESTDIR)$(UDEV_RULES_DIR)/96-$$i; \ done +uninstall-crond: + for i in $(CRONTABS); do \ + $(RM) -f $(DESTDIR)$(CROND_DIR)/$$i; \ + done + +uninstall-libprogs: + for i in $(LIBPROGS); do \ + $(RM) -f $(DESTDIR)$(pkglibdir)/$$i; \ + done + +uninstall-systemd: + for i in $(SERVICE_FILES); do \ + $(RM) -f $(DESTDIR)$(SYSTEMD_SYSTEM_UNIT_DIR)/$$i; \ + done + uninstall: $(UNINSTALL_TGT) for i in $(PROGS); do \ $(RM) -f $(DESTDIR)$(root_sbindir)/$$i; \ diff --git a/scrub/e2scrub.in b/scrub/e2scrub.in index 08b3600..e1965db 100644 --- a/scrub/e2scrub.in +++ b/scrub/e2scrub.in @@ -44,12 +44,34 @@ print_version() { echo "e2scrub @E2FSPROGS_VERSION@ (@E2FSPROGS_DATE@)" } +exitcode() { + ret="$1" + + # If we're being run as a service, the return code must fit the LSB + # init script action error guidelines, which is to say that we + # compress all errors to 1 ("generic or unspecified error", LSB 5.0 + # section 22.2) and hope the admin will scan the log for what + # actually happened. + + # We have to sleep 2 seconds here because journald uses the pid to + # connect our log messages to the systemd service. This is critical + # for capturing all the log messages if the scrub fails, because the + # fail service uses the service name to gather log messages for the + # error report. + if [ -n "${SERVICE_MODE}" ]; then + test "${ret}" -ne 0 && ret=1 + sleep 2 + fi + + exit "${ret}" +} + while getopts "rtV" opt; do case "${opt}" in "r") reap=1;; "t") fstrim=1;; - "V") print_version; exit 0;; - *) print_help; exit 2;; + "V") print_version; exitcode 0;; + *) print_help; exitcode 2;; esac done shift "$((OPTIND - 1))" @@ -57,7 +79,7 @@ shift "$((OPTIND - 1))" arg="$1" if [ -z "${arg}" ]; then print_help - exit 1 + exitcode 1 fi # Find the device for a given mountpoint @@ -112,7 +134,7 @@ fi if [ ! -e "${dev}" ]; then echo "${arg}: Not an ext[234] filesystem." print_help - exit 16 + exitcode 16 fi # Make sure this is an LVM device we can snapshot @@ -122,7 +144,7 @@ if [ -z "${LVM2_VG_NAME}" ] || [ -z "${LVM2_LV_NAME}" ] || echo "${LVM2_LV_ROLE}" | grep -q "snapshot"; then echo "${arg}: Not connnected to a LVM logical volume." print_help - exit 16 + exitcode 16 fi start_time="$(date +'%Y%m%d%H%M%S')" snap="${LVM2_LV_NAME}.e2scrub" @@ -185,7 +207,7 @@ if [ "${reap}" -gt 0 ]; then exit 0 fi if ! setup; then - exit 8 + exitcode 8 fi trap "teardown; exit 1" EXIT INT QUIT TERM @@ -236,4 +258,4 @@ case "$?" in ;; esac -exit "${ret}" +exitcode "${ret}" diff --git a/scrub/e2scrub@.service.in b/scrub/e2scrub@.service.in new file mode 100644 index 0000000..496f894 --- /dev/null +++ b/scrub/e2scrub@.service.in @@ -0,0 +1,20 @@ +[Unit] +Description=Online ext4 Metadata Check for %I +OnFailure=e2scrub_fail@%i.service +Documentation=man:e2scrub(8) + +[Service] +Type=oneshot +WorkingDirectory=/ +PrivateNetwork=true +ProtectSystem=true +ProtectHome=read-only +PrivateTmp=yes +AmbientCapabilities=CAP_SYS_ADMIN CAP_SYS_RAWIO +NoNewPrivileges=yes +User=root +IOSchedulingClass=idle +CPUSchedulingPolicy=idle +Environment=SERVICE_MODE=1 +ExecStart=@root_sbindir@/e2scrub -t %I +SyslogIdentifier=%N diff --git a/scrub/e2scrub_all.cron.in b/scrub/e2scrub_all.cron.in new file mode 100644 index 0000000..7d42c3f --- /dev/null +++ b/scrub/e2scrub_all.cron.in @@ -0,0 +1,2 @@ +30 3 * * 0 root test -e /run/systemd/system || @pkglibdir@/e2scrub_all_cron +10 3 * * * root test -e /run/systemd/system || @root_sbindir@/e2scrub_all -A -r diff --git a/scrub/e2scrub_all.in b/scrub/e2scrub_all.in index b9e5dea..9581dc2 100644 --- a/scrub/e2scrub_all.in +++ b/scrub/e2scrub_all.in @@ -36,12 +36,34 @@ print_version() { echo "e2scrub_all @E2FSPROGS_VERSION@ (@E2FSPROGS_DATE@)" } +exitcode() { + ret="$1" + + # If we're being run as a service, the return code must fit the LSB + # init script action error guidelines, which is to say that we + # compress all errors to 1 ("generic or unspecified error", LSB 5.0 + # section 22.2) and hope the admin will scan the log for what + # actually happened. + + # We have to sleep 2 seconds here because journald uses the pid to + # connect our log messages to the systemd service. This is critical + # for capturing all the log messages if the scrub fails, because the + # fail service uses the service name to gather log messages for the + # error report. + if [ -n "${SERVICE_MODE}" ]; then + test "${ret}" -ne 0 && ret=1 + sleep 2 + fi + + exit "${ret}" +} + while getopts "ArV" opt; do case "${opt}" in "A") scrub_all=1;; "r") scrub_args="${scrub_args} -r";; - "V") print_version; exit 0;; - *) print_help; exit 2;; + "V") print_version; exitcode 0;; + *) print_help; exitcode 2;; esac done shift "$((OPTIND - 1))" @@ -76,10 +98,35 @@ ls_scrub_targets() { done | sort | uniq } +# systemd doesn't know to do path escaping on the instance variable we pass +# to the e2scrub service, which breaks things if there is a dash in the path +# name. Therefore, do the path escaping ourselves if needed. +escape_path_for_systemd() { + local path="$1" + + if echo "${path}" | grep -q -- "-"; then + echo "-$(systemd-escape --path "${path}")" + else + echo "${path}" + fi +} + # Scrub any mounted fs on lvm by creating a snapshot and fscking that. stdin="$(realpath /dev/stdin)" ls_scrub_targets | while read tgt; do + # If we're not reaping and systemd is present, try invoking the + # systemd service. + if [ -z "${scrub_args}" ] && type systemctl > /dev/null 2>&1; then + tgt_esc="$(escape_path_for_systemd "${tgt}")" + ${DBG} systemctl start "e2scrub@${tgt_esc}" 2> /dev/null < "${stdin}" + res=$? + if [ "${res}" -eq 0 ] || [ "${res}" -eq 1 ]; then + continue; + fi + fi + + # Otherwise use direct invocation ${DBG} "@root_sbindir@/e2scrub" ${scrub_args} "${tgt}" < "${stdin}" done -exit 0 +exitcode 0 diff --git a/scrub/e2scrub_all.service.in b/scrub/e2scrub_all.service.in new file mode 100644 index 0000000..bc05184 --- /dev/null +++ b/scrub/e2scrub_all.service.in @@ -0,0 +1,10 @@ +[Unit] +Description=Online ext4 Metadata Check for All Filesystems +ConditionACPower=true +Documentation=man:e2scrub_all(8) + +[Service] +Type=oneshot +Environment=SERVICE_MODE=1 +ExecStart=@root_sbindir@/e2scrub_all +SyslogIdentifier=e2scrub_all diff --git a/scrub/e2scrub_all.timer.in b/scrub/e2scrub_all.timer.in new file mode 100644 index 0000000..3d558bb --- /dev/null +++ b/scrub/e2scrub_all.timer.in @@ -0,0 +1,11 @@ +[Unit] +Description=Periodic ext4 Online Metadata Check for All Filesystems + +[Timer] +# Run on Sunday at 3:10am, to avoid running afoul of DST changes +OnCalendar=Sun *-*-* 03:10:00 +RandomizedDelaySec=60 +Persistent=true + +[Install] +WantedBy=timers.target diff --git a/scrub/e2scrub_all_cron.in b/scrub/e2scrub_all_cron.in new file mode 100644 index 0000000..f9cff87 --- /dev/null +++ b/scrub/e2scrub_all_cron.in @@ -0,0 +1,68 @@ +#!/bin/bash + +# Copyright (C) 2018 Oracle. All Rights Reserved. +# +# Author: Darrick J. Wong +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA. + +# Run e2scrub_all from a cronjob if we don't have systemd and we're not +# running on AC power. + +on_ac_power() { + local any_known=no + + # try sysfs power class first + if [ -d /sys/class/power_supply ]; then + for psu in /sys/class/power_supply/*; do + if [ -r "$psu/type" ]; then + type=$(cat "$psu/type") + + # ignore batteries + [ "$type" = "Battery" ] && continue + + online=$(cat "$psu/online") + + [ "$online" = 1 ] && return 0 + [ "$online" = 0 ] && any_known=yes + fi + done + + [ "$any_known" = "yes" ] && return 1 + fi + + # else fall back to AC adapters in /proc + if [ -d /proc/acpi/ac_adapter ]; then + for ac in /proc/acpi/ac_adapter/*; do + if [ -r "$ac/state" ]; then + grep -q on-line "$ac/state" && return 0 + grep -q off-line "$ac/state" && any_known=yes + elif [ -r "$ac/status" ]; then + grep -q on-line "$ac/status" && return 0 + grep -q off-line "$ac/status" && any_known=yes + fi + done + + [ "$any_known" = "yes" ] && return 1 + fi + + # Can't tell, just assume we're on AC. + return 0 +} + +test -e /run/systemd/system && exit 0 +on_ac_power || exit 0 + +exec @root_sbindir@/e2scrub_all diff --git a/scrub/e2scrub_fail.in b/scrub/e2scrub_fail.in new file mode 100644 index 0000000..f27197a --- /dev/null +++ b/scrub/e2scrub_fail.in @@ -0,0 +1,25 @@ +#!/bin/bash + +# Email logs of failed e2scrub unit runs when the systemd service fails. + +recipient="$1" +test -z "${recipient}" && exit 0 +device="$2" +test -z "${device}" && exit 0 +hostname="$(hostname -f 2>/dev/null)" +test -z "${hostname}" && hostname="${HOSTNAME}" +if ! type sendmail > /dev/null 2>&1; then + echo "$0: sendmail program not found." + exit 1 +fi + +(cat << ENDL +To: $1 +From: +Subject: e2scrub failure on ${device} + +So sorry, the automatic e2scrub of ${device} on ${hostname} failed. + +A log of what happened follows: +ENDL +systemctl status --full --lines 4294967295 "e2scrub@${device}") | sendmail -t -i diff --git a/scrub/e2scrub_fail@.service.in b/scrub/e2scrub_fail@.service.in new file mode 100644 index 0000000..df87949 --- /dev/null +++ b/scrub/e2scrub_fail@.service.in @@ -0,0 +1,10 @@ +[Unit] +Description=Online ext4 Metadata Check Failure Reporting for %I + +[Service] +Type=oneshot +Environment=EMAIL_ADDR=root +ExecStart=@pkglibdir@/e2scrub_fail "${EMAIL_ADDR}" %I +User=mail +Group=mail +SupplementaryGroups=systemd-journal diff --git a/scrub/e2scrub_reap.service.in b/scrub/e2scrub_reap.service.in new file mode 100644 index 0000000..8a32077 --- /dev/null +++ b/scrub/e2scrub_reap.service.in @@ -0,0 +1,21 @@ +[Unit] +Description=Remove Stale Online ext4 Metadata Check Snapshots + +[Service] +Type=oneshot +WorkingDirectory=/ +PrivateNetwork=true +ProtectSystem=true +ProtectHome=read-only +PrivateTmp=yes +AmbientCapabilities=CAP_SYS_ADMIN CAP_SYS_RAWIO +NoNewPrivileges=yes +User=root +IOSchedulingClass=idle +CPUSchedulingPolicy=idle +ExecStart=@root_sbindir@/e2scrub_all -A -r +SyslogIdentifier=%N +RemainAfterExit=no + +[Install] +WantedBy=default.target diff --git a/util/subst.conf.in b/util/subst.conf.in index 6bf658d..0da4554 100644 --- a/util/subst.conf.in +++ b/util/subst.conf.in @@ -21,3 +21,6 @@ JDEV TDB_MAN_COMMENT @TDB_MAN_COMMENT@ root_sbindir @root_sbindir@ root_bindir @root_bindir@ +libdir @libdir@ +$exec_prefix @exec_prefix@ +pkglibdir @libdir@/e2fsprogs