From patchwork Wed Jun 27 15:57:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Artem Blagodarenko X-Patchwork-Id: 935586 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="fZwQUq7F"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41G749152Hz9s1B for ; Thu, 28 Jun 2018 02:03:21 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965634AbeF0QDU (ORCPT ); Wed, 27 Jun 2018 12:03:20 -0400 Received: from mail-lf0-f66.google.com ([209.85.215.66]:44072 "EHLO mail-lf0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S964786AbeF0QDT (ORCPT ); Wed, 27 Jun 2018 12:03:19 -0400 Received: by mail-lf0-f66.google.com with SMTP id j26-v6so1953546lfb.11 for ; Wed, 27 Jun 2018 09:03:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=RClWOjl60Vl5zKwvR99yPFGrV9P21+rKC75krDY5ImE=; b=fZwQUq7FqvjS0gMxGBLLj6mGF1KUQcZWD3uEI6TaZqth+QDhT7bxEvhsiskr7X2e/B vJwmYeKYWL62PYcTMsEk1aB+NvoG0SnTuSXSOF7WPjmTHPHhlJEhSvj4TIeVBl+sVaaR KOTowqK1VZeVp1k5l9R0vBDJx+4oS0DNsGGyy9XrMQpOK3tv28eLHfpr543Nrdx7f2iL sOUxIoxVmNBi8HFzW8sxGRYZbRLM6eYQHhV/eUzTPYwZXpn2Eg86Eg97D0ffA/TaVHCE Dq/FXSS/gU04YmJVFO4DuW1rAvTx6cIF4u8Is9q7JGH4lDut4FL/YpwShu+JJsmltnlK PMqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=RClWOjl60Vl5zKwvR99yPFGrV9P21+rKC75krDY5ImE=; b=Jc4aaIcfYFrsbEV25Nc2+UOInfrPddyhlQVdc9hQV9xluq1k/dzKu2tLWrfpPTQoL0 0NkNQ7UuAgQAWyTi+JC6dqoYl22uocFftkC8GJ/H+dW7N5TX9G77jqQUqYHIZDMnr1Rc gr9reB7dzmeCAm7LyPkvNxplriX/YthcmljixPCa8xI/G7g/u6MUIJMEZog1koF17GT9 7IUP0M3RVY4dfqP9ObDqa3U1Oa0jROAl9mLcVYUbdMWT4WZUvhYrfu7n9S/e0pwI+aG/ LEXkxWyKVKTs8qFpg/z/3xGpa9uFpab57qvP+xIDsvteDQEnFLDXjWquSnDxMBciSoyr jX4A== X-Gm-Message-State: APt69E00zsxRk4JiGobcI/W9cWHV+J7HARivx7N1ZV+9ttA0vgbnuCbH z5/dPG61QC+nOR8zOgUDh43GTLX7 X-Google-Smtp-Source: AAOMgpd31a6FSPDV1OkDDnff0h9B6M3KXgiffSQjS8jIz3lIDLxIhCBljSSmFkmtejbCBJzTP1ctUg== X-Received: by 2002:a19:4b90:: with SMTP id y138-v6mr3623743lfa.118.1530115397557; Wed, 27 Jun 2018 09:03:17 -0700 (PDT) Received: from localhost.localdomain ([95.153.129.199]) by smtp.gmail.com with ESMTPSA id r21-v6sm748840lja.92.2018.06.27.09.03.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 09:03:16 -0700 (PDT) From: c17828 X-Google-Original-From: c17828 To: linux-ext4@vger.kernel.org Cc: adilger.kernel@dilger.ca, alexey.lyashkov@gmail.com, Andreas Dilger , Artem Blagodarenko Subject: [PATCH] filefrag: Lustre changes to filefrag FIEMAP handling Date: Wed, 27 Jun 2018 18:57:31 +0300 Message-Id: <20180627155731.3248-1-c17828@cray.com> X-Mailer: git-send-email 2.14.3 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Andreas Dilger Add support for multiple-device filesystems by defining a new fe_device field in the fiemap_extent structure. This allows printing the filesystem-relative or linux block device number associated with each extent of a file. If a single filesystem extent is mirrored to multiple block devices, the fe_device field can be used to disambiguate the multiple copies. If the "-l" (device-logical) option is given to filefrag, then all extents for a particular device of a file are returned before returning extents for the next device. This makes it easier to see if extent allocation within a single device is contiguous, instead of returning all of the blocks of a file interleaved in file-logical-offset order. Change-Id: Icdefe4dbc319e6652c3d6641e1500f2cfaf63605 Signed-off-by: Andreas Dilger Signed-off-by: Artem Blagodarenko --- lib/ext2fs/fiemap.h | 7 +++++- misc/filefrag.8.in | 5 +++- misc/filefrag.c | 67 +++++++++++++++++++++++++++++++++++++++++++---------- 3 files changed, 65 insertions(+), 14 deletions(-) diff --git a/lib/ext2fs/fiemap.h b/lib/ext2fs/fiemap.h index a331bc12..261a0b2b 100644 --- a/lib/ext2fs/fiemap.h +++ b/lib/ext2fs/fiemap.h @@ -19,7 +19,8 @@ struct fiemap_extent { __u64 fe_length; /* length in bytes for this extent */ __u64 fe_reserved64[2]; __u32 fe_flags; /* FIEMAP_EXTENT_* flags for this extent */ - __u32 fe_reserved[3]; + __u32 fe_device; /* device number (fs-specific if FIEMAP_EXTENT_NET)*/ + __u32 fe_reserved[2]; }; struct fiemap { @@ -42,6 +43,7 @@ struct fiemap { #define FIEMAP_FLAG_SYNC 0x00000001 /* sync file data before map */ #define FIEMAP_FLAG_XATTR 0x00000002 /* map extended attribute tree */ +#define FIEMAP_FLAG_DEVICE_ORDER 0x40000000 /* return device ordered mapping */ #define FIEMAP_FLAGS_COMPAT (FIEMAP_FLAG_SYNC | FIEMAP_FLAG_XATTR) @@ -67,4 +69,7 @@ struct fiemap { #define FIEMAP_EXTENT_SHARED 0x00002000 /* Space shared with other * files. */ +/* Network filesystem flags - use a high bit, don't conflict with upstream */ +#define FIEMAP_EXTENT_NET 0x80000000 /* Data stored remotely. */ + #endif /* _LINUX_FIEMAP_H */ diff --git a/misc/filefrag.8.in b/misc/filefrag.8.in index 292b3b7d..fdba67f7 100644 --- a/misc/filefrag.8.in +++ b/misc/filefrag.8.in @@ -8,7 +8,7 @@ filefrag \- report on file fragmentation .BI \-b blocksize ] [ -.B \-BeksvxX +.B \-BeklsvxX ] [ .I files... @@ -46,6 +46,9 @@ Print output in extent format, even for block-mapped files. .BI \-k Use 1024\-byte blocksize for output (identical to '\-b 1024'). .TP +.B \-l +Extents are displayed in device-logical offset order. +.TP .B \-s Sync the file before requesting the mapping. .TP diff --git a/misc/filefrag.c b/misc/filefrag.c index 9c57ab93..78e13076 100644 --- a/misc/filefrag.c +++ b/misc/filefrag.c @@ -54,14 +54,16 @@ int verbose = 0; int blocksize; /* Use specified blocksize (default 1kB) */ int sync_file = 0; /* fsync file before getting the mapping */ int xattr_map = 0; /* get xattr mapping */ -int force_bmap; /* force use of FIBMAP instead of FIEMAP */ +int force_bmap; /* force use of FIBMAP instead of FIEMAP */ int force_extent; /* print output in extent format always */ +int device_offset; /* extents report device-relative offsets */ int logical_width = 8; int physical_width = 10; const char *ext_fmt = "%4d: %*llu..%*llu: %*llu..%*llu: %6llu: %s\n"; const char *hex_fmt = "%4d: %*llx..%*llx: %*llx..%*llx: %6llx: %s\n"; -#define FILEFRAG_FIEMAP_FLAGS_COMPAT (FIEMAP_FLAG_SYNC | FIEMAP_FLAG_XATTR) +#define FILEFRAG_FIEMAP_FLAGS_COMPAT (FIEMAP_FLAG_SYNC | FIEMAP_FLAG_XATTR |\ + FIEMAP_FLAG_DEVICE_ORDER) #define FIBMAP _IO(0x00, 1) /* bmap access */ #define FIGETBSZ _IO(0x00, 2) /* get the block size used for bmap */ @@ -120,10 +122,10 @@ static void print_extent_header(void) { printf(" ext: %*s %*s length: %*s flags:\n", logical_width * 2 + 3, - "logical_offset:", + device_offset ? "device_logical:" : "logical_offset:", physical_width * 2 + 3, "physical_offset:", - physical_width + 1, - "expected:"); + device_offset ? 5 : physical_width + 1, + device_offset ? " dev:" : "expected:"); } static void print_flag(__u32 *flags, __u32 mask, char *buf, const char *name) @@ -159,11 +161,11 @@ static void print_extent_info(struct fiemap_extent *fm_extent, int cur_ex, physical_blk = fm_extent->fe_physical >> blk_shift; } - if (expected) - sprintf(flags, ext_fmt == hex_fmt ? "%*llx: " : "%*llu: ", + if (device_offset) + sprintf(flags, "%04x: ", fm_extent->fe_device); + else if (expected) + sprintf(flags, ext_fmt == hex_fmt ? "%*llx:" : "%*llu: ", physical_width, expected >> blk_shift); - else - sprintf(flags, "%.*s ", physical_width, " "); fe_flags = fm_extent->fe_flags; print_flag(&fe_flags, FIEMAP_EXTENT_LAST, flags, "last,"); @@ -177,6 +179,8 @@ static void print_extent_info(struct fiemap_extent *fm_extent, int cur_ex, print_flag(&fe_flags, FIEMAP_EXTENT_UNWRITTEN, flags, "unwritten,"); print_flag(&fe_flags, FIEMAP_EXTENT_MERGED, flags, "merged,"); print_flag(&fe_flags, FIEMAP_EXTENT_SHARED, flags, "shared,"); + print_flag(&fe_flags, FIEMAP_EXTENT_NET, flags, "net,"); + /* print any unknown flags as hex values */ for (mask = 1; fe_flags != 0 && mask != 0; mask <<= 1) { char hex[6]; @@ -217,6 +221,7 @@ static int filefrag_fiemap(int fd, int blk_shift, int *num_extents, unsigned int i; int fiemap_header_printed = 0; int tot_extents = 0, n = 0; + int previous_device = 0; int last = 0; int rc; @@ -228,6 +233,12 @@ static int filefrag_fiemap(int fd, int blk_shift, int *num_extents, if (xattr_map) flags |= FIEMAP_FLAG_XATTR; + if (device_offset) { + flags |= FIEMAP_FLAG_DEVICE_ORDER; + memset(fm_ext, 0, sizeof(struct fiemap_extent)); + } + +retry_wo_device_order: do { fiemap->fm_length = ~0ULL; fiemap->fm_flags = flags; @@ -242,6 +253,10 @@ static int filefrag_fiemap(int fd, int blk_shift, int *num_extents, "flags %x\n", fiemap->fm_flags); fiemap_incompat_printed = 1; + } else if (rc == EBADR && (fiemap->fm_flags & + FIEMAP_FLAG_DEVICE_ORDER)) { + flags &= ~FIEMAP_FLAG_DEVICE_ORDER; + goto retry_wo_device_order; } return rc; } @@ -260,6 +275,9 @@ static int filefrag_fiemap(int fd, int blk_shift, int *num_extents, fm_last.fe_length; expected = fm_last.fe_physical + fm_ext[i].fe_logical - fm_last.fe_logical; + if (previous_device != fm_ext[i].fe_device) + previous_device = fm_ext[i].fe_device; + if (fm_ext[i].fe_logical != 0 && fm_ext[i].fe_physical != expected && fm_ext[i].fe_physical != expected_dense) { @@ -278,8 +296,20 @@ static int filefrag_fiemap(int fd, int blk_shift, int *num_extents, n++; } - fiemap->fm_start = (fm_ext[i - 1].fe_logical + - fm_ext[i - 1].fe_length); + /* For DEVICE_ORDER mappings, if EXTENT_LAST not yet found then + * fm_start needs to be the same as it was for earlier ioctl. + * The first extent is used to pass the end offset and device + * of the last FIEMAP call. Otherwise, we ask for extents + * starting from where the last mapping ended. */ + if (flags & FIEMAP_FLAG_DEVICE_ORDER) { + fm_ext[0].fe_logical = fm_ext[i - 1].fe_logical + + fm_ext[i - 1].fe_length; + fm_ext[0].fe_device = fm_ext[i - 1].fe_device; + fiemap->fm_start = 0; + } else { + fiemap->fm_start = fm_ext[i - 1].fe_logical + + fm_ext[i - 1].fe_length; + } } while (last == 0); *num_extents = tot_extents; @@ -303,6 +333,8 @@ static int filefrag_fibmap(int fd, int blk_shift, int *num_extents, memset(&fm_ext, 0, sizeof(fm_ext)); memset(&fm_last, 0, sizeof(fm_last)); if (force_extent) { + memset(&fm_ext, 0, sizeof(fm_ext)); + fm_ext.fe_device = st->st_dev; fm_ext.fe_flags = FIEMAP_EXTENT_MERGED; } @@ -437,6 +469,13 @@ static int frag_report(const char *filename) is_ext2 = 1; } + /* Check if filesystem is Lustre. Always print in extent format + * with 1kB blocks, using the device-relative logical offsets. */ + if (fsinfo.f_type == LUSTRE_SUPER_MAGIC) { + device_offset = 1; + blocksize = blocksize ?: 1024; + } + if (is_ext2) { long cylgroups = div_ceil(fsinfo.f_blocks, blksize * 8); @@ -524,10 +563,11 @@ int main(int argc, char**argv) char **cpp; int rc = 0, c; - while ((c = getopt(argc, argv, "Bb::eksvxX")) != EOF) { + while ((c = getopt(argc, argv, "Bb::eklsvxX")) != EOF) { switch (c) { case 'B': force_bmap++; + force_extent = 0; break; case 'b': if (optarg) { @@ -567,6 +607,9 @@ int main(int argc, char**argv) case 'k': blocksize = 1024; break; + case 'l': + device_offset++; + break; case 's': sync_file++; break;