From patchwork Wed May 27 14:08:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wang Shilong X-Patchwork-Id: 1298948 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=mpABZHQf; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49XCPF6wMcz9sTG for ; Thu, 28 May 2020 00:09:05 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387503AbgE0OJD (ORCPT ); Wed, 27 May 2020 10:09:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725939AbgE0OJD (ORCPT ); Wed, 27 May 2020 10:09:03 -0400 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A83FC08C5C1 for ; Wed, 27 May 2020 07:09:03 -0700 (PDT) Received: by mail-pg1-x542.google.com with SMTP id p21so11791279pgm.13 for ; Wed, 27 May 2020 07:09:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=jQNRI+l8uJJt0mnUF9lyAireteisnpWtnB1IBiEZvZk=; b=mpABZHQfVwzcrFdW24kkfOV79i0ng5zJwbFiek0w9BPRfy2Tu4XTT7FVZ/0gQyEuEj sIYI+DbTZXAcdOyAB+iWQdOX13stltqUjQEqwgAEYLztxUEjBjJX1J6uOZEFuYptrUtw qmnIEch567sZiDbeimIDOf8x/WHEWvstFx9GnasRxKDPKj08epFiR7hDQULbTNV3LavO PE8dU9kf8hI5KrL6KbMtotHSIgXcJUKxkLeqYFOX0rltvoUquLp4Bo/6A6QAEeYKBmAm 3NWbdqUYUiku3IFehZ5wckuDSe69/nvm9PRVMdG3u2ScNC4vy17JVVXWSnWRvJtjDgzo urTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=jQNRI+l8uJJt0mnUF9lyAireteisnpWtnB1IBiEZvZk=; b=b/Tg17BaqHM6ZPD3Pdq+c6KUQxF7b/RXUSiAq+v1V+nQUfH2/CjC4x8ynv8Od36+Qk LiHswCkMLZVRA3tD2FAgU7udcqnmJZ5dU4NI5s1jI7ikJhddykWdxt9we/tgrnDoD+su +iy6rAeUrRBbtvATSeUMBTz9mJS/N500sqQfgB2T1ZDAxKgZxzI8PRPiYFsYcJTxc6px Cl5l76jfEmqBwX8ZCwfbP7hULn0+8WHSGkLPn/s8Jzz3ogyvukYEKuWloijbne0ZhyNo hr1WCbaRn9CoQWdsH1Lf76sWHm/IKU+s4JVRuZx6zTAedeO0mj5XsLuZUMQsMOh0xpUu sXRA== X-Gm-Message-State: AOAM533fMKR4Ph2p/JTk8elbdiOoxx4G73WHCU2IC8Wm5/QCLGlpxpFr dyxwm70QIQYC0cE2oJ/iM6wQViXd0qo= X-Google-Smtp-Source: ABdhPJxkfLaG3/V0grfE6Rs6ipiMpgO9VCDfc5+sd/SFeiFKHyXrdQiE8P7uUZZiyWJxqcFJw7/I0w== X-Received: by 2002:a63:9347:: with SMTP id w7mr4023591pgm.409.1590588542233; Wed, 27 May 2020 07:09:02 -0700 (PDT) Received: from localhost.localdomain (fs276ec80e.tkyc203.ap.nuro.jp. [39.110.200.14]) by smtp.gmail.com with ESMTPSA id q201sm2292580pfq.40.2020.05.27.07.08.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 May 2020 07:09:01 -0700 (PDT) From: Wang Shilong To: linux-ext4@vger.kernel.org Cc: Wang Shilong , Shuichi Ihara , Andreas Dilger , Wang Shilong , Lukas Czerner Subject: [PATCH 1/2] e2fsprogs: add EXT2_FLAG_BG_WAS_TRIMMED to optimize fstrim Date: Wed, 27 May 2020 23:08:43 +0900 Message-Id: <1590588525-29669-1-git-send-email-wangshilong1991@gmail.com> X-Mailer: git-send-email 1.9.1 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong 1)mkfs will set this flag if discard is triggered. 2)whenver we free blocks from block group, this flag will be cleared. 3)make dumpe2fs aware of new flag. Cc: Shuichi Ihara Cc: Andreas Dilger Cc: Wang Shilong Cc: Lukas Czerner Signed-off-by: Wang Shilong --- lib/ext2fs/alloc_stats.c | 8 ++++++-- lib/ext2fs/alloc_tables.c | 2 ++ lib/ext2fs/ext2_fs.h | 1 + lib/ext2fs/ext2fs.h | 1 + misc/dumpe2fs.c | 2 ++ misc/mke2fs.c | 3 +++ 6 files changed, 15 insertions(+), 2 deletions(-) diff --git a/lib/ext2fs/alloc_stats.c b/lib/ext2fs/alloc_stats.c index 3949f618..2a6e0557 100644 --- a/lib/ext2fs/alloc_stats.c +++ b/lib/ext2fs/alloc_stats.c @@ -69,10 +69,12 @@ void ext2fs_block_alloc_stats2(ext2_filsys fs, blk64_t blk, int inuse) #endif return; } - if (inuse > 0) + if (inuse > 0) { ext2fs_mark_block_bitmap2(fs->block_map, blk); - else + } else { ext2fs_unmark_block_bitmap2(fs->block_map, blk); + ext2fs_bg_flags_clear(fs, group, EXT2_BG_WAS_TRIMMED); + } ext2fs_bg_free_blocks_count_set(fs, group, ext2fs_bg_free_blocks_count(fs, group) - inuse); ext2fs_bg_flags_clear(fs, group, EXT2_BG_BLOCK_UNINIT); ext2fs_group_desc_csum_set(fs, group); @@ -138,6 +140,8 @@ void ext2fs_block_alloc_stats_range(ext2_filsys fs, blk64_t blk, ext2fs_bg_free_blocks_count(fs, group) - inuse*n/EXT2FS_CLUSTER_RATIO(fs)); ext2fs_bg_flags_clear(fs, group, EXT2_BG_BLOCK_UNINIT); + if (inuse < 0) + ext2fs_bg_flags_clear(fs, group, EXT2_BG_WAS_TRIMMED); ext2fs_group_desc_csum_set(fs, group); ext2fs_free_blocks_count_add(fs->super, -inuse * (blk64_t) n); blk += n; diff --git a/lib/ext2fs/alloc_tables.c b/lib/ext2fs/alloc_tables.c index 971a6ceb..17087890 100644 --- a/lib/ext2fs/alloc_tables.c +++ b/lib/ext2fs/alloc_tables.c @@ -250,6 +250,8 @@ errcode_t ext2fs_allocate_group_table(ext2_filsys fs, dgrp_t group, } ext2fs_inode_table_loc_set(fs, group, new_blk); } + if (fs->flags & EXT2_FLAG_BG_WAS_TRIMMED) + ext2fs_bg_flags_set(fs, group, EXT2_BG_WAS_TRIMMED); ext2fs_group_desc_csum_set(fs, group); return 0; } diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h index 6c20ea77..86d6f438 100644 --- a/lib/ext2fs/ext2_fs.h +++ b/lib/ext2fs/ext2_fs.h @@ -211,6 +211,7 @@ struct ext4_group_desc #define EXT2_BG_INODE_UNINIT 0x0001 /* Inode table/bitmap not initialized */ #define EXT2_BG_BLOCK_UNINIT 0x0002 /* Block bitmap not initialized */ #define EXT2_BG_INODE_ZEROED 0x0004 /* On-disk itable initialized to zero */ +#define EXT2_BG_WAS_TRIMMED 0x0008 /* Block group was trimmed */ /* * Data structures used by the directory indexing feature diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h index 69c8a3ff..3dd99430 100644 --- a/lib/ext2fs/ext2fs.h +++ b/lib/ext2fs/ext2fs.h @@ -206,6 +206,7 @@ typedef struct ext2_file *ext2_file_t; #define EXT2_FLAG_IGNORE_SB_ERRORS 0x800000 #define EXT2_FLAG_BBITMAP_TAIL_PROBLEM 0x1000000 #define EXT2_FLAG_IBITMAP_TAIL_PROBLEM 0x2000000 +#define EXT2_FLAG_BG_WAS_TRIMMED 0x4000000 /* * Special flag in the ext2 inode i_flag field that means that this is diff --git a/misc/dumpe2fs.c b/misc/dumpe2fs.c index d295ba4d..a305ec9b 100644 --- a/misc/dumpe2fs.c +++ b/misc/dumpe2fs.c @@ -131,6 +131,8 @@ static void print_bg_opts(ext2_filsys fs, dgrp_t i) &first); print_bg_opt(bg_flags, EXT2_BG_INODE_ZEROED, "ITABLE_ZEROED", &first); + print_bg_opt(bg_flags, EXT2_BG_WAS_TRIMMED, "WAS_TRIMMED", + &first); if (!first) fputc(']', stdout); fputc('\n', stdout); diff --git a/misc/mke2fs.c b/misc/mke2fs.c index c90dcf0e..07ee620e 100644 --- a/misc/mke2fs.c +++ b/misc/mke2fs.c @@ -403,6 +403,7 @@ static errcode_t packed_allocate_tables(ext2_filsys fs) ext2fs_block_alloc_stats_range(fs, goal, fs->inode_blocks_per_group, +1); ext2fs_inode_table_loc_set(fs, i, goal); + ext2fs_bg_flags_set(fs, i, EXT2_BG_WAS_TRIMMED); ext2fs_group_desc_csum_set(fs, i); } return 0; @@ -3037,6 +3038,8 @@ int main (int argc, char *argv[]) /* Can't undo discard ... */ if (!noaction && discard && dev_size && (io_ptr != undo_io_manager)) { retval = mke2fs_discard_device(fs); + if (!retval) + fs->flags |= EXT2_FLAG_BG_WAS_TRIMMED; if (!retval && io_channel_discard_zeroes_data(fs->io)) { if (verbose) printf("%s", From patchwork Wed May 27 14:08:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wang Shilong X-Patchwork-Id: 1298949 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=DjpvyuU6; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49XCPJ3Y4Fz9sT3 for ; Thu, 28 May 2020 00:09:08 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388518AbgE0OJI (ORCPT ); Wed, 27 May 2020 10:09:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387711AbgE0OJF (ORCPT ); Wed, 27 May 2020 10:09:05 -0400 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62D4DC08C5C1 for ; Wed, 27 May 2020 07:09:05 -0700 (PDT) Received: by mail-pg1-x543.google.com with SMTP id f21so8803277pgg.12 for ; Wed, 27 May 2020 07:09:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=WMnlzBIzrloadjbjQNPdZUSWLPqow8bmYOKFr77Q5Dw=; b=DjpvyuU64Rum3FaEXB//a04RFK2RSBXu3I+4s8NFIYfn1669gIv/YXIpOzVZk4NA12 BtW0WOw/XNtfJJpT/8hMDT4hkY+btYxFwONuCwP0vlUzXYIMycp5bUgOkbXzfUgeA3o+ mYqKiOkHMF41GxwPeIsJ96ov0w7EEBNH8EtjHCiuDDSa5vTQ4LY72ZZ1DWOLr1yVcG0i xai4ziclp9pz3Kzf7k23tNPKDvPfgcEnDLMz7b795AlxlM+kKXxqaXtZhGL6msjgD65E LYVxkQ33/CYCU+vT37MSZpPDynByKsG0nf6EpC18yh0XXkj91RniOGff5cRh20lY0gvh puDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=WMnlzBIzrloadjbjQNPdZUSWLPqow8bmYOKFr77Q5Dw=; b=SFc0bq8Anun/Mmw+Me9CL8pJ9fOPmdOg//A2bCuaycaHyhCalpCFgoxPU15rtx9lAJ 9C/MgrxGww8yiCYQzJ/482zCLe7oKkJOrxfEc+nMq9JiclIVjh8eIZ155xySF3AS0eUR PAFp4lf6AbX9xmJU3yX1cQoEyzpvG18RJ9MipuKQ5Y5rrLDUHGi3ofhv5EHhzKKQRMzO Y0+5g4YDRAibqbidwiIbg5vo6M+hnQWumvGQfzgfCy1qCSc3FNb8E+Od/kWJ0hEVL5U/ tp3dJR8gKYFETFU6kFLK+8Y/z6lW3qdAagMxmn9VUhZzxd7VEUxKz/zt3MjhgZH5EhnV 0Khg== X-Gm-Message-State: AOAM5301kdx+cZNtVPXZq3mItoDQoAZP1VUzUVTbOsObOX9L1uuup02V E0j0hUH9DVoWhrM+9aA/zrfZDAK6CzM= X-Google-Smtp-Source: ABdhPJwx4AwdCmwG2An9aUNMJD4AFIP5jugEmOEQ5Sin8bcXpQNt+vi8WjKoFQ+G4N/fPAB8r66Z9A== X-Received: by 2002:a63:3c11:: with SMTP id j17mr4300867pga.70.1590588544656; Wed, 27 May 2020 07:09:04 -0700 (PDT) Received: from localhost.localdomain (fs276ec80e.tkyc203.ap.nuro.jp. [39.110.200.14]) by smtp.gmail.com with ESMTPSA id q201sm2292580pfq.40.2020.05.27.07.09.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 May 2020 07:09:04 -0700 (PDT) From: Wang Shilong To: linux-ext4@vger.kernel.org Cc: Wang Shilong , Shuichi Ihara , Andreas Dilger , Wang Shilong , Lukas Czerner Subject: [PATCH 2/2] tune2fs: add clear_was_trimmed option Date: Wed, 27 May 2020 23:08:44 +0900 Message-Id: <1590588525-29669-2-git-send-email-wangshilong1991@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1590588525-29669-1-git-send-email-wangshilong1991@gmail.com> References: <1590588525-29669-1-git-send-email-wangshilong1991@gmail.com> Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong It might be possible that admin want an option for clear existed WAS_TRIMMED flag to force fstrim next run time. Cc: Shuichi Ihara Cc: Andreas Dilger Cc: Wang Shilong Cc: Lukas Czerner Signed-off-by: Wang Shilong --- misc/tune2fs.8.in | 5 +++++ misc/tune2fs.c | 30 +++++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/misc/tune2fs.8.in b/misc/tune2fs.8.in index 3cf1f5ed..b8025949 100644 --- a/misc/tune2fs.8.in +++ b/misc/tune2fs.8.in @@ -249,6 +249,11 @@ mounted using experimental kernel code, such as the ext4dev filesystem. .B ^test_fs Clear the test_fs flag, indicating the filesystem should only be mounted using production-level filesystem code. +.TP +.B clear_was_trimmed +Clear block groups' WAS_TRIMMED flag, this will force fstrim to run every +block group next time. +.TP .RE .TP .B \-f diff --git a/misc/tune2fs.c b/misc/tune2fs.c index 314cc0d0..3e9814e3 100644 --- a/misc/tune2fs.c +++ b/misc/tune2fs.c @@ -101,6 +101,7 @@ static int rewrite_checksums; static int feature_64bit; static int fsck_requested; static char *undo_file; +static int clear_was_trimmed; int journal_size, journal_flags; char *journal_device; @@ -949,6 +950,26 @@ static void rewrite_metadata_checksums(ext2_filsys fs, unsigned int flags) ext2fs_mark_super_dirty(fs); } +static void clear_bg_was_trimmed(ext2_filsys fs) +{ + dgrp_t i; + int dirty = 0; + + for (i = 0; i < fs->group_desc_count; i++) { + if (ext2fs_bg_flags_test(fs, i, EXT2_BG_WAS_TRIMMED)) { + ext2fs_bg_flags_clear(fs, i, EXT2_BG_WAS_TRIMMED); + ext2fs_group_desc_csum_set(fs, i); + dirty = 1; + } + } + + if (dirty) { + fs->flags &= ~EXT2_FLAG_SUPER_ONLY; + ext2fs_mark_bb_dirty(fs); + ext2fs_mark_super_dirty(fs); + } +} + static void enable_uninit_bg(ext2_filsys fs) { struct ext2_group_desc *gd; @@ -2207,6 +2228,9 @@ static int parse_extended_opts(ext2_filsys fs, const char *opts) continue; } ext_mount_opts = strdup(arg); + } else if (!strcmp(token, "clear_was_trimmed") || + !strcmp(token, "clear_was-trimmed")) { + clear_was_trimmed = 1; } else r_usage++; } @@ -2224,7 +2248,8 @@ static int parse_extended_opts(ext2_filsys fs, const char *opts) "\tstripe_width=\n" "\tforce_fsck\n" "\ttest_fs\n" - "\t^test_fs\n")); + "\t^test_fs\n" + "\tclear_was_trimmed\n")); free(buf); return 1; } @@ -3361,6 +3386,9 @@ _("Warning: The journal is dirty. You may wish to replay the journal like:\n\n" } } + if (clear_was_trimmed) + clear_bg_was_trimmed(fs); + if (rewrite_checksums) rewrite_metadata_checksums(fs, rewrite_checksums);