From patchwork Wed Nov 20 04:35:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Dongyang X-Patchwork-Id: 1197819 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=ddn.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=ddn.com header.i=@ddn.com header.b="ZExIvIQg"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 47Hqcb3k9zz9sPv for ; Wed, 20 Nov 2019 15:35:27 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727535AbfKTEf0 (ORCPT ); Tue, 19 Nov 2019 23:35:26 -0500 Received: from mail-eopbgr790084.outbound.protection.outlook.com ([40.107.79.84]:3710 "EHLO NAM03-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727264AbfKTEf0 (ORCPT ); Tue, 19 Nov 2019 23:35:26 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AUZ73t5TgjcVs2Go1aAqoGx0a67OmTOPviqMnPiuWnRoyq/MKK1f5U2PyF6IROi1NdG1s6w4mJY/Qq28gPcejZA4L5l/ZVsic1p1CmvMDZQGJrDVDHFiJ9SlY1KETEk1TtEBhEP6mwLFchjCaGVOaMnHzilLx3KMgPqCndmaHCL3fWwDAB3JTjoMLMRd3Vptsr7Ws6yK+/fdHitYYE379y9PwZIoIyPBp4Gx+IIK4CwkMeFvjubwjl3xvAsVn0CuYzOxSqMojkSHZxgIc1+E2dx/wX/CvU35BcxB61ohcvF3YEm1uT8bKQl7lpl+d+ptjsFG+XJLP4fHNEtHBerRqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wFEDrvto1DzvB4l7IUYyBM27uAy6swDcUDqYVJLMuDs=; b=WSVLi7jSBMyOVq+9MMN19gsXsppEDqU2t6Rl4P+4y266PJiJ6fmaJysZFfZhlbMi+VZ9DlWloyTCaRGbUKcUOhB4fSk/knM3LUMLw4nF9hZl33kg/WGP1f8UnxY/PsXu2GMe8SiJHWCWsPodalg16Y9ITGO+HeaDQpR/fAvSYROJGPnqK197gVzfJamfuIXDr7LoXGO/ypV4lS6BzCS5zGFAO7XlmLpz9mflgcJ9VagqmoI9PMYe1RuMPLl7sMo5km0SxHH23GNOWppTlXhwWvJsTBUOJLKmy/jpOskWLPD1eE6D3C0YBgleQni7htRLZeEPJjwNbfzzHnh57QriDQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ddn.com; dmarc=pass action=none header.from=ddn.com; dkim=pass header.d=ddn.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ddn.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wFEDrvto1DzvB4l7IUYyBM27uAy6swDcUDqYVJLMuDs=; b=ZExIvIQg34+p57GYj5g92qCdDkr1UEZfH9h2pkY7M6TZ7GMWB6yr8T2afNrrTJ7vG8tXRFjR8OPkOjEBBCU0Qcsw7rzJqvWC9HmP5bHTfIFleWriC6pgvmzwbf+byo4Wk7F98DN0PM+l0/Qn2YQ43RU7sAdL+1UbWiyUJnAko3Q= Received: from BL0PR1901MB2004.namprd19.prod.outlook.com (52.132.24.157) by BL0PR1901MB2050.namprd19.prod.outlook.com (52.132.25.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2451.26; Wed, 20 Nov 2019 04:35:22 +0000 Received: from BL0PR1901MB2004.namprd19.prod.outlook.com ([fe80::8587:2f91:6a7b:8655]) by BL0PR1901MB2004.namprd19.prod.outlook.com ([fe80::8587:2f91:6a7b:8655%4]) with mapi id 15.20.2451.031; Wed, 20 Nov 2019 04:35:22 +0000 From: Li Dongyang To: "linux-ext4@vger.kernel.org" CC: "adilger@dilger.ca" Subject: [PATCH v3 1/5] libext2fs: optimize ext2fs_convert_subcluster_bitmap() Thread-Topic: [PATCH v3 1/5] libext2fs: optimize ext2fs_convert_subcluster_bitmap() Thread-Index: AQHVn1vr3nLuCuqfSkeK2sMViUbk/w== Date: Wed, 20 Nov 2019 04:35:21 +0000 Message-ID: <20191120043448.249988-1-dongyangli@ddn.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SY3PR01CA0134.ausprd01.prod.outlook.com (2603:10c6:0:1b::19) To BL0PR1901MB2004.namprd19.prod.outlook.com (2603:10b6:207:38::29) authentication-results: spf=none (sender IP is ) smtp.mailfrom=dli@ddn.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.24.0 x-originating-ip: [150.203.248.39] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5b9be9e9-b55c-4df0-48da-08d76d730dac x-ms-traffictypediagnostic: BL0PR1901MB2050: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4502; x-forefront-prvs: 02272225C5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(39850400004)(396003)(346002)(366004)(136003)(376002)(199004)(189003)(4326008)(5660300002)(25786009)(66476007)(2501003)(102836004)(26005)(7736002)(99286004)(66556008)(256004)(52116002)(2351001)(66946007)(2906002)(36756003)(6512007)(386003)(6506007)(186003)(316002)(6436002)(66446008)(5640700003)(64756008)(6486002)(8936002)(476003)(486006)(6916009)(50226002)(1076003)(305945005)(478600001)(71190400001)(71200400001)(14454004)(66066001)(8676002)(6116002)(3846002)(81156014)(2616005)(81166006); DIR:OUT; SFP:1101; SCL:1; SRVR:BL0PR1901MB2050; H:BL0PR1901MB2004.namprd19.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: ddn.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: boLUOjOgTGyTj8tyRCI5mawxuytl+kPmUGxuiXZ9ZSrVza5NZKFV7FFc7McUgd1huBTKZxqQBxCXOuo/01kb57oO7RdKJP4gh0cl8fVRd8DOdR+9sEbTBzehXKRpbaKm31dsTGSho8gjfig57GsgtHFNx0HqK/XhZOEhOZDlaNLW3xvidzVjoJ9T68Raw75oQeg1Kqcao+9cbnNWY+eYPrG3VYvxWNTVds2F7bV507xirvsoSFmvF9zXRXCC7w0Znvqj/JXSWYkSAIPVO/gM68GeJ6bt7CKfLuIfi3uxFnZKOL71XhSVKYi2VTD9GLxvuMph80YYP56Ib76QX2HZ0XkI13GLFChZzUXfiwcwNNvk+BaE/uHCvPvjaDDLKrmy3/OLmGvuAjRxWVOzpSzxaCaV+xoRgjyN0Dox81gI2ReBuJc0lmI6CyXOsEPS589S x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: ddn.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5b9be9e9-b55c-4df0-48da-08d76d730dac X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Nov 2019 04:35:21.9292 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 753b6e26-6fd3-43e6-8248-3f1735d59bb4 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: WCCdqU5xQQ9Xy1pNJVk4lHZM9NZRXjqSeTMrxkoNAQ2lEqMJsfuH9Qt7aJTfNqwj X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR1901MB2050 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org For a bigalloc filesystem, converting the block bitmap from blocks to chunks in ext2fs_convert_subcluster_bitmap() can take a long time when the device is huge, because we test the bitmap bit-by-bit using ext2fs_test_block_bitmap2(). Use ext2fs_find_first_set_block_bitmap2() which is more efficient for mke2fs when the fs is mostly empty. e2fsck can also benefit from this during pass1 block scanning. Time taken for "mke2fs -O bigalloc,extent -C 131072 -b 4096" on a 1PB device: without patch: real 27m49.457s user 21m36.474s sys 6m9.514s with patch: real 6m31.908s user 0m1.806s sys 6m29.697s Signed-off-by: Li Dongyang Reviewed-by: Andreas Dilger --- lib/ext2fs/gen_bitmap64.c | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-) diff --git a/lib/ext2fs/gen_bitmap64.c b/lib/ext2fs/gen_bitmap64.c index 6e4d8b71..f1dd1891 100644 --- a/lib/ext2fs/gen_bitmap64.c +++ b/lib/ext2fs/gen_bitmap64.c @@ -799,8 +799,7 @@ errcode_t ext2fs_convert_subcluster_bitmap(ext2_filsys fs, ext2fs_generic_bitmap_64 bmap, cmap; ext2fs_block_bitmap gen_bmap = *bitmap, gen_cmap; errcode_t retval; - blk64_t i, b_end, c_end; - int n, ratio; + blk64_t i, next, b_end, c_end; bmap = (ext2fs_generic_bitmap_64) gen_bmap; if (fs->cluster_ratio_bits == ext2fs_get_bitmap_granularity(gen_bmap)) @@ -817,18 +816,13 @@ errcode_t ext2fs_convert_subcluster_bitmap(ext2_filsys fs, bmap->end = bmap->real_end; c_end = cmap->end; cmap->end = cmap->real_end; - n = 0; - ratio = 1 << fs->cluster_ratio_bits; while (i < bmap->real_end) { - if (ext2fs_test_block_bitmap2(gen_bmap, i)) { - ext2fs_mark_block_bitmap2(gen_cmap, i); - i += ratio - n; - n = 0; - continue; - } - i++; n++; - if (n >= ratio) - n = 0; + retval = ext2fs_find_first_set_block_bitmap2(gen_bmap, + i, bmap->real_end, &next); + if (retval) + break; + ext2fs_mark_block_bitmap2(gen_cmap, next); + i = EXT2FS_C2B(fs, EXT2FS_B2C(fs, next) + 1); } bmap->end = b_end; cmap->end = c_end; From patchwork Wed Nov 20 04:35:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Dongyang X-Patchwork-Id: 1197820 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=ddn.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=ddn.com header.i=@ddn.com header.b="VKzGOnRm"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 47Hqcd3Z3Wz9sPZ for ; Wed, 20 Nov 2019 15:35:29 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727556AbfKTEf1 (ORCPT ); Tue, 19 Nov 2019 23:35:27 -0500 Received: from mail-eopbgr740055.outbound.protection.outlook.com ([40.107.74.55]:56019 "EHLO NAM01-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727450AbfKTEf1 (ORCPT ); Tue, 19 Nov 2019 23:35:27 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YuwI/f6wHUv8Udr4GiLdKHWzkJ4pUl4LH7PEFpo3gP/JHfkgidwlFHSuz5g+C+3PYnpghk7RO+DuyfDDK/oqlOmUU9CfKwY/YwaqGRajV3mNHhBTQvqiwMb2kWRvdLgwqDsZhhMvDGbFv5X00BnzxkSLm+roj7vSvlOAYcWNWKIeZSXcga+qoqk+djqxL7HkZmm5oIx/yaJJPHRCByxJPZNoyI1Fz2/Uq2MRos19ceKeGqSVcHzOESRYRMj7croFSKmCc893UAqlGBzp4WCv0QQVeAhNEa4Zwf69V4GrONHK/qM2C+f37hId089M/A1FGV+h7cN+hcdTEnwDHaObWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2oNur8ffPnGc4/SbmWtG8jiZAAyxy+l/JQsDT4dMT3E=; b=LAx7hy3G4UqpriUIFK7RPF3xpxWJn8lfBiRdA/oCxy2HWASysEiWXDn5njdshIJzdldg2KjpqWBmBLajSCCOzooSOBUbzLtjBdH+1BYqmYr/YNz1q4TwAhSr+8DhapHSV0pvbfBdoGNcYtammZO+u4aEnBX8Gnyv8djI0UY/nlBtCHhHwocctfTtbS0ySd+IX/2jY2A5vYutEtjrvRUAkCUnGKp02d5cQ8XKI+wDF+6HlwtckR2VnudVBOlCFoIXSy0mk1FTPf/Lh7mzPyVx9fuXINDUbWjZPA7q76WA9L3oTGeA28lSYG6xU2W6IL5o+D3vdgw+UDb5gjIv+Wf1Cg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ddn.com; dmarc=pass action=none header.from=ddn.com; dkim=pass header.d=ddn.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ddn.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2oNur8ffPnGc4/SbmWtG8jiZAAyxy+l/JQsDT4dMT3E=; b=VKzGOnRmPKwggRyrVbqJaF/X/mTZ1oAdfLgYesDGPkzaHqoXyZDQgnKQMdKIwkTd/UsKsEkqoFpb1WyCWrEMPaCXwHOTOKCpiI+B70lw8lgy7XTOeIMd77SqCrwg3+OBEDoPfFKWHBMfRsjLEoCB6sOeplJn8RWvC/F9OhRspzk= Received: from BL0PR1901MB2004.namprd19.prod.outlook.com (52.132.24.157) by BL0PR1901MB2002.namprd19.prod.outlook.com (52.132.17.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2451.26; Wed, 20 Nov 2019 04:35:24 +0000 Received: from BL0PR1901MB2004.namprd19.prod.outlook.com ([fe80::8587:2f91:6a7b:8655]) by BL0PR1901MB2004.namprd19.prod.outlook.com ([fe80::8587:2f91:6a7b:8655%4]) with mapi id 15.20.2451.031; Wed, 20 Nov 2019 04:35:24 +0000 From: Li Dongyang To: "linux-ext4@vger.kernel.org" CC: "adilger@dilger.ca" Subject: [PATCH v3 2/5] mke2fs: fix setting bad blocks in the block bitmap Thread-Topic: [PATCH v3 2/5] mke2fs: fix setting bad blocks in the block bitmap Thread-Index: AQHVn1vs9FtAuHiukkqPCXsW87kHBA== Date: Wed, 20 Nov 2019 04:35:24 +0000 Message-ID: <20191120043448.249988-2-dongyangli@ddn.com> References: <20191120043448.249988-1-dongyangli@ddn.com> In-Reply-To: <20191120043448.249988-1-dongyangli@ddn.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SY3PR01CA0134.ausprd01.prod.outlook.com (2603:10c6:0:1b::19) To BL0PR1901MB2004.namprd19.prod.outlook.com (2603:10b6:207:38::29) authentication-results: spf=none (sender IP is ) smtp.mailfrom=dli@ddn.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.24.0 x-originating-ip: [150.203.248.39] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5fe6f548-81a0-4150-af05-08d76d730eb8 x-ms-traffictypediagnostic: BL0PR1901MB2002: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:669; x-forefront-prvs: 02272225C5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(396003)(39850400004)(376002)(346002)(136003)(366004)(189003)(199004)(66446008)(64756008)(66556008)(66946007)(26005)(66476007)(6116002)(3846002)(66066001)(76176011)(4744005)(102836004)(5660300002)(1076003)(6916009)(52116002)(305945005)(6512007)(7736002)(36756003)(4326008)(446003)(11346002)(6486002)(8936002)(186003)(2616005)(81166006)(5640700003)(81156014)(50226002)(25786009)(8676002)(256004)(14444005)(6506007)(386003)(2906002)(478600001)(99286004)(2501003)(14454004)(2351001)(71200400001)(71190400001)(316002)(486006)(6436002)(476003); DIR:OUT; SFP:1101; SCL:1; SRVR:BL0PR1901MB2002; H:BL0PR1901MB2004.namprd19.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: ddn.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: jB1Lv9ZgBrGlhjgG6O8MmXJk6kmNZuL6yx6zvezfWKYClm5Ao9OtlqZwGaKapvFXVS+mX7GpYdDc3LYLzoRlaRbubgSemZj3eFbetYQH9Y/pAmIlj8SWMwmo/qGBIpszK6JYb4wgZs9vktFLoopAsB8urAzUTE+LOxEdSHnfGGAjJRnYLpdQiun4Kg5vq2O+IcCCyI8kC1msyTYXsN+tax8V0y3ZJ9xO7zbUCl3HQ8+RwQdz8Ajfs604+fuWRaUvDGnNs8Nc6PV2tZI2oZrezg62urNAnPAeq4g22ZzF5pRyzlETGC0Ne+rbAqYaRjj1fI1euwS0BOtAn7L9YzWaOn336wUSQ6K7kgJmp7yyOo1u0ihShTN9vamS0uSHfpVIDOUg/ug+AcBv2UxWZPh3RnCh5WJqTODdBFWAOgncLn/9quBaNHUF6/cYft3IscpS x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: ddn.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5fe6f548-81a0-4150-af05-08d76d730eb8 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Nov 2019 04:35:24.3788 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 753b6e26-6fd3-43e6-8248-3f1735d59bb4 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ltkC0mfHO91DIVyxJUe6ex8L1Nyv+K3RBESPUWM+86bdx27HqlzbqPbmE0t0UVsA X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR1901MB2002 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org We mark the bad blocks as used on fs->block_map before allocating group tables. Don't translate the block number to cluster number when doing this, the fs->block_map is still a block-granularity allocation map, it will be coverted later by ext2fs_convert_subcluster_bitmap(). Signed-off-by: Li Dongyang --- misc/mke2fs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/misc/mke2fs.c b/misc/mke2fs.c index ffea8233..be38d2c4 100644 --- a/misc/mke2fs.c +++ b/misc/mke2fs.c @@ -344,7 +344,7 @@ _("Warning: the backup superblock/group descriptors at block %u contain\n" exit(1); } while (ext2fs_badblocks_list_iterate(bb_iter, &blk)) - ext2fs_mark_block_bitmap2(fs->block_map, EXT2FS_B2C(fs, blk)); + ext2fs_mark_block_bitmap2(fs->block_map, blk); ext2fs_badblocks_list_iterate_end(bb_iter); } From patchwork Wed Nov 20 04:35:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Dongyang X-Patchwork-Id: 1197821 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=ddn.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=ddn.com header.i=@ddn.com header.b="goGoj+D0"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 47Hqcf0MVsz9sR7 for ; Wed, 20 Nov 2019 15:35:30 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727560AbfKTEf2 (ORCPT ); Tue, 19 Nov 2019 23:35:28 -0500 Received: from mail-eopbgr740055.outbound.protection.outlook.com ([40.107.74.55]:56019 "EHLO NAM01-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727264AbfKTEf1 (ORCPT ); Tue, 19 Nov 2019 23:35:27 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hAXD/+jAjgPtODcAAdITzSaitdnlSv5Uc0Cv40SSUF7AKoCNPcIXAT9bITnPQA+OlpgbAkeCWN7oiUykJZ2hmP+s1hj9ehlMZA012dZEc5KqLlRZKqecS8VS/Ms26us7hdWT92P2BD9/oesX8rbiuj6EWXsBUlnGalZoGXc+fFgz4ufmpRehkF/B3n2qEvo1Duuyt8XYpXm0YQ5FJ2RBOsMhuSW/gfYpbbezQ9mst/ejwsczYVowAXjJKH4afVw1IDJHcopbhFQadCt6S/SDk9+vVBmFrdoxDAXEay5ZJndITYTUxHZC+WE+gjLBw0YVkcoGe2zcwwE+CRacu0KQxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OxAPbM6gPG0GCuvSlAB7lIPmxNVCg4RB9do4hQJJqAc=; b=G3JSht3ctzUnBIZMgYXwPZOckiaoPsOpSkEPedGAjpQKwtnlLVrpIpvKLpBs2EHHzG8s10lHriqyhv6wFoFEmBYBevbEzPJdCmTfiHCIu+VyLmTWL9FXPXxAH2i5hbbvZttXHVqUho6rPuEY5XnVo9DqfkoU8v/u0KzL6t5kHVVVEAPQuJTag+8UYyLwtcdgajPGg/wbWqcQJ87YvwlpaQi+NOk5IYZvcvlwG8gPFUA9KINb2AHQtbk8cro9X70U5TOvYaNrmR7Zisa5PoYCAspjzZkRSlO/Ho7HNXqzeN2poHujJODRO32LSYtFXHLikR1uwXAESSoWFcyg5vAeJw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ddn.com; dmarc=pass action=none header.from=ddn.com; dkim=pass header.d=ddn.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ddn.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OxAPbM6gPG0GCuvSlAB7lIPmxNVCg4RB9do4hQJJqAc=; b=goGoj+D0yRgUgtwvkzp02O1FlAYPq+77ypRsZFtdxiyloCCO3A8TlqkvsXq9ojBdWGToh3ORE8w7uZEk3Hk1t5ahRAQh071VlpWz/gvWLktQkWrOPfSUzdBce1XpBlEcju/ZhQrO/4YeAsknoygFCv9y64iwF4I+Sc9NxRBmx94= Received: from BL0PR1901MB2004.namprd19.prod.outlook.com (52.132.24.157) by BL0PR1901MB2002.namprd19.prod.outlook.com (52.132.17.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2451.26; Wed, 20 Nov 2019 04:35:26 +0000 Received: from BL0PR1901MB2004.namprd19.prod.outlook.com ([fe80::8587:2f91:6a7b:8655]) by BL0PR1901MB2004.namprd19.prod.outlook.com ([fe80::8587:2f91:6a7b:8655%4]) with mapi id 15.20.2451.031; Wed, 20 Nov 2019 04:35:26 +0000 From: Li Dongyang To: "linux-ext4@vger.kernel.org" CC: "adilger@dilger.ca" Subject: [PATCH v3 3/5] ext2fs: rename "s_overhead_blocks" to "s_overhead_clusters" Thread-Topic: [PATCH v3 3/5] ext2fs: rename "s_overhead_blocks" to "s_overhead_clusters" Thread-Index: AQHVn1vtM42MghAqPECfuY10H3Mk5Q== Date: Wed, 20 Nov 2019 04:35:25 +0000 Message-ID: <20191120043448.249988-3-dongyangli@ddn.com> References: <20191120043448.249988-1-dongyangli@ddn.com> In-Reply-To: <20191120043448.249988-1-dongyangli@ddn.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SY3PR01CA0134.ausprd01.prod.outlook.com (2603:10c6:0:1b::19) To BL0PR1901MB2004.namprd19.prod.outlook.com (2603:10b6:207:38::29) authentication-results: spf=none (sender IP is ) smtp.mailfrom=dli@ddn.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.24.0 x-originating-ip: [150.203.248.39] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e0dfba44-0187-46f6-6662-08d76d73104f x-ms-traffictypediagnostic: BL0PR1901MB2002: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:345; x-forefront-prvs: 02272225C5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(396003)(39850400004)(376002)(346002)(136003)(366004)(189003)(199004)(66446008)(64756008)(66556008)(66946007)(26005)(66476007)(6116002)(3846002)(66066001)(76176011)(102836004)(5660300002)(1076003)(6916009)(52116002)(305945005)(6512007)(7736002)(36756003)(4326008)(446003)(11346002)(6486002)(8936002)(186003)(2616005)(81166006)(5640700003)(81156014)(50226002)(25786009)(8676002)(256004)(14444005)(6506007)(386003)(2906002)(478600001)(99286004)(2501003)(14454004)(2351001)(71200400001)(71190400001)(316002)(486006)(6436002)(476003); DIR:OUT; SFP:1101; SCL:1; SRVR:BL0PR1901MB2002; H:BL0PR1901MB2004.namprd19.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: ddn.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: XUZjjen9km0ByVpYh73jVTPL5xxgyamf6qbRxKAoFqFqz7YDOCgPEvm0N/O1C9GQfmAVhtUZ0rzAI3mkQ+pGeNDwNujXroKgFBdBQfy9jbApGIOzZCU6jZukmVI2HV2eETp3alSI5b4fo88DVscwad/y6+P7/Z78Tl7WETQEluTnoZEm8rwk3FzuWJ+qCZOYkjjzFjrn1Z9EkGOhU9F36g6AIpfJxEz4YoZS1n0CaAtsEHJvvScZcd4BYPTzqhKjww3/xGZXI1FQN9/87uyvpCGZE+AHqTxvd+ivtxSMGVCbbQSGZQJJJXcl1Sgxt458EceoMzwYRLRZsD6L17fM4ZpQQJu6JOXAYoKQIkApf7NoSTEFINWMIPneZRgMIWfhGmY0IV/Ho5NJvEpGHRtoBGXpyAmMHW5oNMtLDLHXY5xhdv4JznOyJhDHScd1VrVU x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: ddn.com X-MS-Exchange-CrossTenant-Network-Message-Id: e0dfba44-0187-46f6-6662-08d76d73104f X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Nov 2019 04:35:26.0878 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 753b6e26-6fd3-43e6-8248-3f1735d59bb4 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: V7L7frmj5lK4rKxT6gpUSwgYc+ptKZkKu0LlpaXSK1nPg2SKdIeJ7HpY/Yk1+ycd X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR1901MB2002 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Rename s_overhead_blocks field from struct ext2_super_block to make it consistent with the kernel counterpart. Signed-off-by: Li Dongyang Reviewed-by: Andreas Dilger --- debugfs/set_fields.c | 2 +- lib/e2p/ls.c | 6 +++--- lib/ext2fs/ext2_fs.h | 2 +- lib/ext2fs/swapfs.c | 2 +- lib/ext2fs/tst_super_size.c | 2 +- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/debugfs/set_fields.c b/debugfs/set_fields.c index 5142554d..f497bd92 100644 --- a/debugfs/set_fields.c +++ b/debugfs/set_fields.c @@ -160,7 +160,7 @@ static struct field_set_info super_fields[] = { { "usr_quota_inum", &set_sb.s_usr_quota_inum, NULL, 4, parse_uint }, { "grp_quota_inum", &set_sb.s_grp_quota_inum, NULL, 4, parse_uint }, { "prj_quota_inum", &set_sb.s_prj_quota_inum, NULL, 4, parse_uint }, - { "overhead_blocks", &set_sb.s_overhead_blocks, NULL, 4, parse_uint }, + { "overhead_clusters", &set_sb.s_overhead_clusters, NULL, 4, parse_uint }, { "backup_bgs", &set_sb.s_backup_bgs[0], NULL, 4, parse_uint, FLAG_ARRAY, 2 }, { "checksum", &set_sb.s_checksum, NULL, 4, parse_uint }, diff --git a/lib/e2p/ls.c b/lib/e2p/ls.c index 5a446178..5ca750f6 100644 --- a/lib/e2p/ls.c +++ b/lib/e2p/ls.c @@ -272,9 +272,9 @@ void list_super2(struct ext2_super_block * sb, FILE *f) fprintf(f, "Inode count: %u\n", sb->s_inodes_count); fprintf(f, "Block count: %llu\n", e2p_blocks_count(sb)); fprintf(f, "Reserved block count: %llu\n", e2p_r_blocks_count(sb)); - if (sb->s_overhead_blocks) - fprintf(f, "Overhead blocks: %u\n", - sb->s_overhead_blocks); + if (sb->s_overhead_clusters) + fprintf(f, "Overhead clusters: %u\n", + sb->s_overhead_clusters); fprintf(f, "Free blocks: %llu\n", e2p_free_blocks_count(sb)); fprintf(f, "Free inodes: %u\n", sb->s_free_inodes_count); fprintf(f, "First block: %u\n", sb->s_first_data_block); diff --git a/lib/ext2fs/ext2_fs.h b/lib/ext2fs/ext2_fs.h index 3165b389..7fa8406f 100644 --- a/lib/ext2fs/ext2_fs.h +++ b/lib/ext2fs/ext2_fs.h @@ -742,7 +742,7 @@ struct ext2_super_block { /*200*/ __u8 s_mount_opts[64]; /*240*/ __u32 s_usr_quota_inum; /* inode number of user quota file */ __u32 s_grp_quota_inum; /* inode number of group quota file */ - __u32 s_overhead_blocks; /* overhead blocks/clusters in fs */ + __u32 s_overhead_clusters; /* overhead blocks/clusters in fs */ /*24c*/ __u32 s_backup_bgs[2]; /* If sparse_super2 enabled */ /*254*/ __u8 s_encrypt_algos[4]; /* Encryption algorithms in use */ /*258*/ __u8 s_encrypt_pw_salt[16]; /* Salt used for string2key algorithm */ diff --git a/lib/ext2fs/swapfs.c b/lib/ext2fs/swapfs.c index a1560045..63b24330 100644 --- a/lib/ext2fs/swapfs.c +++ b/lib/ext2fs/swapfs.c @@ -121,7 +121,7 @@ void ext2fs_swap_super(struct ext2_super_block * sb) /* sb->s_mount_opts is __u8 and does not need swabbing */ sb->s_usr_quota_inum = ext2fs_swab32(sb->s_usr_quota_inum); sb->s_grp_quota_inum = ext2fs_swab32(sb->s_grp_quota_inum); - sb->s_overhead_blocks = ext2fs_swab32(sb->s_overhead_blocks); + sb->s_overhead_clusters = ext2fs_swab32(sb->s_overhead_clusters); sb->s_backup_bgs[0] = ext2fs_swab32(sb->s_backup_bgs[0]); sb->s_backup_bgs[1] = ext2fs_swab32(sb->s_backup_bgs[1]); /* sb->s_encrypt_algos is __u8 and does not need swabbing */ diff --git a/lib/ext2fs/tst_super_size.c b/lib/ext2fs/tst_super_size.c index a932685d..ab38dd59 100644 --- a/lib/ext2fs/tst_super_size.c +++ b/lib/ext2fs/tst_super_size.c @@ -135,7 +135,7 @@ int main(int argc, char **argv) check_field(s_mount_opts, 64); check_field(s_usr_quota_inum, 4); check_field(s_grp_quota_inum, 4); - check_field(s_overhead_blocks, 4); + check_field(s_overhead_clusters, 4); check_field(s_backup_bgs, 8); check_field(s_encrypt_algos, 4); check_field(s_encrypt_pw_salt, 16); From patchwork Wed Nov 20 04:35:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Dongyang X-Patchwork-Id: 1197823 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=ddn.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=ddn.com header.i=@ddn.com header.b="J0DQW70q"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 47HqdQ3Szqz9sPZ for ; Wed, 20 Nov 2019 15:36:10 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727363AbfKTEgK (ORCPT ); Tue, 19 Nov 2019 23:36:10 -0500 Received: from mail-eopbgr740088.outbound.protection.outlook.com ([40.107.74.88]:49538 "EHLO NAM01-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727264AbfKTEgJ (ORCPT ); Tue, 19 Nov 2019 23:36:09 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YEJFWJAQbwVXDjDbiwySAx0qbeFPj5bWl6WDAufEAp3i+BzIFTUHV7tIidUJP6fDNFze/adBauMQQ1XXw3nFHlBxJqtXuP6V5IiKMnjCcWE71MhnGPWRbaM3H3F4yMw5OL77HyGmXrzlGIr7DC8QCByqvD2lnAQ3QKiPtWgy4aXebQAdOsgtuP2XIFzzucv9FA8ie1U9FwBnf8wsmm1ZR4pAIl84sO2PP3OSDA/BFBqKNerRpUGJZMGJfhUslj+i23MepW+HXHTgZTdievazljrVNu3BqRfEQ1FN6bAwwFzpwRkbI3Eamq1pAXatZgdt1WqnjaljnWe0K5WpdAOhDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oVLkiMXYD7q5U/7cxTAunPjufK+4PcnJYED1BrrdyJY=; b=KRvlXud1fXObF1LsZysD10X+yDt0EpB3xYndw2faEKWSdOTgqpn3yocOBQmxGa3WvZ1bS8LE/MmLz5KdCkFqOvxvZS35XasJpoinU+N43mTb1PIX/qcOJXV3qaBi0mgt7pp8RadD27MSK/mwUjoN+d2pzj73CKmmTUm/+yqBIsxJrq187HGu24qmA37Xsi8WEwUDDWUhlqBrqrfYqRN9PyPgFwNiaSNFDMgtpl3kUGjEj0dQna6bp80HL2A4K+aTWzl9qu0wFtszm5qkcTsUPKUSbaQMT8fCy2qLSqRtbpoKX+ULXPZRq/zftSUCdVzieEk/n0twmz/f0d8vPGOMGQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ddn.com; dmarc=pass action=none header.from=ddn.com; dkim=pass header.d=ddn.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ddn.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oVLkiMXYD7q5U/7cxTAunPjufK+4PcnJYED1BrrdyJY=; b=J0DQW70qHDZ1CjJt8HYchwrkEZKm16WZ9RayYj1SA+3Vv9zu+2zvKqYJfuC0Gj2egrl6ndum6GADDmsInCjvPcz/S2fs7jlLBwQf5g7Ovk+mGthGonrGIK0aBkOlrReTJKpiEc3oC8dr4m2CPszKx+jkjvWGY6j6P9DKngg6UhA= Received: from BL0PR1901MB2004.namprd19.prod.outlook.com (52.132.24.157) by BL0PR1901MB2002.namprd19.prod.outlook.com (52.132.17.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2451.26; Wed, 20 Nov 2019 04:35:28 +0000 Received: from BL0PR1901MB2004.namprd19.prod.outlook.com ([fe80::8587:2f91:6a7b:8655]) by BL0PR1901MB2004.namprd19.prod.outlook.com ([fe80::8587:2f91:6a7b:8655%4]) with mapi id 15.20.2451.031; Wed, 20 Nov 2019 04:35:28 +0000 From: Li Dongyang To: "linux-ext4@vger.kernel.org" CC: "adilger@dilger.ca" Subject: [PATCH v3 4/5] mke2fs: set overhead in super block Thread-Topic: [PATCH v3 4/5] mke2fs: set overhead in super block Thread-Index: AQHVn1vv8aUzx6r/80yoWojoOcTFqw== Date: Wed, 20 Nov 2019 04:35:27 +0000 Message-ID: <20191120043448.249988-4-dongyangli@ddn.com> References: <20191120043448.249988-1-dongyangli@ddn.com> In-Reply-To: <20191120043448.249988-1-dongyangli@ddn.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SY3PR01CA0134.ausprd01.prod.outlook.com (2603:10c6:0:1b::19) To BL0PR1901MB2004.namprd19.prod.outlook.com (2603:10b6:207:38::29) authentication-results: spf=none (sender IP is ) smtp.mailfrom=dli@ddn.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.24.0 x-originating-ip: [150.203.248.39] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5b9a08a2-6b6b-4644-ea9d-08d76d73115f x-ms-traffictypediagnostic: BL0PR1901MB2002: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3968; x-forefront-prvs: 02272225C5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(396003)(39850400004)(376002)(346002)(136003)(366004)(189003)(199004)(66446008)(64756008)(66556008)(66946007)(26005)(66476007)(6116002)(3846002)(66066001)(76176011)(102836004)(5660300002)(1076003)(6916009)(52116002)(305945005)(6512007)(7736002)(36756003)(4326008)(446003)(11346002)(6486002)(8936002)(186003)(2616005)(81166006)(5640700003)(81156014)(50226002)(25786009)(8676002)(256004)(14444005)(6506007)(386003)(2906002)(478600001)(99286004)(2501003)(14454004)(2351001)(71200400001)(71190400001)(316002)(486006)(6436002)(476003); DIR:OUT; SFP:1101; SCL:1; SRVR:BL0PR1901MB2002; H:BL0PR1901MB2004.namprd19.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: ddn.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ux35iRTHr/slwiM0QLegnXbrt9+bbAd9xuCADwq9H7LgEpfbV69DA87m686AjFAU6/Tul5nw6aIa23b7v5uOujNAe0zAc6Clc81QDgcl+xfbD5KUZZerPtl0mANGTcLm4RpB5CsUqvJ1qQK8SpMvwmrIZbIbnj4RSrdjr6Ss2KNDuQlAJ7tJp1BOhoxWGajhdgJfj0iIOxPfUQvC8+IVABthYZlkIA9xcA8tOz16bL98bBTJYiaP0FtFL1kD3JDmwQZc7t+MmrYY3FtpiV3qHBQNFPf7ygjKsRXOa2GBXCFvDiE4o7ElwJofWnyir+FWWzo0N7aL1NbyxrrwJeVYOyBIYdCBHKmbGJGGg98D1aHvkf6q+GUgLARkqGSTbNvrOelTSAno9kEGqpmVtqM34l2sTmqpoQpC51tSA+yboECPifD31O30o2ztFEDJsclT x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: ddn.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5b9a08a2-6b6b-4644-ea9d-08d76d73115f X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Nov 2019 04:35:27.9587 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 753b6e26-6fd3-43e6-8248-3f1735d59bb4 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: DBb+2RXKNYiJugMHoNT/K4tsl0qRkYwGtVbhc/X6xZu4emcF4iRxvM5I+OamvQJ8 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR1901MB2002 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org If overhead is not recorded in the super block, it is caculated during mount in kernel, for bigalloc file systems the it takes O(groups**2) in time. For a 1PB deivce with 32K cluste size it takes ~12 mins to mount, with most of the time spent on figuring out overhead. While we can not improve the overhead algorithm in kernel due to the nature of bigalloc, we can work out the overhead during mke2fs and set it in the super block, avoiding calculating it every time when it mounts. Overhead is s_first_data_block plus internal journal blocks plus the block and inode bitmaps, inode table, super block backups and group descriptor blocks for every group. This patch introduces ext2fs_count_used_clusters(), which calculates the clusters used in the block bitmap for the given range. When bad blocks are involved, it gets tricky because the blocks counted as overhead and the bad blocks can end up in the same allocation cluster. In this case we will unmark the bad blocks from the block bitmap, convert to cluster bitmap and get the overhead, then mark the bad blocks back in the cluster bitmap. Reset the overhead to zero when resizing, we can not simplly count the used blocks as overhead like we do when mke2fs. The overhead can be calculated by kernel side during mount. Signed-off-by: Li Dongyang --- lib/ext2fs/ext2fs.h | 2 ++ lib/ext2fs/gen_bitmap64.c | 35 +++++++++++++++++++++++++++++ misc/mke2fs.c | 47 +++++++++++++++++++++++++++++++++++++++ resize/resize2fs.c | 1 + 4 files changed, 85 insertions(+) diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h index da309947..78f84632 100644 --- a/lib/ext2fs/ext2fs.h +++ b/lib/ext2fs/ext2fs.h @@ -1442,6 +1442,8 @@ errcode_t ext2fs_set_generic_bmap_range(ext2fs_generic_bitmap bmap, void *in); errcode_t ext2fs_convert_subcluster_bitmap(ext2_filsys fs, ext2fs_block_bitmap *bitmap); +errcode_t ext2fs_count_used_clusters(ext2_filsys fs, blk64_t start, + blk64_t end, blk64_t *out); /* get_num_dirs.c */ extern errcode_t ext2fs_get_num_dirs(ext2_filsys fs, ext2_ino_t *ret_num_dirs); diff --git a/lib/ext2fs/gen_bitmap64.c b/lib/ext2fs/gen_bitmap64.c index f1dd1891..b2370667 100644 --- a/lib/ext2fs/gen_bitmap64.c +++ b/lib/ext2fs/gen_bitmap64.c @@ -940,3 +940,38 @@ errcode_t ext2fs_find_first_set_generic_bmap(ext2fs_generic_bitmap bitmap, return ENOENT; } + +errcode_t ext2fs_count_used_clusters(ext2_filsys fs, blk64_t start, + blk64_t end, blk64_t *out) +{ + blk64_t next; + blk64_t tot_set = 0; + errcode_t retval; + + while (start < end) { + retval = ext2fs_find_first_set_block_bitmap2(fs->block_map, + start, end, &next); + if (retval) { + if (retval == ENOENT) + retval = 0; + break; + } + start = next; + + retval = ext2fs_find_first_zero_block_bitmap2(fs->block_map, + start, end, &next); + if (retval == 0) { + tot_set += next - start; + start = next + 1; + } else if (retval == ENOENT) { + retval = 0; + tot_set += end - start + 1; + break; + } else + break; + } + + if (!retval) + *out = EXT2FS_NUM_B2C(fs, tot_set); + return retval; +} diff --git a/misc/mke2fs.c b/misc/mke2fs.c index be38d2c4..542a3030 100644 --- a/misc/mke2fs.c +++ b/misc/mke2fs.c @@ -2914,6 +2914,8 @@ int main (int argc, char *argv[]) errcode_t retval = 0; ext2_filsys fs; badblocks_list bb_list = 0; + badblocks_iterate bb_iter; + blk_t blk; unsigned int journal_blocks = 0; unsigned int i, checkinterval; int max_mnt_count; @@ -2924,6 +2926,7 @@ int main (int argc, char *argv[]) char opt_string[40]; char *hash_alg_str; int itable_zeroed = 0; + blk64_t overhead; #ifdef ENABLE_NLS setlocale(LC_MESSAGES, ""); @@ -3215,6 +3218,23 @@ int main (int argc, char *argv[]) if (!quiet) printf("%s", _("done \n")); + /* + * Unmark bad blocks to calculate overhead, because metadata + * blocks and bad blocks can land on the same allocation cluster. + */ + if (bb_list) { + retval = ext2fs_badblocks_list_iterate_begin(bb_list, + &bb_iter); + if (retval) { + com_err("ext2fs_badblocks_list_iterate_begin", retval, + "%s", _("while unmarking bad blocks")); + exit(1); + } + while (ext2fs_badblocks_list_iterate(bb_iter, &blk)) + ext2fs_unmark_block_bitmap2(fs->block_map, blk); + ext2fs_badblocks_list_iterate_end(bb_iter); + } + retval = ext2fs_convert_subcluster_bitmap(fs, &fs->block_map); if (retval) { com_err(program_name, retval, "%s", @@ -3222,6 +3242,28 @@ int main (int argc, char *argv[]) exit(1); } + retval = ext2fs_count_used_clusters(fs, fs->super->s_first_data_block, + ext2fs_blocks_count(fs->super) - 1, + &overhead); + if (retval) { + com_err(program_name, retval, "%s", + _("while calculating overhead")); + exit(1); + } + + if (bb_list) { + retval = ext2fs_badblocks_list_iterate_begin(bb_list, + &bb_iter); + if (retval) { + com_err("ext2fs_badblocks_list_iterate_begin", retval, + "%s", _("while marking bad blocks as used")); + exit(1); + } + while (ext2fs_badblocks_list_iterate(bb_iter, &blk)) + ext2fs_mark_block_bitmap2(fs->block_map, blk); + ext2fs_badblocks_list_iterate_end(bb_iter); + } + if (super_only) { check_plausibility(device_name, CHECK_FS_EXIST, NULL); printf(_("%s may be further corrupted by superblock rewrite\n"), @@ -3319,6 +3361,7 @@ int main (int argc, char *argv[]) free(journal_device); } else if ((journal_size) || ext2fs_has_feature_journal(&fs_param)) { + overhead += EXT2FS_NUM_B2C(fs, journal_blocks); if (super_only) { printf("%s", _("Skipping journal creation in super-only mode\n")); fs->super->s_journal_inum = EXT2_JOURNAL_INO; @@ -3361,6 +3404,10 @@ no_journal: fs->super->s_mmp_update_interval); } + overhead += fs->super->s_first_data_block; + if (!super_only) + fs->super->s_overhead_clusters = overhead; + if (ext2fs_has_feature_bigalloc(&fs_param)) fix_cluster_bg_counts(fs); if (ext2fs_has_feature_quota(&fs_param)) diff --git a/resize/resize2fs.c b/resize/resize2fs.c index 8a3d08db..2443ff67 100644 --- a/resize/resize2fs.c +++ b/resize/resize2fs.c @@ -703,6 +703,7 @@ errcode_t adjust_fs_info(ext2_filsys fs, ext2_filsys old_fs, double percent; ext2fs_blocks_count_set(fs->super, new_size); + fs->super->s_overhead_clusters = 0; retry: fs->group_desc_count = ext2fs_div64_ceil(ext2fs_blocks_count(fs->super) - From patchwork Wed Nov 20 04:35:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Dongyang X-Patchwork-Id: 1197822 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=ddn.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=ddn.com header.i=@ddn.com header.b="B17r47U1"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 47HqdK2mH7z9sPZ for ; Wed, 20 Nov 2019 15:36:05 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727465AbfKTEgE (ORCPT ); Tue, 19 Nov 2019 23:36:04 -0500 Received: from mail-eopbgr740042.outbound.protection.outlook.com ([40.107.74.42]:29084 "EHLO NAM01-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727359AbfKTEgE (ORCPT ); Tue, 19 Nov 2019 23:36:04 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N+pnUWQCLnBVfLJTN7vFrryB8ULdU6jvBk/hNXLbrSflKHEazvmNLHMgoVx1v2Inm6+EVEVptK3DGccEB2XGsiAXGqtzhgmoZp7Oq3uJwe21SDWq3Jb2WJGizH/bxbxUFCf5pJBlzgTBusrbKMWdb2kND9loInpxYJzAX40KgsktV5pOxshBZ1kqzWB0rkLUe/TE0Wduzyh7fzuXIIwgiPAWv6y9jp/L9TWI693w/HGSeyABYseWsVuPuJbO8+ikOzSJVsiThxZtNNF6YDF3wRx0toX86R+Pu+Hc8AmMfzGMipLaCmMAUqVC7lRJ8bzyzMhR4UjAa73pIUorBp1Cww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rlxIjpnQGhb+LswSvb9frci0u3Jl+YuVlHoob8qDJag=; b=cVjZL3PezeY/C2qCfSrzDK93fm9gaEqXPnGVh1wEZesMslvBBi0TQQ0SBSsrYQ5LES9Ual+mBZdoFQSQlRClDyR50ZjaD3FpSq3Sv53E8GNO/ghDzt514IEdV1SkhtNWzmwb45H4yuQaa9tUefp/kgHB9Noc0vvz6xXMILGvSUx9EEIz2JozfSljC8AgR1OEtaV89sj1aujiubzsmfwGzp3NbGxTTPTU2YX+evCi5OjOKC5F6QRv04Mdqr8f6D0gQrwYseMQpHuMKU8Au6AIUvvAc6F5PY7Y+h07vqG9mfHdepCHNdXd+xumS9syhKOmFvqpfeQXkac3U0nKDzWCVQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ddn.com; dmarc=pass action=none header.from=ddn.com; dkim=pass header.d=ddn.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ddn.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rlxIjpnQGhb+LswSvb9frci0u3Jl+YuVlHoob8qDJag=; b=B17r47U1K90gpIbOUu5/OATywWWEaItsoc2yBL1KvDkEbzdq5OxBZkeEc8eMKUbOzu6wCOahcSPgWPee5MsDMKb0JsQweJU9bq674YaGM6uMlxhDN9uZ/TVE9GQjirSQY/hsaVJpxhE2cBr893p/4X0KRBn+4nwyJ7E/96lHJnM= Received: from BL0PR1901MB2004.namprd19.prod.outlook.com (52.132.24.157) by BL0PR1901MB2002.namprd19.prod.outlook.com (52.132.17.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2451.26; Wed, 20 Nov 2019 04:35:30 +0000 Received: from BL0PR1901MB2004.namprd19.prod.outlook.com ([fe80::8587:2f91:6a7b:8655]) by BL0PR1901MB2004.namprd19.prod.outlook.com ([fe80::8587:2f91:6a7b:8655%4]) with mapi id 15.20.2451.031; Wed, 20 Nov 2019 04:35:30 +0000 From: Li Dongyang To: "linux-ext4@vger.kernel.org" CC: "adilger@dilger.ca" Subject: [PATCH v3 5/5] ext4: record overhead in super block Thread-Topic: [PATCH v3 5/5] ext4: record overhead in super block Thread-Index: AQHVn1vwCuCxh8H040+zNz2bV0dPog== Date: Wed, 20 Nov 2019 04:35:29 +0000 Message-ID: <20191120043448.249988-5-dongyangli@ddn.com> References: <20191120043448.249988-1-dongyangli@ddn.com> In-Reply-To: <20191120043448.249988-1-dongyangli@ddn.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SY3PR01CA0134.ausprd01.prod.outlook.com (2603:10c6:0:1b::19) To BL0PR1901MB2004.namprd19.prod.outlook.com (2603:10b6:207:38::29) authentication-results: spf=none (sender IP is ) smtp.mailfrom=dli@ddn.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.24.0 x-originating-ip: [150.203.248.39] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0d784180-0be7-4f1c-0da5-08d76d73124d x-ms-traffictypediagnostic: BL0PR1901MB2002: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:51; x-forefront-prvs: 02272225C5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(396003)(39850400004)(376002)(346002)(136003)(366004)(189003)(199004)(66446008)(64756008)(66556008)(66946007)(26005)(66476007)(6116002)(3846002)(66066001)(76176011)(102836004)(5660300002)(1076003)(6916009)(52116002)(305945005)(6512007)(7736002)(36756003)(4326008)(446003)(11346002)(6486002)(8936002)(186003)(2616005)(81166006)(5640700003)(81156014)(50226002)(25786009)(8676002)(256004)(14444005)(6506007)(386003)(2906002)(478600001)(99286004)(2501003)(14454004)(2351001)(71200400001)(71190400001)(316002)(486006)(6436002)(476003); DIR:OUT; SFP:1101; SCL:1; SRVR:BL0PR1901MB2002; H:BL0PR1901MB2004.namprd19.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: ddn.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 62LVaf2hA/DbqWfFDIO+AT1Mh+7tpazYSzcPEAS+gvwXR1mpB38uSnT0UR5H9Juhslkkp4Xdh1y784UM8dgU1RMLSCf0aXQwwit5aSi9jYL4h9H+KvjmED7iybkk1IuUb050LrcJ37zveaseYRmg7AaQQ05R4A2cPrMTYYkKrO/UlX5Ov1xy/WzSMUJNXxVPQG3xkWh0wf8tXkmxSYd7YpRVVRXDPfD6mXPuPfsrplGunMT8QRSHBer7OyZf2O1NsGsJlYHaLv/l934F9fBSO2qrgzPitSD2jkR7O4iN+fErTQfMDtJXrXtTKEc2v2nGRTayrl1W+z102elybmj52bMp3GsXXDNH7LlHZPDxC5xVSBtQwu68kbG7NtT3qEsrk8AnZEv3KzKeqR5zfzYD0Sx3KyYs3FXbGriSfC6XSp33i2UqQ0zHqTpru3XGntJf x-ms-exchange-transport-forked: True MIME-Version: 1.0 X-OriginatorOrg: ddn.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0d784180-0be7-4f1c-0da5-08d76d73124d X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Nov 2019 04:35:29.6318 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 753b6e26-6fd3-43e6-8248-3f1735d59bb4 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: /j3NozRvzDgfnH16F2FDxsyvjMwX8lAwPovP97HKbmB8FqTpZT8/VSlOV/vAX7D4 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR1901MB2002 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Store the overhead in super block so we don't have to calculate again during next mount. It also gets updated after online resizing. Signed-off-by: Li Dongyang --- fs/ext4/resize.c | 1 + fs/ext4/super.c | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/fs/ext4/resize.c b/fs/ext4/resize.c index c0e9aef376a7..edab58c8ff20 100644 --- a/fs/ext4/resize.c +++ b/fs/ext4/resize.c @@ -1431,6 +1431,7 @@ static void ext4_update_super(struct super_block *sb, * Update the fs overhead information */ ext4_calculate_overhead(sb); + es->s_overhead_clusters = cpu_to_le32(sbi->s_overhead); if (test_opt(sb, DEBUG)) printk(KERN_DEBUG "EXT4-fs: added group %u:" diff --git a/fs/ext4/super.c b/fs/ext4/super.c index dd654e53ba3d..c859c67cd5db 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -4467,6 +4467,10 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent) err = ext4_calculate_overhead(sb); if (err) goto failed_mount_wq; + if (!sb_rdonly(sb)) { + es->s_overhead_clusters = cpu_to_le32(sbi->s_overhead); + ext4_commit_super(sb, 1); + } } /*