From patchwork Tue Feb 22 13:54:31 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rogier Wolff X-Patchwork-Id: 83967 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 7D3ECB6EED for ; Wed, 23 Feb 2011 00:54:34 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752215Ab1BVNyd (ORCPT ); Tue, 22 Feb 2011 08:54:33 -0500 Received: from dtp.xs4all.nl ([80.101.171.8]:58102 "HELO abra2.bitwizard.nl" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with SMTP id S1751693Ab1BVNyd (ORCPT ); Tue, 22 Feb 2011 08:54:33 -0500 Received: (qmail 27827 invoked by uid 1000); 22 Feb 2011 14:54:31 +0100 Date: Tue, 22 Feb 2011 14:54:31 +0100 From: Rogier Wolff To: Rogier Wolff Cc: Andreas Dilger , Pawe?? Brodacki , Amir Goldstein , Ted Ts'o , linux-ext4@vger.kernel.org Subject: Re: fsck performance. Message-ID: <20110222135431.GK21917@bitwizard.nl> References: <20110220193406.GC3017@thunk.org> <20110220215531.GA21917@bitwizard.nl> <20110220222013.GA2849@thunk.org> <20110220231514.GC21917@bitwizard.nl> <20110220234131.GC4001@thunk.org> <20110222102056.GH21917@bitwizard.nl> <20110222133652.GI21917@bitwizard.nl> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20110222133652.GI21917@bitwizard.nl> Organization: BitWizard.nl User-Agent: Mutt/1.5.13 (2006-08-11) Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org On Tue, Feb 22, 2011 at 02:36:52PM +0100, Rogier Wolff wrote: > On Tue, Feb 22, 2011 at 11:20:56AM +0100, Rogier Wolff wrote: > > I wouldn't be surprised if I'd need more than 3G of RAM. When I > > extrapolated "more than a few days" it was at under 20% of the > > filesystem and had already allocated on the order of 800Gb of > > memory. Now I'm not entirely sure that this is fair: memory use seems > > to go up quickly in the beginning, and then stabilize: as if it has > > decided that 800M of memory use is "acceptable" and somehow uses a > > different strategy once it hits that limit. > > OK. Good news. It's finished pass1. It is currently using about 2100Mb > of RAM (ehh. mostly swap, I have only 1G in there). Here is the patch. Forgot the patch. Roger. diff --git a/e2fsck/dirinfo.c b/e2fsck/dirinfo.c index 901235c..f0bf2ad 100644 --- a/e2fsck/dirinfo.c +++ b/e2fsck/dirinfo.c @@ -62,7 +62,7 @@ static void setup_tdb(e2fsck_t ctx, ext2_ino_t num_dirs) uuid_unparse(ctx->fs->super->s_uuid, uuid); sprintf(db->tdb_fn, "%s/%s-dirinfo-XXXXXX", tdb_dir, uuid); fd = mkstemp(db->tdb_fn); - db->tdb = tdb_open(db->tdb_fn, 0, TDB_CLEAR_IF_FIRST, + db->tdb = tdb_open(db->tdb_fn, 99931, TDB_CLEAR_IF_FIRST, O_RDWR | O_CREAT | O_TRUNC, 0600); close(fd); } diff --git a/lib/ext2fs/icount.c b/lib/ext2fs/icount.c index bec0f5f..bba740d 100644 --- a/lib/ext2fs/icount.c +++ b/lib/ext2fs/icount.c @@ -193,7 +193,7 @@ errcode_t ext2fs_create_icount_tdb(ext2_filsys fs, char *tdb_dir, fd = mkstemp(fn); icount->tdb_fn = fn; - icount->tdb = tdb_open(fn, 0, TDB_CLEAR_IF_FIRST, + icount->tdb = tdb_open(fn, 999931, TDB_NOLOCK | TDB_NOSYNC, O_RDWR | O_CREAT | O_TRUNC, 0600); if (icount->tdb) { close(fd);