From patchwork Wed Aug 4 16:39:17 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Theodore Ts'o X-Patchwork-Id: 60872 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 5F4C41007D1 for ; Thu, 5 Aug 2010 02:39:22 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755505Ab0HDQjT (ORCPT ); Wed, 4 Aug 2010 12:39:19 -0400 Received: from thunk.org ([69.25.196.29]:46662 "EHLO thunker.thunk.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755682Ab0HDQjS (ORCPT ); Wed, 4 Aug 2010 12:39:18 -0400 Received: from root (helo=tytso-glaptop) by thunker.thunk.org with local-esmtp (Exim 4.50 #1 (Debian)) id 1Oggzp-0007qz-Vh; Wed, 04 Aug 2010 12:39:14 -0400 Received: from tytso by tytso-glaptop with local (Exim 4.71) (envelope-from ) id 1Oggzu-0000rY-4L; Wed, 04 Aug 2010 12:39:18 -0400 From: Theodore Ts'o To: Ext4 Developers List , ocfs2-devel@oss.oracle.com Cc: John Stultz , Keith Maanthey , Eric Whitney , Theodore Ts'o Subject: [PATCH -v2 3/3] jbd2: Remove t_handle_lock from start_this_handle() Date: Wed, 4 Aug 2010 12:39:17 -0400 Message-Id: <1280939957-3277-4-git-send-email-tytso@mit.edu> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1280939957-3277-1-git-send-email-tytso@mit.edu> References: <1280939957-3277-1-git-send-email-tytso@mit.edu> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: tytso@thunk.org X-SA-Exim-Scanned: No (on thunker.thunk.org); SAEximRunCond expanded to false Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org This should remove the last exclusive lock from start_this_handle(), so that we should now be able to start multiple transactions at the same time on large SMP systems. Signed-off-by: "Theodore Ts'o" --- fs/jbd2/commit.c | 3 ++- fs/jbd2/transaction.c | 33 ++++++++++++++++++++++----------- include/linux/jbd2.h | 2 +- 3 files changed, 25 insertions(+), 13 deletions(-) diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c index 67bb0a2..f52e5e8 100644 --- a/fs/jbd2/commit.c +++ b/fs/jbd2/commit.c @@ -1004,7 +1004,8 @@ restart_loop: * File the transaction statistics */ stats.ts_tid = commit_transaction->t_tid; - stats.run.rs_handle_count = commit_transaction->t_handle_count; + stats.run.rs_handle_count = + atomic_read(&commit_transaction->t_handle_count); trace_jbd2_run_stats(journal->j_fs_dev->bd_dev, commit_transaction->t_tid, &stats.run); diff --git a/fs/jbd2/transaction.c b/fs/jbd2/transaction.c index 6630651..0752bcd 100644 --- a/fs/jbd2/transaction.c +++ b/fs/jbd2/transaction.c @@ -57,6 +57,7 @@ jbd2_get_transaction(journal_t *journal, transaction_t *transaction) spin_lock_init(&transaction->t_handle_lock); atomic_set(&transaction->t_updates, 0); atomic_set(&transaction->t_outstanding_credits, 0); + atomic_set(&transaction->t_handle_count, 0); INIT_LIST_HEAD(&transaction->t_inode_list); INIT_LIST_HEAD(&transaction->t_private_list); @@ -180,8 +181,8 @@ repeat: * buffers requested by this operation, we need to stall pending a log * checkpoint to free some more log space. */ - spin_lock(&transaction->t_handle_lock); - needed = atomic_read(&transaction->t_outstanding_credits) + nblocks; + needed = atomic_add_return(nblocks, + &transaction->t_outstanding_credits); if (needed > journal->j_max_transaction_buffers) { /* @@ -192,7 +193,7 @@ repeat: DEFINE_WAIT(wait); jbd_debug(2, "Handle %p starting new commit...\n", handle); - spin_unlock(&transaction->t_handle_lock); + atomic_sub(nblocks, &transaction->t_outstanding_credits); prepare_to_wait(&journal->j_wait_transaction_locked, &wait, TASK_UNINTERRUPTIBLE); __jbd2_log_start_commit(journal, transaction->t_tid); @@ -229,7 +230,7 @@ repeat: */ if (__jbd2_log_space_left(journal) < jbd_space_needed(journal)) { jbd_debug(2, "Handle %p waiting for checkpoint...\n", handle); - spin_unlock(&transaction->t_handle_lock); + atomic_sub(nblocks, &transaction->t_outstanding_credits); read_unlock(&journal->j_state_lock); write_lock(&journal->j_state_lock); if (__jbd2_log_space_left(journal) < jbd_space_needed(journal)) @@ -239,23 +240,33 @@ repeat: } /* OK, account for the buffers that this operation expects to - * use and add the handle to the running transaction. */ - - if (time_after(transaction->t_start, ts)) { + * use and add the handle to the running transaction. + * + * In order for t_max_wait to be reliable, it must be + * protected by a lock. But doing so will mean that + * start_this_handle() can not be run in parallel on SMP + * systems, which limits our scalability. So we only enable + * it when debugging is enabled. We may want to use a + * separate flag, eventually, so we can enable this + * independently of debugging. + */ +#ifdef CONFIG_JBD2_DEBUG + if (jbd2_journal_enable_debug && + time_after(transaction->t_start, ts)) { ts = jbd2_time_diff(ts, transaction->t_start); + spin_lock(&transaction->t_handle_lock); if (ts > transaction->t_max_wait) transaction->t_max_wait = ts; + spin_unlock(&transaction->t_handle_lock); } - +#endif handle->h_transaction = transaction; - atomic_add(nblocks, &transaction->t_outstanding_credits); atomic_inc(&transaction->t_updates); - transaction->t_handle_count++; + atomic_inc(&transaction->t_handle_count); jbd_debug(4, "Handle %p given %d credits (total %d, free %d)\n", handle, nblocks, atomic_read(&transaction->t_outstanding_credits), __jbd2_log_space_left(journal)); - spin_unlock(&transaction->t_handle_lock); read_unlock(&journal->j_state_lock); lock_map_acquire(&handle->h_lockdep_map); diff --git a/include/linux/jbd2.h b/include/linux/jbd2.h index 15d5743..01743b5 100644 --- a/include/linux/jbd2.h +++ b/include/linux/jbd2.h @@ -629,7 +629,7 @@ struct transaction_s /* * How many handles used this transaction? [t_handle_lock] */ - int t_handle_count; + atomic_t t_handle_count; /* * This transaction is being forced and some process is