From patchwork Tue Jul 30 11:24:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 1138938 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linutronix.de Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 45yb162w3pz9sBF for ; Tue, 30 Jul 2019 22:08:10 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729215AbfG3MHw (ORCPT ); Tue, 30 Jul 2019 08:07:52 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:56554 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727206AbfG3MHu (ORCPT ); Tue, 30 Jul 2019 08:07:50 -0400 Received: from localhost ([127.0.0.1] helo=nanos.tec.linutronix.de) by Galois.linutronix.de with esmtp (Exim 4.80) (envelope-from ) id 1hsQuL-0006QF-SX; Tue, 30 Jul 2019 14:07:09 +0200 Message-Id: <20190730120321.193069837@linutronix.de> User-Agent: quilt/0.65 Date: Tue, 30 Jul 2019 13:24:53 +0200 From: Thomas Gleixner To: LKML Cc: Peter Zijlstra , Ingo Molnar , Sebastian Siewior , Anna-Maria Gleixner , Steven Rostedt , Julia Cartwright , "Theodore Tso" , Matthew Wilcox , Alexander Viro , linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, Jan Kara Subject: [patch 1/4] locking/lockdep: Add Kconfig option for bit spinlocks References: <20190730112452.871257694@linutronix.de> MIME-Version: 1.0 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Some usage sites of bit spinlocks have a substitution with regular spinlocks which depends on CONFIG_PREEMPT_RT. But this substitution can also be used to expose these locks to the regular lock debugging infrastructure, e.g. lockdep. As this increases the size of affected data structures significantly this is guarded by a separate Kconfig switch. Note, that only the bit spinlocks which have a substitution implemented will be covered by this. All other bit spinlocks evade lock debugging as before. Signed-off-by: Thomas Gleixner --- lib/Kconfig.debug | 10 ++++++++++ 1 file changed, 10 insertions(+) --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1201,6 +1201,16 @@ config DEBUG_RWSEMS This debugging feature allows mismatched rw semaphore locks and unlocks to be detected and reported. +config DEBUG_BIT_SPINLOCKS + bool "Bit spinlock debugging" + depends on DEBUG_SPINLOCK + help + This debugging feature substitutes bit spinlocks in some use + cases, e.g. buffer head, zram, with with regular spinlocks so + these locks are exposed to lock debugging features. + + Not all bit spinlocks are covered by this. + config DEBUG_LOCK_ALLOC bool "Lock debugging: detect incorrect freeing of live locks" depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT From patchwork Tue Jul 30 11:24:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 1138937 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linutronix.de Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 45yb140Mn3z9sDB for ; Tue, 30 Jul 2019 22:08:08 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729348AbfG3MHx (ORCPT ); Tue, 30 Jul 2019 08:07:53 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:56555 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729088AbfG3MHx (ORCPT ); Tue, 30 Jul 2019 08:07:53 -0400 Received: from localhost ([127.0.0.1] helo=nanos.tec.linutronix.de) by Galois.linutronix.de with esmtp (Exim 4.80) (envelope-from ) id 1hsQuM-0006QI-An; Tue, 30 Jul 2019 14:07:10 +0200 Message-Id: <20190730120321.285095769@linutronix.de> User-Agent: quilt/0.65 Date: Tue, 30 Jul 2019 13:24:54 +0200 From: Thomas Gleixner To: LKML Cc: Peter Zijlstra , Ingo Molnar , Sebastian Siewior , Anna-Maria Gleixner , Steven Rostedt , Julia Cartwright , "Theodore Tso" , Matthew Wilcox , Alexander Viro , linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, Jan Kara Subject: [patch 2/4] fs/buffer: Move BH_Uptodate_Lock locking into wrapper functions References: <20190730112452.871257694@linutronix.de> MIME-Version: 1.0 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Bit spinlocks are problematic if PREEMPT_RT is enabled, because they disable preemption, which is undesired for latency reasons and breaks when regular spinlocks are taken within the bit_spinlock locked region because regular spinlocks are converted to 'sleeping spinlocks' on RT. So RT replaces the bit spinlocks with regular spinlocks to avoid this problem. To avoid ifdeffery at the source level, wrap all BH_Uptodate_Lock bitlock operations with inline functions, so the spinlock substitution can be done at one place. Using regular spinlocks can also be enabled for lock debugging purposes so the lock operations become visible to lockdep. Signed-off-by: Thomas Gleixner Cc: "Theodore Ts'o" Cc: Matthew Wilcox Cc: Alexander Viro Cc: linux-fsdevel@vger.kernel.org Reviewed-by: Jan Kara --- fs/buffer.c | 20 ++++++-------------- fs/ext4/page-io.c | 6 ++---- fs/ntfs/aops.c | 10 +++------- include/linux/buffer_head.h | 16 ++++++++++++++++ 4 files changed, 27 insertions(+), 25 deletions(-) --- a/fs/buffer.c +++ b/fs/buffer.c @@ -275,8 +275,7 @@ static void end_buffer_async_read(struct * decide that the page is now completely done. */ first = page_buffers(page); - local_irq_save(flags); - bit_spin_lock(BH_Uptodate_Lock, &first->b_state); + flags = bh_uptodate_lock_irqsave(first); clear_buffer_async_read(bh); unlock_buffer(bh); tmp = bh; @@ -289,8 +288,7 @@ static void end_buffer_async_read(struct } tmp = tmp->b_this_page; } while (tmp != bh); - bit_spin_unlock(BH_Uptodate_Lock, &first->b_state); - local_irq_restore(flags); + bh_uptodate_unlock_irqrestore(first, flags); /* * If none of the buffers had errors and they are all @@ -302,9 +300,7 @@ static void end_buffer_async_read(struct return; still_busy: - bit_spin_unlock(BH_Uptodate_Lock, &first->b_state); - local_irq_restore(flags); - return; + bh_uptodate_unlock_irqrestore(first, flags); } /* @@ -331,8 +327,7 @@ void end_buffer_async_write(struct buffe } first = page_buffers(page); - local_irq_save(flags); - bit_spin_lock(BH_Uptodate_Lock, &first->b_state); + flags = bh_uptodate_lock_irqsave(first); clear_buffer_async_write(bh); unlock_buffer(bh); @@ -344,15 +339,12 @@ void end_buffer_async_write(struct buffe } tmp = tmp->b_this_page; } - bit_spin_unlock(BH_Uptodate_Lock, &first->b_state); - local_irq_restore(flags); + bh_uptodate_unlock_irqrestore(first, flags); end_page_writeback(page); return; still_busy: - bit_spin_unlock(BH_Uptodate_Lock, &first->b_state); - local_irq_restore(flags); - return; + bh_uptodate_unlock_irqrestore(first, flags); } EXPORT_SYMBOL(end_buffer_async_write); --- a/fs/ext4/page-io.c +++ b/fs/ext4/page-io.c @@ -90,8 +90,7 @@ static void ext4_finish_bio(struct bio * * We check all buffers in the page under BH_Uptodate_Lock * to avoid races with other end io clearing async_write flags */ - local_irq_save(flags); - bit_spin_lock(BH_Uptodate_Lock, &head->b_state); + flags = bh_uptodate_lock_irqsave(head); do { if (bh_offset(bh) < bio_start || bh_offset(bh) + bh->b_size > bio_end) { @@ -103,8 +102,7 @@ static void ext4_finish_bio(struct bio * if (bio->bi_status) buffer_io_error(bh); } while ((bh = bh->b_this_page) != head); - bit_spin_unlock(BH_Uptodate_Lock, &head->b_state); - local_irq_restore(flags); + bh_uptodate_unlock_irqrestore(head, flags); if (!under_io) { fscrypt_free_bounce_page(bounce_page); end_page_writeback(page); --- a/fs/ntfs/aops.c +++ b/fs/ntfs/aops.c @@ -92,8 +92,7 @@ static void ntfs_end_buffer_async_read(s "0x%llx.", (unsigned long long)bh->b_blocknr); } first = page_buffers(page); - local_irq_save(flags); - bit_spin_lock(BH_Uptodate_Lock, &first->b_state); + flags = bh_uptodate_lock_irqsave(first); clear_buffer_async_read(bh); unlock_buffer(bh); tmp = bh; @@ -108,8 +107,7 @@ static void ntfs_end_buffer_async_read(s } tmp = tmp->b_this_page; } while (tmp != bh); - bit_spin_unlock(BH_Uptodate_Lock, &first->b_state); - local_irq_restore(flags); + bh_uptodate_unlock_irqrestore(first, flags); /* * If none of the buffers had errors then we can set the page uptodate, * but we first have to perform the post read mst fixups, if the @@ -142,9 +140,7 @@ static void ntfs_end_buffer_async_read(s unlock_page(page); return; still_busy: - bit_spin_unlock(BH_Uptodate_Lock, &first->b_state); - local_irq_restore(flags); - return; + bh_uptodate_unlock_irqrestore(first, flags); } /** --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -78,6 +78,22 @@ struct buffer_head { atomic_t b_count; /* users using this buffer_head */ }; +static inline unsigned long bh_uptodate_lock_irqsave(struct buffer_head *bh) +{ + unsigned long flags; + + local_irq_save(flags); + bit_spin_lock(BH_Uptodate_Lock, &bh->b_state); + return flags; +} + +static inline void +bh_uptodate_unlock_irqrestore(struct buffer_head *bh, unsigned long flags) +{ + bit_spin_unlock(BH_Uptodate_Lock, &bh->b_state); + local_irq_restore(flags); +} + /* * macro tricks to expand the set_buffer_foo(), clear_buffer_foo() * and buffer_foo() functions. From patchwork Tue Jul 30 11:24:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 1138936 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linutronix.de Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 45yb1208Zhz9sML for ; Tue, 30 Jul 2019 22:08:06 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729462AbfG3MHx (ORCPT ); Tue, 30 Jul 2019 08:07:53 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:56557 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729191AbfG3MHx (ORCPT ); Tue, 30 Jul 2019 08:07:53 -0400 Received: from localhost ([127.0.0.1] helo=nanos.tec.linutronix.de) by Galois.linutronix.de with esmtp (Exim 4.80) (envelope-from ) id 1hsQuM-0006QL-Op; Tue, 30 Jul 2019 14:07:10 +0200 Message-Id: <20190730120321.393759046@linutronix.de> User-Agent: quilt/0.65 Date: Tue, 30 Jul 2019 13:24:55 +0200 From: Thomas Gleixner To: LKML Cc: Peter Zijlstra , Ingo Molnar , Sebastian Siewior , Anna-Maria Gleixner , Steven Rostedt , Julia Cartwright , "Theodore Tso" , Alexander Viro , Matthew Wilcox , linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, Jan Kara Subject: [patch 3/4] fs/buffer: Substitute BH_Uptodate_Lock for RT and bit spinlock debugging References: <20190730112452.871257694@linutronix.de> MIME-Version: 1.0 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Bit spinlocks are problematic if PREEMPT_RT is enabled. They disable preemption, which is undesired for latency reasons and breaks when regular spinlocks are taken within the bit_spinlock locked region because regular spinlocks are converted to 'sleeping spinlocks' on RT. Substitute the BH_Uptodate_Lock bit spinlock with a regular spinlock for PREEMPT_RT enabled kernels. Bit spinlocks are also not covered by lock debugging, e.g. lockdep. With the spinlock substitution in place, they can be exposed via CONFIG_DEBUG_BIT_SPINLOCKS. Signed-off-by: Thomas Gleixner Cc: "Theodore Ts'o" Cc: Alexander Viro Cc: Matthew Wilcox Cc: linux-fsdevel@vger.kernel.org Reviewed-by: Jan Kara --- fs/buffer.c | 1 + include/linux/buffer_head.h | 31 +++++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+) --- a/fs/buffer.c +++ b/fs/buffer.c @@ -3360,6 +3360,7 @@ struct buffer_head *alloc_buffer_head(gf struct buffer_head *ret = kmem_cache_zalloc(bh_cachep, gfp_flags); if (ret) { INIT_LIST_HEAD(&ret->b_assoc_buffers); + buffer_head_init_locks(ret); preempt_disable(); __this_cpu_inc(bh_accounting.nr); recalc_bh_state(); --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -76,8 +76,35 @@ struct buffer_head { struct address_space *b_assoc_map; /* mapping this buffer is associated with */ atomic_t b_count; /* users using this buffer_head */ + +#if defined(CONFIG_PREEMPT_RT) || defined(CONFIG_DEBUG_BIT_SPINLOCKS) + spinlock_t b_uptodate_lock; +#endif }; +#if defined(CONFIG_PREEMPT_RT) || defined(CONFIG_DEBUG_BIT_SPINLOCKS) + +static inline unsigned long bh_uptodate_lock_irqsave(struct buffer_head *bh) +{ + unsigned long flags; + + spin_lock_irqsave(&bh->b_uptodate_lock, flags); + return flags; +} + +static inline void +bh_uptodate_unlock_irqrestore(struct buffer_head *bh, unsigned long flags) +{ + spin_unlock_irqrestore(&bh->b_uptodate_lock, flags); +} + +static inline void buffer_head_init_locks(struct buffer_head *bh) +{ + spin_lock_init(&bh->b_uptodate_lock); +} + +#else /* PREEMPT_RT || DEBUG_BIT_SPINLOCKS */ + static inline unsigned long bh_uptodate_lock_irqsave(struct buffer_head *bh) { unsigned long flags; @@ -94,6 +121,10 @@ bh_uptodate_unlock_irqrestore(struct buf local_irq_restore(flags); } +static inline void buffer_head_init_locks(struct buffer_head *bh) { } + +#endif /* !PREEMPT_RT && !DEBUG_BIT_SPINLOCKS */ + /* * macro tricks to expand the set_buffer_foo(), clear_buffer_foo() * and buffer_foo() functions. From patchwork Tue Jul 30 11:24:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Gleixner X-Patchwork-Id: 1138940 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linutronix.de Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 45yb1w5mxDz9s3Z for ; Tue, 30 Jul 2019 22:08:52 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729279AbfG3MIr (ORCPT ); Tue, 30 Jul 2019 08:08:47 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:56565 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727561AbfG3MIr (ORCPT ); Tue, 30 Jul 2019 08:08:47 -0400 Received: from localhost ([127.0.0.1] helo=nanos.tec.linutronix.de) by Galois.linutronix.de with esmtp (Exim 4.80) (envelope-from ) id 1hsQuN-0006QO-69; Tue, 30 Jul 2019 14:07:11 +0200 Message-Id: <20190730120321.489374435@linutronix.de> User-Agent: quilt/0.65 Date: Tue, 30 Jul 2019 13:24:56 +0200 From: Thomas Gleixner To: LKML Cc: Peter Zijlstra , Ingo Molnar , Sebastian Siewior , Anna-Maria Gleixner , Steven Rostedt , Julia Cartwright , linux-ext4@vger.kernel.org, "Theodore Tso" , Matthew Wilcox , Jan Kara , Alexander Viro , linux-fsdevel@vger.kernel.org Subject: [patch 4/4] fs: jbd/jbd2: Substitute BH locks for RT and lock debugging References: <20190730112452.871257694@linutronix.de> MIME-Version: 1.0 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Bit spinlocks are problematic if PREEMPT_RT is enabled. They disable preemption, which is undesired for latency reasons and breaks when regular spinlocks are taken within the bit_spinlock locked region because regular spinlocks are converted to 'sleeping spinlocks' on RT. Substitute the BH_State and BH_JournalHead bit spinlocks with regular spinlock for PREEMPT_RT enabled kernels. Bit spinlocks are also not covered by lock debugging, e.g. lockdep. With the spinlock substitution in place, they can be exposed via CONFIG_DEBUG_BIT_SPINLOCKS. Originally-by: Steven Rostedt Signed-off-by: Thomas Gleixner Cc: linux-ext4@vger.kernel.org Cc: "Theodore Ts'o" Cc: Matthew Wilcox Cc: Jan Kara --- include/linux/buffer_head.h | 8 ++++++++ include/linux/jbd2.h | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+) --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -79,6 +79,10 @@ struct buffer_head { #if defined(CONFIG_PREEMPT_RT) || defined(CONFIG_DEBUG_BIT_SPINLOCKS) spinlock_t b_uptodate_lock; +# if IS_ENABLED(CONFIG_JBD2) + spinlock_t b_state_lock; + spinlock_t b_journal_head_lock; +# endif #endif }; @@ -101,6 +105,10 @@ bh_uptodate_unlock_irqrestore(struct buf static inline void buffer_head_init_locks(struct buffer_head *bh) { spin_lock_init(&bh->b_uptodate_lock); +#if IS_ENABLED(CONFIG_JBD2) + spin_lock_init(&bh->b_state_lock); + spin_lock_init(&bh->b_journal_head_lock); +#endif } #else /* PREEMPT_RT || DEBUG_BIT_SPINLOCKS */ --- a/include/linux/jbd2.h +++ b/include/linux/jbd2.h @@ -342,6 +342,40 @@ static inline struct journal_head *bh2jh return bh->b_private; } +#if defined(CONFIG_PREEMPT_RT) || defined(CONFIG_DEBUG_BIT_SPINLOCKS) + +static inline void jbd_lock_bh_state(struct buffer_head *bh) +{ + spin_lock(&bh->b_state_lock); +} + +static inline int jbd_trylock_bh_state(struct buffer_head *bh) +{ + return spin_trylock(&bh->b_state_lock); +} + +static inline int jbd_is_locked_bh_state(struct buffer_head *bh) +{ + return spin_is_locked(&bh->b_state_lock); +} + +static inline void jbd_unlock_bh_state(struct buffer_head *bh) +{ + spin_unlock(&bh->b_state_lock); +} + +static inline void jbd_lock_bh_journal_head(struct buffer_head *bh) +{ + spin_lock(&bh->b_journal_head_lock); +} + +static inline void jbd_unlock_bh_journal_head(struct buffer_head *bh) +{ + spin_unlock(&bh->b_journal_head_lock); +} + +#else /* PREEMPT_RT || DEBUG_BIT_SPINLOCKS */ + static inline void jbd_lock_bh_state(struct buffer_head *bh) { bit_spin_lock(BH_State, &bh->b_state); @@ -372,6 +406,8 @@ static inline void jbd_unlock_bh_journal bit_spin_unlock(BH_JournalHead, &bh->b_state); } +#endif /* !PREEMPT_RT && !DEBUG_BIT_SPINLOCKS */ + #define J_ASSERT(assert) BUG_ON(!(assert)) #define J_ASSERT_BH(bh, expr) J_ASSERT(expr)