From patchwork Fri Jun 28 18:51:09 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kamal Mostafa X-Patchwork-Id: 255586 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) by ozlabs.org (Postfix) with ESMTP id D56DE2C0099 for ; Sat, 29 Jun 2013 04:56:58 +1000 (EST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.76) (envelope-from ) id 1Usdqm-0005fD-5t; Fri, 28 Jun 2013 18:56:52 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtp (Exim 4.76) (envelope-from ) id 1Usdqc-0005ZV-Sz for kernel-team@lists.ubuntu.com; Fri, 28 Jun 2013 18:56:42 +0000 Received: from c-67-160-231-42.hsd1.ca.comcast.net ([67.160.231.42] helo=fourier) by youngberry.canonical.com with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1UsdmV-0007Yp-Uo; Fri, 28 Jun 2013 18:52:28 +0000 Received: from kamal by fourier with local (Exim 4.80) (envelope-from ) id 1UsdmT-0006Wp-Qf; Fri, 28 Jun 2013 11:52:25 -0700 From: Kamal Mostafa To: linux-kernel@vger.kernel.org, stable@vger.kernel.org, kernel-team@lists.ubuntu.com Subject: [PATCH 047/105] target/iscsi: don't corrupt bh_count in iscsit_stop_time2retain_timer() Date: Fri, 28 Jun 2013 11:51:09 -0700 Message-Id: <1372445527-24414-48-git-send-email-kamal@canonical.com> X-Mailer: git-send-email 1.8.1.2 In-Reply-To: <1372445527-24414-1-git-send-email-kamal@canonical.com> References: <1372445527-24414-1-git-send-email-kamal@canonical.com> X-Extended-Stable: 3.8 Cc: Joern Engel , Kamal Mostafa , Nicholas Bellinger X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.14 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: kernel-team-bounces@lists.ubuntu.com 3.8.13.4 -stable review patch. If anyone has any objections, please let me know. ------------------ From: =?UTF-8?q?J=C3=B6rn=20Engel?= commit 574780fd5e6ec52bd43e0bdb777a19e4c4c6aa9c upstream. Here is a fun one. Bug seems to have been introduced by commit 140854cb, almost two years ago. I have no idea why we only started seeing it now, but we did. Rough callgraph: core_tpg_set_initiator_node_queue_depth() `-> spin_lock_irqsave(&tpg->session_lock, flags); `-> lio_tpg_shutdown_session() `-> iscsit_stop_time2retain_timer() `-> spin_unlock_bh(&se_tpg->session_lock); `-> spin_lock_bh(&se_tpg->session_lock); `-> spin_unlock_irqrestore(&tpg->session_lock, flags); core_tpg_set_initiator_node_queue_depth() used to call spin_lock_bh(), but 140854cb changed that to spin_lock_irqsave(). However, lio_tpg_shutdown_session() still claims to be called with spin_lock_bh() held, as does iscsit_stop_time2retain_timer(): * Called with spin_lock_bh(&struct se_portal_group->session_lock) held Stale documentation is mostly annoying, but in this case the dropping the lock with the _bh variant is plain wrong. It is also wrong to drop locks two functions below the lock-holder, but I will ignore that bit for now. After some more locking and unlocking we eventually hit this backtrace: ------------[ cut here ]------------ WARNING: at kernel/softirq.c:159 local_bh_enable_ip+0xe8/0x100() Pid: 24645, comm: lio_helper.py Tainted: G O 3.6.11+ Call Trace: [] warn_slowpath_common+0x7f/0xc0 [] ? iscsit_inc_conn_usage_count+0x37/0x50 [iscsi_target_mod] [] warn_slowpath_null+0x1a/0x20 [] local_bh_enable_ip+0xe8/0x100 [] _raw_spin_unlock_bh+0x15/0x20 [] iscsit_inc_conn_usage_count+0x37/0x50 [iscsi_target_mod] [] iscsit_stop_session+0xfa/0x1c0 [iscsi_target_mod] [] lio_tpg_shutdown_session+0x7b/0x90 [iscsi_target_mod] [] core_tpg_set_initiator_node_queue_depth+0xe4/0x290 [target_core_mod] [] iscsit_tpg_set_initiator_node_queue_depth+0x12/0x20 [iscsi_target_mod] [] lio_target_nacl_store_cmdsn_depth+0xa9/0x180 [iscsi_target_mod] [] target_fabric_nacl_base_attr_store+0x39/0x40 [target_core_mod] [] configfs_write_file+0xbd/0x120 [] vfs_write+0xc6/0x180 [] sys_write+0x51/0x90 [] system_call_fastpath+0x16/0x1b ---[ end trace 3747632b9b164652 ]--- As a pure band-aid, this patch drops the _bh. Signed-off-by: Joern Engel Signed-off-by: Nicholas Bellinger Signed-off-by: Kamal Mostafa --- drivers/target/iscsi/iscsi_target_erl0.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/target/iscsi/iscsi_target_erl0.c b/drivers/target/iscsi/iscsi_target_erl0.c index 8e6298c..dcb199d 100644 --- a/drivers/target/iscsi/iscsi_target_erl0.c +++ b/drivers/target/iscsi/iscsi_target_erl0.c @@ -842,11 +842,11 @@ int iscsit_stop_time2retain_timer(struct iscsi_session *sess) return 0; sess->time2retain_timer_flags |= ISCSI_TF_STOP; - spin_unlock_bh(&se_tpg->session_lock); + spin_unlock(&se_tpg->session_lock); del_timer_sync(&sess->time2retain_timer); - spin_lock_bh(&se_tpg->session_lock); + spin_lock(&se_tpg->session_lock); sess->time2retain_timer_flags &= ~ISCSI_TF_RUNNING; pr_debug("Stopped Time2Retain Timer for SID: %u\n", sess->sid);