From patchwork Sun Jan 13 23:27:53 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 211667 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id F15972C0133 for ; Mon, 14 Jan 2013 10:28:18 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755726Ab3AMX17 (ORCPT ); Sun, 13 Jan 2013 18:27:59 -0500 Received: from mail-wi0-f202.google.com ([209.85.212.202]:39341 "EHLO mail-wi0-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756356Ab3AMX14 (ORCPT ); Sun, 13 Jan 2013 18:27:56 -0500 Received: by mail-wi0-f202.google.com with SMTP id hn17so89167wib.1 for ; Sun, 13 Jan 2013 15:27:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:subject:to:from:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-type :content-transfer-encoding; bh=i/5QIvwPqFpVPd2k0bjip3gaMKRmGGw4yFC93eUgqoM=; b=hH6Q9lcKe62//qiUfQ1mb+mAqFVCC0uz2fS4VOPjA6HinlYahSJS6SOsC3yUUegPGS GoqjRdu+nGSgDUnBY3fBAUVQy+Ke2Eu9QDPIQmpxIaJ1+3iB91Y268eYehdK0PyCLb3f nE7thIV49CUt5iidrVsjxx1wRalYOJG7A4MxKDIEJlplzQEiZUw99qGhwEeIxb6r3cTf yYOaAneLeJWqOB2NOrIe8d58NDNJGltAik+2dcOgPUj76l7emGsmzN5BkuEngfUAZome ESDvXpS3diQXeKv1tSgPjYd+YMRkGM41Uz1E79vlmZOvnhYFTBhEtZsSMplukF0A3og7 V0Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:subject:to:from:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-type :content-transfer-encoding:x-gm-message-state; bh=i/5QIvwPqFpVPd2k0bjip3gaMKRmGGw4yFC93eUgqoM=; b=Q9WxGg5GRwIuQJB+qNziwu2u1VZ7ub+RZ03e+9asK8icvI8g9JNyrts5eehISRaQ2i uIwXC0lYoPi8DHV+mX7BayEocryKwUd2haQBku/1Gvy1KaO0ITSaWrVXKSIknQwM4w4J URpiJGJkNDrleFoHOLwPtRVqw3GVDJMBmrbJtF9niWty0sKvHn/a2VsCJUZuNLwUwWQq PPwLKH/NQqWVaDitP3NhHywu8eGn6aR37ahv2OfPW7NaYzmU8IcZoAfs6xN2fSO+W7RC PuK8tA1/Qcpt1uQlhcReyUQ7dDa0reRcmV5VxCNMcsrXb6iBjfD7etoXMp/WjDC6aO1F u1iA== X-Received: by 10.14.179.198 with SMTP id h46mr104322307eem.4.1358119674679; Sun, 13 Jan 2013 15:27:54 -0800 (PST) Received: from hpza10.eem.corp.google.com ([74.125.121.33]) by gmr-mx.google.com with ESMTPS id g9si4440835eeo.1.2013.01.13.15.27.54 (version=TLSv1 cipher=AES128-SHA bits=128/128); Sun, 13 Jan 2013 15:27:54 -0800 (PST) Received: from bhelgaas.mtv.corp.google.com (bhelgaas.mtv.corp.google.com [172.17.131.112]) by hpza10.eem.corp.google.com (Postfix) with ESMTP id E98B0200057; Sun, 13 Jan 2013 15:27:53 -0800 (PST) Received: from bhelgaas.mtv.corp.google.com (unknown [IPv6:::1]) by bhelgaas.mtv.corp.google.com (Postfix) with ESMTP id 5031F18024D; Sun, 13 Jan 2013 15:27:53 -0800 (PST) Subject: [PATCH v7 4/4] PCI: shpchp: Use per-slot workqueues to avoid deadlock To: Yijing Wang , linux-pci@vger.kernel.org From: Bjorn Helgaas Cc: linux-kernel@vger.kernel.org, stable@vger.kernel.org, Daniel J Blueman , Kenji Kaneshige , Tejun Heo , Yinghai Lu , Jiang Liu Date: Sun, 13 Jan 2013 16:27:53 -0700 Message-ID: <20130113232753.7225.31932.stgit@bhelgaas.mtv.corp.google.com> In-Reply-To: <20130113232606.7225.94802.stgit@bhelgaas.mtv.corp.google.com> References: <20130113232606.7225.94802.stgit@bhelgaas.mtv.corp.google.com> User-Agent: StGit/0.15 MIME-Version: 1.0 X-Gm-Message-State: ALoCoQkd5JmpwTwxhPBNnZvvAdaxJowYG63Ax5z1uJdb9UBRv3c6cFY66smk/HLCtFb+gqraJv3jOUADbXQuUipp0KkkVT2w2y+S+8DllUDLlRMNvfyGOLqz2/CnzsMabF432USuO3n731/s0CRMxJdxXRu8N+ML8m9c4VQWttpf5xEf7cQXUlgaBwTacLKBg2sW9RnItV44h+CtpGZCx8VNVKpNqdjsOg== Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org When we have an SHPC-capable bridge with a second SHPC-capable bridge below it, pushing the upstream bridge's attention button causes a deadlock. The deadlock happens because we use the shpchp_wq workqueue to run shpchp_pushbutton_thread(), which uses shpchp_disable_slot() to remove devices below the upstream bridge. When we remove the downstream bridge, we call shpc_remove(), the shpchp driver's .remove() method. That calls flush_workqueue(shpchp_wq), which deadlocks because the shpchp_pushbutton_thread() work item is still running. This patch avoids the deadlock by creating a workqueue for every slot and removing the single shared workqueue. Here's the call path that leads to the deadlock: shpchp_queue_pushbutton_work queue_work(shpchp_wq) # shpchp_pushbutton_thread ... shpchp_pushbutton_thread shpchp_disable_slot remove_board shpchp_unconfigure_device pci_stop_and_remove_bus_device ... shpc_remove # shpchp driver .remove method hpc_release_ctlr cleanup_slots flush_workqueue(shpchp_wq) This change is based on code inspection, since we don't have hardware with this topology. Based-on-patch-by: Yijing Wang Signed-off-by: Bjorn Helgaas CC: stable@vger.kernel.org --- drivers/pci/hotplug/shpchp.h | 2 +- drivers/pci/hotplug/shpchp_core.c | 26 ++++++++++++++------------ drivers/pci/hotplug/shpchp_ctrl.c | 6 +++--- 3 files changed, 18 insertions(+), 16 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/pci/hotplug/shpchp.h b/drivers/pci/hotplug/shpchp.h index 1b69d95..b849f995 100644 --- a/drivers/pci/hotplug/shpchp.h +++ b/drivers/pci/hotplug/shpchp.h @@ -46,7 +46,6 @@ extern bool shpchp_poll_mode; extern int shpchp_poll_time; extern bool shpchp_debug; -extern struct workqueue_struct *shpchp_wq; #define dbg(format, arg...) \ do { \ @@ -90,6 +89,7 @@ struct slot { struct list_head slot_list; struct delayed_work work; /* work for button event */ struct mutex lock; + struct workqueue_struct *wq; u8 hp_slot; }; diff --git a/drivers/pci/hotplug/shpchp_core.c b/drivers/pci/hotplug/shpchp_core.c index 3774e0d..3100c52 100644 --- a/drivers/pci/hotplug/shpchp_core.c +++ b/drivers/pci/hotplug/shpchp_core.c @@ -39,7 +39,6 @@ bool shpchp_debug; bool shpchp_poll_mode; int shpchp_poll_time; -struct workqueue_struct *shpchp_wq; #define DRIVER_VERSION "0.4" #define DRIVER_AUTHOR "Dan Zink , Greg Kroah-Hartman , Dely Sy " @@ -128,6 +127,14 @@ static int init_slots(struct controller *ctrl) slot->device = ctrl->slot_device_offset + i; slot->hpc_ops = ctrl->hpc_ops; slot->number = ctrl->first_slot + (ctrl->slot_num_inc * i); + + snprintf(name, sizeof(name), "shpchp-%d", slot->number); + slot->wq = alloc_workqueue(name, 0, 0); + if (!slot->wq) { + retval = -ENOMEM; + goto error_info; + } + mutex_init(&slot->lock); INIT_DELAYED_WORK(&slot->work, shpchp_queue_pushbutton_work); @@ -147,7 +154,7 @@ static int init_slots(struct controller *ctrl) if (retval) { ctrl_err(ctrl, "pci_hp_register failed with error %d\n", retval); - goto error_info; + goto error_slotwq; } get_power_status(hotplug_slot, &info->power_status); @@ -159,6 +166,8 @@ static int init_slots(struct controller *ctrl) } return 0; +error_slotwq: + destroy_workqueue(slot->wq); error_info: kfree(info); error_hpslot: @@ -179,7 +188,7 @@ void cleanup_slots(struct controller *ctrl) slot = list_entry(tmp, struct slot, slot_list); list_del(&slot->slot_list); cancel_delayed_work(&slot->work); - flush_workqueue(shpchp_wq); + destroy_workqueue(slot->wq); pci_hp_deregister(slot->hotplug_slot); } } @@ -362,18 +371,12 @@ static struct pci_driver shpc_driver = { static int __init shpcd_init(void) { - int retval = 0; - - shpchp_wq = alloc_workqueue("shpchp", 0, 0); - if (!shpchp_wq) - return -ENOMEM; + int retval; retval = pci_register_driver(&shpc_driver); dbg("%s: pci_register_driver = %d\n", __func__, retval); info(DRIVER_DESC " version: " DRIVER_VERSION "\n"); - if (retval) { - destroy_workqueue(shpchp_wq); - } + return retval; } @@ -381,7 +384,6 @@ static void __exit shpcd_cleanup(void) { dbg("unload_shpchpd()\n"); pci_unregister_driver(&shpc_driver); - destroy_workqueue(shpchp_wq); info(DRIVER_DESC " version: " DRIVER_VERSION " unloaded\n"); } diff --git a/drivers/pci/hotplug/shpchp_ctrl.c b/drivers/pci/hotplug/shpchp_ctrl.c index fd2cae9..5849927 100644 --- a/drivers/pci/hotplug/shpchp_ctrl.c +++ b/drivers/pci/hotplug/shpchp_ctrl.c @@ -51,7 +51,7 @@ static int queue_interrupt_event(struct slot *p_slot, u32 event_type) info->p_slot = p_slot; INIT_WORK(&info->work, interrupt_event_handler); - queue_work(shpchp_wq, &info->work); + queue_work(p_slot->wq, &info->work); return 0; } @@ -453,7 +453,7 @@ void shpchp_queue_pushbutton_work(struct work_struct *work) kfree(info); goto out; } - queue_work(shpchp_wq, &info->work); + queue_work(p_slot->wq, &info->work); out: mutex_unlock(&p_slot->lock); } @@ -501,7 +501,7 @@ static void handle_button_press_event(struct slot *p_slot) p_slot->hpc_ops->green_led_blink(p_slot); p_slot->hpc_ops->set_attention_status(p_slot, 0); - queue_delayed_work(shpchp_wq, &p_slot->work, 5*HZ); + queue_delayed_work(p_slot->wq, &p_slot->work, 5*HZ); break; case BLINKINGOFF_STATE: case BLINKINGON_STATE: