From patchwork Fri Jun 29 18:25:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Tatashin X-Patchwork-Id: 937087 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=osuosl.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.b="jgi48upn"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41HQ7p6bTgz9s31 for ; Sat, 30 Jun 2018 04:25:58 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id E8AEA81EB5; Fri, 29 Jun 2018 18:25:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Tm0hhIwCyIdJ; Fri, 29 Jun 2018 18:25:56 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id F246A81EBD; Fri, 29 Jun 2018 18:25:55 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id DFE021C156B for ; Fri, 29 Jun 2018 18:25:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id DB3D0220B3 for ; Fri, 29 Jun 2018 18:25:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QfCvBWzcVdUK for ; Fri, 29 Jun 2018 18:25:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from userp2120.oracle.com (userp2120.oracle.com [156.151.31.85]) by silver.osuosl.org (Postfix) with ESMTPS id 24DC622029 for ; Fri, 29 Jun 2018 18:25:54 +0000 (UTC) Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w5TIO7mk095926; Fri, 29 Jun 2018 18:25:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2017-10-26; bh=63/ZMt3BnYN0KGlKozL16pnCbJ1B0MjOGFxZdYS10XI=; b=jgi48upn8BVksu2NeqnB5qGFuEbnjKPYZtvSMKKFJi+PdF/KODGz9zOaLa2Xa2rVgkuU 3kurI0R9wApFn6J90pGakq7DYlt2lCQMApli6Lg8qYirU43V/TgVsoqsRjhAOJ5rUrC3 MFppGA33oprvAYjRA/q2dkLVivdFtyXDYK/tkREjAnu1BxcTj27T5zYfcHLZ0OCLkqgI MT8wgx9GABJ6o6/cMj+ucphOLDNQ1eOIAP4c2/YK8Tpb5TJD8f+UGBOcIpdEBSuHm9pq nMoLsCyLVj24V6Asq4hmxV8d12cdPI8SbgAmtLtvhllpChtN2emF78NJCJq74DOGFy5s CA== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2jum0afh72-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 29 Jun 2018 18:25:49 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w5TIPmM3017155 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 29 Jun 2018 18:25:49 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w5TIPmkW019460; Fri, 29 Jun 2018 18:25:48 GMT Received: from localhost.localdomain (/73.69.118.222) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 29 Jun 2018 11:25:48 -0700 From: Pavel Tatashin To: pasha.tatashin@oracle.com, steven.sistare@oracle.com, daniel.m.jordan@oracle.com, linux-kernel@vger.kernel.org, jeffrey.t.kirsher@intel.com, intel-wired-lan@lists.osuosl.org, netdev@vger.kernel.org, gregkh@linuxfoundation.org, alexander.duyck@gmail.com, tobin@apporbit.com, andy.shevchenko@gmail.com Date: Fri, 29 Jun 2018 14:25:40 -0400 Message-Id: <20180629182541.6735-3-pasha.tatashin@oracle.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180629182541.6735-1-pasha.tatashin@oracle.com> References: <20180629182541.6735-1-pasha.tatashin@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8939 signatures=668703 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1806290195 Subject: [Intel-wired-lan] [PATCH v6 2/3] drivers core: prepare device_shutdown for multi-threading X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Do all the necessary refactoring to prepare device_shutdown() logic to be multi-threaded. Which includes: 1. Change the direction of traversing the list instead of going backward, we now go forward. 2. Children are shutdown recursively for each root device from bottom-up. 3. Functions that can be multi-threaded have _task() in their name. Signed-off-by: Pavel Tatashin --- drivers/base/core.c | 178 ++++++++++++++++++++++++++++++++++++-------- 1 file changed, 149 insertions(+), 29 deletions(-) diff --git a/drivers/base/core.c b/drivers/base/core.c index 07380aa0683b..5d8623b38d4d 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -2134,6 +2134,59 @@ const char *device_get_devnode(struct device *dev, return *tmp = s; } +/* + * device_children_count - device children count + * @parent: parent struct device. + * + * Returns number of children for this device or 0 if none. + */ +static int device_children_count(struct device *parent) +{ + struct klist_iter i; + int children = 0; + + if (!parent->p) + return 0; + + klist_iter_init(&parent->p->klist_children, &i); + while (next_device(&i)) + children++; + klist_iter_exit(&i); + + return children; +} + +/* + * device_get_child_by_index - Return child using the provided index. + * @parent: parent struct device. + * @index: Index of the child, where 0 is the first child in the children list, + * and so on. + * + * Returns child or NULL if child with this index is not present. + */ +static struct device * +device_get_child_by_index(struct device *parent, int index) +{ + struct klist_iter i; + struct device *dev = NULL, *d; + int child_index = 0; + + if (!parent->p || index < 0) + return NULL; + + klist_iter_init(&parent->p->klist_children, &i); + while ((d = next_device(&i))) { + if (child_index == index) { + dev = d; + break; + } + child_index++; + } + klist_iter_exit(&i); + + return dev; +} + /** * device_for_each_child - device child iterator. * @parent: parent struct device. @@ -2831,50 +2884,117 @@ static void device_shutdown_one(struct device *dev) put_device(dev); } +/* + * Passed as an argument to device_shutdown_child_task(). + * child_next_index the next available child index. + * parent Parent device. + */ +struct device_shutdown_task_data { + atomic_t child_next_index; + struct device *parent; +}; + +static int device_shutdown_child_task(void *data); + +/* + * Shutdown device tree with root started in dev. If dev has no children + * simply shutdown only this device. If dev has children recursively shutdown + * children first, and only then the parent. + */ +static void device_shutdown_tree(struct device *dev) +{ + int children_count; + + device_lock(dev); + children_count = device_children_count(dev); + + if (children_count) { + struct device_shutdown_task_data tdata; + int i; + + atomic_set(&tdata.child_next_index, 0); + tdata.parent = dev; + + for (i = 0; i < children_count; i++) { + device_shutdown_child_task(&tdata); + } + } + device_shutdown_one(dev); + device_unlock(dev); +} + +/* + * Only devices with parent are going through this function. The parent is + * locked and waits for all of its children to finish shutting down before + * calling shutdown function for itself. + */ +static int device_shutdown_child_task(void *data) +{ + struct device_shutdown_task_data *tdata = data; + int cidx = atomic_inc_return(&tdata->child_next_index) - 1; + struct device *dev = device_get_child_by_index(tdata->parent, cidx); + + /* ref. counter is going to be decremented in device_shutdown_one() */ + get_device(dev); + device_shutdown_tree(dev); + return 0; +} + +/* + * On shutdown each root device (the one that does not have a parent) goes + * through this function. + */ +static int device_shutdown_root_task(void *data) +{ + struct device *dev = (struct device *)data; + + device_shutdown_tree(dev); + + return 0; +} + /** * device_shutdown - call ->shutdown() on each device to shutdown. */ void device_shutdown(void) { - struct device *dev, *parent; + struct device *dev; - spin_lock(&devices_kset->list_lock); - /* - * Walk the devices list backward, shutting down each in turn. - * Beware that device unplug events may also start pulling - * devices offline, even as the system is shutting down. + /* Shutdown the root devices. The children are going to be + * shutdown first in device_shutdown_tree(). */ + spin_lock(&devices_kset->list_lock); while (!list_empty(&devices_kset->list)) { - dev = list_entry(devices_kset->list.prev, struct device, - kobj.entry); + dev = list_entry(devices_kset->list.next, struct device, + kobj.entry); - /* - * hold reference count of device's parent to - * prevent it from being freed because parent's - * lock is to be held - */ - parent = get_device(dev->parent); - get_device(dev); /* * Make sure the device is off the kset list, in the * event that dev->*->shutdown() doesn't remove it. */ list_del_init(&dev->kobj.entry); - spin_unlock(&devices_kset->list_lock); - /* hold lock to avoid race with probe/release */ - if (parent) - device_lock(parent); - device_lock(dev); - - device_shutdown_one(dev); - device_unlock(dev); - if (parent) - device_unlock(parent); - - put_device(parent); - - spin_lock(&devices_kset->list_lock); + /* Here we start tasks for root devices only */ + if (!dev->parent) { + /* Prevents devices from being freed. The counter is + * going to be decremented in device_shutdown_one() once + * this root device is shutdown. + */ + get_device(dev); + + /* We unlock list for performance reasons, + * dev->*->shutdown(), may try to take this lock to + * remove us from kset list. To avoid unlocking this + * list we could replace spin lock in: + * dev->kobj.kset->list_lock with a dummy one once + * device is locked in device_shutdown_root_task() and + * in device_shutdown_child_task(). + */ + spin_unlock(&devices_kset->list_lock); + + device_shutdown_root_task(dev); + spin_lock(&devices_kset->list_lock); + } } spin_unlock(&devices_kset->list_lock); } From patchwork Fri Jun 29 18:25:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Tatashin X-Patchwork-Id: 937090 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=osuosl.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.b="rzauaGGY"; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41HQ7s5Khvz9s29 for ; Sat, 30 Jun 2018 04:26:01 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id E9C1A876E4; Fri, 29 Jun 2018 18:25:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I44AyBoEjuTX; Fri, 29 Jun 2018 18:25:58 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 756EF876E7; Fri, 29 Jun 2018 18:25:58 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id B95A21C156B for ; Fri, 29 Jun 2018 18:25:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id B6E3F82165 for ; Fri, 29 Jun 2018 18:25:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r8RSNokbhABq for ; Fri, 29 Jun 2018 18:25:55 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from aserp2120.oracle.com (aserp2120.oracle.com [141.146.126.78]) by fraxinus.osuosl.org (Postfix) with ESMTPS id D01A881E2B for ; Fri, 29 Jun 2018 18:25:55 +0000 (UTC) Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w5TINfXE129267; Fri, 29 Jun 2018 18:25:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2017-10-26; bh=3yWP2VZtGeS1EnJVZim9oC3cvUQxa/TduWADT0HqU1E=; b=rzauaGGYzChLzqrpGoi150cKo4FF64r+6Ly7rXbWLP34VBnWUacc8dyEPFFyfeuKWb2S UPxhqGyop+YPSENfF+LH7NGkSx/Rlow/lOL+q4JCBUIIeAjVutNuoTsDqDRdTPNjvm4y Pl6rSzl7P9li/wAlYiibnGrlb+jTuw18C8LuCFVsdBdrRy4kjpUXFr/tDeiUFxwTwG26 jOsUTBnR4d9a6htJqamGi9YaRCYIAmDIPHwVrg7ftsACjY9/B2Q41Bv+8K714ZlkYFSB L6TLDlxHZn/CwCoZNlmCNaE3MPBENSM6AiQLlGYo+jD6FKzlwdPlKYnKGQD5PufsZob7 mA== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2120.oracle.com with ESMTP id 2jukhsqnyw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 29 Jun 2018 18:25:51 +0000 Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w5TIPoLN028128 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 29 Jun 2018 18:25:50 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w5TIPodd031734; Fri, 29 Jun 2018 18:25:50 GMT Received: from localhost.localdomain (/73.69.118.222) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 29 Jun 2018 11:25:49 -0700 From: Pavel Tatashin To: pasha.tatashin@oracle.com, steven.sistare@oracle.com, daniel.m.jordan@oracle.com, linux-kernel@vger.kernel.org, jeffrey.t.kirsher@intel.com, intel-wired-lan@lists.osuosl.org, netdev@vger.kernel.org, gregkh@linuxfoundation.org, alexander.duyck@gmail.com, tobin@apporbit.com, andy.shevchenko@gmail.com Date: Fri, 29 Jun 2018 14:25:41 -0400 Message-Id: <20180629182541.6735-4-pasha.tatashin@oracle.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180629182541.6735-1-pasha.tatashin@oracle.com> References: <20180629182541.6735-1-pasha.tatashin@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8939 signatures=668703 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1806290195 Subject: [Intel-wired-lan] [PATCH v6 3/3] drivers core: multi-threading device shutdown X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" When system is rebooted, halted or kexeced device_shutdown() is called. This function shuts down every single device by calling either: dev->bus->shutdown(dev) dev->driver->shutdown(dev) Even on a machine with just a moderate amount of devices, device_shutdown() may take multiple seconds to complete. This is because many devices require a specific delays to perform this operation. Here is a sample analysis of time it takes to call device_shutdown() on a two socket Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz machine. device_shutdown 2.95s ----------------------------- mlx4_shutdown 1.14s megasas_shutdown 0.24s ixgbe_shutdown 0.37s x 4 (four ixgbe devices on this machine). the rest 0.09s In mlx4 we spent the most time, but that is because there is a 1 second sleep, which is defined by hardware specifications: mlx4_shutdown mlx4_unload_one mlx4_free_ownership msleep(1000) With megasas we spend a quarter of a second, but sometimes longer (up-to 0.5s) in this path: megasas_shutdown megasas_flush_cache megasas_issue_blocked_cmd wait_event_timeout Finally, with ixgbe_shutdown() it takes 0.37 for each device, but that time is spread all over the place, with bigger offenders: ixgbe_shutdown __ixgbe_shutdown ixgbe_close_suspend ixgbe_down ixgbe_init_hw_generic ixgbe_reset_hw_X540 msleep(100); 0.104483472 ixgbe_get_san_mac_addr_generic 0.048414851 ixgbe_get_wwn_prefix_generic 0.048409893 ixgbe_start_hw_X540 ixgbe_start_hw_generic ixgbe_clear_hw_cntrs_generic 0.048581502 ixgbe_setup_fc_generic 0.024225800 All the ixgbe_*generic functions end-up calling: ixgbe_read_eerd_X540() ixgbe_acquire_swfw_sync_X540 usleep_range(5000, 6000); ixgbe_release_swfw_sync_X540 usleep_range(5000, 6000); While these are short sleeps, they end-up calling them over 24 times! 24 * 0.0055s = 0.132s. Adding-up to 0.528s for four devices. Also we have four msleep(100). Totaling to: 0.928s While we should keep optimizing the individual device drivers, in some cases this is simply a hardware property that forces a specific delay, and we must wait. So, the solution for this problem is to shutdown devices in parallel. However, we must shutdown children before shutting down parents, so parent device must wait for its children to finish. With this patch, on the same machine devices_shutdown() takes 1.142s, and without mlx4 one second delay only 0.38s This feature can be optionally disabled via kernel parameter: device_shutdown_serial. When booted with this parameter, device_shutdown() will shutdown devices one by one. Signed-off-by: Pavel Tatashin --- drivers/base/core.c | 70 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 68 insertions(+), 2 deletions(-) diff --git a/drivers/base/core.c b/drivers/base/core.c index 5d8623b38d4d..b0f9e70daf52 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -2887,19 +2888,39 @@ static void device_shutdown_one(struct device *dev) /* * Passed as an argument to device_shutdown_child_task(). * child_next_index the next available child index. + * tasks_running number of tasks still running. Each tasks decrements it + * when job is finished and the last task signals that the + * job is complete. + * complete Used to signal job competition. * parent Parent device. */ struct device_shutdown_task_data { atomic_t child_next_index; + atomic_t tasks_running; + struct completion complete; struct device *parent; }; static int device_shutdown_child_task(void *data); +static bool device_shutdown_serial; + +/* + * These globals are used by tasks that are started for root devices. + * device_root_tasks_finished Number of root devices finished shutting down. + * device_root_tasks_started Total number of root devices tasks started. + * device_root_tasks_done The completion signal to the main thread. + */ +static atomic_t device_root_tasks_finished; +static atomic_t device_root_tasks_started; +static struct completion device_root_tasks_done; /* * Shutdown device tree with root started in dev. If dev has no children * simply shutdown only this device. If dev has children recursively shutdown * children first, and only then the parent. + * For performance reasons children are shutdown in parallel using kernel + * threads. because we lock dev its children cannot be removed while this + * functions is running. */ static void device_shutdown_tree(struct device *dev) { @@ -2913,11 +2934,20 @@ static void device_shutdown_tree(struct device *dev) int i; atomic_set(&tdata.child_next_index, 0); + atomic_set(&tdata.tasks_running, children_count); + init_completion(&tdata.complete); tdata.parent = dev; for (i = 0; i < children_count; i++) { - device_shutdown_child_task(&tdata); + if (device_shutdown_serial) { + device_shutdown_child_task(&tdata); + } else { + kthread_run(device_shutdown_child_task, + &tdata, "device_shutdown.%s", + dev_name(dev)); + } } + wait_for_completion(&tdata.complete); } device_shutdown_one(dev); device_unlock(dev); @@ -2937,6 +2967,10 @@ static int device_shutdown_child_task(void *data) /* ref. counter is going to be decremented in device_shutdown_one() */ get_device(dev); device_shutdown_tree(dev); + + /* If we are the last to exit, signal the completion */ + if (atomic_dec_return(&tdata->tasks_running) == 0) + complete(&tdata->complete); return 0; } @@ -2947,9 +2981,14 @@ static int device_shutdown_child_task(void *data) static int device_shutdown_root_task(void *data) { struct device *dev = (struct device *)data; + int root_devices; device_shutdown_tree(dev); + /* If we are the last to exit, signal the completion */ + root_devices = atomic_inc_return(&device_root_tasks_finished); + if (root_devices == atomic_read(&device_root_tasks_started)) + complete(&device_root_tasks_done); return 0; } @@ -2958,10 +2997,17 @@ static int device_shutdown_root_task(void *data) */ void device_shutdown(void) { + int root_devices = 0; struct device *dev; + atomic_set(&device_root_tasks_finished, 0); + atomic_set(&device_root_tasks_started, 0); + init_completion(&device_root_tasks_done); + /* Shutdown the root devices. The children are going to be * shutdown first in device_shutdown_tree(). + * We shutdown root devices in parallel by starting thread + * for each root device. */ spin_lock(&devices_kset->list_lock); while (!list_empty(&devices_kset->list)) { @@ -2992,13 +3038,33 @@ void device_shutdown(void) */ spin_unlock(&devices_kset->list_lock); - device_shutdown_root_task(dev); + root_devices++; + if (device_shutdown_serial) { + device_shutdown_root_task(dev); + } else { + kthread_run(device_shutdown_root_task, + dev, "device_root_shutdown.%s", + dev_name(dev)); + } spin_lock(&devices_kset->list_lock); } } spin_unlock(&devices_kset->list_lock); + + /* Set number of root tasks started, and waits for completion */ + atomic_set(&device_root_tasks_started, root_devices); + if (root_devices != atomic_read(&device_root_tasks_finished)) + wait_for_completion(&device_root_tasks_done); +} + +static int __init _device_shutdown_serial(char *arg) +{ + device_shutdown_serial = true; + return 0; } +early_param("device_shutdown_serial", _device_shutdown_serial); + /* * Device logging functions */