From patchwork Mon Oct 23 08:05:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 829169 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yL8DZ4qbcz9sPr for ; Mon, 23 Oct 2017 19:08:46 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="VPFWl2nd"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3yL8DZ3WMYzDqjm for ; Mon, 23 Oct 2017 19:08:46 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="VPFWl2nd"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:400e:c00::243; helo=mail-pf0-x243.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="VPFWl2nd"; dkim-atps=neutral Received: from mail-pf0-x243.google.com (mail-pf0-x243.google.com [IPv6:2607:f8b0:400e:c00::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3yL88f1NRMzDqXj for ; Mon, 23 Oct 2017 19:05:22 +1100 (AEDT) Received: by mail-pf0-x243.google.com with SMTP id n89so16512412pfk.11 for ; Mon, 23 Oct 2017 01:05:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=NLrvTKQq0akKX/y9hZahg0jynAmFQJ7pnCXhgZGVY6Y=; b=VPFWl2nd3rfS9yOGIXAsc8nQlqZots4rQP75KNLtSOxA2oLpo9m1VwaGZwpoh7g3kF 1e5GSR2oB8ac8ZRCNaDBqitR9095RcJP+VEV5+oxkYNd/fCrq/2UaL8C4+9FclMvW9jQ H5WQKyxbh90NklJedwVhLeNzWFq3rjhvWGEg07Gid5swSPMbiMcGmI1SXKtZurkZrEa9 fBxh4R8TbS2hiMW+ZGzdYOfe4R/YXs7rloiPp1wgVWOZ2cIo7n2pWep+5BCj6WaHUAfo JitM6HbEQjjXzbB6lgkKBANjDwpl8nCiG3uIgZTcL619fq3Df7o/HJ4L5bqu2cGLCZBF Wn/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=NLrvTKQq0akKX/y9hZahg0jynAmFQJ7pnCXhgZGVY6Y=; b=igYtgawtg8eOxWf7tr4KuM0B9ZM2nsgv5TfDIpsl4zkmplC2NaHvhFzWdNS+7vZORo bhCh+G3hC/GucnJauhUqyd+Ps3pO/1U1XDpo+Dmx/npHg24GCJVPqPGMGgIBP0ldNWU5 ZWBRUyIGN4i4P923Tjw8VkTZEqtZJ2JkDS6VOxSL+Zz9niG8P+/u0gHUzy1CygWI8E+h X9Go0u805/GIx0FUV7bKuMK1ZsfSqwMMKKuCk6EimUcIyVViUQKw1r1uCfYX6yOokXaC 7lsZdTGfpzFXOlfAMfbkV97Ck4bDm/x03JKBo+QAAe3+AAjtp+b+NZKh+e6IFKKNXF6L n7iA== X-Gm-Message-State: AMCzsaUiEFytiR8nTxhCbyqGSYXFMGYzsGmyX70xQ5wUsVsB+WKRzDHo 5sRrUqmJoFWktFnPi5VmYdC1mw== X-Google-Smtp-Source: ABhQp+Rvsl4mnH7uNaSnyTZfxN0OjYHKbgUm6vSBe8K/xjL4GM96SRKAFz3PvDY71qtwvZufYned+A== X-Received: by 10.98.72.18 with SMTP id v18mr12466402pfa.232.1508745919799; Mon, 23 Oct 2017 01:05:19 -0700 (PDT) Received: from roar.au.ibm.com (59-102-69-156.tpgi.com.au. [59.102.69.156]) by smtp.gmail.com with ESMTPSA id k3sm12043124pfc.44.2017.10.23.01.05.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Oct 2017 01:05:18 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 1/3] powerpc/powernv: Always stop secondaries before reboot/shutdown Date: Mon, 23 Oct 2017 18:05:05 +1000 Message-Id: <20171023080507.21974-2-npiggin@gmail.com> X-Mailer: git-send-email 2.13.3 In-Reply-To: <20171023080507.21974-1-npiggin@gmail.com> References: <20171023080507.21974-1-npiggin@gmail.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Currently powernv reboot and shutdown requests just leave secondaries to do their own things. This is undesirable because they can trigger any number of watchdogs while waiting for reboot, but also we don't know what else they might be doing, or they might be stuck somewhere causing trouble. The opal scheduled flash update code already ran into watchdog problems due to flashing taking a long time, but it's possible for regular reboots to trigger problems too (this is with watchdog_thresh set to 1, but I have seen it with watchdog_thresh at the default value once too): reboot: Restarting system [ 360.038896709,5] OPAL: Reboot request... Watchdog CPU:0 Hard LOCKUP Watchdog CPU:44 detected Hard LOCKUP other CPUS:16 Watchdog CPU:16 Hard LOCKUP watchdog: BUG: soft lockup - CPU#16 stuck for 3s! [swapper/16:0] So remove the special case for flash update, and unconditionally do smp_send_stop before rebooting. Return the CPUs to Linux stop loops rather than OPAL. The reason for this is that the path to firmware is longer, and the CPUs may have been interrupted from firmware, which may cause problems to re-enter it. It's better to put them into a simple spin loop to maximize the chance of a successful reboot. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/opal.h | 2 +- arch/powerpc/platforms/powernv/opal-flash.c | 28 +--------------------------- arch/powerpc/platforms/powernv/setup.c | 15 +++++---------- 3 files changed, 7 insertions(+), 38 deletions(-) diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h index 04c32b08ffa1..ce58f4139ff5 100644 --- a/arch/powerpc/include/asm/opal.h +++ b/arch/powerpc/include/asm/opal.h @@ -317,7 +317,7 @@ struct rtc_time; extern unsigned long opal_get_boot_time(void); extern void opal_nvram_init(void); extern void opal_flash_update_init(void); -extern void opal_flash_term_callback(void); +extern void opal_flash_update_print_message(void); extern int opal_elog_init(void); extern void opal_platform_dump_init(void); extern void opal_sys_param_init(void); diff --git a/arch/powerpc/platforms/powernv/opal-flash.c b/arch/powerpc/platforms/powernv/opal-flash.c index 2fa3ac80cb4e..632871d78576 100644 --- a/arch/powerpc/platforms/powernv/opal-flash.c +++ b/arch/powerpc/platforms/powernv/opal-flash.c @@ -303,26 +303,9 @@ static int opal_flash_update(int op) return rc; } -/* Return CPUs to OPAL before starting FW update */ -static void flash_return_cpu(void *info) -{ - int cpu = smp_processor_id(); - - if (!cpu_online(cpu)) - return; - - /* Disable IRQ */ - hard_irq_disable(); - - /* Return the CPU to OPAL */ - opal_return_cpu(); -} - /* This gets called just before system reboots */ -void opal_flash_term_callback(void) +void opal_flash_update_print_message(void) { - struct cpumask mask; - if (update_flash_data.status != FLASH_IMG_READY) return; @@ -333,15 +316,6 @@ void opal_flash_term_callback(void) /* Small delay to help getting the above message out */ msleep(500); - - /* Return secondary CPUs to firmware */ - cpumask_copy(&mask, cpu_online_mask); - cpumask_clear_cpu(smp_processor_id(), &mask); - if (!cpumask_empty(&mask)) - smp_call_function_many(&mask, - flash_return_cpu, NULL, false); - /* Hard disable interrupts */ - hard_irq_disable(); } /* diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c index b23d8beb0f1e..ff9422776a81 100644 --- a/arch/powerpc/platforms/powernv/setup.c +++ b/arch/powerpc/platforms/powernv/setup.c @@ -112,17 +112,12 @@ static void pnv_prepare_going_down(void) */ opal_event_shutdown(); - /* Soft disable interrupts */ - local_irq_disable(); + /* Print flash update message if one is scheduled. */ + opal_flash_update_print_message(); - /* - * Return secondary CPUs to firwmare if a flash update - * is pending otherwise we will get all sort of error - * messages about CPU being stuck etc.. This will also - * have the side effect of hard disabling interrupts so - * past this point, the kernel is effectively dead. - */ - opal_flash_term_callback(); + smp_send_stop(); + + hard_irq_disable(); } static void __noreturn pnv_restart(char *cmd) From patchwork Mon Oct 23 08:05:06 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 829170 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yL8Gp4Qw9z9sPr for ; Mon, 23 Oct 2017 19:10:42 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="cyQfLjIB"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3yL8Gp3DqvzDqkm for ; Mon, 23 Oct 2017 19:10:42 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="cyQfLjIB"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:400e:c00::241; helo=mail-pf0-x241.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="cyQfLjIB"; dkim-atps=neutral Received: from mail-pf0-x241.google.com (mail-pf0-x241.google.com [IPv6:2607:f8b0:400e:c00::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3yL88j33PgzDqjm for ; Mon, 23 Oct 2017 19:05:25 +1100 (AEDT) Received: by mail-pf0-x241.google.com with SMTP id d28so16533324pfe.2 for ; Mon, 23 Oct 2017 01:05:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=lqq0ndA79OrUCozoYOmY8CACNeS8cdjHWquVaHHiCoI=; b=cyQfLjIBiYDGr3lEukfPyfsXpUxji7zWYd0RHx1TKQjEistDcEXWJFaC9dC26HiX2D wBUEFpEvMvahMj9bSBX2EXDEX+HEULD0Aq7NCzZK09j/YiIHXfI0zXyEDErQhqaSvqNP Unk6y+bMQA2cJLg9M/J7jGQlDmXX0lGm5VULvIJcR09zti1hXsR0ShJ//vjJ9+UooTwe rlENtd+c3XVTOGCCMeR4Tj4WXKNOLXrpBMvh+E6pK3/4emJSzOVurcyuHuDnVouRacyQ EYrgx0Hzxee+TFWVeSwiKKKzsPeguevajf07kfScFGF8TFb+BFInRUV+KBZNKLp+B7/z +GOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=lqq0ndA79OrUCozoYOmY8CACNeS8cdjHWquVaHHiCoI=; b=tx+0iC1mFY68cC90jVGYpD93gb2vrD9DoZu+BY8MLykwQ2crSMqs1HJCEYZLW33tc9 FVIPnjDZ3RN7Jnn6H/jMipbq/cceSogJMcXm6QpNWWe8Y7URE+/WSxHnJGSgi9j8191T fjbdZ2B6/diCFLOYRKjafqGLuhaJf5MxkurT9Lzp4j0ic7FE/IKgn/Cl4T7nPRQLEliU /zDhRsQ9NHvkCcbrnDOGXEgkJmiYmDNf3gzcZB97MAcFztxCT1Eo/v8eRCqTxDWf0tdu 5u2bWAPotminTx80rcJMv/r6/gZ7jSMwtbpn1cSq2ycU7+4KYtao4e6ZrT9jyFrnxyxx rlWQ== X-Gm-Message-State: AMCzsaVWN3CG6/JH00HUUnHEuyjVcp6MkpW/+Hko54RbhOaGcslljaqT sjLdpt/CRIKGeORPzUWlxUdKSQ== X-Google-Smtp-Source: ABhQp+Rg9PS+WjNiKqmyhdFOTaX0iA1wqCKIBfoAWXjKQLz90NAkymRCvGJdfUSfOh9bZRSwlNJaWQ== X-Received: by 10.98.233.21 with SMTP id j21mr12336675pfh.97.1508745922713; Mon, 23 Oct 2017 01:05:22 -0700 (PDT) Received: from roar.au.ibm.com (59-102-69-156.tpgi.com.au. [59.102.69.156]) by smtp.gmail.com with ESMTPSA id k3sm12043124pfc.44.2017.10.23.01.05.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Oct 2017 01:05:21 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 2/3] powerpc: use NMI IPI for smp_send_stop Date: Mon, 23 Oct 2017 18:05:06 +1000 Message-Id: <20171023080507.21974-3-npiggin@gmail.com> X-Mailer: git-send-email 2.13.3 In-Reply-To: <20171023080507.21974-1-npiggin@gmail.com> References: <20171023080507.21974-1-npiggin@gmail.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Use the NMI IPI rather than smp_call_function for smp_send_stop. Have stopped CPUs hard disable interrupts rather than just soft disable. This function is used in crash/panic/shutdown paths to bring other CPUs down as quickly and reliably as possible, and minimizing their potential to cause trouble. Avoiding the Linux smp_call_function infrastructure and (if supported) using true NMI IPIs makes this more robust. Also use spin loop primitives in the stop callback, mainly to help processing speed of the active thread speed in the simulator. Signed-off-by: Nicholas Piggin --- arch/powerpc/kernel/smp.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c index e0a4c1f82e25..ce891030d925 100644 --- a/arch/powerpc/kernel/smp.c +++ b/arch/powerpc/kernel/smp.c @@ -547,19 +547,20 @@ void crash_send_ipi(void (*crash_ipi_callback)(struct pt_regs *)) } #endif -static void stop_this_cpu(void *dummy) +static void __noreturn stop_this_cpu(struct pt_regs *regs) { /* Remove this CPU */ set_cpu_online(smp_processor_id(), false); - local_irq_disable(); + hard_irq_disable(); + spin_begin(); while (1) - ; + spin_cpu_relax(); } void smp_send_stop(void) { - smp_call_function(stop_this_cpu, NULL, 0); + smp_send_nmi_ipi(NMI_IPI_ALL_OTHERS, stop_this_cpu, 1000000); } struct thread_info *current_set[NR_CPUS]; From patchwork Mon Oct 23 08:05:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 829171 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yL8Jy17ddz9t2h for ; Mon, 23 Oct 2017 19:12:34 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="qOz9R8E+"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3yL8Jx6jBFzDqkl for ; Mon, 23 Oct 2017 19:12:33 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="qOz9R8E+"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:400e:c05::242; helo=mail-pg0-x242.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="qOz9R8E+"; dkim-atps=neutral Received: from mail-pg0-x242.google.com (mail-pg0-x242.google.com [IPv6:2607:f8b0:400e:c05::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3yL88m4DHBzDqkl for ; Mon, 23 Oct 2017 19:05:28 +1100 (AEDT) Received: by mail-pg0-x242.google.com with SMTP id p9so11158412pgc.8 for ; Mon, 23 Oct 2017 01:05:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Mv+WrPZjGays+qi/6loKp5dEGi39wKPIEEzorSiAL3A=; b=qOz9R8E+CYSlggLd3CspasjnrMQyETqKk1X5oNnANY+tuv3VKB/UrwRBIFdklUC02f aNxWcnufpw3gLWvVNqgdBwcpex1+v5PYTns5hLmZ5D0wepRP/MefoHe8ixawFWIsyar7 MSa8kmwE04j/kZPPswFFJWoxbZ59fCyi1AYc6+oUgzIbk5dpEXO6ZRCs5et91XgMneu5 o+DBEbvFWKNLO2hVeftCGXJiPf0jtvYvd9UOKZMM6M0EdAe9ra3N/8ROOhGNoU1v/lDx 8w4Cn2PK1LK5TV8Q6H6Ci4Aol44EkrME2ZRkjd6mtF4A0ihu4BKIk6yiuuR9Fj0zpAHu fmyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Mv+WrPZjGays+qi/6loKp5dEGi39wKPIEEzorSiAL3A=; b=hHu89rJa7J/gcaXhlxGEzbNHjIaJkl+WPROh8fQ5/aoSRj6QazJcsLL6Hi49U1d2pB 4/e6W42BR3RXXa4k+x0/Drby0PEFlLvaa3kDU5XpQr3mu5d/PkKAWJcHC75koP3jFJUg 0yXKdupfQV2IcleatoQn5axxm2T5sY4ZKNRduxdCLRvV4eV7FoGngI76zeLcmVxLnLUr 7HOh7EAkGnKS2ezSx4vL6lLYHgk+UmpA8YDqB4GAnBtM7SQebiz6zzs5y0I5OBom7laW A9km3fc+7OECUs2daUD0mrJ7ZVCNjcpszitGJ9PTPhWpw/ukkviTzoa8QrTfx3Y8UjrO qyWg== X-Gm-Message-State: AMCzsaUFckD/wmWoTA3Y4TklnXI6hpX/Hd2pMDGTKxDHpEw2FblJsqez iMkzePoXD/rutECWzD+XaudVsA== X-Google-Smtp-Source: ABhQp+Tdh37gYhqkLLOFwo3+KLFdEMK84qUF1dkcZrLgZOTYRONjwdDvVejPZbBFelQxvT0Hh6ZIRA== X-Received: by 10.98.202.133 with SMTP id y5mr12231323pfk.122.1508745925806; Mon, 23 Oct 2017 01:05:25 -0700 (PDT) Received: from roar.au.ibm.com (59-102-69-156.tpgi.com.au. [59.102.69.156]) by smtp.gmail.com with ESMTPSA id k3sm12043124pfc.44.2017.10.23.01.05.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Oct 2017 01:05:24 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 3/3] powerpc/powernv: Avoid waiting for secondary hold spinloop with OPAL Date: Mon, 23 Oct 2017 18:05:07 +1000 Message-Id: <20171023080507.21974-4-npiggin@gmail.com> X-Mailer: git-send-email 2.13.3 In-Reply-To: <20171023080507.21974-1-npiggin@gmail.com> References: <20171023080507.21974-1-npiggin@gmail.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" OPAL boot does not insert secondaries at 0x60 to wait at the secondary hold spinloop. Instead they are started later, and inserted at generic_secondary_smp_init(), which is after the secondary hold spinloop. Avoid waiting on this spinloop when booting with OPAL firmware. This wait always times out that case. This saves 100ms boot time on powernv, and 10s of seconds of real time when booting on the simulator in SMP. Signed-off-by: Nicholas Piggin --- arch/powerpc/kernel/head_64.S | 16 +++++++++++----- arch/powerpc/kernel/setup_64.c | 10 +++++++++- 2 files changed, 20 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/kernel/head_64.S b/arch/powerpc/kernel/head_64.S index c9e760ec7530..0deef350004f 100644 --- a/arch/powerpc/kernel/head_64.S +++ b/arch/powerpc/kernel/head_64.S @@ -55,12 +55,18 @@ * * For pSeries or server processors: * 1. The MMU is off & open firmware is running in real mode. - * 2. The kernel is entered at __start + * 2. The primary CPU enters at __start. + * 3. If the RTAS supports "query-cpu-stopped-state", then secondary + * CPUs will enter as directed by "start-cpu" RTAS call, which is + * generic_secondary_smp_init, with PIR in r3. + * 4. Else the secondary CPUs will enter at secondary_hold (0x60) as + * directed by the "start-cpu" RTS call, with PIR in r3. * -or- For OPAL entry: - * 1. The MMU is off, processor in HV mode, primary CPU enters at 0 - * with device-tree in gpr3. We also get OPAL base in r8 and - * entry in r9 for debugging purposes - * 2. Secondary processors enter at 0x60 with PIR in gpr3 + * 1. The MMU is off, processor in HV mode. + * 2. The primary CPU enters at 0 with device-tree in r3, OPAL base + * in r8, and entry in r9 for debugging purposes. + * 3. Secondary CPUs enter as directed by OPAL_START_CPU call, which + * is at generic_secondary_smp_init, with PIR in r3. * * For Book3E processors: * 1. The MMU is on running in AS0 in a state defined in ePAPR diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c index 3f2453858f60..ecaab70e4b78 100644 --- a/arch/powerpc/kernel/setup_64.c +++ b/arch/powerpc/kernel/setup_64.c @@ -363,8 +363,16 @@ void early_setup_secondary(void) #if defined(CONFIG_SMP) || defined(CONFIG_KEXEC_CORE) static bool use_spinloop(void) { - if (!IS_ENABLED(CONFIG_PPC_BOOK3E)) + if (IS_ENABLED(CONFIG_PPC_BOOK3S)) { + /* + * See comments in head_64.S -- not all platforms insert + * secondaries at __secondary_hold and wait at the spin + * loop. + */ + if (firmware_has_feature(FW_FEATURE_OPAL)) + return false; return true; + } /* * When book3e boots from kexec, the ePAPR spin table does