From patchwork Thu Mar 14 17:56:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jose Ricardo Ziviani X-Patchwork-Id: 1056630 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44KxHp0Bwhz9s4Y; Fri, 15 Mar 2019 04:57:26 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1h4UbZ-0006M7-5D; Thu, 14 Mar 2019 17:57:21 +0000 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5] helo=mx0a-001b2d01.pphosted.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1h4UbX-0006LN-FW for kernel-team@lists.ubuntu.com; Thu, 14 Mar 2019 17:57:19 +0000 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x2EHrwub072887 for ; Thu, 14 Mar 2019 13:57:18 -0400 Received: from e14.ny.us.ibm.com (e14.ny.us.ibm.com [129.33.205.204]) by mx0a-001b2d01.pphosted.com with ESMTP id 2r7set77t9-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Mar 2019 13:57:18 -0400 Received: from localhost by e14.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 14 Mar 2019 17:57:17 -0000 Received: from b01cxnp22033.gho.pok.ibm.com (9.57.198.23) by e14.ny.us.ibm.com (146.89.104.201) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Thu, 14 Mar 2019 17:57:14 -0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x2EHvD2a25362506 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Mar 2019 17:57:14 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CEB04112061 for ; Thu, 14 Mar 2019 17:57:13 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6BEB9112066 for ; Thu, 14 Mar 2019 17:57:13 +0000 (GMT) Received: from pacoca.br.ibm.com (unknown [9.7.47.58]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP for ; Thu, 14 Mar 2019 17:57:13 +0000 (GMT) From: Jose Ricardo Ziviani To: kernel-team@lists.ubuntu.com Subject: [Bionic][PATCH 02/12] powerpc/powernv: call OPAL_QUIESCE before OPAL_SIGNAL_SYSTEM_RESET Date: Thu, 14 Mar 2019 14:56:42 -0300 X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190314175652.1647-1-joserz@linux.ibm.com> References: <20190314175652.1647-1-joserz@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19031417-0052-0000-0000-0000039B3ABC X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00010758; HX=3.00000242; KW=3.00000007; PH=3.00000004; SC=3.00000281; SDB=6.01174327; UDB=6.00614057; IPR=6.00954992; MB=3.00025978; MTD=3.00000008; XFM=3.00000015; UTC=2019-03-14 17:57:16 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19031417-0053-0000-0000-000060279A02 Message-Id: <20190314175652.1647-3-joserz@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-03-14_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903140126 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Nicholas Piggin BugLink: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1819989 Although it is often possible to recover a CPU that was interrupted from OPAL with a system reset NMI, it's undesirable to interrupt them for a few reasons. Firstly because dump/debug code itself needs to call firmware, so it could hang on a lock or possibly corrupt a per-cpu data structure if it or another CPU was interrupted from OPAL. Secondly, the kexec crash dump code will not return from interrupt to unwind the OPAL call. Call OPAL_QUIESCE with QUIESCE_HOLD before sending an NMI IPI to another CPU, which wait for it to leave firmware (or time out) to avoid this problem in normal conditions. Firmware bugs may still result in a timeout and interrupting OPAL, but that is the best option (stops the CPU, and possibly allows firmware to be debugged). Signed-off-by: Nicholas Piggin Signed-off-by: Michael Ellerman (cherry picked from commit ee03b9b4479d1302d01cebedda3518dc967697b7) Signed-off-by: Jose Ricardo Ziviani --- arch/powerpc/include/asm/opal-api.h | 7 +++++++ arch/powerpc/include/asm/opal.h | 1 + arch/powerpc/platforms/powernv/opal-wrappers.S | 1 + arch/powerpc/platforms/powernv/smp.c | 17 ++++++++++++++++- 4 files changed, 25 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h index 789223f2a095..402f2cd80f9e 100644 --- a/arch/powerpc/include/asm/opal-api.h +++ b/arch/powerpc/include/asm/opal-api.h @@ -201,6 +201,7 @@ #define OPAL_SET_POWER_SHIFT_RATIO 155 #define OPAL_SENSOR_GROUP_CLEAR 156 #define OPAL_PCI_SET_P2P 157 +#define OPAL_QUIESCE 158 #define OPAL_NPU_SPA_SETUP 159 #define OPAL_NPU_SPA_CLEAR_CACHE 160 #define OPAL_NPU_TL_SET 161 @@ -208,6 +209,12 @@ #define OPAL_PCI_SET_PBCQ_TUNNEL_BAR 165 #define OPAL_LAST 165 +#define QUIESCE_HOLD 1 /* Spin all calls at entry */ +#define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */ +#define QUIESCE_LOCK_BREAK 3 /* Set to ignore locks. */ +#define QUIESCE_RESUME 4 /* Un-quiesce */ +#define QUIESCE_RESUME_FAST_REBOOT 5 /* Un-quiesce, fast reboot */ + /* Device tree flags */ /* diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h index c3f0ee833319..5e3dd3bf7602 100644 --- a/arch/powerpc/include/asm/opal.h +++ b/arch/powerpc/include/asm/opal.h @@ -293,6 +293,7 @@ int opal_set_power_shift_ratio(u32 handle, int token, u32 psr); int opal_sensor_group_clear(u32 group_hndl, int token); s64 opal_signal_system_reset(s32 cpu); +s64 opal_quiesce(u64 shutdown_type, s32 cpu); /* Internal functions */ extern int early_init_dt_scan_opal(unsigned long node, const char *uname, diff --git a/arch/powerpc/platforms/powernv/opal-wrappers.S b/arch/powerpc/platforms/powernv/opal-wrappers.S index 3da30c2f26b4..5eb1466665a4 100644 --- a/arch/powerpc/platforms/powernv/opal-wrappers.S +++ b/arch/powerpc/platforms/powernv/opal-wrappers.S @@ -320,6 +320,7 @@ OPAL_CALL(opal_set_powercap, OPAL_SET_POWERCAP); OPAL_CALL(opal_get_power_shift_ratio, OPAL_GET_POWER_SHIFT_RATIO); OPAL_CALL(opal_set_power_shift_ratio, OPAL_SET_POWER_SHIFT_RATIO); OPAL_CALL(opal_sensor_group_clear, OPAL_SENSOR_GROUP_CLEAR); +OPAL_CALL(opal_quiesce, OPAL_QUIESCE); OPAL_CALL(opal_npu_spa_setup, OPAL_NPU_SPA_SETUP); OPAL_CALL(opal_npu_spa_clear_cache, OPAL_NPU_SPA_CLEAR_CACHE); OPAL_CALL(opal_npu_tl_set, OPAL_NPU_TL_SET); diff --git a/arch/powerpc/platforms/powernv/smp.c b/arch/powerpc/platforms/powernv/smp.c index 9664c8461f03..09c156203898 100644 --- a/arch/powerpc/platforms/powernv/smp.c +++ b/arch/powerpc/platforms/powernv/smp.c @@ -334,7 +334,16 @@ static int pnv_cause_nmi_ipi(int cpu) int64_t rc; if (cpu >= 0) { - rc = opal_signal_system_reset(get_hard_smp_processor_id(cpu)); + int h = get_hard_smp_processor_id(cpu); + + if (opal_check_token(OPAL_QUIESCE)) + opal_quiesce(QUIESCE_HOLD, h); + + rc = opal_signal_system_reset(h); + + if (opal_check_token(OPAL_QUIESCE)) + opal_quiesce(QUIESCE_RESUME, h); + if (rc != OPAL_SUCCESS) return 0; return 1; @@ -343,6 +352,8 @@ static int pnv_cause_nmi_ipi(int cpu) bool success = true; int c; + if (opal_check_token(OPAL_QUIESCE)) + opal_quiesce(QUIESCE_HOLD, -1); /* * We do not use broadcasts (yet), because it's not clear @@ -358,6 +369,10 @@ static int pnv_cause_nmi_ipi(int cpu) if (rc != OPAL_SUCCESS) success = false; } + + if (opal_check_token(OPAL_QUIESCE)) + opal_quiesce(QUIESCE_RESUME, -1); + if (success) return 1;