From patchwork Fri Nov 17 14:08:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 839018 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ydgDD3m0Pz9s4s for ; Sat, 18 Nov 2017 01:17:16 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="HmxC8i2j"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3ydgDD2HGfzDrRP for ; Sat, 18 Nov 2017 01:17:16 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="HmxC8i2j"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:400e:c05::241; helo=mail-pg0-x241.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="HmxC8i2j"; dkim-atps=neutral Received: from mail-pg0-x241.google.com (mail-pg0-x241.google.com [IPv6:2607:f8b0:400e:c05::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3ydg2n5pvKzDrST for ; Sat, 18 Nov 2017 01:09:05 +1100 (AEDT) Received: by mail-pg0-x241.google.com with SMTP id s75so2059473pgs.0 for ; Fri, 17 Nov 2017 06:09:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vi7d5xuP8xf4ASzlhnShp6nq3htwVHOhB+/CKezQ5hs=; b=HmxC8i2j2au9wruXrv93QeJ5dsB+t//Q4EHI2A4VZj3gI8l9SD8NlZS1DehG2yCaLJ rP3mt+88v/Ue0f/cAylTorZjkBVlRUXuA9e8C9Padyyv5292o9XqiilAH4OlgNop1e2Z S2pxhYOcOn75wxLUSUYRSTlrk/3rvlH2yvZfKXwRICeGa62m9o/H0bMsJATEZH7L5Zif Fc65yMa4nQvnGb5XgLwQk/GfPQ/C35VhhJuw921vAPbbu+2zhvRVH1UxZRBg+GQOg5TY DOaJMPCP7SXI8un1QjomTLjbVe7M/q6PXCKEXahD+/o778/82xOOmaYf1ryxz4dBjOhY hhRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vi7d5xuP8xf4ASzlhnShp6nq3htwVHOhB+/CKezQ5hs=; b=o0B140ZHKWq2v35AQ6VU65VGF6EQbyk4QcN4mSXK8Qetuoz1ALBBGZ4QHV/bMB/nQy eqo/x5I5d7lawPo+BU1eSUFbM2TLHS40wIspdWwo4iR5t1+b/Yh/HDuw8eJLc6x0kVKO hu+Sp64dRqmOPR3oGkGgTKTon2P4jA4idA1M9HmsB3sIBrdSfi7ipng1RLqxAFqZIofB evMw6nB+VTR7wH4Bbn4wmnwEnpJghCeQpchAfqPSFbkgcgH7fSuW/CW+nMhzpZhH/JZK wTEZNa9/WeOWNaXwfJEjtkdbhs4eoyo60UhXax3+XXT8jBZPUcIXi2mmuyAu6hAr7vT4 zwzg== X-Gm-Message-State: AJaThX7uUjnc2na1S0nfgU6WLbaVlNHhKRkJm4otzSQmYP9Ua3Ez3IEs W5Lx7x6EczYADXOu4vyj5luMBA== X-Google-Smtp-Source: AGs4zMYiB81hIyaMV9PNNBXs68YTxggDwmRix7D7Z0S0FwkpwXBIZN+m1jWXjMJDOzixfP8onJHx6A== X-Received: by 10.98.254.1 with SMTP id z1mr2173612pfh.24.1510927743681; Fri, 17 Nov 2017 06:09:03 -0800 (PST) Received: from roar.au.ibm.com (27-33-241-2.tpgi.com.au. [27.33.241.2]) by smtp.gmail.com with ESMTPSA id w64sm9024357pfj.62.2017.11.17.06.09.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Nov 2017 06:09:02 -0800 (PST) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH 3/3] powerpc/64s/idle: POWER9 ESL=0 stop avoid save/restore overhead Date: Sat, 18 Nov 2017 00:08:07 +1000 Message-Id: <20171117140807.22105-4-npiggin@gmail.com> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20171117140807.22105-1-npiggin@gmail.com> References: <20171117140807.22105-1-npiggin@gmail.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Gautham R . Shenoy" , Nicholas Piggin Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" When stop is executed with EC=ESL=0, it appears to execute like a normal instruction (resuming from NIP when woken by interrupt). So all the save/restore handling can be avoided completely. In particular NV GPRs do not have to be saved, and MSR does not have to be switched back to kernel MSR. So move the test for EC=ESL=0 sleep states out to power9_idle_stop, and return directly to the caller after stop in that case. The mtspr to PSSCR is moved to the top of power9_offline_stop just so it matches power9_idle_stop. This improves performance for ping-pong benchmark with the stop0_lite idle state by 2.54% for 2 threads in the same core, and 2.57% for different cores. Signed-off-by: Nicholas Piggin Reviewed-by: Vaidyanathan Srinivasan --- arch/powerpc/kernel/idle_book3s.S | 43 +++++++++++------------------------ arch/powerpc/platforms/powernv/idle.c | 7 +++++- 2 files changed, 19 insertions(+), 31 deletions(-) diff --git a/arch/powerpc/kernel/idle_book3s.S b/arch/powerpc/kernel/idle_book3s.S index 07a306173c5a..6243da99b26c 100644 --- a/arch/powerpc/kernel/idle_book3s.S +++ b/arch/powerpc/kernel/idle_book3s.S @@ -324,31 +324,8 @@ enter_winkle: /* * r3 - PSSCR value corresponding to the requested stop state. */ -power_enter_stop: -/* - * Check if we are executing the lite variant with ESL=EC=0 - */ - andis. r4,r3,PSSCR_EC_ESL_MASK_SHIFTED +power_enter_stop_esl: clrldi r3,r3,60 /* r3 = Bits[60:63] = Requested Level (RL) */ - bne .Lhandle_esl_ec_set - PPC_STOP - li r3,0 /* Since we didn't lose state, return 0 */ - - /* - * pnv_wakeup_noloss() expects r12 to contain the SRR1 value so - * it can determine if the wakeup reason is an HMI in - * CHECK_HMI_INTERRUPT. - * - * However, when we wakeup with ESL=0, SRR1 will not contain the wakeup - * reason, so there is no point setting r12 to SRR1. - * - * Further, we clear r12 here, so that we don't accidentally enter the - * HMI in pnv_wakeup_noloss() if the value of r12[42:45] == WAKE_HMI. - */ - li r12, 0 - b pnv_wakeup_noloss - -.Lhandle_esl_ec_set: BEGIN_FTR_SECTION /* * POWER9 DD2.0 or earlier can incorrectly set PMAO when waking up after @@ -423,26 +400,32 @@ ALT_FTR_SECTION_END_NESTED_IFSET(CPU_FTR_ARCH_207S, 66); \ * r3 contains desired PSSCR register value. */ _GLOBAL(power9_idle_stop) - std r3, PACA_REQ_PSSCR(r13) mtspr SPRN_PSSCR,r3 - LOAD_REG_ADDR(r4,power_enter_stop) + andis. r4,r3,PSSCR_EC_ESL_MASK_SHIFTED + bne 1f + PPC_STOP + li r3,0 /* Since we didn't lose state, return 0 */ + blr + +1: std r3, PACA_REQ_PSSCR(r13) + LOAD_REG_ADDR(r4,power_enter_stop_esl) b pnv_powersave_common /* No return */ /* - * Entered with MSR[EE]=0 and no soft-masked interrupts pending. - * r3 contains desired PSSCR register value. + * This is the same as the above, but it sets KVM state for secondaries, + * and it must have PSSCR[EC]=1 */ _GLOBAL(power9_offline_stop) - std r3, PACA_REQ_PSSCR(r13) mtspr SPRN_PSSCR,r3 + std r3, PACA_REQ_PSSCR(r13) #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE /* Tell KVM we're entering idle */ li r4,KVM_HWTHREAD_IN_IDLE /* DO THIS IN REAL MODE! See comment above. */ stb r4,HSTATE_HWTHREAD_STATE(r13) #endif - LOAD_REG_ADDR(r4,power_enter_stop) + LOAD_REG_ADDR(r4,power_enter_stop_esl) b pnv_powersave_common /* No return */ diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c index a921d5428d76..610b1637c16f 100644 --- a/arch/powerpc/platforms/powernv/idle.c +++ b/arch/powerpc/platforms/powernv/idle.c @@ -621,7 +621,12 @@ static int __init pnv_power9_idle_init(struct device_node *np, u32 *flags, continue; } - if (max_residency_ns < residency_ns[i]) { + /* + * Deepest stop for unplug must be PSSCR[EC]=1 (wakeup at + * 0x100. + */ + if ((max_residency_ns < residency_ns[i])&& + (psscr_val[i] & PSSCR_EC)) { max_residency_ns = residency_ns[i]; pnv_deepest_stop_psscr_val = psscr_val[i]; pnv_deepest_stop_psscr_mask = psscr_mask[i];