From patchwork Mon Nov 20 10:03:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 839481 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ygPTy4gYyz9s03 for ; Mon, 20 Nov 2017 21:05:10 +1100 (AEDT) Received: from localhost ([::1]:56418 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eGiwu-0000pJ-Nd for incoming@patchwork.ozlabs.org; Mon, 20 Nov 2017 05:05:08 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45854) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eGiw7-0000kM-Uc for qemu-devel@nongnu.org; Mon, 20 Nov 2017 05:04:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eGiw1-0001hW-6p for qemu-devel@nongnu.org; Mon, 20 Nov 2017 05:04:19 -0500 Received: from 7.mo173.mail-out.ovh.net ([46.105.44.159]:49115) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eGiw0-0001Zf-V7 for qemu-devel@nongnu.org; Mon, 20 Nov 2017 05:04:13 -0500 Received: from player739.ha.ovh.net (b9.ovh.net [213.186.33.59]) by mo173.mail-out.ovh.net (Postfix) with ESMTP id C706C88A27 for ; Mon, 20 Nov 2017 11:04:04 +0100 (CET) Received: from zorba.kaod.org.com (LFbn-1-2231-173.w90-76.abo.wanadoo.fr [90.76.52.173]) (Authenticated sender: clg@kaod.org) by player739.ha.ovh.net (Postfix) with ESMTPSA id 628554A009F; Mon, 20 Nov 2017 11:03:58 +0100 (CET) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson , Nikunj A Dadhania , Benjamin Herrenschmidt Date: Mon, 20 Nov 2017 11:03:45 +0100 Message-Id: <20171120100347.8601-2-clg@kaod.org> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20171120100347.8601-1-clg@kaod.org> References: <20171120100347.8601-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 12147334098877123411 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedttddrkeefgddtlecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 46.105.44.159 Subject: [Qemu-devel] [PATCH for-2.12 v3 1/3] spapr/rtas: disable the decrementer interrupt when a CPU is unplugged X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?C=C3=A9dric_Le_Goater?= Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When a CPU is stopped with the 'stop-self' RTAS call, its state 'halted' is switched to 1 and, in this case, the MSR is not taken into account anymore in the cpu_has_work() routine. Only the pending hardware interrupts are checked with their LPCR:PECE* enablement bit. If the DECR timer fires after 'stop-self' is called and before the CPU 'stop' state is reached, the nearly-dead CPU will have some work to do and the guest will crash. This case happens very frequently with the not yet upstream P9 XIVE exploitation mode. In XICS mode, the DECR is occasionally fired but after 'stop' state, so no work is to be done and the guest survives. I suspect there is a race between the QEMU mainloop triggering the timers and the TCG CPU thread but I could not quite identify the root cause. To be safe, let's disable in the LPCR all the exceptions which can cause an exit while the CPU is in power-saving mode and reenable them when the CPU is started. For this purpose, we introduce a little helper routine to calculate the PECE bits for a processor variant. We could also use the mask value LPCR_PECE_L_MASK for the P8 and P9 processors. bit 47 and 48 are reserved on P7 but it is still compatible. Signed-off-by: Cédric Le Goater --- Changes in v3: - introduced a cpu_ppc_papr_pece_bits() helper to gather the PECE bits depending on the CPU family. - enabled Power-saving mode Exit Cause exceptions only on the boot CPU. Changes in v2: - used a new routine ppc_cpu_pvr_match() to discriminate CPU versions - removed the LPCR:PECE* enablement bit when the CPU is initialized if it is a secondary hw/ppc/spapr_rtas.c | 9 +++++++++ target/ppc/cpu.h | 1 + target/ppc/translate_init.c | 33 +++++++++++++++++++++++++-------- 3 files changed, 35 insertions(+), 8 deletions(-) diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c index cdf0b607a0a0..b5cff3ab29d3 100644 --- a/hw/ppc/spapr_rtas.c +++ b/hw/ppc/spapr_rtas.c @@ -174,6 +174,10 @@ static void rtas_start_cpu(PowerPCCPU *cpu_, sPAPRMachineState *spapr, kvm_cpu_synchronize_state(cs); env->msr = (1ULL << MSR_SF) | (1ULL << MSR_ME); + + /* Enable Power-saving mode Exit Cause exceptions for the new CPU */ + env->spr[SPR_LPCR] |= cpu_ppc_papr_pece_bits(env); + env->nip = start; env->gpr[3] = r3; cs->halted = 0; @@ -210,6 +214,11 @@ static void rtas_stop_self(PowerPCCPU *cpu, sPAPRMachineState *spapr, * no need to bother with specific bits, we just clear it. */ env->msr = 0; + + /* Disable Power-saving mode Exit Cause exceptions for the CPU. + * This could deliver an interrupt on a dying CPU and crash the + * guest */ + env->spr[SPR_LPCR] &= ~cpu_ppc_papr_pece_bits(env); } static inline int sysparm_st(target_ulong addr, target_ulong len, diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h index 989761b79569..7c84344421f3 100644 --- a/target/ppc/cpu.h +++ b/target/ppc/cpu.h @@ -1327,6 +1327,7 @@ void store_booke_tsr (CPUPPCState *env, target_ulong val); void ppc_tlb_invalidate_all (CPUPPCState *env); void ppc_tlb_invalidate_one (CPUPPCState *env, target_ulong addr); void cpu_ppc_set_papr(PowerPCCPU *cpu, PPCVirtualHypervisor *vhyp); +target_ulong cpu_ppc_papr_pece_bits(CPUPPCState *env); #endif #endif diff --git a/target/ppc/translate_init.c b/target/ppc/translate_init.c index b9c49c22f29f..a0bf5e01dc52 100644 --- a/target/ppc/translate_init.c +++ b/target/ppc/translate_init.c @@ -8901,11 +8901,28 @@ POWERPC_FAMILY(POWER9)(ObjectClass *oc, void *data) } #if !defined(CONFIG_USER_ONLY) + +target_ulong cpu_ppc_papr_pece_bits(CPUPPCState *env) +{ + switch (env->mmu_model) { + case POWERPC_MMU_3_00: + return LPCR_PDEE | LPCR_HDEE | LPCR_EEE | LPCR_DEE | LPCR_OEE; + default: + /* P7 and P8 has slightly different PECE bits, mostly because P8 adds + * bit 47 and 48 which are reserved on P7. Here we set them all, which + * will work as expected for both implementations + */ + return LPCR_P8_PECE0 | LPCR_P8_PECE1 | LPCR_P8_PECE2 | LPCR_P8_PECE3 | + LPCR_P8_PECE4; + } +} + void cpu_ppc_set_papr(PowerPCCPU *cpu, PPCVirtualHypervisor *vhyp) { CPUPPCState *env = &cpu->env; ppc_spr_t *lpcr = &env->spr_cb[SPR_LPCR]; ppc_spr_t *amor = &env->spr_cb[SPR_AMOR]; + CPUState *cs = CPU(cpu); cpu->vhyp = vhyp; @@ -8947,16 +8964,16 @@ void cpu_ppc_set_papr(PowerPCCPU *cpu, PPCVirtualHypervisor *vhyp) } else { lpcr->default_value &= ~(LPCR_UPRT | LPCR_GTSE); } - lpcr->default_value |= LPCR_PDEE | LPCR_HDEE | LPCR_EEE | LPCR_DEE | - LPCR_OEE; break; default: - /* P7 and P8 has slightly different PECE bits, mostly because P8 adds - * bit 47 and 48 which are reserved on P7. Here we set them all, which - * will work as expected for both implementations - */ - lpcr->default_value |= LPCR_P8_PECE0 | LPCR_P8_PECE1 | LPCR_P8_PECE2 | - LPCR_P8_PECE3 | LPCR_P8_PECE4; + ; + } + + /* Only enable Power-saving mode Exit Cause exceptions on the boot + * CPU. The RTAS command start-cpu will enable them on secondaries. + */ + if (cs == first_cpu) { + lpcr->default_value |= cpu_ppc_papr_pece_bits(env); } /* We should be followed by a CPU reset but update the active value From patchwork Mon Nov 20 10:03:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 839483 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ygPX96ShQz9s03 for ; Mon, 20 Nov 2017 21:07:05 +1100 (AEDT) Received: from localhost ([::1]:56436 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eGiym-0002fa-1t for incoming@patchwork.ozlabs.org; Mon, 20 Nov 2017 05:07:04 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45813) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eGiw6-0000j7-BR for qemu-devel@nongnu.org; Mon, 20 Nov 2017 05:04:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eGiw3-0001j2-3H for qemu-devel@nongnu.org; Mon, 20 Nov 2017 05:04:18 -0500 Received: from 2.mo173.mail-out.ovh.net ([178.33.251.49]:59490) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eGiw2-0001hr-T7 for qemu-devel@nongnu.org; Mon, 20 Nov 2017 05:04:15 -0500 Received: from player739.ha.ovh.net (b9.ovh.net [213.186.33.59]) by mo173.mail-out.ovh.net (Postfix) with ESMTP id A77F288A08 for ; Mon, 20 Nov 2017 11:04:12 +0100 (CET) Received: from zorba.kaod.org.com (LFbn-1-2231-173.w90-76.abo.wanadoo.fr [90.76.52.173]) (Authenticated sender: clg@kaod.org) by player739.ha.ovh.net (Postfix) with ESMTPSA id 8FDE44A007D; Mon, 20 Nov 2017 11:04:04 +0100 (CET) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson , Nikunj A Dadhania , Benjamin Herrenschmidt Date: Mon, 20 Nov 2017 11:03:46 +0100 Message-Id: <20171120100347.8601-3-clg@kaod.org> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20171120100347.8601-1-clg@kaod.org> References: <20171120100347.8601-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 12149022948640787283 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedttddrkeefgddtlecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 178.33.251.49 Subject: [Qemu-devel] [PATCH for-2.12 v3 2/3] spapr/rtas: fix reboot of a a SMP TCG guest X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?C=C3=A9dric_Le_Goater?= Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Just like for hot unplug CPUs, when a guest is rebooted, the secondary CPUs can be awaken by the decrementer and start entering SLOF at the same time the boot CPU is. To be safe, let's disable on the secondaries all the exceptions which can cause an exit while the CPU is in power-saving mode. Based on previous work from Nikunj A Dadhania Signed-off-by: Cédric Le Goater Reviewed-by: David Gibson --- Changes in v3: - used the cpu_ppc_papr_pece_bits() helper hw/ppc/spapr_cpu_core.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c index 3a4c17401226..4ba8563d49e4 100644 --- a/hw/ppc/spapr_cpu_core.c +++ b/hw/ppc/spapr_cpu_core.c @@ -35,6 +35,13 @@ static void spapr_cpu_reset(void *opaque) cs->halted = 1; env->spr[SPR_HIOR] = 0; + + /* Disable Power-saving mode Exit Cause exceptions for the CPU. + * This can cause issues when rebooting the guest if a secondary + * is awaken */ + if (cs != first_cpu) { + env->spr[SPR_LPCR] &= ~cpu_ppc_papr_pece_bits(env); + } } static void spapr_cpu_destroy(PowerPCCPU *cpu) From patchwork Mon Nov 20 10:03:47 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 839484 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ygPYd1hWpz9s7B for ; Mon, 20 Nov 2017 21:08:21 +1100 (AEDT) Received: from localhost ([::1]:56441 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eGizz-0003Lk-Bw for incoming@patchwork.ozlabs.org; Mon, 20 Nov 2017 05:08:19 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45946) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eGiwE-0000zm-Vv for qemu-devel@nongnu.org; Mon, 20 Nov 2017 05:04:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eGiw9-0001oh-CO for qemu-devel@nongnu.org; Mon, 20 Nov 2017 05:04:26 -0500 Received: from 4.mo173.mail-out.ovh.net ([46.105.34.219]:38104) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eGiw8-0001mb-HH for qemu-devel@nongnu.org; Mon, 20 Nov 2017 05:04:21 -0500 Received: from player739.ha.ovh.net (b9.ovh.net [213.186.33.59]) by mo173.mail-out.ovh.net (Postfix) with ESMTP id CC65788A58 for ; Mon, 20 Nov 2017 11:04:17 +0100 (CET) Received: from zorba.kaod.org.com (LFbn-1-2231-173.w90-76.abo.wanadoo.fr [90.76.52.173]) (Authenticated sender: clg@kaod.org) by player739.ha.ovh.net (Postfix) with ESMTPSA id BCB4D4A00B6; Mon, 20 Nov 2017 11:04:10 +0100 (CET) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: qemu-ppc@nongnu.org, qemu-devel@nongnu.org, David Gibson , Nikunj A Dadhania , Benjamin Herrenschmidt Date: Mon, 20 Nov 2017 11:03:47 +0100 Message-Id: <20171120100347.8601-4-clg@kaod.org> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20171120100347.8601-1-clg@kaod.org> References: <20171120100347.8601-1-clg@kaod.org> MIME-Version: 1.0 X-Ovh-Tracer-Id: 12150711798122580819 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedttddrkeefgddtlecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 46.105.34.219 Subject: [Qemu-devel] [PATCH for-2.12 v3 3/3] spapr/rtas: do not reset the MSR in stop-self command X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?C=C3=A9dric_Le_Goater?= Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When a CPU is stopped with the 'stop-self' RTAS call, its state 'halted' is switched to 1 and, in this case, the MSR is not taken into account anymore in the cpu_has_work() routine. Only the pending hardware interrupts are checked with their LPCR:PECE* enablement bit. The CPU is now also protected from the decrementer interrupt by the LPCR:PECE* bits which are disabled in the 'stop-self' RTAS call. Reseting the MSR is pointless. Signed-off-by: Cédric Le Goater Reviewed-by: David Gibson --- hw/ppc/spapr_rtas.c | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c index b5cff3ab29d3..7ddddc1c9c8a 100644 --- a/hw/ppc/spapr_rtas.c +++ b/hw/ppc/spapr_rtas.c @@ -204,16 +204,6 @@ static void rtas_stop_self(PowerPCCPU *cpu, sPAPRMachineState *spapr, cs->halted = 1; qemu_cpu_kick(cs); - /* - * While stopping a CPU, the guest calls H_CPPR which - * effectively disables interrupts on XICS level. - * However decrementer interrupts in TCG can still - * wake the CPU up so here we disable interrupts in MSR - * as well. - * As rtas_start_cpu() resets the whole MSR anyway, there is - * no need to bother with specific bits, we just clear it. - */ - env->msr = 0; /* Disable Power-saving mode Exit Cause exceptions for the CPU. * This could deliver an interrupt on a dying CPU and crash the