[{"id":1779353,"web_url":"http://patchwork.ozlabs.org/comment/1779353/","msgid":"<20171004012921.GQ3260@umbus.fritz.box>","list_archive_url":null,"date":"2017-10-04T01:29:21","subject":"Re: [Qemu-devel] [PATCH v5 4/6] target/ppc: Handle NMI guest exit","submitter":{"id":47,"url":"http://patchwork.ozlabs.org/api/people/47/","name":"David Gibson","email":"david@gibson.dropbear.id.au"},"content":"On Thu, Sep 28, 2017 at 04:08:10PM +0530, Aravinda Prasad wrote:\n> Memory error such as bit flips that cannot be corrected\n> by hardware are passed on to the kernel for handling.\n> If the memory address in error belongs to guest then\n> the guest kernel is responsible for taking suitable action.\n> Patch [1] enhances KVM to exit guest with exit reason\n> set to KVM_EXIT_NMI in such cases.\n> \n> This patch handles KVM_EXIT_NMI exit. If the guest OS\n> has registered the machine check handling routine by\n> calling \"ibm,nmi-register\", then the handler builds\n> the error log and invokes the registered handler else\n> invokes the handler at 0x200.\n> \n> Note that FWNMI handles synchronous machine check exceptions\n> triggered by the hardware and hence we do not extend\n> such support to the \"nmi\" command available in the QEMU\n> monitor. Hence, \"nmi\" command from the monitor will\n> always go through 0x200 vector.\n> \n> [1] https://www.spinics.net/lists/kvm-ppc/msg12637.html\n> \t(e20bbd3d and related commits)\n\nWhat does happen on KVM if an asynchronous machine check exception\noccurs while in the guest? Or under PowerVM for that matter.\n\n> \n> Signed-off-by: Aravinda Prasad \n> Signed-off-by: Mahesh Salgaonkar \n> ---\n> hw/ppc/spapr.c | 4 +++\n> hw/ppc/spapr_events.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++\n> include/hw/ppc/spapr.h | 6 +++++\n> target/ppc/kvm.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++\n> target/ppc/kvm_ppc.h | 14 +++++++++++\n> 5 files changed, 148 insertions(+)\n> \n> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c\n> index d568ea6..7780434 100644\n> --- a/hw/ppc/spapr.c\n> +++ b/hw/ppc/spapr.c\n> @@ -2453,6 +2453,10 @@ static void ppc_spapr_init(MachineState *machine)\n> error_report(\"Could not get size of LPAR rtas '%s'\", filename);\n> exit(1);\n> }\n> +\n> + /* Resize blob to accommodate error log. */\n> + spapr->rtas_size = spapr_get_rtas_size();\n> +\n> spapr->rtas_blob = g_malloc(spapr->rtas_size);\n> if (load_image_size(filename, spapr->rtas_blob, spapr->rtas_size) < 0) {\n> error_report(\"Could not load LPAR rtas '%s'\", filename);\n> diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c\n> index e377fc7..ac93a7b 100644\n> --- a/hw/ppc/spapr_events.c\n> +++ b/hw/ppc/spapr_events.c\n> @@ -41,6 +41,7 @@\n> #include \"qemu/bcd.h\"\n> #include \"hw/ppc/spapr_ovec.h\"\n> #include \n> +#include \n> \n> #define RTAS_LOG_VERSION_MASK 0xff000000\n> #define RTAS_LOG_VERSION_6 0x06000000\n> @@ -174,6 +175,22 @@ struct epow_extended_log {\n> struct rtas_event_log_v6_epow epow;\n> } QEMU_PACKED;\n> \n> +/*\n> + * Data format in RTAS Blob\n> + *\n> + * This structure contains error information related to Machine\n> + * Check exception. This is filled up and copied to rtas blob\n> + * upon machine check exception. The address of rtas blob is\n> + * passed on to OS registered machine check notification\n> + * routines upon machine check exception.\n> + */\n> +struct rtas_event_log_mce {\n> + target_ulong r3;\n> + struct rtas_error_log rtas_error_log;\n> + unsigned char buffer[1]; /* Start of extended log */\n\nI believe we allow C99 extensions in qemu, so you can use buffer[], a\nC99 flexible array member, rather than the length 1 hack.\n\n> +} QEMU_PACKED;\n> +\n> +\n> union drc_identifier {\n> uint32_t index;\n> uint32_t count;\n> @@ -623,6 +640,51 @@ void spapr_hotplug_req_remove_by_count_indexed(sPAPRDRConnectorType drc_type,\n> RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, &drc_id);\n> }\n> \n> +ssize_t spapr_get_rtas_size(void)\n> +{\n> + return RTAS_ERRLOG_OFFSET + sizeof(struct rtas_event_log_mce);\n\nErm.. because of the definition of rtas_event_log_mce, this only\nallows for 1 byte of extended log buffer. That doesn't seem right.\n\n> +}\n> +\n> +target_ulong spapr_mce_req_event(target_ulong r3, hwaddr rtas_addr,\n> + uint16_t flags, bool err_type, bool le)\n\nerr_tpe isn't a very informative name for a boolean. 'uncorrectable'\nwould be better. Although, didn't you say only uncorrectable errors\nare directed to the guest, so does this have any purpose anyway?\n\n> +{\n> + struct rtas_event_log_mce mc_log;\n> + uint32_t summary;\n> +\n> + /* Set error log fields */\n> + mc_log.r3 = r3;\n> +\n> + summary = RTAS_LOG_SEVERITY_ERROR_SYNC;\n> +\n> + if (flags & KVM_RUN_PPC_NMI_DISP_FULLY_RECOV) {\n\nKVM specific flags shouldn't be here, this translation should happen\nin the caller.\n\n> + summary |= RTAS_LOG_DISPOSITION_FULLY_RECOVERED;\n> + } else {\n> + summary |= RTAS_LOG_DISPOSITION_NOT_RECOVERED;\n> + }\n> +\n> + summary |= (RTAS_LOG_INITIATOR_MEMORY | RTAS_LOG_TARGET_MEMORY);\n> +\n> + if (err_type) {\n> + summary |= RTAS_LOG_TYPE_ECC_UNCORR;\n> + } else {\n> + summary |= RTAS_LOG_TYPE_ECC_CORR;\n> + }\n> +\n> + mc_log.rtas_error_log.summary = cpu_to_be32(summary);\n> +\n> + /* Handle all Host/Guest LE/BE combinations */\n\nI'd prefer you do this logic immediate as you store mc_log.r3. I\nreally dislike storing values into a structure in the wrong\nendianness, even temporarily - it makes it harder for someone reading\nthe code to discern what endianness the field is supposed to be in.\n\n> + if (le) {\n> + mc_log.r3 = cpu_to_le64(mc_log.r3);\n> + } else {\n> + mc_log.r3 = cpu_to_be64(mc_log.r3);\n> + }\n> +\n> + cpu_physical_memory_write(rtas_addr + RTAS_ERRLOG_OFFSET,\n> + &mc_log, sizeof(mc_log));\n> +\n> + return rtas_addr + RTAS_ERRLOG_OFFSET;\n> +}\n> +\n> static void check_exception(PowerPCCPU *cpu, sPAPRMachineState *spapr,\n> uint32_t token, uint32_t nargs,\n> target_ulong args,\n> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h\n> index 28b6e2e..a75e9cf 100644\n> --- a/include/hw/ppc/spapr.h\n> +++ b/include/hw/ppc/spapr.h\n> @@ -556,6 +556,9 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,\n> #define DIAGNOSTICS_RUN_MODE_IMMEDIATE 2\n> #define DIAGNOSTICS_RUN_MODE_PERIODIC 3\n> \n> +/* Offset from rtas-base where error log is placed */\n> +#define RTAS_ERRLOG_OFFSET 0x200\n\nIs there any particular rationale for this offset? Our actual RTAS\ncode is 20 bytes, much smaller than this.\n\n> +\n> static inline uint64_t ppc64_phys_to_real(uint64_t addr)\n> {\n> return addr & ~0xF000000000000000ULL;\n> @@ -675,6 +678,9 @@ int spapr_hpt_shift_for_ramsize(uint64_t ramsize);\n> void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift,\n> Error **errp);\n> void spapr_clear_pending_events(sPAPRMachineState *spapr);\n> +ssize_t spapr_get_rtas_size(void);\n> +target_ulong spapr_mce_req_event(target_ulong r3, hwaddr rtas_addr,\n> + uint16_t flags, bool err_type, bool le);\n> \n> /* CPU and LMB DRC release callbacks. */\n> void spapr_core_release(DeviceState *dev);\n> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c\n> index 171d3d8..7e4ce02 100644\n> --- a/target/ppc/kvm.c\n> +++ b/target/ppc/kvm.c\n> @@ -1798,6 +1798,11 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)\n> ret = 0;\n> break;\n> \n> + case KVM_EXIT_NMI:\n> + DPRINTF(\"handle NMI exception\\n\");\n> + ret = kvm_handle_nmi(cpu, run);\n> + break;\n> +\n> default:\n> fprintf(stderr, \"KVM: unknown exit reason %d\\n\", run->exit_reason);\n> ret = -1;\n> @@ -2746,6 +2751,63 @@ int kvm_arch_msi_data_to_gsi(uint32_t data)\n> return data & 0xffff;\n> }\n> \n> +int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run)\n\nMost of the logic here - everything except actually parsing the\nrelevant fields from kvm_run - should move to spapr_events. We may\nnot have any way of generating synchronous MCEs in TCG at the moment,\nbut we shouldn't exclude the possibility (being able to inject\nuncorrectable memory errors in TCG sounds like it could be quite a\nuseful debugging tool).\n\n> +{\n> + CPUPPCState *env = &cpu->env;\n> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());\n> + PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);\n> + target_ulong msr = 0;\n> + bool type, le;\n> +\n> + cpu_synchronize_state(CPU(cpu));\n> +\n> + /*\n> + * Properly set bits in MSR before we invoke the handler.\n> + * SRR0/1, DAR and DSISR are properly set by KVM\n> + */\n> + if (!(*pcc->interrupts_big_endian)(cpu)) {\n> + msr |= (1ULL << MSR_LE);\n> + }\n> +\n> + if (env->msr && (1ULL << MSR_SF)) {\n> + msr |= (1ULL << MSR_SF);\n> + }\n> +\n> + msr |= (1ULL << MSR_ME);\n> + env->msr = msr;\n> +\n> + if (!spapr->guest_machine_check_addr) {\n> + /*\n> + * If OS has not registered with \"ibm,nmi-register\"\n> + * jump to 0x200\n> + */\n> + env->nip = 0x200;\n\nSure you don't need to set some diagnostic registers in this case?\n\n> + return 0;\n> + }\n> +\n> + while (spapr->mc_status != -1) {\n> + /*\n> + * Check whether the same CPU got machine check error\n> + * while still handling the mc error (i.e., before\n> + * that CPU called \"ibm,nmi-interlock\"\n> + */\n> + if (spapr->mc_status == cpu->vcpu_id) {\n> + qemu_system_guest_panicked(NULL);\n> + }\n\nI think you need a check to break out of here if the system has been\nreset. Otherwise if you get:\n\n 1. MC generated on CPUs 0 & 1\n 2. MC delivered CPU0, CPU1 blocked here\n 3. system reset\n\nI think the CPU1 thread will still be stuck here, waiting to get back\nto the main loop that would check for the reset.\n\n> + qemu_cond_wait_iothread(&spapr->mc_delivery_cond);\n> + }\n> +\n> + spapr->mc_status = cpu->vcpu_id;\n> +\n> + type = !!(env->spr[SPR_DSISR] & P7_DSISR_MC_UE);\n> + le = !!(env->msr & (1ULL << MSR_LE));\n> + env->gpr[3] = spapr_mce_req_event(env->gpr[3], spapr->rtas_addr,\n> + run->flags, type, le);\n> + env->nip = spapr->guest_machine_check_addr;\n> +\n> + return 0;\n> +}\n> +\n> int kvmppc_enable_hwrng(void)\n> {\n> if (!kvm_enabled() || !kvm_check_extension(kvm_state, KVM_CAP_PPC_HWRNG)) {\n> diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h\n> index d6be38e..0139dae 100644\n> --- a/target/ppc/kvm_ppc.h\n> +++ b/target/ppc/kvm_ppc.h\n> @@ -71,6 +71,20 @@ bool kvmppc_pvr_workaround_required(PowerPCCPU *cpu);\n> \n> bool kvmppc_is_mem_backend_page_size_ok(const char *obj_path);\n> \n> +int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run);\n> +\n> +/*\n> + * Currently KVM only passes on the uncorrected machine\n> + * check memory error to guest. Other machine check errors\n> + * such as SLB multi-hit and TLB multi-hit are recovered\n> + * in KVM and are not passed on to guest.\n> + *\n> + * DSISR Bit for uncorrected machine check error. Based\n> + * on arch/powerpc/include/asm/mce.h\n> + */\n> +#define PPC_BIT(bit) (0x8000000000000000ULL >> bit)\n> +#define P7_DSISR_MC_UE (PPC_BIT(48)) /* P8 too */\n> +\n> #else\n> \n> static inline uint32_t kvmppc_get_tbfreq(void)\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=gibson.dropbear.id.au\n\theader.i=@gibson.dropbear.id.au header.b=\"jQsMwWi7\"; \n\tdkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6JWK6xTKz9sNc\n\tfor ;\n\tWed, 4 Oct 2017 12:40:28 +1100 (AEDT)","from localhost ([::1]:60922 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dzYfi-00072W-0J\n\tfor incoming@patchwork.ozlabs.org; Tue, 03 Oct 2017 21:40:26 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:57453)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dzYf2-000726-EG\n\tfor qemu-devel@nongnu.org; Tue, 03 Oct 2017 21:39:46 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dzYf0-0007BP-6G\n\tfor qemu-devel@nongnu.org; Tue, 03 Oct 2017 21:39:44 -0400","from ozlabs.org ([2401:3900:2:1::2]:60893)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dzYez-00077u-I4; Tue, 03 Oct 2017 21:39:42 -0400","by ozlabs.org (Postfix, from userid 1007)\n\tid 3y6JVJ2C7Xz9sNc; Wed, 4 Oct 2017 12:39:36 +1100 (AEDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple;\n\td=gibson.dropbear.id.au; s=201602; t=1507081176;\n\tbh=iTnTq+2P10Ef1BmW/wjA3DuIIf3X1NVAusTjFEtt+YU=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=jQsMwWi7cm947muqD7s49k28E/gyiGpQURjotQY6PZGIs8sH/eRZk+7Ua32JZjhQW\n\tR4hwIpkYBFSK41t74OsnT4Wl2yk4FgN9cRDBzlgtCi2Ob9/N1L+GxCizhkCBI4gSTH\n\tQ4kshrsMn38+flKf+XXEGJOmWrAe1pR9C1RLLOhg=","Date":"Wed, 4 Oct 2017 12:29:21 +1100","From":"David Gibson ","To":"Aravinda Prasad ","Message-ID":"<20171004012921.GQ3260@umbus.fritz.box>","References":"<150659494872.25889.2069124544245723984.stgit@aravinda>\n\t<150659509034.25889.15033474935802042526.stgit@aravinda>","MIME-Version":"1.0","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\"; boundary=\"uRjmd8ppyyws0Tml\"","Content-Disposition":"inline","In-Reply-To":"<150659509034.25889.15033474935802042526.stgit@aravinda>","User-Agent":"Mutt/1.9.0 (2017-09-02)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2401:3900:2:1::2","Subject":"Re: [Qemu-devel] [PATCH v5 4/6] target/ppc: Handle NMI guest exit","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"benh@au1.ibm.com, aik@ozlabs.ru, qemu-devel@nongnu.org,\n\tqemu-ppc@nongnu.org, paulus@samba.org, sam.bobroff@au1.ibm.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1782218,"web_url":"http://patchwork.ozlabs.org/comment/1782218/","msgid":"<14aeb1ff-867f-722a-28ce-7d66193051fc@linux.vnet.ibm.com>","list_archive_url":null,"date":"2017-10-08T08:59:26","subject":"Re: [Qemu-devel] [PATCH v5 4/6] target/ppc: Handle NMI guest exit","submitter":{"id":18580,"url":"http://patchwork.ozlabs.org/api/people/18580/","name":"Aravinda Prasad","email":"aravinda@linux.vnet.ibm.com"},"content":"On Wednesday 04 October 2017 06:59 AM, David Gibson wrote:\n> On Thu, Sep 28, 2017 at 04:08:10PM +0530, Aravinda Prasad wrote:\n>> Memory error such as bit flips that cannot be corrected\n>> by hardware are passed on to the kernel for handling.\n>> If the memory address in error belongs to guest then\n>> the guest kernel is responsible for taking suitable action.\n>> Patch [1] enhances KVM to exit guest with exit reason\n>> set to KVM_EXIT_NMI in such cases.\n>>\n>> This patch handles KVM_EXIT_NMI exit. If the guest OS\n>> has registered the machine check handling routine by\n>> calling \"ibm,nmi-register\", then the handler builds\n>> the error log and invokes the registered handler else\n>> invokes the handler at 0x200.\n>>\n>> Note that FWNMI handles synchronous machine check exceptions\n>> triggered by the hardware and hence we do not extend\n>> such support to the \"nmi\" command available in the QEMU\n>> monitor. Hence, \"nmi\" command from the monitor will\n>> always go through 0x200 vector.\n>>\n>> [1] https://www.spinics.net/lists/kvm-ppc/msg12637.html\n>> \t(e20bbd3d and related commits)\n> \n> What does happen on KVM if an asynchronous machine check exception\n> occurs while in the guest? Or under PowerVM for that matter.\n\nAFAIK asynchronous errors take a different path in KVM as it can happen\nin a different process context.\n\n> \n>>\n>> Signed-off-by: Aravinda Prasad \n>> Signed-off-by: Mahesh Salgaonkar \n>> ---\n>> hw/ppc/spapr.c | 4 +++\n>> hw/ppc/spapr_events.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++\n>> include/hw/ppc/spapr.h | 6 +++++\n>> target/ppc/kvm.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++\n>> target/ppc/kvm_ppc.h | 14 +++++++++++\n>> 5 files changed, 148 insertions(+)\n>>\n>> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c\n>> index d568ea6..7780434 100644\n>> --- a/hw/ppc/spapr.c\n>> +++ b/hw/ppc/spapr.c\n>> @@ -2453,6 +2453,10 @@ static void ppc_spapr_init(MachineState *machine)\n>> error_report(\"Could not get size of LPAR rtas '%s'\", filename);\n>> exit(1);\n>> }\n>> +\n>> + /* Resize blob to accommodate error log. */\n>> + spapr->rtas_size = spapr_get_rtas_size();\n>> +\n>> spapr->rtas_blob = g_malloc(spapr->rtas_size);\n>> if (load_image_size(filename, spapr->rtas_blob, spapr->rtas_size) < 0) {\n>> error_report(\"Could not load LPAR rtas '%s'\", filename);\n>> diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c\n>> index e377fc7..ac93a7b 100644\n>> --- a/hw/ppc/spapr_events.c\n>> +++ b/hw/ppc/spapr_events.c\n>> @@ -41,6 +41,7 @@\n>> #include \"qemu/bcd.h\"\n>> #include \"hw/ppc/spapr_ovec.h\"\n>> #include \n>> +#include \n>> \n>> #define RTAS_LOG_VERSION_MASK 0xff000000\n>> #define RTAS_LOG_VERSION_6 0x06000000\n>> @@ -174,6 +175,22 @@ struct epow_extended_log {\n>> struct rtas_event_log_v6_epow epow;\n>> } QEMU_PACKED;\n>> \n>> +/*\n>> + * Data format in RTAS Blob\n>> + *\n>> + * This structure contains error information related to Machine\n>> + * Check exception. This is filled up and copied to rtas blob\n>> + * upon machine check exception. The address of rtas blob is\n>> + * passed on to OS registered machine check notification\n>> + * routines upon machine check exception.\n>> + */\n>> +struct rtas_event_log_mce {\n>> + target_ulong r3;\n>> + struct rtas_error_log rtas_error_log;\n>> + unsigned char buffer[1]; /* Start of extended log */\n> \n> I believe we allow C99 extensions in qemu, so you can use buffer[], a\n> C99 flexible array member, rather than the length 1 hack.\n\nok.\n\n> \n>> +} QEMU_PACKED;\n>> +\n>> +\n>> union drc_identifier {\n>> uint32_t index;\n>> uint32_t count;\n>> @@ -623,6 +640,51 @@ void spapr_hotplug_req_remove_by_count_indexed(sPAPRDRConnectorType drc_type,\n>> RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, &drc_id);\n>> }\n>> \n>> +ssize_t spapr_get_rtas_size(void)\n>> +{\n>> + return RTAS_ERRLOG_OFFSET + sizeof(struct rtas_event_log_mce);\n> \n> Erm.. because of the definition of rtas_event_log_mce, this only\n> allows for 1 byte of extended log buffer. That doesn't seem right.\n\nThis is directly taken from the kernel's RTAS log (struct rtas_error_log\nin arch/powerpc/include/asm/rtas.h). I am not sure why they use 1 byte\nextended log buffer.\n\n> \n>> +}\n>> +\n>> +target_ulong spapr_mce_req_event(target_ulong r3, hwaddr rtas_addr,\n>> + uint16_t flags, bool err_type, bool le)\n> \n> err_tpe isn't a very informative name for a boolean. 'uncorrectable'\n> would be better. Although, didn't you say only uncorrectable errors\n> are directed to the guest, so does this have any purpose anyway?\n\nRight now only uncorrectable errors, but can be enhanced later to pass\non correctable errors to inform the guest.\n\n> \n>> +{\n>> + struct rtas_event_log_mce mc_log;\n>> + uint32_t summary;\n>> +\n>> + /* Set error log fields */\n>> + mc_log.r3 = r3;\n>> +\n>> + summary = RTAS_LOG_SEVERITY_ERROR_SYNC;\n>> +\n>> + if (flags & KVM_RUN_PPC_NMI_DISP_FULLY_RECOV) {\n> \n> KVM specific flags shouldn't be here, this translation should happen\n> in the caller.\n\nok.\n\n> \n>> + summary |= RTAS_LOG_DISPOSITION_FULLY_RECOVERED;\n>> + } else {\n>> + summary |= RTAS_LOG_DISPOSITION_NOT_RECOVERED;\n>> + }\n>> +\n>> + summary |= (RTAS_LOG_INITIATOR_MEMORY | RTAS_LOG_TARGET_MEMORY);\n>> +\n>> + if (err_type) {\n>> + summary |= RTAS_LOG_TYPE_ECC_UNCORR;\n>> + } else {\n>> + summary |= RTAS_LOG_TYPE_ECC_CORR;\n>> + }\n>> +\n>> + mc_log.rtas_error_log.summary = cpu_to_be32(summary);\n>> +\n>> + /* Handle all Host/Guest LE/BE combinations */\n> \n> I'd prefer you do this logic immediate as you store mc_log.r3. I\n> really dislike storing values into a structure in the wrong\n> endianness, even temporarily - it makes it harder for someone reading\n> the code to discern what endianness the field is supposed to be in.\n\nok.\n\n> \n>> + if (le) {\n>> + mc_log.r3 = cpu_to_le64(mc_log.r3);\n>> + } else {\n>> + mc_log.r3 = cpu_to_be64(mc_log.r3);\n>> + }\n>> +\n>> + cpu_physical_memory_write(rtas_addr + RTAS_ERRLOG_OFFSET,\n>> + &mc_log, sizeof(mc_log));\n>> +\n>> + return rtas_addr + RTAS_ERRLOG_OFFSET;\n>> +}\n>> +\n>> static void check_exception(PowerPCCPU *cpu, sPAPRMachineState *spapr,\n>> uint32_t token, uint32_t nargs,\n>> target_ulong args,\n>> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h\n>> index 28b6e2e..a75e9cf 100644\n>> --- a/include/hw/ppc/spapr.h\n>> +++ b/include/hw/ppc/spapr.h\n>> @@ -556,6 +556,9 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,\n>> #define DIAGNOSTICS_RUN_MODE_IMMEDIATE 2\n>> #define DIAGNOSTICS_RUN_MODE_PERIODIC 3\n>> \n>> +/* Offset from rtas-base where error log is placed */\n>> +#define RTAS_ERRLOG_OFFSET 0x200\n> \n> Is there any particular rationale for this offset? Our actual RTAS\n> code is 20 bytes, much smaller than this.\n\nJust to ensure some space if in case RTAS code needs to be extended in\nfuture.\n\n> \n>> +\n>> static inline uint64_t ppc64_phys_to_real(uint64_t addr)\n>> {\n>> return addr & ~0xF000000000000000ULL;\n>> @@ -675,6 +678,9 @@ int spapr_hpt_shift_for_ramsize(uint64_t ramsize);\n>> void spapr_reallocate_hpt(sPAPRMachineState *spapr, int shift,\n>> Error **errp);\n>> void spapr_clear_pending_events(sPAPRMachineState *spapr);\n>> +ssize_t spapr_get_rtas_size(void);\n>> +target_ulong spapr_mce_req_event(target_ulong r3, hwaddr rtas_addr,\n>> + uint16_t flags, bool err_type, bool le);\n>> \n>> /* CPU and LMB DRC release callbacks. */\n>> void spapr_core_release(DeviceState *dev);\n>> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c\n>> index 171d3d8..7e4ce02 100644\n>> --- a/target/ppc/kvm.c\n>> +++ b/target/ppc/kvm.c\n>> @@ -1798,6 +1798,11 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)\n>> ret = 0;\n>> break;\n>> \n>> + case KVM_EXIT_NMI:\n>> + DPRINTF(\"handle NMI exception\\n\");\n>> + ret = kvm_handle_nmi(cpu, run);\n>> + break;\n>> +\n>> default:\n>> fprintf(stderr, \"KVM: unknown exit reason %d\\n\", run->exit_reason);\n>> ret = -1;\n>> @@ -2746,6 +2751,63 @@ int kvm_arch_msi_data_to_gsi(uint32_t data)\n>> return data & 0xffff;\n>> }\n>> \n>> +int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run)\n> \n> Most of the logic here - everything except actually parsing the\n> relevant fields from kvm_run - should move to spapr_events. We may\n> not have any way of generating synchronous MCEs in TCG at the moment,\n> but we shouldn't exclude the possibility (being able to inject\n> uncorrectable memory errors in TCG sounds like it could be quite a\n> useful debugging tool).\n\nok.\n\n> \n>> +{\n>> + CPUPPCState *env = &cpu->env;\n>> + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());\n>> + PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);\n>> + target_ulong msr = 0;\n>> + bool type, le;\n>> +\n>> + cpu_synchronize_state(CPU(cpu));\n>> +\n>> + /*\n>> + * Properly set bits in MSR before we invoke the handler.\n>> + * SRR0/1, DAR and DSISR are properly set by KVM\n>> + */\n>> + if (!(*pcc->interrupts_big_endian)(cpu)) {\n>> + msr |= (1ULL << MSR_LE);\n>> + }\n>> +\n>> + if (env->msr && (1ULL << MSR_SF)) {\n>> + msr |= (1ULL << MSR_SF);\n>> + }\n>> +\n>> + msr |= (1ULL << MSR_ME);\n>> + env->msr = msr;\n>> +\n>> + if (!spapr->guest_machine_check_addr) {\n>> + /*\n>> + * If OS has not registered with \"ibm,nmi-register\"\n>> + * jump to 0x200\n>> + */\n>> + env->nip = 0x200;\n> \n> Sure you don't need to set some diagnostic registers in this case?\n\nNot sure. Let me check.\n\n> \n>> + return 0;\n>> + }\n>> +\n>> + while (spapr->mc_status != -1) {\n>> + /*\n>> + * Check whether the same CPU got machine check error\n>> + * while still handling the mc error (i.e., before\n>> + * that CPU called \"ibm,nmi-interlock\"\n>> + */\n>> + if (spapr->mc_status == cpu->vcpu_id) {\n>> + qemu_system_guest_panicked(NULL);\n>> + }\n> \n> I think you need a check to break out of here if the system has been\n> reset. Otherwise if you get:\n> \n> 1. MC generated on CPUs 0 & 1\n> 2. MC delivered CPU0, CPU1 blocked here\n> 3. system reset\n> \n> I think the CPU1 thread will still be stuck here, waiting to get back\n> to the main loop that would check for the reset.\n\nsure.\n\nRegards,\nAravinda\n> \n>> + qemu_cond_wait_iothread(&spapr->mc_delivery_cond);\n>> + }\n>> +\n>> + spapr->mc_status = cpu->vcpu_id;\n>> +\n>> + type = !!(env->spr[SPR_DSISR] & P7_DSISR_MC_UE);\n>> + le = !!(env->msr & (1ULL << MSR_LE));\n>> + env->gpr[3] = spapr_mce_req_event(env->gpr[3], spapr->rtas_addr,\n>> + run->flags, type, le);\n>> + env->nip = spapr->guest_machine_check_addr;\n>> +\n>> + return 0;\n>> +}\n>> +\n>> int kvmppc_enable_hwrng(void)\n>> {\n>> if (!kvm_enabled() || !kvm_check_extension(kvm_state, KVM_CAP_PPC_HWRNG)) {\n>> diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h\n>> index d6be38e..0139dae 100644\n>> --- a/target/ppc/kvm_ppc.h\n>> +++ b/target/ppc/kvm_ppc.h\n>> @@ -71,6 +71,20 @@ bool kvmppc_pvr_workaround_required(PowerPCCPU *cpu);\n>> \n>> bool kvmppc_is_mem_backend_page_size_ok(const char *obj_path);\n>> \n>> +int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run);\n>> +\n>> +/*\n>> + * Currently KVM only passes on the uncorrected machine\n>> + * check memory error to guest. Other machine check errors\n>> + * such as SLB multi-hit and TLB multi-hit are recovered\n>> + * in KVM and are not passed on to guest.\n>> + *\n>> + * DSISR Bit for uncorrected machine check error. Based\n>> + * on arch/powerpc/include/asm/mce.h\n>> + */\n>> +#define PPC_BIT(bit) (0x8000000000000000ULL >> bit)\n>> +#define P7_DSISR_MC_UE (PPC_BIT(48)) /* P8 too */\n>> +\n>> #else\n>> \n>> static inline uint32_t kvmppc_get_tbfreq(void)\n>>\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y8y5805QLz9t4B\n\tfor ;\n\tSun, 8 Oct 2017 20:00:19 +1100 (AEDT)","from localhost ([::1]:52960 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e17RZ-000630-Se\n\tfor incoming@patchwork.ozlabs.org; Sun, 08 Oct 2017 05:00:17 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:42341)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e17R7-00062Z-6e\n\tfor qemu-devel@nongnu.org; Sun, 08 Oct 2017 04:59:51 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e17Qz-0003Wz-17\n\tfor qemu-devel@nongnu.org; Sun, 08 Oct 2017 04:59:49 -0400","from mx0b-001b2d01.pphosted.com ([148.163.158.5]:34988\n\thelo=mx0a-001b2d01.pphosted.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1e17Qy-0003Wa-Qg\n\tfor qemu-devel@nongnu.org; Sun, 08 Oct 2017 04:59:40 -0400","from pps.filterd (m0098413.ppops.net [127.0.0.1])\n\tby mx0b-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id\n\tv988xJiR119253\n\tfor ; Sun, 8 Oct 2017 04:59:37 -0400","from e32.co.us.ibm.com (e32.co.us.ibm.com [32.97.110.150])\n\tby mx0b-001b2d01.pphosted.com with ESMTP id 2deuguunma-1\n\t(version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT)\n\tfor ; Sun, 08 Oct 2017 04:59:36 -0400","from localhost\n\tby e32.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use\n\tOnly! Violators will be prosecuted\n\tfor from ;\n\tSun, 8 Oct 2017 02:59:36 -0600","from b03cxnp07028.gho.boulder.ibm.com (9.17.130.15)\n\tby e32.co.us.ibm.com (192.168.1.132) with IBM ESMTP SMTP Gateway:\n\tAuthorized Use Only! Violators will be prosecuted; \n\tSun, 8 Oct 2017 02:59:33 -0600","from b03ledav006.gho.boulder.ibm.com\n\t(b03ledav006.gho.boulder.ibm.com [9.17.130.237])\n\tby b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with\n\tESMTP id v988xVp14784626; Sun, 8 Oct 2017 01:59:31 -0700","from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 53D1AC603C;\n\tSun, 8 Oct 2017 02:59:31 -0600 (MDT)","from [9.79.216.34] (unknown [9.79.216.34])\n\tby b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP id 3C08CC6042;\n\tSun, 8 Oct 2017 02:59:27 -0600 (MDT)"],"To":"David Gibson ","References":"<150659494872.25889.2069124544245723984.stgit@aravinda>\n\t<150659509034.25889.15033474935802042526.stgit@aravinda>\n\t<20171004012921.GQ3260@umbus.fritz.box>","From":"Aravinda Prasad ","Date":"Sun, 8 Oct 2017 14:29:26 +0530","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101\n\tThunderbird/45.7.0","MIME-Version":"1.0","In-Reply-To":"<20171004012921.GQ3260@umbus.fritz.box>","Content-Type":"text/plain; charset=windows-1252","Content-Transfer-Encoding":"7bit","X-TM-AS-GCONF":"00","x-cbid":"17100808-0004-0000-0000-000013080AA4","X-IBM-SpamModules-Scores":"","X-IBM-SpamModules-Versions":"BY=3.00007861; HX=3.00000241; KW=3.00000007;\n\tPH=3.00000004; SC=3.00000235; SDB=6.00928089; UDB=6.00467032;\n\tIPR=6.00708360; \n\tBA=6.00005626; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009;\n\tZB=6.00000000; \n\tZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00017446;\n\tXFM=3.00000015; UTC=2017-10-08 08:59:35","X-IBM-AV-DETECTION":"SAVI=unused REMOTE=unused XFE=unused","x-cbparentid":"17100808-0005-0000-0000-00008463EB1D","Message-Id":"<14aeb1ff-867f-722a-28ce-7d66193051fc@linux.vnet.ibm.com>","X-Proofpoint-Virus-Version":"vendor=fsecure engine=2.50.10432:, ,\n\tdefinitions=2017-10-08_02:, , signatures=0","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tspamscore=0 suspectscore=0\n\tmalwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam\n\tadjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000\n\tdefinitions=main-1710080133","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy]","X-Received-From":"148.163.158.5","Subject":"Re: [Qemu-devel] [PATCH v5 4/6] target/ppc: Handle NMI guest exit","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"benh@au1.ibm.com, aik@ozlabs.ru, qemu-devel@nongnu.org,\n\tqemu-ppc@nongnu.org, paulus@samba.org, sam.bobroff@au1.ibm.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1782363,"web_url":"http://patchwork.ozlabs.org/comment/1782363/","msgid":"<20171008234821.GP10050@umbus.fritz.box>","list_archive_url":null,"date":"2017-10-08T23:48:21","subject":"Re: [Qemu-devel] [PATCH v5 4/6] target/ppc: Handle NMI guest exit","submitter":{"id":47,"url":"http://patchwork.ozlabs.org/api/people/47/","name":"David Gibson","email":"david@gibson.dropbear.id.au"},"content":"On Sun, Oct 08, 2017 at 02:29:26PM +0530, Aravinda Prasad wrote:\n> \n> \n> On Wednesday 04 October 2017 06:59 AM, David Gibson wrote:\n> > On Thu, Sep 28, 2017 at 04:08:10PM +0530, Aravinda Prasad wrote:\n> >> Memory error such as bit flips that cannot be corrected\n> >> by hardware are passed on to the kernel for handling.\n> >> If the memory address in error belongs to guest then\n> >> the guest kernel is responsible for taking suitable action.\n> >> Patch [1] enhances KVM to exit guest with exit reason\n> >> set to KVM_EXIT_NMI in such cases.\n> >>\n> >> This patch handles KVM_EXIT_NMI exit. If the guest OS\n> >> has registered the machine check handling routine by\n> >> calling \"ibm,nmi-register\", then the handler builds\n> >> the error log and invokes the registered handler else\n> >> invokes the handler at 0x200.\n> >>\n> >> Note that FWNMI handles synchronous machine check exceptions\n> >> triggered by the hardware and hence we do not extend\n> >> such support to the \"nmi\" command available in the QEMU\n> >> monitor. Hence, \"nmi\" command from the monitor will\n> >> always go through 0x200 vector.\n> >>\n> >> [1] https://www.spinics.net/lists/kvm-ppc/msg12637.html\n> >> \t(e20bbd3d and related commits)\n> > \n> > What does happen on KVM if an asynchronous machine check exception\n> > occurs while in the guest? Or under PowerVM for that matter.\n> \n> AFAIK asynchronous errors take a different path in KVM as it can happen\n> in a different process context.\n\nWell, obviously, I'm wondering what impact it will have on the guest,\none way or another.\n\n[snip]\n> >> +ssize_t spapr_get_rtas_size(void)\n> >> +{\n> >> + return RTAS_ERRLOG_OFFSET + sizeof(struct rtas_event_log_mce);\n> > \n> > Erm.. because of the definition of rtas_event_log_mce, this only\n> > allows for 1 byte of extended log buffer. That doesn't seem right.\n> \n> This is directly taken from the kernel's RTAS log (struct rtas_error_log\n> in arch/powerpc/include/asm/rtas.h). I am not sure why they use 1 byte\n> extended log buffer.\n\nI think you'd better find out, then.\n\n[snip]\n> >> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h\n> >> index 28b6e2e..a75e9cf 100644\n> >> --- a/include/hw/ppc/spapr.h\n> >> +++ b/include/hw/ppc/spapr.h\n> >> @@ -556,6 +556,9 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,\n> >> #define DIAGNOSTICS_RUN_MODE_IMMEDIATE 2\n> >> #define DIAGNOSTICS_RUN_MODE_PERIODIC 3\n> >> \n> >> +/* Offset from rtas-base where error log is placed */\n> >> +#define RTAS_ERRLOG_OFFSET 0x200\n> > \n> > Is there any particular rationale for this offset? Our actual RTAS\n> > code is 20 bytes, much smaller than this.\n> \n> Just to ensure some space if in case RTAS code needs to be extended in\n> future.\n\nHm, but IIUC, we control both sides here. qemu puts the log into the\nRTAS buffer at a particular offset, and qemu tells the guest where to\nfind it at a particular offset within the RTAS buffer.\n\nSo, if we need to extend the RTAS code (unlikely) we can increase our\noffset, and the guest will be none the wiser.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=gibson.dropbear.id.au\n\theader.i=@gibson.dropbear.id.au header.b=\"Z/fevKkB\"; \n\tdkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9L230D9jz9t2c\n\tfor ;\n\tMon, 9 Oct 2017 10:59:07 +1100 (AEDT)","from localhost ([::1]:55512 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e1LTN-00059T-38\n\tfor incoming@patchwork.ozlabs.org; Sun, 08 Oct 2017 19:59:05 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:41317)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e1LSn-000596-Ff\n\tfor qemu-devel@nongnu.org; Sun, 08 Oct 2017 19:58:30 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e1LSm-0000Xy-0D\n\tfor qemu-devel@nongnu.org; Sun, 08 Oct 2017 19:58:29 -0400","from ozlabs.org ([2401:3900:2:1::2]:57713)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1e1LSl-0000X4-CB; Sun, 08 Oct 2017 19:58:27 -0400","by ozlabs.org (Postfix, from userid 1007)\n\tid 3y9L1C0Fwbz9sxR; Mon, 9 Oct 2017 10:58:22 +1100 (AEDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple;\n\td=gibson.dropbear.id.au; s=201602; t=1507507103;\n\tbh=eR8WjENSuAE0Ec4A5clxDjUy22jQjwyJVhlZ0sSvRyM=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=Z/fevKkBi9HtQnxmeKKxuFHv99Pr24DZK72l0Q46OtlY92UpmmZbJnHKj/M2S94mA\n\t+Bh7LUIP4VU3vPN88WIYzubfOJZEjnexHpsbYobT5lV8ME8hL9y7WTIATyVgqXgOwq\n\tRdaiFiHFAmAo57gCTV+p7A24R1zRGdUrnX/EUXas=","Date":"Mon, 9 Oct 2017 10:48:21 +1100","From":"David Gibson ","To":"Aravinda Prasad ","Message-ID":"<20171008234821.GP10050@umbus.fritz.box>","References":"<150659494872.25889.2069124544245723984.stgit@aravinda>\n\t<150659509034.25889.15033474935802042526.stgit@aravinda>\n\t<20171004012921.GQ3260@umbus.fritz.box>\n\t<14aeb1ff-867f-722a-28ce-7d66193051fc@linux.vnet.ibm.com>","MIME-Version":"1.0","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\"; boundary=\"+hz2tM55CCA8Ej21\"","Content-Disposition":"inline","In-Reply-To":"<14aeb1ff-867f-722a-28ce-7d66193051fc@linux.vnet.ibm.com>","User-Agent":"Mutt/1.9.1 (2017-09-22)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2401:3900:2:1::2","Subject":"Re: [Qemu-devel] [PATCH v5 4/6] target/ppc: Handle NMI guest exit","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"benh@au1.ibm.com, aik@ozlabs.ru, qemu-devel@nongnu.org,\n\tqemu-ppc@nongnu.org, paulus@samba.org, sam.bobroff@au1.ibm.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]