[{"id":1759983,"web_url":"http://patchwork.ozlabs.org/comment/1759983/","msgid":"<47f291c6-4e76-685b-b246-81bf04b410ff@arm.com>","list_archive_url":null,"date":"2017-08-30T09:19:08","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"Hi Julien,\n\nOn 30/08/17 10:01, Julien Thierry wrote:\n> Software Step exception is missing after trapping instruction from the guest.\n> \n> We need to set the PSR.SS to 0 for the guest vcpu before resuming guest\n> execution.\n> \n> Signed-off-by: Julien Thierry \n> Cc: Christoffer Dall \n> Cc: Marc Zyngier \n> Cc: Catalin Marinas \n> Cc: Will Deacon \n> \n> ---\n> arch/arm64/include/asm/kvm_asm.h | 2 ++\n> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n> arch/arm64/kvm/debug.c | 12 +++++++++++-\n> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n> 4 files changed, 27 insertions(+), 1 deletion(-)\n> \n> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h\n> index 26a64d0..398bbaa 100644\n> --- a/arch/arm64/include/asm/kvm_asm.h\n> +++ b/arch/arm64/include/asm/kvm_asm.h\n> @@ -32,6 +32,8 @@\n> \n> #define KVM_ARM64_DEBUG_DIRTY_SHIFT\t0\n> #define KVM_ARM64_DEBUG_DIRTY\t\t(1 << KVM_ARM64_DEBUG_DIRTY_SHIFT)\n> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT\t1\n> +#define KVM_ARM64_DEBUG_INST_SKIP\t(1 << KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n> \n> #define kvm_ksym_ref(sym)\t\t\t\t\t\t\\\n> \t({\t\t\t\t\t\t\t\t\\\n> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h\n> index fe39e68..d401c64 100644\n> --- a/arch/arm64/include/asm/kvm_emulate.h\n> +++ b/arch/arm64/include/asm/kvm_emulate.h\n> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu *vcpu, bool is_wide_instr)\n> \t\tkvm_skip_instr32(vcpu, is_wide_instr);\n> \telse\n> \t\t*vcpu_pc(vcpu) += 4;\n> +\t/* Let debug engine know we skipped an instruction */\n> +\tvcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n> }\n> \n> static inline void vcpu_set_thumb(struct kvm_vcpu *vcpu)\n> diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c\n> index dbadfaf..4806e6b 100644\n> --- a/arch/arm64/kvm/debug.c\n> +++ b/arch/arm64/kvm/debug.c\n> @@ -151,12 +151,22 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)\n> \t\t * debugging the system.\n> \t\t */\n> \t\tif (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) {\n> -\t\t\t*vcpu_cpsr(vcpu) |= DBG_SPSR_SS;\n> +\t\t\t/*\n> +\t\t\t * If we skipped an instruction while single stepping,\n> +\t\t\t * spsr.ss needs to be 0 in order to trigger SS\n> +\t\t\t * exception on return.\n> +\t\t\t */\n> +\t\t\tif (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_INST_SKIP))\n> +\t\t\t\t*vcpu_cpsr(vcpu) |= DBG_SPSR_SS;\n> +\t\t\telse\n> +\t\t\t\t*vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;\n> \t\t\tvcpu_sys_reg(vcpu, MDSCR_EL1) |= DBG_MDSCR_SS;\n> \t\t} else {\n> \t\t\tvcpu_sys_reg(vcpu, MDSCR_EL1) &= ~DBG_MDSCR_SS;\n> \t\t}\n> \n> +\t\tvcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_INST_SKIP;\n> +\n> \t\ttrace_kvm_arm_set_dreg32(\"SPSR_EL2\", *vcpu_cpsr(vcpu));\n> \n> \t\t/*\n> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c\n> index 945e79c..6030acd 100644\n> --- a/arch/arm64/kvm/hyp/switch.c\n> +++ b/arch/arm64/kvm/hyp/switch.c\n> @@ -22,6 +22,7 @@\n> #include \n> #include \n> #include \n> +#include \n> \n> static bool __hyp_text __fpsimd_enabled_nvhe(void)\n> {\n> @@ -276,6 +277,10 @@ static void __hyp_text __skip_instr(struct kvm_vcpu *vcpu)\n> \t}\n> \n> \twrite_sysreg_el2(*vcpu_pc(vcpu), elr);\n> +\n> +\t*vcpu_cpsr(vcpu) = read_sysreg_el2(spsr);\n> +\t*vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;\n> +\twrite_sysreg_el2(*vcpu_cpsr(vcpu), spsr);\n> }\n> \n> int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n> @@ -343,6 +348,13 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n> \t\t\tif (ret == -1) {\n> \t\t\t\t/* Promote an illegal access to an SError */\n> \t\t\t\t__skip_instr(vcpu);\n> +\n> +\t\t\t\t/*\n> +\t\t\t\t * We're not jumping back, let debug setup know\n> +\t\t\t\t * we skipped an instruction.\n> +\t\t\t\t */\n> +\t\t\t\tvcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n> +\n\nThis feels odd. You've already cleared the SS bit from the SPSR, and yet\nyou're setting that flag. What is the semantic of this?\n\n> \t\t\t\texit_code = ARM_EXCEPTION_EL1_SERROR;\n> \t\t\t}\n> \n> --\n> 1.9.1\n> \n\nThanks,\n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"foN0anZS\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xj0MK48Lsz9t2Q\n\tfor ;\n\tWed, 30 Aug 2017 19:19:41 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dmz9t-0001Xe-1r; Wed, 30 Aug 2017 09:19:37 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dmz9o-0001NG-9u for linux-arm-kernel@lists.infradead.org;\n\tWed, 30 Aug 2017 09:19:34 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 891A42B;\n\tWed, 30 Aug 2017 02:19:11 -0700 (PDT)","from [10.1.207.16] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t6CF753F483; Wed, 30 Aug 2017 02:19:10 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=4aMbAAJKyo720eTVloejoH0WaFJflAF9fK72zel/sDI=;\n\tb=foN0anZS3PU1ot\n\t11Pjr2dG/hTFeLFswUy877ByPp2wauUxtFkNxAlldGo59FRt0ppvsK2lvQ8WeHHuyi0/GWKSD/Vha\n\tqilZkNtROXl0LmRERSUoBVZnCJgahHndYvksf3MNtOLPlaxbImtTAK/Vnb0JUqqVTR3xq3KAuIp4Z\n\ttl53aAWzZp5OBA6CiGyVVPA8bA8siacBtuIScnJmWjklUFU8ARpLbQN8gQ/kwjtQ/qWyFp/CXpav+\n\tjlxhupQ72vWJH+ME+RVCvMmRmF4K+SvuhDu34Ltfh8nkRL4VB4Iki1ZVW/qEbbx6PoCBMWApfW6e4\n\tnqqDxIr0w10DZvtXqRAg==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"Julien Thierry ,\n\tlinux-arm-kernel@lists.infradead.org","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>","From":"Marc Zyngier ","Organization":"ARM Ltd","Message-ID":"<47f291c6-4e76-685b-b246-81bf04b410ff@arm.com>","Date":"Wed, 30 Aug 2017 10:19:08 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<1504083688-48334-4-git-send-email-julien.thierry@arm.com>","Content-Language":"en-GB","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170830_021932_468948_2B0BB350 ","X-CRM114-Status":"GOOD ( 17.07 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon ,\n\tChristoffer Dall ","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1759999,"web_url":"http://patchwork.ozlabs.org/comment/1759999/","msgid":"<3c5bf8a6-dade-268c-83d3-7f3b31f4224f@arm.com>","list_archive_url":null,"date":"2017-08-30T09:40:39","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":72256,"url":"http://patchwork.ozlabs.org/api/people/72256/","name":"Julien Thierry","email":"julien.thierry@arm.com"},"content":"Hi Marc,\n\nOn 30/08/17 10:19, Marc Zyngier wrote:\n> Hi Julien,\n> \n> On 30/08/17 10:01, Julien Thierry wrote:\n>> Software Step exception is missing after trapping instruction from the guest.\n>>\n>> We need to set the PSR.SS to 0 for the guest vcpu before resuming guest\n>> execution.\n>>\n>> Signed-off-by: Julien Thierry \n>> Cc: Christoffer Dall \n>> Cc: Marc Zyngier \n>> Cc: Catalin Marinas \n>> Cc: Will Deacon \n>>\n>> ---\n>> arch/arm64/include/asm/kvm_asm.h | 2 ++\n>> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n>> arch/arm64/kvm/debug.c | 12 +++++++++++-\n>> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n>> 4 files changed, 27 insertions(+), 1 deletion(-)\n>>\n>> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h\n>> index 26a64d0..398bbaa 100644\n>> --- a/arch/arm64/include/asm/kvm_asm.h\n>> +++ b/arch/arm64/include/asm/kvm_asm.h\n>> @@ -32,6 +32,8 @@\n>>\n>> #define KVM_ARM64_DEBUG_DIRTY_SHIFT\t0\n>> #define KVM_ARM64_DEBUG_DIRTY\t\t(1 << KVM_ARM64_DEBUG_DIRTY_SHIFT)\n>> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT\t1\n>> +#define KVM_ARM64_DEBUG_INST_SKIP\t(1 << KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>>\n>> #define kvm_ksym_ref(sym)\t\t\t\t\t\t\\\n>> \t({\t\t\t\t\t\t\t\t\\\n>> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h\n>> index fe39e68..d401c64 100644\n>> --- a/arch/arm64/include/asm/kvm_emulate.h\n>> +++ b/arch/arm64/include/asm/kvm_emulate.h\n>> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu *vcpu, bool is_wide_instr)\n>> \t\tkvm_skip_instr32(vcpu, is_wide_instr);\n>> \telse\n>> \t\t*vcpu_pc(vcpu) += 4;\n>> +\t/* Let debug engine know we skipped an instruction */\n>> +\tvcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n>> }\n>>\n>> static inline void vcpu_set_thumb(struct kvm_vcpu *vcpu)\n>> diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c\n>> index dbadfaf..4806e6b 100644\n>> --- a/arch/arm64/kvm/debug.c\n>> +++ b/arch/arm64/kvm/debug.c\n>> @@ -151,12 +151,22 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)\n>> \t\t * debugging the system.\n>> \t\t */\n>> \t\tif (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) {\n>> -\t\t\t*vcpu_cpsr(vcpu) |= DBG_SPSR_SS;\n>> +\t\t\t/*\n>> +\t\t\t * If we skipped an instruction while single stepping,\n>> +\t\t\t * spsr.ss needs to be 0 in order to trigger SS\n>> +\t\t\t * exception on return.\n>> +\t\t\t */\n>> +\t\t\tif (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_INST_SKIP))\n>> +\t\t\t\t*vcpu_cpsr(vcpu) |= DBG_SPSR_SS;\n>> +\t\t\telse\n>> +\t\t\t\t*vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;\n>> \t\t\tvcpu_sys_reg(vcpu, MDSCR_EL1) |= DBG_MDSCR_SS;\n>> \t\t} else {\n>> \t\t\tvcpu_sys_reg(vcpu, MDSCR_EL1) &= ~DBG_MDSCR_SS;\n>> \t\t}\n>>\n>> +\t\tvcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_INST_SKIP;\n>> +\n>> \t\ttrace_kvm_arm_set_dreg32(\"SPSR_EL2\", *vcpu_cpsr(vcpu));\n>>\n>> \t\t/*\n>> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c\n>> index 945e79c..6030acd 100644\n>> --- a/arch/arm64/kvm/hyp/switch.c\n>> +++ b/arch/arm64/kvm/hyp/switch.c\n>> @@ -22,6 +22,7 @@\n>> #include \n>> #include \n>> #include \n>> +#include \n>>\n>> static bool __hyp_text __fpsimd_enabled_nvhe(void)\n>> {\n>> @@ -276,6 +277,10 @@ static void __hyp_text __skip_instr(struct kvm_vcpu *vcpu)\n>> \t}\n>>\n>> \twrite_sysreg_el2(*vcpu_pc(vcpu), elr);\n>> +\n>> +\t*vcpu_cpsr(vcpu) = read_sysreg_el2(spsr);\n>> +\t*vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;\n>> +\twrite_sysreg_el2(*vcpu_cpsr(vcpu), spsr);\n>> }\n>>\n>> int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n>> @@ -343,6 +348,13 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n>> \t\t\tif (ret == -1) {\n>> \t\t\t\t/* Promote an illegal access to an SError */\n>> \t\t\t\t__skip_instr(vcpu);\n>> +\n>> +\t\t\t\t/*\n>> +\t\t\t\t * We're not jumping back, let debug setup know\n>> +\t\t\t\t * we skipped an instruction.\n>> +\t\t\t\t */\n>> +\t\t\t\tvcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n>> +\n> \n> This feels odd. You've already cleared the SS bit from the SPSR, and yet\n> you're setting that flag. What is the semantic of this?\n\nThe thing is that whenever you get out of __kvm_vcpu_run, \nkvm_arm_setup_debug will get called before running a guest again. So my \nidea is that kvm_arm_setup_debug needs to be aware whether SPSR.SS must \nbe set or not. Otherwise kvm_arm_setup_debug will always set the SPSR.SS \nflag. So KVM_ARM64_DEBUG_INST_SKIP is to tell the debug setup that we \nskipped an instruction (here as well as in kvm_skip_instr).\n\nThe fact that SPSR.SS is clear in the illegal access case is because it \nuses __skip_instr instead of kvm_skip_instr because we have not saved \nthe guest state at that point. But the clearing of SPSR.SS in \n__skip_instr is more targeted at cases that directly jump back into the \nguest (trapped GICv2/3 CPUIF accesses).\n\nLet me know if you feel there is a cleaner way to do this or something \nthat would make this clearer.\n\nThanks,","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"mPI5dwvK\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xj0r932mqz9sN7\n\tfor ;\n\tWed, 30 Aug 2017 19:41:13 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dmzUk-0007dv-3o; Wed, 30 Aug 2017 09:41:10 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dmzUe-0007b0-3l for linux-arm-kernel@lists.infradead.org;\n\tWed, 30 Aug 2017 09:41:08 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D897F2B;\n\tWed, 30 Aug 2017 02:40:42 -0700 (PDT)","from [10.1.206.31] (e112298-lin.cambridge.arm.com [10.1.206.31])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tCB85F3F483; Wed, 30 Aug 2017 02:40:41 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=y7BC4E/W9eCXHPhKHRXMAbKMxTemK0xyomex96IU8QU=;\n\tb=mPI5dwvKlTiX7MXgBegLjP5ay\n\tRNmN/MOB8oSB0mMnhtuucAmORW7APO4Q3WMWF2olXVoFanzvYfnsTqQ2FIWMsdwQ00vXYBYVTD19P\n\tc5QvqmtzM/Yb3WqresfsBWmG7ZWk+IfI5SOKl1y25lBeD3V9/RGiKTWSK3MPhrohxMMgltUgjAdwZ\n\tTy8DTk7K9LYy2R2JhKq+Y6E8NkrhfvkW6rD2kTF9+5uosIYpEGQcYAmwzi10oNfMq6jNxKrhyOAJr\n\tOS6bpMPzpZIt7+kEQ9krlw8hjSb3h+oyfqUeQrDCmEB7QWyVbsWWgWxOTridNqfiNCceZ1++0tM8T\n\tD5kpk7fUQ==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"Marc Zyngier , linux-arm-kernel@lists.infradead.org","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t<47f291c6-4e76-685b-b246-81bf04b410ff@arm.com>","From":"Julien Thierry ","Message-ID":"<3c5bf8a6-dade-268c-83d3-7f3b31f4224f@arm.com>","Date":"Wed, 30 Aug 2017 10:40:39 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.1.1","MIME-Version":"1.0","In-Reply-To":"<47f291c6-4e76-685b-b246-81bf04b410ff@arm.com>","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170830_024104_255852_D17812CA ","X-CRM114-Status":"GOOD ( 17.21 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon ,\n\tChristoffer Dall ","Content-Transfer-Encoding":"7bit","Content-Type":"text/plain; charset=\"us-ascii\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1760408,"web_url":"http://patchwork.ozlabs.org/comment/1760408/","msgid":"","list_archive_url":null,"date":"2017-08-30T18:53:19","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":26352,"url":"http://patchwork.ozlabs.org/api/people/26352/","name":"Christoffer Dall","email":"christoffer.dall@linaro.org"},"content":"Hi Julien,\n\n[cc'ing Alex Bennée here who wrote the debug code for arm64]\n\nOn Wed, Aug 30, 2017 at 11:01 AM, Julien Thierry wrote:\n> Software Step exception is missing after trapping instruction from the guest.\n>\n> We need to set the PSR.SS to 0 for the guest vcpu before resuming guest\n> execution.\n>\n> Signed-off-by: Julien Thierry \n> Cc: Christoffer Dall \n> Cc: Marc Zyngier \n> Cc: Catalin Marinas \n> Cc: Will Deacon \n>\n> ---\n> arch/arm64/include/asm/kvm_asm.h | 2 ++\n> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n> arch/arm64/kvm/debug.c | 12 +++++++++++-\n> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n> 4 files changed, 27 insertions(+), 1 deletion(-)\n>\n> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h\n> index 26a64d0..398bbaa 100644\n> --- a/arch/arm64/include/asm/kvm_asm.h\n> +++ b/arch/arm64/include/asm/kvm_asm.h\n> @@ -32,6 +32,8 @@\n>\n> #define KVM_ARM64_DEBUG_DIRTY_SHIFT 0\n> #define KVM_ARM64_DEBUG_DIRTY (1 << KVM_ARM64_DEBUG_DIRTY_SHIFT)\n> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT 1\n> +#define KVM_ARM64_DEBUG_INST_SKIP (1 << KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>\n> #define kvm_ksym_ref(sym) \\\n> ({ \\\n> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h\n> index fe39e68..d401c64 100644\n> --- a/arch/arm64/include/asm/kvm_emulate.h\n> +++ b/arch/arm64/include/asm/kvm_emulate.h\n> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu *vcpu, bool is_wide_instr)\n> kvm_skip_instr32(vcpu, is_wide_instr);\n> else\n> *vcpu_pc(vcpu) += 4;\n> + /* Let debug engine know we skipped an instruction */\n> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n\nWhy do we need to defer this action until later?\n\nCan't we simply do clear DBG_SPSR_SS here?\n\nI think even if the guest kernel is single-stepping guest userspace,\nyou're still safe because you'll take another debug exception from\nhaving 'executed' (read: emulated) this instruction in the hypervisor.\n\n> }\n>\n> static inline void vcpu_set_thumb(struct kvm_vcpu *vcpu)\n> diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c\n> index dbadfaf..4806e6b 100644\n> --- a/arch/arm64/kvm/debug.c\n> +++ b/arch/arm64/kvm/debug.c\n> @@ -151,12 +151,22 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)\n> * debugging the system.\n> */\n> if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) {\n> - *vcpu_cpsr(vcpu) |= DBG_SPSR_SS;\n> + /*\n> + * If we skipped an instruction while single stepping,\n> + * spsr.ss needs to be 0 in order to trigger SS\n> + * exception on return.\n> + */\n> + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_INST_SKIP))\n> + *vcpu_cpsr(vcpu) |= DBG_SPSR_SS;\n> + else\n> + *vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;\n\nThen you wouldn't need this.\n\n> vcpu_sys_reg(vcpu, MDSCR_EL1) |= DBG_MDSCR_SS;\n> } else {\n> vcpu_sys_reg(vcpu, MDSCR_EL1) &= ~DBG_MDSCR_SS;\n> }\n>\n> + vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_INST_SKIP;\n> +\n> trace_kvm_arm_set_dreg32(\"SPSR_EL2\", *vcpu_cpsr(vcpu));\n>\n> /*\n> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c\n> index 945e79c..6030acd 100644\n> --- a/arch/arm64/kvm/hyp/switch.c\n> +++ b/arch/arm64/kvm/hyp/switch.c\n> @@ -22,6 +22,7 @@\n> #include \n> #include \n> #include \n> +#include \n>\n> static bool __hyp_text __fpsimd_enabled_nvhe(void)\n> {\n> @@ -276,6 +277,10 @@ static void __hyp_text __skip_instr(struct kvm_vcpu *vcpu)\n> }\n>\n> write_sysreg_el2(*vcpu_pc(vcpu), elr);\n> +\n> + *vcpu_cpsr(vcpu) = read_sysreg_el2(spsr);\n> + *vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;\n> + write_sysreg_el2(*vcpu_cpsr(vcpu), spsr);\n\nI don't think you really need to use *vcpu_cpsr(vcpu) here, I think\nyou can just use a temporary variable, which may make the flow a bit\neasier to read actually.\n\n> }\n>\n> int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n> @@ -343,6 +348,13 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n> if (ret == -1) {\n> /* Promote an illegal access to an SError */\n> __skip_instr(vcpu);\n> +\n> + /*\n> + * We're not jumping back, let debug setup know\n> + * we skipped an instruction.\n> + */\n> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n\nI agree with Marc, I don't think you need this, because you have\nmodified the hardware spsr_el2, which you're going to read back in\n__sysreg_save_guest_state().\n\n> exit_code = ARM_EXCEPTION_EL1_SERROR;\n> }\n>\n> --\n> 1.9.1\n\nThanks,\n-Christoffer","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"Us5+IVKV\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"Jva4wRIy\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xjF5n33J0z9s83\n\tfor ;\n\tThu, 31 Aug 2017 04:53:49 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dn87V-0001nG-Ow; Wed, 30 Aug 2017 18:53:45 +0000","from mail-yw0-x22f.google.com ([2607:f8b0:4002:c05::22f])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dn87R-0001lc-Ey for linux-arm-kernel@lists.infradead.org;\n\tWed, 30 Aug 2017 18:53:43 +0000","by mail-yw0-x22f.google.com with SMTP id s143so35475110ywg.0\n\tfor ;\n\tWed, 30 Aug 2017 11:53:20 -0700 (PDT)","by 10.37.97.23 with HTTP; Wed, 30 Aug 2017 11:53:19 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:\n\tReferences:In-Reply-To:MIME-Version:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=8X3g8PM1HtSk/ZIu++fVFFTbTEsnAe8gT6HVWujgdo4=;\n\tb=Us5+IVKVvXhok7\n\t7nYR4552nBPTc/L9FcC5PDm5EH6ieVg816smitHTCX3z2Gr5j18VOUE/8p8SMfpv1ACveDugMm6M0\n\tNGcCj6YJFQ95O1KzeN2wbd5u+m+1wG9YlqN+yBcZTe3T1dzwHn6VvvYdhI6IITA2S7haEpKaXyncl\n\tIH4cEzn7M+kFREDLuKwoayfMEd12zNCe4rQHCOATCj7248pYp7cHUBx6W72gNRanqhWmNawW93saA\n\tGdMtUYftGYLy9OcjMh0BSNkZOXyHPSyQhoeCUAEIxObgmmYgkHw9AA8o4KHiaFM9PRaxzZITitElm\n\tiH6TwXqSLxdbNLTujiTg==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc:content-transfer-encoding;\n\tbh=N0kfVKa9mtddQsYoNVh4i0z3sVj14RRPvDleeodp7FA=;\n\tb=Jva4wRIyRCQFbFlB5RVrnr4SFILDyYPRDuEQ5FVgRWgJUvidjhsygatKHQC7yLXEgy\n\tu8rlhMtSlbDYl/KgJt0dLBVp088Rkhu2Oc7fL5GaL9TPMCvdfxmOBJ9aOBKLpQZu/9oc\n\td8AM41Bu4xiq1EO1UZnY2PV69HVDl9deRc7r0="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc:content-transfer-encoding;\n\tbh=N0kfVKa9mtddQsYoNVh4i0z3sVj14RRPvDleeodp7FA=;\n\tb=aMKon9JH/C+I+bkbajsuAX2JXRaxEm8exS7BDxCF5JWZZrlDlPt0yKRptCWziIQGo9\n\t3Q/NVtvr+Dpl60BIEsav2+KtBtNyizhp/4bB+zEeoGjH3MUkFhJIhfWZ4CKnp+vIiXcR\n\tYXemg8mO9oGa9Z2uWkoazkzYU6or+zffXFkrtLgfivTK7aQq9wBpY6iy3wCKfQ5ULQ0n\n\twyV1R7ZewU/tRpg/3NbcoYRu+NTMjfzufkMfVAqYcmzhf5Jr0QShwVo7CM9+ZKquuhOv\n\tLyuDGf+dilY3X4eLQ8R/dh5iPFREMhIOZIqQxj/URDnEQOE/9p0RYYzSdPiJvXQk8KKV\n\tehaw==","X-Gm-Message-State":"AHYfb5iWn7KdqyPCAiHNjQU7fjUS/JA/H1Lc9Ei/0wt/n1UE7NZiULyy\n\t+eYgqLtTBJzafziVE5bMqIk1G4iKePOBcX5HGQ==","X-Received":"by 10.37.61.2 with SMTP id k2mr2182608yba.233.1504119199925; Wed,\n\t30 Aug 2017 11:53:19 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<1504083688-48334-4-git-send-email-julien.thierry@arm.com>","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>","From":"Christoffer Dall ","Date":"Wed, 30 Aug 2017 20:53:19 +0200","Message-ID":"","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"Julien Thierry ","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170830_115341_569053_53C2FF15 ","X-CRM114-Status":"GOOD ( 21.06 )","X-Spam-Score":"-2.7 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/,\n\tlow\n\ttrust [2607:f8b0:4002:c05:0:0:0:22f listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier , Catalin Marinas\n\t, Will Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, =?utf-8?q?Alex_Benn=C3=A9e?=\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1760752,"web_url":"http://patchwork.ozlabs.org/comment/1760752/","msgid":"<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>","list_archive_url":null,"date":"2017-08-31T08:45:36","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":72256,"url":"http://patchwork.ozlabs.org/api/people/72256/","name":"Julien Thierry","email":"julien.thierry@arm.com"},"content":"Hi Christoffer,\n\nOn 30/08/17 19:53, Christoffer Dall wrote:\n> Hi Julien,\n> \n> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n> \n> On Wed, Aug 30, 2017 at 11:01 AM, Julien Thierry wrote:\n>> Software Step exception is missing after trapping instruction from the guest.\n>>\n>> We need to set the PSR.SS to 0 for the guest vcpu before resuming guest\n>> execution.\n>>\n>> Signed-off-by: Julien Thierry \n>> Cc: Christoffer Dall \n>> Cc: Marc Zyngier \n>> Cc: Catalin Marinas \n>> Cc: Will Deacon \n>>\n>> ---\n>> arch/arm64/include/asm/kvm_asm.h | 2 ++\n>> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n>> arch/arm64/kvm/debug.c | 12 +++++++++++-\n>> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n>> 4 files changed, 27 insertions(+), 1 deletion(-)\n>>\n>> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h\n>> index 26a64d0..398bbaa 100644\n>> --- a/arch/arm64/include/asm/kvm_asm.h\n>> +++ b/arch/arm64/include/asm/kvm_asm.h\n>> @@ -32,6 +32,8 @@\n>>\n>> #define KVM_ARM64_DEBUG_DIRTY_SHIFT 0\n>> #define KVM_ARM64_DEBUG_DIRTY (1 << KVM_ARM64_DEBUG_DIRTY_SHIFT)\n>> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT 1\n>> +#define KVM_ARM64_DEBUG_INST_SKIP (1 << KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>>\n>> #define kvm_ksym_ref(sym) \\\n>> ({ \\\n>> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h\n>> index fe39e68..d401c64 100644\n>> --- a/arch/arm64/include/asm/kvm_emulate.h\n>> +++ b/arch/arm64/include/asm/kvm_emulate.h\n>> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu *vcpu, bool is_wide_instr)\n>> kvm_skip_instr32(vcpu, is_wide_instr);\n>> else\n>> *vcpu_pc(vcpu) += 4;\n>> + /* Let debug engine know we skipped an instruction */\n>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n> \n> Why do we need to defer this action until later?\n> \n> Can't we simply do clear DBG_SPSR_SS here?\n> \n\nThat was my first intention, but it turns out that the current state of \nthings (without this patch) is that every time we enter a guest, \nkvm_arm_setup_debug gets called and if single step is requested for the \nguest it will set the flag in the SPSR (ignoring the fact that we \ncleared it). This happens even if we exit the guest because of a data abort.\n\nFor normal single step execution, we do need to reset SPSR.SS to 1 \nbefore running the guest since completion of a step should clear that \nbit before taking a software step exception. So what kvm_arm_setup_debug \ndoes seems correct to me but missed the case for trapped/emulated \ninstructions.\n\nSo even if we just clear DBG_SPSR_SS here, we would still need to tell \nkvm_arm_setup_debug not to change the bit. Or resetting SPSR.SS to 1 for \nnormal single stepping needs to be done before we skip instructions in \nKVM but that doesn't sound right to me...\n\n\n> I think even if the guest kernel is single-stepping guest userspace,\n> you're still safe because you'll take another debug exception from\n> having 'executed' (read: emulated) this instruction in the hypervisor.\n> \n>> }\n>>\n>> static inline void vcpu_set_thumb(struct kvm_vcpu *vcpu)\n>> diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c\n>> index dbadfaf..4806e6b 100644\n>> --- a/arch/arm64/kvm/debug.c\n>> +++ b/arch/arm64/kvm/debug.c\n>> @@ -151,12 +151,22 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)\n>> * debugging the system.\n>> */\n>> if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) {\n>> - *vcpu_cpsr(vcpu) |= DBG_SPSR_SS;\n>> + /*\n>> + * If we skipped an instruction while single stepping,\n>> + * spsr.ss needs to be 0 in order to trigger SS\n>> + * exception on return.\n>> + */\n>> + if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_INST_SKIP))\n>> + *vcpu_cpsr(vcpu) |= DBG_SPSR_SS;\n>> + else\n>> + *vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;\n> \n> Then you wouldn't need this.\n> \n>> vcpu_sys_reg(vcpu, MDSCR_EL1) |= DBG_MDSCR_SS;\n>> } else {\n>> vcpu_sys_reg(vcpu, MDSCR_EL1) &= ~DBG_MDSCR_SS;\n>> }\n>>\n>> + vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_INST_SKIP;\n>> +\n>> trace_kvm_arm_set_dreg32(\"SPSR_EL2\", *vcpu_cpsr(vcpu));\n>>\n>> /*\n>> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c\n>> index 945e79c..6030acd 100644\n>> --- a/arch/arm64/kvm/hyp/switch.c\n>> +++ b/arch/arm64/kvm/hyp/switch.c\n>> @@ -22,6 +22,7 @@\n>> #include \n>> #include \n>> #include \n>> +#include \n>>\n>> static bool __hyp_text __fpsimd_enabled_nvhe(void)\n>> {\n>> @@ -276,6 +277,10 @@ static void __hyp_text __skip_instr(struct kvm_vcpu *vcpu)\n>> }\n>>\n>> write_sysreg_el2(*vcpu_pc(vcpu), elr);\n>> +\n>> + *vcpu_cpsr(vcpu) = read_sysreg_el2(spsr);\n>> + *vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;\n>> + write_sysreg_el2(*vcpu_cpsr(vcpu), spsr);\n> \n> I don't think you really need to use *vcpu_cpsr(vcpu) here, I think\n> you can just use a temporary variable, which may make the flow a bit\n> easier to read actually.\n> \n\nGood point, I'll do that.\n\n>> }\n>>\n>> int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n>> @@ -343,6 +348,13 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n>> if (ret == -1) {\n>> /* Promote an illegal access to an SError */\n>> __skip_instr(vcpu);\n>> +\n>> + /*\n>> + * We're not jumping back, let debug setup know\n>> + * we skipped an instruction.\n>> + */\n>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n> \n> I agree with Marc, I don't think you need this, because you have\n> modified the hardware spsr_el2, which you're going to read back in\n> __sysreg_save_guest_state().\n> \n\nWe are exiting the guest, so it is the same case as kvm_skip_instr.\n\nThanks,","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"mq35DbPh\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xjbZk74v1z9s3w\n\tfor ;\n\tThu, 31 Aug 2017 18:46:38 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnL7T-0004L2-E9; Thu, 31 Aug 2017 08:46:35 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnL6t-0003tv-OU for linux-arm-kernel@lists.infradead.org;\n\tThu, 31 Aug 2017 08:46:01 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3469715A2;\n\tThu, 31 Aug 2017 01:45:39 -0700 (PDT)","from [10.1.207.37] (e112298-lin.cambridge.arm.com [10.1.207.37])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tD753F3F540; Thu, 31 Aug 2017 01:45:37 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=2RRekJ9rzPlXS2vDKxBa9JsdYzOP9C9+QEaPNSulbyA=;\n\tb=mq35DbPhoFRYasQDiWnGeXCIh\n\tGAo4zzH1lwYTcTBqH/SxKwQ0JzIzfyLNl746D1tTo+5OPd0DvJibxoUJZ9GdbBsGV91d8JUTB8odR\n\tUz15ohMxmi8aErCwqQBE31hHDwHEm3NXWKYvq0QGBg0+feWHjbmPIJDKakamNLFKVCYy8wkeYD0KB\n\t+W/0ea/kSE3N4ZRsUiaweousTcAS/IeaxzVdGtSXM/QQ9+kNH9uhLKaz1p2a6E/6jP5D0x6tge29y\n\tmSWjewoBqVLJc1JYAwV+HOLQZGckv/imF4ezmdRNQdaleBTuiHnG6eDFsySeWZXblVqpIJ2vXZO5t\n\tPjfcxRYLg==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"Christoffer Dall ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t","From":"Julien Thierry ","Message-ID":"<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>","Date":"Thu, 31 Aug 2017 09:45:36 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170831_014559_834239_FA1E1E14 ","X-CRM114-Status":"GOOD ( 25.63 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier , Catalin Marinas\n\t, Will Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, =?utf-8?q?Alex_Benn=C3=A9e?=\n\t","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1760756,"web_url":"http://patchwork.ozlabs.org/comment/1760756/","msgid":"","list_archive_url":null,"date":"2017-08-31T08:54:41","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":26352,"url":"http://patchwork.ozlabs.org/api/people/26352/","name":"Christoffer Dall","email":"christoffer.dall@linaro.org"},"content":"On Thu, Aug 31, 2017 at 10:45 AM, Julien Thierry wrote:\n> Hi Christoffer,\n>\n>\n> On 30/08/17 19:53, Christoffer Dall wrote:\n>>\n>> Hi Julien,\n>>\n>> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n>>\n>> On Wed, Aug 30, 2017 at 11:01 AM, Julien Thierry \n>> wrote:\n>>>\n>>> Software Step exception is missing after trapping instruction from the\n>>> guest.\n>>>\n>>> We need to set the PSR.SS to 0 for the guest vcpu before resuming guest\n>>> execution.\n>>>\n>>> Signed-off-by: Julien Thierry \n>>> Cc: Christoffer Dall \n>>> Cc: Marc Zyngier \n>>> Cc: Catalin Marinas \n>>> Cc: Will Deacon \n>>>\n>>> ---\n>>> arch/arm64/include/asm/kvm_asm.h | 2 ++\n>>> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n>>> arch/arm64/kvm/debug.c | 12 +++++++++++-\n>>> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n>>> 4 files changed, 27 insertions(+), 1 deletion(-)\n>>>\n>>> diff --git a/arch/arm64/include/asm/kvm_asm.h\n>>> b/arch/arm64/include/asm/kvm_asm.h\n>>> index 26a64d0..398bbaa 100644\n>>> --- a/arch/arm64/include/asm/kvm_asm.h\n>>> +++ b/arch/arm64/include/asm/kvm_asm.h\n>>> @@ -32,6 +32,8 @@\n>>>\n>>> #define KVM_ARM64_DEBUG_DIRTY_SHIFT 0\n>>> #define KVM_ARM64_DEBUG_DIRTY (1 <<\n>>> KVM_ARM64_DEBUG_DIRTY_SHIFT)\n>>> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT 1\n>>> +#define KVM_ARM64_DEBUG_INST_SKIP (1 <<\n>>> KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>>>\n>>> #define kvm_ksym_ref(sym)\n>>> \\\n>>> ({\n>>> \\\n>>> diff --git a/arch/arm64/include/asm/kvm_emulate.h\n>>> b/arch/arm64/include/asm/kvm_emulate.h\n>>> index fe39e68..d401c64 100644\n>>> --- a/arch/arm64/include/asm/kvm_emulate.h\n>>> +++ b/arch/arm64/include/asm/kvm_emulate.h\n>>> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu\n>>> *vcpu, bool is_wide_instr)\n>>> kvm_skip_instr32(vcpu, is_wide_instr);\n>>> else\n>>> *vcpu_pc(vcpu) += 4;\n>>> + /* Let debug engine know we skipped an instruction */\n>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n>>\n>>\n>> Why do we need to defer this action until later?\n>>\n>> Can't we simply do clear DBG_SPSR_SS here?\n>>\n>\n> That was my first intention, but it turns out that the current state of\n> things (without this patch) is that every time we enter a guest,\n> kvm_arm_setup_debug gets called and if single step is requested for the\n> guest it will set the flag in the SPSR (ignoring the fact that we cleared\n> it).\n\nAh, right, duh.\n\n> This happens even if we exit the guest because of a data abort.\n>\n> For normal single step execution, we do need to reset SPSR.SS to 1 before\n> running the guest since completion of a step should clear that bit before\n> taking a software step exception. So what kvm_arm_setup_debug does seems\n> correct to me but missed the case for trapped/emulated instructions.\n>\n> So even if we just clear DBG_SPSR_SS here, we would still need to tell\n> kvm_arm_setup_debug not to change the bit. Or resetting SPSR.SS to 1 for\n> normal single stepping needs to be done before we skip instructions in KVM\n> but that doesn't sound right to me...\n>\n\nSo I'm wondering if we're going about this wrong. Perhaps we need to\ndiscover at the end of the run loop that we were asked to single step\nexecution and simply return to userspace, setting the debug exit\nreason etc., instead of entering the guest with PSTATE.SS==0 and\nrelying on another trap back in to the guest just to set two fields on\nthe kvm_run structure and exit to user space ?\n\n>\n>\n>> I think even if the guest kernel is single-stepping guest userspace,\n>> you're still safe because you'll take another debug exception from\n>> having 'executed' (read: emulated) this instruction in the hypervisor.\n>>\n>>> }\n>>>\n>>> static inline void vcpu_set_thumb(struct kvm_vcpu *vcpu)\n>>> diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c\n>>> index dbadfaf..4806e6b 100644\n>>> --- a/arch/arm64/kvm/debug.c\n>>> +++ b/arch/arm64/kvm/debug.c\n>>> @@ -151,12 +151,22 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)\n>>> * debugging the system.\n>>> */\n>>> if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) {\n>>> - *vcpu_cpsr(vcpu) |= DBG_SPSR_SS;\n>>> + /*\n>>> + * If we skipped an instruction while single\n>>> stepping,\n>>> + * spsr.ss needs to be 0 in order to trigger SS\n>>> + * exception on return.\n>>> + */\n>>> + if (!(vcpu->arch.debug_flags &\n>>> KVM_ARM64_DEBUG_INST_SKIP))\n>>> + *vcpu_cpsr(vcpu) |= DBG_SPSR_SS;\n>>> + else\n>>> + *vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;\n>>\n>>\n>> Then you wouldn't need this.\n>>\n>>> vcpu_sys_reg(vcpu, MDSCR_EL1) |= DBG_MDSCR_SS;\n>>> } else {\n>>> vcpu_sys_reg(vcpu, MDSCR_EL1) &= ~DBG_MDSCR_SS;\n>>> }\n>>>\n>>> + vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_INST_SKIP;\n>>> +\n>>> trace_kvm_arm_set_dreg32(\"SPSR_EL2\", *vcpu_cpsr(vcpu));\n>>>\n>>> /*\n>>> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c\n>>> index 945e79c..6030acd 100644\n>>> --- a/arch/arm64/kvm/hyp/switch.c\n>>> +++ b/arch/arm64/kvm/hyp/switch.c\n>>> @@ -22,6 +22,7 @@\n>>> #include \n>>> #include \n>>> #include \n>>> +#include \n>>>\n>>> static bool __hyp_text __fpsimd_enabled_nvhe(void)\n>>> {\n>>> @@ -276,6 +277,10 @@ static void __hyp_text __skip_instr(struct kvm_vcpu\n>>> *vcpu)\n>>> }\n>>>\n>>> write_sysreg_el2(*vcpu_pc(vcpu), elr);\n>>> +\n>>> + *vcpu_cpsr(vcpu) = read_sysreg_el2(spsr);\n>>> + *vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;\n>>> + write_sysreg_el2(*vcpu_cpsr(vcpu), spsr);\n>>\n>>\n>> I don't think you really need to use *vcpu_cpsr(vcpu) here, I think\n>> you can just use a temporary variable, which may make the flow a bit\n>> easier to read actually.\n>>\n>\n> Good point, I'll do that.\n>\n>>> }\n>>>\n>>> int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n>>> @@ -343,6 +348,13 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n>>> if (ret == -1) {\n>>> /* Promote an illegal access to an\n>>> SError */\n>>> __skip_instr(vcpu);\n>>> +\n>>> + /*\n>>> + * We're not jumping back, let debug\n>>> setup know\n>>> + * we skipped an instruction.\n>>> + */\n>>> + vcpu->arch.debug_flags |=\n>>> KVM_ARM64_DEBUG_INST_SKIP;\n>>\n>>\n>> I agree with Marc, I don't think you need this, because you have\n>> modified the hardware spsr_el2, which you're going to read back in\n>> __sysreg_save_guest_state().\n>>\n>\n> We are exiting the guest, so it is the same case as kvm_skip_instr.\n>\nok, I see this now, you're right. Thanks for explaining it. For some\nreason, this code is just really hard to understand, so I wonder if\nthe suggested approach above is better.\n\nThanks,\n-Christoffer","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"mCoWvzBj\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"QgIV/8ps\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xjbmf63Zwz9sRW\n\tfor ;\n\tThu, 31 Aug 2017 18:55:14 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnLFm-0007NW-FB; Thu, 31 Aug 2017 08:55:10 +0000","from mail-yw0-x22d.google.com ([2607:f8b0:4002:c05::22d])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnLFf-0006h7-FV for linux-arm-kernel@lists.infradead.org;\n\tThu, 31 Aug 2017 08:55:05 +0000","by mail-yw0-x22d.google.com with SMTP id s143so381993ywg.0\n\tfor ;\n\tThu, 31 Aug 2017 01:54:42 -0700 (PDT)","by 10.37.97.23 with HTTP; Thu, 31 Aug 2017 01:54:41 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:\n\tReferences:In-Reply-To:MIME-Version:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=GojckNBWVKOvmvuIR/oSIXgbt77/5S+4FFx4xvN8VPk=;\n\tb=mCoWvzBjd6r+4L\n\tiFOinzgvFTI+j3eWVsLuRhV639WWT01DLQ5oo7A7oj5VBoZh9HKTh1wGQdUYuKQqirfCh8fxDNxsG\n\tAJexiyodEroXqvKkHUJiz1b0PFqL13zu+j2B+qky6GkS9kIXA7O18zVxfuIqG3eJbkr9axx7LDmwn\n\tol9dB4nc5KXjM9ah9lQI201SM6pfQuJrkeJznqBuF2/vjtOkwsajcgalAFwFjjpHdnPA15ukY0t9/\n\tl+Rt8Ds/rHxOd5tZ62kJynskjl3Nha8EM2AnS1MR1e7C7CSrCd1hlZeLyxRTYMSH/LMID9wNZkmAB\n\tx9xNhXehyCy3PfUVw5GQ==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc:content-transfer-encoding;\n\tbh=asDjwC1A7CIV/5mihSNDoM5/5k+MnBCcNP2kzquEl7o=;\n\tb=QgIV/8psMAciKAa3HUuoyjidWBT3Cpwyq/GoTNP4c3fG6n9MNqLLBeIhhglVruHL8Y\n\tMujh3Ds1cBjJb3rhnm68tGkubXfhzWMcVbkX/fODzbCqRmPz4rY9bi3WU/RggDB9VLQ6\n\tWrOENOVGqZF+qKknpzdSx+tE+2fyQ0rLzCqOI="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc:content-transfer-encoding;\n\tbh=asDjwC1A7CIV/5mihSNDoM5/5k+MnBCcNP2kzquEl7o=;\n\tb=BNjmTBUhcAjajS/80H2vWld3knnzSZFTIiMOsw2fx+oYoHjwbav74i7m/Q77oI1x8l\n\tp7sfVtLyZeZAMv+TEVeWlnWNUGDFonkq9eSv9BUspOApSPAELSa0DrjBJi/9t8XtQDE+\n\thjBX4SaBqxDzmzqkvjk6lwjfonTbI3VlGuZdPEnMSd+9FxI7YXdzNlUX90rl/lL3hUIZ\n\tGfUnn1Vuhne6mbwIKdp1z+2krhAfz+gMcQ7RwIxZTyb5EeZiIZLdzs4KzG57JT7sB9f5\n\tMyr+UZ3Q7NcYWy6Vji8bUBxaFdNCE2UFTkETUsZdWVZqvVOtB6E2/5UtOmnkT08zw4i0\n\tAAvg==","X-Gm-Message-State":"AHYfb5hir4ZV4JUanzZeX34N15mtWdNthxy7Zt179CFH6p2h+RuUKzOT\n\tsVO6kQjQxDZu0P+M8+WTWUe69OHUE7qO","X-Received":"by 10.37.41.133 with SMTP id p127mr3659026ybp.329.1504169681912; \n\tThu, 31 Aug 2017 01:54:41 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>","From":"Christoffer Dall ","Date":"Thu, 31 Aug 2017 10:54:41 +0200","Message-ID":"","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"Julien Thierry ","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170831_015503_757811_C5993D02 ","X-CRM114-Status":"GOOD ( 30.85 )","X-Spam-Score":"-2.7 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/,\n\tlow\n\ttrust [2607:f8b0:4002:c05:0:0:0:22d listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier , Catalin Marinas\n\t, Will Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, =?utf-8?q?Alex_Benn=C3=A9e?=\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1760787,"web_url":"http://patchwork.ozlabs.org/comment/1760787/","msgid":"","list_archive_url":null,"date":"2017-08-31T09:37:18","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":72256,"url":"http://patchwork.ozlabs.org/api/people/72256/","name":"Julien Thierry","email":"julien.thierry@arm.com"},"content":"On 31/08/17 09:54, Christoffer Dall wrote:\n> On Thu, Aug 31, 2017 at 10:45 AM, Julien Thierry wrote:\n>> Hi Christoffer,\n>>\n>>\n>> On 30/08/17 19:53, Christoffer Dall wrote:\n>>>\n>>> Hi Julien,\n>>>\n>>> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n>>>\n>>> On Wed, Aug 30, 2017 at 11:01 AM, Julien Thierry \n>>> wrote:\n>>>>\n>>>> Software Step exception is missing after trapping instruction from the\n>>>> guest.\n>>>>\n>>>> We need to set the PSR.SS to 0 for the guest vcpu before resuming guest\n>>>> execution.\n>>>>\n>>>> Signed-off-by: Julien Thierry \n>>>> Cc: Christoffer Dall \n>>>> Cc: Marc Zyngier \n>>>> Cc: Catalin Marinas \n>>>> Cc: Will Deacon \n>>>>\n>>>> ---\n>>>> arch/arm64/include/asm/kvm_asm.h | 2 ++\n>>>> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n>>>> arch/arm64/kvm/debug.c | 12 +++++++++++-\n>>>> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n>>>> 4 files changed, 27 insertions(+), 1 deletion(-)\n>>>>\n>>>> diff --git a/arch/arm64/include/asm/kvm_asm.h\n>>>> b/arch/arm64/include/asm/kvm_asm.h\n>>>> index 26a64d0..398bbaa 100644\n>>>> --- a/arch/arm64/include/asm/kvm_asm.h\n>>>> +++ b/arch/arm64/include/asm/kvm_asm.h\n>>>> @@ -32,6 +32,8 @@\n>>>>\n>>>> #define KVM_ARM64_DEBUG_DIRTY_SHIFT 0\n>>>> #define KVM_ARM64_DEBUG_DIRTY (1 <<\n>>>> KVM_ARM64_DEBUG_DIRTY_SHIFT)\n>>>> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT 1\n>>>> +#define KVM_ARM64_DEBUG_INST_SKIP (1 <<\n>>>> KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>>>>\n>>>> #define kvm_ksym_ref(sym)\n>>>> \\\n>>>> ({\n>>>> \\\n>>>> diff --git a/arch/arm64/include/asm/kvm_emulate.h\n>>>> b/arch/arm64/include/asm/kvm_emulate.h\n>>>> index fe39e68..d401c64 100644\n>>>> --- a/arch/arm64/include/asm/kvm_emulate.h\n>>>> +++ b/arch/arm64/include/asm/kvm_emulate.h\n>>>> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu\n>>>> *vcpu, bool is_wide_instr)\n>>>> kvm_skip_instr32(vcpu, is_wide_instr);\n>>>> else\n>>>> *vcpu_pc(vcpu) += 4;\n>>>> + /* Let debug engine know we skipped an instruction */\n>>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n>>>\n>>>\n>>> Why do we need to defer this action until later?\n>>>\n>>> Can't we simply do clear DBG_SPSR_SS here?\n>>>\n>>\n>> That was my first intention, but it turns out that the current state of\n>> things (without this patch) is that every time we enter a guest,\n>> kvm_arm_setup_debug gets called and if single step is requested for the\n>> guest it will set the flag in the SPSR (ignoring the fact that we cleared\n>> it).\n> \n> Ah, right, duh.\n> \n>> This happens even if we exit the guest because of a data abort.\n>>\n>> For normal single step execution, we do need to reset SPSR.SS to 1 before\n>> running the guest since completion of a step should clear that bit before\n>> taking a software step exception. So what kvm_arm_setup_debug does seems\n>> correct to me but missed the case for trapped/emulated instructions.\n>>\n>> So even if we just clear DBG_SPSR_SS here, we would still need to tell\n>> kvm_arm_setup_debug not to change the bit. Or resetting SPSR.SS to 1 for\n>> normal single stepping needs to be done before we skip instructions in KVM\n>> but that doesn't sound right to me...\n>>\n> \n> So I'm wondering if we're going about this wrong. Perhaps we need to\n> discover at the end of the run loop that we were asked to single step\n> execution and simply return to userspace, setting the debug exit\n> reason etc., instead of entering the guest with PSTATE.SS==0 and\n> relying on another trap back in to the guest just to set two fields on\n> the kvm_run structure and exit to user space ?\n> \n\nSo if I understand correctly, the suggestion is that when we trap an \ninstruction we check whether it was supposed to be single stepped, if it \nwas we set up the vcpu registers as if it had taken a software step \nexception and return from kvm_arch_vcpu_ioctl_run. Is that right?\n\nInteresting idea, I can try to explore that possibility.\n\nThanks for the suggestion,\n\n>>\n>>\n>>> I think even if the guest kernel is single-stepping guest userspace,\n>>> you're still safe because you'll take another debug exception from\n>>> having 'executed' (read: emulated) this instruction in the hypervisor.\n>>>\n>>>> }\n>>>>\n>>>> static inline void vcpu_set_thumb(struct kvm_vcpu *vcpu)\n>>>> diff --git a/arch/arm64/kvm/debug.c b/arch/arm64/kvm/debug.c\n>>>> index dbadfaf..4806e6b 100644\n>>>> --- a/arch/arm64/kvm/debug.c\n>>>> +++ b/arch/arm64/kvm/debug.c\n>>>> @@ -151,12 +151,22 @@ void kvm_arm_setup_debug(struct kvm_vcpu *vcpu)\n>>>> * debugging the system.\n>>>> */\n>>>> if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) {\n>>>> - *vcpu_cpsr(vcpu) |= DBG_SPSR_SS;\n>>>> + /*\n>>>> + * If we skipped an instruction while single\n>>>> stepping,\n>>>> + * spsr.ss needs to be 0 in order to trigger SS\n>>>> + * exception on return.\n>>>> + */\n>>>> + if (!(vcpu->arch.debug_flags &\n>>>> KVM_ARM64_DEBUG_INST_SKIP))\n>>>> + *vcpu_cpsr(vcpu) |= DBG_SPSR_SS;\n>>>> + else\n>>>> + *vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;\n>>>\n>>>\n>>> Then you wouldn't need this.\n>>>\n>>>> vcpu_sys_reg(vcpu, MDSCR_EL1) |= DBG_MDSCR_SS;\n>>>> } else {\n>>>> vcpu_sys_reg(vcpu, MDSCR_EL1) &= ~DBG_MDSCR_SS;\n>>>> }\n>>>>\n>>>> + vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_INST_SKIP;\n>>>> +\n>>>> trace_kvm_arm_set_dreg32(\"SPSR_EL2\", *vcpu_cpsr(vcpu));\n>>>>\n>>>> /*\n>>>> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c\n>>>> index 945e79c..6030acd 100644\n>>>> --- a/arch/arm64/kvm/hyp/switch.c\n>>>> +++ b/arch/arm64/kvm/hyp/switch.c\n>>>> @@ -22,6 +22,7 @@\n>>>> #include \n>>>> #include \n>>>> #include \n>>>> +#include \n>>>>\n>>>> static bool __hyp_text __fpsimd_enabled_nvhe(void)\n>>>> {\n>>>> @@ -276,6 +277,10 @@ static void __hyp_text __skip_instr(struct kvm_vcpu\n>>>> *vcpu)\n>>>> }\n>>>>\n>>>> write_sysreg_el2(*vcpu_pc(vcpu), elr);\n>>>> +\n>>>> + *vcpu_cpsr(vcpu) = read_sysreg_el2(spsr);\n>>>> + *vcpu_cpsr(vcpu) &= ~DBG_SPSR_SS;\n>>>> + write_sysreg_el2(*vcpu_cpsr(vcpu), spsr);\n>>>\n>>>\n>>> I don't think you really need to use *vcpu_cpsr(vcpu) here, I think\n>>> you can just use a temporary variable, which may make the flow a bit\n>>> easier to read actually.\n>>>\n>>\n>> Good point, I'll do that.\n>>\n>>>> }\n>>>>\n>>>> int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n>>>> @@ -343,6 +348,13 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n>>>> if (ret == -1) {\n>>>> /* Promote an illegal access to an\n>>>> SError */\n>>>> __skip_instr(vcpu);\n>>>> +\n>>>> + /*\n>>>> + * We're not jumping back, let debug\n>>>> setup know\n>>>> + * we skipped an instruction.\n>>>> + */\n>>>> + vcpu->arch.debug_flags |=\n>>>> KVM_ARM64_DEBUG_INST_SKIP;\n>>>\n>>>\n>>> I agree with Marc, I don't think you need this, because you have\n>>> modified the hardware spsr_el2, which you're going to read back in\n>>> __sysreg_save_guest_state().\n>>>\n>>\n>> We are exiting the guest, so it is the same case as kvm_skip_instr.\n>>\n> ok, I see this now, you're right. Thanks for explaining it. For some\n> reason, this code is just really hard to understand, so I wonder if\n> the suggested approach above is better.\n> \n> Thanks,\n> -Christoffer\n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"gdJJvpaY\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xjcjs4qxVz9s83\n\tfor ;\n\tThu, 31 Aug 2017 19:37:49 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnLuz-0003oO-WB; Thu, 31 Aug 2017 09:37:46 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnLuw-0003mG-0j for linux-arm-kernel@lists.infradead.org;\n\tThu, 31 Aug 2017 09:37:43 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D661980D;\n\tThu, 31 Aug 2017 02:37:21 -0700 (PDT)","from [10.1.207.37] (e112298-lin.cambridge.arm.com [10.1.207.37])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tB01FE3F540; Thu, 31 Aug 2017 02:37:20 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=3l3cOw/h5JNAwIFklm6eSc0q61s+YEYfD7PTfrmPW2E=;\n\tb=gdJJvpaYUwlvbcUHIVkB5t2Pm\n\t9RSepCCdxU/scH7kJvOLndL/ismX82YdZw9rvc43+7e1va+aM1AOYjcZqQyufHbAxDqZPpAzmPmjd\n\tASSG43pHH3VevYqpqlNdpJgqogewRsNywNzx1WVtl6K96MCD1za3/pZXlJqZBRIkptnbg4pn0CETu\n\tWq4RY5cgwk7fCuE8c3KuMta9vrrYfzExpawAb3Jvex9rWgWcJ+/eONZna2EMWYJX9w8lsk11Tqd8t\n\tg871/vjmooFfoGwDvK4iBf0ysiuNKU7/G2spQi2lR6qjCT7LMz31tEry+B5m3Z+7Zb54mqGeNrGns\n\ta2GmaPbLA==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"Christoffer Dall ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t","From":"Julien Thierry ","Message-ID":"","Date":"Thu, 31 Aug 2017 10:37:18 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170831_023742_086290_34E1DDC7 ","X-CRM114-Status":"GOOD ( 30.38 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier , Catalin Marinas\n\t, Will Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, =?utf-8?q?Alex_Benn=C3=A9e?=\n\t","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1760826,"web_url":"http://patchwork.ozlabs.org/comment/1760826/","msgid":"","list_archive_url":null,"date":"2017-08-31T10:53:39","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":26352,"url":"http://patchwork.ozlabs.org/api/people/26352/","name":"Christoffer Dall","email":"christoffer.dall@linaro.org"},"content":"On Thu, Aug 31, 2017 at 11:37 AM, Julien Thierry wrote:\n>\n>\n> On 31/08/17 09:54, Christoffer Dall wrote:\n>>\n>> On Thu, Aug 31, 2017 at 10:45 AM, Julien Thierry \n>> wrote:\n>>>\n>>> Hi Christoffer,\n>>>\n>>>\n>>> On 30/08/17 19:53, Christoffer Dall wrote:\n>>>>\n>>>>\n>>>> Hi Julien,\n>>>>\n>>>> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n>>>>\n>>>> On Wed, Aug 30, 2017 at 11:01 AM, Julien Thierry\n>>>> \n>>>> wrote:\n>>>>>\n>>>>>\n>>>>> Software Step exception is missing after trapping instruction from the\n>>>>> guest.\n>>>>>\n>>>>> We need to set the PSR.SS to 0 for the guest vcpu before resuming guest\n>>>>> execution.\n>>>>>\n>>>>> Signed-off-by: Julien Thierry \n>>>>> Cc: Christoffer Dall \n>>>>> Cc: Marc Zyngier \n>>>>> Cc: Catalin Marinas \n>>>>> Cc: Will Deacon \n>>>>>\n>>>>> ---\n>>>>> arch/arm64/include/asm/kvm_asm.h | 2 ++\n>>>>> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n>>>>> arch/arm64/kvm/debug.c | 12 +++++++++++-\n>>>>> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n>>>>> 4 files changed, 27 insertions(+), 1 deletion(-)\n>>>>>\n>>>>> diff --git a/arch/arm64/include/asm/kvm_asm.h\n>>>>> b/arch/arm64/include/asm/kvm_asm.h\n>>>>> index 26a64d0..398bbaa 100644\n>>>>> --- a/arch/arm64/include/asm/kvm_asm.h\n>>>>> +++ b/arch/arm64/include/asm/kvm_asm.h\n>>>>> @@ -32,6 +32,8 @@\n>>>>>\n>>>>> #define KVM_ARM64_DEBUG_DIRTY_SHIFT 0\n>>>>> #define KVM_ARM64_DEBUG_DIRTY (1 <<\n>>>>> KVM_ARM64_DEBUG_DIRTY_SHIFT)\n>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT 1\n>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP (1 <<\n>>>>> KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>>>>>\n>>>>> #define kvm_ksym_ref(sym)\n>>>>> \\\n>>>>> ({\n>>>>> \\\n>>>>> diff --git a/arch/arm64/include/asm/kvm_emulate.h\n>>>>> b/arch/arm64/include/asm/kvm_emulate.h\n>>>>> index fe39e68..d401c64 100644\n>>>>> --- a/arch/arm64/include/asm/kvm_emulate.h\n>>>>> +++ b/arch/arm64/include/asm/kvm_emulate.h\n>>>>> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu\n>>>>> *vcpu, bool is_wide_instr)\n>>>>> kvm_skip_instr32(vcpu, is_wide_instr);\n>>>>> else\n>>>>> *vcpu_pc(vcpu) += 4;\n>>>>> + /* Let debug engine know we skipped an instruction */\n>>>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n>>>>\n>>>>\n>>>>\n>>>> Why do we need to defer this action until later?\n>>>>\n>>>> Can't we simply do clear DBG_SPSR_SS here?\n>>>>\n>>>\n>>> That was my first intention, but it turns out that the current state of\n>>> things (without this patch) is that every time we enter a guest,\n>>> kvm_arm_setup_debug gets called and if single step is requested for the\n>>> guest it will set the flag in the SPSR (ignoring the fact that we cleared\n>>> it).\n>>\n>>\n>> Ah, right, duh.\n>>\n>>> This happens even if we exit the guest because of a data abort.\n>>>\n>>> For normal single step execution, we do need to reset SPSR.SS to 1 before\n>>> running the guest since completion of a step should clear that bit before\n>>> taking a software step exception. So what kvm_arm_setup_debug does seems\n>>> correct to me but missed the case for trapped/emulated instructions.\n>>>\n>>> So even if we just clear DBG_SPSR_SS here, we would still need to tell\n>>> kvm_arm_setup_debug not to change the bit. Or resetting SPSR.SS to 1 for\n>>> normal single stepping needs to be done before we skip instructions in\n>>> KVM\n>>> but that doesn't sound right to me...\n>>>\n>>\n>> So I'm wondering if we're going about this wrong. Perhaps we need to\n>> discover at the end of the run loop that we were asked to single step\n>> execution and simply return to userspace, setting the debug exit\n>> reason etc., instead of entering the guest with PSTATE.SS==0 and\n>> relying on another trap back in to the guest just to set two fields on\n>> the kvm_run structure and exit to user space ?\n>>\n>\n> So if I understand correctly, the suggestion is that when we trap an\n> instruction we check whether it was supposed to be single stepped, if it was\n> we set up the vcpu registers as if it had taken a software step exception\n> and return from kvm_arch_vcpu_ioctl_run. Is that right?\n\nyes, that's the idea. If there's a lot of complexity in setting up\nCPU register state, then it may not be a good idea, but if it's\nrelatively clean, I think it can be preferred over the \"let's keep a\nflag aroudn for later\" approach.\n\n>\n> Interesting idea, I can try to explore that possibility.\n>\n> Thanks for the suggestion,\n>\nThanks!\n-Christoffer","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"Ikffw0fX\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"gn900wik\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xjfQJ6HY7z9s83\n\tfor ;\n\tThu, 31 Aug 2017 20:54:32 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnN7F-0005SP-Q4; Thu, 31 Aug 2017 10:54:29 +0000","from mail-yw0-x22a.google.com ([2607:f8b0:4002:c05::22a])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnN6m-0004I1-TG for linux-arm-kernel@lists.infradead.org;\n\tThu, 31 Aug 2017 10:54:08 +0000","by mail-yw0-x22a.google.com with SMTP id s187so1430654ywf.2\n\tfor ;\n\tThu, 31 Aug 2017 03:53:40 -0700 (PDT)","by 10.37.97.23 with HTTP; Thu, 31 Aug 2017 03:53:39 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:\n\tReferences:In-Reply-To:MIME-Version:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=uFIyzb03D/1pfDmjkjTc4rjsTNkWLyh0BfisC54RNXM=;\n\tb=Ikffw0fXG46YKu\n\tWx3tTrvVtVVNKFaffZHrAp4jmBH2hY8qmBCJoUmHxcouT9oPEZLDKeuYDYWCrpyFV0p2MbrYezJ6U\n\tFHQFXolXGKR0MI5ywbfJ5sGLwP/1GbGeDXxT9yOTZwmULuYuMr1foHaI8IlE/0Y0Uy0bwSJ8A1qHd\n\tlucDy1+XRSyZNZfmF+93yDd20QbDjkoNT2DV0GbriaPq9lQqH0NbioYnfca+gQ7pCtVN85cNj+grk\n\tqgyIPLKpASvN5mtFslnN2lIAfCVajM3+j6P2LK+IrbO0+jBpJRg41YbGFLLrsLTY1u7OXklAPNpWv\n\tN1hOLQKNlXsmjjugvfCA==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc:content-transfer-encoding;\n\tbh=E7OzVyKCemq18VqwcKQJUb/Z6KFwIk3hhVwfy4K64OU=;\n\tb=gn900wikfuiZojJUsCH22T+Mx6q5XTuE/V6N23jgYHLCDsjYHV8877AVmlrcEz9IHV\n\tvJ11yH6yYMwHNwI5lpdFgLojgk1RaKnKV73In60xQ/sLQz0Wh7KTdQ9ljnI2NjfK0dog\n\tpstSZhgeWi86qrRintQ0Z3pAa3syEtzpiCb+4="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc:content-transfer-encoding;\n\tbh=E7OzVyKCemq18VqwcKQJUb/Z6KFwIk3hhVwfy4K64OU=;\n\tb=cEzRroVRJsdmt5ew24NYnE4IVH9aYQrxGSaRRB+mzkJ7vvVuzIqLKENNZ48f3TgaE6\n\t52zLs/7N6MN8077hTVoTcu8/riqJpoj5zTUvpwL8XnVzsh/ZZoEMCB4T5ejmxE8nd9Fm\n\tG1ckL8g5te7QSoJFTeFo4xpIbqSop1cdvAN3kS3WWpHArzPknYwkR37HchFVp3Ygmv89\n\tXwp0JqGybyhdN6dpIiTj3/cFV3gfdpmhsTzdKQ+K3SXif7iGPPjLhg9MM4/S7Ryp78HJ\n\tX9qyYCrjJpjLqSNuPQwYtvssgBv1tO8zuMxrp10U+VtSBPX5HXkOsurcfFgC5yRJ/ktR\n\tulNg==","X-Gm-Message-State":"AHYfb5h6Y1aYZShNo1nwRxWs5rNgdgevOzhcJgER5t/mSofCjhpUTN1h\n\tgCbAlpHnpNADehROoiUF4VIjgszBU9XA","X-Received":"by 10.13.217.205 with SMTP id b196mr4218380ywe.369.1504176819627;\n\tThu, 31 Aug 2017 03:53:39 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t","From":"Christoffer Dall ","Date":"Thu, 31 Aug 2017 12:53:39 +0200","Message-ID":"","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"Julien Thierry ","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170831_035401_606554_1978F2A5 ","X-CRM114-Status":"GOOD ( 22.70 )","X-Spam-Score":"-2.7 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/,\n\tlow\n\ttrust [2607:f8b0:4002:c05:0:0:0:22a listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier , Catalin Marinas\n\t, Will Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, =?utf-8?q?Alex_Benn=C3=A9e?=\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1760944,"web_url":"http://patchwork.ozlabs.org/comment/1760944/","msgid":"<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>","list_archive_url":null,"date":"2017-08-31T12:56:58","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":72256,"url":"http://patchwork.ozlabs.org/api/people/72256/","name":"Julien Thierry","email":"julien.thierry@arm.com"},"content":"On 31/08/17 11:53, Christoffer Dall wrote:\n> On Thu, Aug 31, 2017 at 11:37 AM, Julien Thierry wrote:\n>>\n>>\n>> On 31/08/17 09:54, Christoffer Dall wrote:\n>>>\n>>> On Thu, Aug 31, 2017 at 10:45 AM, Julien Thierry \n>>> wrote:\n>>>>\n>>>> Hi Christoffer,\n>>>>\n>>>>\n>>>> On 30/08/17 19:53, Christoffer Dall wrote:\n>>>>>\n>>>>>\n>>>>> Hi Julien,\n>>>>>\n>>>>> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n>>>>>\n>>>>> On Wed, Aug 30, 2017 at 11:01 AM, Julien Thierry\n>>>>> \n>>>>> wrote:\n>>>>>>\n>>>>>>\n>>>>>> Software Step exception is missing after trapping instruction from the\n>>>>>> guest.\n>>>>>>\n>>>>>> We need to set the PSR.SS to 0 for the guest vcpu before resuming guest\n>>>>>> execution.\n>>>>>>\n>>>>>> Signed-off-by: Julien Thierry \n>>>>>> Cc: Christoffer Dall \n>>>>>> Cc: Marc Zyngier \n>>>>>> Cc: Catalin Marinas \n>>>>>> Cc: Will Deacon \n>>>>>>\n>>>>>> ---\n>>>>>> arch/arm64/include/asm/kvm_asm.h | 2 ++\n>>>>>> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n>>>>>> arch/arm64/kvm/debug.c | 12 +++++++++++-\n>>>>>> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n>>>>>> 4 files changed, 27 insertions(+), 1 deletion(-)\n>>>>>>\n>>>>>> diff --git a/arch/arm64/include/asm/kvm_asm.h\n>>>>>> b/arch/arm64/include/asm/kvm_asm.h\n>>>>>> index 26a64d0..398bbaa 100644\n>>>>>> --- a/arch/arm64/include/asm/kvm_asm.h\n>>>>>> +++ b/arch/arm64/include/asm/kvm_asm.h\n>>>>>> @@ -32,6 +32,8 @@\n>>>>>>\n>>>>>> #define KVM_ARM64_DEBUG_DIRTY_SHIFT 0\n>>>>>> #define KVM_ARM64_DEBUG_DIRTY (1 <<\n>>>>>> KVM_ARM64_DEBUG_DIRTY_SHIFT)\n>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT 1\n>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP (1 <<\n>>>>>> KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>>>>>>\n>>>>>> #define kvm_ksym_ref(sym)\n>>>>>> \\\n>>>>>> ({\n>>>>>> \\\n>>>>>> diff --git a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>> b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>> index fe39e68..d401c64 100644\n>>>>>> --- a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>> +++ b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu\n>>>>>> *vcpu, bool is_wide_instr)\n>>>>>> kvm_skip_instr32(vcpu, is_wide_instr);\n>>>>>> else\n>>>>>> *vcpu_pc(vcpu) += 4;\n>>>>>> + /* Let debug engine know we skipped an instruction */\n>>>>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n>>>>>\n>>>>>\n>>>>>\n>>>>> Why do we need to defer this action until later?\n>>>>>\n>>>>> Can't we simply do clear DBG_SPSR_SS here?\n>>>>>\n>>>>\n>>>> That was my first intention, but it turns out that the current state of\n>>>> things (without this patch) is that every time we enter a guest,\n>>>> kvm_arm_setup_debug gets called and if single step is requested for the\n>>>> guest it will set the flag in the SPSR (ignoring the fact that we cleared\n>>>> it).\n>>>\n>>>\n>>> Ah, right, duh.\n>>>\n>>>> This happens even if we exit the guest because of a data abort.\n>>>>\n>>>> For normal single step execution, we do need to reset SPSR.SS to 1 before\n>>>> running the guest since completion of a step should clear that bit before\n>>>> taking a software step exception. So what kvm_arm_setup_debug does seems\n>>>> correct to me but missed the case for trapped/emulated instructions.\n>>>>\n>>>> So even if we just clear DBG_SPSR_SS here, we would still need to tell\n>>>> kvm_arm_setup_debug not to change the bit. Or resetting SPSR.SS to 1 for\n>>>> normal single stepping needs to be done before we skip instructions in\n>>>> KVM\n>>>> but that doesn't sound right to me...\n>>>>\n>>>\n>>> So I'm wondering if we're going about this wrong. Perhaps we need to\n>>> discover at the end of the run loop that we were asked to single step\n>>> execution and simply return to userspace, setting the debug exit\n>>> reason etc., instead of entering the guest with PSTATE.SS==0 and\n>>> relying on another trap back in to the guest just to set two fields on\n>>> the kvm_run structure and exit to user space ?\n>>>\n>>\n>> So if I understand correctly, the suggestion is that when we trap an\n>> instruction we check whether it was supposed to be single stepped, if it was\n>> we set up the vcpu registers as if it had taken a software step exception\n>> and return from kvm_arch_vcpu_ioctl_run. Is that right?\n> \n> yes, that's the idea. If there's a lot of complexity in setting up\n> CPU register state, then it may not be a good idea, but if it's\n> relatively clean, I think it can be preferred over the \"let's keep a\n> flag aroudn for later\" approach.\n> \n\nSo I looked a bit into it.\n\nOne annoying thing is that the single step mechanic is specific to \narm64. MMU and MMIO code is shared between arm and arm64 and do some \nhandling of traps.\n\nSo cleanest way I can think of doing this would be to clear SPSR.SS in \narm64::kvm_skip_instr, then have some function (e.g. \nkvm_handle/manage_debug_state) at the end of the run loop. For arm, the \nfunction is empty. For arm64, the function, if we are in an active \npending state (SPSR.D == 0 && SPSR.SS == 0 && MDSCR_EL1.SS == 1) and not \nabout to return to userland, returns with a \"fake debug exception\".\n\nSo instead of a flag, we would just use SPSR.SS (or more generally the \nvcpu state) to know if we need to exit with a debug exception or not. \nAnd the kvm_arm_setup_debug would be left untouched (always setting \nSPSR.SS when requested by userland).\n\nDoes that sound like what you had in mind? Or does it seem better than \nthe current patch?\n\nThanks,\n\n>>\n>> Interesting idea, I can try to explore that possibility.\n>>\n>> Thanks for the suggestion,\n>>\n> Thanks!\n> -Christoffer\n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"LTNMxZCR\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xjj8B3vJJz9s7g\n\tfor ;\n\tThu, 31 Aug 2017 22:57:30 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnP2F-0008IY-6d; Thu, 31 Aug 2017 12:57:27 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnP2B-0008HS-3V for linux-arm-kernel@lists.infradead.org;\n\tThu, 31 Aug 2017 12:57:24 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C7B5B2B;\n\tThu, 31 Aug 2017 05:57:01 -0700 (PDT)","from [10.1.207.37] (e112298-lin.cambridge.arm.com [10.1.207.37])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t955603F540; Thu, 31 Aug 2017 05:57:00 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=1NWNOvvUGSeIsyD+yBi27oGsNTlWzEkrg1ngd3C2Wyw=;\n\tb=LTNMxZCRKTR0pqYtjJX4T/AYK\n\tk84aLWTQyhFCaxnGTah3+X1mUIZGiKRRw+BdQ24RpXag5kZSQ684uVLOw1Zg8XpHkkgKjCzgccspm\n\tABodclzbUd+MO/L7RxWBd4VAGBlrnHaskuRJifth18oMt4oyL6luQFyON7yMxMyX91hZAHadNDxvY\n\t26PoWN08oFVEZ2283YZJp2SIQ3X7BeJvXVOGL7Hf6rgmIFnofcwDJX+MHEGlhJpPZtW/p7rnbNxZ/\n\tBlc4/liw74VcjZHSrCRo2Qy6jiY6QQrtuQ59+S4OJeblxjTwzduZj1th10FtSETJCf0tZX+nFqUiy\n\tISfkZDfTQ==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"Christoffer Dall ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t","From":"Julien Thierry ","Message-ID":"<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>","Date":"Thu, 31 Aug 2017 13:56:58 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170831_055723_159598_5159CE5B ","X-CRM114-Status":"GOOD ( 24.85 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier , Catalin Marinas\n\t, Will Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, =?utf-8?q?Alex_Benn=C3=A9e?=\n\t","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1760991,"web_url":"http://patchwork.ozlabs.org/comment/1760991/","msgid":"","list_archive_url":null,"date":"2017-08-31T13:28:51","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":26352,"url":"http://patchwork.ozlabs.org/api/people/26352/","name":"Christoffer Dall","email":"christoffer.dall@linaro.org"},"content":"On Thu, Aug 31, 2017 at 2:56 PM, Julien Thierry wrote:\n>\n>\n> On 31/08/17 11:53, Christoffer Dall wrote:\n>>\n>> On Thu, Aug 31, 2017 at 11:37 AM, Julien Thierry \n>> wrote:\n>>>\n>>>\n>>>\n>>> On 31/08/17 09:54, Christoffer Dall wrote:\n>>>>\n>>>>\n>>>> On Thu, Aug 31, 2017 at 10:45 AM, Julien Thierry\n>>>> \n>>>> wrote:\n>>>>>\n>>>>>\n>>>>> Hi Christoffer,\n>>>>>\n>>>>>\n>>>>> On 30/08/17 19:53, Christoffer Dall wrote:\n>>>>>>\n>>>>>>\n>>>>>>\n>>>>>> Hi Julien,\n>>>>>>\n>>>>>> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n>>>>>>\n>>>>>> On Wed, Aug 30, 2017 at 11:01 AM, Julien Thierry\n>>>>>> \n>>>>>> wrote:\n>>>>>>>\n>>>>>>>\n>>>>>>>\n>>>>>>> Software Step exception is missing after trapping instruction from\n>>>>>>> the\n>>>>>>> guest.\n>>>>>>>\n>>>>>>> We need to set the PSR.SS to 0 for the guest vcpu before resuming\n>>>>>>> guest\n>>>>>>> execution.\n>>>>>>>\n>>>>>>> Signed-off-by: Julien Thierry \n>>>>>>> Cc: Christoffer Dall \n>>>>>>> Cc: Marc Zyngier \n>>>>>>> Cc: Catalin Marinas \n>>>>>>> Cc: Will Deacon \n>>>>>>>\n>>>>>>> ---\n>>>>>>> arch/arm64/include/asm/kvm_asm.h | 2 ++\n>>>>>>> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n>>>>>>> arch/arm64/kvm/debug.c | 12 +++++++++++-\n>>>>>>> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n>>>>>>> 4 files changed, 27 insertions(+), 1 deletion(-)\n>>>>>>>\n>>>>>>> diff --git a/arch/arm64/include/asm/kvm_asm.h\n>>>>>>> b/arch/arm64/include/asm/kvm_asm.h\n>>>>>>> index 26a64d0..398bbaa 100644\n>>>>>>> --- a/arch/arm64/include/asm/kvm_asm.h\n>>>>>>> +++ b/arch/arm64/include/asm/kvm_asm.h\n>>>>>>> @@ -32,6 +32,8 @@\n>>>>>>>\n>>>>>>> #define KVM_ARM64_DEBUG_DIRTY_SHIFT 0\n>>>>>>> #define KVM_ARM64_DEBUG_DIRTY (1 <<\n>>>>>>> KVM_ARM64_DEBUG_DIRTY_SHIFT)\n>>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT 1\n>>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP (1 <<\n>>>>>>> KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>>>>>>>\n>>>>>>> #define kvm_ksym_ref(sym)\n>>>>>>> \\\n>>>>>>> ({\n>>>>>>> \\\n>>>>>>> diff --git a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>> b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>> index fe39e68..d401c64 100644\n>>>>>>> --- a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>> +++ b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu\n>>>>>>> *vcpu, bool is_wide_instr)\n>>>>>>> kvm_skip_instr32(vcpu, is_wide_instr);\n>>>>>>> else\n>>>>>>> *vcpu_pc(vcpu) += 4;\n>>>>>>> + /* Let debug engine know we skipped an instruction */\n>>>>>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n>>>>>>\n>>>>>>\n>>>>>>\n>>>>>>\n>>>>>> Why do we need to defer this action until later?\n>>>>>>\n>>>>>> Can't we simply do clear DBG_SPSR_SS here?\n>>>>>>\n>>>>>\n>>>>> That was my first intention, but it turns out that the current state of\n>>>>> things (without this patch) is that every time we enter a guest,\n>>>>> kvm_arm_setup_debug gets called and if single step is requested for the\n>>>>> guest it will set the flag in the SPSR (ignoring the fact that we\n>>>>> cleared\n>>>>> it).\n>>>>\n>>>>\n>>>>\n>>>> Ah, right, duh.\n>>>>\n>>>>> This happens even if we exit the guest because of a data abort.\n>>>>>\n>>>>> For normal single step execution, we do need to reset SPSR.SS to 1\n>>>>> before\n>>>>> running the guest since completion of a step should clear that bit\n>>>>> before\n>>>>> taking a software step exception. So what kvm_arm_setup_debug does\n>>>>> seems\n>>>>> correct to me but missed the case for trapped/emulated instructions.\n>>>>>\n>>>>> So even if we just clear DBG_SPSR_SS here, we would still need to tell\n>>>>> kvm_arm_setup_debug not to change the bit. Or resetting SPSR.SS to 1\n>>>>> for\n>>>>> normal single stepping needs to be done before we skip instructions in\n>>>>> KVM\n>>>>> but that doesn't sound right to me...\n>>>>>\n>>>>\n>>>> So I'm wondering if we're going about this wrong. Perhaps we need to\n>>>> discover at the end of the run loop that we were asked to single step\n>>>> execution and simply return to userspace, setting the debug exit\n>>>> reason etc., instead of entering the guest with PSTATE.SS==0 and\n>>>> relying on another trap back in to the guest just to set two fields on\n>>>> the kvm_run structure and exit to user space ?\n>>>>\n>>>\n>>> So if I understand correctly, the suggestion is that when we trap an\n>>> instruction we check whether it was supposed to be single stepped, if it\n>>> was\n>>> we set up the vcpu registers as if it had taken a software step exception\n>>> and return from kvm_arch_vcpu_ioctl_run. Is that right?\n>>\n>>\n>> yes, that's the idea. If there's a lot of complexity in setting up\n>> CPU register state, then it may not be a good idea, but if it's\n>> relatively clean, I think it can be preferred over the \"let's keep a\n>> flag aroudn for later\" approach.\n>>\n>\n> So I looked a bit into it.\n>\n> One annoying thing is that the single step mechanic is specific to arm64.\n> MMU and MMIO code is shared between arm and arm64 and do some handling of\n> traps.\n>\n> So cleanest way I can think of doing this would be to clear SPSR.SS in\n> arm64::kvm_skip_instr, then have some function (e.g.\n> kvm_handle/manage_debug_state) at the end of the run loop. For arm, the\n> function is empty. For arm64, the function, if we are in an active pending\n> state (SPSR.D == 0 && SPSR.SS == 0 && MDSCR_EL1.SS == 1) and not about to\n> return to userland, returns with a \"fake debug exception\".\n>\n> So instead of a flag, we would just use SPSR.SS (or more generally the vcpu\n> state) to know if we need to exit with a debug exception or not. And the\n> kvm_arm_setup_debug would be left untouched (always setting SPSR.SS when\n> requested by userland).\n>\n> Does that sound like what you had in mind? Or does it seem better than the\n> current patch?\n>\nI was thinking to change the skip_instruction function to return an\nint, and then call kvm_handle_debug_ss() from skip_instruction, which\nwould update the kvm_run structure and exit here and then.\n\nHowever, I'm now thinking that this doesn't really work either,\nbecause we could have to emulate a trapped MMIO instruction in user\nspace, and then it's not clear how to exit with a debug exception at\nthe same time.\n\nSo perhaps we should stick with your original approach.\n\nThanks,\n-Christoffer","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"Mme5AGv8\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"TLjf7Zg/\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xjjrv4pWPz9sPm\n\tfor ;\n\tThu, 31 Aug 2017 23:29:19 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnPX2-0007Jb-C5; Thu, 31 Aug 2017 13:29:16 +0000","from mail-yw0-x236.google.com ([2607:f8b0:4002:c05::236])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnPWy-0007IX-Ng for linux-arm-kernel@lists.infradead.org;\n\tThu, 31 Aug 2017 13:29:14 +0000","by mail-yw0-x236.google.com with SMTP id s143so3277983ywg.0\n\tfor ;\n\tThu, 31 Aug 2017 06:28:52 -0700 (PDT)","by 10.37.97.23 with HTTP; Thu, 31 Aug 2017 06:28:51 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:\n\tReferences:In-Reply-To:MIME-Version:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=/7t/BNXilEaNRvF7tqtGM6dS/jvSYNxoOh0jGpCEGlM=;\n\tb=Mme5AGv8oqNM7b\n\tuAIsHt9XKbzzL8NChBismd1RDT2cf+l2f+KCUF4Hmu9PJSV/UBVZ42FQ28BCkpAztfoXBeVxbmgYZ\n\t1jpKZW5gnAJNmuE7mpVtzk2v6Lko2jHAEqzHnayTHDQNTJQfEa/+QQKk3GggVtOyI28pbLDCtYGgn\n\tTYS6Qu5Y1Qj9bSDjbH3yWWb2MfIrCMmR0ngAQ1tf9F8tmPijJ0+ZVoIeskXH6f19t57ml7WQsPgnP\n\tpkCK3JGmdgXz3s9+dcxijyN2xu2Higgbti3DsYu8IulLZMCbLRjTj0MVJ8c0xX2O3rOGqnRp2pdi7\n\tZHEn36mk7CCHj4wcPKYQ==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc:content-transfer-encoding;\n\tbh=FYiCgrRzGXQvh8PfJ3rYG11K+bo+5m/JxCk8+acxlJA=;\n\tb=TLjf7Zg/uF9amyvSuTiq1Kg/YCtlm0653yot7RrF3IM7qNw3qCBPJHPO7Uiqvv1mSD\n\twUvqbneaxNtqEfwPujlVyJqdg4xRiIY1TZbzRSD0qUeIP3S3O81PRjLXLntexoPMxlRg\n\tsNl8Pw9hXZWvCzyIqGGl6vsW9sFCagR3KmWsw="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc:content-transfer-encoding;\n\tbh=FYiCgrRzGXQvh8PfJ3rYG11K+bo+5m/JxCk8+acxlJA=;\n\tb=W5/UjyLNVAxDX/EveNeK/YREEx3X0+UoyZQPBuzNreNUsAN+BCXyu3OrBlBKZiCjlD\n\taxZdAljwW5KH/MijmitUYOjCABWE/1zFbW67i6ZcS0xi2dNba1n3J7EVxRUedaC3LAk5\n\thC623OJpjLn2lFpnj2gHdVlg+18faAmDlaMwHjSCf/9KBllsXEQIsK4ZSnpHmel7Xxaz\n\tYKjbvWPAgv5kcVW6HMmPPfKsT4zsR3zyxnImULWalFWNMn92zTZFtLPLWOX4r05wmclj\n\tyzaE0WmV5lTnNgosAKob2wtpDvHMnTqyx8COTnIUfy9tleya8KH4RqGJAddlox0hcuGz\n\tXQsA==","X-Gm-Message-State":"AHYfb5icxj0IayLR5qHQeHmnZFf+Ox7zy6GHfmqgtX7Z9HEC68gQ7nFl\n\tDuSTZLP4gre3FpdtI4XYGMOglTTI51lf9xc=","X-Google-Smtp-Source":"ADKCNb6w8+YVFUUKmO86exGFC6u5vpY2J88BOYne1iPoQ2te9Ozsl3HEBIpGAaqbYrtvP5khjUdchi6Zr+wDmQ9FUkY=","X-Received":"by 10.129.174.93 with SMTP id g29mr4259673ywk.435.1504186131522; \n\tThu, 31 Aug 2017 06:28:51 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>","From":"Christoffer Dall ","Date":"Thu, 31 Aug 2017 15:28:51 +0200","Message-ID":"","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"Julien Thierry ","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170831_062912_846067_6E689170 ","X-CRM114-Status":"GOOD ( 29.31 )","X-Spam-Score":"-2.7 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/,\n\tlow\n\ttrust [2607:f8b0:4002:c05:0:0:0:236 listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier , Catalin Marinas\n\t, Will Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, =?utf-8?q?Alex_Benn=C3=A9e?=\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1761037,"web_url":"http://patchwork.ozlabs.org/comment/1761037/","msgid":"<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>","list_archive_url":null,"date":"2017-08-31T13:57:57","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":72256,"url":"http://patchwork.ozlabs.org/api/people/72256/","name":"Julien Thierry","email":"julien.thierry@arm.com"},"content":"On 31/08/17 14:28, Christoffer Dall wrote:\n> On Thu, Aug 31, 2017 at 2:56 PM, Julien Thierry wrote:\n>>\n>>\n>> On 31/08/17 11:53, Christoffer Dall wrote:\n>>>\n>>> On Thu, Aug 31, 2017 at 11:37 AM, Julien Thierry \n>>> wrote:\n>>>>\n>>>>\n>>>>\n>>>> On 31/08/17 09:54, Christoffer Dall wrote:\n>>>>>\n>>>>>\n>>>>> On Thu, Aug 31, 2017 at 10:45 AM, Julien Thierry\n>>>>> \n>>>>> wrote:\n>>>>>>\n>>>>>>\n>>>>>> Hi Christoffer,\n>>>>>>\n>>>>>>\n>>>>>> On 30/08/17 19:53, Christoffer Dall wrote:\n>>>>>>>\n>>>>>>>\n>>>>>>>\n>>>>>>> Hi Julien,\n>>>>>>>\n>>>>>>> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n>>>>>>>\n>>>>>>> On Wed, Aug 30, 2017 at 11:01 AM, Julien Thierry\n>>>>>>> \n>>>>>>> wrote:\n>>>>>>>>\n>>>>>>>>\n>>>>>>>>\n>>>>>>>> Software Step exception is missing after trapping instruction from\n>>>>>>>> the\n>>>>>>>> guest.\n>>>>>>>>\n>>>>>>>> We need to set the PSR.SS to 0 for the guest vcpu before resuming\n>>>>>>>> guest\n>>>>>>>> execution.\n>>>>>>>>\n>>>>>>>> Signed-off-by: Julien Thierry \n>>>>>>>> Cc: Christoffer Dall \n>>>>>>>> Cc: Marc Zyngier \n>>>>>>>> Cc: Catalin Marinas \n>>>>>>>> Cc: Will Deacon \n>>>>>>>>\n>>>>>>>> ---\n>>>>>>>> arch/arm64/include/asm/kvm_asm.h | 2 ++\n>>>>>>>> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n>>>>>>>> arch/arm64/kvm/debug.c | 12 +++++++++++-\n>>>>>>>> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n>>>>>>>> 4 files changed, 27 insertions(+), 1 deletion(-)\n>>>>>>>>\n>>>>>>>> diff --git a/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>> b/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>> index 26a64d0..398bbaa 100644\n>>>>>>>> --- a/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>> +++ b/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>> @@ -32,6 +32,8 @@\n>>>>>>>>\n>>>>>>>> #define KVM_ARM64_DEBUG_DIRTY_SHIFT 0\n>>>>>>>> #define KVM_ARM64_DEBUG_DIRTY (1 <<\n>>>>>>>> KVM_ARM64_DEBUG_DIRTY_SHIFT)\n>>>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT 1\n>>>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP (1 <<\n>>>>>>>> KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>>>>>>>>\n>>>>>>>> #define kvm_ksym_ref(sym)\n>>>>>>>> \\\n>>>>>>>> ({\n>>>>>>>> \\\n>>>>>>>> diff --git a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>> b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>> index fe39e68..d401c64 100644\n>>>>>>>> --- a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>> +++ b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu\n>>>>>>>> *vcpu, bool is_wide_instr)\n>>>>>>>> kvm_skip_instr32(vcpu, is_wide_instr);\n>>>>>>>> else\n>>>>>>>> *vcpu_pc(vcpu) += 4;\n>>>>>>>> + /* Let debug engine know we skipped an instruction */\n>>>>>>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n>>>>>>>\n>>>>>>>\n>>>>>>>\n>>>>>>>\n>>>>>>> Why do we need to defer this action until later?\n>>>>>>>\n>>>>>>> Can't we simply do clear DBG_SPSR_SS here?\n>>>>>>>\n>>>>>>\n>>>>>> That was my first intention, but it turns out that the current state of\n>>>>>> things (without this patch) is that every time we enter a guest,\n>>>>>> kvm_arm_setup_debug gets called and if single step is requested for the\n>>>>>> guest it will set the flag in the SPSR (ignoring the fact that we\n>>>>>> cleared\n>>>>>> it).\n>>>>>\n>>>>>\n>>>>>\n>>>>> Ah, right, duh.\n>>>>>\n>>>>>> This happens even if we exit the guest because of a data abort.\n>>>>>>\n>>>>>> For normal single step execution, we do need to reset SPSR.SS to 1\n>>>>>> before\n>>>>>> running the guest since completion of a step should clear that bit\n>>>>>> before\n>>>>>> taking a software step exception. So what kvm_arm_setup_debug does\n>>>>>> seems\n>>>>>> correct to me but missed the case for trapped/emulated instructions.\n>>>>>>\n>>>>>> So even if we just clear DBG_SPSR_SS here, we would still need to tell\n>>>>>> kvm_arm_setup_debug not to change the bit. Or resetting SPSR.SS to 1\n>>>>>> for\n>>>>>> normal single stepping needs to be done before we skip instructions in\n>>>>>> KVM\n>>>>>> but that doesn't sound right to me...\n>>>>>>\n>>>>>\n>>>>> So I'm wondering if we're going about this wrong. Perhaps we need to\n>>>>> discover at the end of the run loop that we were asked to single step\n>>>>> execution and simply return to userspace, setting the debug exit\n>>>>> reason etc., instead of entering the guest with PSTATE.SS==0 and\n>>>>> relying on another trap back in to the guest just to set two fields on\n>>>>> the kvm_run structure and exit to user space ?\n>>>>>\n>>>>\n>>>> So if I understand correctly, the suggestion is that when we trap an\n>>>> instruction we check whether it was supposed to be single stepped, if it\n>>>> was\n>>>> we set up the vcpu registers as if it had taken a software step exception\n>>>> and return from kvm_arch_vcpu_ioctl_run. Is that right?\n>>>\n>>>\n>>> yes, that's the idea. If there's a lot of complexity in setting up\n>>> CPU register state, then it may not be a good idea, but if it's\n>>> relatively clean, I think it can be preferred over the \"let's keep a\n>>> flag aroudn for later\" approach.\n>>>\n>>\n>> So I looked a bit into it.\n>>\n>> One annoying thing is that the single step mechanic is specific to arm64.\n>> MMU and MMIO code is shared between arm and arm64 and do some handling of\n>> traps.\n>>\n>> So cleanest way I can think of doing this would be to clear SPSR.SS in\n>> arm64::kvm_skip_instr, then have some function (e.g.\n>> kvm_handle/manage_debug_state) at the end of the run loop. For arm, the\n>> function is empty. For arm64, the function, if we are in an active pending\n>> state (SPSR.D == 0 && SPSR.SS == 0 && MDSCR_EL1.SS == 1) and not about to\n>> return to userland, returns with a \"fake debug exception\".\n>>\n>> So instead of a flag, we would just use SPSR.SS (or more generally the vcpu\n>> state) to know if we need to exit with a debug exception or not. And the\n>> kvm_arm_setup_debug would be left untouched (always setting SPSR.SS when\n>> requested by userland).\n>>\n>> Does that sound like what you had in mind? Or does it seem better than the\n>> current patch?\n>>\n> I was thinking to change the skip_instruction function to return an\n> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n> would update the kvm_run structure and exit here and then.\n> \n\nSetting up the debug exception from within kvm_skip_instruction seem to \nchange a bit too much its semantic from arm to arm64. I would find this \neasily confusing.\n\n> However, I'm now thinking that this doesn't really work either,\n> because we could have to emulate a trapped MMIO instruction in user\n> space, and then it's not clear how to exit with a debug exception at\n> the same time.\n> \n> So perhaps we should stick with your original approach.\n> \n\nI had not realized that was possible. This makes things more complicated \nfor avoiding a back and forth with the guest for trapped exceptions. Out \nof luck, having the debug flag does look like single stepping would work \nas expected for userland MMIOs.\n\nI can try to detail the comment in kvm_arm_setup_debug when we set SPSR, \nhopefully making things clearer when seeing that part of the code.\n\nThanks,","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"InnSpd3z\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xjklG6YH6z9sD5\n\tfor ;\n\tFri, 1 Sep 2017 00:09:30 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnQ9r-0007ip-Sw; Thu, 31 Aug 2017 14:09:23 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnPzB-0000wX-9n for linux-arm-kernel@lists.infradead.org;\n\tThu, 31 Aug 2017 13:58:30 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 68D0880D;\n\tThu, 31 Aug 2017 06:58:00 -0700 (PDT)","from [10.1.207.37] (e112298-lin.cambridge.arm.com [10.1.207.37])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t133673F578; Thu, 31 Aug 2017 06:57:58 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=KnM4uTW33QJPvoSbdVKIYdtvCwEWM/A2HT0o92ZpDUg=;\n\tb=InnSpd3zHbKaRU3YwRNIhUCqw\n\t9Jw89PVbZYzTS1aPvq2XHbzQ71AFRl5wG0P+cLYDCgr5C900V0iL8m2N1mxyfb2DWrGL+mXyDb6hk\n\tlCcI5tTZoDFceutctOXfQ12IaanocKz/OhBFOxFxhJiXJTsNFvemsn/Jpgm1JlC3T22uHzz7eCNEV\n\tsdx67UXFeE0PqtIMRnbaIOrbDhmF9MpRk2T7le9NCvCOdbcwLkgV/dOpkJmyc6//VKVedbPaK50Ve\n\tvBAawlOOL8/6Zop9z9OeNrB0J4giV9e0CIg9vNdBcn6fO/XDxx3lX5BeFHqB2L3fupGzmEWQGB2Z0\n\tdJDfaXi8A==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"Christoffer Dall ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t","From":"Julien Thierry ","Message-ID":"<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>","Date":"Thu, 31 Aug 2017 14:57:57 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170831_065821_781690_29A8A40A ","X-CRM114-Status":"GOOD ( 29.57 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier , Catalin Marinas\n\t, Will Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, =?utf-8?q?Alex_Benn=C3=A9e?=\n\t","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1761042,"web_url":"http://patchwork.ozlabs.org/comment/1761042/","msgid":"","list_archive_url":null,"date":"2017-08-31T14:01:19","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":26352,"url":"http://patchwork.ozlabs.org/api/people/26352/","name":"Christoffer Dall","email":"christoffer.dall@linaro.org"},"content":"On Thu, Aug 31, 2017 at 3:57 PM, Julien Thierry wrote:\n>\n>\n> On 31/08/17 14:28, Christoffer Dall wrote:\n>>\n>> On Thu, Aug 31, 2017 at 2:56 PM, Julien Thierry \n>> wrote:\n>>>\n>>>\n>>>\n>>> On 31/08/17 11:53, Christoffer Dall wrote:\n>>>>\n>>>>\n>>>> On Thu, Aug 31, 2017 at 11:37 AM, Julien Thierry\n>>>> \n>>>> wrote:\n>>>>>\n>>>>>\n>>>>>\n>>>>>\n>>>>> On 31/08/17 09:54, Christoffer Dall wrote:\n>>>>>>\n>>>>>>\n>>>>>>\n>>>>>> On Thu, Aug 31, 2017 at 10:45 AM, Julien Thierry\n>>>>>> \n>>>>>> wrote:\n>>>>>>>\n>>>>>>>\n>>>>>>>\n>>>>>>> Hi Christoffer,\n>>>>>>>\n>>>>>>>\n>>>>>>> On 30/08/17 19:53, Christoffer Dall wrote:\n>>>>>>>>\n>>>>>>>>\n>>>>>>>>\n>>>>>>>>\n>>>>>>>> Hi Julien,\n>>>>>>>>\n>>>>>>>> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n>>>>>>>>\n>>>>>>>> On Wed, Aug 30, 2017 at 11:01 AM, Julien Thierry\n>>>>>>>> \n>>>>>>>> wrote:\n>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>> Software Step exception is missing after trapping instruction from\n>>>>>>>>> the\n>>>>>>>>> guest.\n>>>>>>>>>\n>>>>>>>>> We need to set the PSR.SS to 0 for the guest vcpu before resuming\n>>>>>>>>> guest\n>>>>>>>>> execution.\n>>>>>>>>>\n>>>>>>>>> Signed-off-by: Julien Thierry \n>>>>>>>>> Cc: Christoffer Dall \n>>>>>>>>> Cc: Marc Zyngier \n>>>>>>>>> Cc: Catalin Marinas \n>>>>>>>>> Cc: Will Deacon \n>>>>>>>>>\n>>>>>>>>> ---\n>>>>>>>>> arch/arm64/include/asm/kvm_asm.h | 2 ++\n>>>>>>>>> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n>>>>>>>>> arch/arm64/kvm/debug.c | 12 +++++++++++-\n>>>>>>>>> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n>>>>>>>>> 4 files changed, 27 insertions(+), 1 deletion(-)\n>>>>>>>>>\n>>>>>>>>> diff --git a/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>> b/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>> index 26a64d0..398bbaa 100644\n>>>>>>>>> --- a/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>> +++ b/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>> @@ -32,6 +32,8 @@\n>>>>>>>>>\n>>>>>>>>> #define KVM_ARM64_DEBUG_DIRTY_SHIFT 0\n>>>>>>>>> #define KVM_ARM64_DEBUG_DIRTY (1 <<\n>>>>>>>>> KVM_ARM64_DEBUG_DIRTY_SHIFT)\n>>>>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT 1\n>>>>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP (1 <<\n>>>>>>>>> KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>>>>>>>>>\n>>>>>>>>> #define kvm_ksym_ref(sym)\n>>>>>>>>> \\\n>>>>>>>>> ({\n>>>>>>>>> \\\n>>>>>>>>> diff --git a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>> b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>> index fe39e68..d401c64 100644\n>>>>>>>>> --- a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>> +++ b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu\n>>>>>>>>> *vcpu, bool is_wide_instr)\n>>>>>>>>> kvm_skip_instr32(vcpu, is_wide_instr);\n>>>>>>>>> else\n>>>>>>>>> *vcpu_pc(vcpu) += 4;\n>>>>>>>>> + /* Let debug engine know we skipped an instruction */\n>>>>>>>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n>>>>>>>>\n>>>>>>>>\n>>>>>>>>\n>>>>>>>>\n>>>>>>>>\n>>>>>>>> Why do we need to defer this action until later?\n>>>>>>>>\n>>>>>>>> Can't we simply do clear DBG_SPSR_SS here?\n>>>>>>>>\n>>>>>>>\n>>>>>>> That was my first intention, but it turns out that the current state\n>>>>>>> of\n>>>>>>> things (without this patch) is that every time we enter a guest,\n>>>>>>> kvm_arm_setup_debug gets called and if single step is requested for\n>>>>>>> the\n>>>>>>> guest it will set the flag in the SPSR (ignoring the fact that we\n>>>>>>> cleared\n>>>>>>> it).\n>>>>>>\n>>>>>>\n>>>>>>\n>>>>>>\n>>>>>> Ah, right, duh.\n>>>>>>\n>>>>>>> This happens even if we exit the guest because of a data abort.\n>>>>>>>\n>>>>>>> For normal single step execution, we do need to reset SPSR.SS to 1\n>>>>>>> before\n>>>>>>> running the guest since completion of a step should clear that bit\n>>>>>>> before\n>>>>>>> taking a software step exception. So what kvm_arm_setup_debug does\n>>>>>>> seems\n>>>>>>> correct to me but missed the case for trapped/emulated instructions.\n>>>>>>>\n>>>>>>> So even if we just clear DBG_SPSR_SS here, we would still need to\n>>>>>>> tell\n>>>>>>> kvm_arm_setup_debug not to change the bit. Or resetting SPSR.SS to 1\n>>>>>>> for\n>>>>>>> normal single stepping needs to be done before we skip instructions\n>>>>>>> in\n>>>>>>> KVM\n>>>>>>> but that doesn't sound right to me...\n>>>>>>>\n>>>>>>\n>>>>>> So I'm wondering if we're going about this wrong. Perhaps we need to\n>>>>>> discover at the end of the run loop that we were asked to single step\n>>>>>> execution and simply return to userspace, setting the debug exit\n>>>>>> reason etc., instead of entering the guest with PSTATE.SS==0 and\n>>>>>> relying on another trap back in to the guest just to set two fields on\n>>>>>> the kvm_run structure and exit to user space ?\n>>>>>>\n>>>>>\n>>>>> So if I understand correctly, the suggestion is that when we trap an\n>>>>> instruction we check whether it was supposed to be single stepped, if\n>>>>> it\n>>>>> was\n>>>>> we set up the vcpu registers as if it had taken a software step\n>>>>> exception\n>>>>> and return from kvm_arch_vcpu_ioctl_run. Is that right?\n>>>>\n>>>>\n>>>>\n>>>> yes, that's the idea. If there's a lot of complexity in setting up\n>>>> CPU register state, then it may not be a good idea, but if it's\n>>>> relatively clean, I think it can be preferred over the \"let's keep a\n>>>> flag aroudn for later\" approach.\n>>>>\n>>>\n>>> So I looked a bit into it.\n>>>\n>>> One annoying thing is that the single step mechanic is specific to arm64.\n>>> MMU and MMIO code is shared between arm and arm64 and do some handling of\n>>> traps.\n>>>\n>>> So cleanest way I can think of doing this would be to clear SPSR.SS in\n>>> arm64::kvm_skip_instr, then have some function (e.g.\n>>> kvm_handle/manage_debug_state) at the end of the run loop. For arm, the\n>>> function is empty. For arm64, the function, if we are in an active\n>>> pending\n>>> state (SPSR.D == 0 && SPSR.SS == 0 && MDSCR_EL1.SS == 1) and not about to\n>>> return to userland, returns with a \"fake debug exception\".\n>>>\n>>> So instead of a flag, we would just use SPSR.SS (or more generally the\n>>> vcpu\n>>> state) to know if we need to exit with a debug exception or not. And the\n>>> kvm_arm_setup_debug would be left untouched (always setting SPSR.SS when\n>>> requested by userland).\n>>>\n>>> Does that sound like what you had in mind? Or does it seem better than\n>>> the\n>>> current patch?\n>>>\n>> I was thinking to change the skip_instruction function to return an\n>> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n>> would update the kvm_run structure and exit here and then.\n>>\n>\n> Setting up the debug exception from within kvm_skip_instruction seem to\n> change a bit too much its semantic from arm to arm64. I would find this\n> easily confusing.\n>\n>> However, I'm now thinking that this doesn't really work either,\n>> because we could have to emulate a trapped MMIO instruction in user\n>> space, and then it's not clear how to exit with a debug exception at\n>> the same time.\n>>\n>> So perhaps we should stick with your original approach.\n>>\n>\n> I had not realized that was possible. This makes things more complicated for\n> avoiding a back and forth with the guest for trapped exceptions. Out of\n> luck, having the debug flag does look like single stepping would work as\n> expected for userland MMIOs.\n\nYes, I think your approach is the best choice now, considering.\n\n>\n> I can try to detail the comment in kvm_arm_setup_debug when we set SPSR,\n> hopefully making things clearer when seeing that part of the code.\n>\n\nI also think we need to improve the comment in the world-switch return\npath, and I'd like Alex to weigh in here before we merge this. He's\nback from holiday on Monday.\n\nThanks,\n-Christoffer","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"KBW/p759\"; \n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=infradead.org header.i=@infradead.org\n\theader.b=\"s9qaV+L7\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"QRvhtVXX\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xjks46Cmtz9s81\n\tfor ;\n\tFri, 1 Sep 2017 00:14:32 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnQEn-00026F-7p; Thu, 31 Aug 2017 14:14:29 +0000","from casper.infradead.org ([2001:8b0:10b:1236::1])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnQEk-0001xY-Ty for linux-arm-kernel@bombadil.infradead.org;\n\tThu, 31 Aug 2017 14:14:27 +0000","from mail-yw0-x232.google.com ([2607:f8b0:4002:c05::232])\n\tby casper.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dnQ2R-0002Al-3E for linux-arm-kernel@lists.infradead.org;\n\tThu, 31 Aug 2017 14:01:47 +0000","by mail-yw0-x232.google.com with SMTP id s143so3744460ywg.0\n\tfor ;\n\tThu, 31 Aug 2017 07:01:22 -0700 (PDT)","by 10.37.97.23 with HTTP; Thu, 31 Aug 2017 07:01:19 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:\n\tReferences:In-Reply-To:MIME-Version:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=M16SIBux6ePai3/WNCTTFiPJ6tcB55lOBx1tGsvtwn4=;\n\tb=KBW/p759tThS9H\n\tuuIhUkS6TxfIDxsKBKhgKgd33/Axl0g++R6pfQR09U83q2cWNhxBuUyZd132FAcWT/B4Me0IJ8UdM\n\tdeYVjZHYRmKIqnofWHm0TbEYRbsB3nvPTZEZg6bQq/3LSkyZ3mq++fkP5M3hZ9D2fSFhEi8RIITB0\n\tpWrL8oHUqejrSP549WzSwEUUpaDsBMpa3e/j3T1oVH3FS6RjzCAcK1v7jvy3TQmti2JsCS0HXFr1E\n\tbvuKKqicYAfuNXasnVwsjXu9fMGr7TM1B3S3s3x/dETgO7RUgyS3gbSY4aPeM0saiNFEXpCVz2PmI\n\tCgFpZ8dzls5ZcValYhNA==;","v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=infradead.org; s=casper.20170209;\n\th=Content-Transfer-Encoding:Content-Type:\n\tCc:To:Subject:Message-ID:Date:From:References:In-Reply-To:MIME-Version:Sender\n\t:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:\n\tResent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:\n\tList-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;\n\tbh=n415WDI+Cyjykrg3dg6udKfwwBjVOAHxa0/K+wF2zO4=;\n\tb=s9qaV+L7d7+pwAwmAjKnKDt8MP\n\tuVSHCkesRgueSv8tbtwtxFfdFQMtOBwop9FCzPZSPlvoWrWwarjdotYmaXITOg8gVV92O7AHCnT0X\n\tys+CzTMU/F0Y425uHBv0bQu6b0gjUfzXS0As+Czpj0teJMTdA9/OV1OMi6TttOOuxpzk11msygqbN\n\tfdNwKJulHZ4T7OvEFGh5PhqC8s2wOaFuwYxIc0wA5Yy+KwSC8/pgDgxA7/w2OckEYEPUQ0mbQoaod\n\tF6j7guQAN2C8l4/RfBNsgjX+zEM4JDmPy2ZKZvnOqcHywBu6HXoXV54u6ADlQjxACqSvfHlV4glN0\n\tGw5Or7tA==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc:content-transfer-encoding;\n\tbh=n415WDI+Cyjykrg3dg6udKfwwBjVOAHxa0/K+wF2zO4=;\n\tb=QRvhtVXXp9bo6C08bgQue0QbKxEDBkVDvPD+aquIp53Q+leTPVlMsYBobdKJnXzACo\n\thTnfD4JW4gJG0fuVJcVioyplmrBCshNHtyVQHWHfxaHS8bB+oe+FXtmzSRbRvkXVhAyZ\n\taTsIcfhD6B/pBOHDJ0kAWQk1gq9UvLzzF8Qx4="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc:content-transfer-encoding;\n\tbh=n415WDI+Cyjykrg3dg6udKfwwBjVOAHxa0/K+wF2zO4=;\n\tb=ERkNxpG+pPhw+KBEn16jTFW1agyG1qxEI05Q6PH/ucNrIo+BEfoXrIvF3SljIoo+iZ\n\tFyBmvbtWDd6Jg8LCp1Uf15Q7m4+uqqr8L8bvmPXUEETrMebSp7q1oTYTW4O9NpKvtVvA\n\tMTjgrkGQb466TAqRi6ufaPHHZGfNGu5oQqi63Bk2xH0CMpIfXOq85pnQ7scO6rdCi7EC\n\tJuJV5aGtxQLTlOEy5/YFIEKlbftgrHkVr0U4LkeKZ+izyVFyO8inYWOYh7v22F9Sff2C\n\topJqYudkW5MN1XkShZhH4w3sFT5oc138jXKjBFytVsEVG208NEwU06x0rzuZpNv13XI/\n\ttKfg==","X-Gm-Message-State":"AHYfb5gzIKqPkH4U0IHD0GhzTRUFFWjcpYWubg9xNBkpynVai8dPuhIe\n\tTEB0Wd91smsWQsT3/OppDyEIOebqFGl3iAc=","X-Google-Smtp-Source":"ADKCNb5ITQtWiVTUFcYfEO0IhUCAFPtUru57r39K39FhBgwpnfp6H1cDVdtGncobt/EpTMe4TWGRxAwlzW+LrQdgKsY=","X-Received":"by 10.13.217.205 with SMTP id b196mr4793556ywe.369.1504188080297;\n\tThu, 31 Aug 2017 07:01:20 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>","From":"Christoffer Dall ","Date":"Thu, 31 Aug 2017 16:01:19 +0200","Message-ID":"","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"Julien Thierry ","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170831_150143_422440_C130D781 ","X-CRM114-Status":"GOOD ( 44.91 )","X-Spam-Score":"-2.7 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on casper.infradead.org summary:\n\tContent analysis details: (-2.7 points, 5.0 required)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/,\n\tlow\n\ttrust [2607:f8b0:4002:c05:0:0:0:232 listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier , Catalin Marinas\n\t, Will Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, =?utf-8?q?Alex_Benn=C3=A9e?=\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1777519,"web_url":"http://patchwork.ozlabs.org/comment/1777519/","msgid":"<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>","list_archive_url":null,"date":"2017-09-29T12:38:14","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":72256,"url":"http://patchwork.ozlabs.org/api/people/72256/","name":"Julien Thierry","email":"julien.thierry@arm.com"},"content":"On 31/08/17 15:01, Christoffer Dall wrote:\n> On Thu, Aug 31, 2017 at 3:57 PM, Julien Thierry wrote:\n>>\n>>\n>> On 31/08/17 14:28, Christoffer Dall wrote:\n>>>\n>>> On Thu, Aug 31, 2017 at 2:56 PM, Julien Thierry \n>>> wrote:\n>>>>\n>>>>\n>>>>\n>>>> On 31/08/17 11:53, Christoffer Dall wrote:\n>>>>>\n>>>>>\n>>>>> On Thu, Aug 31, 2017 at 11:37 AM, Julien Thierry\n>>>>> \n>>>>> wrote:\n>>>>>>\n>>>>>>\n>>>>>>\n>>>>>>\n>>>>>> On 31/08/17 09:54, Christoffer Dall wrote:\n>>>>>>>\n>>>>>>>\n>>>>>>>\n>>>>>>> On Thu, Aug 31, 2017 at 10:45 AM, Julien Thierry\n>>>>>>> \n>>>>>>> wrote:\n>>>>>>>>\n>>>>>>>>\n>>>>>>>>\n>>>>>>>> Hi Christoffer,\n>>>>>>>>\n>>>>>>>>\n>>>>>>>> On 30/08/17 19:53, Christoffer Dall wrote:\n>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>> Hi Julien,\n>>>>>>>>>\n>>>>>>>>> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n>>>>>>>>>\n>>>>>>>>> On Wed, Aug 30, 2017 at 11:01 AM, Julien Thierry\n>>>>>>>>> \n>>>>>>>>> wrote:\n>>>>>>>>>>\n>>>>>>>>>>\n>>>>>>>>>>\n>>>>>>>>>>\n>>>>>>>>>> Software Step exception is missing after trapping instruction from\n>>>>>>>>>> the\n>>>>>>>>>> guest.\n>>>>>>>>>>\n>>>>>>>>>> We need to set the PSR.SS to 0 for the guest vcpu before resuming\n>>>>>>>>>> guest\n>>>>>>>>>> execution.\n>>>>>>>>>>\n>>>>>>>>>> Signed-off-by: Julien Thierry \n>>>>>>>>>> Cc: Christoffer Dall \n>>>>>>>>>> Cc: Marc Zyngier \n>>>>>>>>>> Cc: Catalin Marinas \n>>>>>>>>>> Cc: Will Deacon \n>>>>>>>>>>\n>>>>>>>>>> ---\n>>>>>>>>>> arch/arm64/include/asm/kvm_asm.h | 2 ++\n>>>>>>>>>> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n>>>>>>>>>> arch/arm64/kvm/debug.c | 12 +++++++++++-\n>>>>>>>>>> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n>>>>>>>>>> 4 files changed, 27 insertions(+), 1 deletion(-)\n>>>>>>>>>>\n>>>>>>>>>> diff --git a/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>>> b/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>>> index 26a64d0..398bbaa 100644\n>>>>>>>>>> --- a/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>>> +++ b/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>>> @@ -32,6 +32,8 @@\n>>>>>>>>>>\n>>>>>>>>>> #define KVM_ARM64_DEBUG_DIRTY_SHIFT 0\n>>>>>>>>>> #define KVM_ARM64_DEBUG_DIRTY (1 <<\n>>>>>>>>>> KVM_ARM64_DEBUG_DIRTY_SHIFT)\n>>>>>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT 1\n>>>>>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP (1 <<\n>>>>>>>>>> KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>>>>>>>>>>\n>>>>>>>>>> #define kvm_ksym_ref(sym)\n>>>>>>>>>> \\\n>>>>>>>>>> ({\n>>>>>>>>>> \\\n>>>>>>>>>> diff --git a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>>> b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>>> index fe39e68..d401c64 100644\n>>>>>>>>>> --- a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>>> +++ b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>>> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu\n>>>>>>>>>> *vcpu, bool is_wide_instr)\n>>>>>>>>>> kvm_skip_instr32(vcpu, is_wide_instr);\n>>>>>>>>>> else\n>>>>>>>>>> *vcpu_pc(vcpu) += 4;\n>>>>>>>>>> + /* Let debug engine know we skipped an instruction */\n>>>>>>>>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>> Why do we need to defer this action until later?\n>>>>>>>>>\n>>>>>>>>> Can't we simply do clear DBG_SPSR_SS here?\n>>>>>>>>>\n>>>>>>>>\n>>>>>>>> That was my first intention, but it turns out that the current state\n>>>>>>>> of\n>>>>>>>> things (without this patch) is that every time we enter a guest,\n>>>>>>>> kvm_arm_setup_debug gets called and if single step is requested for\n>>>>>>>> the\n>>>>>>>> guest it will set the flag in the SPSR (ignoring the fact that we\n>>>>>>>> cleared\n>>>>>>>> it).\n>>>>>>>\n>>>>>>>\n>>>>>>>\n>>>>>>>\n>>>>>>> Ah, right, duh.\n>>>>>>>\n>>>>>>>> This happens even if we exit the guest because of a data abort.\n>>>>>>>>\n>>>>>>>> For normal single step execution, we do need to reset SPSR.SS to 1\n>>>>>>>> before\n>>>>>>>> running the guest since completion of a step should clear that bit\n>>>>>>>> before\n>>>>>>>> taking a software step exception. So what kvm_arm_setup_debug does\n>>>>>>>> seems\n>>>>>>>> correct to me but missed the case for trapped/emulated instructions.\n>>>>>>>>\n>>>>>>>> So even if we just clear DBG_SPSR_SS here, we would still need to\n>>>>>>>> tell\n>>>>>>>> kvm_arm_setup_debug not to change the bit. Or resetting SPSR.SS to 1\n>>>>>>>> for\n>>>>>>>> normal single stepping needs to be done before we skip instructions\n>>>>>>>> in\n>>>>>>>> KVM\n>>>>>>>> but that doesn't sound right to me...\n>>>>>>>>\n>>>>>>>\n>>>>>>> So I'm wondering if we're going about this wrong. Perhaps we need to\n>>>>>>> discover at the end of the run loop that we were asked to single step\n>>>>>>> execution and simply return to userspace, setting the debug exit\n>>>>>>> reason etc., instead of entering the guest with PSTATE.SS==0 and\n>>>>>>> relying on another trap back in to the guest just to set two fields on\n>>>>>>> the kvm_run structure and exit to user space ?\n>>>>>>>\n>>>>>>\n>>>>>> So if I understand correctly, the suggestion is that when we trap an\n>>>>>> instruction we check whether it was supposed to be single stepped, if\n>>>>>> it\n>>>>>> was\n>>>>>> we set up the vcpu registers as if it had taken a software step\n>>>>>> exception\n>>>>>> and return from kvm_arch_vcpu_ioctl_run. Is that right?\n>>>>>\n>>>>>\n>>>>>\n>>>>> yes, that's the idea. If there's a lot of complexity in setting up\n>>>>> CPU register state, then it may not be a good idea, but if it's\n>>>>> relatively clean, I think it can be preferred over the \"let's keep a\n>>>>> flag aroudn for later\" approach.\n>>>>>\n>>>>\n>>>> So I looked a bit into it.\n>>>>\n>>>> One annoying thing is that the single step mechanic is specific to arm64.\n>>>> MMU and MMIO code is shared between arm and arm64 and do some handling of\n>>>> traps.\n>>>>\n>>>> So cleanest way I can think of doing this would be to clear SPSR.SS in\n>>>> arm64::kvm_skip_instr, then have some function (e.g.\n>>>> kvm_handle/manage_debug_state) at the end of the run loop. For arm, the\n>>>> function is empty. For arm64, the function, if we are in an active\n>>>> pending\n>>>> state (SPSR.D == 0 && SPSR.SS == 0 && MDSCR_EL1.SS == 1) and not about to\n>>>> return to userland, returns with a \"fake debug exception\".\n>>>>\n>>>> So instead of a flag, we would just use SPSR.SS (or more generally the\n>>>> vcpu\n>>>> state) to know if we need to exit with a debug exception or not. And the\n>>>> kvm_arm_setup_debug would be left untouched (always setting SPSR.SS when\n>>>> requested by userland).\n>>>>\n>>>> Does that sound like what you had in mind? Or does it seem better than\n>>>> the\n>>>> current patch?\n>>>>\n>>> I was thinking to change the skip_instruction function to return an\n>>> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n>>> would update the kvm_run structure and exit here and then.\n>>>\n>>\n>> Setting up the debug exception from within kvm_skip_instruction seem to\n>> change a bit too much its semantic from arm to arm64. I would find this\n>> easily confusing.\n>>\n>>> However, I'm now thinking that this doesn't really work either,\n>>> because we could have to emulate a trapped MMIO instruction in user\n>>> space, and then it's not clear how to exit with a debug exception at\n>>> the same time.\n>>>\n>>> So perhaps we should stick with your original approach.\n>>>\n>>\n>> I had not realized that was possible. This makes things more complicated for\n>> avoiding a back and forth with the guest for trapped exceptions. Out of\n>> luck, having the debug flag does look like single stepping would work as\n>> expected for userland MMIOs.\n> \n> Yes, I think your approach is the best choice now, considering.\n> \n>>\n>> I can try to detail the comment in kvm_arm_setup_debug when we set SPSR,\n>> hopefully making things clearer when seeing that part of the code.\n>>\n> \n> I also think we need to improve the comment in the world-switch return\n> path, and I'd like Alex to weigh in here before we merge this. He's\n> back from holiday on Monday.\n> \n\nPing Alex?","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"PlMUrKoM\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y3WMH3PPhz9t4R\n\tfor ;\n\tFri, 29 Sep 2017 22:38:50 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dxuZ2-00077J-7I; Fri, 29 Sep 2017 12:38:44 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dxuYv-0006rm-Qg for linux-arm-kernel@lists.infradead.org;\n\tFri, 29 Sep 2017 12:38:41 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0878F1529;\n\tFri, 29 Sep 2017 05:38:17 -0700 (PDT)","from [10.1.206.253] (e112298-lin.cambridge.arm.com [10.1.206.253])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tE5EE23F53D; Fri, 29 Sep 2017 05:38:15 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=STcSne6GVTw8fmB7q19BiseVvqF6WvhHhxCOjbdej1E=;\n\tb=PlMUrKoMDQ+Kp3qfFTIgeyz0e\n\t1665VCgYUIkH2W6nQsozS5znVPU1Hjqo3Sa62J1FqeH388VDuEx3WBcnEP26Hgh/vKQMieiv0ooq5\n\tTsQ7x7N0lc/k4JodU7mfVW60+hurHQaYsJHNeB9IPy+Plp7m0h0uMxA8N/VKPDzNJp7Oz3j8+5WD2\n\tLmSNBHvdLmGKs2FQuPeml+tVs2MYFSV5KmLWBOr8NFg4O0bDASHcQik4CP3qPXt4ICULOR+629vXc\n\tpt4LkS/GD13S28OVZnRM9fWZX2gsOhgqb0Mxs06lxUjQHFSMytXVnc4J+zqkBxswAb507+FsGqbXO\n\t0g34/UdHw==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"Christoffer Dall , =?utf-8?q?Alex_Benn?=\n\t=?utf-8?b?w6ll?= ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t","From":"Julien Thierry ","Message-ID":"<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>","Date":"Fri, 29 Sep 2017 13:38:14 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170929_053838_049655_9B259B6A ","X-CRM114-Status":"GOOD ( 27.35 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779042,"web_url":"http://patchwork.ozlabs.org/comment/1779042/","msgid":"<877ewcz3bv.fsf@linaro.org>","list_archive_url":null,"date":"2017-10-03T14:57:40","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":39532,"url":"http://patchwork.ozlabs.org/api/people/39532/","name":"Alex Bennée","email":"alex.bennee@linaro.org"},"content":"Julien Thierry writes:\n\n> On 31/08/17 15:01, Christoffer Dall wrote:\n>> On Thu, Aug 31, 2017 at 3:57 PM, Julien Thierry wrote:\n>>> On 31/08/17 14:28, Christoffer Dall wrote:\n>>>> On Thu, Aug 31, 2017 at 2:56 PM, Julien Thierry \n>>>> wrote:\n>>>>> On 31/08/17 11:53, Christoffer Dall wrote:\n>>>>>> On Thu, Aug 31, 2017 at 11:37 AM, Julien Thierry\n>>>>>> \n>>>>>> wrote:\n>>>>>>> On 31/08/17 09:54, Christoffer Dall wrote:\n>>>>>>>> On Thu, Aug 31, 2017 at 10:45 AM, Julien Thierry\n>>>>>>>> \n>>>>>>>> wrote:\n>>>>>>>>> Hi Christoffer,\n>>>>>>>>> On 30/08/17 19:53, Christoffer Dall wrote:\n>>>>>>>>>> Hi Julien,\n>>>>>>>>>>\n>>>>>>>>>> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n>>>>>>>>>>\n>>>>>>>>>> On Wed, Aug 30, 2017 at 11:01 AM, Julien Thierry\n>>>>>>>>>> \n>>>>>>>>>> wrote:\n>>>>>>>>>>>\n>>>>>>>>>>> Software Step exception is missing after trapping instruction from\n>>>>>>>>>>> the\n>>>>>>>>>>> guest.\n>>>>>>>>>>>\n>>>>>>>>>>> We need to set the PSR.SS to 0 for the guest vcpu before resuming\n>>>>>>>>>>> guest\n>>>>>>>>>>> execution.\n>>>>>>>>>>>\n>>>>>>>>>>> Signed-off-by: Julien Thierry \n>>>>>>>>>>> Cc: Christoffer Dall \n>>>>>>>>>>> Cc: Marc Zyngier \n>>>>>>>>>>> Cc: Catalin Marinas \n>>>>>>>>>>> Cc: Will Deacon \n>>>>>>>>>>>\n>>>>>>>>>>> ---\n>>>>>>>>>>> arch/arm64/include/asm/kvm_asm.h | 2 ++\n>>>>>>>>>>> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n>>>>>>>>>>> arch/arm64/kvm/debug.c | 12 +++++++++++-\n>>>>>>>>>>> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n>>>>>>>>>>> 4 files changed, 27 insertions(+), 1 deletion(-)\n>>>>>>>>>>>\n>>>>>>>>>>> diff --git a/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>>>> b/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>>>> index 26a64d0..398bbaa 100644\n>>>>>>>>>>> --- a/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>>>> +++ b/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>>>> @@ -32,6 +32,8 @@\n>>>>>>>>>>>\n>>>>>>>>>>> #define KVM_ARM64_DEBUG_DIRTY_SHIFT 0\n>>>>>>>>>>> #define KVM_ARM64_DEBUG_DIRTY (1 <<\n>>>>>>>>>>> KVM_ARM64_DEBUG_DIRTY_SHIFT)\n>>>>>>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT 1\n>>>>>>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP (1 <<\n>>>>>>>>>>> KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>>>>>>>>>>>\n>>>>>>>>>>> #define kvm_ksym_ref(sym)\n>>>>>>>>>>> \\\n>>>>>>>>>>> ({\n>>>>>>>>>>> \\\n>>>>>>>>>>> diff --git a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>>>> b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>>>> index fe39e68..d401c64 100644\n>>>>>>>>>>> --- a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>>>> +++ b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>>>> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu\n>>>>>>>>>>> *vcpu, bool is_wide_instr)\n>>>>>>>>>>> kvm_skip_instr32(vcpu, is_wide_instr);\n>>>>>>>>>>> else\n>>>>>>>>>>> *vcpu_pc(vcpu) += 4;\n>>>>>>>>>>> + /* Let debug engine know we skipped an instruction */\n>>>>>>>>>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n>>>>>>>>>>\n>>>>>>>>>>\n>>>>>>>>>>\n>>>>>>>>>>\n>>>>>>>>>>\n>>>>>>>>>> Why do we need to defer this action until later?\n>>>>>>>>>>\n>>>>>>>>>> Can't we simply do clear DBG_SPSR_SS here?\n>>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>> That was my first intention, but it turns out that the current state\n>>>>>>>>> of\n>>>>>>>>> things (without this patch) is that every time we enter a guest,\n>>>>>>>>> kvm_arm_setup_debug gets called and if single step is requested for\n>>>>>>>>> the\n>>>>>>>>> guest it will set the flag in the SPSR (ignoring the fact that we\n>>>>>>>>> cleared\n>>>>>>>>> it).\n\nSo to be clear kvm_arm_setup_debug only sets SS if we are about to\nsingle-step. This is controlled by the debug ioctl which is done outside\nof the main KVM run loop.\n\n>>>>>>>>\n>>>>>>>> Ah, right, duh.\n>>>>>>>>\n>>>>>>>>> This happens even if we exit the guest because of a data abort.\n>>>>>>>>>\n>>>>>>>>> For normal single step execution, we do need to reset SPSR.SS to 1\n>>>>>>>>> before\n>>>>>>>>> running the guest since completion of a step should clear that bit\n>>>>>>>>> before\n>>>>>>>>> taking a software step exception. So what kvm_arm_setup_debug does\n>>>>>>>>> seems\n>>>>>>>>> correct to me but missed the case for trapped/emulated instructions.\n>>>>>>>>>\n>>>>>>>>> So even if we just clear DBG_SPSR_SS here, we would still need to\n>>>>>>>>> tell\n>>>>>>>>> kvm_arm_setup_debug not to change the bit. Or resetting SPSR.SS to 1\n>>>>>>>>> for\n>>>>>>>>> normal single stepping needs to be done before we skip instructions\n>>>>>>>>> in\n>>>>>>>>> KVM\n>>>>>>>>> but that doesn't sound right to me...\n>>>>>>>>>\n>>>>>>>>\n>>>>>>>> So I'm wondering if we're going about this wrong. Perhaps we need to\n>>>>>>>> discover at the end of the run loop that we were asked to single step\n>>>>>>>> execution and simply return to userspace, setting the debug exit\n>>>>>>>> reason etc., instead of entering the guest with PSTATE.SS==0 and\n>>>>>>>> relying on another trap back in to the guest just to set two fields on\n>>>>>>>> the kvm_run structure and exit to user space ?\n>>>>>>>>\n>>>>>>>\n>>>>>>> So if I understand correctly, the suggestion is that when we trap an\n>>>>>>> instruction we check whether it was supposed to be single stepped, if\n>>>>>>> it\n>>>>>>> was\n>>>>>>> we set up the vcpu registers as if it had taken a software step\n>>>>>>> exception\n>>>>>>> and return from kvm_arch_vcpu_ioctl_run. Is that right?\n>>>>>>\n>>>>>>\n>>>>>>\n>>>>>> yes, that's the idea. If there's a lot of complexity in setting up\n>>>>>> CPU register state, then it may not be a good idea, but if it's\n>>>>>> relatively clean, I think it can be preferred over the \"let's keep a\n>>>>>> flag aroudn for later\" approach.\n>>>>>>\n>>>>>\n>>>>> So I looked a bit into it.\n>>>>>\n>>>>> One annoying thing is that the single step mechanic is specific to arm64.\n>>>>> MMU and MMIO code is shared between arm and arm64 and do some handling of\n>>>>> traps.\n\nThis is because the ARMv7 support isn't complete (and also ARMv7\nhardware is slightly more complex in its corner cases if we were to do it).\n\n>>>>>\n>>>>> So cleanest way I can think of doing this would be to clear SPSR.SS in\n>>>>> arm64::kvm_skip_instr, then have some function (e.g.\n>>>>> kvm_handle/manage_debug_state) at the end of the run loop. For arm, the\n>>>>> function is empty. For arm64, the function, if we are in an active\n>>>>> pending\n>>>>> state (SPSR.D == 0 && SPSR.SS == 0 && MDSCR_EL1.SS == 1) and not about to\n>>>>> return to userland, returns with a \"fake debug exception\".\n>>>>>\n>>>>> So instead of a flag, we would just use SPSR.SS (or more generally the\n>>>>> vcpu\n>>>>> state) to know if we need to exit with a debug exception or not. And the\n>>>>> kvm_arm_setup_debug would be left untouched (always setting SPSR.SS when\n>>>>> requested by userland).\n>>>>>\n>>>>> Does that sound like what you had in mind? Or does it seem better than\n>>>>> the\n>>>>> current patch?\n>>>>>\n>>>> I was thinking to change the skip_instruction function to return an\n>>>> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n>>>> would update the kvm_run structure and exit here and then.\n>>>>\n>>>\n>>> Setting up the debug exception from within kvm_skip_instruction seem to\n>>> change a bit too much its semantic from arm to arm64. I would find this\n>>> easily confusing.\n>>>\n>>>> However, I'm now thinking that this doesn't really work either,\n>>>> because we could have to emulate a trapped MMIO instruction in user\n>>>> space, and then it's not clear how to exit with a debug exception at\n>>>> the same time.\n\nA debug exception at guest exit point is (IIRC) just having the\nappropriate status in the run->exit_reason (KVM_EXIT_DEBUG). If you need\nto exit for MMIO emulation (i.e. the instruction has not run yet) you\nshouldn't do that. Exit, emulate and return. We could handle the ioctl\nto clear SS in userspace but I guess that gets just as messy.\n\n>>>>\n>>>> So perhaps we should stick with your original approach.\n>>>>\n>>>\n>>> I had not realized that was possible. This makes things more complicated for\n>>> avoiding a back and forth with the guest for trapped exceptions. Out of\n>>> luck, having the debug flag does look like single stepping would work as\n>>> expected for userland MMIOs.\n>>\n>> Yes, I think your approach is the best choice now, considering.\n>>\n>>>\n>>> I can try to detail the comment in kvm_arm_setup_debug when we set SPSR,\n>>> hopefully making things clearer when seeing that part of the code.\n>>>\n>>\n>> I also think we need to improve the comment in the world-switch return\n>> path, and I'd like Alex to weigh in here before we merge this. He's\n>> back from holiday on Monday.\n>>\n>\n> Ping Alex?\n\nSorry for the delay getting back to you. I had flagged the email but\nwith holidays and conferences in the way it fell off my queue.\n\nSo to summarise as I understand things:\n\n Host User Space | Host KVM | Host Hyp | Guest VM |\n\n Enable Debug(SS)\n KVM_RUN ----------->\n Guest SPSR.SS set\n --> World Switch ->\n Insn Trap to Hyp\n World Switch <-\n (SS not cleared)\n <--\n Insn Emulated\n pc += 4\n -->\n World Switch\n (SS still set)\n ->\n Insn +4 SS\n <-\n World Switch\n (SS cleared)\n\n <--\n Guest exit (debug)\n <--\n See SS did 2 insns?\n\nDo I understand the problem you are trying to fix correctly?\n\n--\nAlex Bennée","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"EtyMbXHq\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"aifPJ4tW\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y62GF4MWgz9sRV\n\tfor ;\n\tWed, 4 Oct 2017 01:58:13 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzOeA-0002cY-7D; Tue, 03 Oct 2017 14:58:10 +0000","from mail-wr0-x235.google.com ([2a00:1450:400c:c0c::235])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzOe4-0002XZ-HJ for linux-arm-kernel@lists.infradead.org;\n\tTue, 03 Oct 2017 14:58:07 +0000","by mail-wr0-x235.google.com with SMTP id j14so6582590wre.8\n\tfor ;\n\tTue, 03 Oct 2017 07:57:43 -0700 (PDT)","from zen.linaro.local ([81.128.185.34])\n\tby smtp.gmail.com with ESMTPSA id\n\t29sm19504756wrz.77.2017.10.03.07.57.41\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tTue, 03 Oct 2017 07:57:41 -0700 (PDT)","from zen (localhost [127.0.0.1])\n\tby zen.linaro.local (Postfix) with ESMTPS id 9E9183E1337;\n\tTue, 3 Oct 2017 15:57:40 +0100 (BST)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:\n\tIn-reply-to:Subject:To:From:References:Reply-To:Content-ID:\n\tContent-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc\n\t:Resent-Message-ID:List-Owner;\n\tbh=Dx85eH3oLyTqoF3WRd4BpNcacig8bAPrukeKCkcOTRw=;\n\tb=EtyMbXHqsAYCps7W7UQX8OpRgq\n\tvnMEx8RR6w9VEuK+VCZuVdG4c8zw6HRNaFOufdLA9JAlxOZ2qFpIW1o/F/1sA32JmtlLvrf5hzaGz\n\t1r1hj5i/rR6p7iJx4ROdtFfIDOofnsgDGp/mIIdNsdGwSEQyPyssKPF//kaFyvJTr4wHLYecPsEoS\n\t/BAP0z/B7cmq+fuSLOIszzZu7cuXFttasl/zSCuCAlPcBHIRzBjAI/jgf6SZBHqc/a3DnG+YkWysd\n\tgo3e11Wb96vzgBVkE4IJ97eNva10C4UTahVJsK/4LkhMKGma41WH68ZftmfV58TonI3E3Vzphq438\n\tx1lONGHw==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=references:user-agent:from:to:cc:subject:in-reply-to:date\n\t:message-id:mime-version:content-transfer-encoding;\n\tbh=HelVYPP6BwX9eul+kPO8hckTPoVtNDoLroomuuDy3tE=;\n\tb=aifPJ4tWq9uWRYhDR4vhfMUtCA/pLq0l0Cg0Au2YwAxJXDpGiLiJwlDYrRk3PydWpI\n\tjKshS6f66J7gA9jHZ9dojNd636tmNNSKG4GCLcPs6XA50TmpeAROITUfnWxFqqsZA1FN\n\tWT/e4Qci3TrKi1Id8h1rg4LFleEkzA1mWmToc="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:references:user-agent:from:to:cc:subject\n\t:in-reply-to:date:message-id:mime-version:content-transfer-encoding; \n\tbh=HelVYPP6BwX9eul+kPO8hckTPoVtNDoLroomuuDy3tE=;\n\tb=gtsyw5DQMALGhJWPFNiFcZtdMPe70P5sBt8qwhiMPOtbVx/CuwpWI6N559NVGP1qge\n\tjGqCodDrzxBLLbTVd/fqhk+HQRo32DIvcEzpQYUr3H7fC9RdVcuV6AiC3iovm25fC+/G\n\tWG9ldrwNn5ryo6IvPM0qUm4IrYtWMHmziXs14i01KEJ00jE+x2ljzp9HyYIJ7a4tfVI4\n\t1A7LI9S+HvaV4F55Klaf3+mvQSBZ8EcxnVYYJaDYicylmag9aaAXJAfz/8h9Xal8lSuU\n\tZynGINqoORnkPF1K2FU4QLF/tonxvVR/smk0cwxJdevK2DKmTNGKIG/nHHqiVZgpsCQm\n\tz/cA==","X-Gm-Message-State":"AHPjjUh0Y6MTJXbIn0u9g4BWGObdaMgszsRs39LdKiS/fA79/xTxNbaO\n\t3yS7SZ3VFUx5/oKVAEzy7Z6uug==","X-Google-Smtp-Source":"AOwi7QBuSRKpEEHlKW1YZHxg/RT3I6JRptG2eF9EpXGBzPaYkcPTKf5W3N91IWcx8LmaW0LT5tnLCQ==","X-Received":"by 10.223.153.100 with SMTP id\n\tx91mr16130898wrb.202.1507042662021; \n\tTue, 03 Oct 2017 07:57:42 -0700 (PDT)","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>","User-agent":"mu4e 0.9.19; emacs 26.0.60","From":"Alex =?utf-8?q?Benn=C3=A9e?= ","To":"Julien Thierry ","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","In-reply-to":"<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>","Date":"Tue, 03 Oct 2017 15:57:40 +0100","Message-ID":"<877ewcz3bv.fsf@linaro.org>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171003_075804_912395_85B22EF3 ","X-CRM114-Status":"GOOD ( 28.26 )","X-Spam-Score":"-2.0 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.0 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,\n\tno\n\ttrust [2a00:1450:400c:c0c:0:0:0:235 listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, \n\tChristoffer Dall ","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779053,"web_url":"http://patchwork.ozlabs.org/comment/1779053/","msgid":"<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>","list_archive_url":null,"date":"2017-10-03T15:07:38","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":72256,"url":"http://patchwork.ozlabs.org/api/people/72256/","name":"Julien Thierry","email":"julien.thierry@arm.com"},"content":"On 03/10/17 15:57, Alex Bennée wrote:\n> \n> Julien Thierry writes:\n> \n>> On 31/08/17 15:01, Christoffer Dall wrote:\n>>> On Thu, Aug 31, 2017 at 3:57 PM, Julien Thierry wrote:\n>>>> On 31/08/17 14:28, Christoffer Dall wrote:\n>>>>> On Thu, Aug 31, 2017 at 2:56 PM, Julien Thierry \n>>>>> wrote:\n>>>>>> On 31/08/17 11:53, Christoffer Dall wrote:\n>>>>>>> On Thu, Aug 31, 2017 at 11:37 AM, Julien Thierry\n>>>>>>> \n>>>>>>> wrote:\n>>>>>>>> On 31/08/17 09:54, Christoffer Dall wrote:\n>>>>>>>>> On Thu, Aug 31, 2017 at 10:45 AM, Julien Thierry\n>>>>>>>>> \n>>>>>>>>> wrote:\n>>>>>>>>>> Hi Christoffer,\n>>>>>>>>>> On 30/08/17 19:53, Christoffer Dall wrote:\n>>>>>>>>>>> Hi Julien,\n>>>>>>>>>>>\n>>>>>>>>>>> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n>>>>>>>>>>>\n>>>>>>>>>>> On Wed, Aug 30, 2017 at 11:01 AM, Julien Thierry\n>>>>>>>>>>> \n>>>>>>>>>>> wrote:\n>>>>>>>>>>>>\n>>>>>>>>>>>> Software Step exception is missing after trapping instruction from\n>>>>>>>>>>>> the\n>>>>>>>>>>>> guest.\n>>>>>>>>>>>>\n>>>>>>>>>>>> We need to set the PSR.SS to 0 for the guest vcpu before resuming\n>>>>>>>>>>>> guest\n>>>>>>>>>>>> execution.\n>>>>>>>>>>>>\n>>>>>>>>>>>> Signed-off-by: Julien Thierry \n>>>>>>>>>>>> Cc: Christoffer Dall \n>>>>>>>>>>>> Cc: Marc Zyngier \n>>>>>>>>>>>> Cc: Catalin Marinas \n>>>>>>>>>>>> Cc: Will Deacon \n>>>>>>>>>>>>\n>>>>>>>>>>>> ---\n>>>>>>>>>>>> arch/arm64/include/asm/kvm_asm.h | 2 ++\n>>>>>>>>>>>> arch/arm64/include/asm/kvm_emulate.h | 2 ++\n>>>>>>>>>>>> arch/arm64/kvm/debug.c | 12 +++++++++++-\n>>>>>>>>>>>> arch/arm64/kvm/hyp/switch.c | 12 ++++++++++++\n>>>>>>>>>>>> 4 files changed, 27 insertions(+), 1 deletion(-)\n>>>>>>>>>>>>\n>>>>>>>>>>>> diff --git a/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>>>>> b/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>>>>> index 26a64d0..398bbaa 100644\n>>>>>>>>>>>> --- a/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>>>>> +++ b/arch/arm64/include/asm/kvm_asm.h\n>>>>>>>>>>>> @@ -32,6 +32,8 @@\n>>>>>>>>>>>>\n>>>>>>>>>>>> #define KVM_ARM64_DEBUG_DIRTY_SHIFT 0\n>>>>>>>>>>>> #define KVM_ARM64_DEBUG_DIRTY (1 <<\n>>>>>>>>>>>> KVM_ARM64_DEBUG_DIRTY_SHIFT)\n>>>>>>>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP_SHIFT 1\n>>>>>>>>>>>> +#define KVM_ARM64_DEBUG_INST_SKIP (1 <<\n>>>>>>>>>>>> KVM_ARM64_DEBUG_INST_SKIP_SHIFT)\n>>>>>>>>>>>>\n>>>>>>>>>>>> #define kvm_ksym_ref(sym)\n>>>>>>>>>>>> \\\n>>>>>>>>>>>> ({\n>>>>>>>>>>>> \\\n>>>>>>>>>>>> diff --git a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>>>>> b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>>>>> index fe39e68..d401c64 100644\n>>>>>>>>>>>> --- a/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>>>>> +++ b/arch/arm64/include/asm/kvm_emulate.h\n>>>>>>>>>>>> @@ -95,6 +95,8 @@ static inline void kvm_skip_instr(struct kvm_vcpu\n>>>>>>>>>>>> *vcpu, bool is_wide_instr)\n>>>>>>>>>>>> kvm_skip_instr32(vcpu, is_wide_instr);\n>>>>>>>>>>>> else\n>>>>>>>>>>>> *vcpu_pc(vcpu) += 4;\n>>>>>>>>>>>> + /* Let debug engine know we skipped an instruction */\n>>>>>>>>>>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_INST_SKIP;\n>>>>>>>>>>>\n>>>>>>>>>>>\n>>>>>>>>>>>\n>>>>>>>>>>>\n>>>>>>>>>>>\n>>>>>>>>>>> Why do we need to defer this action until later?\n>>>>>>>>>>>\n>>>>>>>>>>> Can't we simply do clear DBG_SPSR_SS here?\n>>>>>>>>>>>\n>>>>>>>>>>\n>>>>>>>>>> That was my first intention, but it turns out that the current state\n>>>>>>>>>> of\n>>>>>>>>>> things (without this patch) is that every time we enter a guest,\n>>>>>>>>>> kvm_arm_setup_debug gets called and if single step is requested for\n>>>>>>>>>> the\n>>>>>>>>>> guest it will set the flag in the SPSR (ignoring the fact that we\n>>>>>>>>>> cleared\n>>>>>>>>>> it).\n> \n> So to be clear kvm_arm_setup_debug only sets SS if we are about to\n> single-step. This is controlled by the debug ioctl which is done outside\n> of the main KVM run loop.\n> \n>>>>>>>>>\n>>>>>>>>> Ah, right, duh.\n>>>>>>>>>\n>>>>>>>>>> This happens even if we exit the guest because of a data abort.\n>>>>>>>>>>\n>>>>>>>>>> For normal single step execution, we do need to reset SPSR.SS to 1\n>>>>>>>>>> before\n>>>>>>>>>> running the guest since completion of a step should clear that bit\n>>>>>>>>>> before\n>>>>>>>>>> taking a software step exception. So what kvm_arm_setup_debug does\n>>>>>>>>>> seems\n>>>>>>>>>> correct to me but missed the case for trapped/emulated instructions.\n>>>>>>>>>>\n>>>>>>>>>> So even if we just clear DBG_SPSR_SS here, we would still need to\n>>>>>>>>>> tell\n>>>>>>>>>> kvm_arm_setup_debug not to change the bit. Or resetting SPSR.SS to 1\n>>>>>>>>>> for\n>>>>>>>>>> normal single stepping needs to be done before we skip instructions\n>>>>>>>>>> in\n>>>>>>>>>> KVM\n>>>>>>>>>> but that doesn't sound right to me...\n>>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>> So I'm wondering if we're going about this wrong. Perhaps we need to\n>>>>>>>>> discover at the end of the run loop that we were asked to single step\n>>>>>>>>> execution and simply return to userspace, setting the debug exit\n>>>>>>>>> reason etc., instead of entering the guest with PSTATE.SS==0 and\n>>>>>>>>> relying on another trap back in to the guest just to set two fields on\n>>>>>>>>> the kvm_run structure and exit to user space ?\n>>>>>>>>>\n>>>>>>>>\n>>>>>>>> So if I understand correctly, the suggestion is that when we trap an\n>>>>>>>> instruction we check whether it was supposed to be single stepped, if\n>>>>>>>> it\n>>>>>>>> was\n>>>>>>>> we set up the vcpu registers as if it had taken a software step\n>>>>>>>> exception\n>>>>>>>> and return from kvm_arch_vcpu_ioctl_run. Is that right?\n>>>>>>>\n>>>>>>>\n>>>>>>>\n>>>>>>> yes, that's the idea. If there's a lot of complexity in setting up\n>>>>>>> CPU register state, then it may not be a good idea, but if it's\n>>>>>>> relatively clean, I think it can be preferred over the \"let's keep a\n>>>>>>> flag aroudn for later\" approach.\n>>>>>>>\n>>>>>>\n>>>>>> So I looked a bit into it.\n>>>>>>\n>>>>>> One annoying thing is that the single step mechanic is specific to arm64.\n>>>>>> MMU and MMIO code is shared between arm and arm64 and do some handling of\n>>>>>> traps.\n> \n> This is because the ARMv7 support isn't complete (and also ARMv7\n> hardware is slightly more complex in its corner cases if we were to do it).\n> \n>>>>>>\n>>>>>> So cleanest way I can think of doing this would be to clear SPSR.SS in\n>>>>>> arm64::kvm_skip_instr, then have some function (e.g.\n>>>>>> kvm_handle/manage_debug_state) at the end of the run loop. For arm, the\n>>>>>> function is empty. For arm64, the function, if we are in an active\n>>>>>> pending\n>>>>>> state (SPSR.D == 0 && SPSR.SS == 0 && MDSCR_EL1.SS == 1) and not about to\n>>>>>> return to userland, returns with a \"fake debug exception\".\n>>>>>>\n>>>>>> So instead of a flag, we would just use SPSR.SS (or more generally the\n>>>>>> vcpu\n>>>>>> state) to know if we need to exit with a debug exception or not. And the\n>>>>>> kvm_arm_setup_debug would be left untouched (always setting SPSR.SS when\n>>>>>> requested by userland).\n>>>>>>\n>>>>>> Does that sound like what you had in mind? Or does it seem better than\n>>>>>> the\n>>>>>> current patch?\n>>>>>>\n>>>>> I was thinking to change the skip_instruction function to return an\n>>>>> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n>>>>> would update the kvm_run structure and exit here and then.\n>>>>>\n>>>>\n>>>> Setting up the debug exception from within kvm_skip_instruction seem to\n>>>> change a bit too much its semantic from arm to arm64. I would find this\n>>>> easily confusing.\n>>>>\n>>>>> However, I'm now thinking that this doesn't really work either,\n>>>>> because we could have to emulate a trapped MMIO instruction in user\n>>>>> space, and then it's not clear how to exit with a debug exception at\n>>>>> the same time.\n> \n> A debug exception at guest exit point is (IIRC) just having the\n> appropriate status in the run->exit_reason (KVM_EXIT_DEBUG). If you need\n> to exit for MMIO emulation (i.e. the instruction has not run yet) you\n> shouldn't do that. Exit, emulate and return. We could handle the ioctl\n> to clear SS in userspace but I guess that gets just as messy.\n> \n>>>>>\n>>>>> So perhaps we should stick with your original approach.\n>>>>>\n>>>>\n>>>> I had not realized that was possible. This makes things more complicated for\n>>>> avoiding a back and forth with the guest for trapped exceptions. Out of\n>>>> luck, having the debug flag does look like single stepping would work as\n>>>> expected for userland MMIOs.\n>>>\n>>> Yes, I think your approach is the best choice now, considering.\n>>>\n>>>>\n>>>> I can try to detail the comment in kvm_arm_setup_debug when we set SPSR,\n>>>> hopefully making things clearer when seeing that part of the code.\n>>>>\n>>>\n>>> I also think we need to improve the comment in the world-switch return\n>>> path, and I'd like Alex to weigh in here before we merge this. He's\n>>> back from holiday on Monday.\n>>>\n>>\n>> Ping Alex?\n> \n> Sorry for the delay getting back to you. I had flagged the email but\n> with holidays and conferences in the way it fell off my queue.\n> \n\nNo problem, thanks for looking at it.\n\n> So to summarise as I understand things:\n> \n> Host User Space | Host KVM | Host Hyp | Guest VM |\n> \n> Enable Debug(SS)\n> KVM_RUN ----------->\n> Guest SPSR.SS set\n> --> World Switch ->\n> Insn Trap to Hyp\n> World Switch <-\n> (SS not cleared)\n> <--\n> Insn Emulated\n> pc += 4\n> -->\n> World Switch\n> (SS still set)\n> ->\n> Insn +4 SS\n> <-\n> World Switch\n> (SS cleared)\n> \n> <--\n> Guest exit (debug)\n> <--\n> See SS did 2 insns?\n> \n> Do I understand the problem you are trying to fix correctly?\n\nYes that's the issue. The debugger is not made aware of the \nemulated/skipped instruction and the hypervisor jumps back into the guest.\n\nClearing SS before jumping back to the guest will simply trigger a debug \nexception as soon as we ERET from EL2 to EL1 (so we end up just getting \nback to EL2).\n\nCheers,","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"tKx65f9g\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y62Tj2SpVz9sBZ\n\tfor ;\n\tWed, 4 Oct 2017 02:08:09 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzOnl-0000TB-UI; Tue, 03 Oct 2017 15:08:05 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzOnh-0000P1-GC for linux-arm-kernel@lists.infradead.org;\n\tTue, 03 Oct 2017 15:08:04 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3A3921529;\n\tTue, 3 Oct 2017 08:07:41 -0700 (PDT)","from [10.1.206.253] (e112298-lin.cambridge.arm.com [10.1.206.253])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tD76083F53D; Tue, 3 Oct 2017 08:07:39 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=kiZj0XYw5tnJ9RoJsanoGLkh83ejgctU4NTYCj6uq6U=;\n\tb=tKx65f9gWE2q2E9lcZZfDdkoe\n\thtrBGSCYQZl+fNwDUK5tsv7NRguJ3RkmYio2eJGancA1WZei6fcNoy1t1ZRIoqFDULS227yl8fgB7\n\tkuN+Z6BYs4ir/K4fyzI7XkrwKGL0kmRg9kd8+zUaQ/Q9YJZ8ct1VOIVvFKzhO1knxFZgAfD9Hi3PG\n\tfUB2IONH0KMhxb5oPmUD4Ze78WDtOG5bZj/3we1lRzARh3YvBhsSSde/aAcdyiijfd4Y2nlSODJhW\n\tAcgrtvgxHhGM+g0N/NTJ6FQvc+wW7+4ocN/K6695iwGVcZZ6EZflW3RSGbBSPrwZxXGpj23rNTBDn\n\tEiAjbu6uA==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"=?utf-8?q?Alex_Benn=C3=A9e?= ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>","From":"Julien Thierry ","Message-ID":"<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>","Date":"Tue, 3 Oct 2017 16:07:38 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<877ewcz3bv.fsf@linaro.org>","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171003_080801_629334_8AE9257D ","X-CRM114-Status":"GOOD ( 34.38 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, \n\tChristoffer Dall ","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779102,"web_url":"http://patchwork.ozlabs.org/comment/1779102/","msgid":"<874lrgz0yf.fsf@linaro.org>","list_archive_url":null,"date":"2017-10-03T15:48:56","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":39532,"url":"http://patchwork.ozlabs.org/api/people/39532/","name":"Alex Bennée","email":"alex.bennee@linaro.org"},"content":"Julien Thierry writes:\n\n> On 03/10/17 15:57, Alex Bennée wrote:\n>>\n>> Julien Thierry writes:\n>>\n>>> On 31/08/17 15:01, Christoffer Dall wrote:\n\n>>>>>>>>>>> On 30/08/17 19:53, Christoffer Dall wrote:\n>>>>>>>>>>>> Hi Julien,\n>>>>>>>>>>>>\n>>>>>>>>>>>> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n>>>>>>>>>>>>\n\n>>>>> I can try to detail the comment in kvm_arm_setup_debug when we set SPSR,\n>>>>> hopefully making things clearer when seeing that part of the code.\n>>>>>\n>>>>\n>>>> I also think we need to improve the comment in the world-switch return\n>>>> path, and I'd like Alex to weigh in here before we merge this. He's\n>>>> back from holiday on Monday.\n>>>>\n>>>\n>>> Ping Alex?\n>>\n>> Sorry for the delay getting back to you. I had flagged the email but\n>> with holidays and conferences in the way it fell off my queue.\n>>\n>\n> No problem, thanks for looking at it.\n>\n>> So to summarise as I understand things:\n>>\n>> Host User Space | Host KVM | Host Hyp | Guest VM |\n>>\n>> Enable Debug(SS)\n>> KVM_RUN ----------->\n>> Guest SPSR.SS set\n>> --> World Switch ->\n>> Insn Trap to Hyp\n>> World Switch <-\n>> (SS not cleared)\n>> <--\n>> Insn Emulated\n>> pc += 4\n>> -->\n>> World Switch\n>> (SS still set)\n>> ->\n>> Insn +4 SS\n>> <-\n>> World Switch\n>> (SS cleared)\n>>\n>> <--\n>> Guest exit (debug)\n>> <--\n>> See SS did 2 insns?\n>>\n>> Do I understand the problem you are trying to fix correctly?\n>\n> Yes that's the issue. The debugger is not made aware of the\n> emulated/skipped instruction and the hypervisor jumps back into the\n> guest.\n>\n> Clearing SS before jumping back to the guest will simply trigger a\n> debug exception as soon as we ERET from EL2 to EL1 (so we end up just\n> getting back to EL2).\n\nWhy don't we just exit KVM after we've emulated the instruction if we\nare under debug? After all at this point whatever needed to be done is\ndone and the guest debug code can get on with life.\n\nI understand there is the problem of exiting for an MMIO emulation but\nmaybe that complexity should be handled by userspace (\"render unto\nuserspace the things that are userspaces\") and it can decide to lift the\nstep ioctl if appropriate.\n\nI guess I should have a look at the series. Are you re-basing anytime\nsoon? It looks like it currently has a few minor merge conflicts with\ncurrent master.\n\n--\nAlex Bennée","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"UvcpxvzE\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"VqFD+lg1\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y63PP1M8Tz9sPt\n\tfor ;\n\tWed, 4 Oct 2017 02:49:27 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzPRj-00007Y-EA; Tue, 03 Oct 2017 15:49:23 +0000","from mail-wm0-x229.google.com ([2a00:1450:400c:c09::229])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzPRf-0008VG-Rb for linux-arm-kernel@lists.infradead.org;\n\tTue, 03 Oct 2017 15:49:21 +0000","by mail-wm0-x229.google.com with SMTP id i124so17870054wmf.3\n\tfor ;\n\tTue, 03 Oct 2017 08:48:59 -0700 (PDT)","from zen.linaro.local ([81.128.185.34])\n\tby smtp.gmail.com with ESMTPSA id\n\t65sm6102602wro.25.2017.10.03.08.48.56\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tTue, 03 Oct 2017 08:48:56 -0700 (PDT)","from zen (localhost [127.0.0.1])\n\tby zen.linaro.local (Postfix) with ESMTPS id 12C703E1337;\n\tTue, 3 Oct 2017 16:48:56 +0100 (BST)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:\n\tIn-reply-to:Subject:To:From:References:Reply-To:Content-ID:\n\tContent-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc\n\t:Resent-Message-ID:List-Owner;\n\tbh=Q2JVQ0T2cqm+5VTC3PC/eAuAW28ZAXScck6YJ7NvzZA=;\n\tb=UvcpxvzE3XD/S0Ariza5sbqfLd\n\tc+pkKltPT/SVFuxA1n5baDmW5oVn5Tda6DB/bKygv3zQhWRJyCIcbsBY998F19hPv5FHY9y6Y+jwY\n\tJjqtzo61eym37hD29PrfAaZGuXmZPHZpPFzuvQYuCaQWToimFUePa5gHdfwsT3pemcyaC3tHAU7ST\n\tzPJKUqD4itLir2ukq0P32fGuntKEFoRLom5prLFWrFPo7XX8pNesnW76bpbjwbrN0d5KfKsmjokaT\n\tcE90cvyrM+hvdFJJSKfM6VJxdXiLabWftYbyqMCHvi2j2yqiWSk0NBCS/+/uIbhDoV8d1qRMaz7r+\n\tMFSbKX9w==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=references:user-agent:from:to:cc:subject:in-reply-to:date\n\t:message-id:mime-version:content-transfer-encoding;\n\tbh=U7JaYscWYutN5Ow6J+YC7iOmTvslx6e3R4p1eP+Hybo=;\n\tb=VqFD+lg1wF2IKGabrU0JaftI7qk1r+vnEnXjSO+uH1c7kAMd4Q1C5xxu0bf76/jY0b\n\tvhzf2/Oe6Irjw2TqeCkGIFw0lOpTpIsnlJsKTYnfDNrCSuoKplQNLxQwd7KtamjlkimD\n\tkA0I2qi5TDxfDHZriOV863dI8ypQID+iyQ8D8="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:references:user-agent:from:to:cc:subject\n\t:in-reply-to:date:message-id:mime-version:content-transfer-encoding; \n\tbh=U7JaYscWYutN5Ow6J+YC7iOmTvslx6e3R4p1eP+Hybo=;\n\tb=tI0zUPpKVXv/YnzTkWxHNVpT0buN382H8dHpGzWuKGwxvv2KN/ndOFR0aYi9AUuolo\n\thkTNOc6LZ8U3OoZX7bj9XrmB3damLJqWMT7fu+J8v7b/khcsi0eLqgVcKkOhZI9BRwTk\n\tCkflzXOKR8iRSX9rO0PRtiFX81BMoLFAh8+FW44YFF5FtAWVvLPsses2Mma+FSwev/DY\n\tJgJ2nFg3RAQ5Ipun5a9xfxLGx4R4BgXtp9KBcEnrd+osl0HbXEXksSFBa9UdvN5GmXY5\n\tY1QgGOj/E3Ysn7t+iuqOFa0CngKQIoZ+j6VZzB5+su+lWnbBNVIv5D0kPEwQmQ4zuwK8\n\tRvbw==","X-Gm-Message-State":"AMCzsaVrY3W3JGkdJVQwqChUsL0ZJJWvq72bJKOVJZMNEEYSFA8rR/kU\n\t7Nrps6Dq6USG5Ujx64x+VGPTYg==","X-Google-Smtp-Source":"AOwi7QAoXguuWNCYlHNWVzVji7GGhtl3yowrCcu8/gHBCfH/zaDxZO14b6lYRXfswhg4VKy+UmLLoQ==","X-Received":"by 10.28.169.21 with SMTP id s21mr9736208wme.87.1507045737387;\n\tTue, 03 Oct 2017 08:48:57 -0700 (PDT)","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>","User-agent":"mu4e 0.9.19; emacs 26.0.60","From":"Alex =?utf-8?q?Benn=C3=A9e?= ","To":"Julien Thierry ","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","In-reply-to":"<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>","Date":"Tue, 03 Oct 2017 16:48:56 +0100","Message-ID":"<874lrgz0yf.fsf@linaro.org>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171003_084920_056913_EB1D2243 ","X-CRM114-Status":"GOOD ( 17.07 )","X-Spam-Score":"-2.7 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/,\n\tlow\n\ttrust [2a00:1450:400c:c09:0:0:0:229 listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, \n\tChristoffer Dall ","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779124,"web_url":"http://patchwork.ozlabs.org/comment/1779124/","msgid":"<37d50b72-58cb-633e-552e-96d5b528b5ba@arm.com>","list_archive_url":null,"date":"2017-10-03T16:17:24","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":72256,"url":"http://patchwork.ozlabs.org/api/people/72256/","name":"Julien Thierry","email":"julien.thierry@arm.com"},"content":"On 03/10/17 16:48, Alex Bennée wrote:\n> \n> Julien Thierry writes:\n> \n>> On 03/10/17 15:57, Alex Bennée wrote:\n>>>\n>>> Julien Thierry writes:\n>>>\n>>>> On 31/08/17 15:01, Christoffer Dall wrote:\n> \n>>>>>>>>>>>> On 30/08/17 19:53, Christoffer Dall wrote:\n>>>>>>>>>>>>> Hi Julien,\n>>>>>>>>>>>>>\n>>>>>>>>>>>>> [cc'ing Alex Bennée here who wrote the debug code for arm64]\n>>>>>>>>>>>>>\n> \n>>>>>> I can try to detail the comment in kvm_arm_setup_debug when we set SPSR,\n>>>>>> hopefully making things clearer when seeing that part of the code.\n>>>>>>\n>>>>>\n>>>>> I also think we need to improve the comment in the world-switch return\n>>>>> path, and I'd like Alex to weigh in here before we merge this. He's\n>>>>> back from holiday on Monday.\n>>>>>\n>>>>\n>>>> Ping Alex?\n>>>\n>>> Sorry for the delay getting back to you. I had flagged the email but\n>>> with holidays and conferences in the way it fell off my queue.\n>>>\n>>\n>> No problem, thanks for looking at it.\n>>\n>>> So to summarise as I understand things:\n>>>\n>>> Host User Space | Host KVM | Host Hyp | Guest VM |\n>>>\n>>> Enable Debug(SS)\n>>> KVM_RUN ----------->\n>>> Guest SPSR.SS set\n>>> --> World Switch ->\n>>> Insn Trap to Hyp\n>>> World Switch <-\n>>> (SS not cleared)\n>>> <--\n>>> Insn Emulated\n>>> pc += 4\n>>> -->\n>>> World Switch\n>>> (SS still set)\n>>> ->\n>>> Insn +4 SS\n>>> <-\n>>> World Switch\n>>> (SS cleared)\n>>>\n>>> <--\n>>> Guest exit (debug)\n>>> <--\n>>> See SS did 2 insns?\n>>>\n>>> Do I understand the problem you are trying to fix correctly?\n>>\n>> Yes that's the issue. The debugger is not made aware of the\n>> emulated/skipped instruction and the hypervisor jumps back into the\n>> guest.\n>>\n>> Clearing SS before jumping back to the guest will simply trigger a\n>> debug exception as soon as we ERET from EL2 to EL1 (so we end up just\n>> getting back to EL2).\n> \n> Why don't we just exit KVM after we've emulated the instruction if we\n> are under debug? After all at this point whatever needed to be done is\n> done and the guest debug code can get on with life.\n> \n> I understand there is the problem of exiting for an MMIO emulation but\n> maybe that complexity should be handled by userspace (\"render unto\n> userspace the things that are userspaces\") and it can decide to lift the\n> step ioctl if appropriate.\n> \n\nI think the issue was that there was no clear way on how to let userland \nknow that it has to deal with both MMIO emulation + debug exception in \none return.\nI guess currently userland relies on having KVM_EXIT_DEBUG exit code to \ninform the debugger. I'm not well aware how MMIO works, so maybe the \nuserland being aware it has set single step it should also deal with the \ndebug case when it perfoms MMIO.\n\nHowever, KVM_EXIT_MMIO code seem to be part of the arch independent \ninterface, so I guess it should have the same semantic on all platforms? \n(and avoid having one where it means \"do MMIO\" and another where it \nmeans \"do MMIO + debug step if needed\")\n\nKVM doc states:\n\n> NOTE: For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_PAPR and\n> KVM_EXIT_EPR the corresponding\n> operations are complete (and guest state is consistent) only after userspace\n> has re-entered the kernel with KVM_RUN. The kernel side will first finish\n> incomplete operations and then check for pending signals. Userspace\n> can re-enter the guest with an unmasked signal pending to complete\n> pending operations.\n\nSince the pausing is supposed to take place after the instruction has \ncompleted, I'm not sure we can expect userland to MMIO + debug \nemulation. But maybe I'm not interpreting this correctly.\n\nOtherwise I can have a look at what other arch expect from userland in \nsuch scenario.\n\n> I guess I should have a look at the series. Are you re-basing anytime\n> soon? It looks like it currently has a few minor merge conflicts with\n> current master.\n> \n\nYes, haven't updated it since v4.13, I guess I can repost it rebased \nsome 4.14-rc.\n\nThanks,","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"Nv84ICEC\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y642F1rpGz9t2x\n\tfor ;\n\tWed, 4 Oct 2017 03:17:57 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzPtJ-0006SP-P9; Tue, 03 Oct 2017 16:17:53 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzPtF-0006Lz-PL for linux-arm-kernel@lists.infradead.org;\n\tTue, 03 Oct 2017 16:17:51 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7FBAE1596;\n\tTue, 3 Oct 2017 09:17:29 -0700 (PDT)","from [10.1.206.253] (e112298-lin.cambridge.arm.com [10.1.206.253])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t519263F578; Tue, 3 Oct 2017 09:17:26 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=vvLQRf9XfnHDoGB9VwGpj7pVJvX0HRSCSIM346oJcQA=;\n\tb=Nv84ICECycMjCpmNZUIGBq5dR\n\tBCzc8adEi0HNbjOUp4TEZSpm1f4zZN4EjkTCto7Wtc7vzCL7AAEQ1DJqEhtK7rsp1/tD+f/9xCHym\n\tBITRSmAfxWv2gxq5s2ylNsxAW3NbU2clYKu645UqSImU5K1C7Z7ZRpOmQHk6r5q3NPuoAA3tFQEc/\n\tq0ISAUP/QmLYxoQHj28vJ0gAf3+QlO275Qmjp0MiQ8lEQ4oNyWNvjbskVLsNNP+h4sPvUlt9JBOVm\n\t4YDWFwrnE+PLEndKtKUfO8cmjY3jb9POD1gxt3iIyXH8wDQ5NwITBA9EKZEgRYgnMj2SebtXYK0BR\n\tv3kYLAHbw==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"=?utf-8?q?Alex_Benn=C3=A9e?= ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>\n\t<874lrgz0yf.fsf@linaro.org>","From":"Julien Thierry ","Message-ID":"<37d50b72-58cb-633e-552e-96d5b528b5ba@arm.com>","Date":"Tue, 3 Oct 2017 17:17:24 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<874lrgz0yf.fsf@linaro.org>","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171003_091749_846131_7D30AB61 ","X-CRM114-Status":"GOOD ( 23.62 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, \n\tChristoffer Dall ","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779130,"web_url":"http://patchwork.ozlabs.org/comment/1779130/","msgid":"<873770yz1o.fsf@linaro.org>","list_archive_url":null,"date":"2017-10-03T16:30:11","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":39532,"url":"http://patchwork.ozlabs.org/api/people/39532/","name":"Alex Bennée","email":"alex.bennee@linaro.org"},"content":"Julien Thierry writes:\n\n> On 03/10/17 15:57, Alex Bennée wrote:\n>>\n>> Julien Thierry writes:\n>>\n>>> On 31/08/17 15:01, Christoffer Dall wrote:\n\n>>>>>>>\n>>>>>>> Does that sound like what you had in mind? Or does it seem better than\n>>>>>>> the\n>>>>>>> current patch?\n>>>>>>>\n>>>>>> I was thinking to change the skip_instruction function to return an\n>>>>>> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n>>>>>> would update the kvm_run structure and exit here and then.\n>>>>>>\n>>>>>\n>>>>> Setting up the debug exception from within kvm_skip_instruction seem to\n>>>>> change a bit too much its semantic from arm to arm64. I would find this\n>>>>> easily confusing.\n>>>>>\n>>>>>> However, I'm now thinking that this doesn't really work either,\n>>>>>> because we could have to emulate a trapped MMIO instruction in user\n>>>>>> space, and then it's not clear how to exit with a debug exception at\n>>>>>> the same time.\n>>\n>> A debug exception at guest exit point is (IIRC) just having the\n>> appropriate status in the run->exit_reason (KVM_EXIT_DEBUG). If you need\n>> to exit for MMIO emulation (i.e. the instruction has not run yet) you\n>> shouldn't do that. Exit, emulate and return. We could handle the ioctl\n>> to clear SS in userspace but I guess that gets just as messy.\n>>\n>>>>>>\n>>>>>> So perhaps we should stick with your original approach.\n>>>>>>\n>>>>>\n>>>>> I had not realized that was possible. This makes things more complicated for\n>>>>> avoiding a back and forth with the guest for trapped exceptions. Out of\n>>>>> luck, having the debug flag does look like single stepping would work as\n>>>>> expected for userland MMIOs.\n\n\n\nThis is my currently untested but otherwise simpler solution:\n\nFrom 46ea80d7dc9b98661fcd51c41090f8ad74a6690f Mon Sep 17 00:00:00 2001\nFrom: =?UTF-8?q?Alex=20Benn=C3=A9e?= \nDate: Tue, 3 Oct 2017 17:17:15 +0100\nSubject: [PATCH] KVM: arm64: handle single-stepping trapped instructions\nMIME-Version: 1.0\nContent-Type: text/plain; charset=UTF-8\nContent-Transfer-Encoding: 8bit\n\nIf we are using guest debug to single-step the guest we need to ensure\nwe exit after emulating the instruction. This only affects\ninstructions emulated by the kernel. If we exit to userspace anyway we\nleave it to userspace to work out what to do.\n\nSigned-off-by: Alex Bennée \n---\n arch/arm64/kvm/handle_exit.c | 51 ++++++++++++++++++++++++++++++++------------\n 1 file changed, 37 insertions(+), 14 deletions(-)\n\ndiff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c\nindex 7debb74843a0..b197ffb10e96 100644\n--- a/arch/arm64/kvm/handle_exit.c\n+++ b/arch/arm64/kvm/handle_exit.c\n@@ -178,6 +178,42 @@ static exit_handle_fn kvm_get_exit_handler(struct kvm_vcpu *vcpu)\n \treturn arm_exit_handlers[hsr_ec];\n }\n\n+/*\n+ * When handling traps we need to ensure exit the guest if we\n+ * successfully emulated the instruction while single-stepping. If we\n+ * have to exit anyway for userspace emulation then it's up to\n+ * userspace to handle the \"while SSing case\".\n+ */\n+\n+static int handle_trap_exceptions(struct kvm_vcpu *vcpu, struct kvm_run *run)\n+{\n+\tint handled;\n+\n+\t/*\n+\t * See ARM ARM B1.14.1: \"Hyp traps on instructions\n+\t * that fail their condition code check\"\n+\t */\n+\tif (!kvm_condition_valid(vcpu)) {\n+\t\tkvm_skip_instr(vcpu, kvm_vcpu_trap_il_is32bit(vcpu));\n+\t\thandled = 1;\n+\t} else {\n+\t\texit_handle_fn exit_handler;\n+\n+\t\texit_handler = kvm_get_exit_handler(vcpu);\n+\t\thandled = exit_handler(vcpu, run);\n+\t}\n+\n+\tif (handled && (vcpu->debug_flags & KVM_GUESTDBG_SINGLESTEP)) {\n+\t\tu32 hsr = kvm_vcpu_get_hsr(vcpu);\n+\n+\t\thandled = 0;\n+\t\trun->exit_reason = KVM_EXIT_DEBUG;\n+\t\trun->debug.arch.hsr = hsr;\n+\t}\n+\n+\treturn handled;\n+}\n+\n /*\n * Return > 0 to return to guest, < 0 on error, 0 (and set exit_reason) on\n * proper exit to userspace.\n@@ -185,8 +221,6 @@ static exit_handle_fn kvm_get_exit_handler(struct kvm_vcpu *vcpu)\n int handle_exit(struct kvm_vcpu *vcpu, struct kvm_run *run,\n \t\t int exception_index)\n {\n-\texit_handle_fn exit_handler;\n-\n \tif (ARM_SERROR_PENDING(exception_index)) {\n \t\tu8 hsr_ec = ESR_ELx_EC(kvm_vcpu_get_hsr(vcpu));\n\n@@ -214,18 +248,7 @@ int handle_exit(struct kvm_vcpu *vcpu, struct kvm_run *run,\n \t\tkvm_inject_vabt(vcpu);\n \t\treturn 1;\n \tcase ARM_EXCEPTION_TRAP:\n-\t\t/*\n-\t\t * See ARM ARM B1.14.1: \"Hyp traps on instructions\n-\t\t * that fail their condition code check\"\n-\t\t */\n-\t\tif (!kvm_condition_valid(vcpu)) {\n-\t\t\tkvm_skip_instr(vcpu, kvm_vcpu_trap_il_is32bit(vcpu));\n-\t\t\treturn 1;\n-\t\t}\n-\n-\t\texit_handler = kvm_get_exit_handler(vcpu);\n-\n-\t\treturn exit_handler(vcpu, run);\n+\t\treturn handle_trap_exceptions(vcpu, run);\n \tcase ARM_EXCEPTION_HYP_GONE:\n \t\t/*\n \t\t * EL2 has been reset to the hyp-stub. This happens when a guest\n--\n2.14.1","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"pZVsWDiM\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"Rs6ZKP5W\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y64Jy2lkgz9t43\n\tfor ;\n\tWed, 4 Oct 2017 03:30:42 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzQ5f-0006IU-7D; Tue, 03 Oct 2017 16:30:39 +0000","from mail-wm0-x22a.google.com ([2a00:1450:400c:c09::22a])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzQ5b-0006FA-0n for linux-arm-kernel@lists.infradead.org;\n\tTue, 03 Oct 2017 16:30:37 +0000","by mail-wm0-x22a.google.com with SMTP id 196so6608434wma.1\n\tfor ;\n\tTue, 03 Oct 2017 09:30:14 -0700 (PDT)","from zen.linaro.local ([81.128.185.34])\n\tby smtp.gmail.com with ESMTPSA id\n\tr123sm3056579wmf.12.2017.10.03.09.30.11\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tTue, 03 Oct 2017 09:30:11 -0700 (PDT)","from zen (localhost [127.0.0.1])\n\tby zen.linaro.local (Postfix) with ESMTPS id 1D87A3E1337;\n\tTue, 3 Oct 2017 17:30:11 +0100 (BST)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:\n\tIn-reply-to:Subject:To:From:References:Reply-To:Content-ID:\n\tContent-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc\n\t:Resent-Message-ID:List-Owner;\n\tbh=fhjSrn7Tz1Rqo+QYxrZYVRYIV+rsrB0VgF/qe3UYOas=;\n\tb=pZVsWDiM00XOgs82bdgJGl1XOy\n\tmGjxq5ogLfci1otNrruRna37yVFOIv+LXgAKw9VklBz9vemdFRXg+OvNU6xGPCqtnS+mhqTSQ8Wdv\n\tPSX3Sygbs+ErHDKDTJTwDiZsLNG/jtDe+aK7YiW1CLVh/Kab/HSdsW0NJg+vQl3t1hzxdwyk/7ZNb\n\tcqQT72UkzhjPEANcUnCJxkxI0crxW6+gijl2N6dKuiSy/t8N3CKCtnHZKJ4qAVOP3KPfZCeAedVZD\n\tAiQPKcHkcNzglXZQBaVXofcvngw1Y1Hfs/F7ioWD4JlP8pypQWjUNGxMqN9c2gwQW8K2z1nsw/3hz\n\tkY1QVaCQ==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=references:user-agent:from:to:cc:subject:in-reply-to:date\n\t:message-id:mime-version:content-transfer-encoding;\n\tbh=CXCNAbZfvhbxKaI+xJQAsq+K0bWLXRcJztPebhAiaUg=;\n\tb=Rs6ZKP5WNCx1awgSiXEJXTNPOVLnu7JolaXi57sbp9eHxs1lEZgQGdEuExGr3uW+5+\n\tvNFn8+Qrd4Ure0ZqUdmEvGbeb5uJyyxMY6Pg7qZI1KVNpd4tktQ5F2H0yUQsaFJ/TgSB\n\tjINvxspYml7UucIHMS6i0OGRT/vb6RILbH1Vc="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:references:user-agent:from:to:cc:subject\n\t:in-reply-to:date:message-id:mime-version:content-transfer-encoding; \n\tbh=CXCNAbZfvhbxKaI+xJQAsq+K0bWLXRcJztPebhAiaUg=;\n\tb=nklm5j+gwKTL8BHf5p2e6GsI3E4y18Qmq8XHD2ob7hm3swA9X2cRl5c6l30Z9brz7/\n\tfTUB8g1CWxOpqjzgahUdBTBGH4uafTkd8XHyic2ksZYq2gEXp8Y/DctlBPGZXl5cjb5z\n\t4JsjsfQ0vrblhbeX8+VYf9AYBTDSn8QRJKjDWRzTnhrGCWOglDud8DoNrefLRNGk9iwk\n\tTiUp8BtonuRKP1kdazANUF2TKJdLdqzkhxK4mC5rOWRaXW/4BB8wccAzpK2RgNTYHIHt\n\t1w3kUpaHs+UB7dn4sefmiM7kYtkHBQSCLr7OeMi64g+sg4D4PZPcN9Wl0qlG89997Wop\n\thnkA==","X-Gm-Message-State":"AHPjjUiUgpTdDQF4ke0w68nOc8arso78KRFI3nqlrElVOGVdlwIQGohi\n\tJ4+B1aHN21M6umw2oxpWF07M0GrSLlY=","X-Google-Smtp-Source":"AOwi7QB3Vwela42YAWwafz2yvOjA6MrNwl6cAsC+v2JGbmoYqXfsHDWkxu8/Kg2SpK6obxBvXBlvGA==","X-Received":"by 10.28.24.7 with SMTP id 7mr15111629wmy.78.1507048212742;\n\tTue, 03 Oct 2017 09:30:12 -0700 (PDT)","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>","User-agent":"mu4e 0.9.19; emacs 26.0.60","From":"Alex =?utf-8?q?Benn=C3=A9e?= ","To":"Julien Thierry ","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","In-reply-to":"<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>","Date":"Tue, 03 Oct 2017 17:30:11 +0100","Message-ID":"<873770yz1o.fsf@linaro.org>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171003_093035_228086_BEB88C02 ","X-CRM114-Status":"GOOD ( 22.69 )","X-Spam-Score":"-2.7 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/,\n\tlow\n\ttrust [2a00:1450:400c:c09:0:0:0:22a listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, \n\tChristoffer Dall ","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779148,"web_url":"http://patchwork.ozlabs.org/comment/1779148/","msgid":"<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>","list_archive_url":null,"date":"2017-10-03T17:08:43","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":72256,"url":"http://patchwork.ozlabs.org/api/people/72256/","name":"Julien Thierry","email":"julien.thierry@arm.com"},"content":"On 03/10/17 17:30, Alex Bennée wrote:\n> \n> Julien Thierry writes:\n> \n>> On 03/10/17 15:57, Alex Bennée wrote:\n>>>\n>>> Julien Thierry writes:\n>>>\n>>>> On 31/08/17 15:01, Christoffer Dall wrote:\n> \n>>>>>>>>\n>>>>>>>> Does that sound like what you had in mind? Or does it seem better than\n>>>>>>>> the\n>>>>>>>> current patch?\n>>>>>>>>\n>>>>>>> I was thinking to change the skip_instruction function to return an\n>>>>>>> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n>>>>>>> would update the kvm_run structure and exit here and then.\n>>>>>>>\n>>>>>>\n>>>>>> Setting up the debug exception from within kvm_skip_instruction seem to\n>>>>>> change a bit too much its semantic from arm to arm64. I would find this\n>>>>>> easily confusing.\n>>>>>>\n>>>>>>> However, I'm now thinking that this doesn't really work either,\n>>>>>>> because we could have to emulate a trapped MMIO instruction in user\n>>>>>>> space, and then it's not clear how to exit with a debug exception at\n>>>>>>> the same time.\n>>>\n>>> A debug exception at guest exit point is (IIRC) just having the\n>>> appropriate status in the run->exit_reason (KVM_EXIT_DEBUG). If you need\n>>> to exit for MMIO emulation (i.e. the instruction has not run yet) you\n>>> shouldn't do that. Exit, emulate and return. We could handle the ioctl\n>>> to clear SS in userspace but I guess that gets just as messy.\n>>>\n>>>>>>>\n>>>>>>> So perhaps we should stick with your original approach.\n>>>>>>>\n>>>>>>\n>>>>>> I had not realized that was possible. This makes things more complicated for\n>>>>>> avoiding a back and forth with the guest for trapped exceptions. Out of\n>>>>>> luck, having the debug flag does look like single stepping would work as\n>>>>>> expected for userland MMIOs.\n> \n> \n> \n> This is my currently untested but otherwise simpler solution:\n> \n> From 46ea80d7dc9b98661fcd51c41090f8ad74a6690f Mon Sep 17 00:00:00 2001\n> From: =?UTF-8?q?Alex=20Benn=C3=A9e?= \n> Date: Tue, 3 Oct 2017 17:17:15 +0100\n> Subject: [PATCH] KVM: arm64: handle single-stepping trapped instructions\n> MIME-Version: 1.0\n> Content-Type: text/plain; charset=UTF-8\n> Content-Transfer-Encoding: 8bit\n> \n> If we are using guest debug to single-step the guest we need to ensure\n> we exit after emulating the instruction. This only affects\n> instructions emulated by the kernel. If we exit to userspace anyway we\n> leave it to userspace to work out what to do.\n> \n> Signed-off-by: Alex Bennée \n> ---\n> arch/arm64/kvm/handle_exit.c | 51 ++++++++++++++++++++++++++++++++------------\n> 1 file changed, 37 insertions(+), 14 deletions(-)\n> \n> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c\n> index 7debb74843a0..b197ffb10e96 100644\n> --- a/arch/arm64/kvm/handle_exit.c\n> +++ b/arch/arm64/kvm/handle_exit.c\n> @@ -178,6 +178,42 @@ static exit_handle_fn kvm_get_exit_handler(struct kvm_vcpu *vcpu)\n> \treturn arm_exit_handlers[hsr_ec];\n> }\n> \n> +/*\n> + * When handling traps we need to ensure exit the guest if we\n> + * successfully emulated the instruction while single-stepping. If we\n> + * have to exit anyway for userspace emulation then it's up to\n> + * userspace to handle the \"while SSing case\".\n> + */\n> +\n\nI have not tested the code but if it work we also need to do something \nsimilar for MMIOs that are handled by the kernel (without returning to \nuserland). But it should be pretty similar.\n\n> +static int handle_trap_exceptions(struct kvm_vcpu *vcpu, struct kvm_run *run)\n> +{\n> +\tint handled;\n> +\n> +\t/*\n> +\t * See ARM ARM B1.14.1: \"Hyp traps on instructions\n> +\t * that fail their condition code check\"\n> +\t */\n> +\tif (!kvm_condition_valid(vcpu)) {\n> +\t\tkvm_skip_instr(vcpu, kvm_vcpu_trap_il_is32bit(vcpu));\n> +\t\thandled = 1;\n> +\t} else {\n> +\t\texit_handle_fn exit_handler;\n> +\n> +\t\texit_handler = kvm_get_exit_handler(vcpu);\n> +\t\thandled = exit_handler(vcpu, run);\n> +\t}\n> +\n> +\tif (handled && (vcpu->debug_flags & KVM_GUESTDBG_SINGLESTEP)) {\n> +\t\tu32 hsr = kvm_vcpu_get_hsr(vcpu);\n> +\n> +\t\thandled = 0;\n> +\t\trun->exit_reason = KVM_EXIT_DEBUG;\n> +\t\trun->debug.arch.hsr = hsr;\n> +\t}\n> +\n> +\treturn handled;\n> +}\n> +\n> /*\n> * Return > 0 to return to guest, < 0 on error, 0 (and set exit_reason) on\n> * proper exit to userspace.\n> @@ -185,8 +221,6 @@ static exit_handle_fn kvm_get_exit_handler(struct kvm_vcpu *vcpu)\n> int handle_exit(struct kvm_vcpu *vcpu, struct kvm_run *run,\n> \t\t int exception_index)\n> {\n> -\texit_handle_fn exit_handler;\n> -\n> \tif (ARM_SERROR_PENDING(exception_index)) {\n> \t\tu8 hsr_ec = ESR_ELx_EC(kvm_vcpu_get_hsr(vcpu));\n> \n> @@ -214,18 +248,7 @@ int handle_exit(struct kvm_vcpu *vcpu, struct kvm_run *run,\n> \t\tkvm_inject_vabt(vcpu);\n> \t\treturn 1;\n> \tcase ARM_EXCEPTION_TRAP:\n> -\t\t/*\n> -\t\t * See ARM ARM B1.14.1: \"Hyp traps on instructions\n> -\t\t * that fail their condition code check\"\n> -\t\t */\n> -\t\tif (!kvm_condition_valid(vcpu)) {\n> -\t\t\tkvm_skip_instr(vcpu, kvm_vcpu_trap_il_is32bit(vcpu));\n> -\t\t\treturn 1;\n> -\t\t}\n> -\n> -\t\texit_handler = kvm_get_exit_handler(vcpu);\n> -\n> -\t\treturn exit_handler(vcpu, run);\n> +\t\treturn handle_trap_exceptions(vcpu, run);\n> \tcase ARM_EXCEPTION_HYP_GONE:\n> \t\t/*\n> \t\t * EL2 has been reset to the hyp-stub. This happens when a guest\n> --\n> 2.14.1\n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"YeN83wkt\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y659P59jNz9ryr\n\tfor ;\n\tWed, 4 Oct 2017 04:09:13 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzQgv-0007Hj-Mp; Tue, 03 Oct 2017 17:09:09 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzQgs-000742-51 for linux-arm-kernel@lists.infradead.org;\n\tTue, 03 Oct 2017 17:09:08 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0D66680D;\n\tTue, 3 Oct 2017 10:08:46 -0700 (PDT)","from [10.1.206.253] (e112298-lin.cambridge.arm.com [10.1.206.253])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tED5A33F578; Tue, 3 Oct 2017 10:08:44 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=dv8E+Nkd+Erv4yDno3krPnOHL2RHg1Rj1hIxl+9zjIY=;\n\tb=YeN83wktlqhw1rSE3ZxoOBGlX\n\tbGtWINkq8ts8Otz7cQ23EQhtzOj2PIqZsNaASf29SLKRghNBlnx5uAm2MshC/BC1iDnvgTFKLLfTv\n\t0bAnQMojSHwlMRkNQbtoRiiQPOb74x3ci0/jJ9xkLnbuv+qflUZznU8OWG53vaZ/sUavZkNGnic7x\n\tr2AeKRnO86Z4xnrfroC+EZvlIis0SwreFiIJVRLVbLUAuND4hv+GKVFizVUz2RQ+Nm2yb7WIUQzZH\n\totTfyNKrovB6iuh6cvj71qw1bBcNxEyRjB4t7XwxJKWiipmrCGkXFfOyv7jHlZrtadz8cAQ0ke2X1\n\thk9Z+FJmg==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"=?utf-8?q?Alex_Benn=C3=A9e?= ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>\n\t<873770yz1o.fsf@linaro.org>","From":"Julien Thierry ","Message-ID":"<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>","Date":"Tue, 3 Oct 2017 18:08:43 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<873770yz1o.fsf@linaro.org>","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171003_100906_220032_EB0E9B89 ","X-CRM114-Status":"GOOD ( 27.90 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, \n\tChristoffer Dall ","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779161,"web_url":"http://patchwork.ozlabs.org/comment/1779161/","msgid":"<871smkywgn.fsf@linaro.org>","list_archive_url":null,"date":"2017-10-03T17:26:00","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":39532,"url":"http://patchwork.ozlabs.org/api/people/39532/","name":"Alex Bennée","email":"alex.bennee@linaro.org"},"content":"Julien Thierry writes:\n\n> On 03/10/17 17:30, Alex Bennée wrote:\n>>\n>> Julien Thierry writes:\n>>\n>>> On 03/10/17 15:57, Alex Bennée wrote:\n>>>>\n>>>> Julien Thierry writes:\n>>>>\n>>>>> On 31/08/17 15:01, Christoffer Dall wrote:\n>> \n>>>>>>>>>\n>>>>>>>>> Does that sound like what you had in mind? Or does it seem better than\n>>>>>>>>> the\n>>>>>>>>> current patch?\n>>>>>>>>>\n>>>>>>>> I was thinking to change the skip_instruction function to return an\n>>>>>>>> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n>>>>>>>> would update the kvm_run structure and exit here and then.\n>>>>>>>>\n>>>>>>>\n>>>>>>> Setting up the debug exception from within kvm_skip_instruction seem to\n>>>>>>> change a bit too much its semantic from arm to arm64. I would find this\n>>>>>>> easily confusing.\n>>>>>>>\n>>>>>>>> However, I'm now thinking that this doesn't really work either,\n>>>>>>>> because we could have to emulate a trapped MMIO instruction in user\n>>>>>>>> space, and then it's not clear how to exit with a debug exception at\n>>>>>>>> the same time.\n>>>>\n>>>> A debug exception at guest exit point is (IIRC) just having the\n>>>> appropriate status in the run->exit_reason (KVM_EXIT_DEBUG). If you need\n>>>> to exit for MMIO emulation (i.e. the instruction has not run yet) you\n>>>> shouldn't do that. Exit, emulate and return. We could handle the ioctl\n>>>> to clear SS in userspace but I guess that gets just as messy.\n>>>>\n>>>>>>>>\n>>>>>>>> So perhaps we should stick with your original approach.\n>>>>>>>>\n>>>>>>>\n>>>>>>> I had not realized that was possible. This makes things more complicated for\n>>>>>>> avoiding a back and forth with the guest for trapped exceptions. Out of\n>>>>>>> luck, having the debug flag does look like single stepping would work as\n>>>>>>> expected for userland MMIOs.\n>>\n>> \n>>\n>> This is my currently untested but otherwise simpler solution:\n>>\n>> From 46ea80d7dc9b98661fcd51c41090f8ad74a6690f Mon Sep 17 00:00:00 2001\n>> From: =?UTF-8?q?Alex=20Benn=C3=A9e?= \n>> Date: Tue, 3 Oct 2017 17:17:15 +0100\n>> Subject: [PATCH] KVM: arm64: handle single-stepping trapped instructions\n>> MIME-Version: 1.0\n>> Content-Type: text/plain; charset=UTF-8\n>> Content-Transfer-Encoding: 8bit\n>>\n>> If we are using guest debug to single-step the guest we need to ensure\n>> we exit after emulating the instruction. This only affects\n>> instructions emulated by the kernel. If we exit to userspace anyway we\n>> leave it to userspace to work out what to do.\n>>\n>> Signed-off-by: Alex Bennée \n>> ---\n>> arch/arm64/kvm/handle_exit.c | 51 ++++++++++++++++++++++++++++++++------------\n>> 1 file changed, 37 insertions(+), 14 deletions(-)\n>>\n>> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c\n>> index 7debb74843a0..b197ffb10e96 100644\n>> --- a/arch/arm64/kvm/handle_exit.c\n>> +++ b/arch/arm64/kvm/handle_exit.c\n>> @@ -178,6 +178,42 @@ static exit_handle_fn kvm_get_exit_handler(struct kvm_vcpu *vcpu)\n>> \treturn arm_exit_handlers[hsr_ec];\n>> }\n>>\n>> +/*\n>> + * When handling traps we need to ensure exit the guest if we\n>> + * successfully emulated the instruction while single-stepping. If we\n>> + * have to exit anyway for userspace emulation then it's up to\n>> + * userspace to handle the \"while SSing case\".\n>> + */\n>> +\n>\n> I have not tested the code but if it work we also need to do something\n> similar for MMIOs that are handled by the kernel (without returning to\n> userland). But it should be pretty similar.\n\n\nWhich path do they take to the mmio emulation?\n\n--\nAlex Bennée","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"JeDYiyhV\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"bslwtmGn\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y65YL5kkZz9sxR\n\tfor ;\n\tWed, 4 Oct 2017 04:26:30 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzQxf-0006I9-JE; Tue, 03 Oct 2017 17:26:27 +0000","from mail-wm0-x229.google.com ([2a00:1450:400c:c09::229])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzQxb-0006GR-N0 for linux-arm-kernel@lists.infradead.org;\n\tTue, 03 Oct 2017 17:26:26 +0000","by mail-wm0-x229.google.com with SMTP id m72so18210607wmc.1\n\tfor ;\n\tTue, 03 Oct 2017 10:26:03 -0700 (PDT)","from zen.linaro.local ([81.128.185.34])\n\tby smtp.gmail.com with ESMTPSA id\n\t193sm10336915wmh.47.2017.10.03.10.26.00\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tTue, 03 Oct 2017 10:26:00 -0700 (PDT)","from zen (localhost [127.0.0.1])\n\tby zen.linaro.local (Postfix) with ESMTPS id 7FAA73E1337;\n\tTue, 3 Oct 2017 18:26:00 +0100 (BST)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:\n\tIn-reply-to:Subject:To:From:References:Reply-To:Content-ID:\n\tContent-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc\n\t:Resent-Message-ID:List-Owner;\n\tbh=PypwQPscCiWuPA+wvkv7v6YdO1g1Bo0tDzUl5ghBr+0=;\n\tb=JeDYiyhV1L+PVFW6V6neYnYwvp\n\t5qmGgqYMSmp9kRc5J//RAw2nkhs0w2rAp8eKdzIfGvmzoMo+5e67XcJbJ6UcnQdxa1hVZx5+GN23J\n\t8/Mtw1tYnjnFdkifJTFGnlX2AGpoX4bT5asI9LW5GxgVgb86n7Xg+r2KGEkMdYMN9ZTyKohLHD5jt\n\t/UeSczfU195VSRSF5JuuaAr6YxE839ebxGZrf0MQsYuOhBJIJ2Pn6kMFtQPelSdasFFWuKvyfIrf/\n\tlanJjuFmpLk/oPt6iWGkVjRWSiLKLwoq57UeubSga1qlaSbsskUi9yGMjYyvk/+7RL+WwhJvaUhAT\n\tYhlIC4WA==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=references:user-agent:from:to:cc:subject:in-reply-to:date\n\t:message-id:mime-version:content-transfer-encoding;\n\tbh=OEMsKdENfjbOypQmq9HBVwUYlyEgYRAYxRg2bakLpxo=;\n\tb=bslwtmGnNoGkq24QPGQb7L28JvP2ebWHGTpl2JI/Fl37MPFpoZ//Y26K5GpyENxG6n\n\tLOkbQEsr8wBPpj/O08/sS0rgiudxGp4bz/Ej7sXVXp+V/3ZdI9V5ywt5QjO+RHQXBtmX\n\t7dx8EDlFJMpo3nEAGdcwG5ofiJaRT+YFcjsgA="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:references:user-agent:from:to:cc:subject\n\t:in-reply-to:date:message-id:mime-version:content-transfer-encoding; \n\tbh=OEMsKdENfjbOypQmq9HBVwUYlyEgYRAYxRg2bakLpxo=;\n\tb=EiDDisEcojQQZEZbVNfbU+pnLBgeBRVCPNI9/15higtfKIGSnFE7d+Ngq9KKjQwh+V\n\toE+uB65g+BaIhxxml9BkN5f8NMs3DHuxjFi8iNVCnkEwXSr5/2l2fGw4ITRbBRG0+5zx\n\ttZTemKfUPt3XQX8xRf4J5kdChgFE3XciKtUNrffQ9sxHrQ+T2q55ivZEogWxZmdQexRg\n\tcuNCbHC+QvfGwpSAbYDpD1SCZavTsqpTRwmt+dJbdoLedQHMvWNkxm0VYUA71zR6Jxd9\n\tfNLeUXm8fk1tf0kArkik8ju12blMS1QW/UTBAc5ecwo1b+B8a3B1PKSn53lqy5mWVLOx\n\tbAtw==","X-Gm-Message-State":"AMCzsaWYjwzukjPxmDJS7NKHI1Fq5zkN/HJZSRfyCrt4iflVTCALCqW5\n\twECzUZ2TCv+Lps0a+fJR6EU85w==","X-Google-Smtp-Source":"AOwi7QD3VttQTBjvFUDDQ7b5e3hmNWeb9rjeePPagk2uqk4g6e9NUBOWwP30Z/nlh0gAkAxek31cXA==","X-Received":"by 10.28.90.87 with SMTP id o84mr7869962wmb.132.1507051561824;\n\tTue, 03 Oct 2017 10:26:01 -0700 (PDT)","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>\n\t<873770yz1o.fsf@linaro.org>\n\t<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>","User-agent":"mu4e 0.9.19; emacs 26.0.60","From":"Alex =?utf-8?q?Benn=C3=A9e?= ","To":"Julien Thierry ","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","In-reply-to":"<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>","Date":"Tue, 03 Oct 2017 18:26:00 +0100","Message-ID":"<871smkywgn.fsf@linaro.org>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171003_102623_910700_6C6DC83A ","X-CRM114-Status":"GOOD ( 19.65 )","X-Spam-Score":"-2.7 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/,\n\tlow\n\ttrust [2a00:1450:400c:c09:0:0:0:229 listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, \n\tChristoffer Dall ","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779497,"web_url":"http://patchwork.ozlabs.org/comment/1779497/","msgid":"","list_archive_url":null,"date":"2017-10-04T08:07:56","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":72256,"url":"http://patchwork.ozlabs.org/api/people/72256/","name":"Julien Thierry","email":"julien.thierry@arm.com"},"content":"On 03/10/17 18:26, Alex Bennée wrote:\n> \n> Julien Thierry writes:\n> \n>> On 03/10/17 17:30, Alex Bennée wrote:\n>>>\n>>> Julien Thierry writes:\n>>>\n>>>> On 03/10/17 15:57, Alex Bennée wrote:\n>>>>>\n>>>>> Julien Thierry writes:\n>>>>>\n>>>>>> On 31/08/17 15:01, Christoffer Dall wrote:\n>>> \n>>>>>>>>>>\n>>>>>>>>>> Does that sound like what you had in mind? Or does it seem better than\n>>>>>>>>>> the\n>>>>>>>>>> current patch?\n>>>>>>>>>>\n>>>>>>>>> I was thinking to change the skip_instruction function to return an\n>>>>>>>>> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n>>>>>>>>> would update the kvm_run structure and exit here and then.\n>>>>>>>>>\n>>>>>>>>\n>>>>>>>> Setting up the debug exception from within kvm_skip_instruction seem to\n>>>>>>>> change a bit too much its semantic from arm to arm64. I would find this\n>>>>>>>> easily confusing.\n>>>>>>>>\n>>>>>>>>> However, I'm now thinking that this doesn't really work either,\n>>>>>>>>> because we could have to emulate a trapped MMIO instruction in user\n>>>>>>>>> space, and then it's not clear how to exit with a debug exception at\n>>>>>>>>> the same time.\n>>>>>\n>>>>> A debug exception at guest exit point is (IIRC) just having the\n>>>>> appropriate status in the run->exit_reason (KVM_EXIT_DEBUG). If you need\n>>>>> to exit for MMIO emulation (i.e. the instruction has not run yet) you\n>>>>> shouldn't do that. Exit, emulate and return. We could handle the ioctl\n>>>>> to clear SS in userspace but I guess that gets just as messy.\n>>>>>\n>>>>>>>>>\n>>>>>>>>> So perhaps we should stick with your original approach.\n>>>>>>>>>\n>>>>>>>>\n>>>>>>>> I had not realized that was possible. This makes things more complicated for\n>>>>>>>> avoiding a back and forth with the guest for trapped exceptions. Out of\n>>>>>>>> luck, having the debug flag does look like single stepping would work as\n>>>>>>>> expected for userland MMIOs.\n>>>\n>>> \n>>>\n>>> This is my currently untested but otherwise simpler solution:\n>>>\n>>> From 46ea80d7dc9b98661fcd51c41090f8ad74a6690f Mon Sep 17 00:00:00 2001\n>>> From: =?UTF-8?q?Alex=20Benn=C3=A9e?= \n>>> Date: Tue, 3 Oct 2017 17:17:15 +0100\n>>> Subject: [PATCH] KVM: arm64: handle single-stepping trapped instructions\n>>> MIME-Version: 1.0\n>>> Content-Type: text/plain; charset=UTF-8\n>>> Content-Transfer-Encoding: 8bit\n>>>\n>>> If we are using guest debug to single-step the guest we need to ensure\n>>> we exit after emulating the instruction. This only affects\n>>> instructions emulated by the kernel. If we exit to userspace anyway we\n>>> leave it to userspace to work out what to do.\n>>>\n>>> Signed-off-by: Alex Bennée \n>>> ---\n>>> arch/arm64/kvm/handle_exit.c | 51 ++++++++++++++++++++++++++++++++------------\n>>> 1 file changed, 37 insertions(+), 14 deletions(-)\n>>>\n>>> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c\n>>> index 7debb74843a0..b197ffb10e96 100644\n>>> --- a/arch/arm64/kvm/handle_exit.c\n>>> +++ b/arch/arm64/kvm/handle_exit.c\n>>> @@ -178,6 +178,42 @@ static exit_handle_fn kvm_get_exit_handler(struct kvm_vcpu *vcpu)\n>>> \treturn arm_exit_handlers[hsr_ec];\n>>> }\n>>>\n>>> +/*\n>>> + * When handling traps we need to ensure exit the guest if we\n>>> + * successfully emulated the instruction while single-stepping. If we\n>>> + * have to exit anyway for userspace emulation then it's up to\n>>> + * userspace to handle the \"while SSing case\".\n>>> + */\n>>> +\n>>\n>> I have not tested the code but if it work we also need to do something\n>> similar for MMIOs that are handled by the kernel (without returning to\n>> userland). But it should be pretty similar.\n> \n> \n> Which path do they take to the mmio emulation?\n> \n\nNevermind, I was mistaken, mmio code will get called by exit_handler and \n\"handled\" will be true if the mmio was handled by KVM. So your patch \nalready handles this.\n\nThere is also the case of GIC CPU inteface accesses being trapped (which \nshouldn't be the default behaviour). If the vgic access fails, we will \nskip the instruction (in __kvm_vcpu_run in arch/arm64/kvm/hyp/switch.c) \nand if we were single stepping we will single step 2 instructions. But \nthis seems to be a corner case of a corner case (GIC CPUIF trapped + \naccess failing + single stepping), so I don't know if we want to take \nthat into account right now?\n\nI'm still a bit concerned about the change of semantics for \nKVM_EXIT_MMIO with regards to userland (cf. my previous mail). But if \nthis is deemed to be a reasonable change, the patch seems fine to me.\n\nThanks,","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"Zi5vxsM3\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6T732D9wz9s5L\n\tfor ;\n\tWed, 4 Oct 2017 19:08:31 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzejE-0004TR-9c; Wed, 04 Oct 2017 08:08:28 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzejA-0004Po-Lr for linux-arm-kernel@lists.infradead.org;\n\tWed, 04 Oct 2017 08:08:26 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7023B80D;\n\tWed, 4 Oct 2017 01:08:00 -0700 (PDT)","from [10.1.206.253] (e112298-lin.cambridge.arm.com [10.1.206.253])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tDE6153F58C; Wed, 4 Oct 2017 01:07:58 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=PWUlQY1jj7yXPzcaUH7vRt5fcCO4ZWav+NBR93CNVEs=;\n\tb=Zi5vxsM3y82hl5KnrO2reECWV\n\tK1inwlKPevQDzgAIz2xxaNLc3A4qFqu0RiGmyINfApE7Izbncm3HMB0SFNLm/+JqrQC9Wit6u6uXH\n\tzoX8NdfjJUBOXgJ5EAfagIN56eI75VuyccfH5xLpzBiAkfvFk0yXlcMo1rOqkaKYUPNS/hkGpil2I\n\twVBtK0wuRKqs1ggtXY96dF9p0lu5cr+riOBmYEVynCGLEeSd5ygg1uP1l9qmD8MGq83wm1ApObSlA\n\t34c5q4EvI6iR8Ms5eL33K7cXgNu6keIvGGEsvTXyaLDXk5+f1LrTae2PAvuJd9KIkKmMwJF4qZ9fT\n\tb3li0Ou+g==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"=?utf-8?q?Alex_Benn=C3=A9e?= ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>\n\t<873770yz1o.fsf@linaro.org>\n\t<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>\n\t<871smkywgn.fsf@linaro.org>","From":"Julien Thierry ","Message-ID":"","Date":"Wed, 4 Oct 2017 09:07:56 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<871smkywgn.fsf@linaro.org>","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171004_010824_731148_A42F8557 ","X-CRM114-Status":"GOOD ( 21.59 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, \n\tChristoffer Dall ","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779603,"web_url":"http://patchwork.ozlabs.org/comment/1779603/","msgid":"<87k20bxm13.fsf@linaro.org>","list_archive_url":null,"date":"2017-10-04T10:08:56","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":39532,"url":"http://patchwork.ozlabs.org/api/people/39532/","name":"Alex Bennée","email":"alex.bennee@linaro.org"},"content":"[Added Paolo, including QEMU userspace patch]\n\nJulien Thierry writes:\n\n> On 03/10/17 18:26, Alex Bennée wrote:\n>>\n>> Julien Thierry writes:\n>>\n>>> On 03/10/17 17:30, Alex Bennée wrote:\n>>>>\n>>>> Julien Thierry writes:\n>>>>\n>>>>> On 03/10/17 15:57, Alex Bennée wrote:\n>>>>>>\n>>>>>> Julien Thierry writes:\n>>>>>>\n>>>>>>> On 31/08/17 15:01, Christoffer Dall wrote:\n>>>> \n>>>>>>>>>>>\n>>>>>>>>>>> Does that sound like what you had in mind? Or does it seem better than\n>>>>>>>>>>> the\n>>>>>>>>>>> current patch?\n>>>>>>>>>>>\n>>>>>>>>>> I was thinking to change the skip_instruction function to return an\n>>>>>>>>>> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n>>>>>>>>>> would update the kvm_run structure and exit here and then.\n>>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>> Setting up the debug exception from within kvm_skip_instruction seem to\n>>>>>>>>> change a bit too much its semantic from arm to arm64. I would find this\n>>>>>>>>> easily confusing.\n>>>>>>>>>\n>>>>>>>>>> However, I'm now thinking that this doesn't really work either,\n>>>>>>>>>> because we could have to emulate a trapped MMIO instruction in user\n>>>>>>>>>> space, and then it's not clear how to exit with a debug exception at\n>>>>>>>>>> the same time.\n>>>>>>\n>>>>>> A debug exception at guest exit point is (IIRC) just having the\n>>>>>> appropriate status in the run->exit_reason (KVM_EXIT_DEBUG). If you need\n>>>>>> to exit for MMIO emulation (i.e. the instruction has not run yet) you\n>>>>>> shouldn't do that. Exit, emulate and return. We could handle the ioctl\n>>>>>> to clear SS in userspace but I guess that gets just as messy.\n>>>>>>\n>>>>>>>>>>\n>>>>>>>>>> So perhaps we should stick with your original approach.\n>>>>>>>>>>\n>>>>>>>>>\n>>>>>>>>> I had not realized that was possible. This makes things more complicated for\n>>>>>>>>> avoiding a back and forth with the guest for trapped exceptions. Out of\n>>>>>>>>> luck, having the debug flag does look like single stepping would work as\n>>>>>>>>> expected for userland MMIOs.\n>>>>\n>>>> \n>>>>\n>>>> This is my currently untested but otherwise simpler solution:\n>>>>\n>>>> From 46ea80d7dc9b98661fcd51c41090f8ad74a6690f Mon Sep 17 00:00:00 2001\n>>>> From: =?UTF-8?q?Alex=20Benn=C3=A9e?= \n>>>> Date: Tue, 3 Oct 2017 17:17:15 +0100\n>>>> Subject: [PATCH] KVM: arm64: handle single-stepping trapped instructions\n>>>> MIME-Version: 1.0\n>>>> Content-Type: text/plain; charset=UTF-8\n>>>> Content-Transfer-Encoding: 8bit\n>>>>\n>>>> If we are using guest debug to single-step the guest we need to ensure\n>>>> we exit after emulating the instruction. This only affects\n>>>> instructions emulated by the kernel. If we exit to userspace anyway we\n>>>> leave it to userspace to work out what to do.\n>>>>\n>>>> Signed-off-by: Alex Bennée \n>>>> ---\n>>>> arch/arm64/kvm/handle_exit.c | 51 ++++++++++++++++++++++++++++++++------------\n>>>> 1 file changed, 37 insertions(+), 14 deletions(-)\n>>>>\n>>>> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c\n>>>> index 7debb74843a0..b197ffb10e96 100644\n>>>> --- a/arch/arm64/kvm/handle_exit.c\n>>>> +++ b/arch/arm64/kvm/handle_exit.c\n>>>> @@ -178,6 +178,42 @@ static exit_handle_fn kvm_get_exit_handler(struct kvm_vcpu *vcpu)\n>>>> \treturn arm_exit_handlers[hsr_ec];\n>>>> }\n>>>>\n>>>> +/*\n>>>> + * When handling traps we need to ensure exit the guest if we\n>>>> + * successfully emulated the instruction while single-stepping. If we\n>>>> + * have to exit anyway for userspace emulation then it's up to\n>>>> + * userspace to handle the \"while SSing case\".\n>>>> + */\n>>>> +\n>>>\n>>> I have not tested the code but if it work we also need to do something\n>>> similar for MMIOs that are handled by the kernel (without returning to\n>>> userland). But it should be pretty similar.\n>> \n>>\n>> Which path do they take to the mmio emulation?\n>>\n>\n> Nevermind, I was mistaken, mmio code will get called by exit_handler\n> and \"handled\" will be true if the mmio was handled by KVM. So your\n> patch already handles this.\n>\n> There is also the case of GIC CPU inteface accesses being trapped\n> (which shouldn't be the default behaviour). If the vgic access fails,\n> we will skip the instruction (in __kvm_vcpu_run in\n> arch/arm64/kvm/hyp/switch.c) and if we were single stepping we will\n> single step 2 instructions. But this seems to be a corner case of a\n> corner case (GIC CPUIF trapped + access failing + single stepping), so\n> I don't know if we want to take that into account right now?\n\nYeah looking at the hyp code I did wonder if it warranted the complexity\nof adding handling there.\n\n> I'm still a bit concerned about the change of semantics for\n> KVM_EXIT_MMIO with regards to userland (cf. my previous mail). But if\n> this is deemed to be a reasonable change, the patch seems fine to me.\n\nHave we changed the semantics? A normal un-handled by the kernel IO/MMIO\nexit needs to be processed before the single step takes effect. I can't\nspeak for other userspace but I think for QEMU it is as simple as the\npatch bellow. That said it wasn't quite clear where the PC gets updated\nin this path - I think the kernel updates the PC before the\nKVM_EXIT_MMIO in the same path as the internal handling.\n\nI'd like to test these patches on some real life examples. I tried\ntracing over the pl011_write code in a kernel boot but I ran into the\nproblem of el1_irq's occurring as I tried to step through the guest\nkernel. Is this something you've come across? What MMIO accesses have\nyou been using in your testing?\n\nQEMU Patch bellow:\n\nFrom 2e8fcea695a9eca67fbeb331d3104d1d9e7e337a Mon Sep 17 00:00:00 2001\nFrom: =?UTF-8?q?Alex=20Benn=C3=A9e?= \nDate: Wed, 4 Oct 2017 09:49:41 +0000\nSubject: [PATCH] kvm: exit run loop after emulating IO when single stepping\nMIME-Version: 1.0\nContent-Type: text/plain; charset=UTF-8\nContent-Transfer-Encoding: 8bit\n\nIf single-stepping is enabled we should exit the run-loop after\nemulating the access. Otherwise single-stepping across emulated IO\naccesses may skip an instruction.\n\nThis only addresses user-space emulation. Stuff done in kernel-mode\nshould be handled there.\n\nSigned-off-by: Alex Bennée \n---\n accel/kvm/kvm-all.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c\nindex 90c88b517d..85bcb2b0d4 100644\n--- a/accel/kvm/kvm-all.c\n+++ b/accel/kvm/kvm-all.c\n@@ -1940,7 +1940,7 @@ int kvm_cpu_exec(CPUState *cpu)\n run->io.direction,\n run->io.size,\n run->io.count);\n- ret = 0;\n+ ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n break;\n case KVM_EXIT_MMIO:\n DPRINTF(\"handle_mmio\\n\");\n@@ -1950,7 +1950,7 @@ int kvm_cpu_exec(CPUState *cpu)\n run->mmio.data,\n run->mmio.len,\n run->mmio.is_write);\n- ret = 0;\n+ ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n break;\n case KVM_EXIT_IRQ_WINDOW_OPEN:\n DPRINTF(\"irq_window_open\\n\");\n--\n2.14.1\n\n\n\n\n>\n> Thanks,\n\n\n--\nAlex Bennée","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"cgvZNYrp\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"SdFQf9pj\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6Wpg4K58z9sRV\n\tfor ;\n\tWed, 4 Oct 2017 21:09:31 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzgcJ-0003TC-UR; Wed, 04 Oct 2017 10:09:27 +0000","from mail-wr0-x22a.google.com ([2a00:1450:400c:c0c::22a])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzgcC-0003Oa-6E for linux-arm-kernel@lists.infradead.org;\n\tWed, 04 Oct 2017 10:09:26 +0000","by mail-wr0-x22a.google.com with SMTP id j14so8313193wre.8\n\tfor ;\n\tWed, 04 Oct 2017 03:08:59 -0700 (PDT)","from zen.linaro.local ([81.128.185.34])\n\tby smtp.gmail.com with ESMTPSA id\n\tz192sm16794939wmz.28.2017.10.04.03.08.57\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tWed, 04 Oct 2017 03:08:57 -0700 (PDT)","from zen (localhost [127.0.0.1])\n\tby zen.linaro.local (Postfix) with ESMTPS id AC4863E0191;\n\tWed, 4 Oct 2017 11:08:56 +0100 (BST)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:\n\tIn-reply-to:Subject:To:From:References:Reply-To:Content-ID:\n\tContent-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc\n\t:Resent-Message-ID:List-Owner;\n\tbh=XAa5zJgqScKMn7+5NXXzc4WXTNMz4sZMQDtpBhpbMKo=;\n\tb=cgvZNYrpeW8s2X3zCwC/ggFhUl\n\tv4w7MVSLacaBCbiRXg1CO0H0EFt0qBkCw4JwaFsc7Ea1h3yqVtqPvRDPHUozJRWpxdliIE48KXioA\n\tfd4RH9fmOKYSa/W6cWi0ooetNXscbonsPt5xc+LnA+QiM1g/DodWaaarJgkGbo1tELKJhCbwNko/L\n\tbQ4X1OHC5ICJnRg+zMnR1qj5/6qqB8Q9k1gPX3bhhNfmtep6uOWoyYF3DBMul0ZXB62anmSmN5UXb\n\tetKmBeNM/ffPHfSMfy5aHC9pKtLS9MFzSOJAglITxpq8QhTpBa02DZrDeKhR0UlqL0kIqYhmPBTPS\n\tIOGrsIzg==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=references:user-agent:from:to:cc:subject:in-reply-to:date\n\t:message-id:mime-version:content-transfer-encoding;\n\tbh=nWVYu6be4Ki4pmDug5Er7INdQYdPORIcYV7EY08d3tU=;\n\tb=SdFQf9pj0Mznqt+AMkSlPic2wK578RTGSRRe+J+7p3PxL2vFhcAuVFDazvIYO7aG0r\n\tbl1cIg7w8pbT2vIxH7qfGUH6++Qzg5uoIUJReGpHSQNMliFR31DEll8jThRy7wXTaWSY\n\thxHi/bVCigD451uhoC8qrSptfnafZro1ubzSw="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:references:user-agent:from:to:cc:subject\n\t:in-reply-to:date:message-id:mime-version:content-transfer-encoding; \n\tbh=nWVYu6be4Ki4pmDug5Er7INdQYdPORIcYV7EY08d3tU=;\n\tb=DAoIDVa1PDWNJzrtCrMntuwXKh+aDBITNSwwBLPZ2swAnE4itP6+870cQpAQdmeFsM\n\t77pWFzEBrbCgpjlScY9e5VurjbxMwo9oTKWYWw0U8oVRDI1sRI2deGeM0FrPeyG4nkgN\n\tq0vhnmwylGBpTkVyR9tdan0bLSDJgvm4cI67bxj+5McPPLF0T3EJBfkoEnSIwwscPXap\n\t0jwJEH9hJ3BE46fBTeRXIOE/aFQOUbRGUsfXby0nVNyldPbxENFnS9kgqpVrG8wgaU3I\n\tCEcYVQ0CllL/jMIfYLuIs3/Y1yBWuTC3+Ot+BqBAYTAxfYm7AxYoGH2G8LDMFuL8YYs6\n\tlDxA==","X-Gm-Message-State":"AMCzsaUOePXVTM4FhYMKa/wtyfQjxZzmLRVzsc25Pbozx5LwBiIogRIB\n\tlpyfq3m3d6qRM1A1LVY+rG78ww==","X-Google-Smtp-Source":"AOwi7QC5Fnah6j3R3Isr8R/ZU9clWejGJI/cuFDdJFJKMwv2keplpm/9bZ/LsuOKG/O6dMJ44v6ojg==","X-Received":"by 10.223.131.226 with SMTP id 89mr2836421wre.122.1507111738016; \n\tWed, 04 Oct 2017 03:08:58 -0700 (PDT)","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>\n\t<873770yz1o.fsf@linaro.org>\n\t<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>\n\t<871smkywgn.fsf@linaro.org>\n\t","User-agent":"mu4e 0.9.19; emacs 26.0.60","From":"Alex =?utf-8?q?Benn=C3=A9e?= ","To":"Julien Thierry ","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","In-reply-to":"","Date":"Wed, 04 Oct 2017 11:08:56 +0100","Message-ID":"<87k20bxm13.fsf@linaro.org>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171004_030920_541998_8F2DB36E ","X-CRM114-Status":"GOOD ( 31.10 )","X-Spam-Score":"-2.0 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.0 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,\n\tno\n\ttrust [2a00:1450:400c:c0c:0:0:0:22a listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\tChristoffer Dall ,\n\tPaolo Bonzini ,\n\t\"linux-arm-kernel@lists.infradead.org\"\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779639,"web_url":"http://patchwork.ozlabs.org/comment/1779639/","msgid":"<93d7d64e-e32c-0c6b-5d02-68704c1d45ba@redhat.com>","list_archive_url":null,"date":"2017-10-04T10:28:40","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":2701,"url":"http://patchwork.ozlabs.org/api/people/2701/","name":"Paolo Bonzini","email":"pbonzini@redhat.com"},"content":"On 04/10/2017 12:08, Alex Bennée wrote:\n> \n> From 2e8fcea695a9eca67fbeb331d3104d1d9e7e337a Mon Sep 17 00:00:00 2001\n> From: =?UTF-8?q?Alex=20Benn=C3=A9e?= \n> Date: Wed, 4 Oct 2017 09:49:41 +0000\n> Subject: [PATCH] kvm: exit run loop after emulating IO when single stepping\n> MIME-Version: 1.0\n> Content-Type: text/plain; charset=UTF-8\n> Content-Transfer-Encoding: 8bit\n> \n> If single-stepping is enabled we should exit the run-loop after\n> emulating the access. Otherwise single-stepping across emulated IO\n> accesses may skip an instruction.\n> \n> This only addresses user-space emulation. Stuff done in kernel-mode\n> should be handled there.\n> \n> Signed-off-by: Alex Bennée \n> ---\n> accel/kvm/kvm-all.c | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n> \n> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c\n> index 90c88b517d..85bcb2b0d4 100644\n> --- a/accel/kvm/kvm-all.c\n> +++ b/accel/kvm/kvm-all.c\n> @@ -1940,7 +1940,7 @@ int kvm_cpu_exec(CPUState *cpu)\n> run->io.direction,\n> run->io.size,\n> run->io.count);\n> - ret = 0;\n> + ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n> break;\n> case KVM_EXIT_MMIO:\n> DPRINTF(\"handle_mmio\\n\");\n> @@ -1950,7 +1950,7 @@ int kvm_cpu_exec(CPUState *cpu)\n> run->mmio.data,\n> run->mmio.len,\n> run->mmio.is_write);\n> - ret = 0;\n> + ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n> break;\n> case KVM_EXIT_IRQ_WINDOW_OPEN:\n> DPRINTF(\"irq_window_open\\n\");\n\nSinglestep mode doesn't make much sense for KVM. For TCG the purpose is\nto build one-instruction translation blocks, but what would it mean for KVM?\n\nPaolo","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"qmy6DtbZ\"; dkim-atps=neutral","ext-mx02.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx02.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=pbonzini@redhat.com"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6XFV2LkZz9sRm\n\tfor ;\n\tWed, 4 Oct 2017 21:29:18 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzgvQ-0004gU-9v; Wed, 04 Oct 2017 10:29:12 +0000","from mx1.redhat.com ([209.132.183.28])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzgvL-0004bm-Gb for linux-arm-kernel@lists.infradead.org;\n\tWed, 04 Oct 2017 10:29:09 +0000","from smtp.corp.redhat.com\n\t(int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id CBC49820F9;\n\tWed, 4 Oct 2017 10:28:45 +0000 (UTC)","from [10.36.118.3] (unknown [10.36.118.3])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id DD5D360842;\n\tWed, 4 Oct 2017 10:28:43 +0000 (UTC)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=MFlD01MhQwuy3rAbdmwOq7iCZ9aZYH5kR2LV5fGqAhA=;\n\tb=qmy6DtbZaoRZzJ\n\tLamhztFf/vz9R1K/uucqd+SbRCj/ph+prlOi+dCZzOOpZDoscyEYzR6rpDv6dDjCgAxPhA+j/vYUW\n\t9W3GkGGQN7LnmYc8MGyReCLQn8yKn+SYe2XP5ZTjF10dEN2vTn+HwsegsLgbktuw2bv0UJ3eHZHdy\n\twIUPpusjzrJPWVDj/HhV6EeIHcoIV9WZx2k4CRSGWM0B8oCNTpOAytqc7ZqECuNgKXRMzwz6g4VNg\n\thDJhaC9RkW+opoM39s60Vmy0jepvkjI5GVKknFC5T4MVehw7GBVMIjSPPwfMMZwgiYivPNVOLs75D\n\tYUpjJyfqPhKwPBdKkghg==;","DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com CBC49820F9","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"=?utf-8?q?Alex_Benn=C3=A9e?= ,\n\tJulien Thierry ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>\n\t<873770yz1o.fsf@linaro.org>\n\t<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>\n\t<871smkywgn.fsf@linaro.org>\n\t\n\t<87k20bxm13.fsf@linaro.org>","From":"Paolo Bonzini ","Message-ID":"<93d7d64e-e32c-0c6b-5d02-68704c1d45ba@redhat.com>","Date":"Wed, 4 Oct 2017 12:28:40 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<87k20bxm13.fsf@linaro.org>","Content-Language":"en-US","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.13","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.26]); Wed, 04 Oct 2017 10:28:46 +0000 (UTC)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171004_032907_610605_1ECE654C ","X-CRM114-Status":"GOOD ( 16.26 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3)\n\t[209.132.183.28 listed in wl.mailspike.net]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-0.0 SPF_HELO_PASS SPF: HELO matches SPF record\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [209.132.183.28 listed in list.dnswl.org]\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.0 RCVD_IN_MSPIKE_WL Mailspike good senders","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, \n\tChristoffer Dall ","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779653,"web_url":"http://patchwork.ozlabs.org/comment/1779653/","msgid":"<1bdaff65-c784-4698-5c04-bfb1c943e6c0@arm.com>","list_archive_url":null,"date":"2017-10-04T10:42:28","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":72256,"url":"http://patchwork.ozlabs.org/api/people/72256/","name":"Julien Thierry","email":"julien.thierry@arm.com"},"content":"On 04/10/17 11:08, Alex Bennée wrote:\n> \n> [Added Paolo, including QEMU userspace patch]\n> \n> Julien Thierry writes:\n> \n>> On 03/10/17 18:26, Alex Bennée wrote:\n>>>\n>>> Julien Thierry writes:\n>>>\n>>>> On 03/10/17 17:30, Alex Bennée wrote:\n>>>>>\n>>>>> Julien Thierry writes:\n>>>>>\n>>>>>> On 03/10/17 15:57, Alex Bennée wrote:\n>>>>>>>\n>>>>>>> Julien Thierry writes:\n>>>>>>>\n>>>>>>>> On 31/08/17 15:01, Christoffer Dall wrote:\n>>>>> \n>>>>>>>>>>>>\n>>>>>>>>>>>> Does that sound like what you had in mind? Or does it seem better than\n>>>>>>>>>>>> the\n>>>>>>>>>>>> current patch?\n>>>>>>>>>>>>\n>>>>>>>>>>> I was thinking to change the skip_instruction function to return an\n>>>>>>>>>>> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n>>>>>>>>>>> would update the kvm_run structure and exit here and then.\n>>>>>>>>>>>\n>>>>>>>>>>\n>>>>>>>>>> Setting up the debug exception from within kvm_skip_instruction seem to\n>>>>>>>>>> change a bit too much its semantic from arm to arm64. I would find this\n>>>>>>>>>> easily confusing.\n>>>>>>>>>>\n>>>>>>>>>>> However, I'm now thinking that this doesn't really work either,\n>>>>>>>>>>> because we could have to emulate a trapped MMIO instruction in user\n>>>>>>>>>>> space, and then it's not clear how to exit with a debug exception at\n>>>>>>>>>>> the same time.\n>>>>>>>\n>>>>>>> A debug exception at guest exit point is (IIRC) just having the\n>>>>>>> appropriate status in the run->exit_reason (KVM_EXIT_DEBUG). If you need\n>>>>>>> to exit for MMIO emulation (i.e. the instruction has not run yet) you\n>>>>>>> shouldn't do that. Exit, emulate and return. We could handle the ioctl\n>>>>>>> to clear SS in userspace but I guess that gets just as messy.\n>>>>>>>\n>>>>>>>>>>>\n>>>>>>>>>>> So perhaps we should stick with your original approach.\n>>>>>>>>>>>\n>>>>>>>>>>\n>>>>>>>>>> I had not realized that was possible. This makes things more complicated for\n>>>>>>>>>> avoiding a back and forth with the guest for trapped exceptions. Out of\n>>>>>>>>>> luck, having the debug flag does look like single stepping would work as\n>>>>>>>>>> expected for userland MMIOs.\n>>>>>\n>>>>> \n>>\n>> There is also the case of GIC CPU inteface accesses being trapped\n>> (which shouldn't be the default behaviour). If the vgic access fails,\n>> we will skip the instruction (in __kvm_vcpu_run in\n>> arch/arm64/kvm/hyp/switch.c) and if we were single stepping we will\n>> single step 2 instructions. But this seems to be a corner case of a\n>> corner case (GIC CPUIF trapped + access failing + single stepping), so\n>> I don't know if we want to take that into account right now?\n> \n> Yeah looking at the hyp code I did wonder if it warranted the complexity\n> of adding handling there.\n> \n>> I'm still a bit concerned about the change of semantics for\n>> KVM_EXIT_MMIO with regards to userland (cf. my previous mail). But if\n>> this is deemed to be a reasonable change, the patch seems fine to me.\n> \n> Have we changed the semantics? A normal un-handled by the kernel IO/MMIO\n> exit needs to be processed before the single step takes effect. I can't\n> speak for other userspace but I think for QEMU it is as simple as the\n> patch bellow. That said it wasn't quite clear where the PC gets updated\n> in this path - I think the kernel updates the PC before the\n> KVM_EXIT_MMIO in the same path as the internal handling.\n> \n\nWell I'm not sure. The part I am concerned about is:\n> NOTE: For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_PAPR and\n> KVM_EXIT_EPR the corresponding\n> operations are complete (and guest state is consistent) only after userspace\n> has re-entered the kernel with KVM_RUN. The kernel side will first finish\n> incomplete operations and then check for pending signals. Userspace\n> can re-enter the guest with an unmasked signal pending to complete\n> pending operations.\n\n From Documentation/virtual/kvm/api.txt.\n\nThe way I interpret this is that userland should not consider the MMIO \ncomplete before running the vcpu again. If that the case it shouldn't \ntrigger the single step since the instruction is not completely finished.\n\nMaybe I don't interpret this correctly or it is not relevant here. \nAlthough I'd like to understand why.\n\n> I'd like to test these patches on some real life examples. I tried\n> tracing over the pl011_write code in a kernel boot but I ran into the\n> problem of el1_irq's occurring as I tried to step through the guest\n> kernel. Is this something you've come across? What MMIO accesses have\n> you been using in your testing?\n> \n\nI didn't know which MMIOs were handled by userland so I have only tested \ntraps and MMIOs handled by the kernel.\n\nThis sounds like an issue when you are debugging an interruptible \ncontext, an issue Pratyush has been looking at [1]. Are you taking a \nguest interrupt when you try to execute the instruction to be stepped? I \ndon't know what's the status of this patch series. Can you test the \nuserland MMIO in a non-interruptible context?\n\n[1] \nhttp://lists.infradead.org/pipermail/linux-arm-kernel/2017-July/517234.html\n\nThanks,\n\n> QEMU Patch bellow:\n> \n> From 2e8fcea695a9eca67fbeb331d3104d1d9e7e337a Mon Sep 17 00:00:00 2001\n> From: =?UTF-8?q?Alex=20Benn=C3=A9e?= \n> Date: Wed, 4 Oct 2017 09:49:41 +0000\n> Subject: [PATCH] kvm: exit run loop after emulating IO when single stepping\n> MIME-Version: 1.0\n> Content-Type: text/plain; charset=UTF-8\n> Content-Transfer-Encoding: 8bit\n> \n> If single-stepping is enabled we should exit the run-loop after\n> emulating the access. Otherwise single-stepping across emulated IO\n> accesses may skip an instruction.\n> \n> This only addresses user-space emulation. Stuff done in kernel-mode\n> should be handled there.\n> \n> Signed-off-by: Alex Bennée \n> ---\n> accel/kvm/kvm-all.c | 4 ++--\n> 1 file changed, 2 insertions(+), 2 deletions(-)\n> \n> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c\n> index 90c88b517d..85bcb2b0d4 100644\n> --- a/accel/kvm/kvm-all.c\n> +++ b/accel/kvm/kvm-all.c\n> @@ -1940,7 +1940,7 @@ int kvm_cpu_exec(CPUState *cpu)\n> run->io.direction,\n> run->io.size,\n> run->io.count);\n> - ret = 0;\n> + ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n> break;\n> case KVM_EXIT_MMIO:\n> DPRINTF(\"handle_mmio\\n\");\n> @@ -1950,7 +1950,7 @@ int kvm_cpu_exec(CPUState *cpu)\n> run->mmio.data,\n> run->mmio.len,\n> run->mmio.is_write);\n> - ret = 0;\n> + ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n> break;\n> case KVM_EXIT_IRQ_WINDOW_OPEN:\n> DPRINTF(\"irq_window_open\\n\");\n> --\n> 2.14.1\n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"mKnCgsH5\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6XYJ1CNTz9t2l\n\tfor ;\n\tWed, 4 Oct 2017 21:43:00 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzh8h-0003HC-SZ; Wed, 04 Oct 2017 10:42:55 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzh8e-0003Cl-1y for linux-arm-kernel@lists.infradead.org;\n\tWed, 04 Oct 2017 10:42:54 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6E3CC1435;\n\tWed, 4 Oct 2017 03:42:31 -0700 (PDT)","from [10.1.206.253] (e112298-lin.cambridge.arm.com [10.1.206.253])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t36CB53F53D; Wed, 4 Oct 2017 03:42:30 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=D+04um/ADzpxL979MfIUXZ2YZL26CnD+rBg7tov4hWg=;\n\tb=mKnCgsH5gMrfThZmPNk+GGceY\n\t2g0ZtkWRg45EZyGcIl2fE+5YKfNZgYRKrqXSQSYbef5IXH6swp1L6RxQzpSsLUkE2/JzHb79b7caj\n\tYW4t2Dkjf6xHs1CsW5f7IONootApUtonBXEIZopgsLaP28eMNuuyxTTM3cTTiMw6IA/GK1ZG7ygsp\n\tWUu8ztwhjt4LjJbesxRdEPGn8ogtBwoJl2iXA7uc3JoF8N/9oKwp67+L/qqYSJ+jLv7FHcIEMiN7k\n\tk/v4D4DMg7Er137DX09bxZeJ1J8MiSK4Gk/zamPhi83zgZ1YOSJgN5UNJmYgxhi0xudscb6CaXoNw\n\tPHjKiXFWQ==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"=?utf-8?q?Alex_Benn=C3=A9e?= ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>\n\t<873770yz1o.fsf@linaro.org>\n\t<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>\n\t<871smkywgn.fsf@linaro.org>\n\t\n\t<87k20bxm13.fsf@linaro.org>","From":"Julien Thierry ","Message-ID":"<1bdaff65-c784-4698-5c04-bfb1c943e6c0@arm.com>","Date":"Wed, 4 Oct 2017 11:42:28 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<87k20bxm13.fsf@linaro.org>","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171004_034252_123980_B1DD4695 ","X-CRM114-Status":"GOOD ( 34.44 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\tChristoffer Dall ,\n\tPaolo Bonzini ,\n\t\"linux-arm-kernel@lists.infradead.org\"\n\t","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779672,"web_url":"http://patchwork.ozlabs.org/comment/1779672/","msgid":"<87infvxk30.fsf@linaro.org>","list_archive_url":null,"date":"2017-10-04T10:50:59","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":39532,"url":"http://patchwork.ozlabs.org/api/people/39532/","name":"Alex Bennée","email":"alex.bennee@linaro.org"},"content":"Paolo Bonzini writes:\n\n> On 04/10/2017 12:08, Alex Bennée wrote:\n>>\n>> From 2e8fcea695a9eca67fbeb331d3104d1d9e7e337a Mon Sep 17 00:00:00 2001\n>> From: =?UTF-8?q?Alex=20Benn=C3=A9e?= \n>> Date: Wed, 4 Oct 2017 09:49:41 +0000\n>> Subject: [PATCH] kvm: exit run loop after emulating IO when single stepping\n>> MIME-Version: 1.0\n>> Content-Type: text/plain; charset=UTF-8\n>> Content-Transfer-Encoding: 8bit\n>>\n>> If single-stepping is enabled we should exit the run-loop after\n>> emulating the access. Otherwise single-stepping across emulated IO\n>> accesses may skip an instruction.\n>>\n>> This only addresses user-space emulation. Stuff done in kernel-mode\n>> should be handled there.\n>>\n>> Signed-off-by: Alex Bennée \n>> ---\n>> accel/kvm/kvm-all.c | 4 ++--\n>> 1 file changed, 2 insertions(+), 2 deletions(-)\n>>\n>> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c\n>> index 90c88b517d..85bcb2b0d4 100644\n>> --- a/accel/kvm/kvm-all.c\n>> +++ b/accel/kvm/kvm-all.c\n>> @@ -1940,7 +1940,7 @@ int kvm_cpu_exec(CPUState *cpu)\n>> run->io.direction,\n>> run->io.size,\n>> run->io.count);\n>> - ret = 0;\n>> + ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n>> break;\n>> case KVM_EXIT_MMIO:\n>> DPRINTF(\"handle_mmio\\n\");\n>> @@ -1950,7 +1950,7 @@ int kvm_cpu_exec(CPUState *cpu)\n>> run->mmio.data,\n>> run->mmio.len,\n>> run->mmio.is_write);\n>> - ret = 0;\n>> + ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n>> break;\n>> case KVM_EXIT_IRQ_WINDOW_OPEN:\n>> DPRINTF(\"irq_window_open\\n\");\n>\n> Singlestep mode doesn't make much sense for KVM. For TCG the purpose is\n> to build one-instruction translation blocks, but what would it mean for KVM?\n\nIt's used by the kvm_arch_handle_debug() code to verify single-stepping\nis enabled when processing debug exceptions. And also kvm_update_debug:\n\n if (cpu->singlestep_enabled) {\n data.dbg.control |= KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_SINGLESTEP;\n }\n\nWe also have an aliased singlestep_enabled in our disas_context for the\ntranslator.\n\n--\nAlex Bennée","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"Ez0x5qDJ\"; \n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=infradead.org header.i=@infradead.org\n\theader.b=\"Mc9kWKNi\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"XF8DWMRq\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6Xvd6XYGz9t2W\n\tfor ;\n\tWed, 4 Oct 2017 21:58:53 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzhO6-0004uO-6A; Wed, 04 Oct 2017 10:58:50 +0000","from casper.infradead.org ([2001:8b0:10b:1236::1])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzhNs-0004cg-Ay for linux-arm-kernel@bombadil.infradead.org;\n\tWed, 04 Oct 2017 10:58:36 +0000","from mail-wr0-x22a.google.com ([2a00:1450:400c:c0c::22a])\n\tby casper.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzhGt-0005XH-5h for linux-arm-kernel@lists.infradead.org;\n\tWed, 04 Oct 2017 10:51:25 +0000","by mail-wr0-x22a.google.com with SMTP id y44so3011473wrd.4\n\tfor ;\n\tWed, 04 Oct 2017 03:51:02 -0700 (PDT)","from zen.linaro.local ([81.128.185.34])\n\tby smtp.gmail.com with ESMTPSA id\n\ts196sm10048232wmb.6.2017.10.04.03.51.00\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tWed, 04 Oct 2017 03:51:00 -0700 (PDT)","from zen (localhost [127.0.0.1])\n\tby zen.linaro.local (Postfix) with ESMTPS id E39933E0191;\n\tWed, 4 Oct 2017 11:50:59 +0100 (BST)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:\n\tIn-reply-to:Subject:To:From:References:Reply-To:Content-ID:\n\tContent-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc\n\t:Resent-Message-ID:List-Owner;\n\tbh=fGq7zApRcBMDzJweEhCpaDsQwOVV6KOIC8zJv2VMDEk=;\n\tb=Ez0x5qDJu4juJeMfpBy5vH29Bt\n\tmiVIbhaypz9KAwkprJd+UVNhGui8PNeO0p1Ikx+2DvqhSaDYjJW/doL4oXaX/QivoPBP243jGobmi\n\tYz6Gc6QrK6sHnTSEu7UrdC7DZnHRp3dNPw3/DCFwiphpqQVoO4tdNSZDG3R0mpBaOuU3H9yzYra34\n\taL3QUbnu1pYkkavtDBTK07k+jL5SvMAvwjQ72tHOvnB42GdedOcVnBg57N/KxDAi/tm+jUOsyyqP0\n\tMNtXpYviSlGfxuvOrhV1d7DhFivBZnfgKqzDhhA2qKR5PZFo19SLWTiM91A1Ho7fiUTktfrvUr1nh\n\tR3xkzE3Q==;","v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=infradead.org; s=casper.20170209;\n\th=Content-Transfer-Encoding:Content-Type:\n\tMIME-Version:Message-ID:Date:In-reply-to:Subject:Cc:To:From:References:Sender\n\t:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:\n\tResent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:\n\tList-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;\n\tbh=QpR1aYwCajRG+m8rh6u1QdAGuAGUIkFuocUwWEGDpG0=;\n\tb=Mc9kWKNijC+J+DLIonfslh8lc6\n\txWSneM/2fq9kpob5zn1AoGzb3IBmYxfXv59vnh3CxgOlMQqx0EqCF7evGAdSm0vQlxCYXPhiZwWIu\n\tyYmcepgyX3G1uSKanZ/T4o7nPqJx3G8T+egGE5kUotl31HpFp/27+ogHIxZPcIEQAbAXIVI79epb9\n\tIVFzPBo+L7cK4/yQNBK9/gaAXprSlsyEuOqTGq9EKXBdTxURTq+RyPAeXYo+5y/mTog5K1uMJYSa4\n\tc6zhPTYxT6xQX7Rai/6bU5KJn2wmsC3FK35hpAKY/xlIprmaqKGXRyFz9Vlh/igz1Pfy+4MIw4ThV\n\t8PkLYrtg==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=references:user-agent:from:to:cc:subject:in-reply-to:date\n\t:message-id:mime-version:content-transfer-encoding;\n\tbh=QpR1aYwCajRG+m8rh6u1QdAGuAGUIkFuocUwWEGDpG0=;\n\tb=XF8DWMRqpW8UCkeuHE7vyPp1H0D8yaDXuo/SL7/9xsVtMEy9HYMzSNIdeNE0O2Z/GV\n\tX0I8PYYk8ECWzcT4lfUVSc5oVRyP3KX4o1zWJhsNyhbvYrmNPULwtOWlr7agUuXN4zy8\n\taq/zRjxlllWUpeSI/0BMbzEZLjD0XhtOmmYZY="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:references:user-agent:from:to:cc:subject\n\t:in-reply-to:date:message-id:mime-version:content-transfer-encoding; \n\tbh=QpR1aYwCajRG+m8rh6u1QdAGuAGUIkFuocUwWEGDpG0=;\n\tb=ohD58y7Hbc3WggOCOAadBoZPUetSEf5dOXbdagK5YeSuytzz+A88BMPh5zmLqkDo36\n\t3mzinyUXd3ynUJ6C7+hpI/JIAPuHXWHweMDr7aiu6u0EBVg0ZPz2cmbt1m7aRsgfoyCw\n\t+yxymStmX4aDHHLJriGLe3Sq1156232iB+/FLCUz3IQdsDHLDCKHAfwVQSEvHmd5eSL+\n\tBymcf92RrnLcJl8NTggAU4rz5/ZXI0NQXETbXkQXqswIuUtQGXC20awov3ZLWujZDAWE\n\t8baN4CMtj2IxHj6y22caSuqAgg1WtJZNxrt2hrghLbbov9eDpNR+lPU1DSHdp+kL4VpL\n\tlvaQ==","X-Gm-Message-State":"AMCzsaUQFadXCXu6B+ewDtLbrcwhz1sn4Et2GBst2MRK1Yr7MhoSf3U9\n\ttrA12XnKNjdATtiMkI4myR7ZWvIup6I=","X-Google-Smtp-Source":"AOwi7QDdAgPJYRRBGHN/906Cu9+ZccbmK3Qt00LPAKHYcB5AwpxeV1XYBsvChh3QS/dXkElcqFNtWg==","X-Received":"by 10.223.185.84 with SMTP id b20mr8407844wrg.87.1507114261445; \n\tWed, 04 Oct 2017 03:51:01 -0700 (PDT)","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>\n\t<873770yz1o.fsf@linaro.org>\n\t<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>\n\t<871smkywgn.fsf@linaro.org>\n\t\n\t<87k20bxm13.fsf@linaro.org>\n\t<93d7d64e-e32c-0c6b-5d02-68704c1d45ba@redhat.com>","User-agent":"mu4e 0.9.19; emacs 26.0.60","From":"Alex =?utf-8?q?Benn=C3=A9e?= ","To":"Paolo Bonzini ","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","In-reply-to":"<93d7d64e-e32c-0c6b-5d02-68704c1d45ba@redhat.com>","Date":"Wed, 04 Oct 2017 11:50:59 +0100","Message-ID":"<87infvxk30.fsf@linaro.org>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171004_115123_324301_82F0DFB3 ","X-CRM114-Status":"GOOD ( 13.60 )","X-Spam-Score":"-2.0 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on casper.infradead.org summary:\n\tContent analysis details: (-2.0 points, 5.0 required)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,\n\tno\n\ttrust [2a00:1450:400c:c0c:0:0:0:22a listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Julien Thierry ,\n\tMarc Zyngier ,\n\tCatalin Marinas , \n\tWill Deacon ,\n\tChristoffer Dall ,\n\t\"linux-arm-kernel@lists.infradead.org\"\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779870,"web_url":"http://patchwork.ozlabs.org/comment/1779870/","msgid":"<94f821e3-feb7-d4b5-137c-84f7deb897ec@redhat.com>","list_archive_url":null,"date":"2017-10-04T14:19:34","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":2701,"url":"http://patchwork.ozlabs.org/api/people/2701/","name":"Paolo Bonzini","email":"pbonzini@redhat.com"},"content":"On 04/10/2017 12:50, Alex Bennée wrote:\n> \n> Paolo Bonzini writes:\n> \n>> On 04/10/2017 12:08, Alex Bennée wrote:\n>>>\n>>> From 2e8fcea695a9eca67fbeb331d3104d1d9e7e337a Mon Sep 17 00:00:00 2001\n>>> From: =?UTF-8?q?Alex=20Benn=C3=A9e?= \n>>> Date: Wed, 4 Oct 2017 09:49:41 +0000\n>>> Subject: [PATCH] kvm: exit run loop after emulating IO when single stepping\n>>> MIME-Version: 1.0\n>>> Content-Type: text/plain; charset=UTF-8\n>>> Content-Transfer-Encoding: 8bit\n>>>\n>>> If single-stepping is enabled we should exit the run-loop after\n>>> emulating the access. Otherwise single-stepping across emulated IO\n>>> accesses may skip an instruction.\n>>>\n>>> This only addresses user-space emulation. Stuff done in kernel-mode\n>>> should be handled there.\n>>>\n>>> Signed-off-by: Alex Bennée \n>>> ---\n>>> accel/kvm/kvm-all.c | 4 ++--\n>>> 1 file changed, 2 insertions(+), 2 deletions(-)\n>>>\n>>> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c\n>>> index 90c88b517d..85bcb2b0d4 100644\n>>> --- a/accel/kvm/kvm-all.c\n>>> +++ b/accel/kvm/kvm-all.c\n>>> @@ -1940,7 +1940,7 @@ int kvm_cpu_exec(CPUState *cpu)\n>>> run->io.direction,\n>>> run->io.size,\n>>> run->io.count);\n>>> - ret = 0;\n>>> + ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n>>> break;\n>>> case KVM_EXIT_MMIO:\n>>> DPRINTF(\"handle_mmio\\n\");\n>>> @@ -1950,7 +1950,7 @@ int kvm_cpu_exec(CPUState *cpu)\n>>> run->mmio.data,\n>>> run->mmio.len,\n>>> run->mmio.is_write);\n>>> - ret = 0;\n>>> + ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n>>> break;\n>>> case KVM_EXIT_IRQ_WINDOW_OPEN:\n>>> DPRINTF(\"irq_window_open\\n\");\n>>\n>> Singlestep mode doesn't make much sense for KVM. For TCG the purpose is\n>> to build one-instruction translation blocks, but what would it mean for KVM?\n> \n> It's used by the kvm_arch_handle_debug() code to verify single-stepping\n> is enabled when processing debug exceptions. And also kvm_update_debug:\n> \n> if (cpu->singlestep_enabled) {\n> data.dbg.control |= KVM_GUESTDBG_ENABLE | KVM_GUESTDBG_SINGLESTEP;\n> }\n> \n> We also have an aliased singlestep_enabled in our disas_context for the\n> translator.\n\nNevermind, I was confusing cpu->singlestep_enabled with the \"singlestep\"\nglobal.\n\nPaolo","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"pKohXQo2\"; dkim-atps=neutral","ext-mx03.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx03.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=pbonzini@redhat.com"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6dMq3NFxz9sRW\n\tfor ;\n\tThu, 5 Oct 2017 01:20:07 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzkWq-0006bs-36; Wed, 04 Oct 2017 14:20:04 +0000","from mx1.redhat.com ([209.132.183.28])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzkWm-0006N0-9o for linux-arm-kernel@lists.infradead.org;\n\tWed, 04 Oct 2017 14:20:02 +0000","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 80E307C838;\n\tWed, 4 Oct 2017 14:19:39 +0000 (UTC)","from [10.36.118.3] (unknown [10.36.118.3])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 0E42F5D962;\n\tWed, 4 Oct 2017 14:19:36 +0000 (UTC)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=YycXCX9Et7rQVnyoJPS4cX6CbQk11PWLXenxm4NBU8U=;\n\tb=pKohXQo2e1zgOP\n\tSd+4RLxRz0yqPIBGAjsHoue8G/67K6SbIrDc1/LvhVu1SqRBfQeTSqHIjn+JeHvJLfuuUoubf2Dwa\n\tMNbuTiqjEaO/1/6886YL/8PVNbBhqUWHVAm8kLl2m+Rm9DvCgX4v4PkD6vwY6jDljS4Cls3M9wQUQ\n\tFkUazt+NFf0C8yB067S4KojPa6PCOQtQeElvlMi6y4WOG400//jc/jEm9+OCKWH+ZeHpQkmZL77q+\n\trN86eZwJLhjZFV4kCsKzKT5ov0qvI4oVnc/Rk8i962k08p9BNV46465K72011enx1GBApnBqcpET0\n\t5Kr/11s0F0fgbfu65kHA==;","DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 80E307C838","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"=?utf-8?q?Alex_Benn=C3=A9e?= ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>\n\t<873770yz1o.fsf@linaro.org>\n\t<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>\n\t<871smkywgn.fsf@linaro.org>\n\t\n\t<87k20bxm13.fsf@linaro.org>\n\t<93d7d64e-e32c-0c6b-5d02-68704c1d45ba@redhat.com>\n\t<87infvxk30.fsf@linaro.org>","From":"Paolo Bonzini ","Message-ID":"<94f821e3-feb7-d4b5-137c-84f7deb897ec@redhat.com>","Date":"Wed, 4 Oct 2017 16:19:34 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<87infvxk30.fsf@linaro.org>","Content-Language":"en-US","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.14","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.27]); Wed, 04 Oct 2017 14:19:39 +0000 (UTC)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171004_072000_391746_6BFFA1F3 ","X-CRM114-Status":"GOOD ( 15.71 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [209.132.183.28 listed in list.dnswl.org]\n\t-0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3)\n\t[209.132.183.28 listed in wl.mailspike.net]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-0.0 SPF_HELO_PASS SPF: HELO matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.0 RCVD_IN_MSPIKE_WL Mailspike good senders","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Julien Thierry ,\n\tMarc Zyngier ,\n\tCatalin Marinas , \n\tWill Deacon ,\n\tChristoffer Dall ,\n\t\"linux-arm-kernel@lists.infradead.org\"\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779938,"web_url":"http://patchwork.ozlabs.org/comment/1779938/","msgid":"<87efqiyl64.fsf@linaro.org>","list_archive_url":null,"date":"2017-10-04T15:42:11","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":39532,"url":"http://patchwork.ozlabs.org/api/people/39532/","name":"Alex Bennée","email":"alex.bennee@linaro.org"},"content":"Julien Thierry writes:\n\n> On 04/10/17 11:08, Alex Bennée wrote:\n>>\n>> [Added Paolo, including QEMU userspace patch]\n>>\n>> Julien Thierry writes:\n>>\n>>> On 03/10/17 18:26, Alex Bennée wrote:\n>>>>\n>>>> Julien Thierry writes:\n>>>>\n>>>>> On 03/10/17 17:30, Alex Bennée wrote:\n>>>>>>\n>>>>>> Julien Thierry writes:\n>>>>>>\n>>>>>>> On 03/10/17 15:57, Alex Bennée wrote:\n>>>>>>>>\n>>>>>>>> Julien Thierry writes:\n>>>>>>>>\n>>>>>>>>> On 31/08/17 15:01, Christoffer Dall wrote:\n>>>>>> \n>>>>>>>>>>>>>\n>>>>>>>>>>>>> Does that sound like what you had in mind? Or does it seem better than\n>>>>>>>>>>>>> the\n>>>>>>>>>>>>> current patch?\n>>>>>>>>>>>>>\n>>>>>>>>>>>> I was thinking to change the skip_instruction function to return an\n>>>>>>>>>>>> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n>>>>>>>>>>>> would update the kvm_run structure and exit here and then.\n>>>>>>>>>>>>\n>>>>>>>>>>>\n>>>>>>>>>>> Setting up the debug exception from within kvm_skip_instruction seem to\n>>>>>>>>>>> change a bit too much its semantic from arm to arm64. I would find this\n>>>>>>>>>>> easily confusing.\n>>>>>>>>>>>\n>>>>>>>>>>>> However, I'm now thinking that this doesn't really work either,\n>>>>>>>>>>>> because we could have to emulate a trapped MMIO instruction in user\n>>>>>>>>>>>> space, and then it's not clear how to exit with a debug exception at\n>>>>>>>>>>>> the same time.\n>>>>>>>>\n>>>>>>>> A debug exception at guest exit point is (IIRC) just having the\n>>>>>>>> appropriate status in the run->exit_reason (KVM_EXIT_DEBUG). If you need\n>>>>>>>> to exit for MMIO emulation (i.e. the instruction has not run yet) you\n>>>>>>>> shouldn't do that. Exit, emulate and return. We could handle the ioctl\n>>>>>>>> to clear SS in userspace but I guess that gets just as messy.\n>>>>>>>>\n>>>>>>>>>>>>\n>>>>>>>>>>>> So perhaps we should stick with your original approach.\n>>>>>>>>>>>>\n>>>>>>>>>>>\n>>>>>>>>>>> I had not realized that was possible. This makes things more complicated for\n>>>>>>>>>>> avoiding a back and forth with the guest for trapped exceptions. Out of\n>>>>>>>>>>> luck, having the debug flag does look like single stepping would work as\n>>>>>>>>>>> expected for userland MMIOs.\n>>>>>>\n>>>>>> \n>>>\n>>> There is also the case of GIC CPU inteface accesses being trapped\n>>> (which shouldn't be the default behaviour). If the vgic access fails,\n>>> we will skip the instruction (in __kvm_vcpu_run in\n>>> arch/arm64/kvm/hyp/switch.c) and if we were single stepping we will\n>>> single step 2 instructions. But this seems to be a corner case of a\n>>> corner case (GIC CPUIF trapped + access failing + single stepping), so\n>>> I don't know if we want to take that into account right now?\n>>\n>> Yeah looking at the hyp code I did wonder if it warranted the complexity\n>> of adding handling there.\n>>\n>>> I'm still a bit concerned about the change of semantics for\n>>> KVM_EXIT_MMIO with regards to userland (cf. my previous mail). But if\n>>> this is deemed to be a reasonable change, the patch seems fine to me.\n>>\n>> Have we changed the semantics? A normal un-handled by the kernel IO/MMIO\n>> exit needs to be processed before the single step takes effect. I can't\n>> speak for other userspace but I think for QEMU it is as simple as the\n>> patch bellow. That said it wasn't quite clear where the PC gets updated\n>> in this path - I think the kernel updates the PC before the\n>> KVM_EXIT_MMIO in the same path as the internal handling.\n>>\n>\n> Well I'm not sure. The part I am concerned about is:\n>> NOTE: For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_PAPR and\n>> KVM_EXIT_EPR the corresponding\n>> operations are complete (and guest state is consistent) only after userspace\n>> has re-entered the kernel with KVM_RUN. The kernel side will first finish\n>> incomplete operations and then check for pending signals. Userspace\n>> can re-enter the guest with an unmasked signal pending to complete\n>> pending operations.\n>\n> From Documentation/virtual/kvm/api.txt.\n>\n> The way I interpret this is that userland should not consider the MMIO\n> complete before running the vcpu again. If that the case it shouldn't\n> trigger the single step since the instruction is not completely\n> finished.\n>\n> Maybe I don't interpret this correctly or it is not relevant here.\n> Although I'd like to understand why.\n\nNo it certainly needs clarifying. The comment was originally added in:\n\n 679613442f84702c06a80f2320cb8a50089200bc\n\nLooking more closely though it has a point. The IO/MMIO exits work\npurely from the address and data entries in the run structure. When we\nreturn to KVM_RUN we do:\n\n\tif (run->exit_reason == KVM_EXIT_MMIO) {\n\t\tret = kvm_handle_mmio_return(vcpu, vcpu->run);\n\t\tif (ret)\n\t\t\treturn ret;\n\t}\n\nSo you are correct the instruction emulation is not complete. Once that\nfixup is done however I think we are good to return. So perhaps we can\navoid involving QEMU entirely in this by generating a debug exit\nhere.\n\nQEMU ->\n kvm_run ->\n switch ->\n guest\n <-\n trap\n <-\n exit mmio\n <-\nQEMU\n -> kvm_run\n handle_mmio_return\n exit debug\n <-\nQEMU\n\nI don't think this affects the handle_exit() case as we only force the\nexit for successfully emulated instructions inside kvm.\n\nLooking at x86 for reference it does seem happy with triggering exits on\nsingle stepped emulation, see kvm_vcpu_do_singlestep(). However it also\nhas a number of comments on IO emulation:\n\n /* FIXME: return into emulator if single-stepping. */\n\nSo ARM is at least not behind the curve on this support ;-)\n\n>> I'd like to test these patches on some real life examples. I tried\n>> tracing over the pl011_write code in a kernel boot but I ran into the\n>> problem of el1_irq's occurring as I tried to step through the guest\n>> kernel. Is this something you've come across? What MMIO accesses have\n>> you been using in your testing?\n>>\n>\n> I didn't know which MMIOs were handled by userland so I have only\n> tested traps and MMIOs handled by the kernel.\n\nAny particular MMIOs I could also use to replicate in my tests?\n\n> This sounds like an issue when you are debugging an interruptible\n> context, an issue Pratyush has been looking at [1]. Are you taking a\n> guest interrupt when you try to execute the instruction to be stepped?\n> I don't know what's the status of this patch series. Can you test the\n> userland MMIO in a non-interruptible context?\n>\n> [1]\n> http://lists.infradead.org/pipermail/linux-arm-kernel/2017-July/517234.html\n\nAgain looking at x86 it looks like the approach is to suspend IRQs if\nyou are single-stepping. I'll have a look at Pratyush's patches.\n\n>\n> Thanks,\n>\n>> QEMU Patch bellow:\n>>\n>> From 2e8fcea695a9eca67fbeb331d3104d1d9e7e337a Mon Sep 17 00:00:00 2001\n>> From: =?UTF-8?q?Alex=20Benn=C3=A9e?= \n>> Date: Wed, 4 Oct 2017 09:49:41 +0000\n>> Subject: [PATCH] kvm: exit run loop after emulating IO when single stepping\n>> MIME-Version: 1.0\n>> Content-Type: text/plain; charset=UTF-8\n>> Content-Transfer-Encoding: 8bit\n>>\n>> If single-stepping is enabled we should exit the run-loop after\n>> emulating the access. Otherwise single-stepping across emulated IO\n>> accesses may skip an instruction.\n>>\n>> This only addresses user-space emulation. Stuff done in kernel-mode\n>> should be handled there.\n>>\n>> Signed-off-by: Alex Bennée \n>> ---\n>> accel/kvm/kvm-all.c | 4 ++--\n>> 1 file changed, 2 insertions(+), 2 deletions(-)\n>>\n>> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c\n>> index 90c88b517d..85bcb2b0d4 100644\n>> --- a/accel/kvm/kvm-all.c\n>> +++ b/accel/kvm/kvm-all.c\n>> @@ -1940,7 +1940,7 @@ int kvm_cpu_exec(CPUState *cpu)\n>> run->io.direction,\n>> run->io.size,\n>> run->io.count);\n>> - ret = 0;\n>> + ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n>> break;\n>> case KVM_EXIT_MMIO:\n>> DPRINTF(\"handle_mmio\\n\");\n>> @@ -1950,7 +1950,7 @@ int kvm_cpu_exec(CPUState *cpu)\n>> run->mmio.data,\n>> run->mmio.len,\n>> run->mmio.is_write);\n>> - ret = 0;\n>> + ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n>> break;\n>> case KVM_EXIT_IRQ_WINDOW_OPEN:\n>> DPRINTF(\"irq_window_open\\n\");\n>> --\n>> 2.14.1\n>>\n\n\n--\nAlex Bennée","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"Ci2SpaaR\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"L6mjr7ol\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6gCC6Mfrz9sRq\n\tfor ;\n\tThu, 5 Oct 2017 02:42:47 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzloo-0000eR-ET; Wed, 04 Oct 2017 15:42:42 +0000","from mail-wr0-x22b.google.com ([2a00:1450:400c:c0c::22b])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzloh-0000Mv-QT for linux-arm-kernel@lists.infradead.org;\n\tWed, 04 Oct 2017 15:42:40 +0000","by mail-wr0-x22b.google.com with SMTP id k62so8932034wrc.9\n\tfor ;\n\tWed, 04 Oct 2017 08:42:15 -0700 (PDT)","from zen.linaro.local ([81.128.185.34])\n\tby smtp.gmail.com with ESMTPSA id\n\t31sm15391079wru.33.2017.10.04.08.42.12\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tWed, 04 Oct 2017 08:42:12 -0700 (PDT)","from zen (localhost [127.0.0.1])\n\tby zen.linaro.local (Postfix) with ESMTPS id BADCB3E0137;\n\tWed, 4 Oct 2017 16:42:11 +0100 (BST)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:\n\tIn-reply-to:Subject:To:From:References:Reply-To:Content-ID:\n\tContent-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc\n\t:Resent-Message-ID:List-Owner;\n\tbh=SUZU+VwQ+yOjw7GVvlZW18rH9Uctl11tyKkXcHRhQYM=;\n\tb=Ci2SpaaRjWm5SMEPUOWQa7dLHa\n\tYG3duWB08fa8QIfXbrYlarX2JYaz0THIN7VLOe6bZ5NHVRBS+iWJxPBw9DsLIAAz7j6VpHxKDVTf2\n\tUZZn/6KrimBZ1h5cq6FDspE3uhp4hL7zCAdzrSBgmisNDkGQoZEYLjQcv/Uk8HlkJ+iTR75ORrSP8\n\tQoR76GVzS0buTuibvf1dwdAk+TgWnwFZbU36d13Mq+yKHBhnE6TinTnp14KLaftNCwooqRBzwZqLS\n\t1P/yu6Nc5ypC6p712G71snkSPRBugIDRumg0IMzG7iAJSNpmtsGv6/kW5YXadZFwtuY/tNx1IsPKT\n\tuAEs+oyQ==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=references:user-agent:from:to:cc:subject:in-reply-to:date\n\t:message-id:mime-version:content-transfer-encoding;\n\tbh=F+4SKSGWFos298Yr4xpxZDcALWNSu9n3VeuQQPCKco8=;\n\tb=L6mjr7ol3mBSiEWeUQiMl0cCIQRNdwd2wZh0/It+MZ3gdLpOWvPVi3uXhWeFOZc5M/\n\tS0NDmSITNy5Sp2Zv70ZfFukEiO6mYs1D7axgABpUQ19gXizIvFxRHEtffBwIqwUonNFj\n\t/ZLqBXK0qQWVxyb1imx//+8pGYSLGp1pGS+eQ="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:references:user-agent:from:to:cc:subject\n\t:in-reply-to:date:message-id:mime-version:content-transfer-encoding; \n\tbh=F+4SKSGWFos298Yr4xpxZDcALWNSu9n3VeuQQPCKco8=;\n\tb=O9nrfPJtnIs/IrAy6AwNAH04pTbsN8DRhJzNqRqemns+ch/+3k5k1OnqKoMla9HRlX\n\tk8JcJvsaWokthUNqTDXidxONKQuYfUZ/OIH0Iu/z8JqAMCZXn/qYfj6eQYZ2e497y3Yp\n\tOxccogM0cXw9XAtDHn1ahgFvj7Mpc9IbOOxVHOGaMIRhqLmh4jiD3bSEsYw4bmOLjgzC\n\tPMkBIWyOnxrz1dSn/NTyOaoYeQnoZx2TRmUA9gDD7i+xp7yibb6tXMLM7fb1TkieSHIw\n\t1TBGMw8Z6hY9LVLBgIHcwty3iO2XrCgLdjxcc1MCu4MQRuEYv8LOjJl0v4JnHp0B9cFK\n\tUMVw==","X-Gm-Message-State":"AMCzsaUTtFkdpBm9a+PbV7QB5xB/hS0inddScBOgF1JRFWxQtj3Kawcm\n\t0BGfi4mdt3s8tOO7HXI+AbcPSQ==","X-Google-Smtp-Source":"AOwi7QDHfR/alHjS7+e6yuOj9ce3Lhdtr/05hvr9CCKxUquo1rQ6VHZgz9IrVis/Pi02fNa7syIxcg==","X-Received":"by 10.223.142.110 with SMTP id\n\tn101mr9743078wrb.234.1507131733411; \n\tWed, 04 Oct 2017 08:42:13 -0700 (PDT)","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t<1504083688-48334-4-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>\n\t<873770yz1o.fsf@linaro.org>\n\t<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>\n\t<871smkywgn.fsf@linaro.org>\n\t\n\t<87k20bxm13.fsf@linaro.org>\n\t<1bdaff65-c784-4698-5c04-bfb1c943e6c0@arm.com>","User-agent":"mu4e 0.9.19; emacs 26.0.60","From":"Alex =?utf-8?q?Benn=C3=A9e?= ","To":"Julien Thierry ","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","In-reply-to":"<1bdaff65-c784-4698-5c04-bfb1c943e6c0@arm.com>","Date":"Wed, 04 Oct 2017 16:42:11 +0100","Message-ID":"<87efqiyl64.fsf@linaro.org>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171004_084236_465666_F6CCF53E ","X-CRM114-Status":"GOOD ( 36.66 )","X-Spam-Score":"-2.0 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.0 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,\n\tno\n\ttrust [2a00:1450:400c:c0c:0:0:0:22b listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\tChristoffer Dall ,\n\tPaolo Bonzini ,\n\t\"linux-arm-kernel@lists.infradead.org\"\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779960,"web_url":"http://patchwork.ozlabs.org/comment/1779960/","msgid":"","list_archive_url":null,"date":"2017-10-04T16:10:15","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":72256,"url":"http://patchwork.ozlabs.org/api/people/72256/","name":"Julien Thierry","email":"julien.thierry@arm.com"},"content":"On 04/10/17 16:42, Alex Bennée wrote:\n> \n> Julien Thierry writes:\n> \n>> On 04/10/17 11:08, Alex Bennée wrote:\n>>>\n>>> [Added Paolo, including QEMU userspace patch]\n>>>\n>>> Julien Thierry writes:\n>>>\n>>>> On 03/10/17 18:26, Alex Bennée wrote:\n>>>>>\n>>>>> Julien Thierry writes:\n>>>>>\n>>>>>> On 03/10/17 17:30, Alex Bennée wrote:\n>>>>>>>\n>>>>>>> Julien Thierry writes:\n>>>>>>>\n>>>>>>>> On 03/10/17 15:57, Alex Bennée wrote:\n>>>>>>>>>\n>>>>>>>>> Julien Thierry writes:\n>>>>>>>>>\n>>>>>>>>>> On 31/08/17 15:01, Christoffer Dall wrote:\n>>>>>>> \n>>>>>>>>>>>>>>\n>>>>>>>>>>>>>> Does that sound like what you had in mind? Or does it seem better than\n>>>>>>>>>>>>>> the\n>>>>>>>>>>>>>> current patch?\n>>>>>>>>>>>>>>\n>>>>>>>>>>>>> I was thinking to change the skip_instruction function to return an\n>>>>>>>>>>>>> int, and then call kvm_handle_debug_ss() from skip_instruction, which\n>>>>>>>>>>>>> would update the kvm_run structure and exit here and then.\n>>>>>>>>>>>>>\n>>>>>>>>>>>>\n>>>>>>>>>>>> Setting up the debug exception from within kvm_skip_instruction seem to\n>>>>>>>>>>>> change a bit too much its semantic from arm to arm64. I would find this\n>>>>>>>>>>>> easily confusing.\n>>>>>>>>>>>>\n>>>>>>>>>>>>> However, I'm now thinking that this doesn't really work either,\n>>>>>>>>>>>>> because we could have to emulate a trapped MMIO instruction in user\n>>>>>>>>>>>>> space, and then it's not clear how to exit with a debug exception at\n>>>>>>>>>>>>> the same time.\n>>>>>>>>>\n>>>>>>>>> A debug exception at guest exit point is (IIRC) just having the\n>>>>>>>>> appropriate status in the run->exit_reason (KVM_EXIT_DEBUG). If you need\n>>>>>>>>> to exit for MMIO emulation (i.e. the instruction has not run yet) you\n>>>>>>>>> shouldn't do that. Exit, emulate and return. We could handle the ioctl\n>>>>>>>>> to clear SS in userspace but I guess that gets just as messy.\n>>>>>>>>>\n>>>>>>>>>>>>>\n>>>>>>>>>>>>> So perhaps we should stick with your original approach.\n>>>>>>>>>>>>>\n>>>>>>>>>>>>\n>>>>>>>>>>>> I had not realized that was possible. This makes things more complicated for\n>>>>>>>>>>>> avoiding a back and forth with the guest for trapped exceptions. Out of\n>>>>>>>>>>>> luck, having the debug flag does look like single stepping would work as\n>>>>>>>>>>>> expected for userland MMIOs.\n>>>>>>>\n>>>>>>> \n>>>>\n>>>> There is also the case of GIC CPU inteface accesses being trapped\n>>>> (which shouldn't be the default behaviour). If the vgic access fails,\n>>>> we will skip the instruction (in __kvm_vcpu_run in\n>>>> arch/arm64/kvm/hyp/switch.c) and if we were single stepping we will\n>>>> single step 2 instructions. But this seems to be a corner case of a\n>>>> corner case (GIC CPUIF trapped + access failing + single stepping), so\n>>>> I don't know if we want to take that into account right now?\n>>>\n>>> Yeah looking at the hyp code I did wonder if it warranted the complexity\n>>> of adding handling there.\n>>>\n>>>> I'm still a bit concerned about the change of semantics for\n>>>> KVM_EXIT_MMIO with regards to userland (cf. my previous mail). But if\n>>>> this is deemed to be a reasonable change, the patch seems fine to me.\n>>>\n>>> Have we changed the semantics? A normal un-handled by the kernel IO/MMIO\n>>> exit needs to be processed before the single step takes effect. I can't\n>>> speak for other userspace but I think for QEMU it is as simple as the\n>>> patch bellow. That said it wasn't quite clear where the PC gets updated\n>>> in this path - I think the kernel updates the PC before the\n>>> KVM_EXIT_MMIO in the same path as the internal handling.\n>>>\n>>\n>> Well I'm not sure. The part I am concerned about is:\n>>> NOTE: For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_PAPR and\n>>> KVM_EXIT_EPR the corresponding\n>>> operations are complete (and guest state is consistent) only after userspace\n>>> has re-entered the kernel with KVM_RUN. The kernel side will first finish\n>>> incomplete operations and then check for pending signals. Userspace\n>>> can re-enter the guest with an unmasked signal pending to complete\n>>> pending operations.\n>>\n>> From Documentation/virtual/kvm/api.txt.\n>>\n>> The way I interpret this is that userland should not consider the MMIO\n>> complete before running the vcpu again. If that the case it shouldn't\n>> trigger the single step since the instruction is not completely\n>> finished.\n>>\n>> Maybe I don't interpret this correctly or it is not relevant here.\n>> Although I'd like to understand why.\n> \n> No it certainly needs clarifying. The comment was originally added in:\n> \n> 679613442f84702c06a80f2320cb8a50089200bc\n> \n> Looking more closely though it has a point. The IO/MMIO exits work\n> purely from the address and data entries in the run structure. When we\n> return to KVM_RUN we do:\n> \n> \tif (run->exit_reason == KVM_EXIT_MMIO) {\n> \t\tret = kvm_handle_mmio_return(vcpu, vcpu->run);\n> \t\tif (ret)\n> \t\t\treturn ret;\n> \t}\n> \n> So you are correct the instruction emulation is not complete. Once that\n> fixup is done however I think we are good to return. So perhaps we can\n> avoid involving QEMU entirely in this by generating a debug exit\n> here.\n> \n> QEMU ->\n> kvm_run ->\n> switch ->\n> guest\n> <-\n> trap\n> <-\n> exit mmio\n> <-\n> QEMU\n> -> kvm_run\n> handle_mmio_return\n> exit debug\n> <-\n> QEMU\n> \n\nThanks for the explanation. This approach sounds good to me. Also, it \nmeans QEMU doesn't need to get patched, is that correct?\n\n> I don't think this affects the handle_exit() case as we only force the\n> exit for successfully emulated instructions inside kvm.\n> \n\nYes, in handle_exit MMIO handled by the kernel will exit with \nKVM_EXIT_DEBUG and MMIO handled by the userland will exit with \nKVM_EXIT_MMIO. So from your patch we just need to add the exit debug \nafter handle_mmio_return.\n\n> Looking at x86 for reference it does seem happy with triggering exits on\n> single stepped emulation, see kvm_vcpu_do_singlestep(). However it also\n> has a number of comments on IO emulation:\n> \n> /* FIXME: return into emulator if single-stepping. */\n> \n> So ARM is at least not behind the curve on this support ;-)\n> \n>>> I'd like to test these patches on some real life examples. I tried\n>>> tracing over the pl011_write code in a kernel boot but I ran into the\n>>> problem of el1_irq's occurring as I tried to step through the guest\n>>> kernel. Is this something you've come across? What MMIO accesses have\n>>> you been using in your testing?\n>>>\n>>\n>> I didn't know which MMIOs were handled by userland so I have only\n>> tested traps and MMIOs handled by the kernel.\n> \n> Any particular MMIOs I could also use to replicate in my tests?\n> \n\nFor MMIOs handled by the kernel, I was testing with the GIC. You could \ntry to break on gic_cpu_config in the guest, where it will write to \ndistributor registers. The function should be called during \ninitialization, before IRQs are enabled, so you shouldn't be bothered by \nthe earlier trouble.\n\n>> This sounds like an issue when you are debugging an interruptible\n>> context, an issue Pratyush has been looking at [1]. Are you taking a\n>> guest interrupt when you try to execute the instruction to be stepped?\n>> I don't know what's the status of this patch series. Can you test the\n>> userland MMIO in a non-interruptible context?\n>>\n>> [1]\n>> http://lists.infradead.org/pipermail/linux-arm-kernel/2017-July/517234.html\n> \n> Again looking at x86 it looks like the approach is to suspend IRQs if\n> you are single-stepping. I'll have a look at Pratyush's patches.\n> \n\nI think that was the idea with Pratyush's patches as well.\n\nSo, I'm happy to replace my patch with yours in this series (unless you \nwant to post it separated since it doesn't depend on my patches).\n\nThank you for looking at this and providing your input.\n\nCheers,\n\n>>\n>>> QEMU Patch bellow:\n>>>\n>>> From 2e8fcea695a9eca67fbeb331d3104d1d9e7e337a Mon Sep 17 00:00:00 2001\n>>> From: =?UTF-8?q?Alex=20Benn=C3=A9e?= \n>>> Date: Wed, 4 Oct 2017 09:49:41 +0000\n>>> Subject: [PATCH] kvm: exit run loop after emulating IO when single stepping\n>>> MIME-Version: 1.0\n>>> Content-Type: text/plain; charset=UTF-8\n>>> Content-Transfer-Encoding: 8bit\n>>>\n>>> If single-stepping is enabled we should exit the run-loop after\n>>> emulating the access. Otherwise single-stepping across emulated IO\n>>> accesses may skip an instruction.\n>>>\n>>> This only addresses user-space emulation. Stuff done in kernel-mode\n>>> should be handled there.\n>>>\n>>> Signed-off-by: Alex Bennée \n>>> ---\n>>> accel/kvm/kvm-all.c | 4 ++--\n>>> 1 file changed, 2 insertions(+), 2 deletions(-)\n>>>\n>>> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c\n>>> index 90c88b517d..85bcb2b0d4 100644\n>>> --- a/accel/kvm/kvm-all.c\n>>> +++ b/accel/kvm/kvm-all.c\n>>> @@ -1940,7 +1940,7 @@ int kvm_cpu_exec(CPUState *cpu)\n>>> run->io.direction,\n>>> run->io.size,\n>>> run->io.count);\n>>> - ret = 0;\n>>> + ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n>>> break;\n>>> case KVM_EXIT_MMIO:\n>>> DPRINTF(\"handle_mmio\\n\");\n>>> @@ -1950,7 +1950,7 @@ int kvm_cpu_exec(CPUState *cpu)\n>>> run->mmio.data,\n>>> run->mmio.len,\n>>> run->mmio.is_write);\n>>> - ret = 0;\n>>> + ret = cpu->singlestep_enabled ? EXCP_DEBUG : 0;\n>>> break;\n>>> case KVM_EXIT_IRQ_WINDOW_OPEN:\n>>> DPRINTF(\"irq_window_open\\n\");\n>>> --\n>>> 2.14.1\n>>>\n> \n> \n> --\n> Alex Bennée\n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"DzZGknRh\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6gqW4QN8z9s7C\n\tfor ;\n\tThu, 5 Oct 2017 03:10:47 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzmFv-0004EI-UD; Wed, 04 Oct 2017 16:10:43 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzmFr-0004AH-LL for linux-arm-kernel@lists.infradead.org;\n\tWed, 04 Oct 2017 16:10:41 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D6D461435;\n\tWed, 4 Oct 2017 09:10:18 -0700 (PDT)","from [10.1.206.253] (e112298-lin.cambridge.arm.com [10.1.206.253])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t60D713F58C; Wed, 4 Oct 2017 09:10:17 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=cPjCvDMIqiB2nSizqelCboXoQADDAqc4Rzl3e/KJoLU=;\n\tb=DzZGknRh4Cgsg0OI9r8sCIbzQ\n\tTagyMsrfWMQFeKSloUXJLM4gwFWVHv6KNzyKK9pK0SAltQkrCDtPdhiX+uZKzNfv4JRK4Djz23AFc\n\tlBMIcMgOax9pT9Yp+XFIGePZfPPpoyfY1Nz2Lx3kPVEEi57SLk8tWAUWVd39rBqD3R6Yu6uZfgqPi\n\tUYvoZR+X8gHnTL9h8oigNyxPuPsKum4kqe3trNRr8966jBsU3ZdPcRAvauWyTNQwmCmFcicAk8DN+\n\tfyc4Td3SAiZ1HggZpGrt6OZH95oFLEFxYea2wvPTckWU0FOlY0kCWPnYzLXrOn4qxdGxBORnhVjAB\n\tzdyjHHx3w==;","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","To":"=?utf-8?q?Alex_Benn=C3=A9e?= ","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>\n\t<873770yz1o.fsf@linaro.org>\n\t<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>\n\t<871smkywgn.fsf@linaro.org>\n\t\n\t<87k20bxm13.fsf@linaro.org>\n\t<1bdaff65-c784-4698-5c04-bfb1c943e6c0@arm.com>\n\t<87efqiyl64.fsf@linaro.org>","From":"Julien Thierry ","Message-ID":"","Date":"Wed, 4 Oct 2017 17:10:15 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<87efqiyl64.fsf@linaro.org>","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171004_091039_725107_35BE77E2 ","X-CRM114-Status":"GOOD ( 42.06 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\tChristoffer Dall ,\n\tPaolo Bonzini ,\n\t\"linux-arm-kernel@lists.infradead.org\"\n\t","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1780041,"web_url":"http://patchwork.ozlabs.org/comment/1780041/","msgid":"<87d162ydq2.fsf@linaro.org>","list_archive_url":null,"date":"2017-10-04T18:23:01","subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","submitter":{"id":39532,"url":"http://patchwork.ozlabs.org/api/people/39532/","name":"Alex Bennée","email":"alex.bennee@linaro.org"},"content":"Julien Thierry writes:\n\n> On 04/10/17 16:42, Alex Bennée wrote:\n>>\n\n>>\n>> Looking more closely though it has a point. The IO/MMIO exits work\n>> purely from the address and data entries in the run structure. When we\n>> return to KVM_RUN we do:\n>>\n>> \tif (run->exit_reason == KVM_EXIT_MMIO) {\n>> \t\tret = kvm_handle_mmio_return(vcpu, vcpu->run);\n>> \t\tif (ret)\n>> \t\t\treturn ret;\n>> \t}\n>>\n>> So you are correct the instruction emulation is not complete. Once that\n>> fixup is done however I think we are good to return. So perhaps we can\n>> avoid involving QEMU entirely in this by generating a debug exit\n>> here.\n>>\n>> QEMU ->\n>> kvm_run ->\n>> switch ->\n>> guest\n>> <-\n>> trap\n>> <-\n>> exit mmio\n>> <-\n>> QEMU\n>> -> kvm_run\n>> handle_mmio_return\n>> exit debug\n>> <-\n>> QEMU\n>>\n>\n> Thanks for the explanation. This approach sounds good to me. Also, it\n> means QEMU doesn't need to get patched, is that correct?\n\nCorrect.\n\n>\n>> I don't think this affects the handle_exit() case as we only force the\n>> exit for successfully emulated instructions inside kvm.\n>>\n>\n> Yes, in handle_exit MMIO handled by the kernel will exit with\n> KVM_EXIT_DEBUG and MMIO handled by the userland will exit with\n> KVM_EXIT_MMIO. So from your patch we just need to add the exit debug\n> after handle_mmio_return.\n\nOne minor wrinkle in kvm_handle_mmio_return() I can't use\nvcpu->debug_flags as the v7 code doesn't have it in kvm_vcpu_arch.\n\n>\n>> Looking at x86 for reference it does seem happy with triggering exits on\n>> single stepped emulation, see kvm_vcpu_do_singlestep(). However it also\n>> has a number of comments on IO emulation:\n>>\n>> /* FIXME: return into emulator if single-stepping. */\n>>\n>> So ARM is at least not behind the curve on this support ;-)\n>>\n>>>> I'd like to test these patches on some real life examples. I tried\n>>>> tracing over the pl011_write code in a kernel boot but I ran into the\n>>>> problem of el1_irq's occurring as I tried to step through the guest\n>>>> kernel. Is this something you've come across? What MMIO accesses have\n>>>> you been using in your testing?\n>>>>\n>>>\n>>> I didn't know which MMIOs were handled by userland so I have only\n>>> tested traps and MMIOs handled by the kernel.\n>>\n>> Any particular MMIOs I could also use to replicate in my tests?\n>>\n>\n> For MMIOs handled by the kernel, I was testing with the GIC. You could\n> try to break on gic_cpu_config in the guest, where it will write to\n> distributor registers. The function should be called during\n> initialization, before IRQs are enabled, so you shouldn't be bothered\n> by the earlier trouble.\n\nThanks - this will be useful.\n\n>>> This sounds like an issue when you are debugging an interruptible\n>>> context, an issue Pratyush has been looking at [1]. Are you taking a\n>>> guest interrupt when you try to execute the instruction to be stepped?\n>>> I don't know what's the status of this patch series. Can you test the\n>>> userland MMIO in a non-interruptible context?\n>>>\n>>> [1]\n>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2017-July/517234.html\n>>\n>> Again looking at x86 it looks like the approach is to suspend IRQs if\n>> you are single-stepping. I'll have a look at Pratyush's patches.\n>>\n>\n> I think that was the idea with Pratyush's patches as well.\n>\n> So, I'm happy to replace my patch with yours in this series (unless\n> you want to post it separated since it doesn't depend on my patches).\n\n\nNope I'm happy for you to carry it to merge. I'll see if I can send you\na v2 once I've addressed the mmio completion.\n\n>\n> Thank you for looking at this and providing your input.\n\nNo problem, sorry it took so long.\n\n--\nAlex Bennée","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"els25KzK\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"ie76HTsY\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6kmm3265z9t1G\n\tfor ;\n\tThu, 5 Oct 2017 05:23:36 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzoKQ-0000IE-KI; Wed, 04 Oct 2017 18:23:30 +0000","from mail-wm0-x22d.google.com ([2a00:1450:400c:c09::22d])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzoKM-0000Ei-Bi for linux-arm-kernel@lists.infradead.org;\n\tWed, 04 Oct 2017 18:23:28 +0000","by mail-wm0-x22d.google.com with SMTP id b189so20613165wmd.4\n\tfor ;\n\tWed, 04 Oct 2017 11:23:05 -0700 (PDT)","from zen.linaro.local ([81.128.185.34])\n\tby smtp.gmail.com with ESMTPSA id\n\t204sm18278408wml.10.2017.10.04.11.23.02\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tWed, 04 Oct 2017 11:23:02 -0700 (PDT)","from zen (localhost [127.0.0.1])\n\tby zen.linaro.local (Postfix) with ESMTPS id D9AE83E0137;\n\tWed, 4 Oct 2017 19:23:01 +0100 (BST)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:\n\tIn-reply-to:Subject:To:From:References:Reply-To:Content-ID:\n\tContent-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc\n\t:Resent-Message-ID:List-Owner;\n\tbh=uRxSDicDWbuN0By30vunBzAI4tDKkkTnlt6JSOgIzKQ=;\n\tb=els25KzKVgdVpqxtR21Suo1QSP\n\toAucuj+d6F+OBMP1XinSAMN78cJaMNvwsxROad7gT4Gvj3UtyG0mlx76D3EQuqZ3795elYJUWQWOt\n\tCMcEh4+/Un+XYBwmBkHiVi3RtE3wfjfPGmsWelXzQ9/Y6dacyz5hKJZf6vWOc2xeSJZqWmQGXdbYr\n\tAydHoHlqLTUS4+sSgUQ6RamK1TPFq8LcGXUTdC5ki3dlPO2aIpRvqvYXJspYo/H5w0guB0ZZoxQC9\n\tYRWY10I+W9pl3nNq8XpWbgfKBKFD8bgKNyRAqj34pv3rtHknd+kfqKXeaNGR6OsoNR5IChD0n8qAF\n\tJBgza0vw==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=references:user-agent:from:to:cc:subject:in-reply-to:date\n\t:message-id:mime-version:content-transfer-encoding;\n\tbh=cT8U3CH9AmETa4A3sapC/14sr2cy7WlB+tlPPBvoEy0=;\n\tb=ie76HTsY+geSc9JTlVIVsiRqMqsrgZKdXJseuysUV8DC1tixkYmedXYM8ZI4yJRraB\n\tl+Cmq/tMZ6Qe8SYxarThDlGPn5iFcJx9MVvm6WXdPKyUMuq7HGxU7F922Zw9Rk4SrCxK\n\tk0mzs9+ViuRVRt9kwgQzLaOQ9jU6ndiV5c6FE="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:references:user-agent:from:to:cc:subject\n\t:in-reply-to:date:message-id:mime-version:content-transfer-encoding; \n\tbh=cT8U3CH9AmETa4A3sapC/14sr2cy7WlB+tlPPBvoEy0=;\n\tb=kWm9JHi+FQXcQoGfaScR+H0UWXnZ2agOl5VyeDjaYAml/FHVP0GbVtCI8axujOHivr\n\tVr9ndY1dZ8BUPJBKBjHEroy6403XFsprWVpV1LGi4ngGDMomQ8oGyozKyc52fIoW7BvV\n\the5wKE4U2ooUzmCN6JPCeeyZQe5qSo02SFNeH3tLrAdQNEuHmx2H9LELU0cCElHpc5e2\n\t7YzA5e1HAseQGV2HtuTsSpeQMU0V75Meds0V09OusDFFKBordXnVYow2mbwi3ZXkoidJ\n\tw1awdScErHJ94ZafQAM0qRQqihDn7tbnnDgMa2gb4dqQl2CydODVh202VbqYvMznkelT\n\tOL/A==","X-Gm-Message-State":"AHPjjUhAwVWeF7e5DWjZ5ZY1NYGcE4wU+LXseH4EJv+uMhCh+ajV71Mc\n\tRB7hazu2RM90o5r8E+Spt5BGBQ==","X-Google-Smtp-Source":"AOwi7QB5TGZohHVwl+A+O9qyoJR2bKC1I8S1z5gESJLb4E8HF11w5sm0XQDC0W8CR0VKlDOuTyQKNg==","X-Received":"by 10.28.63.145 with SMTP id m139mr18792216wma.5.1507141383722; \n\tWed, 04 Oct 2017 11:23:03 -0700 (PDT)","References":"<1504083688-48334-1-git-send-email-julien.thierry@arm.com>\n\t\n\t<3c249a68-45e3-a3a5-7d05-4cfc2d97713b@arm.com>\n\t\n\t\n\t\n\t<3d7d2b36-da2f-04dc-611e-d7aab7666c29@arm.com>\n\t\n\t<9bc5abc2-ab03-3137-82bd-e8afa62624eb@arm.com>\n\t\n\t<861b4e4f-0fbe-cbc6-39ad-4660065449de@arm.com>\n\t<877ewcz3bv.fsf@linaro.org>\n\t<4d9fc0a2-bcf9-ca26-8646-037c2dcc6545@arm.com>\n\t<873770yz1o.fsf@linaro.org>\n\t<5a4f0493-6d3d-85de-fc45-030d4b03b5a8@arm.com>\n\t<871smkywgn.fsf@linaro.org>\n\t\n\t<87k20bxm13.fsf@linaro.org>\n\t<1bdaff65-c784-4698-5c04-bfb1c943e6c0@arm.com>\n\t<87efqiyl64.fsf@linaro.org>\n\t","User-agent":"mu4e 0.9.19; emacs 26.0.60","From":"Alex =?utf-8?q?Benn=C3=A9e?= ","To":"Julien Thierry ","Subject":"Re: [PATCH 3/3] arm64: kvm: Fix single step for guest skipped\n\tinstructions","In-reply-to":"","Date":"Wed, 04 Oct 2017 19:23:01 +0100","Message-ID":"<87d162ydq2.fsf@linaro.org>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171004_112326_748205_84B3A0F0 ","X-CRM114-Status":"GOOD ( 26.14 )","X-Spam-Score":"-2.7 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/,\n\tlow\n\ttrust [2a00:1450:400c:c09:0:0:0:22d listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas ,\n\tWill Deacon , \n\tChristoffer Dall ,\n\tPaolo Bonzini ,\n\t\"linux-arm-kernel@lists.infradead.org\"\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}}]