[{"id":1782975,"web_url":"http://patchwork.ozlabs.org/comment/1782975/","msgid":"<2fb75502-ee42-9178-835f-637179d47937@arm.com>","list_archive_url":null,"date":"2017-10-09T16:22:13","subject":"Re: [PATCH v3 04/20] KVM: arm/arm64: Guard kvm_vgic_map_is_active\n\tagainst !vgic_initialized","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On 23/09/17 01:41, Christoffer Dall wrote:\n> If the vgic is not initialized, don't try to grab its spinlocks or\n> traverse its data structures.\n> \n> This is important because we soon have to start considering the active\n> state of a virtual interrupts when doing vcpu_load, which may happen\n> early on before the vgic is initialized.\n> \n> Signed-off-by: Christoffer Dall \n> ---\n> virt/kvm/arm/vgic/vgic.c | 3 +++\n> 1 file changed, 3 insertions(+)\n> \n> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c\n> index fed717e..e1f7dbc 100644\n> --- a/virt/kvm/arm/vgic/vgic.c\n> +++ b/virt/kvm/arm/vgic/vgic.c\n> @@ -777,6 +777,9 @@ bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int virt_irq)\n> \tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);\n> \tbool map_is_active;\n> \n> +\tif (!vgic_initialized(vcpu->kvm))\n> +\t\treturn false;\n> +\n> \tspin_lock(&irq->irq_lock);\n> \tmap_is_active = irq->hw && irq->active;\n> \tspin_unlock(&irq->irq_lock);\n> \n\nAcked-by: Marc Zyngier \n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"tqLyxukg\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9ls10hKQz9tXx\n\tfor ;\n\tTue, 10 Oct 2017 03:22:45 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1apF-0001Jk-1T; Mon, 09 Oct 2017 16:22:41 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1apA-0001Eh-EL for linux-arm-kernel@lists.infradead.org;\n\tMon, 09 Oct 2017 16:22:38 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 31C091529;\n\tMon, 9 Oct 2017 09:22:16 -0700 (PDT)","from [10.1.207.62] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t228653F483; Mon, 9 Oct 2017 09:22:14 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=gYyRKdmLhOt4Bteo1+atvyKhZwYj8K1F21Liky0pi7M=;\n\tb=tqLyxukgxnl+hY\n\t+SFVvV0iqVXx3SqVnjjjG97XjsVBhQSZ5sRwJXxPeK08pzaiZWCtf6WYvbpO8qcqthYhTAKYaI+Dd\n\twy5XbjhsDXjrRxqTyyhtQGJzQ43nItaDo6AmP12VlC99jHSkA5/SM8r9fvnGkStNt8Ghy2yawCTaN\n\txSxzqznHxNVjy6xHrYPHPy2zFEN1IlFv1gJUmzwUsVkVIrYJxsLZ9lSW04fHIOlWcv0de2oZJKm0k\n\t7czNMTrMwGMI4uCNMO90G8YpDQrBAqIZLnpNhyPwOd5GTDmJk7PIOfSQdoz9wxcx2EFoYEwG6hr6Q\n\tg+5X14SJZvCWjGodjBxw==;","Subject":"Re: [PATCH v3 04/20] KVM: arm/arm64: Guard kvm_vgic_map_is_active\n\tagainst !vgic_initialized","To":"Christoffer Dall , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-5-cdall@linaro.org>","From":"Marc Zyngier ","Organization":"ARM Ltd","Message-ID":"<2fb75502-ee42-9178-835f-637179d47937@arm.com>","Date":"Mon, 9 Oct 2017 17:22:13 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170923004207.22356-5-cdall@linaro.org>","Content-Language":"en-GB","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171009_092236_554843_19A5F3C3 ","X-CRM114-Status":"GOOD ( 13.46 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1782977,"web_url":"http://patchwork.ozlabs.org/comment/1782977/","msgid":"<9b06425f-7c2c-d44a-cd6c-aeaa4b76849c@arm.com>","list_archive_url":null,"date":"2017-10-09T16:21:24","subject":"Re: [PATCH v3 03/20] arm64: Use the physical counter when available\n\tfor read_cycles","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On 23/09/17 01:41, Christoffer Dall wrote:\n> Currently get_cycles() is hardwired to arch_counter_get_cntvct() on\n> arm64, but as we move to using the physical timer for the in-kernel\n> time-keeping, we need to make that more flexible.\n> \n> First, we need to make sure the physical counter can be read on equal\n> terms to the virtual counter, which includes adding physical counter\n> read functions for timers that require errata.\n> \n> Second, we need to make a choice between reading the physical vs virtual\n> counter, depending on which timer is used for time keeping in the kernel\n> otherwise. We can do this using a static key to avoid a performance\n> penalty during runtime when reading the counter.\n> \n> Cc: Catalin Marinas \n> Cc: Will Deacon \n> Cc: Mark Rutland \n> Cc: Marc Zyngier \n> Signed-off-by: Christoffer Dall \n\nRight. I should have read patch #3. I'm an idiot.\n\n> ---\n> arch/arm64/include/asm/arch_timer.h | 15 ++++++++++++---\n> arch/arm64/include/asm/timex.h | 2 +-\n> drivers/clocksource/arm_arch_timer.c | 32 ++++++++++++++++++++++++++++++--\n> 3 files changed, 43 insertions(+), 6 deletions(-)\n> \n> diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h\n> index 1859a1c..c56d8cd 100644\n> --- a/arch/arm64/include/asm/arch_timer.h\n> +++ b/arch/arm64/include/asm/arch_timer.h\n> @@ -30,6 +30,8 @@\n> \n> #include \n> \n> +extern struct static_key_false arch_timer_phys_counter_available;\n> +\n> #if IS_ENABLED(CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND)\n> extern struct static_key_false arch_timer_read_ool_enabled;\n> #define needs_unstable_timer_counter_workaround() \\\n> @@ -52,6 +54,7 @@ struct arch_timer_erratum_workaround {\n> \tconst char *desc;\n> \tu32 (*read_cntp_tval_el0)(void);\n> \tu32 (*read_cntv_tval_el0)(void);\n> +\tu64 (*read_cntpct_el0)(void);\n> \tu64 (*read_cntvct_el0)(void);\n> \tint (*set_next_event_phys)(unsigned long, struct clock_event_device *);\n> \tint (*set_next_event_virt)(unsigned long, struct clock_event_device *);\n> @@ -148,10 +151,8 @@ static inline void arch_timer_set_cntkctl(u32 cntkctl)\n> \n> static inline u64 arch_counter_get_cntpct(void)\n> {\n> -\tu64 cval;\n> \tisb();\n> -\tasm volatile(\"mrs %0, cntpct_el0\" : \"=r\" (cval));\n> -\treturn cval;\n> +\treturn arch_timer_reg_read_stable(cntpct_el0);\n> }\n> \n> static inline u64 arch_counter_get_cntvct(void)\n> @@ -160,6 +161,14 @@ static inline u64 arch_counter_get_cntvct(void)\n> \treturn arch_timer_reg_read_stable(cntvct_el0);\n> }\n> \n> +static inline u64 arch_counter_get_cycles(void)\n> +{\n> +\tif (static_branch_unlikely(&arch_timer_phys_counter_available))\n> +\t return arch_counter_get_cntpct();\n> +\telse\n> +\t return arch_counter_get_cntvct();\n> +}\n> +\n> static inline int arch_timer_arch_init(void)\n> {\n> \treturn 0;\n> diff --git a/arch/arm64/include/asm/timex.h b/arch/arm64/include/asm/timex.h\n> index 81a076e..c0d214c 100644\n> --- a/arch/arm64/include/asm/timex.h\n> +++ b/arch/arm64/include/asm/timex.h\n> @@ -22,7 +22,7 @@\n> * Use the current timer as a cycle counter since this is what we use for\n> * the delay loop.\n> */\n> -#define get_cycles()\tarch_counter_get_cntvct()\n> +#define get_cycles()\tarch_counter_get_cycles()\n\nWhy can't this be arch_timer_read_counter() instead? Is there any \nmeasurable advantage in using a static key compared to a memory \nindirection?\n\n> \n> #include \n> \n> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c\n> index 9b3322a..f35da20 100644\n> --- a/drivers/clocksource/arm_arch_timer.c\n> +++ b/drivers/clocksource/arm_arch_timer.c\n> @@ -77,6 +77,9 @@ static bool arch_timer_mem_use_virtual;\n> static bool arch_counter_suspend_stop;\n> static bool vdso_default = true;\n> \n> +DEFINE_STATIC_KEY_FALSE(arch_timer_phys_counter_available);\n> +EXPORT_SYMBOL_GPL(arch_timer_phys_counter_available);\n> +\n> static bool evtstrm_enable = IS_ENABLED(CONFIG_ARM_ARCH_TIMER_EVTSTREAM);\n> \n> static int __init early_evtstrm_cfg(char *buf)\n> @@ -217,6 +220,11 @@ static u32 notrace fsl_a008585_read_cntv_tval_el0(void)\n> \treturn __fsl_a008585_read_reg(cntv_tval_el0);\n> }\n> \n> +static u64 notrace fsl_a008585_read_cntpct_el0(void)\n> +{\n> +\treturn __fsl_a008585_read_reg(cntpct_el0);\n> +}\n> +\n> static u64 notrace fsl_a008585_read_cntvct_el0(void)\n> {\n> \treturn __fsl_a008585_read_reg(cntvct_el0);\n> @@ -258,6 +266,11 @@ static u32 notrace hisi_161010101_read_cntv_tval_el0(void)\n> \treturn __hisi_161010101_read_reg(cntv_tval_el0);\n> }\n> \n> +static u64 notrace hisi_161010101_read_cntpct_el0(void)\n> +{\n> +\treturn __hisi_161010101_read_reg(cntpct_el0);\n> +}\n> +\n> static u64 notrace hisi_161010101_read_cntvct_el0(void)\n> {\n> \treturn __hisi_161010101_read_reg(cntvct_el0);\n> @@ -288,6 +301,15 @@ static struct ate_acpi_oem_info hisi_161010101_oem_info[] = {\n> #endif\n> \n> #ifdef CONFIG_ARM64_ERRATUM_858921\n> +static u64 notrace arm64_858921_read_cntpct_el0(void)\n> +{\n> +\tu64 old, new;\n> +\n> +\told = read_sysreg(cntpct_el0);\n> +\tnew = read_sysreg(cntpct_el0);\n> +\treturn (((old ^ new) >> 32) & 1) ? old : new;\n> +}\n> +\n> static u64 notrace arm64_858921_read_cntvct_el0(void)\n> {\n> \tu64 old, new;\n> @@ -346,6 +368,7 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {\n> \t\t.desc = \"Freescale erratum a005858\",\n> \t\t.read_cntp_tval_el0 = fsl_a008585_read_cntp_tval_el0,\n> \t\t.read_cntv_tval_el0 = fsl_a008585_read_cntv_tval_el0,\n> +\t\t.read_cntpct_el0 = fsl_a008585_read_cntpct_el0,\n> \t\t.read_cntvct_el0 = fsl_a008585_read_cntvct_el0,\n> \t\t.set_next_event_phys = erratum_set_next_event_tval_phys,\n> \t\t.set_next_event_virt = erratum_set_next_event_tval_virt,\n> @@ -358,6 +381,7 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {\n> \t\t.desc = \"HiSilicon erratum 161010101\",\n> \t\t.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,\n> \t\t.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,\n> +\t\t.read_cntpct_el0 = hisi_161010101_read_cntpct_el0,\n> \t\t.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,\n> \t\t.set_next_event_phys = erratum_set_next_event_tval_phys,\n> \t\t.set_next_event_virt = erratum_set_next_event_tval_virt,\n> @@ -368,6 +392,7 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {\n> \t\t.desc = \"HiSilicon erratum 161010101\",\n> \t\t.read_cntp_tval_el0 = hisi_161010101_read_cntp_tval_el0,\n> \t\t.read_cntv_tval_el0 = hisi_161010101_read_cntv_tval_el0,\n> +\t\t.read_cntpct_el0 = hisi_161010101_read_cntpct_el0,\n> \t\t.read_cntvct_el0 = hisi_161010101_read_cntvct_el0,\n> \t\t.set_next_event_phys = erratum_set_next_event_tval_phys,\n> \t\t.set_next_event_virt = erratum_set_next_event_tval_virt,\n> @@ -378,6 +403,7 @@ static const struct arch_timer_erratum_workaround ool_workarounds[] = {\n> \t\t.match_type = ate_match_local_cap_id,\n> \t\t.id = (void *)ARM64_WORKAROUND_858921,\n> \t\t.desc = \"ARM erratum 858921\",\n> +\t\t.read_cntpct_el0 = arm64_858921_read_cntpct_el0,\n> \t\t.read_cntvct_el0 = arm64_858921_read_cntvct_el0,\n> \t},\n> #endif\n> @@ -890,10 +916,12 @@ static void __init arch_counter_register(unsigned type)\n> \n> \t/* Register the CP15 based counter if we have one */\n> \tif (type & ARCH_TIMER_TYPE_CP15) {\n> -\t\tif (arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI)\n> +\t\tif (arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI) {\n> \t\t\tarch_timer_read_counter = arch_counter_get_cntvct;\n> -\t\telse\n> +\t\t} else {\n> \t\t\tarch_timer_read_counter = arch_counter_get_cntpct;\n> +\t\t\tstatic_branch_enable(&arch_timer_phys_counter_available);\n> +\t\t}\n> \n> \t\tclocksource_counter.archdata.vdso_direct = vdso_default;\n> \t} else {\n> \n\nIn my reply to patch #2, I had the following hunk:\n\n@@ -310,7 +329,7 @@ static void erratum_set_next_event_tval_generic(const int access, unsigned long\n \t\t\t\t\t\tstruct clock_event_device *clk)\n {\n \tunsigned long ctrl;\n-\tu64 cval = evt + arch_counter_get_cntvct();\n+\tu64 cval = evt + arch_timer_read_counter();\n \n \tctrl = arch_timer_reg_read(access, ARCH_TIMER_REG_CTRL, clk);\n \tctrl |= ARCH_TIMER_CTRL_ENABLE;\n\nOnce we start using a different timer, this could well have an effect...\n\nThanks,\n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"FyKU5wmG\"; \n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=infradead.org header.i=@infradead.org\n\theader.b=\"tIv4SzHT\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9lwP0wW5z9tXx\n\tfor ;\n\tTue, 10 Oct 2017 03:25:41 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1as6-0003Vs-VW; Mon, 09 Oct 2017 16:25:39 +0000","from casper.infradead.org ([2001:8b0:10b:1236::1])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1as5-0003Qc-Cb for linux-arm-kernel@bombadil.infradead.org;\n\tMon, 09 Oct 2017 16:25:37 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby casper.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1aoP-0003P4-6J for linux-arm-kernel@lists.infradead.org;\n\tMon, 09 Oct 2017 16:21:51 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0175A1529;\n\tMon, 9 Oct 2017 09:21:28 -0700 (PDT)","from [10.1.207.62] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tF284A3F483; Mon, 9 Oct 2017 09:21:25 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=F4fl+M8Yy4SirUCaEKQCkxCWEXo+r0p3ABRiUHAX3eI=;\n\tb=FyKU5wmG0FgdKD\n\toN1txoFRKJ9Vcbqkx1otJyOzKpUQlikN3zH2hZlhgeQolfPuPzpbBdYXqpoH5On3cEjveIExVNlQj\n\tF8iyYKC78r3h1jYAaAPCmzygR/AC9XorKHxXqyZ++eIBRRY5MCWARpLKTkPdl2/aoH37CCiAzAr1/\n\ti+0tbcRm/3E8LE/YWk/uIvJjaqzZT/Du/Q34KmHvhns0MCT/jeNEYnsChH2wRNOHwxMU9m6VLc6UN\n\teAZl8mlpMPtiO+0Q9qW2MYoL4LP+FcFnhB0KlRPua4+/YBXIYY955jZdIemYaetdyrtrus1HhNjYM\n\t/85M+g17H3/q4fTW6jZQ==;","v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=infradead.org; s=casper.20170209;\n\th=Content-Transfer-Encoding:Content-Type:\n\tIn-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender\n\t:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:\n\tResent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:\n\tList-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;\n\tbh=+/IPfHEx3xH8rMl66wg+iYS5dVc0lFyvHyoTCPZ71Jc=;\n\tb=tIv4SzHT6bCce+2Irf6gWnpv6T\n\tc20e6V4kPOUB7COnYtafEcSTny+vNB3M9NaD7CTpBUClz6PuWLyTHWpF14Z9DFJam1LW/kz00q6Md\n\tr35KkwYyf71tquy3dSr/irvd68XH2fDNzGPegwQL+4ZfSuAzVT+fLIEaIBoAAZLYJk6r3KmuQBNkU\n\tQS3/y/bPYucxny74ERgkfbMSHXo+QMk05YxYyqaPUOchI8V6k0objcig84f/p/tgNYzdUNIrZd4KB\n\tVcHtXX+rQMgAtwjb+W3SCBcccMlsDTCajg2x+RcPN7czcRf+/K3jLJWeUGNKceqrL5xTmlGYrPBpS\n\tY8Ba/2LQ==;"],"Subject":"Re: [PATCH v3 03/20] arm64: Use the physical counter when available\n\tfor read_cycles","To":"Christoffer Dall , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-4-cdall@linaro.org>","From":"Marc Zyngier ","Organization":"ARM Ltd","Message-ID":"<9b06425f-7c2c-d44a-cd6c-aeaa4b76849c@arm.com>","Date":"Mon, 9 Oct 2017 17:21:24 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170923004207.22356-4-cdall@linaro.org>","Content-Language":"en-GB","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171009_172149_535968_542AF5ED ","X-CRM114-Status":"GOOD ( 35.40 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on casper.infradead.org summary:\n\tContent analysis details: (-6.9 points, 5.0 required)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Mark Rutland ,\n\tCatalin Marinas ,\n\tWill Deacon , kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1782984,"web_url":"http://patchwork.ozlabs.org/comment/1782984/","msgid":"<0c3d0855-83d6-645a-23d3-ae143b8a05a9@arm.com>","list_archive_url":null,"date":"2017-10-09T16:37:43","subject":"Re: [PATCH v3 05/20] KVM: arm/arm64: Support calling\n\tvgic_update_irq_pending from irq context","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On 23/09/17 01:41, Christoffer Dall wrote:\n> We are about to optimize our timer handling logic which involves\n> injecting irqs to the vgic directly from the irq handler.\n> \n> Unfortunately, the injection path can take any AP list lock and irq lock\n> and we must therefore make sure to use spin_lock_irqsave where ever\n> interrupts are enabled and we are taking any of those locks, to avoid\n> deadlocking between process context and the ISR.\n> \n> This changes a lot of the VGIC code, but The good news are that the\n> changes are mostly mechanical.\n> \n> Signed-off-by: Christoffer Dall \n> ---\n> virt/kvm/arm/vgic/vgic-its.c | 17 +++++++-----\n> virt/kvm/arm/vgic/vgic-mmio-v2.c | 22 +++++++++------\n> virt/kvm/arm/vgic/vgic-mmio-v3.c | 17 +++++++-----\n> virt/kvm/arm/vgic/vgic-mmio.c | 44 +++++++++++++++++------------\n> virt/kvm/arm/vgic/vgic-v2.c | 5 ++--\n> virt/kvm/arm/vgic/vgic-v3.c | 12 ++++----\n> virt/kvm/arm/vgic/vgic.c | 60 +++++++++++++++++++++++++---------------\n> virt/kvm/arm/vgic/vgic.h | 3 +-\n> 8 files changed, 108 insertions(+), 72 deletions(-)\n> \n> diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c\n> index f51c1e1..9f5e347 100644\n> --- a/virt/kvm/arm/vgic/vgic-its.c\n> +++ b/virt/kvm/arm/vgic/vgic-its.c\n> @@ -278,6 +278,7 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,\n> \tu64 propbase = GICR_PROPBASER_ADDRESS(kvm->arch.vgic.propbaser);\n> \tu8 prop;\n> \tint ret;\n> +\tunsigned long flags;\n> \n> \tret = kvm_read_guest(kvm, propbase + irq->intid - GIC_LPI_OFFSET,\n> \t\t\t &prop, 1);\n> @@ -285,15 +286,15 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,\n> \tif (ret)\n> \t\treturn ret;\n> \n> -\tspin_lock(&irq->irq_lock);\n> +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \tif (!filter_vcpu || filter_vcpu == irq->target_vcpu) {\n> \t\tirq->priority = LPI_PROP_PRIORITY(prop);\n> \t\tirq->enabled = LPI_PROP_ENABLE_BIT(prop);\n> \n> -\t\tvgic_queue_irq_unlock(kvm, irq);\n> +\t\tvgic_queue_irq_unlock(kvm, irq, flags);\n> \t} else {\n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \t}\n> \n> \treturn 0;\n> @@ -393,6 +394,7 @@ static int its_sync_lpi_pending_table(struct kvm_vcpu *vcpu)\n> \tint ret = 0;\n> \tu32 *intids;\n> \tint nr_irqs, i;\n> +\tunsigned long flags;\n> \n> \tnr_irqs = vgic_copy_lpi_list(vcpu, &intids);\n> \tif (nr_irqs < 0)\n> @@ -420,9 +422,9 @@ static int its_sync_lpi_pending_table(struct kvm_vcpu *vcpu)\n> \t\t}\n> \n> \t\tirq = vgic_get_irq(vcpu->kvm, NULL, intids[i]);\n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \t\tirq->pending_latch = pendmask & (1U << bit_nr);\n> -\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> +\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> \n> @@ -515,6 +517,7 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,\n> {\n> \tstruct kvm_vcpu *vcpu;\n> \tstruct its_ite *ite;\n> +\tunsigned long flags;\n> \n> \tif (!its->enabled)\n> \t\treturn -EBUSY;\n> @@ -530,9 +533,9 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,\n> \tif (!vcpu->arch.vgic_cpu.lpis_enabled)\n> \t\treturn -EBUSY;\n> \n> -\tspin_lock(&ite->irq->irq_lock);\n> +\tspin_lock_irqsave(&ite->irq->irq_lock, flags);\n> \tite->irq->pending_latch = true;\n> -\tvgic_queue_irq_unlock(kvm, ite->irq);\n> +\tvgic_queue_irq_unlock(kvm, ite->irq, flags);\n> \n> \treturn 0;\n> }\n> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c\n> index b3d4a10..e21e2f4 100644\n> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c\n> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c\n> @@ -74,6 +74,7 @@ static void vgic_mmio_write_sgir(struct kvm_vcpu *source_vcpu,\n> \tint mode = (val >> 24) & 0x03;\n> \tint c;\n> \tstruct kvm_vcpu *vcpu;\n> +\tunsigned long flags;\n> \n> \tswitch (mode) {\n> \tcase 0x0:\t\t/* as specified by targets */\n> @@ -97,11 +98,11 @@ static void vgic_mmio_write_sgir(struct kvm_vcpu *source_vcpu,\n> \n> \t\tirq = vgic_get_irq(source_vcpu->kvm, vcpu, intid);\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \t\tirq->pending_latch = true;\n> \t\tirq->source |= 1U << source_vcpu->vcpu_id;\n> \n> -\t\tvgic_queue_irq_unlock(source_vcpu->kvm, irq);\n> +\t\tvgic_queue_irq_unlock(source_vcpu->kvm, irq, flags);\n> \t\tvgic_put_irq(source_vcpu->kvm, irq);\n> \t}\n> }\n> @@ -131,6 +132,7 @@ static void vgic_mmio_write_target(struct kvm_vcpu *vcpu,\n> \tu32 intid = VGIC_ADDR_TO_INTID(addr, 8);\n> \tu8 cpu_mask = GENMASK(atomic_read(&vcpu->kvm->online_vcpus) - 1, 0);\n> \tint i;\n> +\tunsigned long flags;\n> \n> \t/* GICD_ITARGETSR[0-7] are read-only */\n> \tif (intid < VGIC_NR_PRIVATE_IRQS)\n> @@ -140,13 +142,13 @@ static void vgic_mmio_write_target(struct kvm_vcpu *vcpu,\n> \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, NULL, intid + i);\n> \t\tint target;\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \t\tirq->targets = (val >> (i * 8)) & cpu_mask;\n> \t\ttarget = irq->targets ? __ffs(irq->targets) : 0;\n> \t\tirq->target_vcpu = kvm_get_vcpu(vcpu->kvm, target);\n> \n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> }\n> @@ -174,17 +176,18 @@ static void vgic_mmio_write_sgipendc(struct kvm_vcpu *vcpu,\n> {\n> \tu32 intid = addr & 0x0f;\n> \tint i;\n> +\tunsigned long flags;\n> \n> \tfor (i = 0; i < len; i++) {\n> \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \t\tirq->source &= ~((val >> (i * 8)) & 0xff);\n> \t\tif (!irq->source)\n> \t\t\tirq->pending_latch = false;\n> \n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> }\n> @@ -195,19 +198,20 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu,\n> {\n> \tu32 intid = addr & 0x0f;\n> \tint i;\n> +\tunsigned long flags;\n> \n> \tfor (i = 0; i < len; i++) {\n> \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \t\tirq->source |= (val >> (i * 8)) & 0xff;\n> \n> \t\tif (irq->source) {\n> \t\t\tirq->pending_latch = true;\n> -\t\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> +\t\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> \t\t} else {\n> -\t\t\tspin_unlock(&irq->irq_lock);\n> +\t\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \t\t}\n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c\n> index 408ef06..8378610 100644\n> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c\n> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c\n> @@ -129,6 +129,7 @@ static void vgic_mmio_write_irouter(struct kvm_vcpu *vcpu,\n> {\n> \tint intid = VGIC_ADDR_TO_INTID(addr, 64);\n> \tstruct vgic_irq *irq;\n> +\tunsigned long flags;\n> \n> \t/* The upper word is WI for us since we don't implement Aff3. */\n> \tif (addr & 4)\n> @@ -139,13 +140,13 @@ static void vgic_mmio_write_irouter(struct kvm_vcpu *vcpu,\n> \tif (!irq)\n> \t\treturn;\n> \n> -\tspin_lock(&irq->irq_lock);\n> +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \t/* We only care about and preserve Aff0, Aff1 and Aff2. */\n> \tirq->mpidr = val & GENMASK(23, 0);\n> \tirq->target_vcpu = kvm_mpidr_to_vcpu(vcpu->kvm, irq->mpidr);\n> \n> -\tspin_unlock(&irq->irq_lock);\n> +\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \tvgic_put_irq(vcpu->kvm, irq);\n> }\n> \n> @@ -241,11 +242,12 @@ static void vgic_v3_uaccess_write_pending(struct kvm_vcpu *vcpu,\n> {\n> \tu32 intid = VGIC_ADDR_TO_INTID(addr, 1);\n> \tint i;\n> +\tunsigned long flags;\n> \n> \tfor (i = 0; i < len * 8; i++) {\n> \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \t\tif (test_bit(i, &val)) {\n> \t\t\t/*\n> \t\t\t * pending_latch is set irrespective of irq type\n> @@ -253,10 +255,10 @@ static void vgic_v3_uaccess_write_pending(struct kvm_vcpu *vcpu,\n> \t\t\t * restore irq config before pending info.\n> \t\t\t */\n> \t\t\tirq->pending_latch = true;\n> -\t\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> +\t\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> \t\t} else {\n> \t\t\tirq->pending_latch = false;\n> -\t\t\tspin_unlock(&irq->irq_lock);\n> +\t\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \t\t}\n> \n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> @@ -799,6 +801,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)\n> \tint sgi, c;\n> \tint vcpu_id = vcpu->vcpu_id;\n> \tbool broadcast;\n> +\tunsigned long flags;\n> \n> \tsgi = (reg & ICC_SGI1R_SGI_ID_MASK) >> ICC_SGI1R_SGI_ID_SHIFT;\n> \tbroadcast = reg & BIT_ULL(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);\n> @@ -837,10 +840,10 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)\n> \n> \t\tirq = vgic_get_irq(vcpu->kvm, c_vcpu, sgi);\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \t\tirq->pending_latch = true;\n> \n> -\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> +\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> }\n> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c\n> index c1e4bdd..deb51ee 100644\n> --- a/virt/kvm/arm/vgic/vgic-mmio.c\n> +++ b/virt/kvm/arm/vgic/vgic-mmio.c\n> @@ -69,13 +69,14 @@ void vgic_mmio_write_senable(struct kvm_vcpu *vcpu,\n> {\n> \tu32 intid = VGIC_ADDR_TO_INTID(addr, 1);\n> \tint i;\n> +\tunsigned long flags;\n> \n> \tfor_each_set_bit(i, &val, len * 8) {\n> \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \t\tirq->enabled = true;\n> -\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> +\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> \n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> @@ -87,15 +88,16 @@ void vgic_mmio_write_cenable(struct kvm_vcpu *vcpu,\n> {\n> \tu32 intid = VGIC_ADDR_TO_INTID(addr, 1);\n> \tint i;\n> +\tunsigned long flags;\n> \n> \tfor_each_set_bit(i, &val, len * 8) {\n> \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \t\tirq->enabled = false;\n> \n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> }\n> @@ -126,14 +128,15 @@ void vgic_mmio_write_spending(struct kvm_vcpu *vcpu,\n> {\n> \tu32 intid = VGIC_ADDR_TO_INTID(addr, 1);\n> \tint i;\n> +\tunsigned long flags;\n> \n> \tfor_each_set_bit(i, &val, len * 8) {\n> \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \t\tirq->pending_latch = true;\n> \n> -\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> +\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> }\n> @@ -144,15 +147,16 @@ void vgic_mmio_write_cpending(struct kvm_vcpu *vcpu,\n> {\n> \tu32 intid = VGIC_ADDR_TO_INTID(addr, 1);\n> \tint i;\n> +\tunsigned long flags;\n> \n> \tfor_each_set_bit(i, &val, len * 8) {\n> \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \t\tirq->pending_latch = false;\n> \n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> }\n> @@ -181,7 +185,8 @@ static void vgic_mmio_change_active(struct kvm_vcpu *vcpu, struct vgic_irq *irq,\n> \t\t\t\t bool new_active_state)\n> {\n> \tstruct kvm_vcpu *requester_vcpu;\n> -\tspin_lock(&irq->irq_lock);\n> +\tunsigned long flags;\n> +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \t/*\n> \t * The vcpu parameter here can mean multiple things depending on how\n> @@ -216,9 +221,9 @@ static void vgic_mmio_change_active(struct kvm_vcpu *vcpu, struct vgic_irq *irq,\n> \n> \tirq->active = new_active_state;\n> \tif (new_active_state)\n> -\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> +\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> \telse\n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> }\n> \n> /*\n> @@ -352,14 +357,15 @@ void vgic_mmio_write_priority(struct kvm_vcpu *vcpu,\n> {\n> \tu32 intid = VGIC_ADDR_TO_INTID(addr, 8);\n> \tint i;\n> +\tunsigned long flags;\n> \n> \tfor (i = 0; i < len; i++) {\n> \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \t\t/* Narrow the priority range to what we actually support */\n> \t\tirq->priority = (val >> (i * 8)) & GENMASK(7, 8 - VGIC_PRI_BITS);\n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> @@ -390,6 +396,7 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,\n> {\n> \tu32 intid = VGIC_ADDR_TO_INTID(addr, 2);\n> \tint i;\n> +\tunsigned long flags;\n> \n> \tfor (i = 0; i < len * 4; i++) {\n> \t\tstruct vgic_irq *irq;\n> @@ -404,14 +411,14 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,\n> \t\t\tcontinue;\n> \n> \t\tirq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \t\tif (test_bit(i * 2 + 1, &val))\n> \t\t\tirq->config = VGIC_CONFIG_EDGE;\n> \t\telse\n> \t\t\tirq->config = VGIC_CONFIG_LEVEL;\n> \n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> }\n> @@ -443,6 +450,7 @@ void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,\n> {\n> \tint i;\n> \tint nr_irqs = vcpu->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;\n> +\tunsigned long flags;\n> \n> \tfor (i = 0; i < 32; i++) {\n> \t\tstruct vgic_irq *irq;\n> @@ -459,12 +467,12 @@ void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,\n> \t\t * restore irq config before line level.\n> \t\t */\n> \t\tnew_level = !!(val & (1U << i));\n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \t\tirq->line_level = new_level;\n> \t\tif (new_level)\n> -\t\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> +\t\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> \t\telse\n> -\t\t\tspin_unlock(&irq->irq_lock);\n> +\t\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c\n> index e4187e5..8089710 100644\n> --- a/virt/kvm/arm/vgic/vgic-v2.c\n> +++ b/virt/kvm/arm/vgic/vgic-v2.c\n> @@ -62,6 +62,7 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu)\n> \tstruct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;\n> \tstruct vgic_v2_cpu_if *cpuif = &vgic_cpu->vgic_v2;\n> \tint lr;\n> +\tunsigned long flags;\n> \n> \tcpuif->vgic_hcr &= ~GICH_HCR_UIE;\n> \n> @@ -77,7 +78,7 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu)\n> \n> \t\tirq = vgic_get_irq(vcpu->kvm, vcpu, intid);\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \t\t/* Always preserve the active bit */\n> \t\tirq->active = !!(val & GICH_LR_ACTIVE_BIT);\n> @@ -104,7 +105,7 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu)\n> \t\t\t\tirq->pending_latch = false;\n> \t\t}\n> \n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> \n> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c\n> index 96ea597..863351c 100644\n> --- a/virt/kvm/arm/vgic/vgic-v3.c\n> +++ b/virt/kvm/arm/vgic/vgic-v3.c\n> @@ -44,6 +44,7 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)\n> \tstruct vgic_v3_cpu_if *cpuif = &vgic_cpu->vgic_v3;\n> \tu32 model = vcpu->kvm->arch.vgic.vgic_model;\n> \tint lr;\n> +\tunsigned long flags;\n> \n> \tcpuif->vgic_hcr &= ~ICH_HCR_UIE;\n> \n> @@ -66,7 +67,7 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)\n> \t\tif (!irq)\t/* An LPI could have been unmapped. */\n> \t\t\tcontinue;\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \t\t/* Always preserve the active bit */\n> \t\tirq->active = !!(val & ICH_LR_ACTIVE_BIT);\n> @@ -94,7 +95,7 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)\n> \t\t\t\tirq->pending_latch = false;\n> \t\t}\n> \n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \t\tvgic_put_irq(vcpu->kvm, irq);\n> \t}\n> \n> @@ -278,6 +279,7 @@ int vgic_v3_lpi_sync_pending_status(struct kvm *kvm, struct vgic_irq *irq)\n> \tbool status;\n> \tu8 val;\n> \tint ret;\n> +\tunsigned long flags;\n> \n> retry:\n> \tvcpu = irq->target_vcpu;\n> @@ -296,13 +298,13 @@ int vgic_v3_lpi_sync_pending_status(struct kvm *kvm, struct vgic_irq *irq)\n> \n> \tstatus = val & (1 << bit_nr);\n> \n> -\tspin_lock(&irq->irq_lock);\n> +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \tif (irq->target_vcpu != vcpu) {\n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \t\tgoto retry;\n> \t}\n> \tirq->pending_latch = status;\n> -\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> +\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> \n> \tif (status) {\n> \t\t/* clear consumed data */\n> diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c\n> index e1f7dbc..b1bd238 100644\n> --- a/virt/kvm/arm/vgic/vgic.c\n> +++ b/virt/kvm/arm/vgic/vgic.c\n> @@ -53,6 +53,10 @@ struct vgic_global kvm_vgic_global_state __ro_after_init = {\n> * vcpuX->vcpu_id < vcpuY->vcpu_id:\n> * spin_lock(vcpuX->arch.vgic_cpu.ap_list_lock);\n> * spin_lock(vcpuY->arch.vgic_cpu.ap_list_lock);\n> + *\n> + * Since the VGIC must support injecting virtual interrupts from ISRs, we have\n> + * to use the spin_lock_irqsave/spin_unlock_irqrestore versions of outer\n> + * spinlocks for any lock that may be taken while injecting an interrupt.\n> */\n> \n> /*\n> @@ -261,7 +265,8 @@ static bool vgic_validate_injection(struct vgic_irq *irq, bool level, void *owne\n> * Needs to be entered with the IRQ lock already held, but will return\n> * with all locks dropped.\n> */\n> -bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq)\n> +bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq,\n> +\t\t\t unsigned long flags)\n> {\n> \tstruct kvm_vcpu *vcpu;\n> \n> @@ -279,7 +284,7 @@ bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq)\n> \t\t * not need to be inserted into an ap_list and there is also\n> \t\t * no more work for us to do.\n> \t\t */\n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \n> \t\t/*\n> \t\t * We have to kick the VCPU here, because we could be\n> @@ -301,11 +306,11 @@ bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq)\n> \t * We must unlock the irq lock to take the ap_list_lock where\n> \t * we are going to insert this new pending interrupt.\n> \t */\n> -\tspin_unlock(&irq->irq_lock);\n> +\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \n> \t/* someone can do stuff here, which we re-check below */\n> \n> -\tspin_lock(&vcpu->arch.vgic_cpu.ap_list_lock);\n> +\tspin_lock_irqsave(&vcpu->arch.vgic_cpu.ap_list_lock, flags);\n> \tspin_lock(&irq->irq_lock);\n> \n> \t/*\n> @@ -322,9 +327,9 @@ bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq)\n> \n> \tif (unlikely(irq->vcpu || vcpu != vgic_target_oracle(irq))) {\n> \t\tspin_unlock(&irq->irq_lock);\n> -\t\tspin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock);\n> +\t\tspin_unlock_irqrestore(&vcpu->arch.vgic_cpu.ap_list_lock, flags);\n> \n> -\t\tspin_lock(&irq->irq_lock);\n> +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \t\tgoto retry;\n> \t}\n> \n> @@ -337,7 +342,7 @@ bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq)\n> \tirq->vcpu = vcpu;\n> \n> \tspin_unlock(&irq->irq_lock);\n> -\tspin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock);\n> +\tspin_unlock_irqrestore(&vcpu->arch.vgic_cpu.ap_list_lock, flags);\n> \n> \tkvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);\n> \tkvm_vcpu_kick(vcpu);\n> @@ -367,6 +372,7 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,\n> {\n> \tstruct kvm_vcpu *vcpu;\n> \tstruct vgic_irq *irq;\n> +\tunsigned long flags;\n> \tint ret;\n> \n> \ttrace_vgic_update_irq_pending(cpuid, intid, level);\n> @@ -383,11 +389,11 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,\n> \tif (!irq)\n> \t\treturn -EINVAL;\n> \n> -\tspin_lock(&irq->irq_lock);\n> +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \tif (!vgic_validate_injection(irq, level, owner)) {\n> \t\t/* Nothing to see here, move along... */\n> -\t\tspin_unlock(&irq->irq_lock);\n> +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \t\tvgic_put_irq(kvm, irq);\n> \t\treturn 0;\n> \t}\n> @@ -397,7 +403,7 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,\n> \telse\n> \t\tirq->pending_latch = true;\n> \n> -\tvgic_queue_irq_unlock(kvm, irq);\n> +\tvgic_queue_irq_unlock(kvm, irq, flags);\n> \tvgic_put_irq(kvm, irq);\n> \n> \treturn 0;\n> @@ -406,15 +412,16 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,\n> int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, u32 virt_irq, u32 phys_irq)\n> {\n> \tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);\n> +\tunsigned long flags;\n> \n> \tBUG_ON(!irq);\n> \n> -\tspin_lock(&irq->irq_lock);\n> +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \tirq->hw = true;\n> \tirq->hwintid = phys_irq;\n> \n> -\tspin_unlock(&irq->irq_lock);\n> +\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \tvgic_put_irq(vcpu->kvm, irq);\n> \n> \treturn 0;\n> @@ -423,6 +430,7 @@ int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, u32 virt_irq, u32 phys_irq)\n> int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int virt_irq)\n> {\n> \tstruct vgic_irq *irq;\n> +\tunsigned long flags;\n> \n> \tif (!vgic_initialized(vcpu->kvm))\n> \t\treturn -EAGAIN;\n> @@ -430,12 +438,12 @@ int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int virt_irq)\n> \tirq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);\n> \tBUG_ON(!irq);\n> \n> -\tspin_lock(&irq->irq_lock);\n> +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> \tirq->hw = false;\n> \tirq->hwintid = 0;\n> \n> -\tspin_unlock(&irq->irq_lock);\n> +\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \tvgic_put_irq(vcpu->kvm, irq);\n> \n> \treturn 0;\n> @@ -486,9 +494,10 @@ static void vgic_prune_ap_list(struct kvm_vcpu *vcpu)\n> {\n> \tstruct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;\n> \tstruct vgic_irq *irq, *tmp;\n> +\tunsigned long flags;\n> \n> retry:\n> -\tspin_lock(&vgic_cpu->ap_list_lock);\n> +\tspin_lock_irqsave(&vgic_cpu->ap_list_lock, flags);\n> \n> \tlist_for_each_entry_safe(irq, tmp, &vgic_cpu->ap_list_head, ap_list) {\n> \t\tstruct kvm_vcpu *target_vcpu, *vcpuA, *vcpuB;\n> @@ -528,7 +537,7 @@ static void vgic_prune_ap_list(struct kvm_vcpu *vcpu)\n> \t\t/* This interrupt looks like it has to be migrated. */\n> \n> \t\tspin_unlock(&irq->irq_lock);\n> -\t\tspin_unlock(&vgic_cpu->ap_list_lock);\n> +\t\tspin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags);\n> \n> \t\t/*\n> \t\t * Ensure locking order by always locking the smallest\n> @@ -542,7 +551,7 @@ static void vgic_prune_ap_list(struct kvm_vcpu *vcpu)\n> \t\t\tvcpuB = vcpu;\n> \t\t}\n> \n> -\t\tspin_lock(&vcpuA->arch.vgic_cpu.ap_list_lock);\n> +\t\tspin_lock_irqsave(&vcpuA->arch.vgic_cpu.ap_list_lock, flags);\n> \t\tspin_lock_nested(&vcpuB->arch.vgic_cpu.ap_list_lock,\n> \t\t\t\t SINGLE_DEPTH_NESTING);\n> \t\tspin_lock(&irq->irq_lock);\n> @@ -566,11 +575,11 @@ static void vgic_prune_ap_list(struct kvm_vcpu *vcpu)\n> \n> \t\tspin_unlock(&irq->irq_lock);\n> \t\tspin_unlock(&vcpuB->arch.vgic_cpu.ap_list_lock);\n> -\t\tspin_unlock(&vcpuA->arch.vgic_cpu.ap_list_lock);\n> +\t\tspin_unlock_irqrestore(&vcpuA->arch.vgic_cpu.ap_list_lock, flags);\n> \t\tgoto retry;\n> \t}\n> \n> -\tspin_unlock(&vgic_cpu->ap_list_lock);\n> +\tspin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags);\n> }\n> \n> static inline void vgic_fold_lr_state(struct kvm_vcpu *vcpu)\n> @@ -703,6 +712,8 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)\n> \tif (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))\n> \t\treturn;\n> \n> +\tDEBUG_SPINLOCK_BUG_ON(!irqs_disabled());\n> +\n> \tspin_lock(&vcpu->arch.vgic_cpu.ap_list_lock);\n> \tvgic_flush_lr_state(vcpu);\n> \tspin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock);\n> @@ -735,11 +746,12 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)\n> \tstruct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;\n> \tstruct vgic_irq *irq;\n> \tbool pending = false;\n> +\tunsigned long flags;\n> \n> \tif (!vcpu->kvm->arch.vgic.enabled)\n> \t\treturn false;\n> \n> -\tspin_lock(&vgic_cpu->ap_list_lock);\n> +\tspin_lock_irqsave(&vgic_cpu->ap_list_lock, flags);\n> \n> \tlist_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) {\n> \t\tspin_lock(&irq->irq_lock);\n> @@ -750,7 +762,7 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)\n> \t\t\tbreak;\n> \t}\n> \n> -\tspin_unlock(&vgic_cpu->ap_list_lock);\n> +\tspin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags);\n> \n> \treturn pending;\n> }\n> @@ -776,13 +788,15 @@ bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int virt_irq)\n> {\n> \tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);\n> \tbool map_is_active;\n> +\tunsigned long flags;\n> \n> \tif (!vgic_initialized(vcpu->kvm))\n> \t\treturn false;\n> +\tDEBUG_SPINLOCK_BUG_ON(!irqs_disabled());\n> \n> -\tspin_lock(&irq->irq_lock);\n> +\tspin_lock_irqsave(&irq->irq_lock, flags);\n\nI'm a bit puzzled by this sequence: Either interrupts are disabled and\nwe don't need the irqsave version, or they aren't and the BUG_ON will\nfire. kvm_vgic_map_is_active is called (indirectly) from\nkvm_timer_flush_hwstate. And at this stage of the patches, we definitely\ncall this function with interrupts enabled.\n\nIs it just a patch splitting snafu? Or something more serious? Same goes\nfor the DEBUG_SPINLOCK_BUG_ON in kvm_vgic_flush_hwstate.\n\n> \tmap_is_active = irq->hw && irq->active;\n> -\tspin_unlock(&irq->irq_lock);\n> +\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> \tvgic_put_irq(vcpu->kvm, irq);\n> \n> \treturn map_is_active;\n> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h\n> index bf9ceab..4f8aecb 100644\n> --- a/virt/kvm/arm/vgic/vgic.h\n> +++ b/virt/kvm/arm/vgic/vgic.h\n> @@ -140,7 +140,8 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,\n> struct vgic_irq *vgic_get_irq(struct kvm *kvm, struct kvm_vcpu *vcpu,\n> \t\t\t u32 intid);\n> void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq);\n> -bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq);\n> +bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq,\n> +\t\t\t unsigned long flags);\n> void vgic_kick_vcpus(struct kvm *kvm);\n> \n> int vgic_check_ioaddr(struct kvm *kvm, phys_addr_t *ioaddr,\n> \n\nOtherwise looks good to me.\n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"ti6nJ5kp\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9mBy6w2rz9tXx\n\tfor ;\n\tTue, 10 Oct 2017 03:38:18 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1b4H-0003BB-Il; Mon, 09 Oct 2017 16:38:13 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1b4B-0002Np-Iu for linux-arm-kernel@lists.infradead.org;\n\tMon, 09 Oct 2017 16:38:11 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6F0361529;\n\tMon, 9 Oct 2017 09:37:46 -0700 (PDT)","from [10.1.207.62] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t126C23F483; Mon, 9 Oct 2017 09:37:44 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=5uopd3lx6PJ9JzMG6EH6Z1bn0lLNAEYZ7QFU45Bcku8=;\n\tb=ti6nJ5kpIQ5RHP\n\txP0hk0NzA0cngHGWSGIb+Lfmddhjlgznu3kNRbjJogpDN/xdKLknbVS449ALIlNnsWfo1Z6SpcN3d\n\tgMs5QHXo6CEjA4Cpo7jkEFBjChDp+KHcpcOqRGhf1tjNjvJFA6TMuDx1e7ahwAkihvpcTR6dLFI+Z\n\th6bCsuK7aQ9dM3CwDvgkyK/dHBKh4myYCiMSp4tXdUqPMPM8+IjyLeduLF9e1CPUlFZe5s97AAH6A\n\tbP4XIHUqeiIFHOFGZNakPuxkSX4qDp07fgzMDR3uCA3kKMCeR1F0s3SilqcO8doBT3Datsvc/H2Qt\n\tkBro9aSRnU3NKqfqkRRw==;","Subject":"Re: [PATCH v3 05/20] KVM: arm/arm64: Support calling\n\tvgic_update_irq_pending from irq context","To":"Christoffer Dall , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-6-cdall@linaro.org>","From":"Marc Zyngier ","Organization":"ARM Ltd","Message-ID":"<0c3d0855-83d6-645a-23d3-ae143b8a05a9@arm.com>","Date":"Mon, 9 Oct 2017 17:37:43 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170923004207.22356-6-cdall@linaro.org>","Content-Language":"en-GB","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171009_093807_732598_5BD67DAD ","X-CRM114-Status":"GOOD ( 21.66 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1782997,"web_url":"http://patchwork.ozlabs.org/comment/1782997/","msgid":"","list_archive_url":null,"date":"2017-10-09T16:47:18","subject":"Re: [PATCH v3 06/20] KVM: arm/arm64: Check that system supports\n\tsplit eoi/deactivate","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On 23/09/17 01:41, Christoffer Dall wrote:\n> Some systems without proper firmware and/or hardware description data\n> don't support the split EOI and deactivate operation.\n> \n> On such systems, we cannot leave the physical interrupt active after the\n> timer handler on the host has run, so we cannot support KVM with an\n> in-kernel GIC with the timer changes we are about to introduce.\n> \n> This patch makes sure that trying to initialize the KVM GIC code will\n> fail on such systems.\n> \n> Cc: Marc Zyngier \n> Signed-off-by: Christoffer Dall \n> ---\n> drivers/irqchip/irq-gic.c | 3 ++-\n> 1 file changed, 2 insertions(+), 1 deletion(-)\n> \n> diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c\n> index f641e8e..ab12bf4 100644\n> --- a/drivers/irqchip/irq-gic.c\n> +++ b/drivers/irqchip/irq-gic.c\n> @@ -1420,7 +1420,8 @@ static void __init gic_of_setup_kvm_info(struct device_node *node)\n> \tif (ret)\n> \t\treturn;\n> \n> -\tgic_set_kvm_info(&gic_v2_kvm_info);\n> +\tif (static_key_true(&supports_deactivate))\n> +\t\tgic_set_kvm_info(&gic_v2_kvm_info);\n> }\n> \n> int __init\n> \n\nShould we add the same level of checking on the ACPI path, just for the\nsake symmetry?\n\nAlso, do we need to add the same thing for GICv3?\n\nOtherwise looks OK to me.\n\nThanks,\n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"FDsdY6c8\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9mQ3561hz9t7D\n\tfor ;\n\tTue, 10 Oct 2017 03:47:55 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1bDZ-0007q1-1n; Mon, 09 Oct 2017 16:47:49 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1bDR-0007hu-7H for linux-arm-kernel@lists.infradead.org;\n\tMon, 09 Oct 2017 16:47:46 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D59C71529;\n\tMon, 9 Oct 2017 09:47:20 -0700 (PDT)","from [10.1.207.62] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tC06073F483; Mon, 9 Oct 2017 09:47:19 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=ZaZnA5iLrg9E2A/wM7Enahm9npdzoUNiQYKfOX0hoXQ=;\n\tb=FDsdY6c8RXyNhW\n\tzcVjDLM0hyuhP7mWxFinxFx+tpd8e8RJlPdWDDbedR23xPFRib6PhCvKbY5an9SO+ntR3qBCI+E0X\n\t9uAFn3oLG7GGgJtAq+s+1YHkKIZHzM0+fpBclKalr9Rc4XACA+sP+RPEqejigwqsBzG7USa8PRz4E\n\tA33Ln2Pvt7PhHbBdJcXySrXWtdcfepmrsaPdxgaXreR60ZjDpaNl/hmQZyvxsaQA5+BGo1wkFAC9J\n\tX++ApRKSz7RNOWVrf+6OQD+Pi5xieLGkwwpa400sOscmPcpzikpxZGJOpszngSkDESzTWbcbibIUK\n\tLNa5XWQGIIH1oxW2gkPA==;","Subject":"Re: [PATCH v3 06/20] KVM: arm/arm64: Check that system supports\n\tsplit eoi/deactivate","To":"Christoffer Dall , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-7-cdall@linaro.org>","From":"Marc Zyngier ","Organization":"ARM Ltd","Message-ID":"","Date":"Mon, 9 Oct 2017 17:47:18 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170923004207.22356-7-cdall@linaro.org>","Content-Language":"en-GB","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171009_094741_402013_E125E1C5 ","X-CRM114-Status":"GOOD ( 15.11 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783021,"web_url":"http://patchwork.ozlabs.org/comment/1783021/","msgid":"","list_archive_url":null,"date":"2017-10-09T17:05:04","subject":"Re: [PATCH v3 07/20] KVM: arm/arm64: Make timer_arm and timer_disarm\n\thelpers more generic","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On 23/09/17 01:41, Christoffer Dall wrote:\n> We are about to add an additional soft timer to the arch timer state for\n> a VCPU and would like to be able to reuse the functions to program and\n> cancel a timer, so we make them slightly more generic and rename to make\n> it more clear that these functions work on soft timers and not the\n> hardware resource that this code is managing.\n> \n> Signed-off-by: Christoffer Dall \n> ---\n> virt/kvm/arm/arch_timer.c | 33 ++++++++++++++++-----------------\n> 1 file changed, 16 insertions(+), 17 deletions(-)\n> \n> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> index 8e89d63..871d8ae 100644\n> --- a/virt/kvm/arm/arch_timer.c\n> +++ b/virt/kvm/arm/arch_timer.c\n> @@ -56,26 +56,22 @@ u64 kvm_phys_timer_read(void)\n> \treturn timecounter->cc->read(timecounter->cc);\n> }\n> \n> -static bool timer_is_armed(struct arch_timer_cpu *timer)\n> +static bool soft_timer_is_armed(struct arch_timer_cpu *timer)\n> {\n> \treturn timer->armed;\n> }\n> \n> -/* timer_arm: as in \"arm the timer\", not as in ARM the company */\n> -static void timer_arm(struct arch_timer_cpu *timer, u64 ns)\n> +static void soft_timer_start(struct hrtimer *hrt, u64 ns)\n> {\n> -\ttimer->armed = true;\n> -\thrtimer_start(&timer->timer, ktime_add_ns(ktime_get(), ns),\n> +\thrtimer_start(hrt, ktime_add_ns(ktime_get(), ns),\n> \t\t HRTIMER_MODE_ABS);\n> }\n> \n> -static void timer_disarm(struct arch_timer_cpu *timer)\n> +static void soft_timer_cancel(struct hrtimer *hrt, struct work_struct *work)\n> {\n> -\tif (timer_is_armed(timer)) {\n> -\t\thrtimer_cancel(&timer->timer);\n> -\t\tcancel_work_sync(&timer->expired);\n> -\t\ttimer->armed = false;\n> -\t}\n> +\thrtimer_cancel(hrt);\n> +\tif (work)\n\nWhen can this happen? Something in a following patch?\n\n> +\t\tcancel_work_sync(work);\n> }\n> \n> static irqreturn_t kvm_arch_timer_handler(int irq, void *dev_id)\n> @@ -271,7 +267,7 @@ static void kvm_timer_emulate(struct kvm_vcpu *vcpu,\n> \t\treturn;\n> \n> \t/* The timer has not yet expired, schedule a background timer */\n> -\ttimer_arm(timer, kvm_timer_compute_delta(timer_ctx));\n> +\tsoft_timer_start(&timer->timer, kvm_timer_compute_delta(timer_ctx));\n> }\n> \n> /*\n> @@ -285,7 +281,7 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)\n> \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> \tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> \n> -\tBUG_ON(timer_is_armed(timer));\n> +\tBUG_ON(soft_timer_is_armed(timer));\n> \n> \t/*\n> \t * No need to schedule a background timer if any guest timer has\n> @@ -306,13 +302,16 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)\n> \t * The guest timers have not yet expired, schedule a background timer.\n> \t * Set the earliest expiration time among the guest timers.\n> \t */\n> -\ttimer_arm(timer, kvm_timer_earliest_exp(vcpu));\n> +\ttimer->armed = true;\n> +\tsoft_timer_start(&timer->timer, kvm_timer_earliest_exp(vcpu));\n> }\n> \n> void kvm_timer_unschedule(struct kvm_vcpu *vcpu)\n> {\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> -\ttimer_disarm(timer);\n> +\n> +\tsoft_timer_cancel(&timer->timer, &timer->expired);\n> +\ttimer->armed = false;\n> }\n> \n> static void kvm_timer_flush_hwstate_vgic(struct kvm_vcpu *vcpu)\n> @@ -448,7 +447,7 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)\n> \t * This is to cancel the background timer for the physical timer\n> \t * emulation if it is set.\n> \t */\n> -\ttimer_disarm(timer);\n> +\tsoft_timer_cancel(&timer->timer, &timer->expired);\n\ntimer_disarm() used to set timer->armed to false, but that's not the\ncase any more. Don't we risk hitting the BUG_ON() in kvm_timer_schedule\nif we hit WFI?\n\n> \n> \t/*\n> \t * The guest could have modified the timer registers or the timer\n> @@ -615,7 +614,7 @@ void kvm_timer_vcpu_terminate(struct kvm_vcpu *vcpu)\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> \n> -\ttimer_disarm(timer);\n> +\tsoft_timer_cancel(&timer->timer, &timer->expired);\n> \tkvm_vgic_unmap_phys_irq(vcpu, vtimer->irq.irq);\n> }\n> \n> \n\nThanks,\n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"Y3EGJ7B2\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9mpR65Rnz9tXx\n\tfor ;\n\tTue, 10 Oct 2017 04:05:35 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1bUg-000814-R8; Mon, 09 Oct 2017 17:05:30 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1bUd-0007ZJ-JP for linux-arm-kernel@lists.infradead.org;\n\tMon, 09 Oct 2017 17:05:29 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 544BB1529;\n\tMon, 9 Oct 2017 10:05:07 -0700 (PDT)","from [10.1.207.62] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t42AC43F483; Mon, 9 Oct 2017 10:05:06 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=xvVoP8mGAPb0gp/Trp0/OdFeqqrRiPvxSUMosguwEhk=;\n\tb=Y3EGJ7B2MU/g7J\n\tsLwiCkKMxHP1LZDnO2Wgb9zN8LWHshVd4jcz3uvavw1pXT3/rciZyuSBhDcAXW10s7J7bejYvLjsq\n\tTbj+AsB7jS4/LQwPrnR1wXiRl7XEYi0y2blYpGvwrWBABbMe7o878n6dX9BF97e3aScyxNP+I+AOM\n\t/iqGEFqupRrn+xsJXdU7BmFIrROpARd7JNnVMzw725dYwix6VjEslSmPFA4htgFjHkb3ogKvi2Tbr\n\tZ+NkAjDf4E+5zWlYKg5+bO0XRIEewUEhUwMYXdULNkyucxf3O/Veg5RllgYE+IYyhZBWDW0yhKH0Q\n\tRcX0Mmxu9776dTm7ZPnA==;","Subject":"Re: [PATCH v3 07/20] KVM: arm/arm64: Make timer_arm and timer_disarm\n\thelpers more generic","To":"Christoffer Dall , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-8-cdall@linaro.org>","From":"Marc Zyngier ","Organization":"ARM Ltd","Message-ID":"","Date":"Mon, 9 Oct 2017 18:05:04 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170923004207.22356-8-cdall@linaro.org>","Content-Language":"en-GB","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171009_100527_656883_9489E3E9 ","X-CRM114-Status":"GOOD ( 22.08 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783060,"web_url":"http://patchwork.ozlabs.org/comment/1783060/","msgid":"","list_archive_url":null,"date":"2017-10-09T17:23:45","subject":"Re: [PATCH v3 09/20] KVM: arm/arm64: Use separate timer for phys\n\ttimer emulation","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On 23/09/17 01:41, Christoffer Dall wrote:\n> We were using the same hrtimer for emulating the physical timer and for\n> making sure a blocking VCPU thread would be eventually woken up. That\n> worked fine in the previous arch timer design, but as we are about to\n> actually use the soft timer expire function for the physical timer\n> emulation, change the logic to use a dedicated hrtimer.\n> \n> This has the added benefit of not having to cancel any work in the sync\n> path, which in turn allows us to run the flush and sync with IRQs\n> disabled.\n> \n> Signed-off-by: Christoffer Dall \n> ---\n> include/kvm/arm_arch_timer.h | 3 +++\n> virt/kvm/arm/arch_timer.c | 18 ++++++++++++++----\n> 2 files changed, 17 insertions(+), 4 deletions(-)\n> \n> diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h\n> index dcbb2e1..16887c0 100644\n> --- a/include/kvm/arm_arch_timer.h\n> +++ b/include/kvm/arm_arch_timer.h\n> @@ -47,6 +47,9 @@ struct arch_timer_cpu {\n> \t/* Work queued with the above timer expires */\n> \tstruct work_struct\t\texpired;\n> \n> +\t/* Physical timer emulation */\n> +\tstruct hrtimer\t\t\tphys_timer;\n> +\n> \t/* Background timer active */\n> \tbool\t\t\t\tarmed;\n> \n> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> index c2e8326..7f87099 100644\n> --- a/virt/kvm/arm/arch_timer.c\n> +++ b/virt/kvm/arm/arch_timer.c\n> @@ -178,6 +178,12 @@ static enum hrtimer_restart kvm_bg_timer_expire(struct hrtimer *hrt)\n> \treturn HRTIMER_NORESTART;\n> }\n> \n> +static enum hrtimer_restart kvm_phys_timer_expire(struct hrtimer *hrt)\n> +{\n> +\tWARN(1, \"Timer only used to ensure guest exit - unexpected event.\");\n> +\treturn HRTIMER_NORESTART;\n> +}\n> +\n\nSo what prevents this handler from actually firing? Is it that we cancel\nthe hrtimer while interrupts are still disabled, hence the timer never\nfires? If that's the intention, then this patch is slightly out of\nplace, as we haven't moved the timer sync within the irq_disable() section.\n\nOr am I missing something obvious?\n\n> bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)\n> {\n> \tu64 cval, now;\n> @@ -255,7 +261,7 @@ static void kvm_timer_update_state(struct kvm_vcpu *vcpu)\n> }\n> \n> /* Schedule the background timer for the emulated timer. */\n> -static void kvm_timer_emulate(struct kvm_vcpu *vcpu,\n> +static void phys_timer_emulate(struct kvm_vcpu *vcpu,\n> \t\t\t struct arch_timer_context *timer_ctx)\n> {\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> @@ -267,7 +273,7 @@ static void kvm_timer_emulate(struct kvm_vcpu *vcpu,\n> \t\treturn;\n> \n> \t/* The timer has not yet expired, schedule a background timer */\n> -\tsoft_timer_start(&timer->bg_timer, kvm_timer_compute_delta(timer_ctx));\n> +\tsoft_timer_start(&timer->phys_timer, kvm_timer_compute_delta(timer_ctx));\n> }\n> \n> /*\n> @@ -424,7 +430,7 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)\n> \tkvm_timer_update_state(vcpu);\n> \n> \t/* Set the background timer for the physical timer emulation. */\n> -\tkvm_timer_emulate(vcpu, vcpu_ptimer(vcpu));\n> +\tphys_timer_emulate(vcpu, vcpu_ptimer(vcpu));\n> \n> \tif (unlikely(!irqchip_in_kernel(vcpu->kvm)))\n> \t\tkvm_timer_flush_hwstate_user(vcpu);\n> @@ -447,7 +453,7 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)\n> \t * This is to cancel the background timer for the physical timer\n> \t * emulation if it is set.\n> \t */\n> -\tsoft_timer_cancel(&timer->bg_timer, &timer->expired);\n> +\tsoft_timer_cancel(&timer->phys_timer, NULL);\n\nRight, that now explains the \"work\" test in one of the previous patches.\n\n> \n> \t/*\n> \t * The guest could have modified the timer registers or the timer\n> @@ -507,6 +513,9 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)\n> \thrtimer_init(&timer->bg_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);\n> \ttimer->bg_timer.function = kvm_bg_timer_expire;\n> \n> +\thrtimer_init(&timer->phys_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);\n> +\ttimer->phys_timer.function = kvm_phys_timer_expire;\n> +\n> \tvtimer->irq.irq = default_vtimer_irq.irq;\n> \tptimer->irq.irq = default_ptimer_irq.irq;\n> }\n> @@ -615,6 +624,7 @@ void kvm_timer_vcpu_terminate(struct kvm_vcpu *vcpu)\n> \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> \n> \tsoft_timer_cancel(&timer->bg_timer, &timer->expired);\n> +\tsoft_timer_cancel(&timer->phys_timer, NULL);\n> \tkvm_vgic_unmap_phys_irq(vcpu, vtimer->irq.irq);\n> }\n> \n> \n\nThanks,\n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"aSeOTLlt\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9nD13Zd9z9t6C\n\tfor ;\n\tTue, 10 Oct 2017 04:24:17 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1bmn-0006nM-K9; Mon, 09 Oct 2017 17:24:13 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1bmi-0006kW-VI for linux-arm-kernel@lists.infradead.org;\n\tMon, 09 Oct 2017 17:24:11 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F184B1529;\n\tMon, 9 Oct 2017 10:23:47 -0700 (PDT)","from [10.1.207.62] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tC90D43F483; Mon, 9 Oct 2017 10:23:46 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=VL0NNSkTPFu6gMjxbHahuO6Sld1LeRSr1l3LHJmhWzE=;\n\tb=aSeOTLlts6Mauj\n\tJ7a6/YGijRHRn9ZBvLGbhutYRm1r2Cd2WfohFP5kAbuWq581kbujcjSop+CcHaj3C0k8Em47n2VwJ\n\tMW3vCiAA02CBGb2xoLfQlFrNuHEkRVE5m0cPtehYlWUFI2AKJMBbRkyqBjDaw2oQvOARuVTmfe8x9\n\tyMIGYLvP3TkYbSO9KFH/BrYdFxQjoB5q7I2zVGFHo+dFcyFMjF5nb0z8EbnGr20Xzx5wu+hArrSxJ\n\trE53ctcKZTW9aBeYtPzBgrlKG0iOjoiJrYRz3tB4VzybjMuoK04cuLjalxrU/Z9aEJnar3WjEsGXT\n\thZtwyh7EbjTkNcH/ThYg==;","Subject":"Re: [PATCH v3 09/20] KVM: arm/arm64: Use separate timer for phys\n\ttimer emulation","To":"Christoffer Dall , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-10-cdall@linaro.org>","From":"Marc Zyngier ","Organization":"ARM Ltd","Message-ID":"","Date":"Mon, 9 Oct 2017 18:23:45 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170923004207.22356-10-cdall@linaro.org>","Content-Language":"en-GB","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171009_102409_139191_2F360882 ","X-CRM114-Status":"GOOD ( 21.44 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783072,"web_url":"http://patchwork.ozlabs.org/comment/1783072/","msgid":"<512939dd-1f04-f653-d069-e468936eeabb@arm.com>","list_archive_url":null,"date":"2017-10-09T17:34:45","subject":"Re: [PATCH v3 10/20] KVM: arm/arm64: Move timer/vgic flush/sync\n\tunder disabled irq","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On 23/09/17 01:41, Christoffer Dall wrote:\n> As we are about to play tricks with the timer to be more lazy in saving\n> and restoring state, we need to move the timer sync and flush functions\n> under a disabled irq section and since we have to flush the vgic state\n> after the timer and PMU state, we do the whole flush/sync sequence with\n> disabled irqs.\n> \n> The only downside is a slightly longer delay before being able to\n> process hardware interrupts and run softirqs.\n> \n> Signed-off-by: Christoffer Dall \n> ---\n> virt/kvm/arm/arm.c | 26 +++++++++++++-------------\n> 1 file changed, 13 insertions(+), 13 deletions(-)\n> \n> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c\n> index b9f68e4..27db222 100644\n> --- a/virt/kvm/arm/arm.c\n> +++ b/virt/kvm/arm/arm.c\n> @@ -654,11 +654,11 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)\n> \n> \t\tkvm_pmu_flush_hwstate(vcpu);\n> \n> +\t\tlocal_irq_disable();\n> +\n> \t\tkvm_timer_flush_hwstate(vcpu);\n> \t\tkvm_vgic_flush_hwstate(vcpu);\n> \n> -\t\tlocal_irq_disable();\n> -\n> \t\t/*\n> \t\t * If we have a singal pending, or need to notify a userspace\n> \t\t * irqchip about timer or PMU level changes, then we exit (and\n> @@ -683,10 +683,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)\n> \t\tif (ret <= 0 || need_new_vmid_gen(vcpu->kvm) ||\n> \t\t kvm_request_pending(vcpu)) {\n> \t\t\tvcpu->mode = OUTSIDE_GUEST_MODE;\n> -\t\t\tlocal_irq_enable();\n> \t\t\tkvm_pmu_sync_hwstate(vcpu);\n> \t\t\tkvm_timer_sync_hwstate(vcpu);\n> \t\t\tkvm_vgic_sync_hwstate(vcpu);\n> +\t\t\tlocal_irq_enable();\n> \t\t\tpreempt_enable();\n> \t\t\tcontinue;\n> \t\t}\n> @@ -710,6 +710,16 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)\n> \t\tkvm_arm_clear_debug(vcpu);\n> \n> \t\t/*\n> +\t\t * We must sync the PMU and timer state before the vgic state so\n> +\t\t * that the vgic can properly sample the updated state of the\n> +\t\t * interrupt line.\n> +\t\t */\n> +\t\tkvm_pmu_sync_hwstate(vcpu);\n> +\t\tkvm_timer_sync_hwstate(vcpu);\n> +\n> +\t\tkvm_vgic_sync_hwstate(vcpu);\n> +\n> +\t\t/*\n> \t\t * We may have taken a host interrupt in HYP mode (ie\n> \t\t * while executing the guest). This interrupt is still\n> \t\t * pending, as we haven't serviced it yet!\n> @@ -732,16 +742,6 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)\n> \t\tguest_exit();\n> \t\ttrace_kvm_exit(ret, kvm_vcpu_trap_get_class(vcpu), *vcpu_pc(vcpu));\n> \n> -\t\t/*\n> -\t\t * We must sync the PMU and timer state before the vgic state so\n> -\t\t * that the vgic can properly sample the updated state of the\n> -\t\t * interrupt line.\n> -\t\t */\n> -\t\tkvm_pmu_sync_hwstate(vcpu);\n> -\t\tkvm_timer_sync_hwstate(vcpu);\n> -\n> -\t\tkvm_vgic_sync_hwstate(vcpu);\n> -\n> \t\tpreempt_enable();\n> \n> \t\tret = handle_exit(vcpu, run, ret);\n> \n\nReviewed-by: Marc Zyngier \n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"ueUWkTur\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9nSh6p1vz9sCZ\n\tfor ;\n\tTue, 10 Oct 2017 04:35:16 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1bxQ-0005Kp-R1; Mon, 09 Oct 2017 17:35:12 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1bxM-0004Qa-MF for linux-arm-kernel@lists.infradead.org;\n\tMon, 09 Oct 2017 17:35:10 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 533781529;\n\tMon, 9 Oct 2017 10:34:48 -0700 (PDT)","from [10.1.207.62] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t32FA93F483; Mon, 9 Oct 2017 10:34:47 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=r7HQ9ruDc0m93hOi64V4tkRoNnDVEnXJp5ev63lWjdw=;\n\tb=ueUWkTur5Hi+1v\n\tzCOy8juOkVcTBl9ixpAvYNoz7De3IP3+6EtGCDAX66GRFumKaOEARmUk+g2J3uPjaMRUWD8W8CdIG\n\tUBSTYMdM7PzzZz6yPe3OT6dhGHv2taosNK5i8Hjvt7qpKuoZr+OKdrAXltzCi9kfiHc+RN3a9sHp7\n\tw8Q79ADQjr+9Y2fn/kFhxStB1oYOgoYUeDZLZQtWAo0+b1EgeWyskAV8aKy2sZ3dlD8v2/i/2Jz4x\n\tA661pMPb1W19PcGc69Vg8roXXujXq86D8VELUFbPdDT1xhMBQ49IRqfx9rYxi8z+QeA7HnuH8RbJS\n\tlvJCjoBpVSwpbbNgO+gQ==;","Subject":"Re: [PATCH v3 10/20] KVM: arm/arm64: Move timer/vgic flush/sync\n\tunder disabled irq","To":"Christoffer Dall , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-11-cdall@linaro.org>","From":"Marc Zyngier ","Organization":"ARM Ltd","Message-ID":"<512939dd-1f04-f653-d069-e468936eeabb@arm.com>","Date":"Mon, 9 Oct 2017 18:34:45 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170923004207.22356-11-cdall@linaro.org>","Content-Language":"en-GB","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171009_103508_855770_A87F7A3A ","X-CRM114-Status":"GOOD ( 16.90 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783073,"web_url":"http://patchwork.ozlabs.org/comment/1783073/","msgid":"","list_archive_url":null,"date":"2017-10-09T17:06:06","subject":"Re: [PATCH v3 08/20] KVM: arm/arm64: Rename soft timer to bg_timer","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On 23/09/17 01:41, Christoffer Dall wrote:\n> As we are about to introduce a separate hrtimer for the physical timer,\n> call this timer bg_timer, because we refer to this timer as the\n> background timer in the code and comments elsewhere.\n> \n> Signed-off-by: Christoffer Dall \nAcked-by: Marc Zyngier \n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"hELP5LAO\"; \n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=infradead.org header.i=@infradead.org\n\theader.b=\"Rtew2wKh\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9nT050FVz9sCZ\n\tfor ;\n\tTue, 10 Oct 2017 04:35:32 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1bxg-0005sP-Gr; Mon, 09 Oct 2017 17:35:28 +0000","from merlin.infradead.org ([2001:8b0:10b:1231::1])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1brE-0001R0-1G for linux-arm-kernel@bombadil.infradead.org;\n\tMon, 09 Oct 2017 17:28:48 +0000","from foss.arm.com ([217.140.101.70])\n\tby merlin.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1bVf-0001ho-7A for linux-arm-kernel@lists.infradead.org;\n\tMon, 09 Oct 2017 17:06:32 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E40C51529;\n\tMon, 9 Oct 2017 10:06:09 -0700 (PDT)","from [10.1.207.62] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tD1B293F483; Mon, 9 Oct 2017 10:06:08 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=eDJ83qmEMHsugjflNhL5GIBu0PR51c5h7EA7jI5W0Ms=;\n\tb=hELP5LAOlH/e+t\n\tti/926OGI4sm7TPyZbqzxlM2UJsg6K6p4oJDsXwcr+wYF9OZCpddtJ8VEpBXbAryiBqi4gAULuNVl\n\tl7AgJ3xpBxsfVCuNZ7U8Bsi7iOl9VOgRJZEUD/rAtS0A+p/m++7rDOoIULDRA8Q7J9vSHC5FpB3sE\n\tIP5HP+rwVODGAdQogxhHrSIFK2nCe3mobXDHGC9/GpaYfBjKrmbBd7SwP1NzC7RtfB5LO99j4Qpqj\n\tmMf8lsp87/TShgefPI58WbAJbr6nx306evkSnvzSPB2qRL+lupBXk95v381+c8epmbJJiEYHFaMjR\n\tD3xmXkrKkntFCQEiHU3Q==;","v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=infradead.org; s=merlin.20170209;\n\th=Content-Transfer-Encoding:Content-Type:\n\tIn-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender\n\t:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:\n\tResent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:\n\tList-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;\n\tbh=Rz9mOei6xKVIJPualOhuJz3SU0E2+tCQ0DAMlVZe4bY=;\n\tb=Rtew2wKhgw7JeHd3DWjhHv4gWB\n\tFXBSFQ2ofBexL6vo5KWlkf4k85Ao3GdYcdTOPBQhJ3sI6JHNvFEFZIUaoRApYW74iveBLrQ7bq/e+\n\tg5fI8aQueV1IeZ7LC0yjlrKsl33aME5hB3/WNXuHkhQCpmbJsNpaLlGn5muL3EorpctNkZ0GRrpPH\n\tLCfwCARmZrITVAqUrRfXDcMQ4+xiQ56sZC1n0zmAl8pFGyF7i92QZO5vF7TxAitiyfaws9n9lWBPR\n\tQasR4/x5pnK9ZBqN3Amjb4H0KqwBrQc72IbYF0CVdeMilFNH92gAKGoeg0DuAK7T8hzej4FOUlCvl\n\t4QbdNvBg==;"],"Subject":"Re: [PATCH v3 08/20] KVM: arm/arm64: Rename soft timer to bg_timer","To":"Christoffer Dall , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-9-cdall@linaro.org>","From":"Marc Zyngier ","Organization":"ARM Ltd","Message-ID":"","Date":"Mon, 9 Oct 2017 18:06:06 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170923004207.22356-9-cdall@linaro.org>","Content-Language":"en-GB","X-Spam-Note":"CRM114 invocation failed","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on merlin.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783089,"web_url":"http://patchwork.ozlabs.org/comment/1783089/","msgid":"<3aebbbaa-bad2-32ed-822d-2db23a759f78@arm.com>","list_archive_url":null,"date":"2017-10-09T17:47:42","subject":"Re: [PATCH v3 11/20] KVM: arm/arm64: Move timer save/restore out of\n\tthe hyp code","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On 23/09/17 01:41, Christoffer Dall wrote:\n> As we are about to be lazy with saving and restoring the timer\n> registers, we prepare by moving all possible timer configuration logic\n> out of the hyp code. All virtual timer registers can be programmed from\n> EL1 and since the arch timer is always a level triggered interrupt we\n> can safely do this with interrupts disabled in the host kernel on the\n> way to the guest without taking vtimer interrupts in the host kernel\n> (yet).\n> \n> The downside is that the cntvoff register can only be programmed from\n> hyp mode, so we jump into hyp mode and back to program it. This is also\n> safe, because the host kernel doesn't use the virtual timer in the KVM\n> code. It may add a little performance performance penalty, but only\n> until following commits where we move this operation to vcpu load/put.\n> \n> Signed-off-by: Christoffer Dall \n> ---\n> arch/arm/include/asm/kvm_asm.h | 2 ++\n> arch/arm/include/asm/kvm_hyp.h | 4 +--\n> arch/arm/kvm/hyp/switch.c | 7 ++--\n> arch/arm64/include/asm/kvm_asm.h | 2 ++\n> arch/arm64/include/asm/kvm_hyp.h | 4 +--\n> arch/arm64/kvm/hyp/switch.c | 6 ++--\n> virt/kvm/arm/arch_timer.c | 40 ++++++++++++++++++++++\n> virt/kvm/arm/hyp/timer-sr.c | 74 +++++++++++++++++-----------------------\n> 8 files changed, 87 insertions(+), 52 deletions(-)\n> \n> diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h\n> index 14d68a4..36dd296 100644\n> --- a/arch/arm/include/asm/kvm_asm.h\n> +++ b/arch/arm/include/asm/kvm_asm.h\n> @@ -68,6 +68,8 @@ extern void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);\n> extern void __kvm_tlb_flush_vmid(struct kvm *kvm);\n> extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);\n> \n> +extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);\n> +\n> extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);\n> \n> extern void __init_stage2_translation(void);\n> diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h\n> index 14b5903..ab20ffa 100644\n> --- a/arch/arm/include/asm/kvm_hyp.h\n> +++ b/arch/arm/include/asm/kvm_hyp.h\n> @@ -98,8 +98,8 @@\n> #define cntvoff_el2\t\t\tCNTVOFF\n> #define cnthctl_el2\t\t\tCNTHCTL\n> \n> -void __timer_save_state(struct kvm_vcpu *vcpu);\n> -void __timer_restore_state(struct kvm_vcpu *vcpu);\n> +void __timer_enable_traps(struct kvm_vcpu *vcpu);\n> +void __timer_disable_traps(struct kvm_vcpu *vcpu);\n> \n> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);\n> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);\n> diff --git a/arch/arm/kvm/hyp/switch.c b/arch/arm/kvm/hyp/switch.c\n> index ebd2dd4..330c9ce 100644\n> --- a/arch/arm/kvm/hyp/switch.c\n> +++ b/arch/arm/kvm/hyp/switch.c\n> @@ -174,7 +174,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n> \t__activate_vm(vcpu);\n> \n> \t__vgic_restore_state(vcpu);\n> -\t__timer_restore_state(vcpu);\n> +\t__timer_enable_traps(vcpu);\n> \n> \t__sysreg_restore_state(guest_ctxt);\n> \t__banked_restore_state(guest_ctxt);\n> @@ -191,7 +191,8 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n> \n> \t__banked_save_state(guest_ctxt);\n> \t__sysreg_save_state(guest_ctxt);\n> -\t__timer_save_state(vcpu);\n> +\t__timer_disable_traps(vcpu);\n> +\n> \t__vgic_save_state(vcpu);\n> \n> \t__deactivate_traps(vcpu);\n> @@ -237,7 +238,7 @@ void __hyp_text __noreturn __hyp_panic(int cause)\n> \n> \t\tvcpu = (struct kvm_vcpu *)read_sysreg(HTPIDR);\n> \t\thost_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);\n> -\t\t__timer_save_state(vcpu);\n> +\t\t__timer_disable_traps(vcpu);\n> \t\t__deactivate_traps(vcpu);\n> \t\t__deactivate_vm(vcpu);\n> \t\t__banked_restore_state(host_ctxt);\n> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h\n> index 26a64d0..ab4d0a9 100644\n> --- a/arch/arm64/include/asm/kvm_asm.h\n> +++ b/arch/arm64/include/asm/kvm_asm.h\n> @@ -55,6 +55,8 @@ extern void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);\n> extern void __kvm_tlb_flush_vmid(struct kvm *kvm);\n> extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);\n> \n> +extern void __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high);\n> +\n> extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);\n> \n> extern u64 __vgic_v3_get_ich_vtr_el2(void);\n> diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h\n> index 4572a9b..08d3bb6 100644\n> --- a/arch/arm64/include/asm/kvm_hyp.h\n> +++ b/arch/arm64/include/asm/kvm_hyp.h\n> @@ -129,8 +129,8 @@ void __vgic_v3_save_state(struct kvm_vcpu *vcpu);\n> void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);\n> int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);\n> \n> -void __timer_save_state(struct kvm_vcpu *vcpu);\n> -void __timer_restore_state(struct kvm_vcpu *vcpu);\n> +void __timer_enable_traps(struct kvm_vcpu *vcpu);\n> +void __timer_disable_traps(struct kvm_vcpu *vcpu);\n> \n> void __sysreg_save_host_state(struct kvm_cpu_context *ctxt);\n> void __sysreg_restore_host_state(struct kvm_cpu_context *ctxt);\n> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c\n> index 945e79c..4994f4b 100644\n> --- a/arch/arm64/kvm/hyp/switch.c\n> +++ b/arch/arm64/kvm/hyp/switch.c\n> @@ -298,7 +298,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n> \t__activate_vm(vcpu);\n> \n> \t__vgic_restore_state(vcpu);\n> -\t__timer_restore_state(vcpu);\n> +\t__timer_enable_traps(vcpu);\n> \n> \t/*\n> \t * We must restore the 32-bit state before the sysregs, thanks\n> @@ -368,7 +368,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)\n> \n> \t__sysreg_save_guest_state(guest_ctxt);\n> \t__sysreg32_save_state(vcpu);\n> -\t__timer_save_state(vcpu);\n> +\t__timer_disable_traps(vcpu);\n> \t__vgic_save_state(vcpu);\n> \n> \t__deactivate_traps(vcpu);\n> @@ -436,7 +436,7 @@ void __hyp_text __noreturn __hyp_panic(void)\n> \n> \t\tvcpu = (struct kvm_vcpu *)read_sysreg(tpidr_el2);\n> \t\thost_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);\n> -\t\t__timer_save_state(vcpu);\n> +\t\t__timer_disable_traps(vcpu);\n> \t\t__deactivate_traps(vcpu);\n> \t\t__deactivate_vm(vcpu);\n> \t\t__sysreg_restore_host_state(host_ctxt);\n> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> index 7f87099..4254f88 100644\n> --- a/virt/kvm/arm/arch_timer.c\n> +++ b/virt/kvm/arm/arch_timer.c\n> @@ -276,6 +276,20 @@ static void phys_timer_emulate(struct kvm_vcpu *vcpu,\n> \tsoft_timer_start(&timer->phys_timer, kvm_timer_compute_delta(timer_ctx));\n> }\n> \n> +static void timer_save_state(struct kvm_vcpu *vcpu)\n> +{\n> +\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> +\n> +\tif (timer->enabled) {\n> +\t\tvtimer->cnt_ctl = read_sysreg_el0(cntv_ctl);\n> +\t\tvtimer->cnt_cval = read_sysreg_el0(cntv_cval);\n> +\t}\n> +\n> +\t/* Disable the virtual timer */\n> +\twrite_sysreg_el0(0, cntv_ctl);\n> +}\n> +\n> /*\n> * Schedule the background timer before calling kvm_vcpu_block, so that this\n> * thread is removed from its waitqueue and made runnable when there's a timer\n> @@ -312,6 +326,18 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)\n> \tsoft_timer_start(&timer->bg_timer, kvm_timer_earliest_exp(vcpu));\n> }\n> \n> +static void timer_restore_state(struct kvm_vcpu *vcpu)\n> +{\n> +\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> +\n> +\tif (timer->enabled) {\n> +\t\twrite_sysreg_el0(vtimer->cnt_cval, cntv_cval);\n> +\t\tisb();\n> +\t\twrite_sysreg_el0(vtimer->cnt_ctl, cntv_ctl);\n> +\t}\n> +}\n> +\n> void kvm_timer_unschedule(struct kvm_vcpu *vcpu)\n> {\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> @@ -320,6 +346,13 @@ void kvm_timer_unschedule(struct kvm_vcpu *vcpu)\n> \ttimer->armed = false;\n> }\n> \n> +static void set_cntvoff(u64 cntvoff)\n> +{\n> +\tu32 low = cntvoff & GENMASK(31, 0);\n> +\tu32 high = (cntvoff >> 32) & GENMASK(31, 0);\n\nupper_32_bits/lower_32_bits?\n\n> +\tkvm_call_hyp(__kvm_timer_set_cntvoff, low, high);\n\nMaybe a comment as to why we need to split the 64bit value in two 32bit\nwords (32bit ARM PCS is getting in the way).\n\n> +}\n> +\n> static void kvm_timer_flush_hwstate_vgic(struct kvm_vcpu *vcpu)\n> {\n> \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> @@ -423,6 +456,7 @@ static void kvm_timer_flush_hwstate_user(struct kvm_vcpu *vcpu)\n> void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)\n> {\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> \n> \tif (unlikely(!timer->enabled))\n> \t\treturn;\n> @@ -436,6 +470,9 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)\n> \t\tkvm_timer_flush_hwstate_user(vcpu);\n> \telse\n> \t\tkvm_timer_flush_hwstate_vgic(vcpu);\n> +\n> +\tset_cntvoff(vtimer->cntvoff);\n> +\ttimer_restore_state(vcpu);\n> }\n> \n> /**\n> @@ -455,6 +492,9 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)\n> \t */\n> \tsoft_timer_cancel(&timer->phys_timer, NULL);\n> \n> +\ttimer_save_state(vcpu);\n> +\tset_cntvoff(0);\n> +\n> \t/*\n> \t * The guest could have modified the timer registers or the timer\n> \t * could have expired, update the timer state.\n> diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c\n> index 4734915..a6c3b10 100644\n> --- a/virt/kvm/arm/hyp/timer-sr.c\n> +++ b/virt/kvm/arm/hyp/timer-sr.c\n> @@ -21,58 +21,48 @@\n> \n> #include \n> \n> -/* vcpu is already in the HYP VA space */\n> -void __hyp_text __timer_save_state(struct kvm_vcpu *vcpu)\n> +void __hyp_text __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high)\n> +{\n> +\tu64 cntvoff = (u64)cntvoff_high << 32 | cntvoff_low;\n> +\twrite_sysreg(cntvoff, cntvoff_el2);\n> +}\n> +\n> +void __hyp_text enable_phys_timer(void)\n> {\n> -\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> -\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> \tu64 val;\n> \n> -\tif (timer->enabled) {\n> -\t\tvtimer->cnt_ctl = read_sysreg_el0(cntv_ctl);\n> -\t\tvtimer->cnt_cval = read_sysreg_el0(cntv_cval);\n> -\t}\n> +\t/* Allow physical timer/counter access for the host */\n> +\tval = read_sysreg(cnthctl_el2);\n> +\tval |= CNTHCTL_EL1PCTEN | CNTHCTL_EL1PCEN;\n> +\twrite_sysreg(val, cnthctl_el2);\n> +}\n> \n> -\t/* Disable the virtual timer */\n> -\twrite_sysreg_el0(0, cntv_ctl);\n> +void __hyp_text disable_phys_timer(void)\n> +{\n> +\tu64 val;\n> \n> \t/*\n> +\t * Disallow physical timer access for the guest\n> +\t * Physical counter access is allowed\n> +\t */\n> +\tval = read_sysreg(cnthctl_el2);\n> +\tval &= ~CNTHCTL_EL1PCEN;\n> +\tval |= CNTHCTL_EL1PCTEN;\n> +\twrite_sysreg(val, cnthctl_el2);\n> +}\n> +\n> +void __hyp_text __timer_disable_traps(struct kvm_vcpu *vcpu)\n> +{\n> +\t/*\n> \t * We don't need to do this for VHE since the host kernel runs in EL2\n> \t * with HCR_EL2.TGE ==1, which makes those bits have no impact.\n> \t */\n> -\tif (!has_vhe()) {\n> -\t\t/* Allow physical timer/counter access for the host */\n> -\t\tval = read_sysreg(cnthctl_el2);\n> -\t\tval |= CNTHCTL_EL1PCTEN | CNTHCTL_EL1PCEN;\n> -\t\twrite_sysreg(val, cnthctl_el2);\n> -\t}\n> -\n> -\t/* Clear cntvoff for the host */\n> -\twrite_sysreg(0, cntvoff_el2);\n> +\tif (!has_vhe())\n> +\t\tenable_phys_timer();\n> }\n> \n> -void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)\n> +void __hyp_text __timer_enable_traps(struct kvm_vcpu *vcpu)\n> {\n> -\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> -\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> -\tu64 val;\n> -\n> -\t/* Those bits are already configured at boot on VHE-system */\n> -\tif (!has_vhe()) {\n> -\t\t/*\n> -\t\t * Disallow physical timer access for the guest\n> -\t\t * Physical counter access is allowed\n> -\t\t */\n> -\t\tval = read_sysreg(cnthctl_el2);\n> -\t\tval &= ~CNTHCTL_EL1PCEN;\n> -\t\tval |= CNTHCTL_EL1PCTEN;\n> -\t\twrite_sysreg(val, cnthctl_el2);\n> -\t}\n> -\n> -\tif (timer->enabled) {\n> -\t\twrite_sysreg(vtimer->cntvoff, cntvoff_el2);\n> -\t\twrite_sysreg_el0(vtimer->cnt_cval, cntv_cval);\n> -\t\tisb();\n> -\t\twrite_sysreg_el0(vtimer->cnt_ctl, cntv_ctl);\n> -\t}\n> +\tif (!has_vhe())\n> +\t\tdisable_phys_timer();\n> }\n> \n\nOtherwise:\n\nReviewed-by: Marc Zyngier \n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"Kr8Xsb1N\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9nlk6vdYz9sBd\n\tfor ;\n\tTue, 10 Oct 2017 04:48:15 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1c9y-00025R-Fp; Mon, 09 Oct 2017 17:48:10 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1c9u-000232-UE for linux-arm-kernel@lists.infradead.org;\n\tMon, 09 Oct 2017 17:48:09 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5550C1529;\n\tMon, 9 Oct 2017 10:47:46 -0700 (PDT)","from [10.1.207.62] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t183B23F483; Mon, 9 Oct 2017 10:47:44 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=FyCE4AW/QC9YvG1PWSt0pFgEQ563M5JBEKH/ESI/r4Y=;\n\tb=Kr8Xsb1Nckg3nR\n\t+I/sDV1WnnqUxtyysL+edv/TDqnLFIRIYRecIz56KIt8ZW7PjZogRvKWx58JS3rgxA4evgkPoEB2M\n\t1zaIp+RNN0Lu90ZnHIhc7o4hFBeW+GzVyc7EMYdDXx2vXBb5X/0gvjCN9rOFQFPSFcFLmcjjkH6+V\n\tt6NW8b7P95g7PF4erIsCjigXTIUxgPQu4vafhjvfK1SiKJRqkL8Dv0MmmDo70rErAflv7f1oHmS5X\n\t3l3JOzJy3CYYb2sCGl/o7djsQOkRtFfYU+z+56vFZ3l/Vrkx1mpqM3I0IU1hhj5HWRkATDGzbadcP\n\txD8OqHDAAmMoYn6n8V2A==;","Subject":"Re: [PATCH v3 11/20] KVM: arm/arm64: Move timer save/restore out of\n\tthe hyp code","To":"Christoffer Dall , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-12-cdall@linaro.org>","From":"Marc Zyngier ","Organization":"ARM Ltd","Message-ID":"<3aebbbaa-bad2-32ed-822d-2db23a759f78@arm.com>","Date":"Mon, 9 Oct 2017 18:47:42 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170923004207.22356-12-cdall@linaro.org>","Content-Language":"en-GB","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171009_104806_992714_21ED31D8 ","X-CRM114-Status":"GOOD ( 25.96 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783092,"web_url":"http://patchwork.ozlabs.org/comment/1783092/","msgid":"<1a21ed0e-39eb-a5a3-5cda-3a51d9f9c8da@arm.com>","list_archive_url":null,"date":"2017-10-09T17:48:33","subject":"Re: [PATCH v3 12/20] genirq: Document vcpu_info usage for\n\tpercpu_devid interrupts","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On 23/09/17 01:41, Christoffer Dall wrote:\n> It is currently unclear how to set the VCPU affinity for a percpu_devid\n> interrupt , since the Linux irq_data structure describes the state for\n> multiple interrupts, one for each physical CPU on the system. Since\n> each such interrupt can be associated with different VCPUs or none at\n> all, associating a single VCPU state with such an interrupt does not\n> capture the necessary semantics.\n> \n> The implementers of irq_set_affinity are the Intel and AMD IOMMUs, and\n> the ARM GIC irqchip. The Intel and AMD callers do not appear to use\n> percpu_devid interrupts, and the ARM GIC implementation only checks the\n> pointer against NULL vs. non-NULL.\n> \n> Therefore, simply update the function documentation to explain the\n> expected use in the context of percpu_devid interrupts, allowing future\n> changes or additions to irqchip implementers to do the right thing.\n> \n> This allows us to set the VCPU affinity for the virtual timer interrupt\n> in KVM/ARM, which is a percpu_devid (PPI) interrupt.\n> \n> Cc: Thomas Gleixner \n> Cc: Marc Zyngier \n> Signed-off-by: Christoffer Dall \n> ---\n> kernel/irq/manage.c | 3 ++-\n> 1 file changed, 2 insertions(+), 1 deletion(-)\n> \n> diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c\n> index 573dc52..2b2c94f 100644\n> --- a/kernel/irq/manage.c\n> +++ b/kernel/irq/manage.c\n> @@ -381,7 +381,8 @@ int irq_select_affinity_usr(unsigned int irq)\n> /**\n> *\tirq_set_vcpu_affinity - Set vcpu affinity for the interrupt\n> *\t@irq: interrupt number to set affinity\n> - *\t@vcpu_info: vCPU specific data\n> + *\t@vcpu_info: vCPU specific data or pointer to a percpu array of vCPU\n> + *\t specific data for percpu_devid interrupts\n> *\n> *\tThis function uses the vCPU specific data to set the vCPU\n> *\taffinity for an irq. The vCPU specific data is passed from\n> \n\nAcked-by: Marc Zyngier \n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"dg2bBIZG\"; \n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=infradead.org header.i=@infradead.org\n\theader.b=\"Podw/ZeW\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9nmg0r7vz9t5Q\n\tfor ;\n\tTue, 10 Oct 2017 04:49:07 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1cAp-0002T6-2N; Mon, 09 Oct 2017 17:49:03 +0000","from casper.infradead.org ([2001:8b0:10b:1236::1])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1cAl-0002Sg-VU for linux-arm-kernel@bombadil.infradead.org;\n\tMon, 09 Oct 2017 17:49:00 +0000","from foss.arm.com ([217.140.101.70])\n\tby casper.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1cAi-0007aG-Ih for linux-arm-kernel@lists.infradead.org;\n\tMon, 09 Oct 2017 17:48:58 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B5C081529;\n\tMon, 9 Oct 2017 10:48:35 -0700 (PDT)","from [10.1.207.62] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t804703F483; Mon, 9 Oct 2017 10:48:34 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=FrRnR9UoZKumdWmhOenjHdApXlbBartoCXVCuz82ymo=;\n\tb=dg2bBIZG9zUVy1\n\tgJb3KJIMqJhucPCo8J/5rMp+ZW/jgWbESnAJ2i6Xcyrb8zxuqIo73/IgTeFWsUTybN4506QTFIuVS\n\tw9Gl9e/VEmpbdQzw3K5kAJakxZnkRYqYHQvd+woj9H/aU+I6Csg5uc/aEZo9RzljDjqbty3UfQ/Dg\n\tsE7iDNNDIlUS0R2DudY3LVXS7zXOCUOk/59tKxKoiytBh7TWqPPcNO8IhvyyG46oP4G0PKjT2DBo3\n\tdcz4koLkf6PiaHHb5S+d7rpk3PbuwVqxTCzLL7o2q138AX/Pr6djb+zEdbji09H49/kzRRUaM6PAx\n\tJ9xDY2HWtUBD46DXMTNw==;","v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=infradead.org; s=casper.20170209;\n\th=Content-Transfer-Encoding:Content-Type:\n\tIn-Reply-To:MIME-Version:Date:Message-ID:From:References:Cc:To:Subject:Sender\n\t:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:\n\tResent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:\n\tList-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;\n\tbh=rnKDpjSzHID6jzWRnuMlHO56nc/Ui4oFn8cBytvXQCM=;\n\tb=Podw/ZeWAA1GIGn4ve0qBcq/4s\n\tlC1Brg3gqm7q4cPJaf+WsVuGjwvDeRI0eu56uzkIF9JpMlmC+Y8q2HUPtYk6iKC3J2mJLkAC/15A4\n\t49Xokg+h3ck5aYaVQn+nyxGRv5FnWInbtKYiOrgpEpRsNQi7uFwEkQzJHLxxU6A4yiyOAaGv2pg74\n\t+0QmijwJMpbg3uO3w6TWwxeY0rKOsUXwbdPyl9QAszHyvgGoQHegbOaYDo/K04NpSFk/vpaTN9yUT\n\tulTOdL13mqRjO0oOVVI9Rayop7Yq8pGSPM2HMngPv+EZ/RDnZ22o5TbeR0tTi05s/tEFptD3ttE85\n\tzg1OUAGQ==;"],"Subject":"Re: [PATCH v3 12/20] genirq: Document vcpu_info usage for\n\tpercpu_devid interrupts","To":"Christoffer Dall , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-13-cdall@linaro.org>","From":"Marc Zyngier ","Organization":"ARM Ltd","Message-ID":"<1a21ed0e-39eb-a5a3-5cda-3a51d9f9c8da@arm.com>","Date":"Mon, 9 Oct 2017 18:48:33 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170923004207.22356-13-cdall@linaro.org>","Content-Language":"en-GB","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171009_184856_764351_A8039E51 ","X-CRM114-Status":"GOOD ( 21.62 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on casper.infradead.org summary:\n\tContent analysis details: (-6.9 points, 5.0 required)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tThomas Gleixner , Will Deacon , \n\tkvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783114,"web_url":"http://patchwork.ozlabs.org/comment/1783114/","msgid":"<916c971f-861e-092e-3b3a-325eb558e599@arm.com>","list_archive_url":null,"date":"2017-10-09T17:52:16","subject":"Re: [PATCH v3 13/20] KVM: arm/arm64: Set VCPU affinity for virt\n\ttimer irq","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On 23/09/17 01:42, Christoffer Dall wrote:\n> As we are about to take physical interrupts for the virtual timer on the\n> host but want to leave those active while running the VM (and let the VM\n> deactivate them), we need to set the vtimer PPI affinity accordingly.\n> \n> Signed-off-by: Christoffer Dall \n> ---\n> virt/kvm/arm/arch_timer.c | 9 +++++++++\n> 1 file changed, 9 insertions(+)\n> \n> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> index 4254f88..4275f8f 100644\n> --- a/virt/kvm/arm/arch_timer.c\n> +++ b/virt/kvm/arm/arch_timer.c\n> @@ -650,11 +650,20 @@ int kvm_timer_hyp_init(void)\n> \t\treturn err;\n> \t}\n> \n> +\terr = irq_set_vcpu_affinity(host_vtimer_irq, kvm_get_running_vcpus());\n> +\tif (err) {\n> +\t\tkvm_err(\"kvm_arch_timer: error setting vcpu affinity\\n\");\n> +\t\tgoto out_free_irq;\n> +\t}\n> +\n> \tkvm_info(\"virtual timer IRQ%d\\n\", host_vtimer_irq);\n> \n> \tcpuhp_setup_state(CPUHP_AP_KVM_ARM_TIMER_STARTING,\n> \t\t\t \"kvm/arm/timer:starting\", kvm_timer_starting_cpu,\n> \t\t\t kvm_timer_dying_cpu);\n> +\treturn 0;\n> +out_free_irq:\n> +\tfree_percpu_irq(host_vtimer_irq, kvm_get_running_vcpus());\n> \treturn err;\n> }\n> \n> \n\nReviewed-by: Marc Zyngier \n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"mvTmHOkJ\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9p7V1qtcz9ryQ\n\tfor ;\n\tTue, 10 Oct 2017 05:05:26 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1cQe-0002rj-7D; Mon, 09 Oct 2017 18:05:24 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1cEI-0004M8-VJ for linux-arm-kernel@lists.infradead.org;\n\tMon, 09 Oct 2017 17:52:40 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CC4AB1529;\n\tMon, 9 Oct 2017 10:52:18 -0700 (PDT)","from [10.1.207.62] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tB62903F483; Mon, 9 Oct 2017 10:52:17 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=Je5L/uuQqhIJqFhgS2j2wW/eoCpH/zrHypq+ka3tuFk=;\n\tb=mvTmHOkJaUyV9I\n\tTocsiXZmNryUhZ+r8I6t23ZCXJGuCdx89WQpJBhBeO04UfTQ4qyG/3uZo+DFSCG9ahSxN+a+z/XqS\n\tZ6IkL8QhgFnxUu01WExxZtgf5row6YSkzEwBvjfz8h+m3BlyE54cAST3zDZrIuRbWDWNj8HLaCbYp\n\teLdvH3P+iINJGM6TDmhwah0rHoE69JTk2PQYDuBf4IBg/L/ul+D6ziPzcWCzJ80az+Tf6S3vjUgk3\n\tj6kjFQuxMyMxLQnhK9IxYDBnlyWYdprsPcYEZNtikt/SkdTqssKKI7nJeGkQA4YWbsb8+RScmbVyF\n\txaDG5tWisqmmsQ2/K4tQ==;","Subject":"Re: [PATCH v3 13/20] KVM: arm/arm64: Set VCPU affinity for virt\n\ttimer irq","To":"Christoffer Dall , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-14-cdall@linaro.org>","From":"Marc Zyngier ","Organization":"ARM Ltd","Message-ID":"<916c971f-861e-092e-3b3a-325eb558e599@arm.com>","Date":"Mon, 9 Oct 2017 18:52:16 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170923004207.22356-14-cdall@linaro.org>","Content-Language":"en-GB","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171009_105239_037945_D06E9493 ","X-CRM114-Status":"GOOD ( 12.49 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783517,"web_url":"http://patchwork.ozlabs.org/comment/1783517/","msgid":"<87mv4zwfru.fsf@on-the-bus.cambridge.arm.com>","list_archive_url":null,"date":"2017-10-10T08:47:33","subject":"Re: [PATCH v3 14/20] KVM: arm/arm64: Avoid timer save/restore in\n\tvcpu entry/exit","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On Sat, Sep 23 2017 at 2:42:01 am BST, Christoffer Dall wrote:\n> We don't need to save and restore the hardware timer state and examine\n> if it generates interrupts on on every entry/exit to the guest. The\n> timer hardware is perfectly capable of telling us when it has expired\n> by signaling interrupts.\n>\n> When taking a vtimer interrupt in the host, we don't want to mess with\n> the timer configuration, we just want to forward the physical interrupt\n> to the guest as a virtual interrupt. We can use the split priority drop\n> and deactivate feature of the GIC to do this, which leaves an EOI'ed\n> interrupt active on the physical distributor, making sure we don't keep\n> taking timer interrupts which would prevent the guest from running. We\n> can then forward the physical interrupt to the VM using the HW bit in\n> the LR of the GIC VE, like we do already, which lets the guest directly\n\nVE?\n\n> deactivate both the physical and virtual timer simultaneously, allowing\n> the timer hardware to exit the VM and generate a new physical interrupt\n> when the timer output is again asserted later on.\n>\n> We do need to capture this state when migrating VCPUs between physical\n> CPUs, however, which we use the vcpu put/load functions for, which are\n> called through preempt notifiers whenever the thread is scheduled away\n> from the CPU or called directly if we return from the ioctl to\n> userspace.\n>\n> One caveat is that we cannot restore the timer state during\n> kvm_timer_vcpu_load, because the flow of sleeping a VCPU is:\n>\n> 1. kvm_vcpu_block\n> 2. kvm_timer_schedule\n> 3. schedule\n> 4. kvm_timer_vcpu_put (preempt notifier)\n> 5. schedule (vcpu thread gets scheduled back)\n> 6. kvm_timer_vcpu_load\n> <---- We restore the hardware state here, but the bg_timer\n> \t hrtimer may have scheduled a work function that also\n> \t changes the timer state here.\n> 7. kvm_timer_unschedule\n> <---- We can restore the state here instead\n>\n> So, while we do need to restore the timer state in step (6) in all other\n> cases than when we called kvm_vcpu_block(), we have to defer the restore\n> to step (7) when coming back after kvm_vcpu_block(). Note that we\n> cannot simply call cancel_work_sync() in step (6), because vcpu_load can\n> be called from a preempt notifier.\n>\n> An added benefit beyond not having to read and write the timer sysregs\n> on every entry and exit is that we no longer have to actively write the\n> active state to the physical distributor, because we set the affinity of\n\nI don't understand this thing about the affinity of the timer. It is a\nPPI, so it cannot go anywhere else.\n\n> the vtimer interrupt when loading the timer state, so that the interrupt\n> automatically stays active after firing.\n>\n> Signed-off-by: Christoffer Dall \n> ---\n> include/kvm/arm_arch_timer.h | 9 +-\n> virt/kvm/arm/arch_timer.c | 238 +++++++++++++++++++++++++++----------------\n> virt/kvm/arm/arm.c | 19 +++-\n> virt/kvm/arm/hyp/timer-sr.c | 8 +-\n> 4 files changed, 174 insertions(+), 100 deletions(-)\n>\n> diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h\n> index 16887c0..8e5ed54 100644\n> --- a/include/kvm/arm_arch_timer.h\n> +++ b/include/kvm/arm_arch_timer.h\n> @@ -31,8 +31,8 @@ struct arch_timer_context {\n> \t/* Timer IRQ */\n> \tstruct kvm_irq_level\t\tirq;\n> \n> -\t/* Active IRQ state caching */\n> -\tbool\t\t\t\tactive_cleared_last;\n> +\t/* Is the timer state loaded on the hardware timer */\n> +\tbool\t\t\tloaded;\n\nI think this little guy is pretty crucial to understand the flow, as\nthere is now two points where we save/restore the timer:\nvcpu_load/vcpu_put and timer_schedule/timer_unschedule. Both can be\nexecuted on the blocking path, and this is the predicate to find out if\nthere is actually something to do.\n\nWould you mind adding a small comment to that effect?\n\n> \n> \t/* Virtual offset */\n> \tu64\t\t\tcntvoff;\n> @@ -80,10 +80,15 @@ void kvm_timer_unschedule(struct kvm_vcpu *vcpu);\n> \n> u64 kvm_phys_timer_read(void);\n> \n> +void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu);\n> void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu);\n> \n> void kvm_timer_init_vhe(void);\n> \n> #define vcpu_vtimer(v)\t(&(v)->arch.timer_cpu.vtimer)\n> #define vcpu_ptimer(v)\t(&(v)->arch.timer_cpu.ptimer)\n> +\n> +void enable_el1_phys_timer_access(void);\n> +void disable_el1_phys_timer_access(void);\n> +\n> #endif\n> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> index 4275f8f..70110ea 100644\n> --- a/virt/kvm/arm/arch_timer.c\n> +++ b/virt/kvm/arm/arch_timer.c\n> @@ -46,10 +46,9 @@ static const struct kvm_irq_level default_vtimer_irq = {\n> \t.level\t= 1,\n> };\n> \n> -void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)\n> -{\n> -\tvcpu_vtimer(vcpu)->active_cleared_last = false;\n> -}\n> +static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx);\n> +static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,\n> +\t\t\t\t struct arch_timer_context *timer_ctx);\n> \n> u64 kvm_phys_timer_read(void)\n> {\n> @@ -74,17 +73,37 @@ static void soft_timer_cancel(struct hrtimer *hrt, struct work_struct *work)\n> \t\tcancel_work_sync(work);\n> }\n> \n> -static irqreturn_t kvm_arch_timer_handler(int irq, void *dev_id)\n> +static void kvm_vtimer_update_mask_user(struct kvm_vcpu *vcpu)\n> {\n> -\tstruct kvm_vcpu *vcpu = *(struct kvm_vcpu **)dev_id;\n> +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> \n> \t/*\n> -\t * We disable the timer in the world switch and let it be\n> -\t * handled by kvm_timer_sync_hwstate(). Getting a timer\n> -\t * interrupt at this point is a sure sign of some major\n> -\t * breakage.\n> +\t * To prevent continuously exiting from the guest, we mask the\n> +\t * physical interrupt when the virtual level is high, such that the\n> +\t * guest can make forward progress. Once we detect the output level\n> +\t * being deasserted, we unmask the interrupt again so that we exit\n> +\t * from the guest when the timer fires.\n\nMaybe an additional comment indicating that this only makes sense when\nwe don't have an in-kernel GIC? I know this wasn't in the original code,\nbut I started asking myself all kind of questions until I realised what\nthis was for...\n\n> \t */\n> -\tpr_warn(\"Unexpected interrupt %d on vcpu %p\\n\", irq, vcpu);\n> +\tif (vtimer->irq.level)\n> +\t\tdisable_percpu_irq(host_vtimer_irq);\n> +\telse\n> +\t\tenable_percpu_irq(host_vtimer_irq, 0);\n> +}\n> +\n> +static irqreturn_t kvm_arch_timer_handler(int irq, void *dev_id)\n> +{\n> +\tstruct kvm_vcpu *vcpu = *(struct kvm_vcpu **)dev_id;\n> +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> +\n> +\tif (!vtimer->irq.level) {\n> +\t\tvtimer->cnt_ctl = read_sysreg_el0(cntv_ctl);\n> +\t\tif (kvm_timer_irq_can_fire(vtimer))\n> +\t\t\tkvm_timer_update_irq(vcpu, true, vtimer);\n> +\t}\n> +\n> +\tif (unlikely(!irqchip_in_kernel(vcpu->kvm)))\n> +\t\tkvm_vtimer_update_mask_user(vcpu);\n> +\n> \treturn IRQ_HANDLED;\n> }\n> \n> @@ -220,7 +239,6 @@ static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,\n> {\n> \tint ret;\n> \n> -\ttimer_ctx->active_cleared_last = false;\n> \ttimer_ctx->irq.level = new_level;\n> \ttrace_kvm_timer_update_irq(vcpu->vcpu_id, timer_ctx->irq.irq,\n> \t\t\t\t timer_ctx->irq.level);\n> @@ -276,10 +294,16 @@ static void phys_timer_emulate(struct kvm_vcpu *vcpu,\n> \tsoft_timer_start(&timer->phys_timer, kvm_timer_compute_delta(timer_ctx));\n> }\n> \n> -static void timer_save_state(struct kvm_vcpu *vcpu)\n> +static void vtimer_save_state(struct kvm_vcpu *vcpu)\n> {\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> +\tunsigned long flags;\n> +\n> +\tlocal_irq_save(flags);\n\nIs that to avoid racing against the timer when doing a\nvcpu_put/timer/schedule?\n\n> +\n> +\tif (!vtimer->loaded)\n> +\t\tgoto out;\n> \n> \tif (timer->enabled) {\n> \t\tvtimer->cnt_ctl = read_sysreg_el0(cntv_ctl);\n> @@ -288,6 +312,10 @@ static void timer_save_state(struct kvm_vcpu *vcpu)\n> \n> \t/* Disable the virtual timer */\n> \twrite_sysreg_el0(0, cntv_ctl);\n> +\n> +\tvtimer->loaded = false;\n> +out:\n> +\tlocal_irq_restore(flags);\n> }\n> \n> /*\n> @@ -303,6 +331,8 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)\n> \n> \tBUG_ON(bg_timer_is_armed(timer));\n> \n> +\tvtimer_save_state(vcpu);\n> +\n> \t/*\n> \t * No need to schedule a background timer if any guest timer has\n> \t * already expired, because kvm_vcpu_block will return before putting\n> @@ -326,16 +356,26 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)\n> \tsoft_timer_start(&timer->bg_timer, kvm_timer_earliest_exp(vcpu));\n> }\n> \n> -static void timer_restore_state(struct kvm_vcpu *vcpu)\n> +static void vtimer_restore_state(struct kvm_vcpu *vcpu)\n> {\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> +\tunsigned long flags;\n> +\n> +\tlocal_irq_save(flags);\n> +\n> +\tif (vtimer->loaded)\n> +\t\tgoto out;\n> \n> \tif (timer->enabled) {\n> \t\twrite_sysreg_el0(vtimer->cnt_cval, cntv_cval);\n> \t\tisb();\n> \t\twrite_sysreg_el0(vtimer->cnt_ctl, cntv_ctl);\n> \t}\n> +\n> +\tvtimer->loaded = true;\n> +out:\n> +\tlocal_irq_restore(flags);\n> }\n> \n> void kvm_timer_unschedule(struct kvm_vcpu *vcpu)\n> @@ -344,6 +384,8 @@ void kvm_timer_unschedule(struct kvm_vcpu *vcpu)\n> \n> \tsoft_timer_cancel(&timer->bg_timer, &timer->expired);\n> \ttimer->armed = false;\n> +\n> +\tvtimer_restore_state(vcpu);\n> }\n> \n> static void set_cntvoff(u64 cntvoff)\n> @@ -353,61 +395,56 @@ static void set_cntvoff(u64 cntvoff)\n> \tkvm_call_hyp(__kvm_timer_set_cntvoff, low, high);\n> }\n> \n> -static void kvm_timer_flush_hwstate_vgic(struct kvm_vcpu *vcpu)\n> +static void kvm_timer_vcpu_load_vgic(struct kvm_vcpu *vcpu)\n> {\n> \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> \tbool phys_active;\n> \tint ret;\n> \n> -\t/*\n> -\t* If we enter the guest with the virtual input level to the VGIC\n> -\t* asserted, then we have already told the VGIC what we need to, and\n> -\t* we don't need to exit from the guest until the guest deactivates\n> -\t* the already injected interrupt, so therefore we should set the\n> -\t* hardware active state to prevent unnecessary exits from the guest.\n> -\t*\n> -\t* Also, if we enter the guest with the virtual timer interrupt active,\n> -\t* then it must be active on the physical distributor, because we set\n> -\t* the HW bit and the guest must be able to deactivate the virtual and\n> -\t* physical interrupt at the same time.\n> -\t*\n> -\t* Conversely, if the virtual input level is deasserted and the virtual\n> -\t* interrupt is not active, then always clear the hardware active state\n> -\t* to ensure that hardware interrupts from the timer triggers a guest\n> -\t* exit.\n> -\t*/\n> -\tphys_active = vtimer->irq.level ||\n> -\t\t\tkvm_vgic_map_is_active(vcpu, vtimer->irq.irq);\n> -\n> -\t/*\n> -\t * We want to avoid hitting the (re)distributor as much as\n> -\t * possible, as this is a potentially expensive MMIO access\n> -\t * (not to mention locks in the irq layer), and a solution for\n> -\t * this is to cache the \"active\" state in memory.\n> -\t *\n> -\t * Things to consider: we cannot cache an \"active set\" state,\n> -\t * because the HW can change this behind our back (it becomes\n> -\t * \"clear\" in the HW). We must then restrict the caching to\n> -\t * the \"clear\" state.\n> -\t *\n> -\t * The cache is invalidated on:\n> -\t * - vcpu put, indicating that the HW cannot be trusted to be\n> -\t * in a sane state on the next vcpu load,\n> -\t * - any change in the interrupt state\n> -\t *\n> -\t * Usage conditions:\n> -\t * - cached value is \"active clear\"\n> -\t * - value to be programmed is \"active clear\"\n> -\t */\n> -\tif (vtimer->active_cleared_last && !phys_active)\n> -\t\treturn;\n> -\n> +\tif (vtimer->irq.level || kvm_vgic_map_is_active(vcpu, vtimer->irq.irq))\n> +\t\tphys_active = true;\n> +\telse\n> +\t\tphys_active = false;\n\nnit: this can be written as:\n\n phys_active = (vtimer->irq.level ||\n \t\t kvm_vgic_map_is_active(vcpu, vtimer->irq.irq));\n\nNot that it matters in the slightest...\n\n> \tret = irq_set_irqchip_state(host_vtimer_irq,\n> \t\t\t\t IRQCHIP_STATE_ACTIVE,\n> \t\t\t\t phys_active);\n> \tWARN_ON(ret);\n> +}\n> +\n> +static void kvm_timer_vcpu_load_user(struct kvm_vcpu *vcpu)\n> +{\n> +\tkvm_vtimer_update_mask_user(vcpu);\n> +}\n> +\n> +void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)\n> +{\n> +\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> +\n> +\tif (unlikely(!timer->enabled))\n> +\t\treturn;\n> +\n> +\tif (unlikely(!irqchip_in_kernel(vcpu->kvm)))\n> +\t\tkvm_timer_vcpu_load_user(vcpu);\n> +\telse\n> +\t\tkvm_timer_vcpu_load_vgic(vcpu);\n> \n> -\tvtimer->active_cleared_last = !phys_active;\n> +\tset_cntvoff(vtimer->cntvoff);\n> +\n> +\t/*\n> +\t * If we armed a soft timer and potentially queued work, we have to\n> +\t * cancel this, but cannot do it here, because canceling work can\n> +\t * sleep and we can be in the middle of a preempt notifier call.\n> +\t * Instead, when the timer has been armed, we know the return path\n> +\t * from kvm_vcpu_block will call kvm_timer_unschedule, so we can defer\n> +\t * restoring the state and canceling any soft timers and work items\n> +\t * until then.\n> +\t */\n> +\tif (!bg_timer_is_armed(timer))\n> +\t\tvtimer_restore_state(vcpu);\n> +\n> +\tif (has_vhe())\n> +\t\tdisable_el1_phys_timer_access();\n> }\n> \n> bool kvm_timer_should_notify_user(struct kvm_vcpu *vcpu)\n> @@ -427,23 +464,6 @@ bool kvm_timer_should_notify_user(struct kvm_vcpu *vcpu)\n> \t ptimer->irq.level != plevel;\n> }\n> \n> -static void kvm_timer_flush_hwstate_user(struct kvm_vcpu *vcpu)\n> -{\n> -\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> -\n> -\t/*\n> -\t * To prevent continuously exiting from the guest, we mask the\n> -\t * physical interrupt such that the guest can make forward progress.\n> -\t * Once we detect the output level being deasserted, we unmask the\n> -\t * interrupt again so that we exit from the guest when the timer\n> -\t * fires.\n> -\t*/\n> -\tif (vtimer->irq.level)\n> -\t\tdisable_percpu_irq(host_vtimer_irq);\n> -\telse\n> -\t\tenable_percpu_irq(host_vtimer_irq, 0);\n> -}\n> -\n> /**\n> * kvm_timer_flush_hwstate - prepare timers before running the vcpu\n> * @vcpu: The vcpu pointer\n> @@ -456,23 +476,55 @@ static void kvm_timer_flush_hwstate_user(struct kvm_vcpu *vcpu)\n> void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)\n> {\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> -\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> +\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> \n> \tif (unlikely(!timer->enabled))\n> \t\treturn;\n> \n> -\tkvm_timer_update_state(vcpu);\n> +\tif (kvm_timer_should_fire(ptimer) != ptimer->irq.level)\n> +\t\tkvm_timer_update_irq(vcpu, !ptimer->irq.level, ptimer);\n> \n> \t/* Set the background timer for the physical timer emulation. */\n> \tphys_timer_emulate(vcpu, vcpu_ptimer(vcpu));\n> +}\n> \n> -\tif (unlikely(!irqchip_in_kernel(vcpu->kvm)))\n> -\t\tkvm_timer_flush_hwstate_user(vcpu);\n> -\telse\n> -\t\tkvm_timer_flush_hwstate_vgic(vcpu);\n> +void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)\n> +{\n> +\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> \n> -\tset_cntvoff(vtimer->cntvoff);\n> -\ttimer_restore_state(vcpu);\n> +\tif (unlikely(!timer->enabled))\n> +\t\treturn;\n> +\n> +\tif (has_vhe())\n> +\t\tenable_el1_phys_timer_access();\n> +\n> +\tvtimer_save_state(vcpu);\n> +\n> +\tset_cntvoff(0);\n\nCan this be moved into vtimer_save_state()? And thinking of it, why\ndon't we reset cntvoff in kvm_timer_schedule() as well? \n\n> +}\n> +\n> +static void unmask_vtimer_irq(struct kvm_vcpu *vcpu)\n> +{\n> +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> +\n> +\tif (unlikely(!irqchip_in_kernel(vcpu->kvm))) {\n> +\t\tkvm_vtimer_update_mask_user(vcpu);\n> +\t\treturn;\n> +\t}\n> +\n> +\t/*\n> +\t * If the guest disabled the timer without acking the interrupt, then\n> +\t * we must make sure the physical and virtual active states are in\n> +\t * sync by deactivating the physical interrupt, because otherwise we\n> +\t * wouldn't see the next timer interrupt in the host.\n> +\t */\n> +\tif (!kvm_vgic_map_is_active(vcpu, vtimer->irq.irq)) {\n> +\t\tint ret;\n> +\t\tret = irq_set_irqchip_state(host_vtimer_irq,\n> +\t\t\t\t\t IRQCHIP_STATE_ACTIVE,\n> +\t\t\t\t\t false);\n> +\t\tWARN_ON(ret);\n> +\t}\n> }\n> \n> /**\n> @@ -485,6 +537,7 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)\n> void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)\n> {\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> \n> \t/*\n> \t * This is to cancel the background timer for the physical timer\n> @@ -492,14 +545,19 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)\n> \t */\n> \tsoft_timer_cancel(&timer->phys_timer, NULL);\n> \n> -\ttimer_save_state(vcpu);\n> -\tset_cntvoff(0);\n> -\n> \t/*\n> -\t * The guest could have modified the timer registers or the timer\n> -\t * could have expired, update the timer state.\n> +\t * If we entered the guest with the vtimer output asserted we have to\n> +\t * check if the guest has modified the timer so that we should lower\n> +\t * the line at this point.\n> \t */\n> -\tkvm_timer_update_state(vcpu);\n> +\tif (vtimer->irq.level) {\n> +\t\tvtimer->cnt_ctl = read_sysreg_el0(cntv_ctl);\n> +\t\tvtimer->cnt_cval = read_sysreg_el0(cntv_cval);\n> +\t\tif (!kvm_timer_should_fire(vtimer)) {\n> +\t\t\tkvm_timer_update_irq(vcpu, false, vtimer);\n> +\t\t\tunmask_vtimer_irq(vcpu);\n> +\t\t}\n> +\t}\n> }\n> \n> int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)\n> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c\n> index 27db222..132d39a 100644\n> --- a/virt/kvm/arm/arm.c\n> +++ b/virt/kvm/arm/arm.c\n> @@ -354,18 +354,18 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)\n> \tvcpu->arch.host_cpu_context = this_cpu_ptr(kvm_host_cpu_state);\n> \n> \tkvm_arm_set_running_vcpu(vcpu);\n> -\n> \tkvm_vgic_load(vcpu);\n> +\tkvm_timer_vcpu_load(vcpu);\n> }\n> \n> void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)\n> {\n> +\tkvm_timer_vcpu_put(vcpu);\n> \tkvm_vgic_put(vcpu);\n> \n> \tvcpu->cpu = -1;\n> \n> \tkvm_arm_set_running_vcpu(NULL);\n> -\tkvm_timer_vcpu_put(vcpu);\n> }\n> \n> static void vcpu_power_off(struct kvm_vcpu *vcpu)\n> @@ -710,16 +710,27 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)\n> \t\tkvm_arm_clear_debug(vcpu);\n> \n> \t\t/*\n> -\t\t * We must sync the PMU and timer state before the vgic state so\n> +\t\t * We must sync the PMU state before the vgic state so\n> \t\t * that the vgic can properly sample the updated state of the\n> \t\t * interrupt line.\n> \t\t */\n> \t\tkvm_pmu_sync_hwstate(vcpu);\n> -\t\tkvm_timer_sync_hwstate(vcpu);\n> \n> +\t\t/*\n> +\t\t * Sync the vgic state before syncing the timer state because\n> +\t\t * the timer code needs to know if the virtual timer\n> +\t\t * interrupts are active.\n> +\t\t */\n> \t\tkvm_vgic_sync_hwstate(vcpu);\n> \n> \t\t/*\n> +\t\t * Sync the timer hardware state before enabling interrupts as\n> +\t\t * we don't want vtimer interrupts to race with syncing the\n> +\t\t * timer virtual interrupt state.\n> +\t\t */\n> +\t\tkvm_timer_sync_hwstate(vcpu);\n> +\n> +\t\t/*\n> \t\t * We may have taken a host interrupt in HYP mode (ie\n> \t\t * while executing the guest). This interrupt is still\n> \t\t * pending, as we haven't serviced it yet!\n> diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c\n> index a6c3b10..f398616 100644\n> --- a/virt/kvm/arm/hyp/timer-sr.c\n> +++ b/virt/kvm/arm/hyp/timer-sr.c\n> @@ -27,7 +27,7 @@ void __hyp_text __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high)\n> \twrite_sysreg(cntvoff, cntvoff_el2);\n> }\n> \n> -void __hyp_text enable_phys_timer(void)\n> +void __hyp_text enable_el1_phys_timer_access(void)\n> {\n> \tu64 val;\n> \n> @@ -37,7 +37,7 @@ void __hyp_text enable_phys_timer(void)\n> \twrite_sysreg(val, cnthctl_el2);\n> }\n> \n> -void __hyp_text disable_phys_timer(void)\n> +void __hyp_text disable_el1_phys_timer_access(void)\n> {\n> \tu64 val;\n> \n> @@ -58,11 +58,11 @@ void __hyp_text __timer_disable_traps(struct kvm_vcpu *vcpu)\n> \t * with HCR_EL2.TGE ==1, which makes those bits have no impact.\n> \t */\n> \tif (!has_vhe())\n> -\t\tenable_phys_timer();\n> +\t\tenable_el1_phys_timer_access();\n> }\n> \n> void __hyp_text __timer_enable_traps(struct kvm_vcpu *vcpu)\n> {\n> \tif (!has_vhe())\n> -\t\tdisable_phys_timer();\n> +\t\tdisable_el1_phys_timer_access();\n> }\n\nIt'd be nice to move this renaming to the patch that introduce these two\nfunctions.\n\nThanks,\n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"MdJUZbE9\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yB9k530b6z9tY0\n\tfor ;\n\tTue, 10 Oct 2017 19:48:13 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1qCx-0001Kn-If; Tue, 10 Oct 2017 08:48:11 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1qCo-0001Dm-FZ for linux-arm-kernel@lists.infradead.org;\n\tTue, 10 Oct 2017 08:48:09 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9D87780D;\n\tTue, 10 Oct 2017 01:47:41 -0700 (PDT)","from on-the-bus (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t6EC603F483; Tue, 10 Oct 2017 01:47:37 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References\n\t:In-Reply-To:Subject:To:From:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=qJ8J5dXQedu/u+JZz4loCauqxdtJW0WhLSR/+0IEw/I=;\n\tb=MdJUZbE97zeflx\n\tjT8YGa6NENY1FGHi/6xl3cflWQ40/ndE4UzEdSD59+FXYXPDIGZzKICAUrfwZp0Bw7NWMsY1aBuAg\n\tjGIToLgXPqe6q0plVvtR0IRxlV67pju9M+TPBKKEZQ699DqCLuxT3DKy5/sAqUWXtsokPs0zhq9Im\n\tEcnSdObVgG4gkPVMJ549pTSKeaVgVhmH2sMs8XZSkWQLZ/qwAVsT5/3yR6UGbS9dYPlD/dFZ8bhGA\n\tPrpyV6iR3wwzYI02vsMW/iQa1cngD15WMC/trKkCgtdhhDC0KqpQTOhP/nlItaPrs12TUKcULDCFd\n\tnnSreJSI+aJ3mUKgmYhQ==;","From":"Marc Zyngier ","To":"Christoffer Dall ","Subject":"Re: [PATCH v3 14/20] KVM: arm/arm64: Avoid timer save/restore in\n\tvcpu entry/exit","In-Reply-To":"<20170923004207.22356-15-cdall@linaro.org> (Christoffer Dall's\n\tmessage of \"Sat, 23 Sep 2017 02:42:01 +0200\")","Organization":"ARM Ltd","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-15-cdall@linaro.org>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)","Date":"Tue, 10 Oct 2017 09:47:33 +0100","Message-ID":"<87mv4zwfru.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171010_014802_546640_91F1B580 ","X-CRM114-Status":"GOOD ( 39.02 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783534,"web_url":"http://patchwork.ozlabs.org/comment/1783534/","msgid":"<87infnwepo.fsf@on-the-bus.cambridge.arm.com>","list_archive_url":null,"date":"2017-10-10T09:10:27","subject":"Re: [PATCH v3 15/20] KVM: arm/arm64: Support EL1 phys timer register\n\taccess in set/get reg","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On Sat, Sep 23 2017 at 2:42:02 am BST, Christoffer Dall wrote:\n> Add suport for the physical timer registers in kvm_arm_timer_set_reg and\n> kvm_arm_timer_get_reg so that these functions can be reused to interact\n> with the rest of the system.\n>\n> Note that this paves part of the way for the physical timer state\n> save/restore, but we still need to add those registers to\n> KVM_GET_REG_LIST before we support migrating the physical timer state.\n>\n> Signed-off-by: Christoffer Dall \n> ---\n> arch/arm/include/uapi/asm/kvm.h | 6 ++++++\n> arch/arm64/include/uapi/asm/kvm.h | 6 ++++++\n> virt/kvm/arm/arch_timer.c | 33 +++++++++++++++++++++++++++++++--\n> 3 files changed, 43 insertions(+), 2 deletions(-)\n>\n> diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h\n> index 5db2d4c..665c454 100644\n> --- a/arch/arm/include/uapi/asm/kvm.h\n> +++ b/arch/arm/include/uapi/asm/kvm.h\n> @@ -151,6 +151,12 @@ struct kvm_arch_memory_slot {\n> \t(__ARM_CP15_REG(op1, 0, crm, 0) | KVM_REG_SIZE_U64)\n> #define ARM_CP15_REG64(...) __ARM_CP15_REG64(__VA_ARGS__)\n> \n> +/* PL1 Physical Timer Registers */\n> +#define KVM_REG_ARM_PTIMER_CTL\t\tARM_CP15_REG32(0, 14, 2, 1)\n> +#define KVM_REG_ARM_PTIMER_CNT\t\tARM_CP15_REG64(0, 14)\n> +#define KVM_REG_ARM_PTIMER_CVAL\t\tARM_CP15_REG64(2, 14)\n> +\n> +/* Virtual Timer Registers */\n> #define KVM_REG_ARM_TIMER_CTL\t\tARM_CP15_REG32(0, 14, 3, 1)\n> #define KVM_REG_ARM_TIMER_CNT\t\tARM_CP15_REG64(1, 14)\n> #define KVM_REG_ARM_TIMER_CVAL\t\tARM_CP15_REG64(3, 14)\n> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h\n> index 9f3ca24..07be6e2 100644\n> --- a/arch/arm64/include/uapi/asm/kvm.h\n> +++ b/arch/arm64/include/uapi/asm/kvm.h\n> @@ -195,6 +195,12 @@ struct kvm_arch_memory_slot {\n> \n> #define ARM64_SYS_REG(...) (__ARM64_SYS_REG(__VA_ARGS__) | KVM_REG_SIZE_U64)\n> \n> +/* EL1 Physical Timer Registers */\n\nThese are EL0 registers, even if we tend to restrict them to EL1. Even\nthe 32bit version is not strictly a PL1 register, since PL1 can delegate\nit to userspace (but the ARMv7 ARM still carries this PL1 thing...).\n\n> +#define KVM_REG_ARM_PTIMER_CTL\t\tARM64_SYS_REG(3, 3, 14, 2, 1)\n> +#define KVM_REG_ARM_PTIMER_CVAL\t\tARM64_SYS_REG(3, 3, 14, 2, 2)\n> +#define KVM_REG_ARM_PTIMER_CNT\t\tARM64_SYS_REG(3, 3, 14, 0, 1)\n> +\n> +/* EL0 Virtual Timer Registers */\n> #define KVM_REG_ARM_TIMER_CTL\t\tARM64_SYS_REG(3, 3, 14, 3, 1)\n> #define KVM_REG_ARM_TIMER_CNT\t\tARM64_SYS_REG(3, 3, 14, 3, 2)\n> #define KVM_REG_ARM_TIMER_CVAL\t\tARM64_SYS_REG(3, 3, 14, 0, 2)\n> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> index 70110ea..d5b632d 100644\n> --- a/virt/kvm/arm/arch_timer.c\n> +++ b/virt/kvm/arm/arch_timer.c\n> @@ -626,10 +626,11 @@ static void kvm_timer_init_interrupt(void *info)\n> int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)\n> {\n> \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> +\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> \n> \tswitch (regid) {\n> \tcase KVM_REG_ARM_TIMER_CTL:\n> -\t\tvtimer->cnt_ctl = value;\n> +\t\tvtimer->cnt_ctl = value & ~ARCH_TIMER_CTRL_IT_STAT;\n\nAh, interesting. Does this change anything to userspace behaviour?\n\n> \t\tbreak;\n> \tcase KVM_REG_ARM_TIMER_CNT:\n> \t\tupdate_vtimer_cntvoff(vcpu, kvm_phys_timer_read() - value);\n> @@ -637,6 +638,13 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)\n> \tcase KVM_REG_ARM_TIMER_CVAL:\n> \t\tvtimer->cnt_cval = value;\n> \t\tbreak;\n> +\tcase KVM_REG_ARM_PTIMER_CTL:\n> +\t\tptimer->cnt_ctl = value & ~ARCH_TIMER_CTRL_IT_STAT;\n> +\t\tbreak;\n> +\tcase KVM_REG_ARM_PTIMER_CVAL:\n> +\t\tptimer->cnt_cval = value;\n> +\t\tbreak;\n> +\n> \tdefault:\n> \t\treturn -1;\n> \t}\n> @@ -645,17 +653,38 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)\n> \treturn 0;\n> }\n> \n> +static u64 read_timer_ctl(struct arch_timer_context *timer)\n> +{\n> +\t/*\n> +\t * Set ISTATUS bit if it's expired.\n> +\t * Note that according to ARMv8 ARM Issue A.k, ISTATUS bit is\n> +\t * UNKNOWN when ENABLE bit is 0, so we chose to set ISTATUS bit\n> +\t * regardless of ENABLE bit for our implementation convenience.\n> +\t */\n> +\tif (!kvm_timer_compute_delta(timer))\n> +\t\treturn timer->cnt_ctl | ARCH_TIMER_CTRL_IT_STAT;\n> +\telse\n> +\t\treturn timer->cnt_ctl;\n\nCan't we end-up with a stale IT_STAT bit here if the timer has been\nsnapshoted with an interrupt pending, and then CVAL updated to expire\nlater?\n\n> +}\n> +\n> u64 kvm_arm_timer_get_reg(struct kvm_vcpu *vcpu, u64 regid)\n> {\n> +\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> \n> \tswitch (regid) {\n> \tcase KVM_REG_ARM_TIMER_CTL:\n> -\t\treturn vtimer->cnt_ctl;\n> +\t\treturn read_timer_ctl(vtimer);\n> \tcase KVM_REG_ARM_TIMER_CNT:\n> \t\treturn kvm_phys_timer_read() - vtimer->cntvoff;\n> \tcase KVM_REG_ARM_TIMER_CVAL:\n> \t\treturn vtimer->cnt_cval;\n> +\tcase KVM_REG_ARM_PTIMER_CTL:\n> +\t\treturn read_timer_ctl(ptimer);\n> +\tcase KVM_REG_ARM_PTIMER_CVAL:\n> +\t\treturn ptimer->cnt_cval;\n> +\tcase KVM_REG_ARM_PTIMER_CNT:\n> +\t\treturn kvm_phys_timer_read();\n> \t}\n> \treturn (u64)-1;\n> }\n\nThanks,\n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"R0jTiVNe\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yBBDX0q9Wz9sPm\n\tfor ;\n\tTue, 10 Oct 2017 20:11:08 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1qZ5-0004yc-Qb; Tue, 10 Oct 2017 09:11:03 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1qYy-0004rZ-CB for linux-arm-kernel@lists.infradead.org;\n\tTue, 10 Oct 2017 09:11:02 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7B35780D;\n\tTue, 10 Oct 2017 02:10:34 -0700 (PDT)","from on-the-bus (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t02C893F483; Tue, 10 Oct 2017 02:10:30 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References\n\t:In-Reply-To:Subject:To:From:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=husvCWkEk6s5IL3f9IH7PKQZNlLhfHE690rmud8RGlA=;\n\tb=R0jTiVNeK9e5QD\n\tVr0en4nmG3CqdzFHn7yBmUDQO7hy7GPSH39FU+7gjjbnnhtq8YRavsBeJgyms9fyMoQlKfjk2KLjG\n\twoxpOj3E7MLzHXvc3koKzhs+oU7oF7qHL5+GONkbAECWpMvBbgx6hNQLBI/isgJc46Eyobz3Qh2la\n\tabeU34I0SoV/rs4Rxs9xP9e4b2VcokN0Z9Xd6QamD+h1DtfkpoJugFWZ4/IsMxdT0Q0xphQGH6Sbf\n\tCsH3vlB1EG2ODnZGcVU9YdHiCa0LmXOXAygmsvrnGuacxB8bm3odou9gLJDdFS577sD9shW3oX9Mu\n\tpvc1BHznWALK/bMK7j7Q==;","From":"Marc Zyngier ","To":"Christoffer Dall ","Subject":"Re: [PATCH v3 15/20] KVM: arm/arm64: Support EL1 phys timer register\n\taccess in set/get reg","In-Reply-To":"<20170923004207.22356-16-cdall@linaro.org> (Christoffer Dall's\n\tmessage of \"Sat, 23 Sep 2017 02:42:02 +0200\")","Organization":"ARM Ltd","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-16-cdall@linaro.org>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)","Date":"Tue, 10 Oct 2017 10:10:27 +0100","Message-ID":"<87infnwepo.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171010_021056_500141_050146DF ","X-CRM114-Status":"GOOD ( 20.23 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783535,"web_url":"http://patchwork.ozlabs.org/comment/1783535/","msgid":"<87efqbwen0.fsf@on-the-bus.cambridge.arm.com>","list_archive_url":null,"date":"2017-10-10T09:12:03","subject":"Re: [PATCH v3 16/20] KVM: arm/arm64: Use kvm_arm_timer_set/get_reg\n\tfor guest register traps","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On Sat, Sep 23 2017 at 2:42:03 am BST, Christoffer Dall wrote:\n> When trapping on a guest access to one of the timer registers, we were\n> messing with the internals of the timer state from the sysregs handling\n> code, and that logic was about to receive more added complexity when\n> optimizing the timer handling code.\n>\n> Therefore, since we already have timer register access functions (to\n> access registers from userspace), reuse those for the timer register\n> traps from a VM and let the timer code maintain its own consistency.\n>\n> Signed-off-by: Christoffer Dall \n> ---\n> arch/arm64/kvm/sys_regs.c | 41 ++++++++++++++---------------------------\n> 1 file changed, 14 insertions(+), 27 deletions(-)\n>\n> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c\n> index 2e070d3..bb0e41b 100644\n> --- a/arch/arm64/kvm/sys_regs.c\n> +++ b/arch/arm64/kvm/sys_regs.c\n> @@ -841,13 +841,16 @@ static bool access_cntp_tval(struct kvm_vcpu *vcpu,\n> \t\tstruct sys_reg_params *p,\n> \t\tconst struct sys_reg_desc *r)\n> {\n> -\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> \tu64 now = kvm_phys_timer_read();\n> +\tu64 cval;\n> \n> -\tif (p->is_write)\n> -\t\tptimer->cnt_cval = p->regval + now;\n> -\telse\n> -\t\tp->regval = ptimer->cnt_cval - now;\n> +\tif (p->is_write) {\n> +\t\tkvm_arm_timer_set_reg(vcpu, KVM_REG_ARM_PTIMER_CVAL,\n> +\t\t\t\t p->regval + now);\n> +\t} else {\n> +\t\tcval = kvm_arm_timer_get_reg(vcpu, KVM_REG_ARM_PTIMER_CVAL);\n> +\t\tp->regval = cval - now;\n> +\t}\n> \n> \treturn true;\n> }\n> @@ -856,24 +859,10 @@ static bool access_cntp_ctl(struct kvm_vcpu *vcpu,\n> \t\tstruct sys_reg_params *p,\n> \t\tconst struct sys_reg_desc *r)\n> {\n> -\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> -\n> -\tif (p->is_write) {\n> -\t\t/* ISTATUS bit is read-only */\n> -\t\tptimer->cnt_ctl = p->regval & ~ARCH_TIMER_CTRL_IT_STAT;\n> -\t} else {\n> -\t\tu64 now = kvm_phys_timer_read();\n> -\n> -\t\tp->regval = ptimer->cnt_ctl;\n> -\t\t/*\n> -\t\t * Set ISTATUS bit if it's expired.\n> -\t\t * Note that according to ARMv8 ARM Issue A.k, ISTATUS bit is\n> -\t\t * UNKNOWN when ENABLE bit is 0, so we chose to set ISTATUS bit\n> -\t\t * regardless of ENABLE bit for our implementation convenience.\n> -\t\t */\n> -\t\tif (ptimer->cnt_cval <= now)\n> -\t\t\tp->regval |= ARCH_TIMER_CTRL_IT_STAT;\n> -\t}\n> +\tif (p->is_write)\n> +\t\tkvm_arm_timer_set_reg(vcpu, KVM_REG_ARM_PTIMER_CTL, p->regval);\n> +\telse\n> +\t\tp->regval = kvm_arm_timer_get_reg(vcpu, KVM_REG_ARM_PTIMER_CTL);\n> \n> \treturn true;\n> }\n> @@ -882,12 +871,10 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,\n> \t\tstruct sys_reg_params *p,\n> \t\tconst struct sys_reg_desc *r)\n> {\n> -\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> -\n> \tif (p->is_write)\n> -\t\tptimer->cnt_cval = p->regval;\n> +\t\tkvm_arm_timer_set_reg(vcpu, KVM_REG_ARM_PTIMER_CVAL, p->regval);\n> \telse\n> -\t\tp->regval = ptimer->cnt_cval;\n> +\t\tp->regval = kvm_arm_timer_get_reg(vcpu, KVM_REG_ARM_PTIMER_CVAL);\n> \n> \treturn true;\n> }\n\nAcked-by: Marc Zyngier \n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"M2j3vyTO\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yBBGQ2KrHz9sPm\n\tfor ;\n\tTue, 10 Oct 2017 20:12:46 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1qac-0005UT-D5; Tue, 10 Oct 2017 09:12:38 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1qaV-0005Pt-6F for linux-arm-kernel@lists.infradead.org;\n\tTue, 10 Oct 2017 09:12:36 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0F42580D;\n\tTue, 10 Oct 2017 02:12:11 -0700 (PDT)","from on-the-bus (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t148A03F483; Tue, 10 Oct 2017 02:12:06 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References\n\t:In-Reply-To:Subject:To:From:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=X41uSPAk1vUBJuenEDsZdGn/bHzVXCgXDuBN7m7paeQ=;\n\tb=M2j3vyTOWXBnEm\n\tWhXCtEuh5uxRcdMPXoPXCQ/63UranmoAGeruAUCLVRCmmwNxG3NsJOyUwBHJDS+yJZ/wK3N72X5iX\n\txGed/rAypMWPJyRxjYSttrhxtezV9aVVROtlOOSPAwLhZTLLDIiB7uFYfGMdcE8qc7QaVFiFG8k2B\n\t8Bw9piYh/DCpMAOEFtN6OafxSBsLpm1iCxSIAiRSLg+8nnKl8pXGTDrE8IWXwcbyckoF4qDRYtuz4\n\tduAQVw6TFzJa7dMCAx7Gf/xcAfllJK3ExYj5thK1KhDTk7La8oR4zwrm6CDA8aQ02nIwNP7EhB0k0\n\t7KH3SWPT9kI/rUV+cgcQ==;","From":"Marc Zyngier ","To":"Christoffer Dall ","Subject":"Re: [PATCH v3 16/20] KVM: arm/arm64: Use kvm_arm_timer_set/get_reg\n\tfor guest register traps","In-Reply-To":"<20170923004207.22356-17-cdall@linaro.org> (Christoffer Dall's\n\tmessage of \"Sat, 23 Sep 2017 02:42:03 +0200\")","Organization":"ARM Ltd","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-17-cdall@linaro.org>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)","Date":"Tue, 10 Oct 2017 10:12:03 +0100","Message-ID":"<87efqbwen0.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171010_021231_260628_224A089A ","X-CRM114-Status":"GOOD ( 15.28 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783553,"web_url":"http://patchwork.ozlabs.org/comment/1783553/","msgid":"<87a80zwe7h.fsf@on-the-bus.cambridge.arm.com>","list_archive_url":null,"date":"2017-10-10T09:21:22","subject":"Re: [PATCH v3 17/20] KVM: arm/arm64: Move phys_timer_emulate\n\tfunction","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On Sat, Sep 23 2017 at 2:42:04 am BST, Christoffer Dall wrote:\n> We are about to call phys_timer_emulate() from kvm_timer_update_state()\n> and modify phys_timer_emulate() at the same time. Moving the function\n> and modifying it in a single patch makes the diff hard to read, so do\n> this separately first.\n>\n> No functional change.\n>\n> Signed-off-by: Christoffer Dall \n> ---\n> virt/kvm/arm/arch_timer.c | 32 ++++++++++++++++----------------\n> 1 file changed, 16 insertions(+), 16 deletions(-)\n>\n> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> index d5b632d..1f82c21 100644\n> --- a/virt/kvm/arm/arch_timer.c\n> +++ b/virt/kvm/arm/arch_timer.c\n> @@ -252,6 +252,22 @@ static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,\n> \t}\n> }\n> \n> +/* Schedule the background timer for the emulated timer. */\n> +static void phys_timer_emulate(struct kvm_vcpu *vcpu,\n> +\t\t\t struct arch_timer_context *timer_ctx)\n> +{\n> +\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> +\n> +\tif (kvm_timer_should_fire(timer_ctx))\n> +\t\treturn;\n> +\n> +\tif (!kvm_timer_irq_can_fire(timer_ctx))\n> +\t\treturn;\n> +\n> +\t/* The timer has not yet expired, schedule a background timer */\n> +\tsoft_timer_start(&timer->phys_timer, kvm_timer_compute_delta(timer_ctx));\n> +}\n> +\n> /*\n> * Check if there was a change in the timer state (should we raise or lower\n> * the line level to the GIC).\n> @@ -278,22 +294,6 @@ static void kvm_timer_update_state(struct kvm_vcpu *vcpu)\n> \t\tkvm_timer_update_irq(vcpu, !ptimer->irq.level, ptimer);\n> }\n> \n> -/* Schedule the background timer for the emulated timer. */\n> -static void phys_timer_emulate(struct kvm_vcpu *vcpu,\n> -\t\t\t struct arch_timer_context *timer_ctx)\n> -{\n> -\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> -\n> -\tif (kvm_timer_should_fire(timer_ctx))\n> -\t\treturn;\n> -\n> -\tif (!kvm_timer_irq_can_fire(timer_ctx))\n> -\t\treturn;\n> -\n> -\t/* The timer has not yet expired, schedule a background timer */\n> -\tsoft_timer_start(&timer->phys_timer, kvm_timer_compute_delta(timer_ctx));\n> -}\n> -\n> static void vtimer_save_state(struct kvm_vcpu *vcpu)\n> {\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n\nAcked-by: Marc Zyngier \n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"cGIXF4Ol\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yBBmm2hPTz9sBW\n\tfor ;\n\tTue, 10 Oct 2017 20:35:36 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1qwo-0004bH-6u; Tue, 10 Oct 2017 09:35:34 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1qjU-0002Bk-Lf for linux-arm-kernel@lists.infradead.org;\n\tTue, 10 Oct 2017 09:21:52 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9209D80D;\n\tTue, 10 Oct 2017 02:21:28 -0700 (PDT)","from on-the-bus (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tC85193F483; Tue, 10 Oct 2017 02:21:25 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References\n\t:In-Reply-To:Subject:To:From:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=2j4k3179wG6zQMJUC6mxCrwdUSv4dV6pQUnfkMnLkYs=;\n\tb=cGIXF4OlndxDqa\n\tyiDmdmjbJhmZnkyqnUaCQ3JUywsQEZI3dcnOHUAuSSnGxiOHXYF40qJitSrph/e/VX/JgTfpqahRV\n\txt3GEmpsRkIvIO5XUBMGKfZ/5GTkC39pAW8ZvETumiCYNSDUbBIppu6FGSmSsrejLZNOtGZIJ3dD4\n\tHa1FRGRic8z60oZb3n2lOrbdzOj2+dP1R7aLB5D0QdtY4ICfEiBdr+Hq0sicdkLOOTLk3Wy/xrPz/\n\tvWroLm/kuJobVhPIUdkAEeZMdxt1NFZUUHeqwecTYqCy1woY80Va0TePVvCi4L7Mfg63kOVhHOomY\n\tTZAf9kYpZZK3W/8gOuGA==;","From":"Marc Zyngier ","To":"Christoffer Dall ","Subject":"Re: [PATCH v3 17/20] KVM: arm/arm64: Move phys_timer_emulate\n\tfunction","In-Reply-To":"<20170923004207.22356-18-cdall@linaro.org> (Christoffer Dall's\n\tmessage of \"Sat, 23 Sep 2017 02:42:04 +0200\")","Organization":"ARM Ltd","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-18-cdall@linaro.org>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)","Date":"Tue, 10 Oct 2017 10:21:22 +0100","Message-ID":"<87a80zwe7h.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171010_022149_100739_A131B43E ","X-CRM114-Status":"GOOD ( 15.17 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783565,"web_url":"http://patchwork.ozlabs.org/comment/1783565/","msgid":"<8760bnwd3o.fsf@on-the-bus.cambridge.arm.com>","list_archive_url":null,"date":"2017-10-10T09:45:15","subject":"Re: [PATCH v3 18/20] KVM: arm/arm64: Avoid phys timer emulation in\n\tvcpu entry/exit","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On Sat, Sep 23 2017 at 2:42:05 am BST, Christoffer Dall wrote:\n> There is no need to schedule and cancel a hrtimer when entering and\n> exiting the guest, because we know when the physical timer is going to\n> fire when the guest programs it, and we can simply program the hrtimer\n> at that point.\n>\n> Now when the register modifications from the guest go through the\n> kvm_arm_timer_set/get_reg functions, which always call\n> kvm_timer_update_state(), we can simply consider the timer state in this\n> function and schedule and cancel the timers as needed.\n>\n> This avoids looking at the physical timer emulation state when entering\n> and exiting the VCPU, allowing for faster servicing of the VM when\n> needed.\n>\n> Signed-off-by: Christoffer Dall \n> ---\n> virt/kvm/arm/arch_timer.c | 75 ++++++++++++++++++++++++++++++++---------------\n> 1 file changed, 51 insertions(+), 24 deletions(-)\n>\n> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> index 1f82c21..aa18a5d 100644\n> --- a/virt/kvm/arm/arch_timer.c\n> +++ b/virt/kvm/arm/arch_timer.c\n> @@ -199,7 +199,27 @@ static enum hrtimer_restart kvm_bg_timer_expire(struct hrtimer *hrt)\n> \n> static enum hrtimer_restart kvm_phys_timer_expire(struct hrtimer *hrt)\n> {\n> -\tWARN(1, \"Timer only used to ensure guest exit - unexpected event.\");\n> +\tstruct arch_timer_context *ptimer;\n> +\tstruct arch_timer_cpu *timer;\n> +\tstruct kvm_vcpu *vcpu;\n> +\tu64 ns;\n> +\n> +\ttimer = container_of(hrt, struct arch_timer_cpu, phys_timer);\n> +\tvcpu = container_of(timer, struct kvm_vcpu, arch.timer_cpu);\n> +\tptimer = vcpu_ptimer(vcpu);\n> +\n> +\t/*\n> +\t * Check that the timer has really expired from the guest's\n> +\t * PoV (NTP on the host may have forced it to expire\n> +\t * early). If not ready, schedule for a later time.\n> +\t */\n> +\tns = kvm_timer_compute_delta(ptimer);\n> +\tif (unlikely(ns)) {\n> +\t\thrtimer_forward_now(hrt, ns_to_ktime(ns));\n> +\t\treturn HRTIMER_RESTART;\n> +\t}\n\nDon't we already have a similar logic for the background timer (I must\nadmit I've lost track of how we changed things in this series)? If so,\ncan we make this common code?\n\n> +\n> +\tkvm_timer_update_irq(vcpu, true, ptimer);\n> \treturn HRTIMER_NORESTART;\n> }\n> \n> @@ -253,24 +273,28 @@ static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,\n> }\n> \n> /* Schedule the background timer for the emulated timer. */\n> -static void phys_timer_emulate(struct kvm_vcpu *vcpu,\n> -\t\t\t struct arch_timer_context *timer_ctx)\n> +static void phys_timer_emulate(struct kvm_vcpu *vcpu)\n> {\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> +\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> \n> -\tif (kvm_timer_should_fire(timer_ctx))\n> -\t\treturn;\n> -\n> -\tif (!kvm_timer_irq_can_fire(timer_ctx))\n> +\t/*\n> +\t * If the timer can fire now we have just raised the IRQ line and we\n> +\t * don't need to have a soft timer scheduled for the future. If the\n> +\t * timer cannot fire at all, then we also don't need a soft timer.\n> +\t */\n> +\tif (kvm_timer_should_fire(ptimer) || !kvm_timer_irq_can_fire(ptimer)) {\n> +\t\tsoft_timer_cancel(&timer->phys_timer, NULL);\n> \t\treturn;\n> +\t}\n> \n> -\t/* The timer has not yet expired, schedule a background timer */\n> -\tsoft_timer_start(&timer->phys_timer, kvm_timer_compute_delta(timer_ctx));\n> +\tsoft_timer_start(&timer->phys_timer, kvm_timer_compute_delta(ptimer));\n> }\n> \n> /*\n> - * Check if there was a change in the timer state (should we raise or lower\n> - * the line level to the GIC).\n> + * Check if there was a change in the timer state, so that we should either\n> + * raise or lower the line level to the GIC or schedule a background timer to\n> + * emulate the physical timer.\n> */\n> static void kvm_timer_update_state(struct kvm_vcpu *vcpu)\n> {\n> @@ -292,6 +316,8 @@ static void kvm_timer_update_state(struct kvm_vcpu *vcpu)\n> \n> \tif (kvm_timer_should_fire(ptimer) != ptimer->irq.level)\n> \t\tkvm_timer_update_irq(vcpu, !ptimer->irq.level, ptimer);\n> +\n> +\tphys_timer_emulate(vcpu);\n> }\n> \n> static void vtimer_save_state(struct kvm_vcpu *vcpu)\n> @@ -445,6 +471,9 @@ void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)\n> \n> \tif (has_vhe())\n> \t\tdisable_el1_phys_timer_access();\n> +\n> +\t/* Set the background timer for the physical timer emulation. */\n> +\tphys_timer_emulate(vcpu);\n> }\n> \n> bool kvm_timer_should_notify_user(struct kvm_vcpu *vcpu)\n> @@ -480,12 +509,6 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)\n> \n> \tif (unlikely(!timer->enabled))\n> \t\treturn;\n> -\n> -\tif (kvm_timer_should_fire(ptimer) != ptimer->irq.level)\n> -\t\tkvm_timer_update_irq(vcpu, !ptimer->irq.level, ptimer);\n> -\n> -\t/* Set the background timer for the physical timer emulation. */\n> -\tphys_timer_emulate(vcpu, vcpu_ptimer(vcpu));\n> }\n> \n> void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)\n> @@ -500,6 +523,17 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)\n> \n> \tvtimer_save_state(vcpu);\n> \n> +\t/*\n> +\t * Cancel the physical timer emulation, because the only case where we\n> +\t * need it after a vcpu_put is in the context of a sleeping VCPU, and\n> +\t * in that case we already factor in the deadline for the physical\n> +\t * timer when scheduling the bg_timer.\n> +\t *\n> +\t * In any case, we re-schedule the hrtimer for the physical timer when\n> +\t * coming back to the VCPU thread in kvm_timer_vcpu_load().\n> +\t */\n> +\tsoft_timer_cancel(&timer->phys_timer, NULL);\n> +\n> \tset_cntvoff(0);\n> }\n> \n> @@ -536,16 +570,9 @@ static void unmask_vtimer_irq(struct kvm_vcpu *vcpu)\n> */\n> void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)\n> {\n> -\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> \n> \t/*\n> -\t * This is to cancel the background timer for the physical timer\n> -\t * emulation if it is set.\n> -\t */\n> -\tsoft_timer_cancel(&timer->phys_timer, NULL);\n> -\n> -\t/*\n> \t * If we entered the guest with the vtimer output asserted we have to\n> \t * check if the guest has modified the timer so that we should lower\n> \t * the line at this point.\n\nOtherwise:\n\nReviewed-by: Marc Zyngier \n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"tSuV2N0E\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yBC0h2NS9z9tYB\n\tfor ;\n\tTue, 10 Oct 2017 20:45:56 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1r6j-00044J-32; Tue, 10 Oct 2017 09:45:49 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1r6c-0003RO-DQ for linux-arm-kernel@lists.infradead.org;\n\tTue, 10 Oct 2017 09:45:45 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D950F80D;\n\tTue, 10 Oct 2017 02:45:21 -0700 (PDT)","from on-the-bus (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t8BE683F483; Tue, 10 Oct 2017 02:45:19 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References\n\t:In-Reply-To:Subject:To:From:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=QIMBLUqA/lCEAhRWlvLTxAa1qXdEOIln9MproOZYU4w=;\n\tb=tSuV2N0EEG9jD2\n\tQwfTO9ddtWlJys1r2XCWupVFe2WtHH3R0vetCD8xQnCrVL3ap7i25hMHF+yELKR8YoRJnVC34/1UE\n\tR1U4THOcPHKaxlT7ZQf2nmesq+GzEhak7UUxxel52SZevm1wYJ0rOm8vXc2E8FxU9vw5DBmgkB4D4\n\t4U6LdF5ppgESi0fKJ35dOfP6xDt/VWM3Du9uTGvcO4oVSi7bjd251xkkKq2Ve0Lp7eEuWDMldqImH\n\tdearbICBVRiOUp74dmTWnkpW25npNNMOk3unC0Dwdq3QqJBfQmS+PhxWVUNZmCmmpsWwsSrqWJBTE\n\t3OGqlqWGbAQpKOxJSCuA==;","From":"Marc Zyngier ","To":"Christoffer Dall ","Subject":"Re: [PATCH v3 18/20] KVM: arm/arm64: Avoid phys timer emulation in\n\tvcpu entry/exit","In-Reply-To":"<20170923004207.22356-19-cdall@linaro.org> (Christoffer Dall's\n\tmessage of \"Sat, 23 Sep 2017 02:42:05 +0200\")","Organization":"ARM Ltd","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-19-cdall@linaro.org>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)","Date":"Tue, 10 Oct 2017 10:45:15 +0100","Message-ID":"<8760bnwd3o.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171010_024542_511723_D98D2BEB ","X-CRM114-Status":"GOOD ( 25.81 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783567,"web_url":"http://patchwork.ozlabs.org/comment/1783567/","msgid":"<871smbwd0t.fsf@on-the-bus.cambridge.arm.com>","list_archive_url":null,"date":"2017-10-10T09:46:58","subject":"Re: [PATCH v3 19/20] KVM: arm/arm64: Get rid of\n\tkvm_timer_flush_hwstate","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On Sat, Sep 23 2017 at 2:42:06 am BST, Christoffer Dall wrote:\n> Now when both the vtimer and the ptimer when using both the in-kernel\n> vgic emulation and a userspace IRQ chip are driven by the timer signals\n> and at the vcpu load/put boundaries, instead of recomputing the timer\n> state at every entry/exit to/from the guest, we can get entirely rid of\n> the flush hwstate function.\n>\n> Signed-off-by: Christoffer Dall \n> ---\n> include/kvm/arm_arch_timer.h | 1 -\n> virt/kvm/arm/arch_timer.c | 24 ------------------------\n> virt/kvm/arm/arm.c | 1 -\n> 3 files changed, 26 deletions(-)\n>\n> diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h\n> index 8e5ed54..af29563 100644\n> --- a/include/kvm/arm_arch_timer.h\n> +++ b/include/kvm/arm_arch_timer.h\n> @@ -61,7 +61,6 @@ int kvm_timer_hyp_init(void);\n> int kvm_timer_enable(struct kvm_vcpu *vcpu);\n> int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu);\n> void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);\n> -void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu);\n> void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu);\n> bool kvm_timer_should_notify_user(struct kvm_vcpu *vcpu);\n> void kvm_timer_update_run(struct kvm_vcpu *vcpu);\n> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> index aa18a5d..f92459a 100644\n> --- a/virt/kvm/arm/arch_timer.c\n> +++ b/virt/kvm/arm/arch_timer.c\n> @@ -302,12 +302,6 @@ static void kvm_timer_update_state(struct kvm_vcpu *vcpu)\n> \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> \tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> \n> -\t/*\n> -\t * If userspace modified the timer registers via SET_ONE_REG before\n> -\t * the vgic was initialized, we mustn't set the vtimer->irq.level value\n> -\t * because the guest would never see the interrupt. Instead wait\n> -\t * until we call this function from kvm_timer_flush_hwstate.\n> -\t */\n> \tif (unlikely(!timer->enabled))\n> \t\treturn;\n> \n> @@ -493,24 +487,6 @@ bool kvm_timer_should_notify_user(struct kvm_vcpu *vcpu)\n> \t ptimer->irq.level != plevel;\n> }\n> \n> -/**\n> - * kvm_timer_flush_hwstate - prepare timers before running the vcpu\n> - * @vcpu: The vcpu pointer\n> - *\n> - * Check if the virtual timer has expired while we were running in the host,\n> - * and inject an interrupt if that was the case, making sure the timer is\n> - * masked or disabled on the host so that we keep executing. Also schedule a\n> - * software timer for the physical timer if it is enabled.\n> - */\n> -void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)\n> -{\n> -\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> -\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> -\n> -\tif (unlikely(!timer->enabled))\n> -\t\treturn;\n> -}\n> -\n> void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)\n> {\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c\n> index 132d39a..14c50d1 100644\n> --- a/virt/kvm/arm/arm.c\n> +++ b/virt/kvm/arm/arm.c\n> @@ -656,7 +656,6 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)\n> \n> \t\tlocal_irq_disable();\n> \n> -\t\tkvm_timer_flush_hwstate(vcpu);\n> \t\tkvm_vgic_flush_hwstate(vcpu);\n> \n> \t\t/*\n\nAcked-by: Marc Zyngier \n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"WkMkpFTx\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yBC2W2ZBfz9tXl\n\tfor ;\n\tTue, 10 Oct 2017 20:47:31 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1r8L-00058S-Mq; Tue, 10 Oct 2017 09:47:29 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1r8H-000528-1S for linux-arm-kernel@lists.infradead.org;\n\tTue, 10 Oct 2017 09:47:28 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A740880D;\n\tTue, 10 Oct 2017 02:47:04 -0700 (PDT)","from on-the-bus (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t07C843F483; Tue, 10 Oct 2017 02:47:01 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References\n\t:In-Reply-To:Subject:To:From:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=TKdhO3NDNo9J1/y15GjmMBunyry7lwgDf2RtwIq48DU=;\n\tb=WkMkpFTxXyvIDn\n\twNl7PYr59zsnwxTYJ+ATnG2UulPfHFNuHO7LF+G0oCjHpnTC8RrjZhHFSL5w+VSE7LvCvzHmi4aAw\n\tg0T5LGfaJRrj43pWnSQY0ieyL/+GLNaqOd9CAGhLMzawhj6DydJbvhHBV9a/ck+dr1VpnkrSMQcp1\n\tUabM6oBBujswtvqUTwUUwjAM4UJCTJsx2EXMrlZ6Si31Ur1ijK9lLTdeld4ZrRrsIC+o7VbuRTDw1\n\tLZCbW9RbQ2OpwKUJNaYhyD/yTsO9O6tZFGqv6FzJColSH/G6YhuN+4/YhRuKZA16+Qkyz2BHHa7ym\n\tmiTRi91ckTTBcH1Me3Iw==;","From":"Marc Zyngier ","To":"Christoffer Dall ","Subject":"Re: [PATCH v3 19/20] KVM: arm/arm64: Get rid of\n\tkvm_timer_flush_hwstate","In-Reply-To":"<20170923004207.22356-20-cdall@linaro.org> (Christoffer Dall's\n\tmessage of \"Sat, 23 Sep 2017 02:42:06 +0200\")","Organization":"ARM Ltd","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-20-cdall@linaro.org>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)","Date":"Tue, 10 Oct 2017 10:46:58 +0100","Message-ID":"<871smbwd0t.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171010_024725_159189_B7BD360E ","X-CRM114-Status":"GOOD ( 15.22 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1783573,"web_url":"http://patchwork.ozlabs.org/comment/1783573/","msgid":"<87wp43uxvz.fsf@on-the-bus.cambridge.arm.com>","list_archive_url":null,"date":"2017-10-10T09:59:12","subject":"Re: [PATCH v3 20/20] KVM: arm/arm64: Rework kvm_timer_should_fire","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On Sat, Sep 23 2017 at 2:42:07 am BST, Christoffer Dall wrote:\n> kvm_timer_should_fire() can be called in two different situations from\n> the kvm_vcpu_block().\n>\n> The first case is before calling kvm_timer_schedule(), used for wait\n> polling, and in this case the VCPU thread is running and the timer state\n> is loaded onto the hardware so all we have to do is check if the virtual\n> interrupt lines are asserted, becasue the timer interrupt handler\n> functions will raise those lines as appropriate.\n>\n> The second case is inside the wait loop of kvm_vcpu_block(), where we\n> have already called kvm_timer_schedule() and therefore the hardware will\n> be disabled and the software view of the timer state is up to date\n> (timer->loaded is false), and so we can simply check if the timer should\n> fire by looking at the software state.\n>\n> Signed-off-by: Christoffer Dall \n> ---\n> include/kvm/arm_arch_timer.h | 3 ++-\n> virt/kvm/arm/arch_timer.c | 22 +++++++++++++++++++++-\n> virt/kvm/arm/arm.c | 3 +--\n> 3 files changed, 24 insertions(+), 4 deletions(-)\n>\n> diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h\n> index af29563..250db34 100644\n> --- a/include/kvm/arm_arch_timer.h\n> +++ b/include/kvm/arm_arch_timer.h\n> @@ -73,7 +73,8 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr);\n> int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr);\n> int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr);\n> \n> -bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx);\n> +bool kvm_timer_is_pending(struct kvm_vcpu *vcpu);\n> +\n> void kvm_timer_schedule(struct kvm_vcpu *vcpu);\n> void kvm_timer_unschedule(struct kvm_vcpu *vcpu);\n> \n> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> index f92459a..1d0cd3a 100644\n> --- a/virt/kvm/arm/arch_timer.c\n> +++ b/virt/kvm/arm/arch_timer.c\n> @@ -49,6 +49,7 @@ static const struct kvm_irq_level default_vtimer_irq = {\n> static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx);\n> static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,\n> \t\t\t\t struct arch_timer_context *timer_ctx);\n> +static bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx);\n> \n> u64 kvm_phys_timer_read(void)\n> {\n> @@ -223,7 +224,7 @@ static enum hrtimer_restart kvm_phys_timer_expire(struct hrtimer *hrt)\n> \treturn HRTIMER_NORESTART;\n> }\n> \n> -bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)\n> +static bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)\n> {\n> \tu64 cval, now;\n> \n> @@ -236,6 +237,25 @@ bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)\n> \treturn cval <= now;\n> }\n> \n> +bool kvm_timer_is_pending(struct kvm_vcpu *vcpu)\n> +{\n> +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> +\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> +\n> +\tif (vtimer->irq.level || ptimer->irq.level)\n> +\t\treturn true;\n> +\n> +\t/*\n> +\t * When this is called from withing the wait loop of kvm_vcpu_block(),\n> +\t * the software view of the timer state is up to date (timer->loaded\n> +\t * is false), and so we can simply check if the timer should fire now.\n> +\t */\n> +\tif (!vtimer->loaded && kvm_timer_should_fire(vtimer))\n> +\t\treturn true;\n> +\n> +\treturn kvm_timer_should_fire(ptimer);\n> +}\n> +\n> /*\n> * Reflect the timer output level into the kvm_run structure\n> */\n> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c\n> index 14c50d1..bc126fb 100644\n> --- a/virt/kvm/arm/arm.c\n> +++ b/virt/kvm/arm/arm.c\n> @@ -307,8 +307,7 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)\n> \n> int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)\n> {\n> -\treturn kvm_timer_should_fire(vcpu_vtimer(vcpu)) ||\n> -\t kvm_timer_should_fire(vcpu_ptimer(vcpu));\n> +\treturn kvm_timer_is_pending(vcpu);\n> }\n> \n> void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)\n\nReviewed-by: Marc Zyngier \n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"o7eDBm71\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yBCJd3t3Sz9tY3\n\tfor ;\n\tTue, 10 Oct 2017 20:59:45 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1rK9-0004v0-Ut; Tue, 10 Oct 2017 09:59:41 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e1rK6-0004qQ-Bf for linux-arm-kernel@lists.infradead.org;\n\tTue, 10 Oct 2017 09:59:40 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 03B2E80D;\n\tTue, 10 Oct 2017 02:59:18 -0700 (PDT)","from on-the-bus (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tA5A8D3F58C; Tue, 10 Oct 2017 02:59:15 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References\n\t:In-Reply-To:Subject:To:From:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=wrpEuldwNwhDnHWHfnfryBF78RfQqxktLCLmkdLYIJI=;\n\tb=o7eDBm71kp9soW\n\tOEjoLt2X9lalKlLcs7nz2r15PQ9MeJ7dXhtOQnzBdHrCXSexwso5RgmRc6e0LMWVRLAszlUU3Lmcz\n\tQsSsS2+zoMzbpeZhebcNKCeG8ZGjhrOPTmD0KvSOhW70a3Qp/ik17VKfA2KeDfZw2etsNALsuUd/H\n\tajBjfmzbN8jI8euphHzpnhE1BZiPRCwOTxAN20HTezOlhxWDyVfCL7BttAg0/6nwnHh2DasxApsUr\n\tPuthrCFOog5ADe2h4fcgu/WBuF9oIBYv7W3G2V1TKnulKCeRXZ4bOhnQ/aIpxmF9RGL6vMhxLAlsw\n\tb1MplIVUIr7tMDctnmzg==;","From":"Marc Zyngier ","To":"Christoffer Dall ","Subject":"Re: [PATCH v3 20/20] KVM: arm/arm64: Rework kvm_timer_should_fire","In-Reply-To":"<20170923004207.22356-21-cdall@linaro.org> (Christoffer Dall's\n\tmessage of \"Sat, 23 Sep 2017 02:42:07 +0200\")","Organization":"ARM Ltd","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-21-cdall@linaro.org>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)","Date":"Tue, 10 Oct 2017 10:59:12 +0100","Message-ID":"<87wp43uxvz.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171010_025938_411942_3E9D9B9C ","X-CRM114-Status":"GOOD ( 18.92 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1788556,"web_url":"http://patchwork.ozlabs.org/comment/1788556/","msgid":"<20171017153305.GB19711@arm.com>","list_archive_url":null,"date":"2017-10-17T15:33:05","subject":"Re: [PATCH v3 02/20] arm64: Use physical counter for in-kernel reads","submitter":{"id":7916,"url":"http://patchwork.ozlabs.org/api/people/7916/","name":"Will Deacon","email":"will.deacon@arm.com"},"content":"Hi Christoffer,\n\nOn Sat, Sep 23, 2017 at 02:41:49AM +0200, Christoffer Dall wrote:\n> Using the physical counter allows KVM to retain the offset between the\n> virtual and physical counter as long as it is actively running a VCPU.\n> \n> As soon as a VCPU is released, another thread is scheduled or we start\n> running userspace applications, we reset the offset to 0, so that\n> userspace accessing the virtual timer can still read the cirtual counter\n> and get the same view of time as the kernel.\n> \n> This opens up potential improvements for KVM performance.\n> \n> VHE kernels or kernels continuing to use the virtual timer are\n> unaffected.\n> \n> Signed-off-by: Christoffer Dall \n> ---\n> arch/arm64/include/asm/arch_timer.h | 9 ++++-----\n> drivers/clocksource/arm_arch_timer.c | 3 +--\n> 2 files changed, 5 insertions(+), 7 deletions(-)\n> \n> diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h\n> index a652ce0..1859a1c 100644\n> --- a/arch/arm64/include/asm/arch_timer.h\n> +++ b/arch/arm64/include/asm/arch_timer.h\n> @@ -148,11 +148,10 @@ static inline void arch_timer_set_cntkctl(u32 cntkctl)\n> \n> static inline u64 arch_counter_get_cntpct(void)\n> {\n> -\t/*\n> -\t * AArch64 kernel and user space mandate the use of CNTVCT.\n> -\t */\n> -\tBUG();\n> -\treturn 0;\n> +\tu64 cval;\n> +\tisb();\n> +\tasm volatile(\"mrs %0, cntpct_el0\" : \"=r\" (cval));\n> +\treturn cval;\n> }\n> \n> static inline u64 arch_counter_get_cntvct(void)\n> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c\n> index fd4b7f6..9b3322a 100644\n> --- a/drivers/clocksource/arm_arch_timer.c\n> +++ b/drivers/clocksource/arm_arch_timer.c\n> @@ -890,8 +890,7 @@ static void __init arch_counter_register(unsigned type)\n> \n> \t/* Register the CP15 based counter if we have one */\n> \tif (type & ARCH_TIMER_TYPE_CP15) {\n> -\t\tif (IS_ENABLED(CONFIG_ARM64) ||\n> -\t\t arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI)\n> +\t\tif (arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI)\n\nPlease can you add an is_hyp_mode_available() check here, as you suggested\nlast time?\n\nhttp://lists.infradead.org/pipermail/linux-arm-kernel/2017-July/521542.html\n\nWithout it, I worry that the kernel timekeeper will be out of sync with the\nvDSO (which uses the virtual counter) on systems where CNTVOFF is\ninitialised to a consistent non-zero offset and Linux was loaded at EL1.\n\nThanks,\n\nWill","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"SOyjf0fJ\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yGfNT4kVbz9t2m\n\tfor ;\n\tWed, 18 Oct 2017 02:33:29 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4Trw-0001Ha-Hv; Tue, 17 Oct 2017 15:33:24 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4Trs-00016Z-NX for linux-arm-kernel@lists.infradead.org;\n\tTue, 17 Oct 2017 15:33:22 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 79A6BF;\n\tTue, 17 Oct 2017 08:33:00 -0700 (PDT)","from edgewater-inn.cambridge.arm.com\n\t(usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id\n\t4BD183F590; Tue, 17 Oct 2017 08:33:00 -0700 (PDT)","by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000)\n\tid A3F9F1AE2F39; Tue, 17 Oct 2017 16:33:05 +0100 (BST)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:\n\tMessage-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=5nKFcq5c+smXs1+e4lLC3XvL2vDiPZXwfBhUEos5pkk=;\n\tb=SOyjf0fJNwJr2s\n\tD/m6c0J14HjN/ZBnc0Bt1cQZaxbBY62VyJSPOq5OKdUTzR6ZNqdAyZOOaWDtuDLXaEnK20QYQLmVV\n\tIB5cD0D+cu00hfW9JNcSwn+5jDHn8lIWcGhhkZ30BVsXlnY7zXNHMnyHAaarX0kkSgroY0eVVjf9i\n\ti7dcH8VFtD9J5F2n793h6a2EGJUHmL6I03tPaEo8J5v30jlL1aBT3/44XVcUK8UhQ2hYODsbeOvpw\n\t+LTpQPgGiMrB8cYuB0ehFYMP/X3zhobCQA4vih//AWHEBBylZXVOmdAYS5dnpc8j6FbaXih/1tIh7\n\tTTGMWbsGYhLPE2Sat9aA==;","Date":"Tue, 17 Oct 2017 16:33:05 +0100","From":"Will Deacon ","To":"Christoffer Dall ","Subject":"Re: [PATCH v3 02/20] arm64: Use physical counter for in-kernel reads","Message-ID":"<20171017153305.GB19711@arm.com>","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-3-cdall@linaro.org>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20170923004207.22356-3-cdall@linaro.org>","User-Agent":"Mutt/1.5.23 (2014-03-12)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171017_083320_774779_0D47F49C ","X-CRM114-Status":"GOOD ( 17.17 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas , kvmarm@lists.cs.columbia.edu, \n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1789318,"web_url":"http://patchwork.ozlabs.org/comment/1789318/","msgid":"<20171018100007.GK8326@lvm>","list_archive_url":null,"date":"2017-10-18T10:00:07","subject":"Re: [PATCH v3 02/20] arm64: Use physical counter for in-kernel reads","submitter":{"id":71350,"url":"http://patchwork.ozlabs.org/api/people/71350/","name":"Christoffer Dall","email":"cdall@linaro.org"},"content":"On Tue, Oct 17, 2017 at 04:33:05PM +0100, Will Deacon wrote:\n> Hi Christoffer,\n> \n> On Sat, Sep 23, 2017 at 02:41:49AM +0200, Christoffer Dall wrote:\n> > Using the physical counter allows KVM to retain the offset between the\n> > virtual and physical counter as long as it is actively running a VCPU.\n> > \n> > As soon as a VCPU is released, another thread is scheduled or we start\n> > running userspace applications, we reset the offset to 0, so that\n> > userspace accessing the virtual timer can still read the cirtual counter\n> > and get the same view of time as the kernel.\n> > \n> > This opens up potential improvements for KVM performance.\n> > \n> > VHE kernels or kernels continuing to use the virtual timer are\n> > unaffected.\n> > \n> > Signed-off-by: Christoffer Dall \n> > ---\n> > arch/arm64/include/asm/arch_timer.h | 9 ++++-----\n> > drivers/clocksource/arm_arch_timer.c | 3 +--\n> > 2 files changed, 5 insertions(+), 7 deletions(-)\n> > \n> > diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h\n> > index a652ce0..1859a1c 100644\n> > --- a/arch/arm64/include/asm/arch_timer.h\n> > +++ b/arch/arm64/include/asm/arch_timer.h\n> > @@ -148,11 +148,10 @@ static inline void arch_timer_set_cntkctl(u32 cntkctl)\n> > \n> > static inline u64 arch_counter_get_cntpct(void)\n> > {\n> > -\t/*\n> > -\t * AArch64 kernel and user space mandate the use of CNTVCT.\n> > -\t */\n> > -\tBUG();\n> > -\treturn 0;\n> > +\tu64 cval;\n> > +\tisb();\n> > +\tasm volatile(\"mrs %0, cntpct_el0\" : \"=r\" (cval));\n> > +\treturn cval;\n> > }\n> > \n> > static inline u64 arch_counter_get_cntvct(void)\n> > diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c\n> > index fd4b7f6..9b3322a 100644\n> > --- a/drivers/clocksource/arm_arch_timer.c\n> > +++ b/drivers/clocksource/arm_arch_timer.c\n> > @@ -890,8 +890,7 @@ static void __init arch_counter_register(unsigned type)\n> > \n> > \t/* Register the CP15 based counter if we have one */\n> > \tif (type & ARCH_TIMER_TYPE_CP15) {\n> > -\t\tif (IS_ENABLED(CONFIG_ARM64) ||\n> > -\t\t arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI)\n> > +\t\tif (arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI)\n> \n> Please can you add an is_hyp_mode_available() check here, as you suggested\n> last time?\n> \n> http://lists.infradead.org/pipermail/linux-arm-kernel/2017-July/521542.html\n> \n> Without it, I worry that the kernel timekeeper will be out of sync with the\n> vDSO (which uses the virtual counter) on systems where CNTVOFF is\n> initialised to a consistent non-zero offset and Linux was loaded at EL1.\n> \n\nYes, will do.\n\nThanks,\n-Christoffer","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"RXSCn3Bp\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"JHLMbVsP\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yH6y34MHBz9t3l\n\tfor ;\n\tWed, 18 Oct 2017 21:00:43 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4l9R-0007M1-9z; Wed, 18 Oct 2017 10:00:37 +0000","from mail-wm0-x232.google.com ([2a00:1450:400c:c09::232])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4l9L-0006vz-PD for linux-arm-kernel@lists.infradead.org;\n\tWed, 18 Oct 2017 10:00:34 +0000","by mail-wm0-x232.google.com with SMTP id f4so9092919wme.0\n\tfor ;\n\tWed, 18 Oct 2017 03:00:10 -0700 (PDT)","from localhost (xd93dd96b.cust.hiper.dk. [217.61.217.107])\n\tby smtp.gmail.com with ESMTPSA id\n\to55sm9161559edc.90.2017.10.18.03.00.08\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tWed, 18 Oct 2017 03:00:08 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:\n\tMessage-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=GgbHtKpKOH88X80d5b3lYaMDbA+BvB/3UyPOTNPpSbA=;\n\tb=RXSCn3Bp2osZeb\n\td2668sCejjNu19NT++xvdwgSxHrjnbYmWQ3hXIowgF7OY2L3R9miuLfYRNai8+ZLk2XX9qP5+yo2P\n\tc4ARGJR6UF8HlSvzWd7oVS3Jyzvbe6feoYSkKY5nmsvxsOsL8vMXxHImA3OlA4Yk66b/6AqNBGC9J\n\t6w8IS5XHW6BszeWaV+7QMNYzTVvqbftISTcYLOKF2q5xOYjtqImte7notUJ0Y9T+GQRRhluzB4Vac\n\tVuytjSYKLPTQP7wk0Sr8V6Nb/aRotJgeeN/xmocxgKbZxTzYYLlimpmVQBgHYLaMXPDGDspzNOElL\n\tlMttuwgC89/LmD8pwOgA==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=aS37278T1kbkhgu5sIxmZi40Vk2ti7DcbaE8orjVAJ4=;\n\tb=JHLMbVsPTK0QEYNQVv4yrltcdd2FZ13Eu0CL+9H8ki6Yn4jMTCiesvQReY/O7lN3Q7\n\tvRGfyfXwuHQ15mK7+FhQ6HdVipHrocqVWAup01sIDlcTk6rZ6rkZER9j9GNWyCdWXax4\n\tBH8nd+y2Y+gm9NuUOCbOGnjO+bgP8hCvwYWSA="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=aS37278T1kbkhgu5sIxmZi40Vk2ti7DcbaE8orjVAJ4=;\n\tb=nxo3bAtF9hAipjHZIXkwBkLhX3rxTmZswKDc+W/LBv2bzalAzTN0vL1dDBVfINnji9\n\tsSUVWFDsLN0VWLEVnYftL/Q01gaUm7txPMksAWUzMuub7q1vB28WDkT2TLlRm/x/6PuY\n\tpiqawV1qsu/+A7UYkRL3T49KtJFBClVyBl0emEu4B3Zz5BhHg3Edb4yTHGnNgfBNNM6Q\n\tRFitHv+nS+UCVHT+RNH/3dLOmRJKuBDYIetij/VDnxtUgvUzwWdes1w74m+Xh2Veir6q\n\tPMAa6KFAZZjNZktIL7DP4t7IU8PQvFggbr4FIDHHrS0Vo4Dte5TC+6/e/Qr6VYORj4N+\n\t8yAw==","X-Gm-Message-State":"AMCzsaWzNEtpj46sgUbF3jC4NbVCeMeOeidQMJSusE8iai37l8kwL3Xr\n\tn0SwdAKFPmxDgSEIrRcuq9EzdA==","X-Google-Smtp-Source":"AOwi7QCOgElctAG6be4WqvnlLOesCm7BYJDWuWBOCznr/+5pwcWyNMxdpA+vCYzbvQW31TLY4hZQCA==","X-Received":"by 10.80.146.98 with SMTP id j31mr20693215eda.298.1508320809497; \n\tWed, 18 Oct 2017 03:00:09 -0700 (PDT)","Date":"Wed, 18 Oct 2017 12:00:07 +0200","From":"Christoffer Dall ","To":"Will Deacon ","Subject":"Re: [PATCH v3 02/20] arm64: Use physical counter for in-kernel reads","Message-ID":"<20171018100007.GK8326@lvm>","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-3-cdall@linaro.org>\n\t<20171017153305.GB19711@arm.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20171017153305.GB19711@arm.com>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171018_030031_979019_E5D53CB8 ","X-CRM114-Status":"GOOD ( 21.74 )","X-Spam-Score":"-2.7 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/,\n\tlow\n\ttrust [2a00:1450:400c:c09:0:0:0:232 listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Marc Zyngier ,\n\tCatalin Marinas , kvmarm@lists.cs.columbia.edu, \n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1789452,"web_url":"http://patchwork.ozlabs.org/comment/1789452/","msgid":"<20171018115436.GC8900@cbox>","list_archive_url":null,"date":"2017-10-18T11:54:36","subject":"Re: [PATCH v3 05/20] KVM: arm/arm64: Support calling\n\tvgic_update_irq_pending from irq context","submitter":{"id":71350,"url":"http://patchwork.ozlabs.org/api/people/71350/","name":"Christoffer Dall","email":"cdall@linaro.org"},"content":"On Mon, Oct 09, 2017 at 05:37:43PM +0100, Marc Zyngier wrote:\n> On 23/09/17 01:41, Christoffer Dall wrote:\n> > We are about to optimize our timer handling logic which involves\n> > injecting irqs to the vgic directly from the irq handler.\n> > \n> > Unfortunately, the injection path can take any AP list lock and irq lock\n> > and we must therefore make sure to use spin_lock_irqsave where ever\n> > interrupts are enabled and we are taking any of those locks, to avoid\n> > deadlocking between process context and the ISR.\n> > \n> > This changes a lot of the VGIC code, but The good news are that the\n> > changes are mostly mechanical.\n> > \n> > Signed-off-by: Christoffer Dall \n> > ---\n> > virt/kvm/arm/vgic/vgic-its.c | 17 +++++++-----\n> > virt/kvm/arm/vgic/vgic-mmio-v2.c | 22 +++++++++------\n> > virt/kvm/arm/vgic/vgic-mmio-v3.c | 17 +++++++-----\n> > virt/kvm/arm/vgic/vgic-mmio.c | 44 +++++++++++++++++------------\n> > virt/kvm/arm/vgic/vgic-v2.c | 5 ++--\n> > virt/kvm/arm/vgic/vgic-v3.c | 12 ++++----\n> > virt/kvm/arm/vgic/vgic.c | 60 +++++++++++++++++++++++++---------------\n> > virt/kvm/arm/vgic/vgic.h | 3 +-\n> > 8 files changed, 108 insertions(+), 72 deletions(-)\n> > \n> > diff --git a/virt/kvm/arm/vgic/vgic-its.c b/virt/kvm/arm/vgic/vgic-its.c\n> > index f51c1e1..9f5e347 100644\n> > --- a/virt/kvm/arm/vgic/vgic-its.c\n> > +++ b/virt/kvm/arm/vgic/vgic-its.c\n> > @@ -278,6 +278,7 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,\n> > \tu64 propbase = GICR_PROPBASER_ADDRESS(kvm->arch.vgic.propbaser);\n> > \tu8 prop;\n> > \tint ret;\n> > +\tunsigned long flags;\n> > \n> > \tret = kvm_read_guest(kvm, propbase + irq->intid - GIC_LPI_OFFSET,\n> > \t\t\t &prop, 1);\n> > @@ -285,15 +286,15 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,\n> > \tif (ret)\n> > \t\treturn ret;\n> > \n> > -\tspin_lock(&irq->irq_lock);\n> > +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \tif (!filter_vcpu || filter_vcpu == irq->target_vcpu) {\n> > \t\tirq->priority = LPI_PROP_PRIORITY(prop);\n> > \t\tirq->enabled = LPI_PROP_ENABLE_BIT(prop);\n> > \n> > -\t\tvgic_queue_irq_unlock(kvm, irq);\n> > +\t\tvgic_queue_irq_unlock(kvm, irq, flags);\n> > \t} else {\n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \t}\n> > \n> > \treturn 0;\n> > @@ -393,6 +394,7 @@ static int its_sync_lpi_pending_table(struct kvm_vcpu *vcpu)\n> > \tint ret = 0;\n> > \tu32 *intids;\n> > \tint nr_irqs, i;\n> > +\tunsigned long flags;\n> > \n> > \tnr_irqs = vgic_copy_lpi_list(vcpu, &intids);\n> > \tif (nr_irqs < 0)\n> > @@ -420,9 +422,9 @@ static int its_sync_lpi_pending_table(struct kvm_vcpu *vcpu)\n> > \t\t}\n> > \n> > \t\tirq = vgic_get_irq(vcpu->kvm, NULL, intids[i]);\n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \t\tirq->pending_latch = pendmask & (1U << bit_nr);\n> > -\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> > +\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > \n> > @@ -515,6 +517,7 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,\n> > {\n> > \tstruct kvm_vcpu *vcpu;\n> > \tstruct its_ite *ite;\n> > +\tunsigned long flags;\n> > \n> > \tif (!its->enabled)\n> > \t\treturn -EBUSY;\n> > @@ -530,9 +533,9 @@ static int vgic_its_trigger_msi(struct kvm *kvm, struct vgic_its *its,\n> > \tif (!vcpu->arch.vgic_cpu.lpis_enabled)\n> > \t\treturn -EBUSY;\n> > \n> > -\tspin_lock(&ite->irq->irq_lock);\n> > +\tspin_lock_irqsave(&ite->irq->irq_lock, flags);\n> > \tite->irq->pending_latch = true;\n> > -\tvgic_queue_irq_unlock(kvm, ite->irq);\n> > +\tvgic_queue_irq_unlock(kvm, ite->irq, flags);\n> > \n> > \treturn 0;\n> > }\n> > diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c\n> > index b3d4a10..e21e2f4 100644\n> > --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c\n> > +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c\n> > @@ -74,6 +74,7 @@ static void vgic_mmio_write_sgir(struct kvm_vcpu *source_vcpu,\n> > \tint mode = (val >> 24) & 0x03;\n> > \tint c;\n> > \tstruct kvm_vcpu *vcpu;\n> > +\tunsigned long flags;\n> > \n> > \tswitch (mode) {\n> > \tcase 0x0:\t\t/* as specified by targets */\n> > @@ -97,11 +98,11 @@ static void vgic_mmio_write_sgir(struct kvm_vcpu *source_vcpu,\n> > \n> > \t\tirq = vgic_get_irq(source_vcpu->kvm, vcpu, intid);\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \t\tirq->pending_latch = true;\n> > \t\tirq->source |= 1U << source_vcpu->vcpu_id;\n> > \n> > -\t\tvgic_queue_irq_unlock(source_vcpu->kvm, irq);\n> > +\t\tvgic_queue_irq_unlock(source_vcpu->kvm, irq, flags);\n> > \t\tvgic_put_irq(source_vcpu->kvm, irq);\n> > \t}\n> > }\n> > @@ -131,6 +132,7 @@ static void vgic_mmio_write_target(struct kvm_vcpu *vcpu,\n> > \tu32 intid = VGIC_ADDR_TO_INTID(addr, 8);\n> > \tu8 cpu_mask = GENMASK(atomic_read(&vcpu->kvm->online_vcpus) - 1, 0);\n> > \tint i;\n> > +\tunsigned long flags;\n> > \n> > \t/* GICD_ITARGETSR[0-7] are read-only */\n> > \tif (intid < VGIC_NR_PRIVATE_IRQS)\n> > @@ -140,13 +142,13 @@ static void vgic_mmio_write_target(struct kvm_vcpu *vcpu,\n> > \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, NULL, intid + i);\n> > \t\tint target;\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \t\tirq->targets = (val >> (i * 8)) & cpu_mask;\n> > \t\ttarget = irq->targets ? __ffs(irq->targets) : 0;\n> > \t\tirq->target_vcpu = kvm_get_vcpu(vcpu->kvm, target);\n> > \n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > }\n> > @@ -174,17 +176,18 @@ static void vgic_mmio_write_sgipendc(struct kvm_vcpu *vcpu,\n> > {\n> > \tu32 intid = addr & 0x0f;\n> > \tint i;\n> > +\tunsigned long flags;\n> > \n> > \tfor (i = 0; i < len; i++) {\n> > \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \t\tirq->source &= ~((val >> (i * 8)) & 0xff);\n> > \t\tif (!irq->source)\n> > \t\t\tirq->pending_latch = false;\n> > \n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > }\n> > @@ -195,19 +198,20 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu,\n> > {\n> > \tu32 intid = addr & 0x0f;\n> > \tint i;\n> > +\tunsigned long flags;\n> > \n> > \tfor (i = 0; i < len; i++) {\n> > \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \t\tirq->source |= (val >> (i * 8)) & 0xff;\n> > \n> > \t\tif (irq->source) {\n> > \t\t\tirq->pending_latch = true;\n> > -\t\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> > +\t\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> > \t\t} else {\n> > -\t\t\tspin_unlock(&irq->irq_lock);\n> > +\t\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \t\t}\n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c\n> > index 408ef06..8378610 100644\n> > --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c\n> > +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c\n> > @@ -129,6 +129,7 @@ static void vgic_mmio_write_irouter(struct kvm_vcpu *vcpu,\n> > {\n> > \tint intid = VGIC_ADDR_TO_INTID(addr, 64);\n> > \tstruct vgic_irq *irq;\n> > +\tunsigned long flags;\n> > \n> > \t/* The upper word is WI for us since we don't implement Aff3. */\n> > \tif (addr & 4)\n> > @@ -139,13 +140,13 @@ static void vgic_mmio_write_irouter(struct kvm_vcpu *vcpu,\n> > \tif (!irq)\n> > \t\treturn;\n> > \n> > -\tspin_lock(&irq->irq_lock);\n> > +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \t/* We only care about and preserve Aff0, Aff1 and Aff2. */\n> > \tirq->mpidr = val & GENMASK(23, 0);\n> > \tirq->target_vcpu = kvm_mpidr_to_vcpu(vcpu->kvm, irq->mpidr);\n> > \n> > -\tspin_unlock(&irq->irq_lock);\n> > +\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \tvgic_put_irq(vcpu->kvm, irq);\n> > }\n> > \n> > @@ -241,11 +242,12 @@ static void vgic_v3_uaccess_write_pending(struct kvm_vcpu *vcpu,\n> > {\n> > \tu32 intid = VGIC_ADDR_TO_INTID(addr, 1);\n> > \tint i;\n> > +\tunsigned long flags;\n> > \n> > \tfor (i = 0; i < len * 8; i++) {\n> > \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \t\tif (test_bit(i, &val)) {\n> > \t\t\t/*\n> > \t\t\t * pending_latch is set irrespective of irq type\n> > @@ -253,10 +255,10 @@ static void vgic_v3_uaccess_write_pending(struct kvm_vcpu *vcpu,\n> > \t\t\t * restore irq config before pending info.\n> > \t\t\t */\n> > \t\t\tirq->pending_latch = true;\n> > -\t\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> > +\t\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> > \t\t} else {\n> > \t\t\tirq->pending_latch = false;\n> > -\t\t\tspin_unlock(&irq->irq_lock);\n> > +\t\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \t\t}\n> > \n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > @@ -799,6 +801,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)\n> > \tint sgi, c;\n> > \tint vcpu_id = vcpu->vcpu_id;\n> > \tbool broadcast;\n> > +\tunsigned long flags;\n> > \n> > \tsgi = (reg & ICC_SGI1R_SGI_ID_MASK) >> ICC_SGI1R_SGI_ID_SHIFT;\n> > \tbroadcast = reg & BIT_ULL(ICC_SGI1R_IRQ_ROUTING_MODE_BIT);\n> > @@ -837,10 +840,10 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)\n> > \n> > \t\tirq = vgic_get_irq(vcpu->kvm, c_vcpu, sgi);\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \t\tirq->pending_latch = true;\n> > \n> > -\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> > +\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > }\n> > diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c\n> > index c1e4bdd..deb51ee 100644\n> > --- a/virt/kvm/arm/vgic/vgic-mmio.c\n> > +++ b/virt/kvm/arm/vgic/vgic-mmio.c\n> > @@ -69,13 +69,14 @@ void vgic_mmio_write_senable(struct kvm_vcpu *vcpu,\n> > {\n> > \tu32 intid = VGIC_ADDR_TO_INTID(addr, 1);\n> > \tint i;\n> > +\tunsigned long flags;\n> > \n> > \tfor_each_set_bit(i, &val, len * 8) {\n> > \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \t\tirq->enabled = true;\n> > -\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> > +\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> > \n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > @@ -87,15 +88,16 @@ void vgic_mmio_write_cenable(struct kvm_vcpu *vcpu,\n> > {\n> > \tu32 intid = VGIC_ADDR_TO_INTID(addr, 1);\n> > \tint i;\n> > +\tunsigned long flags;\n> > \n> > \tfor_each_set_bit(i, &val, len * 8) {\n> > \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \t\tirq->enabled = false;\n> > \n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > }\n> > @@ -126,14 +128,15 @@ void vgic_mmio_write_spending(struct kvm_vcpu *vcpu,\n> > {\n> > \tu32 intid = VGIC_ADDR_TO_INTID(addr, 1);\n> > \tint i;\n> > +\tunsigned long flags;\n> > \n> > \tfor_each_set_bit(i, &val, len * 8) {\n> > \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \t\tirq->pending_latch = true;\n> > \n> > -\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> > +\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > }\n> > @@ -144,15 +147,16 @@ void vgic_mmio_write_cpending(struct kvm_vcpu *vcpu,\n> > {\n> > \tu32 intid = VGIC_ADDR_TO_INTID(addr, 1);\n> > \tint i;\n> > +\tunsigned long flags;\n> > \n> > \tfor_each_set_bit(i, &val, len * 8) {\n> > \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \t\tirq->pending_latch = false;\n> > \n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > }\n> > @@ -181,7 +185,8 @@ static void vgic_mmio_change_active(struct kvm_vcpu *vcpu, struct vgic_irq *irq,\n> > \t\t\t\t bool new_active_state)\n> > {\n> > \tstruct kvm_vcpu *requester_vcpu;\n> > -\tspin_lock(&irq->irq_lock);\n> > +\tunsigned long flags;\n> > +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \t/*\n> > \t * The vcpu parameter here can mean multiple things depending on how\n> > @@ -216,9 +221,9 @@ static void vgic_mmio_change_active(struct kvm_vcpu *vcpu, struct vgic_irq *irq,\n> > \n> > \tirq->active = new_active_state;\n> > \tif (new_active_state)\n> > -\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> > +\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> > \telse\n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > }\n> > \n> > /*\n> > @@ -352,14 +357,15 @@ void vgic_mmio_write_priority(struct kvm_vcpu *vcpu,\n> > {\n> > \tu32 intid = VGIC_ADDR_TO_INTID(addr, 8);\n> > \tint i;\n> > +\tunsigned long flags;\n> > \n> > \tfor (i = 0; i < len; i++) {\n> > \t\tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \t\t/* Narrow the priority range to what we actually support */\n> > \t\tirq->priority = (val >> (i * 8)) & GENMASK(7, 8 - VGIC_PRI_BITS);\n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > @@ -390,6 +396,7 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,\n> > {\n> > \tu32 intid = VGIC_ADDR_TO_INTID(addr, 2);\n> > \tint i;\n> > +\tunsigned long flags;\n> > \n> > \tfor (i = 0; i < len * 4; i++) {\n> > \t\tstruct vgic_irq *irq;\n> > @@ -404,14 +411,14 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,\n> > \t\t\tcontinue;\n> > \n> > \t\tirq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);\n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \t\tif (test_bit(i * 2 + 1, &val))\n> > \t\t\tirq->config = VGIC_CONFIG_EDGE;\n> > \t\telse\n> > \t\t\tirq->config = VGIC_CONFIG_LEVEL;\n> > \n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > }\n> > @@ -443,6 +450,7 @@ void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,\n> > {\n> > \tint i;\n> > \tint nr_irqs = vcpu->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;\n> > +\tunsigned long flags;\n> > \n> > \tfor (i = 0; i < 32; i++) {\n> > \t\tstruct vgic_irq *irq;\n> > @@ -459,12 +467,12 @@ void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,\n> > \t\t * restore irq config before line level.\n> > \t\t */\n> > \t\tnew_level = !!(val & (1U << i));\n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \t\tirq->line_level = new_level;\n> > \t\tif (new_level)\n> > -\t\t\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> > +\t\t\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> > \t\telse\n> > -\t\t\tspin_unlock(&irq->irq_lock);\n> > +\t\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > diff --git a/virt/kvm/arm/vgic/vgic-v2.c b/virt/kvm/arm/vgic/vgic-v2.c\n> > index e4187e5..8089710 100644\n> > --- a/virt/kvm/arm/vgic/vgic-v2.c\n> > +++ b/virt/kvm/arm/vgic/vgic-v2.c\n> > @@ -62,6 +62,7 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu)\n> > \tstruct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;\n> > \tstruct vgic_v2_cpu_if *cpuif = &vgic_cpu->vgic_v2;\n> > \tint lr;\n> > +\tunsigned long flags;\n> > \n> > \tcpuif->vgic_hcr &= ~GICH_HCR_UIE;\n> > \n> > @@ -77,7 +78,7 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu)\n> > \n> > \t\tirq = vgic_get_irq(vcpu->kvm, vcpu, intid);\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \t\t/* Always preserve the active bit */\n> > \t\tirq->active = !!(val & GICH_LR_ACTIVE_BIT);\n> > @@ -104,7 +105,7 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu)\n> > \t\t\t\tirq->pending_latch = false;\n> > \t\t}\n> > \n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > \n> > diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c\n> > index 96ea597..863351c 100644\n> > --- a/virt/kvm/arm/vgic/vgic-v3.c\n> > +++ b/virt/kvm/arm/vgic/vgic-v3.c\n> > @@ -44,6 +44,7 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)\n> > \tstruct vgic_v3_cpu_if *cpuif = &vgic_cpu->vgic_v3;\n> > \tu32 model = vcpu->kvm->arch.vgic.vgic_model;\n> > \tint lr;\n> > +\tunsigned long flags;\n> > \n> > \tcpuif->vgic_hcr &= ~ICH_HCR_UIE;\n> > \n> > @@ -66,7 +67,7 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)\n> > \t\tif (!irq)\t/* An LPI could have been unmapped. */\n> > \t\t\tcontinue;\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \t\t/* Always preserve the active bit */\n> > \t\tirq->active = !!(val & ICH_LR_ACTIVE_BIT);\n> > @@ -94,7 +95,7 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)\n> > \t\t\t\tirq->pending_latch = false;\n> > \t\t}\n> > \n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \t\tvgic_put_irq(vcpu->kvm, irq);\n> > \t}\n> > \n> > @@ -278,6 +279,7 @@ int vgic_v3_lpi_sync_pending_status(struct kvm *kvm, struct vgic_irq *irq)\n> > \tbool status;\n> > \tu8 val;\n> > \tint ret;\n> > +\tunsigned long flags;\n> > \n> > retry:\n> > \tvcpu = irq->target_vcpu;\n> > @@ -296,13 +298,13 @@ int vgic_v3_lpi_sync_pending_status(struct kvm *kvm, struct vgic_irq *irq)\n> > \n> > \tstatus = val & (1 << bit_nr);\n> > \n> > -\tspin_lock(&irq->irq_lock);\n> > +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \tif (irq->target_vcpu != vcpu) {\n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \t\tgoto retry;\n> > \t}\n> > \tirq->pending_latch = status;\n> > -\tvgic_queue_irq_unlock(vcpu->kvm, irq);\n> > +\tvgic_queue_irq_unlock(vcpu->kvm, irq, flags);\n> > \n> > \tif (status) {\n> > \t\t/* clear consumed data */\n> > diff --git a/virt/kvm/arm/vgic/vgic.c b/virt/kvm/arm/vgic/vgic.c\n> > index e1f7dbc..b1bd238 100644\n> > --- a/virt/kvm/arm/vgic/vgic.c\n> > +++ b/virt/kvm/arm/vgic/vgic.c\n> > @@ -53,6 +53,10 @@ struct vgic_global kvm_vgic_global_state __ro_after_init = {\n> > * vcpuX->vcpu_id < vcpuY->vcpu_id:\n> > * spin_lock(vcpuX->arch.vgic_cpu.ap_list_lock);\n> > * spin_lock(vcpuY->arch.vgic_cpu.ap_list_lock);\n> > + *\n> > + * Since the VGIC must support injecting virtual interrupts from ISRs, we have\n> > + * to use the spin_lock_irqsave/spin_unlock_irqrestore versions of outer\n> > + * spinlocks for any lock that may be taken while injecting an interrupt.\n> > */\n> > \n> > /*\n> > @@ -261,7 +265,8 @@ static bool vgic_validate_injection(struct vgic_irq *irq, bool level, void *owne\n> > * Needs to be entered with the IRQ lock already held, but will return\n> > * with all locks dropped.\n> > */\n> > -bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq)\n> > +bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq,\n> > +\t\t\t unsigned long flags)\n> > {\n> > \tstruct kvm_vcpu *vcpu;\n> > \n> > @@ -279,7 +284,7 @@ bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq)\n> > \t\t * not need to be inserted into an ap_list and there is also\n> > \t\t * no more work for us to do.\n> > \t\t */\n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \n> > \t\t/*\n> > \t\t * We have to kick the VCPU here, because we could be\n> > @@ -301,11 +306,11 @@ bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq)\n> > \t * We must unlock the irq lock to take the ap_list_lock where\n> > \t * we are going to insert this new pending interrupt.\n> > \t */\n> > -\tspin_unlock(&irq->irq_lock);\n> > +\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \n> > \t/* someone can do stuff here, which we re-check below */\n> > \n> > -\tspin_lock(&vcpu->arch.vgic_cpu.ap_list_lock);\n> > +\tspin_lock_irqsave(&vcpu->arch.vgic_cpu.ap_list_lock, flags);\n> > \tspin_lock(&irq->irq_lock);\n> > \n> > \t/*\n> > @@ -322,9 +327,9 @@ bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq)\n> > \n> > \tif (unlikely(irq->vcpu || vcpu != vgic_target_oracle(irq))) {\n> > \t\tspin_unlock(&irq->irq_lock);\n> > -\t\tspin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock);\n> > +\t\tspin_unlock_irqrestore(&vcpu->arch.vgic_cpu.ap_list_lock, flags);\n> > \n> > -\t\tspin_lock(&irq->irq_lock);\n> > +\t\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \t\tgoto retry;\n> > \t}\n> > \n> > @@ -337,7 +342,7 @@ bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq)\n> > \tirq->vcpu = vcpu;\n> > \n> > \tspin_unlock(&irq->irq_lock);\n> > -\tspin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock);\n> > +\tspin_unlock_irqrestore(&vcpu->arch.vgic_cpu.ap_list_lock, flags);\n> > \n> > \tkvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);\n> > \tkvm_vcpu_kick(vcpu);\n> > @@ -367,6 +372,7 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,\n> > {\n> > \tstruct kvm_vcpu *vcpu;\n> > \tstruct vgic_irq *irq;\n> > +\tunsigned long flags;\n> > \tint ret;\n> > \n> > \ttrace_vgic_update_irq_pending(cpuid, intid, level);\n> > @@ -383,11 +389,11 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,\n> > \tif (!irq)\n> > \t\treturn -EINVAL;\n> > \n> > -\tspin_lock(&irq->irq_lock);\n> > +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \tif (!vgic_validate_injection(irq, level, owner)) {\n> > \t\t/* Nothing to see here, move along... */\n> > -\t\tspin_unlock(&irq->irq_lock);\n> > +\t\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \t\tvgic_put_irq(kvm, irq);\n> > \t\treturn 0;\n> > \t}\n> > @@ -397,7 +403,7 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,\n> > \telse\n> > \t\tirq->pending_latch = true;\n> > \n> > -\tvgic_queue_irq_unlock(kvm, irq);\n> > +\tvgic_queue_irq_unlock(kvm, irq, flags);\n> > \tvgic_put_irq(kvm, irq);\n> > \n> > \treturn 0;\n> > @@ -406,15 +412,16 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int intid,\n> > int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, u32 virt_irq, u32 phys_irq)\n> > {\n> > \tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);\n> > +\tunsigned long flags;\n> > \n> > \tBUG_ON(!irq);\n> > \n> > -\tspin_lock(&irq->irq_lock);\n> > +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \tirq->hw = true;\n> > \tirq->hwintid = phys_irq;\n> > \n> > -\tspin_unlock(&irq->irq_lock);\n> > +\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \tvgic_put_irq(vcpu->kvm, irq);\n> > \n> > \treturn 0;\n> > @@ -423,6 +430,7 @@ int kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu, u32 virt_irq, u32 phys_irq)\n> > int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int virt_irq)\n> > {\n> > \tstruct vgic_irq *irq;\n> > +\tunsigned long flags;\n> > \n> > \tif (!vgic_initialized(vcpu->kvm))\n> > \t\treturn -EAGAIN;\n> > @@ -430,12 +438,12 @@ int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int virt_irq)\n> > \tirq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);\n> > \tBUG_ON(!irq);\n> > \n> > -\tspin_lock(&irq->irq_lock);\n> > +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> > \n> > \tirq->hw = false;\n> > \tirq->hwintid = 0;\n> > \n> > -\tspin_unlock(&irq->irq_lock);\n> > +\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \tvgic_put_irq(vcpu->kvm, irq);\n> > \n> > \treturn 0;\n> > @@ -486,9 +494,10 @@ static void vgic_prune_ap_list(struct kvm_vcpu *vcpu)\n> > {\n> > \tstruct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;\n> > \tstruct vgic_irq *irq, *tmp;\n> > +\tunsigned long flags;\n> > \n> > retry:\n> > -\tspin_lock(&vgic_cpu->ap_list_lock);\n> > +\tspin_lock_irqsave(&vgic_cpu->ap_list_lock, flags);\n> > \n> > \tlist_for_each_entry_safe(irq, tmp, &vgic_cpu->ap_list_head, ap_list) {\n> > \t\tstruct kvm_vcpu *target_vcpu, *vcpuA, *vcpuB;\n> > @@ -528,7 +537,7 @@ static void vgic_prune_ap_list(struct kvm_vcpu *vcpu)\n> > \t\t/* This interrupt looks like it has to be migrated. */\n> > \n> > \t\tspin_unlock(&irq->irq_lock);\n> > -\t\tspin_unlock(&vgic_cpu->ap_list_lock);\n> > +\t\tspin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags);\n> > \n> > \t\t/*\n> > \t\t * Ensure locking order by always locking the smallest\n> > @@ -542,7 +551,7 @@ static void vgic_prune_ap_list(struct kvm_vcpu *vcpu)\n> > \t\t\tvcpuB = vcpu;\n> > \t\t}\n> > \n> > -\t\tspin_lock(&vcpuA->arch.vgic_cpu.ap_list_lock);\n> > +\t\tspin_lock_irqsave(&vcpuA->arch.vgic_cpu.ap_list_lock, flags);\n> > \t\tspin_lock_nested(&vcpuB->arch.vgic_cpu.ap_list_lock,\n> > \t\t\t\t SINGLE_DEPTH_NESTING);\n> > \t\tspin_lock(&irq->irq_lock);\n> > @@ -566,11 +575,11 @@ static void vgic_prune_ap_list(struct kvm_vcpu *vcpu)\n> > \n> > \t\tspin_unlock(&irq->irq_lock);\n> > \t\tspin_unlock(&vcpuB->arch.vgic_cpu.ap_list_lock);\n> > -\t\tspin_unlock(&vcpuA->arch.vgic_cpu.ap_list_lock);\n> > +\t\tspin_unlock_irqrestore(&vcpuA->arch.vgic_cpu.ap_list_lock, flags);\n> > \t\tgoto retry;\n> > \t}\n> > \n> > -\tspin_unlock(&vgic_cpu->ap_list_lock);\n> > +\tspin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags);\n> > }\n> > \n> > static inline void vgic_fold_lr_state(struct kvm_vcpu *vcpu)\n> > @@ -703,6 +712,8 @@ void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu)\n> > \tif (list_empty(&vcpu->arch.vgic_cpu.ap_list_head))\n> > \t\treturn;\n> > \n> > +\tDEBUG_SPINLOCK_BUG_ON(!irqs_disabled());\n> > +\n> > \tspin_lock(&vcpu->arch.vgic_cpu.ap_list_lock);\n> > \tvgic_flush_lr_state(vcpu);\n> > \tspin_unlock(&vcpu->arch.vgic_cpu.ap_list_lock);\n> > @@ -735,11 +746,12 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)\n> > \tstruct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;\n> > \tstruct vgic_irq *irq;\n> > \tbool pending = false;\n> > +\tunsigned long flags;\n> > \n> > \tif (!vcpu->kvm->arch.vgic.enabled)\n> > \t\treturn false;\n> > \n> > -\tspin_lock(&vgic_cpu->ap_list_lock);\n> > +\tspin_lock_irqsave(&vgic_cpu->ap_list_lock, flags);\n> > \n> > \tlist_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) {\n> > \t\tspin_lock(&irq->irq_lock);\n> > @@ -750,7 +762,7 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)\n> > \t\t\tbreak;\n> > \t}\n> > \n> > -\tspin_unlock(&vgic_cpu->ap_list_lock);\n> > +\tspin_unlock_irqrestore(&vgic_cpu->ap_list_lock, flags);\n> > \n> > \treturn pending;\n> > }\n> > @@ -776,13 +788,15 @@ bool kvm_vgic_map_is_active(struct kvm_vcpu *vcpu, unsigned int virt_irq)\n> > {\n> > \tstruct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, virt_irq);\n> > \tbool map_is_active;\n> > +\tunsigned long flags;\n> > \n> > \tif (!vgic_initialized(vcpu->kvm))\n> > \t\treturn false;\n> > +\tDEBUG_SPINLOCK_BUG_ON(!irqs_disabled());\n> > \n> > -\tspin_lock(&irq->irq_lock);\n> > +\tspin_lock_irqsave(&irq->irq_lock, flags);\n> \n> I'm a bit puzzled by this sequence: Either interrupts are disabled and\n> we don't need the irqsave version, or they aren't and the BUG_ON will\n> fire. kvm_vgic_map_is_active is called (indirectly) from\n> kvm_timer_flush_hwstate. And at this stage of the patches, we definitely\n> call this function with interrupts enabled.\n> \n> Is it just a patch splitting snafu? Or something more serious? Same goes\n> for the DEBUG_SPINLOCK_BUG_ON in kvm_vgic_flush_hwstate.\n\nIt's a leftover thing from before I realized that this also needs to be\ncalled from kvm_timer_vcpu_load_vgic, which has interrupts enabled, and\nso I changed the simple spin_lock/spin_unlock to the irqsave/irqrestore\nversions, but apparently forgot to take out the assert. (And apparently\ndidn't run this with spinlock debugging enabled).\n\nThanks for spotting it.\n-Christoffer\n\n> > \tmap_is_active = irq->hw && irq->active;\n> > -\tspin_unlock(&irq->irq_lock);\n> > +\tspin_unlock_irqrestore(&irq->irq_lock, flags);\n> > \tvgic_put_irq(vcpu->kvm, irq);\n> > \n> > \treturn map_is_active;\n> > diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h\n> > index bf9ceab..4f8aecb 100644\n> > --- a/virt/kvm/arm/vgic/vgic.h\n> > +++ b/virt/kvm/arm/vgic/vgic.h\n> > @@ -140,7 +140,8 @@ vgic_get_mmio_region(struct kvm_vcpu *vcpu, struct vgic_io_device *iodev,\n> > struct vgic_irq *vgic_get_irq(struct kvm *kvm, struct kvm_vcpu *vcpu,\n> > \t\t\t u32 intid);\n> > void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq);\n> > -bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq);\n> > +bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq,\n> > +\t\t\t unsigned long flags);\n> > void vgic_kick_vcpus(struct kvm *kvm);\n> > \n> > int vgic_check_ioaddr(struct kvm *kvm, phys_addr_t *ioaddr,\n> > \n> \n> Otherwise looks good to me.\n> \n> \tM.\n> -- \n> Jazz is not dead. It just smells funny...","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"tTSaEAjV\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"aoPaTqsw\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yH9V045Byz9sBW\n\tfor ;\n\tWed, 18 Oct 2017 22:55:04 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4mwA-0004pa-1o; Wed, 18 Oct 2017 11:55:02 +0000","from mail-wm0-x241.google.com ([2a00:1450:400c:c09::241])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4mw4-0004in-4n for linux-arm-kernel@lists.infradead.org;\n\tWed, 18 Oct 2017 11:54:59 +0000","by mail-wm0-x241.google.com with SMTP id u138so9779216wmu.4\n\tfor ;\n\tWed, 18 Oct 2017 04:54:36 -0700 (PDT)","from localhost (xd93dd96b.cust.hiper.dk. [217.61.217.107])\n\tby smtp.gmail.com with ESMTPSA id\n\t88sm8352109edq.49.2017.10.18.04.54.32\n\t(version=TLS1_2 cipher=AES128-SHA bits=128/128);\n\tWed, 18 Oct 2017 04:54:33 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:\n\tMessage-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=x4Kfsi1qaZt0j3l8u9syGo2/zEf1lYINpGlV1em177M=;\n\tb=tTSaEAjVeSPhH0\n\t1WLjKlphWrQyjBm/TRMX3n0qkjcWDq0ih0RRj+NEToRzdydBrpEsAE5MFNjEBHCX2fGrpWAKFR706\n\txQxKZaTDSOdeIf8RRakmCsYOzsBXmYOLpVZGp8bcdNxGxz9SMqv4que1THUSwUq3XV9fEsdeck0PM\n\tdSuoWOYGpz/Mi++/LvVqwo+rR+EE1ZHDa1JGFF0Yn71imoeUdSE8qV+LR1lRY/VjnMAqVRAYcAd+A\n\tyPDQDMhfxDtbzKD2F4F5aigHPuE7HQDqkfYwBjdwRxD3/QJZeAM+ZzRz6kt3QOv7sz/51LqGlhB7g\n\tNtGVxj/JumJgxdxzxAXQ==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=eI4BF2wPQG5F7eA4Se3r2AGMCffA64Yvq0rqYBkMUDY=;\n\tb=aoPaTqswlCi5bwlLI486aCcjAna1mHxHrsz4SOqgLM2XM2mGWn+RxSVqBzlR1CcBxQ\n\tFjzyiFSTweTyfK52B/F2EOb0QYL+GPY7wdOvLSJcqy/Ae0H5qkCLUxcCu5DydY0zVX8r\n\tkmOR+duGB2+s68+Gyr4qXHZy2cXQTZ4csHa5U="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=eI4BF2wPQG5F7eA4Se3r2AGMCffA64Yvq0rqYBkMUDY=;\n\tb=H11lZ5SaIlVKIhMhRNeE9ZoWdTRdhRCr656xmQ66NFixWv1DcXcr0jgoXwqSxgvKtv\n\tqCRRJWuJ5K4l83BxLirJLzwABw1d8/4AkB8yWrEnz8UOgwAklZ8V0Woew8E/hChfC+Re\n\tcIVhDgS1Yv/nz90GjABv7ZLjqHOowK5jo2sMwY7yiTqmDBECemWC/ljg9DU7kO5Ze8aE\n\tFs7ysbpq8imqXDmD3TTrQNA3Hy3dcJBySUpjzVkVbPVrf5/49bvvDi+lMp7U4Td+notw\n\t2/Bsxi86TqRbe9Pk7bdIBYE5OXbXvKNy8nxtVNzuN2GjEx8mp+62oBS0lSaGvn4TTywf\n\t5j0w==","X-Gm-Message-State":"AMCzsaUBcbo/csV8a33QCEZb0o6Gj0bcSvP1UNEKzI9prDXL1gDudlj3\n\tXvZW1GrlUl9KjRXBMOkQSjbJ4Q==","X-Google-Smtp-Source":"AOwi7QCeeCxjHqrxZhYBg7lGw9rYVAP4GDrH57wGNdC2A6iQMZzD0rekfvN4TaGVaMLL3idUKdkusw==","X-Received":"by 10.80.206.84 with SMTP id k20mr21112960edj.145.1508327674079; \n\tWed, 18 Oct 2017 04:54:34 -0700 (PDT)","Date":"Wed, 18 Oct 2017 13:54:36 +0200","From":"Christoffer Dall ","To":"Marc Zyngier ","Subject":"Re: [PATCH v3 05/20] KVM: arm/arm64: Support calling\n\tvgic_update_irq_pending from irq context","Message-ID":"<20171018115436.GC8900@cbox>","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-6-cdall@linaro.org>\n\t<0c3d0855-83d6-645a-23d3-ae143b8a05a9@arm.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<0c3d0855-83d6-645a-23d3-ae143b8a05a9@arm.com>","User-Agent":"Mutt/1.5.21 (2010-09-15)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171018_045456_649481_FCB31D6A ","X-CRM114-Status":"GOOD ( 23.49 )","X-Spam-Score":"-2.0 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.0 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1789718,"web_url":"http://patchwork.ozlabs.org/comment/1789718/","msgid":"<878tg8jvwl.fsf@on-the-bus.cambridge.arm.com>","list_archive_url":null,"date":"2017-10-18T15:52:26","subject":"Re: [PATCH v3 03/20] arm64: Use the physical counter when available\n\tfor read_cycles","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On Wed, Oct 18 2017 at 1:34:05 pm BST, Christoffer Dall wrote:\n> On Mon, Oct 09, 2017 at 05:21:24PM +0100, Marc Zyngier wrote:\n>> On 23/09/17 01:41, Christoffer Dall wrote:\n>> > Currently get_cycles() is hardwired to arch_counter_get_cntvct() on\n>> > arm64, but as we move to using the physical timer for the in-kernel\n>> > time-keeping, we need to make that more flexible.\n>> > \n>> > First, we need to make sure the physical counter can be read on equal\n>> > terms to the virtual counter, which includes adding physical counter\n>> > read functions for timers that require errata.\n>> > \n>> > Second, we need to make a choice between reading the physical vs virtual\n>> > counter, depending on which timer is used for time keeping in the kernel\n>> > otherwise. We can do this using a static key to avoid a performance\n>> > penalty during runtime when reading the counter.\n>> > \n>> > Cc: Catalin Marinas \n>> > Cc: Will Deacon \n>> > Cc: Mark Rutland \n>> > Cc: Marc Zyngier \n>> > Signed-off-by: Christoffer Dall \n\n[...]\n\n>> In my reply to patch #2, I had the following hunk:\n>> \n>> @@ -310,7 +329,7 @@ static void erratum_set_next_event_tval_generic(const int access, unsigned long\n>> \t\t\t\t\t\tstruct clock_event_device *clk)\n>> {\n>> \tunsigned long ctrl;\n>> -\tu64 cval = evt + arch_counter_get_cntvct();\n>> +\tu64 cval = evt + arch_timer_read_counter();\n>> \n>> \tctrl = arch_timer_reg_read(access, ARCH_TIMER_REG_CTRL, clk);\n>> \tctrl |= ARCH_TIMER_CTRL_ENABLE;\n>> \n>> Once we start using a different timer, this could well have an effect...\n>> \n>\n> Right, but wouldn't the following be a more correct way to go about it then:\n>\n> diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c\n> index 9a7b359..07f19db 100644\n> --- a/drivers/clocksource/arm_arch_timer.c\n> +++ b/drivers/clocksource/arm_arch_timer.c\n> @@ -329,16 +329,19 @@ static void erratum_set_next_event_tval_generic(const int access, unsigned long\n> \t\t\t\t\t\tstruct clock_event_device *clk)\n> {\n> \tunsigned long ctrl;\n> -\tu64 cval = evt + arch_timer_read_counter();\n> +\tu64 cval;\n> \n> \tctrl = arch_timer_reg_read(access, ARCH_TIMER_REG_CTRL, clk);\n> \tctrl |= ARCH_TIMER_CTRL_ENABLE;\n> \tctrl &= ~ARCH_TIMER_CTRL_IT_MASK;\n> \n> -\tif (access == ARCH_TIMER_PHYS_ACCESS)\n> +\tif (access == ARCH_TIMER_PHYS_ACCESS) {\n> +\t\tcval = evt + arch_counter_get_cntpct();\n> \t\twrite_sysreg(cval, cntp_cval_el0);\n> -\telse\n> +\t} else {\n> +\t\tcval = evt + arch_counter_get_cntvct();\n> \t\twrite_sysreg(cval, cntv_cval_el0);\n> +\t}\n> \n> \tarch_timer_reg_write(access, ARCH_TIMER_REG_CTRL, ctrl, clk);\n> }\n\nYup, that's much better.\n\nThanks,\n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"E0o8cJIm\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yHGmX0MTXz9t4P\n\tfor ;\n\tThu, 19 Oct 2017 02:53:00 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4qeM-0004Fg-6z; Wed, 18 Oct 2017 15:52:54 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4qeI-0003mq-FN for linux-arm-kernel@lists.infradead.org;\n\tWed, 18 Oct 2017 15:52:52 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B7E16F;\n\tWed, 18 Oct 2017 08:52:29 -0700 (PDT)","from on-the-bus (on-the-bus.cambridge.arm.com [10.3.33.38])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t5F9AE3F483; Wed, 18 Oct 2017 08:52:28 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References\n\t:In-Reply-To:Subject:To:From:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=8nhlO+QNj+7jWig7BIQX23YVy8zvB12TyhaF0Vaj+vY=;\n\tb=E0o8cJIm0I2Gp3\n\tJGsfEcgNX+vvNi4Q5EVoJX37LTY5ciD3K/pr7l7TupRSptWEYn6foLt39jh3nvM2C8E6cd44AtVaC\n\tuvGfLM4WJR55gqVd/Fc4xTwdbXoI+6Ce/4qNP2xPuJoMtUeh/B68HgZRnUjxa3dgM+EYuejUFqD7y\n\tpu1R4wnozfYCP3ELYZQGwzZSYZkQfhKz61x4ZBv+r0i5OIB59u1+72ZAHqVw7zNmI1rtrHrI0U1ZX\n\tUMnJ5MFu4aJW0IcNCx7o2GbC54Z1nJLHUmqHw29PaD6l0qBVlG7oDkDeySsz0twe7aRU2uK450fc4\n\tmZz/0Ex7KHsojf4s2TNg==;","From":"Marc Zyngier ","To":"Christoffer Dall ","Subject":"Re: [PATCH v3 03/20] arm64: Use the physical counter when available\n\tfor read_cycles","In-Reply-To":"<20171018113405.GA8900@cbox> (Christoffer Dall's message of\n\t\"Wed, 18 Oct 2017 13:34:05 +0200\")","Organization":"ARM Ltd","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-4-cdall@linaro.org>\n\t<9b06425f-7c2c-d44a-cd6c-aeaa4b76849c@arm.com>\n\t<20171018113405.GA8900@cbox>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)","Date":"Wed, 18 Oct 2017 16:52:26 +0100","Message-ID":"<878tg8jvwl.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171018_085250_609328_CD677E5B ","X-CRM114-Status":"GOOD ( 17.70 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Mark Rutland , kvm@vger.kernel.org,\n\tCatalin Marinas ,\n\tWill Deacon , \n\tkvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1789729,"web_url":"http://patchwork.ozlabs.org/comment/1789729/","msgid":"<874lqwjvdv.fsf@on-the-bus.cambridge.arm.com>","list_archive_url":null,"date":"2017-10-18T16:03:40","subject":"Re: [PATCH v3 06/20] KVM: arm/arm64: Check that system supports\n\tsplit eoi/deactivate","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On Wed, Oct 18 2017 at 3:41:45 pm BST, Christoffer Dall wrote:\n> On Mon, Oct 09, 2017 at 05:47:18PM +0100, Marc Zyngier wrote:\n>> On 23/09/17 01:41, Christoffer Dall wrote:\n>> > Some systems without proper firmware and/or hardware description data\n>> > don't support the split EOI and deactivate operation.\n>> > \n>> > On such systems, we cannot leave the physical interrupt active after the\n>> > timer handler on the host has run, so we cannot support KVM with an\n>> > in-kernel GIC with the timer changes we are about to introduce.\n>> > \n>> > This patch makes sure that trying to initialize the KVM GIC code will\n>> > fail on such systems.\n>> > \n>> > Cc: Marc Zyngier \n>> > Signed-off-by: Christoffer Dall \n>> > ---\n>> > drivers/irqchip/irq-gic.c | 3 ++-\n>> > 1 file changed, 2 insertions(+), 1 deletion(-)\n>> > \n>> > diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c\n>> > index f641e8e..ab12bf4 100644\n>> > --- a/drivers/irqchip/irq-gic.c\n>> > +++ b/drivers/irqchip/irq-gic.c\n>> > @@ -1420,7 +1420,8 @@ static void __init gic_of_setup_kvm_info(struct device_node *node)\n>> > \tif (ret)\n>> > \t\treturn;\n>> > \n>> > -\tgic_set_kvm_info(&gic_v2_kvm_info);\n>> > +\tif (static_key_true(&supports_deactivate))\n>> > +\t\tgic_set_kvm_info(&gic_v2_kvm_info);\n>> > }\n>> > \n>> > int __init\n>> > \n>> \n>> Should we add the same level of checking on the ACPI path, just for the\n>> sake symmetry?\n>\n> Yes, we should, if anyone is crazy enough to use ACPI :)\n\nSadly, the madness is becoming commonplace.\n\n>> \n>> Also, do we need to add the same thing for GICv3?\n>> \n>\n> Why would split EOI/deactivate not be available on GICv3, actually? It\n> looks like this is not supported unless you have EL2, but I can't seem\n> to find anything in the spec for this, and KVM should support\n> EOI/deactivate for GICv3 guests I think. Am I missing something?\n\nNo, you're not. This is just a Linux choice (or rather mine) not to use\nEOImode=1 in guests (or anything booted at EL1), as we don't really need\nthe two-stage deactivate in that situation (it is pure overhead).\n\nI'm just worried of potentially broken HW, and would like to make sure\nthat when we force EOImode=0 on these systems, we truly tell KVM about\nit.\n\n> Assuming I'm wrong about GICv3, which I probably am, how does this look\n> (on top of the posted patch):\n>\n> diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c\n> index 519149e..aed524c 100644\n> --- a/drivers/irqchip/irq-gic-v3.c\n> +++ b/drivers/irqchip/irq-gic-v3.c\n> @@ -1228,7 +1228,9 @@ static int __init gic_of_init(struct device_node *node, struct device_node *pare\n> \t\tgoto out_unmap_rdist;\n> \n> \tgic_populate_ppi_partitions(node);\n> -\tgic_of_setup_kvm_info(node);\n> +\n> +\tif (static_key_true(&supports_deactivate))\n> +\t\tgic_of_setup_kvm_info(node);\n> \treturn 0;\n> \n> out_unmap_rdist:\n> @@ -1517,7 +1519,9 @@ gic_acpi_init(struct acpi_subtable_header *header, const unsigned long end)\n> \t\tgoto out_fwhandle_free;\n> \n> \tacpi_set_irq_model(ACPI_IRQ_MODEL_GIC, domain_handle);\n> -\tgic_acpi_setup_kvm_info();\n> +\n> +\tif (static_key_true(&supports_deactivate))\n> +\t\tgic_acpi_setup_kvm_info();\n> \n> \treturn 0;\n> \n> diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c\n> index ab12bf4..121af5c 100644\n> --- a/drivers/irqchip/irq-gic.c\n> +++ b/drivers/irqchip/irq-gic.c\n> @@ -1653,7 +1653,8 @@ static int __init gic_v2_acpi_init(struct acpi_subtable_header *header,\n> \tif (IS_ENABLED(CONFIG_ARM_GIC_V2M))\n> \t\tgicv2m_init(NULL, gic_data[0].domain);\n> \n> -\tgic_acpi_setup_kvm_info();\n> +\tif (static_key_true(&supports_deactivate))\n> +\t\tgic_acpi_setup_kvm_info();\n> \n> \treturn 0;\n> }\n\nYup, looks good to me!\n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"LYsueGJj\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yHH1k0mJtz9t4P\n\tfor ;\n\tThu, 19 Oct 2017 03:04:26 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4qpQ-0006ou-6m; Wed, 18 Oct 2017 16:04:20 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4qp9-0006Ki-JY for linux-arm-kernel@lists.infradead.org;\n\tWed, 18 Oct 2017 16:04:12 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 45D95F;\n\tWed, 18 Oct 2017 09:03:43 -0700 (PDT)","from on-the-bus (unknown [10.3.33.38])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t128833F25D; Wed, 18 Oct 2017 09:03:41 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References\n\t:In-Reply-To:Subject:To:From:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=604U7LVRTheNq2TgRPqQLLJdSEfTt7kgj4bKAPiYL74=;\n\tb=LYsueGJjT6XfgU\n\t2zCve4nxQ8a/cYPpKER++Fo3gjxzNtCKLN/Gt28818VKL6xwP1tSOgxwnsxenkIDKTIYZ/T7As1wp\n\tYsnBIodbNxwpSOvL0NsWGMgfRoDKMiKWCGuu6cFC/uAslkR+B8fWgSHMqOprZF0RhArObIu3AAAWu\n\tU3cq4/QeJsjGhOPymJx6SvVWabFECOd5ABBrBjv/ZhJJ4pYm1Yr2vW5dyij79zHxmdhgXVVd16DU5\n\tcaQoNSMW/xPJMXR2cNWqP7wMtCx/f7wwOZc5st4YmnAhzDQoGTofMJZN3224ifquuWI7TteULi1mu\n\tEOZ7bB8fbsBiB7vUtBXA==;","From":"Marc Zyngier ","To":"Christoffer Dall ","Subject":"Re: [PATCH v3 06/20] KVM: arm/arm64: Check that system supports\n\tsplit eoi/deactivate","In-Reply-To":"<20171018134145.GH8900@cbox> (Christoffer Dall's message of\n\t\"Wed, 18 Oct 2017 15:41:45 +0200\")","Organization":"ARM Ltd","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-7-cdall@linaro.org>\n\t\n\t<20171018134145.GH8900@cbox>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)","Date":"Wed, 18 Oct 2017 17:03:40 +0100","Message-ID":"<874lqwjvdv.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171018_090403_848924_4BEBCD88 ","X-CRM114-Status":"GOOD ( 22.40 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1789797,"web_url":"http://patchwork.ozlabs.org/comment/1789797/","msgid":"<87o9p4ieik.fsf@on-the-bus.cambridge.arm.com>","list_archive_url":null,"date":"2017-10-18T16:53:23","subject":"Re: [PATCH v3 07/20] KVM: arm/arm64: Make timer_arm and timer_disarm\n\thelpers more generic","submitter":{"id":7353,"url":"http://patchwork.ozlabs.org/api/people/7353/","name":"Marc Zyngier","email":"marc.zyngier@arm.com"},"content":"On Wed, Oct 18 2017 at 6:47:50 pm BST, Christoffer Dall wrote:\n> On Mon, Oct 09, 2017 at 06:05:04PM +0100, Marc Zyngier wrote:\n>> On 23/09/17 01:41, Christoffer Dall wrote:\n>> > We are about to add an additional soft timer to the arch timer state for\n>> > a VCPU and would like to be able to reuse the functions to program and\n>> > cancel a timer, so we make them slightly more generic and rename to make\n>> > it more clear that these functions work on soft timers and not the\n>> > hardware resource that this code is managing.\n>> > \n>> > Signed-off-by: Christoffer Dall \n>> > ---\n>> > virt/kvm/arm/arch_timer.c | 33 ++++++++++++++++-----------------\n>> > 1 file changed, 16 insertions(+), 17 deletions(-)\n>> > \n>> > diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n>> > index 8e89d63..871d8ae 100644\n>> > --- a/virt/kvm/arm/arch_timer.c\n>> > +++ b/virt/kvm/arm/arch_timer.c\n>> > @@ -56,26 +56,22 @@ u64 kvm_phys_timer_read(void)\n>> > \treturn timecounter->cc->read(timecounter->cc);\n>> > }\n>> > \n>> > -static bool timer_is_armed(struct arch_timer_cpu *timer)\n>> > +static bool soft_timer_is_armed(struct arch_timer_cpu *timer)\n>> > {\n>> > \treturn timer->armed;\n>> > }\n>> > \n>> > -/* timer_arm: as in \"arm the timer\", not as in ARM the company */\n>> > -static void timer_arm(struct arch_timer_cpu *timer, u64 ns)\n>> > +static void soft_timer_start(struct hrtimer *hrt, u64 ns)\n>> > {\n>> > -\ttimer->armed = true;\n>> > -\thrtimer_start(&timer->timer, ktime_add_ns(ktime_get(), ns),\n>> > +\thrtimer_start(hrt, ktime_add_ns(ktime_get(), ns),\n>> > \t\t HRTIMER_MODE_ABS);\n>> > }\n>> > \n>> > -static void timer_disarm(struct arch_timer_cpu *timer)\n>> > +static void soft_timer_cancel(struct hrtimer *hrt, struct work_struct *work)\n>> > {\n>> > -\tif (timer_is_armed(timer)) {\n>> > -\t\thrtimer_cancel(&timer->timer);\n>> > -\t\tcancel_work_sync(&timer->expired);\n>> > -\t\ttimer->armed = false;\n>> > -\t}\n>> > +\thrtimer_cancel(hrt);\n>> > +\tif (work)\n>> \n>> When can this happen? Something in a following patch?\n>> \n>\n> Yeah, sorry about that. I will point this out in the commit message.\n>\n>> > +\t\tcancel_work_sync(work);\n>> > }\n>> > \n>> > static irqreturn_t kvm_arch_timer_handler(int irq, void *dev_id)\n>> > @@ -271,7 +267,7 @@ static void kvm_timer_emulate(struct kvm_vcpu *vcpu,\n>> > \t\treturn;\n>> > \n>> > \t/* The timer has not yet expired, schedule a background timer */\n>> > -\ttimer_arm(timer, kvm_timer_compute_delta(timer_ctx));\n>> > +\tsoft_timer_start(&timer->timer, kvm_timer_compute_delta(timer_ctx));\n>> > }\n>> > \n>> > /*\n>> > @@ -285,7 +281,7 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)\n>> > \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n>> > \tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n>> > \n>> > -\tBUG_ON(timer_is_armed(timer));\n>> > +\tBUG_ON(soft_timer_is_armed(timer));\n>> > \n>> > \t/*\n>> > \t * No need to schedule a background timer if any guest timer has\n>> > @@ -306,13 +302,16 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)\n>> > \t * The guest timers have not yet expired, schedule a background timer.\n>> > \t * Set the earliest expiration time among the guest timers.\n>> > \t */\n>> > -\ttimer_arm(timer, kvm_timer_earliest_exp(vcpu));\n>> > +\ttimer->armed = true;\n>> > +\tsoft_timer_start(&timer->timer, kvm_timer_earliest_exp(vcpu));\n>> > }\n>> > \n>> > void kvm_timer_unschedule(struct kvm_vcpu *vcpu)\n>> > {\n>> > \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n>> > -\ttimer_disarm(timer);\n>> > +\n>> > +\tsoft_timer_cancel(&timer->timer, &timer->expired);\n>> > +\ttimer->armed = false;\n>> > }\n>> > \n>> > static void kvm_timer_flush_hwstate_vgic(struct kvm_vcpu *vcpu)\n>> > @@ -448,7 +447,7 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)\n>> > \t * This is to cancel the background timer for the physical timer\n>> > \t * emulation if it is set.\n>> > \t */\n>> > -\ttimer_disarm(timer);\n>> > +\tsoft_timer_cancel(&timer->timer, &timer->expired);\n>> \n>> timer_disarm() used to set timer->armed to false, but that's not the\n>> case any more. Don't we risk hitting the BUG_ON() in kvm_timer_schedule\n>> if we hit WFI?\n>> \n>\n> We do, and I just didn't hit that because this goes away at the end of\n> the series, and I didn't vigurously test every single patch in the\n> series (just a compile test).\n>\n> We actually only use the armed flag for the BUG_ON(), and I don't think\n> we need that check really. So I suggest simply merging this logic into\n> this patch:\n>\n> diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h\n> index f0053f884b4a..d0beae98f755 100644\n> --- a/include/kvm/arm_arch_timer.h\n> +++ b/include/kvm/arm_arch_timer.h\n> @@ -48,9 +48,6 @@ struct arch_timer_cpu {\n> \t/* Work queued with the above timer expires */\n> \tstruct work_struct\t\texpired;\n> \n> -\t/* Background timer active */\n> -\tbool\t\t\t\tarmed;\n> -\n> \t/* Is the timer enabled */\n> \tbool\t\t\tenabled;\n> };\n> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> index 871d8ae52f9b..98643bc696a9 100644\n> --- a/virt/kvm/arm/arch_timer.c\n> +++ b/virt/kvm/arm/arch_timer.c\n> @@ -56,11 +56,6 @@ u64 kvm_phys_timer_read(void)\n> \treturn timecounter->cc->read(timecounter->cc);\n> }\n> \n> -static bool soft_timer_is_armed(struct arch_timer_cpu *timer)\n> -{\n> -\treturn timer->armed;\n> -}\n> -\n> static void soft_timer_start(struct hrtimer *hrt, u64 ns)\n> {\n> \thrtimer_start(hrt, ktime_add_ns(ktime_get(), ns),\n> @@ -281,8 +276,6 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)\n> \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> \tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> \n> -\tBUG_ON(soft_timer_is_armed(timer));\n> -\n> \t/*\n> \t * No need to schedule a background timer if any guest timer has\n> \t * already expired, because kvm_vcpu_block will return before putting\n> @@ -302,7 +295,6 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)\n> \t * The guest timers have not yet expired, schedule a background timer.\n> \t * Set the earliest expiration time among the guest timers.\n> \t */\n> -\ttimer->armed = true;\n> \tsoft_timer_start(&timer->timer, kvm_timer_earliest_exp(vcpu));\n> }\n> \n> @@ -311,7 +303,6 @@ void kvm_timer_unschedule(struct kvm_vcpu *vcpu)\n> \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> \n> \tsoft_timer_cancel(&timer->timer, &timer->expired);\n> -\ttimer->armed = false;\n> }\n> \n> static void kvm_timer_flush_hwstate_vgic(struct kvm_vcpu *vcpu)\n\nYes, this seems like a sensible thing to do.\n\nThanks,\n\n\tM.","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"CdJBDisV\"; \n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=infradead.org header.i=@infradead.org\n\theader.b=\"K8WR0Xfp\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yHJFm5TXpz9t3l\n\tfor ;\n\tThu, 19 Oct 2017 03:59:56 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4rhB-00022I-8P; Wed, 18 Oct 2017 16:59:53 +0000","from casper.infradead.org ([2001:8b0:10b:1236::1])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4rh9-0001vR-3y for linux-arm-kernel@bombadil.infradead.org;\n\tWed, 18 Oct 2017 16:59:51 +0000","from foss.arm.com ([217.140.101.70])\n\tby casper.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4rbH-0002Ax-Es for linux-arm-kernel@lists.infradead.org;\n\tWed, 18 Oct 2017 16:53:49 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DAF20F;\n\tWed, 18 Oct 2017 09:53:25 -0700 (PDT)","from on-the-bus (unknown [10.3.33.38])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tA6FB03F25D; Wed, 18 Oct 2017 09:53:24 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:References\n\t:In-Reply-To:Subject:To:From:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=z64rOTpsnNnnsvrQ5UVHw8O3rmcj/Fa6ewbzFyNxY1k=;\n\tb=CdJBDisVrpfIc5\n\tHOdTc4F/VPaC6NroY7Wub3KFVvFkw3JUf0XS58+xwtZZdYeY9qByLvg50rNblvlTiHKAiGTsw+u0G\n\tmq6wzy77Vu/wQK3Ip8nlmE+BPbiElNmTy8do1edE5JwjWC4dWEIYNKwIsofwAN8kY48vgmQuKEPuF\n\tjMIMqM/8jEh6LmLdC+w0Cpc+BNp5L+Fza9lRc/WObGyjUCNjT/uiNQgkS4GOdpSZBo5GGp02JdO4b\n\tTKOLorE5JS9Far8GdUpmDBHqkF7wEAfwFzvDp985ITEo1c9kZEJd12NinQv7mKo2BylvdKMw5Dgt1\n\tp+LvIhN7zB5ysZs4xRwQ==;","v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=infradead.org; s=casper.20170209;\n\th=Content-Type:MIME-Version:Message-ID:\n\tDate:References:In-Reply-To:Subject:Cc:To:From:Sender:Reply-To:\n\tContent-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:\n\tList-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;\n\tbh=ow6jKtSDOQv6gspWpjyC+Wi/HiQJZqH/TJ77RjNy3cE=;\n\tb=K8WR0XfprV5w32fZtzHeE6gA1\n\thw4BPyhFFFg6KtA17SdtWGsqnj9qS+AxLhftwHqO/j13LEkgfUknacbYE9u6zbe+5fyAdFFSax3Zu\n\tGhMm4hITyfbfyFco9Fw3KDsWw7VyUlCMLDSo6s5mv3Gv7uasIKtZniPOKD8O9wI4s9AP4zOpaIEV0\n\tHiPLTIkKGWZiUHM3Mrbp8OuwiLARQFS1oF2rCroNq3afZTK3w1WgwqMNVZ2BgmMRe1+LIKExUJAgA\n\tiaGEN7m9vV8zsuz6RHwnv2+oTB+o0Vo7HNEnh7PBZSu9cqdxqMQsH81HhWld5suoypYZE7m0chGRz\n\tFZBkH6cvw==;"],"From":"Marc Zyngier ","To":"Christoffer Dall ","Subject":"Re: [PATCH v3 07/20] KVM: arm/arm64: Make timer_arm and timer_disarm\n\thelpers more generic","In-Reply-To":"<20171018164750.GI8900@cbox> (Christoffer Dall's message of\n\t\"Wed, 18 Oct 2017 18:47:50 +0200\")","Organization":"ARM Ltd","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-8-cdall@linaro.org>\n\t\n\t<20171018164750.GI8900@cbox>","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)","Date":"Wed, 18 Oct 2017 17:53:23 +0100","Message-ID":"<87o9p4ieik.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171018_175347_799710_CE3FF308 ","X-CRM114-Status":"GOOD ( 37.55 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on casper.infradead.org summary:\n\tContent analysis details: (-6.9 points, 5.0 required)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1789907,"web_url":"http://patchwork.ozlabs.org/comment/1789907/","msgid":"<20171018191638.GJ8900@cbox>","list_archive_url":null,"date":"2017-10-18T19:16:38","subject":"Re: [PATCH v3 06/20] KVM: arm/arm64: Check that system supports\n\tsplit eoi/deactivate","submitter":{"id":71350,"url":"http://patchwork.ozlabs.org/api/people/71350/","name":"Christoffer Dall","email":"cdall@linaro.org"},"content":"On Wed, Oct 18, 2017 at 05:03:40PM +0100, Marc Zyngier wrote:\n> On Wed, Oct 18 2017 at 3:41:45 pm BST, Christoffer Dall wrote:\n> > On Mon, Oct 09, 2017 at 05:47:18PM +0100, Marc Zyngier wrote:\n> >> On 23/09/17 01:41, Christoffer Dall wrote:\n> >> > Some systems without proper firmware and/or hardware description data\n> >> > don't support the split EOI and deactivate operation.\n> >> > \n> >> > On such systems, we cannot leave the physical interrupt active after the\n> >> > timer handler on the host has run, so we cannot support KVM with an\n> >> > in-kernel GIC with the timer changes we are about to introduce.\n> >> > \n> >> > This patch makes sure that trying to initialize the KVM GIC code will\n> >> > fail on such systems.\n> >> > \n> >> > Cc: Marc Zyngier \n> >> > Signed-off-by: Christoffer Dall \n> >> > ---\n> >> > drivers/irqchip/irq-gic.c | 3 ++-\n> >> > 1 file changed, 2 insertions(+), 1 deletion(-)\n> >> > \n> >> > diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c\n> >> > index f641e8e..ab12bf4 100644\n> >> > --- a/drivers/irqchip/irq-gic.c\n> >> > +++ b/drivers/irqchip/irq-gic.c\n> >> > @@ -1420,7 +1420,8 @@ static void __init gic_of_setup_kvm_info(struct device_node *node)\n> >> > \tif (ret)\n> >> > \t\treturn;\n> >> > \n> >> > -\tgic_set_kvm_info(&gic_v2_kvm_info);\n> >> > +\tif (static_key_true(&supports_deactivate))\n> >> > +\t\tgic_set_kvm_info(&gic_v2_kvm_info);\n> >> > }\n> >> > \n> >> > int __init\n> >> > \n> >> \n> >> Should we add the same level of checking on the ACPI path, just for the\n> >> sake symmetry?\n> >\n> > Yes, we should, if anyone is crazy enough to use ACPI :)\n> \n> Sadly, the madness is becoming commonplace.\n> \n> >> \n> >> Also, do we need to add the same thing for GICv3?\n> >> \n> >\n> > Why would split EOI/deactivate not be available on GICv3, actually? It\n> > looks like this is not supported unless you have EL2, but I can't seem\n> > to find anything in the spec for this, and KVM should support\n> > EOI/deactivate for GICv3 guests I think. Am I missing something?\n> \n> No, you're not. This is just a Linux choice (or rather mine) not to use\n> EOImode=1 in guests (or anything booted at EL1), as we don't really need\n> the two-stage deactivate in that situation (it is pure overhead).\n> \n> I'm just worried of potentially broken HW, and would like to make sure\n> that when we force EOImode=0 on these systems, we truly tell KVM about\n> it.\n> \n\nYes, makes sense, it's also more cosistent that way.\n\n> > Assuming I'm wrong about GICv3, which I probably am, how does this look\n> > (on top of the posted patch):\n> >\n> > diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c\n> > index 519149e..aed524c 100644\n> > --- a/drivers/irqchip/irq-gic-v3.c\n> > +++ b/drivers/irqchip/irq-gic-v3.c\n> > @@ -1228,7 +1228,9 @@ static int __init gic_of_init(struct device_node *node, struct device_node *pare\n> > \t\tgoto out_unmap_rdist;\n> > \n> > \tgic_populate_ppi_partitions(node);\n> > -\tgic_of_setup_kvm_info(node);\n> > +\n> > +\tif (static_key_true(&supports_deactivate))\n> > +\t\tgic_of_setup_kvm_info(node);\n> > \treturn 0;\n> > \n> > out_unmap_rdist:\n> > @@ -1517,7 +1519,9 @@ gic_acpi_init(struct acpi_subtable_header *header, const unsigned long end)\n> > \t\tgoto out_fwhandle_free;\n> > \n> > \tacpi_set_irq_model(ACPI_IRQ_MODEL_GIC, domain_handle);\n> > -\tgic_acpi_setup_kvm_info();\n> > +\n> > +\tif (static_key_true(&supports_deactivate))\n> > +\t\tgic_acpi_setup_kvm_info();\n> > \n> > \treturn 0;\n> > \n> > diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c\n> > index ab12bf4..121af5c 100644\n> > --- a/drivers/irqchip/irq-gic.c\n> > +++ b/drivers/irqchip/irq-gic.c\n> > @@ -1653,7 +1653,8 @@ static int __init gic_v2_acpi_init(struct acpi_subtable_header *header,\n> > \tif (IS_ENABLED(CONFIG_ARM_GIC_V2M))\n> > \t\tgicv2m_init(NULL, gic_data[0].domain);\n> > \n> > -\tgic_acpi_setup_kvm_info();\n> > +\tif (static_key_true(&supports_deactivate))\n> > +\t\tgic_acpi_setup_kvm_info();\n> > \n> > \treturn 0;\n> > }\n> \n> Yup, looks good to me!\n> \n\nThanks,\n-Christoffer","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"KBz1uj/z\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"c3iSONR+\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yHMJ21spWz9t4P\n\tfor ;\n\tThu, 19 Oct 2017 06:17:06 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4tpu-0005BQ-4E; Wed, 18 Oct 2017 19:17:02 +0000","from mail-wm0-x244.google.com ([2a00:1450:400c:c09::244])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4tpp-000477-IM for linux-arm-kernel@lists.infradead.org;\n\tWed, 18 Oct 2017 19:16:59 +0000","by mail-wm0-x244.google.com with SMTP id u138so11807191wmu.5\n\tfor ;\n\tWed, 18 Oct 2017 12:16:36 -0700 (PDT)","from localhost (xd93dd96b.cust.hiper.dk. [217.61.217.107])\n\tby smtp.gmail.com with ESMTPSA id\n\ta1sm9016418edb.37.2017.10.18.12.16.34\n\t(version=TLS1_2 cipher=AES128-SHA bits=128/128);\n\tWed, 18 Oct 2017 12:16:34 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:\n\tMessage-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=+7f/Ij1cB1c5wVtCWan00MmKpn/UbXqNufD2fgjo5Sk=;\n\tb=KBz1uj/zcebasv\n\tSysCpDQr05uYI8MJYiAt8lVy88GDB1MQZMOHs94LDSHcg2a1iN/1gZj1oe6HxEaeYquZoT28cFxDf\n\ta9LoPHq8kzLpewXfeLDKtNJC2HY8W8rVKnztCdsSYCCQDIVpdwtRjBbsUwbt7G+cwz2/tccbvwmei\n\t4rsBb9y7BfrCt6zm5LTVBB2wJzSx1oBhTqNyW0u6m0j8M4CbV9O5SO2sCrM9ctTPl4O1+ojwJ9zvi\n\tYD18Uwa8MKN9MfyWbcJioEoPldKXSjOZFWONIvxbEirG4XUCcKYd2kJCDVe2omsxPwJo/TjPAR0jq\n\th1G3v2XQ0qD4r931eq/g==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=lmcUrs8d3mZLC/ZE3oTTl9BJ03GsL441PLgkQISbXT8=;\n\tb=c3iSONR+uGd7G+0RxxV5KRXsjuZpTGRxHSxrSjRU+tDKuAt0o4u01o6wXuYUzRjPsg\n\tbJV05Gsq6E0pytysnyB0NIuT3+TRJLTK81FAERXy3vba9W1sz/wTP5vTHUh9Uhwq4iNf\n\t1q/O3K6n/MwuKXC7/7kyXO1+aYs5iiYDC1WE0="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=lmcUrs8d3mZLC/ZE3oTTl9BJ03GsL441PLgkQISbXT8=;\n\tb=GjPja3i3DxtNCAEntdtp6/oKB0zn9/VNVBbwKvWZcOTjAUPBvKDh1N7UNoh6rQyXvZ\n\t3/OdsUh8poT4RyYg02SMEeRYmnjumVa1+8cuJmN+QxLPsydbKyLqt37AePpHqo7U02u6\n\tkDaGc9n2az36F/HtTtyHUbASIqaWUvLYbzZINK6bF4s1jjNvqRTbCIb12iRzZNHCBVEQ\n\tk+76YrSYfcTFxxYSup55IcqPaaXyKqgYMzXPKSagQbW6a5IClLwYoib4CpZZxT6fe4fh\n\tki2x/3xfjVEZdh/FBPJP5eyaSOnf0cjKnSYIiYa4FH9LUOlOExooTqKwKv7cj3YfZqw9\n\tcwrQ==","X-Gm-Message-State":"AMCzsaWmjqicUqFv8TVD3nRIu/VkMnBbdLAxDYCD46BuLNgskdCVQNp0\n\tcwJO23NHDdTO+OeqN44iG1tLJw==","X-Google-Smtp-Source":"AOwi7QAMJ+se5gpi+91P8jPFIF5kWvWKzk55ViO5J87Zf5q4M631rD89I7h3j+UffKNZDoDmKW4M9Q==","X-Received":"by 10.80.173.150 with SMTP id a22mr20968953edd.49.1508354195268; \n\tWed, 18 Oct 2017 12:16:35 -0700 (PDT)","Date":"Wed, 18 Oct 2017 21:16:38 +0200","From":"Christoffer Dall ","To":"Marc Zyngier ","Subject":"Re: [PATCH v3 06/20] KVM: arm/arm64: Check that system supports\n\tsplit eoi/deactivate","Message-ID":"<20171018191638.GJ8900@cbox>","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-7-cdall@linaro.org>\n\t\n\t<20171018134145.GH8900@cbox>\n\t<874lqwjvdv.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<874lqwjvdv.fsf@on-the-bus.cambridge.arm.com>","User-Agent":"Mutt/1.5.21 (2010-09-15)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171018_121657_771996_C564BE7D ","X-CRM114-Status":"GOOD ( 28.48 )","X-Spam-Score":"-2.0 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.0 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1790178,"web_url":"http://patchwork.ozlabs.org/comment/1790178/","msgid":"<20171019073809.GM8900@cbox>","list_archive_url":null,"date":"2017-10-19T07:38:09","subject":"Re: [PATCH v3 09/20] KVM: arm/arm64: Use separate timer for phys\n\ttimer emulation","submitter":{"id":71350,"url":"http://patchwork.ozlabs.org/api/people/71350/","name":"Christoffer Dall","email":"cdall@linaro.org"},"content":"On Mon, Oct 09, 2017 at 06:23:45PM +0100, Marc Zyngier wrote:\n> On 23/09/17 01:41, Christoffer Dall wrote:\n> > We were using the same hrtimer for emulating the physical timer and for\n> > making sure a blocking VCPU thread would be eventually woken up. That\n> > worked fine in the previous arch timer design, but as we are about to\n> > actually use the soft timer expire function for the physical timer\n> > emulation, change the logic to use a dedicated hrtimer.\n> > \n> > This has the added benefit of not having to cancel any work in the sync\n> > path, which in turn allows us to run the flush and sync with IRQs\n> > disabled.\n> > \n> > Signed-off-by: Christoffer Dall \n> > ---\n> > include/kvm/arm_arch_timer.h | 3 +++\n> > virt/kvm/arm/arch_timer.c | 18 ++++++++++++++----\n> > 2 files changed, 17 insertions(+), 4 deletions(-)\n> > \n> > diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h\n> > index dcbb2e1..16887c0 100644\n> > --- a/include/kvm/arm_arch_timer.h\n> > +++ b/include/kvm/arm_arch_timer.h\n> > @@ -47,6 +47,9 @@ struct arch_timer_cpu {\n> > \t/* Work queued with the above timer expires */\n> > \tstruct work_struct\t\texpired;\n> > \n> > +\t/* Physical timer emulation */\n> > +\tstruct hrtimer\t\t\tphys_timer;\n> > +\n> > \t/* Background timer active */\n> > \tbool\t\t\t\tarmed;\n> > \n> > diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> > index c2e8326..7f87099 100644\n> > --- a/virt/kvm/arm/arch_timer.c\n> > +++ b/virt/kvm/arm/arch_timer.c\n> > @@ -178,6 +178,12 @@ static enum hrtimer_restart kvm_bg_timer_expire(struct hrtimer *hrt)\n> > \treturn HRTIMER_NORESTART;\n> > }\n> > \n> > +static enum hrtimer_restart kvm_phys_timer_expire(struct hrtimer *hrt)\n> > +{\n> > +\tWARN(1, \"Timer only used to ensure guest exit - unexpected event.\");\n> > +\treturn HRTIMER_NORESTART;\n> > +}\n> > +\n> \n> So what prevents this handler from actually firing? Is it that we cancel\n> the hrtimer while interrupts are still disabled, hence the timer never\n> fires? If that's the intention, then this patch is slightly out of\n> place, as we haven't moved the timer sync within the irq_disable() section.\n> \n> Or am I missing something obvious?\n> \n\nNo you're not missing anything, indeed, that is broken. I think I had\nin the back of my mind that we disable stuff in the world-switch still,\nbut that obviously doesn't apply to the soft timers.\n\nI'll just move this patch following the next one where interrupts are\ndisabled.\n\nNice catch!\n\n> > bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)\n> > {\n> > \tu64 cval, now;\n> > @@ -255,7 +261,7 @@ static void kvm_timer_update_state(struct kvm_vcpu *vcpu)\n> > }\n> > \n> > /* Schedule the background timer for the emulated timer. */\n> > -static void kvm_timer_emulate(struct kvm_vcpu *vcpu,\n> > +static void phys_timer_emulate(struct kvm_vcpu *vcpu,\n> > \t\t\t struct arch_timer_context *timer_ctx)\n> > {\n> > \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> > @@ -267,7 +273,7 @@ static void kvm_timer_emulate(struct kvm_vcpu *vcpu,\n> > \t\treturn;\n> > \n> > \t/* The timer has not yet expired, schedule a background timer */\n> > -\tsoft_timer_start(&timer->bg_timer, kvm_timer_compute_delta(timer_ctx));\n> > +\tsoft_timer_start(&timer->phys_timer, kvm_timer_compute_delta(timer_ctx));\n> > }\n> > \n> > /*\n> > @@ -424,7 +430,7 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)\n> > \tkvm_timer_update_state(vcpu);\n> > \n> > \t/* Set the background timer for the physical timer emulation. */\n> > -\tkvm_timer_emulate(vcpu, vcpu_ptimer(vcpu));\n> > +\tphys_timer_emulate(vcpu, vcpu_ptimer(vcpu));\n> > \n> > \tif (unlikely(!irqchip_in_kernel(vcpu->kvm)))\n> > \t\tkvm_timer_flush_hwstate_user(vcpu);\n> > @@ -447,7 +453,7 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)\n> > \t * This is to cancel the background timer for the physical timer\n> > \t * emulation if it is set.\n> > \t */\n> > -\tsoft_timer_cancel(&timer->bg_timer, &timer->expired);\n> > +\tsoft_timer_cancel(&timer->phys_timer, NULL);\n> \n> Right, that now explains the \"work\" test in one of the previous patches.\n> \n\nYes, I've moved the addition of the test to this patch which actually\nuses is.\n\n> > \n> > \t/*\n> > \t * The guest could have modified the timer registers or the timer\n> > @@ -507,6 +513,9 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)\n> > \thrtimer_init(&timer->bg_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);\n> > \ttimer->bg_timer.function = kvm_bg_timer_expire;\n> > \n> > +\thrtimer_init(&timer->phys_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);\n> > +\ttimer->phys_timer.function = kvm_phys_timer_expire;\n> > +\n> > \tvtimer->irq.irq = default_vtimer_irq.irq;\n> > \tptimer->irq.irq = default_ptimer_irq.irq;\n> > }\n> > @@ -615,6 +624,7 @@ void kvm_timer_vcpu_terminate(struct kvm_vcpu *vcpu)\n> > \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > \n> > \tsoft_timer_cancel(&timer->bg_timer, &timer->expired);\n> > +\tsoft_timer_cancel(&timer->phys_timer, NULL);\n> > \tkvm_vgic_unmap_phys_irq(vcpu, vtimer->irq.irq);\n> > }\n> > \n> > \n> \n\nThanks,\n-Christoffer","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"I8HJaVpD\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"LfIZMy2B\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yHglq2xv5z9tX4\n\tfor ;\n\tThu, 19 Oct 2017 18:38:47 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e55Pd-0002qv-BI; Thu, 19 Oct 2017 07:38:41 +0000","from mail-wm0-x242.google.com ([2a00:1450:400c:c09::242])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e55PT-0002el-Md for linux-arm-kernel@lists.infradead.org;\n\tThu, 19 Oct 2017 07:38:38 +0000","by mail-wm0-x242.google.com with SMTP id u138so14053227wmu.5\n\tfor ;\n\tThu, 19 Oct 2017 00:38:11 -0700 (PDT)","from localhost (xd93dd96b.cust.hiper.dk. [217.61.217.107])\n\tby smtp.gmail.com with ESMTPSA id\n\ts26sm9391020eds.68.2017.10.19.00.38.06\n\t(version=TLS1_2 cipher=AES128-SHA bits=128/128);\n\tThu, 19 Oct 2017 00:38:07 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:\n\tMessage-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=2QK1pMQkmx6I54QBUZcn1tZu/VCGQ4sDRq1tutHDJlI=;\n\tb=I8HJaVpDmAMejg\n\twXdKIlmJQTas4ndQkoPaXj8V7iS0zUZpxFBL89+VzP0gxCFiEnJvGEFqzHGUlAzcPYo3weyc6Twb/\n\tf+Laxzu6EfYFcZP7tmzCCCQBLD+SfoE10nIuS8x3D+Volb5UUO5ArCyyGsMvwzRrWM+L9AcBgEL9M\n\tN0QXyCAS4kkQBQe+smeQ44JN2U+BFgAM++UVpPhbRbbbr22f2VNWjI1yf6JUUMEQbr6VX1Hcbv/GI\n\txO9hOa7ajdV9+dZ4WyVFmA+slVieTINs0TajtYa1MI3nuJ12X54ArAyOuoQ4yrk3yAHkEoJgP5jGe\n\tYMELTypzoy5dOOlWRFAQ==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=ZlwtXnSnAlxhJEqvm/CPja2I5om+btx7mU31G8pjfkc=;\n\tb=LfIZMy2BZ+p5CWFHEhhkEjeBUb1RT+4eOiWAeAXbz3KsR7tTW13wG+FUW04SOnz2qb\n\tMFAUsEjxVav7wigfAcuqT5tkOowgVJWejiURe+pWE0T/RnisVHBPxME7US7NJutPjqQO\n\t3DRVVShtGtaEMQAyEnjwaDgJzsZ7ESE2o5x8w="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=ZlwtXnSnAlxhJEqvm/CPja2I5om+btx7mU31G8pjfkc=;\n\tb=eWjMX62eyouB759wem6vbFVqBi3doFNnW1aprXosF6nr2c5S6g+8En5MYauPSJqBwJ\n\tJzm2k8jWeZ08uNuUVOkFpbtFmlnjR+YNHtg1pGSqKJS2KaUq1JHFwf4JXuQvHkMGLiM4\n\tBP6OwCWATFfpnonBXVjlsnah5vckdQCdx0KwrvqZ9+nBa/9DuUsC681TnysNTv4zhklP\n\tq809cJb6s0hKBNvpCBDO057EtvyYuuIhsZvM+HA6VqDd6NWb9ZFqB/S0Wwtjjj0BNR6l\n\tCwuuWILPWiinD+LEReK6JP9gEKNVUzSxKaNTX+rhVecKYgYuwvOwauv5wNKx3OfLeIiW\n\tEpjw==","X-Gm-Message-State":"AMCzsaU3WMghQ6BK+UKG41AUdWjqRerg2DRTHVi3IE3UZn+YJYbHyQkO\n\tsgorpGMgf+8vY54R22EaxSGcxQ==","X-Google-Smtp-Source":"ABhQp+Rb4F15ulerAFwhjCEvkQxlGLqHKaAMmfP5vd4W4U3+DUS2tHQj1DDXoxWJtXzBHFxoPFgobA==","X-Received":"by 10.80.245.204 with SMTP id x12mr1300154edm.172.1508398689833; \n\tThu, 19 Oct 2017 00:38:09 -0700 (PDT)","Date":"Thu, 19 Oct 2017 09:38:09 +0200","From":"Christoffer Dall ","To":"Marc Zyngier ","Subject":"Re: [PATCH v3 09/20] KVM: arm/arm64: Use separate timer for phys\n\ttimer emulation","Message-ID":"<20171019073809.GM8900@cbox>","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-10-cdall@linaro.org>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.5.21 (2010-09-15)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171019_003832_177590_2F389CE1 ","X-CRM114-Status":"GOOD ( 27.94 )","X-Spam-Score":"-2.0 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.0 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,\n\tno\n\ttrust [2a00:1450:400c:c09:0:0:0:242 listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1790208,"web_url":"http://patchwork.ozlabs.org/comment/1790208/","msgid":"<20171019081529.GO8900@cbox>","list_archive_url":null,"date":"2017-10-19T08:15:29","subject":"Re: [PATCH v3 14/20] KVM: arm/arm64: Avoid timer save/restore in\n\tvcpu entry/exit","submitter":{"id":71350,"url":"http://patchwork.ozlabs.org/api/people/71350/","name":"Christoffer Dall","email":"cdall@linaro.org"},"content":"On Tue, Oct 10, 2017 at 09:47:33AM +0100, Marc Zyngier wrote:\n> On Sat, Sep 23 2017 at 2:42:01 am BST, Christoffer Dall wrote:\n> > We don't need to save and restore the hardware timer state and examine\n> > if it generates interrupts on on every entry/exit to the guest. The\n> > timer hardware is perfectly capable of telling us when it has expired\n> > by signaling interrupts.\n> >\n> > When taking a vtimer interrupt in the host, we don't want to mess with\n> > the timer configuration, we just want to forward the physical interrupt\n> > to the guest as a virtual interrupt. We can use the split priority drop\n> > and deactivate feature of the GIC to do this, which leaves an EOI'ed\n> > interrupt active on the physical distributor, making sure we don't keep\n> > taking timer interrupts which would prevent the guest from running. We\n> > can then forward the physical interrupt to the VM using the HW bit in\n> > the LR of the GIC VE, like we do already, which lets the guest directly\n> \n> VE?\n> \n\nVirtualization Extensions. I can use GIC hardware virtualization\nsupport or VGIC instead.\n\n> > deactivate both the physical and virtual timer simultaneously, allowing\n> > the timer hardware to exit the VM and generate a new physical interrupt\n> > when the timer output is again asserted later on.\n> >\n> > We do need to capture this state when migrating VCPUs between physical\n> > CPUs, however, which we use the vcpu put/load functions for, which are\n> > called through preempt notifiers whenever the thread is scheduled away\n> > from the CPU or called directly if we return from the ioctl to\n> > userspace.\n> >\n> > One caveat is that we cannot restore the timer state during\n> > kvm_timer_vcpu_load, because the flow of sleeping a VCPU is:\n> >\n> > 1. kvm_vcpu_block\n> > 2. kvm_timer_schedule\n> > 3. schedule\n> > 4. kvm_timer_vcpu_put (preempt notifier)\n> > 5. schedule (vcpu thread gets scheduled back)\n> > 6. kvm_timer_vcpu_load\n> > <---- We restore the hardware state here, but the bg_timer\n> > \t hrtimer may have scheduled a work function that also\n> > \t changes the timer state here.\n> > 7. kvm_timer_unschedule\n> > <---- We can restore the state here instead\n> >\n> > So, while we do need to restore the timer state in step (6) in all other\n> > cases than when we called kvm_vcpu_block(), we have to defer the restore\n> > to step (7) when coming back after kvm_vcpu_block(). Note that we\n> > cannot simply call cancel_work_sync() in step (6), because vcpu_load can\n> > be called from a preempt notifier.\n> >\n> > An added benefit beyond not having to read and write the timer sysregs\n> > on every entry and exit is that we no longer have to actively write the\n> > active state to the physical distributor, because we set the affinity of\n> \n> I don't understand this thing about the affinity of the timer. It is a\n> PPI, so it cannot go anywhere else.\n> \n\nAh, silly wording perhaps. I mean that we call irq_set_vcpu_affinity()\nso that the interrupt doesn't get deactivated by the GIC driver. I can\ntry to reword.\n\nHow about:\n\n An added benefit beyond not having to read and write the timer sysregs\n on every entry and exit is that we no longer have to actively write the\n active state to the physical distributor, because we configured the\n irq for the vtimer to only get a priority drop when handling the\n interrupt in the GIC driver (we called irq_set_vcpu_affinity()), and\n the interrupt stays active after firing on the host.\n\n\n> > the vtimer interrupt when loading the timer state, so that the interrupt\n> > automatically stays active after firing.\n> >\n> > Signed-off-by: Christoffer Dall \n> > ---\n> > include/kvm/arm_arch_timer.h | 9 +-\n> > virt/kvm/arm/arch_timer.c | 238 +++++++++++++++++++++++++++----------------\n> > virt/kvm/arm/arm.c | 19 +++-\n> > virt/kvm/arm/hyp/timer-sr.c | 8 +-\n> > 4 files changed, 174 insertions(+), 100 deletions(-)\n> >\n> > diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h\n> > index 16887c0..8e5ed54 100644\n> > --- a/include/kvm/arm_arch_timer.h\n> > +++ b/include/kvm/arm_arch_timer.h\n> > @@ -31,8 +31,8 @@ struct arch_timer_context {\n> > \t/* Timer IRQ */\n> > \tstruct kvm_irq_level\t\tirq;\n> > \n> > -\t/* Active IRQ state caching */\n> > -\tbool\t\t\t\tactive_cleared_last;\n> > +\t/* Is the timer state loaded on the hardware timer */\n> > +\tbool\t\t\tloaded;\n> \n> I think this little guy is pretty crucial to understand the flow, as\n> there is now two points where we save/restore the timer:\n> vcpu_load/vcpu_put and timer_schedule/timer_unschedule. Both can be\n> executed on the blocking path, and this is the predicate to find out if\n> there is actually something to do.\n> \n> Would you mind adding a small comment to that effect?\n> \n\nI don't mind at all, will add a comment. How about:\n\n\t/*\n\t * We have multiple paths which can save/restore the timer state\n\t * onto the hardware, so we need some way of keeping track of\n\t * where the latest state is.\n\t *\n\t * loaded == true: State is loaded on the hardware registers.\n\t * loaded == false: State is stored in memory.\n\t */\n\n> > \n> > \t/* Virtual offset */\n> > \tu64\t\t\tcntvoff;\n> > @@ -80,10 +80,15 @@ void kvm_timer_unschedule(struct kvm_vcpu *vcpu);\n> > \n> > u64 kvm_phys_timer_read(void);\n> > \n> > +void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu);\n> > void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu);\n> > \n> > void kvm_timer_init_vhe(void);\n> > \n> > #define vcpu_vtimer(v)\t(&(v)->arch.timer_cpu.vtimer)\n> > #define vcpu_ptimer(v)\t(&(v)->arch.timer_cpu.ptimer)\n> > +\n> > +void enable_el1_phys_timer_access(void);\n> > +void disable_el1_phys_timer_access(void);\n> > +\n> > #endif\n> > diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> > index 4275f8f..70110ea 100644\n> > --- a/virt/kvm/arm/arch_timer.c\n> > +++ b/virt/kvm/arm/arch_timer.c\n> > @@ -46,10 +46,9 @@ static const struct kvm_irq_level default_vtimer_irq = {\n> > \t.level\t= 1,\n> > };\n> > \n> > -void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)\n> > -{\n> > -\tvcpu_vtimer(vcpu)->active_cleared_last = false;\n> > -}\n> > +static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx);\n> > +static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,\n> > +\t\t\t\t struct arch_timer_context *timer_ctx);\n> > \n> > u64 kvm_phys_timer_read(void)\n> > {\n> > @@ -74,17 +73,37 @@ static void soft_timer_cancel(struct hrtimer *hrt, struct work_struct *work)\n> > \t\tcancel_work_sync(work);\n> > }\n> > \n> > -static irqreturn_t kvm_arch_timer_handler(int irq, void *dev_id)\n> > +static void kvm_vtimer_update_mask_user(struct kvm_vcpu *vcpu)\n> > {\n> > -\tstruct kvm_vcpu *vcpu = *(struct kvm_vcpu **)dev_id;\n> > +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > \n> > \t/*\n> > -\t * We disable the timer in the world switch and let it be\n> > -\t * handled by kvm_timer_sync_hwstate(). Getting a timer\n> > -\t * interrupt at this point is a sure sign of some major\n> > -\t * breakage.\n> > +\t * To prevent continuously exiting from the guest, we mask the\n> > +\t * physical interrupt when the virtual level is high, such that the\n> > +\t * guest can make forward progress. Once we detect the output level\n> > +\t * being deasserted, we unmask the interrupt again so that we exit\n> > +\t * from the guest when the timer fires.\n> \n> Maybe an additional comment indicating that this only makes sense when\n> we don't have an in-kernel GIC? I know this wasn't in the original code,\n> but I started asking myself all kind of questions until I realised what\n> this was for...\n> \n\nYes, I'll clarify. How about:\n\n\t/*\n\t * When using a userspace irqchip with the architected timers,\n\t * we disable...\n\t [...]\n\t * ...we mask the physical interrupt by disabling it on the host\n\t * interrupt controller when the...\n\n> > \t */\n> > -\tpr_warn(\"Unexpected interrupt %d on vcpu %p\\n\", irq, vcpu);\n> > +\tif (vtimer->irq.level)\n> > +\t\tdisable_percpu_irq(host_vtimer_irq);\n> > +\telse\n> > +\t\tenable_percpu_irq(host_vtimer_irq, 0);\n> > +}\n> > +\n> > +static irqreturn_t kvm_arch_timer_handler(int irq, void *dev_id)\n> > +{\n> > +\tstruct kvm_vcpu *vcpu = *(struct kvm_vcpu **)dev_id;\n> > +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > +\n> > +\tif (!vtimer->irq.level) {\n> > +\t\tvtimer->cnt_ctl = read_sysreg_el0(cntv_ctl);\n> > +\t\tif (kvm_timer_irq_can_fire(vtimer))\n> > +\t\t\tkvm_timer_update_irq(vcpu, true, vtimer);\n> > +\t}\n> > +\n> > +\tif (unlikely(!irqchip_in_kernel(vcpu->kvm)))\n> > +\t\tkvm_vtimer_update_mask_user(vcpu);\n> > +\n> > \treturn IRQ_HANDLED;\n> > }\n> > \n> > @@ -220,7 +239,6 @@ static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,\n> > {\n> > \tint ret;\n> > \n> > -\ttimer_ctx->active_cleared_last = false;\n> > \ttimer_ctx->irq.level = new_level;\n> > \ttrace_kvm_timer_update_irq(vcpu->vcpu_id, timer_ctx->irq.irq,\n> > \t\t\t\t timer_ctx->irq.level);\n> > @@ -276,10 +294,16 @@ static void phys_timer_emulate(struct kvm_vcpu *vcpu,\n> > \tsoft_timer_start(&timer->phys_timer, kvm_timer_compute_delta(timer_ctx));\n> > }\n> > \n> > -static void timer_save_state(struct kvm_vcpu *vcpu)\n> > +static void vtimer_save_state(struct kvm_vcpu *vcpu)\n> > {\n> > \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> > \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > +\tunsigned long flags;\n> > +\n> > +\tlocal_irq_save(flags);\n> \n> Is that to avoid racing against the timer when doing a\n> vcpu_put/timer/schedule?\n> \n\nDepends on where it's called from. When called from kvm_timer_schedule,\nthis is because we need to know the state of the timer, so we know when\nto schedule the timer in the future, which is only done in\nkvm_timer_schedule (not kvm_timer_vcpu_put). When called from\nkvm_timer_vcpu_put, it is to save the state so that we can preserve it.\n\n> > +\n> > +\tif (!vtimer->loaded)\n> > +\t\tgoto out;\n> > \n> > \tif (timer->enabled) {\n> > \t\tvtimer->cnt_ctl = read_sysreg_el0(cntv_ctl);\n> > @@ -288,6 +312,10 @@ static void timer_save_state(struct kvm_vcpu *vcpu)\n> > \n> > \t/* Disable the virtual timer */\n> > \twrite_sysreg_el0(0, cntv_ctl);\n> > +\n> > +\tvtimer->loaded = false;\n> > +out:\n> > +\tlocal_irq_restore(flags);\n> > }\n> > \n> > /*\n> > @@ -303,6 +331,8 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)\n> > \n> > \tBUG_ON(bg_timer_is_armed(timer));\n> > \n> > +\tvtimer_save_state(vcpu);\n> > +\n> > \t/*\n> > \t * No need to schedule a background timer if any guest timer has\n> > \t * already expired, because kvm_vcpu_block will return before putting\n> > @@ -326,16 +356,26 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)\n> > \tsoft_timer_start(&timer->bg_timer, kvm_timer_earliest_exp(vcpu));\n> > }\n> > \n> > -static void timer_restore_state(struct kvm_vcpu *vcpu)\n> > +static void vtimer_restore_state(struct kvm_vcpu *vcpu)\n> > {\n> > \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> > \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > +\tunsigned long flags;\n> > +\n> > +\tlocal_irq_save(flags);\n> > +\n> > +\tif (vtimer->loaded)\n> > +\t\tgoto out;\n> > \n> > \tif (timer->enabled) {\n> > \t\twrite_sysreg_el0(vtimer->cnt_cval, cntv_cval);\n> > \t\tisb();\n> > \t\twrite_sysreg_el0(vtimer->cnt_ctl, cntv_ctl);\n> > \t}\n> > +\n> > +\tvtimer->loaded = true;\n> > +out:\n> > +\tlocal_irq_restore(flags);\n> > }\n> > \n> > void kvm_timer_unschedule(struct kvm_vcpu *vcpu)\n> > @@ -344,6 +384,8 @@ void kvm_timer_unschedule(struct kvm_vcpu *vcpu)\n> > \n> > \tsoft_timer_cancel(&timer->bg_timer, &timer->expired);\n> > \ttimer->armed = false;\n> > +\n> > +\tvtimer_restore_state(vcpu);\n> > }\n> > \n> > static void set_cntvoff(u64 cntvoff)\n> > @@ -353,61 +395,56 @@ static void set_cntvoff(u64 cntvoff)\n> > \tkvm_call_hyp(__kvm_timer_set_cntvoff, low, high);\n> > }\n> > \n> > -static void kvm_timer_flush_hwstate_vgic(struct kvm_vcpu *vcpu)\n> > +static void kvm_timer_vcpu_load_vgic(struct kvm_vcpu *vcpu)\n> > {\n> > \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > \tbool phys_active;\n> > \tint ret;\n> > \n> > -\t/*\n> > -\t* If we enter the guest with the virtual input level to the VGIC\n> > -\t* asserted, then we have already told the VGIC what we need to, and\n> > -\t* we don't need to exit from the guest until the guest deactivates\n> > -\t* the already injected interrupt, so therefore we should set the\n> > -\t* hardware active state to prevent unnecessary exits from the guest.\n> > -\t*\n> > -\t* Also, if we enter the guest with the virtual timer interrupt active,\n> > -\t* then it must be active on the physical distributor, because we set\n> > -\t* the HW bit and the guest must be able to deactivate the virtual and\n> > -\t* physical interrupt at the same time.\n> > -\t*\n> > -\t* Conversely, if the virtual input level is deasserted and the virtual\n> > -\t* interrupt is not active, then always clear the hardware active state\n> > -\t* to ensure that hardware interrupts from the timer triggers a guest\n> > -\t* exit.\n> > -\t*/\n> > -\tphys_active = vtimer->irq.level ||\n> > -\t\t\tkvm_vgic_map_is_active(vcpu, vtimer->irq.irq);\n> > -\n> > -\t/*\n> > -\t * We want to avoid hitting the (re)distributor as much as\n> > -\t * possible, as this is a potentially expensive MMIO access\n> > -\t * (not to mention locks in the irq layer), and a solution for\n> > -\t * this is to cache the \"active\" state in memory.\n> > -\t *\n> > -\t * Things to consider: we cannot cache an \"active set\" state,\n> > -\t * because the HW can change this behind our back (it becomes\n> > -\t * \"clear\" in the HW). We must then restrict the caching to\n> > -\t * the \"clear\" state.\n> > -\t *\n> > -\t * The cache is invalidated on:\n> > -\t * - vcpu put, indicating that the HW cannot be trusted to be\n> > -\t * in a sane state on the next vcpu load,\n> > -\t * - any change in the interrupt state\n> > -\t *\n> > -\t * Usage conditions:\n> > -\t * - cached value is \"active clear\"\n> > -\t * - value to be programmed is \"active clear\"\n> > -\t */\n> > -\tif (vtimer->active_cleared_last && !phys_active)\n> > -\t\treturn;\n> > -\n> > +\tif (vtimer->irq.level || kvm_vgic_map_is_active(vcpu, vtimer->irq.irq))\n> > +\t\tphys_active = true;\n> > +\telse\n> > +\t\tphys_active = false;\n> \n> nit: this can be written as:\n> \n> phys_active = (vtimer->irq.level ||\n> \t\t kvm_vgic_map_is_active(vcpu, vtimer->irq.irq));\n> \n> Not that it matters in the slightest...\n> \n\nI don't mind changing it.\n\n> > \tret = irq_set_irqchip_state(host_vtimer_irq,\n> > \t\t\t\t IRQCHIP_STATE_ACTIVE,\n> > \t\t\t\t phys_active);\n> > \tWARN_ON(ret);\n> > +}\n> > +\n> > +static void kvm_timer_vcpu_load_user(struct kvm_vcpu *vcpu)\n> > +{\n> > +\tkvm_vtimer_update_mask_user(vcpu);\n> > +}\n> > +\n> > +void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)\n> > +{\n> > +\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> > +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > +\n> > +\tif (unlikely(!timer->enabled))\n> > +\t\treturn;\n> > +\n> > +\tif (unlikely(!irqchip_in_kernel(vcpu->kvm)))\n> > +\t\tkvm_timer_vcpu_load_user(vcpu);\n> > +\telse\n> > +\t\tkvm_timer_vcpu_load_vgic(vcpu);\n> > \n> > -\tvtimer->active_cleared_last = !phys_active;\n> > +\tset_cntvoff(vtimer->cntvoff);\n> > +\n> > +\t/*\n> > +\t * If we armed a soft timer and potentially queued work, we have to\n> > +\t * cancel this, but cannot do it here, because canceling work can\n> > +\t * sleep and we can be in the middle of a preempt notifier call.\n> > +\t * Instead, when the timer has been armed, we know the return path\n> > +\t * from kvm_vcpu_block will call kvm_timer_unschedule, so we can defer\n> > +\t * restoring the state and canceling any soft timers and work items\n> > +\t * until then.\n> > +\t */\n> > +\tif (!bg_timer_is_armed(timer))\n> > +\t\tvtimer_restore_state(vcpu);\n> > +\n> > +\tif (has_vhe())\n> > +\t\tdisable_el1_phys_timer_access();\n> > }\n> > \n> > bool kvm_timer_should_notify_user(struct kvm_vcpu *vcpu)\n> > @@ -427,23 +464,6 @@ bool kvm_timer_should_notify_user(struct kvm_vcpu *vcpu)\n> > \t ptimer->irq.level != plevel;\n> > }\n> > \n> > -static void kvm_timer_flush_hwstate_user(struct kvm_vcpu *vcpu)\n> > -{\n> > -\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > -\n> > -\t/*\n> > -\t * To prevent continuously exiting from the guest, we mask the\n> > -\t * physical interrupt such that the guest can make forward progress.\n> > -\t * Once we detect the output level being deasserted, we unmask the\n> > -\t * interrupt again so that we exit from the guest when the timer\n> > -\t * fires.\n> > -\t*/\n> > -\tif (vtimer->irq.level)\n> > -\t\tdisable_percpu_irq(host_vtimer_irq);\n> > -\telse\n> > -\t\tenable_percpu_irq(host_vtimer_irq, 0);\n> > -}\n> > -\n> > /**\n> > * kvm_timer_flush_hwstate - prepare timers before running the vcpu\n> > * @vcpu: The vcpu pointer\n> > @@ -456,23 +476,55 @@ static void kvm_timer_flush_hwstate_user(struct kvm_vcpu *vcpu)\n> > void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)\n> > {\n> > \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> > -\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > +\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> > \n> > \tif (unlikely(!timer->enabled))\n> > \t\treturn;\n> > \n> > -\tkvm_timer_update_state(vcpu);\n> > +\tif (kvm_timer_should_fire(ptimer) != ptimer->irq.level)\n> > +\t\tkvm_timer_update_irq(vcpu, !ptimer->irq.level, ptimer);\n> > \n> > \t/* Set the background timer for the physical timer emulation. */\n> > \tphys_timer_emulate(vcpu, vcpu_ptimer(vcpu));\n> > +}\n> > \n> > -\tif (unlikely(!irqchip_in_kernel(vcpu->kvm)))\n> > -\t\tkvm_timer_flush_hwstate_user(vcpu);\n> > -\telse\n> > -\t\tkvm_timer_flush_hwstate_vgic(vcpu);\n> > +void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)\n> > +{\n> > +\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> > \n> > -\tset_cntvoff(vtimer->cntvoff);\n> > -\ttimer_restore_state(vcpu);\n> > +\tif (unlikely(!timer->enabled))\n> > +\t\treturn;\n> > +\n> > +\tif (has_vhe())\n> > +\t\tenable_el1_phys_timer_access();\n> > +\n> > +\tvtimer_save_state(vcpu);\n> > +\n> > +\tset_cntvoff(0);\n> \n> Can this be moved into vtimer_save_state()?\n\nIt can, I just kept it out of there, because it's technically not saving\nany state, but managing some other piece of host hardware, which only\nneeds to get reset when doing kvm_timer_vcpu_put, ...\n\n> And thinking of it, why\n> don't we reset cntvoff in kvm_timer_schedule() as well? \n> \n\n... because kvm_timer_vcpu_put will get called any time we're going to\nrun userspace or some other kernel thread, even after we've gotten\nkvm_timer_schedule, and that's what made the most semantic sense to me;\nhere we need to make sure that userspace which accesses the virtual\ncounter sees a zero offset to the physical counter.\n\nI can put a comment in kvm_timer_vcpu_put, or move it into\nvtimer_save_state, with a slight preference to the first option. What\ndo you think?\n\n> > +}\n> > +\n> > +static void unmask_vtimer_irq(struct kvm_vcpu *vcpu)\n> > +{\n> > +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > +\n> > +\tif (unlikely(!irqchip_in_kernel(vcpu->kvm))) {\n> > +\t\tkvm_vtimer_update_mask_user(vcpu);\n> > +\t\treturn;\n> > +\t}\n> > +\n> > +\t/*\n> > +\t * If the guest disabled the timer without acking the interrupt, then\n> > +\t * we must make sure the physical and virtual active states are in\n> > +\t * sync by deactivating the physical interrupt, because otherwise we\n> > +\t * wouldn't see the next timer interrupt in the host.\n> > +\t */\n> > +\tif (!kvm_vgic_map_is_active(vcpu, vtimer->irq.irq)) {\n> > +\t\tint ret;\n> > +\t\tret = irq_set_irqchip_state(host_vtimer_irq,\n> > +\t\t\t\t\t IRQCHIP_STATE_ACTIVE,\n> > +\t\t\t\t\t false);\n> > +\t\tWARN_ON(ret);\n> > +\t}\n> > }\n> > \n> > /**\n> > @@ -485,6 +537,7 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)\n> > void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)\n> > {\n> > \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> > +\tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > \n> > \t/*\n> > \t * This is to cancel the background timer for the physical timer\n> > @@ -492,14 +545,19 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)\n> > \t */\n> > \tsoft_timer_cancel(&timer->phys_timer, NULL);\n> > \n> > -\ttimer_save_state(vcpu);\n> > -\tset_cntvoff(0);\n> > -\n> > \t/*\n> > -\t * The guest could have modified the timer registers or the timer\n> > -\t * could have expired, update the timer state.\n> > +\t * If we entered the guest with the vtimer output asserted we have to\n> > +\t * check if the guest has modified the timer so that we should lower\n> > +\t * the line at this point.\n> > \t */\n> > -\tkvm_timer_update_state(vcpu);\n> > +\tif (vtimer->irq.level) {\n> > +\t\tvtimer->cnt_ctl = read_sysreg_el0(cntv_ctl);\n> > +\t\tvtimer->cnt_cval = read_sysreg_el0(cntv_cval);\n> > +\t\tif (!kvm_timer_should_fire(vtimer)) {\n> > +\t\t\tkvm_timer_update_irq(vcpu, false, vtimer);\n> > +\t\t\tunmask_vtimer_irq(vcpu);\n> > +\t\t}\n> > +\t}\n> > }\n> > \n> > int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)\n> > diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c\n> > index 27db222..132d39a 100644\n> > --- a/virt/kvm/arm/arm.c\n> > +++ b/virt/kvm/arm/arm.c\n> > @@ -354,18 +354,18 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)\n> > \tvcpu->arch.host_cpu_context = this_cpu_ptr(kvm_host_cpu_state);\n> > \n> > \tkvm_arm_set_running_vcpu(vcpu);\n> > -\n> > \tkvm_vgic_load(vcpu);\n> > +\tkvm_timer_vcpu_load(vcpu);\n> > }\n> > \n> > void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)\n> > {\n> > +\tkvm_timer_vcpu_put(vcpu);\n> > \tkvm_vgic_put(vcpu);\n> > \n> > \tvcpu->cpu = -1;\n> > \n> > \tkvm_arm_set_running_vcpu(NULL);\n> > -\tkvm_timer_vcpu_put(vcpu);\n> > }\n> > \n> > static void vcpu_power_off(struct kvm_vcpu *vcpu)\n> > @@ -710,16 +710,27 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)\n> > \t\tkvm_arm_clear_debug(vcpu);\n> > \n> > \t\t/*\n> > -\t\t * We must sync the PMU and timer state before the vgic state so\n> > +\t\t * We must sync the PMU state before the vgic state so\n> > \t\t * that the vgic can properly sample the updated state of the\n> > \t\t * interrupt line.\n> > \t\t */\n> > \t\tkvm_pmu_sync_hwstate(vcpu);\n> > -\t\tkvm_timer_sync_hwstate(vcpu);\n> > \n> > +\t\t/*\n> > +\t\t * Sync the vgic state before syncing the timer state because\n> > +\t\t * the timer code needs to know if the virtual timer\n> > +\t\t * interrupts are active.\n> > +\t\t */\n> > \t\tkvm_vgic_sync_hwstate(vcpu);\n> > \n> > \t\t/*\n> > +\t\t * Sync the timer hardware state before enabling interrupts as\n> > +\t\t * we don't want vtimer interrupts to race with syncing the\n> > +\t\t * timer virtual interrupt state.\n> > +\t\t */\n> > +\t\tkvm_timer_sync_hwstate(vcpu);\n> > +\n> > +\t\t/*\n> > \t\t * We may have taken a host interrupt in HYP mode (ie\n> > \t\t * while executing the guest). This interrupt is still\n> > \t\t * pending, as we haven't serviced it yet!\n> > diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c\n> > index a6c3b10..f398616 100644\n> > --- a/virt/kvm/arm/hyp/timer-sr.c\n> > +++ b/virt/kvm/arm/hyp/timer-sr.c\n> > @@ -27,7 +27,7 @@ void __hyp_text __kvm_timer_set_cntvoff(u32 cntvoff_low, u32 cntvoff_high)\n> > \twrite_sysreg(cntvoff, cntvoff_el2);\n> > }\n> > \n> > -void __hyp_text enable_phys_timer(void)\n> > +void __hyp_text enable_el1_phys_timer_access(void)\n> > {\n> > \tu64 val;\n> > \n> > @@ -37,7 +37,7 @@ void __hyp_text enable_phys_timer(void)\n> > \twrite_sysreg(val, cnthctl_el2);\n> > }\n> > \n> > -void __hyp_text disable_phys_timer(void)\n> > +void __hyp_text disable_el1_phys_timer_access(void)\n> > {\n> > \tu64 val;\n> > \n> > @@ -58,11 +58,11 @@ void __hyp_text __timer_disable_traps(struct kvm_vcpu *vcpu)\n> > \t * with HCR_EL2.TGE ==1, which makes those bits have no impact.\n> > \t */\n> > \tif (!has_vhe())\n> > -\t\tenable_phys_timer();\n> > +\t\tenable_el1_phys_timer_access();\n> > }\n> > \n> > void __hyp_text __timer_enable_traps(struct kvm_vcpu *vcpu)\n> > {\n> > \tif (!has_vhe())\n> > -\t\tdisable_phys_timer();\n> > +\t\tdisable_el1_phys_timer_access();\n> > }\n> \n> It'd be nice to move this renaming to the patch that introduce these two\n> functions.\n\nAh, yes, absolutely. Patch splitting madness.\n\nThanks,\n-Christoffer","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"hyZo110V\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"NNygrPGX\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yHhZt66RLz9t44\n\tfor ;\n\tThu, 19 Oct 2017 19:16:06 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e55zk-0001er-8Q; Thu, 19 Oct 2017 08:16:00 +0000","from mail-wm0-x241.google.com ([2a00:1450:400c:c09::241])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e55zd-00019e-DU for linux-arm-kernel@lists.infradead.org;\n\tThu, 19 Oct 2017 08:15:57 +0000","by mail-wm0-x241.google.com with SMTP id b189so14165460wmd.4\n\tfor ;\n\tThu, 19 Oct 2017 01:15:32 -0700 (PDT)","from localhost (xd93dd96b.cust.hiper.dk. [217.61.217.107])\n\tby smtp.gmail.com with ESMTPSA id\n\tm1sm9796736edd.76.2017.10.19.01.15.26\n\t(version=TLS1_2 cipher=AES128-SHA bits=128/128);\n\tThu, 19 Oct 2017 01:15:26 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:\n\tMessage-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=a2JcD/rBKNYizTGUI57gd1M5Jf56fh5m0ixzVuT1SD0=;\n\tb=hyZo110VmaFuQn\n\taaoSuDdRNqgWSBXOjTVNm6e7cbEgE/ys5tRnwEstyjB38TKsISadLMNaAraQrIT/TPGnreS/OI3t4\n\tK6meFeHsYSDIXrFNmhVZgW/7ddIzlA5ACCWYOD4Nav4xuEW5FJa8HMVfTznMEw9M7bntiMjwARqkW\n\tyWR9/tZWMfWVDft23mOIkWS5FhiFfOCdRMXKYQA0e8hRDmyIMzLctctC7f3kmKKSVRJmxAQgkobsq\n\tqf1r5nU5tIvjNhyUAPPkSAyTCQfjFaReBXgPjfz0ydGr9Oiat+1SrpSFqa541NNnSXWeR6JVgPsTu\n\ts/lctiLd7YEg+6IZShHA==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=gOPom/JNVT7TJC/peaeXlayVvBGt5Ha64+F4iOI/1U8=;\n\tb=NNygrPGXr+UWhyl7Pal3VfDcJBg8MMMbd+qW8NnsqpEXbJApOGOR+yvgZBTDgDCKqw\n\tJ8a3FkYwhtccbjP1NfYbixnDI06qewa/Nn6MUgFuebrfMOQnuyiwQQlfgonEJj53kff1\n\tfXPHwOomQjAJoisT2gxT8bX3ZOVwAdAPZ5vao="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=gOPom/JNVT7TJC/peaeXlayVvBGt5Ha64+F4iOI/1U8=;\n\tb=ALiforfEG7h0WAMCEqYIIgv+llkmgKvyyzj4syLKwzAQIlf/vAXYh+BvZugRzAmk3g\n\t6OZCl+lBo6Wsk4caEK3oPGfD4TW4wbojFwCRjcMummH7y8TEsdYlyD02VVPGffpsfTCW\n\tDC16Lr+MjunGMDVI9nJDzBaHFGCo8wrVckJYOHR1rA4/mimMLGZm9bPsn3z2QO0nUCbf\n\tjMcZREthOhyRVa5utkredI0oTpbas0YBY8bbKB7YM7vyAAbvLX2b8RSZcA4dbBzpvlI/\n\t0Dlt644NSF2UAJ8bKCfJ1ZJe0Spg7iauDvx/bgRDdW6pD5NMhnnUn+MUgAXjjdLN0YcT\n\tEwxg==","X-Gm-Message-State":"AMCzsaXtFKnGd31RM3AE8xUXQjmuGXVfpgXIj7/vMP6V3WROg1QmNilZ\n\t08PzF96yU24/fAoV2KE3ZYDtGw==","X-Google-Smtp-Source":"ABhQp+QLrjsWETJzrK7jkxaD868gFzdtu8jXWegw4mPvAcSxWHBvNEPke0WUYyVzizJUpfdzAoQbZw==","X-Received":"by 10.80.152.21 with SMTP id g21mr1390337edb.305.1508400930351; \n\tThu, 19 Oct 2017 01:15:30 -0700 (PDT)","Date":"Thu, 19 Oct 2017 10:15:29 +0200","From":"Christoffer Dall ","To":"Marc Zyngier ","Subject":"Re: [PATCH v3 14/20] KVM: arm/arm64: Avoid timer save/restore in\n\tvcpu entry/exit","Message-ID":"<20171019081529.GO8900@cbox>","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-15-cdall@linaro.org>\n\t<87mv4zwfru.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<87mv4zwfru.fsf@on-the-bus.cambridge.arm.com>","User-Agent":"Mutt/1.5.21 (2010-09-15)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171019_011553_804419_B5C83F3B ","X-CRM114-Status":"GOOD ( 45.44 )","X-Spam-Score":"-2.0 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.0 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1790228,"web_url":"http://patchwork.ozlabs.org/comment/1790228/","msgid":"<20171019083233.GP8900@cbox>","list_archive_url":null,"date":"2017-10-19T08:32:33","subject":"Re: [PATCH v3 15/20] KVM: arm/arm64: Support EL1 phys timer register\n\taccess in set/get reg","submitter":{"id":71350,"url":"http://patchwork.ozlabs.org/api/people/71350/","name":"Christoffer Dall","email":"cdall@linaro.org"},"content":"On Tue, Oct 10, 2017 at 10:10:27AM +0100, Marc Zyngier wrote:\n> On Sat, Sep 23 2017 at 2:42:02 am BST, Christoffer Dall wrote:\n> > Add suport for the physical timer registers in kvm_arm_timer_set_reg and\n> > kvm_arm_timer_get_reg so that these functions can be reused to interact\n> > with the rest of the system.\n> >\n> > Note that this paves part of the way for the physical timer state\n> > save/restore, but we still need to add those registers to\n> > KVM_GET_REG_LIST before we support migrating the physical timer state.\n> >\n> > Signed-off-by: Christoffer Dall \n> > ---\n> > arch/arm/include/uapi/asm/kvm.h | 6 ++++++\n> > arch/arm64/include/uapi/asm/kvm.h | 6 ++++++\n> > virt/kvm/arm/arch_timer.c | 33 +++++++++++++++++++++++++++++++--\n> > 3 files changed, 43 insertions(+), 2 deletions(-)\n> >\n> > diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h\n> > index 5db2d4c..665c454 100644\n> > --- a/arch/arm/include/uapi/asm/kvm.h\n> > +++ b/arch/arm/include/uapi/asm/kvm.h\n> > @@ -151,6 +151,12 @@ struct kvm_arch_memory_slot {\n> > \t(__ARM_CP15_REG(op1, 0, crm, 0) | KVM_REG_SIZE_U64)\n> > #define ARM_CP15_REG64(...) __ARM_CP15_REG64(__VA_ARGS__)\n> > \n> > +/* PL1 Physical Timer Registers */\n> > +#define KVM_REG_ARM_PTIMER_CTL\t\tARM_CP15_REG32(0, 14, 2, 1)\n> > +#define KVM_REG_ARM_PTIMER_CNT\t\tARM_CP15_REG64(0, 14)\n> > +#define KVM_REG_ARM_PTIMER_CVAL\t\tARM_CP15_REG64(2, 14)\n> > +\n> > +/* Virtual Timer Registers */\n> > #define KVM_REG_ARM_TIMER_CTL\t\tARM_CP15_REG32(0, 14, 3, 1)\n> > #define KVM_REG_ARM_TIMER_CNT\t\tARM_CP15_REG64(1, 14)\n> > #define KVM_REG_ARM_TIMER_CVAL\t\tARM_CP15_REG64(3, 14)\n> > diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h\n> > index 9f3ca24..07be6e2 100644\n> > --- a/arch/arm64/include/uapi/asm/kvm.h\n> > +++ b/arch/arm64/include/uapi/asm/kvm.h\n> > @@ -195,6 +195,12 @@ struct kvm_arch_memory_slot {\n> > \n> > #define ARM64_SYS_REG(...) (__ARM64_SYS_REG(__VA_ARGS__) | KVM_REG_SIZE_U64)\n> > \n> > +/* EL1 Physical Timer Registers */\n> \n> These are EL0 registers, even if we tend to restrict them to EL1. Even\n> the 32bit version is not strictly a PL1 register, since PL1 can delegate\n> it to userspace (but the ARMv7 ARM still carries this PL1 thing...).\n> \n\nThe latest publicly available ARM ARM also refers to the timer as the\n\"EL1 Physical Timer\", for example, the EL0 register CNTP_CTL_EL0 is\ndescribed as \"Control register for the EL1 physical timer\", so the\nassociativity in my comment was \"(EL1 Physical Timer) Registers\", and\nnot \"Physical Timer (EL1 Registers)\" :)\n\nHow about \"EL0 Registers for the EL1 Physical Timer\" or \"Physical Timer\nEL0 Registers\" or \"EL1 Physical Timer EL0 Registers\". Take your pick...\n\n> > +#define KVM_REG_ARM_PTIMER_CTL\t\tARM64_SYS_REG(3, 3, 14, 2, 1)\n> > +#define KVM_REG_ARM_PTIMER_CVAL\t\tARM64_SYS_REG(3, 3, 14, 2, 2)\n> > +#define KVM_REG_ARM_PTIMER_CNT\t\tARM64_SYS_REG(3, 3, 14, 0, 1)\n> > +\n> > +/* EL0 Virtual Timer Registers */\n> > #define KVM_REG_ARM_TIMER_CTL\t\tARM64_SYS_REG(3, 3, 14, 3, 1)\n> > #define KVM_REG_ARM_TIMER_CNT\t\tARM64_SYS_REG(3, 3, 14, 3, 2)\n> > #define KVM_REG_ARM_TIMER_CVAL\t\tARM64_SYS_REG(3, 3, 14, 0, 2)\n> > diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> > index 70110ea..d5b632d 100644\n> > --- a/virt/kvm/arm/arch_timer.c\n> > +++ b/virt/kvm/arm/arch_timer.c\n> > @@ -626,10 +626,11 @@ static void kvm_timer_init_interrupt(void *info)\n> > int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)\n> > {\n> > \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > +\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> > \n> > \tswitch (regid) {\n> > \tcase KVM_REG_ARM_TIMER_CTL:\n> > -\t\tvtimer->cnt_ctl = value;\n> > +\t\tvtimer->cnt_ctl = value & ~ARCH_TIMER_CTRL_IT_STAT;\n> \n> Ah, interesting. Does this change anything to userspace behaviour?\n> \n\nThe only effect is that you don't get read-as-written from userspace\nbehavior, but we don't guarantee that anywhere in the API and the\ncurrent QEMU code doesn't rely on it.\n\nIt can't have any meaningful effect, because ISTATUS is purely a\nfunction of the remaining state of the timer.\n\n> > \t\tbreak;\n> > \tcase KVM_REG_ARM_TIMER_CNT:\n> > \t\tupdate_vtimer_cntvoff(vcpu, kvm_phys_timer_read() - value);\n> > @@ -637,6 +638,13 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)\n> > \tcase KVM_REG_ARM_TIMER_CVAL:\n> > \t\tvtimer->cnt_cval = value;\n> > \t\tbreak;\n> > +\tcase KVM_REG_ARM_PTIMER_CTL:\n> > +\t\tptimer->cnt_ctl = value & ~ARCH_TIMER_CTRL_IT_STAT;\n> > +\t\tbreak;\n> > +\tcase KVM_REG_ARM_PTIMER_CVAL:\n> > +\t\tptimer->cnt_cval = value;\n> > +\t\tbreak;\n> > +\n> > \tdefault:\n> > \t\treturn -1;\n> > \t}\n> > @@ -645,17 +653,38 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)\n> > \treturn 0;\n> > }\n> > \n> > +static u64 read_timer_ctl(struct arch_timer_context *timer)\n> > +{\n> > +\t/*\n> > +\t * Set ISTATUS bit if it's expired.\n> > +\t * Note that according to ARMv8 ARM Issue A.k, ISTATUS bit is\n> > +\t * UNKNOWN when ENABLE bit is 0, so we chose to set ISTATUS bit\n> > +\t * regardless of ENABLE bit for our implementation convenience.\n> > +\t */\n> > +\tif (!kvm_timer_compute_delta(timer))\n> > +\t\treturn timer->cnt_ctl | ARCH_TIMER_CTRL_IT_STAT;\n> > +\telse\n> > +\t\treturn timer->cnt_ctl;\n> \n> Can't we end-up with a stale IT_STAT bit here if the timer has been\n> snapshoted with an interrupt pending, and then CVAL updated to expire\n> later?\n> \n\nYes, but that's just the nature of doing business with the timer, and no\ndifferent from the behavior we had before, where you could have run the\nguest, read the cnt_ctl as it was saved in the world-switch, run the\nVCPU again which changes cval, and then the bit would be stale.\n\nDo you see us changing that behavior in some worse way here?\n\n> > +}\n> > +\n> > u64 kvm_arm_timer_get_reg(struct kvm_vcpu *vcpu, u64 regid)\n> > {\n> > +\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> > \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > \n> > \tswitch (regid) {\n> > \tcase KVM_REG_ARM_TIMER_CTL:\n> > -\t\treturn vtimer->cnt_ctl;\n> > +\t\treturn read_timer_ctl(vtimer);\n> > \tcase KVM_REG_ARM_TIMER_CNT:\n> > \t\treturn kvm_phys_timer_read() - vtimer->cntvoff;\n> > \tcase KVM_REG_ARM_TIMER_CVAL:\n> > \t\treturn vtimer->cnt_cval;\n> > +\tcase KVM_REG_ARM_PTIMER_CTL:\n> > +\t\treturn read_timer_ctl(ptimer);\n> > +\tcase KVM_REG_ARM_PTIMER_CVAL:\n> > +\t\treturn ptimer->cnt_cval;\n> > +\tcase KVM_REG_ARM_PTIMER_CNT:\n> > +\t\treturn kvm_phys_timer_read();\n> > \t}\n> > \treturn (u64)-1;\n> > }\n> \n\nThanks,\n-Christoffer","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"SLkxB41J\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"jY5Zazjs\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yHhyZ0S1lz9t48\n\tfor ;\n\tThu, 19 Oct 2017 19:33:10 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e56GI-0002YE-BS; Thu, 19 Oct 2017 08:33:06 +0000","from mail-wm0-x241.google.com ([2a00:1450:400c:c09::241])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e56G5-0001xX-Gb for linux-arm-kernel@lists.infradead.org;\n\tThu, 19 Oct 2017 08:33:01 +0000","by mail-wm0-x241.google.com with SMTP id i124so14704886wmf.3\n\tfor ;\n\tThu, 19 Oct 2017 01:32:33 -0700 (PDT)","from localhost (xd93dd96b.cust.hiper.dk. [217.61.217.107])\n\tby smtp.gmail.com with ESMTPSA id\n\tj39sm10691320ede.10.2017.10.19.01.32.30\n\t(version=TLS1_2 cipher=AES128-SHA bits=128/128);\n\tThu, 19 Oct 2017 01:32:31 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:\n\tMessage-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=XQgRW5J+j8f2womvVZjbPuPE7PoW6k39S0B5Q56/oMw=;\n\tb=SLkxB41JDC7Tli\n\toIAmyWIbYZ3KtImLTU1NLZv/M9qrtRlnUwPAl9BriXk9WSTO9iGwhGeox4c51bi5cK8DpzHB5B+u7\n\tnNH2GGN6MtAtdc6a55XKwXLRy785+5jAQuxXluE0fwh/ZuVSI0+XVJOLJLhouyyHnOjoI3jzqVlNo\n\tbJbbkX2rvAy/c1+dcuiWzD0gr/+U2q1CFEq9WgtdBJP+DHugqptwTfkkzlEbMwkE6V1dmp9KLSLOa\n\taU38JorJ7NHnC/FCgcZVuI+Sr7XuOj9X9+dsO9A8ngpBeLCKxyQZQA0yyXvzQE2hxjiy4KdSSZ0NM\n\tR/i9Y/1wXvT6g0yK5jCQ==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=ObtegTrYe+JyjGt52fiqJ9XjzRt70xI7UtSULGkrYA4=;\n\tb=jY5Zazjs7rEd+Hjjg3vVrqGnN7VS9VVrIWv4cB9GObRVcOLRXgn2EKRsKOWIUmbobr\n\tSkbA7opjL3putIDjl2C8TGP42l5wRbZ4Wd+GjP/NG3cZvINcWoJrFIEwVXlVbUz0VYUF\n\tYXMEAKIwkE/T1n4vqqRHlUulwDyxS3NxEc79k="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=ObtegTrYe+JyjGt52fiqJ9XjzRt70xI7UtSULGkrYA4=;\n\tb=C/UzvvTu+k2rWPpcPGU23cE1mZkgJQ2g3KWE2rP71pk+jSVdOQB1yAuijSpvBZLwD6\n\t1lm3ihrRSP9IekVECMO7iJoylqbZTJYO0MHhieubfPipE0im4Vo6xPd22kGIjGhOvvj5\n\tEy2UIbk7xGs8R3LfI9kJ+PWQUCmsMn14AL3i5vfO8RvWNT4uscyCjiIIM3MwyVQXNtGz\n\twlKFjgXfFZkiSlWqmYzGIEvE04fh+im0WzlG7r1wFIgl579QeP7QCpAb371qvD7hO5jQ\n\tSSJTXihFC69DGHXlFvv9R2taZw8DgbJD4eg91q5+cMpCuCsXFMC7JTBMMp9Q+sGvXoFf\n\teRsQ==","X-Gm-Message-State":"AMCzsaWUF/7fPOv3N2IUG54pljS7RVREAY5cdV+98I5rl/kp+/Mml09Q\n\tpvzf7OlXBGh1W1ReA3tc6phWyw==","X-Google-Smtp-Source":"ABhQp+Qx2JtPOiT8DR/GH/KykrYcLEfBsrlP1KaM1rwzoMP0wypDkYILY/5oiflHVq9YxGpWyv/uiQ==","X-Received":"by 10.80.185.68 with SMTP id m62mr1473977ede.239.1508401951808; \n\tThu, 19 Oct 2017 01:32:31 -0700 (PDT)","Date":"Thu, 19 Oct 2017 10:32:33 +0200","From":"Christoffer Dall ","To":"Marc Zyngier ","Subject":"Re: [PATCH v3 15/20] KVM: arm/arm64: Support EL1 phys timer register\n\taccess in set/get reg","Message-ID":"<20171019083233.GP8900@cbox>","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-16-cdall@linaro.org>\n\t<87infnwepo.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<87infnwepo.fsf@on-the-bus.cambridge.arm.com>","User-Agent":"Mutt/1.5.21 (2010-09-15)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171019_013254_382761_5220684C ","X-CRM114-Status":"GOOD ( 28.84 )","X-Spam-Score":"-2.0 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.0 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1790238,"web_url":"http://patchwork.ozlabs.org/comment/1790238/","msgid":"<20171019084443.GQ8900@cbox>","list_archive_url":null,"date":"2017-10-19T08:44:43","subject":"Re: [PATCH v3 18/20] KVM: arm/arm64: Avoid phys timer emulation in\n\tvcpu entry/exit","submitter":{"id":71350,"url":"http://patchwork.ozlabs.org/api/people/71350/","name":"Christoffer Dall","email":"cdall@linaro.org"},"content":"On Tue, Oct 10, 2017 at 10:45:15AM +0100, Marc Zyngier wrote:\n> On Sat, Sep 23 2017 at 2:42:05 am BST, Christoffer Dall wrote:\n> > There is no need to schedule and cancel a hrtimer when entering and\n> > exiting the guest, because we know when the physical timer is going to\n> > fire when the guest programs it, and we can simply program the hrtimer\n> > at that point.\n> >\n> > Now when the register modifications from the guest go through the\n> > kvm_arm_timer_set/get_reg functions, which always call\n> > kvm_timer_update_state(), we can simply consider the timer state in this\n> > function and schedule and cancel the timers as needed.\n> >\n> > This avoids looking at the physical timer emulation state when entering\n> > and exiting the VCPU, allowing for faster servicing of the VM when\n> > needed.\n> >\n> > Signed-off-by: Christoffer Dall \n> > ---\n> > virt/kvm/arm/arch_timer.c | 75 ++++++++++++++++++++++++++++++++---------------\n> > 1 file changed, 51 insertions(+), 24 deletions(-)\n> >\n> > diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c\n> > index 1f82c21..aa18a5d 100644\n> > --- a/virt/kvm/arm/arch_timer.c\n> > +++ b/virt/kvm/arm/arch_timer.c\n> > @@ -199,7 +199,27 @@ static enum hrtimer_restart kvm_bg_timer_expire(struct hrtimer *hrt)\n> > \n> > static enum hrtimer_restart kvm_phys_timer_expire(struct hrtimer *hrt)\n> > {\n> > -\tWARN(1, \"Timer only used to ensure guest exit - unexpected event.\");\n> > +\tstruct arch_timer_context *ptimer;\n> > +\tstruct arch_timer_cpu *timer;\n> > +\tstruct kvm_vcpu *vcpu;\n> > +\tu64 ns;\n> > +\n> > +\ttimer = container_of(hrt, struct arch_timer_cpu, phys_timer);\n> > +\tvcpu = container_of(timer, struct kvm_vcpu, arch.timer_cpu);\n> > +\tptimer = vcpu_ptimer(vcpu);\n> > +\n> > +\t/*\n> > +\t * Check that the timer has really expired from the guest's\n> > +\t * PoV (NTP on the host may have forced it to expire\n> > +\t * early). If not ready, schedule for a later time.\n> > +\t */\n> > +\tns = kvm_timer_compute_delta(ptimer);\n> > +\tif (unlikely(ns)) {\n> > +\t\thrtimer_forward_now(hrt, ns_to_ktime(ns));\n> > +\t\treturn HRTIMER_RESTART;\n> > +\t}\n> \n> Don't we already have a similar logic for the background timer (I must\n> admit I've lost track of how we changed things in this series)? If so,\n> can we make this common code?\n> \n\nI looked at it, but the functions are slightly different. In the\nphys_timer, we just have to calculate the delta and figure out if we\nneed to restart it. In the bg_timer, we have to compute the delta of\nboth the vtimer and the ptimer, figure out the earliest one, and figure\nout if we have to restart it. Therefore, kvm_timer_earliest_exp()\nalrady calls kvm_timer_compute_delate() to share code, and the only code\nwe can really share is:\n\n\tif (unlikely(ns)) {\n\t\thrtimer_forward_now(hrt, ns_to_ktime(ns));\n\t\treturn HRTIMER_RESTART;\n\t}\n\nThe following code also cannot really be shared because in one case we\nhave to schedule work (the bg timer) and in the other case we have to\ninject an irq (the phys timer). Ther alternative would be:\n\nstatic enum hrtimer_restart kvm_soft_timer_expire(struct hrtimer *hrt, bool is_bg_timer)\n{\n\tstruct arch_timer_context *ptimer;\n\tstruct arch_timer_cpu *timer;\n\tstruct kvm_vcpu *vcpu;\n\tu64 ns;\n\n\tif (is_bg_timer)\n\t\ttimer = container_of(hrt, struct arch_timer_cpu, bg_timer);\n\telse\n\t\ttimer = container_of(hrt, struct arch_timer_cpu, phys_timer);\n\tvcpu = container_of(timer, struct kvm_vcpu, arch.timer_cpu);\n\tptimer = vcpu_ptimer(vcpu);\n\n\t/*\n\t * Check that the timer has really expired from the guest's\n\t * PoV (NTP on the host may have forced it to expire\n\t * early). If we should have slept longer, restart it.\n\t */\n\tif (is_bg_timer)\n\t\tns = kvm_timer_earliest_exp(vcpu);\n\telse\n\t\tns = kvm_timer_compute_delta(ptimer);\n\tif (unlikely(ns)) {\n\t\thrtimer_forward_now(hrt, ns_to_ktime(ns));\n\t\treturn HRTIMER_RESTART;\n\t}\n\n\tif (is_bg_timer)\n\t\tschedule_work(&timer->expired);\n\telse\n\t\tkvm_timer_update_irq(vcpu, true, ptimer);\n\treturn HRTIMER_NORESTART;\n}\n\nBut I prefer just having them separate.\n\n\n> > +\n> > +\tkvm_timer_update_irq(vcpu, true, ptimer);\n> > \treturn HRTIMER_NORESTART;\n> > }\n> > \n> > @@ -253,24 +273,28 @@ static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,\n> > }\n> > \n> > /* Schedule the background timer for the emulated timer. */\n> > -static void phys_timer_emulate(struct kvm_vcpu *vcpu,\n> > -\t\t\t struct arch_timer_context *timer_ctx)\n> > +static void phys_timer_emulate(struct kvm_vcpu *vcpu)\n> > {\n> > \tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> > +\tstruct arch_timer_context *ptimer = vcpu_ptimer(vcpu);\n> > \n> > -\tif (kvm_timer_should_fire(timer_ctx))\n> > -\t\treturn;\n> > -\n> > -\tif (!kvm_timer_irq_can_fire(timer_ctx))\n> > +\t/*\n> > +\t * If the timer can fire now we have just raised the IRQ line and we\n> > +\t * don't need to have a soft timer scheduled for the future. If the\n> > +\t * timer cannot fire at all, then we also don't need a soft timer.\n> > +\t */\n> > +\tif (kvm_timer_should_fire(ptimer) || !kvm_timer_irq_can_fire(ptimer)) {\n> > +\t\tsoft_timer_cancel(&timer->phys_timer, NULL);\n> > \t\treturn;\n> > +\t}\n> > \n> > -\t/* The timer has not yet expired, schedule a background timer */\n> > -\tsoft_timer_start(&timer->phys_timer, kvm_timer_compute_delta(timer_ctx));\n> > +\tsoft_timer_start(&timer->phys_timer, kvm_timer_compute_delta(ptimer));\n> > }\n> > \n> > /*\n> > - * Check if there was a change in the timer state (should we raise or lower\n> > - * the line level to the GIC).\n> > + * Check if there was a change in the timer state, so that we should either\n> > + * raise or lower the line level to the GIC or schedule a background timer to\n> > + * emulate the physical timer.\n> > */\n> > static void kvm_timer_update_state(struct kvm_vcpu *vcpu)\n> > {\n> > @@ -292,6 +316,8 @@ static void kvm_timer_update_state(struct kvm_vcpu *vcpu)\n> > \n> > \tif (kvm_timer_should_fire(ptimer) != ptimer->irq.level)\n> > \t\tkvm_timer_update_irq(vcpu, !ptimer->irq.level, ptimer);\n> > +\n> > +\tphys_timer_emulate(vcpu);\n> > }\n> > \n> > static void vtimer_save_state(struct kvm_vcpu *vcpu)\n> > @@ -445,6 +471,9 @@ void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu)\n> > \n> > \tif (has_vhe())\n> > \t\tdisable_el1_phys_timer_access();\n> > +\n> > +\t/* Set the background timer for the physical timer emulation. */\n> > +\tphys_timer_emulate(vcpu);\n> > }\n> > \n> > bool kvm_timer_should_notify_user(struct kvm_vcpu *vcpu)\n> > @@ -480,12 +509,6 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)\n> > \n> > \tif (unlikely(!timer->enabled))\n> > \t\treturn;\n> > -\n> > -\tif (kvm_timer_should_fire(ptimer) != ptimer->irq.level)\n> > -\t\tkvm_timer_update_irq(vcpu, !ptimer->irq.level, ptimer);\n> > -\n> > -\t/* Set the background timer for the physical timer emulation. */\n> > -\tphys_timer_emulate(vcpu, vcpu_ptimer(vcpu));\n> > }\n> > \n> > void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)\n> > @@ -500,6 +523,17 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)\n> > \n> > \tvtimer_save_state(vcpu);\n> > \n> > +\t/*\n> > +\t * Cancel the physical timer emulation, because the only case where we\n> > +\t * need it after a vcpu_put is in the context of a sleeping VCPU, and\n> > +\t * in that case we already factor in the deadline for the physical\n> > +\t * timer when scheduling the bg_timer.\n> > +\t *\n> > +\t * In any case, we re-schedule the hrtimer for the physical timer when\n> > +\t * coming back to the VCPU thread in kvm_timer_vcpu_load().\n> > +\t */\n> > +\tsoft_timer_cancel(&timer->phys_timer, NULL);\n> > +\n> > \tset_cntvoff(0);\n> > }\n> > \n> > @@ -536,16 +570,9 @@ static void unmask_vtimer_irq(struct kvm_vcpu *vcpu)\n> > */\n> > void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)\n> > {\n> > -\tstruct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;\n> > \tstruct arch_timer_context *vtimer = vcpu_vtimer(vcpu);\n> > \n> > \t/*\n> > -\t * This is to cancel the background timer for the physical timer\n> > -\t * emulation if it is set.\n> > -\t */\n> > -\tsoft_timer_cancel(&timer->phys_timer, NULL);\n> > -\n> > -\t/*\n> > \t * If we entered the guest with the vtimer output asserted we have to\n> > \t * check if the guest has modified the timer so that we should lower\n> > \t * the line at this point.\n> \n> Otherwise:\n> \n> Reviewed-by: Marc Zyngier \n> \nThanks,\n-Christoffer","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"qgxFGq8J\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"a40KaQLd\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yHjDd2Vpzz9t38\n\tfor ;\n\tThu, 19 Oct 2017 19:45:17 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e56Rz-0001Wl-6K; Thu, 19 Oct 2017 08:45:11 +0000","from mail-wm0-x242.google.com ([2a00:1450:400c:c09::242])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e56Rs-0000VA-M7 for linux-arm-kernel@lists.infradead.org;\n\tThu, 19 Oct 2017 08:45:09 +0000","by mail-wm0-x242.google.com with SMTP id t69so14479480wmt.2\n\tfor ;\n\tThu, 19 Oct 2017 01:44:44 -0700 (PDT)","from localhost (xd93dd96b.cust.hiper.dk. [217.61.217.107])\n\tby smtp.gmail.com with ESMTPSA id\n\td60sm9247111edc.44.2017.10.19.01.44.40\n\t(version=TLS1_2 cipher=AES128-SHA bits=128/128);\n\tThu, 19 Oct 2017 01:44:40 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:\n\tMessage-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=aDlijD9fBJ0/H+5b2HxK+N72MZQKU32TTlETUkX3QLU=;\n\tb=qgxFGq8Jd+wYn7\n\tk91TNTUkC3pWzHg2zB/yJPEor2tGj5viogg/0ns7IzyTy7rOSyUl5xtHlS6V6Gp2PTDntZAaRJWwG\n\tVwQsSyigWSlO9VovsIodXq2v/6m74MLyjfpUdrGQq6poTWpPU/ckR2x6RTKzQjTF+jXQxJlJn8A/j\n\tdqvr0gMFVt2KojBD52mG8u5YuMnFR0JVF/THiqUdiqHSgorTvIfQ7MayOTpF79i/7c6UFNiQo6G4i\n\tY9SvP3dkNHetnrWuFuKsjLX56gTnMNf0jL5rrLq/uUIufidougAZ2Iny+KsC/7BnG6AGbTvs6qhjD\n\tXQZQklfXS091TAQwgPbw==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=NNiDFm6NXYccUerwb9kmF6hk+m6Oz/SGlHnEjoNBFic=;\n\tb=a40KaQLdZUqkiU0pF1kjUPwS8msYOyUpV6DBMDLqCOGJ9k4fJ7ICIdbmzZniKnKXMD\n\ty9JQ6przentv39CLVr0fVbrypzu9yh0y2sNY0nFYy+h5jWxZEeZ9UFeH/dpDUAQGGOoO\n\t3QindY/eMYS8GFO4Itb0WrhlG2Xl7caHXrrf0="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=NNiDFm6NXYccUerwb9kmF6hk+m6Oz/SGlHnEjoNBFic=;\n\tb=It0V/srB4Q9NhGsMWBdgQ4zvXluJ9e59X3XuvCFz2KB+st2uJwLCgP/KM+UPAlCt7o\n\tKb5AbY4qgy1/iJ5fKYE2Yu2A4ObZbi2RsIhrO8pNYsdIWPFYOGU+7wTVEX+QzcocAVRl\n\tutC2vtCjG+9D9tBmdbEnJwuIIWoOh8PB8BXcrHPSCdrIit/IdVJVsAEFi8k/poVYnySM\n\tRmbFudEioQewGeOi8VsncVyVA6qjuzYCz1A7JWKXd4dPUY/GOydbNkdu5RffwCtEuHq/\n\tEJzUlz+DrwoeQw7lak8XRVjYMhPX0mZq17K3vUcQ58Bg4+fwK5Cgqz+KMe+gx7ALEQ5c\n\t/qUg==","X-Gm-Message-State":"AMCzsaWmMlHB8c65EU4TeHGQPA1zxj4DzKqDiWro4LoTWgSjmpghuh5K\n\tDRvWWjxyEOgzY/v2gK4vmU7vcg==","X-Google-Smtp-Source":"ABhQp+TMaiPrWDQ/3xZbGCvwAvBgfi15N0lq4MGZKqGhm/xI8rh3gcoR4FdPzi3nALkdSZmjiy6R+A==","X-Received":"by 10.80.194.1 with SMTP id n1mr1469520edf.1.1508402682782;\n\tThu, 19 Oct 2017 01:44:42 -0700 (PDT)","Date":"Thu, 19 Oct 2017 10:44:43 +0200","From":"Christoffer Dall ","To":"Marc Zyngier ","Subject":"Re: [PATCH v3 18/20] KVM: arm/arm64: Avoid phys timer emulation in\n\tvcpu entry/exit","Message-ID":"<20171019084443.GQ8900@cbox>","References":"<20170923004207.22356-1-cdall@linaro.org>\n\t<20170923004207.22356-19-cdall@linaro.org>\n\t<8760bnwd3o.fsf@on-the-bus.cambridge.arm.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<8760bnwd3o.fsf@on-the-bus.cambridge.arm.com>","User-Agent":"Mutt/1.5.21 (2010-09-15)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171019_014505_058206_24C4507E ","X-CRM114-Status":"GOOD ( 32.62 )","X-Spam-Score":"-2.0 (--)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-2.0 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,\n\tno\n\ttrust [2a00:1450:400c:c09:0:0:0:242 listed in] [list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Catalin Marinas ,\n\tWill Deacon , kvmarm@lists.cs.columbia.edu,\n\tlinux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}}]