From patchwork Wed May 26 05:15:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1483869 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FqfLG1vYhz9sW8; Wed, 26 May 2021 15:16:05 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1llltg-0001gi-H2; Wed, 26 May 2021 05:16:00 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1lllte-0001gD-7q for kernel-team@lists.ubuntu.com; Wed, 26 May 2021 05:15:58 +0000 Received: from mail-ed1-f72.google.com ([209.85.208.72]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1llltd-0004Cd-Vv for kernel-team@lists.ubuntu.com; Wed, 26 May 2021 05:15:58 +0000 Received: by mail-ed1-f72.google.com with SMTP id cn20-20020a0564020cb4b029038d0b0e183fso18657396edb.22 for ; Tue, 25 May 2021 22:15:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iQ9mK0vq5EBhFUEALzcqcs+T1NcRxmtIdIcd7EsOF5E=; b=chDlY1jcbNNnCv/vgF27vErWUOTw6Lj1PBXgEn/dcvqGb4m8D1HRHEqxmz1Ar+uGsJ QfuXBgm2x5vIXr6ispjZa49lWCKJ97180G6CM3cR/P++jh4PXjJrKVNUJuoSIKGxCADU 9dteIRNIfSGwMaFAG0+8NP0vz3f0+IzDDvaYVbhNq06AFtN1OPXypaj8EYEdJaDzqBB5 31kC8K/M46gXoWGVecO8CkAoJ/P0keo9FoA7Kh96bC3y1+1frtXueQEc3zUOSzA8rzu0 wptsh+/TdUQ2NztOOGtsm1cMpuZilK8RR+OaoG3g74eW3BewoX9zyeU3eGXo2dib3Kb9 158A== X-Gm-Message-State: AOAM531TtbiRBwB1Wxxf7QUuBuJqqHLAEMfyEGtBDPxWOn5mr9bV6Xf3 MSNcklUtAo497i3PkCnGlT87w9/2pQwjPW2z0MbEj3e0JtDv3ze2rFgxFoBTAFbL25ids58HI9D EbPTt7Cr01pWg0lMOzG+8iVfflV732SwnRWD66IM3og== X-Received: by 2002:a05:6402:203c:: with SMTP id ay28mr35264897edb.100.1622006157754; Tue, 25 May 2021 22:15:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzDPEno4ums0Y14JffCGGzUshPCC5D0/liqaU0lOwkkAveZkj7eh8IUEKmbBtQN+ODKjy5Pzw== X-Received: by 2002:a05:6402:203c:: with SMTP id ay28mr35264877edb.100.1622006157579; Tue, 25 May 2021 22:15:57 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id z4sm11704098edc.1.2021.05.25.22.15.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 May 2021 22:15:57 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH v2 1/5] x86/kvm: Fix pr_info() for async PF setup/teardown Date: Wed, 26 May 2021 07:15:43 +0200 Message-Id: <20210526051547.217632-2-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210526051547.217632-1-andrea.righi@canonical.com> References: <20210526051547.217632-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov BugLink: https://bugs.launchpad.net/bugs/1920944 'pr_fmt' already has 'kvm-guest: ' so 'KVM' prefix is redundant. "Unregister pv shared memory" is very ambiguous, it's hard to say which particular PV feature it relates to. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-2-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (backported from commit 0a269a008f837e76ce285679ab3005059fadc2a6) Signed-off-by: Andrea Righi --- arch/x86/kernel/kvm.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index e820568ed4d5..ee595588b98a 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -321,8 +321,7 @@ static void kvm_guest_cpu_init(void) wrmsrl(MSR_KVM_ASYNC_PF_EN, pa); __this_cpu_write(apf_reason.enabled, 1); - printk(KERN_INFO"KVM setup async PF for cpu %d\n", - smp_processor_id()); + pr_info("setup async PF for cpu %d\n", smp_processor_id()); } if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) { @@ -347,8 +346,7 @@ static void kvm_pv_disable_apf(void) wrmsrl(MSR_KVM_ASYNC_PF_EN, 0); __this_cpu_write(apf_reason.enabled, 0); - printk(KERN_INFO"Unregister pv shared memory for cpu %d\n", - smp_processor_id()); + pr_info("disable async PF for cpu %d\n", smp_processor_id()); } static void kvm_pv_guest_cpu_reboot(void *unused) From patchwork Wed May 26 05:15:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1483867 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FqfLG1j93z9sVt; Wed, 26 May 2021 15:16:05 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1llltg-0001gu-PG; Wed, 26 May 2021 05:16:00 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1llltf-0001gJ-4D for kernel-team@lists.ubuntu.com; Wed, 26 May 2021 05:15:59 +0000 Received: from mail-ed1-f71.google.com ([209.85.208.71]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1lllte-0004Cq-SL for kernel-team@lists.ubuntu.com; Wed, 26 May 2021 05:15:58 +0000 Received: by mail-ed1-f71.google.com with SMTP id i3-20020aa7dd030000b029038ce772ffe4so18764138edv.12 for ; Tue, 25 May 2021 22:15:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aQeTYROlqEIR450wBaubK1KRGHXVtkadaqFYqbA58hU=; b=tS2xjXKQTsvVekbB+jsAlT7CtAPU2mamiR2YeHDARPQFGQ+a4oudcvZGhp8kbKRsAf AkXQDAwUwK/H9SxwZx5cMFpdnJrZMP4DHceZYV4ZxbxexlDDe7bldvqz6KB+6tmhwP+A ksO/WfwmOwHQU/yKonQ4Jziy3P9NQJ6x/o9DDkMOD1JNEDCOE2u4lmXpqMyCZUzkuKMc LT8oC2BL/qs6vwiqzUJ8HsVoD/3eX3kksIvoBayBq7KB6yrUxHM/IJp7EgpNfqlIYoDd gJH6wGP/hdbk2OLGguk8Z2Gjo81ZTuUxSGDK3MGQDMPmqIg3/2gTDV43Y3ZLcE4UOf0q jYJQ== X-Gm-Message-State: AOAM530S654eObhhvmsEoCfAiWu8CQc+PTm6IXuNv92uTTaXmtBR0/S4 tD9mIKb+iiKeZAbNhsy6Sw+qFXA/k7yQFnLMJtzKZaBz0+H3fXCPSEPY5PYwmt2nFzjFZdEe3sh rItVFQpY5y7okpsuEUHiwuXDFo7jbMyA/4P1jk2fOLA== X-Received: by 2002:a50:ab5e:: with SMTP id t30mr35920583edc.50.1622006158594; Tue, 25 May 2021 22:15:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy318NKZFUs6Dhy+UwqLI5vtmVIrTTLPZtNl+Hk0D2criw+6thFOjTPQA+94yFdAHudT316nA== X-Received: by 2002:a50:ab5e:: with SMTP id t30mr35920570edc.50.1622006158375; Tue, 25 May 2021 22:15:58 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id z4sm11704098edc.1.2021.05.25.22.15.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 May 2021 22:15:57 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH v2 2/5] x86/kvm: Teardown PV features on boot CPU as well Date: Wed, 26 May 2021 07:15:44 +0200 Message-Id: <20210526051547.217632-3-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210526051547.217632-1-andrea.righi@canonical.com> References: <20210526051547.217632-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov BugLink: https://bugs.launchpad.net/bugs/1920944 Various PV features (Async PF, PV EOI, steal time) work through memory shared with hypervisor and when we restore from hibernation we must properly teardown all these features to make sure hypervisor doesn't write to stale locations after we jump to the previously hibernated kernel (which can try to place anything there). For secondary CPUs the job is already done by kvm_cpu_down_prepare(), register syscore ops to do the same for boot CPU. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-3-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (backported from commit 8b79feffeca28c5459458fe78676b081e87c93a4) Signed-off-by: Andrea Righi --- arch/x86/kernel/kvm.c | 57 +++++++++++++++++++++++++++++++------------ 1 file changed, 41 insertions(+), 16 deletions(-) diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index ee595588b98a..29082944956b 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -426,6 +427,25 @@ static void __init sev_map_percpu_data(void) } } +static void kvm_guest_cpu_offline(void) +{ + kvm_disable_steal_time(); + if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) + wrmsrl(MSR_KVM_PV_EOI_EN, 0); + kvm_pv_disable_apf(); + apf_task_wake_all(); +} + +static int kvm_cpu_online(unsigned int cpu) +{ + unsigned long flags; + + local_irq_save(flags); + kvm_guest_cpu_init(); + local_irq_restore(flags); + return 0; +} + #ifdef CONFIG_SMP #define KVM_IPI_CLUSTER_SIZE (2 * BITS_PER_LONG) @@ -545,31 +565,34 @@ static void __init kvm_smp_prepare_boot_cpu(void) kvm_spinlock_init(); } -static void kvm_guest_cpu_offline(void) +static int kvm_cpu_down_prepare(unsigned int cpu) { - kvm_disable_steal_time(); - if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) - wrmsrl(MSR_KVM_PV_EOI_EN, 0); - kvm_pv_disable_apf(); - apf_task_wake_all(); -} + unsigned long flags; -static int kvm_cpu_online(unsigned int cpu) -{ - local_irq_disable(); - kvm_guest_cpu_init(); - local_irq_enable(); + local_irq_save(flags); + kvm_guest_cpu_offline(); + local_irq_restore(flags); return 0; } -static int kvm_cpu_down_prepare(unsigned int cpu) +#endif + +static int kvm_suspend(void) { - local_irq_disable(); kvm_guest_cpu_offline(); - local_irq_enable(); + return 0; } -#endif + +static void kvm_resume(void) +{ + kvm_cpu_online(raw_smp_processor_id()); +} + +static struct syscore_ops kvm_syscore_ops = { + .suspend = kvm_suspend, + .resume = kvm_resume, +}; static void __init kvm_apf_trap_init(void) { @@ -647,6 +670,8 @@ static void __init kvm_guest_init(void) kvm_guest_cpu_init(); #endif + register_syscore_ops(&kvm_syscore_ops); + /* * Hard lockup detection is enabled by default. Disable it, as guests * can get false positives too easily, for example if the host is From patchwork Wed May 26 05:15:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1483871 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FqfLG3MLGz9sWT; Wed, 26 May 2021 15:16:05 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1llltg-0001hC-UE; Wed, 26 May 2021 05:16:00 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1llltf-0001gY-W1 for kernel-team@lists.ubuntu.com; Wed, 26 May 2021 05:15:59 +0000 Received: from mail-ed1-f69.google.com ([209.85.208.69]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1llltf-0004D0-Nl for kernel-team@lists.ubuntu.com; Wed, 26 May 2021 05:15:59 +0000 Received: by mail-ed1-f69.google.com with SMTP id x3-20020a50ba830000b029038caed0dd2eso18678521ede.7 for ; Tue, 25 May 2021 22:15:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=H0642PfIAa7cyDD8OKUv04YPqH7qULLAc61bNuB6j3A=; b=tJDwe1zBZoCjaTtyZX/WIEWlYunjNsormbWWT43tQaZAxgeCdSvqzaj9YWAs8zITLQ XAZFF+4t3fUFC5JlPVQSkAxu35M6svpSC3DS/FCb5gz5t9z9k/EN1t3jpeZAnENmTPEi ZenvM2WRprQUJiMsEE59DrbLmTPOtAXEyKFdT6o5U1jFI1aC42gI0X63vGYvqArPx/v2 HsNWAQrmi7CI85e3f6YWJ8Ys5hXHExqVQ43tA4O/Yhv0HYcffD+3LuFrLqfgEMPpx3zW VH22rc0ajLWL998BS6TSmgVWxio19uDtIwtBhqp+406SpqQyT0Lk8FraWXSQ1xIcKOGS z1YA== X-Gm-Message-State: AOAM531VbOceaTdRk+3XLNQ+lcGBGV9fQbaYQFpRveC3KpMbfC1XMPXR mo0xiEhD3Y78vFzExIc9X3s52ZLgedBHJQYCX3FJ4bI91mZBlDWih0Z5jdOp1lzsBFZO6jjpZ2m l5xuMHcUaK8HFRHOxZgGMeU9M+Dvaux7tgLmLcXHXxQ== X-Received: by 2002:a05:6402:10c9:: with SMTP id p9mr6789746edu.370.1622006159480; Tue, 25 May 2021 22:15:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyX0ERu9MH+YhdIC0iP+eUkEIE7HaJ0YmeIkn0SLwuCBeI5lDHNO/cJlP9f2ez9Evd1e1cNUw== X-Received: by 2002:a05:6402:10c9:: with SMTP id p9mr6789723edu.370.1622006159162; Tue, 25 May 2021 22:15:59 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id z4sm11704098edc.1.2021.05.25.22.15.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 May 2021 22:15:58 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH v2 3/5] x86/kvm: Disable kvmclock on all CPUs on shutdown Date: Wed, 26 May 2021 07:15:45 +0200 Message-Id: <20210526051547.217632-4-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210526051547.217632-1-andrea.righi@canonical.com> References: <20210526051547.217632-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov BugLink: https://bugs.launchpad.net/bugs/1920944 Currenly, we disable kvmclock from machine_shutdown() hook and this only happens for boot CPU. We need to disable it for all CPUs to guard against memory corruption e.g. on restore from hibernate. Note, writing '0' to kvmclock MSR doesn't clear memory location, it just prevents hypervisor from updating the location so for the short while after write and while CPU is still alive, the clock remains usable and correct so we don't need to switch to some other clocksource. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-4-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (cherry picked from commit c02027b5742b5aa804ef08a4a9db433295533046) Signed-off-by: Andrea Righi --- arch/x86/include/asm/kvm_para.h | 4 ++-- arch/x86/kernel/kvm.c | 1 + arch/x86/kernel/kvmclock.c | 5 +---- 3 files changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h index 9b4df6eaa11a..a617fd360023 100644 --- a/arch/x86/include/asm/kvm_para.h +++ b/arch/x86/include/asm/kvm_para.h @@ -6,8 +6,6 @@ #include #include -extern void kvmclock_init(void); - #ifdef CONFIG_KVM_GUEST bool kvm_check_and_clear_guest_paused(void); #else @@ -85,6 +83,8 @@ static inline long kvm_hypercall4(unsigned int nr, unsigned long p1, } #ifdef CONFIG_KVM_GUEST +void kvmclock_init(void); +void kvmclock_disable(void); bool kvm_para_available(void); unsigned int kvm_arch_para_features(void); unsigned int kvm_arch_para_hints(void); diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 29082944956b..35cd7baf2ad7 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -434,6 +434,7 @@ static void kvm_guest_cpu_offline(void) wrmsrl(MSR_KVM_PV_EOI_EN, 0); kvm_pv_disable_apf(); apf_task_wake_all(); + kvmclock_disable(); } static int kvm_cpu_online(unsigned int cpu) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 904494b924c1..bd3962953f78 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -214,11 +214,9 @@ static void kvm_crash_shutdown(struct pt_regs *regs) } #endif -static void kvm_shutdown(void) +void kvmclock_disable(void) { native_write_msr(msr_kvm_system_time, 0, 0); - kvm_disable_steal_time(); - native_machine_shutdown(); } static void __init kvmclock_init_mem(void) @@ -346,7 +344,6 @@ void __init kvmclock_init(void) #endif x86_platform.save_sched_clock_state = kvm_save_sched_clock_state; x86_platform.restore_sched_clock_state = kvm_restore_sched_clock_state; - machine_ops.shutdown = kvm_shutdown; #ifdef CONFIG_KEXEC_CORE machine_ops.crash_shutdown = kvm_crash_shutdown; #endif From patchwork Wed May 26 05:15:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1483870 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FqfLG2LQ7z9sWD; Wed, 26 May 2021 15:16:05 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1lllti-0001iI-4r; Wed, 26 May 2021 05:16:02 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1llltg-0001gq-Ol for kernel-team@lists.ubuntu.com; Wed, 26 May 2021 05:16:00 +0000 Received: from mail-ej1-f70.google.com ([209.85.218.70]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1llltg-0004DE-EX for kernel-team@lists.ubuntu.com; Wed, 26 May 2021 05:16:00 +0000 Received: by mail-ej1-f70.google.com with SMTP id z6-20020a17090665c6b02903700252d1ccso46038ejn.10 for ; Tue, 25 May 2021 22:16:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0/kdgeExGSLxIBfMIgJVyeUjnpApk5pLqjSp2YnJNi4=; b=m/VjvmCkhdxIehZT7Fe8LMRQrnnO9pnFum5ADmfyzeAcIHcDCab+P5evyrdWQMc4hk zQ8nf3NEqoh2Wkuhrldpn1uZCyFiYSB59g6Of6t5sGSwHMv6HViF83QG69gcasO8vCVx +KrziKWM0FlBt+VkkIfx3q27AUNCKHlg/iZ3sJsoCsq6edbGSE5u2xepuxQqNEyAg3U6 MTFsmQHUIPhETl/Ow0bzKaRAHDjbzYpRXE3BFmiwy3c16OLg12HJZrtpI6lZ1QpnYvU0 fpabjCAr0g7PdOWLlnkyMFBXGUmCtjX1ZpwuVd/RwVRV5DL+IosQXC/CMXWWOa/23C5j hJ+Q== X-Gm-Message-State: AOAM5305Dctfnht/YuDmxGqpg8W0AXOfEc2IgRMM+3Bmhw8OFiATDmYQ 7BER7agzjvxONylQdNn6jzHE4n17Zo7C4THNM1XBDsEImV65OhmP20bTINLGJaYVX1NlukjtSnQ jMYbjllu7UCOFgRIbF8axhpmb1v3QCgJyPoXlf5/W9A== X-Received: by 2002:a05:6402:14c1:: with SMTP id f1mr34654462edx.334.1622006160160; Tue, 25 May 2021 22:16:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyMoQ0sGzWtjlkgt6vy7IM9JYYML2qN1CrxJNZnqT1mhLTqAjGVKDnnY3g62D18bLbs9zP96w== X-Received: by 2002:a05:6402:14c1:: with SMTP id f1mr34654441edx.334.1622006159882; Tue, 25 May 2021 22:15:59 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id z4sm11704098edc.1.2021.05.25.22.15.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 May 2021 22:15:59 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH v2 4/5] x86/kvm: Disable all PV features on crash Date: Wed, 26 May 2021 07:15:46 +0200 Message-Id: <20210526051547.217632-5-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210526051547.217632-1-andrea.righi@canonical.com> References: <20210526051547.217632-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov BugLink: https://bugs.launchpad.net/bugs/1920944 Crash shutdown handler only disables kvmclock and steal time, other PV features remain active so we risk corrupting memory or getting some side-effects in kdump kernel. Move crash handler to kvm.c and unify with CPU offline. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-5-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (backported from commit 3d6b84132d2a57b5a74100f6923a8feb679ac2ce) Signed-off-by: Andrea Righi --- arch/x86/include/asm/kvm_para.h | 5 ---- arch/x86/kernel/kvm.c | 44 ++++++++++++++++++++++++--------- arch/x86/kernel/kvmclock.c | 21 ---------------- 3 files changed, 32 insertions(+), 38 deletions(-) diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h index a617fd360023..7ff8ad490a78 100644 --- a/arch/x86/include/asm/kvm_para.h +++ b/arch/x86/include/asm/kvm_para.h @@ -91,7 +91,6 @@ unsigned int kvm_arch_para_hints(void); void kvm_async_pf_task_wait(u32 token, int interrupt_kernel); void kvm_async_pf_task_wake(u32 token); u32 kvm_read_and_reset_pf_reason(void); -extern void kvm_disable_steal_time(void); void do_async_page_fault(struct pt_regs *regs, unsigned long error_code, unsigned long address); #ifdef CONFIG_PARAVIRT_SPINLOCKS @@ -126,10 +125,6 @@ static inline u32 kvm_read_and_reset_pf_reason(void) return 0; } -static inline void kvm_disable_steal_time(void) -{ - return; -} #endif #endif /* _ASM_X86_KVM_PARA_H */ diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 35cd7baf2ad7..3bba064da7ff 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -34,6 +34,7 @@ #include #include #include +#include static int kvmapf = 1; @@ -350,6 +351,14 @@ static void kvm_pv_disable_apf(void) pr_info("disable async PF for cpu %d\n", smp_processor_id()); } +static void kvm_disable_steal_time(void) +{ + if (!has_steal_clock) + return; + + wrmsr(MSR_KVM_STEAL_TIME, 0, 0); +} + static void kvm_pv_guest_cpu_reboot(void *unused) { /* @@ -392,14 +401,6 @@ static u64 kvm_steal_clock(int cpu) return steal; } -void kvm_disable_steal_time(void) -{ - if (!has_steal_clock) - return; - - wrmsr(MSR_KVM_STEAL_TIME, 0, 0); -} - static inline void __set_percpu_decrypted(void *ptr, unsigned long size) { early_set_memory_decrypted((unsigned long) ptr, size); @@ -427,13 +428,14 @@ static void __init sev_map_percpu_data(void) } } -static void kvm_guest_cpu_offline(void) +static void kvm_guest_cpu_offline(bool shutdown) { kvm_disable_steal_time(); if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) wrmsrl(MSR_KVM_PV_EOI_EN, 0); kvm_pv_disable_apf(); - apf_task_wake_all(); + if (!shutdown) + apf_task_wake_all(); kvmclock_disable(); } @@ -571,7 +573,7 @@ static int kvm_cpu_down_prepare(unsigned int cpu) unsigned long flags; local_irq_save(flags); - kvm_guest_cpu_offline(); + kvm_guest_cpu_offline(false); local_irq_restore(flags); return 0; } @@ -580,7 +582,7 @@ static int kvm_cpu_down_prepare(unsigned int cpu) static int kvm_suspend(void) { - kvm_guest_cpu_offline(); + kvm_guest_cpu_offline(false); return 0; } @@ -628,6 +630,20 @@ static void kvm_flush_tlb_others(const struct cpumask *cpumask, native_flush_tlb_others(flushmask, info); } +/* + * After a PV feature is registered, the host will keep writing to the + * registered memory location. If the guest happens to shutdown, this memory + * won't be valid. In cases like kexec, in which you install a new kernel, this + * means a random memory location will be kept being written. + */ +#ifdef CONFIG_KEXEC_CORE +static void kvm_crash_shutdown(struct pt_regs *regs) +{ + kvm_guest_cpu_offline(true); + native_machine_crash_shutdown(regs); +} +#endif + static void __init kvm_guest_init(void) { int i; @@ -671,6 +687,10 @@ static void __init kvm_guest_init(void) kvm_guest_cpu_init(); #endif +#ifdef CONFIG_KEXEC_CORE + machine_ops.crash_shutdown = kvm_crash_shutdown; +#endif + register_syscore_ops(&kvm_syscore_ops); /* diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index bd3962953f78..4a0802af2e3e 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -20,7 +20,6 @@ #include #include #include -#include #include static int kvmclock __initdata = 1; @@ -197,23 +196,6 @@ static void kvm_setup_secondary_clock(void) } #endif -/* - * After the clock is registered, the host will keep writing to the - * registered memory location. If the guest happens to shutdown, this memory - * won't be valid. In cases like kexec, in which you install a new kernel, this - * means a random memory location will be kept being written. So before any - * kind of shutdown from our side, we unregister the clock by writing anything - * that does not have the 'enable' bit set in the msr - */ -#ifdef CONFIG_KEXEC_CORE -static void kvm_crash_shutdown(struct pt_regs *regs) -{ - native_write_msr(msr_kvm_system_time, 0, 0); - kvm_disable_steal_time(); - native_machine_crash_shutdown(regs); -} -#endif - void kvmclock_disable(void) { native_write_msr(msr_kvm_system_time, 0, 0); @@ -344,9 +326,6 @@ void __init kvmclock_init(void) #endif x86_platform.save_sched_clock_state = kvm_save_sched_clock_state; x86_platform.restore_sched_clock_state = kvm_restore_sched_clock_state; -#ifdef CONFIG_KEXEC_CORE - machine_ops.crash_shutdown = kvm_crash_shutdown; -#endif kvm_get_preset_lpj(); /* From patchwork Wed May 26 05:15:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1483872 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FqfLK0vfqz9sWW; Wed, 26 May 2021 15:16:08 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1llltj-0001jI-G5; Wed, 26 May 2021 05:16:03 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1lllth-0001hg-GS for kernel-team@lists.ubuntu.com; Wed, 26 May 2021 05:16:01 +0000 Received: from mail-ed1-f72.google.com ([209.85.208.72]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1lllth-0004DP-5j for kernel-team@lists.ubuntu.com; Wed, 26 May 2021 05:16:01 +0000 Received: by mail-ed1-f72.google.com with SMTP id h18-20020a05640250d2b029038cc3938914so18667348edb.17 for ; Tue, 25 May 2021 22:16:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=alf6IFyQSuQl77FnfeOkt/pSpJewyeh8BG5wTmiNw7E=; b=LmVhN77AqtlA/XRTsO9WaqWPFUdloycESHkHK7/p0Hy9PXS+hYIIJQO0QdgScYW4eU XG6HYz9pTG+U/0q11gho2B09yomFE1P5DLMisteKXZjbeKjjOxCbPV9M5FhpMBOcgEq1 Vic2typs3LzTmsOwUchSkcvG5rY27eRPgTEUXSaJbr2CajZaSDEziKcBmIMHPF+XeioU JUK8wX0EbbvhT1lJeR1Xn/d7tXGxtmDtNCivHRyXmSLH94Q8ePWp8zddXrUk+Y84CPx5 NWdmdOek5dJSPeKpbau9Y+vJ8RcK/6P1vNsgykq3GJDDb3ObiZdrntUnW/2sVMhAMep0 CWHw== X-Gm-Message-State: AOAM53392n9Y4pqHqAunUmQ1/JinkqPKdJ85esiPtalrpak6BSiBwmvN sRijucZzCAxeaT22VUlm3EhN/5Lhq0euVM2Mng70cuCKCZ9XNtoAVuwQQyQVcSsCAJxG4abijBR 4mjf9ri0rS0q0xGp2U99yHksKW6Oq1Z8Xy+UPq5ggWw== X-Received: by 2002:aa7:c7cd:: with SMTP id o13mr13212581eds.269.1622006160851; Tue, 25 May 2021 22:16:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxATyg+TB+Et4lEuvzapIY4ayxyvb8eVEvUj9TqCBRSyVzq/qjawBZNk03KVcm20P8eyioLhw== X-Received: by 2002:aa7:c7cd:: with SMTP id o13mr13212563eds.269.1622006160654; Tue, 25 May 2021 22:16:00 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id z4sm11704098edc.1.2021.05.25.22.16.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 May 2021 22:16:00 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH v2 5/5] x86/kvm: Unify kvm_pv_guest_cpu_reboot() with kvm_guest_cpu_offline() Date: Wed, 26 May 2021 07:15:47 +0200 Message-Id: <20210526051547.217632-6-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210526051547.217632-1-andrea.righi@canonical.com> References: <20210526051547.217632-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov BugLink: https://bugs.launchpad.net/bugs/1920944 Simplify the code by making PV features shutdown happen in one place. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-6-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (backported from commit 384fc672f528d3b84eacd9a86ecf35df3363b8ba) Signed-off-by: Andrea Righi --- arch/x86/kernel/kvm.c | 42 +++++++++++++++++------------------------- 1 file changed, 17 insertions(+), 25 deletions(-) diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 3bba064da7ff..4d895bbe9f1e 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -359,31 +359,6 @@ static void kvm_disable_steal_time(void) wrmsr(MSR_KVM_STEAL_TIME, 0, 0); } -static void kvm_pv_guest_cpu_reboot(void *unused) -{ - /* - * We disable PV EOI before we load a new kernel by kexec, - * since MSR_KVM_PV_EOI_EN stores a pointer into old kernel's memory. - * New kernel can re-enable when it boots. - */ - if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) - wrmsrl(MSR_KVM_PV_EOI_EN, 0); - kvm_pv_disable_apf(); - kvm_disable_steal_time(); -} - -static int kvm_pv_reboot_notify(struct notifier_block *nb, - unsigned long code, void *unused) -{ - if (code == SYS_RESTART) - on_each_cpu(kvm_pv_guest_cpu_reboot, NULL, 1); - return NOTIFY_DONE; -} - -static struct notifier_block kvm_pv_reboot_nb = { - .notifier_call = kvm_pv_reboot_notify, -}; - static u64 kvm_steal_clock(int cpu) { u64 steal; @@ -630,6 +605,23 @@ static void kvm_flush_tlb_others(const struct cpumask *cpumask, native_flush_tlb_others(flushmask, info); } +static void kvm_pv_guest_cpu_reboot(void *unused) +{ + kvm_guest_cpu_offline(true); +} + +static int kvm_pv_reboot_notify(struct notifier_block *nb, + unsigned long code, void *unused) +{ + if (code == SYS_RESTART) + on_each_cpu(kvm_pv_guest_cpu_reboot, NULL, 1); + return NOTIFY_DONE; +} + +static struct notifier_block kvm_pv_reboot_nb = { + .notifier_call = kvm_pv_reboot_notify, +}; + /* * After a PV feature is registered, the host will keep writing to the * registered memory location. If the guest happens to shutdown, this memory