From patchwork Thu May 20 13:39:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1481637 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fm9pQ2Dk4z9sV5; Thu, 20 May 2021 23:39:58 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1ljiu2-000272-49; Thu, 20 May 2021 13:39:54 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ljiu0-00026k-5B for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:39:52 +0000 Received: from mail-ed1-f71.google.com ([209.85.208.71]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1ljitz-0000fL-Ua for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:39:51 +0000 Received: by mail-ed1-f71.google.com with SMTP id h18-20020a05640250d2b029038cc3938914so9564676edb.17 for ; Thu, 20 May 2021 06:39:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xvpiaIcyS3PvwaFkqMHVlRqC3115dgP+mghHRfXUcn4=; b=jdp5Xcy5LCtsgQ1SYUkmvv5UI/xdSiQIZE0hCtV0Nlm6zCutUbZP6krregWAPUWSLc b2QjDkCqamL/yHr3FpGofORrpRX8VpZAylulnH0c5BRhyRPpNDkraFkccjAZet58ivxR 1npYDjr+KpY5XVvkZKVKLXJlKEd1dNTm8g9hO9tlVyS4Nch/+/wL52po4hZSbdZ+6xCf udKBp+JVCQF2kv78qc6HworkMF18a4YVHnklicWGdkYZEqVTqwhvL1CTDEB75ea+gbZm y3n1QKa1VuPcclhxejMdN/+XW3gy2+r0JLn4o/ryVdJ6uIBdpQSFKIV+Ibv02e9nsKz0 B6WQ== X-Gm-Message-State: AOAM532hfyx2jvc3t6CHJiMiZjhZTYX7Bhv7v1XoP+xzHGCkkkYi3lnF WuMz/MUmaUsKPeNFG0prKj91d7yRNhED0EifXnemLHORh+oObZyynZeZm9FrO22fn3tGZIeBp0t /H2xYbjxW5MyPLrt4y2IdD0ZhpnjTD9SOXHE0ght6Hg== X-Received: by 2002:a05:6402:896:: with SMTP id e22mr5124030edy.256.1621517991667; Thu, 20 May 2021 06:39:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwY9T55Rwwbo0Iwey6zggP5DIZqM1NNCjwfwpaAYe+889S2qwDW8/6FNoHC0eGRhfMOA7Q02A== X-Received: by 2002:a05:6402:896:: with SMTP id e22mr5124002edy.256.1621517991432; Thu, 20 May 2021 06:39:51 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id m16sm1555414edq.56.2021.05.20.06.39.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 06:39:51 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][G][PATCH 1/5] x86/kvm: Fix pr_info() for async PF setup/teardown Date: Thu, 20 May 2021 15:39:44 +0200 Message-Id: <20210520133948.39779-2-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520133948.39779-1-andrea.righi@canonical.com> References: <20210520133948.39779-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov 'pr_fmt' already has 'kvm-guest: ' so 'KVM' prefix is redundant. "Unregister pv shared memory" is very ambiguous, it's hard to say which particular PV feature it relates to. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-2-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (cherry picked from commit 0a269a008f837e76ce285679ab3005059fadc2a6) Signed-off-by: Andrea Righi --- arch/x86/kernel/kvm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index df63786e7bfa..0fee42222a8f 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -324,7 +324,7 @@ static void kvm_guest_cpu_init(void) wrmsrl(MSR_KVM_ASYNC_PF_EN, pa); __this_cpu_write(apf_reason.enabled, 1); - pr_info("KVM setup async PF for cpu %d\n", smp_processor_id()); + pr_info("setup async PF for cpu %d\n", smp_processor_id()); } if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) { @@ -350,7 +350,7 @@ static void kvm_pv_disable_apf(void) wrmsrl(MSR_KVM_ASYNC_PF_EN, 0); __this_cpu_write(apf_reason.enabled, 0); - pr_info("Unregister pv shared memory for cpu %d\n", smp_processor_id()); + pr_info("disable async PF for cpu %d\n", smp_processor_id()); } static void kvm_pv_guest_cpu_reboot(void *unused) From patchwork Thu May 20 13:39:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1481640 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fm9pX1Hfjz9sWW; Thu, 20 May 2021 23:40:04 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1ljiu7-00029J-Qw; Thu, 20 May 2021 13:39:59 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ljiu0-00026q-WA for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:39:52 +0000 Received: from mail-ed1-f69.google.com ([209.85.208.69]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1ljiu0-0000fW-Ne for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:39:52 +0000 Received: by mail-ed1-f69.google.com with SMTP id n6-20020a0564020606b029038cdc241890so9518954edv.20 for ; Thu, 20 May 2021 06:39:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BzywhRca+JC3rk5v+emNt2o8fIXwWMXhvHg6NVAweiY=; b=i+zGx005AQV2tab4T79m8M7NTQIK+PtszCDviQZ3blDb5oWYecJQU9YjynD+F4BBRi IZvGZV8IvPHpgwbYzLZHhWI6K9aG/OY36A9V4O862O2mkeXPOt9oo+dor/7db6iHWGDI c8afDgCb5RPpNGbRDZFjg3788QD/PdkFDfXYIT5PAQ5br1z70KgqEIs4u5Bvta22/hU1 rl+qKh9mVh/hIUYqRRs/pAxMCaVkO7fe9pjNBcpx1nQW4wsaoHZoK8QqxhBfJ5/s4s9C Q6M8DmSiIlDgVRxKHsUxC2QZ7VYEU/Yv0xnZCvLfZY+n4wkr4zf5NF03j2jVSkRs/5jz nY7Q== X-Gm-Message-State: AOAM533fSOcdINfHT0TRerFMDnZ8Iv4rz0l0Tzw5z/Uuhagasv3p0Cb9 U9X7egG6LQedIgjUxuyXyqZrR/lUsO/C0l/Zlp+OnU20DGpV9YSyQmgrtJEnYSuhbGc4ll+B9n2 aGJ1tbzglt+7rBdotqOuql4QmqriBe9B68ypXSWkrFQ== X-Received: by 2002:aa7:d04a:: with SMTP id n10mr5083947edo.33.1621517992394; Thu, 20 May 2021 06:39:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyWCR05Ziw5NyNTTwqZxMHd9XS5o0jj/dLRw5GEnVF+2wra7rPY3y55S8SJHrYheEi43C6W0g== X-Received: by 2002:aa7:d04a:: with SMTP id n10mr5083918edo.33.1621517992084; Thu, 20 May 2021 06:39:52 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id m16sm1555414edq.56.2021.05.20.06.39.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 06:39:51 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][G][PATCH 2/5] x86/kvm: Teardown PV features on boot CPU as well Date: Thu, 20 May 2021 15:39:45 +0200 Message-Id: <20210520133948.39779-3-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520133948.39779-1-andrea.righi@canonical.com> References: <20210520133948.39779-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov Various PV features (Async PF, PV EOI, steal time) work through memory shared with hypervisor and when we restore from hibernation we must properly teardown all these features to make sure hypervisor doesn't write to stale locations after we jump to the previously hibernated kernel (which can try to place anything there). For secondary CPUs the job is already done by kvm_cpu_down_prepare(), register syscore ops to do the same for boot CPU. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-3-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (backported from commit 8b79feffeca28c5459458fe78676b081e87c93a4) Signed-off-by: Andrea Righi --- arch/x86/kernel/kvm.c | 57 +++++++++++++++++++++++++++++++------------ 1 file changed, 41 insertions(+), 16 deletions(-) diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 0fee42222a8f..4dc69777417a 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -430,6 +431,25 @@ static void __init sev_map_percpu_data(void) } } +static void kvm_guest_cpu_offline(void) +{ + kvm_disable_steal_time(); + if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) + wrmsrl(MSR_KVM_PV_EOI_EN, 0); + kvm_pv_disable_apf(); + apf_task_wake_all(); +} + +static int kvm_cpu_online(unsigned int cpu) +{ + unsigned long flags; + + local_irq_save(flags); + kvm_guest_cpu_init(); + local_irq_restore(flags); + return 0; +} + static bool pv_tlb_flush_supported(void) { return (kvm_para_has_feature(KVM_FEATURE_PV_TLB_FLUSH) && @@ -571,31 +591,34 @@ static void __init kvm_smp_prepare_boot_cpu(void) kvm_spinlock_init(); } -static void kvm_guest_cpu_offline(void) +static int kvm_cpu_down_prepare(unsigned int cpu) { - kvm_disable_steal_time(); - if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) - wrmsrl(MSR_KVM_PV_EOI_EN, 0); - kvm_pv_disable_apf(); - apf_task_wake_all(); -} + unsigned long flags; -static int kvm_cpu_online(unsigned int cpu) -{ - local_irq_disable(); - kvm_guest_cpu_init(); - local_irq_enable(); + local_irq_save(flags); + kvm_guest_cpu_offline(); + local_irq_restore(flags); return 0; } -static int kvm_cpu_down_prepare(unsigned int cpu) +#endif + +static int kvm_suspend(void) { - local_irq_disable(); kvm_guest_cpu_offline(); - local_irq_enable(); + return 0; } -#endif + +static void kvm_resume(void) +{ + kvm_cpu_online(raw_smp_processor_id()); +} + +static struct syscore_ops kvm_syscore_ops = { + .suspend = kvm_suspend, + .resume = kvm_resume, +}; static void kvm_flush_tlb_others(const struct cpumask *cpumask, const struct flush_tlb_info *info) @@ -664,6 +687,8 @@ static void __init kvm_guest_init(void) kvm_guest_cpu_init(); #endif + register_syscore_ops(&kvm_syscore_ops); + /* * Hard lockup detection is enabled by default. Disable it, as guests * can get false positives too easily, for example if the host is From patchwork Thu May 20 13:39:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1481642 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fm9ph46Z0z9sW1; Thu, 20 May 2021 23:40:12 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1ljiuG-0002FZ-67; Thu, 20 May 2021 13:40:08 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ljiu1-00026w-Kj for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:39:53 +0000 Received: from mail-ej1-f72.google.com ([209.85.218.72]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1ljiu1-0000fh-Cs for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:39:53 +0000 Received: by mail-ej1-f72.google.com with SMTP id p18-20020a1709067852b02903dab2a3e1easo2034344ejm.17 for ; Thu, 20 May 2021 06:39:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dMXPPb944oq00QqTAKny0rWmgpVKrJLFoPdHE6duxEc=; b=mKEFR6sLtfzgZTp7mZoOxi+Ole5QGMMIGqYRSumc5VF0wdm1fzMFNOv5HnoTAFVRii ME6X8sgGZjOrJIvObL25R8s9Vq2j/4PKorsEFpWybibw4XO3bGO5c/QVdboKYLw9a30Q /dqZWxQf9tfKMQv0e/h+gMI/RmazzxYTAqDNZRMrAlSP1ndH2bYjGYSZXZX/qh10hgjG 5wvMKRRxB5JeP6rpt66etyb1q/1miltBeCL+1P1e1yc4wh1fzmL9m11SR4V0tUppgkpa uTSJHXPYU1NYK20+eTIlMder6suOvK5yU2sq7NWpaw8MTRmYCBs1aZCCHsxBKjfx9zhd B0tw== X-Gm-Message-State: AOAM531vWtFT98lpe7iDl+cYlhlTjZqnhTwRd8aLYzYc2xDKdz97ORFh o527KAAFb4HRQI4GtM3cu1UPiOx6GfKuq/4WA4Gtn4sstXioCNT3a3ytCJy3ASvPt2KhCbjZiKJ N3ZlCLXEeD3Diupfgvn4FoypJmZOzagIBQpg6iMSQyQ== X-Received: by 2002:a50:c09a:: with SMTP id k26mr5101502edf.27.1621517993009; Thu, 20 May 2021 06:39:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyaKaGS4lBqjsAIYuMxY1rO5ljGP2gYZiB0UsdZKcN31rmKHve8xrChcPLVtFtTO2PEqjJX6g== X-Received: by 2002:a50:c09a:: with SMTP id k26mr5101482edf.27.1621517992816; Thu, 20 May 2021 06:39:52 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id m16sm1555414edq.56.2021.05.20.06.39.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 06:39:52 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][G][PATCH 3/5] x86/kvm: Disable kvmclock on all CPUs on shutdown Date: Thu, 20 May 2021 15:39:46 +0200 Message-Id: <20210520133948.39779-4-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520133948.39779-1-andrea.righi@canonical.com> References: <20210520133948.39779-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov Currenly, we disable kvmclock from machine_shutdown() hook and this only happens for boot CPU. We need to disable it for all CPUs to guard against memory corruption e.g. on restore from hibernate. Note, writing '0' to kvmclock MSR doesn't clear memory location, it just prevents hypervisor from updating the location so for the short while after write and while CPU is still alive, the clock remains usable and correct so we don't need to switch to some other clocksource. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-4-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (cherry picked from commit c02027b5742b5aa804ef08a4a9db433295533046) Signed-off-by: Andrea Righi --- arch/x86/include/asm/kvm_para.h | 4 ++-- arch/x86/kernel/kvm.c | 1 + arch/x86/kernel/kvmclock.c | 5 +---- 3 files changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h index 49d3a9edb06f..43cc4a396bee 100644 --- a/arch/x86/include/asm/kvm_para.h +++ b/arch/x86/include/asm/kvm_para.h @@ -6,8 +6,6 @@ #include #include -extern void kvmclock_init(void); - #ifdef CONFIG_KVM_GUEST bool kvm_check_and_clear_guest_paused(void); #else @@ -85,6 +83,8 @@ static inline long kvm_hypercall4(unsigned int nr, unsigned long p1, } #ifdef CONFIG_KVM_GUEST +void kvmclock_init(void); +void kvmclock_disable(void); bool kvm_para_available(void); unsigned int kvm_arch_para_features(void); unsigned int kvm_arch_para_hints(void); diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 4dc69777417a..74d6c0e344ba 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -438,6 +438,7 @@ static void kvm_guest_cpu_offline(void) wrmsrl(MSR_KVM_PV_EOI_EN, 0); kvm_pv_disable_apf(); apf_task_wake_all(); + kvmclock_disable(); } static int kvm_cpu_online(unsigned int cpu) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 5ee705b44560..327a0de01066 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -221,11 +221,9 @@ static void kvm_crash_shutdown(struct pt_regs *regs) } #endif -static void kvm_shutdown(void) +void kvmclock_disable(void) { native_write_msr(msr_kvm_system_time, 0, 0); - kvm_disable_steal_time(); - native_machine_shutdown(); } static void __init kvmclock_init_mem(void) @@ -352,7 +350,6 @@ void __init kvmclock_init(void) #endif x86_platform.save_sched_clock_state = kvm_save_sched_clock_state; x86_platform.restore_sched_clock_state = kvm_restore_sched_clock_state; - machine_ops.shutdown = kvm_shutdown; #ifdef CONFIG_KEXEC_CORE machine_ops.crash_shutdown = kvm_crash_shutdown; #endif From patchwork Thu May 20 13:39:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1481638 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fm9pT44LVz9sTD; Thu, 20 May 2021 23:40:01 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1ljiu5-00027l-A1; Thu, 20 May 2021 13:39:57 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ljiu2-00027B-Cd for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:39:54 +0000 Received: from mail-ed1-f71.google.com ([209.85.208.71]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1ljiu2-0000fr-3T for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:39:54 +0000 Received: by mail-ed1-f71.google.com with SMTP id w1-20020aa7da410000b029038d323eeee3so7511003eds.8 for ; Thu, 20 May 2021 06:39:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0jqgFDUODh4aXUkctLjauRHUDbv5JTCQtUoI4f3zoEk=; b=sUSb9m2F+PQz8bIG0V8AjAPDq544Mdzpop7OekToHqRVB4SVp+By+EArDlmvoWYGyo X+exqaNBq+/RWL5ZAg5E4U1jZZvlipHl5MIECFQLFB/O+hJtG5OOB+eSz5rEoAr8Dj/V 7sSCwrk7H77MAtoyuBre9NGJdB6xz0qrLPD13yySsg4gdQVAF8EhQbQPF2G4HdVN2pK1 mU4LQMVpkrkVp7ju5iBzHSgmblt8ymgeCO0ubhrw7XAVgOptTF1eDf4Rif/aKVODEFZ4 mJej1w+Y/ryGm8u/FULGbKVJ72I1p+z0JTeGvQCdXiOtyWzPmplTAbroPSrC5WNXAszI OHqw== X-Gm-Message-State: AOAM532DseWBLqTcZaz6gTG+eK6Anu3xtrLhWpDLmb3dHO38EIItPS9s pE9dCdS0IgxAb6Mg/SsEBE/WHqmLpMFGI205LXhZthY++lg9zNOqFbQTO+10myig8WIg7e/vNKz u2XhDYAYoCFUC5AEpbBTFmgLdsHAQ2PdxJTfsybjzlg== X-Received: by 2002:a50:ec91:: with SMTP id e17mr5137875edr.228.1621517993726; Thu, 20 May 2021 06:39:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwBlhdw1TMk8S7WvUn/cAfvD/h7OukqCpEGfKfFbK1KBrUGTs6GzVSQTrysZT5ZZLjTCHz5Sw== X-Received: by 2002:a50:ec91:: with SMTP id e17mr5137854edr.228.1621517993429; Thu, 20 May 2021 06:39:53 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id m16sm1555414edq.56.2021.05.20.06.39.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 06:39:53 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][G][PATCH 4/5] x86/kvm: Disable all PV features on crash Date: Thu, 20 May 2021 15:39:47 +0200 Message-Id: <20210520133948.39779-5-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520133948.39779-1-andrea.righi@canonical.com> References: <20210520133948.39779-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov Crash shutdown handler only disables kvmclock and steal time, other PV features remain active so we risk corrupting memory or getting some side-effects in kdump kernel. Move crash handler to kvm.c and unify with CPU offline. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-5-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (backported from commit 3d6b84132d2a57b5a74100f6923a8feb679ac2ce) Signed-off-by: Andrea Righi --- arch/x86/include/asm/kvm_para.h | 6 ----- arch/x86/kernel/kvm.c | 46 +++++++++++++++++++++++---------- arch/x86/kernel/kvmclock.c | 21 --------------- 3 files changed, 33 insertions(+), 40 deletions(-) diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h index 43cc4a396bee..b2bfcbffcd74 100644 --- a/arch/x86/include/asm/kvm_para.h +++ b/arch/x86/include/asm/kvm_para.h @@ -91,7 +91,6 @@ unsigned int kvm_arch_para_hints(void); void kvm_async_pf_task_wait_schedule(u32 token); void kvm_async_pf_task_wake(u32 token); u32 kvm_read_and_reset_apf_flags(void); -void kvm_disable_steal_time(void); bool __kvm_handle_async_pf(struct pt_regs *regs, u32 token); DECLARE_STATIC_KEY_FALSE(kvm_async_pf_enabled); @@ -136,11 +135,6 @@ static inline u32 kvm_read_and_reset_apf_flags(void) return 0; } -static inline void kvm_disable_steal_time(void) -{ - return; -} - static __always_inline bool kvm_handle_async_pf(struct pt_regs *regs, u32 token) { return false; diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 74d6c0e344ba..adedf0fcb12b 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -34,6 +34,7 @@ #include #include #include +#include DEFINE_STATIC_KEY_FALSE(kvm_async_pf_enabled); @@ -354,6 +355,14 @@ static void kvm_pv_disable_apf(void) pr_info("disable async PF for cpu %d\n", smp_processor_id()); } +static void kvm_disable_steal_time(void) +{ + if (!has_steal_clock) + return; + + wrmsr(MSR_KVM_STEAL_TIME, 0, 0); +} + static void kvm_pv_guest_cpu_reboot(void *unused) { /* @@ -396,14 +405,6 @@ static u64 kvm_steal_clock(int cpu) return steal; } -void kvm_disable_steal_time(void) -{ - if (!has_steal_clock) - return; - - wrmsr(MSR_KVM_STEAL_TIME, 0, 0); -} - static inline void __set_percpu_decrypted(void *ptr, unsigned long size) { early_set_memory_decrypted((unsigned long) ptr, size); @@ -431,13 +432,14 @@ static void __init sev_map_percpu_data(void) } } -static void kvm_guest_cpu_offline(void) +static void kvm_guest_cpu_offline(bool shutdown) { kvm_disable_steal_time(); if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) wrmsrl(MSR_KVM_PV_EOI_EN, 0); kvm_pv_disable_apf(); - apf_task_wake_all(); + if (!shutdown) + apf_task_wake_all(); kvmclock_disable(); } @@ -597,7 +599,7 @@ static int kvm_cpu_down_prepare(unsigned int cpu) unsigned long flags; local_irq_save(flags); - kvm_guest_cpu_offline(); + kvm_guest_cpu_offline(false); local_irq_restore(flags); return 0; } @@ -606,7 +608,7 @@ static int kvm_cpu_down_prepare(unsigned int cpu) static int kvm_suspend(void) { - kvm_guest_cpu_offline(); + kvm_guest_cpu_offline(false); return 0; } @@ -647,6 +649,20 @@ static void kvm_flush_tlb_others(const struct cpumask *cpumask, native_flush_tlb_others(flushmask, info); } +/* + * After a PV feature is registered, the host will keep writing to the + * registered memory location. If the guest happens to shutdown, this memory + * won't be valid. In cases like kexec, in which you install a new kernel, this + * means a random memory location will be kept being written. + */ +#ifdef CONFIG_KEXEC_CORE +static void kvm_crash_shutdown(struct pt_regs *regs) +{ + kvm_guest_cpu_offline(true); + native_machine_crash_shutdown(regs); +} +#endif + static void __init kvm_guest_init(void) { int i; @@ -665,7 +681,7 @@ static void __init kvm_guest_init(void) pv_ops.mmu.flush_tlb_others = kvm_flush_tlb_others; pv_ops.mmu.tlb_remove_table = tlb_remove_table; pr_info("KVM setup pv remote TLB flush\n"); - } +} if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) apic_set_eoi_write(kvm_guest_apic_eoi_write); @@ -688,6 +704,10 @@ static void __init kvm_guest_init(void) kvm_guest_cpu_init(); #endif +#ifdef CONFIG_KEXEC_CORE + machine_ops.crash_shutdown = kvm_crash_shutdown; +#endif + register_syscore_ops(&kvm_syscore_ops); /* diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 327a0de01066..c4ac26333bc4 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -20,7 +20,6 @@ #include #include #include -#include #include static int kvmclock __initdata = 1; @@ -204,23 +203,6 @@ static void kvm_setup_secondary_clock(void) } #endif -/* - * After the clock is registered, the host will keep writing to the - * registered memory location. If the guest happens to shutdown, this memory - * won't be valid. In cases like kexec, in which you install a new kernel, this - * means a random memory location will be kept being written. So before any - * kind of shutdown from our side, we unregister the clock by writing anything - * that does not have the 'enable' bit set in the msr - */ -#ifdef CONFIG_KEXEC_CORE -static void kvm_crash_shutdown(struct pt_regs *regs) -{ - native_write_msr(msr_kvm_system_time, 0, 0); - kvm_disable_steal_time(); - native_machine_crash_shutdown(regs); -} -#endif - void kvmclock_disable(void) { native_write_msr(msr_kvm_system_time, 0, 0); @@ -350,9 +332,6 @@ void __init kvmclock_init(void) #endif x86_platform.save_sched_clock_state = kvm_save_sched_clock_state; x86_platform.restore_sched_clock_state = kvm_restore_sched_clock_state; -#ifdef CONFIG_KEXEC_CORE - machine_ops.crash_shutdown = kvm_crash_shutdown; -#endif kvm_get_preset_lpj(); /* From patchwork Thu May 20 13:39:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1481641 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Fm9pY6gZ0z9sWX; Thu, 20 May 2021 23:40:05 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1ljiu7-00029b-Vo; Thu, 20 May 2021 13:39:59 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ljiu2-00027Q-W2 for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:39:54 +0000 Received: from mail-ej1-f72.google.com ([209.85.218.72]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1ljiu2-0000g1-Mx for kernel-team@lists.ubuntu.com; Thu, 20 May 2021 13:39:54 +0000 Received: by mail-ej1-f72.google.com with SMTP id k9-20020a17090646c9b029039d323bd239so4992039ejs.16 for ; Thu, 20 May 2021 06:39:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nn419IHVxkYgh3tSQcApKNGvfcr6EdSijkXQUWlP4tE=; b=ff4EHhx9TLp/CpbnAJVxHnrOGq2xmWdJjjpzLwsDGLcy1zgzZhnlb06R4wAw3Skw3R Oq43Ne6gF0bDgegRblVn0qpghw89/h/XgEwJgj4WZeb+hPVDnmDDA4h/Mllnos3+C+p9 daMrd1KHZ+wjGYQiAhVR04u4lXUtR03Wf8Am00ohwGud2bIrJjArBB3iiBi/S0njbKHc hAYvGwTYgP7Fdk9U0z+HQtbslS7btch4mSrxuT1o1hRi6uCmAyDX/Ld8WtBN4pPuNLeJ Ls/lOKsi/k1YRWZqroumgSndbJUUTBV0nG4PjkcBMtdDvLS9m2KGM95DH1P+xaZLGzqS ddFQ== X-Gm-Message-State: AOAM532EH87e28WMkwcUf4POQ/iyKDHugl5tnu7xpLbSu9VOH3/9x6zu pCXXRxh6x9bwZPW/lyW3rngKUAOg2R6IKmIr4v/FUl1oT/Brmq7Jm+8yorHtGJ3u9sMybNznx1c INZAzRHWlDb+w61U5jLWqRc7VeUjQ+8liA31MhPw2ZA== X-Received: by 2002:a17:906:e15:: with SMTP id l21mr4707378eji.538.1621517994357; Thu, 20 May 2021 06:39:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy3j6N/+5XQB7KNEHzxU6X0cw1WeQySRMelJd+WBmmvMmVzWk2jIuBKPQQeOmBrN3NJKNvoYw== X-Received: by 2002:a17:906:e15:: with SMTP id l21mr4707357eji.538.1621517994137; Thu, 20 May 2021 06:39:54 -0700 (PDT) Received: from xps-13-7390.homenet.telecomitalia.it (host-87-19-3-42.retail.telecomitalia.it. [87.19.3.42]) by smtp.gmail.com with ESMTPSA id m16sm1555414edq.56.2021.05.20.06.39.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 May 2021 06:39:53 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][G][PATCH 5/5] x86/kvm: Unify kvm_pv_guest_cpu_reboot() with kvm_guest_cpu_offline() Date: Thu, 20 May 2021 15:39:48 +0200 Message-Id: <20210520133948.39779-6-andrea.righi@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210520133948.39779-1-andrea.righi@canonical.com> References: <20210520133948.39779-1-andrea.righi@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov Simplify the code by making PV features shutdown happen in one place. Signed-off-by: Vitaly Kuznetsov Message-Id: <20210414123544.1060604-6-vkuznets@redhat.com> Signed-off-by: Paolo Bonzini (backported from commit 384fc672f528d3b84eacd9a86ecf35df3363b8ba) Signed-off-by: Andrea Righi --- arch/x86/kernel/kvm.c | 42 +++++++++++++++++------------------------- 1 file changed, 17 insertions(+), 25 deletions(-) diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index adedf0fcb12b..bdc939d573ec 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -363,31 +363,6 @@ static void kvm_disable_steal_time(void) wrmsr(MSR_KVM_STEAL_TIME, 0, 0); } -static void kvm_pv_guest_cpu_reboot(void *unused) -{ - /* - * We disable PV EOI before we load a new kernel by kexec, - * since MSR_KVM_PV_EOI_EN stores a pointer into old kernel's memory. - * New kernel can re-enable when it boots. - */ - if (kvm_para_has_feature(KVM_FEATURE_PV_EOI)) - wrmsrl(MSR_KVM_PV_EOI_EN, 0); - kvm_pv_disable_apf(); - kvm_disable_steal_time(); -} - -static int kvm_pv_reboot_notify(struct notifier_block *nb, - unsigned long code, void *unused) -{ - if (code == SYS_RESTART) - on_each_cpu(kvm_pv_guest_cpu_reboot, NULL, 1); - return NOTIFY_DONE; -} - -static struct notifier_block kvm_pv_reboot_nb = { - .notifier_call = kvm_pv_reboot_notify, -}; - static u64 kvm_steal_clock(int cpu) { u64 steal; @@ -649,6 +624,23 @@ static void kvm_flush_tlb_others(const struct cpumask *cpumask, native_flush_tlb_others(flushmask, info); } +static void kvm_pv_guest_cpu_reboot(void *unused) +{ + kvm_guest_cpu_offline(true); +} + +static int kvm_pv_reboot_notify(struct notifier_block *nb, + unsigned long code, void *unused) +{ + if (code == SYS_RESTART) + on_each_cpu(kvm_pv_guest_cpu_reboot, NULL, 1); + return NOTIFY_DONE; +} + +static struct notifier_block kvm_pv_reboot_nb = { + .notifier_call = kvm_pv_reboot_notify, +}; + /* * After a PV feature is registered, the host will keep writing to the * registered memory location. If the guest happens to shutdown, this memory