From patchwork Tue Jan 26 13:20:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Henrique Cerri X-Patchwork-Id: 1431680 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DQ6my2Stlz9sWD; Wed, 27 Jan 2021 00:20:50 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1l4OH0-0008DW-KC; Tue, 26 Jan 2021 13:20:46 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1l4OGx-0008D8-Ho for kernel-team@lists.ubuntu.com; Tue, 26 Jan 2021 13:20:43 +0000 Received: from mail-qt1-f198.google.com ([209.85.160.198]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1l4OGx-0005DO-70 for kernel-team@lists.ubuntu.com; Tue, 26 Jan 2021 13:20:43 +0000 Received: by mail-qt1-f198.google.com with SMTP id f5so9178922qtf.15 for ; Tue, 26 Jan 2021 05:20:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hBo3l2kcZEV/+w5ktFklPqaTWJYkAOORT+3/XNf+xuE=; b=AjnxBOO79K/Nw3EAlNEquAZOZxTKLOOPQsZX4IupHrFrHxzp5yZzibshmi2sDGrVoF M37SWiAvMLTGeWSlCecN6Yct1wmBalvEobHwbO0Bl7J8NPWcHsyjmBLclm7r5LhY+/cY j8FD6wtqeTcVafaR9MUcBBBEf8kIYGQ2ZNH5Y5t0iP5RtTmca+wm3WB2YSG0aSHT3noh d4PQgWlXG7sJcNzTnwU8vrbDeYjSukTxb7IyqaF6fDKLDOoFRvsfEI4X3FG3EHNm6W8W CCm4GsGhnOjxTDbpkv55Edf5I+0ezKB0QzUKtV/i6YZcOTPYXpEPew1iXNS1D1xkHR84 rYBQ== X-Gm-Message-State: AOAM533oAlaWrU7A3u2ddizwjFRgXmaVVH+ZIdVzoPOh1Y9qIJwMwDdN Iuj5DdpT6vVXiPlI6E43l9U4IXyRSTM4/ijJAZcQouCTQT1yGjhceQTPykt8eRwVfTH3AZLV6Lg MPB4gvx4ul6z3VTygdIjaqrk5E588EghfPg1d62oX X-Received: by 2002:a37:884:: with SMTP id 126mr5585910qki.404.1611667241993; Tue, 26 Jan 2021 05:20:41 -0800 (PST) X-Google-Smtp-Source: ABdhPJxbeXQ0LnQ/q2KE+IZnyI6eu2yI41JVBhBdpezxoNvCOrc1VfvoeazQpG4T2weBWvTnzNqkvA== X-Received: by 2002:a37:884:: with SMTP id 126mr5585891qki.404.1611667241711; Tue, 26 Jan 2021 05:20:41 -0800 (PST) Received: from localhost.localdomain ([2804:431:cfed:edc:c86c:ab75:eda1:1e6c]) by smtp.gmail.com with ESMTPSA id 18sm7068007qkl.20.2021.01.26.05.20.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 05:20:41 -0800 (PST) From: Marcelo Henrique Cerri To: kernel-team@lists.ubuntu.com Subject: [bionic:linux-azure-4.15][PATCH 1/3] x86/kvm/vmx: read MSR_{FS, KERNEL_GS}_BASE from current->thread Date: Tue, 26 Jan 2021 10:20:22 -0300 Message-Id: <20210126132024.1764493-2-marcelo.cerri@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210126132024.1764493-1-marcelo.cerri@canonical.com> References: <20210126131712.1744754-1-marcelo.cerri@canonical.com> <20210126132024.1764493-1-marcelo.cerri@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Vitaly Kuznetsov BugLink: https://bugs.launchpad.net/bugs/1913294 vmx_save_host_state() is only called from kvm_arch_vcpu_ioctl_run() so the context is pretty well defined. Read MSR_{FS,KERNEL_GS}_BASE from current->thread after calling save_fsgs() which takes care of X86_BUG_NULL_SEG case now and will do RD[FG,GS]BASE when FSGSBASE extensions are exposed to userspace (currently they are not). Acked-by: Andy Lutomirski Signed-off-by: Vitaly Kuznetsov Signed-off-by: Paolo Bonzini (cherry picked from commit 42b933b59721f288e3ce23ca79a17a973808dab9) Signed-off-by: Marcelo Henrique Cerri --- arch/x86/include/asm/processor.h | 5 +++++ arch/x86/kernel/process_64.c | 14 ++++++++++++++ arch/x86/kvm/vmx.c | 13 ++++++++++--- 3 files changed, 29 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index 2c5ecfb72aae..286bc96c67c8 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -408,6 +408,11 @@ DECLARE_INIT_PER_CPU(irq_stack_union); DECLARE_PER_CPU(char *, irq_stack_ptr); DECLARE_PER_CPU(unsigned int, irq_count); extern asmlinkage void ignore_sysret(void); + +#if IS_ENABLED(CONFIG_KVM) +/* Save actual FS/GS selectors and bases to current->thread */ +void save_fsgs_for_kvm(void); +#endif #else /* X86_64 */ #ifdef CONFIG_CC_STACKPROTECTOR /* diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index 55ff408b98bd..e976b6c1c26e 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -218,6 +218,20 @@ static __always_inline void save_fsgs(struct task_struct *task) } } +#if IS_ENABLED(CONFIG_KVM) +/* + * While a process is running,current->thread.fsbase and current->thread.gsbase + * may not match the corresponding CPU registers (see save_base_legacy()). KVM + * wants an efficient way to save and restore FSBASE and GSBASE. + * When FSGSBASE extensions are enabled, this will have to use RD{FS,GS}BASE. + */ +void save_fsgs_for_kvm(void) +{ + save_fsgs(current); +} +EXPORT_SYMBOL_GPL(save_fsgs_for_kvm); +#endif + static __always_inline void loadseg(enum which_selector which, unsigned short sel) { diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 2773499f3578..ca29acc1721f 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -2322,7 +2322,15 @@ static void vmx_save_host_state(struct kvm_vcpu *vcpu) */ vmx->host_state.ldt_sel = kvm_read_ldt(); vmx->host_state.gs_ldt_reload_needed = vmx->host_state.ldt_sel; + +#ifdef CONFIG_X86_64 + save_fsgs_for_kvm(); + vmx->host_state.fs_sel = current->thread.fsindex; + vmx->host_state.gs_sel = current->thread.gsindex; +#else savesegment(fs, vmx->host_state.fs_sel); + savesegment(gs, vmx->host_state.gs_sel); +#endif if (!(vmx->host_state.fs_sel & 7)) { vmcs_write16(HOST_FS_SELECTOR, vmx->host_state.fs_sel); vmx->host_state.fs_reload_needed = 0; @@ -2330,7 +2338,6 @@ static void vmx_save_host_state(struct kvm_vcpu *vcpu) vmcs_write16(HOST_FS_SELECTOR, 0); vmx->host_state.fs_reload_needed = 1; } - savesegment(gs, vmx->host_state.gs_sel); if (!(vmx->host_state.gs_sel & 7)) vmcs_write16(HOST_GS_SELECTOR, vmx->host_state.gs_sel); else { @@ -2344,7 +2351,7 @@ static void vmx_save_host_state(struct kvm_vcpu *vcpu) #endif #ifdef CONFIG_X86_64 - vmcs_writel(HOST_FS_BASE, read_msr(MSR_FS_BASE)); + vmcs_writel(HOST_FS_BASE, current->thread.fsbase); vmcs_writel(HOST_GS_BASE, read_msr(MSR_GS_BASE)); #else vmcs_writel(HOST_FS_BASE, segment_base(vmx->host_state.fs_sel)); @@ -2352,7 +2359,7 @@ static void vmx_save_host_state(struct kvm_vcpu *vcpu) #endif #ifdef CONFIG_X86_64 - rdmsrl(MSR_KERNEL_GS_BASE, vmx->msr_host_kernel_gs_base); + vmx->msr_host_kernel_gs_base = current->thread.gsbase; if (is_long_mode(&vmx->vcpu)) wrmsrl(MSR_KERNEL_GS_BASE, vmx->msr_guest_kernel_gs_base); #endif From patchwork Tue Jan 26 13:20:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Henrique Cerri X-Patchwork-Id: 1431681 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DQ6n01Zlwz9sW8; Wed, 27 Jan 2021 00:20:51 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1l4OH1-0008ED-RF; Tue, 26 Jan 2021 13:20:47 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1l4OGz-0008DF-6l for kernel-team@lists.ubuntu.com; Tue, 26 Jan 2021 13:20:45 +0000 Received: from mail-qv1-f70.google.com ([209.85.219.70]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1l4OGy-0005DT-Rv for kernel-team@lists.ubuntu.com; Tue, 26 Jan 2021 13:20:44 +0000 Received: by mail-qv1-f70.google.com with SMTP id b1so558969qvk.17 for ; Tue, 26 Jan 2021 05:20:44 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5lBruM8nkgMArKc+Rx5olhpzkOZ9jKtB3bKCgYvt/MQ=; b=U0RHpumqVo1U2Vjs0rMjzf3FRowgafgf+3WGsL51s1/StXC7+1HORssjSYW3tM5s1A a6urxZqWrBhRI/MwAMeiRgmMogutiVyUacbFtPdNFPb4VXYPNx20GMiJ9yo0oYntQGDk Nod/IlOR4Phm63VwnbozapLqLgWlKkg8gHECCMWlnOkwwlHk7HOsWkvNLIbI096aaFL5 7kHyqB8ijoPRlgr0KpQm2TXkC14wfn0xVBnc5/15v9aM0QkPkK4USM2uqWOye+L7WRYm 7d8YuNSyW0mcmzyL24I3TmeiBOFIuCXkOkyDbtXj7otPJAHJN5/mf88cqSAFXwlBit8L ud7Q== X-Gm-Message-State: AOAM531q93cyp0VVtCvJ//DjPESlqWAoYHKTRY6skIzECZ70NGCtXch6 qiq9b2ZjyU6jy65JC/GlM7IgFMeNsbAJaK3FsunLRY8oeI0ewsYL65fpZQdyeRvwiSb0CI4oeYZ hVN1IVD+Ws5IRPHYg761q2oD+NlsKJ/+gU8E0/kXX X-Received: by 2002:ac8:6902:: with SMTP id e2mr4873879qtr.162.1611667243649; Tue, 26 Jan 2021 05:20:43 -0800 (PST) X-Google-Smtp-Source: ABdhPJze8OwM20ebLcEfPWQCY0N3kehdIy/ayKhxO7h6KVrEhRDV6qVuPmZQDYJsurOx5jIyNIJkWw== X-Received: by 2002:ac8:6902:: with SMTP id e2mr4873853qtr.162.1611667243392; Tue, 26 Jan 2021 05:20:43 -0800 (PST) Received: from localhost.localdomain ([2804:431:cfed:edc:c86c:ab75:eda1:1e6c]) by smtp.gmail.com with ESMTPSA id 18sm7068007qkl.20.2021.01.26.05.20.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 05:20:42 -0800 (PST) From: Marcelo Henrique Cerri To: kernel-team@lists.ubuntu.com Subject: [bionic:linux-azure-4.15][PATCH 2/3] x86/process/64: Make save_fsgs_for_kvm() ready for FSGSBASE Date: Tue, 26 Jan 2021 10:20:23 -0300 Message-Id: <20210126132024.1764493-3-marcelo.cerri@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210126132024.1764493-1-marcelo.cerri@canonical.com> References: <20210126131712.1744754-1-marcelo.cerri@canonical.com> <20210126132024.1764493-1-marcelo.cerri@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Thomas Gleixner BugLink: https://bugs.launchpad.net/bugs/1913294 save_fsgs_for_kvm() is invoked via vcpu_enter_guest() kvm_x86_ops.prepare_guest_switch(vcpu) vmx_prepare_switch_to_guest() save_fsgs_for_kvm() with preemption disabled, but interrupts enabled. The upcoming FSGSBASE based GS safe needs interrupts to be disabled. This could be done in the helper function, but that function is also called from switch_to() which has interrupts disabled already. Disable interrupts inside save_fsgs_for_kvm() and rename the function to current_save_fsgs() so it can be invoked from other places. Signed-off-by: Thomas Gleixner Signed-off-by: Sasha Levin Signed-off-by: Thomas Gleixner Link: https://lkml.kernel.org/r/20200528201402.1708239-7-sashal@kernel.org (backported from commit 6758034e4d6a7f0e26b748789ab1f83f3116d1b9) Signed-off-by: Marcelo Henrique Cerri --- arch/x86/include/asm/processor.h | 4 +--- arch/x86/kernel/process_64.c | 15 +++++++++------ arch/x86/kvm/vmx.c | 2 +- 3 files changed, 11 insertions(+), 10 deletions(-) diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index 286bc96c67c8..c3a2392fa7ec 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -409,10 +409,8 @@ DECLARE_PER_CPU(char *, irq_stack_ptr); DECLARE_PER_CPU(unsigned int, irq_count); extern asmlinkage void ignore_sysret(void); -#if IS_ENABLED(CONFIG_KVM) /* Save actual FS/GS selectors and bases to current->thread */ -void save_fsgs_for_kvm(void); -#endif +void current_save_fsgs(void); #else /* X86_64 */ #ifdef CONFIG_CC_STACKPROTECTOR /* diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index e976b6c1c26e..381c478184b1 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -218,18 +218,21 @@ static __always_inline void save_fsgs(struct task_struct *task) } } -#if IS_ENABLED(CONFIG_KVM) /* * While a process is running,current->thread.fsbase and current->thread.gsbase - * may not match the corresponding CPU registers (see save_base_legacy()). KVM - * wants an efficient way to save and restore FSBASE and GSBASE. - * When FSGSBASE extensions are enabled, this will have to use RD{FS,GS}BASE. + * may not match the corresponding CPU registers (see save_base_legacy()). */ -void save_fsgs_for_kvm(void) +void current_save_fsgs(void) { + unsigned long flags; + + /* Interrupts need to be off for FSGSBASE */ + local_irq_save(flags); save_fsgs(current); + local_irq_restore(flags); } -EXPORT_SYMBOL_GPL(save_fsgs_for_kvm); +#if IS_ENABLED(CONFIG_KVM) +EXPORT_SYMBOL_GPL(current_save_fsgs); #endif static __always_inline void loadseg(enum which_selector which, diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index ca29acc1721f..79893adbcdac 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -2324,7 +2324,7 @@ static void vmx_save_host_state(struct kvm_vcpu *vcpu) vmx->host_state.gs_ldt_reload_needed = vmx->host_state.ldt_sel; #ifdef CONFIG_X86_64 - save_fsgs_for_kvm(); + current_save_fsgs(); vmx->host_state.fs_sel = current->thread.fsindex; vmx->host_state.gs_sel = current->thread.gsindex; #else From patchwork Tue Jan 26 13:20:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Henrique Cerri X-Patchwork-Id: 1431682 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DQ6n52K6Rz9sWD; Wed, 27 Jan 2021 00:20:57 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1l4OH7-0008HK-3h; Tue, 26 Jan 2021 13:20:53 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1l4OH1-0008Dl-1E for kernel-team@lists.ubuntu.com; Tue, 26 Jan 2021 13:20:47 +0000 Received: from mail-qv1-f70.google.com ([209.85.219.70]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1l4OH0-0005Da-LX for kernel-team@lists.ubuntu.com; Tue, 26 Jan 2021 13:20:46 +0000 Received: by mail-qv1-f70.google.com with SMTP id dj13so10409591qvb.20 for ; Tue, 26 Jan 2021 05:20:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EM8BCQMgwZQN5gy8nGYbYTBJzz0CBLHirMaUbtbLA1I=; b=RV4qSBud+zYloK38JvUuHKRmQPg7aVoUd2BMA3qcciQl5OBwqyU1MfGR/4FUHliYSq fM4cUM85jZzB5kcJDHx/hXXKDZ4dpLvu4vBXGcJ6xVgB5rieJ2U/5jPBW8vhQhZqnOHj Yjn/tuG5EVZ2dkttYnvQo+BrxQpVDg58M5CgWT3p+Jd8vV/7Pcmp5D9QKSxhXdjkTPBU qSgsBm/pczmypXn2fBMvf90nr2KQbocwXx5UMvLPI1RmSh+Wy0MLEqzQSmcFpDkpQCW6 nTvDQDHauFZsXuhfMy1U5boST+qTlfKEUDUtLfglVM4RPWkQ+upblSc870rvFxzrWmLo N1ag== X-Gm-Message-State: AOAM531tnmR6QYN9O0NQzfmpcW/68WT7g6d0tZSyIZ6S0UeuIwvPnlRA XBfYKVMXCncm/LR+yfq3/gYb9rA09Tau2Ea2vXLrZmg89zDVV3QOrB3i1X6nBdPueZmRD/xbynM tp9bRnqJ7DNZ9TXdpbzB2dD+FmfO1vrbsMbMK81Gq X-Received: by 2002:a05:620a:1ed:: with SMTP id x13mr5472999qkn.260.1611667245358; Tue, 26 Jan 2021 05:20:45 -0800 (PST) X-Google-Smtp-Source: ABdhPJyHO7CFyeiIPqxwcLaBoOvzgEmJnBCiLd71wCmgf+hm3rENAwvJlgKpz7u2zOk7tXhGcPKWfg== X-Received: by 2002:a05:620a:1ed:: with SMTP id x13mr5472979qkn.260.1611667245042; Tue, 26 Jan 2021 05:20:45 -0800 (PST) Received: from localhost.localdomain ([2804:431:cfed:edc:c86c:ab75:eda1:1e6c]) by smtp.gmail.com with ESMTPSA id 18sm7068007qkl.20.2021.01.26.05.20.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Jan 2021 05:20:44 -0800 (PST) From: Marcelo Henrique Cerri To: kernel-team@lists.ubuntu.com Subject: [bionic:linux-azure-4.15][PATCH 3/3] x86/entry/64: Do not use RDPID in paranoid entry to accomodate KVM Date: Tue, 26 Jan 2021 10:20:24 -0300 Message-Id: <20210126132024.1764493-4-marcelo.cerri@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210126132024.1764493-1-marcelo.cerri@canonical.com> References: <20210126131712.1744754-1-marcelo.cerri@canonical.com> <20210126132024.1764493-1-marcelo.cerri@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Sean Christopherson BugLink: https://bugs.launchpad.net/bugs/1913294 KVM has an optmization to avoid expensive MRS read/writes on VMENTER/EXIT. It caches the MSR values and restores them either when leaving the run loop, on preemption or when going out to user space. The affected MSRs are not required for kernel context operations. This changed with the recently introduced mechanism to handle FSGSBASE in the paranoid entry code which has to retrieve the kernel GSBASE value by accessing per CPU memory. The mechanism needs to retrieve the CPU number and uses either LSL or RDPID if the processor supports it. Unfortunately RDPID uses MSR_TSC_AUX which is in the list of cached and lazily restored MSRs, which means between the point where the guest value is written and the point of restore, MSR_TSC_AUX contains a random number. If an NMI or any other exception which uses the paranoid entry path happens in such a context, then RDPID returns the random guest MSR_TSC_AUX value. As a consequence this reads from the wrong memory location to retrieve the kernel GSBASE value. Kernel GS is used to for all regular this_cpu_*() operations. If the GSBASE in the exception handler points to the per CPU memory of a different CPU then this has the obvious consequences of data corruption and crashes. As the paranoid entry path is the only place which accesses MSR_TSX_AUX (via RDPID) and the fallback via LSL is not significantly slower, remove the RDPID alternative from the entry path and always use LSL. The alternative would be to write MSR_TSC_AUX on every VMENTER and VMEXIT which would be inflicting massive overhead on that code path. [ tglx: Rewrote changelog ] Fixes: eaad981291ee3 ("x86/entry/64: Introduce the FIND_PERCPU_BASE macro") Reported-by: Tom Lendacky Debugged-by: Tom Lendacky Suggested-by: Andy Lutomirski Suggested-by: Peter Zijlstra Signed-off-by: Sean Christopherson Signed-off-by: Paolo Bonzini Signed-off-by: Thomas Gleixner Link: https://lore.kernel.org/r/20200821105229.18938-1-pbonzini@redhat.com (cherry picked from commit 6a3ea3e68b8a8a26c4aaac03432ed92269c9a14e) Signed-off-by: Marcelo Henrique Cerri --- arch/x86/entry/calling.h | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h index 8ab5b4731a3e..550c6c161662 100644 --- a/arch/x86/entry/calling.h +++ b/arch/x86/entry/calling.h @@ -377,12 +377,14 @@ For 32-bit we have the following conventions - kernel is built with * Fetch the per-CPU GS base value for this processor and put it in @reg. * We normally use %gs for accessing per-CPU data, but we are setting up * %gs here and obviously can not use %gs itself to access per-CPU data. + * + * Do not use RDPID, because KVM loads guest's TSC_AUX on vm-entry and + * may not restore the host's value until the CPU returns to userspace. + * Thus the kernel would consume a guest's TSC_AUX if an NMI arrives + * while running KVM's run loop. */ .macro GET_PERCPU_BASE reg:req - ALTERNATIVE \ - "LOAD_CPU_AND_NODE_SEG_LIMIT \reg", \ - "RDPID \reg", \ - X86_FEATURE_RDPID + LOAD_CPU_AND_NODE_SEG_LIMIT \reg andq $VDSO_CPUNODE_MASK, \reg movq __per_cpu_offset(, \reg, 8), \reg .endm