From patchwork Thu Feb 25 13:46:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444361 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=i3qCvJVz; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYyX0rNLz9sWH for ; Fri, 26 Feb 2021 00:48:04 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230386AbhBYNrt (ORCPT ); Thu, 25 Feb 2021 08:47:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44548 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230166AbhBYNrs (ORCPT ); Thu, 25 Feb 2021 08:47:48 -0500 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BD56C06174A for ; Thu, 25 Feb 2021 05:47:08 -0800 (PST) Received: by mail-pj1-x1035.google.com with SMTP id b15so3542430pjb.0 for ; Thu, 25 Feb 2021 05:47:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cO00I9izdaV4ZgywXXN1ZhPvhTZLsxjfozyqUloYMJs=; b=i3qCvJVzCcy802+nvTxBiHekWsPWWkwBLd4OSSo9Y5NMCgrCrskQQfjoKVRNw5HivF S/c7t1ymihEz2eEWtuJtIaiONYmuAZxUVbZD3ztWfuitdn/E63hdnaR+rEV+naH1tg5U NHOEkMIsTXeOEuqsEvvtOzA10v83xTArUnMoq+o5hgii+5vEHAuD37XP3d7JpOvcRU8F fda7X6oj3xHFTnsm8jttHHil1BWy1jc9jDohcO77l2dCxc6eFQmR4laYrAytNn1zyid1 5WmIVXHAOoBhV4xcAwpOo7VB96ATJIeWs4IoEXi9CZAc0i0rS9nX00gnac8dysrxIUyU cgPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cO00I9izdaV4ZgywXXN1ZhPvhTZLsxjfozyqUloYMJs=; b=Ua24tjlgIX1qsji73VSINlhBDctIr+dbXXX01r/XGjJvQSRCx/3iecF73WAunsW0YL JFHEcWL2cQyqoN4/KjtladPIV8Df+j8PSHlvAkKbwdCXKZ6jB8loepsuMKMitlfF/jqZ YQ43n2ByCBJV0r6mIV4Gevuwo88Mv5jHcXQR2byNhR4Y2ChIp9H8SwRuT7NeXXJMTBII sMByYKP6ngB1Y3DBeVpdPIY6FOe5HlVku9jXuWXvG5gZRXFV9BCphdFwW3dNbHZRt0gp 0n46en0G6pQ9svDNWHEcUB136ANcNvaUsOXPnMgDh9/C1e79UvoEmYafgqwTsjiDfMZH egpQ== X-Gm-Message-State: AOAM5325AB04+Sh8QimXkK/aFBT3anZSsOYW3c+xwvWmJawQquXi+BFe t2LvIHRu8x4KQbUFGva90PVStcB69bc= X-Google-Smtp-Source: ABdhPJxXmN8PHnlMQg/qlEiR4RRkwJOHlBHBokTevTL+jVA25EQK2114P/4WHlVjDdD4tMGFMIHhkA== X-Received: by 2002:a17:90b:1290:: with SMTP id fw16mr3369002pjb.99.1614260827569; Thu, 25 Feb 2021 05:47:07 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:07 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 01/37] KVM: PPC: Book3S 64: remove unused kvmppc_h_protect argument Date: Thu, 25 Feb 2021 23:46:16 +1000 Message-Id: <20210225134652.2127648-2-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org The va argument is not used in the function or set by its asm caller, so remove it to be safe. Signed-off-by: Nicholas Piggin Reviewed-by: Daniel Axtens --- arch/powerpc/include/asm/kvm_ppc.h | 3 +-- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 3 +-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h index 8aacd76bb702..9531b1c1b190 100644 --- a/arch/powerpc/include/asm/kvm_ppc.h +++ b/arch/powerpc/include/asm/kvm_ppc.h @@ -767,8 +767,7 @@ long kvmppc_h_remove(struct kvm_vcpu *vcpu, unsigned long flags, unsigned long pte_index, unsigned long avpn); long kvmppc_h_bulk_remove(struct kvm_vcpu *vcpu); long kvmppc_h_protect(struct kvm_vcpu *vcpu, unsigned long flags, - unsigned long pte_index, unsigned long avpn, - unsigned long va); + unsigned long pte_index, unsigned long avpn); long kvmppc_h_read(struct kvm_vcpu *vcpu, unsigned long flags, unsigned long pte_index); long kvmppc_h_clear_ref(struct kvm_vcpu *vcpu, unsigned long flags, diff --git a/arch/powerpc/kvm/book3s_hv_rm_mmu.c b/arch/powerpc/kvm/book3s_hv_rm_mmu.c index 88da2764c1bb..7af7c70f1468 100644 --- a/arch/powerpc/kvm/book3s_hv_rm_mmu.c +++ b/arch/powerpc/kvm/book3s_hv_rm_mmu.c @@ -673,8 +673,7 @@ long kvmppc_h_bulk_remove(struct kvm_vcpu *vcpu) } long kvmppc_h_protect(struct kvm_vcpu *vcpu, unsigned long flags, - unsigned long pte_index, unsigned long avpn, - unsigned long va) + unsigned long pte_index, unsigned long avpn) { struct kvm *kvm = vcpu->kvm; __be64 *hpte; From patchwork Thu Feb 25 13:46:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444362 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=Vv7+ddpk; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYyX73Qwz9sVy for ; Fri, 26 Feb 2021 00:48:04 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230494AbhBYNrx (ORCPT ); Thu, 25 Feb 2021 08:47:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230467AbhBYNrw (ORCPT ); Thu, 25 Feb 2021 08:47:52 -0500 Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 29653C061756 for ; Thu, 25 Feb 2021 05:47:12 -0800 (PST) Received: by mail-pg1-x52e.google.com with SMTP id l2so3826937pgb.1 for ; Thu, 25 Feb 2021 05:47:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5WGq83w4nIFDwUXoN/x8ZP7+yYqxduj9t3wxP04w6lk=; b=Vv7+ddpkzgRhh9YE/LxLdBIQv0wC/AEbSC/bFbZXXuaa04hFdzuuivvmoimJ1F8RgM rURFyGBMf3CY46OvGP3qV/6CE2gdfRA25GbmWoxe+FozAoLGbgB9fp16C9oCNU/33VdL wj+pt8WStmbUIeAIBonVDS5qxfDUh6GHHnPDyDQDN0TaryzBgAVzyezFz9T+qIIWBE1C 0/YmFmkKiuwDSSS6/GqRocgRctogoEVWVwQCzxjfax7PfSgfIyMV8sdSBo3x0LntDxE9 533fO0ezxc+9TEb2vvdEBYzmomcIVNAIO5Sxov/zf6pNYPvKFGeOEiooEgpV5r3oxz7m GaTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5WGq83w4nIFDwUXoN/x8ZP7+yYqxduj9t3wxP04w6lk=; b=ZkLT2evREu3h+mA/DGp8vG9OI+zge4cU6gAoDhLUzcsTNpAkskSaU4sa8Z2oMfRduZ SvDU3fCTfIlbdwdLG9cg6nhuEFgvlnbfC4shyLOuKkZdrFmxP7R18w3b/uuiDOXVvnXy jOSmi83grF/OENJSwEGLBnSHmTlqkUwGTJWq1ifGcRGGByndXmHjcjc7ASa8U4PlCV4L bmBV4UcOI77LtJbzrd4pvU/mDpjm0C8nwTzFu4x9+DJK/XNQ7GCj/+sTGdpWDOdlVRtf /EbNGHKj8Xfx/GNRGSq2IwUH/yQzYABTGHAceVATwm75M7ZwNg4shktbegERqrpLEk5s gaDA== X-Gm-Message-State: AOAM530hyW+25DFMkliXkSdxOktao/SSJQcFN++GKBL/3H/ysLxOKZa5 gj9KJXmX90wZG0eJeKFExwrB8R6l+q8= X-Google-Smtp-Source: ABdhPJy4stBtzZ4ef0AuMSuecUMTr51LG8sJdPr/qiCdGBmQ4BlF+bu+U1GlqC0zzjjuUvXb7e/S3w== X-Received: by 2002:a62:25c7:0:b029:156:72a3:b0c0 with SMTP id l190-20020a6225c70000b029015672a3b0c0mr3312010pfl.59.1614260831378; Thu, 25 Feb 2021 05:47:11 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:10 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 02/37] KVM: PPC: Book3S HV: Fix CONFIG_SPAPR_TCE_IOMMU=n default hcalls Date: Thu, 25 Feb 2021 23:46:17 +1000 Message-Id: <20210225134652.2127648-3-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org This config option causes the warning in init_default_hcalls to fire because the TCE handlers are in the default hcall list but not implemented. Signed-off-by: Nicholas Piggin Reviewed-by: Daniel Axtens --- arch/powerpc/kvm/book3s_hv.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 13bad6bf4c95..895090636295 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -5369,8 +5369,10 @@ static unsigned int default_hcall_list[] = { H_READ, H_PROTECT, H_BULK_REMOVE, +#ifdef CONFIG_SPAPR_TCE_IOMMU H_GET_TCE, H_PUT_TCE, +#endif H_SET_DABR, H_SET_XDABR, H_CEDE, From patchwork Thu Feb 25 13:46:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444363 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=HhY4kn+4; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYyY4q94z9sWW for ; Fri, 26 Feb 2021 00:48:05 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230467AbhBYNr5 (ORCPT ); Thu, 25 Feb 2021 08:47:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44576 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230166AbhBYNr4 (ORCPT ); Thu, 25 Feb 2021 08:47:56 -0500 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EAF9CC061786 for ; Thu, 25 Feb 2021 05:47:15 -0800 (PST) Received: by mail-pj1-x1034.google.com with SMTP id u12so3318670pjr.2 for ; Thu, 25 Feb 2021 05:47:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PQ5MsB/7FGz/hp4mvby/IngotkcaBU3resUOZZYOYwQ=; b=HhY4kn+4A52rg4eUJqgs1XKXan9wsa8JPqLC3eA0f+yqOdX2ZpkIqvbuN/g77kpbkD sPosXd4uKndvPb0ihaPDv7+wXzvO7bTKNf4+YOQfFXFg7WlCjbNtZyLVMix4LhCNq1MS TcnJOg7D1xW1OLTdamucwA0lIJb4V5zOiCYse5gp17AH+N+61tdX5Ylnd9zCOKDnjtIK U+gPNs+IUnsZzg1XFxNslhW8ra5VO7msMCPm2/u+I8PgeXXbsQcKn8EE+J7UkvBAekM+ WfsDSIr5hEQQd+35KEtOInM7B1WUn8lNrzuUDVox84JVfLnQjSyL5ih6W4WeXJfj0nJd 1/UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PQ5MsB/7FGz/hp4mvby/IngotkcaBU3resUOZZYOYwQ=; b=fQ42iwRaLWHxNTnUxDJbwzpAC0o+YFas+K2gt69OFXaH3DjSXlBNHx4b7ArC8n5R/X BEM2PQZoEyZkr2taVBzqFl0gebhR97ED9lqmPkgm/n8tlIpE16016Q9N19++qyixoRNx 5u9BcITnLoBWMOyP472qdH9/0kSFKrfl8/TEmqHC35Oq1qYhxz1a9bSHaBeQaGCHcWdO Hu436OoprObSS5ab+1TFvOtsS93UJaJiRWNOhQhACpGZhejc4kRGlyrToaN3R0IUTMxk 0Ag4FWnfSGRsOj7f0/2bOIRXFZJh89o2B/iAtnZxqQAnlfvxj1xWonBr52U+7YI0OpNI 0vAQ== X-Gm-Message-State: AOAM533Mztuy8je6JfpGYroIf/dU0BXmUcuGFChmZev9+2FECQOQhJgT p7mW/VrV+39kv7scEKsn8NUTACGKUyU= X-Google-Smtp-Source: ABdhPJysSqXiQZVbglCTK6LYw197cPu65igol+mtRJ9WMgCDydIOdFnpcLBayFgC7m+/QntFqOgZKQ== X-Received: by 2002:a17:90a:5302:: with SMTP id x2mr3506949pjh.232.1614260835067; Thu, 25 Feb 2021 05:47:15 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:14 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org, Fabiano Rosas Subject: [PATCH v2 03/37] powerpc/64s: Remove KVM handler support from CBE_RAS interrupts Date: Thu, 25 Feb 2021 23:46:18 +1000 Message-Id: <20210225134652.2127648-4-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Cell does not support KVM. Reviewed-by: Fabiano Rosas Signed-off-by: Nicholas Piggin --- arch/powerpc/kernel/exceptions-64s.S | 6 ------ 1 file changed, 6 deletions(-) diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index 60d3051a8bc8..a027600beeb1 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S @@ -2530,8 +2530,6 @@ EXC_VIRT_NONE(0x5100, 0x100) INT_DEFINE_BEGIN(cbe_system_error) IVEC=0x1200 IHSRR=1 - IKVM_SKIP=1 - IKVM_REAL=1 INT_DEFINE_END(cbe_system_error) EXC_REAL_BEGIN(cbe_system_error, 0x1200, 0x100) @@ -2701,8 +2699,6 @@ EXC_COMMON_BEGIN(denorm_exception_common) INT_DEFINE_BEGIN(cbe_maintenance) IVEC=0x1600 IHSRR=1 - IKVM_SKIP=1 - IKVM_REAL=1 INT_DEFINE_END(cbe_maintenance) EXC_REAL_BEGIN(cbe_maintenance, 0x1600, 0x100) @@ -2754,8 +2750,6 @@ EXC_COMMON_BEGIN(altivec_assist_common) INT_DEFINE_BEGIN(cbe_thermal) IVEC=0x1800 IHSRR=1 - IKVM_SKIP=1 - IKVM_REAL=1 INT_DEFINE_END(cbe_thermal) EXC_REAL_BEGIN(cbe_thermal, 0x1800, 0x100) From patchwork Thu Feb 25 13:46:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444364 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=OPHu1jGz; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYyZ2Qzxz9sVn for ; Fri, 26 Feb 2021 00:48:06 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230498AbhBYNsA (ORCPT ); Thu, 25 Feb 2021 08:48:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44592 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230166AbhBYNr7 (ORCPT ); Thu, 25 Feb 2021 08:47:59 -0500 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 428B1C061788 for ; Thu, 25 Feb 2021 05:47:19 -0800 (PST) Received: by mail-pj1-x1036.google.com with SMTP id o22so5063271pjs.1 for ; Thu, 25 Feb 2021 05:47:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eJhojtxV4qkzw509yP6Eq/ZPkxtU7/fUkZd3OCkgR3c=; b=OPHu1jGz9QvQ28B3ue7zbnd81xaCapKmz+ha0rh06dZZFqf6JHdholBZwFdPYXZuSi QJyFsGVusNacpanQWnLBFrpXQieY/L2gm9N/DQ1PCGTpgFgunUCRBE7f3Fsu4lZrnA73 RHHANpxa1j9t/WQdLs1KurNglGkt8wH0HNQcFWkgk4b+5IRpv+N402x8YH9lwnatE00V uztsvC/8Jo59deOsARo6I/XKl4SbnJABkrQvKWhOEotTC/7/iE1hYVYpuRd7/LO8rl9A hmR31ce1LT94V7wuB3CJKOHaFmdzppfgQSpkt6RRtX51Q27OXmbv9S1dB8zduUBVNvO1 ixLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eJhojtxV4qkzw509yP6Eq/ZPkxtU7/fUkZd3OCkgR3c=; b=mHevcUg1x459YqLHxKZsAqlRN5xz+BHqrYj+pWs4cAW6LXDehT+6WgK/EVHMkXGiT6 Ype0rQKGUiPXQSLjdcxPU8PX0zX9CjvAh5RNlNFny3a6CxBsrKFgLuPraYOZGKlT6lVH +pvAjWuJDNronTOhLpDKcnXzXamOZQ8RNjrcR/VrS59QNQ3XhJsVFs6Pe/CTe4K8eRmS yuoacgnedhyRLPxTIp8vjgIt9uTg/0XI3XGE+KR+Dqt0X+bPGFT+IU8DSndCy8X0nhvQ T7p0zxTQrsXdHf+98XkUAnXWuBeZQ6moHPZsWSbIkCMW/I1WWkBT60c3yqoJiWQcbGeg GTiA== X-Gm-Message-State: AOAM532Qk9YKA3gsayAJPJJJjwABCjvAIf17WauYqEC9tsrByCp4gXnI pZ2NAbmZirdwDAOKJj14X6uQWdIEtYw= X-Google-Smtp-Source: ABdhPJyED+odZHpmKwwIYH5DXmA56S7BR1QQijuJXU5K+rDhmPiyYkAyb/pfD9IzVSieJVFjAaUYyw== X-Received: by 2002:a17:902:e54e:b029:de:8c70:2ec4 with SMTP id n14-20020a170902e54eb02900de8c702ec4mr3260154plf.56.1614260838408; Thu, 25 Feb 2021 05:47:18 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:17 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org, Fabiano Rosas Subject: [PATCH v2 04/37] powerpc/64s: remove KVM SKIP test from instruction breakpoint handler Date: Thu, 25 Feb 2021 23:46:19 +1000 Message-Id: <20210225134652.2127648-5-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org The code being executed in KVM_GUEST_MODE_SKIP is hypervisor code with MSR[IR]=0, so the faults of concern are the d-side ones caused by access to guest context by the hypervisor. Instruction breakpoint interrupts are not a concern here. It's unlikely any good would come of causing breaks in this code, but skipping the instruction that caused it won't help matters (e.g., skip the mtmsr that sets MSR[DR]=0 or clears KVM_GUEST_MODE_SKIP). Reviewed-by: Fabiano Rosas Signed-off-by: Nicholas Piggin Reviewed-by: Daniel Axtens --- arch/powerpc/kernel/exceptions-64s.S | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index a027600beeb1..0097e0676ed7 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S @@ -2553,7 +2553,6 @@ EXC_VIRT_NONE(0x5200, 0x100) INT_DEFINE_BEGIN(instruction_breakpoint) IVEC=0x1300 #ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE - IKVM_SKIP=1 IKVM_REAL=1 #endif INT_DEFINE_END(instruction_breakpoint) From patchwork Thu Feb 25 13:46:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444368 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=eDa097mP; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzH49kfz9sW4 for ; Fri, 26 Feb 2021 00:48:43 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230166AbhBYNs1 (ORCPT ); Thu, 25 Feb 2021 08:48:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229498AbhBYNs0 (ORCPT ); Thu, 25 Feb 2021 08:48:26 -0500 Received: from mail-pg1-x534.google.com (mail-pg1-x534.google.com [IPv6:2607:f8b0:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C63BDC06178A for ; Thu, 25 Feb 2021 05:47:22 -0800 (PST) Received: by mail-pg1-x534.google.com with SMTP id t25so3819110pga.2 for ; Thu, 25 Feb 2021 05:47:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OagdvsHRZnah4S5WUxjULUBGe/jVBE2xruZ6F/aPWm4=; b=eDa097mPBQ2HN5Wk6JrVBt8+mc8GSBtE/2ml74f0TmQVw5xhfZY3Do9HNUqn8rz7Oy FCWDL5OG79mQr9q8ho0/lM93W0xsPP9ZgXp6O4Z/67t3oSZVb8EVV/w+BbqcJzH1HOy0 VL3c7uLbqzofxXGCv0Kqs0Ypq0OsHj9oAFRHbVOXDJDT04lI6enOnD0he1Y5nUXH0+zp Kzl6/6bHFyFcgzhzePL7UjpGWJY3YmNO0QSycY/Lh/kZNYWmc3WWI5nAax+gkmOiJFXz 33pxbYlZyxz5v82Inqc8ig8xUAFVktR0KfPJfxzMcMF0peJG0YCezrnCIEnlkUgpbPi5 8mFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OagdvsHRZnah4S5WUxjULUBGe/jVBE2xruZ6F/aPWm4=; b=N0EkXdy5R3KgjECMXXL32b5z9dWGRJ0kzRyboSOBz9ayQc80aP6WWzqC5OqDEotMYC KdPZmuyaOy8UU4LaWDs8sefpXSfl/z+0+f8LRkqQd0KV8dKECtQnR4Z6F3tP6Ndaz/68 ZItlc7Yv9FvMbLUoC6P2hRGQfydv5iYJnq6YPFkgjxipQv87FRxl4WV3el15+IHv1oe4 ZfgIeEhqq+R3ZlZmu8yBPbX1o7Et/+UzyJPtvTNMjI5zMcnl2wghkGw5yoWCVL20RJLN j6JCHoyQOjaEGnUcesHsz5LocFGOdn9xtb+c1bg1a+GcELRHxinVoP8TnWgBfOTuxbud iHDQ== X-Gm-Message-State: AOAM532etgJflZwHyO64uw0HKocN53lPvtVpClD4hSISv1zKCn34MCI7 2JCFLm4H3hv2HeHbJklkxpOZ01isYbE= X-Google-Smtp-Source: ABdhPJy8fbq/xLrcGgmcY7flvo9j52FHjYt9Qm3IAbEWt0os+CsKbepaRJ5n9rqP6jX4L8W97RCXYw== X-Received: by 2002:a63:1c13:: with SMTP id c19mr2925019pgc.359.1614260841901; Thu, 25 Feb 2021 05:47:21 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:21 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org, Fabiano Rosas Subject: [PATCH v2 05/37] KVM: PPC: Book3S HV: Ensure MSR[ME] is always set in guest MSR Date: Thu, 25 Feb 2021 23:46:20 +1000 Message-Id: <20210225134652.2127648-6-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Rather than add the ME bit to the MSR when the guest is entered, make it clear that the hypervisor does not allow the guest to clear the bit. The ME addition is kept in the code for now, but a future patch will warn if it's not present. Reviewed-by: Fabiano Rosas Signed-off-by: Nicholas Piggin Reviewed-by: Daniel Axtens --- arch/powerpc/kvm/book3s_hv_builtin.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c index 158d309b42a3..1ca484160636 100644 --- a/arch/powerpc/kvm/book3s_hv_builtin.c +++ b/arch/powerpc/kvm/book3s_hv_builtin.c @@ -662,6 +662,13 @@ static void kvmppc_end_cede(struct kvm_vcpu *vcpu) void kvmppc_set_msr_hv(struct kvm_vcpu *vcpu, u64 msr) { + /* + * Guest must always run with machine check interrupt + * enabled. + */ + if (!(msr & MSR_ME)) + msr |= MSR_ME; + /* * Check for illegal transactional state bit combination * and if we find it, force the TS field to a safe state. From patchwork Thu Feb 25 13:46:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444369 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=qjwpcnky; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzJ0NpNz9sVn for ; Fri, 26 Feb 2021 00:48:44 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229498AbhBYNs2 (ORCPT ); Thu, 25 Feb 2021 08:48:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229993AbhBYNs0 (ORCPT ); Thu, 25 Feb 2021 08:48:26 -0500 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 25D51C06178B for ; Thu, 25 Feb 2021 05:47:26 -0800 (PST) Received: by mail-pf1-x42f.google.com with SMTP id w18so3636172pfu.9 for ; Thu, 25 Feb 2021 05:47:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kDl47vuNwaIEo8T2SK3luJ13JGVpmR0EeDbltJicvt0=; b=qjwpcnkyc8NfBbGmoyWtqTAzPEqmdH2AZcHh/SCfM7JwBTKqWQzHxUsxbhYFep105/ wqYo73cerVQUjvfBy83D2LUts2tiAN5B5PTFKunY1A6WU6aLDVDm+MFSCsIhfzL8YyYR w1cNf5X3gZzsdAMp723HmWAnkBShKdf7kXMcIb30hB1AhE7k/h+ui0JhOwCQagS9ZtAo RSQi4JTmRTJw5hIDVisP4CBgkseeeXuwHy/ZrPf0wC8cTl4ZDESukuJSsB5fo1ELGofk 056hHbtEi2knuosMdzOO0Hz0NvYlT+dXeQF6f3dW1LIlY5dwxM1cvWZoOby/aGT3rHE1 B2Sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kDl47vuNwaIEo8T2SK3luJ13JGVpmR0EeDbltJicvt0=; b=t748ckiED0ev+/6p7d9DqJRT7Zi4+qnf9Y4osYs4tJs6DnMmOH7SBZcVFscwre0/h8 ZY6IhAoCfvoEbz4afEaRxN2ZYyrtOSZJMwRqE1cB9yIolm6w1wTwvwABMPaX9gUAa5O8 +nQ8tE1NKgpT75yAaeW++4QAsOpYdu4IDC80k2/a/lSxv/hk4cvThx3ae2Xn2c8WOEzw CMN5mK5o6G4GHKXvcqsYLLRysQ1ijoq81ac8vWvS88PGsh8Xa8GskQQA5RqdcYLTF1Vm hx7mbObSDS9BdFxqOziIzZryUqLOmvz/5HDMLnoLFFT8kFZ6I8MnZ1i2Pf8FgcYETpRA kP6Q== X-Gm-Message-State: AOAM530opWXpcMvcu1kXwAL7dyktHkYU/NbpN6W3ezdB/xSBs+JJ0YNA c3TXyzq9sAYjP6Pu6tff6RL2EfmvrIc= X-Google-Smtp-Source: ABdhPJwEFnmuCnptZE6hr5JOtVkTS9LJIalqDUOnD/EZjsoUscmXBBhk7NAqSTi68cRzHF6RxHOzgg== X-Received: by 2002:a63:1826:: with SMTP id y38mr3060314pgl.252.1614260845210; Thu, 25 Feb 2021 05:47:25 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:24 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org, Fabiano Rosas Subject: [PATCH v2 06/37] KVM: PPC: Book3S 64: move KVM interrupt entry to a common entry point Date: Thu, 25 Feb 2021 23:46:21 +1000 Message-Id: <20210225134652.2127648-7-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Rather than bifurcate the call depending on whether or not HV is possible, and have the HV entry test for PR, just make a single common point which does the demultiplexing. This makes it simpler to add another type of exit handler. Reviewed-by: Fabiano Rosas Signed-off-by: Nicholas Piggin Reviewed-by: Daniel Axtens --- arch/powerpc/kernel/exceptions-64s.S | 8 +----- arch/powerpc/kvm/Makefile | 3 +++ arch/powerpc/kvm/book3s_64_entry.S | 35 +++++++++++++++++++++++++ arch/powerpc/kvm/book3s_hv_rmhandlers.S | 11 ++------ 4 files changed, 41 insertions(+), 16 deletions(-) create mode 100644 arch/powerpc/kvm/book3s_64_entry.S diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index 0097e0676ed7..ba13d749d203 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S @@ -208,7 +208,6 @@ do_define_int n .endm #ifdef CONFIG_KVM_BOOK3S_64_HANDLER -#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE /* * All interrupts which set HSRR registers, as well as SRESET and MCE and * syscall when invoked with "sc 1" switch to MSR[HV]=1 (HVMODE) to be taken, @@ -238,13 +237,8 @@ do_define_int n /* * If an interrupt is taken while a guest is running, it is immediately routed - * to KVM to handle. If both HV and PR KVM arepossible, KVM interrupts go first - * to kvmppc_interrupt_hv, which handles the PR guest case. + * to KVM to handle. */ -#define kvmppc_interrupt kvmppc_interrupt_hv -#else -#define kvmppc_interrupt kvmppc_interrupt_pr -#endif .macro KVMTEST name lbz r10,HSTATE_IN_GUEST(r13) diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile index 2bfeaa13befb..cdd119028f64 100644 --- a/arch/powerpc/kvm/Makefile +++ b/arch/powerpc/kvm/Makefile @@ -59,6 +59,9 @@ kvm-pr-y := \ kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \ tm.o +kvm-book3s_64-builtin-objs-y += \ + book3s_64_entry.o + ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE kvm-book3s_64-builtin-objs-$(CONFIG_KVM_BOOK3S_64_HANDLER) += \ book3s_rmhandlers.o diff --git a/arch/powerpc/kvm/book3s_64_entry.S b/arch/powerpc/kvm/book3s_64_entry.S new file mode 100644 index 000000000000..e9a6a8fbb164 --- /dev/null +++ b/arch/powerpc/kvm/book3s_64_entry.S @@ -0,0 +1,35 @@ +#include +#include +#include +#include +#include +#include + +/* + * This is branched to from interrupt handlers in exception-64s.S which set + * IKVM_REAL or IKVM_VIRT, if HSTATE_IN_GUEST was found to be non-zero. + */ +.global kvmppc_interrupt +.balign IFETCH_ALIGN_BYTES +kvmppc_interrupt: + /* + * Register contents: + * R12 = (guest CR << 32) | interrupt vector + * R13 = PACA + * guest R12 saved in shadow VCPU SCRATCH0 + * guest R13 saved in SPRN_SCRATCH0 + */ +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE + std r9,HSTATE_SCRATCH2(r13) + lbz r9,HSTATE_IN_GUEST(r13) + cmpwi r9,KVM_GUEST_MODE_HOST_HV + beq kvmppc_bad_host_intr +#ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE + cmpwi r9,KVM_GUEST_MODE_GUEST + ld r9,HSTATE_SCRATCH2(r13) + beq kvmppc_interrupt_pr +#endif + b kvmppc_interrupt_hv +#else + b kvmppc_interrupt_pr +#endif diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index 5e634db4809b..f976efb7e4a9 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -1269,16 +1269,8 @@ kvmppc_interrupt_hv: * R13 = PACA * guest R12 saved in shadow VCPU SCRATCH0 * guest R13 saved in SPRN_SCRATCH0 + * guest R9 saved in HSTATE_SCRATCH2 */ - std r9, HSTATE_SCRATCH2(r13) - lbz r9, HSTATE_IN_GUEST(r13) - cmpwi r9, KVM_GUEST_MODE_HOST_HV - beq kvmppc_bad_host_intr -#ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE - cmpwi r9, KVM_GUEST_MODE_GUEST - ld r9, HSTATE_SCRATCH2(r13) - beq kvmppc_interrupt_pr -#endif /* We're now back in the host but in guest MMU context */ li r9, KVM_GUEST_MODE_HOST_HV stb r9, HSTATE_IN_GUEST(r13) @@ -3280,6 +3272,7 @@ END_FTR_SECTION_IFCLR(CPU_FTR_P9_TM_HV_ASSIST) * cfar is saved in HSTATE_CFAR(r13) * ppr is saved in HSTATE_PPR(r13) */ +.global kvmppc_bad_host_intr kvmppc_bad_host_intr: /* * Switch to the emergency stack, but start half-way down in From patchwork Thu Feb 25 13:46:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444370 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=TNTdcoIa; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzJ5vtyz9sWW for ; Fri, 26 Feb 2021 00:48:44 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231274AbhBYNs3 (ORCPT ); Thu, 25 Feb 2021 08:48:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229993AbhBYNs3 (ORCPT ); Thu, 25 Feb 2021 08:48:29 -0500 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7738CC06178C for ; Thu, 25 Feb 2021 05:47:29 -0800 (PST) Received: by mail-pj1-x1034.google.com with SMTP id e9so867027pjs.2 for ; Thu, 25 Feb 2021 05:47:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dnJgPi6pS049zycWvC+smiQcgR5eT/YM0fJSrNlfUGQ=; b=TNTdcoIaotV6bhnyDlk4HBo9c+hJ9K5LNMDyL99YvuYMqWv1IBFpm10/KSoyfqqqgK Gzhjvm14BIv3DZCXoV3nUkcFdBOkILnYtOmBlx/d1uVmMi/ypk+CkuDOOn8+xDfkbHrL F9aDqViKJ+h/T/AREAf/iT/2X5HY7KxUBDFNNIflamucp1vBPNKwnCV4RhRWz9z+plxx CNPYOFcLoge6JKfFw+XVRHQdl7CTDDtIskpni8HDxhzZtUAGg+bmBLgeux/8UUPrWKAm TUfMh/nHye1mc4GRPhobdkVMvEfNAc3yUhkV3nu4SMjxmWmJoz7LGzl8mVJU3TOek3Jg ymbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dnJgPi6pS049zycWvC+smiQcgR5eT/YM0fJSrNlfUGQ=; b=R1TEFk3zE8U720mIA4I4HBWQpLEX7ZDH4LwyBZmW6o54rL4sK1dglG9L5V/mKbY7Rq ulNIEsnBMJSICXW3okXfzb/6/M3Uhm9xGhUt2IwFM7kpSaysAs2VPmWjWo0X2ygLe5V4 wIdv/JEI7zk2Kg6gpgV3AOwK05G0n3HwLh0KLF95V++lhpumoj4/NxNgXxlQgH8tnnC4 b1eEgUSjBswjQbsuazW/TKW66gEjxbQPqJVo0fX+pQ8VxpeiItfxw9SnExxXuVEPi1lt 3TCq6UwPkYv7JKdyv5Xl1dCHhoazKtNJgS0jqMW8y1dEPdDfWYQkZTr1bIfffzbT0dav vNbw== X-Gm-Message-State: AOAM533DJv/xysOTs28fnhkqyH6h+81l8Csb1vga1PLDmgLjY94b6khc 5CUJm5Rey+YR/NoEjEHJckGSgy9OOsw= X-Google-Smtp-Source: ABdhPJwsRomrwb/S97K4G/nsNhT7HFem+VzokZcRLWEvpXd0w9g2bPiiM7p7yIdhQyIJrqXhgY0Shw== X-Received: by 2002:a17:902:b68f:b029:e3:b18:7a48 with SMTP id c15-20020a170902b68fb02900e30b187a48mr3168862pls.65.1614260848494; Thu, 25 Feb 2021 05:47:28 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:27 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org, Fabiano Rosas Subject: [PATCH v2 07/37] KVM: PPC: Book3S 64: Move GUEST_MODE_SKIP test into KVM Date: Thu, 25 Feb 2021 23:46:22 +1000 Message-Id: <20210225134652.2127648-8-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Move the GUEST_MODE_SKIP logic into KVM code. This is quite a KVM internal detail that has no real need to be in common handlers. Also add a comment explaining why this thing exists. Reviewed-by: Fabiano Rosas Signed-off-by: Nicholas Piggin Reviewed-by: Daniel Axtens --- arch/powerpc/kernel/exceptions-64s.S | 60 ---------------------------- arch/powerpc/kvm/book3s_64_entry.S | 51 ++++++++++++++++++++++- 2 files changed, 50 insertions(+), 61 deletions(-) diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index ba13d749d203..d956dd9ed61f 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S @@ -133,7 +133,6 @@ name: #define IBRANCH_TO_COMMON .L_IBRANCH_TO_COMMON_\name\() /* ENTRY branch to common */ #define IREALMODE_COMMON .L_IREALMODE_COMMON_\name\() /* Common runs in realmode */ #define IMASK .L_IMASK_\name\() /* IRQ soft-mask bit */ -#define IKVM_SKIP .L_IKVM_SKIP_\name\() /* Generate KVM skip handler */ #define IKVM_REAL .L_IKVM_REAL_\name\() /* Real entry tests KVM */ #define __IKVM_REAL(name) .L_IKVM_REAL_ ## name #define IKVM_VIRT .L_IKVM_VIRT_\name\() /* Virt entry tests KVM */ @@ -190,9 +189,6 @@ do_define_int n .ifndef IMASK IMASK=0 .endif - .ifndef IKVM_SKIP - IKVM_SKIP=0 - .endif .ifndef IKVM_REAL IKVM_REAL=0 .endif @@ -250,15 +246,10 @@ do_define_int n .balign IFETCH_ALIGN_BYTES \name\()_kvm: - .if IKVM_SKIP - cmpwi r10,KVM_GUEST_MODE_SKIP - beq 89f - .else BEGIN_FTR_SECTION ld r10,IAREA+EX_CFAR(r13) std r10,HSTATE_CFAR(r13) END_FTR_SECTION_IFSET(CPU_FTR_CFAR) - .endif ld r10,IAREA+EX_CTR(r13) mtctr r10 @@ -285,27 +276,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) ori r12,r12,(IVEC) .endif b kvmppc_interrupt - - .if IKVM_SKIP -89: mtocrf 0x80,r9 - ld r10,IAREA+EX_CTR(r13) - mtctr r10 - ld r9,IAREA+EX_R9(r13) - ld r10,IAREA+EX_R10(r13) - ld r11,IAREA+EX_R11(r13) - ld r12,IAREA+EX_R12(r13) - .if IHSRR_IF_HVMODE - BEGIN_FTR_SECTION - b kvmppc_skip_Hinterrupt - FTR_SECTION_ELSE - b kvmppc_skip_interrupt - ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206) - .elseif IHSRR - b kvmppc_skip_Hinterrupt - .else - b kvmppc_skip_interrupt - .endif - .endif .endm #else @@ -1083,7 +1053,6 @@ INT_DEFINE_BEGIN(machine_check) ISET_RI=0 IDAR=1 IDSISR=1 - IKVM_SKIP=1 IKVM_REAL=1 INT_DEFINE_END(machine_check) @@ -1356,7 +1325,6 @@ INT_DEFINE_BEGIN(data_access) IVEC=0x300 IDAR=1 IDSISR=1 - IKVM_SKIP=1 IKVM_REAL=1 INT_DEFINE_END(data_access) @@ -1410,7 +1378,6 @@ ALT_MMU_FTR_SECTION_END_IFCLR(MMU_FTR_TYPE_RADIX) INT_DEFINE_BEGIN(data_access_slb) IVEC=0x380 IDAR=1 - IKVM_SKIP=1 IKVM_REAL=1 INT_DEFINE_END(data_access_slb) @@ -2080,7 +2047,6 @@ INT_DEFINE_BEGIN(h_data_storage) IHSRR=1 IDAR=1 IDSISR=1 - IKVM_SKIP=1 IKVM_REAL=1 IKVM_VIRT=1 INT_DEFINE_END(h_data_storage) @@ -3024,32 +2990,6 @@ EXPORT_SYMBOL(do_uaccess_flush) MASKED_INTERRUPT MASKED_INTERRUPT hsrr=1 -#ifdef CONFIG_KVM_BOOK3S_64_HANDLER -kvmppc_skip_interrupt: - /* - * Here all GPRs are unchanged from when the interrupt happened - * except for r13, which is saved in SPRG_SCRATCH0. - */ - mfspr r13, SPRN_SRR0 - addi r13, r13, 4 - mtspr SPRN_SRR0, r13 - GET_SCRATCH0(r13) - RFI_TO_KERNEL - b . - -kvmppc_skip_Hinterrupt: - /* - * Here all GPRs are unchanged from when the interrupt happened - * except for r13, which is saved in SPRG_SCRATCH0. - */ - mfspr r13, SPRN_HSRR0 - addi r13, r13, 4 - mtspr SPRN_HSRR0, r13 - GET_SCRATCH0(r13) - HRFI_TO_KERNEL - b . -#endif - /* * Relocation-on interrupts: A subset of the interrupts can be delivered * with IR=1/DR=1, if AIL==2 and MSR.HV won't be changed by delivering diff --git a/arch/powerpc/kvm/book3s_64_entry.S b/arch/powerpc/kvm/book3s_64_entry.S index e9a6a8fbb164..c1276f616af4 100644 --- a/arch/powerpc/kvm/book3s_64_entry.S +++ b/arch/powerpc/kvm/book3s_64_entry.S @@ -1,5 +1,6 @@ #include #include +#include #include #include #include @@ -19,9 +20,12 @@ kvmppc_interrupt: * guest R12 saved in shadow VCPU SCRATCH0 * guest R13 saved in SPRN_SCRATCH0 */ -#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE std r9,HSTATE_SCRATCH2(r13) lbz r9,HSTATE_IN_GUEST(r13) + cmpwi r9,KVM_GUEST_MODE_SKIP + beq- .Lmaybe_skip +.Lno_skip: +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE cmpwi r9,KVM_GUEST_MODE_HOST_HV beq kvmppc_bad_host_intr #ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE @@ -33,3 +37,48 @@ kvmppc_interrupt: #else b kvmppc_interrupt_pr #endif + +/* + * KVM uses a trick where it is running in MSR[HV]=1 mode in real-mode with the + * guest MMU context loaded, and it sets KVM_GUEST_MODE_SKIP and enables + * MSR[DR]=1 while leaving MSR[IR]=0, so it continues to fetch HV instructions + * but loads and stores will access the guest context. This is used to load + * the faulting instruction without walking page tables. + * + * However the guest context may not be able to translate, or it may cause a + * machine check or other issue, which will result in a fault in the host + * (even with KVM-HV). + * + * These faults are caught here and if the fault was (or was likely) due to + * that load, then we just return with the PC advanced +4 and skip the load, + * which then goes via the slow path. + */ +.Lmaybe_skip: + cmpwi r12,BOOK3S_INTERRUPT_MACHINE_CHECK + beq 1f + cmpwi r12,BOOK3S_INTERRUPT_DATA_STORAGE + beq 1f + cmpwi r12,BOOK3S_INTERRUPT_DATA_SEGMENT + beq 1f +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE + /* HSRR interrupts have 2 added to trap vector */ + cmpwi r12,BOOK3S_INTERRUPT_H_DATA_STORAGE | 0x2 + beq 2f +#endif + b .Lno_skip +1: mfspr r9,SPRN_SRR0 + addi r9,r9,4 + mtspr SPRN_SRR0,r9 + ld r12,HSTATE_SCRATCH0(r13) + ld r9,HSTATE_SCRATCH2(r13) + GET_SCRATCH0(r13) + RFI_TO_KERNEL +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE +2: mfspr r9,SPRN_HSRR0 + addi r9,r9,4 + mtspr SPRN_HSRR0,r9 + ld r12,HSTATE_SCRATCH0(r13) + ld r9,HSTATE_SCRATCH2(r13) + GET_SCRATCH0(r13) + HRFI_TO_KERNEL +#endif From patchwork Thu Feb 25 13:46:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444371 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=aCwoSEX3; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzK2xh2z9sVt for ; Fri, 26 Feb 2021 00:48:45 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229993AbhBYNsa (ORCPT ); Thu, 25 Feb 2021 08:48:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231154AbhBYNs3 (ORCPT ); Thu, 25 Feb 2021 08:48:29 -0500 Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C833C061793 for ; Thu, 25 Feb 2021 05:47:33 -0800 (PST) Received: by mail-pg1-x531.google.com with SMTP id o63so3811135pgo.6 for ; Thu, 25 Feb 2021 05:47:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AWnzQkuEUPMoWm/4qDOGdHuIkxWe5CpH2emCUPPeV1U=; b=aCwoSEX3CLiQ454/HcUwBUvX4+4SdXsdFI7ce0eGhYjN7OO6qXwC8Zpv30cSV3NsN+ xN58WSX4Mp50JLz1QtcjNVuA3YOeONO6rzxaQUp5KLZM3KbVDY1iwMRQSnUhCtqbxb+C KKIUuUyjjzvnd2//gHKsW0eN/UB6FJg1Qm2lCVrXfMlEVmJLqV+XZvDdkSEhetgGmsDf iKrSVGVsSFJqR79TrKrZbgl+HnU8/6czzDuGqUYxb7h9gL/ZireF8m2Dcld6tfh4Qnq2 CHLsIwppPaqa0Wf8v/0uMej+HhKGkXU977LRRyiDZKMOSqJYXlBL6fjfrjzkIhdSLTLU lTtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AWnzQkuEUPMoWm/4qDOGdHuIkxWe5CpH2emCUPPeV1U=; b=P+pgOoGN5tRIIOBXf5+la8C8njv222ec5T3LCJqjwvBl+5nTtlr3QiVG/lnH1RxK1G jDVqCdAIoBZBA3j29MD1UsHHh6z0VvNIA5P00B06BVMjAYNU+whN14zgHLWq8NYxs0R/ BHh5Bqin1LhwdEjoac71hdOI9Xb9P6maF6fDjebN/IiKWENhvVngRKykfIYAgJQxgsTD RO+BfenhzGS+B8OhCKRoNSTbpVRNl4sJCjzhRSI5bCYo3qpHcXQmKpw+vNymNYlY6StA XLTOLPO2LehaAxO8xGR3cAdc1FdtxQaIxyR76ljz/tQRMasHCBX06d+DDvPR+tSwTn8v 9pIQ== X-Gm-Message-State: AOAM533wWFjT5QsSgZQDuhOrSTBzE4XuN6M7kQOhReUQ/rWwveNgwYIj pyCVR/kRCy682VRJKFpLA0CIwfl0k0Q= X-Google-Smtp-Source: ABdhPJwEMI7Log06mCQHoCSRxy9UFT2J7w4Jc1vzc5d7sNcJknEZi9OvrFEJZxi8S0j9BRjhUpvJwQ== X-Received: by 2002:a63:1350:: with SMTP id 16mr3031040pgt.85.1614260852602; Thu, 25 Feb 2021 05:47:32 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:31 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org, Fabiano Rosas Subject: [PATCH v2 08/37] KVM: PPC: Book3S 64: add hcall interrupt handler Date: Thu, 25 Feb 2021 23:46:23 +1000 Message-Id: <20210225134652.2127648-9-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Add a separate hcall entry point. This can be used to deal with the different calling convention. Reviewed-by: Fabiano Rosas Signed-off-by: Nicholas Piggin Reviewed-by: Daniel Axtens --- arch/powerpc/kernel/exceptions-64s.S | 4 ++-- arch/powerpc/kvm/book3s_64_entry.S | 6 +++++- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index d956dd9ed61f..9ae463e8522b 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S @@ -1992,13 +1992,13 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) * Requires __LOAD_FAR_HANDLER beause kvmppc_interrupt lives * outside the head section. */ - __LOAD_FAR_HANDLER(r10, kvmppc_interrupt) + __LOAD_FAR_HANDLER(r10, kvmppc_hcall) mtctr r10 ld r10,PACA_EXGEN+EX_R10(r13) bctr #else ld r10,PACA_EXGEN+EX_R10(r13) - b kvmppc_interrupt + b kvmppc_hcall #endif #endif diff --git a/arch/powerpc/kvm/book3s_64_entry.S b/arch/powerpc/kvm/book3s_64_entry.S index c1276f616af4..9572f759255c 100644 --- a/arch/powerpc/kvm/book3s_64_entry.S +++ b/arch/powerpc/kvm/book3s_64_entry.S @@ -7,9 +7,13 @@ #include /* - * This is branched to from interrupt handlers in exception-64s.S which set + * These are branched to from interrupt handlers in exception-64s.S which set * IKVM_REAL or IKVM_VIRT, if HSTATE_IN_GUEST was found to be non-zero. */ +.global kvmppc_hcall +.balign IFETCH_ALIGN_BYTES +kvmppc_hcall: + .global kvmppc_interrupt .balign IFETCH_ALIGN_BYTES kvmppc_interrupt: From patchwork Thu Feb 25 13:46:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444372 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=DpImZxPF; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzK63vzz9sVv for ; Fri, 26 Feb 2021 00:48:45 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231414AbhBYNse (ORCPT ); Thu, 25 Feb 2021 08:48:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231154AbhBYNsd (ORCPT ); Thu, 25 Feb 2021 08:48:33 -0500 Received: from mail-pl1-x634.google.com (mail-pl1-x634.google.com [IPv6:2607:f8b0:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5CE3C061794 for ; Thu, 25 Feb 2021 05:47:36 -0800 (PST) Received: by mail-pl1-x634.google.com with SMTP id w18so3200339plc.12 for ; Thu, 25 Feb 2021 05:47:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qqX19oS/0kUyK315AzpZ4ksxHH5Hk/Pabuzt0JgR9ug=; b=DpImZxPF43lWc1GhwnW69j+AZUttLbvEu9PabgH2/ce4o3n9vP8hedwDwUI2i64Jcf E8RTy7C+YrWSfuzrv14T8AC8eMwVjoFr0YDq0Ak8FxC1k7DSuRgisrmYTRqCQH6zoMwa pJc90HoW7P8prRk8V4UM79lpW5ajJ7ce/hDGu0s2rnW2ZP0IwC9/eVnmTScb9odR8yKW 1R5H7PqbccZfD0mKby7OFn8PDg4MD3TqpurGAkxMRMG7uIzL9JNsWd7qMwIepH9+GgwZ /9/gD4Q9abrRowAxCVFaxcYAHAgLB+PF5ujjRAtTDHDKHQqEGnFU0agK/B5zuwqhjtwt 3eUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qqX19oS/0kUyK315AzpZ4ksxHH5Hk/Pabuzt0JgR9ug=; b=fm5C7eu1c4YRTjuqyW2lcVEJ6CIqgD3B+smY1L7g9QESgci5EZ+a7pKMlYz5EDgQE4 zPvTRGRvW9MFKXfG5e+ZI8ugmwd325M39z/uT4xoSkuI2wAimyxwCA7BnlwKaLYXYMIC yS8PlK57ETMYGiZZYe+EBMoJkFn8SA6f2V7Lwn9CuOlUOe6jaDU0ni2Uyi5E4dp7ikis MQuNk605gZ/d/WwXfMX38aytb8okaM5AZVphU7n8H4xXUwzLj/zn1tom5tl+W4/Jk1cO E+FayODQ1d+LtCVfiQ1VPk31lrT3AoC5HfdfSYJAwkfyM1tyPKMSzIfIkFWqvyjy4WiK f1fA== X-Gm-Message-State: AOAM531Um3wp87qgYrnU1OL6qQLDe1i+vm8v04cSXs+R0fcGmAi3ckn6 u7G3M3+njIzV6c6WV7In9RCIHeCT0Po= X-Google-Smtp-Source: ABdhPJy0LoWmelDQ3cXgBvHTDz3l5isyvONZwGo0ZJqbOTllZfHYtzLBlaFs2lFS7U5zc4dTqHBO7g== X-Received: by 2002:a17:90a:c7d5:: with SMTP id gf21mr3286267pjb.165.1614260855795; Thu, 25 Feb 2021 05:47:35 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:35 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 09/37] KVM: PPC: Book3S 64: Move hcall early register setup to KVM Date: Thu, 25 Feb 2021 23:46:24 +1000 Message-Id: <20210225134652.2127648-10-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org System calls / hcalls have a different calling convention than other interrupts, so there is code in the KVMTEST to massage these into the same form as other interrupt handlers. Move this work into the KVM hcall handler. This means teaching KVM a little more about the low level interrupt handler setup, PACA save areas, etc., although that's not obviously worse than the current approach of coming up with an entirely different interrupt register / save convention. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/exception-64s.h | 13 +++++++ arch/powerpc/kernel/exceptions-64s.S | 44 ++---------------------- arch/powerpc/kvm/book3s_64_entry.S | 17 +++++++++ 3 files changed, 32 insertions(+), 42 deletions(-) diff --git a/arch/powerpc/include/asm/exception-64s.h b/arch/powerpc/include/asm/exception-64s.h index c1a8aac01cf9..bb6f78fcf981 100644 --- a/arch/powerpc/include/asm/exception-64s.h +++ b/arch/powerpc/include/asm/exception-64s.h @@ -35,6 +35,19 @@ /* PACA save area size in u64 units (exgen, exmc, etc) */ #define EX_SIZE 10 +/* PACA save area offsets */ +#define EX_R9 0 +#define EX_R10 8 +#define EX_R11 16 +#define EX_R12 24 +#define EX_R13 32 +#define EX_DAR 40 +#define EX_DSISR 48 +#define EX_CCR 52 +#define EX_CFAR 56 +#define EX_PPR 64 +#define EX_CTR 72 + /* * maximum recursive depth of MCE exceptions */ diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index 9ae463e8522b..b7092ba87da8 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S @@ -21,22 +21,6 @@ #include #include -/* PACA save area offsets (exgen, exmc, etc) */ -#define EX_R9 0 -#define EX_R10 8 -#define EX_R11 16 -#define EX_R12 24 -#define EX_R13 32 -#define EX_DAR 40 -#define EX_DSISR 48 -#define EX_CCR 52 -#define EX_CFAR 56 -#define EX_PPR 64 -#define EX_CTR 72 -.if EX_SIZE != 10 - .error "EX_SIZE is wrong" -.endif - /* * Following are fixed section helper macros. * @@ -1964,45 +1948,21 @@ EXC_VIRT_END(system_call, 0x4c00, 0x100) #ifdef CONFIG_KVM_BOOK3S_64_HANDLER TRAMP_REAL_BEGIN(system_call_kvm) - /* - * This is a hcall, so register convention is as above, with these - * differences: - * r13 = PACA - * ctr = orig r13 - * orig r10 saved in PACA - */ - /* - * Save the PPR (on systems that support it) before changing to - * HMT_MEDIUM. That allows the KVM code to save that value into the - * guest state (it is the guest's PPR value). - */ -BEGIN_FTR_SECTION - mfspr r10,SPRN_PPR - std r10,HSTATE_PPR(r13) -END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) - HMT_MEDIUM mfctr r10 - SET_SCRATCH0(r10) - mfcr r10 - std r12,HSTATE_SCRATCH0(r13) - sldi r12,r10,32 - ori r12,r12,0xc00 + SET_SCRATCH0(r10) /* Save r13 in SCRATCH0 */ #ifdef CONFIG_RELOCATABLE /* - * Requires __LOAD_FAR_HANDLER beause kvmppc_interrupt lives + * Requires __LOAD_FAR_HANDLER beause kvmppc_hcall lives * outside the head section. */ __LOAD_FAR_HANDLER(r10, kvmppc_hcall) mtctr r10 - ld r10,PACA_EXGEN+EX_R10(r13) bctr #else - ld r10,PACA_EXGEN+EX_R10(r13) b kvmppc_hcall #endif #endif - /** * Interrupt 0xd00 - Trace Interrupt. * This is a synchronous interrupt in response to instruction step or diff --git a/arch/powerpc/kvm/book3s_64_entry.S b/arch/powerpc/kvm/book3s_64_entry.S index 9572f759255c..1c9518ab7d96 100644 --- a/arch/powerpc/kvm/book3s_64_entry.S +++ b/arch/powerpc/kvm/book3s_64_entry.S @@ -13,6 +13,23 @@ .global kvmppc_hcall .balign IFETCH_ALIGN_BYTES kvmppc_hcall: + /* + * This is a hcall, so register convention is as + * Documentation/powerpc/papr_hcalls.rst, with these additions: + * R13 = PACA + * guest R13 saved in SPRN_SCRATCH0 + * R10 = free + */ +BEGIN_FTR_SECTION + mfspr r10,SPRN_PPR + std r10,HSTATE_PPR(r13) +END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) + HMT_MEDIUM + mfcr r10 + std r12,HSTATE_SCRATCH0(r13) + sldi r12,r10,32 + ori r12,r12,0xc00 + ld r10,PACA_EXGEN+EX_R10(r13) .global kvmppc_interrupt .balign IFETCH_ALIGN_BYTES From patchwork Thu Feb 25 13:46:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444373 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=XZQFGamC; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzL3glWz9sWV for ; Fri, 26 Feb 2021 00:48:46 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231154AbhBYNse (ORCPT ); Thu, 25 Feb 2021 08:48:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44720 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231284AbhBYNsd (ORCPT ); Thu, 25 Feb 2021 08:48:33 -0500 Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46C7EC061797 for ; Thu, 25 Feb 2021 05:47:40 -0800 (PST) Received: by mail-pg1-x52b.google.com with SMTP id p21so3788703pgl.12 for ; Thu, 25 Feb 2021 05:47:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zNXZNgDKBi0C8KqgpIDfTtmWQKfiUrv4N0CE+Ra0iZg=; b=XZQFGamCesp/F62MM4QGZ1Kl1kSNWJx/Opd/Eqt7w/0IM4SyHDUPL0LGgMzccqNsOM W3qr6FNGpbw5PAuo2XkPBD37wQFoXet/sAwdE6iEdK4TEBRkLU+oSe++teJZuc/vWtia 3rdUjmr1Nb1pMPYfQIccI7smhjTv/rbJp1U/Nti4qyIRQ8kmI5dc/BUA8QGU9aLaY2U1 5DUj2l2S3+2gB07pBBeaQXTzPuxLqLD0M8FbTkKN2WRaxfVDV2R/ApaFfyeiqtml7Yh6 8gzG1hJYmMQZSUj6KsPItl3LwSDVF8b3WeAO7WidhNfPB16PJGQGvt9uyOEoSb/F3lhk NiSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zNXZNgDKBi0C8KqgpIDfTtmWQKfiUrv4N0CE+Ra0iZg=; b=IeOpzJ4WU+3KhUB1/cBRswOfH+mxbF1D2PE/cTCrQ2KeCX7aHRQ0V7rqSbs+RB+JIO u/gAe4CC50AEog5btfFHUs+m6erpatDstzA3PJ+9KTnT4Dy8cBpGT0i7bLS1Y+bSBK9H m4r7xXK3BgENFzWpcyL6wum7WMCjpKN060Rkw7oVQwTzWI0Ml5dX0Z3Byp7zje0kzbIq RfAT02OEFX36li4dAirP/oHNTPIl2jXrC4zH8+SLSAOiWimp2kicOpXIDcFtju2/Ii2u RqjZJpeUpiugPRn9MbfJ+0A1rHqD6OpY/thh5NwCKdNZt/iwWPwhmc9kQqc7wZ0Vs4Ta Ddgw== X-Gm-Message-State: AOAM531mNrAJl6PHki6VjCEP5zM9GJqJWxPcriV+UJ0xGaK3IRKsyC+I AULoZPcFYumLILGRzzOVEC8NYN5Xm8Q= X-Google-Smtp-Source: ABdhPJyM8kIs/7CzzirzwwiHni0pUpG6wfANGqcHaQ5Ldjag8TXRN1ja9TnB+4iX4wFIXsJsvN3CQg== X-Received: by 2002:a63:1f46:: with SMTP id q6mr3008343pgm.411.1614260859172; Thu, 25 Feb 2021 05:47:39 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:38 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 10/37] KVM: PPC: Book3S 64: Move interrupt early register setup to KVM Date: Thu, 25 Feb 2021 23:46:25 +1000 Message-Id: <20210225134652.2127648-11-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Like the earlier patch for hcalls, KVM interrupt entry requires a different calling convention than the Linux interrupt handlers set up. Move the code that converts from one to the other into KVM. Signed-off-by: Nicholas Piggin --- arch/powerpc/kernel/exceptions-64s.S | 126 ++++----------------------- arch/powerpc/kvm/book3s_64_entry.S | 34 +++++++- 2 files changed, 50 insertions(+), 110 deletions(-) diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index b7092ba87da8..bbda628ab344 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S @@ -187,7 +187,6 @@ do_define_int n .endif .endm -#ifdef CONFIG_KVM_BOOK3S_64_HANDLER /* * All interrupts which set HSRR registers, as well as SRESET and MCE and * syscall when invoked with "sc 1" switch to MSR[HV]=1 (HVMODE) to be taken, @@ -220,54 +219,25 @@ do_define_int n * to KVM to handle. */ -.macro KVMTEST name +.macro KVMTEST name handler +#ifdef CONFIG_KVM_BOOK3S_64_HANDLER lbz r10,HSTATE_IN_GUEST(r13) cmpwi r10,0 - bne \name\()_kvm -.endm - -.macro GEN_KVM name - .balign IFETCH_ALIGN_BYTES -\name\()_kvm: - -BEGIN_FTR_SECTION - ld r10,IAREA+EX_CFAR(r13) - std r10,HSTATE_CFAR(r13) -END_FTR_SECTION_IFSET(CPU_FTR_CFAR) - - ld r10,IAREA+EX_CTR(r13) - mtctr r10 -BEGIN_FTR_SECTION - ld r10,IAREA+EX_PPR(r13) - std r10,HSTATE_PPR(r13) -END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) - ld r11,IAREA+EX_R11(r13) - ld r12,IAREA+EX_R12(r13) - std r12,HSTATE_SCRATCH0(r13) - sldi r12,r9,32 - ld r9,IAREA+EX_R9(r13) - ld r10,IAREA+EX_R10(r13) /* HSRR variants have the 0x2 bit added to their trap number */ .if IHSRR_IF_HVMODE BEGIN_FTR_SECTION - ori r12,r12,(IVEC + 0x2) + li r10,(IVEC + 0x2) FTR_SECTION_ELSE - ori r12,r12,(IVEC) + li r10,(IVEC) ALT_FTR_SECTION_END_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206) .elseif IHSRR - ori r12,r12,(IVEC+ 0x2) + li r10,(IVEC + 0x2) .else - ori r12,r12,(IVEC) + li r10,(IVEC) .endif - b kvmppc_interrupt -.endm - -#else -.macro KVMTEST name -.endm -.macro GEN_KVM name -.endm + bne \handler #endif +.endm /* * This is the BOOK3S interrupt entry code macro. @@ -409,7 +379,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_CFAR) DEFINE_FIXED_SYMBOL(\name\()_common_real) \name\()_common_real: .if IKVM_REAL - KVMTEST \name + KVMTEST \name kvm_interrupt .endif ld r10,PACAKMSR(r13) /* get MSR value for kernel */ @@ -432,7 +402,7 @@ DEFINE_FIXED_SYMBOL(\name\()_common_real) DEFINE_FIXED_SYMBOL(\name\()_common_virt) \name\()_common_virt: .if IKVM_VIRT - KVMTEST \name + KVMTEST \name kvm_interrupt 1: .endif .endif /* IVIRT */ @@ -446,7 +416,7 @@ DEFINE_FIXED_SYMBOL(\name\()_common_virt) DEFINE_FIXED_SYMBOL(\name\()_common_real) \name\()_common_real: .if IKVM_REAL - KVMTEST \name + KVMTEST \name kvm_interrupt .endif .endm @@ -967,8 +937,6 @@ EXC_COMMON_BEGIN(system_reset_common) EXCEPTION_RESTORE_REGS RFI_TO_USER_OR_KERNEL - GEN_KVM system_reset - /** * Interrupt 0x200 - Machine Check Interrupt (MCE). @@ -1132,7 +1100,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206) /* * Check if we are coming from guest. If yes, then run the normal * exception handler which will take the - * machine_check_kvm->kvmppc_interrupt branch to deliver the MC event + * machine_check_kvm->kvm_interrupt branch to deliver the MC event * to guest. */ lbz r11,HSTATE_IN_GUEST(r13) @@ -1203,8 +1171,6 @@ EXC_COMMON_BEGIN(machine_check_common) bl machine_check_exception b interrupt_return - GEN_KVM machine_check - #ifdef CONFIG_PPC_P7_NAP /* @@ -1339,8 +1305,6 @@ ALT_MMU_FTR_SECTION_END_IFCLR(MMU_FTR_TYPE_RADIX) REST_NVGPRS(r1) b interrupt_return - GEN_KVM data_access - /** * Interrupt 0x380 - Data Segment Interrupt (DSLB). @@ -1390,8 +1354,6 @@ ALT_MMU_FTR_SECTION_END_IFCLR(MMU_FTR_TYPE_RADIX) bl do_bad_slb_fault b interrupt_return - GEN_KVM data_access_slb - /** * Interrupt 0x400 - Instruction Storage Interrupt (ISI). @@ -1428,8 +1390,6 @@ MMU_FTR_SECTION_ELSE ALT_MMU_FTR_SECTION_END_IFCLR(MMU_FTR_TYPE_RADIX) b interrupt_return - GEN_KVM instruction_access - /** * Interrupt 0x480 - Instruction Segment Interrupt (ISLB). @@ -1474,8 +1434,6 @@ ALT_MMU_FTR_SECTION_END_IFCLR(MMU_FTR_TYPE_RADIX) bl do_bad_slb_fault b interrupt_return - GEN_KVM instruction_access_slb - /** * Interrupt 0x500 - External Interrupt. @@ -1521,8 +1479,6 @@ EXC_COMMON_BEGIN(hardware_interrupt_common) bl do_IRQ b interrupt_return - GEN_KVM hardware_interrupt - /** * Interrupt 0x600 - Alignment Interrupt @@ -1550,8 +1506,6 @@ EXC_COMMON_BEGIN(alignment_common) REST_NVGPRS(r1) /* instruction emulation may change GPRs */ b interrupt_return - GEN_KVM alignment - /** * Interrupt 0x700 - Program Interrupt (program check). @@ -1659,8 +1613,6 @@ EXC_COMMON_BEGIN(program_check_common) REST_NVGPRS(r1) /* instruction emulation may change GPRs */ b interrupt_return - GEN_KVM program_check - /* * Interrupt 0x800 - Floating-Point Unavailable Interrupt. @@ -1710,8 +1662,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_TM) b interrupt_return #endif - GEN_KVM fp_unavailable - /** * Interrupt 0x900 - Decrementer Interrupt. @@ -1751,8 +1701,6 @@ EXC_COMMON_BEGIN(decrementer_common) bl timer_interrupt b interrupt_return - GEN_KVM decrementer - /** * Interrupt 0x980 - Hypervisor Decrementer Interrupt. @@ -1798,8 +1746,6 @@ EXC_COMMON_BEGIN(hdecrementer_common) ld r13,PACA_EXGEN+EX_R13(r13) HRFI_TO_KERNEL - GEN_KVM hdecrementer - /** * Interrupt 0xa00 - Directed Privileged Doorbell Interrupt. @@ -1840,8 +1786,6 @@ EXC_COMMON_BEGIN(doorbell_super_common) #endif b interrupt_return - GEN_KVM doorbell_super - EXC_REAL_NONE(0xb00, 0x100) EXC_VIRT_NONE(0x4b00, 0x100) @@ -1891,7 +1835,7 @@ INT_DEFINE_END(system_call) GET_PACA(r13) std r10,PACA_EXGEN+EX_R10(r13) INTERRUPT_TO_KERNEL - KVMTEST system_call /* uses r10, branch to system_call_kvm */ + KVMTEST system_call kvm_hcall /* uses r10, branch to kvm_hcall */ mfctr r9 #else mr r9,r13 @@ -1947,7 +1891,7 @@ EXC_VIRT_BEGIN(system_call, 0x4c00, 0x100) EXC_VIRT_END(system_call, 0x4c00, 0x100) #ifdef CONFIG_KVM_BOOK3S_64_HANDLER -TRAMP_REAL_BEGIN(system_call_kvm) +TRAMP_REAL_BEGIN(kvm_hcall) mfctr r10 SET_SCRATCH0(r10) /* Save r13 in SCRATCH0 */ #ifdef CONFIG_RELOCATABLE @@ -1987,8 +1931,6 @@ EXC_COMMON_BEGIN(single_step_common) bl single_step_exception b interrupt_return - GEN_KVM single_step - /** * Interrupt 0xe00 - Hypervisor Data Storage Interrupt (HDSI). @@ -2027,8 +1969,6 @@ MMU_FTR_SECTION_ELSE ALT_MMU_FTR_SECTION_END_IFSET(MMU_FTR_TYPE_RADIX) b interrupt_return - GEN_KVM h_data_storage - /** * Interrupt 0xe20 - Hypervisor Instruction Storage Interrupt (HISI). @@ -2054,8 +1994,6 @@ EXC_COMMON_BEGIN(h_instr_storage_common) bl unknown_exception b interrupt_return - GEN_KVM h_instr_storage - /** * Interrupt 0xe40 - Hypervisor Emulation Assistance Interrupt. @@ -2080,8 +2018,6 @@ EXC_COMMON_BEGIN(emulation_assist_common) REST_NVGPRS(r1) /* instruction emulation may change GPRs */ b interrupt_return - GEN_KVM emulation_assist - /** * Interrupt 0xe60 - Hypervisor Maintenance Interrupt (HMI). @@ -2153,8 +2089,6 @@ EXC_COMMON_BEGIN(hmi_exception_early_common) EXCEPTION_RESTORE_REGS hsrr=1 GEN_INT_ENTRY hmi_exception, virt=0 - GEN_KVM hmi_exception_early - EXC_COMMON_BEGIN(hmi_exception_common) GEN_COMMON hmi_exception FINISH_NAP @@ -2162,8 +2096,6 @@ EXC_COMMON_BEGIN(hmi_exception_common) bl handle_hmi_exception b interrupt_return - GEN_KVM hmi_exception - /** * Interrupt 0xe80 - Directed Hypervisor Doorbell Interrupt. @@ -2195,8 +2127,6 @@ EXC_COMMON_BEGIN(h_doorbell_common) #endif b interrupt_return - GEN_KVM h_doorbell - /** * Interrupt 0xea0 - Hypervisor Virtualization Interrupt. @@ -2224,8 +2154,6 @@ EXC_COMMON_BEGIN(h_virt_irq_common) bl do_IRQ b interrupt_return - GEN_KVM h_virt_irq - EXC_REAL_NONE(0xec0, 0x20) EXC_VIRT_NONE(0x4ec0, 0x20) @@ -2270,8 +2198,6 @@ EXC_COMMON_BEGIN(performance_monitor_common) bl performance_monitor_exception b interrupt_return - GEN_KVM performance_monitor - /** * Interrupt 0xf20 - Vector Unavailable Interrupt. @@ -2321,8 +2247,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC) bl altivec_unavailable_exception b interrupt_return - GEN_KVM altivec_unavailable - /** * Interrupt 0xf40 - VSX Unavailable Interrupt. @@ -2371,8 +2295,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX) bl vsx_unavailable_exception b interrupt_return - GEN_KVM vsx_unavailable - /** * Interrupt 0xf60 - Facility Unavailable Interrupt. @@ -2401,8 +2323,6 @@ EXC_COMMON_BEGIN(facility_unavailable_common) REST_NVGPRS(r1) /* instruction emulation may change GPRs */ b interrupt_return - GEN_KVM facility_unavailable - /** * Interrupt 0xf60 - Hypervisor Facility Unavailable Interrupt. @@ -2431,8 +2351,6 @@ EXC_COMMON_BEGIN(h_facility_unavailable_common) REST_NVGPRS(r1) /* XXX Shouldn't be necessary in practice */ b interrupt_return - GEN_KVM h_facility_unavailable - EXC_REAL_NONE(0xfa0, 0x20) EXC_VIRT_NONE(0x4fa0, 0x20) @@ -2462,8 +2380,6 @@ EXC_COMMON_BEGIN(cbe_system_error_common) bl cbe_system_error_exception b interrupt_return - GEN_KVM cbe_system_error - #else /* CONFIG_CBE_RAS */ EXC_REAL_NONE(0x1200, 0x100) EXC_VIRT_NONE(0x5200, 0x100) @@ -2489,8 +2405,6 @@ EXC_COMMON_BEGIN(instruction_breakpoint_common) bl instruction_breakpoint_exception b interrupt_return - GEN_KVM instruction_breakpoint - EXC_REAL_NONE(0x1400, 0x100) EXC_VIRT_NONE(0x5400, 0x100) @@ -2611,8 +2525,6 @@ EXC_COMMON_BEGIN(denorm_exception_common) bl unknown_exception b interrupt_return - GEN_KVM denorm_exception - #ifdef CONFIG_CBE_RAS INT_DEFINE_BEGIN(cbe_maintenance) @@ -2630,8 +2542,6 @@ EXC_COMMON_BEGIN(cbe_maintenance_common) bl cbe_maintenance_exception b interrupt_return - GEN_KVM cbe_maintenance - #else /* CONFIG_CBE_RAS */ EXC_REAL_NONE(0x1600, 0x100) EXC_VIRT_NONE(0x5600, 0x100) @@ -2662,8 +2572,6 @@ EXC_COMMON_BEGIN(altivec_assist_common) #endif b interrupt_return - GEN_KVM altivec_assist - #ifdef CONFIG_CBE_RAS INT_DEFINE_BEGIN(cbe_thermal) @@ -2681,8 +2589,6 @@ EXC_COMMON_BEGIN(cbe_thermal_common) bl cbe_thermal_exception b interrupt_return - GEN_KVM cbe_thermal - #else /* CONFIG_CBE_RAS */ EXC_REAL_NONE(0x1800, 0x100) EXC_VIRT_NONE(0x5800, 0x100) @@ -2935,6 +2841,10 @@ TRAMP_REAL_BEGIN(rfscv_flush_fallback) USE_TEXT_SECTION() + /* conditional branch in KVMTEST can't reach all the way, make a stub */ +kvm_interrupt: + b kvmppc_interrupt + _GLOBAL(do_uaccess_flush) UACCESS_FLUSH_FIXUP_SECTION nop diff --git a/arch/powerpc/kvm/book3s_64_entry.S b/arch/powerpc/kvm/book3s_64_entry.S index 1c9518ab7d96..4603c0709ae3 100644 --- a/arch/powerpc/kvm/book3s_64_entry.S +++ b/arch/powerpc/kvm/book3s_64_entry.S @@ -30,15 +30,45 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) sldi r12,r10,32 ori r12,r12,0xc00 ld r10,PACA_EXGEN+EX_R10(r13) + b do_kvm_interrupt .global kvmppc_interrupt .balign IFETCH_ALIGN_BYTES kvmppc_interrupt: + li r11,PACA_EXGEN + cmpdi r10,0x200 + bgt+ 1f + li r11,PACA_EXMC + beq 1f + li r11,PACA_EXNMI +1: add r11,r11,r13 + +BEGIN_FTR_SECTION + ld r12,EX_CFAR(r11) + std r12,HSTATE_CFAR(r13) +END_FTR_SECTION_IFSET(CPU_FTR_CFAR) + ld r12,EX_CTR(r11) + mtctr r12 +BEGIN_FTR_SECTION + ld r12,EX_PPR(r11) + std r12,HSTATE_PPR(r13) +END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) + ld r12,EX_R12(r11) + std r12,HSTATE_SCRATCH0(r13) + sldi r12,r9,32 + or r12,r12,r10 + ld r9,EX_R9(r11) + ld r10,EX_R10(r11) + ld r11,EX_R11(r11) + +do_kvm_interrupt: /* - * Register contents: + * Hcalls and other interrupts come here after normalising register + * contents and save locations: + * * R12 = (guest CR << 32) | interrupt vector * R13 = PACA - * guest R12 saved in shadow VCPU SCRATCH0 + * guest R12 saved in shadow HSTATE_SCRATCH0 * guest R13 saved in SPRN_SCRATCH0 */ std r9,HSTATE_SCRATCH2(r13) From patchwork Thu Feb 25 13:46:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444375 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=oN/KqaNu; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzS0rnbz9sWV for ; Fri, 26 Feb 2021 00:48:51 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231439AbhBYNsj (ORCPT ); Thu, 25 Feb 2021 08:48:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44736 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231284AbhBYNsh (ORCPT ); Thu, 25 Feb 2021 08:48:37 -0500 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 114A0C0617A7 for ; Thu, 25 Feb 2021 05:47:43 -0800 (PST) Received: by mail-pj1-x1031.google.com with SMTP id t5so5068339pjd.0 for ; Thu, 25 Feb 2021 05:47:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/KksQ3NoTja4HR9q6bStGFmmuukVXyNVWmRp3h0EkF4=; b=oN/KqaNurMawdiWHffSoWaWasXoxM0KkFY2RzTaxl0tQoQsqV/XKeu55fOlkpLafvq t1Kr+vg5b5tMjNHBUS6hAqlABB+hpOdvinrTNUj1jVZU6qMH/vPag8Pgpz/ketWram/X H+89HwZUmYDpJFzxSl4bWIVDj/hNlgvRvnCEvYhAF+NIkcLlRuhLyIvJuhWvUa8YNbq2 MBsJmkCVtJC+QDaFvKmeGy85V0MPI+xjM2M1tFluyakhZbYw4bz5HyuNN8rI+izRGFEm jgBfKLhcvs4h35DPgAQ90PhKNKIj0jBHPVC+r6TjoPfm0kfRTZOWfZ7hKLW/uccnFG4u m6Qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/KksQ3NoTja4HR9q6bStGFmmuukVXyNVWmRp3h0EkF4=; b=XQrKXWY+eVy9r+SdDypCGR9vG4F7FCxI0/LJ92iQjCROMlNb6lX3vKpIjsl+7evkyp fWPFLuILYF2WqQRGpJmK35okBSQNDtxYhvnk8liOfMqS8qNlUjqwb2cOC7Ya1S6nCx9R OEb4ihSFFhMupEihHcF0UHNAqB9mZT42EtxlVWLPLtsQ2sRZ79dI5YNxjKEl4NtMey7/ IH8T/+3ZjuI0UbZomT630SW9jXkzV0p1a87u87VbAh4mrXAkEv1aBWRvBlNvafaN3tVc n6Hxyo9IEXxpE/F7YpjQywJ4tOVwj5LXFgqdEY2FSBV1jI5ddlA+M7uBXaH6CBlFkgS/ JhqA== X-Gm-Message-State: AOAM531qibpJI49VtRBkmaf5iToLbCOCBhFoFULaoKfePu2kuNBhdHxl 74AxIO1YOMHgaxC1Dt/GS15FQFGfX8U= X-Google-Smtp-Source: ABdhPJylqeBoTtCQhFBwmeRqFOcRawDo/ooewyuCSswjXpp1ZIYz5XY+O9HoV7jNNDlAzLZFqxoX5g== X-Received: by 2002:a17:90a:b782:: with SMTP id m2mr3378164pjr.220.1614260862150; Thu, 25 Feb 2021 05:47:42 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:41 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 11/37] KVM: PPC: Book3S 64: move bad_host_intr check to HV handler Date: Thu, 25 Feb 2021 23:46:26 +1000 Message-Id: <20210225134652.2127648-12-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org This is not used by PR KVM. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_64_entry.S | 3 --- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 4 +++- arch/powerpc/kvm/book3s_segment.S | 7 +++++++ 3 files changed, 10 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_entry.S b/arch/powerpc/kvm/book3s_64_entry.S index 4603c0709ae3..75accb1321c9 100644 --- a/arch/powerpc/kvm/book3s_64_entry.S +++ b/arch/powerpc/kvm/book3s_64_entry.S @@ -77,11 +77,8 @@ do_kvm_interrupt: beq- .Lmaybe_skip .Lno_skip: #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE - cmpwi r9,KVM_GUEST_MODE_HOST_HV - beq kvmppc_bad_host_intr #ifdef CONFIG_KVM_BOOK3S_PR_POSSIBLE cmpwi r9,KVM_GUEST_MODE_GUEST - ld r9,HSTATE_SCRATCH2(r13) beq kvmppc_interrupt_pr #endif b kvmppc_interrupt_hv diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index f976efb7e4a9..75405ef53238 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -1265,6 +1265,7 @@ hdec_soon: kvmppc_interrupt_hv: /* * Register contents: + * R9 = HSTATE_IN_GUEST * R12 = (guest CR << 32) | interrupt vector * R13 = PACA * guest R12 saved in shadow VCPU SCRATCH0 @@ -1272,6 +1273,8 @@ kvmppc_interrupt_hv: * guest R9 saved in HSTATE_SCRATCH2 */ /* We're now back in the host but in guest MMU context */ + cmpwi r9,KVM_GUEST_MODE_HOST_HV + beq kvmppc_bad_host_intr li r9, KVM_GUEST_MODE_HOST_HV stb r9, HSTATE_IN_GUEST(r13) @@ -3272,7 +3275,6 @@ END_FTR_SECTION_IFCLR(CPU_FTR_P9_TM_HV_ASSIST) * cfar is saved in HSTATE_CFAR(r13) * ppr is saved in HSTATE_PPR(r13) */ -.global kvmppc_bad_host_intr kvmppc_bad_host_intr: /* * Switch to the emergency stack, but start half-way down in diff --git a/arch/powerpc/kvm/book3s_segment.S b/arch/powerpc/kvm/book3s_segment.S index 1f492aa4c8d6..ef1d88b869bf 100644 --- a/arch/powerpc/kvm/book3s_segment.S +++ b/arch/powerpc/kvm/book3s_segment.S @@ -167,8 +167,15 @@ kvmppc_interrupt_pr: * R12 = (guest CR << 32) | exit handler id * R13 = PACA * HSTATE.SCRATCH0 = guest R12 + * + * If HV is possible, additionally: + * R9 = HSTATE_IN_GUEST + * HSTATE.SCRATCH2 = guest R9 */ #ifdef CONFIG_PPC64 +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE + ld r9,HSTATE_SCRATCH2(r13) +#endif /* Match 32-bit entry */ rotldi r12, r12, 32 /* Flip R12 halves for stw */ stw r12, HSTATE_SCRATCH1(r13) /* CR is now in the low half */ From patchwork Thu Feb 25 13:46:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444374 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=kb3bUEGX; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzM5Rknz9sWc for ; Fri, 26 Feb 2021 00:48:47 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231591AbhBYNsj (ORCPT ); Thu, 25 Feb 2021 08:48:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231439AbhBYNsg (ORCPT ); Thu, 25 Feb 2021 08:48:36 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35A28C0617A9 for ; Thu, 25 Feb 2021 05:47:46 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id e9so867514pjs.2 for ; Thu, 25 Feb 2021 05:47:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KauVfKK9v1PaXpgGlh5JvIFa1U0JoxcWq/+O2Lu+hJo=; b=kb3bUEGXfY0FtDkkBhOm7FxpBiB6LA3hXoDpBRNvQtSHgqdsqOPKnGJNb+h8i+W2kD jBvQOUt4nMvg9L6Opxa49XpQRqHPpZaaelv1uVGcbPWkAjon7gcGXDb+XAx33i1WvGT4 1rzuPh3WYDWF/YumOqdY+dlH9WCPTDeLhqrX6e6JCjKam0TS6d+WpadaUpz9Vmb9jtRH UUUIzipFNl6u8OjfpLcYZUBmFqgKjh9QwPCHWpkBxTJ3tF7+HYult0YpRaXJojWuHFYR nt7vtHx8X8R0IiWd6pM3M1wIBt87yhdYIrQLRO1L+AFCzDOm9BRhfYo9ScQ5dehdCrCb mFbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KauVfKK9v1PaXpgGlh5JvIFa1U0JoxcWq/+O2Lu+hJo=; b=EGoaSvdvLCffuRDJkI7/CXJ9YoDJjxL17OSYmk82BfYab8aYO4UNFQW+7K42QP2llM jAcbzwIUzkAXoEeygLIMp0Zth+N1JEqN0+FmpLEgN7TArBKTDznjvJtMOnjep3F77xnN X/Z1D06rxe+l9GCYUb6gs7mJMN1uUCNnWUgbvsqoJG24+FaTzZmwlJxbtlMqyLGQ/5lK 81LdHV4aBeXiDjZStxZ73Hw2eoo4Snv349qotVR/i++QuNytdFOYdTtMSa8/I5QNaFU6 vpLP7r25rZt3Tk4cF74T338emKRKEuOfW3o57RMpqUFKS2ODypJJ8UuxWKfanSH99Mtd YeGw== X-Gm-Message-State: AOAM530HdZE7zw+Y5J8eYhgliNO47tWZkkYbgagqx/HMqfuyabtonIcd FNEnGJl9qgheq41mvBdkJ01VAukK6aM= X-Google-Smtp-Source: ABdhPJw2SWurXvHLMFh8Yl6Ug+xl7KifH4v7V10LgKHzNwmgS8UAexzi5tmBK9eQ9ZhWhO8GE2nX0A== X-Received: by 2002:a17:90a:be16:: with SMTP id a22mr3561956pjs.32.1614260865303; Thu, 25 Feb 2021 05:47:45 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:44 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 12/37] KVM: PPC: Book3S 64: Minimise hcall handler calling convention differences Date: Thu, 25 Feb 2021 23:46:27 +1000 Message-Id: <20210225134652.2127648-13-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org This sets up the same calling convention from interrupt entry to KVM interrupt handler for system calls as exists for other interrupt types. This is a better API, it uses a save area rather than SPR, and it has more registers free to use. Using a single common API helps maintain it, and it becomes easier to use in C in a later patch. Signed-off-by: Nicholas Piggin --- arch/powerpc/kernel/exceptions-64s.S | 16 +++++++++++++++- arch/powerpc/kvm/book3s_64_entry.S | 22 +++------------------- 2 files changed, 18 insertions(+), 20 deletions(-) diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index bbda628ab344..dcd71d9e7913 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S @@ -1892,8 +1892,22 @@ EXC_VIRT_END(system_call, 0x4c00, 0x100) #ifdef CONFIG_KVM_BOOK3S_64_HANDLER TRAMP_REAL_BEGIN(kvm_hcall) + std r9,PACA_EXGEN+EX_R9(r13) + std r11,PACA_EXGEN+EX_R11(r13) + std r12,PACA_EXGEN+EX_R12(r13) + mfcr r9 mfctr r10 - SET_SCRATCH0(r10) /* Save r13 in SCRATCH0 */ + std r10,PACA_EXGEN+EX_R13(r13) + li r10,0 + std r10,PACA_EXGEN+EX_CFAR(r13) + std r10,PACA_EXGEN+EX_CTR(r13) +BEGIN_FTR_SECTION + mfspr r10,SPRN_PPR + std r10,PACA_EXGEN+EX_PPR(r13) +END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) + + HMT_MEDIUM + #ifdef CONFIG_RELOCATABLE /* * Requires __LOAD_FAR_HANDLER beause kvmppc_hcall lives diff --git a/arch/powerpc/kvm/book3s_64_entry.S b/arch/powerpc/kvm/book3s_64_entry.S index 75accb1321c9..f826c8dc2e19 100644 --- a/arch/powerpc/kvm/book3s_64_entry.S +++ b/arch/powerpc/kvm/book3s_64_entry.S @@ -13,24 +13,9 @@ .global kvmppc_hcall .balign IFETCH_ALIGN_BYTES kvmppc_hcall: - /* - * This is a hcall, so register convention is as - * Documentation/powerpc/papr_hcalls.rst, with these additions: - * R13 = PACA - * guest R13 saved in SPRN_SCRATCH0 - * R10 = free - */ -BEGIN_FTR_SECTION - mfspr r10,SPRN_PPR - std r10,HSTATE_PPR(r13) -END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) - HMT_MEDIUM - mfcr r10 - std r12,HSTATE_SCRATCH0(r13) - sldi r12,r10,32 - ori r12,r12,0xc00 - ld r10,PACA_EXGEN+EX_R10(r13) - b do_kvm_interrupt + ld r10,PACA_EXGEN+EX_R13(r13) + SET_SCRATCH0(r10) + li r10,0xc00 .global kvmppc_interrupt .balign IFETCH_ALIGN_BYTES @@ -61,7 +46,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) ld r10,EX_R10(r11) ld r11,EX_R11(r11) -do_kvm_interrupt: /* * Hcalls and other interrupts come here after normalising register * contents and save locations: From patchwork Thu Feb 25 13:46:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444376 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=SX/9GTjd; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzT0NM6z9sWc for ; Fri, 26 Feb 2021 00:48:53 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231771AbhBYNsk (ORCPT ); Thu, 25 Feb 2021 08:48:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231284AbhBYNsk (ORCPT ); Thu, 25 Feb 2021 08:48:40 -0500 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43A10C061574 for ; Thu, 25 Feb 2021 05:47:49 -0800 (PST) Received: by mail-pl1-x636.google.com with SMTP id z7so3211994plk.7 for ; Thu, 25 Feb 2021 05:47:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WWGbwKAIWEIlTWS3f4xQdZ+FNhX22RSH9NDIuj/AOdE=; b=SX/9GTjdxLHR0i6tC9CK7s/Bo7xMW8QqezlgCk7B0bQjX5/8u4WRH9QmSHNb3klckT f6BuMszO986JMqcIt1smfdINNDVFuRik6i5SVOF3hANtUCPtsz+1PayA8mNk4f/WwyZH noLVEHD6GR/2EhZmSsd0aHWvGKvN7KzXVGXvF/sQPVksSg9MOvSXV5IL7pE1ajqgjUyQ OuqHuYXHqkqc1KTkRa2MT54rUCO89ZTuCbfuEXqnY0m3VQD0e+XS2Mt+XdFxxGCR1doQ M1kVn3ufAxN58rOkd3Tr/td6tJdMEv5rGCAtlQidPx9W4GaNrNpBL4eZiR4xs5pfvmjM xgyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WWGbwKAIWEIlTWS3f4xQdZ+FNhX22RSH9NDIuj/AOdE=; b=o+0U/4/kGYc2ZftTJQo51dmc4DdQlobR5hieBw/3eArPc7aUNjGsS6/nOJpL7sDNIt QXfaI5jCghqEyGjYWNa+1mEZdVbyTSjsogApRQlG53xvBr8Ku9VINhMnEzQjR4151MVf oXoc/MDbcTCBtiRVuuIX6yTjyy0VEv3Ftb9x7KfYAsyQT7EAujj6kG10/gIVczgAcJlv vgCACJy0DDOyfP3mzDQLCY8tVd/4Q/9ZmWGp+CP5N9Xq/LAdAC1aCocryytxJMHJ7DA5 1FywCvr719x9LBDGu6W/70x0fdRVt/9nPEUAy1I8Sy5ojgedIInkQrqOU0NEvJG3mxKb 9/Yg== X-Gm-Message-State: AOAM531pcMyIzaRScoRUTUkHeqJTDq0RgUBGeCeu+qz9qk05HcWNvYyC hvk+lCTwZm1CCU49n6oThzrOjuU/CIE= X-Google-Smtp-Source: ABdhPJyb4g9ISGJKSwuEbgJE6KTFhMOWM/A48OmDFY6Fgn6f+cohk0PturkrhOi6vYubZU8jkNgCHg== X-Received: by 2002:a17:90a:71c3:: with SMTP id m3mr3372964pjs.186.1614260868341; Thu, 25 Feb 2021 05:47:48 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:47 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 13/37] KVM: PPC: Book3S HV P9: Move radix MMU switching instructions together Date: Thu, 25 Feb 2021 23:46:28 +1000 Message-Id: <20210225134652.2127648-14-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Switching the MMU from radix<->radix mode is tricky particularly as the MMU can remain enabled and requires a certain sequence of SPR updates. Move these together into their own functions. This also includes the radix TLB check / flush because it's tied in to MMU switching due to tlbiel getting LPID from LPIDR. (XXX: isync / hwsync synchronisation TBD) Signed-off-by: Nicholas Piggin Reviewed-by: Fabiano Rosas --- arch/powerpc/kvm/book3s_hv.c | 55 +++++++++++++++++++++--------------- 1 file changed, 32 insertions(+), 23 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 895090636295..23d6dc04b0e9 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3440,12 +3440,38 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc) trace_kvmppc_run_core(vc, 1); } +static void switch_mmu_to_guest_radix(struct kvm *kvm, struct kvm_vcpu *vcpu, u64 lpcr) +{ + struct kvmppc_vcore *vc = vcpu->arch.vcore; + struct kvm_nested_guest *nested = vcpu->arch.nested; + u32 lpid; + + lpid = nested ? nested->shadow_lpid : kvm->arch.lpid; + + mtspr(SPRN_LPID, lpid); + mtspr(SPRN_LPCR, lpcr); + mtspr(SPRN_PID, vcpu->arch.pid); + isync(); + + /* TLBIEL must have LPIDR set, so set guest LPID before flushing. */ + kvmppc_check_need_tlb_flush(kvm, vc->pcpu, nested); +} + +static void switch_mmu_to_host_radix(struct kvm *kvm, u32 pid) +{ + mtspr(SPRN_PID, pid); + mtspr(SPRN_LPID, kvm->arch.host_lpid); + mtspr(SPRN_LPCR, kvm->arch.host_lpcr); + isync(); +} + /* * Load up hypervisor-mode registers on P9. */ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpcr) { + struct kvm *kvm = vcpu->kvm; struct kvmppc_vcore *vc = vcpu->arch.vcore; s64 hdec; u64 tb, purr, spurr; @@ -3468,12 +3494,12 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, * P8 and P9 suppress the HDEC exception when LPCR[HDICE] = 0, * so set HDICE before writing HDEC. */ - mtspr(SPRN_LPCR, vcpu->kvm->arch.host_lpcr | LPCR_HDICE); + mtspr(SPRN_LPCR, kvm->arch.host_lpcr | LPCR_HDICE); isync(); hdec = time_limit - mftb(); if (hdec < 0) { - mtspr(SPRN_LPCR, vcpu->kvm->arch.host_lpcr); + mtspr(SPRN_LPCR, kvm->arch.host_lpcr); isync(); return BOOK3S_INTERRUPT_HV_DECREMENTER; } @@ -3508,7 +3534,6 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, } mtspr(SPRN_CIABR, vcpu->arch.ciabr); mtspr(SPRN_IC, vcpu->arch.ic); - mtspr(SPRN_PID, vcpu->arch.pid); mtspr(SPRN_PSSCR, vcpu->arch.psscr | PSSCR_EC | (local_paca->kvm_hstate.fake_suspend << PSSCR_FAKE_SUSPEND_LG)); @@ -3522,8 +3547,7 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, mtspr(SPRN_AMOR, ~0UL); - mtspr(SPRN_LPCR, lpcr); - isync(); + switch_mmu_to_guest_radix(kvm, vcpu, lpcr); kvmppc_xive_push_vcpu(vcpu); @@ -3562,7 +3586,6 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, mtspr(SPRN_DAWR1, host_dawr1); mtspr(SPRN_DAWRX1, host_dawrx1); } - mtspr(SPRN_PID, host_pidr); /* * Since this is radix, do a eieio; tlbsync; ptesync sequence in @@ -3577,9 +3600,6 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, if (cpu_has_feature(CPU_FTR_ARCH_31)) asm volatile(PPC_CP_ABORT); - mtspr(SPRN_LPID, vcpu->kvm->arch.host_lpid); /* restore host LPID */ - isync(); - vc->dpdes = mfspr(SPRN_DPDES); vc->vtb = mfspr(SPRN_VTB); mtspr(SPRN_DPDES, 0); @@ -3596,7 +3616,8 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, } mtspr(SPRN_HDEC, 0x7fffffff); - mtspr(SPRN_LPCR, vcpu->kvm->arch.host_lpcr); + + switch_mmu_to_host_radix(kvm, host_pidr); return trap; } @@ -4130,7 +4151,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, { struct kvm_run *run = vcpu->run; int trap, r, pcpu; - int srcu_idx, lpid; + int srcu_idx; struct kvmppc_vcore *vc; struct kvm *kvm = vcpu->kvm; struct kvm_nested_guest *nested = vcpu->arch.nested; @@ -4204,13 +4225,6 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, vc->vcore_state = VCORE_RUNNING; trace_kvmppc_run_core(vc, 0); - if (cpu_has_feature(CPU_FTR_HVMODE)) { - lpid = nested ? nested->shadow_lpid : kvm->arch.lpid; - mtspr(SPRN_LPID, lpid); - isync(); - kvmppc_check_need_tlb_flush(kvm, pcpu, nested); - } - guest_enter_irqoff(); srcu_idx = srcu_read_lock(&kvm->srcu); @@ -4229,11 +4243,6 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, srcu_read_unlock(&kvm->srcu, srcu_idx); - if (cpu_has_feature(CPU_FTR_HVMODE)) { - mtspr(SPRN_LPID, kvm->arch.host_lpid); - isync(); - } - set_irq_happened(trap); kvmppc_set_host_core(pcpu); From patchwork Thu Feb 25 13:46:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444377 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=FH2lfXzV; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzV0htvz9sjD for ; Fri, 26 Feb 2021 00:48:54 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231284AbhBYNsl (ORCPT ); Thu, 25 Feb 2021 08:48:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231604AbhBYNsk (ORCPT ); Thu, 25 Feb 2021 08:48:40 -0500 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51472C06174A for ; Thu, 25 Feb 2021 05:47:53 -0800 (PST) Received: by mail-pl1-x62e.google.com with SMTP id b8so400634plh.0 for ; Thu, 25 Feb 2021 05:47:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lclwaLxX2NLn9E/+rl7sv8a7z7kaIpmV7064pmVpvB8=; b=FH2lfXzV0haQmWcjE+qg0irxB/IKAqQDAmv9zDDCskGxihsrFLZBqeIm0xd/jWOj/7 LkbgHpQM0bAMkEYKWhS4TPsaVxEsgXVTRuWbrYX6/BJOdvdWMErBvVQVP/z5B3aJ+XMo 4rXmyDzJKWls9fxZqN0OFROkLIjjc2ObguxyzmhHo7WtrOl/iqZSz9GPbMtdbB4O7db0 xU1575FdbEabAqJFwPeHffxmw3ErzR3kbb5jDPmdf+hruHg22nCqyCI18FB+zfWAK431 PEXC5h/CtAG3fkBF4IjHjFpCTcmD2MKxJ0Mt1svFCAZI9cfl1yt/huEZxejuUF1ClAR6 BRAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lclwaLxX2NLn9E/+rl7sv8a7z7kaIpmV7064pmVpvB8=; b=KogDB6cBYPO6mMCrFDqIKA9/Omf23ZB7r8mxXJx/8qfXEgNhisuHkUW/gXG9A5iy+Z qPKV60kDiTna9t6wXJvHKuDa2DDQKgPKCYKq1c4Nx4aQP1qfKzMRfi5yMHChtuzYQ/kA fbTI1uTlN7jYcO5XBaQ6WXJVs1kdKVSOcSRQEgXU3/ywwxwwMpUkntfSF8gvZp5/JV2y s8WEL47U1EU6Mx6mXICHNCMua20nMF53RkWAYlJVsLkBCkMLBMqr+wqQwhtJ9r5dh3L8 t+uvWTsKC9l82aMyympscjQYGUtr3/8+CTDhtPq3oNjg8imC/nxsuTeuWmEWYJs2B7KZ y1Pg== X-Gm-Message-State: AOAM533U1MQSPs2hMQ7HwjGpMFcFG7ARADFKRD+/BFHDaC4mu/Ga3DkC HnOYhW8mXzN66xy0lCVLYvl6tpwOZSY= X-Google-Smtp-Source: ABdhPJyotqTRpIhZJOOVk4kAb+HujsJoUqRA3pV56bKCrgBWFWuguD3FK4Ps+mw7GANorVgLvl5bWg== X-Received: by 2002:a17:90a:9909:: with SMTP id b9mr3257736pjp.46.1614260872441; Thu, 25 Feb 2021 05:47:52 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:51 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 14/37] KVM: PPC: Book3S HV P9: implement kvmppc_xive_pull_vcpu in C Date: Thu, 25 Feb 2021 23:46:29 +1000 Message-Id: <20210225134652.2127648-15-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org This is more symmetric with kvmppc_xive_push_vcpu. The extra test in the asm will go away in a later change. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/kvm_ppc.h | 2 ++ arch/powerpc/kvm/book3s_hv.c | 2 ++ arch/powerpc/kvm/book3s_hv_rmhandlers.S | 5 ++++ arch/powerpc/kvm/book3s_xive.c | 34 +++++++++++++++++++++++++ 4 files changed, 43 insertions(+) diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h index 9531b1c1b190..73b1ca5a6471 100644 --- a/arch/powerpc/include/asm/kvm_ppc.h +++ b/arch/powerpc/include/asm/kvm_ppc.h @@ -672,6 +672,7 @@ extern int kvmppc_xive_set_icp(struct kvm_vcpu *vcpu, u64 icpval); extern int kvmppc_xive_set_irq(struct kvm *kvm, int irq_source_id, u32 irq, int level, bool line_status); extern void kvmppc_xive_push_vcpu(struct kvm_vcpu *vcpu); +extern void kvmppc_xive_pull_vcpu(struct kvm_vcpu *vcpu); static inline int kvmppc_xive_enabled(struct kvm_vcpu *vcpu) { @@ -712,6 +713,7 @@ static inline int kvmppc_xive_set_icp(struct kvm_vcpu *vcpu, u64 icpval) { retur static inline int kvmppc_xive_set_irq(struct kvm *kvm, int irq_source_id, u32 irq, int level, bool line_status) { return -ENODEV; } static inline void kvmppc_xive_push_vcpu(struct kvm_vcpu *vcpu) { } +static inline void kvmppc_xive_pull_vcpu(struct kvm_vcpu *vcpu) { } static inline int kvmppc_xive_enabled(struct kvm_vcpu *vcpu) { return 0; } diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 23d6dc04b0e9..e3344d58537d 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3556,6 +3556,8 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, trap = __kvmhv_vcpu_entry_p9(vcpu); + kvmppc_xive_pull_vcpu(vcpu); + /* Advance host PURR/SPURR by the amount used by guest */ purr = mfspr(SPRN_PURR); spurr = mfspr(SPRN_SPURR); diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index 75405ef53238..c11597f815e4 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -1442,6 +1442,11 @@ guest_exit_cont: /* r9 = vcpu, r12 = trap, r13 = paca */ bl kvmhv_accumulate_time #endif #ifdef CONFIG_KVM_XICS + /* If we came in through the P9 short path, xive pull is done in C */ + lwz r0, STACK_SLOT_SHORT_PATH(r1) + cmpwi r0, 0 + bne 1f + /* We are exiting, pull the VP from the XIVE */ lbz r0, VCPU_XIVE_PUSHED(r9) cmpwi cr0, r0, 0 diff --git a/arch/powerpc/kvm/book3s_xive.c b/arch/powerpc/kvm/book3s_xive.c index e7219b6f5f9a..8632fb998a55 100644 --- a/arch/powerpc/kvm/book3s_xive.c +++ b/arch/powerpc/kvm/book3s_xive.c @@ -127,6 +127,40 @@ void kvmppc_xive_push_vcpu(struct kvm_vcpu *vcpu) } EXPORT_SYMBOL_GPL(kvmppc_xive_push_vcpu); +/* + * Pull a vcpu's context from the XIVE on guest exit. + * This assumes we are in virtual mode (MMU on) + */ +void kvmppc_xive_pull_vcpu(struct kvm_vcpu *vcpu) +{ + void __iomem *tima = local_paca->kvm_hstate.xive_tima_virt; + + BUG_ON(!(mfmsr() & MSR_IR)); + BUG_ON(!(mfmsr() & MSR_DR)); + + if (!vcpu->arch.xive_pushed) + return; + + /* + * Sould not have been pushed if there is no tima + */ + if (WARN_ON(!tima)) + return; + + eieio(); + /* First load to pull the context, we ignore the value */ + __raw_readl(tima + TM_SPC_PULL_OS_CTX); + /* Second load to recover the context state (Words 0 and 1) */ + vcpu->arch.xive_saved_state.w01 = __raw_readq(tima + TM_QW1_OS); + + /* Fixup some of the state for the next load */ + vcpu->arch.xive_saved_state.lsmfb = 0; + vcpu->arch.xive_saved_state.ack = 0xff; + vcpu->arch.xive_pushed = 0; + eieio(); +} +EXPORT_SYMBOL_GPL(kvmppc_xive_pull_vcpu); + /* * This is a simple trigger for a generic XIVE IRQ. This must * only be called for interrupts that support a trigger page From patchwork Thu Feb 25 13:46:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444378 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=Zd9/+fJB; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzW4Pw2z9ssP for ; Fri, 26 Feb 2021 00:48:55 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231786AbhBYNsm (ORCPT ); Thu, 25 Feb 2021 08:48:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231604AbhBYNsm (ORCPT ); Thu, 25 Feb 2021 08:48:42 -0500 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BAFAC061756 for ; Thu, 25 Feb 2021 05:47:56 -0800 (PST) Received: by mail-pf1-x42f.google.com with SMTP id m6so3656519pfk.1 for ; Thu, 25 Feb 2021 05:47:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FGYCVyl5qXjwebi/u46IYNl3I3VlFCFB0eRTWRiaPFU=; b=Zd9/+fJBaQaHATcB1cKjB+Ry16+u0gjeugQgYRGrma7/1Creqsv7nrs8HiS92vyF0o fEbKWEflW7epS08ZrVD7aFrcgVfDNKhexxzLhAvURVWEdWQdNxpOe8cvmyjsMSYzXYZh 8iMh4efteUhbF0CQjTRndsRJeP/a19rsBiShhVExmFfQ4iw/t+KTs8SKya0YZO12bUUo bu5Jiks3vQL1kV8wQejRs1drG7Ktxr4m27SZljiUSqYTrkZNK9VwTtOE3ArAtQUZEEXD dmkanXuxSNsFNiBM8rzflOr77vVUnK83Eex99prKmTKpFDUHST5Rvo/P8bAvpyerYQyV bmdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FGYCVyl5qXjwebi/u46IYNl3I3VlFCFB0eRTWRiaPFU=; b=mVvfmTxQXfSleCo3TY1Qy/x36xNYcUfPnM5IrevWhVIpVKCYZAT6uCQYUw5iEpNbT7 auVc9qBWiBskUXqr5S7s3a6ZSHvPti/c/AwCccBp/Kil6K6jnBUoJRzzyNd+E2nD/C/k zhNYv6HrVQ6JqQ4fUgU9unRAtGYIgCOHiQiG6NKduAqjZagxDGtKY5mH2oeMHmu1/uV9 SrDTWWXghIgi4Yz1Mhv9JlLZzWOqgbnNb6VsvMN6ltMgfrDTkkxkwBjOMSnrBZHIwDX7 BLMud6+p5oQQNvti2dJt3N5FRgkfIOlzPYfa7NZQ0hiDAqLIG12udwyKqyVJwIVdo0Bf UB2A== X-Gm-Message-State: AOAM5314be3u7wJzeT+rZ6Defiec72mcT4dqKUU7g7ooxki0gZlQ9t8K zT/kJdxYY5jaeQ2fAsq67jqQGSm1RsA= X-Google-Smtp-Source: ABdhPJx5BZk10husdXQ7k44Rx1C1FfwF1Vsz1f+vgH6EJfpVQbwBLoV90YqFP98X0H1jrjStXY+ZoQ== X-Received: by 2002:a05:6a00:2353:b029:1ba:d824:f1dc with SMTP id j19-20020a056a002353b02901bad824f1dcmr3363293pfj.9.1614260875507; Thu, 25 Feb 2021 05:47:55 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:54 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 15/37] KVM: PPC: Book3S HV P9: Move xive vcpu context management into kvmhv_p9_guest_entry Date: Thu, 25 Feb 2021 23:46:30 +1000 Message-Id: <20210225134652.2127648-16-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Move the xive management up so the low level register switching can be pushed further down in a later patch. XIVE MMIO CI operations can run in higher level code with machine checks, tracing, etc., available. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index e3344d58537d..7e23838b7f9b 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3549,15 +3549,11 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, switch_mmu_to_guest_radix(kvm, vcpu, lpcr); - kvmppc_xive_push_vcpu(vcpu); - mtspr(SPRN_SRR0, vcpu->arch.shregs.srr0); mtspr(SPRN_SRR1, vcpu->arch.shregs.srr1); trap = __kvmhv_vcpu_entry_p9(vcpu); - kvmppc_xive_pull_vcpu(vcpu); - /* Advance host PURR/SPURR by the amount used by guest */ purr = mfspr(SPRN_PURR); spurr = mfspr(SPRN_SPURR); @@ -3740,7 +3736,10 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, trap = 0; } } else { + kvmppc_xive_push_vcpu(vcpu); trap = kvmhv_load_hv_regs_and_go(vcpu, time_limit, lpcr); + kvmppc_xive_pull_vcpu(vcpu); + } vcpu->arch.slb_max = 0; From patchwork Thu Feb 25 13:46:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444390 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=StdzBC0e; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ096YcSz9sWb for ; Fri, 26 Feb 2021 00:49:29 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229890AbhBYNtN (ORCPT ); Thu, 25 Feb 2021 08:49:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44852 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232237AbhBYNtH (ORCPT ); Thu, 25 Feb 2021 08:49:07 -0500 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A70A4C061786 for ; Thu, 25 Feb 2021 05:47:59 -0800 (PST) Received: by mail-pl1-x62b.google.com with SMTP id a24so3201164plm.11 for ; Thu, 25 Feb 2021 05:47:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZdOYv8+nnZM3a+E6C6u4dc+uTen+WkrGUatgkgkogK4=; b=StdzBC0eEVJ9WgNSNszS5UOUmm2MlwItvdM2++qKNG31X+dDJfL8y1/FbivVnxW5R5 KQx1RYsU9PPY8KpzwB5/RvcXHnVoxFs1+q2vZDiR+QUzqh4aoObq+3afXp7Miu0gF43g 6oMVe9FwjgTCs0cwcUUY/dV0RYNvDgWkxRRHy/M87E2UyPtHdBNhYnJYvhJRns4gI4SI THmRU0YOTy4LTQvfsp1191QRRtAhHkzjWMAtxwE8gc7MtJh28RfuZGqNUQGJkNZyuHVR Pc1QfkBMpmSPFzSzuAQW/U5i30hl1ziOl6C3Tqg9pSJPyOX3jQCq4DZAsUsLfYZXxz3a 2uSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZdOYv8+nnZM3a+E6C6u4dc+uTen+WkrGUatgkgkogK4=; b=r1w0iNj8jSRYTulkTwOlK93OTVBT8zbDs3kIMt6nIQir4AjvRlqoiW4qpDbj3Zr7xz x3hguoru1JWhNilf0bQNYEkzT6Jzlo//hZz/G1xOVnuIg5dmVH2RIGHNebCkcaX7TB7Q dPe3cl6zEp0DWHk494h9DudUz23Kppvf8h79zXgH2Oz34sDW88KnHqtxPA07UuCkuQCa ZxVln/+kk9YBbdadZGYCRx8X15UGjOz8whRqos5Tx0hWS7KYDsHrUxhTc2ZuZJcKufBg dYishb+urSBJcZuE7zw5jZIgfztg5gde+ZihsB5mTlRgH+mcVDdyQEO4zKv+kjrY6cXW 1YHw== X-Gm-Message-State: AOAM530FSF78a/EArhFRJgCnO0rfFUaSM0c2RslpBd645wA4p8f1L4JG ASghZrV/rdrWYUN7IktRYTdcZIm41Xg= X-Google-Smtp-Source: ABdhPJxE4Ai4Jq/cmQXxsnBQq6UKSowfA2PoZk0/AuHyqwVUVi8xPVcj7vR6uqQJ2gOhoQ8FKCmxKA== X-Received: by 2002:a17:90b:1c0c:: with SMTP id oc12mr3364744pjb.180.1614260878614; Thu, 25 Feb 2021 05:47:58 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:47:57 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 16/37] KVM: PPC: Book3S HV P9: Stop handling hcalls in real-mode in the P9 path Date: Thu, 25 Feb 2021 23:46:31 +1000 Message-Id: <20210225134652.2127648-17-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org In the interest of minimising the amount of code that is run in "real-mode", don't handle hcalls in real mode in the P9 path. POWER8 and earlier are much more expensive to exit from HV real mode and switch to host mode, because on those processors HV interrupts get to the hypervisor with the MMU off, and the other threads in the core need to be pulled out of the guest, and SLBs all need to be saved, ERATs invalidated, and host SLB reloaded before the MMU is re-enabled in host mode. Hash guests also require a lot of hcalls to run. The XICS interrupt controller requires hcalls to run. By contrast, POWER9 has independent thread switching, and in radix mode the hypervisor is already in a host virtual memory mode when the HV interrupt is taken. Radix + xive guests don't need hcalls to handle interrupts or manage translations. So it's much less important to handle hcalls in real mode in P9. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/kvm_ppc.h | 5 +++++ arch/powerpc/kvm/book3s_hv.c | 25 ++++++++++++++++++++++--- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 5 +++++ arch/powerpc/kvm/book3s_xive.c | 25 +++++++++++++++++++++++++ 4 files changed, 57 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h index 73b1ca5a6471..db6646c2ade2 100644 --- a/arch/powerpc/include/asm/kvm_ppc.h +++ b/arch/powerpc/include/asm/kvm_ppc.h @@ -607,6 +607,7 @@ extern void kvmppc_free_pimap(struct kvm *kvm); extern int kvmppc_xics_rm_complete(struct kvm_vcpu *vcpu, u32 hcall); extern void kvmppc_xics_free_icp(struct kvm_vcpu *vcpu); extern int kvmppc_xics_hcall(struct kvm_vcpu *vcpu, u32 cmd); +extern int kvmppc_xive_xics_hcall(struct kvm_vcpu *vcpu, u32 req); extern u64 kvmppc_xics_get_icp(struct kvm_vcpu *vcpu); extern int kvmppc_xics_set_icp(struct kvm_vcpu *vcpu, u64 icpval); extern int kvmppc_xics_connect_vcpu(struct kvm_device *dev, @@ -639,6 +640,8 @@ static inline int kvmppc_xics_enabled(struct kvm_vcpu *vcpu) static inline void kvmppc_xics_free_icp(struct kvm_vcpu *vcpu) { } static inline int kvmppc_xics_hcall(struct kvm_vcpu *vcpu, u32 cmd) { return 0; } +static inline int kvmppc_xive_xics_hcall(struct kvm_vcpu *vcpu, u32 req) + { return 0; } #endif #ifdef CONFIG_KVM_XIVE @@ -673,6 +676,7 @@ extern int kvmppc_xive_set_irq(struct kvm *kvm, int irq_source_id, u32 irq, int level, bool line_status); extern void kvmppc_xive_push_vcpu(struct kvm_vcpu *vcpu); extern void kvmppc_xive_pull_vcpu(struct kvm_vcpu *vcpu); +extern void kvmppc_xive_cede_vcpu(struct kvm_vcpu *vcpu); static inline int kvmppc_xive_enabled(struct kvm_vcpu *vcpu) { @@ -714,6 +718,7 @@ static inline int kvmppc_xive_set_irq(struct kvm *kvm, int irq_source_id, u32 ir int level, bool line_status) { return -ENODEV; } static inline void kvmppc_xive_push_vcpu(struct kvm_vcpu *vcpu) { } static inline void kvmppc_xive_pull_vcpu(struct kvm_vcpu *vcpu) { } +static inline void kvmppc_xive_cede_vcpu(struct kvm_vcpu *vcpu) { } static inline int kvmppc_xive_enabled(struct kvm_vcpu *vcpu) { return 0; } diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 7e23838b7f9b..d4770b222d7e 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -1144,7 +1144,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) * This has to be done early, not in kvmppc_pseries_do_hcall(), so * that the cede logic in kvmppc_run_single_vcpu() works properly. */ -static void kvmppc_nested_cede(struct kvm_vcpu *vcpu) +static void kvmppc_cede(struct kvm_vcpu *vcpu) { vcpu->arch.shregs.msr |= MSR_EE; vcpu->arch.ceded = 1; @@ -3731,15 +3731,34 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, /* H_CEDE has to be handled now, not later */ if (trap == BOOK3S_INTERRUPT_SYSCALL && !vcpu->arch.nested && kvmppc_get_gpr(vcpu, 3) == H_CEDE) { - kvmppc_nested_cede(vcpu); + kvmppc_cede(vcpu); kvmppc_set_gpr(vcpu, 3, 0); trap = 0; } } else { kvmppc_xive_push_vcpu(vcpu); trap = kvmhv_load_hv_regs_and_go(vcpu, time_limit, lpcr); - kvmppc_xive_pull_vcpu(vcpu); + /* H_CEDE has to be handled now, not later */ + /* XICS hcalls must be handled before xive is pulled */ + if (trap == BOOK3S_INTERRUPT_SYSCALL && + !(vcpu->arch.shregs.msr & MSR_PR)) { + unsigned long req = kvmppc_get_gpr(vcpu, 3); + if (req == H_CEDE) { + kvmppc_cede(vcpu); + kvmppc_xive_cede_vcpu(vcpu); /* may un-cede */ + kvmppc_set_gpr(vcpu, 3, 0); + trap = 0; + } + if (req == H_EOI || req == H_CPPR || req == H_IPI || + req == H_IPOLL || req == H_XIRR || req == H_XIRR_X) { + unsigned long ret; + ret = kvmppc_xive_xics_hcall(vcpu, req); + kvmppc_set_gpr(vcpu, 3, ret); + trap = 0; + } + } + kvmppc_xive_pull_vcpu(vcpu); } vcpu->arch.slb_max = 0; diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index c11597f815e4..2d0d14ed1d92 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -1397,9 +1397,14 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) mr r4,r9 bge fast_guest_return 2: + /* If we came in through the P9 short path, no real mode hcalls */ + lwz r0, STACK_SLOT_SHORT_PATH(r1) + cmpwi r0, 0 + bne no_try_real /* See if this is an hcall we can handle in real mode */ cmpwi r12,BOOK3S_INTERRUPT_SYSCALL beq hcall_try_real_mode +no_try_real: /* Hypervisor doorbell - exit only if host IPI flag set */ cmpwi r12, BOOK3S_INTERRUPT_H_DOORBELL diff --git a/arch/powerpc/kvm/book3s_xive.c b/arch/powerpc/kvm/book3s_xive.c index 8632fb998a55..d2266d36a7c7 100644 --- a/arch/powerpc/kvm/book3s_xive.c +++ b/arch/powerpc/kvm/book3s_xive.c @@ -2109,6 +2109,31 @@ static int kvmppc_xive_create(struct kvm_device *dev, u32 type) return 0; } +int kvmppc_xive_xics_hcall(struct kvm_vcpu *vcpu, u32 req) +{ + struct kvmppc_vcore *vc = vcpu->arch.vcore; + + switch (req) { + case H_XIRR: + return xive_vm_h_xirr(vcpu); + case H_CPPR: + return xive_vm_h_cppr(vcpu, kvmppc_get_gpr(vcpu, 4)); + case H_EOI: + return xive_vm_h_eoi(vcpu, kvmppc_get_gpr(vcpu, 4)); + case H_IPI: + return xive_vm_h_ipi(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5)); + case H_IPOLL: + return xive_vm_h_ipoll(vcpu, kvmppc_get_gpr(vcpu, 4)); + case H_XIRR_X: + xive_vm_h_xirr(vcpu); + kvmppc_set_gpr(vcpu, 5, get_tb() + vc->tb_offset); + return H_SUCCESS; + } + + return H_UNSUPPORTED; +} + int kvmppc_xive_debug_show_queues(struct seq_file *m, struct kvm_vcpu *vcpu) { struct kvmppc_xive_vcpu *xc = vcpu->arch.xive_vcpu; From patchwork Thu Feb 25 13:46:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444379 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=cZG/TmfI; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzX2s8Nz9sW4 for ; Fri, 26 Feb 2021 00:48:56 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231818AbhBYNsn (ORCPT ); Thu, 25 Feb 2021 08:48:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231604AbhBYNsn (ORCPT ); Thu, 25 Feb 2021 08:48:43 -0500 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8088C061788 for ; Thu, 25 Feb 2021 05:48:02 -0800 (PST) Received: by mail-pj1-x1034.google.com with SMTP id l18so3562155pji.3 for ; Thu, 25 Feb 2021 05:48:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kbXDPVn/6/K9PcQafkWtS0R94LJwb94j0RZ+KsMZVko=; b=cZG/TmfIJYP9ySL3cURRthWTojPynINIW6XqbZoJb+2omF6EgjZKQ61a8JJDuzRdnr x2VV63vHuuObIA/2cChPJyXmZZdlz3zssKpWwv1u+BBAa9OZ/ekPV4Z22dy4TZ7xe4Op RjENg68UGSrRwE+0RTXgFnQVRzkJfLx/3rgCHv/wI/fphgGFf23U+7MJcxCEvgIkQ+PH iKC1pb7XyL7H9L6eiCAoow9XILu+EUvmCmqRWRLjMJYTkISODOPcg9BEXkh3VKv0FwSb R0BI91c1vvduiIZL4dbjl2hXPCWQ2qmgPhskY0/NSBe7/Ux2aJtfIxt2KqJOldHHi/ur DAMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kbXDPVn/6/K9PcQafkWtS0R94LJwb94j0RZ+KsMZVko=; b=gALVSzJsrVxYToE+vytk/T+5QC6eYw+GWveu6uowHpV7ZNo0x0pMK+NRvua8M2GulY jxRXGe24DzRjZk7JQ7Q1m83GSrjz3VfZaKYbDcVmZaW6iuh8U0Ok8f7AvvslyY5lylq9 wFJkZQkDYSqJAqUWv4dYtMK6Zmc56SzStqaFKF3nNuyJsRWhQY6jW0RN8bw7TCPGnjcq 9if9jRvcj5Xum7Y5O6kPemp9CdMJzvqfXmF8iGLqDvBLnLDJ/FaRFa7U81ze0l7WK0tC dNJSE82MMFD9IAeFVAxebsw9llGpfYhw9HEcAEsGgmtUVgG8YWLAIeVeKnFea9tJeH9d csXg== X-Gm-Message-State: AOAM531dA128QlB2E5UjGv2Yllz29Am60ST5PtkJihf34EUGXiNE8Q+q 0NePpfiOn3BnsXJpxzlR1Eg/PMJwI/w= X-Google-Smtp-Source: ABdhPJyc5gaQwKOIAt8FWXSy3UqBYnhW/0DQWRY2+1dIg+EUjt5VYmoWzjive5AP27muZzAdO2kOfA== X-Received: by 2002:a17:90a:7f87:: with SMTP id m7mr3354890pjl.64.1614260881707; Thu, 25 Feb 2021 05:48:01 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.47.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:01 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 17/37] KVM: PPC: Book3S HV P9: Move setting HDEC after switching to guest LPCR Date: Thu, 25 Feb 2021 23:46:32 +1000 Message-Id: <20210225134652.2127648-18-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org LPCR[HDICE]=0 suppresses hypervisor decrementer exceptions on some processors, so it must be enabled before HDEC is set. Rather than set it in the host LPCR then setting HDEC, move the HDEC update to after the guest MMU context (including LPCR) is loaded. There shouldn't be much concern with delaying HDEC by some 10s or 100s of nanoseconds by setting it a bit later. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index d4770b222d7e..63cc92c45c5d 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3490,23 +3490,13 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, host_dawrx1 = mfspr(SPRN_DAWRX1); } - /* - * P8 and P9 suppress the HDEC exception when LPCR[HDICE] = 0, - * so set HDICE before writing HDEC. - */ - mtspr(SPRN_LPCR, kvm->arch.host_lpcr | LPCR_HDICE); - isync(); - - hdec = time_limit - mftb(); - if (hdec < 0) { - mtspr(SPRN_LPCR, kvm->arch.host_lpcr); - isync(); + tb = mftb(); + hdec = time_limit - tb; + if (hdec < 0) return BOOK3S_INTERRUPT_HV_DECREMENTER; - } - mtspr(SPRN_HDEC, hdec); if (vc->tb_offset) { - u64 new_tb = mftb() + vc->tb_offset; + u64 new_tb = tb + vc->tb_offset; mtspr(SPRN_TBU40, new_tb); tb = mftb(); if ((tb & 0xffffff) < (new_tb & 0xffffff)) @@ -3549,6 +3539,12 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, switch_mmu_to_guest_radix(kvm, vcpu, lpcr); + /* + * P9 suppresses the HDEC exception when LPCR[HDICE] = 0, + * so set guest LPCR (with HDICE) before writing HDEC. + */ + mtspr(SPRN_HDEC, hdec); + mtspr(SPRN_SRR0, vcpu->arch.shregs.srr0); mtspr(SPRN_SRR1, vcpu->arch.shregs.srr1); From patchwork Thu Feb 25 13:46:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444391 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=mpNscLPS; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ0C04JFz9sWW for ; Fri, 26 Feb 2021 00:49:31 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231960AbhBYNtO (ORCPT ); Thu, 25 Feb 2021 08:49:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229566AbhBYNtI (ORCPT ); Thu, 25 Feb 2021 08:49:08 -0500 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D6942C0617AA for ; Thu, 25 Feb 2021 05:48:05 -0800 (PST) Received: by mail-pj1-x102d.google.com with SMTP id u12so3320002pjr.2 for ; Thu, 25 Feb 2021 05:48:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=q3MbbxrELn+TwojMnsr2Dw3fkqeHU+FUDlPMBwDlaCY=; b=mpNscLPSdgLzWsUAfE8EpRXRIM51BpyecMaKWajFv1yfnBmWJ17F6txO5IPVjUJ74L r5jJWJYgkwBGx10hHRj2advd8SPjai+3tWj3jnG48SQiYw/bHkdJ9czxHN8BWWnZjOI7 7mfMQ/xwvhiz8ckx974mi2t3H1AhMbmjYI+XXPpXkt8OY6hz+g+dnCRT1MjHu5NvIy/v WTM4NrDFeV/p3R/4FF912bryse8S5F6W1CUXn/BKF5hnuUHMY0vcga/zvSXtyTfyWZDZ UguZpSvU4Q2Pdys1uOaG6Hf0aQxhBJp8qoqJJhGhNp500fCOdNpdYP3N1NKzOmt7P5GH md6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q3MbbxrELn+TwojMnsr2Dw3fkqeHU+FUDlPMBwDlaCY=; b=n0bsOgpD55eMNd5R23I6PQffP53pq9juoVVpX/8aZwz/PfPc40iK62vGPi7PBvc5P8 jQix8mF0jsH3u2VM56QDYrM9J88ZjsdYc3h7MZOqjqLerYYCI1em/TcfdVJpaeiyJrAn xSjy08M91yzjp8PMrl/BJ3aLi4rnlja/lNHBrNkWYZqsYzZT0dP8HqoaXsuVXs07yM7t kFJjqPM/+kFOntZAYi2IZYIOffqS7ekCOs+TJNRRV7mPN6x3sDoHQSZ07jiY4FHRWaP9 nag68Xx5gwy4YiitTqp2oRRQv4/9DpcifR1Ue6OokGaXQyez62k/Csnb9OJrG1mr6Zdr 1cVg== X-Gm-Message-State: AOAM532u7nTo3G7kU/QaMLhV2IAnjYomF53RAcb4bxvO3Gu5JoEAOOm6 JXmQUNjd2aie8KXr329IaTTV2vp88ME= X-Google-Smtp-Source: ABdhPJxmjVYicu31+Ovtl+bVMxJEtCuzruCMimQ9v0cIdWxEQisZVMGiS7vRy4ElZuhl5mATjEWgpA== X-Received: by 2002:a17:90a:3804:: with SMTP id w4mr3468161pjb.189.1614260885095; Thu, 25 Feb 2021 05:48:05 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:04 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 18/37] KVM: PPC: Book3S HV P9: Use large decrementer for HDEC Date: Thu, 25 Feb 2021 23:46:33 +1000 Message-Id: <20210225134652.2127648-19-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org On processors that don't suppress the HDEC exceptions when LPCR[HDICE]=0, this could help reduce needless guest exits due to leftover exceptions on entering the guest. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/time.h | 2 ++ arch/powerpc/kvm/book3s_hv.c | 3 ++- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/include/asm/time.h b/arch/powerpc/include/asm/time.h index 8dd3cdb25338..68d94711811e 100644 --- a/arch/powerpc/include/asm/time.h +++ b/arch/powerpc/include/asm/time.h @@ -18,6 +18,8 @@ #include /* time.c */ +extern u64 decrementer_max; + extern unsigned long tb_ticks_per_jiffy; extern unsigned long tb_ticks_per_usec; extern unsigned long tb_ticks_per_sec; diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 63cc92c45c5d..913582bd848f 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3609,7 +3609,8 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, vc->tb_offset_applied = 0; } - mtspr(SPRN_HDEC, 0x7fffffff); + /* HDEC must be at least as large as DEC, so decrementer_max fits */ + mtspr(SPRN_HDEC, decrementer_max); switch_mmu_to_host_radix(kvm, host_pidr); From patchwork Thu Feb 25 13:46:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444380 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=I9SIJ+2J; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzY4N0Hz9sf9 for ; Fri, 26 Feb 2021 00:48:57 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231895AbhBYNsq (ORCPT ); Thu, 25 Feb 2021 08:48:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231604AbhBYNso (ORCPT ); Thu, 25 Feb 2021 08:48:44 -0500 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C20CFC0617AB for ; Thu, 25 Feb 2021 05:48:09 -0800 (PST) Received: by mail-pl1-x632.google.com with SMTP id g20so3230695plo.2 for ; Thu, 25 Feb 2021 05:48:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IAvUmJ5x6WP1qzuhU/NEz1W7bJ05d4CA4Gd/jBet5Cc=; b=I9SIJ+2JSZBissr8WFV6oI788xYTeZUYkdHy2Fr7d3QjcaxD9ob+STuQ2EJnJoiJM/ VSens5ZyguuoSXAG60mLHo1UzagDqGXhIyY6GUB6q8uY/XLVWozxhNCL8mjVtpHLceiB 6oUdrEZ0s/TQclr1y4zjdzwUnzXNNc2XMUWPfrPi364Z5mi39rSQx5vqULiVXaemf36N zz1n59rK9ML+F9r632G1SxUGArcy1ysEGQzXUDiLkpvE26ej5RGw+VY4V8zS68+/+n7w cN85mExtgr7JiZ0deMVGstQVPWCd/XWs6W6GhfD5WASULvwXHOTouXUct6f/Mpcigqmd Rh2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IAvUmJ5x6WP1qzuhU/NEz1W7bJ05d4CA4Gd/jBet5Cc=; b=CR4SUgb7AiBtBCDc1BC6wHcsLffJ4VQJQu7x/Ja811xcCX0QDh9X+GLesNjnI2W0gQ +HhCWse2EHKIqUEaF3VdChG1daqHOqMdNM+mc317PwlA5LPWD4uQwUpi4tlAKxDKzLKJ DNSxKeaxUYEeOQsZKxnPKlyzwwzAqRiYYmYFoZXUNlmcZrQz2EHKfWXo/bdU5Fx1Embu jHLL5oMQqjMVldCjWAmsyXiImyiBmUfEFsZV3V3lbOdkDKQ9QH0E6BULO7W0PkefAYKO Zk72E5s7t5aSF1Usan8GTuzrm+HqkGDjcbqhxPJwOhyd9jjLKQuIKgI6tgLBK3qHty3M GT2A== X-Gm-Message-State: AOAM5309vCc90JiIhgOCzSnBy7NkrO5xEU0VqU17L9SKnI9Tm4LC34Ah 72b/Af1UAuEMfLEYtdosN1/Q3NQ+inU= X-Google-Smtp-Source: ABdhPJwwyG9yKV5GO5sIoo2EFzuVqiNXSgeaLsc4YQlwyGDnXto2BpBBrTCwAO9r4VQwzIe2fKpiQQ== X-Received: by 2002:a17:902:9349:b029:df:fab3:64b8 with SMTP id g9-20020a1709029349b02900dffab364b8mr2985259plp.72.1614260888800; Thu, 25 Feb 2021 05:48:08 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:07 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 19/37] KVM: PPC: Book3S HV P9: Use host timer accounting to avoid decrementer read Date: Thu, 25 Feb 2021 23:46:34 +1000 Message-Id: <20210225134652.2127648-20-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org There is no need to save away the host DEC value, as it is derived from the host timer subsystem, which maintains the next timer time. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/time.h | 5 +++++ arch/powerpc/kvm/book3s_hv.c | 12 ++++++------ 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/arch/powerpc/include/asm/time.h b/arch/powerpc/include/asm/time.h index 68d94711811e..0128cd9769bc 100644 --- a/arch/powerpc/include/asm/time.h +++ b/arch/powerpc/include/asm/time.h @@ -101,6 +101,11 @@ extern void __init time_init(void); DECLARE_PER_CPU(u64, decrementers_next_tb); +static inline u64 timer_get_next_tb(void) +{ + return __this_cpu_read(decrementers_next_tb); +} + /* Convert timebase ticks to nanoseconds */ unsigned long long tb_to_ns(unsigned long long tb_ticks); diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 913582bd848f..735ec40ece86 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3631,16 +3631,16 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long host_amr = mfspr(SPRN_AMR); unsigned long host_fscr = mfspr(SPRN_FSCR); s64 dec; - u64 tb; + u64 tb, next_timer; int trap, save_pmu; - dec = mfspr(SPRN_DEC); tb = mftb(); - if (dec < 0) + next_timer = timer_get_next_tb(); + if (tb >= next_timer) return BOOK3S_INTERRUPT_HV_DECREMENTER; - local_paca->kvm_hstate.dec_expires = dec + tb; - if (local_paca->kvm_hstate.dec_expires < time_limit) - time_limit = local_paca->kvm_hstate.dec_expires; + local_paca->kvm_hstate.dec_expires = next_timer; + if (next_timer < time_limit) + time_limit = next_timer; vcpu->arch.ceded = 0; From patchwork Thu Feb 25 13:46:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444392 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=otc6vdNM; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ0D1Rdrz9sWq for ; Fri, 26 Feb 2021 00:49:32 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232290AbhBYNtP (ORCPT ); Thu, 25 Feb 2021 08:49:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229613AbhBYNtJ (ORCPT ); Thu, 25 Feb 2021 08:49:09 -0500 Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4E15C061356 for ; Thu, 25 Feb 2021 05:48:12 -0800 (PST) Received: by mail-pl1-x629.google.com with SMTP id f8so3216730plg.5 for ; Thu, 25 Feb 2021 05:48:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5gBupD6w9IAV3hK4OXn3cqVs1/uQmmRToC//irklCQ4=; b=otc6vdNMX1R7wpi+EcZ6Sx/eMagFElg1z8C/hMvENJbfsoJiLAhL5xwPj8SfUnDy+W aY1OVgUMSruJ8aIiwH1p0tsLDQQrwXxFjbYjHWkWd+fjJxnSKkoNSZjqx/HjCAP2D35X ZjEKcid0iFjqrvMztOaDevAI2MgEJRv3m+WOn9j0x8sEj6n+uT+PZRN7BeiP6FDUQKbo EsdcsF/UTVK4qvvy5c9jNADiI6+zBA5wx2TM1h3YATs+IyB1i+tc+F1ScFXkdIMzsOvc JvbLWs3rJESGSXoCiHzOH40ijL+qUZ25IPJFtwkw4tt61odrQzzqMa32Fpnh8bJbURSy oyeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5gBupD6w9IAV3hK4OXn3cqVs1/uQmmRToC//irklCQ4=; b=K3bPvimTOxWlPlSqwEyFfB7zjPf8qTzIMbVx/R1oWGxjl5nRfuzE5AzeJjSBNq81EA zdDY1RB+VpOT9neeL6ALQ365nd8ghNCHp7h6GITWBZ2QfGYmtf0b9XaRUs/zRT2gq5Op 2+WCAQ40FmQz08kzmcNDvnts+ujxoxbXA/WjUvPQR49Ftlm8AP/H3+2yuEBC38aA8U2i V7byz6wsblxCBaL2mQ3I4CRFqtg42NAIrJtmhR9d9sFgmGn7cqcCzKNxqqrZ+sSkTGFX ZBS9tcCISM5i84stX2yvrSo54H5tEFlyU1NFDFLYkjNAFC7ZxDSkK0sQTkvFz+cKzmzf O5fA== X-Gm-Message-State: AOAM533STMkzCUY43Jc2u3UL7nPZAs4JH9fVc6BxW8+l7/pF0GxskCm2 XQmdsIsH4Je8FDA/1AduN93GTW1+h/s= X-Google-Smtp-Source: ABdhPJw55Z0GXGwEI7LASJoBShS68vWHDeVV6WM1gcGrjR0KrZyT0feHuIU12BgvwOse21jbUSEqfw== X-Received: by 2002:a17:902:ced2:b029:e2:8db8:266f with SMTP id d18-20020a170902ced2b02900e28db8266fmr3008957plg.34.1614260891850; Thu, 25 Feb 2021 05:48:11 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:11 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 20/37] KVM: PPC: Book3S HV P9: Reduce mftb per guest entry/exit Date: Thu, 25 Feb 2021 23:46:35 +1000 Message-Id: <20210225134652.2127648-21-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org mftb is serialising (dispatch next-to-complete) so it is heavy weight for a mfspr. Avoid reading it multiple times in the entry or exit paths. A small number of cycles delay to timers is tolerable. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 735ec40ece86..d98958b78830 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3689,7 +3689,7 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, if (!(vcpu->arch.ctrl & 1)) mtspr(SPRN_CTRLT, mfspr(SPRN_CTRLF) & ~1); - mtspr(SPRN_DEC, vcpu->arch.dec_expires - mftb()); + mtspr(SPRN_DEC, vcpu->arch.dec_expires - tb); if (kvmhv_on_pseries()) { /* @@ -3822,7 +3822,7 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, vc->entry_exit_map = 0x101; vc->in_guest = 0; - mtspr(SPRN_DEC, local_paca->kvm_hstate.dec_expires - mftb()); + mtspr(SPRN_DEC, local_paca->kvm_hstate.dec_expires - tb); mtspr(SPRN_SPRG_VDSO_WRITE, local_paca->sprg_vdso); kvmhv_load_host_pmu(); From patchwork Thu Feb 25 13:46:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444381 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=IUX2I6OT; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzZ1p9lz9ssP for ; Fri, 26 Feb 2021 00:48:58 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231604AbhBYNsr (ORCPT ); Thu, 25 Feb 2021 08:48:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231822AbhBYNsp (ORCPT ); Thu, 25 Feb 2021 08:48:45 -0500 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8DF4CC06121D for ; Thu, 25 Feb 2021 05:48:16 -0800 (PST) Received: by mail-pl1-x633.google.com with SMTP id u11so3198591plg.13 for ; Thu, 25 Feb 2021 05:48:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wxpw7NhXAqb/o/SA9VUSgxQVhLanI9uq2XV1Nk13yao=; b=IUX2I6OTL090eZq9oSuMpuIzDkT/w0BAUY9dM4H/gj8fOb1/0snrh1axsLvw7P0TY+ +Ee8uqBb8mHfvPTSBnOXJ0Oqg/56xDgKtlV3wtBwexeyFoUbn8leVkYvYqYP8wZAXJZI Iod6zIfWQrSzZqcQFrrCcqEwP4GNhRoJ/XiOauKwDqhHA4cpZUnXF8INaqtBzMBmEG9a p4tR3H+wbQGidMvLF/Wd215bSCbbLPrjQiVTNsHSgqPc5XDLGqvEmEB11k4Jqd9h+2YA ITImTlSSyJoZTxeRQSq1WNzRtBgw2cOWnyDuQlKYashzkv3qHwjaz+Pc/UiGr2NvT77O DzZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wxpw7NhXAqb/o/SA9VUSgxQVhLanI9uq2XV1Nk13yao=; b=oWVramweRpN128QL3ExRuqQMBP4c8HjzZQMKeewlM6cXZAc0O4pnIevm7E4E0HbVPk S6r7B0sVg3PFnTWHFPurKoHrfQe9hkjYkuR1BNKykKdezJEzPKF3Tt8jKUUDPthAVpEi hzn6R5Jz1GNjQkz8X1u88LdD2Z8ehynAOmlVkY2xKPA7vXxfOK+eCUmAyGLGSM7WhYuQ ba/N7jsCup1v3qpclzGajc6jOshIEpBsGyrXNdWpfFZjn1/j+hkUM3z1PiLRo2wbiKYu vo+KPgmJT4pVTFcLikhkfYuLiCPCNobFcJeaiFKn6gAzSygy8W9DHCeiBWPlbGHzeeea r/PA== X-Gm-Message-State: AOAM533qSAEoQ0I+lOZhr4WLf3WbCSCCJprLpgBYNoV0nba68UvMn3Xu CdnFde65QfvoRYhcAZuNrd4G8pVx9Xc= X-Google-Smtp-Source: ABdhPJzWKocZMPhw3t5VEokOjJwf9KuPQSGR3JgqR13o4xrBqpGbZyYOou3wP+GEte7t6dMLVwz5VA== X-Received: by 2002:a17:902:eccb:b029:e3:b2e3:a21b with SMTP id a11-20020a170902eccbb02900e3b2e3a21bmr3149912plh.41.1614260895676; Thu, 25 Feb 2021 05:48:15 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:15 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 21/37] powerpc: add set_dec_or_work API for safely updating decrementer Date: Thu, 25 Feb 2021 23:46:36 +1000 Message-Id: <20210225134652.2127648-22-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Decrementer updates must always check for new irq work to avoid an irq work decrementer interrupt being lost. Add an API for this in the timer code so callers don't have to care about details. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/time.h | 9 +++++++++ arch/powerpc/kernel/time.c | 20 +++++++++++--------- 2 files changed, 20 insertions(+), 9 deletions(-) diff --git a/arch/powerpc/include/asm/time.h b/arch/powerpc/include/asm/time.h index 0128cd9769bc..d62bde57bf02 100644 --- a/arch/powerpc/include/asm/time.h +++ b/arch/powerpc/include/asm/time.h @@ -78,6 +78,15 @@ static inline void set_dec(u64 val) mtspr(SPRN_DEC, val - 1); } +#ifdef CONFIG_IRQ_WORK +void set_dec_or_work(u64 val); +#else +static inline void set_dec_or_work(u64 val) +{ + set_dec(val); +} +#endif + static inline unsigned long tb_ticks_since(unsigned long tstamp) { return mftb() - tstamp; diff --git a/arch/powerpc/kernel/time.c b/arch/powerpc/kernel/time.c index b67d93a609a2..e35156858e6e 100644 --- a/arch/powerpc/kernel/time.c +++ b/arch/powerpc/kernel/time.c @@ -561,6 +561,15 @@ void arch_irq_work_raise(void) preempt_enable(); } +void set_dec_or_work(u64 val) +{ + set_dec(val); + /* We may have raced with new irq work */ + if (unlikely(test_irq_work_pending())) + set_dec(1); +} +EXPORT_SYMBOL_GPL(set_dec_or_work); + #else /* CONFIG_IRQ_WORK */ #define test_irq_work_pending() 0 @@ -628,10 +637,7 @@ DEFINE_INTERRUPT_HANDLER_ASYNC(timer_interrupt) } else { now = *next_tb - now; if (now <= decrementer_max) - set_dec(now); - /* We may have raced with new irq work */ - if (test_irq_work_pending()) - set_dec(1); + set_dec_or_work(now); __this_cpu_inc(irq_stat.timer_irqs_others); } @@ -873,11 +879,7 @@ static int decrementer_set_next_event(unsigned long evt, struct clock_event_device *dev) { __this_cpu_write(decrementers_next_tb, get_tb() + evt); - set_dec(evt); - - /* We may have raced with new irq work */ - if (test_irq_work_pending()) - set_dec(1); + set_dec_or_work(evt); return 0; } From patchwork Thu Feb 25 13:46:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444393 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=SOqp7vt2; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ0F0M1Mz9sWg for ; Fri, 26 Feb 2021 00:49:33 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232296AbhBYNtP (ORCPT ); Thu, 25 Feb 2021 08:49:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44868 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229722AbhBYNtK (ORCPT ); Thu, 25 Feb 2021 08:49:10 -0500 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0054C06121E for ; Thu, 25 Feb 2021 05:48:19 -0800 (PST) Received: by mail-pf1-x433.google.com with SMTP id j12so3621356pfj.12 for ; Thu, 25 Feb 2021 05:48:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5VNlGtaRSctLYlPeChdxDgtGvbZrSWxq9sa/ajZPH1c=; b=SOqp7vt2cwBvh3FOLS+ZqkyswmxMv+IhcaCzXAtn+qMSGhlYztYsdA+BBQIvrLXkZL X4cBl5NndAKHtxlH8P51LKzFrkCqqor39tC0shbwkzh4plUzUOftmEvgTUDGCueuP0rp Q/JNsUYw86WBdNOy1B9Yx1RQ94x3/98ZM9ixvpS2rCBPwWd0wSz8sLKDQHWbmsWcMC51 DdSr8XQuNpgyqpcQiwuimBggX2WntJfAUYU6qcBDldyTu/fzPfWZG4BXvo2MpDZ9xoOo JiohBow/zRFTOsV1f7sKDCNxuZIJ+JAfkwzg1V1Iwuiyc0IMBZTBhSOlyV2+x5UxSXlm 8WYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5VNlGtaRSctLYlPeChdxDgtGvbZrSWxq9sa/ajZPH1c=; b=f/EuXpuot/+9iGuNGyZ12nca45ghNYfX5yytbYeA3T8pER3k5ph9aLaR47/nMRPz2Y 6Q0Ia/iQkAjEigrwFCiFWg2ZNWHPUnHP85mfGBsR3wduG1B5ygy91k2nRp6TV+TrlVRo UU6e6BV/1fKgAOKeEHHkOeFNvarYXKyQ75chEmcAeg9kO8JdxUWpuMmeGy8dqlbXOzNz UX/PzhynCE04gZZyDRtGfoQmwFyroCLEQNKWpVf//uZqDruya1ENUhWYOUzO62BL+vUf GRk1BmuABpCEffORUf/A9GW+4pRn29lf7DQE+WMNhQU9doYlOBEkQ2xhE+6C3vTQhD4M s3jw== X-Gm-Message-State: AOAM533ENedBw/4moTeZpoBMzjKamF5m2go+S+SqhJwO0ImFMu6rJsOn OsjqvGfBz89NEwq+j0HkKitrN5POR+M= X-Google-Smtp-Source: ABdhPJx1UcBDTKpvP1zGz70sEDd9oVgfptBMzSZBjBnBZSh09cKJ2k7xVwMFEO7HEGjA0y5ByNwwng== X-Received: by 2002:a62:8686:0:b029:1ed:7a8b:4933 with SMTP id x128-20020a6286860000b02901ed7a8b4933mr3470531pfd.9.1614260898881; Thu, 25 Feb 2021 05:48:18 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:18 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 22/37] KVM: PPC: Book3S HV P9: Reduce irq_work vs guest decrementer races Date: Thu, 25 Feb 2021 23:46:37 +1000 Message-Id: <20210225134652.2127648-23-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org irq_work's use of the DEC SPR is racy with guest<->host switch and guest entry which flips the DEC interrupt to guest, which could lose a host work interrupt. This patch closes one race, and attempts to comment several others. (XXX: should think a bit harder about this) Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/paca.h | 1 + arch/powerpc/kvm/book3s_hv.c | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/include/asm/paca.h b/arch/powerpc/include/asm/paca.h index ec18ac818e3a..23c12048fbc9 100644 --- a/arch/powerpc/include/asm/paca.h +++ b/arch/powerpc/include/asm/paca.h @@ -174,6 +174,7 @@ struct paca_struct { u8 irq_happened; /* irq happened while soft-disabled */ u8 irq_work_pending; /* IRQ_WORK interrupt while soft-disable */ #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE + /* Could have irq_work_using_hdec here, but what about nested HV entry modifying DEC? Could have a pointer to the hv struct time limit */ u8 pmcregs_in_use; /* pseries puts this in lppaca */ #endif u64 sprg_vdso; /* Saved user-visible sprg */ diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index d98958b78830..1997cf347d3e 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3689,6 +3689,18 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, if (!(vcpu->arch.ctrl & 1)) mtspr(SPRN_CTRLT, mfspr(SPRN_CTRLF) & ~1); + /* + * XXX: must always deal with irq_work_raise via NMI vs setting DEC. + * The problem occurs right as we switch into guest mode if a NMI + * hits and sets pending work and sets DEC, then that will apply to + * the guest and not bring us back to the host. + * + * irq_work_raise could check a flag (or possibly LPCR[HDICE] for + * example) and set HDEC to 1? That wouldn't solve the nested hv + * case which needs to abort the hcall or zero the time limit. + * + * Another day's problem. + */ mtspr(SPRN_DEC, vcpu->arch.dec_expires - tb); if (kvmhv_on_pseries()) { @@ -3822,7 +3834,8 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, vc->entry_exit_map = 0x101; vc->in_guest = 0; - mtspr(SPRN_DEC, local_paca->kvm_hstate.dec_expires - tb); + set_dec_or_work(local_paca->kvm_hstate.dec_expires - tb); + mtspr(SPRN_SPRG_VDSO_WRITE, local_paca->sprg_vdso); kvmhv_load_host_pmu(); From patchwork Thu Feb 25 13:46:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444382 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=kq06fiUQ; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzZ6XL9z9sWp for ; Fri, 26 Feb 2021 00:48:58 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231974AbhBYNsu (ORCPT ); Thu, 25 Feb 2021 08:48:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231822AbhBYNst (ORCPT ); Thu, 25 Feb 2021 08:48:49 -0500 Received: from mail-pg1-x535.google.com (mail-pg1-x535.google.com [IPv6:2607:f8b0:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB3FFC06121F for ; Thu, 25 Feb 2021 05:48:24 -0800 (PST) Received: by mail-pg1-x535.google.com with SMTP id t11so3808659pgu.8 for ; Thu, 25 Feb 2021 05:48:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iugbjjW8P6YeBsxWVVXF7ID8mX0uXVwpCpjEUpz/KX0=; b=kq06fiUQcoCHZfL9k7u/oCZIOJBp6aZsbxAPlsVpRghCm/OcNEx/7feAkPmWqGVeHr dk9vDxl0XAii7VzgEZsLcXoRjGnJ2X4IORFsV8FYbSjkNQe/mHMXrAiP4dzYqj649/ax b+NzbARSkiyOTTMkaeEIqx3JOcElc817ARhcNbprsMqqzvnGd2MEoocK+1PyejbCX/ox PyCOzO//VU6vFeMW09fORblYfVadWadjm2w0ezKp3TR7Hnfek7+OpxQoFB8Xfu50rDUn JdYYqf2MI5dJ198dWz8ptYlwoOEZFj3ZWwh4lBA5OcWkURVHHWUghckqm9D3FEANXrDo bbYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iugbjjW8P6YeBsxWVVXF7ID8mX0uXVwpCpjEUpz/KX0=; b=XvL/j/rVLoRJ98zpHUF9vvyE5H6RJRVUxcEESSdoQ8LDJo/bm3pTE1O1i3T8lS3EGB 9ikjJl4lBklqguZo4c+V5MDgYSbeNUr3U7DYkwAkO8lR4G2BqqhM2S6yIa7Rxdj9EDT8 1QvviuAjiExVTZtEbe+uxcfAOQxMMIPgJonE5pUhxZFQ5uV9CA/xyOGqHflOcTZv5A8M LH5jfgqdBr49mBW6Pu2m+uR5H/kpYot1MfZBuRI4IK4y03stdxRo+UpgZuEkVZ+L5Ccw snKHVq0K2fkABuDQiPCJ9CUUSHPmWLWEhZXloBFOjtQ1IRYjDxIJNNRQ92EFoDlQWrmX y99g== X-Gm-Message-State: AOAM531BjgfvlX2+AXa99hwKb5Ni64G9zDQlhcqPTEluMqCi3fW8CqC6 YwMxoBmrkKhFebjtVOjlBGz3/FtsjAA= X-Google-Smtp-Source: ABdhPJw7h4MLpxaOvYjAiH0IV0geHpya+fR3SmZ/FmPT/qi6dyWIHAo5dpWzSXPfyPSFHXvD78xNUg== X-Received: by 2002:aa7:9e04:0:b029:1ed:996a:abb4 with SMTP id y4-20020aa79e040000b02901ed996aabb4mr3325005pfq.5.1614260903040; Thu, 25 Feb 2021 05:48:23 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:22 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 23/37] KVM: PPC: Book3S HV P9: Implement the rest of the P9 path in C Date: Thu, 25 Feb 2021 23:46:38 +1000 Message-Id: <20210225134652.2127648-24-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Almost all logic is moved to C, by introducing a new in_guest mode that selects and branches very early in the interrupt handler to the P9 exit code. The remaining assembly is only about 160 lines of low level stack setup, with VCPU vs host register save and restore, plus a small shim to the legacy paths in the interrupt handler. There are two motivations for this, the first is just make the code more maintainable being in C. The second is to reduce the amount of code running in a special KVM mode, "realmode". I put that in quotes because with radix it is no longer necessarily real-mode in the MMU, but it still has to be treated specially because it may be in real-mode, and has various important registers like PID, DEC, TB, etc set to guest. This is hostile to the rest of Linux and can't use arbitrary kernel functionality or be instrumented well. This initial patch is a reasonably faithful conversion of the asm code. It does lack any loop to return quickly back into the guest without switching out of realmode in the case of unimportant or easily handled interrupts, as explained in the previous change, handling HV interrupts in real mode is not so important for P9. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/asm-prototypes.h | 3 +- arch/powerpc/include/asm/kvm_asm.h | 3 +- arch/powerpc/include/asm/kvm_book3s_64.h | 8 + arch/powerpc/kernel/security.c | 5 +- arch/powerpc/kvm/Makefile | 3 + arch/powerpc/kvm/book3s_64_entry.S | 181 ++++++++++++++++++ arch/powerpc/kvm/book3s_hv.c | 27 ++- arch/powerpc/kvm/book3s_hv_interrupt.c | 221 ++++++++++++++++++++++ arch/powerpc/kvm/book3s_hv_rmhandlers.S | 123 +----------- arch/powerpc/kvm/book3s_xive.c | 34 ++++ 10 files changed, 480 insertions(+), 128 deletions(-) create mode 100644 arch/powerpc/kvm/book3s_hv_interrupt.c diff --git a/arch/powerpc/include/asm/asm-prototypes.h b/arch/powerpc/include/asm/asm-prototypes.h index 939f3c94c8f3..8677d27929fe 100644 --- a/arch/powerpc/include/asm/asm-prototypes.h +++ b/arch/powerpc/include/asm/asm-prototypes.h @@ -122,6 +122,7 @@ extern s32 patch__call_flush_branch_caches3; extern s32 patch__flush_count_cache_return; extern s32 patch__flush_link_stack_return; extern s32 patch__call_kvm_flush_link_stack; +extern s32 patch__call_kvm_flush_link_stack_2; extern s32 patch__memset_nocache, patch__memcpy_nocache; extern long flush_branch_caches; @@ -142,7 +143,7 @@ void kvmhv_load_host_pmu(void); void kvmhv_save_guest_pmu(struct kvm_vcpu *vcpu, bool pmu_in_use); void kvmhv_load_guest_pmu(struct kvm_vcpu *vcpu); -int __kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu); +void kvmppc_p9_enter_guest(struct kvm_vcpu *vcpu); long kvmppc_h_set_dabr(struct kvm_vcpu *vcpu, unsigned long dabr); long kvmppc_h_set_xdabr(struct kvm_vcpu *vcpu, unsigned long dabr, diff --git a/arch/powerpc/include/asm/kvm_asm.h b/arch/powerpc/include/asm/kvm_asm.h index a3633560493b..b4f9996bd331 100644 --- a/arch/powerpc/include/asm/kvm_asm.h +++ b/arch/powerpc/include/asm/kvm_asm.h @@ -146,7 +146,8 @@ #define KVM_GUEST_MODE_GUEST 1 #define KVM_GUEST_MODE_SKIP 2 #define KVM_GUEST_MODE_GUEST_HV 3 -#define KVM_GUEST_MODE_HOST_HV 4 +#define KVM_GUEST_MODE_GUEST_HV_FAST 4 /* ISA v3.0 with host radix mode */ +#define KVM_GUEST_MODE_HOST_HV 5 #define KVM_INST_FETCH_FAILED -1 diff --git a/arch/powerpc/include/asm/kvm_book3s_64.h b/arch/powerpc/include/asm/kvm_book3s_64.h index 9bb9bb370b53..c214bcffb441 100644 --- a/arch/powerpc/include/asm/kvm_book3s_64.h +++ b/arch/powerpc/include/asm/kvm_book3s_64.h @@ -153,9 +153,17 @@ static inline bool kvmhv_vcpu_is_radix(struct kvm_vcpu *vcpu) return radix; } +int __kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu); + #define KVM_DEFAULT_HPT_ORDER 24 /* 16MB HPT by default */ #endif +/* + * Invalid HDSISR value which is used to indicate when HW has not set the reg. + * Used to work around an errata. + */ +#define HDSISR_CANARY 0x7fff + /* * We use a lock bit in HPTE dword 0 to synchronize updates and * accesses to each HPTE, and another bit to indicate non-present diff --git a/arch/powerpc/kernel/security.c b/arch/powerpc/kernel/security.c index e4e1a94ccf6a..6c37aeed0650 100644 --- a/arch/powerpc/kernel/security.c +++ b/arch/powerpc/kernel/security.c @@ -430,16 +430,19 @@ device_initcall(stf_barrier_debugfs_init); static void update_branch_cache_flush(void) { - u32 *site; + u32 *site, __maybe_unused *site2; #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE site = &patch__call_kvm_flush_link_stack; + site2 = &patch__call_kvm_flush_link_stack_2; // This controls the branch from guest_exit_cont to kvm_flush_link_stack if (link_stack_flush_type == BRANCH_CACHE_FLUSH_NONE) { patch_instruction_site(site, ppc_inst(PPC_INST_NOP)); + patch_instruction_site(site2, ppc_inst(PPC_INST_NOP)); } else { // Could use HW flush, but that could also flush count cache patch_branch_site(site, (u64)&kvm_flush_link_stack, BRANCH_SET_LINK); + patch_branch_site(site2, (u64)&kvm_flush_link_stack, BRANCH_SET_LINK); } #endif diff --git a/arch/powerpc/kvm/Makefile b/arch/powerpc/kvm/Makefile index cdd119028f64..40a55a5ba4ff 100644 --- a/arch/powerpc/kvm/Makefile +++ b/arch/powerpc/kvm/Makefile @@ -43,6 +43,9 @@ kvm-objs-$(CONFIG_KVM_E500MC) := $(kvm-e500mc-objs) kvm-book3s_64-builtin-objs-$(CONFIG_SPAPR_TCE_IOMMU) := \ book3s_64_vio_hv.o +kvm-book3s_64-builtin-objs-y += \ + book3s_hv_interrupt.o + kvm-pr-y := \ fpu.o \ emulate.o \ diff --git a/arch/powerpc/kvm/book3s_64_entry.S b/arch/powerpc/kvm/book3s_64_entry.S index f826c8dc2e19..cc7b76865a16 100644 --- a/arch/powerpc/kvm/book3s_64_entry.S +++ b/arch/powerpc/kvm/book3s_64_entry.S @@ -1,10 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ #include #include +#include #include +#include #include #include #include #include +#include /* * These are branched to from interrupt handlers in exception-64s.S which set @@ -13,13 +17,24 @@ .global kvmppc_hcall .balign IFETCH_ALIGN_BYTES kvmppc_hcall: + lbz r10,HSTATE_IN_GUEST(r13) + cmpwi r10,KVM_GUEST_MODE_GUEST_HV_FAST + beq kvmppc_p9_exit_hcall ld r10,PACA_EXGEN+EX_R13(r13) SET_SCRATCH0(r10) li r10,0xc00 + li r11,PACA_EXGEN + b 1f .global kvmppc_interrupt .balign IFETCH_ALIGN_BYTES kvmppc_interrupt: + std r10,HSTATE_SCRATCH0(r13) + lbz r10,HSTATE_IN_GUEST(r13) + cmpwi r10,KVM_GUEST_MODE_GUEST_HV_FAST + beq kvmppc_p9_exit_interrupt + ld r10,HSTATE_SCRATCH0(r13) + lbz r11,HSTATE_IN_GUEST(r13) li r11,PACA_EXGEN cmpdi r10,0x200 bgt+ 1f @@ -114,3 +129,169 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) GET_SCRATCH0(r13) HRFI_TO_KERNEL #endif + +/* Stack frame offsets for kvmppc_hv_entry */ +#define SFS 208 +#define STACK_SLOT_VCPU (SFS-8) +#define STACK_SLOT_NVGPRS (SFS-152) /* 18 gprs */ + +/* + * void kvmppc_p9_enter_guest(struct vcpu *vcpu); + * + * Enter the guest on a ISAv3.0 or later system where we have exactly + * one vcpu per vcore, and both the host and guest are radix, and threads + * are set to "indepdent mode". + */ +.balign IFETCH_ALIGN_BYTES +_GLOBAL(kvmppc_p9_enter_guest) +EXPORT_SYMBOL_GPL(kvmppc_p9_enter_guest) + mflr r0 + std r0,PPC_LR_STKOFF(r1) + stdu r1,-SFS(r1) + + std r1,HSTATE_HOST_R1(r13) + std r3,STACK_SLOT_VCPU(r1) + + mfcr r4 + stw r4,SFS+8(r1) + + reg = 14 + .rept 18 + std reg,STACK_SLOT_NVGPRS + ((reg - 14) * 8)(r1) + reg = reg + 1 + .endr + + ld r4,VCPU_LR(r3) + mtlr r4 + ld r4,VCPU_CTR(r3) + mtctr r4 + ld r4,VCPU_XER(r3) + mtspr SPRN_XER,r4 + + ld r1,VCPU_CR(r3) + +BEGIN_FTR_SECTION + ld r4,VCPU_CFAR(r3) + mtspr SPRN_CFAR,r4 +END_FTR_SECTION_IFSET(CPU_FTR_CFAR) +BEGIN_FTR_SECTION + ld r4,VCPU_PPR(r3) + mtspr SPRN_PPR,r4 +END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) + + reg = 4 + .rept 28 + ld reg,__VCPU_GPR(reg)(r3) + reg = reg + 1 + .endr + + ld r4,VCPU_KVM(r3) + lbz r4,KVM_SECURE_GUEST(r4) + cmpdi r4,0 + ld r4,VCPU_GPR(R4)(r3) + bne .Lret_to_ultra + + mtcr r1 + + ld r0,VCPU_GPR(R0)(r3) + ld r1,VCPU_GPR(R1)(r3) + ld r2,VCPU_GPR(R2)(r3) + ld r3,VCPU_GPR(R3)(r3) + + HRFI_TO_GUEST + b . + + /* + * Use UV_RETURN ultracall to return control back to the Ultravisor + * after processing an hypercall or interrupt that was forwarded + * (a.k.a. reflected) to the Hypervisor. + * + * All registers have already been reloaded except the ucall requires: + * R0 = hcall result + * R2 = SRR1, so UV can detect a synthesized interrupt (if any) + * R3 = UV_RETURN + */ +.Lret_to_ultra: + mtcr r1 + ld r1,VCPU_GPR(R1)(r3) + + ld r0,VCPU_GPR(R3)(r3) + mfspr r2,SPRN_SRR1 + LOAD_REG_IMMEDIATE(r3, UV_RETURN) + sc 2 + +/* + * kvmppc_p9_exit_hcall and kvmppc_p9_exit_interrupt are branched to from + * above if the interrupt was taken for a guest that was entered via + * kvmppc_p9_enter_guest(). + * + * This code recovers the host stack and vcpu pointer, saves all GPRs and + * CR, LR, CTR, XER as well as guest MSR and NIA into the VCPU, then re- + * establishes the host stack and registers to return from the + * kvmppc_p9_enter_guest() function. + */ +.balign IFETCH_ALIGN_BYTES +kvmppc_p9_exit_hcall: + mfspr r11,SPRN_SRR0 + mfspr r12,SPRN_SRR1 + li r10,0xc00 + std r10,HSTATE_SCRATCH0(r13) + +.balign IFETCH_ALIGN_BYTES +kvmppc_p9_exit_interrupt: + std r1,HSTATE_SCRATCH1(r13) + std r3,HSTATE_SCRATCH2(r13) + ld r1,HSTATE_HOST_R1(r13) + ld r3,STACK_SLOT_VCPU(r1) + + std r9,VCPU_CR(r3) + +1: + std r11,VCPU_PC(r3) + std r12,VCPU_MSR(r3) + + reg = 14 + .rept 18 + std reg,__VCPU_GPR(reg)(r3) + reg = reg + 1 + .endr + + /* r1, r3, r9-r13 are saved to vcpu by C code */ + std r0,VCPU_GPR(R0)(r3) + std r2,VCPU_GPR(R2)(r3) + reg = 4 + .rept 5 + std reg,__VCPU_GPR(reg)(r3) + reg = reg + 1 + .endr + + ld r2,PACATOC(r13) + + mflr r4 + std r4,VCPU_LR(r3) + mfspr r4,SPRN_XER + std r4,VCPU_XER(r3) + + reg = 14 + .rept 18 + ld reg,STACK_SLOT_NVGPRS + ((reg - 14) * 8)(r1) + reg = reg + 1 + .endr + + lwz r4,SFS+8(r1) + mtcr r4 + + /* + * Flush the link stack here, before executing the first blr on the + * way out of the guest. + * + * The link stack won't match coming out of the guest anyway so the + * only cost is the flush itself. The call clobbers r0. + */ +1: nop + patch_site 1b patch__call_kvm_flush_link_stack_2 + + addi r1,r1,SFS + ld r0,PPC_LR_STKOFF(r1) + mtlr r0 + blr diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 1997cf347d3e..28a2761515e3 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -1421,6 +1421,8 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu, */ case BOOK3S_INTERRUPT_H_DATA_STORAGE: r = RESUME_PAGE_FAULT; + if (vcpu->arch.fault_dsisr == HDSISR_CANARY) + r = RESUME_GUEST; /* Just retry if it's the canary */ break; case BOOK3S_INTERRUPT_H_INST_STORAGE: vcpu->arch.fault_dar = kvmppc_get_pc(vcpu); @@ -3736,14 +3738,14 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, vcpu->arch.shregs.dsisr = mfspr(SPRN_DSISR); vcpu->arch.psscr = mfspr(SPRN_PSSCR_PR); mtspr(SPRN_PSSCR_PR, host_psscr); - /* H_CEDE has to be handled now, not later */ - if (trap == BOOK3S_INTERRUPT_SYSCALL && !vcpu->arch.nested && + if (trap == BOOK3S_INTERRUPT_SYSCALL && kvmppc_get_gpr(vcpu, 3) == H_CEDE) { kvmppc_cede(vcpu); kvmppc_set_gpr(vcpu, 3, 0); trap = 0; } + } else { kvmppc_xive_push_vcpu(vcpu); trap = kvmhv_load_hv_regs_and_go(vcpu, time_limit, lpcr); @@ -3768,9 +3770,10 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, } } kvmppc_xive_pull_vcpu(vcpu); + + vcpu->arch.slb_max = 0; } - vcpu->arch.slb_max = 0; dec = mfspr(SPRN_DEC); if (!(lpcr & LPCR_LD)) /* Sign extend if not using large decrementer */ dec = (s32) dec; @@ -4429,11 +4432,19 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu) else r = kvmppc_run_vcpu(vcpu); - if (run->exit_reason == KVM_EXIT_PAPR_HCALL && - !(vcpu->arch.shregs.msr & MSR_PR)) { - trace_kvm_hcall_enter(vcpu); - r = kvmppc_pseries_do_hcall(vcpu); - trace_kvm_hcall_exit(vcpu, r); + if (run->exit_reason == KVM_EXIT_PAPR_HCALL) { + if (unlikely(vcpu->arch.shregs.msr & MSR_PR)) { + /* + * Guest userspace executed sc 1, reflect it + * back as a privileged program check interrupt. + */ + kvmppc_core_queue_program(vcpu, SRR1_PROGPRIV); + r = RESUME_GUEST; + } else { + trace_kvm_hcall_enter(vcpu); + r = kvmppc_pseries_do_hcall(vcpu); + trace_kvm_hcall_exit(vcpu, r); + } kvmppc_core_prepare_to_enter(vcpu); } else if (r == RESUME_PAGE_FAULT) { srcu_idx = srcu_read_lock(&kvm->srcu); diff --git a/arch/powerpc/kvm/book3s_hv_interrupt.c b/arch/powerpc/kvm/book3s_hv_interrupt.c new file mode 100644 index 000000000000..5a7b036c447f --- /dev/null +++ b/arch/powerpc/kvm/book3s_hv_interrupt.c @@ -0,0 +1,221 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include +#include +#include +#include + +#ifdef CONFIG_KVM_BOOK3S_HV_EXIT_TIMING +static void __start_timing(struct kvm_vcpu *vcpu, struct kvmhv_tb_accumulator *next) +{ + struct kvmppc_vcore *vc = vcpu->arch.vcore; + u64 tb = mftb() - vc->tb_offset_applied; + + vcpu->arch.cur_activity = next; + vcpu->arch.cur_tb_start = tb; +} + +static void __accumulate_time(struct kvm_vcpu *vcpu, struct kvmhv_tb_accumulator *next) +{ + struct kvmppc_vcore *vc = vcpu->arch.vcore; + struct kvmhv_tb_accumulator *curr; + u64 tb = mftb() - vc->tb_offset_applied; + u64 prev_tb; + u64 delta; + u64 seq; + + curr = vcpu->arch.cur_activity; + vcpu->arch.cur_activity = next; + prev_tb = vcpu->arch.cur_tb_start; + vcpu->arch.cur_tb_start = tb; + + if (!curr) + return; + + delta = tb - prev_tb; + + seq = curr->seqcount; + curr->seqcount = seq + 1; + smp_wmb(); + curr->tb_total += delta; + if (seq == 0 || delta < curr->tb_min) + curr->tb_min = delta; + if (delta > curr->tb_max) + curr->tb_max = delta; + smp_wmb(); + curr->seqcount = seq + 2; +} + +#define start_timing(vcpu, next) __start_timing(vcpu, next) +#define end_timing(vcpu) __start_timing(vcpu, NULL) +#define accumulate_time(vcpu, next) __accumulate_time(vcpu, next) +#else +#define start_timing(vcpu, next) do {} while (0) +#define end_timing(vcpu) do {} while (0) +#define accumulate_time(vcpu, next) do {} while (0) +#endif + +static inline void mfslb(unsigned int idx, u64 *slbee, u64 *slbev) +{ + asm volatile("slbmfev %0,%1" : "=r" (*slbev) : "r" (idx)); + asm volatile("slbmfee %0,%1" : "=r" (*slbee) : "r" (idx)); +} + +static inline void mtslb(unsigned int idx, u64 slbee, u64 slbev) +{ + BUG_ON((slbee & 0xfff) != idx); + + asm volatile("slbmte %0,%1" :: "r" (slbev), "r" (slbee)); +} + +static inline void slb_invalidate(unsigned int ih) +{ + asm volatile("slbia %0" :: "i"(ih)); +} + +/* + * Malicious or buggy radix guests may have inserted SLB entries + * (only 0..3 because radix always runs with UPRT=1), so these must + * be cleared here to avoid side-channels. slbmte is used rather + * than slbia, as it won't clear cached translations. + */ +static void radix_clear_slb(void) +{ + u64 slbee, slbev; + int i; + + for (i = 0; i < 4; i++) { + mfslb(i, &slbee, &slbev); + if (unlikely(slbee || slbev)) { + slbee = i; + slbev = 0; + mtslb(i, slbee, slbev); + } + } +} + +int __kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu) +{ + u64 *exsave; + unsigned long msr = mfmsr(); + int trap; + + start_timing(vcpu, &vcpu->arch.rm_entry); + + vcpu->arch.ceded = 0; + + WARN_ON_ONCE(vcpu->arch.shregs.msr & MSR_HV); + WARN_ON_ONCE(!(vcpu->arch.shregs.msr & MSR_ME)); + + mtspr(SPRN_HSRR0, vcpu->arch.regs.nip); + mtspr(SPRN_HSRR1, (vcpu->arch.shregs.msr & ~MSR_HV) | MSR_ME); + + /* + * On POWER9 DD2.1 and below, sometimes on a Hypervisor Data Storage + * Interrupt (HDSI) the HDSISR is not be updated at all. + * + * To work around this we put a canary value into the HDSISR before + * returning to a guest and then check for this canary when we take a + * HDSI. If we find the canary on a HDSI, we know the hardware didn't + * update the HDSISR. In this case we return to the guest to retake the + * HDSI which should correctly update the HDSISR the second time HDSI + * entry. + * + * Just do this on all p9 processors for now. + */ + mtspr(SPRN_HDSISR, HDSISR_CANARY); + + accumulate_time(vcpu, &vcpu->arch.guest_time); + + local_paca->kvm_hstate.in_guest = KVM_GUEST_MODE_GUEST_HV_FAST; + kvmppc_p9_enter_guest(vcpu); + // Radix host and guest means host never runs with guest MMU state + local_paca->kvm_hstate.in_guest = KVM_GUEST_MODE_NONE; + + accumulate_time(vcpu, &vcpu->arch.rm_intr); + + /* Get these from r11/12 and paca exsave */ + vcpu->arch.shregs.srr0 = mfspr(SPRN_SRR0); + vcpu->arch.shregs.srr1 = mfspr(SPRN_SRR1); + vcpu->arch.shregs.dar = mfspr(SPRN_DAR); + vcpu->arch.shregs.dsisr = mfspr(SPRN_DSISR); + + trap = local_paca->kvm_hstate.scratch0 & ~0x2; + if (likely(trap > BOOK3S_INTERRUPT_MACHINE_CHECK)) { + exsave = local_paca->exgen; + } else if (trap == BOOK3S_INTERRUPT_SYSTEM_RESET) { + exsave = local_paca->exnmi; + } else { /* trap == 0x200 */ + exsave = local_paca->exmc; + } + + vcpu->arch.regs.gpr[1] = local_paca->kvm_hstate.scratch1; + vcpu->arch.regs.gpr[3] = local_paca->kvm_hstate.scratch2; + vcpu->arch.regs.gpr[9] = exsave[EX_R9/sizeof(u64)]; + vcpu->arch.regs.gpr[10] = exsave[EX_R10/sizeof(u64)]; + vcpu->arch.regs.gpr[11] = exsave[EX_R11/sizeof(u64)]; + vcpu->arch.regs.gpr[12] = exsave[EX_R12/sizeof(u64)]; + vcpu->arch.regs.gpr[13] = exsave[EX_R13/sizeof(u64)]; + vcpu->arch.ppr = exsave[EX_PPR/sizeof(u64)]; + vcpu->arch.cfar = exsave[EX_CFAR/sizeof(u64)]; + vcpu->arch.regs.ctr = exsave[EX_CTR/sizeof(u64)]; + + vcpu->arch.last_inst = KVM_INST_FETCH_FAILED; + + if (unlikely(trap == BOOK3S_INTERRUPT_MACHINE_CHECK)) { + vcpu->arch.fault_dar = exsave[EX_DAR/sizeof(u64)]; + vcpu->arch.fault_dsisr = exsave[EX_DSISR/sizeof(u64)]; + kvmppc_realmode_machine_check(vcpu); + + } else if (unlikely(trap == BOOK3S_INTERRUPT_HMI)) { + kvmppc_realmode_hmi_handler(); + + } else if (trap == BOOK3S_INTERRUPT_H_EMUL_ASSIST) { + vcpu->arch.emul_inst = mfspr(SPRN_HEIR); + + } else if (trap == BOOK3S_INTERRUPT_H_DATA_STORAGE) { + vcpu->arch.fault_dar = exsave[EX_DAR/sizeof(u64)]; + vcpu->arch.fault_dsisr = exsave[EX_DSISR/sizeof(u64)]; + vcpu->arch.fault_gpa = mfspr(SPRN_ASDR); + + } else if (trap == BOOK3S_INTERRUPT_H_INST_STORAGE) { + vcpu->arch.fault_gpa = mfspr(SPRN_ASDR); + + } else if (trap == BOOK3S_INTERRUPT_H_FAC_UNAVAIL) { + vcpu->arch.hfscr = mfspr(SPRN_HFSCR); + +#ifdef CONFIG_PPC_TRANSACTIONAL_MEM + /* + * Softpatch interrupt for transactional memory emulation cases + * on POWER9 DD2.2. This is early in the guest exit path - we + * haven't saved registers or done a treclaim yet. + */ + } else if (trap == BOOK3S_INTERRUPT_HV_SOFTPATCH) { + vcpu->arch.emul_inst = mfspr(SPRN_HEIR); + + /* + * The cases we want to handle here are those where the guest + * is in real suspend mode and is trying to transition to + * transactional mode. + */ + if (local_paca->kvm_hstate.fake_suspend && + (vcpu->arch.shregs.msr & MSR_TS_S)) { + if (kvmhv_p9_tm_emulation_early(vcpu)) { + /* Prevent it being handled again. */ + trap = 0; + } + } +#endif + } + + radix_clear_slb(); + + __mtmsrd(msr, 0); + + accumulate_time(vcpu, &vcpu->arch.rm_exit); + + end_timing(vcpu); + + return trap; +} +EXPORT_SYMBOL_GPL(__kvmhv_vcpu_entry_p9); diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index 2d0d14ed1d92..6118e8a97ddd 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -44,9 +44,8 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) #define NAPPING_UNSPLIT 3 /* Stack frame offsets for kvmppc_hv_entry */ -#define SFS 208 +#define SFS 160 #define STACK_SLOT_TRAP (SFS-4) -#define STACK_SLOT_SHORT_PATH (SFS-8) #define STACK_SLOT_TID (SFS-16) #define STACK_SLOT_PSSCR (SFS-24) #define STACK_SLOT_PID (SFS-32) @@ -59,8 +58,6 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) #define STACK_SLOT_UAMOR (SFS-88) #define STACK_SLOT_DAWR1 (SFS-96) #define STACK_SLOT_DAWRX1 (SFS-104) -/* the following is used by the P9 short path */ -#define STACK_SLOT_NVGPRS (SFS-152) /* 18 gprs */ /* * Call kvmppc_hv_entry in real mode. @@ -1008,9 +1005,6 @@ END_MMU_FTR_SECTION_IFCLR(MMU_FTR_TYPE_RADIX) no_xive: #endif /* CONFIG_KVM_XICS */ - li r0, 0 - stw r0, STACK_SLOT_SHORT_PATH(r1) - deliver_guest_interrupt: /* r4 = vcpu, r13 = paca */ /* Check if we can deliver an external or decrementer interrupt now */ ld r0, VCPU_PENDING_EXC(r4) @@ -1030,7 +1024,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) mtspr SPRN_SRR0, r6 mtspr SPRN_SRR1, r7 -fast_guest_entry_c: ld r10, VCPU_PC(r4) ld r11, VCPU_MSR(r4) /* r11 = vcpu->arch.msr & ~MSR_HV */ @@ -1135,97 +1128,6 @@ ret_to_ultra: ld r4, VCPU_GPR(R4)(r4) sc 2 -/* - * Enter the guest on a P9 or later system where we have exactly - * one vcpu per vcore and we don't need to go to real mode - * (which implies that host and guest are both using radix MMU mode). - * r3 = vcpu pointer - * Most SPRs and all the VSRs have been loaded already. - */ -_GLOBAL(__kvmhv_vcpu_entry_p9) -EXPORT_SYMBOL_GPL(__kvmhv_vcpu_entry_p9) - mflr r0 - std r0, PPC_LR_STKOFF(r1) - stdu r1, -SFS(r1) - - li r0, 1 - stw r0, STACK_SLOT_SHORT_PATH(r1) - - std r3, HSTATE_KVM_VCPU(r13) - mfcr r4 - stw r4, SFS+8(r1) - - std r1, HSTATE_HOST_R1(r13) - - reg = 14 - .rept 18 - std reg, STACK_SLOT_NVGPRS + ((reg - 14) * 8)(r1) - reg = reg + 1 - .endr - - reg = 14 - .rept 18 - ld reg, __VCPU_GPR(reg)(r3) - reg = reg + 1 - .endr - - mfmsr r10 - std r10, HSTATE_HOST_MSR(r13) - - mr r4, r3 - b fast_guest_entry_c -guest_exit_short_path: - /* - * Malicious or buggy radix guests may have inserted SLB entries - * (only 0..3 because radix always runs with UPRT=1), so these must - * be cleared here to avoid side-channels. slbmte is used rather - * than slbia, as it won't clear cached translations. - */ - li r0,0 - slbmte r0,r0 - li r4,1 - slbmte r0,r4 - li r4,2 - slbmte r0,r4 - li r4,3 - slbmte r0,r4 - - li r0, KVM_GUEST_MODE_NONE - stb r0, HSTATE_IN_GUEST(r13) - - reg = 14 - .rept 18 - std reg, __VCPU_GPR(reg)(r9) - reg = reg + 1 - .endr - - reg = 14 - .rept 18 - ld reg, STACK_SLOT_NVGPRS + ((reg - 14) * 8)(r1) - reg = reg + 1 - .endr - - lwz r4, SFS+8(r1) - mtcr r4 - - mr r3, r12 /* trap number */ - - addi r1, r1, SFS - ld r0, PPC_LR_STKOFF(r1) - mtlr r0 - - /* If we are in real mode, do a rfid to get back to the caller */ - mfmsr r4 - andi. r5, r4, MSR_IR - bnelr - rldicl r5, r4, 64 - MSR_TS_S_LG, 62 /* extract TS field */ - mtspr SPRN_SRR0, r0 - ld r10, HSTATE_HOST_MSR(r13) - rldimi r10, r5, MSR_TS_S_LG, 63 - MSR_TS_T_LG - mtspr SPRN_SRR1, r10 - RFI_TO_KERNEL - b . - secondary_too_late: li r12, 0 stw r12, STACK_SLOT_TRAP(r1) @@ -1397,14 +1299,9 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) mr r4,r9 bge fast_guest_return 2: - /* If we came in through the P9 short path, no real mode hcalls */ - lwz r0, STACK_SLOT_SHORT_PATH(r1) - cmpwi r0, 0 - bne no_try_real /* See if this is an hcall we can handle in real mode */ cmpwi r12,BOOK3S_INTERRUPT_SYSCALL beq hcall_try_real_mode -no_try_real: /* Hypervisor doorbell - exit only if host IPI flag set */ cmpwi r12, BOOK3S_INTERRUPT_H_DOORBELL @@ -1447,11 +1344,6 @@ guest_exit_cont: /* r9 = vcpu, r12 = trap, r13 = paca */ bl kvmhv_accumulate_time #endif #ifdef CONFIG_KVM_XICS - /* If we came in through the P9 short path, xive pull is done in C */ - lwz r0, STACK_SLOT_SHORT_PATH(r1) - cmpwi r0, 0 - bne 1f - /* We are exiting, pull the VP from the XIVE */ lbz r0, VCPU_XIVE_PUSHED(r9) cmpwi cr0, r0, 0 @@ -1496,11 +1388,6 @@ guest_exit_cont: /* r9 = vcpu, r12 = trap, r13 = paca */ 1: nop patch_site 1b patch__call_kvm_flush_link_stack - /* If we came in through the P9 short path, go back out to C now */ - lwz r0, STACK_SLOT_SHORT_PATH(r1) - cmpwi r0, 0 - bne guest_exit_short_path - /* For hash guest, read the guest SLB and save it away */ ld r5, VCPU_KVM(r9) lbz r0, KVM_RADIX(r5) @@ -1548,8 +1435,10 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_RADIX) b guest_bypass 0: /* - * Sanitise radix guest SLB, see guest_exit_short_path comment. - * We clear vcpu->arch.slb_max to match earlier behaviour. + * Malicious or buggy radix guests may have inserted SLB entries + * (only 0..3 because radix always runs with UPRT=1), so these must + * be cleared here to avoid side-channels. slbmte is used rather + * than slbia, as it won't clear cached translations. */ li r0,0 stw r0,VCPU_SLB_MAX(r9) @@ -3362,7 +3251,7 @@ BEGIN_FTR_SECTION mtspr SPRN_DAWRX1, r0 END_FTR_SECTION_IFSET(CPU_FTR_DAWR1) - /* Clear hash and radix guest SLB, see guest_exit_short_path comment. */ + /* Clear hash and radix guest SLB. */ slbmte r0, r0 PPC_SLBIA(6) diff --git a/arch/powerpc/kvm/book3s_xive.c b/arch/powerpc/kvm/book3s_xive.c index d2266d36a7c7..cb03ed147b42 100644 --- a/arch/powerpc/kvm/book3s_xive.c +++ b/arch/powerpc/kvm/book3s_xive.c @@ -161,6 +161,40 @@ void kvmppc_xive_pull_vcpu(struct kvm_vcpu *vcpu) } EXPORT_SYMBOL_GPL(kvmppc_xive_pull_vcpu); +void kvmppc_xive_cede_vcpu(struct kvm_vcpu *vcpu) +{ + void __iomem *esc_vaddr = (void __iomem *)vcpu->arch.xive_esc_vaddr; + + if (!esc_vaddr) + return; + + /* we are using XIVE with single escalation */ + + if (vcpu->arch.xive_esc_on) { + /* + * If we still have a pending escalation, abort the cede, + * and we must set PQ to 10 rather than 00 so that we don't + * potentially end up with two entries for the escalation + * interrupt in the XIVE interrupt queue. In that case + * we also don't want to set xive_esc_on to 1 here in + * case we race with xive_esc_irq(). + */ + vcpu->arch.ceded = 0; + /* + * The escalation interrupts are special as we don't EOI them. + * There is no need to use the load-after-store ordering offset + * to set PQ to 10 as we won't use StoreEOI. + */ + __raw_readq(esc_vaddr + XIVE_ESB_SET_PQ_10); + } else { + vcpu->arch.xive_esc_on = true; + mb(); + __raw_readq(esc_vaddr + XIVE_ESB_SET_PQ_00); + } + mb(); +} +EXPORT_SYMBOL_GPL(kvmppc_xive_cede_vcpu); + /* * This is a simple trigger for a generic XIVE IRQ. This must * only be called for interrupts that support a trigger page From patchwork Thu Feb 25 13:46:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444394 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=po96VUhV; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ0F68Qhz9sWW for ; Fri, 26 Feb 2021 00:49:33 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229596AbhBYNtR (ORCPT ); Thu, 25 Feb 2021 08:49:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229954AbhBYNtO (ORCPT ); Thu, 25 Feb 2021 08:49:14 -0500 Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BCCE6C06178A for ; Thu, 25 Feb 2021 05:48:27 -0800 (PST) Received: by mail-pf1-x435.google.com with SMTP id 201so3646577pfw.5 for ; Thu, 25 Feb 2021 05:48:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WOBDp9+4bbeXpaMJ2aqQcKk8uET1HUF6JVdPf/T7HGY=; b=po96VUhV6ewN5kgH0dxIV1O0LoEsfULxB7F1Mw85RetpUF+36GjyXrXmyMlCPNI7qw 0bpSrQBVbnlNG5fxp+Yl0WM6QFLDBTlY1NJsuMQj6GnQTgykB/yphuBM6DZVH2abTqRn vPX8FLtB48cboqAwc1CBtG64/PVcMA33CjHpHvoeEKSY/SQXdrr2vUo5OhLzJQrntLJ2 5WBCDLKqqUEJ5O9vQ/9tK8OtTQqHBnKQU3Cri6RTVnJ/HgPEpQTithpmOR/CW+zftEPy 3srHn3zwHiSnVTesjA8D0u/hCoSdkkcmJTMMIBJjySaLEm2HI1EZM7KbsP6q/inCWj9/ rNnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WOBDp9+4bbeXpaMJ2aqQcKk8uET1HUF6JVdPf/T7HGY=; b=MULGFWR4US0Cb3KdPqEWVIlmmhU31zvgNIzKCGkKcGqm3GpIAvPQEFXlrjZXq4K1aY ezCM1QXgXSpyQrN+FeOTxFqsSmc9nYn3tSiFi72sNDb6u2y6XU1SCGZY8AZr+/OFX1gF /Ct9SFy0vsxWev9k5X6M0sWmya56XqhU//4sogA545g4vx1YEFHJOad0fKy2gBm8b0i2 m5Fb2k3iUt2QkYZv/teIaxzFgrLoOjccHwWIkvMpQqb630vwVuJa5a6xH6YJIfgOt5Uj OMVMi/I2i0mnKVMd/BVOdV73RdWW9rKvoTfYwZylYPjyDOABRCDSNq27ub11Cmi7OUVi kvmQ== X-Gm-Message-State: AOAM5318L5Tj8od4wts7ym1CyNOWwoyKjmZAhlQzkHwbmjAmwHhC/cda /6xPcUOCq3sDmOsDdsnZ8DemByOOHnc= X-Google-Smtp-Source: ABdhPJw0ALIgIEy19/MZdaDIg8UdWZk5Odqes6JB35w8D18J5DjyGMySMxgvQ80QfbODwMV8php7lw== X-Received: by 2002:a05:6a00:23c5:b029:1e6:2f2e:a438 with SMTP id g5-20020a056a0023c5b02901e62f2ea438mr3426435pfc.75.1614260906630; Thu, 25 Feb 2021 05:48:26 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:25 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 24/37] KVM: PPC: Book3S HV P9: inline kvmhv_load_hv_regs_and_go into __kvmhv_vcpu_entry_p9 Date: Thu, 25 Feb 2021 23:46:39 +1000 Message-Id: <20210225134652.2127648-25-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Now the initial C implementation is done, inline more HV code to make rearranging things easier. And rename __kvmhv_vcpu_entry_p9 to drop the leading underscores as it's now C, and is now a more complete vcpu entry. Signed-off-by: Nicholas Piggin Reviewed-by: Fabiano Rosas --- arch/powerpc/include/asm/kvm_book3s_64.h | 2 +- arch/powerpc/kvm/book3s_hv.c | 181 +---------------------- arch/powerpc/kvm/book3s_hv_interrupt.c | 168 ++++++++++++++++++++- 3 files changed, 169 insertions(+), 182 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_book3s_64.h b/arch/powerpc/include/asm/kvm_book3s_64.h index c214bcffb441..eaf3a562bf1e 100644 --- a/arch/powerpc/include/asm/kvm_book3s_64.h +++ b/arch/powerpc/include/asm/kvm_book3s_64.h @@ -153,7 +153,7 @@ static inline bool kvmhv_vcpu_is_radix(struct kvm_vcpu *vcpu) return radix; } -int __kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu); +int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpcr); #define KVM_DEFAULT_HPT_ORDER 24 /* 16MB HPT by default */ #endif diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 28a2761515e3..f99503acdda5 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3442,183 +3442,6 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc) trace_kvmppc_run_core(vc, 1); } -static void switch_mmu_to_guest_radix(struct kvm *kvm, struct kvm_vcpu *vcpu, u64 lpcr) -{ - struct kvmppc_vcore *vc = vcpu->arch.vcore; - struct kvm_nested_guest *nested = vcpu->arch.nested; - u32 lpid; - - lpid = nested ? nested->shadow_lpid : kvm->arch.lpid; - - mtspr(SPRN_LPID, lpid); - mtspr(SPRN_LPCR, lpcr); - mtspr(SPRN_PID, vcpu->arch.pid); - isync(); - - /* TLBIEL must have LPIDR set, so set guest LPID before flushing. */ - kvmppc_check_need_tlb_flush(kvm, vc->pcpu, nested); -} - -static void switch_mmu_to_host_radix(struct kvm *kvm, u32 pid) -{ - mtspr(SPRN_PID, pid); - mtspr(SPRN_LPID, kvm->arch.host_lpid); - mtspr(SPRN_LPCR, kvm->arch.host_lpcr); - isync(); -} - -/* - * Load up hypervisor-mode registers on P9. - */ -static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit, - unsigned long lpcr) -{ - struct kvm *kvm = vcpu->kvm; - struct kvmppc_vcore *vc = vcpu->arch.vcore; - s64 hdec; - u64 tb, purr, spurr; - int trap; - unsigned long host_hfscr = mfspr(SPRN_HFSCR); - unsigned long host_ciabr = mfspr(SPRN_CIABR); - unsigned long host_dawr0 = mfspr(SPRN_DAWR0); - unsigned long host_dawrx0 = mfspr(SPRN_DAWRX0); - unsigned long host_psscr = mfspr(SPRN_PSSCR); - unsigned long host_pidr = mfspr(SPRN_PID); - unsigned long host_dawr1 = 0; - unsigned long host_dawrx1 = 0; - - if (cpu_has_feature(CPU_FTR_DAWR1)) { - host_dawr1 = mfspr(SPRN_DAWR1); - host_dawrx1 = mfspr(SPRN_DAWRX1); - } - - tb = mftb(); - hdec = time_limit - tb; - if (hdec < 0) - return BOOK3S_INTERRUPT_HV_DECREMENTER; - - if (vc->tb_offset) { - u64 new_tb = tb + vc->tb_offset; - mtspr(SPRN_TBU40, new_tb); - tb = mftb(); - if ((tb & 0xffffff) < (new_tb & 0xffffff)) - mtspr(SPRN_TBU40, new_tb + 0x1000000); - vc->tb_offset_applied = vc->tb_offset; - } - - if (vc->pcr) - mtspr(SPRN_PCR, vc->pcr | PCR_MASK); - mtspr(SPRN_DPDES, vc->dpdes); - mtspr(SPRN_VTB, vc->vtb); - - local_paca->kvm_hstate.host_purr = mfspr(SPRN_PURR); - local_paca->kvm_hstate.host_spurr = mfspr(SPRN_SPURR); - mtspr(SPRN_PURR, vcpu->arch.purr); - mtspr(SPRN_SPURR, vcpu->arch.spurr); - - if (dawr_enabled()) { - mtspr(SPRN_DAWR0, vcpu->arch.dawr0); - mtspr(SPRN_DAWRX0, vcpu->arch.dawrx0); - if (cpu_has_feature(CPU_FTR_DAWR1)) { - mtspr(SPRN_DAWR1, vcpu->arch.dawr1); - mtspr(SPRN_DAWRX1, vcpu->arch.dawrx1); - } - } - mtspr(SPRN_CIABR, vcpu->arch.ciabr); - mtspr(SPRN_IC, vcpu->arch.ic); - - mtspr(SPRN_PSSCR, vcpu->arch.psscr | PSSCR_EC | - (local_paca->kvm_hstate.fake_suspend << PSSCR_FAKE_SUSPEND_LG)); - - mtspr(SPRN_HFSCR, vcpu->arch.hfscr); - - mtspr(SPRN_SPRG0, vcpu->arch.shregs.sprg0); - mtspr(SPRN_SPRG1, vcpu->arch.shregs.sprg1); - mtspr(SPRN_SPRG2, vcpu->arch.shregs.sprg2); - mtspr(SPRN_SPRG3, vcpu->arch.shregs.sprg3); - - mtspr(SPRN_AMOR, ~0UL); - - switch_mmu_to_guest_radix(kvm, vcpu, lpcr); - - /* - * P9 suppresses the HDEC exception when LPCR[HDICE] = 0, - * so set guest LPCR (with HDICE) before writing HDEC. - */ - mtspr(SPRN_HDEC, hdec); - - mtspr(SPRN_SRR0, vcpu->arch.shregs.srr0); - mtspr(SPRN_SRR1, vcpu->arch.shregs.srr1); - - trap = __kvmhv_vcpu_entry_p9(vcpu); - - /* Advance host PURR/SPURR by the amount used by guest */ - purr = mfspr(SPRN_PURR); - spurr = mfspr(SPRN_SPURR); - mtspr(SPRN_PURR, local_paca->kvm_hstate.host_purr + - purr - vcpu->arch.purr); - mtspr(SPRN_SPURR, local_paca->kvm_hstate.host_spurr + - spurr - vcpu->arch.spurr); - vcpu->arch.purr = purr; - vcpu->arch.spurr = spurr; - - vcpu->arch.ic = mfspr(SPRN_IC); - vcpu->arch.pid = mfspr(SPRN_PID); - vcpu->arch.psscr = mfspr(SPRN_PSSCR) & PSSCR_GUEST_VIS; - - vcpu->arch.shregs.sprg0 = mfspr(SPRN_SPRG0); - vcpu->arch.shregs.sprg1 = mfspr(SPRN_SPRG1); - vcpu->arch.shregs.sprg2 = mfspr(SPRN_SPRG2); - vcpu->arch.shregs.sprg3 = mfspr(SPRN_SPRG3); - - /* Preserve PSSCR[FAKE_SUSPEND] until we've called kvmppc_save_tm_hv */ - mtspr(SPRN_PSSCR, host_psscr | - (local_paca->kvm_hstate.fake_suspend << PSSCR_FAKE_SUSPEND_LG)); - mtspr(SPRN_HFSCR, host_hfscr); - mtspr(SPRN_CIABR, host_ciabr); - mtspr(SPRN_DAWR0, host_dawr0); - mtspr(SPRN_DAWRX0, host_dawrx0); - if (cpu_has_feature(CPU_FTR_DAWR1)) { - mtspr(SPRN_DAWR1, host_dawr1); - mtspr(SPRN_DAWRX1, host_dawrx1); - } - - /* - * Since this is radix, do a eieio; tlbsync; ptesync sequence in - * case we interrupted the guest between a tlbie and a ptesync. - */ - asm volatile("eieio; tlbsync; ptesync"); - - /* - * cp_abort is required if the processor supports local copy-paste - * to clear the copy buffer that was under control of the guest. - */ - if (cpu_has_feature(CPU_FTR_ARCH_31)) - asm volatile(PPC_CP_ABORT); - - vc->dpdes = mfspr(SPRN_DPDES); - vc->vtb = mfspr(SPRN_VTB); - mtspr(SPRN_DPDES, 0); - if (vc->pcr) - mtspr(SPRN_PCR, PCR_MASK); - - if (vc->tb_offset_applied) { - u64 new_tb = mftb() - vc->tb_offset_applied; - mtspr(SPRN_TBU40, new_tb); - tb = mftb(); - if ((tb & 0xffffff) < (new_tb & 0xffffff)) - mtspr(SPRN_TBU40, new_tb + 0x1000000); - vc->tb_offset_applied = 0; - } - - /* HDEC must be at least as large as DEC, so decrementer_max fits */ - mtspr(SPRN_HDEC, decrementer_max); - - switch_mmu_to_host_radix(kvm, host_pidr); - - return trap; -} - /* * Virtual-mode guest entry for POWER9 and later when the host and * guest are both using the radix MMU. The LPIDR has already been set. @@ -3710,7 +3533,7 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, * We need to save and restore the guest visible part of the * psscr (i.e. using SPRN_PSSCR_PR) since the hypervisor * doesn't do this for us. Note only required if pseries since - * this is done in kvmhv_load_hv_regs_and_go() below otherwise. + * this is done in kvmhv_vcpu_entry_p9() below otherwise. */ unsigned long host_psscr; /* call our hypervisor to load up HV regs and go */ @@ -3748,7 +3571,7 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, } else { kvmppc_xive_push_vcpu(vcpu); - trap = kvmhv_load_hv_regs_and_go(vcpu, time_limit, lpcr); + trap = kvmhv_vcpu_entry_p9(vcpu, time_limit, lpcr); /* H_CEDE has to be handled now, not later */ /* XICS hcalls must be handled before xive is pulled */ if (trap == BOOK3S_INTERRUPT_SYSCALL && diff --git a/arch/powerpc/kvm/book3s_hv_interrupt.c b/arch/powerpc/kvm/book3s_hv_interrupt.c index 5a7b036c447f..dea3eca3648a 100644 --- a/arch/powerpc/kvm/book3s_hv_interrupt.c +++ b/arch/powerpc/kvm/book3s_hv_interrupt.c @@ -55,6 +55,31 @@ static void __accumulate_time(struct kvm_vcpu *vcpu, struct kvmhv_tb_accumulator #define accumulate_time(vcpu, next) do {} while (0) #endif +static void switch_mmu_to_guest_radix(struct kvm *kvm, struct kvm_vcpu *vcpu, u64 lpcr) +{ + struct kvmppc_vcore *vc = vcpu->arch.vcore; + struct kvm_nested_guest *nested = vcpu->arch.nested; + u32 lpid; + + lpid = nested ? nested->shadow_lpid : kvm->arch.lpid; + + mtspr(SPRN_LPID, lpid); + mtspr(SPRN_LPCR, lpcr); + mtspr(SPRN_PID, vcpu->arch.pid); + isync(); + + /* TLBIEL must have LPIDR set, so set guest LPID before flushing. */ + kvmppc_check_need_tlb_flush(kvm, vc->pcpu, nested); +} + +static void switch_mmu_to_host_radix(struct kvm *kvm, u32 pid) +{ + mtspr(SPRN_PID, pid); + mtspr(SPRN_LPID, kvm->arch.host_lpid); + mtspr(SPRN_LPCR, kvm->arch.host_lpcr); + isync(); +} + static inline void mfslb(unsigned int idx, u64 *slbee, u64 *slbev) { asm volatile("slbmfev %0,%1" : "=r" (*slbev) : "r" (idx)); @@ -94,11 +119,86 @@ static void radix_clear_slb(void) } } -int __kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu) +int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpcr) { + struct kvm *kvm = vcpu->kvm; + struct kvmppc_vcore *vc = vcpu->arch.vcore; + s64 hdec; + u64 tb, purr, spurr; u64 *exsave; unsigned long msr = mfmsr(); int trap; + unsigned long host_hfscr = mfspr(SPRN_HFSCR); + unsigned long host_ciabr = mfspr(SPRN_CIABR); + unsigned long host_dawr0 = mfspr(SPRN_DAWR0); + unsigned long host_dawrx0 = mfspr(SPRN_DAWRX0); + unsigned long host_psscr = mfspr(SPRN_PSSCR); + unsigned long host_pidr = mfspr(SPRN_PID); + unsigned long host_dawr1 = 0; + unsigned long host_dawrx1 = 0; + + if (cpu_has_feature(CPU_FTR_DAWR1)) { + host_dawr1 = mfspr(SPRN_DAWR1); + host_dawrx1 = mfspr(SPRN_DAWRX1); + } + + tb = mftb(); + hdec = time_limit - tb; + if (hdec < 0) + return BOOK3S_INTERRUPT_HV_DECREMENTER; + + if (vc->tb_offset) { + u64 new_tb = tb + vc->tb_offset; + mtspr(SPRN_TBU40, new_tb); + tb = mftb(); + if ((tb & 0xffffff) < (new_tb & 0xffffff)) + mtspr(SPRN_TBU40, new_tb + 0x1000000); + vc->tb_offset_applied = vc->tb_offset; + } + + if (vc->pcr) + mtspr(SPRN_PCR, vc->pcr | PCR_MASK); + mtspr(SPRN_DPDES, vc->dpdes); + mtspr(SPRN_VTB, vc->vtb); + + local_paca->kvm_hstate.host_purr = mfspr(SPRN_PURR); + local_paca->kvm_hstate.host_spurr = mfspr(SPRN_SPURR); + mtspr(SPRN_PURR, vcpu->arch.purr); + mtspr(SPRN_SPURR, vcpu->arch.spurr); + + if (dawr_enabled()) { + mtspr(SPRN_DAWR0, vcpu->arch.dawr0); + mtspr(SPRN_DAWRX0, vcpu->arch.dawrx0); + if (cpu_has_feature(CPU_FTR_DAWR1)) { + mtspr(SPRN_DAWR1, vcpu->arch.dawr1); + mtspr(SPRN_DAWRX1, vcpu->arch.dawrx1); + } + } + mtspr(SPRN_CIABR, vcpu->arch.ciabr); + mtspr(SPRN_IC, vcpu->arch.ic); + + mtspr(SPRN_PSSCR, vcpu->arch.psscr | PSSCR_EC | + (local_paca->kvm_hstate.fake_suspend << PSSCR_FAKE_SUSPEND_LG)); + + mtspr(SPRN_HFSCR, vcpu->arch.hfscr); + + mtspr(SPRN_SPRG0, vcpu->arch.shregs.sprg0); + mtspr(SPRN_SPRG1, vcpu->arch.shregs.sprg1); + mtspr(SPRN_SPRG2, vcpu->arch.shregs.sprg2); + mtspr(SPRN_SPRG3, vcpu->arch.shregs.sprg3); + + mtspr(SPRN_AMOR, ~0UL); + + switch_mmu_to_guest_radix(kvm, vcpu, lpcr); + + /* + * P9 suppresses the HDEC exception when LPCR[HDICE] = 0, + * so set guest LPCR (with HDICE) before writing HDEC. + */ + mtspr(SPRN_HDEC, hdec); + + mtspr(SPRN_SRR0, vcpu->arch.shregs.srr0); + mtspr(SPRN_SRR1, vcpu->arch.shregs.srr1); start_timing(vcpu, &vcpu->arch.rm_entry); @@ -216,6 +316,70 @@ int __kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu) end_timing(vcpu); + /* Advance host PURR/SPURR by the amount used by guest */ + purr = mfspr(SPRN_PURR); + spurr = mfspr(SPRN_SPURR); + mtspr(SPRN_PURR, local_paca->kvm_hstate.host_purr + + purr - vcpu->arch.purr); + mtspr(SPRN_SPURR, local_paca->kvm_hstate.host_spurr + + spurr - vcpu->arch.spurr); + vcpu->arch.purr = purr; + vcpu->arch.spurr = spurr; + + vcpu->arch.ic = mfspr(SPRN_IC); + vcpu->arch.pid = mfspr(SPRN_PID); + vcpu->arch.psscr = mfspr(SPRN_PSSCR) & PSSCR_GUEST_VIS; + + vcpu->arch.shregs.sprg0 = mfspr(SPRN_SPRG0); + vcpu->arch.shregs.sprg1 = mfspr(SPRN_SPRG1); + vcpu->arch.shregs.sprg2 = mfspr(SPRN_SPRG2); + vcpu->arch.shregs.sprg3 = mfspr(SPRN_SPRG3); + + /* Preserve PSSCR[FAKE_SUSPEND] until we've called kvmppc_save_tm_hv */ + mtspr(SPRN_PSSCR, host_psscr | + (local_paca->kvm_hstate.fake_suspend << PSSCR_FAKE_SUSPEND_LG)); + mtspr(SPRN_HFSCR, host_hfscr); + mtspr(SPRN_CIABR, host_ciabr); + mtspr(SPRN_DAWR0, host_dawr0); + mtspr(SPRN_DAWRX0, host_dawrx0); + if (cpu_has_feature(CPU_FTR_DAWR1)) { + mtspr(SPRN_DAWR1, host_dawr1); + mtspr(SPRN_DAWRX1, host_dawrx1); + } + + /* + * Since this is radix, do a eieio; tlbsync; ptesync sequence in + * case we interrupted the guest between a tlbie and a ptesync. + */ + asm volatile("eieio; tlbsync; ptesync"); + + /* + * cp_abort is required if the processor supports local copy-paste + * to clear the copy buffer that was under control of the guest. + */ + if (cpu_has_feature(CPU_FTR_ARCH_31)) + asm volatile(PPC_CP_ABORT); + + vc->dpdes = mfspr(SPRN_DPDES); + vc->vtb = mfspr(SPRN_VTB); + mtspr(SPRN_DPDES, 0); + if (vc->pcr) + mtspr(SPRN_PCR, PCR_MASK); + + if (vc->tb_offset_applied) { + u64 new_tb = mftb() - vc->tb_offset_applied; + mtspr(SPRN_TBU40, new_tb); + tb = mftb(); + if ((tb & 0xffffff) < (new_tb & 0xffffff)) + mtspr(SPRN_TBU40, new_tb + 0x1000000); + vc->tb_offset_applied = 0; + } + + /* HDEC must be at least as large as DEC, so decrementer_max fits */ + mtspr(SPRN_HDEC, decrementer_max); + + switch_mmu_to_host_radix(kvm, host_pidr); + return trap; } -EXPORT_SYMBOL_GPL(__kvmhv_vcpu_entry_p9); +EXPORT_SYMBOL_GPL(kvmhv_vcpu_entry_p9); From patchwork Thu Feb 25 13:46:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444383 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=cq9K5pem; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzc0fNcz9sWv for ; Fri, 26 Feb 2021 00:49:00 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231822AbhBYNsu (ORCPT ); Thu, 25 Feb 2021 08:48:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44720 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231960AbhBYNst (ORCPT ); Thu, 25 Feb 2021 08:48:49 -0500 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A45C6C06178B for ; Thu, 25 Feb 2021 05:48:30 -0800 (PST) Received: by mail-pf1-x42e.google.com with SMTP id u26so3639394pfn.6 for ; Thu, 25 Feb 2021 05:48:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ughaSHH/tmuQhZEXSNwZbQ9e2et0A2HKfcBWJCpA71U=; b=cq9K5pem1WGdvui+SUXzh3YALeEpu0WY7nv0OhDWuZlwZbD//vv/onpE1o53RQnSAi eve0cC9bgbdTz5kdxSno47dB4DvF5ySSxhijsh8L3SIUPJKkKz+CEQNZELgSTUjCxAtu hSzDsshIAi1U3aRAyDDdHH/rtAnaXv0WgDWHMDherhqSojDsEXYZgsMWon66Pc0S4QV2 eii0ZTizl9Gv3bb693b9CvHWSbXvsZK9RhMon+aGb81zGJ0HrvSgv4f6QL8sHO0we68N dqK3FXt/m6KJYk5u68UApW/eGZRhZux9M9YlUjWqH7QhHWkShv0/IRyZJYD2kpENZMD5 vB8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ughaSHH/tmuQhZEXSNwZbQ9e2et0A2HKfcBWJCpA71U=; b=ps736BNEKzrB8aBxUGnmWHp3OX8QHNm/hPFl6nXWIyX1D4PxTsOEmG8MwgAW7THkCQ IUIGMQKfa4NQ/tmdYis5rWj+vxOmKPCugiuacG+7AjypdK+EWwcipES9CNynCGYBcUZ8 mbGzqxr3ntBY3y9hodKtiMXcpNL/fYgS7XuIDPinVWHZdfpQdYUjpScUZnP0tuFFKoGG 5/L1wFX/meik4iqe9tw0YsVGTxMfl9tpMNw+d5NGD7sPHm2O/X3I+iHrZXnL+xJfnLee I/rSQ+84q3h8S9RmHNv2kvYCWdpaCKRXeCLGNrvIO0RODO0h8IL66zuyHp/3VRBi4ZX3 R+Rg== X-Gm-Message-State: AOAM532oiYSXjOS6xw+ITVlt3I4LD99eYx9FaeP5hCYIPWYhJ33PcW31 eIiXlImRnwwPUEoDdqvrC+GQof6Z2XM= X-Google-Smtp-Source: ABdhPJxJqz21DM/2kF4hrsQowLl3bgC4xxF4G8pprrbaZhPi5zNrQ/tTUUNg3tUBRu+qm/em8eRNxQ== X-Received: by 2002:a63:d144:: with SMTP id c4mr3003929pgj.196.1614260909712; Thu, 25 Feb 2021 05:48:29 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:29 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 25/37] KVM: PPC: Book3S HV P9: Read machine check registers while MSR[RI] is 0 Date: Thu, 25 Feb 2021 23:46:40 +1000 Message-Id: <20210225134652.2127648-26-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org SRR0/1, DAR, DSISR must all be protected from machine check which can clobber them. Ensure MSR[RI] is clear while they are live. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 5 +++-- arch/powerpc/kvm/book3s_hv_interrupt.c | 26 +++++++++++++++++++++++--- arch/powerpc/kvm/book3s_hv_ras.c | 5 +++++ 3 files changed, 31 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index f99503acdda5..94989fe2fdfe 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3506,8 +3506,7 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, mtspr(SPRN_BESCR, vcpu->arch.bescr); mtspr(SPRN_WORT, vcpu->arch.wort); mtspr(SPRN_TIDR, vcpu->arch.tid); - mtspr(SPRN_DAR, vcpu->arch.shregs.dar); - mtspr(SPRN_DSISR, vcpu->arch.shregs.dsisr); + /* XXX: DAR, DSISR must be set with MSR[RI] clear (or hstate as appropriate) */ mtspr(SPRN_AMR, vcpu->arch.amr); mtspr(SPRN_UAMOR, vcpu->arch.uamor); @@ -3553,6 +3552,8 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, hvregs.vcpu_token = vcpu->vcpu_id; } hvregs.hdec_expiry = time_limit; + mtspr(SPRN_DAR, vcpu->arch.shregs.dar); + mtspr(SPRN_DSISR, vcpu->arch.shregs.dsisr); trap = plpar_hcall_norets(H_ENTER_NESTED, __pa(&hvregs), __pa(&vcpu->arch.regs)); kvmhv_restore_hv_return_state(vcpu, &hvregs); diff --git a/arch/powerpc/kvm/book3s_hv_interrupt.c b/arch/powerpc/kvm/book3s_hv_interrupt.c index dea3eca3648a..f5fef7398e37 100644 --- a/arch/powerpc/kvm/book3s_hv_interrupt.c +++ b/arch/powerpc/kvm/book3s_hv_interrupt.c @@ -126,6 +126,7 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc s64 hdec; u64 tb, purr, spurr; u64 *exsave; + bool ri_clear; unsigned long msr = mfmsr(); int trap; unsigned long host_hfscr = mfspr(SPRN_HFSCR); @@ -197,9 +198,6 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc */ mtspr(SPRN_HDEC, hdec); - mtspr(SPRN_SRR0, vcpu->arch.shregs.srr0); - mtspr(SPRN_SRR1, vcpu->arch.shregs.srr1); - start_timing(vcpu, &vcpu->arch.rm_entry); vcpu->arch.ceded = 0; @@ -225,6 +223,13 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc */ mtspr(SPRN_HDSISR, HDSISR_CANARY); + __mtmsrd(0, 1); /* clear RI */ + + mtspr(SPRN_DAR, vcpu->arch.shregs.dar); + mtspr(SPRN_DSISR, vcpu->arch.shregs.dsisr); + mtspr(SPRN_SRR0, vcpu->arch.shregs.srr0); + mtspr(SPRN_SRR1, vcpu->arch.shregs.srr1); + accumulate_time(vcpu, &vcpu->arch.guest_time); local_paca->kvm_hstate.in_guest = KVM_GUEST_MODE_GUEST_HV_FAST; @@ -240,6 +245,13 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc vcpu->arch.shregs.dar = mfspr(SPRN_DAR); vcpu->arch.shregs.dsisr = mfspr(SPRN_DSISR); + /* HSRR interrupts leave MSR[RI] unchanged, SRR interrupts clear it. */ + if ((local_paca->kvm_hstate.scratch0 & 0x2) && + (vcpu->arch.shregs.msr & MSR_RI)) + ri_clear = false; + else + ri_clear = true; + trap = local_paca->kvm_hstate.scratch0 & ~0x2; if (likely(trap > BOOK3S_INTERRUPT_MACHINE_CHECK)) { exsave = local_paca->exgen; @@ -251,6 +263,14 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc vcpu->arch.regs.gpr[1] = local_paca->kvm_hstate.scratch1; vcpu->arch.regs.gpr[3] = local_paca->kvm_hstate.scratch2; + + if (ri_clear) { +/// XXX this fires maybe on syscalls on mambo WARN_ON((mfmsr() & MSR_RI)); + __mtmsrd(MSR_RI, 1); /* set RI after reading machine check regs (DAR, DSISR, SRR0/1) and hstate scratch (which we need to move into exsave) */ + } else { + WARN_ON(!(mfmsr() & MSR_RI)); + } + vcpu->arch.regs.gpr[9] = exsave[EX_R9/sizeof(u64)]; vcpu->arch.regs.gpr[10] = exsave[EX_R10/sizeof(u64)]; vcpu->arch.regs.gpr[11] = exsave[EX_R11/sizeof(u64)]; diff --git a/arch/powerpc/kvm/book3s_hv_ras.c b/arch/powerpc/kvm/book3s_hv_ras.c index d4bca93b79f6..7a645f4428c2 100644 --- a/arch/powerpc/kvm/book3s_hv_ras.c +++ b/arch/powerpc/kvm/book3s_hv_ras.c @@ -198,6 +198,7 @@ static void kvmppc_tb_resync_done(void) * value. Hence the idea is to resync the TB on every HMI, so that we * know about the exact state of the TB value. Resync TB call will * restore TB to host timebase. + * XXX: could use new opal hmi handler flags for this * * Things to consider: * - On TB error, HMI interrupt is reported on all the threads of the core @@ -290,6 +291,10 @@ long kvmppc_realmode_hmi_handler(void) */ wait_for_subcore_guest_exit(); + /* + * XXX: Is this safe with independent threads mode? + */ + /* * At this point we are sure that primary threads from each * subcore on this core have completed guest->host partition From patchwork Thu Feb 25 13:46:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444395 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=s/3/KaxD; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ0G6xRkz9sVR for ; Fri, 26 Feb 2021 00:49:34 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229954AbhBYNtY (ORCPT ); Thu, 25 Feb 2021 08:49:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229566AbhBYNtO (ORCPT ); Thu, 25 Feb 2021 08:49:14 -0500 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8BE1DC06178C for ; Thu, 25 Feb 2021 05:48:33 -0800 (PST) Received: by mail-pj1-x1030.google.com with SMTP id o22so5064690pjs.1 for ; Thu, 25 Feb 2021 05:48:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z1msQbx0Hbt6eMglomvgJz+el+SEDc6l3SYc2zTEVxU=; b=s/3/KaxDPHyERPd/FnBJhlvLUnkzX9y1IsZx0j3QaTcL/Gipm0lPbo8XGLZvDRSftk hBlGclraQunoazuEo+vX574p2QYOejng4GO1m/lIeZQxSVJ1xNHpuLaj2IYY5l4A43JM 02KT3G4q3jFJTj/YQ1HoGkesgqqDKw/IC6A5VCiJfRT8fbkwOQN6PpQnYhFXDDn/AcuS ZN8H7/WEIFJvmeTAumrTGRJeiLilgigsFAZkXaEUR80YIk9fI79TC6vc3Vhs1Z7YEm2R dj5MQ9cZ81OZEECp/nklW5t5o4x3MOhDtCV78/bNYfYPF8L6ItybhStJ2rRbW7KEvEG2 UPFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z1msQbx0Hbt6eMglomvgJz+el+SEDc6l3SYc2zTEVxU=; b=gSW2ufTtpU6M1qPicqiUCCWDN3m+Yxq/yUT3SL6sb+Gvm1B/5YfWa19IBKY8YiJ/X0 uJMdvXsPfvIN0g4j1Sc/kvVDMrdR9qJZjkBrtk6qniQHjha7Ii1VBQfBAgijVcO9/KDz rg1zY5HVQBgfYM4F/ek843Be3zA4b5yEBVycJM7BaWs9JFJnAivmGn2MYQmoT4SLd0Wx CzzR9gpoYxZx0IGorFdBwYOLPRKZtAalclvCH3AsMC2XRxRS0qWbljjb5hOhwuUBnGfR 5A3uXQjpED6CnYPcWpptWnTqdg7bBLP6bHWcFmhAkfOuh1c6brxqIJ5Fi5bjGRcXc5jP 7d6w== X-Gm-Message-State: AOAM532gn3/QqHUr5SVLru6gOh8ctAvzFlBo163RyjpPsLfPSdZxx7dt deMLIMau/Q86sQh1hXm1qbQEwAIA8PU= X-Google-Smtp-Source: ABdhPJwHr4Jo55/pa7u26cgrj7FDeHBSv5PrK2PSvYxQlz7IUuv26xNR9pMVXf3fZ7hr1Iwpv+dYsg== X-Received: by 2002:a17:90b:1290:: with SMTP id fw16mr3374278pjb.99.1614260912722; Thu, 25 Feb 2021 05:48:32 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:32 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 26/37] KVM: PPC: Book3S HV P9: Improve exit timing accounting coverage Date: Thu, 25 Feb 2021 23:46:41 +1000 Message-Id: <20210225134652.2127648-27-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org The C conversion caused exit timing to become a bit cramped. Expand it to cover more of the entry and exit code. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv_interrupt.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_interrupt.c b/arch/powerpc/kvm/book3s_hv_interrupt.c index f5fef7398e37..4a158c8fc0bc 100644 --- a/arch/powerpc/kvm/book3s_hv_interrupt.c +++ b/arch/powerpc/kvm/book3s_hv_interrupt.c @@ -148,6 +148,8 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc if (hdec < 0) return BOOK3S_INTERRUPT_HV_DECREMENTER; + start_timing(vcpu, &vcpu->arch.rm_entry); + if (vc->tb_offset) { u64 new_tb = tb + vc->tb_offset; mtspr(SPRN_TBU40, new_tb); @@ -198,8 +200,6 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc */ mtspr(SPRN_HDEC, hdec); - start_timing(vcpu, &vcpu->arch.rm_entry); - vcpu->arch.ceded = 0; WARN_ON_ONCE(vcpu->arch.shregs.msr & MSR_HV); @@ -334,8 +334,6 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc accumulate_time(vcpu, &vcpu->arch.rm_exit); - end_timing(vcpu); - /* Advance host PURR/SPURR by the amount used by guest */ purr = mfspr(SPRN_PURR); spurr = mfspr(SPRN_SPURR); @@ -400,6 +398,8 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc switch_mmu_to_host_radix(kvm, host_pidr); + end_timing(vcpu); + return trap; } EXPORT_SYMBOL_GPL(kvmhv_vcpu_entry_p9); From patchwork Thu Feb 25 13:46:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444385 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=oEq6KFmH; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzd2MQYz9sVv for ; Fri, 26 Feb 2021 00:49:01 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231978AbhBYNsy (ORCPT ); Thu, 25 Feb 2021 08:48:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231960AbhBYNsx (ORCPT ); Thu, 25 Feb 2021 08:48:53 -0500 Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D740BC061793 for ; Thu, 25 Feb 2021 05:48:36 -0800 (PST) Received: by mail-pg1-x52e.google.com with SMTP id a4so3788436pgc.11 for ; Thu, 25 Feb 2021 05:48:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4wpRUIXYsph30RZ+o04YHt9M47A98ys3vHV9XOremLs=; b=oEq6KFmHXRFXCmls+yOQT9IO5CWXZ8JmfrXrz1qZvOOvuCFPvSWwvnMUfjvlaXcWnM k7//repmAwqZY2KIyLPmjx5+ciDhQP23uZ9UQycPgelC3yaUnX/LsJiFI2nsUwpRnwIg kwHUVq63k15vh9zvYWG/Oqs2ZxBuBptilpvff3t0ODpz6qChGSNE0hcvZPxekx5e8Tde fA9VCIsCi3xotst0DZCD4gSJu0qSzwTVXv4dKCba4BT3iqOio7uABDRUuksSOmpSrqBU Ehn7X/0gjEjvJrcUUfMUsttC2cdxxp9ugQ5Be2hGRxqH5oUqDUtkl94/3SIpWugFU3lw I0wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4wpRUIXYsph30RZ+o04YHt9M47A98ys3vHV9XOremLs=; b=JbbNe32DQZoMtks1yhuBM/4gdq3FcquePGUcxC4+UwN9oYn9/IQBwSxVejT+TBrnxT CFMtt1dNbIPus/9bdf8GvndfFKbri/yX0EUI9tBs/IRI4oXrIubSbGMsWqKBYac0kWFa pXDeErjlbNGuTdv+sh81oJVMnKk4bTWN+qPOlSnxRGA5OK6zHxdAOEQQ4BVgFe/frsoK 5htaPVehmH4iPyJ553f6t+upZne7azGUCY176jl6qSv0xo9uwGZCT4AnMZlzBkr5h1ky CiKp3w+q0zIP2aiknY/B12O/uL6hwNTAyCAUDjrVPmACNBeeTY6ltREI31irpA8E1ICX +H8w== X-Gm-Message-State: AOAM53347TQO2B0q0WKxWstafzKi9jo5KD954OD29QVS9OTBSOuc/ElW otWBCWWcPE83C3gB0QMF3jyqHYVijkM= X-Google-Smtp-Source: ABdhPJzooV747wcG/f0eQ83XVGXoIqrpt9BT1iLlSxgTTidzwjL+5hbDniMdc1wEKts8PZDfFFnsFA== X-Received: by 2002:a63:df01:: with SMTP id u1mr3052930pgg.341.1614260916109; Thu, 25 Feb 2021 05:48:36 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:35 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 27/37] KVM: PPC: Book3S HV P9: Move SPR loading after expiry time check Date: Thu, 25 Feb 2021 23:46:42 +1000 Message-Id: <20210225134652.2127648-28-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org This is wasted work if the time limit is exceeded. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv_interrupt.c | 38 ++++++++++++++++---------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_interrupt.c b/arch/powerpc/kvm/book3s_hv_interrupt.c index 4a158c8fc0bc..dd0a78a69f49 100644 --- a/arch/powerpc/kvm/book3s_hv_interrupt.c +++ b/arch/powerpc/kvm/book3s_hv_interrupt.c @@ -126,22 +126,17 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc s64 hdec; u64 tb, purr, spurr; u64 *exsave; - bool ri_clear; - unsigned long msr = mfmsr(); int trap; - unsigned long host_hfscr = mfspr(SPRN_HFSCR); - unsigned long host_ciabr = mfspr(SPRN_CIABR); - unsigned long host_dawr0 = mfspr(SPRN_DAWR0); - unsigned long host_dawrx0 = mfspr(SPRN_DAWRX0); - unsigned long host_psscr = mfspr(SPRN_PSSCR); - unsigned long host_pidr = mfspr(SPRN_PID); - unsigned long host_dawr1 = 0; - unsigned long host_dawrx1 = 0; - - if (cpu_has_feature(CPU_FTR_DAWR1)) { - host_dawr1 = mfspr(SPRN_DAWR1); - host_dawrx1 = mfspr(SPRN_DAWRX1); - } + bool ri_clear; + unsigned long msr; + unsigned long host_hfscr; + unsigned long host_ciabr; + unsigned long host_dawr0; + unsigned long host_dawrx0; + unsigned long host_psscr; + unsigned long host_pidr; + unsigned long host_dawr1; + unsigned long host_dawrx1; tb = mftb(); hdec = time_limit - tb; @@ -159,6 +154,19 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc vc->tb_offset_applied = vc->tb_offset; } + msr = mfmsr(); + + host_hfscr = mfspr(SPRN_HFSCR); + host_ciabr = mfspr(SPRN_CIABR); + host_dawr0 = mfspr(SPRN_DAWR0); + host_dawrx0 = mfspr(SPRN_DAWRX0); + host_psscr = mfspr(SPRN_PSSCR); + host_pidr = mfspr(SPRN_PID); + if (cpu_has_feature(CPU_FTR_DAWR1)) { + host_dawr1 = mfspr(SPRN_DAWR1); + host_dawrx1 = mfspr(SPRN_DAWRX1); + } + if (vc->pcr) mtspr(SPRN_PCR, vc->pcr | PCR_MASK); mtspr(SPRN_DPDES, vc->dpdes); From patchwork Thu Feb 25 13:46:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444386 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=ay7Nqv9o; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzd5zGjz9sRN for ; Fri, 26 Feb 2021 00:49:01 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232000AbhBYNs4 (ORCPT ); Thu, 25 Feb 2021 08:48:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231960AbhBYNsz (ORCPT ); Thu, 25 Feb 2021 08:48:55 -0500 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A0E2C061794 for ; Thu, 25 Feb 2021 05:48:40 -0800 (PST) Received: by mail-pl1-x631.google.com with SMTP id s16so3208395plr.9 for ; Thu, 25 Feb 2021 05:48:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BdUM20RYIE/jm7f4nL8DUwkMLUwopPt6d3FqQLRKZzI=; b=ay7Nqv9oX16z0rS/UJufxRfupXdWME1rJIFGqpqtUDVXfEr1oqBT+SoH7ICU/HbagX wS9MLNjSHb/zlJZ+/RHKOGNIh2xPg3EFx6iaeH8SoLt8YGHqXd8AS2rkV1Tmct5HQp7q /VdnrVZgRf0nk9aD2Z5jRElOm02p7/TCsNDb9zoWYUQWgrMKQZ/qHsLCCSOWDfgH5dVU jSFW5bhRxGPCp3w6wrKSapS4Vi2imJWEKgM+7IipkaZvPNTzj3pu8saUS+GXsYlmIgt2 KlAYtKlxSha494el+4YcidKb+pq1nXcWQBLpMsCgdty8Pru6cYKnp77rZZCiE4p4+U13 K07A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BdUM20RYIE/jm7f4nL8DUwkMLUwopPt6d3FqQLRKZzI=; b=Ht1GgZGBU1LXDSc19SHsW2jOXL24dLMv+PYPakfuKI3GCiUqp2InlvmoLs8nArGQvr PSC+bvSE30UTsqKU2c6e/KpBvRqox0Olor+rxs9K1+et4j5wJkaHNB4GADibJpP7ORgL ih8QibeUXGfWm2Zkahb6g0E+A39flGvIE4kTeypOz/I1V3sdoAredMOZ6VeFz0Ui74mS gEv1XR1i5ES5m49jkhneBRDZL6Fb/+IawX2km05NM2rUsZiPp1Ho2xohQzhFmdlzGMXH G45C3Jmtw+rxlRW+D54aqkD105l3UlI7w0m2KkQ0K+ApqZbyCVfbq8cS8SzafDQWaqdN 1Y9g== X-Gm-Message-State: AOAM533eKfzlRevEQT3eOcnJi8y8i33vkN5tWqAFeCCL6I+onRhJjV4l kOCTrDt3KlmhQZh/PdtfdjIYwjB0xPw= X-Google-Smtp-Source: ABdhPJxeiXR7nziq7swxRFZBtB68BlgktFfgZM1CueNq+YO24amjZgqnd2mPjLf4elbkBeSrrgvYBw== X-Received: by 2002:a17:902:aa8f:b029:e3:df7f:fd51 with SMTP id d15-20020a170902aa8fb02900e3df7ffd51mr3162731plr.71.1614260919412; Thu, 25 Feb 2021 05:48:39 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:38 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 28/37] KVM: PPC: Book3S HV P9: Add helpers for OS SPR handling Date: Thu, 25 Feb 2021 23:46:43 +1000 Message-Id: <20210225134652.2127648-29-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org This is a first step to wrapping supervisor and user SPR saving and loading up into helpers, which will then be called independently in bare metal and nested HV cases in order to optimise SPR access. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 131 ++++++++++++++++++++++------------- 1 file changed, 84 insertions(+), 47 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 94989fe2fdfe..ad16331c3370 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3442,6 +3442,84 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc) trace_kvmppc_run_core(vc, 1); } +static void load_spr_state(struct kvm_vcpu *vcpu) +{ + mtspr(SPRN_DSCR, vcpu->arch.dscr); + mtspr(SPRN_IAMR, vcpu->arch.iamr); + mtspr(SPRN_PSPB, vcpu->arch.pspb); + mtspr(SPRN_FSCR, vcpu->arch.fscr); + mtspr(SPRN_TAR, vcpu->arch.tar); + mtspr(SPRN_EBBHR, vcpu->arch.ebbhr); + mtspr(SPRN_EBBRR, vcpu->arch.ebbrr); + mtspr(SPRN_BESCR, vcpu->arch.bescr); + mtspr(SPRN_WORT, vcpu->arch.wort); + mtspr(SPRN_TIDR, vcpu->arch.tid); + /* XXX: DAR, DSISR must be set with MSR[RI] clear (or hstate as appropriate) */ + mtspr(SPRN_AMR, vcpu->arch.amr); + mtspr(SPRN_UAMOR, vcpu->arch.uamor); + + if (!(vcpu->arch.ctrl & 1)) + mtspr(SPRN_CTRLT, mfspr(SPRN_CTRLF) & ~1); +} + +static void store_spr_state(struct kvm_vcpu *vcpu) +{ + vcpu->arch.ctrl = mfspr(SPRN_CTRLF); + + vcpu->arch.iamr = mfspr(SPRN_IAMR); + vcpu->arch.pspb = mfspr(SPRN_PSPB); + vcpu->arch.fscr = mfspr(SPRN_FSCR); + vcpu->arch.tar = mfspr(SPRN_TAR); + vcpu->arch.ebbhr = mfspr(SPRN_EBBHR); + vcpu->arch.ebbrr = mfspr(SPRN_EBBRR); + vcpu->arch.bescr = mfspr(SPRN_BESCR); + vcpu->arch.wort = mfspr(SPRN_WORT); + vcpu->arch.tid = mfspr(SPRN_TIDR); + vcpu->arch.amr = mfspr(SPRN_AMR); + vcpu->arch.uamor = mfspr(SPRN_UAMOR); + vcpu->arch.dscr = mfspr(SPRN_DSCR); +} + +/* + * Privileged (non-hypervisor) host registers to save. + */ +struct p9_host_os_sprs { + unsigned long dscr; + unsigned long tidr; + unsigned long iamr; + unsigned long amr; + unsigned long fscr; +}; + +static void save_p9_host_os_sprs(struct p9_host_os_sprs *host_os_sprs) +{ + host_os_sprs->dscr = mfspr(SPRN_DSCR); + host_os_sprs->tidr = mfspr(SPRN_TIDR); + host_os_sprs->iamr = mfspr(SPRN_IAMR); + host_os_sprs->amr = mfspr(SPRN_AMR); + host_os_sprs->fscr = mfspr(SPRN_FSCR); +} + +/* vcpu guest regs must already be saved */ +static void restore_p9_host_os_sprs(struct kvm_vcpu *vcpu, + struct p9_host_os_sprs *host_os_sprs) +{ + mtspr(SPRN_PSPB, 0); + mtspr(SPRN_WORT, 0); + mtspr(SPRN_UAMOR, 0); + mtspr(SPRN_PSPB, 0); + + mtspr(SPRN_DSCR, host_os_sprs->dscr); + mtspr(SPRN_TIDR, host_os_sprs->tidr); + mtspr(SPRN_IAMR, host_os_sprs->iamr); + + if (host_os_sprs->amr != vcpu->arch.amr) + mtspr(SPRN_AMR, host_os_sprs->amr); + + if (host_os_sprs->fscr != vcpu->arch.fscr) + mtspr(SPRN_FSCR, host_os_sprs->fscr); +} + /* * Virtual-mode guest entry for POWER9 and later when the host and * guest are both using the radix MMU. The LPIDR has already been set. @@ -3450,11 +3528,7 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpcr) { struct kvmppc_vcore *vc = vcpu->arch.vcore; - unsigned long host_dscr = mfspr(SPRN_DSCR); - unsigned long host_tidr = mfspr(SPRN_TIDR); - unsigned long host_iamr = mfspr(SPRN_IAMR); - unsigned long host_amr = mfspr(SPRN_AMR); - unsigned long host_fscr = mfspr(SPRN_FSCR); + struct p9_host_os_sprs host_os_sprs; s64 dec; u64 tb, next_timer; int trap, save_pmu; @@ -3469,6 +3543,8 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, vcpu->arch.ceded = 0; + save_p9_host_os_sprs(&host_os_sprs); + kvmhv_save_host_pmu(); /* saves it to PACA kvm_hstate */ kvmppc_subcore_enter_guest(); @@ -3496,22 +3572,7 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, #endif mtspr(SPRN_VRSAVE, vcpu->arch.vrsave); - mtspr(SPRN_DSCR, vcpu->arch.dscr); - mtspr(SPRN_IAMR, vcpu->arch.iamr); - mtspr(SPRN_PSPB, vcpu->arch.pspb); - mtspr(SPRN_FSCR, vcpu->arch.fscr); - mtspr(SPRN_TAR, vcpu->arch.tar); - mtspr(SPRN_EBBHR, vcpu->arch.ebbhr); - mtspr(SPRN_EBBRR, vcpu->arch.ebbrr); - mtspr(SPRN_BESCR, vcpu->arch.bescr); - mtspr(SPRN_WORT, vcpu->arch.wort); - mtspr(SPRN_TIDR, vcpu->arch.tid); - /* XXX: DAR, DSISR must be set with MSR[RI] clear (or hstate as appropriate) */ - mtspr(SPRN_AMR, vcpu->arch.amr); - mtspr(SPRN_UAMOR, vcpu->arch.uamor); - - if (!(vcpu->arch.ctrl & 1)) - mtspr(SPRN_CTRLT, mfspr(SPRN_CTRLF) & ~1); + load_spr_state(vcpu); /* * XXX: must always deal with irq_work_raise via NMI vs setting DEC. @@ -3605,34 +3666,10 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, vcpu->arch.dec_expires = dec + tb; vcpu->cpu = -1; vcpu->arch.thread_cpu = -1; - vcpu->arch.ctrl = mfspr(SPRN_CTRLF); - - vcpu->arch.iamr = mfspr(SPRN_IAMR); - vcpu->arch.pspb = mfspr(SPRN_PSPB); - vcpu->arch.fscr = mfspr(SPRN_FSCR); - vcpu->arch.tar = mfspr(SPRN_TAR); - vcpu->arch.ebbhr = mfspr(SPRN_EBBHR); - vcpu->arch.ebbrr = mfspr(SPRN_EBBRR); - vcpu->arch.bescr = mfspr(SPRN_BESCR); - vcpu->arch.wort = mfspr(SPRN_WORT); - vcpu->arch.tid = mfspr(SPRN_TIDR); - vcpu->arch.amr = mfspr(SPRN_AMR); - vcpu->arch.uamor = mfspr(SPRN_UAMOR); - vcpu->arch.dscr = mfspr(SPRN_DSCR); - - mtspr(SPRN_PSPB, 0); - mtspr(SPRN_WORT, 0); - mtspr(SPRN_UAMOR, 0); - mtspr(SPRN_DSCR, host_dscr); - mtspr(SPRN_TIDR, host_tidr); - mtspr(SPRN_IAMR, host_iamr); - mtspr(SPRN_PSPB, 0); - if (host_amr != vcpu->arch.amr) - mtspr(SPRN_AMR, host_amr); + restore_p9_host_os_sprs(vcpu, &host_os_sprs); - if (host_fscr != vcpu->arch.fscr) - mtspr(SPRN_FSCR, host_fscr); + store_spr_state(vcpu); msr_check_and_set(MSR_FP | MSR_VEC | MSR_VSX); store_fp_state(&vcpu->arch.fp); From patchwork Thu Feb 25 13:46:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444387 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=EADf74ji; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzf4XcPz9sVy for ; Fri, 26 Feb 2021 00:49:02 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232008AbhBYNs7 (ORCPT ); Thu, 25 Feb 2021 08:48:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231960AbhBYNs6 (ORCPT ); Thu, 25 Feb 2021 08:48:58 -0500 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75629C061574 for ; Thu, 25 Feb 2021 05:48:43 -0800 (PST) Received: by mail-pf1-x42d.google.com with SMTP id q204so2591131pfq.10 for ; Thu, 25 Feb 2021 05:48:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nFaLjtbHvhG9RPsmAn2L9v8PlI/yIaIQI9IuG70b8yw=; b=EADf74jia1v+Dl6Sggk63dBVE/91g56OzdKAcCr7OH87ZEbwTUUsyIYSYtUebnuqut RYYC17/N81cipa9HgiuVssN//6TD3rDIDGN1irGlLuJ1m721pGVwnDW5wwwg1dFbp2Ol eWRpz6/s54aUApKSIGxi/DdNTFrEhtFzoMVWVUU4kDY4o34RAc0JhuP9BVP7TJOA88pW NDWGl5cSGpo8WPjtWxVGgM6/eclmiSBMj9hnAnoDD54m77l5KhxFk2yqCA2zNouz7JGW olbzgujULdHZzqDNwVsaXMY7UbfDDxokHWzIWz+0tKBEQGpIzzfN5HqLgUulHtuzhLKF wCdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nFaLjtbHvhG9RPsmAn2L9v8PlI/yIaIQI9IuG70b8yw=; b=Z3sAHvNoPs9dUHQ+uXA0wqoG86yE4Olktv9V3JF6/bYOAs5Y9ui4/ArvQQmuNb1REg N8IEKRTLOC5q+YkmVjXEbGbUtEVnQEe/JfegElbytgW/uLVzT4NVc0r/vAovzt4D7qVM P5Q4URUXZRVSrpcxAyMyVFx7TNsCaPGi3c3bu68c2ncJ3DNquduWhUhf5wtYsK/1H5cl hiwFFgg5Rdn6ltRi7MYd3maDLG4Fci6ZfrT5TNcBvhtO/tTAIwQhv/l5Nrun0QZmcBrz HzRpwLoHfH7AD0DBFR7W7Vr7qsysuVkax4Ln76/LVgXQgLb9kBQJ3cqX5rgSdPvgOT3L kqVg== X-Gm-Message-State: AOAM530M2pewVmElCRy55VCqTMmYiwHxup6IRBUeh4IbGg+m1LN8NUBv qoLwPnyAL+0mODL9zsqNLhGtPJpHdRA= X-Google-Smtp-Source: ABdhPJwUbThXbDyvaI7aS0Fet3uIuMuLbgnDOCcKp5h4bhbUy/L5JdKF4Iq5km8L76+G2R6KN4SepA== X-Received: by 2002:a63:1c13:: with SMTP id c19mr2929117pgc.359.1614260922556; Thu, 25 Feb 2021 05:48:42 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:42 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 29/37] KVM: PPC: Book3S HV P9: Switch to guest MMU context as late as possible Date: Thu, 25 Feb 2021 23:46:44 +1000 Message-Id: <20210225134652.2127648-30-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Move WARN_ON traps early so they are less likely to get tangled on CPU switching to guest. Move MMU context switch as late as reasonably possible to minimise code running with guest context switched in. This becomes more important when this code may run in real-mode, with later changes. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv_interrupt.c | 40 +++++++++++++------------- arch/powerpc/kvm/book3s_hv_nested.c | 1 + 2 files changed, 21 insertions(+), 20 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_interrupt.c b/arch/powerpc/kvm/book3s_hv_interrupt.c index dd0a78a69f49..b93d861d8538 100644 --- a/arch/powerpc/kvm/book3s_hv_interrupt.c +++ b/arch/powerpc/kvm/book3s_hv_interrupt.c @@ -143,8 +143,13 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc if (hdec < 0) return BOOK3S_INTERRUPT_HV_DECREMENTER; + WARN_ON_ONCE(vcpu->arch.shregs.msr & MSR_HV); + WARN_ON_ONCE(!(vcpu->arch.shregs.msr & MSR_ME)); + start_timing(vcpu, &vcpu->arch.rm_entry); + vcpu->arch.ceded = 0; + if (vc->tb_offset) { u64 new_tb = tb + vc->tb_offset; mtspr(SPRN_TBU40, new_tb); @@ -193,26 +198,6 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc mtspr(SPRN_HFSCR, vcpu->arch.hfscr); - mtspr(SPRN_SPRG0, vcpu->arch.shregs.sprg0); - mtspr(SPRN_SPRG1, vcpu->arch.shregs.sprg1); - mtspr(SPRN_SPRG2, vcpu->arch.shregs.sprg2); - mtspr(SPRN_SPRG3, vcpu->arch.shregs.sprg3); - - mtspr(SPRN_AMOR, ~0UL); - - switch_mmu_to_guest_radix(kvm, vcpu, lpcr); - - /* - * P9 suppresses the HDEC exception when LPCR[HDICE] = 0, - * so set guest LPCR (with HDICE) before writing HDEC. - */ - mtspr(SPRN_HDEC, hdec); - - vcpu->arch.ceded = 0; - - WARN_ON_ONCE(vcpu->arch.shregs.msr & MSR_HV); - WARN_ON_ONCE(!(vcpu->arch.shregs.msr & MSR_ME)); - mtspr(SPRN_HSRR0, vcpu->arch.regs.nip); mtspr(SPRN_HSRR1, (vcpu->arch.shregs.msr & ~MSR_HV) | MSR_ME); @@ -231,6 +216,21 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc */ mtspr(SPRN_HDSISR, HDSISR_CANARY); + mtspr(SPRN_SPRG0, vcpu->arch.shregs.sprg0); + mtspr(SPRN_SPRG1, vcpu->arch.shregs.sprg1); + mtspr(SPRN_SPRG2, vcpu->arch.shregs.sprg2); + mtspr(SPRN_SPRG3, vcpu->arch.shregs.sprg3); + + mtspr(SPRN_AMOR, ~0UL); + + switch_mmu_to_guest_radix(kvm, vcpu, lpcr); + + /* + * P9 suppresses the HDEC exception when LPCR[HDICE] = 0, + * so set guest LPCR (with HDICE) before writing HDEC. + */ + mtspr(SPRN_HDEC, hdec); + __mtmsrd(0, 1); /* clear RI */ mtspr(SPRN_DAR, vcpu->arch.shregs.dar); diff --git a/arch/powerpc/kvm/book3s_hv_nested.c b/arch/powerpc/kvm/book3s_hv_nested.c index 0cd0e7aad588..cdf3ee2145ab 100644 --- a/arch/powerpc/kvm/book3s_hv_nested.c +++ b/arch/powerpc/kvm/book3s_hv_nested.c @@ -323,6 +323,7 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) vcpu->arch.shregs.msr = vcpu->arch.regs.msr; mask = LPCR_DPFD | LPCR_ILE | LPCR_TC | LPCR_AIL | LPCR_LD | LPCR_LPES | LPCR_MER; + /* XXX: set lpcr in sanitise hv regs? Why is it plumbed through? */ lpcr = (vc->lpcr & ~mask) | (l2_hv.lpcr & mask); sanitise_hv_regs(vcpu, &l2_hv); restore_hv_regs(vcpu, &l2_hv); From patchwork Thu Feb 25 13:46:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444388 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=asJulepH; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmYzh11nDz9sWK for ; Fri, 26 Feb 2021 00:49:04 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232033AbhBYNtD (ORCPT ); Thu, 25 Feb 2021 08:49:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231960AbhBYNtB (ORCPT ); Thu, 25 Feb 2021 08:49:01 -0500 Received: from mail-pg1-x52a.google.com (mail-pg1-x52a.google.com [IPv6:2607:f8b0:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5587C06174A for ; Thu, 25 Feb 2021 05:48:46 -0800 (PST) Received: by mail-pg1-x52a.google.com with SMTP id t11so3809280pgu.8 for ; Thu, 25 Feb 2021 05:48:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vNkKwEjppVpjoVv8a8mKAjNqn2HlCTwStuqi2y+CGmg=; b=asJulepHZSVOSdggo9uHfye1fTDK1zueRgrbABTsssDO37MSKeJO7ZWp8elGRfeZOs sG+HQx8JyJBNTAwP22o2e4121T4v1KeNSoZ0O/3ujFlFO/qdOsbgtst4rsxKpoqinofe hkTV8dXVKeHpK1Sanf8tvbbffkKKwCo7BKqk2bK0r9N/06gQBEsEROJDIlL2n+EL7Y65 cQfVPTuW8Y4hn1SP01y9N1+GHEWM3bTPhe3RPLGm0ZehqJZN6RZmZq26+HEZVhb7QInb SCh4e7eEzCoPNYo7PhhRgCAaq1dhfVOuM7Q2HdK4qPTdPDchZ6Gn5w9VGvJk5bjVhQ/6 UwOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vNkKwEjppVpjoVv8a8mKAjNqn2HlCTwStuqi2y+CGmg=; b=KwB0DECVslcA9QFTSZAe7BGv/F9ZB4N++B1nv1jGbD/2xo2lob5wgGqJLZciH84Yre m6tDWYsQHqUeP9MNrrxSOBcnZ8+hJd9N7Co45arUmg7NwaYMdcIkEhkV+66yurXapePC yd70RcjTa2x4a99igfiPp89lpHB+uUcYRhydXpwhrF8NRCNsANdtkPKgI6huXoKzC7H8 Qgy34u1pwnADquiGBP7S97w6yPRC9V+P3Y2RtxlkltTtG0YW6DXJabiPxY+ayUm7nsj4 r6OTJhf03zzSdyD7MGnSU+yzVy1xGnpOPydMeqqnv4yt5/ORB/II2RT/v0R7BLh0yUS9 W8hg== X-Gm-Message-State: AOAM530lS61HGrVYvpc+8Jv3yWkBTHRFm6zVyUe2u2Ov2edO8nRcstkb W3NgfKH5rcDqXuwyXUSgQ5r57uxi/+Q= X-Google-Smtp-Source: ABdhPJxKrS0xPtgYriVmo2v7WcY6tXYRW+KROmQS6gb0W4TtCktzyAezdS7GFA3iXm/Nz90OiWrGvQ== X-Received: by 2002:a05:6a00:1e:b029:1ed:b82c:bb64 with SMTP id h30-20020a056a00001eb02901edb82cbb64mr3297740pfk.78.1614260925647; Thu, 25 Feb 2021 05:48:45 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:45 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 30/37] KVM: PPC: Book3S HV: Implement radix prefetch workaround by disabling MMU Date: Thu, 25 Feb 2021 23:46:45 +1000 Message-Id: <20210225134652.2127648-31-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Rather than partition the guest PID space and catch and flush a rogue guest, instead work around this issue by ensuring the MMU is always disabled in HV mode while the guest MMU context is switched in. This may be a bit less efficient, but it is a lot less complicated and allows the P9 path to trivally implement the workaround too. Newer CPUs are not subject to this issue. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/mmu_context.h | 6 ---- arch/powerpc/kvm/book3s_hv.c | 10 ++++-- arch/powerpc/kvm/book3s_hv_interrupt.c | 14 ++++++-- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 34 ------------------ arch/powerpc/mm/book3s64/radix_pgtable.c | 27 +++++--------- arch/powerpc/mm/book3s64/radix_tlb.c | 46 ------------------------ arch/powerpc/mm/mmu_context.c | 4 +-- 7 files changed, 28 insertions(+), 113 deletions(-) diff --git a/arch/powerpc/include/asm/mmu_context.h b/arch/powerpc/include/asm/mmu_context.h index 652ce85f9410..bb5c7e5e142e 100644 --- a/arch/powerpc/include/asm/mmu_context.h +++ b/arch/powerpc/include/asm/mmu_context.h @@ -122,12 +122,6 @@ static inline bool need_extra_context(struct mm_struct *mm, unsigned long ea) } #endif -#if defined(CONFIG_KVM_BOOK3S_HV_POSSIBLE) && defined(CONFIG_PPC_RADIX_MMU) -extern void radix_kvm_prefetch_workaround(struct mm_struct *mm); -#else -static inline void radix_kvm_prefetch_workaround(struct mm_struct *mm) { } -#endif - extern void switch_cop(struct mm_struct *next); extern int use_cop(unsigned long acop, struct mm_struct *mm); extern void drop_cop(unsigned long acop, struct mm_struct *mm); diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index ad16331c3370..c3064075f1d7 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -806,6 +806,10 @@ static int kvmppc_h_set_mode(struct kvm_vcpu *vcpu, unsigned long mflags, /* KVM does not support mflags=2 (AIL=2) */ if (mflags != 0 && mflags != 3) return H_UNSUPPORTED_FLAG_START; + /* Prefetch bug */ + if (cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG) && + kvmhv_vcpu_is_radix(vcpu) && mflags == 3) + return H_UNSUPPORTED_FLAG_START; return H_TOO_HARD; default: return H_TOO_HARD; @@ -4286,8 +4290,7 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu) * The TLB prefetch bug fixup is only in the kvmppc_run_vcpu * path, which also handles hash and dependent threads mode. */ - if (kvm->arch.threads_indep && kvm_is_radix(kvm) && - !cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) + if (kvm->arch.threads_indep && kvm_is_radix(kvm)) r = kvmhv_run_single_vcpu(vcpu, ~(u64)0, vcpu->arch.vcore->lpcr); else @@ -4914,6 +4917,9 @@ static int kvmppc_core_init_vm_hv(struct kvm *kvm) if (!indep_threads_mode && !cpu_has_feature(CPU_FTR_HVMODE)) { pr_warn("KVM: Ignoring indep_threads_mode=N in nested hypervisor\n"); kvm->arch.threads_indep = true; + } else if (!indep_threads_mode && cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) { + pr_warn("KVM: Ignoring indep_threads_mode=N on pre-DD2.2 POWER9\n"); + kvm->arch.threads_indep = true; } else { kvm->arch.threads_indep = indep_threads_mode; } diff --git a/arch/powerpc/kvm/book3s_hv_interrupt.c b/arch/powerpc/kvm/book3s_hv_interrupt.c index b93d861d8538..9784da3f8565 100644 --- a/arch/powerpc/kvm/book3s_hv_interrupt.c +++ b/arch/powerpc/kvm/book3s_hv_interrupt.c @@ -223,6 +223,9 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc mtspr(SPRN_AMOR, ~0UL); + if (cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) + __mtmsrd(msr & ~(MSR_IR|MSR_DR|MSR_RI), 0); + switch_mmu_to_guest_radix(kvm, vcpu, lpcr); /* @@ -231,7 +234,8 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc */ mtspr(SPRN_HDEC, hdec); - __mtmsrd(0, 1); /* clear RI */ + if (!cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) + __mtmsrd(0, 1); /* clear RI */ mtspr(SPRN_DAR, vcpu->arch.shregs.dar); mtspr(SPRN_DSISR, vcpu->arch.shregs.dsisr); @@ -338,8 +342,6 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc radix_clear_slb(); - __mtmsrd(msr, 0); - accumulate_time(vcpu, &vcpu->arch.rm_exit); /* Advance host PURR/SPURR by the amount used by guest */ @@ -406,6 +408,12 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc switch_mmu_to_host_radix(kvm, host_pidr); + /* + * If we are in real mode, don't switch MMU on until the MMU is + * switched to host, to avoid the P9 radix prefetch bug. + */ + __mtmsrd(msr, 0); + end_timing(vcpu); return trap; diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index 6118e8a97ddd..61f71a7df238 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -1710,40 +1710,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) eieio tlbsync ptesync - -BEGIN_FTR_SECTION - /* Radix: Handle the case where the guest used an illegal PID */ - LOAD_REG_ADDR(r4, mmu_base_pid) - lwz r3, VCPU_GUEST_PID(r9) - lwz r5, 0(r4) - cmpw cr0,r3,r5 - blt 2f - - /* - * Illegal PID, the HW might have prefetched and cached in the TLB - * some translations for the LPID 0 / guest PID combination which - * Linux doesn't know about, so we need to flush that PID out of - * the TLB. First we need to set LPIDR to 0 so tlbiel applies to - * the right context. - */ - li r0,0 - mtspr SPRN_LPID,r0 - isync - - /* Then do a congruence class local flush */ - ld r6,VCPU_KVM(r9) - lwz r0,KVM_TLB_SETS(r6) - mtctr r0 - li r7,0x400 /* IS field = 0b01 */ - ptesync - sldi r0,r3,32 /* RS has PID */ -1: PPC_TLBIEL(7,0,2,1,1) /* RIC=2, PRS=1, R=1 */ - addi r7,r7,0x1000 - bdnz 1b - ptesync -END_FTR_SECTION_IFSET(CPU_FTR_P9_RADIX_PREFETCH_BUG) - -2: #endif /* CONFIG_PPC_RADIX_MMU */ /* diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index 98f0b243c1ab..1ea95891a79e 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -357,30 +357,19 @@ static void __init radix_init_pgtable(void) } /* Find out how many PID bits are supported */ - if (!cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) { - if (!mmu_pid_bits) - mmu_pid_bits = 20; - mmu_base_pid = 1; - } else if (cpu_has_feature(CPU_FTR_HVMODE)) { - if (!mmu_pid_bits) - mmu_pid_bits = 20; -#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE + if (!cpu_has_feature(CPU_FTR_HVMODE) && + cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) { /* - * When KVM is possible, we only use the top half of the - * PID space to avoid collisions between host and guest PIDs - * which can cause problems due to prefetch when exiting the - * guest with AIL=3 + * Older versions of KVM on these machines perfer if the + * guest only uses the low 19 PID bits. */ - mmu_base_pid = 1 << (mmu_pid_bits - 1); -#else - mmu_base_pid = 1; -#endif - } else { - /* The guest uses the bottom half of the PID space */ if (!mmu_pid_bits) mmu_pid_bits = 19; - mmu_base_pid = 1; + } else { + if (!mmu_pid_bits) + mmu_pid_bits = 20; } + mmu_base_pid = 1; /* * Allocate Partition table and process table for the diff --git a/arch/powerpc/mm/book3s64/radix_tlb.c b/arch/powerpc/mm/book3s64/radix_tlb.c index 409e61210789..312236a6b085 100644 --- a/arch/powerpc/mm/book3s64/radix_tlb.c +++ b/arch/powerpc/mm/book3s64/radix_tlb.c @@ -1336,49 +1336,3 @@ void radix__flush_tlb_all(void) : : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(0) : "memory"); asm volatile("eieio; tlbsync; ptesync": : :"memory"); } - -#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE -extern void radix_kvm_prefetch_workaround(struct mm_struct *mm) -{ - unsigned long pid = mm->context.id; - - if (unlikely(pid == MMU_NO_CONTEXT)) - return; - - if (!cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) - return; - - /* - * If this context hasn't run on that CPU before and KVM is - * around, there's a slim chance that the guest on another - * CPU just brought in obsolete translation into the TLB of - * this CPU due to a bad prefetch using the guest PID on - * the way into the hypervisor. - * - * We work around this here. If KVM is possible, we check if - * any sibling thread is in KVM. If it is, the window may exist - * and thus we flush that PID from the core. - * - * A potential future improvement would be to mark which PIDs - * have never been used on the system and avoid it if the PID - * is new and the process has no other cpumask bit set. - */ - if (cpu_has_feature(CPU_FTR_HVMODE) && radix_enabled()) { - int cpu = smp_processor_id(); - int sib = cpu_first_thread_sibling(cpu); - bool flush = false; - - for (; sib <= cpu_last_thread_sibling(cpu) && !flush; sib++) { - if (sib == cpu) - continue; - if (!cpu_possible(sib)) - continue; - if (paca_ptrs[sib]->kvm_hstate.kvm_vcpu) - flush = true; - } - if (flush) - _tlbiel_pid(pid, RIC_FLUSH_ALL); - } -} -EXPORT_SYMBOL_GPL(radix_kvm_prefetch_workaround); -#endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */ diff --git a/arch/powerpc/mm/mmu_context.c b/arch/powerpc/mm/mmu_context.c index 18f20da0d348..7479d39976c9 100644 --- a/arch/powerpc/mm/mmu_context.c +++ b/arch/powerpc/mm/mmu_context.c @@ -81,9 +81,7 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, if (cpu_has_feature(CPU_FTR_ALTIVEC)) asm volatile ("dssall"); - if (new_on_cpu) - radix_kvm_prefetch_workaround(next); - else + if (!new_on_cpu) membarrier_arch_switch_mm(prev, next, tsk); /* From patchwork Thu Feb 25 13:46:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444398 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=aF54FXJ8; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ0K5243z9sWQ for ; Fri, 26 Feb 2021 00:49:37 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232394AbhBYNtf (ORCPT ); Thu, 25 Feb 2021 08:49:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44944 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232237AbhBYNt3 (ORCPT ); Thu, 25 Feb 2021 08:49:29 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A19C9C061756 for ; Thu, 25 Feb 2021 05:48:49 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id t5so5069607pjd.0 for ; Thu, 25 Feb 2021 05:48:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jZ1wfYQFjuRbdyvcRSS6ix6mGNgmk39buWP9mBEV9/M=; b=aF54FXJ8C41bl5cKZY58N0m8jnWaI7Nj6Q6FgN1HlMLmfL1QeZcH3TNLg0EAx3gmyN +qAx3jSBopJo8GAetxELd0lNX7C6Q75l/0hMJF12CSNbwZPQRv9b9wcI52NctexkWTKE 98eaYPmqE6WGr1C5Fh9kIyIx0lEVlftuUmUjrTq7lMGAnpEoi/IJATt3OemEW+mn0X5J GLAo4Ut7CE6bZHW/gtlU2KleJ7NcbhFJ41PAAcH0pB9YxcHPI9hp+9KhYzHbtB5qwkZo B3xeJP4DDGLZVP3TNw99IAsIAFBtz1QZ1/YkvO77w1JqEE1IeEU5Ej3LWIFfMGea41TD vpSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jZ1wfYQFjuRbdyvcRSS6ix6mGNgmk39buWP9mBEV9/M=; b=Xg0mle44ZdtRh7p5BSQlkGoErgCXF2fZ7oQWg7uCZgXFPQqGdCnVBMGRuh71YXGs2h vvMrGKfqoG6jS4xEH22ttazCPP0Mi09kTBkowgfjdMIK1K6W8sektFm0KSsfLwognHPc h7+LKbTF+OMKAKey63IUEucN4gVj/ARpPWF+zllC2Zv+GI5DwR4vVkespyoFtzYQJrBn dEQsvoNvursxNSmGMXUd/xRYqD9txr4mHwSeXIXWeMSJ2ahHKzkM5MeTGis7yAhyYHgl bA4efrCXpaTesING36MYPM/0jJrTmuOMQBhDr3ikDERhgXi+HRP3qHD+baWu1CMQlbge +BqA== X-Gm-Message-State: AOAM532wKQ1ATdhS8/oxZfXonYcHX+0S7jSZQ8Z7UlGG8l3pAFGGuOTQ VkM3wFbY1jiOOsAS0TC0zDQiXTZBFsw= X-Google-Smtp-Source: ABdhPJwMBUWfvwykU+tx48kV3tQeLk29DGRGlwSw7c67hvySaKDQiqoeNcwT0v45yjz2hih1yc/CgA== X-Received: by 2002:a17:902:f702:b029:e3:5e25:85bb with SMTP id h2-20020a170902f702b02900e35e2585bbmr3190850plo.56.1614260928742; Thu, 25 Feb 2021 05:48:48 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:48 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 31/37] KVM: PPC: Book3S HV: Remove support for dependent threads mode on P9 Date: Thu, 25 Feb 2021 23:46:46 +1000 Message-Id: <20210225134652.2127648-32-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Radix guest support will be removed from the P7/8 path, so disallow dependent threads mode on P9. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/kvm_host.h | 1 - arch/powerpc/kvm/book3s_hv.c | 27 +++++---------------------- 2 files changed, 5 insertions(+), 23 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h index 05fb00d37609..dd017dfa4e65 100644 --- a/arch/powerpc/include/asm/kvm_host.h +++ b/arch/powerpc/include/asm/kvm_host.h @@ -304,7 +304,6 @@ struct kvm_arch { u8 fwnmi_enabled; u8 secure_guest; u8 svm_enabled; - bool threads_indep; bool nested_enable; bool dawr1_enabled; pgd_t *pgtable; diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index c3064075f1d7..1f27187ff1e7 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -103,13 +103,9 @@ static int target_smt_mode; module_param(target_smt_mode, int, 0644); MODULE_PARM_DESC(target_smt_mode, "Target threads per core (0 = max)"); -static bool indep_threads_mode = true; -module_param(indep_threads_mode, bool, S_IRUGO | S_IWUSR); -MODULE_PARM_DESC(indep_threads_mode, "Independent-threads mode (only on POWER9)"); - static bool one_vm_per_core; module_param(one_vm_per_core, bool, S_IRUGO | S_IWUSR); -MODULE_PARM_DESC(one_vm_per_core, "Only run vCPUs from the same VM on a core (requires indep_threads_mode=N)"); +MODULE_PARM_DESC(one_vm_per_core, "Only run vCPUs from the same VM on a core (requires POWER8 or older)"); #ifdef CONFIG_KVM_XICS static const struct kernel_param_ops module_param_ops = { @@ -2201,7 +2197,7 @@ static int kvmppc_set_one_reg_hv(struct kvm_vcpu *vcpu, u64 id, */ static int threads_per_vcore(struct kvm *kvm) { - if (kvm->arch.threads_indep) + if (cpu_has_feature(CPU_FTR_ARCH_300)) return 1; return threads_per_subcore; } @@ -4290,7 +4286,7 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu) * The TLB prefetch bug fixup is only in the kvmppc_run_vcpu * path, which also handles hash and dependent threads mode. */ - if (kvm->arch.threads_indep && kvm_is_radix(kvm)) + if (kvm_is_radix(kvm)) r = kvmhv_run_single_vcpu(vcpu, ~(u64)0, vcpu->arch.vcore->lpcr); else @@ -4910,21 +4906,8 @@ static int kvmppc_core_init_vm_hv(struct kvm *kvm) /* * Track that we now have a HV mode VM active. This blocks secondary * CPU threads from coming online. - * On POWER9, we only need to do this if the "indep_threads_mode" - * module parameter has been set to N. */ - if (cpu_has_feature(CPU_FTR_ARCH_300)) { - if (!indep_threads_mode && !cpu_has_feature(CPU_FTR_HVMODE)) { - pr_warn("KVM: Ignoring indep_threads_mode=N in nested hypervisor\n"); - kvm->arch.threads_indep = true; - } else if (!indep_threads_mode && cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) { - pr_warn("KVM: Ignoring indep_threads_mode=N on pre-DD2.2 POWER9\n"); - kvm->arch.threads_indep = true; - } else { - kvm->arch.threads_indep = indep_threads_mode; - } - } - if (!kvm->arch.threads_indep) + if (!cpu_has_feature(CPU_FTR_ARCH_300)) kvm_hv_vm_activated(); /* @@ -4965,7 +4948,7 @@ static void kvmppc_core_destroy_vm_hv(struct kvm *kvm) { debugfs_remove_recursive(kvm->arch.debugfs_dir); - if (!kvm->arch.threads_indep) + if (!cpu_has_feature(CPU_FTR_ARCH_300)) kvm_hv_vm_deactivated(); kvmppc_free_vcores(kvm); From patchwork Thu Feb 25 13:46:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444400 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=RtWUGw48; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ0m6LPlz9sRN for ; Fri, 26 Feb 2021 00:50:00 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232495AbhBYNth (ORCPT ); Thu, 25 Feb 2021 08:49:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44956 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232447AbhBYNtd (ORCPT ); Thu, 25 Feb 2021 08:49:33 -0500 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DFB2C061788 for ; Thu, 25 Feb 2021 05:48:53 -0800 (PST) Received: by mail-pl1-x631.google.com with SMTP id d11so3212249plo.8 for ; Thu, 25 Feb 2021 05:48:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kUC0xUCe92YtsXcbm6F3+IuMhubuwTkAZ1cDtpWPJRs=; b=RtWUGw48QDe1rF3+nvkAD8UlBns0Bq1LCpMQUUv/A5UL1V3GjTuCDBRyxe9IfhkBzI +NLOOphilQxLEk+0rCyHVO9HKG4vzCPXhXl7bIcOKZ6Q3jojPrCze27wbGxc2YlPVKZc wmByXuCdCRlFlgZ4y6g2aS7uwQ127P36Aq/d94iB0hXzI7DlvCTeDoh5wcMxQMSV9acN 0lunDWEKfXR2RYVGmzighm6DqG1LyLnLCxfDUVTh9yvhwBoCKc6lK9OYX8cusWdlBgCL 1F114BZOQaToWS3QSAyWZ3CDU6aZaOzgJzbl23l9BSg3letHgVX+3zV16GYk93bWj6Hn JcTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kUC0xUCe92YtsXcbm6F3+IuMhubuwTkAZ1cDtpWPJRs=; b=HwbL1rDX1zcNlKTpi41gjZsavia8lTMj6botifzz2mt0KFoxYhcmlKYRQp9Q/zAvkB gRGlnwpLC4qGE5lPGF8F0Digzpwe4nve20n5/aszTtCnsqvFSTRxdhBgMei2s6F7vD8d NYPvVYLaZa1alWYGLn4MKhonNkK6jbNt5Bcq77v3JfrDjknuFVg6u9P2VUocoHKe0L44 hvSoB1RTKvAdtQu8LcMsmoICjEaFO4YCk9npZJN8r5qm7ryimcTTF6d2T4hQcUH5K+zB LF7oTwsWKhHxgSuz3eGCiY4zhbZMQqwxr7+WHpqJIO4VhzYIM5K81r0fk87hOCur7Taw sMdQ== X-Gm-Message-State: AOAM530wjVWEUo5M6fetUPQuaSdbXed+ED3L4xk3WN2/r58cG9WlryIE ZV8VnWze8kic9zEhGnhe5rLAMNp3PBM= X-Google-Smtp-Source: ABdhPJxRRtoKmswDfRI27mwf2fhRtlR3vvkjfE6V1GhkJAiKOLjD9myzUnA6gzhm4SN9/9uoyA6YwA== X-Received: by 2002:a17:90b:1bc6:: with SMTP id oa6mr3520457pjb.86.1614260931994; Thu, 25 Feb 2021 05:48:51 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:51 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 32/37] KVM: PPC: Book3S HV: Remove radix guest support from P7/8 path Date: Thu, 25 Feb 2021 23:46:47 +1000 Message-Id: <20210225134652.2127648-33-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org The P9 path will run all supported radix guest combinations now, so remove support from the old path. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 65 ++----------------------- 1 file changed, 3 insertions(+), 62 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index 61f71a7df238..a8ce68eed13e 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -899,11 +899,6 @@ ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_300) cmpdi r3, 512 /* 1 microsecond */ blt hdec_soon - ld r6, VCPU_KVM(r4) - lbz r0, KVM_RADIX(r6) - cmpwi r0, 0 - bne 9f - /* For hash guest, clear out and reload the SLB */ BEGIN_MMU_FTR_SECTION /* Radix host won't have populated the SLB, so no need to clear */ @@ -1389,11 +1384,7 @@ guest_exit_cont: /* r9 = vcpu, r12 = trap, r13 = paca */ patch_site 1b patch__call_kvm_flush_link_stack /* For hash guest, read the guest SLB and save it away */ - ld r5, VCPU_KVM(r9) - lbz r0, KVM_RADIX(r5) li r5, 0 - cmpwi r0, 0 - bne 0f /* for radix, save 0 entries */ lwz r0,VCPU_SLB_NR(r9) /* number of entries in SLB */ mtctr r0 li r6,0 @@ -1432,23 +1423,6 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_RADIX) slbmte r6,r5 1: addi r8,r8,16 .endr - b guest_bypass - -0: /* - * Malicious or buggy radix guests may have inserted SLB entries - * (only 0..3 because radix always runs with UPRT=1), so these must - * be cleared here to avoid side-channels. slbmte is used rather - * than slbia, as it won't clear cached translations. - */ - li r0,0 - stw r0,VCPU_SLB_MAX(r9) - slbmte r0,r0 - li r4,1 - slbmte r0,r4 - li r4,2 - slbmte r0,r4 - li r4,3 - slbmte r0,r4 guest_bypass: stw r12, STACK_SLOT_TRAP(r1) @@ -1694,24 +1668,6 @@ BEGIN_FTR_SECTION mtspr SPRN_PID, r7 END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) -#ifdef CONFIG_PPC_RADIX_MMU - /* - * Are we running hash or radix ? - */ - ld r5, VCPU_KVM(r9) - lbz r0, KVM_RADIX(r5) - cmpwi cr2, r0, 0 - beq cr2, 2f - - /* - * Radix: do eieio; tlbsync; ptesync sequence in case we - * interrupted the guest between a tlbie and a ptesync. - */ - eieio - tlbsync - ptesync -#endif /* CONFIG_PPC_RADIX_MMU */ - /* * cp_abort is required if the processor supports local copy-paste * to clear the copy buffer that was under control of the guest. @@ -1970,8 +1926,6 @@ kvmppc_tm_emul: * reflect the HDSI to the guest as a DSI. */ kvmppc_hdsi: - ld r3, VCPU_KVM(r9) - lbz r0, KVM_RADIX(r3) mfspr r4, SPRN_HDAR mfspr r6, SPRN_HDSISR BEGIN_FTR_SECTION @@ -1979,8 +1933,6 @@ BEGIN_FTR_SECTION cmpdi r6, 0x7fff beq 6f END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) - cmpwi r0, 0 - bne .Lradix_hdsi /* on radix, just save DAR/DSISR/ASDR */ /* HPTE not found fault or protection fault? */ andis. r0, r6, (DSISR_NOHPTE | DSISR_PROTFAULT)@h beq 1f /* if not, send it to the guest */ @@ -2057,23 +2009,11 @@ fast_interrupt_c_return: stb r0, HSTATE_IN_GUEST(r13) b guest_exit_cont -.Lradix_hdsi: - std r4, VCPU_FAULT_DAR(r9) - stw r6, VCPU_FAULT_DSISR(r9) -.Lradix_hisi: - mfspr r5, SPRN_ASDR - std r5, VCPU_FAULT_GPA(r9) - b guest_exit_cont - /* * Similarly for an HISI, reflect it to the guest as an ISI unless * it is an HPTE not found fault for a page that we have paged out. */ kvmppc_hisi: - ld r3, VCPU_KVM(r9) - lbz r0, KVM_RADIX(r3) - cmpwi r0, 0 - bne .Lradix_hisi /* for radix, just save ASDR */ andis. r0, r11, SRR1_ISI_NOPT@h beq 1f andi. r0, r11, MSR_IR /* instruction relocation enabled? */ @@ -3217,15 +3157,16 @@ BEGIN_FTR_SECTION mtspr SPRN_DAWRX1, r0 END_FTR_SECTION_IFSET(CPU_FTR_DAWR1) - /* Clear hash and radix guest SLB. */ + /* Clear guest SLB. */ slbmte r0, r0 PPC_SLBIA(6) + ptesync BEGIN_MMU_FTR_SECTION b 4f END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_RADIX) - ptesync + /* load host SLB entries */ ld r8, PACA_SLBSHADOWPTR(r13) .rept SLB_NUM_BOLTED li r3, SLBSHADOW_SAVEAREA From patchwork Thu Feb 25 13:46:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444401 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=bIvzWMw7; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ0n2Y9vz9sVv for ; Fri, 26 Feb 2021 00:50:01 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232447AbhBYNtr (ORCPT ); Thu, 25 Feb 2021 08:49:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232491AbhBYNtg (ORCPT ); Thu, 25 Feb 2021 08:49:36 -0500 Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FEE5C06178B for ; Thu, 25 Feb 2021 05:48:56 -0800 (PST) Received: by mail-pf1-x42c.google.com with SMTP id u26so3640111pfn.6 for ; Thu, 25 Feb 2021 05:48:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=veJ44tY4H1b4cNQs/+ikD/U07gxfyKVeQOKDpom/Qlo=; b=bIvzWMw7UyfnfE3dxFo/LOmOsyj6lCHMn9htGdUAKrD1h57jVAzaW+cTmRQ6MprAzo LLyROP07D4p09AfGlneIf8mQLUSp9m6C3c6sWlbdp1heDCCCZ73gYVIAs7sPuce9+jm+ fvSY7ch69daYTM1WtnyRLkwoTN0CACzuW//5g0Orh4AVzfyu/UuXt554N3qu0lQPdrvw oJlJQ6leXpfEZ5YvHSHxZPUUr1cZUM/TWCblWil0mUD0eOtuIWg51p7g0/2ISSUsWxYN XedrKGPiqh8QZ+k4RpX1k5mNfwRyM7M3yGx3+7rQgo27ZmHQg7HMu1WWFW4BbY5tpJpP uZeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=veJ44tY4H1b4cNQs/+ikD/U07gxfyKVeQOKDpom/Qlo=; b=WUSnq4GAR7kMefAgUtAFuDJrPtDXDD/GOUyce2f/e96ojACf92lcigRpHktVjq+A43 yhxMZZgkTP6/u2Y4VEsCmQgRFWMeUlicKln2+0g80WU21oD9DSkoeC9fpzfG3AJ1eBzd XPmXpZiN7WXp03bgLQlXw44KKrEgub1l9TPPvMH9RGZqgNjnPycRDHfWnaPt5y+EZRNo F56sufjP6G3LsFdAvL4WaoxEOhqpaH02LFQxrrq2nnuwomWfnHOg2REJYXn7iOjMFVc6 iEw8gdIdpatOpWVdsCCe5qyn3kS64t7q8ZYc/Bl49vZvqkbZxR4Frh5fto9Hgu38gOEQ Tvew== X-Gm-Message-State: AOAM531vVo/Ao6AfRDxUW8QJDs7Ck2Of2XzqNLZmQznlXnSiWD6BsegL nrehTduUXbrR94fMRWsnCUfngVnY1+M= X-Google-Smtp-Source: ABdhPJxq3tMoWnWb0JY9fjh/na9nQWenmY9OE5PZW5fALA538ouxAlTXUPGyJTgn2rEH91+7PngU5g== X-Received: by 2002:a63:2a17:: with SMTP id q23mr3026533pgq.223.1614260935198; Thu, 25 Feb 2021 05:48:55 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:54 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 33/37] KVM: PPC: Book3S HV: small pseries_do_hcall cleanup Date: Thu, 25 Feb 2021 23:46:48 +1000 Message-Id: <20210225134652.2127648-34-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Functionality should not be changed. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 1f27187ff1e7..9d2fa21201c1 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -923,6 +923,7 @@ static int kvmppc_get_yield_count(struct kvm_vcpu *vcpu) int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) { + struct kvm *kvm = vcpu->kvm; unsigned long req = kvmppc_get_gpr(vcpu, 3); unsigned long target, ret = H_SUCCESS; int yield_count; @@ -938,7 +939,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) break; case H_PROD: target = kvmppc_get_gpr(vcpu, 4); - tvcpu = kvmppc_find_vcpu(vcpu->kvm, target); + tvcpu = kvmppc_find_vcpu(kvm, target); if (!tvcpu) { ret = H_PARAMETER; break; @@ -952,7 +953,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) target = kvmppc_get_gpr(vcpu, 4); if (target == -1) break; - tvcpu = kvmppc_find_vcpu(vcpu->kvm, target); + tvcpu = kvmppc_find_vcpu(kvm, target); if (!tvcpu) { ret = H_PARAMETER; break; @@ -968,12 +969,12 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) kvmppc_get_gpr(vcpu, 6)); break; case H_RTAS: - if (list_empty(&vcpu->kvm->arch.rtas_tokens)) + if (list_empty(&kvm->arch.rtas_tokens)) return RESUME_HOST; - idx = srcu_read_lock(&vcpu->kvm->srcu); + idx = srcu_read_lock(&kvm->srcu); rc = kvmppc_rtas_hcall(vcpu); - srcu_read_unlock(&vcpu->kvm->srcu, idx); + srcu_read_unlock(&kvm->srcu, idx); if (rc == -ENOENT) return RESUME_HOST; @@ -1060,12 +1061,12 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) case H_SET_PARTITION_TABLE: ret = H_FUNCTION; - if (nesting_enabled(vcpu->kvm)) + if (nesting_enabled(kvm)) ret = kvmhv_set_partition_table(vcpu); break; case H_ENTER_NESTED: ret = H_FUNCTION; - if (!nesting_enabled(vcpu->kvm)) + if (!nesting_enabled(kvm)) break; ret = kvmhv_enter_nested_guest(vcpu); if (ret == H_INTERRUPT) { @@ -1080,12 +1081,12 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) break; case H_TLB_INVALIDATE: ret = H_FUNCTION; - if (nesting_enabled(vcpu->kvm)) + if (nesting_enabled(kvm)) ret = kvmhv_do_nested_tlbie(vcpu); break; case H_COPY_TOFROM_GUEST: ret = H_FUNCTION; - if (nesting_enabled(vcpu->kvm)) + if (nesting_enabled(kvm)) ret = kvmhv_copy_tofrom_guest_nested(vcpu); break; case H_PAGE_INIT: @@ -1096,7 +1097,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) case H_SVM_PAGE_IN: ret = H_UNSUPPORTED; if (kvmppc_get_srr1(vcpu) & MSR_S) - ret = kvmppc_h_svm_page_in(vcpu->kvm, + ret = kvmppc_h_svm_page_in(kvm, kvmppc_get_gpr(vcpu, 4), kvmppc_get_gpr(vcpu, 5), kvmppc_get_gpr(vcpu, 6)); @@ -1104,7 +1105,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) case H_SVM_PAGE_OUT: ret = H_UNSUPPORTED; if (kvmppc_get_srr1(vcpu) & MSR_S) - ret = kvmppc_h_svm_page_out(vcpu->kvm, + ret = kvmppc_h_svm_page_out(kvm, kvmppc_get_gpr(vcpu, 4), kvmppc_get_gpr(vcpu, 5), kvmppc_get_gpr(vcpu, 6)); @@ -1112,12 +1113,12 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) case H_SVM_INIT_START: ret = H_UNSUPPORTED; if (kvmppc_get_srr1(vcpu) & MSR_S) - ret = kvmppc_h_svm_init_start(vcpu->kvm); + ret = kvmppc_h_svm_init_start(kvm); break; case H_SVM_INIT_DONE: ret = H_UNSUPPORTED; if (kvmppc_get_srr1(vcpu) & MSR_S) - ret = kvmppc_h_svm_init_done(vcpu->kvm); + ret = kvmppc_h_svm_init_done(kvm); break; case H_SVM_INIT_ABORT: /* @@ -1127,7 +1128,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) * Instead the kvm->arch.secure_guest flag is checked inside * kvmppc_h_svm_init_abort(). */ - ret = kvmppc_h_svm_init_abort(vcpu->kvm); + ret = kvmppc_h_svm_init_abort(kvm); break; default: From patchwork Thu Feb 25 13:46:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444396 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=lO8j5A1A; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ0H5NRTz9sWH for ; Fri, 26 Feb 2021 00:49:35 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229566AbhBYNtZ (ORCPT ); Thu, 25 Feb 2021 08:49:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232237AbhBYNtO (ORCPT ); Thu, 25 Feb 2021 08:49:14 -0500 Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A10CC061574 for ; Thu, 25 Feb 2021 05:48:59 -0800 (PST) Received: by mail-pg1-x52e.google.com with SMTP id o38so3795497pgm.9 for ; Thu, 25 Feb 2021 05:48:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uEQIGr8Qp8LFoR1y4PL0b7+g3Mw1hvzNAwtrHtHvVdU=; b=lO8j5A1AGxcM10U9YGyU7kGNYkYvI5h+IB9BX3Gy2rw0u8flT32H5Q96VFCkBVzM4i 6ubVdEoi4bljklOLXBqtBhol/jlSwhiN6TakhwbID81h8ASr2H33wVAas7Z0T93nVamg UvMuWZRWhIi9Kg8sNEFoVZKkiCpdlGjhNTeyJtWpQNAuqLNzL/cyYDbQ/l+U4TKUJv26 QVubzlr1OWdWsOmr9YOELifPL/kRAsA2pMV1ICiRV6SfUcdR+0BpSm+y/2K6tNNPCris HzkmfY8DGYYrUjLD4Opp5zmMtIZYApDHMcimWfPNml9troOZ7jquiptYJz+8zUjHY6Dm D0Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uEQIGr8Qp8LFoR1y4PL0b7+g3Mw1hvzNAwtrHtHvVdU=; b=QbQIZQKkxD+GWwfoNnPV3vV8E+1ST20vGbUtHtcR2kDXTks4ol/0VCiEA1ut1L5ZSN qpOmVUbZ1rgdGF0Yj/dHjqXp35r2XTMcK/hd9+mwW47AQKYTKaWGl4EvNZp5fMZOBMj8 mEfvqPB41hxYgMkm7BdK3jsq+XdMS5bS7TkZlgncKGDctNWgBzBuLywNZskfUaIcUEAk JK9og6l9nxXEla+AtuVr2JWfj/oz2MVWq1mAIeAOo8RBYDRpJ0Yu4+MOAie7CtBhoWtn kwH2CWhxb9GzqWU6CGEGEzWerl3+3eQhYW0xYumsftcpJCcQPp+wivQkbbUAdyhgv80b 56Sg== X-Gm-Message-State: AOAM5326/dDVRgnsNn3v6jr+lC6reJsPwNb19IV/Mzv5GEYj5hHYsD4A 4Hs7yDf6L/q831qT3h5NZpUl3ng8cGQ= X-Google-Smtp-Source: ABdhPJzTgPyR7/GY4LHlXD8W7cxZ9ohC9HH4hNz/Lz3TX4tnQt7EqG2yJmSc29rGHYiqhEbqva7PhQ== X-Received: by 2002:a63:520d:: with SMTP id g13mr3098033pgb.234.1614260938490; Thu, 25 Feb 2021 05:48:58 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:48:57 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 34/37] KVM: PPC: Book3S HV: add virtual mode handlers for HPT hcalls and page faults Date: Thu, 25 Feb 2021 23:46:49 +1000 Message-Id: <20210225134652.2127648-35-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org In order to support hash guests in the P9 path (which does not do real mode hcalls or page fault handling), these real-mode hash specific interrupts need to be implemented in virt mode. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 118 +++++++++++++++++++++++++++++++++-- 1 file changed, 113 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 9d2fa21201c1..1bbc46f2cfbf 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -935,6 +935,52 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) return RESUME_HOST; switch (req) { + case H_REMOVE: + ret = kvmppc_h_remove(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5), + kvmppc_get_gpr(vcpu, 6)); + if (ret == H_TOO_HARD) + return RESUME_HOST; + break; + case H_ENTER: + ret = kvmppc_h_enter(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5), + kvmppc_get_gpr(vcpu, 6), + kvmppc_get_gpr(vcpu, 7)); + if (ret == H_TOO_HARD) + return RESUME_HOST; + break; + case H_READ: + ret = kvmppc_h_read(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5)); + if (ret == H_TOO_HARD) + return RESUME_HOST; + break; + case H_CLEAR_MOD: + ret = kvmppc_h_clear_mod(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5)); + if (ret == H_TOO_HARD) + return RESUME_HOST; + break; + case H_CLEAR_REF: + ret = kvmppc_h_clear_ref(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5)); + if (ret == H_TOO_HARD) + return RESUME_HOST; + break; + case H_PROTECT: + ret = kvmppc_h_protect(vcpu, kvmppc_get_gpr(vcpu, 4), + kvmppc_get_gpr(vcpu, 5), + kvmppc_get_gpr(vcpu, 6)); + if (ret == H_TOO_HARD) + return RESUME_HOST; + break; + case H_BULK_REMOVE: + ret = kvmppc_h_bulk_remove(vcpu); + if (ret == H_TOO_HARD) + return RESUME_HOST; + break; + case H_CEDE: break; case H_PROD: @@ -1134,6 +1180,7 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu) default: return RESUME_HOST; } + WARN_ON_ONCE(ret == H_TOO_HARD); kvmppc_set_gpr(vcpu, 3, ret); vcpu->arch.hcall_needed = 0; return RESUME_GUEST; @@ -1420,19 +1467,80 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu, * host page has been paged out. Any other HDSI/HISI interrupts * have been handled already. */ - case BOOK3S_INTERRUPT_H_DATA_STORAGE: - r = RESUME_PAGE_FAULT; - if (vcpu->arch.fault_dsisr == HDSISR_CANARY) + case BOOK3S_INTERRUPT_H_DATA_STORAGE: { + unsigned long vsid; + long err; + + if (vcpu->arch.fault_dsisr == HDSISR_CANARY) { r = RESUME_GUEST; /* Just retry if it's the canary */ + break; + } + + if (kvm_is_radix(vcpu->kvm)) { + r = RESUME_PAGE_FAULT; + break; + } + + if (!(vcpu->arch.fault_dsisr & (DSISR_NOHPTE | DSISR_PROTFAULT))) { + kvmppc_core_queue_data_storage(vcpu, vcpu->arch.fault_dar, vcpu->arch.fault_dsisr); + r = RESUME_GUEST; + break; + } + if (!(vcpu->arch.shregs.msr & MSR_DR)) { + vsid = vcpu->kvm->arch.vrma_slb_v; + } else { + vsid = vcpu->arch.fault_gpa; + } + err = kvmppc_hpte_hv_fault(vcpu, vcpu->arch.fault_dar, + vsid, vcpu->arch.fault_dsisr, true); + if (err == 0) { + r = RESUME_GUEST; + } else if (err == -1 || err == -2) { + r = RESUME_PAGE_FAULT; + } else { + kvmppc_core_queue_data_storage(vcpu, vcpu->arch.fault_dar, err); + r = RESUME_GUEST; + } break; - case BOOK3S_INTERRUPT_H_INST_STORAGE: + } + case BOOK3S_INTERRUPT_H_INST_STORAGE: { + unsigned long vsid; + long err; + vcpu->arch.fault_dar = kvmppc_get_pc(vcpu); vcpu->arch.fault_dsisr = vcpu->arch.shregs.msr & DSISR_SRR1_MATCH_64S; if (vcpu->arch.shregs.msr & HSRR1_HISI_WRITE) vcpu->arch.fault_dsisr |= DSISR_ISSTORE; - r = RESUME_PAGE_FAULT; + if (kvm_is_radix(vcpu->kvm)) { + r = RESUME_PAGE_FAULT; + break; + } + + if (!(vcpu->arch.fault_dsisr & SRR1_ISI_NOPT)) { + /* XXX: clear DSISR_ISSTORE? */ + kvmppc_core_queue_inst_storage(vcpu, vcpu->arch.fault_dsisr); + r = RESUME_GUEST; + break; + } + if (!(vcpu->arch.shregs.msr & MSR_DR)) { + vsid = vcpu->kvm->arch.vrma_slb_v; + } else { + vsid = vcpu->arch.fault_gpa; + } + err = kvmppc_hpte_hv_fault(vcpu, vcpu->arch.fault_dar, + vsid, vcpu->arch.fault_dsisr, false); + if (err == 0) { + r = RESUME_GUEST; + } else if (err == -1) { + r = RESUME_PAGE_FAULT; + } else { + kvmppc_core_queue_inst_storage(vcpu, err); + r = RESUME_GUEST; + } break; + } + /* * This occurs if the guest executes an illegal instruction. * If the guest debug is disabled, generate a program interrupt From patchwork Thu Feb 25 13:46:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444397 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=uXLoBGpA; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ0J3RRTz9sWv for ; Fri, 26 Feb 2021 00:49:36 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232469AbhBYNte (ORCPT ); Thu, 25 Feb 2021 08:49:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232394AbhBYNtR (ORCPT ); Thu, 25 Feb 2021 08:49:17 -0500 Received: from mail-pl1-x634.google.com (mail-pl1-x634.google.com [IPv6:2607:f8b0:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8C2CC06174A for ; Thu, 25 Feb 2021 05:49:02 -0800 (PST) Received: by mail-pl1-x634.google.com with SMTP id u11so3199656plg.13 for ; Thu, 25 Feb 2021 05:49:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SmX0ilCm5r3iBQEzRqeBZLJnlYFu+crfS8nmIsYYJDw=; b=uXLoBGpAjTZqt/CpNZhcU6GsLIjcp2bzhYjOrLoYXtjXKQcRTlyTr+w8Eq1b5pTVMj UvWvysgRHvEkykZlxUTfZOGa1Vc3UjA7lXSCe3/+MJyOUzbpnKW1eLLSfzy3sR1D3pNY uQ5kin9p8mMvpD2P/hmfe75Z/VGi35xVJhkUxHcKdHrw/ojuXX0wlqgttBubDl51Np5j CKzp4te1i00CLLzaaEAGCNOenW0xtg4F8AomBI/cKBqUDq0AmYgsSvUWIHooLy0Ore4K aDXvtcro2rwKDC0c+xxRWUD8Vy4uYZH+zgkvmmqeJ18WtpVGlH6ofYBVFdG4nXetit4D 5sXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SmX0ilCm5r3iBQEzRqeBZLJnlYFu+crfS8nmIsYYJDw=; b=JfmVgLq5YMBHDiv3fm13G8L/MeccRmr3XieANomI7zU4XddZTNf4NUv9fBZbxXXgB1 QI9uxeODQ3acGqDjzy3gT9VfbL42ghFD3JdSzOBoj8FUcXbwUz5pbOolvnvyx2llpCxW Jzju12FVsMo4sgVeXBrXke1ZwHsPoF3oRGUZTy9SH/+o3j70XkOyYNLZ9mrpNMJ/xaVS R7f0Xod5a3Hi+aBJ8Wt9N47zzyx1CQw1lzHsrUb9cMppWmQ5+XShEadB3cu5VxCGRdko phCl3Te7DEKx2HzFmZfDTaeNdBLBNXC2MCbLe123I+bM1cT5FXjvMFfz3pGP9+kmMMNr YW1w== X-Gm-Message-State: AOAM533Fp2KDx9G4TeWkRt3SZMYVzeCHK73p5w5skgcZif4PkFwN1kPN 5w2UQJf3FemKU8jzNhpBO/EYCyei2GA= X-Google-Smtp-Source: ABdhPJyL9Qt8ekL1P4yo2LZl0CmhKG7RFIglDLdC29J9K0z/Z/LexWTEXyBoxE8XlpI1qqSANzEyKA== X-Received: by 2002:a17:90a:5501:: with SMTP id b1mr3435726pji.57.1614260941681; Thu, 25 Feb 2021 05:49:01 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.48.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:49:01 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 35/37] KVM: PPC: Book3S HV P9: implement hash guest support Date: Thu, 25 Feb 2021 23:46:50 +1000 Message-Id: <20210225134652.2127648-36-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 20 ++-- arch/powerpc/kvm/book3s_hv_interrupt.c | 123 +++++++++++++++++------- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 4 + arch/powerpc/kvm/book3s_hv_rmhandlers.S | 14 +-- 4 files changed, 109 insertions(+), 52 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 1bbc46f2cfbf..97320531f37c 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3765,7 +3765,8 @@ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, } kvmppc_xive_pull_vcpu(vcpu); - vcpu->arch.slb_max = 0; + if (kvm_is_radix(vcpu->kvm)) + vcpu->arch.slb_max = 0; } dec = mfspr(SPRN_DEC); @@ -3998,7 +3999,6 @@ static void kvmppc_vcore_blocked(struct kvmppc_vcore *vc) /* * This never fails for a radix guest, as none of the operations it does * for a radix guest can fail or have a way to report failure. - * kvmhv_run_single_vcpu() relies on this fact. */ static int kvmhv_setup_mmu(struct kvm_vcpu *vcpu) { @@ -4177,8 +4177,15 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, vc->runner = vcpu; /* See if the MMU is ready to go */ - if (!kvm->arch.mmu_ready) - kvmhv_setup_mmu(vcpu); + if (!kvm->arch.mmu_ready) { + r = kvmhv_setup_mmu(vcpu); + if (r) { + run->exit_reason = KVM_EXIT_FAIL_ENTRY; + run->fail_entry.hardware_entry_failure_reason = 0; + vcpu->arch.ret = r; + return r; + } + } if (need_resched()) cond_resched(); @@ -4191,7 +4198,8 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, preempt_disable(); pcpu = smp_processor_id(); vc->pcpu = pcpu; - kvmppc_prepare_radix_vcpu(vcpu, pcpu); + if (kvm_is_radix(kvm)) + kvmppc_prepare_radix_vcpu(vcpu, pcpu); local_irq_disable(); hard_irq_disable(); @@ -4395,7 +4403,7 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu) * The TLB prefetch bug fixup is only in the kvmppc_run_vcpu * path, which also handles hash and dependent threads mode. */ - if (kvm_is_radix(kvm)) + if (radix_enabled()) r = kvmhv_run_single_vcpu(vcpu, ~(u64)0, vcpu->arch.vcore->lpcr); else diff --git a/arch/powerpc/kvm/book3s_hv_interrupt.c b/arch/powerpc/kvm/book3s_hv_interrupt.c index 9784da3f8565..d79c6f4f330c 100644 --- a/arch/powerpc/kvm/book3s_hv_interrupt.c +++ b/arch/powerpc/kvm/book3s_hv_interrupt.c @@ -55,44 +55,25 @@ static void __accumulate_time(struct kvm_vcpu *vcpu, struct kvmhv_tb_accumulator #define accumulate_time(vcpu, next) do {} while (0) #endif -static void switch_mmu_to_guest_radix(struct kvm *kvm, struct kvm_vcpu *vcpu, u64 lpcr) -{ - struct kvmppc_vcore *vc = vcpu->arch.vcore; - struct kvm_nested_guest *nested = vcpu->arch.nested; - u32 lpid; - - lpid = nested ? nested->shadow_lpid : kvm->arch.lpid; - - mtspr(SPRN_LPID, lpid); - mtspr(SPRN_LPCR, lpcr); - mtspr(SPRN_PID, vcpu->arch.pid); - isync(); - - /* TLBIEL must have LPIDR set, so set guest LPID before flushing. */ - kvmppc_check_need_tlb_flush(kvm, vc->pcpu, nested); -} - -static void switch_mmu_to_host_radix(struct kvm *kvm, u32 pid) -{ - mtspr(SPRN_PID, pid); - mtspr(SPRN_LPID, kvm->arch.host_lpid); - mtspr(SPRN_LPCR, kvm->arch.host_lpcr); - isync(); -} - static inline void mfslb(unsigned int idx, u64 *slbee, u64 *slbev) { asm volatile("slbmfev %0,%1" : "=r" (*slbev) : "r" (idx)); asm volatile("slbmfee %0,%1" : "=r" (*slbee) : "r" (idx)); } +static inline void __mtslb(u64 slbee, u64 slbev) +{ + asm volatile("slbmte %0,%1" :: "r" (slbev), "r" (slbee)); +} + static inline void mtslb(unsigned int idx, u64 slbee, u64 slbev) { BUG_ON((slbee & 0xfff) != idx); - asm volatile("slbmte %0,%1" :: "r" (slbev), "r" (slbee)); + __mtslb(slbee, slbev); } + static inline void slb_invalidate(unsigned int ih) { asm volatile("slbia %0" :: "i"(ih)); @@ -119,6 +100,54 @@ static void radix_clear_slb(void) } } +static void switch_mmu_to_guest_radix(struct kvm *kvm, struct kvm_vcpu *vcpu, u64 lpcr) +{ + struct kvmppc_vcore *vc = vcpu->arch.vcore; + struct kvm_nested_guest *nested = vcpu->arch.nested; + u32 lpid; + + lpid = nested ? nested->shadow_lpid : kvm->arch.lpid; + + mtspr(SPRN_LPID, lpid); + mtspr(SPRN_LPCR, lpcr); + mtspr(SPRN_PID, vcpu->arch.pid); + isync(); + + /* TLBIEL must have LPIDR set, so set guest LPID before flushing. */ + kvmppc_check_need_tlb_flush(kvm, vc->pcpu, nested); +} + +static void switch_mmu_to_guest_hpt(struct kvm *kvm, struct kvm_vcpu *vcpu, u64 lpcr) +{ + struct kvm_nested_guest *nested = vcpu->arch.nested; + u32 lpid; + int i; + + BUG_ON(nested); + + lpid = kvm->arch.lpid; + + mtspr(SPRN_LPID, lpid); + mtspr(SPRN_LPCR, lpcr); + mtspr(SPRN_PID, vcpu->arch.pid); + + for (i = 0; i < vcpu->arch.slb_max; i++) + __mtslb(vcpu->arch.slb[i].orige, vcpu->arch.slb[i].origv); + + isync(); + + /* XXX: TLBIEL not virtualised for HPT guests */ +} + + +static void switch_mmu_to_host_radix(struct kvm *kvm, u32 pid) +{ + mtspr(SPRN_PID, pid); + mtspr(SPRN_LPID, kvm->arch.host_lpid); + mtspr(SPRN_LPCR, kvm->arch.host_lpcr); + isync(); +} + int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpcr) { struct kvm *kvm = vcpu->kvm; @@ -223,10 +252,17 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc mtspr(SPRN_AMOR, ~0UL); - if (cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) - __mtmsrd(msr & ~(MSR_IR|MSR_DR|MSR_RI), 0); + if (kvm_is_radix(kvm)) { + if (cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) + __mtmsrd(msr & ~(MSR_IR|MSR_DR|MSR_RI), 0); + switch_mmu_to_guest_radix(kvm, vcpu, lpcr); + if (!cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) + __mtmsrd(0, 1); /* clear RI */ - switch_mmu_to_guest_radix(kvm, vcpu, lpcr); + } else { + __mtmsrd(msr & ~(MSR_IR|MSR_DR|MSR_RI), 0); + switch_mmu_to_guest_hpt(kvm, vcpu, lpcr); + } /* * P9 suppresses the HDEC exception when LPCR[HDICE] = 0, @@ -234,9 +270,6 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc */ mtspr(SPRN_HDEC, hdec); - if (!cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) - __mtmsrd(0, 1); /* clear RI */ - mtspr(SPRN_DAR, vcpu->arch.shregs.dar); mtspr(SPRN_DSISR, vcpu->arch.shregs.dsisr); mtspr(SPRN_SRR0, vcpu->arch.shregs.srr0); @@ -340,8 +373,6 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc #endif } - radix_clear_slb(); - accumulate_time(vcpu, &vcpu->arch.rm_exit); /* Advance host PURR/SPURR by the amount used by guest */ @@ -406,6 +437,30 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc /* HDEC must be at least as large as DEC, so decrementer_max fits */ mtspr(SPRN_HDEC, decrementer_max); + if (kvm_is_radix(kvm)) { + radix_clear_slb(); + } else { + int i; + int nr = 0; + + /* + * This must run before switching to host (radix host can't + * access all SLBs). + */ + for (i = 0; i < vcpu->arch.slb_nr; i++) { + u64 slbee, slbev; + mfslb(i, &slbee, &slbev); + if (slbee & SLB_ESID_V) { + vcpu->arch.slb[nr].orige = slbee | i; + vcpu->arch.slb[nr].origv = slbev; + nr++; + } + } + vcpu->arch.slb_max = nr; + mtslb(0, 0, 0); + slb_invalidate(6); + } + switch_mmu_to_host_radix(kvm, host_pidr); /* diff --git a/arch/powerpc/kvm/book3s_hv_rm_mmu.c b/arch/powerpc/kvm/book3s_hv_rm_mmu.c index 7af7c70f1468..b5532b85c7fd 100644 --- a/arch/powerpc/kvm/book3s_hv_rm_mmu.c +++ b/arch/powerpc/kvm/book3s_hv_rm_mmu.c @@ -57,6 +57,10 @@ static int global_invalidates(struct kvm *kvm) else global = 1; + /* LPID has been switched to host if in virt mode so can't do local */ + if (!global && (mfmsr() & (MSR_IR|MSR_DR))) + global = 1; + if (!global) { /* any other core might now have stale TLB entries... */ smp_wmb(); diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index a8ce68eed13e..be5742640780 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -899,14 +899,11 @@ ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_300) cmpdi r3, 512 /* 1 microsecond */ blt hdec_soon - /* For hash guest, clear out and reload the SLB */ -BEGIN_MMU_FTR_SECTION - /* Radix host won't have populated the SLB, so no need to clear */ + /* Clear out and reload the SLB */ li r6, 0 slbmte r6, r6 PPC_SLBIA(6) ptesync -END_MMU_FTR_SECTION_IFCLR(MMU_FTR_TYPE_RADIX) /* Load up guest SLB entries (N.B. slb_max will be 0 for radix) */ lwz r5,VCPU_SLB_MAX(r4) @@ -1408,9 +1405,6 @@ guest_exit_cont: /* r9 = vcpu, r12 = trap, r13 = paca */ stw r5,VCPU_SLB_MAX(r9) /* load host SLB entries */ -BEGIN_MMU_FTR_SECTION - b guest_bypass -END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_RADIX) ld r8,PACA_SLBSHADOWPTR(r13) .rept SLB_NUM_BOLTED @@ -3162,10 +3156,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_DAWR1) PPC_SLBIA(6) ptesync -BEGIN_MMU_FTR_SECTION - b 4f -END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_RADIX) - /* load host SLB entries */ ld r8, PACA_SLBSHADOWPTR(r13) .rept SLB_NUM_BOLTED @@ -3179,7 +3169,7 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_RADIX) 3: addi r8, r8, 16 .endr -4: lwz r7, KVM_HOST_LPID(r10) + lwz r7, KVM_HOST_LPID(r10) mtspr SPRN_LPID, r7 mtspr SPRN_PID, r0 ld r8, KVM_HOST_LPCR(r10) From patchwork Thu Feb 25 13:46:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444402 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=M2IA6lAX; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ0p1gt1z9sWK for ; Fri, 26 Feb 2021 00:50:02 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232600AbhBYNtu (ORCPT ); Thu, 25 Feb 2021 08:49:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230142AbhBYNtq (ORCPT ); Thu, 25 Feb 2021 08:49:46 -0500 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10E4DC061793 for ; Thu, 25 Feb 2021 05:49:06 -0800 (PST) Received: by mail-pf1-x42e.google.com with SMTP id d12so1108992pfo.7 for ; Thu, 25 Feb 2021 05:49:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/d8HsRevOUAthNw1fJ9lQVtVP6HhjoYaa+NO0SoIY3c=; b=M2IA6lAXNli1kQg+PS6A/2dAZxaIQ3h50JKVW0pDdSSnm2OgTFr0UinELigHncu6vb rx6umZeJ0/v2NA3svwU7JHJ/hr/su59OXdh2+q0maBRejgMz42m99o/PdlMrH/SfUJ0n JUMmwgZr0nFYknRklSq+zA2RxFlsxr/WUfxPpozxWQ1rYRcSs2u1nPCBMwy/jhQFhaMk arGzIP5Ct4btdKarOx1Mx9qk7746mpkYTDhXktGyvLLHQm8HeRgS3Dr6Ofx8lXN+x6j9 9zYFFF1ZfzMkhRi6nZRW5Rw034g/d9ywe7R2hZIc6rM5Df8jKd79s3A8Z3zwN6QbEZzW OKEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/d8HsRevOUAthNw1fJ9lQVtVP6HhjoYaa+NO0SoIY3c=; b=R48f0mbSraXL0/pmH966ofK99bIAu50ceCtZL/IpW7fnTpNFnMuf4Mifvj8aUtNvcM b5mGjsQhEQtARTCAjVE0ZvCEoOGqteLjAW+J8yklnfeq2FQhpz6vMvFeds86iyqHMgkq MS65iW4mRs8XihLPXRsEsq6CLcUvhGkVc4O9VzXX0jly8un5QrQL5/B2DseQabaixCqZ vUL0EMU1tHzJ8YwJ2AE7X2lnGHhhzM6Z7GaKn1dUA0aPyllTjwScN6kcDVhWbWCGgmq6 +3wJ9AcBXFFIMi8QZgEmWuqMkx428OTmFVjOwOyUN7JyWYqSr+BLK0qYzfbaC1j1Ycgx NGvw== X-Gm-Message-State: AOAM532Aeauy9/8D9fnR6N5rQl98OBvWJeggKYFbcZEBHNOfHdJmFWxu LFucUyj35BdOvzpxUfoN7t5tvN8JSqg= X-Google-Smtp-Source: ABdhPJx9GutxHKPn6nwNeeltugDeXxOtwMZO1Xkh8v+cvO3T43BAiOeR/hDF3Ae4FQVT1rax4dOQjQ== X-Received: by 2002:a05:6a00:22ca:b029:1ed:f915:ca98 with SMTP id f10-20020a056a0022cab02901edf915ca98mr3317549pfj.68.1614260944724; Thu, 25 Feb 2021 05:49:04 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.49.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:49:04 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 36/37] KVM: PPC: Book3S HV P9: implement hash host / hash guest support Date: Thu, 25 Feb 2021 23:46:51 +1000 Message-Id: <20210225134652.2127648-37-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 2 +- arch/powerpc/kvm/book3s_hv_interrupt.c | 75 ++++++++++++++++---------- 2 files changed, 47 insertions(+), 30 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 97320531f37c..10d5c7ea80ca 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -4403,7 +4403,7 @@ static int kvmppc_vcpu_run_hv(struct kvm_vcpu *vcpu) * The TLB prefetch bug fixup is only in the kvmppc_run_vcpu * path, which also handles hash and dependent threads mode. */ - if (radix_enabled()) + if (cpu_has_feature(CPU_FTR_ARCH_300)) r = kvmhv_run_single_vcpu(vcpu, ~(u64)0, vcpu->arch.vcore->lpcr); else diff --git a/arch/powerpc/kvm/book3s_hv_interrupt.c b/arch/powerpc/kvm/book3s_hv_interrupt.c index d79c6f4f330c..af4772755e5d 100644 --- a/arch/powerpc/kvm/book3s_hv_interrupt.c +++ b/arch/powerpc/kvm/book3s_hv_interrupt.c @@ -140,12 +140,51 @@ static void switch_mmu_to_guest_hpt(struct kvm *kvm, struct kvm_vcpu *vcpu, u64 } -static void switch_mmu_to_host_radix(struct kvm *kvm, u32 pid) +static void switch_mmu_to_host(struct kvm *kvm, u32 pid) { mtspr(SPRN_PID, pid); mtspr(SPRN_LPID, kvm->arch.host_lpid); mtspr(SPRN_LPCR, kvm->arch.host_lpcr); isync(); + + /* XXX: could save and restore host SLBs to reduce SLB faults */ + if (!radix_enabled()) + slb_restore_bolted_realmode(); +} + +static void save_host_mmu(struct kvm *kvm) +{ + if (!radix_enabled()) { + mtslb(0, 0, 0); + slb_invalidate(6); + } +} + +static void save_guest_mmu(struct kvm *kvm, struct kvm_vcpu *vcpu) +{ + if (kvm_is_radix(kvm)) { + radix_clear_slb(); + } else { + int i; + int nr = 0; + + /* + * This must run before switching to host (radix host can't + * access all SLBs). + */ + for (i = 0; i < vcpu->arch.slb_nr; i++) { + u64 slbee, slbev; + mfslb(i, &slbee, &slbev); + if (slbee & SLB_ESID_V) { + vcpu->arch.slb[nr].orige = slbee | i; + vcpu->arch.slb[nr].origv = slbev; + nr++; + } + } + vcpu->arch.slb_max = nr; + mtslb(0, 0, 0); + slb_invalidate(6); + } } int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpcr) @@ -252,15 +291,16 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc mtspr(SPRN_AMOR, ~0UL); + if (!radix_enabled() || !kvm_is_radix(kvm) || cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) + __mtmsrd(msr & ~(MSR_IR|MSR_DR|MSR_RI), 0); + + save_host_mmu(kvm); if (kvm_is_radix(kvm)) { - if (cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) - __mtmsrd(msr & ~(MSR_IR|MSR_DR|MSR_RI), 0); switch_mmu_to_guest_radix(kvm, vcpu, lpcr); if (!cpu_has_feature(CPU_FTR_P9_RADIX_PREFETCH_BUG)) __mtmsrd(0, 1); /* clear RI */ } else { - __mtmsrd(msr & ~(MSR_IR|MSR_DR|MSR_RI), 0); switch_mmu_to_guest_hpt(kvm, vcpu, lpcr); } @@ -437,31 +477,8 @@ int kvmhv_vcpu_entry_p9(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpc /* HDEC must be at least as large as DEC, so decrementer_max fits */ mtspr(SPRN_HDEC, decrementer_max); - if (kvm_is_radix(kvm)) { - radix_clear_slb(); - } else { - int i; - int nr = 0; - - /* - * This must run before switching to host (radix host can't - * access all SLBs). - */ - for (i = 0; i < vcpu->arch.slb_nr; i++) { - u64 slbee, slbev; - mfslb(i, &slbee, &slbev); - if (slbee & SLB_ESID_V) { - vcpu->arch.slb[nr].orige = slbee | i; - vcpu->arch.slb[nr].origv = slbev; - nr++; - } - } - vcpu->arch.slb_max = nr; - mtslb(0, 0, 0); - slb_invalidate(6); - } - - switch_mmu_to_host_radix(kvm, host_pidr); + save_guest_mmu(kvm, vcpu); + switch_mmu_to_host(kvm, host_pidr); /* * If we are in real mode, don't switch MMU on until the MMU is From patchwork Thu Feb 25 13:46:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1444403 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=kvm-ppc-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=lLX2nmdc; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4DmZ0q0HQgz9sRN for ; Fri, 26 Feb 2021 00:50:03 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232631AbhBYNt5 (ORCPT ); Thu, 25 Feb 2021 08:49:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232596AbhBYNtu (ORCPT ); Thu, 25 Feb 2021 08:49:50 -0500 Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EFDFEC061794 for ; Thu, 25 Feb 2021 05:49:09 -0800 (PST) Received: by mail-pg1-x529.google.com with SMTP id o10so3816084pgg.4 for ; Thu, 25 Feb 2021 05:49:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6BhTvQDmZm0ut9vvLy3QqC+iijopDtXPGxngNhgEr6A=; b=lLX2nmdcGxZEOKWe09zyvZ3WEidwFySNWoM5n+EFdxilWjrrjRy1DYm4FbJmU+XAdn Z686Txna9BwOXmEAR/uhLtJWewXWdaQ8ElU/EQZs9qtcaQB5fF0Jqwi3mLKvavQEYxaT RkR/LV6y47TAkVjDQ/GV8Ja850pR1m6IWppx0NlfwREy0xY0tbqtdNmXWkOkaBnJ1VbI oQDyDerzCMx//KCRHLm2ElUTgfPVunhduoS2xUgX7j0Y2db9LSqYPgR9m+hU5EnqBbtZ HcyhGdvFpCo8xCT8KueIScTjfOC5h+S5shqe9wpGTJIDaZEY86Ii4meuiErJGsbF+eYU tcOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6BhTvQDmZm0ut9vvLy3QqC+iijopDtXPGxngNhgEr6A=; b=ar8mSmNpU0VpSmAZhpJxm9i6DTq9dcTqM5+J+56/PykNaUh0jeotJkWR+b4tuutbfp oBIE5TH5xb3tSXBanL/m1mm/vqd6mrutPE1ouUuRF29nboYqVLEr/NfofG1GTLSbWCNl RgQ1NOT+H5uhMqQDrPMWv97rtRdKnHW2kj3xPn/Kr8/6irAaa6PikcsoqSKt26m7MbtD 9E9vwS8O80qM+OBKdZzFQrkXpryiGe+HVklzw6KB9J8OcZltnp7b6o/JQn1bVcGKGYZj MwOFZHMGsNLjkYpc/848F/tCKNXqm8hT39C01f3sGtn3eCRka0Xj0xOoF/fkFEEHvWhq rguw== X-Gm-Message-State: AOAM533Km9UAZtMYiNbjxhqZY+w1cQs7kpqWq7ifuiiA5WAG1foQEeK2 qGn/JqfQEtKCH0OpH/Cset5x/sv8/vo= X-Google-Smtp-Source: ABdhPJwYeAGGQ63xR7CFS90xUM/MOWPvISGyKOAsBvcIy8zJWzYMMaOeyI9ExN8Ov39+/JjMQjLi5A== X-Received: by 2002:aa7:9e05:0:b029:1ed:e159:cbaa with SMTP id y5-20020aa79e050000b02901ede159cbaamr3422650pfq.49.1614260948335; Thu, 25 Feb 2021 05:49:08 -0800 (PST) Received: from bobo.ibm.com (58-6-239-121.tpgi.com.au. [58.6.239.121]) by smtp.gmail.com with ESMTPSA id a9sm5925868pjq.17.2021.02.25.05.49.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Feb 2021 05:49:07 -0800 (PST) From: Nicholas Piggin To: kvm-ppc@vger.kernel.org Cc: Nicholas Piggin , linuxppc-dev@lists.ozlabs.org Subject: [PATCH v2 37/37] KVM: PPC: Book3S HV: remove POWER9 support from P7/8 paths Date: Thu, 25 Feb 2021 23:46:52 +1000 Message-Id: <20210225134652.2127648-38-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210225134652.2127648-1-npiggin@gmail.com> References: <20210225134652.2127648-1-npiggin@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm-ppc@vger.kernel.org This is dead code now. Signed-off-by: Nicholas Piggin --- arch/powerpc/kvm/book3s_hv.c | 27 +- arch/powerpc/kvm/book3s_hv_interrupts.S | 9 +- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 404 +----------------------- arch/powerpc/platforms/powernv/idle.c | 52 +-- 4 files changed, 23 insertions(+), 469 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 10d5c7ea80ca..483a1a821ea4 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3040,8 +3040,7 @@ static void prepare_threads(struct kvmppc_vcore *vc) for_each_runnable_thread(i, vcpu, vc) { if (signal_pending(vcpu->arch.run_task)) vcpu->arch.ret = -EINTR; - else if (no_mixing_hpt_and_radix && - kvm_is_radix(vc->kvm) != radix_enabled()) + else if (kvm_is_radix(vc->kvm)) vcpu->arch.ret = -EINVAL; else if (vcpu->arch.vpa.update_pending || vcpu->arch.slb_shadow.update_pending || @@ -3249,6 +3248,9 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc) int trap; bool is_power8; + if (WARN_ON_ONCE(cpu_has_feature(CPU_FTR_ARCH_300))) + return; + /* * Remove from the list any threads that have a signal pending * or need a VPA update done @@ -3276,9 +3278,6 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc) * Make sure we are running on primary threads, and that secondary * threads are offline. Also check if the number of threads in this * guest are greater than the current system threads per guest. - * On POWER9, we need to be not in independent-threads mode if - * this is a HPT guest on a radix host machine where the - * CPU threads may not be in different MMU modes. */ if ((controlled_threads > 1) && ((vc->num_threads > threads_per_subcore) || !on_primary_thread())) { @@ -3302,18 +3301,6 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc) if (vc->num_threads < target_threads) collect_piggybacks(&core_info, target_threads); - /* - * On radix, arrange for TLB flushing if necessary. - * This has to be done before disabling interrupts since - * it uses smp_call_function(). - */ - pcpu = smp_processor_id(); - if (kvm_is_radix(vc->kvm)) { - for (sub = 0; sub < core_info.n_subcores; ++sub) - for_each_runnable_thread(i, vcpu, core_info.vc[sub]) - kvmppc_prepare_radix_vcpu(vcpu, pcpu); - } - /* * Hard-disable interrupts, and check resched flag and signals. * If we need to reschedule or deliver a signal, clean up @@ -3346,8 +3333,7 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc) cmd_bit = stat_bit = 0; split = core_info.n_subcores; sip = NULL; - is_power8 = cpu_has_feature(CPU_FTR_ARCH_207S) - && !cpu_has_feature(CPU_FTR_ARCH_300); + is_power8 = cpu_has_feature(CPU_FTR_ARCH_207S); if (split > 1) { sip = &split_info; @@ -3630,8 +3616,7 @@ static void restore_p9_host_os_sprs(struct kvm_vcpu *vcpu, } /* - * Virtual-mode guest entry for POWER9 and later when the host and - * guest are both using the radix MMU. The LPIDR has already been set. + * Guest entry for POWER9 and later CPUs. */ static int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, unsigned long lpcr) diff --git a/arch/powerpc/kvm/book3s_hv_interrupts.S b/arch/powerpc/kvm/book3s_hv_interrupts.S index 327417d79eac..4444f83cb133 100644 --- a/arch/powerpc/kvm/book3s_hv_interrupts.S +++ b/arch/powerpc/kvm/book3s_hv_interrupts.S @@ -58,7 +58,7 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_207S) /* * Put whatever is in the decrementer into the * hypervisor decrementer. - * Because of a hardware deviation in P8 and P9, + * Because of a hardware deviation in P8, * we need to set LPCR[HDICE] before writing HDEC. */ ld r5, HSTATE_KVM_VCORE(r13) @@ -67,15 +67,10 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_207S) ori r8, r9, LPCR_HDICE mtspr SPRN_LPCR, r8 isync - andis. r0, r9, LPCR_LD@h mfspr r8,SPRN_DEC mftb r7 -BEGIN_FTR_SECTION - /* On POWER9, don't sign-extend if host LPCR[LD] bit is set */ - bne 32f -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) extsw r8,r8 -32: mtspr SPRN_HDEC,r8 + mtspr SPRN_HDEC,r8 add r8,r8,r7 std r8,HSTATE_DECEXP(r13) diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index be5742640780..4e9808a2c3a9 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -25,7 +25,6 @@ #include #include #include -#include #include #include #include @@ -34,9 +33,7 @@ /* Sign-extend HDEC if not on POWER9 */ #define EXTEND_HDEC(reg) \ -BEGIN_FTR_SECTION; \ - extsw reg, reg; \ -END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) + extsw reg, reg /* Values in HSTATE_NAPPING(r13) */ #define NAPPING_CEDE 1 @@ -56,8 +53,6 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) #define STACK_SLOT_HFSCR (SFS-72) #define STACK_SLOT_AMR (SFS-80) #define STACK_SLOT_UAMOR (SFS-88) -#define STACK_SLOT_DAWR1 (SFS-96) -#define STACK_SLOT_DAWRX1 (SFS-104) /* * Call kvmppc_hv_entry in real mode. @@ -347,7 +342,6 @@ kvm_secondary_got_guest: LOAD_REG_ADDR(r6, decrementer_max) ld r6, 0(r6) mtspr SPRN_HDEC, r6 -BEGIN_FTR_SECTION /* and set per-LPAR registers, if doing dynamic micro-threading */ ld r6, HSTATE_SPLIT_MODE(r13) cmpdi r6, 0 @@ -359,7 +353,6 @@ BEGIN_FTR_SECTION ld r0, KVM_SPLIT_LDBAR(r6) mtspr SPRN_LDBAR, r0 isync -END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) 63: /* Order load of vcpu after load of vcore */ lwsync @@ -430,7 +423,6 @@ kvm_no_guest: blr 53: -BEGIN_FTR_SECTION HMT_LOW ld r5, HSTATE_KVM_VCORE(r13) cmpdi r5, 0 @@ -445,14 +437,6 @@ BEGIN_FTR_SECTION b kvm_unsplit_nap 60: HMT_MEDIUM b kvm_secondary_got_guest -FTR_SECTION_ELSE - HMT_LOW - ld r5, HSTATE_KVM_VCORE(r13) - cmpdi r5, 0 - beq kvm_no_guest - HMT_MEDIUM - b kvm_secondary_got_guest -ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_300) 54: li r0, KVM_HWTHREAD_IN_KVM stb r0, HSTATE_HWTHREAD_STATE(r13) @@ -578,13 +562,11 @@ kvmppc_hv_entry: bne 10f lwz r7,KVM_LPID(r9) -BEGIN_FTR_SECTION ld r6,KVM_SDR1(r9) li r0,LPID_RSVD /* switch to reserved LPID */ mtspr SPRN_LPID,r0 ptesync mtspr SPRN_SDR1,r6 /* switch to partition page table */ -END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) mtspr SPRN_LPID,r7 isync @@ -664,16 +646,6 @@ kvmppc_got_guest: mtspr SPRN_SPURR,r8 /* Save host values of some registers */ -BEGIN_FTR_SECTION - mfspr r5, SPRN_TIDR - mfspr r6, SPRN_PSSCR - mfspr r7, SPRN_PID - std r5, STACK_SLOT_TID(r1) - std r6, STACK_SLOT_PSSCR(r1) - std r7, STACK_SLOT_PID(r1) - mfspr r5, SPRN_HFSCR - std r5, STACK_SLOT_HFSCR(r1) -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) BEGIN_FTR_SECTION mfspr r5, SPRN_CIABR mfspr r6, SPRN_DAWR0 @@ -684,12 +656,6 @@ BEGIN_FTR_SECTION std r7, STACK_SLOT_DAWRX0(r1) std r8, STACK_SLOT_IAMR(r1) END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S) -BEGIN_FTR_SECTION - mfspr r6, SPRN_DAWR1 - mfspr r7, SPRN_DAWRX1 - std r6, STACK_SLOT_DAWR1(r1) - std r7, STACK_SLOT_DAWRX1(r1) -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S | CPU_FTR_DAWR1) mfspr r5, SPRN_AMR std r5, STACK_SLOT_AMR(r1) @@ -707,13 +673,9 @@ BEGIN_FTR_SECTION END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_207S) #ifdef CONFIG_PPC_TRANSACTIONAL_MEM -/* - * Branch around the call if both CPU_FTR_TM and - * CPU_FTR_P9_TM_HV_ASSIST are off. - */ BEGIN_FTR_SECTION b 91f -END_FTR_SECTION(CPU_FTR_TM | CPU_FTR_P9_TM_HV_ASSIST, 0) +END_FTR_SECTION_IFCLR(CPU_FTR_TM) /* * NOTE THAT THIS TRASHES ALL NON-VOLATILE REGISTERS (but not CR) */ @@ -780,12 +742,6 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_207S) ld r6, VCPU_DAWRX0(r4) mtspr SPRN_DAWR0, r5 mtspr SPRN_DAWRX0, r6 -BEGIN_FTR_SECTION - ld r5, VCPU_DAWR1(r4) - ld r6, VCPU_DAWRX1(r4) - mtspr SPRN_DAWR1, r5 - mtspr SPRN_DAWRX1, r6 -END_FTR_SECTION_IFSET(CPU_FTR_DAWR1) 1: ld r7, VCPU_CIABR(r4) ld r8, VCPU_TAR(r4) @@ -803,7 +759,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_DAWR1) mtspr SPRN_BESCR, r6 mtspr SPRN_PID, r7 mtspr SPRN_WORT, r8 -BEGIN_FTR_SECTION /* POWER8-only registers */ ld r5, VCPU_TCSCR(r4) ld r6, VCPU_ACOP(r4) @@ -814,18 +769,6 @@ BEGIN_FTR_SECTION mtspr SPRN_CSIGR, r7 mtspr SPRN_TACR, r8 nop -FTR_SECTION_ELSE - /* POWER9-only registers */ - ld r5, VCPU_TID(r4) - ld r6, VCPU_PSSCR(r4) - lbz r8, HSTATE_FAKE_SUSPEND(r13) - oris r6, r6, PSSCR_EC@h /* This makes stop trap to HV */ - rldimi r6, r8, PSSCR_FAKE_SUSPEND_LG, 63 - PSSCR_FAKE_SUSPEND_LG - ld r7, VCPU_HFSCR(r4) - mtspr SPRN_TIDR, r5 - mtspr SPRN_PSSCR, r6 - mtspr SPRN_HFSCR, r7 -ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_300) 8: ld r5, VCPU_SPRG0(r4) @@ -918,93 +861,9 @@ ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_300) bdnz 1b 9: -#ifdef CONFIG_KVM_XICS - /* We are entering the guest on that thread, push VCPU to XIVE */ - ld r11, VCPU_XIVE_SAVED_STATE(r4) - li r9, TM_QW1_OS - lwz r8, VCPU_XIVE_CAM_WORD(r4) - cmpwi r8, 0 - beq no_xive - li r7, TM_QW1_OS + TM_WORD2 - mfmsr r0 - andi. r0, r0, MSR_DR /* in real mode? */ - beq 2f - ld r10, HSTATE_XIVE_TIMA_VIRT(r13) - cmpldi cr1, r10, 0 - beq cr1, no_xive - eieio - stdx r11,r9,r10 - stwx r8,r7,r10 - b 3f -2: ld r10, HSTATE_XIVE_TIMA_PHYS(r13) - cmpldi cr1, r10, 0 - beq cr1, no_xive - eieio - stdcix r11,r9,r10 - stwcix r8,r7,r10 -3: li r9, 1 - stb r9, VCPU_XIVE_PUSHED(r4) - eieio - - /* - * We clear the irq_pending flag. There is a small chance of a - * race vs. the escalation interrupt happening on another - * processor setting it again, but the only consequence is to - * cause a spurrious wakeup on the next H_CEDE which is not an - * issue. - */ - li r0,0 - stb r0, VCPU_IRQ_PENDING(r4) - - /* - * In single escalation mode, if the escalation interrupt is - * on, we mask it. - */ - lbz r0, VCPU_XIVE_ESC_ON(r4) - cmpwi cr1, r0,0 - beq cr1, 1f - li r9, XIVE_ESB_SET_PQ_01 - beq 4f /* in real mode? */ - ld r10, VCPU_XIVE_ESC_VADDR(r4) - ldx r0, r10, r9 - b 5f -4: ld r10, VCPU_XIVE_ESC_RADDR(r4) - ldcix r0, r10, r9 -5: sync - - /* We have a possible subtle race here: The escalation interrupt might - * have fired and be on its way to the host queue while we mask it, - * and if we unmask it early enough (re-cede right away), there is - * a theorical possibility that it fires again, thus landing in the - * target queue more than once which is a big no-no. - * - * Fortunately, solving this is rather easy. If the above load setting - * PQ to 01 returns a previous value where P is set, then we know the - * escalation interrupt is somewhere on its way to the host. In that - * case we simply don't clear the xive_esc_on flag below. It will be - * eventually cleared by the handler for the escalation interrupt. - * - * Then, when doing a cede, we check that flag again before re-enabling - * the escalation interrupt, and if set, we abort the cede. - */ - andi. r0, r0, XIVE_ESB_VAL_P - bne- 1f - - /* Now P is 0, we can clear the flag */ - li r0, 0 - stb r0, VCPU_XIVE_ESC_ON(r4) -1: -no_xive: -#endif /* CONFIG_KVM_XICS */ - deliver_guest_interrupt: /* r4 = vcpu, r13 = paca */ /* Check if we can deliver an external or decrementer interrupt now */ ld r0, VCPU_PENDING_EXC(r4) -BEGIN_FTR_SECTION - /* On POWER9, also check for emulated doorbell interrupt */ - lbz r3, VCPU_DBELL_REQ(r4) - or r0, r0, r3 -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) cmpdi r0, 0 beq 71f mr r3, r4 @@ -1077,12 +936,6 @@ BEGIN_FTR_SECTION mtspr SPRN_PPR, r0 END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) -/* Move canary into DSISR to check for later */ -BEGIN_FTR_SECTION - li r0, 0x7fff - mtspr SPRN_HDSISR, r0 -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) - ld r6, VCPU_KVM(r4) lbz r7, KVM_SECURE_GUEST(r6) cmpdi r7, 0 @@ -1298,14 +1151,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) /* Hypervisor doorbell - exit only if host IPI flag set */ cmpwi r12, BOOK3S_INTERRUPT_H_DOORBELL bne 3f -BEGIN_FTR_SECTION - PPC_MSGSYNC - lwsync - /* always exit if we're running a nested guest */ - ld r0, VCPU_NESTED(r9) - cmpdi r0, 0 - bne guest_exit_cont -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) lbz r0, HSTATE_HOST_IPI(r13) cmpwi r0, 0 beq maybe_reenter_guest @@ -1335,43 +1180,6 @@ guest_exit_cont: /* r9 = vcpu, r12 = trap, r13 = paca */ mr r4, r9 bl kvmhv_accumulate_time #endif -#ifdef CONFIG_KVM_XICS - /* We are exiting, pull the VP from the XIVE */ - lbz r0, VCPU_XIVE_PUSHED(r9) - cmpwi cr0, r0, 0 - beq 1f - li r7, TM_SPC_PULL_OS_CTX - li r6, TM_QW1_OS - mfmsr r0 - andi. r0, r0, MSR_DR /* in real mode? */ - beq 2f - ld r10, HSTATE_XIVE_TIMA_VIRT(r13) - cmpldi cr0, r10, 0 - beq 1f - /* First load to pull the context, we ignore the value */ - eieio - lwzx r11, r7, r10 - /* Second load to recover the context state (Words 0 and 1) */ - ldx r11, r6, r10 - b 3f -2: ld r10, HSTATE_XIVE_TIMA_PHYS(r13) - cmpldi cr0, r10, 0 - beq 1f - /* First load to pull the context, we ignore the value */ - eieio - lwzcix r11, r7, r10 - /* Second load to recover the context state (Words 0 and 1) */ - ldcix r11, r6, r10 -3: std r11, VCPU_XIVE_SAVED_STATE(r9) - /* Fixup some of the state for the next load */ - li r10, 0 - li r0, 0xff - stb r10, VCPU_XIVE_PUSHED(r9) - stb r10, (VCPU_XIVE_SAVED_STATE+3)(r9) - stb r0, (VCPU_XIVE_SAVED_STATE+4)(r9) - eieio -1: -#endif /* CONFIG_KVM_XICS */ /* * Possibly flush the link stack here, before we do a blr in @@ -1426,12 +1234,6 @@ guest_bypass: ld r3, HSTATE_KVM_VCORE(r13) mfspr r5,SPRN_DEC mftb r6 - /* On P9, if the guest has large decr enabled, don't sign extend */ -BEGIN_FTR_SECTION - ld r4, VCORE_LPCR(r3) - andis. r4, r4, LPCR_LD@h - bne 16f -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) extsw r5,r5 16: add r5,r5,r6 /* r5 is a guest timebase value here, convert to host TB */ @@ -1505,7 +1307,6 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_207S) std r6, VCPU_BESCR(r9) stw r7, VCPU_GUEST_PID(r9) std r8, VCPU_WORT(r9) -BEGIN_FTR_SECTION mfspr r5, SPRN_TCSCR mfspr r6, SPRN_ACOP mfspr r7, SPRN_CSIGR @@ -1514,17 +1315,6 @@ BEGIN_FTR_SECTION std r6, VCPU_ACOP(r9) std r7, VCPU_CSIGR(r9) std r8, VCPU_TACR(r9) -FTR_SECTION_ELSE - mfspr r5, SPRN_TIDR - mfspr r6, SPRN_PSSCR - std r5, VCPU_TID(r9) - rldicl r6, r6, 4, 50 /* r6 &= PSSCR_GUEST_VIS */ - rotldi r6, r6, 60 - std r6, VCPU_PSSCR(r9) - /* Restore host HFSCR value */ - ld r7, STACK_SLOT_HFSCR(r1) - mtspr SPRN_HFSCR, r7 -ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_300) /* * Restore various registers to 0, where non-zero values * set by the guest could disrupt the host. @@ -1532,13 +1322,11 @@ ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_300) li r0, 0 mtspr SPRN_PSPB, r0 mtspr SPRN_WORT, r0 -BEGIN_FTR_SECTION mtspr SPRN_TCSCR, r0 /* Set MMCRS to 1<<31 to freeze and disable the SPMC counters */ li r0, 1 sldi r0, r0, 31 mtspr SPRN_MMCRS, r0 -END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) /* Save and restore AMR, IAMR and UAMOR before turning on the MMU */ ld r8, STACK_SLOT_IAMR(r1) @@ -1595,13 +1383,9 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) bl kvmppc_save_fp #ifdef CONFIG_PPC_TRANSACTIONAL_MEM -/* - * Branch around the call if both CPU_FTR_TM and - * CPU_FTR_P9_TM_HV_ASSIST are off. - */ BEGIN_FTR_SECTION b 91f -END_FTR_SECTION(CPU_FTR_TM | CPU_FTR_P9_TM_HV_ASSIST, 0) +END_FTR_SECTION_IFCLR(CPU_FTR_TM) /* * NOTE THAT THIS TRASHES ALL NON-VOLATILE REGISTERS (but not CR) */ @@ -1647,28 +1431,6 @@ BEGIN_FTR_SECTION mtspr SPRN_DAWR0, r6 mtspr SPRN_DAWRX0, r7 END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S) -BEGIN_FTR_SECTION - ld r6, STACK_SLOT_DAWR1(r1) - ld r7, STACK_SLOT_DAWRX1(r1) - mtspr SPRN_DAWR1, r6 - mtspr SPRN_DAWRX1, r7 -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S | CPU_FTR_DAWR1) -BEGIN_FTR_SECTION - ld r5, STACK_SLOT_TID(r1) - ld r6, STACK_SLOT_PSSCR(r1) - ld r7, STACK_SLOT_PID(r1) - mtspr SPRN_TIDR, r5 - mtspr SPRN_PSSCR, r6 - mtspr SPRN_PID, r7 -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) - - /* - * cp_abort is required if the processor supports local copy-paste - * to clear the copy buffer that was under control of the guest. - */ -BEGIN_FTR_SECTION - PPC_CP_ABORT -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_31) /* * POWER7/POWER8 guest -> host partition switch code. @@ -1705,13 +1467,11 @@ kvmhv_switch_to_host: /* Primary thread switches back to host partition */ lwz r7,KVM_HOST_LPID(r4) -BEGIN_FTR_SECTION ld r6,KVM_HOST_SDR1(r4) li r8,LPID_RSVD /* switch to reserved LPID */ mtspr SPRN_LPID,r8 ptesync mtspr SPRN_SDR1,r6 /* switch to host page table */ -END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) mtspr SPRN_LPID,r7 isync @@ -1922,20 +1682,11 @@ kvmppc_tm_emul: kvmppc_hdsi: mfspr r4, SPRN_HDAR mfspr r6, SPRN_HDSISR -BEGIN_FTR_SECTION - /* Look for DSISR canary. If we find it, retry instruction */ - cmpdi r6, 0x7fff - beq 6f -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) /* HPTE not found fault or protection fault? */ andis. r0, r6, (DSISR_NOHPTE | DSISR_PROTFAULT)@h beq 1f /* if not, send it to the guest */ andi. r0, r11, MSR_DR /* data relocation enabled? */ beq 3f -BEGIN_FTR_SECTION - mfspr r5, SPRN_ASDR /* on POWER9, use ASDR to get VSID */ - b 4f -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) clrrdi r0, r4, 28 PPC_SLBFEE_DOT(R5, R0) /* if so, look up SLB */ li r0, BOOK3S_INTERRUPT_DATA_SEGMENT @@ -2012,10 +1763,6 @@ kvmppc_hisi: beq 1f andi. r0, r11, MSR_IR /* instruction relocation enabled? */ beq 3f -BEGIN_FTR_SECTION - mfspr r5, SPRN_ASDR /* on POWER9, use ASDR to get VSID */ - b 4f -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) clrrdi r0, r10, 28 PPC_SLBFEE_DOT(R5, R0) /* if so, look up SLB */ li r0, BOOK3S_INTERRUPT_INST_SEGMENT @@ -2063,10 +1810,6 @@ hcall_try_real_mode: andi. r0,r11,MSR_PR /* sc 1 from userspace - reflect to guest syscall */ bne sc_1_fast_return - /* sc 1 from nested guest - give it to L1 to handle */ - ld r0, VCPU_NESTED(r9) - cmpdi r0, 0 - bne guest_exit_cont clrrdi r3,r3,2 cmpldi r3,hcall_real_table_end - hcall_real_table bge guest_exit_cont @@ -2462,13 +2205,9 @@ _GLOBAL(kvmppc_h_cede) /* r3 = vcpu pointer, r11 = msr, r13 = paca */ bl kvmppc_save_fp #ifdef CONFIG_PPC_TRANSACTIONAL_MEM -/* - * Branch around the call if both CPU_FTR_TM and - * CPU_FTR_P9_TM_HV_ASSIST are off. - */ BEGIN_FTR_SECTION b 91f -END_FTR_SECTION(CPU_FTR_TM | CPU_FTR_P9_TM_HV_ASSIST, 0) +END_FTR_SECTION_IFCLR(CPU_FTR_TM) /* * NOTE THAT THIS TRASHES ALL NON-VOLATILE REGISTERS (but not CR) */ @@ -2488,15 +2227,8 @@ END_FTR_SECTION(CPU_FTR_TM | CPU_FTR_P9_TM_HV_ASSIST, 0) mfspr r3, SPRN_DEC mfspr r4, SPRN_HDEC mftb r5 -BEGIN_FTR_SECTION - /* On P9 check whether the guest has large decrementer mode enabled */ - ld r6, HSTATE_KVM_VCORE(r13) - ld r6, VCORE_LPCR(r6) - andis. r6, r6, LPCR_LD@h - bne 68f -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) extsw r3, r3 -68: EXTEND_HDEC(r4) + EXTEND_HDEC(r4) cmpd r3, r4 ble 67f mtspr SPRN_DEC, r4 @@ -2541,28 +2273,14 @@ BEGIN_FTR_SECTION END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S) kvm_nap_sequence: /* desired LPCR value in r5 */ -BEGIN_FTR_SECTION - /* - * PSSCR bits: exit criterion = 1 (wakeup based on LPCR at sreset) - * enable state loss = 1 (allow SMT mode switch) - * requested level = 0 (just stop dispatching) - */ - lis r3, (PSSCR_EC | PSSCR_ESL)@h /* Set LPCR_PECE_HVEE bit to enable wakeup by HV interrupts */ li r4, LPCR_PECE_HVEE@higher sldi r4, r4, 32 or r5, r5, r4 -FTR_SECTION_ELSE - li r3, PNV_THREAD_NAP -ALT_FTR_SECTION_END_IFSET(CPU_FTR_ARCH_300) mtspr SPRN_LPCR,r5 isync -BEGIN_FTR_SECTION - bl isa300_idle_stop_mayloss -FTR_SECTION_ELSE bl isa206_idle_insn_mayloss -ALT_FTR_SECTION_END_IFSET(CPU_FTR_ARCH_300) mfspr r0, SPRN_CTRLF ori r0, r0, 1 @@ -2581,10 +2299,8 @@ ALT_FTR_SECTION_END_IFSET(CPU_FTR_ARCH_300) beq kvm_end_cede cmpwi r0, NAPPING_NOVCPU beq kvm_novcpu_wakeup -BEGIN_FTR_SECTION cmpwi r0, NAPPING_UNSPLIT beq kvm_unsplit_wakeup -END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) twi 31,0,0 /* Nap state must not be zero */ 33: mr r4, r3 @@ -2604,13 +2320,9 @@ kvm_end_cede: #endif #ifdef CONFIG_PPC_TRANSACTIONAL_MEM -/* - * Branch around the call if both CPU_FTR_TM and - * CPU_FTR_P9_TM_HV_ASSIST are off. - */ BEGIN_FTR_SECTION b 91f -END_FTR_SECTION(CPU_FTR_TM | CPU_FTR_P9_TM_HV_ASSIST, 0) +END_FTR_SECTION_IFCLR(CPU_FTR_TM) /* * NOTE THAT THIS TRASHES ALL NON-VOLATILE REGISTERS (but not CR) */ @@ -2700,47 +2412,7 @@ kvm_cede_prodded: /* we've ceded but we want to give control to the host */ kvm_cede_exit: ld r9, HSTATE_KVM_VCPU(r13) -#ifdef CONFIG_KVM_XICS - /* are we using XIVE with single escalation? */ - ld r10, VCPU_XIVE_ESC_VADDR(r9) - cmpdi r10, 0 - beq 3f - li r6, XIVE_ESB_SET_PQ_00 - /* - * If we still have a pending escalation, abort the cede, - * and we must set PQ to 10 rather than 00 so that we don't - * potentially end up with two entries for the escalation - * interrupt in the XIVE interrupt queue. In that case - * we also don't want to set xive_esc_on to 1 here in - * case we race with xive_esc_irq(). - */ - lbz r5, VCPU_XIVE_ESC_ON(r9) - cmpwi r5, 0 - beq 4f - li r0, 0 - stb r0, VCPU_CEDED(r9) - /* - * The escalation interrupts are special as we don't EOI them. - * There is no need to use the load-after-store ordering offset - * to set PQ to 10 as we won't use StoreEOI. - */ - li r6, XIVE_ESB_SET_PQ_10 - b 5f -4: li r0, 1 - stb r0, VCPU_XIVE_ESC_ON(r9) - /* make sure store to xive_esc_on is seen before xive_esc_irq runs */ - sync -5: /* Enable XIVE escalation */ - mfmsr r0 - andi. r0, r0, MSR_DR /* in real mode? */ - beq 1f - ldx r0, r10, r6 - b 2f -1: ld r10, VCPU_XIVE_ESC_RADDR(r9) - ldcix r0, r10, r6 -2: sync -#endif /* CONFIG_KVM_XICS */ -3: b guest_exit_cont + b guest_exit_cont /* Try to do machine check recovery in real mode */ machine_check_realmode: @@ -2817,10 +2489,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S) PPC_MSGCLR(6) /* see if it's a host IPI */ li r3, 1 -BEGIN_FTR_SECTION - PPC_MSGSYNC - lwsync -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) lbz r0, HSTATE_HOST_IPI(r13) cmpwi r0, 0 bnelr @@ -3129,70 +2797,12 @@ kvmppc_bad_host_intr: std r3, STACK_FRAME_OVERHEAD-16(r1) /* - * On POWER9 do a minimal restore of the MMU and call C code, - * which will print a message and panic. * XXX On POWER7 and POWER8, we just spin here since we don't * know what the other threads are doing (and we don't want to * coordinate with them) - but at least we now have register state * in memory that we might be able to look at from another CPU. */ -BEGIN_FTR_SECTION b . -END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_300) - ld r9, HSTATE_KVM_VCPU(r13) - ld r10, VCPU_KVM(r9) - - li r0, 0 - mtspr SPRN_AMR, r0 - mtspr SPRN_IAMR, r0 - mtspr SPRN_CIABR, r0 - mtspr SPRN_DAWRX0, r0 -BEGIN_FTR_SECTION - mtspr SPRN_DAWRX1, r0 -END_FTR_SECTION_IFSET(CPU_FTR_DAWR1) - - /* Clear guest SLB. */ - slbmte r0, r0 - PPC_SLBIA(6) - ptesync - - /* load host SLB entries */ - ld r8, PACA_SLBSHADOWPTR(r13) - .rept SLB_NUM_BOLTED - li r3, SLBSHADOW_SAVEAREA - LDX_BE r5, r8, r3 - addi r3, r3, 8 - LDX_BE r6, r8, r3 - andis. r7, r5, SLB_ESID_V@h - beq 3f - slbmte r6, r5 -3: addi r8, r8, 16 - .endr - - lwz r7, KVM_HOST_LPID(r10) - mtspr SPRN_LPID, r7 - mtspr SPRN_PID, r0 - ld r8, KVM_HOST_LPCR(r10) - mtspr SPRN_LPCR, r8 - isync - li r0, KVM_GUEST_MODE_NONE - stb r0, HSTATE_IN_GUEST(r13) - - /* - * Turn on the MMU and jump to C code - */ - bcl 20, 31, .+4 -5: mflr r3 - addi r3, r3, 9f - 5b - li r4, -1 - rldimi r3, r4, 62, 0 /* ensure 0xc000000000000000 bits are set */ - ld r4, PACAKMSR(r13) - mtspr SPRN_SRR0, r3 - mtspr SPRN_SRR1, r4 - RFI_TO_KERNEL -9: addi r3, r1, STACK_FRAME_OVERHEAD - bl kvmppc_bad_interrupt - b 9b /* * This mimics the MSR transition on IRQ delivery. The new guest MSR is taken diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c index 999997d9e9a9..528a7e0cf83a 100644 --- a/arch/powerpc/platforms/powernv/idle.c +++ b/arch/powerpc/platforms/powernv/idle.c @@ -604,7 +604,7 @@ struct p9_sprs { u64 uamor; }; -static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on) +static unsigned long power9_idle_stop(unsigned long psscr) { int cpu = raw_smp_processor_id(); int first = cpu_first_thread_sibling(cpu); @@ -620,8 +620,6 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on) if (!(psscr & (PSSCR_EC|PSSCR_ESL))) { /* EC=ESL=0 case */ - BUG_ON(!mmu_on); - /* * Wake synchronously. SRESET via xscom may still cause * a 0x100 powersave wakeup with SRR1 reason! @@ -803,8 +801,7 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on) __slb_restore_bolted_realmode(); out: - if (mmu_on) - mtmsr(MSR_KERNEL); + mtmsr(MSR_KERNEL); return srr1; } @@ -895,7 +892,7 @@ struct p10_sprs { */ }; -static unsigned long power10_idle_stop(unsigned long psscr, bool mmu_on) +static unsigned long power10_idle_stop(unsigned long psscr) { int cpu = raw_smp_processor_id(); int first = cpu_first_thread_sibling(cpu); @@ -909,8 +906,6 @@ static unsigned long power10_idle_stop(unsigned long psscr, bool mmu_on) if (!(psscr & (PSSCR_EC|PSSCR_ESL))) { /* EC=ESL=0 case */ - BUG_ON(!mmu_on); - /* * Wake synchronously. SRESET via xscom may still cause * a 0x100 powersave wakeup with SRR1 reason! @@ -991,8 +986,7 @@ static unsigned long power10_idle_stop(unsigned long psscr, bool mmu_on) __slb_restore_bolted_realmode(); out: - if (mmu_on) - mtmsr(MSR_KERNEL); + mtmsr(MSR_KERNEL); return srr1; } @@ -1002,40 +996,10 @@ static unsigned long arch300_offline_stop(unsigned long psscr) { unsigned long srr1; -#ifndef CONFIG_KVM_BOOK3S_HV_POSSIBLE - __ppc64_runlatch_off(); if (cpu_has_feature(CPU_FTR_ARCH_31)) - srr1 = power10_idle_stop(psscr, true); + srr1 = power10_idle_stop(psscr); else - srr1 = power9_idle_stop(psscr, true); - __ppc64_runlatch_on(); -#else - /* - * Tell KVM we're entering idle. - * This does not have to be done in real mode because the P9 MMU - * is independent per-thread. Some steppings share radix/hash mode - * between threads, but in that case KVM has a barrier sync in real - * mode before and after switching between radix and hash. - * - * kvm_start_guest must still be called in real mode though, hence - * the false argument. - */ - local_paca->kvm_hstate.hwthread_state = KVM_HWTHREAD_IN_IDLE; - - __ppc64_runlatch_off(); - if (cpu_has_feature(CPU_FTR_ARCH_31)) - srr1 = power10_idle_stop(psscr, false); - else - srr1 = power9_idle_stop(psscr, false); - __ppc64_runlatch_on(); - - local_paca->kvm_hstate.hwthread_state = KVM_HWTHREAD_IN_KERNEL; - /* Order setting hwthread_state vs. testing hwthread_req */ - smp_mb(); - if (local_paca->kvm_hstate.hwthread_req) - srr1 = idle_kvm_start_guest(srr1); - mtmsr(MSR_KERNEL); -#endif + srr1 = power9_idle_stop(psscr); return srr1; } @@ -1055,9 +1019,9 @@ void arch300_idle_type(unsigned long stop_psscr_val, __ppc64_runlatch_off(); if (cpu_has_feature(CPU_FTR_ARCH_31)) - srr1 = power10_idle_stop(psscr, true); + srr1 = power10_idle_stop(psscr); else - srr1 = power9_idle_stop(psscr, true); + srr1 = power9_idle_stop(psscr); __ppc64_runlatch_on(); fini_irq_for_idle_irqsoff();