{"id":831067,"url":"http://patchwork.ozlabs.org/api/1.2/patches/831067/?format=json","web_url":"http://patchwork.ozlabs.org/project/kvm-ppc/patch/1509079594-28977-5-git-send-email-paulus@ozlabs.org/","project":{"id":23,"url":"http://patchwork.ozlabs.org/api/1.2/projects/23/?format=json","name":"KVM PowerPC development","link_name":"kvm-ppc","list_id":"kvm-ppc.vger.kernel.org","list_email":"kvm-ppc@vger.kernel.org","web_url":null,"scm_url":null,"webscm_url":null,"list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<1509079594-28977-5-git-send-email-paulus@ozlabs.org>","list_archive_url":null,"date":"2017-10-27T04:46:32","name":"[v2,4/6] KVM: PPC: Book3S HV: Add infrastructure for running HPT guests on radix host","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"6ad348a15ad1adbdc37fdbf857f3b6b9f4966536","submitter":{"id":67079,"url":"http://patchwork.ozlabs.org/api/1.2/people/67079/?format=json","name":"Paul Mackerras","email":"paulus@ozlabs.org"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/kvm-ppc/patch/1509079594-28977-5-git-send-email-paulus@ozlabs.org/mbox/","series":[{"id":10490,"url":"http://patchwork.ozlabs.org/api/1.2/series/10490/?format=json","web_url":"http://patchwork.ozlabs.org/project/kvm-ppc/list/?series=10490","date":"2017-10-27T04:46:28","name":"KVM: PPC: Book3S HV: Run HPT guests on radix hosts","version":2,"mbox":"http://patchwork.ozlabs.org/series/10490/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/831067/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/831067/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=kvm-ppc-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"rQbwZiRk\";\n\tdkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yNWZ10drhz9t30\n\tfor ;\n\tFri, 27 Oct 2017 15:47:05 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751961AbdJ0Eqz (ORCPT );\n\tFri, 27 Oct 2017 00:46:55 -0400","from ozlabs.org ([103.22.144.67]:56007 \"EHLO ozlabs.org\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1751158AbdJ0Eqs (ORCPT );\n\tFri, 27 Oct 2017 00:46:48 -0400","from authenticated.ozlabs.org (localhost [127.0.0.1])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPSA id 3yNWYc6yWfz9t3F;\n\tFri, 27 Oct 2017 15:46:44 +1100 (AEDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ozlabs.org; s=201707; \n\tt=1509079605; bh=y+pPZQsDgnJpJndDpWjjbXmC7JMAKoxBi3XS5CZLXjU=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=rQbwZiRkOojRkip4q4eUbswsb/dl9tA2oZ6IUfqT1twHWO0aFFG6uqSkKRytXpSxH\n\tVI4arRv3wYtXQ3b6Jrvg5z3lwo//3wLz/A9JS3lnk3WutWicQk4Go9z7kTxt6vzzIG\n\tFgpLmt4z4o7Cloxr4Q8oyffGNz+L9pKMDIRxMmruW2MwkRWQCpJfOlbJwRGzp3dR4n\n\tvgKwgjN0gwqB1dnrXEg43YLXsD4tZi5kcCKYXmHfHarAc4A7A0o2oaVMeHazi0SKBD\n\t+Sl0kQ5AWiHlJft9mbmCoq2cJ41Qio7IZFA9+FjGFSslA9rvWaYP8JgOcJT4RsdRSA\n\tpwJjsZgnkXdYw==","From":"Paul Mackerras ","To":"kvm@vger.kernel.org, kvm-ppc@vger.kernel.org","Cc":"david@gibson.dropbear.id.au","Subject":"[PATCH v2 4/6] KVM: PPC: Book3S HV: Add infrastructure for running\n\tHPT guests on radix host","Date":"Fri, 27 Oct 2017 15:46:32 +1100","Message-Id":"<1509079594-28977-5-git-send-email-paulus@ozlabs.org>","X-Mailer":"git-send-email 2.7.4","In-Reply-To":"<1509079594-28977-1-git-send-email-paulus@ozlabs.org>","References":"<1509079594-28977-1-git-send-email-paulus@ozlabs.org>","Sender":"kvm-ppc-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"kvm-ppc@vger.kernel.org"},"content":"This sets up the machinery for switching a guest between HPT (hashed\npage table) and radix MMU modes, so that in future we can run a HPT\nguest on a radix host on POWER9 machines.\n\n* The KVM_PPC_CONFIGURE_V3_MMU ioctl can now specify either HPT or\n radix mode, on a radix host.\n\n* The KVM_CAP_PPC_MMU_HASH_V3 capability now returns 1 on POWER9\n with HV KVM on a radix host.\n\n* The KVM_PPC_GET_SMMU_INFO returns information about the HPT MMU on a\n radix host.\n\n* The KVM_PPC_ALLOCATE_HTAB ioctl on a radix host will switch the\n guest to HPT mode and allocate a HPT.\n\n* For simplicity, we now allocate the rmap array for each memslot,\n even on a radix host, since it will be needed if the guest switches\n to HPT mode.\n\n* Since we cannot yet run a HPT guest on a radix host, the KVM_RUN\n ioctl will return an EINVAL error in that case.\n\nSigned-off-by: Paul Mackerras \n---\n arch/powerpc/include/asm/kvm_ppc.h | 3 ++\n arch/powerpc/kvm/book3s_64_mmu_hv.c | 22 +++++----\n arch/powerpc/kvm/book3s_64_mmu_radix.c | 1 +\n arch/powerpc/kvm/book3s_hv.c | 87 ++++++++++++++++++++++++----------\n arch/powerpc/kvm/powerpc.c | 3 +-\n 5 files changed, 79 insertions(+), 37 deletions(-)","diff":"diff --git a/arch/powerpc/include/asm/kvm_ppc.h b/arch/powerpc/include/asm/kvm_ppc.h\nindex ba5fadd..96753f3 100644\n--- a/arch/powerpc/include/asm/kvm_ppc.h\n+++ b/arch/powerpc/include/asm/kvm_ppc.h\n@@ -168,6 +168,7 @@ extern int kvmppc_allocate_hpt(struct kvm_hpt_info *info, u32 order);\n extern void kvmppc_set_hpt(struct kvm *kvm, struct kvm_hpt_info *info);\n extern long kvmppc_alloc_reset_hpt(struct kvm *kvm, int order);\n extern void kvmppc_free_hpt(struct kvm_hpt_info *info);\n+extern void kvmppc_rmap_reset(struct kvm *kvm);\n extern long kvmppc_prepare_vrma(struct kvm *kvm,\n \t\t\t\tstruct kvm_userspace_memory_region *mem);\n extern void kvmppc_map_vrma(struct kvm_vcpu *vcpu,\n@@ -177,6 +178,8 @@ extern long kvm_spapr_tce_attach_iommu_group(struct kvm *kvm, int tablefd,\n \t\tstruct iommu_group *grp);\n extern void kvm_spapr_tce_release_iommu_group(struct kvm *kvm,\n \t\tstruct iommu_group *grp);\n+extern int kvmppc_switch_mmu_to_hpt(struct kvm *kvm);\n+extern int kvmppc_switch_mmu_to_radix(struct kvm *kvm);\n \n extern long kvm_vm_ioctl_create_spapr_tce(struct kvm *kvm,\n \t\t\t\tstruct kvm_create_spapr_tce_64 *args);\ndiff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c\nindex 944f7a5..6aec8a2 100644\n--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c\n+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c\n@@ -73,8 +73,6 @@ struct kvm_resize_hpt {\n \tstruct kvm_hpt_info hpt;\n };\n \n-static void kvmppc_rmap_reset(struct kvm *kvm);\n-\n int kvmppc_allocate_hpt(struct kvm_hpt_info *info, u32 order)\n {\n \tunsigned long hpt = 0;\n@@ -136,9 +134,6 @@ long kvmppc_alloc_reset_hpt(struct kvm *kvm, int order)\n \tlong err = -EBUSY;\n \tstruct kvm_hpt_info info;\n \n-\tif (kvm_is_radix(kvm))\n-\t\treturn -EINVAL;\n-\n \tmutex_lock(&kvm->lock);\n \tif (kvm->arch.mmu_ready) {\n \t\tkvm->arch.mmu_ready = 0;\n@@ -149,6 +144,12 @@ long kvmppc_alloc_reset_hpt(struct kvm *kvm, int order)\n \t\t\tgoto out;\n \t\t}\n \t}\n+\tif (kvm_is_radix(kvm)) {\n+\t\terr = kvmppc_switch_mmu_to_hpt(kvm);\n+\t\tif (err)\n+\t\t\tgoto out;\n+\t}\n+\n \tif (kvm->arch.hpt.order == order) {\n \t\t/* We already have a suitable HPT */\n \n@@ -182,6 +183,7 @@ long kvmppc_alloc_reset_hpt(struct kvm *kvm, int order)\n void kvmppc_free_hpt(struct kvm_hpt_info *info)\n {\n \tvfree(info->rev);\n+\tinfo->rev = NULL;\n \tif (info->cma)\n \t\tkvm_free_hpt_cma(virt_to_page(info->virt),\n \t\t\t\t 1 << (info->order - PAGE_SHIFT));\n@@ -349,6 +351,9 @@ static int kvmppc_mmu_book3s_64_hv_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,\n \tint index;\n \tint virtmode = vcpu->arch.shregs.msr & (data ? MSR_DR : MSR_IR);\n \n+\tif (kvm_is_radix(vcpu->kvm))\n+\t\treturn kvmppc_mmu_radix_xlate(vcpu, eaddr, gpte, data, iswrite);\n+\n \t/* Get SLB entry */\n \tif (virtmode) {\n \t\tslbe = kvmppc_mmu_book3s_hv_find_slbe(vcpu, eaddr);\n@@ -710,7 +715,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,\n \tgoto out_put;\n }\n \n-static void kvmppc_rmap_reset(struct kvm *kvm)\n+void kvmppc_rmap_reset(struct kvm *kvm)\n {\n \tstruct kvm_memslots *slots;\n \tstruct kvm_memory_slot *memslot;\n@@ -2089,10 +2094,7 @@ void kvmppc_mmu_book3s_hv_init(struct kvm_vcpu *vcpu)\n \n \tvcpu->arch.slb_nr = 32;\t\t/* POWER7/POWER8 */\n \n-\tif (kvm_is_radix(vcpu->kvm))\n-\t\tmmu->xlate = kvmppc_mmu_radix_xlate;\n-\telse\n-\t\tmmu->xlate = kvmppc_mmu_book3s_64_hv_xlate;\n+\tmmu->xlate = kvmppc_mmu_book3s_64_hv_xlate;\n \tmmu->reset_msr = kvmppc_mmu_book3s_64_hv_reset_msr;\n \n \tvcpu->arch.hflags |= BOOK3S_HFLAG_SLB;\ndiff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c\nindex 6336b13..58618f6 100644\n--- a/arch/powerpc/kvm/book3s_64_mmu_radix.c\n+++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c\n@@ -662,6 +662,7 @@ void kvmppc_free_radix(struct kvm *kvm)\n \t\tpgd_clear(pgd);\n \t}\n \tpgd_free(kvm->mm, kvm->arch.pgtable);\n+\tkvm->arch.pgtable = NULL;\n }\n \n static void pte_ctor(void *addr)\ndiff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c\nindex 96b92d4..040e102 100644\n--- a/arch/powerpc/kvm/book3s_hv.c\n+++ b/arch/powerpc/kvm/book3s_hv.c\n@@ -2442,6 +2442,9 @@ static void prepare_threads(struct kvmppc_vcore *vc)\n \tfor_each_runnable_thread(i, vcpu, vc) {\n \t\tif (signal_pending(vcpu->arch.run_task))\n \t\t\tvcpu->arch.ret = -EINTR;\n+\t\telse if (kvm_is_radix(vc->kvm) != radix_enabled())\n+\t\t\t/* can't actually run HPT guest on radix host yet... */\n+\t\t\tvcpu->arch.ret = -EINVAL;\n \t\telse if (vcpu->arch.vpa.update_pending ||\n \t\t\t vcpu->arch.slb_shadow.update_pending ||\n \t\t\t vcpu->arch.dtl.update_pending)\n@@ -3339,13 +3342,6 @@ static int kvm_vm_ioctl_get_smmu_info_hv(struct kvm *kvm,\n \tstruct kvm_ppc_one_seg_page_size *sps;\n \n \t/*\n-\t * Since we don't yet support HPT guests on a radix host,\n-\t * return an error if the host uses radix.\n-\t */\n-\tif (radix_enabled())\n-\t\treturn -EINVAL;\n-\n-\t/*\n \t * POWER7, POWER8 and POWER9 all support 32 storage keys for data.\n \t * POWER7 doesn't support keys for instruction accesses,\n \t * POWER8 and POWER9 do.\n@@ -3447,15 +3443,6 @@ static void kvmppc_core_free_memslot_hv(struct kvm_memory_slot *free,\n static int kvmppc_core_create_memslot_hv(struct kvm_memory_slot *slot,\n \t\t\t\t\t unsigned long npages)\n {\n-\t/*\n-\t * For now, if radix_enabled() then we only support radix guests,\n-\t * and in that case we don't need the rmap array.\n-\t */\n-\tif (radix_enabled()) {\n-\t\tslot->arch.rmap = NULL;\n-\t\treturn 0;\n-\t}\n-\n \tslot->arch.rmap = vzalloc(npages * sizeof(*slot->arch.rmap));\n \tif (!slot->arch.rmap)\n \t\treturn -ENOMEM;\n@@ -3628,6 +3615,34 @@ static int kvmppc_hv_setup_htab_rma(struct kvm_vcpu *vcpu)\n \tgoto out_srcu;\n }\n \n+/* Must be called with kvm->lock held and mmu_ready = 0 and no vcpus running */\n+int kvmppc_switch_mmu_to_hpt(struct kvm *kvm)\n+{\n+\tkvmppc_free_radix(kvm);\n+\tkvmppc_update_lpcr(kvm, LPCR_VPM1,\n+\t\t\t LPCR_VPM1 | LPCR_UPRT | LPCR_GTSE | LPCR_HR);\n+\tkvmppc_rmap_reset(kvm);\n+\tkvm->arch.radix = 0;\n+\tkvm->arch.process_table = 0;\n+\treturn 0;\n+}\n+\n+/* Must be called with kvm->lock held and mmu_ready = 0 and no vcpus running */\n+int kvmppc_switch_mmu_to_radix(struct kvm *kvm)\n+{\n+\tint err;\n+\n+\terr = kvmppc_init_vm_radix(kvm);\n+\tif (err)\n+\t\treturn err;\n+\n+\tkvmppc_free_hpt(&kvm->arch.hpt);\n+\tkvmppc_update_lpcr(kvm, LPCR_UPRT | LPCR_GTSE | LPCR_HR,\n+\t\t\t LPCR_VPM1 | LPCR_UPRT | LPCR_GTSE | LPCR_HR);\n+\tkvm->arch.radix = 1;\n+\treturn 0;\n+}\n+\n #ifdef CONFIG_KVM_XICS\n /*\n * Allocate a per-core structure for managing state about which cores are\n@@ -3771,7 +3786,7 @@ static int kvmppc_core_init_vm_hv(struct kvm *kvm)\n \t}\n \n \t/*\n-\t * For now, if the host uses radix, the guest must be radix.\n+\t * If the host uses radix, the guest starts out as radix.\n \t */\n \tif (radix_enabled()) {\n \t\tkvm->arch.radix = 1;\n@@ -3795,7 +3810,7 @@ static int kvmppc_core_init_vm_hv(struct kvm *kvm)\n \t * Work out how many sets the TLB has, for the use of\n \t * the TLB invalidation loop in book3s_hv_rmhandlers.S.\n \t */\n-\tif (kvm_is_radix(kvm))\n+\tif (radix_enabled())\n \t\tkvm->arch.tlb_sets = POWER9_TLB_SETS_RADIX;\t/* 128 */\n \telse if (cpu_has_feature(CPU_FTR_ARCH_300))\n \t\tkvm->arch.tlb_sets = POWER9_TLB_SETS_HASH;\t/* 256 */\n@@ -4185,6 +4200,7 @@ static int kvmhv_configure_mmu(struct kvm *kvm, struct kvm_ppc_mmuv3_cfg *cfg)\n {\n \tunsigned long lpcr;\n \tint radix;\n+\tint err;\n \n \t/* If not on a POWER9, reject it */\n \tif (!cpu_has_feature(CPU_FTR_ARCH_300))\n@@ -4194,12 +4210,8 @@ static int kvmhv_configure_mmu(struct kvm *kvm, struct kvm_ppc_mmuv3_cfg *cfg)\n \tif (cfg->flags & ~(KVM_PPC_MMUV3_RADIX | KVM_PPC_MMUV3_GTSE))\n \t\treturn -EINVAL;\n \n-\t/* We can't change a guest to/from radix yet */\n-\tradix = !!(cfg->flags & KVM_PPC_MMUV3_RADIX);\n-\tif (radix != kvm_is_radix(kvm))\n-\t\treturn -EINVAL;\n-\n \t/* GR (guest radix) bit in process_table field must match */\n+\tradix = !!(cfg->flags & KVM_PPC_MMUV3_RADIX);\n \tif (!!(cfg->process_table & PATB_GR) != radix)\n \t\treturn -EINVAL;\n \n@@ -4207,15 +4219,40 @@ static int kvmhv_configure_mmu(struct kvm *kvm, struct kvm_ppc_mmuv3_cfg *cfg)\n \tif ((cfg->process_table & PRTS_MASK) > 24)\n \t\treturn -EINVAL;\n \n+\t/* We can change a guest to/from radix now, if the host is radix */\n+\tif (radix && !radix_enabled())\n+\t\treturn -EINVAL;\n+\n \tmutex_lock(&kvm->lock);\n+\tif (radix != kvm_is_radix(kvm)) {\n+\t\tif (kvm->arch.mmu_ready) {\n+\t\t\tkvm->arch.mmu_ready = 0;\n+\t\t\t/* order mmu_ready vs. vcpus_running */\n+\t\t\tsmp_mb();\n+\t\t\tif (atomic_read(&kvm->arch.vcpus_running)) {\n+\t\t\t\tkvm->arch.mmu_ready = 1;\n+\t\t\t\terr = -EBUSY;\n+\t\t\t\tgoto out_unlock;\n+\t\t\t}\n+\t\t}\n+\t\tif (radix)\n+\t\t\terr = kvmppc_switch_mmu_to_radix(kvm);\n+\t\telse\n+\t\t\terr = kvmppc_switch_mmu_to_hpt(kvm);\n+\t\tif (err)\n+\t\t\tgoto out_unlock;\n+\t}\n+\n \tkvm->arch.process_table = cfg->process_table;\n \tkvmppc_setup_partition_table(kvm);\n \n \tlpcr = (cfg->flags & KVM_PPC_MMUV3_GTSE) ? LPCR_GTSE : 0;\n \tkvmppc_update_lpcr(kvm, lpcr, LPCR_GTSE);\n-\tmutex_unlock(&kvm->lock);\n+\terr = 0;\n \n-\treturn 0;\n+ out_unlock:\n+\tmutex_unlock(&kvm->lock);\n+\treturn err;\n }\n \n static struct kvmppc_ops kvm_ops_hv = {\ndiff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c\nindex a3746b9..a0b7f09 100644\n--- a/arch/powerpc/kvm/powerpc.c\n+++ b/arch/powerpc/kvm/powerpc.c\n@@ -590,8 +590,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)\n \t\tr = !!(hv_enabled && radix_enabled());\n \t\tbreak;\n \tcase KVM_CAP_PPC_MMU_HASH_V3:\n-\t\tr = !!(hv_enabled && !radix_enabled() &&\n-\t\t cpu_has_feature(CPU_FTR_ARCH_300));\n+\t\tr = !!(hv_enabled && cpu_has_feature(CPU_FTR_ARCH_300));\n \t\tbreak;\n #endif\n \tcase KVM_CAP_SYNC_MMU:\n","prefixes":["v2","4/6"]}