{"id":831065,"url":"http://patchwork.ozlabs.org/api/1.2/patches/831065/?format=json","web_url":"http://patchwork.ozlabs.org/project/kvm-ppc/patch/1509079594-28977-4-git-send-email-paulus@ozlabs.org/","project":{"id":23,"url":"http://patchwork.ozlabs.org/api/1.2/projects/23/?format=json","name":"KVM PowerPC development","link_name":"kvm-ppc","list_id":"kvm-ppc.vger.kernel.org","list_email":"kvm-ppc@vger.kernel.org","web_url":null,"scm_url":null,"webscm_url":null,"list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<1509079594-28977-4-git-send-email-paulus@ozlabs.org>","list_archive_url":null,"date":"2017-10-27T04:46:31","name":"[v2,3/6] KVM: PPC: Book3S HV: Unify dirty page map between HPT and radix","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"74ede252e1f1f5ade4f1683a81c8b66fba84b564","submitter":{"id":67079,"url":"http://patchwork.ozlabs.org/api/1.2/people/67079/?format=json","name":"Paul Mackerras","email":"paulus@ozlabs.org"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/kvm-ppc/patch/1509079594-28977-4-git-send-email-paulus@ozlabs.org/mbox/","series":[{"id":10490,"url":"http://patchwork.ozlabs.org/api/1.2/series/10490/?format=json","web_url":"http://patchwork.ozlabs.org/project/kvm-ppc/list/?series=10490","date":"2017-10-27T04:46:28","name":"KVM: PPC: Book3S HV: Run HPT guests on radix hosts","version":2,"mbox":"http://patchwork.ozlabs.org/series/10490/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/831065/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/831065/checks/","tags":{},"related":[],"headers":{"Return-Path":"<kvm-ppc-owner@vger.kernel.org>","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=kvm-ppc-owner@vger.kernel.org;\n\treceiver=<UNKNOWN>)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"TVj+pRlR\";\n\tdkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3yNWYx5jFXz9t34\n\tfor <incoming@patchwork.ozlabs.org>;\n\tFri, 27 Oct 2017 15:47:01 +1100 (AEDT)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751675AbdJ0Eq6 (ORCPT <rfc822;incoming@patchwork.ozlabs.org>);\n\tFri, 27 Oct 2017 00:46:58 -0400","from ozlabs.org ([103.22.144.67]:35921 \"EHLO ozlabs.org\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1751530AbdJ0Eqq (ORCPT <rfc822;kvm-ppc@vger.kernel.org>);\n\tFri, 27 Oct 2017 00:46:46 -0400","from authenticated.ozlabs.org (localhost [127.0.0.1])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPSA id 3yNWYc4pSyz9t39;\n\tFri, 27 Oct 2017 15:46:44 +1100 (AEDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ozlabs.org; s=201707; \n\tt=1509079604; bh=vaw3cmqVoUr9GjkOutFtwLAjmpvWVu9eHWmvhEHJXdQ=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=TVj+pRlR0uUuZ4R1Ok831mjk+ARfr3AcmpaOLRBH22PNOyPUO0jjUVvWx2tVBkSM+\n\tVVVAQxIXPNCa7cxcd3jTz/l6Tr+DzNcXvN77qDd0xjjKxsZRaLixXTdwgm3WucomAU\n\ttYFjjqEL2S4eC19ku/JX+NjnCAKQQuXsl/G9ok+ni9lGLrjMLfRMFwTiFOE30ZG8zt\n\tluIKRL4dDR+tRfFJZzixTz8iLutYMqe9HrAwLR2qIYangaatJ3jc0xicS9juahMjQK\n\toZyEXY+lQ7qwcyatBBVQZGOwY8jZyIXKpC5iUBe3GuBDD0wRueXalYip74Eq5WQ8n0\n\tOYBl5E5nQW7hA==","From":"Paul Mackerras <paulus@ozlabs.org>","To":"kvm@vger.kernel.org, kvm-ppc@vger.kernel.org","Cc":"david@gibson.dropbear.id.au","Subject":"[PATCH v2 3/6] KVM: PPC: Book3S HV: Unify dirty page map between\n\tHPT and radix","Date":"Fri, 27 Oct 2017 15:46:31 +1100","Message-Id":"<1509079594-28977-4-git-send-email-paulus@ozlabs.org>","X-Mailer":"git-send-email 2.7.4","In-Reply-To":"<1509079594-28977-1-git-send-email-paulus@ozlabs.org>","References":"<1509079594-28977-1-git-send-email-paulus@ozlabs.org>","Sender":"kvm-ppc-owner@vger.kernel.org","Precedence":"bulk","List-ID":"<kvm-ppc.vger.kernel.org>","X-Mailing-List":"kvm-ppc@vger.kernel.org"},"content":"Currently, the HPT code in HV KVM maintains a dirty bit per guest page\nin the rmap array, whether or not dirty page tracking has been enabled\nfor the memory slot.  In contrast, the radix code maintains a dirty\nbit per guest page in memslot->dirty_bitmap, and only does so when\ndirty page tracking has been enabled.\n\nThis changes the HPT code to maintain the dirty bits in the memslot\ndirty_bitmap like radix does.  This results in slightly less code\noverall, and will mean that we do not lose the dirty bits when\ntransitioning between HPT and radix mode in future.\n\nThere is one minor change to behaviour as a result.  With HPT, when\ndirty tracking was enabled for a memslot, we would previously clear\nall the dirty bits at that point (both in the HPT entries and in the\nrmap arrays), meaning that a KVM_GET_DIRTY_LOG ioctl immediately\nfollowing would show no pages as dirty (assuming no vcpus have run\nin the meantime).  With this change, the dirty bits on HPT entries\nare not cleared at the point where dirty tracking is enabled, so\nKVM_GET_DIRTY_LOG would show as dirty any guest pages that are\nresident in the HPT and dirty.  This is consistent with what happens\non radix.\n\nThis also fixes a bug in the mark_pages_dirty() function for radix\n(in the sense that the function no longer exists).  In the case where\na large page of 64 normal pages or more is marked dirty, the\naddressing of the dirty bitmap was incorrect and could write past\nthe end of the bitmap.  Fortunately this case was never hit in\npractice because a 2MB large page is only 32 x 64kB pages, and we\ndon't support backing the guest with 1GB huge pages at this point.\n\nSigned-off-by: Paul Mackerras <paulus@ozlabs.org>\n---\n arch/powerpc/include/asm/kvm_book3s.h    |  3 +-\n arch/powerpc/include/asm/kvm_book3s_64.h | 24 +++++++++++++\n arch/powerpc/include/asm/kvm_host.h      |  3 --\n arch/powerpc/kvm/book3s_64_mmu_hv.c      | 39 ++++++---------------\n arch/powerpc/kvm/book3s_64_mmu_radix.c   | 50 +++++---------------------\n arch/powerpc/kvm/book3s_hv.c             | 30 +++++++---------\n arch/powerpc/kvm/book3s_hv_rm_mmu.c      | 60 ++++++++++++++++++++------------\n 7 files changed, 96 insertions(+), 113 deletions(-)","diff":"diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h\nindex b8d5b8e..9a66700 100644\n--- a/arch/powerpc/include/asm/kvm_book3s.h\n+++ b/arch/powerpc/include/asm/kvm_book3s.h\n@@ -216,7 +216,8 @@ extern kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gpa_t gpa,\n \t\t\tbool writing, bool *writable);\n extern void kvmppc_add_revmap_chain(struct kvm *kvm, struct revmap_entry *rev,\n \t\t\tunsigned long *rmap, long pte_index, int realmode);\n-extern void kvmppc_update_rmap_change(unsigned long *rmap, unsigned long psize);\n+extern void kvmppc_update_dirty_map(struct kvm_memory_slot *memslot,\n+\t\t\tunsigned long gfn, unsigned long psize);\n extern void kvmppc_invalidate_hpte(struct kvm *kvm, __be64 *hptep,\n \t\t\tunsigned long pte_index);\n void kvmppc_clear_ref_hpte(struct kvm *kvm, __be64 *hptep,\ndiff --git a/arch/powerpc/include/asm/kvm_book3s_64.h b/arch/powerpc/include/asm/kvm_book3s_64.h\nindex b21936c..735cfa3 100644\n--- a/arch/powerpc/include/asm/kvm_book3s_64.h\n+++ b/arch/powerpc/include/asm/kvm_book3s_64.h\n@@ -20,6 +20,8 @@\n #ifndef __ASM_KVM_BOOK3S_64_H__\n #define __ASM_KVM_BOOK3S_64_H__\n \n+#include <linux/string.h>\n+#include <asm/bitops.h>\n #include <asm/book3s/64/mmu-hash.h>\n \n /* Power architecture requires HPT is at least 256kiB, at most 64TiB */\n@@ -444,6 +446,28 @@ static inline unsigned long kvmppc_hpt_mask(struct kvm_hpt_info *hpt)\n \treturn (1UL << (hpt->order - 7)) - 1;\n }\n \n+/* Set bits in a dirty bitmap, which is in LE format */\n+static inline void set_dirty_bits(unsigned long *map, unsigned long i,\n+\t\t\t\t  unsigned long npages)\n+{\n+\n+\tif (npages >= 8)\n+\t\tmemset((char *)map + i / 8, 0xff, npages / 8);\n+\telse\n+\t\tfor (; npages; ++i, --npages)\n+\t\t\t__set_bit_le(i, map);\n+}\n+\n+static inline void set_dirty_bits_atomic(unsigned long *map, unsigned long i,\n+\t\t\t\t\t unsigned long npages)\n+{\n+\tif (npages >= 8)\n+\t\tmemset((char *)map + i / 8, 0xff, npages / 8);\n+\telse\n+\t\tfor (; npages; ++i, --npages)\n+\t\t\tset_bit_le(i, map);\n+}\n+\n #endif /* CONFIG_KVM_BOOK3S_HV_POSSIBLE */\n \n #endif /* __ASM_KVM_BOOK3S_64_H__ */\ndiff --git a/arch/powerpc/include/asm/kvm_host.h b/arch/powerpc/include/asm/kvm_host.h\nindex 49493ea..d831a38 100644\n--- a/arch/powerpc/include/asm/kvm_host.h\n+++ b/arch/powerpc/include/asm/kvm_host.h\n@@ -235,10 +235,7 @@ struct revmap_entry {\n  */\n #define KVMPPC_RMAP_LOCK_BIT\t63\n #define KVMPPC_RMAP_RC_SHIFT\t32\n-#define KVMPPC_RMAP_CHG_SHIFT\t48\n #define KVMPPC_RMAP_REFERENCED\t(HPTE_R_R << KVMPPC_RMAP_RC_SHIFT)\n-#define KVMPPC_RMAP_CHANGED\t(HPTE_R_C << KVMPPC_RMAP_RC_SHIFT)\n-#define KVMPPC_RMAP_CHG_ORDER\t(0x3ful << KVMPPC_RMAP_CHG_SHIFT)\n #define KVMPPC_RMAP_PRESENT\t0x100000000ul\n #define KVMPPC_RMAP_INDEX\t0xfffffffful\n \ndiff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c\nindex 8472803..944f7a5 100644\n--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c\n+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c\n@@ -776,6 +776,7 @@ static int kvm_handle_hva(struct kvm *kvm, unsigned long hva,\n \n /* Must be called with both HPTE and rmap locked */\n static void kvmppc_unmap_hpte(struct kvm *kvm, unsigned long i,\n+\t\t\t      struct kvm_memory_slot *memslot,\n \t\t\t      unsigned long *rmapp, unsigned long gfn)\n {\n \t__be64 *hptep = (__be64 *) (kvm->arch.hpt.virt + (i << 4));\n@@ -807,8 +808,8 @@ static void kvmppc_unmap_hpte(struct kvm *kvm, unsigned long i,\n \t\t/* Harvest R and C */\n \t\trcbits = be64_to_cpu(hptep[1]) & (HPTE_R_R | HPTE_R_C);\n \t\t*rmapp |= rcbits << KVMPPC_RMAP_RC_SHIFT;\n-\t\tif (rcbits & HPTE_R_C)\n-\t\t\tkvmppc_update_rmap_change(rmapp, psize);\n+\t\tif ((rcbits & HPTE_R_C) && memslot->dirty_bitmap)\n+\t\t\tkvmppc_update_dirty_map(memslot, gfn, psize);\n \t\tif (rcbits & ~rev[i].guest_rpte) {\n \t\t\trev[i].guest_rpte = ptel | rcbits;\n \t\t\tnote_hpte_modification(kvm, &rev[i]);\n@@ -846,7 +847,7 @@ static int kvm_unmap_rmapp(struct kvm *kvm, struct kvm_memory_slot *memslot,\n \t\t\tcontinue;\n \t\t}\n \n-\t\tkvmppc_unmap_hpte(kvm, i, rmapp, gfn);\n+\t\tkvmppc_unmap_hpte(kvm, i, memslot, rmapp, gfn);\n \t\tunlock_rmap(rmapp);\n \t\t__unlock_hpte(hptep, be64_to_cpu(hptep[0]));\n \t}\n@@ -1029,14 +1030,6 @@ static int kvm_test_clear_dirty_npages(struct kvm *kvm, unsigned long *rmapp)\n \n  retry:\n \tlock_rmap(rmapp);\n-\tif (*rmapp & KVMPPC_RMAP_CHANGED) {\n-\t\tlong change_order = (*rmapp & KVMPPC_RMAP_CHG_ORDER)\n-\t\t\t>> KVMPPC_RMAP_CHG_SHIFT;\n-\t\t*rmapp &= ~(KVMPPC_RMAP_CHANGED | KVMPPC_RMAP_CHG_ORDER);\n-\t\tnpages_dirty = 1;\n-\t\tif (change_order > PAGE_SHIFT)\n-\t\t\tnpages_dirty = 1ul << (change_order - PAGE_SHIFT);\n-\t}\n \tif (!(*rmapp & KVMPPC_RMAP_PRESENT)) {\n \t\tunlock_rmap(rmapp);\n \t\treturn npages_dirty;\n@@ -1128,7 +1121,7 @@ void kvmppc_harvest_vpa_dirty(struct kvmppc_vpa *vpa,\n long kvmppc_hv_get_dirty_log_hpt(struct kvm *kvm,\n \t\t\tstruct kvm_memory_slot *memslot, unsigned long *map)\n {\n-\tunsigned long i, j;\n+\tunsigned long i;\n \tunsigned long *rmapp;\n \n \tpreempt_disable();\n@@ -1140,9 +1133,8 @@ long kvmppc_hv_get_dirty_log_hpt(struct kvm *kvm,\n \t\t * since we always put huge-page HPTEs in the rmap chain\n \t\t * corresponding to their page base address.\n \t\t */\n-\t\tif (npages && map)\n-\t\t\tfor (j = i; npages; ++j, --npages)\n-\t\t\t\t__set_bit_le(j, map);\n+\t\tif (npages)\n+\t\t\tset_dirty_bits(map, i, npages);\n \t\t++rmapp;\n \t}\n \tpreempt_enable();\n@@ -1186,7 +1178,6 @@ void kvmppc_unpin_guest_page(struct kvm *kvm, void *va, unsigned long gpa,\n \tstruct page *page = virt_to_page(va);\n \tstruct kvm_memory_slot *memslot;\n \tunsigned long gfn;\n-\tunsigned long *rmap;\n \tint srcu_idx;\n \n \tput_page(page);\n@@ -1194,20 +1185,12 @@ void kvmppc_unpin_guest_page(struct kvm *kvm, void *va, unsigned long gpa,\n \tif (!dirty)\n \t\treturn;\n \n-\t/* We need to mark this page dirty in the rmap chain */\n+\t/* We need to mark this page dirty in the memslot dirty_bitmap, if any */\n \tgfn = gpa >> PAGE_SHIFT;\n \tsrcu_idx = srcu_read_lock(&kvm->srcu);\n \tmemslot = gfn_to_memslot(kvm, gfn);\n-\tif (memslot) {\n-\t\tif (!kvm_is_radix(kvm)) {\n-\t\t\trmap = &memslot->arch.rmap[gfn - memslot->base_gfn];\n-\t\t\tlock_rmap(rmap);\n-\t\t\t*rmap |= KVMPPC_RMAP_CHANGED;\n-\t\t\tunlock_rmap(rmap);\n-\t\t} else if (memslot->dirty_bitmap) {\n-\t\t\tmark_page_dirty(kvm, gfn);\n-\t\t}\n-\t}\n+\tif (memslot && memslot->dirty_bitmap)\n+\t\tset_bit_le(gfn - memslot->base_gfn, memslot->dirty_bitmap);\n \tsrcu_read_unlock(&kvm->srcu, srcu_idx);\n }\n \n@@ -1282,7 +1265,7 @@ static unsigned long resize_hpt_rehash_hpte(struct kvm_resize_hpt *resize,\n \t\t\trmapp = &memslot->arch.rmap[gfn - memslot->base_gfn];\n \n \t\t\tlock_rmap(rmapp);\n-\t\t\tkvmppc_unmap_hpte(kvm, idx, rmapp, gfn);\n+\t\t\tkvmppc_unmap_hpte(kvm, idx, memslot, rmapp, gfn);\n \t\t\tunlock_rmap(rmapp);\n \t\t}\n \ndiff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c\nindex c5d7435..6336b13 100644\n--- a/arch/powerpc/kvm/book3s_64_mmu_radix.c\n+++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c\n@@ -474,26 +474,6 @@ int kvmppc_book3s_radix_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,\n \treturn ret;\n }\n \n-static void mark_pages_dirty(struct kvm *kvm, struct kvm_memory_slot *memslot,\n-\t\t\t     unsigned long gfn, unsigned int order)\n-{\n-\tunsigned long i, limit;\n-\tunsigned long *dp;\n-\n-\tif (!memslot->dirty_bitmap)\n-\t\treturn;\n-\tlimit = 1ul << order;\n-\tif (limit < BITS_PER_LONG) {\n-\t\tfor (i = 0; i < limit; ++i)\n-\t\t\tmark_page_dirty(kvm, gfn + i);\n-\t\treturn;\n-\t}\n-\tdp = memslot->dirty_bitmap + (gfn - memslot->base_gfn);\n-\tlimit /= BITS_PER_LONG;\n-\tfor (i = 0; i < limit; ++i)\n-\t\t*dp++ = ~0ul;\n-}\n-\n /* Called with kvm->lock held */\n int kvm_unmap_radix(struct kvm *kvm, struct kvm_memory_slot *memslot,\n \t\t    unsigned long gfn)\n@@ -508,12 +488,11 @@ int kvm_unmap_radix(struct kvm *kvm, struct kvm_memory_slot *memslot,\n \t\told = kvmppc_radix_update_pte(kvm, ptep, _PAGE_PRESENT, 0,\n \t\t\t\t\t      gpa, shift);\n \t\tkvmppc_radix_tlbie_page(kvm, gpa, shift);\n-\t\tif (old & _PAGE_DIRTY) {\n-\t\t\tif (!shift)\n-\t\t\t\tmark_page_dirty(kvm, gfn);\n-\t\t\telse\n-\t\t\t\tmark_pages_dirty(kvm, memslot,\n-\t\t\t\t\t\t gfn, shift - PAGE_SHIFT);\n+\t\tif ((old & _PAGE_DIRTY) && memslot->dirty_bitmap) {\n+\t\t\tunsigned long npages = 1;\n+\t\t\tif (shift)\n+\t\t\t\tnpages = 1ul << (shift - PAGE_SHIFT);\n+\t\t\tkvmppc_update_dirty_map(memslot, gfn, npages);\n \t\t}\n \t}\n \treturn 0;\t\t\t\t\n@@ -579,20 +558,8 @@ long kvmppc_hv_get_dirty_log_radix(struct kvm *kvm,\n \t\t\tstruct kvm_memory_slot *memslot, unsigned long *map)\n {\n \tunsigned long i, j;\n-\tunsigned long n, *p;\n \tint npages;\n \n-\t/*\n-\t * Radix accumulates dirty bits in the first half of the\n-\t * memslot's dirty_bitmap area, for when pages are paged\n-\t * out or modified by the host directly.  Pick up these\n-\t * bits and add them to the map.\n-\t */\n-\tn = kvm_dirty_bitmap_bytes(memslot) / sizeof(long);\n-\tp = memslot->dirty_bitmap;\n-\tfor (i = 0; i < n; ++i)\n-\t\tmap[i] |= xchg(&p[i], 0);\n-\n \tfor (i = 0; i < memslot->npages; i = j) {\n \t\tnpages = kvm_radix_test_clear_dirty(kvm, memslot, i);\n \n@@ -604,9 +571,10 @@ long kvmppc_hv_get_dirty_log_radix(struct kvm *kvm,\n \t\t * real address, if npages > 1 we can skip to i + npages.\n \t\t */\n \t\tj = i + 1;\n-\t\tif (npages)\n-\t\t\tfor (j = i; npages; ++j, --npages)\n-\t\t\t\t__set_bit_le(j, map);\n+\t\tif (npages) {\n+\t\t\tset_dirty_bits(map, i, npages);\n+\t\t\ti = j + npages;\n+\t\t}\n \t}\n \treturn 0;\n }\ndiff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c\nindex f652207..96b92d4 100644\n--- a/arch/powerpc/kvm/book3s_hv.c\n+++ b/arch/powerpc/kvm/book3s_hv.c\n@@ -3376,7 +3376,7 @@ static int kvm_vm_ioctl_get_dirty_log_hv(struct kvm *kvm,\n \tstruct kvm_memory_slot *memslot;\n \tint i, r;\n \tunsigned long n;\n-\tunsigned long *buf;\n+\tunsigned long *buf, *p;\n \tstruct kvm_vcpu *vcpu;\n \n \tmutex_lock(&kvm->slots_lock);\n@@ -3392,8 +3392,8 @@ static int kvm_vm_ioctl_get_dirty_log_hv(struct kvm *kvm,\n \t\tgoto out;\n \n \t/*\n-\t * Use second half of bitmap area because radix accumulates\n-\t * bits in the first half.\n+\t * Use second half of bitmap area because both HPT and radix\n+\t * accumulate bits in the first half.\n \t */\n \tn = kvm_dirty_bitmap_bytes(memslot);\n \tbuf = memslot->dirty_bitmap + n / sizeof(long);\n@@ -3406,6 +3406,16 @@ static int kvm_vm_ioctl_get_dirty_log_hv(struct kvm *kvm,\n \tif (r)\n \t\tgoto out;\n \n+\t/*\n+\t * We accumulate dirty bits in the first half of the\n+\t * memslot's dirty_bitmap area, for when pages are paged\n+\t * out or modified by the host directly.  Pick up these\n+\t * bits and add them to the map.\n+\t */\n+\tp = memslot->dirty_bitmap;\n+\tfor (i = 0; i < n / sizeof(long); ++i)\n+\t\tbuf[i] |= xchg(&p[i], 0);\n+\n \t/* Harvest dirty bits from VPA and DTL updates */\n \t/* Note: we never modify the SLB shadow buffer areas */\n \tkvm_for_each_vcpu(i, vcpu, kvm) {\n@@ -3466,8 +3476,6 @@ static void kvmppc_core_commit_memory_region_hv(struct kvm *kvm,\n \t\t\t\tconst struct kvm_memory_slot *new)\n {\n \tunsigned long npages = mem->memory_size >> PAGE_SHIFT;\n-\tstruct kvm_memslots *slots;\n-\tstruct kvm_memory_slot *memslot;\n \n \t/*\n \t * If we are making a new memslot, it might make\n@@ -3477,18 +3485,6 @@ static void kvmppc_core_commit_memory_region_hv(struct kvm *kvm,\n \t */\n \tif (npages)\n \t\tatomic64_inc(&kvm->arch.mmio_update);\n-\n-\tif (npages && old->npages && !kvm_is_radix(kvm)) {\n-\t\t/*\n-\t\t * If modifying a memslot, reset all the rmap dirty bits.\n-\t\t * If this is a new memslot, we don't need to do anything\n-\t\t * since the rmap array starts out as all zeroes,\n-\t\t * i.e. no pages are dirty.\n-\t\t */\n-\t\tslots = kvm_memslots(kvm);\n-\t\tmemslot = id_to_memslot(slots, mem->slot);\n-\t\tkvmppc_hv_get_dirty_log_hpt(kvm, memslot, NULL);\n-\t}\n }\n \n /*\ndiff --git a/arch/powerpc/kvm/book3s_hv_rm_mmu.c b/arch/powerpc/kvm/book3s_hv_rm_mmu.c\nindex cf98f17..26c11f6 100644\n--- a/arch/powerpc/kvm/book3s_hv_rm_mmu.c\n+++ b/arch/powerpc/kvm/book3s_hv_rm_mmu.c\n@@ -107,23 +107,39 @@ void kvmppc_add_revmap_chain(struct kvm *kvm, struct revmap_entry *rev,\n }\n EXPORT_SYMBOL_GPL(kvmppc_add_revmap_chain);\n \n-/* Update the changed page order field of an rmap entry */\n-void kvmppc_update_rmap_change(unsigned long *rmap, unsigned long psize)\n+/* Update the dirty bitmap of a memslot */\n+void kvmppc_update_dirty_map(struct kvm_memory_slot *memslot,\n+\t\t\t     unsigned long gfn, unsigned long psize)\n {\n-\tunsigned long order;\n+\tunsigned long npages;\n \n-\tif (!psize)\n+\tif (!psize || !memslot->dirty_bitmap)\n \t\treturn;\n-\torder = ilog2(psize);\n-\torder <<= KVMPPC_RMAP_CHG_SHIFT;\n-\tif (order > (*rmap & KVMPPC_RMAP_CHG_ORDER))\n-\t\t*rmap = (*rmap & ~KVMPPC_RMAP_CHG_ORDER) | order;\n+\tnpages = (psize + PAGE_SIZE - 1) / PAGE_SIZE;\n+\tgfn -= memslot->base_gfn;\n+\tset_dirty_bits_atomic(memslot->dirty_bitmap, gfn, npages);\n+}\n+EXPORT_SYMBOL_GPL(kvmppc_update_dirty_map);\n+\n+static void kvmppc_set_dirty_from_hpte(struct kvm *kvm,\n+\t\t\t\tunsigned long hpte_v, unsigned long hpte_gr)\n+{\n+\tstruct kvm_memory_slot *memslot;\n+\tunsigned long gfn;\n+\tunsigned long psize;\n+\n+\tpsize = kvmppc_actual_pgsz(hpte_v, hpte_gr);\n+\tgfn = hpte_rpn(hpte_gr, psize);\n+\tmemslot = __gfn_to_memslot(kvm_memslots_raw(kvm), gfn);\n+\tif (memslot && memslot->dirty_bitmap)\n+\t\tkvmppc_update_dirty_map(memslot, gfn, psize);\n }\n-EXPORT_SYMBOL_GPL(kvmppc_update_rmap_change);\n \n /* Returns a pointer to the revmap entry for the page mapped by a HPTE */\n static unsigned long *revmap_for_hpte(struct kvm *kvm, unsigned long hpte_v,\n-\t\t\t\t      unsigned long hpte_gr)\n+\t\t\t\t      unsigned long hpte_gr,\n+\t\t\t\t      struct kvm_memory_slot **memslotp,\n+\t\t\t\t      unsigned long *gfnp)\n {\n \tstruct kvm_memory_slot *memslot;\n \tunsigned long *rmap;\n@@ -131,6 +147,10 @@ static unsigned long *revmap_for_hpte(struct kvm *kvm, unsigned long hpte_v,\n \n \tgfn = hpte_rpn(hpte_gr, kvmppc_actual_pgsz(hpte_v, hpte_gr));\n \tmemslot = __gfn_to_memslot(kvm_memslots_raw(kvm), gfn);\n+\tif (memslotp)\n+\t\t*memslotp = memslot;\n+\tif (gfnp)\n+\t\t*gfnp = gfn;\n \tif (!memslot)\n \t\treturn NULL;\n \n@@ -147,10 +167,12 @@ static void remove_revmap_chain(struct kvm *kvm, long pte_index,\n \tunsigned long ptel, head;\n \tunsigned long *rmap;\n \tunsigned long rcbits;\n+\tstruct kvm_memory_slot *memslot;\n+\tunsigned long gfn;\n \n \trcbits = hpte_r & (HPTE_R_R | HPTE_R_C);\n \tptel = rev->guest_rpte |= rcbits;\n-\trmap = revmap_for_hpte(kvm, hpte_v, ptel);\n+\trmap = revmap_for_hpte(kvm, hpte_v, ptel, &memslot, &gfn);\n \tif (!rmap)\n \t\treturn;\n \tlock_rmap(rmap);\n@@ -169,8 +191,8 @@ static void remove_revmap_chain(struct kvm *kvm, long pte_index,\n \t}\n \t*rmap |= rcbits << KVMPPC_RMAP_RC_SHIFT;\n \tif (rcbits & HPTE_R_C)\n-\t\tkvmppc_update_rmap_change(rmap,\n-\t\t\t\t\t  kvmppc_actual_pgsz(hpte_v, hpte_r));\n+\t\tkvmppc_update_dirty_map(memslot, gfn,\n+\t\t\t\t\tkvmppc_actual_pgsz(hpte_v, hpte_r));\n \tunlock_rmap(rmap);\n }\n \n@@ -798,7 +820,7 @@ long kvmppc_h_clear_ref(struct kvm_vcpu *vcpu, unsigned long flags,\n \t\tgr |= r & (HPTE_R_R | HPTE_R_C);\n \t\tif (r & HPTE_R_R) {\n \t\t\tkvmppc_clear_ref_hpte(kvm, hpte, pte_index);\n-\t\t\trmap = revmap_for_hpte(kvm, v, gr);\n+\t\t\trmap = revmap_for_hpte(kvm, v, gr, NULL, NULL);\n \t\t\tif (rmap) {\n \t\t\t\tlock_rmap(rmap);\n \t\t\t\t*rmap |= KVMPPC_RMAP_REFERENCED;\n@@ -820,7 +842,6 @@ long kvmppc_h_clear_mod(struct kvm_vcpu *vcpu, unsigned long flags,\n \t__be64 *hpte;\n \tunsigned long v, r, gr;\n \tstruct revmap_entry *rev;\n-\tunsigned long *rmap;\n \tlong ret = H_NOT_FOUND;\n \n \tif (kvm_is_radix(kvm))\n@@ -849,16 +870,9 @@ long kvmppc_h_clear_mod(struct kvm_vcpu *vcpu, unsigned long flags,\n \t\tr = be64_to_cpu(hpte[1]);\n \t\tgr |= r & (HPTE_R_R | HPTE_R_C);\n \t\tif (r & HPTE_R_C) {\n-\t\t\tunsigned long psize = kvmppc_actual_pgsz(v, r);\n \t\t\thpte[1] = cpu_to_be64(r & ~HPTE_R_C);\n \t\t\teieio();\n-\t\t\trmap = revmap_for_hpte(kvm, v, gr);\n-\t\t\tif (rmap) {\n-\t\t\t\tlock_rmap(rmap);\n-\t\t\t\t*rmap |= KVMPPC_RMAP_CHANGED;\n-\t\t\t\tkvmppc_update_rmap_change(rmap, psize);\n-\t\t\t\tunlock_rmap(rmap);\n-\t\t\t}\n+\t\t\tkvmppc_set_dirty_from_hpte(kvm, v, gr);\n \t\t}\n \t}\n \tvcpu->arch.gpr[4] = gr;\n","prefixes":["v2","3/6"]}