Patchwork [RESEND,v4] powerpc: kvm: fix rare but potential deadlock scene

login
register
mail settings
Submitter Liu Ping Fan
Date Nov. 19, 2013, 6:12 a.m.
Message ID <1384841568-20986-1-git-send-email-pingfank@linux.vnet.ibm.com>
Download mbox | patch
Permalink /patch/292284/
State New
Headers show

Comments

Liu Ping Fan - Nov. 19, 2013, 6:12 a.m.
Since kvmppc_hv_find_lock_hpte() is called from both virtmode and
realmode, so it can trigger the deadlock.

Suppose the following scene:

Two physical cpuM, cpuN, two VM instances A, B, each VM has a group of
vcpus.

If on cpuM, vcpu_A_1 holds bitlock X (HPTE_V_HVLOCK), then is switched
out, and on cpuN, vcpu_A_2 try to lock X in realmode, then cpuN will be
caught in realmode for a long time.

What makes things even worse if the following happens,
  On cpuM, bitlockX is hold, on cpuN, Y is hold.
  vcpu_B_2 try to lock Y on cpuM in realmode
  vcpu_A_2 try to lock X on cpuN in realmode

Oops! deadlock happens

Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
---
 arch/powerpc/kvm/book3s_64_mmu_hv.c | 6 +++++-
 arch/powerpc/kvm/book3s_hv_rm_mmu.c | 4 ++++
 2 files changed, 9 insertions(+), 1 deletion(-)
Alexander Graf - Nov. 19, 2013, 10:39 a.m.
On 19.11.2013, at 07:12, Liu Ping Fan <kernelfans@gmail.com> wrote:

> Since kvmppc_hv_find_lock_hpte() is called from both virtmode and
> realmode, so it can trigger the deadlock.
> 
> Suppose the following scene:
> 
> Two physical cpuM, cpuN, two VM instances A, B, each VM has a group of
> vcpus.
> 
> If on cpuM, vcpu_A_1 holds bitlock X (HPTE_V_HVLOCK), then is switched
> out, and on cpuN, vcpu_A_2 try to lock X in realmode, then cpuN will be
> caught in realmode for a long time.
> 
> What makes things even worse if the following happens,
>  On cpuM, bitlockX is hold, on cpuN, Y is hold.
>  vcpu_B_2 try to lock Y on cpuM in realmode
>  vcpu_A_2 try to lock X on cpuN in realmode
> 
> Oops! deadlock happens
> 
> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>

Any particular reason for the resend? The patch is already applied, no?


Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Liu Ping Fan - Nov. 20, 2013, 2:42 a.m.
On Tue, Nov 19, 2013 at 6:39 PM, Alexander Graf <agraf@suse.de> wrote:
>
> On 19.11.2013, at 07:12, Liu Ping Fan <kernelfans@gmail.com> wrote:
>
>> Since kvmppc_hv_find_lock_hpte() is called from both virtmode and
>> realmode, so it can trigger the deadlock.
>>
>> Suppose the following scene:
>>
>> Two physical cpuM, cpuN, two VM instances A, B, each VM has a group of
>> vcpus.
>>
>> If on cpuM, vcpu_A_1 holds bitlock X (HPTE_V_HVLOCK), then is switched
>> out, and on cpuN, vcpu_A_2 try to lock X in realmode, then cpuN will be
>> caught in realmode for a long time.
>>
>> What makes things even worse if the following happens,
>>  On cpuM, bitlockX is hold, on cpuN, Y is hold.
>>  vcpu_B_2 try to lock Y on cpuM in realmode
>>  vcpu_A_2 try to lock X on cpuN in realmode
>>
>> Oops! deadlock happens
>>
>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>
> Any particular reason for the resend? The patch is already applied, no?
>
Oh, seems that I misunderstood your meaning. You said "Actually, I've
changed my mind and moved the patch to the for-3.13 branch instead.
Please make sure to CC kvm@vger on all patches you submit though". So
I think it is necessary to resend with cc kvm@vger

Regards,
Pingfan
--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Alexander Graf - Nov. 20, 2013, 10:53 a.m.
> Am 20.11.2013 um 03:42 schrieb Liu ping fan <kernelfans@gmail.com>:
> 
>> On Tue, Nov 19, 2013 at 6:39 PM, Alexander Graf <agraf@suse.de> wrote:
>> 
>>> On 19.11.2013, at 07:12, Liu Ping Fan <kernelfans@gmail.com> wrote:
>>> 
>>> Since kvmppc_hv_find_lock_hpte() is called from both virtmode and
>>> realmode, so it can trigger the deadlock.
>>> 
>>> Suppose the following scene:
>>> 
>>> Two physical cpuM, cpuN, two VM instances A, B, each VM has a group of
>>> vcpus.
>>> 
>>> If on cpuM, vcpu_A_1 holds bitlock X (HPTE_V_HVLOCK), then is switched
>>> out, and on cpuN, vcpu_A_2 try to lock X in realmode, then cpuN will be
>>> caught in realmode for a long time.
>>> 
>>> What makes things even worse if the following happens,
>>> On cpuM, bitlockX is hold, on cpuN, Y is hold.
>>> vcpu_B_2 try to lock Y on cpuM in realmode
>>> vcpu_A_2 try to lock X on cpuN in realmode
>>> 
>>> Oops! deadlock happens
>>> 
>>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>> 
>> Any particular reason for the resend? The patch is already applied, no?
> Oh, seems that I misunderstood your meaning. You said "Actually, I've
> changed my mind and moved the patch to the for-3.13 branch instead.
> Please make sure to CC kvm@vger on all patches you submit though". So
> I think it is necessary to resend with cc kvm@vger

Oh. I meant "for next time" :). But now that it happened for this patch already it's even better.

Thanks a lot!

Alex

> 
> Regards,
> Pingfan
--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
index 842f081..abf81fe 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -473,11 +473,14 @@  static int kvmppc_mmu_book3s_64_hv_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,
 		slb_v = vcpu->kvm->arch.vrma_slb_v;
 	}
 
+	preempt_disable();
 	/* Find the HPTE in the hash table */
 	index = kvmppc_hv_find_lock_hpte(kvm, eaddr, slb_v,
 					 HPTE_V_VALID | HPTE_V_ABSENT);
-	if (index < 0)
+	if (index < 0) {
+		preempt_enable();
 		return -ENOENT;
+	}
 	hptep = (unsigned long *)(kvm->arch.hpt_virt + (index << 4));
 	v = hptep[0] & ~HPTE_V_HVLOCK;
 	gr = kvm->arch.revmap[index].guest_rpte;
@@ -485,6 +488,7 @@  static int kvmppc_mmu_book3s_64_hv_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,
 	/* Unlock the HPTE */
 	asm volatile("lwsync" : : : "memory");
 	hptep[0] = v;
+	preempt_enable();
 
 	gpte->eaddr = eaddr;
 	gpte->vpage = ((v & HPTE_V_AVPN) << 4) | ((eaddr >> 12) & 0xfff);
diff --git a/arch/powerpc/kvm/book3s_hv_rm_mmu.c b/arch/powerpc/kvm/book3s_hv_rm_mmu.c
index 9c51544..ea17b30 100644
--- a/arch/powerpc/kvm/book3s_hv_rm_mmu.c
+++ b/arch/powerpc/kvm/book3s_hv_rm_mmu.c
@@ -749,6 +749,10 @@  static int slb_base_page_shift[4] = {
 	20,	/* 1M, unsupported */
 };
 
+/* When called from virtmode, this func should be protected by
+ * preempt_disable(), otherwise, the holding of HPTE_V_HVLOCK
+ * can trigger deadlock issue.
+ */
 long kvmppc_hv_find_lock_hpte(struct kvm *kvm, gva_t eaddr, unsigned long slb_v,
 			      unsigned long valid)
 {