Patchwork [4/4] KVM: PPC: align vcpu_kick with x86

login
register
mail settings
Submitter Alexander Graf
Date Dec. 9, 2011, 3:26 p.m.
Message ID <1323444412-18482-5-git-send-email-agraf@suse.de>
Download mbox | patch
Permalink /patch/130402/
State New
Headers show

Comments

Alexander Graf - Dec. 9, 2011, 3:26 p.m.
Our vcpu kick implementation differs a bit from x86 which resulted in us not
disabling preemption during the kick. Get it a bit closer to what x86 does.

Signed-off-by: Alexander Graf <agraf@suse.de>
---
 arch/powerpc/kvm/powerpc.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)
Scott Wood - Dec. 9, 2011, 6:19 p.m.
On 12/09/2011 09:26 AM, Alexander Graf wrote:
> Our vcpu kick implementation differs a bit from x86 which resulted in us not
> disabling preemption during the kick. Get it a bit closer to what x86 does.

Disabling preemption only matters due to the other bit of functionality
you brought over -- avoiding kicking the current CPU.

Probably doesn't even matter all that much with it, since avoiding that
is just an optimization, and any race that causes us to fail to
reschedule a vcpu of a different thread means that thread just
rescheduled anyway.  Not something I feel any great need to rely on,
though. :-)

> Signed-off-by: Alexander Graf <agraf@suse.de>
> ---
>  arch/powerpc/kvm/powerpc.c |    7 ++++++-
>  1 files changed, 6 insertions(+), 1 deletions(-)
> 
> diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
> index c952f13..ef8c990 100644
> --- a/arch/powerpc/kvm/powerpc.c
> +++ b/arch/powerpc/kvm/powerpc.c
> @@ -557,12 +557,17 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
>  
>  void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
>  {
> +        int me;
> +        int cpu = vcpu->cpu;
> +
> +        me = get_cpu();
>  	if (waitqueue_active(&vcpu->wq)) {
>  		wake_up_interruptible(vcpu->arch.wqp);
>  		vcpu->stat.halt_wakeup++;
> -	} else if (vcpu->cpu != -1) {
> +	} else if (cpu != me && cpu != -1) {
>  		smp_send_reschedule(vcpu->cpu);
>  	}
> +        put_cpu();
>  }

Whitespace.

-Scott

--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Alexander Graf - Dec. 9, 2011, 7:10 p.m.
On 09.12.2011, at 19:19, Scott Wood <scottwood@freescale.com> wrote:

> On 12/09/2011 09:26 AM, Alexander Graf wrote:
>> Our vcpu kick implementation differs a bit from x86 which resulted in us not
>> disabling preemption during the kick. Get it a bit closer to what x86 does.
> 
> Disabling preemption only matters due to the other bit of functionality
> you brought over -- avoiding kicking the current CPU.

Nope, I had BUG_ON warnings in the kick code because preemption was on. get_cpu() disables preemption indirectly though, getting rid of the oops :)

Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Scott Wood - Dec. 9, 2011, 7:15 p.m.
On 12/09/2011 01:10 PM, Alexander Graf wrote:
> 
> On 09.12.2011, at 19:19, Scott Wood <scottwood@freescale.com> wrote:
> 
>> On 12/09/2011 09:26 AM, Alexander Graf wrote:
>>> Our vcpu kick implementation differs a bit from x86 which resulted in us not
>>> disabling preemption during the kick. Get it a bit closer to what x86 does.
>>
>> Disabling preemption only matters due to the other bit of functionality
>> you brought over -- avoiding kicking the current CPU.
> 
> Nope, I had BUG_ON warnings in the kick code because preemption was on.

Coming from where?

-Scott

--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Alexander Graf - Dec. 9, 2011, 11:15 p.m.
On 09.12.2011, at 20:15, Scott Wood wrote:

> On 12/09/2011 01:10 PM, Alexander Graf wrote:
>> 
>> On 09.12.2011, at 19:19, Scott Wood <scottwood@freescale.com> wrote:
>> 
>>> On 12/09/2011 09:26 AM, Alexander Graf wrote:
>>>> Our vcpu kick implementation differs a bit from x86 which resulted in us not
>>>> disabling preemption during the kick. Get it a bit closer to what x86 does.
>>> 
>>> Disabling preemption only matters due to the other bit of functionality
>>> you brought over -- avoiding kicking the current CPU.
>> 
>> Nope, I had BUG_ON warnings in the kick code because preemption was on.
> 
> Coming from where?

From here:

BUG: using smp_processor_id() in preemptible [00000000] code: qemu-system-ppc/17448
caller is .smp_mpic_message_pass+0x88/0x10c
Call Trace:
[c000000078d83600] [c000000000013e70] .show_stack+0x6c/0x16c (unreliable)
[c000000078d836b0] [c00000000037b614] .debug_smp_processor_id+0xe4/0x11c
[c000000078d83740] [c000000000048988] .smp_mpic_message_pass+0x88/0x10c
[c000000078d837d0] [c00000000002f2b4] .smp_send_reschedule+0x4c/0x80
[c000000078d83850] [d000000005e68984] .kvm_vcpu_kick+0x5c/0x74 [kvm]
[c000000078d838d0] [d000000005e689d8] .kvm_vcpu_ioctl_interrupt+0x3c/0x54 [kvm]
[c000000078d83950] [d000000005e68a8c] .kvm_arch_vcpu_ioctl+0x9c/0x21c [kvm]
[c000000078d83b60] [d000000005e64520] .kvm_vcpu_ioctl+0x7c/0x6b8 [kvm]
[c000000078d83c20] [d000000005e64c2c] .kvm_vcpu_compat_ioctl+0xd0/0xfc [kvm]
[c000000078d83cd0] [c0000000001be750] .compat_sys_ioctl+0x160/0x1864
[c000000078d83e30] [c00000000000988c] syscall_exit+0x0/0x40


Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Scott Wood - Dec. 12, 2011, 5:32 p.m.
On 12/09/2011 05:15 PM, Alexander Graf wrote:
> 
> On 09.12.2011, at 20:15, Scott Wood wrote:
> 
>> On 12/09/2011 01:10 PM, Alexander Graf wrote:
>>>
>>> On 09.12.2011, at 19:19, Scott Wood <scottwood@freescale.com> wrote:
>>>
>>>> On 12/09/2011 09:26 AM, Alexander Graf wrote:
>>>>> Our vcpu kick implementation differs a bit from x86 which resulted in us not
>>>>> disabling preemption during the kick. Get it a bit closer to what x86 does.
>>>>
>>>> Disabling preemption only matters due to the other bit of functionality
>>>> you brought over -- avoiding kicking the current CPU.
>>>
>>> Nope, I had BUG_ON warnings in the kick code because preemption was on.
>>
>> Coming from where?
> 
> From here:
> 
> BUG: using smp_processor_id() in preemptible [00000000] code: qemu-system-ppc/17448
> caller is .smp_mpic_message_pass+0x88/0x10c
> Call Trace:
> [c000000078d83600] [c000000000013e70] .show_stack+0x6c/0x16c (unreliable)
> [c000000078d836b0] [c00000000037b614] .debug_smp_processor_id+0xe4/0x11c
> [c000000078d83740] [c000000000048988] .smp_mpic_message_pass+0x88/0x10c
> [c000000078d837d0] [c00000000002f2b4] .smp_send_reschedule+0x4c/0x80
> [c000000078d83850] [d000000005e68984] .kvm_vcpu_kick+0x5c/0x74 [kvm]
> [c000000078d838d0] [d000000005e689d8] .kvm_vcpu_ioctl_interrupt+0x3c/0x54 [kvm]
> [c000000078d83950] [d000000005e68a8c] .kvm_arch_vcpu_ioctl+0x9c/0x21c [kvm]
> [c000000078d83b60] [d000000005e64520] .kvm_vcpu_ioctl+0x7c/0x6b8 [kvm]
> [c000000078d83c20] [d000000005e64c2c] .kvm_vcpu_compat_ioctl+0xd0/0xfc [kvm]
> [c000000078d83cd0] [c0000000001be750] .compat_sys_ioctl+0x160/0x1864
> [c000000078d83e30] [c00000000000988c] syscall_exit+0x0/0x40

Hmm, unless smp_send_reschedule must be called with preemption disabled
(doesn't seem like an obvious requirement, even if callers commonly have
a reason to do so, and the only documentation of it I see is on the ia64
implementation), that seems like a bug in the MPIC IPI implementation.

It wouldn't be an issue (and would make the MPIC driver slightly faster,
too) if the MPIC driver were to use the magic CPU region instead of
manually indexing the non-magic array of per-CPU regions.  Maybe it
doesn't work on all hardware the driver supports?

Not that it matters much here -- we want this patch anyway, because we
want to check that it's not the current CPU.

-Scott

--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Alexander Graf - Dec. 23, 2011, 12:56 p.m.
On 12.12.2011, at 18:32, Scott Wood wrote:

> On 12/09/2011 05:15 PM, Alexander Graf wrote:
>> 
>> On 09.12.2011, at 20:15, Scott Wood wrote:
>> 
>>> On 12/09/2011 01:10 PM, Alexander Graf wrote:
>>>> 
>>>> On 09.12.2011, at 19:19, Scott Wood <scottwood@freescale.com> wrote:
>>>> 
>>>>> On 12/09/2011 09:26 AM, Alexander Graf wrote:
>>>>>> Our vcpu kick implementation differs a bit from x86 which resulted in us not
>>>>>> disabling preemption during the kick. Get it a bit closer to what x86 does.
>>>>> 
>>>>> Disabling preemption only matters due to the other bit of functionality
>>>>> you brought over -- avoiding kicking the current CPU.
>>>> 
>>>> Nope, I had BUG_ON warnings in the kick code because preemption was on.
>>> 
>>> Coming from where?
>> 
>> From here:
>> 
>> BUG: using smp_processor_id() in preemptible [00000000] code: qemu-system-ppc/17448
>> caller is .smp_mpic_message_pass+0x88/0x10c
>> Call Trace:
>> [c000000078d83600] [c000000000013e70] .show_stack+0x6c/0x16c (unreliable)
>> [c000000078d836b0] [c00000000037b614] .debug_smp_processor_id+0xe4/0x11c
>> [c000000078d83740] [c000000000048988] .smp_mpic_message_pass+0x88/0x10c
>> [c000000078d837d0] [c00000000002f2b4] .smp_send_reschedule+0x4c/0x80
>> [c000000078d83850] [d000000005e68984] .kvm_vcpu_kick+0x5c/0x74 [kvm]
>> [c000000078d838d0] [d000000005e689d8] .kvm_vcpu_ioctl_interrupt+0x3c/0x54 [kvm]
>> [c000000078d83950] [d000000005e68a8c] .kvm_arch_vcpu_ioctl+0x9c/0x21c [kvm]
>> [c000000078d83b60] [d000000005e64520] .kvm_vcpu_ioctl+0x7c/0x6b8 [kvm]
>> [c000000078d83c20] [d000000005e64c2c] .kvm_vcpu_compat_ioctl+0xd0/0xfc [kvm]
>> [c000000078d83cd0] [c0000000001be750] .compat_sys_ioctl+0x160/0x1864
>> [c000000078d83e30] [c00000000000988c] syscall_exit+0x0/0x40
> 
> Hmm, unless smp_send_reschedule must be called with preemption disabled
> (doesn't seem like an obvious requirement, even if callers commonly have
> a reason to do so, and the only documentation of it I see is on the ia64
> implementation), that seems like a bug in the MPIC IPI implementation.
> 
> It wouldn't be an issue (and would make the MPIC driver slightly faster,
> too) if the MPIC driver were to use the magic CPU region instead of
> manually indexing the non-magic array of per-CPU regions.  Maybe it
> doesn't work on all hardware the driver supports?

I'd say that's a question I'll redirect to Ben :).

> Not that it matters much here -- we want this patch anyway, because we
> want to check that it's not the current CPU.

Yeah. Well - eventually we want the exact same vcpu_kick function across architectures. But this is a good step in the right direction.


Alex

--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Benjamin Herrenschmidt - Dec. 23, 2011, 9:50 p.m.
On Fri, 2011-12-23 at 13:56 +0100, Alexander Graf wrote:

> > Hmm, unless smp_send_reschedule must be called with preemption disabled
> > (doesn't seem like an obvious requirement, even if callers commonly have
> > a reason to do so, and the only documentation of it I see is on the ia64
> > implementation), that seems like a bug in the MPIC IPI implementation.
> > 
> > It wouldn't be an issue (and would make the MPIC driver slightly faster,
> > too) if the MPIC driver were to use the magic CPU region instead of
> > manually indexing the non-magic array of per-CPU regions.  Maybe it
> > doesn't work on all hardware the driver supports?

Right, the magic per-cpu region doesn't work on all HW.

> I'd say that's a question I'll redirect to Ben :).
> 
> > Not that it matters much here -- we want this patch anyway, because we
> > want to check that it's not the current CPU.
> 
> Yeah. Well - eventually we want the exact same vcpu_kick function across architectures.
> But this is a good step in the right direction.

Cheers,
Ben.


--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index c952f13..ef8c990 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -557,12 +557,17 @@  int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
 
 void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
 {
+        int me;
+        int cpu = vcpu->cpu;
+
+        me = get_cpu();
 	if (waitqueue_active(&vcpu->wq)) {
 		wake_up_interruptible(vcpu->arch.wqp);
 		vcpu->stat.halt_wakeup++;
-	} else if (vcpu->cpu != -1) {
+	} else if (cpu != me && cpu != -1) {
 		smp_send_reschedule(vcpu->cpu);
 	}
+        put_cpu();
 }
 
 int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu, struct kvm_interrupt *irq)