Patchwork cpu: smp_wmb before lauching cpus.

login
register
mail settings
Submitter pingfan liu
Date July 5, 2012, 2:18 a.m.
Message ID <1341454689-8129-1-git-send-email-qemulist@gmail.com>
Download mbox | patch
Permalink /patch/169059/
State New
Headers show

Comments

pingfan liu - July 5, 2012, 2:18 a.m.
Vcpu state must be set completely before receiving INIT-IPI,SIPI

Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
---
 kvm.h |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
Jan Kiszka - July 5, 2012, 6:46 a.m.
On 2012-07-05 04:18, Liu Ping Fan wrote:
> Vcpu state must be set completely before receiving INIT-IPI,SIPI
> 
> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
> ---
>  kvm.h |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/kvm.h b/kvm.h
> index 9c7b0ea..5b3c228 100644
> --- a/kvm.h
> +++ b/kvm.h
> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>  {
>      if (kvm_enabled()) {
>          kvm_cpu_synchronize_post_init(env);
> +        smp_wmb();
>      }
>  }
>  
> 

In theory, there should be no vcpu kick-off after this without some
locking operations involved that imply barriers. Did you see real
inconsistencies without this explicit one?

Jan
pingfan liu - July 5, 2012, 10:10 a.m.
On Thu, Jul 5, 2012 at 2:46 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
> On 2012-07-05 04:18, Liu Ping Fan wrote:
>> Vcpu state must be set completely before receiving INIT-IPI,SIPI
>>
>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>> ---
>>  kvm.h |    1 +
>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>
>> diff --git a/kvm.h b/kvm.h
>> index 9c7b0ea..5b3c228 100644
>> --- a/kvm.h
>> +++ b/kvm.h
>> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>>  {
>>      if (kvm_enabled()) {
>>          kvm_cpu_synchronize_post_init(env);
>> +        smp_wmb();
>>      }
>>  }
>>
>>
>
> In theory, there should be no vcpu kick-off after this without some
> locking operations involved that imply barriers. Did you see real

Yeah, but what if it is non-x86?
> inconsistencies without this explicit one?
>
> Jan
>
Jan Kiszka - July 5, 2012, 10:16 a.m.
On 2012-07-05 12:10, liu ping fan wrote:
> On Thu, Jul 5, 2012 at 2:46 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
>> On 2012-07-05 04:18, Liu Ping Fan wrote:
>>> Vcpu state must be set completely before receiving INIT-IPI,SIPI
>>>
>>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>>> ---
>>>  kvm.h |    1 +
>>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/kvm.h b/kvm.h
>>> index 9c7b0ea..5b3c228 100644
>>> --- a/kvm.h
>>> +++ b/kvm.h
>>> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>>>  {
>>>      if (kvm_enabled()) {
>>>          kvm_cpu_synchronize_post_init(env);
>>> +        smp_wmb();
>>>      }
>>>  }
>>>
>>>
>>
>> In theory, there should be no vcpu kick-off after this without some
>> locking operations involved that imply barriers. Did you see real
> 
> Yeah, but what if it is non-x86?

The locking I'm referring to is arch independent.

>> inconsistencies without this explicit one?

Again: Did you see real issues or is this based on static analysis?

Jan
pingfan liu - July 5, 2012, 11:02 a.m.
On Thu, Jul 5, 2012 at 6:16 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
> On 2012-07-05 12:10, liu ping fan wrote:
>> On Thu, Jul 5, 2012 at 2:46 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
>>> On 2012-07-05 04:18, Liu Ping Fan wrote:
>>>> Vcpu state must be set completely before receiving INIT-IPI,SIPI
>>>>
>>>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>>>> ---
>>>>  kvm.h |    1 +
>>>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>>>
>>>> diff --git a/kvm.h b/kvm.h
>>>> index 9c7b0ea..5b3c228 100644
>>>> --- a/kvm.h
>>>> +++ b/kvm.h
>>>> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>>>>  {
>>>>      if (kvm_enabled()) {
>>>>          kvm_cpu_synchronize_post_init(env);
>>>> +        smp_wmb();
>>>>      }
>>>>  }
>>>>
>>>>
>>>
>>> In theory, there should be no vcpu kick-off after this without some
>>> locking operations involved that imply barriers. Did you see real
>>
>> Yeah, but what if it is non-x86?
>
> The locking I'm referring to is arch independent.
>
>>> inconsistencies without this explicit one?
>
> Again: Did you see real issues or is this based on static analysis?
>
Just on  static analysis

Regards,
pingfan
> Jan
>
> --
> Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
> Corporate Competence Center Embedded Linux
>
Jan Kiszka - July 5, 2012, 11:58 a.m.
On 2012-07-05 13:02, liu ping fan wrote:
> On Thu, Jul 5, 2012 at 6:16 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>> On 2012-07-05 12:10, liu ping fan wrote:
>>> On Thu, Jul 5, 2012 at 2:46 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
>>>> On 2012-07-05 04:18, Liu Ping Fan wrote:
>>>>> Vcpu state must be set completely before receiving INIT-IPI,SIPI
>>>>>
>>>>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>>>>> ---
>>>>>  kvm.h |    1 +
>>>>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>>>>
>>>>> diff --git a/kvm.h b/kvm.h
>>>>> index 9c7b0ea..5b3c228 100644
>>>>> --- a/kvm.h
>>>>> +++ b/kvm.h
>>>>> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>>>>>  {
>>>>>      if (kvm_enabled()) {
>>>>>          kvm_cpu_synchronize_post_init(env);
>>>>> +        smp_wmb();
>>>>>      }
>>>>>  }
>>>>>
>>>>>
>>>>
>>>> In theory, there should be no vcpu kick-off after this without some
>>>> locking operations involved that imply barriers. Did you see real
>>>
>>> Yeah, but what if it is non-x86?
>>
>> The locking I'm referring to is arch independent.
>>
>>>> inconsistencies without this explicit one?
>>
>> Again: Did you see real issues or is this based on static analysis?
>>
> Just on  static analysis

Then please describe - also for the changelog - at least one case in
details where this is needed.

Thanks,
Jan
pingfan liu - July 6, 2012, 7:46 a.m.
On Thu, Jul 5, 2012 at 7:58 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
> On 2012-07-05 13:02, liu ping fan wrote:
>> On Thu, Jul 5, 2012 at 6:16 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>> On 2012-07-05 12:10, liu ping fan wrote:
>>>> On Thu, Jul 5, 2012 at 2:46 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
>>>>> On 2012-07-05 04:18, Liu Ping Fan wrote:
>>>>>> Vcpu state must be set completely before receiving INIT-IPI,SIPI
>>>>>>
>>>>>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>>>>>> ---
>>>>>>  kvm.h |    1 +
>>>>>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>>>>>
>>>>>> diff --git a/kvm.h b/kvm.h
>>>>>> index 9c7b0ea..5b3c228 100644
>>>>>> --- a/kvm.h
>>>>>> +++ b/kvm.h
>>>>>> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>>>>>>  {
>>>>>>      if (kvm_enabled()) {
>>>>>>          kvm_cpu_synchronize_post_init(env);
>>>>>> +        smp_wmb();
>>>>>>      }
>>>>>>  }
>>>>>>
>>>>>>
>>>>>
>>>>> In theory, there should be no vcpu kick-off after this without some
>>>>> locking operations involved that imply barriers. Did you see real
>>>>
>>>> Yeah, but what if it is non-x86?
>>>
>>> The locking I'm referring to is arch independent.
>>>
>>>>> inconsistencies without this explicit one?
>>>
>>> Again: Did you see real issues or is this based on static analysis?
>>>
>> Just on  static analysis
>
> Then please describe - also for the changelog - at least one case in
> details where this is needed.
>
I dived into code. And yes, as you said, ACPI eject does involve some
locking operation. So the only thing left is for starting up.  There
seems no potential lock operation from cpu_synchronize_all_post_init()
to  resume_all_vcpus().
If correct, I will update this info for the changelog

Regards,
pingfa

> Thanks,
> Jan
>
> --
> Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
> Corporate Competence Center Embedded Linux
>
>
Jan Kiszka - July 6, 2012, 8:14 a.m.
On 2012-07-06 09:46, liu ping fan wrote:
> On Thu, Jul 5, 2012 at 7:58 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>> On 2012-07-05 13:02, liu ping fan wrote:
>>> On Thu, Jul 5, 2012 at 6:16 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>>> On 2012-07-05 12:10, liu ping fan wrote:
>>>>> On Thu, Jul 5, 2012 at 2:46 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
>>>>>> On 2012-07-05 04:18, Liu Ping Fan wrote:
>>>>>>> Vcpu state must be set completely before receiving INIT-IPI,SIPI
>>>>>>>
>>>>>>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>>>>>>> ---
>>>>>>>  kvm.h |    1 +
>>>>>>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>>>>>>
>>>>>>> diff --git a/kvm.h b/kvm.h
>>>>>>> index 9c7b0ea..5b3c228 100644
>>>>>>> --- a/kvm.h
>>>>>>> +++ b/kvm.h
>>>>>>> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>>>>>>>  {
>>>>>>>      if (kvm_enabled()) {
>>>>>>>          kvm_cpu_synchronize_post_init(env);
>>>>>>> +        smp_wmb();
>>>>>>>      }
>>>>>>>  }
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> In theory, there should be no vcpu kick-off after this without some
>>>>>> locking operations involved that imply barriers. Did you see real
>>>>>
>>>>> Yeah, but what if it is non-x86?
>>>>
>>>> The locking I'm referring to is arch independent.
>>>>
>>>>>> inconsistencies without this explicit one?
>>>>
>>>> Again: Did you see real issues or is this based on static analysis?
>>>>
>>> Just on  static analysis
>>
>> Then please describe - also for the changelog - at least one case in
>> details where this is needed.
>>
> I dived into code. And yes, as you said, ACPI eject does involve some
> locking operation. So the only thing left is for starting up.  There
> seems no potential lock operation from cpu_synchronize_all_post_init()
> to  resume_all_vcpus().

There is no difference in this scenario as well: the vcpu is waiting on
a condvar for the initial kick-off and will synchronize with the main
thread via the global mutex. When in doubt, try using a debugger.

Jan

Patch

diff --git a/kvm.h b/kvm.h
index 9c7b0ea..5b3c228 100644
--- a/kvm.h
+++ b/kvm.h
@@ -198,6 +198,7 @@  static inline void cpu_synchronize_post_init(CPUArchState *env)
 {
     if (kvm_enabled()) {
         kvm_cpu_synchronize_post_init(env);
+        smp_wmb();
     }
 }