cpu: smp_wmb before lauching cpus.

Submitted by pingfan liu on July 5, 2012, 2:18 a.m.

Details

Message ID 1341454689-8129-1-git-send-email-qemulist@gmail.com
State New
Headers show

Commit Message

pingfan liu July 5, 2012, 2:18 a.m.
Vcpu state must be set completely before receiving INIT-IPI,SIPI

Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
---
 kvm.h |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

Comments

Jan Kiszka July 5, 2012, 6:46 a.m.
On 2012-07-05 04:18, Liu Ping Fan wrote:
> Vcpu state must be set completely before receiving INIT-IPI,SIPI
> 
> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
> ---
>  kvm.h |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)
> 
> diff --git a/kvm.h b/kvm.h
> index 9c7b0ea..5b3c228 100644
> --- a/kvm.h
> +++ b/kvm.h
> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>  {
>      if (kvm_enabled()) {
>          kvm_cpu_synchronize_post_init(env);
> +        smp_wmb();
>      }
>  }
>  
> 

In theory, there should be no vcpu kick-off after this without some
locking operations involved that imply barriers. Did you see real
inconsistencies without this explicit one?

Jan
pingfan liu July 5, 2012, 10:10 a.m.
On Thu, Jul 5, 2012 at 2:46 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
> On 2012-07-05 04:18, Liu Ping Fan wrote:
>> Vcpu state must be set completely before receiving INIT-IPI,SIPI
>>
>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>> ---
>>  kvm.h |    1 +
>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>
>> diff --git a/kvm.h b/kvm.h
>> index 9c7b0ea..5b3c228 100644
>> --- a/kvm.h
>> +++ b/kvm.h
>> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>>  {
>>      if (kvm_enabled()) {
>>          kvm_cpu_synchronize_post_init(env);
>> +        smp_wmb();
>>      }
>>  }
>>
>>
>
> In theory, there should be no vcpu kick-off after this without some
> locking operations involved that imply barriers. Did you see real

Yeah, but what if it is non-x86?
> inconsistencies without this explicit one?
>
> Jan
>
Jan Kiszka July 5, 2012, 10:16 a.m.
On 2012-07-05 12:10, liu ping fan wrote:
> On Thu, Jul 5, 2012 at 2:46 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
>> On 2012-07-05 04:18, Liu Ping Fan wrote:
>>> Vcpu state must be set completely before receiving INIT-IPI,SIPI
>>>
>>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>>> ---
>>>  kvm.h |    1 +
>>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/kvm.h b/kvm.h
>>> index 9c7b0ea..5b3c228 100644
>>> --- a/kvm.h
>>> +++ b/kvm.h
>>> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>>>  {
>>>      if (kvm_enabled()) {
>>>          kvm_cpu_synchronize_post_init(env);
>>> +        smp_wmb();
>>>      }
>>>  }
>>>
>>>
>>
>> In theory, there should be no vcpu kick-off after this without some
>> locking operations involved that imply barriers. Did you see real
> 
> Yeah, but what if it is non-x86?

The locking I'm referring to is arch independent.

>> inconsistencies without this explicit one?

Again: Did you see real issues or is this based on static analysis?

Jan
pingfan liu July 5, 2012, 11:02 a.m.
On Thu, Jul 5, 2012 at 6:16 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
> On 2012-07-05 12:10, liu ping fan wrote:
>> On Thu, Jul 5, 2012 at 2:46 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
>>> On 2012-07-05 04:18, Liu Ping Fan wrote:
>>>> Vcpu state must be set completely before receiving INIT-IPI,SIPI
>>>>
>>>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>>>> ---
>>>>  kvm.h |    1 +
>>>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>>>
>>>> diff --git a/kvm.h b/kvm.h
>>>> index 9c7b0ea..5b3c228 100644
>>>> --- a/kvm.h
>>>> +++ b/kvm.h
>>>> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>>>>  {
>>>>      if (kvm_enabled()) {
>>>>          kvm_cpu_synchronize_post_init(env);
>>>> +        smp_wmb();
>>>>      }
>>>>  }
>>>>
>>>>
>>>
>>> In theory, there should be no vcpu kick-off after this without some
>>> locking operations involved that imply barriers. Did you see real
>>
>> Yeah, but what if it is non-x86?
>
> The locking I'm referring to is arch independent.
>
>>> inconsistencies without this explicit one?
>
> Again: Did you see real issues or is this based on static analysis?
>
Just on  static analysis

Regards,
pingfan
> Jan
>
> --
> Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
> Corporate Competence Center Embedded Linux
>
Jan Kiszka July 5, 2012, 11:58 a.m.
On 2012-07-05 13:02, liu ping fan wrote:
> On Thu, Jul 5, 2012 at 6:16 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>> On 2012-07-05 12:10, liu ping fan wrote:
>>> On Thu, Jul 5, 2012 at 2:46 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
>>>> On 2012-07-05 04:18, Liu Ping Fan wrote:
>>>>> Vcpu state must be set completely before receiving INIT-IPI,SIPI
>>>>>
>>>>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>>>>> ---
>>>>>  kvm.h |    1 +
>>>>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>>>>
>>>>> diff --git a/kvm.h b/kvm.h
>>>>> index 9c7b0ea..5b3c228 100644
>>>>> --- a/kvm.h
>>>>> +++ b/kvm.h
>>>>> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>>>>>  {
>>>>>      if (kvm_enabled()) {
>>>>>          kvm_cpu_synchronize_post_init(env);
>>>>> +        smp_wmb();
>>>>>      }
>>>>>  }
>>>>>
>>>>>
>>>>
>>>> In theory, there should be no vcpu kick-off after this without some
>>>> locking operations involved that imply barriers. Did you see real
>>>
>>> Yeah, but what if it is non-x86?
>>
>> The locking I'm referring to is arch independent.
>>
>>>> inconsistencies without this explicit one?
>>
>> Again: Did you see real issues or is this based on static analysis?
>>
> Just on  static analysis

Then please describe - also for the changelog - at least one case in
details where this is needed.

Thanks,
Jan
pingfan liu July 6, 2012, 7:46 a.m.
On Thu, Jul 5, 2012 at 7:58 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
> On 2012-07-05 13:02, liu ping fan wrote:
>> On Thu, Jul 5, 2012 at 6:16 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>> On 2012-07-05 12:10, liu ping fan wrote:
>>>> On Thu, Jul 5, 2012 at 2:46 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
>>>>> On 2012-07-05 04:18, Liu Ping Fan wrote:
>>>>>> Vcpu state must be set completely before receiving INIT-IPI,SIPI
>>>>>>
>>>>>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>>>>>> ---
>>>>>>  kvm.h |    1 +
>>>>>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>>>>>
>>>>>> diff --git a/kvm.h b/kvm.h
>>>>>> index 9c7b0ea..5b3c228 100644
>>>>>> --- a/kvm.h
>>>>>> +++ b/kvm.h
>>>>>> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>>>>>>  {
>>>>>>      if (kvm_enabled()) {
>>>>>>          kvm_cpu_synchronize_post_init(env);
>>>>>> +        smp_wmb();
>>>>>>      }
>>>>>>  }
>>>>>>
>>>>>>
>>>>>
>>>>> In theory, there should be no vcpu kick-off after this without some
>>>>> locking operations involved that imply barriers. Did you see real
>>>>
>>>> Yeah, but what if it is non-x86?
>>>
>>> The locking I'm referring to is arch independent.
>>>
>>>>> inconsistencies without this explicit one?
>>>
>>> Again: Did you see real issues or is this based on static analysis?
>>>
>> Just on  static analysis
>
> Then please describe - also for the changelog - at least one case in
> details where this is needed.
>
I dived into code. And yes, as you said, ACPI eject does involve some
locking operation. So the only thing left is for starting up.  There
seems no potential lock operation from cpu_synchronize_all_post_init()
to  resume_all_vcpus().
If correct, I will update this info for the changelog

Regards,
pingfa

> Thanks,
> Jan
>
> --
> Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
> Corporate Competence Center Embedded Linux
>
>
Jan Kiszka July 6, 2012, 8:14 a.m.
On 2012-07-06 09:46, liu ping fan wrote:
> On Thu, Jul 5, 2012 at 7:58 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>> On 2012-07-05 13:02, liu ping fan wrote:
>>> On Thu, Jul 5, 2012 at 6:16 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>>> On 2012-07-05 12:10, liu ping fan wrote:
>>>>> On Thu, Jul 5, 2012 at 2:46 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
>>>>>> On 2012-07-05 04:18, Liu Ping Fan wrote:
>>>>>>> Vcpu state must be set completely before receiving INIT-IPI,SIPI
>>>>>>>
>>>>>>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>>>>>>> ---
>>>>>>>  kvm.h |    1 +
>>>>>>>  1 files changed, 1 insertions(+), 0 deletions(-)
>>>>>>>
>>>>>>> diff --git a/kvm.h b/kvm.h
>>>>>>> index 9c7b0ea..5b3c228 100644
>>>>>>> --- a/kvm.h
>>>>>>> +++ b/kvm.h
>>>>>>> @@ -198,6 +198,7 @@ static inline void cpu_synchronize_post_init(CPUArchState *env)
>>>>>>>  {
>>>>>>>      if (kvm_enabled()) {
>>>>>>>          kvm_cpu_synchronize_post_init(env);
>>>>>>> +        smp_wmb();
>>>>>>>      }
>>>>>>>  }
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> In theory, there should be no vcpu kick-off after this without some
>>>>>> locking operations involved that imply barriers. Did you see real
>>>>>
>>>>> Yeah, but what if it is non-x86?
>>>>
>>>> The locking I'm referring to is arch independent.
>>>>
>>>>>> inconsistencies without this explicit one?
>>>>
>>>> Again: Did you see real issues or is this based on static analysis?
>>>>
>>> Just on  static analysis
>>
>> Then please describe - also for the changelog - at least one case in
>> details where this is needed.
>>
> I dived into code. And yes, as you said, ACPI eject does involve some
> locking operation. So the only thing left is for starting up.  There
> seems no potential lock operation from cpu_synchronize_all_post_init()
> to  resume_all_vcpus().

There is no difference in this scenario as well: the vcpu is waiting on
a condvar for the initial kick-off and will synchronize with the main
thread via the global mutex. When in doubt, try using a debugger.

Jan

Patch hide | download patch | download mbox

diff --git a/kvm.h b/kvm.h
index 9c7b0ea..5b3c228 100644
--- a/kvm.h
+++ b/kvm.h
@@ -198,6 +198,7 @@  static inline void cpu_synchronize_post_init(CPUArchState *env)
 {
     if (kvm_enabled()) {
         kvm_cpu_synchronize_post_init(env);
+        smp_wmb();
     }
 }