[RFC,04/11] powerpc: kvm: introduce a kthread on primary thread to anti tickless
diff mbox

Message ID 1413487800-7162-5-git-send-email-kernelfans@gmail.com
State RFC
Headers show

Commit Message

Pingfan Liu Oct. 16, 2014, 7:29 p.m. UTC
(This patch is a place holder.)

If there is only one vcpu thread is ready(the other vcpu thread can
wait for it to execute), the primary thread can enter tickless mode,
which causes the primary keeps running, so the secondary has no
opportunity to exit to host, even they have other tsk on them.

Introduce a kthread (anti_tickless) on primary, so when there is only
one vcpu thread on primary, the secondary can resort to anti_tickless
to keep the primary out of tickless mode.
(I thought that anti_tickless thread can goto NAP, so we can let the
secondary run).

Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
---
 arch/powerpc/kernel/sysfs.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

Comments

Preeti U Murthy Oct. 27, 2014, 4:45 a.m. UTC | #1
On 10/17/2014 12:59 AM, kernelfans@gmail.com wrote:
> (This patch is a place holder.)
> 
> If there is only one vcpu thread is ready(the other vcpu thread can
> wait for it to execute), the primary thread can enter tickless mode,

We do not configure NOHZ_FULL to y by default. Hence no thread would
enter tickless mode.

> which causes the primary keeps running, so the secondary has no
> opportunity to exit to host, even they have other tsk on them.

The secondary threads can still get scheduling ticks. The decrementer of
the secondary threads is still active. So as long as secondary threads
are busy, scheduling ticks will fire and try to schedule a new task on them.

Regards
Preeti U Murthy
> 
> Introduce a kthread (anti_tickless) on primary, so when there is only
> one vcpu thread on primary, the secondary can resort to anti_tickless
> to keep the primary out of tickless mode.
> (I thought that anti_tickless thread can goto NAP, so we can let the
> secondary run).
> 
> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
> ---
>  arch/powerpc/kernel/sysfs.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/powerpc/kernel/sysfs.c b/arch/powerpc/kernel/sysfs.c
> index a2595dd..f0b110e 100644
> --- a/arch/powerpc/kernel/sysfs.c
> +++ b/arch/powerpc/kernel/sysfs.c
> @@ -575,9 +575,11 @@ static ssize_t __used store_kvm_enable(struct device *dev,
>  	if (!test_bit(core, &kvm_on_core))
>  		for (thr = 1; thr< threads_per_core; thr++)
>  			if (cpu_online(thr * threads_per_core + thr))
> -				cpumask_set_cpu(thr * threads_per_core + thr, &stop_cpus);
> +				cpumask_set_cpu(core * threads_per_core + thr, &stop_cpus);
>  
>  	stop_machine(xics_migrate_irqs_away_secondary, NULL, &stop_cpus);
> +	/* fixme, create a kthread on primary hwthread to handle tickless mode */
> +	//kthread_create_on_cpu(prevent_tickless, NULL, core * threads_per_core, "ppckvm_prevent_tickless");
>  	set_bit(core, &kvm_on_core);
>  	return count;
>  }
>
Pingfan Liu Nov. 18, 2014, 5:24 a.m. UTC | #2
On Mon, Oct 27, 2014 at 12:45 PM, Preeti U Murthy
<preeti@linux.vnet.ibm.com> wrote:
> On 10/17/2014 12:59 AM, kernelfans@gmail.com wrote:
>> (This patch is a place holder.)
>>
>> If there is only one vcpu thread is ready(the other vcpu thread can
>> wait for it to execute), the primary thread can enter tickless mode,
>
> We do not configure NOHZ_FULL to y by default. Hence no thread would
> enter tickless mode.
>
But NOHZ_FULL can be chosen by user, and we should survive from it :)

>> which causes the primary keeps running, so the secondary has no
>> opportunity to exit to host, even they have other tsk on them.
>
> The secondary threads can still get scheduling ticks. The decrementer of
> the secondary threads is still active. So as long as secondary threads
> are busy, scheduling ticks will fire and try to schedule a new task on them.
>
No. As my original thought, after enable KVM on core, the HDEC on
secondary is disabled, otherwise the host exit will be too frequent.
Any suggestion?

Thx,
Fan

> Regards
> Preeti U Murthy
>>
>> Introduce a kthread (anti_tickless) on primary, so when there is only
>> one vcpu thread on primary, the secondary can resort to anti_tickless
>> to keep the primary out of tickless mode.
>> (I thought that anti_tickless thread can goto NAP, so we can let the
>> secondary run).
>>
>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>> ---
>>  arch/powerpc/kernel/sysfs.c | 4 +++-
>>  1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/powerpc/kernel/sysfs.c b/arch/powerpc/kernel/sysfs.c
>> index a2595dd..f0b110e 100644
>> --- a/arch/powerpc/kernel/sysfs.c
>> +++ b/arch/powerpc/kernel/sysfs.c
>> @@ -575,9 +575,11 @@ static ssize_t __used store_kvm_enable(struct device *dev,
>>       if (!test_bit(core, &kvm_on_core))
>>               for (thr = 1; thr< threads_per_core; thr++)
>>                       if (cpu_online(thr * threads_per_core + thr))
>> -                             cpumask_set_cpu(thr * threads_per_core + thr, &stop_cpus);
>> +                             cpumask_set_cpu(core * threads_per_core + thr, &stop_cpus);
>>
>>       stop_machine(xics_migrate_irqs_away_secondary, NULL, &stop_cpus);
>> +     /* fixme, create a kthread on primary hwthread to handle tickless mode */
>> +     //kthread_create_on_cpu(prevent_tickless, NULL, core * threads_per_core, "ppckvm_prevent_tickless");
>>       set_bit(core, &kvm_on_core);
>>       return count;
>>  }
>>
>

Patch
diff mbox

diff --git a/arch/powerpc/kernel/sysfs.c b/arch/powerpc/kernel/sysfs.c
index a2595dd..f0b110e 100644
--- a/arch/powerpc/kernel/sysfs.c
+++ b/arch/powerpc/kernel/sysfs.c
@@ -575,9 +575,11 @@  static ssize_t __used store_kvm_enable(struct device *dev,
 	if (!test_bit(core, &kvm_on_core))
 		for (thr = 1; thr< threads_per_core; thr++)
 			if (cpu_online(thr * threads_per_core + thr))
-				cpumask_set_cpu(thr * threads_per_core + thr, &stop_cpus);
+				cpumask_set_cpu(core * threads_per_core + thr, &stop_cpus);
 
 	stop_machine(xics_migrate_irqs_away_secondary, NULL, &stop_cpus);
+	/* fixme, create a kthread on primary hwthread to handle tickless mode */
+	//kthread_create_on_cpu(prevent_tickless, NULL, core * threads_per_core, "ppckvm_prevent_tickless");
 	set_bit(core, &kvm_on_core);
 	return count;
 }