diff mbox

[qemu] sysemu: support up to 1024 vCPUs

Message ID 1475541180-16363-1-git-send-email-aik@ozlabs.ru
State New
Headers show

Commit Message

Alexey Kardashevskiy Oct. 4, 2016, 12:33 a.m. UTC
From: Greg Kurz <gkurz@linux.vnet.ibm.com>

Some systems can already provide more than 255 hardware threads.

Bumping the QEMU limit to 1024 seems reasonable:
- it has no visible overhead in top;
- the limit itself has no effect on hot paths.

Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
 include/sysemu/sysemu.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Alexey Kardashevskiy Oct. 10, 2016, 10:19 p.m. UTC | #1
Ping, anyone?


On 04/10/16 11:33, Alexey Kardashevskiy wrote:
> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> 
> Some systems can already provide more than 255 hardware threads.
> 
> Bumping the QEMU limit to 1024 seems reasonable:
> - it has no visible overhead in top;
> - the limit itself has no effect on hot paths.
> 
> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
>  include/sysemu/sysemu.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> index ef2c50b..2ec0bd8 100644
> --- a/include/sysemu/sysemu.h
> +++ b/include/sysemu/sysemu.h
> @@ -173,7 +173,7 @@ extern int mem_prealloc;
>   *
>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
>   */
> -#define MAX_CPUMASK_BITS 255
> +#define MAX_CPUMASK_BITS 1024
>  
>  #define MAX_OPTION_ROMS 16
>  typedef struct QEMUOptionRom {
>
Alexey Kardashevskiy Oct. 18, 2016, 1:07 a.m. UTC | #2
Ping, anyone?

I rather expected floods of mails on such a controversial topic :)


On 11/10/16 09:19, Alexey Kardashevskiy wrote:
> Ping, anyone?
> 
> 
> On 04/10/16 11:33, Alexey Kardashevskiy wrote:
>> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
>>
>> Some systems can already provide more than 255 hardware threads.
>>
>> Bumping the QEMU limit to 1024 seems reasonable:
>> - it has no visible overhead in top;
>> - the limit itself has no effect on hot paths.
>>
>> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>> ---
>>  include/sysemu/sysemu.h | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
>> index ef2c50b..2ec0bd8 100644
>> --- a/include/sysemu/sysemu.h
>> +++ b/include/sysemu/sysemu.h
>> @@ -173,7 +173,7 @@ extern int mem_prealloc;
>>   *
>>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
>>   */
>> -#define MAX_CPUMASK_BITS 255
>> +#define MAX_CPUMASK_BITS 1024
>>  
>>  #define MAX_OPTION_ROMS 16
>>  typedef struct QEMUOptionRom {
>>
> 
>
David Gibson Oct. 18, 2016, 1:40 a.m. UTC | #3
On Tue, Oct 18, 2016 at 12:07:20PM +1100, Alexey Kardashevskiy wrote:
> Ping, anyone?
> 
> I rather expected floods of mails on such a controversial topic :)

I think it's the reverse of the bike-shed problem.  Hardly anyone
feels qualified to comment on it.

It looks ok to me, but I just don't know if there are other subtle
dependencies on the size of the cpumask buried in the code.

Oh well,  let's say:

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>

and find any problems as they arise.

Not sure who we need to convince to take this into their tree though.

> 
> 
> On 11/10/16 09:19, Alexey Kardashevskiy wrote:
> > Ping, anyone?
> > 
> > 
> > On 04/10/16 11:33, Alexey Kardashevskiy wrote:
> >> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> >>
> >> Some systems can already provide more than 255 hardware threads.
> >>
> >> Bumping the QEMU limit to 1024 seems reasonable:
> >> - it has no visible overhead in top;
> >> - the limit itself has no effect on hot paths.
> >>
> >> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> >> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >> ---
> >>  include/sysemu/sysemu.h | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> >> index ef2c50b..2ec0bd8 100644
> >> --- a/include/sysemu/sysemu.h
> >> +++ b/include/sysemu/sysemu.h
> >> @@ -173,7 +173,7 @@ extern int mem_prealloc;
> >>   *
> >>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
> >>   */
> >> -#define MAX_CPUMASK_BITS 255
> >> +#define MAX_CPUMASK_BITS 1024
> >>  
> >>  #define MAX_OPTION_ROMS 16
> >>  typedef struct QEMUOptionRom {
> >>
> > 
> > 
> 
>
Igor Mammedov Oct. 18, 2016, 11 a.m. UTC | #4
On Tue, 11 Oct 2016 09:19:10 +1100
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> Ping, anyone?
I have a similar patch
http://patchwork.ozlabs.org/patch/681709/
which bumps limit to 288 and does a little bit more
so it wouldn't affect current users.

After that's merged, I plan to get rid of this limit and
make that part of numa parsing code dynamic so that it
wouldn't impose such limit/any limits on target code.

> 
> 
> On 04/10/16 11:33, Alexey Kardashevskiy wrote:
> > From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > 
> > Some systems can already provide more than 255 hardware threads.
> > 
> > Bumping the QEMU limit to 1024 seems reasonable:
> > - it has no visible overhead in top;
> > - the limit itself has no effect on hot paths.
> > 
> > Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> > ---
> >  include/sysemu/sysemu.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> > index ef2c50b..2ec0bd8 100644
> > --- a/include/sysemu/sysemu.h
> > +++ b/include/sysemu/sysemu.h
> > @@ -173,7 +173,7 @@ extern int mem_prealloc;
> >   *
> >   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
> >   */
> > -#define MAX_CPUMASK_BITS 255
> > +#define MAX_CPUMASK_BITS 1024
> >  
> >  #define MAX_OPTION_ROMS 16
> >  typedef struct QEMUOptionRom {
> >   
> 
>
Greg Kurz Oct. 18, 2016, 11:56 a.m. UTC | #5
On Tue, 18 Oct 2016 13:00:07 +0200
Igor Mammedov <imammedo@redhat.com> wrote:

> On Tue, 11 Oct 2016 09:19:10 +1100
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
> > Ping, anyone?  
> I have a similar patch
> http://patchwork.ozlabs.org/patch/681709/
> which bumps limit to 288 and does a little bit more
> so it wouldn't affect current users.
> 

Heh the bumping to 1024 was kinda arbitrary but 288 would be an
improvement anyway. FYI, only Alpine class servers would benefit
from that since they have 320 threads, even if they're not officially
supported to run a KVM based hypervisor (biggest supported model is
Tuleta with 192 threads).

BTW, the afore mentioned patch keeps the current 255 limit for
pseries machine types. I guess David's answer means that the
hw/ppc/spapr.c hunk can be safely dropped.

> After that's merged, I plan to get rid of this limit and
> make that part of numa parsing code dynamic so that it
> wouldn't impose such limit/any limits on target code.
> 
> > 
> > 
> > On 04/10/16 11:33, Alexey Kardashevskiy wrote:  
> > > From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > > 
> > > Some systems can already provide more than 255 hardware threads.
> > > 
> > > Bumping the QEMU limit to 1024 seems reasonable:
> > > - it has no visible overhead in top;
> > > - the limit itself has no effect on hot paths.
> > > 
> > > Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> > > ---
> > >  include/sysemu/sysemu.h | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > 
> > > diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> > > index ef2c50b..2ec0bd8 100644
> > > --- a/include/sysemu/sysemu.h
> > > +++ b/include/sysemu/sysemu.h
> > > @@ -173,7 +173,7 @@ extern int mem_prealloc;
> > >   *
> > >   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
> > >   */
> > > -#define MAX_CPUMASK_BITS 255
> > > +#define MAX_CPUMASK_BITS 1024
> > >  
> > >  #define MAX_OPTION_ROMS 16
> > >  typedef struct QEMUOptionRom {
> > >     
> > 
> >   
>
Alexey Kardashevskiy Oct. 18, 2016, 11:39 p.m. UTC | #6
On 18/10/16 22:00, Igor Mammedov wrote:
> On Tue, 11 Oct 2016 09:19:10 +1100
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
>> Ping, anyone?
> I have a similar patch
> http://patchwork.ozlabs.org/patch/681709/
> which bumps limit to 288 and does a little bit more
> so it wouldn't affect current users.

Why 288 (not oldlimit<<n)? :)

What does happen to the Greg's patch now?


> 
> After that's merged, I plan to get rid of this limit and
> make that part of numa parsing code dynamic so that it
> wouldn't impose such limit/any limits on target code.
> 
>>
>>
>> On 04/10/16 11:33, Alexey Kardashevskiy wrote:
>>> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
>>>
>>> Some systems can already provide more than 255 hardware threads.
>>>
>>> Bumping the QEMU limit to 1024 seems reasonable:
>>> - it has no visible overhead in top;
>>> - the limit itself has no effect on hot paths.
>>>
>>> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>> ---
>>>  include/sysemu/sysemu.h | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
>>> index ef2c50b..2ec0bd8 100644
>>> --- a/include/sysemu/sysemu.h
>>> +++ b/include/sysemu/sysemu.h
>>> @@ -173,7 +173,7 @@ extern int mem_prealloc;
>>>   *
>>>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
>>>   */
>>> -#define MAX_CPUMASK_BITS 255
>>> +#define MAX_CPUMASK_BITS 1024
>>>  
>>>  #define MAX_OPTION_ROMS 16
>>>  typedef struct QEMUOptionRom {
>>>   
>>
>>
>
Igor Mammedov Oct. 19, 2016, 12:23 p.m. UTC | #7
On Wed, 19 Oct 2016 10:39:12 +1100
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 18/10/16 22:00, Igor Mammedov wrote:
> > On Tue, 11 Oct 2016 09:19:10 +1100
> > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> >   
> >> Ping, anyone?  
> > I have a similar patch
> > http://patchwork.ozlabs.org/patch/681709/
> > which bumps limit to 288 and does a little bit more
> > so it wouldn't affect current users.  
> 
> Why 288 (not oldlimit<<n)? :)
That's how many vCPU we plan to support for target-i386.


> What does happen to the Greg's patch now?
it could be amended to apply top of above patch,

where it would change not only MAX_CPUMASK_BITS but also
maxcpus in ppc target since above patch unties maxcpus
from MAX_CPUMASK_BITS for all targets.

So followup increases of MAX_CPUMASK_BITS won't affect
other targets.

> 
> 
> > 
> > After that's merged, I plan to get rid of this limit and
> > make that part of numa parsing code dynamic so that it
> > wouldn't impose such limit/any limits on target code.
> >   
> >>
> >>
> >> On 04/10/16 11:33, Alexey Kardashevskiy wrote:  
> >>> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> >>>
> >>> Some systems can already provide more than 255 hardware threads.
> >>>
> >>> Bumping the QEMU limit to 1024 seems reasonable:
> >>> - it has no visible overhead in top;
> >>> - the limit itself has no effect on hot paths.
> >>>
> >>> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> >>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >>> ---
> >>>  include/sysemu/sysemu.h | 2 +-
> >>>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>>
> >>> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> >>> index ef2c50b..2ec0bd8 100644
> >>> --- a/include/sysemu/sysemu.h
> >>> +++ b/include/sysemu/sysemu.h
> >>> @@ -173,7 +173,7 @@ extern int mem_prealloc;
> >>>   *
> >>>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
> >>>   */
> >>> -#define MAX_CPUMASK_BITS 255
> >>> +#define MAX_CPUMASK_BITS 1024
> >>>  
> >>>  #define MAX_OPTION_ROMS 16
> >>>  typedef struct QEMUOptionRom {
> >>>     
> >>
> >>  
> >   
> 
>
Alexey Kardashevskiy Oct. 24, 2016, 1:53 a.m. UTC | #8
On 19/10/16 23:23, Igor Mammedov wrote:
> On Wed, 19 Oct 2016 10:39:12 +1100
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
>> On 18/10/16 22:00, Igor Mammedov wrote:
>>> On Tue, 11 Oct 2016 09:19:10 +1100
>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>   
>>>> Ping, anyone?  
>>> I have a similar patch
>>> http://patchwork.ozlabs.org/patch/681709/
>>> which bumps limit to 288 and does a little bit more
>>> so it wouldn't affect current users.  
>>
>> Why 288 (not oldlimit<<n)? :)
> That's how many vCPU we plan to support for target-i386.


So that patch from patchworks should actually be 3 patches -
- untie MAX_CPUMASK_BITS from maxcpus;
- increase MAX_CPUMASK_BITS to 1024 (till we get this thing totally dynamic);
- increase maxcpus for x86 to 288.

and on top of this I (or Greg) would repost pseries change to maxcpus=1024

Correct? Has your patch gone anywhere yet?



> 
> 
>> What does happen to the Greg's patch now?
> it could be amended to apply top of above patch,
> 
> where it would change not only MAX_CPUMASK_BITS but also
> maxcpus in ppc target since above patch unties maxcpus
> from MAX_CPUMASK_BITS for all targets.
> 
> So followup increases of MAX_CPUMASK_BITS won't affect
> other targets.
> 
>>
>>
>>>
>>> After that's merged, I plan to get rid of this limit and
>>> make that part of numa parsing code dynamic so that it
>>> wouldn't impose such limit/any limits on target code.
>>>   
>>>>
>>>>
>>>> On 04/10/16 11:33, Alexey Kardashevskiy wrote:  
>>>>> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
>>>>>
>>>>> Some systems can already provide more than 255 hardware threads.
>>>>>
>>>>> Bumping the QEMU limit to 1024 seems reasonable:
>>>>> - it has no visible overhead in top;
>>>>> - the limit itself has no effect on hot paths.
>>>>>
>>>>> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>>>> ---
>>>>>  include/sysemu/sysemu.h | 2 +-
>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
>>>>> index ef2c50b..2ec0bd8 100644
>>>>> --- a/include/sysemu/sysemu.h
>>>>> +++ b/include/sysemu/sysemu.h
>>>>> @@ -173,7 +173,7 @@ extern int mem_prealloc;
>>>>>   *
>>>>>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
>>>>>   */
>>>>> -#define MAX_CPUMASK_BITS 255
>>>>> +#define MAX_CPUMASK_BITS 1024
>>>>>  
>>>>>  #define MAX_OPTION_ROMS 16
>>>>>  typedef struct QEMUOptionRom {
>>>>>     
>>>>
>>>>  
>>>   
>>
>>
>
Igor Mammedov Oct. 24, 2016, 9:22 a.m. UTC | #9
On Mon, 24 Oct 2016 12:53:10 +1100
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 19/10/16 23:23, Igor Mammedov wrote:
> > On Wed, 19 Oct 2016 10:39:12 +1100
> > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> >   
> >> On 18/10/16 22:00, Igor Mammedov wrote:  
> >>> On Tue, 11 Oct 2016 09:19:10 +1100
> >>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> >>>     
> >>>> Ping, anyone?    
> >>> I have a similar patch
> >>> http://patchwork.ozlabs.org/patch/681709/
> >>> which bumps limit to 288 and does a little bit more
> >>> so it wouldn't affect current users.    
> >>
> >> Why 288 (not oldlimit<<n)? :)  
> > That's how many vCPU we plan to support for target-i386.  
> 
> 
> So that patch from patchworks should actually be 3 patches -
> - untie MAX_CPUMASK_BITS from maxcpus;
> - increase MAX_CPUMASK_BITS to 1024 (till we get this thing totally dynamic);
> - increase maxcpus for x86 to 288.
> 
> and on top of this I (or Greg) would repost pseries change to maxcpus=1024
> 
> Correct? Has your patch gone anywhere yet?
It's been reviewed and I expect it merged soon.

on top of it 
MAX_CPUMASK_BITS could be bumped up along with pseries.maxcpus

 
> >   
> >> What does happen to the Greg's patch now?  
> > it could be amended to apply top of above patch,
> > 
> > where it would change not only MAX_CPUMASK_BITS but also
> > maxcpus in ppc target since above patch unties maxcpus
> > from MAX_CPUMASK_BITS for all targets.
> > 
> > So followup increases of MAX_CPUMASK_BITS won't affect
> > other targets.
> >   
> >>
> >>  
> >>>
> >>> After that's merged, I plan to get rid of this limit and
> >>> make that part of numa parsing code dynamic so that it
> >>> wouldn't impose such limit/any limits on target code.
> >>>     
> >>>>
> >>>>
> >>>> On 04/10/16 11:33, Alexey Kardashevskiy wrote:    
> >>>>> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> >>>>>
> >>>>> Some systems can already provide more than 255 hardware threads.
> >>>>>
> >>>>> Bumping the QEMU limit to 1024 seems reasonable:
> >>>>> - it has no visible overhead in top;
> >>>>> - the limit itself has no effect on hot paths.
> >>>>>
> >>>>> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> >>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >>>>> ---
> >>>>>  include/sysemu/sysemu.h | 2 +-
> >>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>>>>
> >>>>> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> >>>>> index ef2c50b..2ec0bd8 100644
> >>>>> --- a/include/sysemu/sysemu.h
> >>>>> +++ b/include/sysemu/sysemu.h
> >>>>> @@ -173,7 +173,7 @@ extern int mem_prealloc;
> >>>>>   *
> >>>>>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
> >>>>>   */
> >>>>> -#define MAX_CPUMASK_BITS 255
> >>>>> +#define MAX_CPUMASK_BITS 1024
> >>>>>  
> >>>>>  #define MAX_OPTION_ROMS 16
> >>>>>  typedef struct QEMUOptionRom {
> >>>>>       
> >>>>
> >>>>    
> >>>     
> >>
> >>  
> >   
> 
>
diff mbox

Patch

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index ef2c50b..2ec0bd8 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -173,7 +173,7 @@  extern int mem_prealloc;
  *
  * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
  */
-#define MAX_CPUMASK_BITS 255
+#define MAX_CPUMASK_BITS 1024
 
 #define MAX_OPTION_ROMS 16
 typedef struct QEMUOptionRom {