Patchwork vhost: force vhost off for non-MSI guests

login
register
mail settings
Submitter Michael S. Tsirkin
Date Jan. 20, 2011, 4:07 p.m.
Message ID <20110120160718.GA24652@redhat.com>
Download mbox | patch
Permalink /patch/79713/
State New
Headers show

Comments

Michael S. Tsirkin - Jan. 20, 2011, 4:07 p.m.
On Thu, Jan 20, 2011 at 09:43:57AM -0600, Anthony Liguori wrote:
> On 01/20/2011 09:35 AM, Michael S. Tsirkin wrote:
> >When MSI is off, each interrupt needs to be bounced through the io
> >thread when it's set/cleared, so vhost-net causes more context switches and
> >higher CPU utilization than userspace virtio which handles networking in
> >the same thread.
> >
> >We'll need to fix this by adding level irq support in kvm irqfd,
> >for now disable vhost-net in these configurations.
> >
> >Signed-off-by: Michael S. Tsirkin<mst@redhat.com>
> 
> I actually think this should be a terminal error.  The user asks for
> vhost-net, if we cannot enable it, we should exit.
> 
> Or we should warn the user that they should expect bad performance.
> Silently doing something that the user has explicitly asked us not
> to do is not a good behavior.
> 
> Regards,
> 
> Anthony Liguori

The issue is that user has no control of the guest, and can not know
whether the guest enables MSI. So what you ask for will just make
some guests fail, and others fail sometimes.
The user also has no way to know that version X of kvm does not expose a
way to inject level interrupts with irqfd.

We could have *another* flag that says "use vhost where it helps" but
then I think this is what everyone wants to do, anyway, and libvirt
already sets vhost=on so I prefer redefining the meaning of an existing
flag.

Maybe this is best handled by a documentation update?

We always said:
	    "                use vhost=on to enable experimental in kernel accelerator\n"

note 'enable' not 'require'. This is similar to how we specify
nvectors : you can not make guest use the feature.

How about this:
Anthony Liguori - Jan. 21, 2011, 12:23 a.m.
On 01/20/2011 10:07 AM, Michael S. Tsirkin wrote:
> On Thu, Jan 20, 2011 at 09:43:57AM -0600, Anthony Liguori wrote:
>    
>> On 01/20/2011 09:35 AM, Michael S. Tsirkin wrote:
>>      
>>> When MSI is off, each interrupt needs to be bounced through the io
>>> thread when it's set/cleared, so vhost-net causes more context switches and
>>> higher CPU utilization than userspace virtio which handles networking in
>>> the same thread.
>>>
>>> We'll need to fix this by adding level irq support in kvm irqfd,
>>> for now disable vhost-net in these configurations.
>>>
>>> Signed-off-by: Michael S. Tsirkin<mst@redhat.com>
>>>        
>> I actually think this should be a terminal error.  The user asks for
>> vhost-net, if we cannot enable it, we should exit.
>>
>> Or we should warn the user that they should expect bad performance.
>> Silently doing something that the user has explicitly asked us not
>> to do is not a good behavior.
>>
>> Regards,
>>
>> Anthony Liguori
>>      
> The issue is that user has no control of the guest, and can not know
> whether the guest enables MSI. So what you ask for will just make
> some guests fail, and others fail sometimes.
> The user also has no way to know that version X of kvm does not expose a
> way to inject level interrupts with irqfd.
>
> We could have *another* flag that says "use vhost where it helps" but
> then I think this is what everyone wants to do, anyway, and libvirt
> already sets vhost=on so I prefer redefining the meaning of an existing
> flag.
>    

In the very least, there needs to be a vhost=force.

Having some sort of friendly default policy is fine but we need to 
provide a mechanism for a user to have the final say.  If you want to 
redefine vhost=on to really mean, use the friendly default, that's fine 
by me, but only if the vhost=force option exists.

I actually would think libvirt would want to use vhost=force.  Debugging 
with vhost=on is going to be a royal pain in the ass if a user reports 
bad performance.  Given the libvirt XML, you can't actually tell from 
the guest and the XML whether or not vhost was actually in use or not.

Regards,

Anthony Liguori

> Maybe this is best handled by a documentation update?
>
> We always said:
> 	    "                use vhost=on to enable experimental in kernel accelerator\n"
>
> note 'enable' not 'require'. This is similar to how we specify
> nvectors : you can not make guest use the feature.
>
> How about this:
>
> diff --git a/qemu-options.hx b/qemu-options.hx
> index 898561d..3c937c1 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -1061,6 +1061,7 @@ DEF("net", HAS_ARG, QEMU_OPTION_net,
>       "                use vnet_hdr=off to avoid enabling the IFF_VNET_HDR tap flag\n"
>       "                use vnet_hdr=on to make the lack of IFF_VNET_HDR support an error condition\n"
>       "                use vhost=on to enable experimental in kernel accelerator\n"
> +    "                (note: vhost=on has no effect unless guest uses MSI-X)\n"
>       "                use 'vhostfd=h' to connect to an already opened vhost net device\n"
>   #endif
>       "-net socket[,vlan=n][,name=str][,fd=h][,listen=[host]:port][,connect=host:port]\n"
>
>
>
Alex Williamson - Jan. 21, 2011, 1:35 a.m.
On Thu, 2011-01-20 at 18:23 -0600, Anthony Liguori wrote:
> On 01/20/2011 10:07 AM, Michael S. Tsirkin wrote:
> > On Thu, Jan 20, 2011 at 09:43:57AM -0600, Anthony Liguori wrote:
> >    
> >> On 01/20/2011 09:35 AM, Michael S. Tsirkin wrote:
> >>      
> >>> When MSI is off, each interrupt needs to be bounced through the io
> >>> thread when it's set/cleared, so vhost-net causes more context switches and
> >>> higher CPU utilization than userspace virtio which handles networking in
> >>> the same thread.
> >>>
> >>> We'll need to fix this by adding level irq support in kvm irqfd,
> >>> for now disable vhost-net in these configurations.
> >>>
> >>> Signed-off-by: Michael S. Tsirkin<mst@redhat.com>
> >>>        
> >> I actually think this should be a terminal error.  The user asks for
> >> vhost-net, if we cannot enable it, we should exit.
> >>
> >> Or we should warn the user that they should expect bad performance.
> >> Silently doing something that the user has explicitly asked us not
> >> to do is not a good behavior.
> >>
> >> Regards,
> >>
> >> Anthony Liguori
> >>      
> > The issue is that user has no control of the guest, and can not know
> > whether the guest enables MSI. So what you ask for will just make
> > some guests fail, and others fail sometimes.
> > The user also has no way to know that version X of kvm does not expose a
> > way to inject level interrupts with irqfd.
> >
> > We could have *another* flag that says "use vhost where it helps" but
> > then I think this is what everyone wants to do, anyway, and libvirt
> > already sets vhost=on so I prefer redefining the meaning of an existing
> > flag.
> >    
> 
> In the very least, there needs to be a vhost=force.
> 
> Having some sort of friendly default policy is fine but we need to 
> provide a mechanism for a user to have the final say.  If you want to 
> redefine vhost=on to really mean, use the friendly default, that's fine 
> by me, but only if the vhost=force option exists.
> 
> I actually would think libvirt would want to use vhost=force.  Debugging 
> with vhost=on is going to be a royal pain in the ass if a user reports 
> bad performance.  Given the libvirt XML, you can't actually tell from 
> the guest and the XML whether or not vhost was actually in use or not.

If we add a force option, let's please distinguish hotplug from VM
creation time.  The latter can abort.  Hotplug should print an error and
fail the initfn.  Thanks,

Alex
Michael S. Tsirkin - Jan. 21, 2011, 9:48 a.m.
On Thu, Jan 20, 2011 at 06:23:36PM -0600, Anthony Liguori wrote:
> On 01/20/2011 10:07 AM, Michael S. Tsirkin wrote:
> >On Thu, Jan 20, 2011 at 09:43:57AM -0600, Anthony Liguori wrote:
> >>On 01/20/2011 09:35 AM, Michael S. Tsirkin wrote:
> >>>When MSI is off, each interrupt needs to be bounced through the io
> >>>thread when it's set/cleared, so vhost-net causes more context switches and
> >>>higher CPU utilization than userspace virtio which handles networking in
> >>>the same thread.
> >>>
> >>>We'll need to fix this by adding level irq support in kvm irqfd,
> >>>for now disable vhost-net in these configurations.
> >>>
> >>>Signed-off-by: Michael S. Tsirkin<mst@redhat.com>
> >>I actually think this should be a terminal error.  The user asks for
> >>vhost-net, if we cannot enable it, we should exit.
> >>
> >>Or we should warn the user that they should expect bad performance.
> >>Silently doing something that the user has explicitly asked us not
> >>to do is not a good behavior.
> >>
> >>Regards,
> >>
> >>Anthony Liguori
> >The issue is that user has no control of the guest, and can not know
> >whether the guest enables MSI. So what you ask for will just make
> >some guests fail, and others fail sometimes.
> >The user also has no way to know that version X of kvm does not expose a
> >way to inject level interrupts with irqfd.
> >
> >We could have *another* flag that says "use vhost where it helps" but
> >then I think this is what everyone wants to do, anyway, and libvirt
> >already sets vhost=on so I prefer redefining the meaning of an existing
> >flag.
> 
> In the very least, there needs to be a vhost=force.
> Having some sort of friendly default policy is fine but we need to
> provide a mechanism for a user to have the final say.  If you want
> to redefine vhost=on to really mean, use the friendly default,
> that's fine by me, but only if the vhost=force option exists.

OK, I will add that, probably as a separate flag as vhost
is a boolean.  This will get worse performance but it will be what the
user asked for.

> 
> I actually would think libvirt would want to use vhost=force.
> Debugging with vhost=on is going to be a royal pain in the ass if a
> user reports bad performance.  Given the libvirt XML, you can't
> actually tell from the guest and the XML whether or not vhost was
> actually in use or not.

Yes you can: check MSI enabled in the guest, if it is -
check vhost enabled in the XML. Not that bad at all, is it?

> 
> Regards,
> 
> Anthony Liguori

We get worse performance without MSI anyway, how is this different?

> >Maybe this is best handled by a documentation update?
> >
> >We always said:
> >	    "                use vhost=on to enable experimental in kernel accelerator\n"
> >
> >note 'enable' not 'require'. This is similar to how we specify
> >nvectors : you can not make guest use the feature.
> >
> >How about this:
> >
> >diff --git a/qemu-options.hx b/qemu-options.hx
> >index 898561d..3c937c1 100644
> >--- a/qemu-options.hx
> >+++ b/qemu-options.hx
> >@@ -1061,6 +1061,7 @@ DEF("net", HAS_ARG, QEMU_OPTION_net,
> >      "                use vnet_hdr=off to avoid enabling the IFF_VNET_HDR tap flag\n"
> >      "                use vnet_hdr=on to make the lack of IFF_VNET_HDR support an error condition\n"
> >      "                use vhost=on to enable experimental in kernel accelerator\n"
> >+    "                (note: vhost=on has no effect unless guest uses MSI-X)\n"
> >      "                use 'vhostfd=h' to connect to an already opened vhost net device\n"
> >  #endif
> >      "-net socket[,vlan=n][,name=str][,fd=h][,listen=[host]:port][,connect=host:port]\n"
> >
> >
Michael S. Tsirkin - Jan. 21, 2011, 9:55 a.m.
On Thu, Jan 20, 2011 at 06:35:46PM -0700, Alex Williamson wrote:
> On Thu, 2011-01-20 at 18:23 -0600, Anthony Liguori wrote:
> > On 01/20/2011 10:07 AM, Michael S. Tsirkin wrote:
> > > On Thu, Jan 20, 2011 at 09:43:57AM -0600, Anthony Liguori wrote:
> > >    
> > >> On 01/20/2011 09:35 AM, Michael S. Tsirkin wrote:
> > >>      
> > >>> When MSI is off, each interrupt needs to be bounced through the io
> > >>> thread when it's set/cleared, so vhost-net causes more context switches and
> > >>> higher CPU utilization than userspace virtio which handles networking in
> > >>> the same thread.
> > >>>
> > >>> We'll need to fix this by adding level irq support in kvm irqfd,
> > >>> for now disable vhost-net in these configurations.
> > >>>
> > >>> Signed-off-by: Michael S. Tsirkin<mst@redhat.com>
> > >>>        
> > >> I actually think this should be a terminal error.  The user asks for
> > >> vhost-net, if we cannot enable it, we should exit.
> > >>
> > >> Or we should warn the user that they should expect bad performance.
> > >> Silently doing something that the user has explicitly asked us not
> > >> to do is not a good behavior.
> > >>
> > >> Regards,
> > >>
> > >> Anthony Liguori
> > >>      
> > > The issue is that user has no control of the guest, and can not know
> > > whether the guest enables MSI. So what you ask for will just make
> > > some guests fail, and others fail sometimes.
> > > The user also has no way to know that version X of kvm does not expose a
> > > way to inject level interrupts with irqfd.
> > >
> > > We could have *another* flag that says "use vhost where it helps" but
> > > then I think this is what everyone wants to do, anyway, and libvirt
> > > already sets vhost=on so I prefer redefining the meaning of an existing
> > > flag.
> > >    
> > 
> > In the very least, there needs to be a vhost=force.
> > 
> > Having some sort of friendly default policy is fine but we need to 
> > provide a mechanism for a user to have the final say.  If you want to 
> > redefine vhost=on to really mean, use the friendly default, that's fine 
> > by me, but only if the vhost=force option exists.
> > 
> > I actually would think libvirt would want to use vhost=force.  Debugging 
> > with vhost=on is going to be a royal pain in the ass if a user reports 
> > bad performance.  Given the libvirt XML, you can't actually tell from 
> > the guest and the XML whether or not vhost was actually in use or not.
> 
> If we add a force option, let's please distinguish hotplug from VM
> creation time.  The latter can abort.  Hotplug should print an error and
> fail the initfn.

It can't abort at init - MSI is disabled at init, it needs to be enabled
by the guest later. And aborting the guest in the middle of the run
is a very bad idea.

What vhostforce=true will do is force vhost backend to be used even if
it is slower.

>  Thanks,
> 
> Alex
Alex Williamson - Jan. 21, 2011, 1:19 p.m.
On Fri, 2011-01-21 at 11:55 +0200, Michael S. Tsirkin wrote:
> On Thu, Jan 20, 2011 at 06:35:46PM -0700, Alex Williamson wrote:
> > On Thu, 2011-01-20 at 18:23 -0600, Anthony Liguori wrote:
> > > On 01/20/2011 10:07 AM, Michael S. Tsirkin wrote:
> > > > On Thu, Jan 20, 2011 at 09:43:57AM -0600, Anthony Liguori wrote:
> > > >    
> > > >> On 01/20/2011 09:35 AM, Michael S. Tsirkin wrote:
> > > >>      
> > > >>> When MSI is off, each interrupt needs to be bounced through the io
> > > >>> thread when it's set/cleared, so vhost-net causes more context switches and
> > > >>> higher CPU utilization than userspace virtio which handles networking in
> > > >>> the same thread.
> > > >>>
> > > >>> We'll need to fix this by adding level irq support in kvm irqfd,
> > > >>> for now disable vhost-net in these configurations.
> > > >>>
> > > >>> Signed-off-by: Michael S. Tsirkin<mst@redhat.com>
> > > >>>        
> > > >> I actually think this should be a terminal error.  The user asks for
> > > >> vhost-net, if we cannot enable it, we should exit.
> > > >>
> > > >> Or we should warn the user that they should expect bad performance.
> > > >> Silently doing something that the user has explicitly asked us not
> > > >> to do is not a good behavior.
> > > >>
> > > >> Regards,
> > > >>
> > > >> Anthony Liguori
> > > >>      
> > > > The issue is that user has no control of the guest, and can not know
> > > > whether the guest enables MSI. So what you ask for will just make
> > > > some guests fail, and others fail sometimes.
> > > > The user also has no way to know that version X of kvm does not expose a
> > > > way to inject level interrupts with irqfd.
> > > >
> > > > We could have *another* flag that says "use vhost where it helps" but
> > > > then I think this is what everyone wants to do, anyway, and libvirt
> > > > already sets vhost=on so I prefer redefining the meaning of an existing
> > > > flag.
> > > >    
> > > 
> > > In the very least, there needs to be a vhost=force.
> > > 
> > > Having some sort of friendly default policy is fine but we need to 
> > > provide a mechanism for a user to have the final say.  If you want to 
> > > redefine vhost=on to really mean, use the friendly default, that's fine 
> > > by me, but only if the vhost=force option exists.
> > > 
> > > I actually would think libvirt would want to use vhost=force.  Debugging 
> > > with vhost=on is going to be a royal pain in the ass if a user reports 
> > > bad performance.  Given the libvirt XML, you can't actually tell from 
> > > the guest and the XML whether or not vhost was actually in use or not.
> > 
> > If we add a force option, let's please distinguish hotplug from VM
> > creation time.  The latter can abort.  Hotplug should print an error and
> > fail the initfn.
> 
> It can't abort at init - MSI is disabled at init, it needs to be enabled
> by the guest later. And aborting the guest in the middle of the run
> is a very bad idea.

Yeah, I was thinking about the ordering of device being added vs guest
enabling MSI this morning.  Waiting until the guest decides to try to
start using the device to NAK it with an abort is very undesirable.
What if when we have vhost=on,force, the device doesn't advertise an
INTx (PCI_INTERRUPT_PIN = 0)?

Alex
Michael S. Tsirkin - Jan. 21, 2011, 1:43 p.m.
On Fri, Jan 21, 2011 at 06:19:13AM -0700, Alex Williamson wrote:
> On Fri, 2011-01-21 at 11:55 +0200, Michael S. Tsirkin wrote:
> > On Thu, Jan 20, 2011 at 06:35:46PM -0700, Alex Williamson wrote:
> > > On Thu, 2011-01-20 at 18:23 -0600, Anthony Liguori wrote:
> > > > On 01/20/2011 10:07 AM, Michael S. Tsirkin wrote:
> > > > > On Thu, Jan 20, 2011 at 09:43:57AM -0600, Anthony Liguori wrote:
> > > > >    
> > > > >> On 01/20/2011 09:35 AM, Michael S. Tsirkin wrote:
> > > > >>      
> > > > >>> When MSI is off, each interrupt needs to be bounced through the io
> > > > >>> thread when it's set/cleared, so vhost-net causes more context switches and
> > > > >>> higher CPU utilization than userspace virtio which handles networking in
> > > > >>> the same thread.
> > > > >>>
> > > > >>> We'll need to fix this by adding level irq support in kvm irqfd,
> > > > >>> for now disable vhost-net in these configurations.
> > > > >>>
> > > > >>> Signed-off-by: Michael S. Tsirkin<mst@redhat.com>
> > > > >>>        
> > > > >> I actually think this should be a terminal error.  The user asks for
> > > > >> vhost-net, if we cannot enable it, we should exit.
> > > > >>
> > > > >> Or we should warn the user that they should expect bad performance.
> > > > >> Silently doing something that the user has explicitly asked us not
> > > > >> to do is not a good behavior.
> > > > >>
> > > > >> Regards,
> > > > >>
> > > > >> Anthony Liguori
> > > > >>      
> > > > > The issue is that user has no control of the guest, and can not know
> > > > > whether the guest enables MSI. So what you ask for will just make
> > > > > some guests fail, and others fail sometimes.
> > > > > The user also has no way to know that version X of kvm does not expose a
> > > > > way to inject level interrupts with irqfd.
> > > > >
> > > > > We could have *another* flag that says "use vhost where it helps" but
> > > > > then I think this is what everyone wants to do, anyway, and libvirt
> > > > > already sets vhost=on so I prefer redefining the meaning of an existing
> > > > > flag.
> > > > >    
> > > > 
> > > > In the very least, there needs to be a vhost=force.
> > > > 
> > > > Having some sort of friendly default policy is fine but we need to 
> > > > provide a mechanism for a user to have the final say.  If you want to 
> > > > redefine vhost=on to really mean, use the friendly default, that's fine 
> > > > by me, but only if the vhost=force option exists.
> > > > 
> > > > I actually would think libvirt would want to use vhost=force.  Debugging 
> > > > with vhost=on is going to be a royal pain in the ass if a user reports 
> > > > bad performance.  Given the libvirt XML, you can't actually tell from 
> > > > the guest and the XML whether or not vhost was actually in use or not.
> > > 
> > > If we add a force option, let's please distinguish hotplug from VM
> > > creation time.  The latter can abort.  Hotplug should print an error and
> > > fail the initfn.
> > 
> > It can't abort at init - MSI is disabled at init, it needs to be enabled
> > by the guest later. And aborting the guest in the middle of the run
> > is a very bad idea.
> 
> Yeah, I was thinking about the ordering of device being added vs guest
> enabling MSI this morning.  Waiting until the guest decides to try to
> start using the device to NAK it with an abort is very undesirable.
> What if when we have vhost=on,force, the device doesn't advertise an
> INTx (PCI_INTERRUPT_PIN = 0)?
> 
> Alex

Then we break backward compatibility with old guests.
I don't see what the issue is really:
It is trivial to check that the guest uses MSIX.
Anthony Liguori - Jan. 21, 2011, 2:37 p.m.
On 01/21/2011 03:48 AM, Michael S. Tsirkin wrote:
> On Thu, Jan 20, 2011 at 06:23:36PM -0600, Anthony Liguori wrote:
>    
>> On 01/20/2011 10:07 AM, Michael S. Tsirkin wrote:
>>      
>>> On Thu, Jan 20, 2011 at 09:43:57AM -0600, Anthony Liguori wrote:
>>>        
>>>> On 01/20/2011 09:35 AM, Michael S. Tsirkin wrote:
>>>>          
>>>>> When MSI is off, each interrupt needs to be bounced through the io
>>>>> thread when it's set/cleared, so vhost-net causes more context switches and
>>>>> higher CPU utilization than userspace virtio which handles networking in
>>>>> the same thread.
>>>>>
>>>>> We'll need to fix this by adding level irq support in kvm irqfd,
>>>>> for now disable vhost-net in these configurations.
>>>>>
>>>>> Signed-off-by: Michael S. Tsirkin<mst@redhat.com>
>>>>>            
>>>> I actually think this should be a terminal error.  The user asks for
>>>> vhost-net, if we cannot enable it, we should exit.
>>>>
>>>> Or we should warn the user that they should expect bad performance.
>>>> Silently doing something that the user has explicitly asked us not
>>>> to do is not a good behavior.
>>>>
>>>> Regards,
>>>>
>>>> Anthony Liguori
>>>>          
>>> The issue is that user has no control of the guest, and can not know
>>> whether the guest enables MSI. So what you ask for will just make
>>> some guests fail, and others fail sometimes.
>>> The user also has no way to know that version X of kvm does not expose a
>>> way to inject level interrupts with irqfd.
>>>
>>> We could have *another* flag that says "use vhost where it helps" but
>>> then I think this is what everyone wants to do, anyway, and libvirt
>>> already sets vhost=on so I prefer redefining the meaning of an existing
>>> flag.
>>>        
>> In the very least, there needs to be a vhost=force.
>> Having some sort of friendly default policy is fine but we need to
>> provide a mechanism for a user to have the final say.  If you want
>> to redefine vhost=on to really mean, use the friendly default,
>> that's fine by me, but only if the vhost=force option exists.
>>      
> OK, I will add that, probably as a separate flag as vhost
> is a boolean.  This will get worse performance but it will be what the
> user asked for.
>
>    
>> I actually would think libvirt would want to use vhost=force.
>> Debugging with vhost=on is going to be a royal pain in the ass if a
>> user reports bad performance.  Given the libvirt XML, you can't
>> actually tell from the guest and the XML whether or not vhost was
>> actually in use or not.
>>      
> Yes you can: check MSI enabled in the guest, if it is -
> check vhost enabled in the XML. Not that bad at all, is it?
>    

Until you automatically detect level triggered interrupt support for 
irqfd.  This means it's also dependent on a kernel feature too.

Is there any way to tell in QEMU that vhost was silently disabled?

Regards,

Anthony Liguori

>    
>> Regards,
>>
>> Anthony Liguori
>>      
> We get worse performance without MSI anyway, how is this different?
>
>    
>>> Maybe this is best handled by a documentation update?
>>>
>>> We always said:
>>> 	    "                use vhost=on to enable experimental in kernel accelerator\n"
>>>
>>> note 'enable' not 'require'. This is similar to how we specify
>>> nvectors : you can not make guest use the feature.
>>>
>>> How about this:
>>>
>>> diff --git a/qemu-options.hx b/qemu-options.hx
>>> index 898561d..3c937c1 100644
>>> --- a/qemu-options.hx
>>> +++ b/qemu-options.hx
>>> @@ -1061,6 +1061,7 @@ DEF("net", HAS_ARG, QEMU_OPTION_net,
>>>       "                use vnet_hdr=off to avoid enabling the IFF_VNET_HDR tap flag\n"
>>>       "                use vnet_hdr=on to make the lack of IFF_VNET_HDR support an error condition\n"
>>>       "                use vhost=on to enable experimental in kernel accelerator\n"
>>> +    "                (note: vhost=on has no effect unless guest uses MSI-X)\n"
>>>       "                use 'vhostfd=h' to connect to an already opened vhost net device\n"
>>>   #endif
>>>       "-net socket[,vlan=n][,name=str][,fd=h][,listen=[host]:port][,connect=host:port]\n"
>>>
>>>
>>>        
>
Anthony Liguori - Jan. 21, 2011, 2:40 p.m.
On 01/21/2011 03:55 AM, Michael S. Tsirkin wrote:
> On Thu, Jan 20, 2011 at 06:35:46PM -0700, Alex Williamson wrote:
>    
>> On Thu, 2011-01-20 at 18:23 -0600, Anthony Liguori wrote:
>>      
>>> On 01/20/2011 10:07 AM, Michael S. Tsirkin wrote:
>>>        
>>>> On Thu, Jan 20, 2011 at 09:43:57AM -0600, Anthony Liguori wrote:
>>>>
>>>>          
>>>>> On 01/20/2011 09:35 AM, Michael S. Tsirkin wrote:
>>>>>
>>>>>            
>>>>>> When MSI is off, each interrupt needs to be bounced through the io
>>>>>> thread when it's set/cleared, so vhost-net causes more context switches and
>>>>>> higher CPU utilization than userspace virtio which handles networking in
>>>>>> the same thread.
>>>>>>
>>>>>> We'll need to fix this by adding level irq support in kvm irqfd,
>>>>>> for now disable vhost-net in these configurations.
>>>>>>
>>>>>> Signed-off-by: Michael S. Tsirkin<mst@redhat.com>
>>>>>>
>>>>>>              
>>>>> I actually think this should be a terminal error.  The user asks for
>>>>> vhost-net, if we cannot enable it, we should exit.
>>>>>
>>>>> Or we should warn the user that they should expect bad performance.
>>>>> Silently doing something that the user has explicitly asked us not
>>>>> to do is not a good behavior.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Anthony Liguori
>>>>>
>>>>>            
>>>> The issue is that user has no control of the guest, and can not know
>>>> whether the guest enables MSI. So what you ask for will just make
>>>> some guests fail, and others fail sometimes.
>>>> The user also has no way to know that version X of kvm does not expose a
>>>> way to inject level interrupts with irqfd.
>>>>
>>>> We could have *another* flag that says "use vhost where it helps" but
>>>> then I think this is what everyone wants to do, anyway, and libvirt
>>>> already sets vhost=on so I prefer redefining the meaning of an existing
>>>> flag.
>>>>
>>>>          
>>> In the very least, there needs to be a vhost=force.
>>>
>>> Having some sort of friendly default policy is fine but we need to
>>> provide a mechanism for a user to have the final say.  If you want to
>>> redefine vhost=on to really mean, use the friendly default, that's fine
>>> by me, but only if the vhost=force option exists.
>>>
>>> I actually would think libvirt would want to use vhost=force.  Debugging
>>> with vhost=on is going to be a royal pain in the ass if a user reports
>>> bad performance.  Given the libvirt XML, you can't actually tell from
>>> the guest and the XML whether or not vhost was actually in use or not.
>>>        
>> If we add a force option, let's please distinguish hotplug from VM
>> creation time.  The latter can abort.  Hotplug should print an error and
>> fail the initfn.
>>      
> It can't abort at init - MSI is disabled at init, it needs to be enabled
> by the guest later. And aborting the guest in the middle of the run
> is a very bad idea.
>
> What vhostforce=true will do is force vhost backend to be used even if
> it is slower.
>    

vhost=on,vhostforce=false              use vhost if we think it will 
improve performance
vhost=on,vhostforce=true               always use vhost
vhost=off,vhostforce=*                    do not use vhost

Regards,

Anthony Liguori

>    
>>   Thanks,
>>
>> Alex
>>      
>

Patch

diff --git a/qemu-options.hx b/qemu-options.hx
index 898561d..3c937c1 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -1061,6 +1061,7 @@  DEF("net", HAS_ARG, QEMU_OPTION_net,
     "                use vnet_hdr=off to avoid enabling the IFF_VNET_HDR tap flag\n"
     "                use vnet_hdr=on to make the lack of IFF_VNET_HDR support an error condition\n"
     "                use vhost=on to enable experimental in kernel accelerator\n"
+    "                (note: vhost=on has no effect unless guest uses MSI-X)\n"
     "                use 'vhostfd=h' to connect to an already opened vhost net device\n"
 #endif
     "-net socket[,vlan=n][,name=str][,fd=h][,listen=[host]:port][,connect=host:port]\n"