Patchwork [4/4] xen: Fix PV-on-HVM

login
register
mail settings
Submitter Anthony PERARD
Date May 15, 2012, 3:26 p.m.
Message ID <1337095599-28836-5-git-send-email-anthony.perard@citrix.com>
Download mbox | patch
Permalink /patch/159367/
State New
Headers show

Comments

Anthony PERARD - May 15, 2012, 3:26 p.m.
In the context of PV-on-HVM under Xen, the emulated nics are supposed to be
unplug before the guest drivers are initialized. This mean that there must be
unplug without the consent of the guest.

Without this patch, the guest end up with two nics with the same MAC, the
emulated nic and the PV nic.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 hw/xen_platform.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
Michael S. Tsirkin - May 15, 2012, 9 p.m.
On Tue, May 15, 2012 at 04:26:39PM +0100, Anthony PERARD wrote:
> In the context of PV-on-HVM under Xen, the emulated nics are supposed to be
> unplug before the guest drivers are initialized. This mean that there must be
> unplug without the consent of the guest.
> 
> Without this patch, the guest end up with two nics with the same MAC, the
> emulated nic and the PV nic.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

OK, so on Xen there are special devices that can be safely removed
without telling the guest?  Does there need to be regular hotplug for
these devices too? Or can it be always surprise removal?

> ---
>  hw/xen_platform.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
> 
> diff --git a/hw/xen_platform.c b/hw/xen_platform.c
> index a9c52a6..2e47129 100644
> --- a/hw/xen_platform.c
> +++ b/hw/xen_platform.c
> @@ -87,7 +87,7 @@ static void unplug_nic(PCIBus *b, PCIDevice *d)
>  {
>      if (pci_get_word(d->config + PCI_CLASS_DEVICE) ==
>              PCI_CLASS_NETWORK_ETHERNET) {
> -        qdev_unplug(&(d->qdev), NULL);
> +        qdev_force_unplug(&(d->qdev), NULL);
>      }
>  }
>  
> -- 
> Anthony PERARD
Paolo Bonzini - May 16, 2012, 8:06 a.m.
Il 15/05/2012 23:00, Michael S. Tsirkin ha scritto:
> On Tue, May 15, 2012 at 04:26:39PM +0100, Anthony PERARD wrote:
>> In the context of PV-on-HVM under Xen, the emulated nics are supposed to be
>> unplug before the guest drivers are initialized. This mean that there must be
>> unplug without the consent of the guest.
>>
>> Without this patch, the guest end up with two nics with the same MAC, the
>> emulated nic and the PV nic.
>>
>> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> OK, so on Xen there are special devices that can be safely removed
> without telling the guest?  Does there need to be regular hotplug for
> these devices too? Or can it be always surprise removal?

On Xen the PV drivers can ask the firmware to surprise-remove the
emulated NICs.  Of course it has to do it early enough so that the guest
doesn't crash.

Paolo
Michael S. Tsirkin - May 16, 2012, 8:13 a.m.
On Wed, May 16, 2012 at 10:06:25AM +0200, Paolo Bonzini wrote:
> Il 15/05/2012 23:00, Michael S. Tsirkin ha scritto:
> > On Tue, May 15, 2012 at 04:26:39PM +0100, Anthony PERARD wrote:
> >> In the context of PV-on-HVM under Xen, the emulated nics are supposed to be
> >> unplug before the guest drivers are initialized. This mean that there must be
> >> unplug without the consent of the guest.
> >>
> >> Without this patch, the guest end up with two nics with the same MAC, the
> >> emulated nic and the PV nic.
> >>
> >> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> > 
> > OK, so on Xen there are special devices that can be safely removed
> > without telling the guest?  Does there need to be regular hotplug for
> > these devices too? Or can it be always surprise removal?
> 
> On Xen the PV drivers can ask the firmware to surprise-remove the
> emulated NICs.

So driver tells firmware (meaning acpi? how?) that it's ok
to do surprize removal?

> Of course it has to do it early enough so that the guest
> doesn't crash.
> 
> Paolo

What does early enough mean and how do we ensure that?
Paolo Bonzini - May 16, 2012, 8:42 a.m.
Il 16/05/2012 10:13, Michael S. Tsirkin ha scritto:
> On Wed, May 16, 2012 at 10:06:25AM +0200, Paolo Bonzini wrote:
>> On Xen the PV drivers can ask the firmware to surprise-remove the
>> emulated NICs.
> 
> So driver tells firmware (meaning acpi? how?) that it's ok
> to do surprize removal?

It writes something to some I/O port, and then QEMU surprise-removes the
NICs.

>> Of course it has to do it early enough so that the guest
>> doesn't crash.
> 
> What does early enough mean and how do we ensure that?

Early enough means that the I/O port is written very early in the boot
process, even before the PCI bus is scanned by the OS.

You don't ensure it, it's up to the OS.  The OS knows whether its
drivers can cope properly with surprise removal.  If they can, in
principle it could write the magic value whenever it wants to.

Paolo
Stefano Stabellini - May 16, 2012, 10:19 a.m.
On Wed, 16 May 2012, Paolo Bonzini wrote:
> Il 16/05/2012 10:13, Michael S. Tsirkin ha scritto:
> > On Wed, May 16, 2012 at 10:06:25AM +0200, Paolo Bonzini wrote:
> >> On Xen the PV drivers can ask the firmware to surprise-remove the
> >> emulated NICs.
> > 
> > So driver tells firmware (meaning acpi? how?) that it's ok
> > to do surprize removal?
> 
> It writes something to some I/O port, and then QEMU surprise-removes the
> NICs.

Yes, writing to a static I/O port provided by the Xen platform PCI
device, see hw/xen_platform.c:platform_fixed_ioport_writew.

The guest can ask to unplug emulated NICs and disks this way.
Surprise-removal is OK in these cases.


> >> Of course it has to do it early enough so that the guest
> >> doesn't crash.
> > 
> > What does early enough mean and how do we ensure that?
> 
> Early enough means that the I/O port is written very early in the boot
> process, even before the PCI bus is scanned by the OS.
> 
> You don't ensure it, it's up to the OS.  The OS knows whether its
> drivers can cope properly with surprise removal.  If they can, in
> principle it could write the magic value whenever it wants to.

Right, it is up to the OS, in general before the PCI bus is scanned.
In Linux we do it from hypervisor_x86->init_platform.
Michael S. Tsirkin - May 16, 2012, 10:30 a.m.
On Wed, May 16, 2012 at 11:19:53AM +0100, Stefano Stabellini wrote:
> On Wed, 16 May 2012, Paolo Bonzini wrote:
> > Il 16/05/2012 10:13, Michael S. Tsirkin ha scritto:
> > > On Wed, May 16, 2012 at 10:06:25AM +0200, Paolo Bonzini wrote:
> > >> On Xen the PV drivers can ask the firmware to surprise-remove the
> > >> emulated NICs.
> > > 
> > > So driver tells firmware (meaning acpi? how?) that it's ok
> > > to do surprize removal?
> > 
> > It writes something to some I/O port, and then QEMU surprise-removes the
> > NICs.
> 
> Yes, writing to a static I/O port provided by the Xen platform PCI
> device, see hw/xen_platform.c:platform_fixed_ioport_writew.
> 
> The guest can ask to unplug emulated NICs and disks this way.
> Surprise-removal is OK in these cases.

Confused.
Don't you want to just remove the device on unplug?
In fact the equivalent of guest calling _EJ0?

> > >> Of course it has to do it early enough so that the guest
> > >> doesn't crash.
> > > 
> > > What does early enough mean and how do we ensure that?
> > 
> > Early enough means that the I/O port is written very early in the boot
> > process, even before the PCI bus is scanned by the OS.
> > 
> > You don't ensure it, it's up to the OS.  The OS knows whether its
> > drivers can cope properly with surprise removal.  If they can, in
> > principle it could write the magic value whenever it wants to.
> 
> Right, it is up to the OS, in general before the PCI bus is scanned.
> In Linux we do it from hypervisor_x86->init_platform.

So early on boot you decide you want PV and so you unplug all emulated
devices?
Stefano Stabellini - May 16, 2012, 10:37 a.m.
On Wed, 16 May 2012, Michael S. Tsirkin wrote:
> On Wed, May 16, 2012 at 11:19:53AM +0100, Stefano Stabellini wrote:
> > On Wed, 16 May 2012, Paolo Bonzini wrote:
> > > Il 16/05/2012 10:13, Michael S. Tsirkin ha scritto:
> > > > On Wed, May 16, 2012 at 10:06:25AM +0200, Paolo Bonzini wrote:
> > > >> On Xen the PV drivers can ask the firmware to surprise-remove the
> > > >> emulated NICs.
> > > > 
> > > > So driver tells firmware (meaning acpi? how?) that it's ok
> > > > to do surprize removal?
> > > 
> > > It writes something to some I/O port, and then QEMU surprise-removes the
> > > NICs.
> > 
> > Yes, writing to a static I/O port provided by the Xen platform PCI
> > device, see hw/xen_platform.c:platform_fixed_ioport_writew.
> > 
> > The guest can ask to unplug emulated NICs and disks this way.
> > Surprise-removal is OK in these cases.
> 
> Confused.
> Don't you want to just remove the device on unplug?

Yes, the NIC needs to "disappear" from the PCI bus.


> In fact the equivalent of guest calling _EJ0?

Except that _EJ0 can or cannot be implemented, while this doesn't have
to go through ACPI or PCI hotplug and it is supposed to always work.


> > > >> Of course it has to do it early enough so that the guest
> > > >> doesn't crash.
> > > > 
> > > > What does early enough mean and how do we ensure that?
> > > 
> > > Early enough means that the I/O port is written very early in the boot
> > > process, even before the PCI bus is scanned by the OS.
> > > 
> > > You don't ensure it, it's up to the OS.  The OS knows whether its
> > > drivers can cope properly with surprise removal.  If they can, in
> > > principle it could write the magic value whenever it wants to.
> > 
> > Right, it is up to the OS, in general before the PCI bus is scanned.
> > In Linux we do it from hypervisor_x86->init_platform.
> 
> So early on boot you decide you want PV and so you unplug all emulated
> devices?

Yes, but only the emulated devices that can collide with PV devices:
disk and network.
Michael S. Tsirkin - May 16, 2012, 1:24 p.m.
On Wed, May 16, 2012 at 11:37:02AM +0100, Stefano Stabellini wrote:
> On Wed, 16 May 2012, Michael S. Tsirkin wrote:
> > On Wed, May 16, 2012 at 11:19:53AM +0100, Stefano Stabellini wrote:
> > > On Wed, 16 May 2012, Paolo Bonzini wrote:
> > > > Il 16/05/2012 10:13, Michael S. Tsirkin ha scritto:
> > > > > On Wed, May 16, 2012 at 10:06:25AM +0200, Paolo Bonzini wrote:
> > > > >> On Xen the PV drivers can ask the firmware to surprise-remove the
> > > > >> emulated NICs.
> > > > > 
> > > > > So driver tells firmware (meaning acpi? how?) that it's ok
> > > > > to do surprize removal?
> > > > 
> > > > It writes something to some I/O port, and then QEMU surprise-removes the
> > > > NICs.
> > > 
> > > Yes, writing to a static I/O port provided by the Xen platform PCI
> > > device, see hw/xen_platform.c:platform_fixed_ioport_writew.
> > > 
> > > The guest can ask to unplug emulated NICs and disks this way.
> > > Surprise-removal is OK in these cases.
> > 
> > Confused.
> > Don't you want to just remove the device on unplug?
> 
> Yes, the NIC needs to "disappear" from the PCI bus.
> 
> 
> > In fact the equivalent of guest calling _EJ0?
> 
> Except that _EJ0 can or cannot be implemented, while this doesn't have
> to go through ACPI or PCI hotplug and it is supposed to always work.

So the answer is to simply free on unplug exactly
like _EJ0 does. There's no forcing and no surprise removal here.

Patch

diff --git a/hw/xen_platform.c b/hw/xen_platform.c
index a9c52a6..2e47129 100644
--- a/hw/xen_platform.c
+++ b/hw/xen_platform.c
@@ -87,7 +87,7 @@  static void unplug_nic(PCIBus *b, PCIDevice *d)
 {
     if (pci_get_word(d->config + PCI_CLASS_DEVICE) ==
             PCI_CLASS_NETWORK_ETHERNET) {
-        qdev_unplug(&(d->qdev), NULL);
+        qdev_force_unplug(&(d->qdev), NULL);
     }
 }