diff mbox

Hard and silent lock up since linux 3.14 with PCIe pass through (vfio)

Message ID 20141010225408.GA24493@google.com
State Not Applicable
Headers show

Commit Message

Bjorn Helgaas Oct. 10, 2014, 10:54 p.m. UTC 
On Sat, Oct 11, 2014 at 12:32:19AM +0200, Andreas Hartmann wrote:
> Bjorn Helgaas wrote:
> > On Fri, Oct 10, 2014 at 10:09 AM, Andreas Hartmann
> > <andihartmann@freenet.de> wrote:
> >> Bjorn Helgaas wrote:
> >>> On Fri, Oct 10, 2014 at 8:49 AM, Andreas Hartmann
> >>> <andihartmann@freenet.de> wrote:
> >>>> Bjorn Helgaas wrote:
> >>>>> On Fri, Oct 10, 2014 at 3:39 AM, Andreas Hartmann
> >>>>> <andihartmann@freenet.de> wrote:
> >>>>>> shortly: I retested w/ qemu 2.1.0 and Linux 3.17.0 - no change in behaviour.
> >>>>>>
> >>>>>> Alex Williamson wrote:
> >>>>>>> On Tue, 2014-09-23 at 21:03 +0200, Andreas Hartmann wrote:
> >>>>>>>> Hello!
> >>>>>>>>
> >>>>>>>> Since long time now, I'm using w/o any problem PCIe pass through with a
> >>>>>>>> Gigabyte GA-990XA-UD3/GA-990XA-UD3 mainboard (AMD 990X chipset) and
> >>>>>>>> enabled IOMMU with vfio-pci.
> >>>>>>>>
> >>>>>>>> The last kernel working w/o any problem is kernel 3.13.7 (I didn't use
> >>>>>>>> .8 and .9, but I do not think they would have been problematic).
> >>>>>>>>
> >>>>>>>> Since 3.14.19 (I didn't test any 3.14 kernel before) I'm encountering a
> >>>>>>>> hard and silent lock up of the complete machine when starting the VM
> >>>>>>>> with the PCIe card passed through.
> >>>>>
> >>>>> Since we're not really making any progress on this yet, would it be
> >>>>> possible to bisect it?  We already know that 3.13.7 works and 3.14.19
> >>>>> fails, and "git bisect start v3.14 v3.13" says it's about 13 steps.  I
> >>>>> know that's still quite a bit of work, but at least it sounds like the
> >>>>> problem is easy to reproduce.
> >>>>
> >>>> Which git repository should I use best?
> >>>
> >>> The linux-stable repository [1] contains both the v3.13.x and the
> >>> v3.14.x branches, but apparently you can't bisect directly between
> >>> v3.13.7 and v3.14.19:
> >>
> >> I know that the first version after 3.13.0 (patch-v3.13-next-20140121)
> >> is already broken. Therefore, it must be between 3.13.7 and
> >> patch-v3.13-next-20140121.
> 
> 
> Ok, this is the result of git bisect:
> 
> 425c1b223dac456d00a61fd6b451b6d1cf00d065 is the first bad commit
> commit 425c1b223dac456d00a61fd6b451b6d1cf00d065
> Author: Alex Williamson <alex.williamson@redhat.com>
> Date:   Tue Dec 17 16:43:51 2013 -0700
> 
>     PCI: Add Virtual Channel to save/restore support
>     
>     While we don't really have any infrastructure for making use of VC
>     support, the system BIOS can configure the topology to non-default
>     VC values prior to boot.  This may be due to silicon bugs, desire to
>     reserve traffic classes, or perhaps just BIOS bugs.  When we reset
>     devices, the VC configuration may return to default values, which can
>     be incompatible with devices upstream.  For instance, Nvidia GRID 
>     cards provide a PCIe switch and some number of GPUs, all supporting 
>     VC.  The power-on default for VC is to support TC0-7 across VC0,
>     however some platforms will only enable TC0/VC0 mapping across the 
>     topology.  When we do a secondary bus reset on the downstream switch 
>     port, the GPU is reset to a TC0-7/VC0 mapping while the opposite end 
>     of the link only enables TC0/VC0.  If the GPU attempts to use TC1-7, 
>     it fails. 
> 
>     This patch attempts to provide complete support for VC save/restore, 
>     even beyond the minimally required use case above.  This includes 
>     save/restore and reload of the arbitration table, save/restore and 
>     reload of the port arbitration tables, and re-enabling of the 
>     channels for VC, VC9, and MFVC capabilities. 
> 
>     Signed-off-by: Alex Williamson <alex.williamson@redhat.com> 
>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Wow, I'm amazed that you could get that done so fast... you must have spent
your whole day working on this!

To double-check this, can you try applying the patch below?  It should be
enough to make things work if 425c1b223dac is really what's causing the
trouble.

This patch is based on v3.17, but 425c1b223dac appeared in v3.14, so you
should be able to apply it to v3.14 or any later kernel.

Bjorn


--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Andreas Hartmann Oct. 11, 2014, 6:20 a.m. UTC | #1 
Bjorn Helgaas wrote:
> On Sat, Oct 11, 2014 at 12:32:19AM +0200, Andreas Hartmann wrote:
>> Bjorn Helgaas wrote:
>>> On Fri, Oct 10, 2014 at 10:09 AM, Andreas Hartmann
>>> <andihartmann@freenet.de> wrote:
>>>> Bjorn Helgaas wrote:
>>>>> On Fri, Oct 10, 2014 at 8:49 AM, Andreas Hartmann
>>>>> <andihartmann@freenet.de> wrote:
>>>>>> Bjorn Helgaas wrote:
>>>>>>> On Fri, Oct 10, 2014 at 3:39 AM, Andreas Hartmann
>>>>>>> <andihartmann@freenet.de> wrote:
>>>>>>>> shortly: I retested w/ qemu 2.1.0 and Linux 3.17.0 - no change in behaviour.
>>>>>>>>
>>>>>>>> Alex Williamson wrote:
>>>>>>>>> On Tue, 2014-09-23 at 21:03 +0200, Andreas Hartmann wrote:
>>>>>>>>>> Hello!
>>>>>>>>>>
>>>>>>>>>> Since long time now, I'm using w/o any problem PCIe pass through with a
>>>>>>>>>> Gigabyte GA-990XA-UD3/GA-990XA-UD3 mainboard (AMD 990X chipset) and
>>>>>>>>>> enabled IOMMU with vfio-pci.
>>>>>>>>>>
>>>>>>>>>> The last kernel working w/o any problem is kernel 3.13.7 (I didn't use
>>>>>>>>>> .8 and .9, but I do not think they would have been problematic).
>>>>>>>>>>
>>>>>>>>>> Since 3.14.19 (I didn't test any 3.14 kernel before) I'm encountering a
>>>>>>>>>> hard and silent lock up of the complete machine when starting the VM
>>>>>>>>>> with the PCIe card passed through.
>>>>>>>
>>>>>>> Since we're not really making any progress on this yet, would it be
>>>>>>> possible to bisect it?  We already know that 3.13.7 works and 3.14.19
>>>>>>> fails, and "git bisect start v3.14 v3.13" says it's about 13 steps.  I
>>>>>>> know that's still quite a bit of work, but at least it sounds like the
>>>>>>> problem is easy to reproduce.
>>>>>>
>>>>>> Which git repository should I use best?
>>>>>
>>>>> The linux-stable repository [1] contains both the v3.13.x and the
>>>>> v3.14.x branches, but apparently you can't bisect directly between
>>>>> v3.13.7 and v3.14.19:
>>>>
>>>> I know that the first version after 3.13.0 (patch-v3.13-next-20140121)
>>>> is already broken. Therefore, it must be between 3.13.7 and
>>>> patch-v3.13-next-20140121.
>>
>>
>> Ok, this is the result of git bisect:
>>
>> 425c1b223dac456d00a61fd6b451b6d1cf00d065 is the first bad commit
>> commit 425c1b223dac456d00a61fd6b451b6d1cf00d065
>> Author: Alex Williamson <alex.williamson@redhat.com>
>> Date:   Tue Dec 17 16:43:51 2013 -0700
>>
>>     PCI: Add Virtual Channel to save/restore support
>>     
>>     While we don't really have any infrastructure for making use of VC
>>     support, the system BIOS can configure the topology to non-default
>>     VC values prior to boot.  This may be due to silicon bugs, desire to
>>     reserve traffic classes, or perhaps just BIOS bugs.  When we reset
>>     devices, the VC configuration may return to default values, which can
>>     be incompatible with devices upstream.  For instance, Nvidia GRID 
>>     cards provide a PCIe switch and some number of GPUs, all supporting 
>>     VC.  The power-on default for VC is to support TC0-7 across VC0,
>>     however some platforms will only enable TC0/VC0 mapping across the 
>>     topology.  When we do a secondary bus reset on the downstream switch 
>>     port, the GPU is reset to a TC0-7/VC0 mapping while the opposite end 
>>     of the link only enables TC0/VC0.  If the GPU attempts to use TC1-7, 
>>     it fails. 
>>
>>     This patch attempts to provide complete support for VC save/restore, 
>>     even beyond the minimally required use case above.  This includes 
>>     save/restore and reload of the arbitration table, save/restore and 
>>     reload of the port arbitration tables, and re-enabling of the 
>>     channels for VC, VC9, and MFVC capabilities. 
>>
>>     Signed-off-by: Alex Williamson <alex.williamson@redhat.com> 
>>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> 
> Wow, I'm amazed that you could get that done so fast... you must have spent
> your whole day working on this!

If I would have been more familiar with the versioning of the kernels
and if I would have a faster internet connection and if there wouldn't
be another bug in systemd, which has bitten me on booting with broken fs
(but I found a cool workaround now :-)), I would have been much faster:
my 8 core machine and 8 GB of RAM, where I've been compiling the kernel
in and my special kernel config (which Im using since 3.10) only
containing my requests, with parts of the process automated makes it
possible to have a turn around of ~ 7 minutes :-).
I too had no problem with reproducibility, because the problem always
comes up at the start of the vm after 1 or 2 secs.

> 
> To double-check this, can you try applying the patch below?  It should be
> enough to make things work if 425c1b223dac is really what's causing the
> trouble.
> 
> This patch is based on v3.17, but 425c1b223dac appeared in v3.14, so you
> should be able to apply it to v3.14 or any later kernel.
> 
> Bjorn
> 
> 
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 2c9ac70254e2..8ef8bc56a584 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -1007,8 +1007,6 @@ int pci_save_state(struct pci_dev *dev)
>  		return i;
>  	if ((i = pci_save_pcix_state(dev)) != 0)
>  		return i;
> -	if ((i = pci_save_vc_state(dev)) != 0)
> -		return i;
>  	return 0;
>  }
>  EXPORT_SYMBOL(pci_save_state);
> @@ -1072,7 +1070,6 @@ void pci_restore_state(struct pci_dev *dev)
>  	/* PCI Express register must be restored first */
>  	pci_restore_pcie_state(dev);
>  	pci_restore_ats_state(dev);
> -	pci_restore_vc_state(dev);
>  
>  	pci_restore_config_space(dev);
>  
> @@ -2170,8 +2167,6 @@ void pci_allocate_cap_save_buffers(struct pci_dev *dev)
>  	if (error)
>  		dev_err(&dev->dev,
>  			"unable to preallocate PCI-X save buffer\n");
> -
> -	pci_allocate_vc_save_buffers(dev);
>  }
>  
>  void pci_free_cap_save_buffers(struct pci_dev *dev)
> 

This patch proofed the git bisect result. I applied it to
patch-v3.13-next-20140122 and the machine worked pretty fine :-).


Thanks,
Regards,
Andreas
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Alex Williamson Oct. 15, 2014, 8:04 a.m. UTC | #2 
On Sat, 2014-10-11 at 08:20 +0200, Andreas Hartmann wrote:
> Bjorn Helgaas wrote:
> > On Sat, Oct 11, 2014 at 12:32:19AM +0200, Andreas Hartmann wrote:
> >> Bjorn Helgaas wrote:
> >>> On Fri, Oct 10, 2014 at 10:09 AM, Andreas Hartmann
> >>> <andihartmann@freenet.de> wrote:
> >>>> Bjorn Helgaas wrote:
> >>>>> On Fri, Oct 10, 2014 at 8:49 AM, Andreas Hartmann
> >>>>> <andihartmann@freenet.de> wrote:
> >>>>>> Bjorn Helgaas wrote:
> >>>>>>> On Fri, Oct 10, 2014 at 3:39 AM, Andreas Hartmann
> >>>>>>> <andihartmann@freenet.de> wrote:
> >>>>>>>> shortly: I retested w/ qemu 2.1.0 and Linux 3.17.0 - no change in behaviour.
> >>>>>>>>
> >>>>>>>> Alex Williamson wrote:
> >>>>>>>>> On Tue, 2014-09-23 at 21:03 +0200, Andreas Hartmann wrote:
> >>>>>>>>>> Hello!
> >>>>>>>>>>
> >>>>>>>>>> Since long time now, I'm using w/o any problem PCIe pass through with a
> >>>>>>>>>> Gigabyte GA-990XA-UD3/GA-990XA-UD3 mainboard (AMD 990X chipset) and
> >>>>>>>>>> enabled IOMMU with vfio-pci.
> >>>>>>>>>>
> >>>>>>>>>> The last kernel working w/o any problem is kernel 3.13.7 (I didn't use
> >>>>>>>>>> .8 and .9, but I do not think they would have been problematic).
> >>>>>>>>>>
> >>>>>>>>>> Since 3.14.19 (I didn't test any 3.14 kernel before) I'm encountering a
> >>>>>>>>>> hard and silent lock up of the complete machine when starting the VM
> >>>>>>>>>> with the PCIe card passed through.
> >>>>>>>
> >>>>>>> Since we're not really making any progress on this yet, would it be
> >>>>>>> possible to bisect it?  We already know that 3.13.7 works and 3.14.19
> >>>>>>> fails, and "git bisect start v3.14 v3.13" says it's about 13 steps.  I
> >>>>>>> know that's still quite a bit of work, but at least it sounds like the
> >>>>>>> problem is easy to reproduce.
> >>>>>>
> >>>>>> Which git repository should I use best?
> >>>>>
> >>>>> The linux-stable repository [1] contains both the v3.13.x and the
> >>>>> v3.14.x branches, but apparently you can't bisect directly between
> >>>>> v3.13.7 and v3.14.19:
> >>>>
> >>>> I know that the first version after 3.13.0 (patch-v3.13-next-20140121)
> >>>> is already broken. Therefore, it must be between 3.13.7 and
> >>>> patch-v3.13-next-20140121.
> >>
> >>
> >> Ok, this is the result of git bisect:
> >>
> >> 425c1b223dac456d00a61fd6b451b6d1cf00d065 is the first bad commit
> >> commit 425c1b223dac456d00a61fd6b451b6d1cf00d065
> >> Author: Alex Williamson <alex.williamson@redhat.com>
> >> Date:   Tue Dec 17 16:43:51 2013 -0700
> >>
> >>     PCI: Add Virtual Channel to save/restore support
> >>     
> >>     While we don't really have any infrastructure for making use of VC
> >>     support, the system BIOS can configure the topology to non-default
> >>     VC values prior to boot.  This may be due to silicon bugs, desire to
> >>     reserve traffic classes, or perhaps just BIOS bugs.  When we reset
> >>     devices, the VC configuration may return to default values, which can
> >>     be incompatible with devices upstream.  For instance, Nvidia GRID 
> >>     cards provide a PCIe switch and some number of GPUs, all supporting 
> >>     VC.  The power-on default for VC is to support TC0-7 across VC0,
> >>     however some platforms will only enable TC0/VC0 mapping across the 
> >>     topology.  When we do a secondary bus reset on the downstream switch 
> >>     port, the GPU is reset to a TC0-7/VC0 mapping while the opposite end 
> >>     of the link only enables TC0/VC0.  If the GPU attempts to use TC1-7, 
> >>     it fails. 
> >>
> >>     This patch attempts to provide complete support for VC save/restore, 
> >>     even beyond the minimally required use case above.  This includes 
> >>     save/restore and reload of the arbitration table, save/restore and 
> >>     reload of the port arbitration tables, and re-enabling of the 
> >>     channels for VC, VC9, and MFVC capabilities. 
> >>
> >>     Signed-off-by: Alex Williamson <alex.williamson@redhat.com> 
> >>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> > 
> > Wow, I'm amazed that you could get that done so fast... you must have spent
> > your whole day working on this!
> 
> If I would have been more familiar with the versioning of the kernels
> and if I would have a faster internet connection and if there wouldn't
> be another bug in systemd, which has bitten me on booting with broken fs
> (but I found a cool workaround now :-)), I would have been much faster:
> my 8 core machine and 8 GB of RAM, where I've been compiling the kernel
> in and my special kernel config (which Im using since 3.10) only
> containing my requests, with parts of the process automated makes it
> possible to have a turn around of ~ 7 minutes :-).
> I too had no problem with reproducibility, because the problem always
> comes up at the start of the vm after 1 or 2 secs.

Hi Andreas,

Sorry for the breakage.  Is it possible to run lspci on the device in a
loop from the host and capture whether we're failing to restore some of
the VC bits to their previous state?  Does the problem also occur if you
unbind from host driver, echo 1 > reset in pci-sysfs, and re-bind to the
host?  I'll also try to reproduce on my 990fx system, but I won't be
able to do that until next week due to travel.  Thanks,

Alex

> > To double-check this, can you try applying the patch below?  It should be
> > enough to make things work if 425c1b223dac is really what's causing the
> > trouble.
> > 
> > This patch is based on v3.17, but 425c1b223dac appeared in v3.14, so you
> > should be able to apply it to v3.14 or any later kernel.
> > 
> > Bjorn
> > 
> > 
> > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> > index 2c9ac70254e2..8ef8bc56a584 100644
> > --- a/drivers/pci/pci.c
> > +++ b/drivers/pci/pci.c
> > @@ -1007,8 +1007,6 @@ int pci_save_state(struct pci_dev *dev)
> >  		return i;
> >  	if ((i = pci_save_pcix_state(dev)) != 0)
> >  		return i;
> > -	if ((i = pci_save_vc_state(dev)) != 0)
> > -		return i;
> >  	return 0;
> >  }
> >  EXPORT_SYMBOL(pci_save_state);
> > @@ -1072,7 +1070,6 @@ void pci_restore_state(struct pci_dev *dev)
> >  	/* PCI Express register must be restored first */
> >  	pci_restore_pcie_state(dev);
> >  	pci_restore_ats_state(dev);
> > -	pci_restore_vc_state(dev);
> >  
> >  	pci_restore_config_space(dev);
> >  
> > @@ -2170,8 +2167,6 @@ void pci_allocate_cap_save_buffers(struct pci_dev *dev)
> >  	if (error)
> >  		dev_err(&dev->dev,
> >  			"unable to preallocate PCI-X save buffer\n");
> > -
> > -	pci_allocate_vc_save_buffers(dev);
> >  }
> >  
> >  void pci_free_cap_save_buffers(struct pci_dev *dev)
> > 
> 
> This patch proofed the git bisect result. I applied it to
> patch-v3.13-next-20140122 and the machine worked pretty fine :-).
> 
> 
> Thanks,
> Regards,
> Andreas



--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Andreas Hartmann Oct. 17, 2014, 1:04 a.m. UTC | #3 
Hello Alex,

Alex Williamson wrote:
> Hi Andreas,
[...]
> Sorry for the breakage.  Is it possible to run lspci on the device in a
> loop from the host and capture whether we're failing to restore some of
> the VC bits to their previous state? 

> Does the problem also occur if you
> unbind from host driver,

The machine is booted w/ blacklisted ath9k. Then, the device is bound to
vfio:

echo "168c 0030" > /sys/bus/pci/drivers/vfio-pci/new_id
echo 0000:03:00.0 > /sys/bus/pci/devices/0000:03:00.0/driver/unbind
echo 0000:03:00.0 > /sys/bus/pci/drivers/vfio-pci/bind

afterwards the VM is started -> hang.

W/o starting th VM, I can bind it to vfio and unbind it from vfio w/o
any problem.

> echo 1 > reset in pci-sysfs,

echo 1 > /sys/bus/pci/devices/0000:03:00.0 works w/o any problem while
bound to vfio. Even after unbinding from vfio and rebinding to vfio
again ... .

> and re-bind to the

Do you mean loading ath9k in host system after unbinding from vfio? If
yes: Works w/o any problem. It's even possible to reset it or do a
ifconfig wlan0 up, ifconfig wlan0 down, rmmod ath9k, bind it to vfio
again and reset it, ....

Looks like the hang only is triggered by qemu-system_x86_64 on startup
the VM.

> host?  I'll also try to reproduce on my 990fx system, but I won't be
> able to do that until next week due to travel.  Thanks,


Regards,
Andreas
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Alex Williamson Oct. 21, 2014, 9:06 p.m. UTC | #4 
Hi Andreas,

On Fri, 2014-10-17 at 03:04 +0200, Andreas Hartmann wrote:
> Hello Alex,
> 
> Alex Williamson wrote:
> > Hi Andreas,
> [...]
> > Sorry for the breakage.  Is it possible to run lspci on the device in a
> > loop from the host and capture whether we're failing to restore some of
> > the VC bits to their previous state? 
> 
> > Does the problem also occur if you
> > unbind from host driver,
> 
> The machine is booted w/ blacklisted ath9k. Then, the device is bound to
> vfio:
> 
> echo "168c 0030" > /sys/bus/pci/drivers/vfio-pci/new_id
> echo 0000:03:00.0 > /sys/bus/pci/devices/0000:03:00.0/driver/unbind
> echo 0000:03:00.0 > /sys/bus/pci/drivers/vfio-pci/bind
> 
> afterwards the VM is started -> hang.
> 
> W/o starting th VM, I can bind it to vfio and unbind it from vfio w/o
> any problem.
> 
> > echo 1 > reset in pci-sysfs,
> 
> echo 1 > /sys/bus/pci/devices/0000:03:00.0 works w/o any problem while
> bound to vfio. Even after unbinding from vfio and rebinding to vfio
> again ... .
> 
> > and re-bind to the
> 
> Do you mean loading ath9k in host system after unbinding from vfio? If
> yes: Works w/o any problem. It's even possible to reset it or do a
> ifconfig wlan0 up, ifconfig wlan0 down, rmmod ath9k, bind it to vfio
> again and reset it, ....
> 
> Looks like the hang only is triggered by qemu-system_x86_64 on startup
> the VM.
> 
> > host?  I'll also try to reproduce on my 990fx system, but I won't be
> > able to do that until next week due to travel.  Thanks,

Could you send me the lspci -vvvxxxx for the device and parent root
port?  Thanks,

Alex

--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Andreas Hartmann Oct. 22, 2014, 3:34 p.m. UTC | #5 
Alex Williamson schrieb:
> Hi Andreas,
> 
> On Fri, 2014-10-17 at 03:04 +0200, Andreas Hartmann wrote:
>> Hello Alex,
>>
>> Alex Williamson wrote:
>>> Hi Andreas,
>> [...]
>>> Sorry for the breakage.  Is it possible to run lspci on the device in a
>>> loop from the host and capture whether we're failing to restore some of
>>> the VC bits to their previous state? 
>>
>>> Does the problem also occur if you
>>> unbind from host driver,
>>
>> The machine is booted w/ blacklisted ath9k. Then, the device is bound to
>> vfio:
>>
>> echo "168c 0030" > /sys/bus/pci/drivers/vfio-pci/new_id
>> echo 0000:03:00.0 > /sys/bus/pci/devices/0000:03:00.0/driver/unbind
>> echo 0000:03:00.0 > /sys/bus/pci/drivers/vfio-pci/bind
>>
>> afterwards the VM is started -> hang.
>>
>> W/o starting th VM, I can bind it to vfio and unbind it from vfio w/o
>> any problem.
>>
>>> echo 1 > reset in pci-sysfs,
>>
>> echo 1 > /sys/bus/pci/devices/0000:03:00.0 works w/o any problem while
>> bound to vfio. Even after unbinding from vfio and rebinding to vfio
>> again ... .
>>
>>> and re-bind to the
>>
>> Do you mean loading ath9k in host system after unbinding from vfio? If
>> yes: Works w/o any problem. It's even possible to reset it or do a
>> ifconfig wlan0 up, ifconfig wlan0 down, rmmod ath9k, bind it to vfio
>> again and reset it, ....
>>
>> Looks like the hang only is triggered by qemu-system_x86_64 on startup
>> the VM.
>>
>>> host?  I'll also try to reproduce on my 990fx system, but I won't be
>>> able to do that until next week due to travel.  Thanks,
> 
> Could you send me the lspci -vvvxxxx for the device and parent root
> port?  Thanks,


Done with kernel 3.12.28 in host while the device was used in VM:

# lspci -vt
-[0000:00]-+-00.0  Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx0 port B)
           +-00.2  Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
           +-02.0-[01]--+-00.0  Advanced Micro Devices, Inc. [AMD/ATI] Turks PRO [Radeon HD 6570/7570]
           |            \-00.1  Advanced Micro Devices, Inc. [AMD/ATI] Turks/Whistler HDMI Audio [Radeon HD 6000 Series]
           +-04.0-[02]----00.0  Etron Technology, Inc. EJ168 USB 3.0 Host Controller
           +-05.0-[03]----00.0  Qualcomm Atheros AR93xx Wireless Network Adapter
           +-09.0-[04]----00.0  Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
           +-0a.0-[05]----00.0  Etron Technology, Inc. EJ168 USB 3.0 Host Controller
           +-11.0  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode]
           +-12.0  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
           +-12.2  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
           +-13.0  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
           +-13.2  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
           +-14.0  Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller
           +-14.2  Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA)
           +-14.3  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller
           +-14.4-[06]--+-06.0  Intel Corporation 82557/8/9/0/1 Ethernet Pro 100
           |            \-0e.0  VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller
           +-14.5  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
           +-15.0-[07]--
           +-16.0  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
           +-16.2  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
           +-18.0  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
           +-18.1  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
           +-18.2  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
           +-18.3  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
           +-18.4  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
           \-18.5  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5


# lspci -s 03:00 -vvvxxxx
03:00.0 Network controller: Qualcomm Atheros AR93xx Wireless Network Adapter (rev 01)
        Subsystem: Qualcomm Atheros Device 3112
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 17
        Region 0: Memory at fdbc0000 (64-bit, non-prefetchable) [size=128K]
        [virtual] Expansion ROM at fda00000 [disabled] [size=64K]
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/4 Maskable+ 64bit+
                Address: 0000000000000000  Data: 0000
                Masking: 00000000  Pending: 00000000
        Capabilities: [70] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <1us, L1 <8us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <2us, L1 <64us
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout+ NonFatalErr+
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
        Capabilities: [140 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed- WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
                        Status: NegoPending- InProgress-
        Capabilities: [300 v1] Device Serial Number 00-00-00-00-00-00-00-00
        Kernel driver in use: vfio-pci
        Kernel modules: ath9k
00: 8c 16 30 00 07 01 10 00 01 00 80 02 10 00 00 00
10: 04 00 bc fd 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 8c 16 12 31
30: 00 00 00 00 40 00 00 00 00 00 00 00 05 01 00 00
40: 01 50 c3 5b 00 00 00 00 00 00 00 00 00 00 00 00
50: 05 70 84 01 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 10 00 02 00 00 87 2c 01 10 20 09 00 11 5c 03 00
80: 40 00 11 10 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 10 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
100: 01 00 01 14 00 00 00 00 00 00 00 00 30 20 06 00
110: 00 30 00 00 00 20 00 00 00 00 00 00 00 00 00 00
120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
140: 02 00 01 30 00 00 00 00 00 00 00 00 00 00 00 00
150: 00 00 00 00 ff 00 00 80 00 00 00 00 00 00 00 00
160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
300: 03 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
700: 76 00 63 01 ff ff ff ff 04 00 00 07 01 3f 3f 17
710: 20 01 01 00 00 00 00 00 aa 83 00 00 80 02 00 00
720: 00 00 00 00 03 00 00 00 51 5c ae 03 10 01 00 08
730: 40 00 01 00 ff 0f 01 00 ff ff 0f 00 00 00 00 00
740: 0f 00 00 00 00 00 00 00 08 40 20 00 03 40 20 00
750: 00 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
7a0: 00 00 00 00 00 00 00 00 2c 00 08 00 00 00 00 00
7b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
7c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
7d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
7e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
7f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
800: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
810: ff ff ff ff 00 00 00 00 00 03 00 00 00 00 00 00
820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00


I'm not sure what you mean with "parent root port". Could it be this:

# lspci -s 00:00 -vvvxxxx
00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx0 port B) (rev 02)
        Subsystem: Gigabyte Technology Co., Ltd Device 5000
        Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
        Region 3: Memory at <ignored> (64-bit, non-prefetchable) [size=512M]
        Capabilities: [f0] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [c4] HyperTransport: Slave or Primary Interface
                Command: BaseUnitID=0 UnitCnt=20 MastHost- DefDir- DUL-
                Link Control 0: CFlE- CST- CFE- <LkFail- Init+ EOC- TXO- <CRCErr=0 IsocEn- LSEn- ExtCTL- 64b-
                Link Config 0: MLWI=16bit DwFcIn- MLWO=16bit DwFcOut- LWI=16bit DwFcInEn- LWO=16bit DwFcOutEn-
                Link Control 1: CFlE- CST- CFE- <LkFail+ Init- EOC+ TXO+ <CRCErr=0 IsocEn- LSEn- ExtCTL- 64b-
                Link Config 1: MLWI=8bit DwFcIn- MLWO=8bit DwFcOut- LWI=8bit DwFcInEn- LWO=8bit DwFcOutEn-
                Revision ID: 3.00
                Link Frequency 0: [e]
                Link Error 0: <Prot- <Ovfl- <EOC- CTLTm-
                Link Frequency Capability 0: 200MHz+ 300MHz- 400MHz+ 500MHz- 600MHz+ 800MHz+ 1.0GHz+ 1.2GHz+ 1.4GHz- 1.6GHz- Vend-
                Feature Capability: IsocFC+ LDTSTOP+ CRCTM- ECTLT- 64bA+ UIDRD-
                Link Frequency 1: 200MHz
                Link Error 1: <Prot- <Ovfl- <EOC- CTLTm-
                Link Frequency Capability 1: 200MHz- 300MHz- 400MHz- 500MHz- 600MHz- 800MHz- 1.0GHz- 1.2GHz- 1.4GHz- 1.6GHz- Vend-
                Error Handling: PFlE- OFlE- PFE- OFE- EOCFE- RFE- CRCFE- SERRFE- CF- RE- PNFE- ONFE- EOCNFE- RNFE- CRCNFE- SERRNFE-
                Prefetchable memory behind bridge Upper: 00-00
                Bus Number: 00
        Capabilities: [40] HyperTransport: Retry Mode
        Capabilities: [54] HyperTransport: UnitID Clumping
        Capabilities: [9c] HyperTransport: #1a
        Capabilities: [70] MSI: Enable- Count=1/4 Maskable- 64bit-
                Address: 00000000  Data: 0000
00: 02 10 14 5a 02 00 10 20 02 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 04 00 00 e0
20: 00 00 00 00 00 00 00 00 00 00 00 00 58 14 00 50
30: 00 00 00 00 f0 00 00 00 00 00 00 00 ff 00 00 00
40: 08 54 00 c0 c1 00 00 00 00 00 00 00 42 20 05 00
50: 58 14 00 50 08 9c 00 90 08 10 00 00 08 10 00 00
60: 00 00 00 00 86 01 00 00 00 00 00 40 64 56 00 78
70: 05 00 04 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 10 00 00 03 20 02 30 00 31 20 00 00
90: 00 00 00 d0 00 00 00 00 10 09 00 00 08 70 3c d0
a0: 66 00 00 00 00 00 00 05 00 00 00 00 79 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 80 08 40 80 02 20 00 11 11 d0 00 00 00
d0: 60 0e f5 7f 13 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 05 00 ff ff ff ff 00 00 00 00 00 00 00 00
f0: 08 c4 03 a8 00 80 80 00 01 00 00 00 08 00 c0 fe

00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
        Subsystem: Gigabyte Technology Co., Ltd Device 5000
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 40
        Capabilities: [40] Secure device <?>
        Capabilities: [54] MSI: Enable+ Count=1/1 Maskable- 64bit+
                Address: 00000000feeff00c  Data: 4171
        Capabilities: [64] HyperTransport: MSI Mapping Enable+ Fixed+
00: 02 10 23 5a 00 04 10 00 00 00 06 08 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 58 14 00 50
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 01 00 00
40: 0f 54 0b 01 01 00 c3 fe 00 00 00 00 00 00 00 00
50: 00 34 20 00 05 64 81 00 0c f0 ef fe 00 00 00 00
60: 71 41 00 00 08 00 03 a8 58 14 00 50 01 01 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 82 00 00 00 00 00 00 00 11 00 05 00 00 00 00 00


Hope this helps!

Thanks,
Andreas
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Alex Williamson Oct. 22, 2014, 4:02 p.m. UTC | #6 
On Wed, 2014-10-22 at 17:34 +0200, Andreas Hartmann wrote:
> Alex Williamson schrieb:
> > Hi Andreas,
> > 
> > On Fri, 2014-10-17 at 03:04 +0200, Andreas Hartmann wrote:
> >> Hello Alex,
> >>
> >> Alex Williamson wrote:
> >>> Hi Andreas,
> >> [...]
> >>> Sorry for the breakage.  Is it possible to run lspci on the device in a
> >>> loop from the host and capture whether we're failing to restore some of
> >>> the VC bits to their previous state? 
> >>
> >>> Does the problem also occur if you
> >>> unbind from host driver,
> >>
> >> The machine is booted w/ blacklisted ath9k. Then, the device is bound to
> >> vfio:
> >>
> >> echo "168c 0030" > /sys/bus/pci/drivers/vfio-pci/new_id
> >> echo 0000:03:00.0 > /sys/bus/pci/devices/0000:03:00.0/driver/unbind
> >> echo 0000:03:00.0 > /sys/bus/pci/drivers/vfio-pci/bind
> >>
> >> afterwards the VM is started -> hang.
> >>
> >> W/o starting th VM, I can bind it to vfio and unbind it from vfio w/o
> >> any problem.
> >>
> >>> echo 1 > reset in pci-sysfs,
> >>
> >> echo 1 > /sys/bus/pci/devices/0000:03:00.0 works w/o any problem while
> >> bound to vfio. Even after unbinding from vfio and rebinding to vfio
> >> again ... .
> >>
> >>> and re-bind to the
> >>
> >> Do you mean loading ath9k in host system after unbinding from vfio? If
> >> yes: Works w/o any problem. It's even possible to reset it or do a
> >> ifconfig wlan0 up, ifconfig wlan0 down, rmmod ath9k, bind it to vfio
> >> again and reset it, ....
> >>
> >> Looks like the hang only is triggered by qemu-system_x86_64 on startup
> >> the VM.
> >>
> >>> host?  I'll also try to reproduce on my 990fx system, but I won't be
> >>> able to do that until next week due to travel.  Thanks,
> > 
> > Could you send me the lspci -vvvxxxx for the device and parent root
> > port?  Thanks,
> 
> 
> Done with kernel 3.12.28 in host while the device was used in VM:
> 
> # lspci -vt
> -[0000:00]-+-00.0  Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx0 port B)
>            +-00.2  Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
>            +-02.0-[01]--+-00.0  Advanced Micro Devices, Inc. [AMD/ATI] Turks PRO [Radeon HD 6570/7570]
>            |            \-00.1  Advanced Micro Devices, Inc. [AMD/ATI] Turks/Whistler HDMI Audio [Radeon HD 6000 Series]
>            +-04.0-[02]----00.0  Etron Technology, Inc. EJ168 USB 3.0 Host Controller
>            +-05.0-[03]----00.0  Qualcomm Atheros AR93xx Wireless Network Adapter
>            +-09.0-[04]----00.0  Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
>            +-0a.0-[05]----00.0  Etron Technology, Inc. EJ168 USB 3.0 Host Controller
>            +-11.0  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode]
>            +-12.0  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>            +-12.2  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
>            +-13.0  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>            +-13.2  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
>            +-14.0  Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller
>            +-14.2  Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA)
>            +-14.3  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller
>            +-14.4-[06]--+-06.0  Intel Corporation 82557/8/9/0/1 Ethernet Pro 100
>            |            \-0e.0  VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller
>            +-14.5  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
>            +-15.0-[07]--
>            +-16.0  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>            +-16.2  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
>            +-18.0  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
>            +-18.1  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
>            +-18.2  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
>            +-18.3  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
>            +-18.4  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
>            \-18.5  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5
> 
> 
> # lspci -s 03:00 -vvvxxxx
> 03:00.0 Network controller: Qualcomm Atheros AR93xx Wireless Network Adapter (rev 01)
[snip]
> 
> 
> I'm not sure what you mean with "parent root port". Could it be this:

No, it's 00:05.0

--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Andreas Hartmann Oct. 22, 2014, 4:20 p.m. UTC | #7 
Am Wed, 22 Oct 2014 10:02:29 -0600
schrieb Alex Williamson <alex.williamson@redhat.com>:

> On Wed, 2014-10-22 at 17:34 +0200, Andreas Hartmann wrote:
> > Alex Williamson schrieb:
> > > Hi Andreas,
> > > 
> > > On Fri, 2014-10-17 at 03:04 +0200, Andreas Hartmann wrote:
> > >> Hello Alex,
> > >>
> > >> Alex Williamson wrote:
> > >>> Hi Andreas,
> > >> [...]
> > >>> Sorry for the breakage.  Is it possible to run lspci on the device in a
> > >>> loop from the host and capture whether we're failing to restore some of
> > >>> the VC bits to their previous state? 
> > >>
> > >>> Does the problem also occur if you
> > >>> unbind from host driver,
> > >>
> > >> The machine is booted w/ blacklisted ath9k. Then, the device is bound to
> > >> vfio:
> > >>
> > >> echo "168c 0030" > /sys/bus/pci/drivers/vfio-pci/new_id
> > >> echo 0000:03:00.0 > /sys/bus/pci/devices/0000:03:00.0/driver/unbind
> > >> echo 0000:03:00.0 > /sys/bus/pci/drivers/vfio-pci/bind
> > >>
> > >> afterwards the VM is started -> hang.
> > >>
> > >> W/o starting th VM, I can bind it to vfio and unbind it from vfio w/o
> > >> any problem.
> > >>
> > >>> echo 1 > reset in pci-sysfs,
> > >>
> > >> echo 1 > /sys/bus/pci/devices/0000:03:00.0 works w/o any problem while
> > >> bound to vfio. Even after unbinding from vfio and rebinding to vfio
> > >> again ... .
> > >>
> > >>> and re-bind to the
> > >>
> > >> Do you mean loading ath9k in host system after unbinding from vfio? If
> > >> yes: Works w/o any problem. It's even possible to reset it or do a
> > >> ifconfig wlan0 up, ifconfig wlan0 down, rmmod ath9k, bind it to vfio
> > >> again and reset it, ....
> > >>
> > >> Looks like the hang only is triggered by qemu-system_x86_64 on startup
> > >> the VM.
> > >>
> > >>> host?  I'll also try to reproduce on my 990fx system, but I won't be
> > >>> able to do that until next week due to travel.  Thanks,
> > > 
> > > Could you send me the lspci -vvvxxxx for the device and parent root
> > > port?  Thanks,
> > 
> > 
> > Done with kernel 3.12.28 in host while the device was used in VM:
> > 
> > # lspci -vt
> > -[0000:00]-+-00.0  Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx0 port B)
> >            +-00.2  Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
> >            +-02.0-[01]--+-00.0  Advanced Micro Devices, Inc. [AMD/ATI] Turks PRO [Radeon HD 6570/7570]
> >            |            \-00.1  Advanced Micro Devices, Inc. [AMD/ATI] Turks/Whistler HDMI Audio [Radeon HD 6000 Series]
> >            +-04.0-[02]----00.0  Etron Technology, Inc. EJ168 USB 3.0 Host Controller
> >            +-05.0-[03]----00.0  Qualcomm Atheros AR93xx Wireless Network Adapter
> >            +-09.0-[04]----00.0  Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
> >            +-0a.0-[05]----00.0  Etron Technology, Inc. EJ168 USB 3.0 Host Controller
> >            +-11.0  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode]
> >            +-12.0  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> >            +-12.2  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
> >            +-13.0  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> >            +-13.2  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
> >            +-14.0  Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller
> >            +-14.2  Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA)
> >            +-14.3  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller
> >            +-14.4-[06]--+-06.0  Intel Corporation 82557/8/9/0/1 Ethernet Pro 100
> >            |            \-0e.0  VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller
> >            +-14.5  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
> >            +-15.0-[07]--
> >            +-16.0  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> >            +-16.2  Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
> >            +-18.0  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
> >            +-18.1  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
> >            +-18.2  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
> >            +-18.3  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
> >            +-18.4  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
> >            \-18.5  Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5
> > 
> > 
> > # lspci -s 03:00 -vvvxxxx
> > 03:00.0 Network controller: Qualcomm Atheros AR93xx Wireless Network Adapter (rev 01)
> [snip]
> > 
> > 
> > I'm not sure what you mean with "parent root port". Could it be this:
> 
> No, it's 00:05.0

# lspci -s 00:05.0 -vvvxxxx
00:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port E) (prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
        I/O behind bridge: 0000c000-0000cfff
        Memory behind bridge: fdb00000-fdbfffff
        Prefetchable memory behind bridge: 00000000fda00000-00000000fdafffff
        Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [50] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [58] Express (v2) Root Port (Slot+), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag+ RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Latency L0 <1us, L1 <8us
                        ClockPM- Surprise- LLActRep+ BwNot+
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
                SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
                        Slot #5, PowerLimit 75.000W; Interlock- NoCompl+
                SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
                        Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
                SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
                        Changed: MRL- PresDet+ LinkState+
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported ARIFwd+
                DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit-
                Address: 00000000  Data: 0000
        Capabilities: [b0] Subsystem: Gigabyte Technology Co., Ltd Device 5000
        Capabilities: [b8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [190 v1] Access Control Services
                ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+
                ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
        Kernel driver in use: pcieport
        Kernel modules: shpchp
00: 02 10 19 5a 07 00 10 00 00 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 00 03 03 00 c1 c1 00 20
20: b0 fd b0 fd a1 fd a1 fd 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 05 01 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 01 58 03 c8 00 00 00 00 10 a0 42 01 20 80 00 00
60: 10 08 00 00 12 cc 31 01 40 00 11 70 80 25 2c 00
70: 00 00 48 01 00 00 01 00 00 00 00 00 3f 00 00 00
80: 06 00 00 00 00 00 00 00 42 00 01 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 05 b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 0d b8 00 00 58 14 00 50 08 00 03 a8 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 ff ff ff ff 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
100: 0b 00 01 19 01 00 01 01 00 58 22 00 00 00 00 00
110: 02 00 01 19 00 00 00 00 00 00 00 00 00 00 00 00
120: 01 00 00 00 ff 00 00 80 00 00 00 00 01 00 00 00
130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
140: 03 00 01 19 00 00 00 00 00 00 00 00 00 00 00 00
150: 01 00 01 19 00 00 00 00 00 00 00 00 30 20 06 00
160: 00 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00
170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
190: 0d 00 01 00 5f 00 1d 00 00 00 00 00 00 00 00 00
1a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
250: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
260: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
270: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
290: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
4f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
510: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
520: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
580: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
5f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
670: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
690: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
6f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
710: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
720: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
790: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
7a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
7b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
7c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
7d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
7e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
7f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
810: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
860: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
8f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
900: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
910: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
920: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
930: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
940: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
9f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ad0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ae0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
af0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ba0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
bb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
bc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
bd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
be0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
bf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ca0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
cb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
cc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
cd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
da0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
db0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
dc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
dd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
de0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
df0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
fa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00


Thanks,
Andreas
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 2c9ac70254e2..8ef8bc56a584 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -1007,8 +1007,6 @@  int pci_save_state(struct pci_dev *dev)
 		return i;
 	if ((i = pci_save_pcix_state(dev)) != 0)
 		return i;
-	if ((i = pci_save_vc_state(dev)) != 0)
-		return i;
 	return 0;
 }
 EXPORT_SYMBOL(pci_save_state);
@@ -1072,7 +1070,6 @@  void pci_restore_state(struct pci_dev *dev)
 	/* PCI Express register must be restored first */
 	pci_restore_pcie_state(dev);
 	pci_restore_ats_state(dev);
-	pci_restore_vc_state(dev);
 
 	pci_restore_config_space(dev);
 
@@ -2170,8 +2167,6 @@  void pci_allocate_cap_save_buffers(struct pci_dev *dev)
 	if (error)
 		dev_err(&dev->dev,
 			"unable to preallocate PCI-X save buffer\n");
-
-	pci_allocate_vc_save_buffers(dev);
 }
 
 void pci_free_cap_save_buffers(struct pci_dev *dev)