diff mbox

[RFC,v2,part3,07/11] PCI, xen-pcifront: use new PCI interfaces to simplify implementation

Message ID 51B4B266.5010100@gmail.com
State Superseded
Headers show

Commit Message

Jiang Liu June 9, 2013, 4:50 p.m. UTC 
On Sat 08 Jun 2013 01:07:06 AM CST, Konrad Rzeszutek Wilk wrote:
> On Sat, Jun 08, 2013 at 12:50:31AM +0800, Jiang Liu wrote:
>> On 06/07/2013 11:38 PM, Konrad Rzeszutek Wilk wrote:
>>> On Fri, Jun 07, 2013 at 10:50:24AM -0400, Konrad Rzeszutek Wilk wrote:
>>>> On Thu, May 16, 2013 at 11:50:55PM +0800, Jiang Liu wrote:
>>>>> Use new PCI interfaces to simplify xen-pcifront implementation:
>>>>> 1) Use pci_create_root_bus() instead of pci_scan_bus_parented()
>>>>>    because pci_scan_bus_parented() is marked as __deprecated.This
>>>>>    also gets rid of a duplicated call of pci_bus_start_devices().
>>>>> 2) Use pci_stop_root_bus() and pci_remove_root_bus() instead of
>>>>>    open-coded private implementation.
>>>>> 3) Use pci_set_host_bridge_release() to release data structures
>>>>>    associated with PCI root buses.
>>>>> 4) Use pci_bus_get()/pci_bus_put() to manage PCI root bus reference
>>>>>    count.
>>>>>
>>>>> This is also a preparation for coming PCI bus lock enhancement.
>>>
>>> With this patch from :
>>>
>>>  Merge branch 'pci_lock_v3' of https://github.com/jiangliu/linux into testing
>>>
>>>
>>> it blows up when detaching the device.
>> Hi Konrad,
>>    Thanks for testing! According to the log messages, this issue should
>> be related to pci bus reference counter management. Seems we have done
>> an extra(unbalanced) release of pci bus device.
>>    Will investigate it tomorrow!
>
> That is quite commendable that you are willing to look over this on
> the weekend but I am not going to be able to rerun this test until
> some time in the week. You could enjoy the weekend and just look at
> this during the week.

Hi Konrad,
     We should have root-caused this bug, which is caused by 
for_each_pci_root_bus().
Current implementation doesn't support root bus deletion when walking 
PCI root
buses by for_each_pci_root_bus(). The reference counter 
(pci_bus->dev.knode_class.n_ref)
becomes zero after returning from pci_remove_root_bus(), so it triggers 
kref warnings
and double-free of klist_node object when we call 
pci_get_next_root_bus() to get the
next PCI root bus.
     So we will first revert to  list_for_each_entry_safe(bus, temp, 
&pci_root_buses, node)
and solve this issue in next version of for_each_pci_root_bus().

Regards!
Gerry

---
                        pci_stop_root_bus(bus);
---

--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Konrad Rzeszutek Wilk June 10, 2013, 4:58 p.m. UTC | #1 
On Mon, Jun 10, 2013 at 12:50:46AM +0800, Jiang Liu wrote:
> On Sat 08 Jun 2013 01:07:06 AM CST, Konrad Rzeszutek Wilk wrote:
> > On Sat, Jun 08, 2013 at 12:50:31AM +0800, Jiang Liu wrote:
> >> On 06/07/2013 11:38 PM, Konrad Rzeszutek Wilk wrote:
> >>> On Fri, Jun 07, 2013 at 10:50:24AM -0400, Konrad Rzeszutek Wilk wrote:
> >>>> On Thu, May 16, 2013 at 11:50:55PM +0800, Jiang Liu wrote:
> >>>>> Use new PCI interfaces to simplify xen-pcifront implementation:
> >>>>> 1) Use pci_create_root_bus() instead of pci_scan_bus_parented()
> >>>>>    because pci_scan_bus_parented() is marked as __deprecated.This
> >>>>>    also gets rid of a duplicated call of pci_bus_start_devices().
> >>>>> 2) Use pci_stop_root_bus() and pci_remove_root_bus() instead of
> >>>>>    open-coded private implementation.
> >>>>> 3) Use pci_set_host_bridge_release() to release data structures
> >>>>>    associated with PCI root buses.
> >>>>> 4) Use pci_bus_get()/pci_bus_put() to manage PCI root bus reference
> >>>>>    count.
> >>>>>
> >>>>> This is also a preparation for coming PCI bus lock enhancement.
> >>>
> >>> With this patch from :
> >>>
> >>>  Merge branch 'pci_lock_v3' of https://github.com/jiangliu/linux into testing
> >>>
> >>>
> >>> it blows up when detaching the device.
> >> Hi Konrad,
> >>    Thanks for testing! According to the log messages, this issue should
> >> be related to pci bus reference counter management. Seems we have done
> >> an extra(unbalanced) release of pci bus device.
> >>    Will investigate it tomorrow!
> >
> > That is quite commendable that you are willing to look over this on
> > the weekend but I am not going to be able to rerun this test until
> > some time in the week. You could enjoy the weekend and just look at
> > this during the week.
> 
> Hi Konrad,
>      We should have root-caused this bug, which is caused by 
> for_each_pci_root_bus().
> Current implementation doesn't support root bus deletion when walking 
> PCI root
> buses by for_each_pci_root_bus(). The reference counter 
> (pci_bus->dev.knode_class.n_ref)
> becomes zero after returning from pci_remove_root_bus(), so it triggers 
> kref warnings
> and double-free of klist_node object when we call 
> pci_get_next_root_bus() to get the
> next PCI root bus.
>      So we will first revert to  list_for_each_entry_safe(bus, temp, 
> &pci_root_buses, node)
> and solve this issue in next version of for_each_pci_root_bus().

That definitly solves the issue. Thanks!

I used pci_lock_v3 of https://github.com/jiangliu/linux and the fix
below.

Is there a new patchset you are going to be posting or a git branch
that I can look over?

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Jiang Liu June 10, 2013, 5:08 p.m. UTC | #2 
On Tue 11 Jun 2013 12:58:01 AM CST, Konrad Rzeszutek Wilk wrote:
> On Mon, Jun 10, 2013 at 12:50:46AM +0800, Jiang Liu wrote:
>> On Sat 08 Jun 2013 01:07:06 AM CST, Konrad Rzeszutek Wilk wrote:
>>> On Sat, Jun 08, 2013 at 12:50:31AM +0800, Jiang Liu wrote:
>>>> On 06/07/2013 11:38 PM, Konrad Rzeszutek Wilk wrote:
>>>>> On Fri, Jun 07, 2013 at 10:50:24AM -0400, Konrad Rzeszutek Wilk wrote:
>>>>>> On Thu, May 16, 2013 at 11:50:55PM +0800, Jiang Liu wrote:
>>>>>>> Use new PCI interfaces to simplify xen-pcifront implementation:
>>>>>>> 1) Use pci_create_root_bus() instead of pci_scan_bus_parented()
>>>>>>>    because pci_scan_bus_parented() is marked as __deprecated.This
>>>>>>>    also gets rid of a duplicated call of pci_bus_start_devices().
>>>>>>> 2) Use pci_stop_root_bus() and pci_remove_root_bus() instead of
>>>>>>>    open-coded private implementation.
>>>>>>> 3) Use pci_set_host_bridge_release() to release data structures
>>>>>>>    associated with PCI root buses.
>>>>>>> 4) Use pci_bus_get()/pci_bus_put() to manage PCI root bus reference
>>>>>>>    count.
>>>>>>>
>>>>>>> This is also a preparation for coming PCI bus lock enhancement.
>>>>>
>>>>> With this patch from :
>>>>>
>>>>>  Merge branch 'pci_lock_v3' of https://github.com/jiangliu/linux into testing
>>>>>
>>>>>
>>>>> it blows up when detaching the device.
>>>> Hi Konrad,
>>>>    Thanks for testing! According to the log messages, this issue should
>>>> be related to pci bus reference counter management. Seems we have done
>>>> an extra(unbalanced) release of pci bus device.
>>>>    Will investigate it tomorrow!
>>>
>>> That is quite commendable that you are willing to look over this on
>>> the weekend but I am not going to be able to rerun this test until
>>> some time in the week. You could enjoy the weekend and just look at
>>> this during the week.
>>
>> Hi Konrad,
>>      We should have root-caused this bug, which is caused by
>> for_each_pci_root_bus().
>> Current implementation doesn't support root bus deletion when walking
>> PCI root
>> buses by for_each_pci_root_bus(). The reference counter
>> (pci_bus->dev.knode_class.n_ref)
>> becomes zero after returning from pci_remove_root_bus(), so it triggers
>> kref warnings
>> and double-free of klist_node object when we call
>> pci_get_next_root_bus() to get the
>> next PCI root bus.
>>      So we will first revert to  list_for_each_entry_safe(bus, temp,
>> &pci_root_buses, node)
>> and solve this issue in next version of for_each_pci_root_bus().
>
> That definitly solves the issue. Thanks!
>
> I used pci_lock_v3 of https://github.com/jiangliu/linux and the fix
> below.
>
> Is there a new patchset you are going to be posting or a git branch
> that I can look over?
>
> Thanks.
Hi Konrad,
     Really appreciate your support! I will try to post a new version 
tomorrow,
but I think there won't be big changes except addressing some review
comments from you.
    Thanks!
    Gerry
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Konrad Rzeszutek Wilk June 14, 2013, 6:07 p.m. UTC | #3 
On Tue, Jun 11, 2013 at 01:08:14AM +0800, Jiang Liu wrote:
> On Tue 11 Jun 2013 12:58:01 AM CST, Konrad Rzeszutek Wilk wrote:
> > On Mon, Jun 10, 2013 at 12:50:46AM +0800, Jiang Liu wrote:
> >> On Sat 08 Jun 2013 01:07:06 AM CST, Konrad Rzeszutek Wilk wrote:
> >>> On Sat, Jun 08, 2013 at 12:50:31AM +0800, Jiang Liu wrote:
> >>>> On 06/07/2013 11:38 PM, Konrad Rzeszutek Wilk wrote:
> >>>>> On Fri, Jun 07, 2013 at 10:50:24AM -0400, Konrad Rzeszutek Wilk wrote:
> >>>>>> On Thu, May 16, 2013 at 11:50:55PM +0800, Jiang Liu wrote:
> >>>>>>> Use new PCI interfaces to simplify xen-pcifront implementation:
> >>>>>>> 1) Use pci_create_root_bus() instead of pci_scan_bus_parented()
> >>>>>>>    because pci_scan_bus_parented() is marked as __deprecated.This
> >>>>>>>    also gets rid of a duplicated call of pci_bus_start_devices().
> >>>>>>> 2) Use pci_stop_root_bus() and pci_remove_root_bus() instead of
> >>>>>>>    open-coded private implementation.
> >>>>>>> 3) Use pci_set_host_bridge_release() to release data structures
> >>>>>>>    associated with PCI root buses.
> >>>>>>> 4) Use pci_bus_get()/pci_bus_put() to manage PCI root bus reference
> >>>>>>>    count.
> >>>>>>>
> >>>>>>> This is also a preparation for coming PCI bus lock enhancement.
> >>>>>
> >>>>> With this patch from :
> >>>>>
> >>>>>  Merge branch 'pci_lock_v3' of https://github.com/jiangliu/linux into testing
> >>>>>
> >>>>>
> >>>>> it blows up when detaching the device.
> >>>> Hi Konrad,
> >>>>    Thanks for testing! According to the log messages, this issue should
> >>>> be related to pci bus reference counter management. Seems we have done
> >>>> an extra(unbalanced) release of pci bus device.
> >>>>    Will investigate it tomorrow!
> >>>
> >>> That is quite commendable that you are willing to look over this on
> >>> the weekend but I am not going to be able to rerun this test until
> >>> some time in the week. You could enjoy the weekend and just look at
> >>> this during the week.
> >>
> >> Hi Konrad,
> >>      We should have root-caused this bug, which is caused by
> >> for_each_pci_root_bus().
> >> Current implementation doesn't support root bus deletion when walking
> >> PCI root
> >> buses by for_each_pci_root_bus(). The reference counter
> >> (pci_bus->dev.knode_class.n_ref)
> >> becomes zero after returning from pci_remove_root_bus(), so it triggers
> >> kref warnings
> >> and double-free of klist_node object when we call
> >> pci_get_next_root_bus() to get the
> >> next PCI root bus.
> >>      So we will first revert to  list_for_each_entry_safe(bus, temp,
> >> &pci_root_buses, node)
> >> and solve this issue in next version of for_each_pci_root_bus().
> >
> > That definitly solves the issue. Thanks!
> >
> > I used pci_lock_v3 of https://github.com/jiangliu/linux and the fix
> > below.
> >
> > Is there a new patchset you are going to be posting or a git branch
> > that I can look over?
> >
> > Thanks.
> Hi Konrad,
>      Really appreciate your support! I will try to post a new version 
> tomorrow,
> but I think there won't be big changes except addressing some review
> comments from you.

I also get this:

ERROR: "pci_set_host_bridge_release" [drivers/pci/xen-pcifront.ko] undefined!
ERROR: "pci_create_root_bus" [drivers/pci/xen-pcifront.ko] undefined!
ERROR: "pci_remove_root_bus" [drivers/pci/xen-pcifront.ko] undefined!
ERROR: "pci_stop_root_bus" [drivers/pci/xen-pcifront.ko] undefined!

when building it as a module.
--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/drivers/pci/xen-pcifront.c b/drivers/pci/xen-pcifront.c
index 6aa2c0f..6e577db 100644
--- a/drivers/pci/xen-pcifront.c
+++ b/drivers/pci/xen-pcifront.c
@@ -553,11 +553,11 @@  static int pcifront_rescan_root(struct 
pcifront_device *pdev,
 static void pcifront_free_roots(struct pcifront_device *pdev)
 {
        struct pcifront_sd *sd;
-       struct pci_bus *bus;
+       struct pci_bus *bus, *temp;

        dev_dbg(&pdev->xdev->dev, "cleaning up root buses\n");

-       for_each_pci_root_bus(bus) {
+       list_for_each_entry_safe(bus, temp, &pci_root_buses, node) {
                sd = bus->sysdata;
                if (sd->pdev == pdev) {