Message ID | 1470789844-16401-2-git-send-email-mauricfo@linux.vnet.ibm.com (mailing list archive) |
---|---|
State | Superseded |
Headers | show |
On 10/08/16 10:44, Mauricio Faria de Oliveira wrote: > This commit introduces the 'refcount' field in struct pci_controller, > along with the corresponding functions 'controller_(get|put|free)()'. > > The functions 'pcibios_(alloc|free)_controller()' are modified to use > that in a backwards compatible manner. (i.e., kfree(phb) is performed > when pcibios_free_controller() is called.) > > So, this patch adds the infrastructure with no functional differences > to current users of pcibios_(alloc|free)_controller(). Notably, only > the pseries platform calls pcibios_free_controller() for some purpose > other than to release the pci_controller in case of errors just after > the call to pcibios_alloc_controller() (i.e., 'goto error' scenarios). cxl's vPHB API also uses pcibios_free_controller() (which is why we export the symbol, as it's called from within the cxl module). When we remove/shutdown the underlying cxl device, we remove all the devices sitting on the vPHB and then free the vPHB. I'm currently working on a cxl defect found by an IBM test team where we run into this - will review this patch more thoroughly and test it shortly. Andrew
On 08/09/2016 10:45 PM, Andrew Donnellan wrote: >> [snip] Notably, only >> the pseries platform calls pcibios_free_controller() for some purpose >> other than to release the pci_controller in case of errors just after >> the call to pcibios_alloc_controller() (i.e., 'goto error' scenarios). > > cxl's vPHB API also uses pcibios_free_controller() [snip] Cool. I see I missed this report line from grep; thanks. I was mostly biased at arch/powerpc/ and driver/pci/ these days. > I'm currently working on a cxl defect found by an IBM test team where we > run into this - will review this patch more thoroughly and test it shortly. That's great; thanks!
On 08/09/2016 10:45 PM, Andrew Donnellan wrote: > I'm currently working on a cxl defect found by an IBM test team where we > run into this - will review this patch more thoroughly and test it shortly. Gavin provided a review/suggestions via chat, pointing to rely on the refcount that already exists in the PCI subsystem (not reinvent another) and leverage the release of the PCI root bus -- which is much simpler! He replied there should be no problems w/ the EEH reset path (PCI root bus not released) nor w/ other structs w/ refs to the PHB (PCI DN, EEH PE, EEH DEV). I'll go down that path for a PATCH v3.
On 08/10/2016 10:53 AM, Mauricio Faria de Oliveira wrote:
> I'll go down that path for a PATCH v3.
That is,
'powerpc/pseries: use pci_host_bridge.release_fn() to kfree(phb)'
diff --git a/arch/powerpc/include/asm/pci-bridge.h b/arch/powerpc/include/asm/pci-bridge.h index b5e88e4..6fde0a9 100644 --- a/arch/powerpc/include/asm/pci-bridge.h +++ b/arch/powerpc/include/asm/pci-bridge.h @@ -10,6 +10,7 @@ #include <linux/pci.h> #include <linux/list.h> #include <linux/ioport.h> +#include <linux/kref.h> struct device_node; @@ -128,9 +129,23 @@ struct pci_controller { struct pci_dn *pci_data; #endif /* CONFIG_PPC64 */ + /* + * Reference counting for the structures: + * - TODO pci_dev + * - TODO pci_bus + * - TODO pci_dn + * - TODO eeh_pe + * - TODO eeh_dev + */ + struct kref refcount; + void *private_data; }; +void controller_get(struct pci_controller *phb); +void controller_put(struct pci_controller *phb); +void controller_free(struct kref *kref); + /* These are used for config access before all the PCI probing has been done. */ extern int early_read_config_byte(struct pci_controller *hose, int bus, diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c index 08afddf..29b37d3 100644 --- a/arch/powerpc/kernel/pci-common.c +++ b/arch/powerpc/kernel/pci-common.c @@ -114,6 +114,8 @@ struct pci_controller *pcibios_alloc_controller(struct device_node *dev) phb = zalloc_maybe_bootmem(sizeof(struct pci_controller), GFP_KERNEL); if (phb == NULL) return NULL; + + kref_init(&phb->refcount); /* use first reference for hose_list entry */ spin_lock(&hose_spinlock); phb->global_number = get_phb_number(dev); list_add_tail(&phb->list_node, &hose_list); @@ -130,12 +132,53 @@ struct pci_controller *pcibios_alloc_controller(struct device_node *dev) PHB_SET_NODE(phb, nid); } #endif + + pr_debug("PCI domain %d, phb %p, phb->is_dynamic %d\n", + phb->global_number, phb, phb->is_dynamic); + return phb; } EXPORT_SYMBOL_GPL(pcibios_alloc_controller); +void controller_get(struct pci_controller *phb) +{ + if (unlikely(!phb)) { + pr_warn("%s: null PHB; refcount bug!\n", __func__); + WARN_ON(1); + } else { + pr_debug("PCI domain %d, phb %p\n", phb->global_number, phb); + kref_get(&phb->refcount); + } +} + +void controller_put(struct pci_controller *phb) +{ + if (unlikely(!phb)) { + pr_warn("%s: null PHB; refcount bug!\n", __func__); + WARN_ON(1); + } else { + pr_debug("PCI domain %d, phb %p\n", phb->global_number, phb); + kref_put(&phb->refcount, controller_free); + } +} + +void controller_free(struct kref *kref) +{ + struct pci_controller *phb = container_of(kref, struct pci_controller, + refcount); + + pr_info("%s: PCI domain: %d, phb %p, phb->is_dynamic %d\n", + __func__, phb->global_number, phb, phb->is_dynamic); + + if (phb->is_dynamic) + kfree(phb); +} + void pcibios_free_controller(struct pci_controller *phb) { + pr_debug("PCI domain %d, phb %p, phb->is_dynamic %d\n", + phb->global_number, phb, phb->is_dynamic); + spin_lock(&hose_spinlock); /* Clear bit of phb_bitmap to allow reuse of this PHB number. */ @@ -143,10 +186,8 @@ void pcibios_free_controller(struct pci_controller *phb) clear_bit(phb->global_number, phb_bitmap); list_del(&phb->list_node); + controller_put(phb); spin_unlock(&hose_spinlock); - - if (phb->is_dynamic) - kfree(phb); } EXPORT_SYMBOL_GPL(pcibios_free_controller);
This commit introduces the 'refcount' field in struct pci_controller, along with the corresponding functions 'controller_(get|put|free)()'. The functions 'pcibios_(alloc|free)_controller()' are modified to use that in a backwards compatible manner. (i.e., kfree(phb) is performed when pcibios_free_controller() is called.) So, this patch adds the infrastructure with no functional differences to current users of pcibios_(alloc|free)_controller(). Notably, only the pseries platform calls pcibios_free_controller() for some purpose other than to release the pci_controller in case of errors just after the call to pcibios_alloc_controller() (i.e., 'goto error' scenarios). Signed-off-by: Mauricio Faria de Oliveira <mauricfo@linux.vnet.ibm.com> --- arch/powerpc/include/asm/pci-bridge.h | 15 +++++++++++ arch/powerpc/kernel/pci-common.c | 47 ++++++++++++++++++++++++++++++++--- 2 files changed, 59 insertions(+), 3 deletions(-)