| Message ID | 20170309182205.GB3685@bhelgaas-glaptop.roam.corp.google.com |
|---|---|
| State | New |
| Headers | show |
On 03/09/2017 07:22 PM, Bjorn Helgaas wrote: > On Thu, Mar 09, 2017 at 08:39:07AM +0100, Rafał Miłecki wrote: >> On 03/08/2017 01:56 PM, Rafał Miłecki wrote: >>> I just tried upgrading BCM5301X from 4.4 to 4.9 and noticed I don't see card >>> connected to the 2nd controller. >>> >>> pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00 >>> pci_bus 0000:00: root bus resource [mem 0x08000000-0x0fffffff] >>> pcie_iproc_bcma bcma0:7: link: UP >>> PCI: bus0: Fast back to back transfers disabled >>> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring >>> PCI: bus1: Fast back to back transfers disabled >>> pci 0000:00:00.0: BAR 8: assigned [mem 0x08000000-0x080fffff] >>> pci 0000:01:00.0: BAR 0: assigned [mem 0x08000000-0x08007fff 64bit] >>> pci 0000:00:00.0: PCI bridge to [bus 01] >>> pci 0000:00:00.0: bridge window [mem 0x08000000-0x080fffff] >>> >>> pcie_iproc_bcma bcma0:8: resource collision: [mem 0x40000000-0x47ffffff] conflicts with PCIe MEM space [mem 0x40000000-0x47ffffff] >>> pcie_iproc_bcma bcma0:8: PCIe controller setup failed >>> pcie_iproc_bcma: probe of bcma0:8 failed with error -16 >>> >>> >>> This used to work with older kernels because there wasn't any collision check: >>> >>> pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00 >>> pci_bus 0000:00: root bus resource [mem 0x08000000-0x0fffffff] >>> pcie_iproc_bcma bcma0:7: link: UP >>> PCI: bus0: Fast back to back transfers disabled >>> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring >>> PCI: bus1: Fast back to back transfers disabled >>> pci 0000:00:00.0: BAR 8: assigned [mem 0x08000000-0x080fffff] >>> pci 0000:01:00.0: BAR 0: assigned [mem 0x08000000-0x08007fff 64bit] >>> pci 0000:00:00.0: PCI bridge to [bus 01] >>> pci 0000:00:00.0: bridge window [mem 0x08000000-0x080fffff] >>> >>> pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00 >>> pci_bus 0001:00: root bus resource [mem 0x40000000-0x47ffffff] >>> pcie_iproc_bcma bcma0:8: link: UP >>> PCI: bus0: Fast back to back transfers disabled >>> pci 0001:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring >>> PCI: bus1: Fast back to back transfers disabled >>> pci 0001:00:00.0: BAR 8: assigned [mem 0x40000000-0x400fffff] >>> pci 0001:01:00.0: BAR 0: assigned [mem 0x40000000-0x40007fff 64bit] >>> pci 0001:00:00.0: PCI bridge to [bus 01] >>> pci 0001:00:00.0: bridge window [mem 0x40000000-0x400fffff] >>> >>> >>> I guess the check is OK after all and the real problem is iproc driver assigning >>> the same resource. >>> >>> Broadcom team: could you take a look at this, please? >> >> I found a reason of this conflict (and probably random crashes I started >> seeing with 4.9). I believe we have a memory corruption. > > Yep, we're using a resource structure on the stack when we shouldn't. Can > you try the patch below? Works great! Will you push it with Cc: stable@vger.kernel.org # v4.8+ please? Thanks for looking at this! Also see below... > (...) > if (ret) > dev_err(dev, "PCIe controller setup failed\n"); > - > - pci_free_resource_list(&res); > + pci_free_resource_list(&resources); > + return ret; > + } Works great, except it doesn't compile (braces) ;)
Hi Bjorn, Thanks for helping take care of this. Ray On 3/10/2017 8:02 AM, Rafał Miłecki wrote: > On 03/09/2017 07:22 PM, Bjorn Helgaas wrote: >> On Thu, Mar 09, 2017 at 08:39:07AM +0100, Rafał Miłecki wrote: >>> On 03/08/2017 01:56 PM, Rafał Miłecki wrote: >>>> I just tried upgrading BCM5301X from 4.4 to 4.9 and noticed I don't >>>> see card >>>> connected to the 2nd controller. >>>> >>>> pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00 >>>> pci_bus 0000:00: root bus resource [mem 0x08000000-0x0fffffff] >>>> pcie_iproc_bcma bcma0:7: link: UP >>>> PCI: bus0: Fast back to back transfers disabled >>>> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), >>>> reconfiguring >>>> PCI: bus1: Fast back to back transfers disabled >>>> pci 0000:00:00.0: BAR 8: assigned [mem 0x08000000-0x080fffff] >>>> pci 0000:01:00.0: BAR 0: assigned [mem 0x08000000-0x08007fff 64bit] >>>> pci 0000:00:00.0: PCI bridge to [bus 01] >>>> pci 0000:00:00.0: bridge window [mem 0x08000000-0x080fffff] >>>> >>>> pcie_iproc_bcma bcma0:8: resource collision: [mem >>>> 0x40000000-0x47ffffff] conflicts with PCIe MEM space [mem >>>> 0x40000000-0x47ffffff] >>>> pcie_iproc_bcma bcma0:8: PCIe controller setup failed >>>> pcie_iproc_bcma: probe of bcma0:8 failed with error -16 >>>> >>>> >>>> This used to work with older kernels because there wasn't any >>>> collision check: >>>> >>>> pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00 >>>> pci_bus 0000:00: root bus resource [mem 0x08000000-0x0fffffff] >>>> pcie_iproc_bcma bcma0:7: link: UP >>>> PCI: bus0: Fast back to back transfers disabled >>>> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), >>>> reconfiguring >>>> PCI: bus1: Fast back to back transfers disabled >>>> pci 0000:00:00.0: BAR 8: assigned [mem 0x08000000-0x080fffff] >>>> pci 0000:01:00.0: BAR 0: assigned [mem 0x08000000-0x08007fff 64bit] >>>> pci 0000:00:00.0: PCI bridge to [bus 01] >>>> pci 0000:00:00.0: bridge window [mem 0x08000000-0x080fffff] >>>> >>>> pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00 >>>> pci_bus 0001:00: root bus resource [mem 0x40000000-0x47ffffff] >>>> pcie_iproc_bcma bcma0:8: link: UP >>>> PCI: bus0: Fast back to back transfers disabled >>>> pci 0001:00:00.0: bridge configuration invalid ([bus 00-00]), >>>> reconfiguring >>>> PCI: bus1: Fast back to back transfers disabled >>>> pci 0001:00:00.0: BAR 8: assigned [mem 0x40000000-0x400fffff] >>>> pci 0001:01:00.0: BAR 0: assigned [mem 0x40000000-0x40007fff 64bit] >>>> pci 0001:00:00.0: PCI bridge to [bus 01] >>>> pci 0001:00:00.0: bridge window [mem 0x40000000-0x400fffff] >>>> >>>> >>>> I guess the check is OK after all and the real problem is iproc >>>> driver assigning >>>> the same resource. >>>> >>>> Broadcom team: could you take a look at this, please? >>> >>> I found a reason of this conflict (and probably random crashes I started >>> seeing with 4.9). I believe we have a memory corruption. >> >> Yep, we're using a resource structure on the stack when we shouldn't. >> Can >> you try the patch below? > > Works great! Will you push it with > Cc: stable@vger.kernel.org # v4.8+ > please? > > Thanks for looking at this! > > Also see below... > > >> (...) >> if (ret) >> dev_err(dev, "PCIe controller setup failed\n"); >> - >> - pci_free_resource_list(&res); >> + pci_free_resource_list(&resources); >> + return ret; >> + } > > Works great, except it doesn't compile (braces) ;)
On Fri, Mar 10, 2017 at 05:02:48PM +0100, Rafał Miłecki wrote: > On 03/09/2017 07:22 PM, Bjorn Helgaas wrote: > >On Thu, Mar 09, 2017 at 08:39:07AM +0100, Rafał Miłecki wrote: > >>On 03/08/2017 01:56 PM, Rafał Miłecki wrote: > >>>I just tried upgrading BCM5301X from 4.4 to 4.9 and noticed I don't see card > >>>connected to the 2nd controller. > >>> > >>>pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00 > >>>pci_bus 0000:00: root bus resource [mem 0x08000000-0x0fffffff] > >>>pcie_iproc_bcma bcma0:7: link: UP > >>>PCI: bus0: Fast back to back transfers disabled > >>>pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring > >>>PCI: bus1: Fast back to back transfers disabled > >>>pci 0000:00:00.0: BAR 8: assigned [mem 0x08000000-0x080fffff] > >>>pci 0000:01:00.0: BAR 0: assigned [mem 0x08000000-0x08007fff 64bit] > >>>pci 0000:00:00.0: PCI bridge to [bus 01] > >>>pci 0000:00:00.0: bridge window [mem 0x08000000-0x080fffff] > >>> > >>>pcie_iproc_bcma bcma0:8: resource collision: [mem 0x40000000-0x47ffffff] conflicts with PCIe MEM space [mem 0x40000000-0x47ffffff] > >>>pcie_iproc_bcma bcma0:8: PCIe controller setup failed > >>>pcie_iproc_bcma: probe of bcma0:8 failed with error -16 > >>> > >>> > >>>This used to work with older kernels because there wasn't any collision check: > >>> > >>>pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00 > >>>pci_bus 0000:00: root bus resource [mem 0x08000000-0x0fffffff] > >>>pcie_iproc_bcma bcma0:7: link: UP > >>>PCI: bus0: Fast back to back transfers disabled > >>>pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring > >>>PCI: bus1: Fast back to back transfers disabled > >>>pci 0000:00:00.0: BAR 8: assigned [mem 0x08000000-0x080fffff] > >>>pci 0000:01:00.0: BAR 0: assigned [mem 0x08000000-0x08007fff 64bit] > >>>pci 0000:00:00.0: PCI bridge to [bus 01] > >>>pci 0000:00:00.0: bridge window [mem 0x08000000-0x080fffff] > >>> > >>>pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00 > >>>pci_bus 0001:00: root bus resource [mem 0x40000000-0x47ffffff] > >>>pcie_iproc_bcma bcma0:8: link: UP > >>>PCI: bus0: Fast back to back transfers disabled > >>>pci 0001:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring > >>>PCI: bus1: Fast back to back transfers disabled > >>>pci 0001:00:00.0: BAR 8: assigned [mem 0x40000000-0x400fffff] > >>>pci 0001:01:00.0: BAR 0: assigned [mem 0x40000000-0x40007fff 64bit] > >>>pci 0001:00:00.0: PCI bridge to [bus 01] > >>>pci 0001:00:00.0: bridge window [mem 0x40000000-0x400fffff] > >>> > >>> > >>>I guess the check is OK after all and the real problem is iproc driver assigning > >>>the same resource. > >>> > >>>Broadcom team: could you take a look at this, please? > >> > >>I found a reason of this conflict (and probably random crashes I started > >>seeing with 4.9). I believe we have a memory corruption. > > > >Yep, we're using a resource structure on the stack when we shouldn't. Can > >you try the patch below? > > Works great! Will you push it with > Cc: stable@vger.kernel.org # v4.8+ > please? Thanks for testing this. I applied the patch (with the braces fix) to for-linus for v4.11. > >(...) > > if (ret) > > dev_err(dev, "PCIe controller setup failed\n"); > >- > >- pci_free_resource_list(&res); > >+ pci_free_resource_list(&resources); > >+ return ret; > >+ } > > Works great, except it doesn't compile (braces) ;)
On 03/10/2017 06:47 PM, Bjorn Helgaas wrote: > On Fri, Mar 10, 2017 at 05:02:48PM +0100, Rafał Miłecki wrote: >> On 03/09/2017 07:22 PM, Bjorn Helgaas wrote: >>> On Thu, Mar 09, 2017 at 08:39:07AM +0100, Rafał Miłecki wrote: >>>> On 03/08/2017 01:56 PM, Rafał Miłecki wrote: >>>>> I just tried upgrading BCM5301X from 4.4 to 4.9 and noticed I don't see card >>>>> connected to the 2nd controller. >>>>> >>>>> pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00 >>>>> pci_bus 0000:00: root bus resource [mem 0x08000000-0x0fffffff] >>>>> pcie_iproc_bcma bcma0:7: link: UP >>>>> PCI: bus0: Fast back to back transfers disabled >>>>> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring >>>>> PCI: bus1: Fast back to back transfers disabled >>>>> pci 0000:00:00.0: BAR 8: assigned [mem 0x08000000-0x080fffff] >>>>> pci 0000:01:00.0: BAR 0: assigned [mem 0x08000000-0x08007fff 64bit] >>>>> pci 0000:00:00.0: PCI bridge to [bus 01] >>>>> pci 0000:00:00.0: bridge window [mem 0x08000000-0x080fffff] >>>>> >>>>> pcie_iproc_bcma bcma0:8: resource collision: [mem 0x40000000-0x47ffffff] conflicts with PCIe MEM space [mem 0x40000000-0x47ffffff] >>>>> pcie_iproc_bcma bcma0:8: PCIe controller setup failed >>>>> pcie_iproc_bcma: probe of bcma0:8 failed with error -16 >>>>> >>>>> >>>>> This used to work with older kernels because there wasn't any collision check: >>>>> >>>>> pcie_iproc_bcma bcma0:7: PCI host bridge to bus 0000:00 >>>>> pci_bus 0000:00: root bus resource [mem 0x08000000-0x0fffffff] >>>>> pcie_iproc_bcma bcma0:7: link: UP >>>>> PCI: bus0: Fast back to back transfers disabled >>>>> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring >>>>> PCI: bus1: Fast back to back transfers disabled >>>>> pci 0000:00:00.0: BAR 8: assigned [mem 0x08000000-0x080fffff] >>>>> pci 0000:01:00.0: BAR 0: assigned [mem 0x08000000-0x08007fff 64bit] >>>>> pci 0000:00:00.0: PCI bridge to [bus 01] >>>>> pci 0000:00:00.0: bridge window [mem 0x08000000-0x080fffff] >>>>> >>>>> pcie_iproc_bcma bcma0:8: PCI host bridge to bus 0001:00 >>>>> pci_bus 0001:00: root bus resource [mem 0x40000000-0x47ffffff] >>>>> pcie_iproc_bcma bcma0:8: link: UP >>>>> PCI: bus0: Fast back to back transfers disabled >>>>> pci 0001:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring >>>>> PCI: bus1: Fast back to back transfers disabled >>>>> pci 0001:00:00.0: BAR 8: assigned [mem 0x40000000-0x400fffff] >>>>> pci 0001:01:00.0: BAR 0: assigned [mem 0x40000000-0x40007fff 64bit] >>>>> pci 0001:00:00.0: PCI bridge to [bus 01] >>>>> pci 0001:00:00.0: bridge window [mem 0x40000000-0x400fffff] >>>>> >>>>> >>>>> I guess the check is OK after all and the real problem is iproc driver assigning >>>>> the same resource. >>>>> >>>>> Broadcom team: could you take a look at this, please? >>>> >>>> I found a reason of this conflict (and probably random crashes I started >>>> seeing with 4.9). I believe we have a memory corruption. >>> >>> Yep, we're using a resource structure on the stack when we shouldn't. Can >>> you try the patch below? >> >> Works great! Will you push it with >> Cc: stable@vger.kernel.org # v4.8+ >> please? > > Thanks for testing this. I applied the patch (with the braces fix) to > for-linus for v4.11. Thanks! Btw. this is how it looks now (all good it seems): root@LEDE:/# cat /proc/iomem 00000000-07ffffff : System RAM 00008000-0043cfff : Kernel code 00476000-004d95df : Kernel data 08000000-0fffffff : PCIe MEM space 08000000-085fffff : PCI Bus 0000:01 08000000-083fffff : 0000:01:00.0 08400000-08407fff : 0000:01:00.0 18000300-180003ff : serial 18003000-18003fff : ccb-mii 1800c000-1800cfff : dmu 18021000-18021fff : /axi@18000000/usb2@21000/ehci@21000 18022000-18022fff : /axi@18000000/usb2@21000/ohci@22000 18023000-18023fff : /axi@18000000/usb3@23000/xhci@23000 18028000-180285ff : nand 18028f00-18028f1f : iproc-ext 18105000-18105fff : dmp 1811a408-1811aa07 : iproc-idm 40000000-47ffffff : PCIe MEM space 40000000-405fffff : PCI Bus 0001:01 40000000-403fffff : 0001:01:00.0 40400000-40407fff : 0001:01:00.0 88000000-8fffffff : System RAM
diff --git a/drivers/pci/host/pcie-iproc-bcma.c b/drivers/pci/host/pcie-iproc-bcma.c index bd4c9ec25edc..8cebbbff1e72 100644 --- a/drivers/pci/host/pcie-iproc-bcma.c +++ b/drivers/pci/host/pcie-iproc-bcma.c @@ -44,8 +44,7 @@ static int iproc_pcie_bcma_probe(struct bcma_device *bdev) { struct device *dev = &bdev->dev; struct iproc_pcie *pcie; - LIST_HEAD(res); - struct resource res_mem; + LIST_HEAD(resources); int ret; pcie = devm_kzalloc(dev, sizeof(*pcie), GFP_KERNEL); @@ -63,22 +62,23 @@ static int iproc_pcie_bcma_probe(struct bcma_device *bdev) pcie->base_addr = bdev->addr; - res_mem.start = bdev->addr_s[0]; - res_mem.end = bdev->addr_s[0] + SZ_128M - 1; - res_mem.name = "PCIe MEM space"; - res_mem.flags = IORESOURCE_MEM; - pci_add_resource(&res, &res_mem); + pcie->mem.start = bdev->addr_s[0]; + pcie->mem.end = bdev->addr_s[0] + SZ_128M - 1; + pcie->mem.name = "PCIe MEM space"; + pcie->mem.flags = IORESOURCE_MEM; + pci_add_resource(&resources, &pcie->mem); pcie->map_irq = iproc_pcie_bcma_map_irq; - ret = iproc_pcie_setup(pcie, &res); + ret = iproc_pcie_setup(pcie, &resources); if (ret) dev_err(dev, "PCIe controller setup failed\n"); - - pci_free_resource_list(&res); + pci_free_resource_list(&resources); + return ret; + } bcma_set_drvdata(bdev, pcie); - return ret; + return 0; } static void iproc_pcie_bcma_remove(struct bcma_device *bdev) diff --git a/drivers/pci/host/pcie-iproc-platform.c b/drivers/pci/host/pcie-iproc-platform.c index f4909bb0b2ad..5f6361f27c69 100644 --- a/drivers/pci/host/pcie-iproc-platform.c +++ b/drivers/pci/host/pcie-iproc-platform.c @@ -51,7 +51,7 @@ static int iproc_pcie_pltfm_probe(struct platform_device *pdev) struct device_node *np = dev->of_node; struct resource reg; resource_size_t iobase = 0; - LIST_HEAD(res); + LIST_HEAD(resources); int ret; pcie = devm_kzalloc(dev, sizeof(*pcie), GFP_KERNEL); @@ -96,10 +96,10 @@ static int iproc_pcie_pltfm_probe(struct platform_device *pdev) pcie->phy = NULL; } - ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res, &iobase); + ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &resources, + &iobase); if (ret) { - dev_err(dev, - "unable to get PCI host bridge resources\n"); + dev_err(dev, "unable to get PCI host bridge resources\n"); return ret; } @@ -112,14 +112,15 @@ static int iproc_pcie_pltfm_probe(struct platform_device *pdev) pcie->map_irq = of_irq_parse_and_map_pci; } - ret = iproc_pcie_setup(pcie, &res); + ret = iproc_pcie_setup(pcie, &resources); if (ret) dev_err(dev, "PCIe controller setup failed\n"); - - pci_free_resource_list(&res); + pci_free_resource_list(&resources); + return ret; + } platform_set_drvdata(pdev, pcie); - return ret; + return 0; } static int iproc_pcie_pltfm_remove(struct platform_device *pdev) diff --git a/drivers/pci/host/pcie-iproc.h b/drivers/pci/host/pcie-iproc.h index 04fed8e907f1..0bbe2ea44f3e 100644 --- a/drivers/pci/host/pcie-iproc.h +++ b/drivers/pci/host/pcie-iproc.h @@ -90,6 +90,7 @@ struct iproc_pcie { #ifdef CONFIG_ARM struct pci_sys_data sysdata; #endif + struct resource mem; struct pci_bus *root_bus; struct phy *phy; int (*map_irq)(const struct pci_dev *, u8, u8);