[U-Boot] U-boot hangs on imx6 pci driver

Fabio Estevam wrote:
> On Wed, May 28, 2014 at 4:40 AM, "David Müller (ELSOFT AG)"
> <d.mueller@elsoft.ch> wrote:
>> I use an additional delay in imx6_add_pcie_port() as a workaround so far.
> 
> How much of additional delay? Could you please share your patch?



Dave

On Fri, May 30, 2014 at 12:04 AM, "David Müller (ELSOFT AG)"
<d.mueller@elsoft.ch> wrote:
> Fabio Estevam wrote:
>> On Wed, May 28, 2014 at 4:40 AM, "David Müller (ELSOFT AG)"
>> <d.mueller@elsoft.ch> wrote:
>>> I use an additional delay in imx6_add_pcie_port() as a workaround so far.
>>
>> How much of additional delay? Could you please share your patch?
>
> diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-imx6.c
> index ee08250..1accc0e 100644
> --- a/drivers/pci/host/pci-imx6.c
> +++ b/drivers/pci/host/pci-imx6.c
> @@ -503,6 +532,9 @@ static int imx6_add_pcie_port(struct pcie_port *pp,
>         pp->root_bus_nr = -1;
>         pp->ops = &imx6_pcie_host_ops;
>
> +       /* FIXME  kernel hangs without this delay */
> +       usleep_range(20000, 25000);
> +
>         spin_lock_init(&pp->conf_lock);
>         ret = dw_pcie_host_init(pp);
>         if (ret) {
>
>
> Dave
> _______________________________________________
> U-Boot mailing list
> U-Boot@lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot

David / Marek,

I run into this issue with our Gateworks Ventana IMX6 boards as well.

When enabling PCI support in u-boot my 3.14 kernel hangs somewhere
during PCI init or enumeration (can't tell as uart is not up yet) with
PCI devices behind a PCIe switch. Applying your patch does seem to
work-around the problem on my end as well. Note that I do not see this
issue unless I'm using one of our boards with a PCIe switch and If I
disable PCI support in u-boot this does not occur. This feels like a
PCIE_RST# issue however I do have a PCIE_RST# signal being pinmuxed
and configured properly in my u-boot board support as well as the
kernel.

There seems to be some sort of timing thing that we don't understand here.

David, on your board(s) do you have a PCIe switch as well?

Tim

Tim Harvey wrote:

> When enabling PCI support in u-boot my 3.14 kernel hangs somewhere 
> during PCI init or enumeration (can't tell as uart is not up yet)

Enabling "earlyprintk" support may help.

> David, on your board(s) do you have a PCIe switch as well?

Not yet, but this (using a PEX8603) is planned for the near future.

Dave

On Wed, Jun 4, 2014 at 11:30 PM, "David Müller (ELSOFT AG)"
<d.mueller@elsoft.ch> wrote:
> Tim Harvey wrote:
>
>> When enabling PCI support in u-boot my 3.14 kernel hangs somewhere
>> during PCI init or enumeration (can't tell as uart is not up yet)
>
> Enabling "earlyprintk" support may help.

David,

This is definitely related to PCI_RST# as the delay you inserted is
essentially after imx6_pcie_probe() gets the reset_gpio from OF and
asserts it low. Moving the delay around I find that it must come
before imx6_pcie_assert_core_reset(), specifically before
IMOUX_GPR1:18 is set (phy power down request).

Enabling earlyprintk to try to debug the 'hang' on my boards further I
find that I hang in in imx6_pcie_link_up() during the
readl(pp->dbi_base + PCIE_PHY_DEBUG_R1) which is called after setting
IOMUXC_GPR12:1 to start LTSSM. If I add a udelay(55) (55us determined
via trial and error) directly after setting IOMUXC_GPR12:1 I get
passed 'this' hang however during pci_scan_bridge() I find that
PCI_PRIMARY_BUS config dword comes back as 0x00000000 instead of
0x00010100. This appears to cause the causes the pci code to think the
RC is a bridge, tries to scan behind it, and hangs (because its not a
bridge and those transactions are not valid).

All of this seems to indicate to me that the PLX bridge I have on my
boards requires a longer minimum time to be held in reset 'before' the
PCIe PHY is powered down 'if' it has already been enumerated (or
something to that nature) as I only see this if I enable PCI in uboot.
Why I also need the extra udelay after enabling LTSSM in this scenario
I can't say.

This may correlate with your findings as I believe you say you have an
i210 attached to the IMX6 RC and have found an errata indicating the
i210 needs a longer time in reset. Do you find that this is the case
even if you disable PCI in uboot? My first thought when I read about
that i210 errata you pointed out was that it wasn't an issue because
we do hold reset low for 100ms in the kernel but if the issue has
something to do with holding it in reset with the PHY being powered
down then perhaps this explains things.

Merek,

you've done much more work on IMX6 link than I... any of this make
sense to you? I believe you have never encountered this 100%
repeatable issue on your board(s) that David and I do encounter, but
you have encountered a 1% PCIe link failure rate (which I'm inclined
to say is something completely different?).

Regards,

Tim

>
>> David, on your board(s) do you have a PCIe switch as well?
>
> Not yet, but this (using a PEX8603) is planned for the near future.
>
> Dave
>

On Thursday, June 05, 2014 at 12:13:39 PM, Tim Harvey wrote:
> On Wed, Jun 4, 2014 at 11:30 PM, "David Müller (ELSOFT AG)"
> 
> <d.mueller@elsoft.ch> wrote:
> > Tim Harvey wrote:
> >> When enabling PCI support in u-boot my 3.14 kernel hangs somewhere
> >> during PCI init or enumeration (can't tell as uart is not up yet)
> > 
> > Enabling "earlyprintk" support may help.
> 
> David,
> 
> This is definitely related to PCI_RST# as the delay you inserted is
> essentially after imx6_pcie_probe() gets the reset_gpio from OF and
> asserts it low. Moving the delay around I find that it must come
> before imx6_pcie_assert_core_reset(), specifically before
> IMOUX_GPR1:18 is set (phy power down request).
> 
> Enabling earlyprintk to try to debug the 'hang' on my boards further I
> find that I hang in in imx6_pcie_link_up() during the
> readl(pp->dbi_base + PCIE_PHY_DEBUG_R1) which is called after setting
> IOMUXC_GPR12:1 to start LTSSM. If I add a udelay(55) (55us determined
> via trial and error) directly after setting IOMUXC_GPR12:1 I get
> passed 'this' hang however during pci_scan_bridge() I find that
> PCI_PRIMARY_BUS config dword comes back as 0x00000000 instead of
> 0x00010100. This appears to cause the causes the pci code to think the
> RC is a bridge, tries to scan behind it, and hangs (because its not a
> bridge and those transactions are not valid).
> 
> All of this seems to indicate to me that the PLX bridge I have on my
> boards requires a longer minimum time to be held in reset 'before' the
> PCIe PHY is powered down 'if' it has already been enumerated (or
> something to that nature) as I only see this if I enable PCI in uboot.
> Why I also need the extra udelay after enabling LTSSM in this scenario
> I can't say.
> 
> This may correlate with your findings as I believe you say you have an
> i210 attached to the IMX6 RC and have found an errata indicating the
> i210 needs a longer time in reset. Do you find that this is the case
> even if you disable PCI in uboot? My first thought when I read about
> that i210 errata you pointed out was that it wasn't an issue because
> we do hold reset low for 100ms in the kernel but if the issue has
> something to do with holding it in reset with the PHY being powered
> down then perhaps this explains things.
> 
> Merek,
> 
> you've done much more work on IMX6 link than I... any of this make
> sense to you? I believe you have never encountered this 100%
> repeatable issue on your board(s) that David and I do encounter, but
> you have encountered a 1% PCIe link failure rate (which I'm inclined
> to say is something completely different?).

Sorry for the later reply. Most certainly, I observe a failure rate of 1-out-
of-200 or so. I also use i210 though, so it might be related somehow. Can you 
tell me which i210 errata are you talking about here please?

I am currently discussing this with FSL, but it didn't yield any results yet.

Best regards,
Marek Vasut