diff mbox

[U-Boot] U-boot hangs on imx6 pci driver

Message ID CAOMZO5DU_AbK59WuU8kypC4aoq0KmCN9wa7oFMOHRyXaNEUWOQ@mail.gmail.com
State Not Applicable
Delegated to: Stefano Babic
Headers show

Commit Message

Fabio Estevam May 27, 2014, 12:30 p.m. UTC
Hi,

Working on a mx6solo board with PCIe driver enabled in U-boot, I
notice that after doing several reboots a hang is seen on the PCIe
driver:

PCI Autoconfig: Bus Memory region: [0x1100000-0x1efffff],
                Physical Memory [1100000-1efffffx]
PCI Autoconfig: Bus I/O region: [0x1000000-0x10fffff],
                Physical Memory: [1000000-10fffff]
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc00c
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc000
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc000
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc008
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
  00:01.0     - 16c3:abcd - Bridge device
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc008
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc004
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc010
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
PCI Autoconfig: BAR 0, I/O, size=0xfff4, address=0x1000000
bus_lower=0x100fff4### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config

### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc014
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc004
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc00c
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc00c
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
PCI Autoconfig: Found P2P bridge, device 1
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc004
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc018
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc018
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc018
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc020
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc024
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc024
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc01c
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc030
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc004
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1f0000c

(Hang)

As a quick workaround I tried the following:

the PCI switch can be discovered.

Any suggestions?

Thanks,

Fabio Estevam

Comments

Marek Vasut May 27, 2014, 1:25 p.m. UTC | #1
On Tuesday, May 27, 2014 at 02:30:27 PM, Fabio Estevam wrote:
> Hi,
> 
> Working on a mx6solo board with PCIe driver enabled in U-boot, I
> notice that after doing several reboots a hang is seen on the PCIe
> driver:

[...]

> Any suggestions?

Take a look at SR# 1-1347946851 in the FSL internal bug tracker. It looks like 
the PCIe IP core implementation in the MX6 is bugged in my opinion.

Best regards,
Marek Vasut
David Müller (ELSOFT AG) May 27, 2014, 2:43 p.m. UTC | #2
Marek Vasut wrote:
> Take a look at SR# 1-1347946851 in the FSL internal bug tracker. It
> looks like the PCIe IP core implementation in the MX6 is bugged in my
> opinion.

Are there any publicly available info regarding this SR?

Dave
Marek Vasut May 27, 2014, 2:56 p.m. UTC | #3
On Tuesday, May 27, 2014 at 04:43:08 PM, David Müller (ELSOFT AG) wrote:
> Marek Vasut wrote:
> > Take a look at SR# 1-1347946851 in the FSL internal bug tracker. It
> > looks like the PCIe IP core implementation in the MX6 is bugged in my
> > opinion.
> 
> Are there any publicly available info regarding this SR?

Scrubbed the irrelevant bits:

-->8--
Please find a defect report below for the i.MX6DL PCI express driver in current 
Linux 3.10.17-1.0.0-ga released by Freescale:

Priority: High

Issue Type: Defect Report
Problem Category: Freescale Software
Problem Domain: Serial Connectivity

Project Name: Freescale SabreSDP
Industry Segment: Industrial

Target Processor: i.MX6DL Processors
Target HW Platform: i.MX6DL SabreSDP
Target OS: Linux
Target Software Package: Linux imx_3.10.17_1.0.0_ga

PC Host System O/S, Version: Not Applicable in this case

Reproducibility: Rarely
Steps to Reproduce:
  Prep:
  1) Take MX6DL SabreSDP platform
  2) Attach Intel i210 PCI express card
  3) Install BSP with kernel 3.10.17-1.0.0-ga onto SD card
  4) Boot the platform and confirm the i210 is recognized

  Test:
  1) Power on the MX6DL SabreSDP
  2) Boot the kernel
  3) Verify the SabreSDP recognized the i210
  4) When the platform reaches the init process, trigger a software-reboot
  5) The software-reboot will put the platform back into U-Boot, so the SabreSDP
     will again continue from step 2) by booting the kernel automatically.

  Test steps 2)...5) must be performed at least 10000 times.

Expected Results:
  10000 of 10000 times, the PCI express i210 card will be recognized by the
  platform. The PCI express link will always reliably come up.

Observed Results:
  After some hundreds of the soft-reboot cycles, the i210 is not recognized. 
  This is caused by the PCI express link failing to come up. We see "link never 
  came up" message in the kernel log.

Description:
  We perform the test above -- soft-restarting the MX6DL SabreSDP platform with 
  Linux 3.10.17-1.0.0-ga in quick sequences. We would expect for the PCIe link 
  to reliably come up in all of the 10000 cycles, but in some of those cycles, 
  the link fails to come up.

  Can you please confirm/replicate the issue and provide us with a fix ?
--8<--

Best regards,
Marek Vasut
David Müller (ELSOFT AG) May 28, 2014, 7:40 a.m. UTC | #4
Marek Vasut wrote:
> Observed Results: After some hundreds of the soft-reboot cycles, the
> i210 is not recognized. This is caused by the PCI express link
> failing to come up. We see "link never came up" message in the kernel
> log.

Just a guest but maybe this is "errata #18" of the i210.

A little bit off-topic but i'm facing the problem that Linux (kernel
3.14 + some patches) hangs regularly during PCIe initialisation on our
custom iMX6 / i210 board.

I use an additional delay in imx6_add_pcie_port() as a workaround so far.


Dave
Fabio Estevam May 28, 2014, 4:42 p.m. UTC | #5
On Tue, May 27, 2014 at 9:30 AM, Fabio Estevam <festevam@gmail.com> wrote:
> Hi,
>
> Working on a mx6solo board with PCIe driver enabled in U-boot, I
> notice that after doing several reboots a hang is seen on the PCIe
> driver:

Just an update: we are able now to run overnight tests without PCI hangs.

Sorry for the noise, but the hang was caused by a debug code.

We still get "link never came up" issue on some boots, but this is a
different one.
Fabio Estevam May 28, 2014, 4:43 p.m. UTC | #6
Hi David,

On Wed, May 28, 2014 at 4:40 AM, "David Müller (ELSOFT AG)"
<d.mueller@elsoft.ch> wrote:
> Marek Vasut wrote:
>> Observed Results: After some hundreds of the soft-reboot cycles, the
>> i210 is not recognized. This is caused by the PCI express link
>> failing to come up. We see "link never came up" message in the kernel
>> log.
>
> Just a guest but maybe this is "errata #18" of the i210.
>
> A little bit off-topic but i'm facing the problem that Linux (kernel
> 3.14 + some patches) hangs regularly during PCIe initialisation on our
> custom iMX6 / i210 board.
>
> I use an additional delay in imx6_add_pcie_port() as a workaround so far.

How much of additional delay? Could you please share your patch?
Marek Vasut May 28, 2014, 6:31 p.m. UTC | #7
On Wednesday, May 28, 2014 at 06:42:41 PM, Fabio Estevam wrote:
> On Tue, May 27, 2014 at 9:30 AM, Fabio Estevam <festevam@gmail.com> wrote:
> > Hi,
> > 
> > Working on a mx6solo board with PCIe driver enabled in U-boot, I
> > notice that after doing several reboots a hang is seen on the PCIe
> 
> > driver:
> Just an update: we are able now to run overnight tests without PCI hangs.
> 
> Sorry for the noise, but the hang was caused by a debug code.
> 
> We still get "link never came up" issue on some boots, but this is a
> different one.

Does that mean FSL is unable to reliably bring PCIe link up on every boot on 
their platform please?

Best regards,
Marek Vasut
diff mbox

Patch

--- a/drivers/pci/pcie_imx.c
+++ b/drivers/pci/pcie_imx.c
@@ -385,7 +385,8 @@  static int imx_pcie_read_config(struct pci_controller *hose,
         */
        imx_pcie_fix_dabt_handler(true);
        writel(0xffffffff, val);
-       *val = readl(va_address);
+       if (0x01ffc000 <= va_address && va_address <= 0x01ffffff)
+               *val = readl(va_address);
        imx_pcie_fix_dabt_handler(false);

and the hang does not happen. However, not all devices connected to