| Message ID | 1590023130-137406-1-git-send-email-shawn.lin@rock-chips.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/2] PCI: rockchip: Enable IO base and limit registers | expand |
On Thu, 21 May 2020 at 06:35, Shawn Lin <shawn.lin@rock-chips.com> wrote: > > According to RK3399 user manual, bit 9 in PCIE_RC_BAR_CONF should > be set, otherwise accessing to IO base and limit registers would > fail. > > [ 0.411318] pci_bus 0000:00: root bus resource [bus 00-1f] > [ 0.411822] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff] > [ 0.412440] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) > [ 0.413665] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring > [ 0.414698] pci 0000:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid > [ 0.415412] pci 0000:01:00.0: reg 0x18: initial BAR value 0x00000000 invalid > [ 0.418456] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] > [ 0.419065] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa000000-0xfa007fff pref] > [ 0.419728] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa008000-0xfa00ffff pref] > [ 0.420377] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] > [ 0.420935] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] > [ 0.421526] pci 0000:01:00.0: BAR 2: no space for [io size 0x0004] > [ 0.422084] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0004] > [ 0.422687] pci 0000:00:00.0: PCI bridge to [bus 01] > [ 0.423135] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] > [ 0.423794] pcieport 0000:00:00.0: enabling device (0000 -> 0002) > [ 0.424566] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt > [ 0.425182] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt > > 01:00.0 Class 0700: Device 1c00:3853 (rev 10) (prog-if 05) > Subsystem: Device 1c00:3853 > Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Interrupt: pin A routed to IRQ 230 > Region 0: I/O ports at <unassigned> [disabled] > Region 1: Memory at fa000000 (32-bit, prefetchable) [disabled] [size=32K] > Region 2: I/O ports at <unassigned> [disabled] > [virtual] Expansion ROM at fa008000 [disabled] [size=32K] > > Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> > --- > > drivers/pci/controller/pcie-rockchip.c | 7 +++++-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/drivers/pci/controller/pcie-rockchip.c b/drivers/pci/controller/pcie-rockchip.c > index c53d132..f82452b 100644 > --- a/drivers/pci/controller/pcie-rockchip.c > +++ b/drivers/pci/controller/pcie-rockchip.c > @@ -407,8 +407,11 @@ void rockchip_pcie_cfg_configuration_accesses( > { > u32 ob_desc_0; > > - /* Configuration Accesses for region 0 */ > - rockchip_pcie_write(rockchip, 0x0, PCIE_RC_BAR_CONF); > + /* > + * Configuration Accesses for region 0. > + * Bit 9 is for enabling IO base and limit registers. > + */ > + rockchip_pcie_write(rockchip, BIT(9), PCIE_RC_BAR_CONF); > > rockchip_pcie_write(rockchip, > (RC_REGION_0_ADDR_TRANS_L + RC_REGION_0_PASS_BITS), > -- > 2.7.4 > > > > > _______________________________________________ > Linux-rockchip mailing list > Linux-rockchip@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-rockchip
Hi Shawn, On Thu, 21 May 2020 at 06:35, Shawn Lin <shawn.lin@rock-chips.com> wrote: > > According to RK3399 user manual, bit 9 in PCIE_RC_BAR_CONF should > be set, otherwise accessing to IO base and limit registers would > fail. > > [ 0.411318] pci_bus 0000:00: root bus resource [bus 00-1f] > [ 0.411822] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff] > [ 0.412440] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) > [ 0.413665] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring > [ 0.414698] pci 0000:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid > [ 0.415412] pci 0000:01:00.0: reg 0x18: initial BAR value 0x00000000 invalid > [ 0.418456] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] > [ 0.419065] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa000000-0xfa007fff pref] > [ 0.419728] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa008000-0xfa00ffff pref] > [ 0.420377] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] > [ 0.420935] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] > [ 0.421526] pci 0000:01:00.0: BAR 2: no space for [io size 0x0004] > [ 0.422084] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0004] > [ 0.422687] pci 0000:00:00.0: PCI bridge to [bus 01] > [ 0.423135] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] > [ 0.423794] pcieport 0000:00:00.0: enabling device (0000 -> 0002) > [ 0.424566] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt > [ 0.425182] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt > > 01:00.0 Class 0700: Device 1c00:3853 (rev 10) (prog-if 05) > Subsystem: Device 1c00:3853 > Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > Interrupt: pin A routed to IRQ 230 > Region 0: I/O ports at <unassigned> [disabled] > Region 1: Memory at fa000000 (32-bit, prefetchable) [disabled] [size=32K] > Region 2: I/O ports at <unassigned> [disabled] > [virtual] Expansion ROM at fa008000 [disabled] [size=32K] > > Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> > --- I have old development board Odroid N1 (RK3399), It has onboard PCIe 2 dual sata bridge. I have tested this patch, but I am still getting following log on Odroid N1 board. Is their any more configuration needed for sata ports ? [ 7.444504] pci_bus 0000:01: busn_res: [bus 01-1f] end is updated to 01 [ 7.445521] panfrost ff9a0000.gpu: Features: L2:0x07120206 Shader:0x00000000 Tiler:0x00000809 Mem:0x1 MMU:0x00002830 AS:0xff JS:0x7 [ 7.452246] pci 0000:00:00.0: BAR 14: assigned [mem 0xfa000000-0xfa0fffff] [ 7.460106] panfrost ff9a0000.gpu: shader_present=0xf l2_present=0x1 [ 7.466459] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa000000-0xfa00ffff pref] [ 7.473679] panfrost ff9a0000.gpu: [drm:panfrost_devfreq_init [panfrost]] Failed to register cooling device [ 7.479703] pci 0000:01:00.0: BAR 5: assigned [mem 0xfa010000-0xfa0101ff] [ 7.487706] [drm] Initialized panfrost 1.1.0 20180908 for ff9a0000.gpu on minor 0 [ 7.494343] pci 0000:01:00.0: BAR 4: no space for [io size 0x0010] [ 7.494348] pci 0000:01:00.0: BAR 4: failed to assign [io size 0x0010] [ 7.494352] pci 0000:01:00.0: BAR 0: no space for [io size 0x0008] [ 7.494356] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0008] [ 7.494360] pci 0000:01:00.0: BAR 2: no space for [io size 0x0008] [ 7.494364] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0008] [ 7.494368] pci 0000:01:00.0: BAR 1: no space for [io size 0x0004] [ 7.494372] pci 0000:01:00.0: BAR 1: failed to assign [io size 0x0004] [ 7.578910] rockchip-vop ff8f0000.vop: Adding to iommu group 3 [ 7.587074] pci 0000:01:00.0: BAR 3: no space for [io size 0x0004] [ 7.594780] rockchip-vop ff900000.vop: Adding to iommu group 4 [ 7.607701] pci 0000:01:00.0: BAR 3: failed to assign [io size 0x0004] # lspci -v 00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd RK3399 PCI Express Root Port (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0, IRQ 237 Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 00000000-00000fff [size=4K] Memory behind bridge: fa000000-fa0fffff [size=1M] Prefetchable memory behind bridge: 00000000-000fffff [size=1M] Capabilities: [80] Power Management version 3 Capabilities: [90] MSI: Enable+ Count=1/1 Maskable+ 64bit+ Capabilities: [b0] MSI-X: Enable- Count=1 Masked- Capabilities: [c0] Express Root Port (Slot+), MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [274] Transaction Processing Hints Kernel driver in use: pcieport 01:00.0 IDE interface: ASMedia Technology Inc. ASM1061 SATA IDE Controller (rev 02) (prog-if 85 [PCI native mode-only controller, supports bus mastering]) Subsystem: ASMedia Technology Inc. ASM1061 SATA IDE Controller Flags: bus master, fast devsel, latency 0, IRQ 238 I/O ports at <unassigned> [disabled] I/O ports at <unassigned> [disabled] I/O ports at <unassigned> [disabled] I/O ports at <unassigned> [disabled] I/O ports at <unassigned> [disabled] Memory at fa010000 (32-bit, non-prefetchable) [size=512] Expansion ROM at fa000000 [virtual] [disabled] [size=64K] Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit- Capabilities: [78] Power Management version 3 Capabilities: [80] Express Legacy Endpoint, MSI 00 Capabilities: [100] Virtual Channel Kernel driver in use: ahci Best Regards -Anand
在 2020/5/21 18:51, Anand Moon 写道: > Hi Shawn, > > On Thu, 21 May 2020 at 06:35, Shawn Lin <shawn.lin@rock-chips.com> wrote: >> >> According to RK3399 user manual, bit 9 in PCIE_RC_BAR_CONF should >> be set, otherwise accessing to IO base and limit registers would >> fail. >> >> [ 0.411318] pci_bus 0000:00: root bus resource [bus 00-1f] >> [ 0.411822] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff] >> [ 0.412440] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) >> [ 0.413665] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring >> [ 0.414698] pci 0000:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid >> [ 0.415412] pci 0000:01:00.0: reg 0x18: initial BAR value 0x00000000 invalid >> [ 0.418456] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] >> [ 0.419065] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa000000-0xfa007fff pref] >> [ 0.419728] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa008000-0xfa00ffff pref] >> [ 0.420377] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] >> [ 0.420935] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] >> [ 0.421526] pci 0000:01:00.0: BAR 2: no space for [io size 0x0004] >> [ 0.422084] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0004] >> [ 0.422687] pci 0000:00:00.0: PCI bridge to [bus 01] >> [ 0.423135] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] >> [ 0.423794] pcieport 0000:00:00.0: enabling device (0000 -> 0002) >> [ 0.424566] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt >> [ 0.425182] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt >> >> 01:00.0 Class 0700: Device 1c00:3853 (rev 10) (prog-if 05) >> Subsystem: Device 1c00:3853 >> Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- >> Interrupt: pin A routed to IRQ 230 >> Region 0: I/O ports at <unassigned> [disabled] >> Region 1: Memory at fa000000 (32-bit, prefetchable) [disabled] [size=32K] >> Region 2: I/O ports at <unassigned> [disabled] >> [virtual] Expansion ROM at fa008000 [disabled] [size=32K] >> >> Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> >> --- > > I have old development board Odroid N1 (RK3399), It has onboard PCIe > 2 dual sata bridge. > I have tested this patch, but I am still getting following log on > Odroid N1 board. > Is their any more configuration needed for sata ports ? Thanks for testing. I made a mistake that it should be bit 19, so can you try using BIT(19)? > > [ 7.444504] pci_bus 0000:01: busn_res: [bus 01-1f] end is updated to 01 > [ 7.445521] panfrost ff9a0000.gpu: Features: L2:0x07120206 > Shader:0x00000000 Tiler:0x00000809 Mem:0x1 MMU:0x00002830 AS:0xff > JS:0x7 > [ 7.452246] pci 0000:00:00.0: BAR 14: assigned [mem 0xfa000000-0xfa0fffff] > [ 7.460106] panfrost ff9a0000.gpu: shader_present=0xf l2_present=0x1 > [ 7.466459] pci 0000:01:00.0: BAR 6: assigned [mem > 0xfa000000-0xfa00ffff pref] > [ 7.473679] panfrost ff9a0000.gpu: [drm:panfrost_devfreq_init > [panfrost]] Failed to register cooling device > [ 7.479703] pci 0000:01:00.0: BAR 5: assigned [mem 0xfa010000-0xfa0101ff] > [ 7.487706] [drm] Initialized panfrost 1.1.0 20180908 for > ff9a0000.gpu on minor 0 > [ 7.494343] pci 0000:01:00.0: BAR 4: no space for [io size 0x0010] > [ 7.494348] pci 0000:01:00.0: BAR 4: failed to assign [io size 0x0010] > [ 7.494352] pci 0000:01:00.0: BAR 0: no space for [io size 0x0008] > [ 7.494356] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0008] > [ 7.494360] pci 0000:01:00.0: BAR 2: no space for [io size 0x0008] > [ 7.494364] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0008] > [ 7.494368] pci 0000:01:00.0: BAR 1: no space for [io size 0x0004] > [ 7.494372] pci 0000:01:00.0: BAR 1: failed to assign [io size 0x0004] > [ 7.578910] rockchip-vop ff8f0000.vop: Adding to iommu group 3 > [ 7.587074] pci 0000:01:00.0: BAR 3: no space for [io size 0x0004] > [ 7.594780] rockchip-vop ff900000.vop: Adding to iommu group 4 > [ 7.607701] pci 0000:01:00.0: BAR 3: failed to assign [io size 0x0004] > > # lspci -v > 00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd RK3399 PCI > Express Root Port (prog-if 00 [Normal decode]) > Flags: bus master, fast devsel, latency 0, IRQ 237 > Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 > I/O behind bridge: 00000000-00000fff [size=4K] > Memory behind bridge: fa000000-fa0fffff [size=1M] > Prefetchable memory behind bridge: 00000000-000fffff [size=1M] > Capabilities: [80] Power Management version 3 > Capabilities: [90] MSI: Enable+ Count=1/1 Maskable+ 64bit+ > Capabilities: [b0] MSI-X: Enable- Count=1 Masked- > Capabilities: [c0] Express Root Port (Slot+), MSI 00 > Capabilities: [100] Advanced Error Reporting > Capabilities: [274] Transaction Processing Hints > Kernel driver in use: pcieport > > 01:00.0 IDE interface: ASMedia Technology Inc. ASM1061 SATA IDE > Controller (rev 02) (prog-if 85 [PCI native mode-only controller, > supports bus mastering]) > Subsystem: ASMedia Technology Inc. ASM1061 SATA IDE Controller > Flags: bus master, fast devsel, latency 0, IRQ 238 > I/O ports at <unassigned> [disabled] > I/O ports at <unassigned> [disabled] > I/O ports at <unassigned> [disabled] > I/O ports at <unassigned> [disabled] > I/O ports at <unassigned> [disabled] > Memory at fa010000 (32-bit, non-prefetchable) [size=512] > Expansion ROM at fa000000 [virtual] [disabled] [size=64K] > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit- > Capabilities: [78] Power Management version 3 > Capabilities: [80] Express Legacy Endpoint, MSI 00 > Capabilities: [100] Virtual Channel > Kernel driver in use: ahci > > Best Regards > -Anand > > _______________________________________________ > Linux-rockchip mailing list > Linux-rockchip@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-rockchip > > >
Hi Shawn On Fri, 22 May 2020 at 08:30, Shawn Lin <shawn.lin@rock-chips.com> wrote: > > > 在 2020/5/21 18:51, Anand Moon 写道: > > Hi Shawn, > > > > On Thu, 21 May 2020 at 06:35, Shawn Lin <shawn.lin@rock-chips.com> wrote: > >> > >> According to RK3399 user manual, bit 9 in PCIE_RC_BAR_CONF should > >> be set, otherwise accessing to IO base and limit registers would > >> fail. > >> > >> [ 0.411318] pci_bus 0000:00: root bus resource [bus 00-1f] > >> [ 0.411822] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff] > >> [ 0.412440] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) > >> [ 0.413665] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring > >> [ 0.414698] pci 0000:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid > >> [ 0.415412] pci 0000:01:00.0: reg 0x18: initial BAR value 0x00000000 invalid > >> [ 0.418456] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] > >> [ 0.419065] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa000000-0xfa007fff pref] > >> [ 0.419728] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa008000-0xfa00ffff pref] > >> [ 0.420377] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] > >> [ 0.420935] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] > >> [ 0.421526] pci 0000:01:00.0: BAR 2: no space for [io size 0x0004] > >> [ 0.422084] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0004] > >> [ 0.422687] pci 0000:00:00.0: PCI bridge to [bus 01] > >> [ 0.423135] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] > >> [ 0.423794] pcieport 0000:00:00.0: enabling device (0000 -> 0002) > >> [ 0.424566] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt > >> [ 0.425182] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt > >> > >> 01:00.0 Class 0700: Device 1c00:3853 (rev 10) (prog-if 05) > >> Subsystem: Device 1c00:3853 > >> Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > >> Interrupt: pin A routed to IRQ 230 > >> Region 0: I/O ports at <unassigned> [disabled] > >> Region 1: Memory at fa000000 (32-bit, prefetchable) [disabled] [size=32K] > >> Region 2: I/O ports at <unassigned> [disabled] > >> [virtual] Expansion ROM at fa008000 [disabled] [size=32K] > >> > >> Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> > >> --- > > > > I have old development board Odroid N1 (RK3399), It has onboard PCIe > > 2 dual sata bridge. > > I have tested this patch, but I am still getting following log on > > Odroid N1 board. > > Is their any more configuration needed for sata ports ? > > Thanks for testing. I made a mistake that it should be bit 19, so > can you try using BIT(19)? > Nop enable this bit dose not solve the issue see at my end. But as per RK3399 TMR 17.6.7.1.45 Root Complex BAR Configuration Register their are many bits that are not tuned correctly. I tried to set some bit to BAR Configuration register. but it dose not work at my end. I feel some more core configuration is missing. If I have some update I will share it with you. -Anand
On Fri, May 22, 2020 at 05:59:14PM +0530, Anand Moon wrote: > Hi Shawn > > On Fri, 22 May 2020 at 08:30, Shawn Lin <shawn.lin@rock-chips.com> wrote: > > > > > > 在 2020/5/21 18:51, Anand Moon 写道: > > > Hi Shawn, > > > > > > On Thu, 21 May 2020 at 06:35, Shawn Lin <shawn.lin@rock-chips.com> wrote: > > >> > > >> According to RK3399 user manual, bit 9 in PCIE_RC_BAR_CONF should > > >> be set, otherwise accessing to IO base and limit registers would > > >> fail. > > >> > > >> [ 0.411318] pci_bus 0000:00: root bus resource [bus 00-1f] > > >> [ 0.411822] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff] > > >> [ 0.412440] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) > > >> [ 0.413665] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring > > >> [ 0.414698] pci 0000:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid > > >> [ 0.415412] pci 0000:01:00.0: reg 0x18: initial BAR value 0x00000000 invalid > > >> [ 0.418456] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] > > >> [ 0.419065] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa000000-0xfa007fff pref] > > >> [ 0.419728] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa008000-0xfa00ffff pref] > > >> [ 0.420377] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] > > >> [ 0.420935] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] > > >> [ 0.421526] pci 0000:01:00.0: BAR 2: no space for [io size 0x0004] > > >> [ 0.422084] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0004] > > >> [ 0.422687] pci 0000:00:00.0: PCI bridge to [bus 01] > > >> [ 0.423135] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] > > >> [ 0.423794] pcieport 0000:00:00.0: enabling device (0000 -> 0002) > > >> [ 0.424566] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt > > >> [ 0.425182] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt > > >> > > >> 01:00.0 Class 0700: Device 1c00:3853 (rev 10) (prog-if 05) > > >> Subsystem: Device 1c00:3853 > > >> Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > > >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > > >> Interrupt: pin A routed to IRQ 230 > > >> Region 0: I/O ports at <unassigned> [disabled] > > >> Region 1: Memory at fa000000 (32-bit, prefetchable) [disabled] [size=32K] > > >> Region 2: I/O ports at <unassigned> [disabled] > > >> [virtual] Expansion ROM at fa008000 [disabled] [size=32K] > > >> > > >> Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> > > >> --- > > > > > > I have old development board Odroid N1 (RK3399), It has onboard PCIe > > > 2 dual sata bridge. > > > I have tested this patch, but I am still getting following log on > > > Odroid N1 board. > > > Is their any more configuration needed for sata ports ? > > > > Thanks for testing. I made a mistake that it should be bit 19, so > > can you try using BIT(19)? > > > > Nop enable this bit dose not solve the issue see at my end. > > But as per RK3399 TMR 17.6.7.1.45 Root Complex BAR Configuration Register > their are many bits that are not tuned correctly. > I tried to set some bit to BAR Configuration register. but it dose not > work at my end. > I feel some more core configuration is missing. > If I have some update I will share it with you. What's the status of this discussion and therefore this series ? Thanks, Lorenzo
hi Lorenzo, On Wed, 8 Jul 2020 at 20:31, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote: > > On Fri, May 22, 2020 at 05:59:14PM +0530, Anand Moon wrote: > > Hi Shawn > > > > On Fri, 22 May 2020 at 08:30, Shawn Lin <shawn.lin@rock-chips.com> wrote: > > > > > > > > > 在 2020/5/21 18:51, Anand Moon 写道: > > > > Hi Shawn, > > > > > > > > On Thu, 21 May 2020 at 06:35, Shawn Lin <shawn.lin@rock-chips.com> wrote: > > > >> > > > >> According to RK3399 user manual, bit 9 in PCIE_RC_BAR_CONF should > > > >> be set, otherwise accessing to IO base and limit registers would > > > >> fail. > > > >> > > > >> [ 0.411318] pci_bus 0000:00: root bus resource [bus 00-1f] > > > >> [ 0.411822] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff] > > > >> [ 0.412440] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) > > > >> [ 0.413665] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring > > > >> [ 0.414698] pci 0000:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid > > > >> [ 0.415412] pci 0000:01:00.0: reg 0x18: initial BAR value 0x00000000 invalid > > > >> [ 0.418456] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] > > > >> [ 0.419065] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa000000-0xfa007fff pref] > > > >> [ 0.419728] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa008000-0xfa00ffff pref] > > > >> [ 0.420377] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] > > > >> [ 0.420935] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] > > > >> [ 0.421526] pci 0000:01:00.0: BAR 2: no space for [io size 0x0004] > > > >> [ 0.422084] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0004] > > > >> [ 0.422687] pci 0000:00:00.0: PCI bridge to [bus 01] > > > >> [ 0.423135] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] > > > >> [ 0.423794] pcieport 0000:00:00.0: enabling device (0000 -> 0002) > > > >> [ 0.424566] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt > > > >> [ 0.425182] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt > > > >> > > > >> 01:00.0 Class 0700: Device 1c00:3853 (rev 10) (prog-if 05) > > > >> Subsystem: Device 1c00:3853 > > > >> Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > > > >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > > > >> Interrupt: pin A routed to IRQ 230 > > > >> Region 0: I/O ports at <unassigned> [disabled] > > > >> Region 1: Memory at fa000000 (32-bit, prefetchable) [disabled] [size=32K] > > > >> Region 2: I/O ports at <unassigned> [disabled] > > > >> [virtual] Expansion ROM at fa008000 [disabled] [size=32K] > > > >> > > > >> Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> > > > >> --- > > > > > > > > I have old development board Odroid N1 (RK3399), It has onboard PCIe > > > > 2 dual sata bridge. > > > > I have tested this patch, but I am still getting following log on > > > > Odroid N1 board. > > > > Is their any more configuration needed for sata ports ? > > > > > > Thanks for testing. I made a mistake that it should be bit 19, so > > > can you try using BIT(19)? > > > > > > > Nop enable this bit dose not solve the issue see at my end. > > > > But as per RK3399 TMR 17.6.7.1.45 Root Complex BAR Configuration Register > > their are many bits that are not tuned correctly. > > I tried to set some bit to BAR Configuration register. but it dose not > > work at my end. > > I feel some more core configuration is missing. > > If I have some update I will share it with you. > > What's the status of this discussion and therefore this series ? > > Thanks, > Lorenzo Well I have looked into the RK3399 TRM (Rockchip RK3399 TRM V1.3 Part2.pdf) There seems to be some core configuration missing, but I could not resolve this on my board. Best Regards -Anand
On Thu, Jul 09, 2020 at 09:18:27AM +0530, Anand Moon wrote: > hi Lorenzo, > > On Wed, 8 Jul 2020 at 20:31, Lorenzo Pieralisi > <lorenzo.pieralisi@arm.com> wrote: > > > > On Fri, May 22, 2020 at 05:59:14PM +0530, Anand Moon wrote: > > > Hi Shawn > > > > > > On Fri, 22 May 2020 at 08:30, Shawn Lin <shawn.lin@rock-chips.com> wrote: > > > > > > > > > > > > 在 2020/5/21 18:51, Anand Moon 写道: > > > > > Hi Shawn, > > > > > > > > > > On Thu, 21 May 2020 at 06:35, Shawn Lin <shawn.lin@rock-chips.com> wrote: > > > > >> > > > > >> According to RK3399 user manual, bit 9 in PCIE_RC_BAR_CONF should > > > > >> be set, otherwise accessing to IO base and limit registers would > > > > >> fail. > > > > >> > > > > >> [ 0.411318] pci_bus 0000:00: root bus resource [bus 00-1f] > > > > >> [ 0.411822] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff] > > > > >> [ 0.412440] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) > > > > >> [ 0.413665] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring > > > > >> [ 0.414698] pci 0000:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid > > > > >> [ 0.415412] pci 0000:01:00.0: reg 0x18: initial BAR value 0x00000000 invalid > > > > >> [ 0.418456] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] > > > > >> [ 0.419065] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa000000-0xfa007fff pref] > > > > >> [ 0.419728] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa008000-0xfa00ffff pref] > > > > >> [ 0.420377] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] > > > > >> [ 0.420935] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] > > > > >> [ 0.421526] pci 0000:01:00.0: BAR 2: no space for [io size 0x0004] > > > > >> [ 0.422084] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0004] > > > > >> [ 0.422687] pci 0000:00:00.0: PCI bridge to [bus 01] > > > > >> [ 0.423135] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] > > > > >> [ 0.423794] pcieport 0000:00:00.0: enabling device (0000 -> 0002) > > > > >> [ 0.424566] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt > > > > >> [ 0.425182] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt > > > > >> > > > > >> 01:00.0 Class 0700: Device 1c00:3853 (rev 10) (prog-if 05) > > > > >> Subsystem: Device 1c00:3853 > > > > >> Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- > > > > >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- > > > > >> Interrupt: pin A routed to IRQ 230 > > > > >> Region 0: I/O ports at <unassigned> [disabled] > > > > >> Region 1: Memory at fa000000 (32-bit, prefetchable) [disabled] [size=32K] > > > > >> Region 2: I/O ports at <unassigned> [disabled] > > > > >> [virtual] Expansion ROM at fa008000 [disabled] [size=32K] > > > > >> > > > > >> Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> > > > > >> --- > > > > > > > > > > I have old development board Odroid N1 (RK3399), It has onboard PCIe > > > > > 2 dual sata bridge. > > > > > I have tested this patch, but I am still getting following log on > > > > > Odroid N1 board. > > > > > Is their any more configuration needed for sata ports ? > > > > > > > > Thanks for testing. I made a mistake that it should be bit 19, so > > > > can you try using BIT(19)? > > > > > > > > > > Nop enable this bit dose not solve the issue see at my end. > > > > > > But as per RK3399 TMR 17.6.7.1.45 Root Complex BAR Configuration Register > > > their are many bits that are not tuned correctly. > > > I tried to set some bit to BAR Configuration register. but it dose not > > > work at my end. > > > I feel some more core configuration is missing. > > > If I have some update I will share it with you. > > > > What's the status of this discussion and therefore this series ? > > > > Thanks, > > Lorenzo > > Well I have looked into the RK3399 TRM (Rockchip RK3399 TRM V1.3 Part2.pdf) > There seems to be some core configuration missing, but I could not > resolve this on my board. So what are we going to do with this series ? Lorenzo
On 2020/7/14 0:45, Lorenzo Pieralisi wrote: > On Thu, Jul 09, 2020 at 09:18:27AM +0530, Anand Moon wrote: >> hi Lorenzo, >> >> On Wed, 8 Jul 2020 at 20:31, Lorenzo Pieralisi >> <lorenzo.pieralisi@arm.com> wrote: >>> >>> On Fri, May 22, 2020 at 05:59:14PM +0530, Anand Moon wrote: >>>> Hi Shawn >>>> >>>> On Fri, 22 May 2020 at 08:30, Shawn Lin <shawn.lin@rock-chips.com> wrote: >>>>> >>>>> >>>>> 在 2020/5/21 18:51, Anand Moon 写道: >>>>>> Hi Shawn, >>>>>> >>>>>> On Thu, 21 May 2020 at 06:35, Shawn Lin <shawn.lin@rock-chips.com> wrote: >>>>>>> >>>>>>> According to RK3399 user manual, bit 9 in PCIE_RC_BAR_CONF should >>>>>>> be set, otherwise accessing to IO base and limit registers would >>>>>>> fail. >>>>>>> >>>>>>> [ 0.411318] pci_bus 0000:00: root bus resource [bus 00-1f] >>>>>>> [ 0.411822] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff] >>>>>>> [ 0.412440] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) >>>>>>> [ 0.413665] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring >>>>>>> [ 0.414698] pci 0000:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid >>>>>>> [ 0.415412] pci 0000:01:00.0: reg 0x18: initial BAR value 0x00000000 invalid >>>>>>> [ 0.418456] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] >>>>>>> [ 0.419065] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa000000-0xfa007fff pref] >>>>>>> [ 0.419728] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa008000-0xfa00ffff pref] >>>>>>> [ 0.420377] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] >>>>>>> [ 0.420935] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] >>>>>>> [ 0.421526] pci 0000:01:00.0: BAR 2: no space for [io size 0x0004] >>>>>>> [ 0.422084] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0004] >>>>>>> [ 0.422687] pci 0000:00:00.0: PCI bridge to [bus 01] >>>>>>> [ 0.423135] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] >>>>>>> [ 0.423794] pcieport 0000:00:00.0: enabling device (0000 -> 0002) >>>>>>> [ 0.424566] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt >>>>>>> [ 0.425182] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt >>>>>>> >>>>>>> 01:00.0 Class 0700: Device 1c00:3853 (rev 10) (prog-if 05) >>>>>>> Subsystem: Device 1c00:3853 >>>>>>> Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- >>>>>>> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- >>>>>>> Interrupt: pin A routed to IRQ 230 >>>>>>> Region 0: I/O ports at <unassigned> [disabled] >>>>>>> Region 1: Memory at fa000000 (32-bit, prefetchable) [disabled] [size=32K] >>>>>>> Region 2: I/O ports at <unassigned> [disabled] >>>>>>> [virtual] Expansion ROM at fa008000 [disabled] [size=32K] >>>>>>> >>>>>>> Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> >>>>>>> --- >>>>>> >>>>>> I have old development board Odroid N1 (RK3399), It has onboard PCIe >>>>>> 2 dual sata bridge. >>>>>> I have tested this patch, but I am still getting following log on >>>>>> Odroid N1 board. >>>>>> Is their any more configuration needed for sata ports ? >>>>> >>>>> Thanks for testing. I made a mistake that it should be bit 19, so >>>>> can you try using BIT(19)? >>>>> >>>> >>>> Nop enable this bit dose not solve the issue see at my end. >>>> >>>> But as per RK3399 TMR 17.6.7.1.45 Root Complex BAR Configuration Register >>>> their are many bits that are not tuned correctly. >>>> I tried to set some bit to BAR Configuration register. but it dose not >>>> work at my end. >>>> I feel some more core configuration is missing. >>>> If I have some update I will share it with you. >>> >>> What's the status of this discussion and therefore this series ? >>> >>> Thanks, >>> Lorenzo >> >> Well I have looked into the RK3399 TRM (Rockchip RK3399 TRM V1.3 Part2.pdf) >> There seems to be some core configuration missing, but I could not >> resolve this on my board. > > So what are we going to do with this series ? I didn't test it on N1 board so I cannot say what happened there, but I incline to suspend this series untile I have a sufficient offlist debugging with Anand. > > Lorenzo > >
> On 2020/7/14 0:45, Lorenzo Pieralisi wrote: >> On Thu, Jul 09, 2020 at 09:18:27AM +0530, Anand Moon wrote: >>> hi Lorenzo, >>> >>> On Wed, 8 Jul 2020 at 20:31, Lorenzo Pieralisi >>> <lorenzo.pieralisi@arm.com> wrote: >>>> >>>> On Fri, May 22, 2020 at 05:59:14PM +0530, Anand Moon wrote: >>>>> Hi Shawn >>>>> >>>>> On Fri, 22 May 2020 at 08:30, Shawn Lin <shawn.lin@rock-chips.com> wrote: >>>>>> >>>>>> >>>>>> 在 2020/5/21 18:51, Anand Moon 写道: >>>>>>> Hi Shawn, >>>>>>> >>>>>>> On Thu, 21 May 2020 at 06:35, Shawn Lin <shawn.lin@rock-chips.com> wrote: >>>>>>>> >>>>>>>> According to RK3399 user manual, bit 9 in PCIE_RC_BAR_CONF should >>>>>>>> be set, otherwise accessing to IO base and limit registers would >>>>>>>> fail. >>>>>>>> >>>>>>>> [ 0.411318] pci_bus 0000:00: root bus resource [bus 00-1f] >>>>>>>> [ 0.411822] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff] >>>>>>>> [ 0.412440] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) >>>>>>>> [ 0.413665] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring >>>>>>>> [ 0.414698] pci 0000:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid >>>>>>>> [ 0.415412] pci 0000:01:00.0: reg 0x18: initial BAR value 0x00000000 invalid >>>>>>>> [ 0.418456] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] >>>>>>>> [ 0.419065] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa000000-0xfa007fff pref] >>>>>>>> [ 0.419728] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa008000-0xfa00ffff pref] >>>>>>>> [ 0.420377] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] >>>>>>>> [ 0.420935] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] >>>>>>>> [ 0.421526] pci 0000:01:00.0: BAR 2: no space for [io size 0x0004] >>>>>>>> [ 0.422084] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0004] >>>>>>>> [ 0.422687] pci 0000:00:00.0: PCI bridge to [bus 01] >>>>>>>> [ 0.423135] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] >>>>>>>> [ 0.423794] pcieport 0000:00:00.0: enabling device (0000 -> 0002) >>>>>>>> [ 0.424566] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt >>>>>>>> [ 0.425182] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt >>>>>>>> >>>>>>>> 01:00.0 Class 0700: Device 1c00:3853 (rev 10) (prog-if 05) >>>>>>>> Subsystem: Device 1c00:3853 >>>>>>>> Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- >>>>>>>> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- >>>>>>>> Interrupt: pin A routed to IRQ 230 >>>>>>>> Region 0: I/O ports at <unassigned> [disabled] >>>>>>>> Region 1: Memory at fa000000 (32-bit, prefetchable) [disabled] [size=32K] >>>>>>>> Region 2: I/O ports at <unassigned> [disabled] >>>>>>>> [virtual] Expansion ROM at fa008000 [disabled] [size=32K] >>>>>>>> >>>>>>>> Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> >>>>>>>> --- >>>>>>> >>>>>>> I have old development board Odroid N1 (RK3399), It has onboard PCIe >>>>>>> 2 dual sata bridge. >>>>>>> I have tested this patch, but I am still getting following log on >>>>>>> Odroid N1 board. >>>>>>> Is their any more configuration needed for sata ports ? >>>>>> >>>>>> Thanks for testing. I made a mistake that it should be bit 19, so >>>>>> can you try using BIT(19)? >>>>>> >>>>> >>>>> Nop enable this bit dose not solve the issue see at my end. >>>>> >>>>> But as per RK3399 TMR 17.6.7.1.45 Root Complex BAR Configuration Register >>>>> their are many bits that are not tuned correctly. >>>>> I tried to set some bit to BAR Configuration register. but it dose not >>>>> work at my end. >>>>> I feel some more core configuration is missing. >>>>> If I have some update I will share it with you. >>>> >>>> What's the status of this discussion and therefore this series ? >>>> >>>> Thanks, >>>> Lorenzo >>> >>> Well I have looked into the RK3399 TRM (Rockchip RK3399 TRM V1.3 Part2.pdf) >>> There seems to be some core configuration missing, but I could not >>> resolve this on my board. >> >> So what are we going to do with this series ? > > I didn't test it on N1 board so I cannot say what happened there, but I > incline to suspend this series untile I have a sufficient offlist > debugging with Anand. > >> >> Lorenzo >> >> Hello Shawn and all, Is there any news about this series? I happened to stumble upon this while searching anything PCIe related for my bus scan crash workaround [1]. This series still seems to apply cleanly on v5.11-rc6 so I applied the BIT(9) to BIT(19) change mentioned earlier and tested it with four SAS adapter cards and two SATA adapter cards. For all of them this series fixed "no space for io" in dmesg and "I/O ports at <unassigned> [disabled]" in lspci output. Below are few dmesg and lspci -vvnn snippets before and after applying these patches (SAS cards need bus scan crash workaround too). LSI SAS2008 before applying patch: [ 2.664846] pci_bus 0000:00: bus scan returning with max=01 [ 2.665412] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] [ 2.666083] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa000000-0xfa07ffff pref] [ 2.666796] pci 0000:01:00.0: BAR 3: assigned [mem 0xfa080000-0xfa0bffff 64bit] [ 2.667550] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa0c0000-0xfa0c3fff 64bit] [ 2.668301] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] [ 2.668917] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] [ 2.669565] pci 0000:00:00.0: PCI bridge to [bus 01] [ 2.670075] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] 00:00.0 PCI bridge [0604]: Fuzhou Rockchip Electronics Co., Ltd RK3399 PCI Express Root Port [1d87:0100] (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0, IRQ 78 Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 00000000-00000fff [size=4K] Memory behind bridge: fa000000-fa0fffff [size=1M] ... 01:00.0 RAID bus controller [0104]: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] [1000:0072] (rev 03) Subsystem: Fujitsu Technology Solutions HBA Ctrl SAS 6G 0/1 [D2607] [1734:1177] Flags: bus master, fast devsel, latency 0, IRQ 77 I/O ports at <unassigned> [disabled] Memory at fa0c0000 (64-bit, non-prefetchable) [size=16K] LSI SAS2008 after applying patch: [ 2.746453] pci_bus 0000:00: bus scan returning with max=01 [ 2.747021] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] [ 2.747689] pci 0000:00:00.0: BAR 7: assigned [io 0x1000-0x1fff] [ 2.748294] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa000000-0xfa07ffff pref] [ 2.749008] pci 0000:01:00.0: BAR 3: assigned [mem 0xfa080000-0xfa0bffff 64bit] [ 2.749761] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa0c0000-0xfa0c3fff 64bit] [ 2.750515] pci 0000:01:00.0: BAR 0: assigned [io 0x1000-0x10ff] [ 2.751128] pci 0000:00:00.0: PCI bridge to [bus 01] [ 2.751638] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff] [ 2.752242] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] 00:00.0 PCI bridge [0604]: Fuzhou Rockchip Electronics Co., Ltd RK3399 PCI Express Root Port [1d87:0100] (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0, IRQ 78 Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 00001000-00001fff [size=4K] Memory behind bridge: fa000000-fa0fffff [size=1M] ... 01:00.0 RAID bus controller [0104]: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] [1000:0072] (rev 03) Subsystem: Fujitsu Technology Solutions HBA Ctrl SAS 6G 0/1 [D2607] [1734:1177] Flags: bus master, fast devsel, latency 0, IRQ 77 I/O ports at 1000 [disabled] [size=256] Memory at fa0c0000 (64-bit, non-prefetchable) [size=16K] I haven't yet tested if this change actually makes these card work any better or worse but I could do more testing if that BIT(9) to BIT(19) is fixed and this series is otherwise able to get merged. [1] https://lore.kernel.org/linux-pci/20201231125214.25733-1-nuumiofi@gmail.com/ Regards, Jari
> On 2020/7/14 0:45, Lorenzo Pieralisi wrote: >> On Thu, Jul 09, 2020 at 09:18:27AM +0530, Anand Moon wrote: >>> hi Lorenzo, >>> >>> On Wed, 8 Jul 2020 at 20:31, Lorenzo Pieralisi >>> <lorenzo.pieralisi@arm.com> wrote: >>>> >>>> On Fri, May 22, 2020 at 05:59:14PM +0530, Anand Moon wrote: >>>>> Hi Shawn >>>>> >>>>> On Fri, 22 May 2020 at 08:30, Shawn Lin <shawn.lin@rock-chips.com> wrote: >>>>>> >>>>>> >>>>>> 在 2020/5/21 18:51, Anand Moon 写道: >>>>>>> Hi Shawn, >>>>>>> >>>>>>> On Thu, 21 May 2020 at 06:35, Shawn Lin <shawn.lin@rock-chips.com> wrote: >>>>>>>> >>>>>>>> According to RK3399 user manual, bit 9 in PCIE_RC_BAR_CONF should >>>>>>>> be set, otherwise accessing to IO base and limit registers would >>>>>>>> fail. >>>>>>>> >>>>>>>> [ 0.411318] pci_bus 0000:00: root bus resource [bus 00-1f] >>>>>>>> [ 0.411822] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff] >>>>>>>> [ 0.412440] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) >>>>>>>> [ 0.413665] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring >>>>>>>> [ 0.414698] pci 0000:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid >>>>>>>> [ 0.415412] pci 0000:01:00.0: reg 0x18: initial BAR value 0x00000000 invalid >>>>>>>> [ 0.418456] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] >>>>>>>> [ 0.419065] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa000000-0xfa007fff pref] >>>>>>>> [ 0.419728] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa008000-0xfa00ffff pref] >>>>>>>> [ 0.420377] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] >>>>>>>> [ 0.420935] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] >>>>>>>> [ 0.421526] pci 0000:01:00.0: BAR 2: no space for [io size 0x0004] >>>>>>>> [ 0.422084] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0004] >>>>>>>> [ 0.422687] pci 0000:00:00.0: PCI bridge to [bus 01] >>>>>>>> [ 0.423135] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] >>>>>>>> [ 0.423794] pcieport 0000:00:00.0: enabling device (0000 -> 0002) >>>>>>>> [ 0.424566] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt >>>>>>>> [ 0.425182] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt >>>>>>>> >>>>>>>> 01:00.0 Class 0700: Device 1c00:3853 (rev 10) (prog-if 05) >>>>>>>> Subsystem: Device 1c00:3853 >>>>>>>> Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- >>>>>>>> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- >>>>>>>> Interrupt: pin A routed to IRQ 230 >>>>>>>> Region 0: I/O ports at <unassigned> [disabled] >>>>>>>> Region 1: Memory at fa000000 (32-bit, prefetchable) [disabled] [size=32K] >>>>>>>> Region 2: I/O ports at <unassigned> [disabled] >>>>>>>> [virtual] Expansion ROM at fa008000 [disabled] [size=32K] >>>>>>>> >>>>>>>> Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> >>>>>>>> --- >>>>>>> >>>>>>> I have old development board Odroid N1 (RK3399), It has onboard PCIe >>>>>>> 2 dual sata bridge. >>>>>>> I have tested this patch, but I am still getting following log on >>>>>>> Odroid N1 board. >>>>>>> Is their any more configuration needed for sata ports ? >>>>>> >>>>>> Thanks for testing. I made a mistake that it should be bit 19, so >>>>>> can you try using BIT(19)? >>>>>> >>>>> >>>>> Nop enable this bit dose not solve the issue see at my end. >>>>> >>>>> But as per RK3399 TMR 17.6.7.1.45 Root Complex BAR Configuration Register >>>>> their are many bits that are not tuned correctly. >>>>> I tried to set some bit to BAR Configuration register. but it dose not >>>>> work at my end. >>>>> I feel some more core configuration is missing. >>>>> If I have some update I will share it with you. >>>> >>>> What's the status of this discussion and therefore this series ? >>>> >>>> Thanks, >>>> Lorenzo >>> >>> Well I have looked into the RK3399 TRM (Rockchip RK3399 TRM V1.3 Part2.pdf) >>> There seems to be some core configuration missing, but I could not >>> resolve this on my board. >> >> So what are we going to do with this series ? > > I didn't test it on N1 board so I cannot say what happened there, but I > incline to suspend this series untile I have a sufficient offlist > debugging with Anand. > >> >> Lorenzo >> >> Hello Shawn and all, Sorry about my previous garbled message. I'm trying to use git send-email and subject failed badly. Hopefully this one works better. Is there any news about this series? I happened to stumble upon this while searching anything PCIe related for my bus scan crash workaround [1]. This series still seems to apply cleanly on v5.11-rc6 so I applied the BIT(9) to BIT(19) change mentioned earlier and tested it with four SAS adapter cards and two SATA adapter cards. For all of them this series fixed "no space for io" in dmesg and "I/O ports at <unassigned> [disabled]" in lspci output. Below are few dmesg and lspci -vvnn snippets before and after applying these patches (SAS cards need bus scan crash workaround too). LSI SAS2008 before applying patch: [ 2.664846] pci_bus 0000:00: bus scan returning with max=01 [ 2.665412] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] [ 2.666083] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa000000-0xfa07ffff pref] [ 2.666796] pci 0000:01:00.0: BAR 3: assigned [mem 0xfa080000-0xfa0bffff 64bit] [ 2.667550] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa0c0000-0xfa0c3fff 64bit] [ 2.668301] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] [ 2.668917] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] [ 2.669565] pci 0000:00:00.0: PCI bridge to [bus 01] [ 2.670075] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] 00:00.0 PCI bridge [0604]: Fuzhou Rockchip Electronics Co., Ltd RK3399 PCI Express Root Port [1d87:0100] (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0, IRQ 78 Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 00000000-00000fff [size=4K] Memory behind bridge: fa000000-fa0fffff [size=1M] ... 01:00.0 RAID bus controller [0104]: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] [1000:0072] (rev 03) Subsystem: Fujitsu Technology Solutions HBA Ctrl SAS 6G 0/1 [D2607] [1734:1177] Flags: bus master, fast devsel, latency 0, IRQ 77 I/O ports at <unassigned> [disabled] Memory at fa0c0000 (64-bit, non-prefetchable) [size=16K] LSI SAS2008 after applying patch: [ 2.746453] pci_bus 0000:00: bus scan returning with max=01 [ 2.747021] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] [ 2.747689] pci 0000:00:00.0: BAR 7: assigned [io 0x1000-0x1fff] [ 2.748294] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa000000-0xfa07ffff pref] [ 2.749008] pci 0000:01:00.0: BAR 3: assigned [mem 0xfa080000-0xfa0bffff 64bit] [ 2.749761] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa0c0000-0xfa0c3fff 64bit] [ 2.750515] pci 0000:01:00.0: BAR 0: assigned [io 0x1000-0x10ff] [ 2.751128] pci 0000:00:00.0: PCI bridge to [bus 01] [ 2.751638] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff] [ 2.752242] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] 00:00.0 PCI bridge [0604]: Fuzhou Rockchip Electronics Co., Ltd RK3399 PCI Express Root Port [1d87:0100] (prog-if 00 [Normal decode]) Flags: bus master, fast devsel, latency 0, IRQ 78 Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 00001000-00001fff [size=4K] Memory behind bridge: fa000000-fa0fffff [size=1M] ... 01:00.0 RAID bus controller [0104]: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] [1000:0072] (rev 03) Subsystem: Fujitsu Technology Solutions HBA Ctrl SAS 6G 0/1 [D2607] [1734:1177] Flags: bus master, fast devsel, latency 0, IRQ 77 I/O ports at 1000 [disabled] [size=256] Memory at fa0c0000 (64-bit, non-prefetchable) [size=16K] I haven't yet tested if this change actually makes these card work any better or worse but I could do more testing if that BIT(9) to BIT(19) is fixed and this series is otherwise able to get merged. [1] https://lore.kernel.org/linux-pci/20201231125214.25733-1-nuumiofi@gmail.com/ Regards, Jari
diff --git a/drivers/pci/controller/pcie-rockchip.c b/drivers/pci/controller/pcie-rockchip.c index c53d132..f82452b 100644 --- a/drivers/pci/controller/pcie-rockchip.c +++ b/drivers/pci/controller/pcie-rockchip.c @@ -407,8 +407,11 @@ void rockchip_pcie_cfg_configuration_accesses( { u32 ob_desc_0; - /* Configuration Accesses for region 0 */ - rockchip_pcie_write(rockchip, 0x0, PCIE_RC_BAR_CONF); + /* + * Configuration Accesses for region 0. + * Bit 9 is for enabling IO base and limit registers. + */ + rockchip_pcie_write(rockchip, BIT(9), PCIE_RC_BAR_CONF); rockchip_pcie_write(rockchip, (RC_REGION_0_ADDR_TRANS_L + RC_REGION_0_PASS_BITS),
According to RK3399 user manual, bit 9 in PCIE_RC_BAR_CONF should be set, otherwise accessing to IO base and limit registers would fail. [ 0.411318] pci_bus 0000:00: root bus resource [bus 00-1f] [ 0.411822] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfbdfffff] [ 0.412440] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfbe00000-0xfbefffff]) [ 0.413665] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring [ 0.414698] pci 0000:01:00.0: reg 0x10: initial BAR value 0x00000000 invalid [ 0.415412] pci 0000:01:00.0: reg 0x18: initial BAR value 0x00000000 invalid [ 0.418456] pci 0000:00:00.0: BAR 8: assigned [mem 0xfa000000-0xfa0fffff] [ 0.419065] pci 0000:01:00.0: BAR 1: assigned [mem 0xfa000000-0xfa007fff pref] [ 0.419728] pci 0000:01:00.0: BAR 6: assigned [mem 0xfa008000-0xfa00ffff pref] [ 0.420377] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] [ 0.420935] pci 0000:01:00.0: BAR 0: failed to assign [io size 0x0100] [ 0.421526] pci 0000:01:00.0: BAR 2: no space for [io size 0x0004] [ 0.422084] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0004] [ 0.422687] pci 0000:00:00.0: PCI bridge to [bus 01] [ 0.423135] pci 0000:00:00.0: bridge window [mem 0xfa000000-0xfa0fffff] [ 0.423794] pcieport 0000:00:00.0: enabling device (0000 -> 0002) [ 0.424566] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt [ 0.425182] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt 01:00.0 Class 0700: Device 1c00:3853 (rev 10) (prog-if 05) Subsystem: Device 1c00:3853 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin A routed to IRQ 230 Region 0: I/O ports at <unassigned> [disabled] Region 1: Memory at fa000000 (32-bit, prefetchable) [disabled] [size=32K] Region 2: I/O ports at <unassigned> [disabled] [virtual] Expansion ROM at fa008000 [disabled] [size=32K] Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com> --- drivers/pci/controller/pcie-rockchip.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)