[OpenWrt-Devel,2/3] mvebu: reduce speed to gen1 for armada 37xx devices pcie

Message ID 20180609141342.23948-2-tomek_n@o2.pl
State Superseded
Delegated to: John Crispin
Headers show
Series
  • [OpenWrt-Devel,1/3] mvebu: add fix for armada 37xx cpufreq driver
Related show

Commit Message

Tomasz Maciej Nowak June 9, 2018, 2:13 p.m.
Since the beginning there's been an issue with initializing the Atheros
based MiniPCIe wlan cards. Here's an example of kerenel log:

advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x3c
advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x44
advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
ath9k 0000:00:00.0: enabling device (0000 -> 0002)
advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x3c
advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0xc
advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x40
ath9k 0000:00:00.0: request_irq failed
advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
ath9k: probe of 0000:00:00.0 failed with error -22

The same happens for ath5k cards, while ath10k card didn't appear at
all (not detected). Following the issue on esppressobin.net forum [1]
the workaround seems to be limiting the speed of PCIe bridge to 1st
generation. This fixed the initialization of ath5k, ath9k and ath10k
cards. The change shouldn't affect the performance for wireless cards,
it could reduce the performance of storage controller cards but since
OpenWrt focuses on wireless connectivity, fixing compatibility with
wireless cards should be a priority.
For the record, the iwlwifi and mt76 cards were not affected by this
issue.

1. http://espressobin.net/forums/topic/which-pcie-wlan-cards-are-supported

Signed-off-by: Tomasz Maciej Nowak <tomek_n@o2.pl>
---
Please also cherry-pick this to OpenWrt 18.06.
---
 .../527-pci-aardvark-reduce-speed-to-gen1.patch   | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
 create mode 100644 target/linux/mvebu/patches-4.14/527-pci-aardvark-reduce-speed-to-gen1.patch

Comments

Kevin Darbyshire-Bryant via openwrt-devel June 9, 2018, 5:02 p.m. | #1
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.

To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.
On Sat, Jun 9, 2018 at 4:15 PM Tomasz Maciej Nowak <tomek_n@o2.pl> wrote:
>
> Since the beginning there's been an issue with initializing the Atheros
> based MiniPCIe wlan cards. Here's an example of kerenel log:
>
> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x3c
> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x44
> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
> ath9k 0000:00:00.0: enabling device (0000 -> 0002)
> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x3c
> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0xc
> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x40
> ath9k 0000:00:00.0: request_irq failed
> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
> ath9k: probe of 0000:00:00.0 failed with error -22
>
> The same happens for ath5k cards, while ath10k card didn't appear at
> all (not detected). Following the issue on esppressobin.net forum [1]
> the workaround seems to be limiting the speed of PCIe bridge to 1st
> generation. This fixed the initialization of ath5k, ath9k and ath10k
> cards. The change shouldn't affect the performance for wireless cards,
> it could reduce the performance of storage controller cards but since
> OpenWrt focuses on wireless connectivity, fixing compatibility with
> wireless cards should be a priority.
> For the record, the iwlwifi and mt76 cards were not affected by this
> issue.
does this meant that the PCIe link speed depends on the board?

the PCI dt-bindings already specify a "max-link-speed" property, see [0]
there's even a helper function to parse that property: of_pci_get_max_link_speed

this would give you control over the PCIe link speed per board (I am
assuming that the mvebu target uses devicetree).
additionally this would allow you to send the patch upstream so
OpenWrt doesn't have to carry custom patches around forever

what do you think?


Regards
Martin


[0] https://github.com/torvalds/linux/blob/master/Documentation/devicetree/bindings/pci/pci.txt
Tomasz Maciej Nowak June 9, 2018, 9:09 p.m. | #2
W dniu 09.06.2018 o 19:02, Martin Blumenstingl pisze:
> On Sat, Jun 9, 2018 at 4:15 PM Tomasz Maciej Nowak <tomek_n@o2.pl> wrote:
>>
>> Since the beginning there's been an issue with initializing the Atheros
>> based MiniPCIe wlan cards. Here's an example of kerenel log:
>>
>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x3c
>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x44
>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
>> ath9k 0000:00:00.0: enabling device (0000 -> 0002)
>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x3c
>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0xc
>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x40
>> ath9k 0000:00:00.0: request_irq failed
>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
>> ath9k: probe of 0000:00:00.0 failed with error -22
>>
>> The same happens for ath5k cards, while ath10k card didn't appear at
>> all (not detected). Following the issue on esppressobin.net forum [1]
>> the workaround seems to be limiting the speed of PCIe bridge to 1st
>> generation. This fixed the initialization of ath5k, ath9k and ath10k
>> cards. The change shouldn't affect the performance for wireless cards,
>> it could reduce the performance of storage controller cards but since
>> OpenWrt focuses on wireless connectivity, fixing compatibility with
>> wireless cards should be a priority.
>> For the record, the iwlwifi and mt76 cards were not affected by this
>> issue.
> does this meant that the PCIe link speed depends on the board?

No, that depends on the SoC board has and PCIe card which negotiates the speed and capabilities. It shouldn't depend on the board, of course if there are no design faults. Maybe some code in Atheros drivers also affects this or there's bug in aardvark enumerating code. The assessment of this is outside of my skills.

> 
> the PCI dt-bindings already specify a "max-link-speed" property, see [0]
> there's even a helper function to parse that property: of_pci_get_max_link_speed
> 
> this would give you control over the PCIe link speed per board (I am
> assuming that the mvebu target uses devicetree).

I tried this before submitting the patch, just to be sure retried it after Your mail. Setting this had no effect, the state was as if there was no change.

> additionally this would allow you to send the patch upstream so
> OpenWrt doesn't have to carry custom patches around forever
> 
> what do you think?

This driver still is in early stage, there are still some issues, until it'll be more mature I'm reluctant to send this workaround upstream or sending bug report. Thomas Petazzoni from Bootlin is working on this driver and still has some pending changes but that will take few months. Until his changes will hit upstream I'm inclined to keep this.
There is also upcoming device from CZ.nic, namely Turris MOX, which is based on the same processor as ESPRESSObin. They already submitted watchdog driver to LKML and U-Boot tree and minor fixes to U-Boot. Maybe their involvement will speed up the changes and we'll see this workaround unnecessary.

Just to be clear this issue and workaround affects only devices based on Armada 37xx SoC living in cortexa53 subtarget.

Regards, Tomasz

> 
> 
> Regards
> Martin
> 
> 
> [0] https://github.com/torvalds/linux/blob/master/Documentation/devicetree/bindings/pci/pci.txt
>
Kevin Darbyshire-Bryant via openwrt-devel June 10, 2018, 1:23 p.m. | #3
The sender domain has a DMARC Reject/Quarantine policy which disallows
sending mailing list messages using the original "From" header.

To mitigate this problem, the original message has been wrapped
automatically by the mailing list software.
On Sat, Jun 9, 2018 at 11:09 PM Tomasz Maciej Nowak <tmn505@gmail.com> wrote:
>
> W dniu 09.06.2018 o 19:02, Martin Blumenstingl pisze:
> > On Sat, Jun 9, 2018 at 4:15 PM Tomasz Maciej Nowak <tomek_n@o2.pl> wrote:
> >>
> >> Since the beginning there's been an issue with initializing the Atheros
> >> based MiniPCIe wlan cards. Here's an example of kerenel log:
> >>
> >> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x3c
> >> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x44
> >> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
> >> ath9k 0000:00:00.0: enabling device (0000 -> 0002)
> >> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x3c
> >> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0xc
> >> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
> >> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x40
> >> ath9k 0000:00:00.0: request_irq failed
> >> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
> >> ath9k: probe of 0000:00:00.0 failed with error -22
> >>
> >> The same happens for ath5k cards, while ath10k card didn't appear at
> >> all (not detected). Following the issue on esppressobin.net forum [1]
> >> the workaround seems to be limiting the speed of PCIe bridge to 1st
> >> generation. This fixed the initialization of ath5k, ath9k and ath10k
> >> cards. The change shouldn't affect the performance for wireless cards,
> >> it could reduce the performance of storage controller cards but since
> >> OpenWrt focuses on wireless connectivity, fixing compatibility with
> >> wireless cards should be a priority.
> >> For the record, the iwlwifi and mt76 cards were not affected by this
> >> issue.
> > does this meant that the PCIe link speed depends on the board?
>
> No, that depends on the SoC board has and PCIe card which negotiates the speed and capabilities. It shouldn't depend on the board, of course if there are no design faults. Maybe some code in Atheros drivers also affects this or there's bug in aardvark enumerating code. The assessment of this is outside of my skills.
I was curious because officially the SoCs from the Armada 3700 series
support PCIe 2.0, so it seems strange to force/limit the hardware to
PCIe 1.0

> >
> > the PCI dt-bindings already specify a "max-link-speed" property, see [0]
> > there's even a helper function to parse that property: of_pci_get_max_link_speed
> >
> > this would give you control over the PCIe link speed per board (I am
> > assuming that the mvebu target uses devicetree).
>
> I tried this before submitting the patch, just to be sure retried it after Your mail. Setting this had no effect, the state was as if there was no change.
as far as I can see the pci-aardvark driver does not parse the
"max-link-speed" devicetree property yet (because it does not use/call
of_pci_get_max_link_speed).
so to use the "max-link-speed" property one would:
- implement support for it in the driver
- add the property to the .dts file

> > additionally this would allow you to send the patch upstream so
> > OpenWrt doesn't have to carry custom patches around forever
> >
> > what do you think?
>
> This driver still is in early stage, there are still some issues, until it'll be more mature I'm reluctant to send this workaround upstream or sending bug report. Thomas Petazzoni from Bootlin is working on this driver and still has some pending changes but that will take few months. Until his changes will hit upstream I'm inclined to keep this.
I'm not against adding this patch
it just seems to me that all PCI related kernel patches in OpenWrt are
backports from upstream patches
so why not report your issue upstream, work on a fix with them and
backport the result (this will make kernel updates on the mvebu target
easier, since it'll allow dropping patches when updating rather then
having to figure out why they are still needed, rebasing and testing
them again, ...)

> There is also upcoming device from CZ.nic, namely Turris MOX, which is based on the same processor as ESPRESSObin. They already submitted watchdog driver to LKML and U-Boot tree and minor fixes to U-Boot. Maybe their involvement will speed up the changes and we'll see this workaround unnecessary.
by sending your patch (even if you think that it's not in shape to be
applied upstream - then just state that explicitly) upstream you may
save other developers lots of time since you already found a way to
make some Atheros PCIe devices work with this controller/driver :)


Regards
Martin
Tomasz Maciej Nowak June 14, 2018, 7:31 p.m. | #4
W dniu 10.06.2018 o 15:23, Martin Blumenstingl pisze:
> On Sat, Jun 9, 2018 at 11:09 PM Tomasz Maciej Nowak <tmn505@gmail.com> wrote:
>>
>> W dniu 09.06.2018 o 19:02, Martin Blumenstingl pisze:
>>> On Sat, Jun 9, 2018 at 4:15 PM Tomasz Maciej Nowak <tomek_n@o2.pl> wrote:
>>>>
>>>> Since the beginning there's been an issue with initializing the Atheros
>>>> based MiniPCIe wlan cards. Here's an example of kerenel log:
>>>>
>>>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x3c
>>>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x44
>>>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
>>>> ath9k 0000:00:00.0: enabling device (0000 -> 0002)
>>>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x3c
>>>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0xc
>>>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
>>>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x40
>>>> ath9k 0000:00:00.0: request_irq failed
>>>> advk-pcie d0070000.pcie: Posted PIO Response Status: CA,0xe00 @ 0x4
>>>> ath9k: probe of 0000:00:00.0 failed with error -22
>>>>
>>>> The same happens for ath5k cards, while ath10k card didn't appear at
>>>> all (not detected). Following the issue on esppressobin.net forum [1]
>>>> the workaround seems to be limiting the speed of PCIe bridge to 1st
>>>> generation. This fixed the initialization of ath5k, ath9k and ath10k
>>>> cards. The change shouldn't affect the performance for wireless cards,
>>>> it could reduce the performance of storage controller cards but since
>>>> OpenWrt focuses on wireless connectivity, fixing compatibility with
>>>> wireless cards should be a priority.
>>>> For the record, the iwlwifi and mt76 cards were not affected by this
>>>> issue.
>>> does this meant that the PCIe link speed depends on the board?
>>
>> No, that depends on the SoC board has and PCIe card which negotiates the speed and capabilities. It shouldn't depend on the board, of course if there are no design faults. Maybe some code in Atheros drivers also affects this or there's bug in aardvark enumerating code. The assessment of this is outside of my skills.
> I was curious because officially the SoCs from the Armada 3700 series
> support PCIe 2.0, so it seems strange to force/limit the hardware to
> PCIe 1.0
> 
>>>
>>> the PCI dt-bindings already specify a "max-link-speed" property, see [0]
>>> there's even a helper function to parse that property: of_pci_get_max_link_speed
>>>
>>> this would give you control over the PCIe link speed per board (I am
>>> assuming that the mvebu target uses devicetree).
>>
>> I tried this before submitting the patch, just to be sure retried it after Your mail. Setting this had no effect, the state was as if there was no change.
> as far as I can see the pci-aardvark driver does not parse the
> "max-link-speed" devicetree property yet (because it does not use/call
> of_pci_get_max_link_speed).
> so to use the "max-link-speed" property one would:
> - implement support for it in the driver
> - add the property to the .dts file
> 

Done. Yesterday using my copy-pasta fu (don't know any programming language, only shell scripting) I managed to whip a patch from Your pointers.

>>> additionally this would allow you to send the patch upstream so
>>> OpenWrt doesn't have to carry custom patches around forever
>>>
>>> what do you think?
>>
>> This driver still is in early stage, there are still some issues, until it'll be more mature I'm reluctant to send this workaround upstream or sending bug report. Thomas Petazzoni from Bootlin is working on this driver and still has some pending changes but that will take few months. Until his changes will hit upstream I'm inclined to keep this.
> I'm not against adding this patch
> it just seems to me that all PCI related kernel patches in OpenWrt are
> backports from upstream patches
> so why not report your issue upstream, work on a fix with them and
> backport the result (this will make kernel updates on the mvebu target
> easier, since it'll allow dropping patches when updating rather then
> having to figure out why they are still needed, rebasing and testing
> them again, ..
I'm not confident to do this and defend my standpoint given the lack in skills.

>> There is also upcoming device from CZ.nic, namely Turris MOX, which is based on the same processor as ESPRESSObin. They already submitted watchdog driver to LKML and U-Boot tree and minor fixes to U-Boot. Maybe their involvement will speed up the changes and we'll see this workaround unnecessary.
> by sending your patch (even if you think that it's not in shape to be
> applied upstream - then just state that explicitly) upstream you may
> save other developers lots of time since you already found a way to
> make some Atheros PCIe devices work with this controller/driver :)

That's a valid point. I'll report the bug on bugzilla after some tests with linux-next.

> 
> 
> Regards
> Martin
> 

Regards, Tomasz.

Patch

diff --git a/target/linux/mvebu/patches-4.14/527-pci-aardvark-reduce-speed-to-gen1.patch b/target/linux/mvebu/patches-4.14/527-pci-aardvark-reduce-speed-to-gen1.patch
new file mode 100644
index 0000000000..1974c5684f
--- /dev/null
+++ b/target/linux/mvebu/patches-4.14/527-pci-aardvark-reduce-speed-to-gen1.patch
@@ -0,0 +1,15 @@ 
+--- a/drivers/pci/host/pci-aardvark.c
++++ b/drivers/pci/host/pci-aardvark.c
+@@ -311,10 +311,10 @@ static void advk_pcie_setup_hw(struct ad
+ 		PCIE_CORE_CTRL2_TD_ENABLE;
+ 	advk_writel(pcie, reg, PCIE_CORE_CTRL2_REG);
+ 
+-	/* Set GEN2 */
++	/* Set GEN1 */
+ 	reg = advk_readl(pcie, PCIE_CORE_CTRL0_REG);
+ 	reg &= ~PCIE_GEN_SEL_MSK;
+-	reg |= SPEED_GEN_2;
++	reg |= SPEED_GEN_1;
+ 	advk_writel(pcie, reg, PCIE_CORE_CTRL0_REG);
+ 
+ 	/* Set lane X1 */