mbox series

[Aspeed,ncsi-rx,v1,0/1] net: ftgmac100: Fix AST2600EVB NCSI RX issue

Message ID 20201215192323.24359-1-hongweiz@ami.com
Headers show
Series net: ftgmac100: Fix AST2600EVB NCSI RX issue | expand

Message

Hongwei Zhang Dec. 15, 2020, 7:23 p.m. UTC 
Dear Reviewer,

Fix AST2600 EVB NCSI RX timeout issue by removing FTGMAC100_RXDES0_RX_ERR bit 
from macro RXDES0_ANY_ERROR.

Hongwei Zhang (1):
  net: ftgmac100: Fix AST2600 EVB NCSI RX issue

 drivers/net/ethernet/faraday/ftgmac100.h | 1 -
 1 file changed, 1 deletion(-)

Comments

Hongwei Zhang Dec. 21, 2020, 5 p.m. UTC | #1 
Dear Reviewer,

When FTGMAC100 driver is used on other NCSI Ethernet controllers, few
controllers have compatible issue. One example is Intel I210 Ethernet
controller on AST2600 BMC, with FTGMAC100 driver, it always trigger
RXDES0_RX_ERR error, cause NCSI initialization failure, removing
FTGMAC100_RXDES0_RX_ERR bit from RXDES0_ANY_ERROR fix the issue.

Here are part of the debug logs:
......
[   35.075552] ftgmac100_hard_start_xmit TXDESO=b000003c
[   35.080843] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 55
[   35.087141] ftgmac100 1e660000.ethernet eth0: rx_packet_error RXDES0=0xb0070040
[   35.094448] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000 RXDES2=88f8 
[   35.101498] ftgmac100 1e660000.ethernet eth0: rx_packet_error 0xb0070040
[   35.108205] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040: RX_ERR
[   35.287808] i2c i2c-1: new_device: Instantiated device slave-mqueue at 0x12
[   35.428379] ftgmac100_hard_start_xmit TXDESO=b000003c
[   35.433624] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 56
[   35.439915] ftgmac100 1e660000.ethernet eth0: rx_packet_error RXDES0=0xb0070040
[   35.447225] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000 RXDES2=88f8
[   35.454273] ftgmac100 1e660000.ethernet eth0: rx_packet_error 0xb0070040
[   35.460972] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040: RX_ERR
[   35.797825] ftgmac100_hard_start_xmit TXDESO=b000003c
[   35.803241] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 57
[   35.809541] ftgmac100 1e660000.ethernet eth0: rx_packet_error RXDES0=0xb0070040
[   35.816848] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000 RXDES2=88f8
[   35.823899] ftgmac100 1e660000.ethernet eth0: rx_packet_error 0xb0070040
[   35.830597] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040: RX_ERR
[   36.179914] ftgmac100_hard_start_xmit TXDESO=b000003c
[   36.185160] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 58
[   36.191454] ftgmac100 1e660000.ethernet eth0: rx_packet_error RXDES0=0xb0070040
[   36.198761] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000 RXDES2=88f8
[   36.205813] ftgmac100 1e660000.ethernet eth0: rx_packet_error 0xb0070040
[   36.212513] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040: RX_ERR
[   36.593688] ftgmac100_hard_start_xmit TXDESO=b000003c
[   36.602937] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 59
[   36.609244] ftgmac100 1e660000.ethernet eth0: rx_packet_error RXDES0=0xb0070040
[   36.616558] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000 RXDES2=88f8
[   36.623608] ftgmac100 1e660000.ethernet eth0: rx_packet_error 0xb0070040
[   36.630315] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040: RX_ERR
[   37.031524] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   37.067831] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
............

This patch add a configurable flag, FTGMAC100_RXDES0_RX_ERR_CHK, in FTGMAC100
 driver, it is YES by default, so keep the orignal define of
RXDES0_ANY_ERROR. If it is needed, user can set the flag to NO to remove
the RXDES0_RX_ERR bit, to fix the issue.

Hongwei Zhang (1):
  net: ftgmac100: Fix AST2600 EVB NCSI RX issue

 drivers/net/ethernet/faraday/Kconfig     | 9 +++++++++
 drivers/net/ethernet/faraday/ftgmac100.h | 8 ++++++++
 2 files changed, 17 insertions(+)
Hongwei Zhang Dec. 21, 2020, 5 p.m. UTC | #2 
Hi Andrew,

> From:	Andrew Jeffery <andrew@aj.id.au>
> 
> > Fix AST2600 EVB NCSI RX timeout issue by removing 
> > FTGMAC100_RXDES0_RX_ERR bit from macro RXDES0_ANY_ERROR.
> 
> But why? What is wrong with the EVB that this change resolves? Which revision of the EVB?
> 
> The change affects all designs using the MAC, not just the AST2600 EVB. Why is this patch an 
> appropriate course of action? Can we not add a quirk targeting the specific board (e.g. a devicetree 
> property)?

You are correct. I was in a rush and didn't put this patch under whole picture 
of the driver it affected. I revised the cover letter with more information, also
use a configurable flag to isolate the change. please review the new approach.

> 
> Andrew
> 
Thanks,
--Hongwei
Hongwei Zhang Dec. 21, 2020, 5 p.m. UTC | #3 
Hi Jakub,

> From:	Jakub Kicinski <kuba@kernel.org>
> 
> > ... 
> > Signed-off-by: Hongwei Zhang <hongweiz@ami.com>
> 
> Thanks for the patch. Please repost CCing the netdev mailing list so it can be merged to the networking 
> tree (which I assume is your intent).
> Please also include a Fixes tag pointing to the commit where the timeout issue started (even if it's the 
> first commit of the driver).
> 
I updated the cc list and cover letter accordingly, also addressed
Andrew's question. please review.

Thanks,
--Hongwei
Joel Stanley Dec. 22, 2020, 2:26 a.m. UTC | #4 
On Mon, 21 Dec 2020 at 17:01, Hongwei Zhang <hongweiz@ami.com> wrote:
>
> Dear Reviewer,
>
> When FTGMAC100 driver is used on other NCSI Ethernet controllers, few
> controllers have compatible issue. One example is Intel I210 Ethernet
> controller on AST2600 BMC, with FTGMAC100 driver, it always trigger
> RXDES0_RX_ERR error, cause NCSI initialization failure, removing
> FTGMAC100_RXDES0_RX_ERR bit from RXDES0_ANY_ERROR fix the issue.

I work with a few systems that use the i210 on the 2600. We haven't
seen this issue in our testing.

Is there something specific about the setup that you use to trigger this?

Ryan, is this an issue that Aspeed is aware of?

Cheers,

Joel

>
> Here are part of the debug logs:
> ......
> [   35.075552] ftgmac100_hard_start_xmit TXDESO=b000003c
> [   35.080843] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 55
> [   35.087141] ftgmac100 1e660000.ethernet eth0: rx_packet_error RXDES0=0xb0070040
> [   35.094448] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000 RXDES2=88f8
> [   35.101498] ftgmac100 1e660000.ethernet eth0: rx_packet_error 0xb0070040
> [   35.108205] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040: RX_ERR
> [   35.287808] i2c i2c-1: new_device: Instantiated device slave-mqueue at 0x12
> [   35.428379] ftgmac100_hard_start_xmit TXDESO=b000003c
> [   35.433624] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 56
> [   35.439915] ftgmac100 1e660000.ethernet eth0: rx_packet_error RXDES0=0xb0070040
> [   35.447225] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000 RXDES2=88f8
> [   35.454273] ftgmac100 1e660000.ethernet eth0: rx_packet_error 0xb0070040
> [   35.460972] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040: RX_ERR
> [   35.797825] ftgmac100_hard_start_xmit TXDESO=b000003c
> [   35.803241] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 57
> [   35.809541] ftgmac100 1e660000.ethernet eth0: rx_packet_error RXDES0=0xb0070040
> [   35.816848] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000 RXDES2=88f8
> [   35.823899] ftgmac100 1e660000.ethernet eth0: rx_packet_error 0xb0070040
> [   35.830597] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040: RX_ERR
> [   36.179914] ftgmac100_hard_start_xmit TXDESO=b000003c
> [   36.185160] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 58
> [   36.191454] ftgmac100 1e660000.ethernet eth0: rx_packet_error RXDES0=0xb0070040
> [   36.198761] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000 RXDES2=88f8
> [   36.205813] ftgmac100 1e660000.ethernet eth0: rx_packet_error 0xb0070040
> [   36.212513] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040: RX_ERR
> [   36.593688] ftgmac100_hard_start_xmit TXDESO=b000003c
> [   36.602937] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 59
> [   36.609244] ftgmac100 1e660000.ethernet eth0: rx_packet_error RXDES0=0xb0070040
> [   36.616558] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000 RXDES2=88f8
> [   36.623608] ftgmac100 1e660000.ethernet eth0: rx_packet_error 0xb0070040
> [   36.630315] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040: RX_ERR
> [   37.031524] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
> [   37.067831] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
> ............
>
> This patch add a configurable flag, FTGMAC100_RXDES0_RX_ERR_CHK, in FTGMAC100
>  driver, it is YES by default, so keep the orignal define of
> RXDES0_ANY_ERROR. If it is needed, user can set the flag to NO to remove
> the RXDES0_RX_ERR bit, to fix the issue.
>
> Hongwei Zhang (1):
>   net: ftgmac100: Fix AST2600 EVB NCSI RX issue
>
>  drivers/net/ethernet/faraday/Kconfig     | 9 +++++++++
>  drivers/net/ethernet/faraday/ftgmac100.h | 8 ++++++++
>  2 files changed, 17 insertions(+)
>
> --
> 2.17.1
>
Dylan Hung Dec. 22, 2020, 2:43 a.m. UTC | #5 
Hi Hongwei,

The NCSI should run on 3.3V RMII.  According your log, you enabled NCSI on ftgmac100@1e660000 which can only support 1.8V I/O voltage.
Did you observe the same error on ftgmac100@1e670000 (MAC3) or ftgmac100@1e690000 (MAC4)?

--
Dylan

> -----Original Message-----
> From: Linux-aspeed
> [mailto:linux-aspeed-bounces+dylan_hung=aspeedtech.com@lists.ozlabs.org]
> On Behalf Of Joel Stanley
> Sent: 2020年12月22日 10:26 AM
> To: Hongwei Zhang <hongweiz@ami.com>; Ryan Chen
> <ryan_chen@aspeedtech.com>
> Cc: linux-aspeed <linux-aspeed@lists.ozlabs.org>; netdev
> <netdev@vger.kernel.org>; OpenBMC Maillist <openbmc@lists.ozlabs.org>;
> Linux Kernel Mailing List <linux-kernel@vger.kernel.org>; Jakub Kicinski
> <kuba@kernel.org>; David S Miller <davem@davemloft.net>
> Subject: Re: [Aspeed, ncsi-rx, v1 0/1] net: ftgmac100: Fix AST2600EVB NCSI RX
> issue
> 
> On Mon, 21 Dec 2020 at 17:01, Hongwei Zhang <hongweiz@ami.com> wrote:
> >
> > Dear Reviewer,
> >
> > When FTGMAC100 driver is used on other NCSI Ethernet controllers, few
> > controllers have compatible issue. One example is Intel I210 Ethernet
> > controller on AST2600 BMC, with FTGMAC100 driver, it always trigger
> > RXDES0_RX_ERR error, cause NCSI initialization failure, removing
> > FTGMAC100_RXDES0_RX_ERR bit from RXDES0_ANY_ERROR fix the issue.
> 
> I work with a few systems that use the i210 on the 2600. We haven't seen this
> issue in our testing.
> 
> Is there something specific about the setup that you use to trigger this?
> 
> Ryan, is this an issue that Aspeed is aware of?
> 
> Cheers,
> 
> Joel
> 
> >
> > Here are part of the debug logs:
> > ......
> > [   35.075552] ftgmac100_hard_start_xmit TXDESO=b000003c
> > [   35.080843] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 55
> > [   35.087141] ftgmac100 1e660000.ethernet eth0: rx_packet_error
> RXDES0=0xb0070040
> > [   35.094448] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000
> RXDES2=88f8
> > [   35.101498] ftgmac100 1e660000.ethernet eth0: rx_packet_error
> 0xb0070040
> > [   35.108205] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040:
> RX_ERR
> > [   35.287808] i2c i2c-1: new_device: Instantiated device slave-mqueue at
> 0x12
> > [   35.428379] ftgmac100_hard_start_xmit TXDESO=b000003c
> > [   35.433624] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 56
> > [   35.439915] ftgmac100 1e660000.ethernet eth0: rx_packet_error
> RXDES0=0xb0070040
> > [   35.447225] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000
> RXDES2=88f8
> > [   35.454273] ftgmac100 1e660000.ethernet eth0: rx_packet_error
> 0xb0070040
> > [   35.460972] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040:
> RX_ERR
> > [   35.797825] ftgmac100_hard_start_xmit TXDESO=b000003c
> > [   35.803241] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 57
> > [   35.809541] ftgmac100 1e660000.ethernet eth0: rx_packet_error
> RXDES0=0xb0070040
> > [   35.816848] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000
> RXDES2=88f8
> > [   35.823899] ftgmac100 1e660000.ethernet eth0: rx_packet_error
> 0xb0070040
> > [   35.830597] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040:
> RX_ERR
> > [   36.179914] ftgmac100_hard_start_xmit TXDESO=b000003c
> > [   36.185160] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 58
> > [   36.191454] ftgmac100 1e660000.ethernet eth0: rx_packet_error
> RXDES0=0xb0070040
> > [   36.198761] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000
> RXDES2=88f8
> > [   36.205813] ftgmac100 1e660000.ethernet eth0: rx_packet_error
> 0xb0070040
> > [   36.212513] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040:
> RX_ERR
> > [   36.593688] ftgmac100_hard_start_xmit TXDESO=b000003c
> > [   36.602937] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 59
> > [   36.609244] ftgmac100 1e660000.ethernet eth0: rx_packet_error
> RXDES0=0xb0070040
> > [   36.616558] ftgmac100_rx_packet RXDES0=b0070040 RXDES1=f0800000
> RXDES2=88f8
> > [   36.623608] ftgmac100 1e660000.ethernet eth0: rx_packet_error
> 0xb0070040
> > [   36.630315] ftgmac100 1e660000.ethernet eth0: [ISR] = 0xb0070040:
> RX_ERR
> > [   37.031524] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
> > [   37.067831] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
> > ............
> >
> > This patch add a configurable flag, FTGMAC100_RXDES0_RX_ERR_CHK, in
> > FTGMAC100  driver, it is YES by default, so keep the orignal define of
> > RXDES0_ANY_ERROR. If it is needed, user can set the flag to NO to
> > remove the RXDES0_RX_ERR bit, to fix the issue.
> >
> > Hongwei Zhang (1):
> >   net: ftgmac100: Fix AST2600 EVB NCSI RX issue
> >
> >  drivers/net/ethernet/faraday/Kconfig     | 9 +++++++++
> >  drivers/net/ethernet/faraday/ftgmac100.h | 8 ++++++++
> >  2 files changed, 17 insertions(+)
> >
> > --
> > 2.17.1
> >
Hongwei Zhang Jan. 11, 2021, 8:58 p.m. UTC | #6 
> 
> Hi Hongwei,
> 
> The NCSI should run on 3.3V RMII.  According your log, you enabled NCSI on 
> ftgmac100@1e660000 which can only support 1.8V I/O voltage.
> Did you observe the same error on ftgmac100@1e670000 (MAC3) or ftgmac100@1e690000 (MAC4)?
> 

Hi Dylan,

Thanks for your review and input, you're correct, this issue is not observed on
AST2600 MAC4 (ftgmac100@1e690000).

Though this issue is caused by using NCSI incompatible MAC ftgmac100@1e660000,
we thought this patch is still having value, by providing an extra option to
user to be able to use ftgmac100@1e660000 for NCSI, and this is also true for
AST2500.

--Hongwei
 
> > -----Original Message-----
> > From: Linux-aspeed
> > [mailto:linux-aspeed-bounces+dylan_hung=aspeedtech.com@lists.ozlabs.or
> > g]
> > On Behalf Of Joel Stanley
> > Sent: 2020?12?22? 10:26 AM
> > To: Hongwei Zhang <hongweiz@ami.com>; Ryan Chen 
> > <ryan_chen@aspeedtech.com>
> > Cc: linux-aspeed <linux-aspeed@lists.ozlabs.org>; netdev 
> > <netdev@vger.kernel.org>; OpenBMC Maillist <openbmc@lists.ozlabs.org>; 
> > Linux Kernel Mailing List <linux-kernel@vger.kernel.org>; Jakub 
> > Kicinski <kuba@kernel.org>; David S Miller <davem@davemloft.net>
> > Subject: Re: [Aspeed, ncsi-rx, v1 0/1] net: ftgmac100: Fix AST2600EVB 
> > NCSI RX issue
> > 
> > On Mon, 21 Dec 2020 at 17:01, Hongwei Zhang <hongweiz@ami.com> wrote:
> > >
> > > Dear Reviewer,
> > >
> > > When FTGMAC100 driver is used on other NCSI Ethernet controllers, 
> > > few controllers have compatible issue. One example is Intel I210 
> > > Ethernet controller on AST2600 BMC, with FTGMAC100 driver, it always 
> > > trigger RXDES0_RX_ERR error, cause NCSI initialization failure, 
> > > removing FTGMAC100_RXDES0_RX_ERR bit from RXDES0_ANY_ERROR fix the issue.
> > 
> > I work with a few systems that use the i210 on the 2600. We haven't 
> > seen this issue in our testing.
> > 
> > Is there something specific about the setup that you use to trigger this?
> > 
> > Ryan, is this an issue that Aspeed is aware of?
> > 
> > Cheers,
> > 
> > Joel

Hello Joel,

Thanks for your review, please see my response to Dylan, he pointed out
the root cause of the issue.

-- Hongwei

> > 
> > >
> > > Here are part of the debug logs:
> > > ......
> > > [   35.075552] ftgmac100_hard_start_xmit TXDESO=b000003c
> > > [   35.080843] ftgmac100 1e660000.ethernet eth0: tx_complete_packet 55
> > > [   35.087141] ftgmac100 1e660000.ethernet eth0: rx_packet_error
> > RXDES0=0xb0070040
> > > [   37.067831] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
> > > ............
> > >
> > > This patch add a configurable flag, FTGMAC100_RXDES0_RX_ERR_CHK, in
> > > FTGMAC100  driver, it is YES by default, so keep the orignal define 
> > > of RXDES0_ANY_ERROR. If it is needed, user can set the flag to NO to 
> > > remove the RXDES0_RX_ERR bit, to fix the issue.
> > >
> > > Hongwei Zhang (1):
> > >   net: ftgmac100: Fix AST2600 EVB NCSI RX issue
> > >
> > >  drivers/net/ethernet/faraday/Kconfig     | 9 +++++++++
> > >  drivers/net/ethernet/faraday/ftgmac100.h | 8 ++++++++
> > >  2 files changed, 17 insertions(+)
> > >
> > > --
> > > 2.17.1
> > >
>
Hongwei Zhang Jan. 11, 2021, 8:59 p.m. UTC | #7 
> 
> From:	Jakub Kicinski <kuba@kernel.org>
> Sent:	Monday, December 21, 2020 5:10 PM
> To:	Hongwei Zhang
> 
> On Mon, 21 Dec 2020 14:40:26 -0500 Hongwei Zhang wrote:
> > When FTGMAC100 driver is used on other NCSI Ethernet controllers, few
> 
> When you say NCSI Ethernet controller here you mean the main system NIC, right? The MAC on the NCSI 
> side is FTGMAC100, correct?
> 

Hi Jakub,

The use case for us is the MAC is configured as NCSI, so it provides network
 access for both BMC and Host, the Ethernet controller driver is from BMC kernel
side.

please see my response to Dylan in another thread, <20201215192323.24359-1-hongweiz@ami.com>,
he points out the root cause of the issue.

> In that case I'm not sure how user is supposed to control this setting at build time. The system NIC is 
> often pluggable on the PCIe bus, and can be changed at will.
> 
> > controllers have compatible issue, removing FTGMAC100_RXDES0_RX_ERR 
> > bit from RXDES0_ANY_ERROR can fix the issue.
> > 
> > Fixes: 7ee2d5b4d4340353 ("ARM: dts: nuvoton: Add Fii Kudo system")
> 
> Please fix the commit hash, this hash does not exist upstream:
> 

will do.

Thanks
--Hongwei

> Commit: 8711d4ef64fa ("net: ftgmac100: Fix AST2600 EVB NCSI RX issue")
> 	Fixes tag: Fixes: 7ee2d5b4d4340353 ("ARM: dts: nuvoton: Add Fii Kudo system")
> 	Has these problem(s):
> 		- Target SHA1 does not exist
>