| Message ID | 49eec816-9238-c893-0860-602aa8965515@bell.net |
|---|---|
| State | RFC |
| Delegated to: | David Miller |
| Headers | show |
| Series | net: phylink: dsa: mv88e6xxx: flaky link detection on switch ports with internal PHYs | expand |
On Tue, Jan 22, 2019 at 02:16:09PM -0500, John David Anglin wrote: > I've been hacking on a espressobin board to try to improve ptp support, > etc. However, I have > a big problem with link detection on the wan, lan0 and lan1 ports. > > I have a standard bridge configuration using systemd-networkd. Hi John Does it make a difference if you do it by hand? Bring up the master interface, wan, lan0, lan1, add any bridge you need, etc. > From power on, none of the wan, lan0, lan1 or br0 achieve link > (LOWER_UP). Is the master interface up? What does ip link show give? Andrew
Hi Andrew, On 1/22/2019 3:28 PM, Andrew Lunn wrote: > Does it make a difference if you do it by hand? Bring up the master > interface, wan, lan0, lan1, add any bridge you need, etc. > >> From power on, none of the wan, lan0, lan1 or br0 achieve link >> (LOWER_UP). I can explore this but I don't know at the moment. I believe that all interfaces are configured. In serdes.c, interrupts are used to monitor link changes. However, phy.c doesn't do this and it doesn't call phylink_mac_change(). > Is the master interface up? What does ip link show give? The master interface (eth0 on mvneta) always comes up. The link status of this interface appears to be checked by polling at 1 second intervals. After a power on boot, this is what I see: root@espressobin:~# ip link show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: bond0: <BROADCAST,MULTICAST,MASTER> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether 96:80:ba:01:e9:31 brd ff:ff:ff:ff:ff:ff 3: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether 46:42:f8:2b:d1:4e brd ff:ff:ff:ff:ff:ff 4: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1024 link/ether 3e:c0:66:0a:db:ba brd ff:ff:ff:ff:ff:ff 5: wan@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br0 state LOWERLAYERDOWN mode DEFAULT group default qlen 1000 link/ether 3e:c0:66:0a:db:ba brd ff:ff:ff:ff:ff:ff 6: lan0@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br0 state LOWERLAYERDOWN mode DEFAULT group default qlen 1000 link/ether 3e:c0:66:0a:db:ba brd ff:ff:ff:ff:ff:ff 7: lan1@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br0 state LOWERLAYERDOWN mode DEFAULT group default qlen 1000 link/ether 3e:c0:66:0a:db:ba brd ff:ff:ff:ff:ff:ff 8: br0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 link/ether 00:50:43:25:fb:84 brd ff:ff:ff:ff:ff:ff root@espressobin:~# networkctl IDX LINK TYPE OPERATIONAL SETUP 1 lo loopback carrier unmanaged 2 bond0 bond off unmanaged 3 dummy0 ether off unmanaged 4 eth0 ether degraded configured 5 wan dsa no-carrier configuring 6 lan0 dsa no-carrier configuring 7 lan1 dsa no-carrier configuring 8 br0 bridge no-carrier configuring 8 links listed. r0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 ether 00:50:43:25:fb:84 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::3cc0:66ff:fe0a:dbba prefixlen 64 scopeid 0x20<link> ether 3e:c0:66:0a:db:ba txqueuelen 1024 (Ethernet) RX packets 2229 bytes 160191 (156.4 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 14 bytes 1116 (1.0 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 11 lan0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 ether 3e:c0:66:0a:db:ba txqueuelen 1000 (Ethernet) RX packets 1075 bytes 55969 (54.6 KiB) RX errors 0 dropped 551 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lan1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 ether 3e:c0:66:0a:db:ba txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 wan: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 ether 3e:c0:66:0a:db:ba txqueuelen 1000 (Ethernet) RX packets 1154 bytes 55184 (53.8 KiB) RX errors 0 dropped 14 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 Then, if I do a "shutdown -r +0", I see: root@espressobin:~# ip link show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: bond0: <BROADCAST,MULTICAST,MASTER> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether 26:dd:a4:7c:26:3f brd ff:ff:ff:ff:ff:ff 3: dummy0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether 0a:73:61:5a:ae:dc brd ff:ff:ff:ff:ff:ff 4: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1024 link/ether f6:2d:a2:ab:32:4d brd ff:ff:ff:ff:ff:ff 5: wan@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP mode DEFAULT group default qlen 1000 link/ether f6:2d:a2:ab:32:4d brd ff:ff:ff:ff:ff:ff 6: lan0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP mode DEFAULT group default qlen 1000 link/ether f6:2d:a2:ab:32:4d brd ff:ff:ff:ff:ff:ff 7: lan1@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br0 state LOWERLAYERDOWN mode DEFAULT group default qlen 1000 link/ether f6:2d:a2:ab:32:4d brd ff:ff:ff:ff:ff:ff 8: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether 00:50:43:25:fb:84 brd ff:ff:ff:ff:ff:ff root@espressobin:~# networkctl IDX LINK TYPE OPERATIONAL SETUP 1 lo loopback carrier unmanaged 2 bond0 bond off unmanaged 3 dummy0 ether off unmanaged 4 eth0 ether degraded configured 5 wan dsa degraded configured 6 lan0 dsa degraded configured 7 lan1 dsa no-carrier configuring 8 br0 bridge routable configured 8 links listed. root@espressobin:~# ifconfig br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.0.55 netmask 255.255.255.0 broadcast 192.168.0.255 inet6 fe80::250:43ff:fe25:fb84 prefixlen 64 scopeid 0x20<link> ether 00:50:43:25:fb:84 txqueuelen 1000 (Ethernet) RX packets 1184 bytes 237936 (232.3 KiB) RX errors 0 dropped 261 overruns 0 frame 0 TX packets 193 bytes 19444 (18.9 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::f42d:a2ff:feab:324d prefixlen 64 scopeid 0x20<link> ether f6:2d:a2:ab:32:4d txqueuelen 1024 (Ethernet) RX packets 1331 bytes 276209 (269.7 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 233 bytes 24352 (23.7 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 11 lan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::f42d:a2ff:feab:324d prefixlen 64 scopeid 0x20<link> ether f6:2d:a2:ab:32:4d txqueuelen 1000 (Ethernet) RX packets 264 bytes 13773 (13.4 KiB) RX errors 0 dropped 135 overruns 0 frame 0 TX packets 18 bytes 1368 (1.3 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lan1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500 ether f6:2d:a2:ab:32:4d txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 wan: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::f42d:a2ff:feab:324d prefixlen 64 scopeid 0x20<link> ether f6:2d:a2:ab:32:4d txqueuelen 1000 (Ethernet) RX packets 1067 bytes 233154 (227.6 KiB) RX errors 0 dropped 4 overruns 0 frame 0 TX packets 203 bytes 20240 (19.7 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 Everything seems normal. The wan and lan0 ports are connected. From syslog for p2 (lan0): Jan 22 16:02:34 localhost kernel: [ 5.249693] mv88e6085 d0032004.mdio-mii:01: p2: PortState set to Disabled Jan 22 16:02:34 localhost kernel: [ 5.394717] mv88e6085 d0032004.mdio-mii:01: p2: Force link down Jan 22 16:02:34 localhost kernel: [ 5.400554] mv88e6085 d0032004.mdio-mii:01: p2: Speed unforced Jan 22 16:02:34 localhost kernel: [ 5.406558] mv88e6085 d0032004.mdio-mii:01: p2: Unforce half duplex Jan 22 16:02:34 localhost kernel: [ 5.413007] mv88e6085 d0032004.mdio-mii:01: p2: Unforce link down Jan 22 16:02:34 localhost kernel: [ 5.421850] mv88e6085 d0032004.mdio-mii:01: p2: 802.1QMode set to Disabled Jan 22 16:02:34 localhost kernel: [ 5.430197] mv88e6085 d0032004.mdio-mii:01: p2: FID set to 0 Jan 22 16:02:34 localhost kernel: [ 5.433696] mv88e6085 d0032004.mdio-mii:01: p2: VLANTable set to 001 Jan 22 16:02:34 localhost kernel: [ 6.136188] mv88e6085 d0032004.mdio-mii:01 lan0 (uninitialized): PHY [!soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:12] driver [Marvell 88E6390] Jan 22 16:02:34 localhost kernel: [ 8.890947] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this. Jan 22 16:02:34 localhost systemd-networkd[358]: lan0: Joined netdev Jan 22 16:02:34 localhost systemd-networkd[358]: lan0: Bringing link up Jan 22 16:02:34 localhost systemd-networkd[358]: lan0: Flags change: +UP Jan 22 16:02:34 localhost systemd-networkd[358]: lan0: Set link Jan 22 16:02:34 localhost kernel: [ 9.017442] br0: port 2(lan0) entered blocking state Jan 22 16:02:34 localhost kernel: [ 9.017478] br0: port 2(lan0) entered disabled state Jan 22 16:02:34 localhost kernel: [ 9.018314] mv88e6085 d0032004.mdio-mii:01: p2: VLANTable set to 009 Jan 22 16:02:34 localhost kernel: [ 9.021257] device lan0 entered promiscuous mode Jan 22 16:02:34 localhost kernel: [ 9.033658] mv88e6085 d0032004.mdio-mii:01: p2: VLANTable set to 00b an 22 16:02:34 localhost kernel: [ 9.138112] mv88e6085 d0032004.mdio-mii:01: p2: PortState set to Blocking/Listening Jan 22 16:02:34 localhost kernel: [ 9.138149] mv88e6085 d0032004.mdio-mii:01 lan0: configuring for phy/ link mode Jan 22 16:02:34 localhost kernel: [ 9.138158] mv88e6085 d0032004.mdio-mii:01: port=2, mode=0, link=0 Jan 22 16:02:34 localhost kernel: [ 9.138267] mv88e6085 d0032004.mdio-mii:01: port=2, mode=0, link=0 Jan 22 16:02:34 localhost kernel: [ 9.138274] mv88e6085 d0032004.mdio-mii:01 lan0: Link is Down - mac_link_dropped=0 Jan 22 16:02:34 localhost kernel: [ 9.138885] IPv6: ADDRCONF(NETDEV_UP): lan0: link is not ready Jan 22 16:02:34 localhost kernel: [ 9.213888] mv88e6085 d0032004.mdio-mii:01: port=2, mode=0, link=1 Jan 22 16:02:34 localhost kernel: [ 9.213919] mv88e6085 d0032004.mdio-mii:01 lan0: Link is Up - 1Gbps/Full - flow control rx/tx Jan 22 16:02:34 localhost kernel: [ 9.213962] IPv6: ADDRCONF(NETDEV_CHANGE): lan0: link becomes ready Jan 22 16:02:34 localhost kernel: [ 11.356713] br0: port 2(lan0) entered blocking state Jan 22 16:02:34 localhost kernel: [ 11.356719] br0: port 2(lan0) entered forwarding state Jan 22 16:02:34 localhost kernel: [ 11.359228] mv88e6085 d0032004.mdio-mii:01: p2: PortState set to Blocking/Listening Jan 22 16:02:34 localhost kernel: [ 11.360339] mv88e6085 d0032004.mdio-mii:01: p2: PortState set to Forwarding Jan 22 16:02:34 localhost systemd-networkd[358]: lan0: Flags change: +LOWER_UP +RUNNING Jan 22 16:02:34 localhost systemd-networkd[358]: lan0: Gained carrier Jan 22 16:02:34 localhost systemd-networkd[358]: lan0: Configured
On Tue, Jan 22, 2019 at 04:40:27PM -0500, John David Anglin wrote: > Hi Andrew, > > On 1/22/2019 3:28 PM, Andrew Lunn wrote: > >Does it make a difference if you do it by hand? Bring up the master > >interface, wan, lan0, lan1, add any bridge you need, etc. > > > >> From power on, none of the wan, lan0, lan1 or br0 achieve link > >>(LOWER_UP). > I can explore this but I don't know at the moment. I believe that all > interfaces are configured. Hi John In what order? The master interface has to be up first before any slave interface is configured up. > In serdes.c, interrupts are used to monitor link changes. However, phy.c > doesn't do this and > it doesn't call phylink_mac_change(). It does not need to. There are two options here: 1) The PHY has no interrupt. phylib will poll the PHY once per second for link changes. 2) The PHY has in interrupt. Link changes will cause the interrupt to fire, and the phylib will then read the current state. For PHYs embedded within a switch driver by mv88e6xxx interrupts should always be used. I will do some testing on my espressobin. Andrew
On Tue, Jan 22, 2019 at 02:16:09PM -0500, John David Anglin wrote: > I've been hacking on a espressobin board to try to improve ptp support, > etc. However, I have > a big problem with link detection on the wan, lan0 and lan1 ports. Hi John I just booted my espressobin with net-next. It is running Debian, and i have the following in /etc/network/interfaces source /etc/network/interfaces.d/* # The loopback network interface auto lo iface lo inet loopback # The primary network interface allow-hotplug wan iface wan inet dhcp pre-up ip link set eth0 up allow-hotplug lan0 iface lan0 inet static pre-up ip link set eth0 up address 10.42.42.42 netmask 255.255.255.0 my wan port got its IP address from DHCP. root@espressobin:~# ip addr show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group defaul t qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1508 qdisc mq state UP group defa ult qlen 1024 link/ether f0:ad:4e:03:69:9c brd ff:ff:ff:ff:ff:ff inet6 fe80::f2ad:4eff:fe03:699c/64 scope link valid_lft forever preferred_lft forever 3: wan@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP g roup default qlen 1000 link/ether f0:ad:4e:03:69:9c brd ff:ff:ff:ff:ff:ff inet 10.0.0.11/24 brd 10.0.0.255 scope global wan valid_lft forever preferred_lft forever inet6 fe80::f2ad:4eff:fe03:699c/64 scope link valid_lft forever preferred_lft forever 4: lan0@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state L OWERLAYERDOWN group default qlen 1000 link/ether f0:ad:4e:03:69:9c brd ff:ff:ff:ff:ff:ff inet 10.42.42.42/24 brd 10.42.42.255 scope global lan0 valid_lft forever preferred_lft forever 5: lan1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether f0:ad:4e:03:69:9c brd ff:ff:ff:ff:ff:ff lan0 is correctly down, because the machine on the other end is down. I then manually configured lan1 up and powered on the peer. I then see: [ 543.113227] IPv6: ADDRCONF(NETDEV_UP): lan1: link is not ready [ 546.680276] mv88e6085 d0032004.mdio-mii:01 lan1: Link is Up - 1Gbps/Full - fl ow control off [ 546.686106] IPv6: ADDRCONF(NETDEV_CHANGE): lan1: link becomes ready So for me, everything is working as it should. I would suggest you manually configure your networking, just to test. I would suspect systemd is not doing things correctly. Andrew
On 2019-01-22 6:12 p.m., Andrew Lunn wrote: > I would suggest you manually configure your networking, just to > test. I would suspect systemd is not doing things correctly. I have armbian with debian stretch installed. I did find that both systemd-networkd and networkmanager were enabled. I disabled the networkmanager service. However, DHCP on br0 didn't always get an IP. It was after disabling networkmanager that the ports didn't come up on boot. Dave
On 2019-01-22 5:36 p.m., Andrew Lunn wrote: > In what order? The master interface has to be up first before any > slave interface is configured up. Possibly, that's a clue. If I recall, that's determined by the naming on the .network files in /etc/systemd/network. The slave interfaces are coming up in reverse order: lan1->lan0->wan. Need to check eth0 relative to the others. Dave
On 2019-01-22 6:12 p.m., Andrew Lunn wrote: > I just booted my espressobin with net-next. It is running Debian, and > i have the following in /etc/network/interfaces > > source /etc/network/interfaces.d/* > > # The loopback network interface > auto lo > iface lo inet loopback > > # The primary network interface > allow-hotplug wan > iface wan inet dhcp > pre-up ip link set eth0 up > > allow-hotplug lan0 > iface lan0 inet static > pre-up ip link set eth0 up > address 10.42.42.42 > netmask 255.255.255.0 > > my wan port got its IP address from DHCP. Your configuration is different. I have the wan, lan0 and lan1 ports configured as a bridge and they don't get IP addresses. The only port that gets an IP is br0. Dave
On 1/22/19 4:00 PM, John David Anglin wrote: > On 2019-01-22 6:12 p.m., Andrew Lunn wrote: >> I just booted my espressobin with net-next. It is running Debian, and >> i have the following in /etc/network/interfaces >> >> source /etc/network/interfaces.d/* >> >> # The loopback network interface >> auto lo >> iface lo inet loopback >> >> # The primary network interface >> allow-hotplug wan >> iface wan inet dhcp >> pre-up ip link set eth0 up >> >> allow-hotplug lan0 >> iface lan0 inet static >> pre-up ip link set eth0 up >> address 10.42.42.42 >> netmask 255.255.255.0 >> >> my wan port got its IP address from DHCP. > Your configuration is different. I have the wan, lan0 and lan1 ports > configured as a bridge and they > don't get IP addresses. The only port that gets an IP is br0. If you bridge all devices together that's expected. If you leave the network devices interfaces outside of the bridge, then they act as individual/standalone network ports, and that is where a DHCP client would be running. The point Andrew as trying to make is that if you make sure you always bring-up the DSA master network device, typically ethX first, and then the DSA network interfaces that are per-port, then you should not have any issues with link being UP/DOWN. Whether IP stack works is a different thing.
On 2019-01-22 5:36 p.m., Andrew Lunn wrote: > It does not need to. There are two options here: > > 1) The PHY has no interrupt. phylib will poll the PHY once per second > for link changes. > > 2) The PHY has in interrupt. Link changes will cause the interrupt to > fire, and the phylib will then read the current state. > > For PHYs embedded within a switch driver by mv88e6xxx interrupts > should always be used. I don't think option 2) is implemented. Didn't see any irq code in phy.c. If I remember correctly, one needs to use clause 45 accesses to get at the PHY registers in the 88E6341. Can you point me to the phylib code that does the polling? Thanks, Dave
> > It does not need to. There are two options here: > > > > 1) The PHY has no interrupt. phylib will poll the PHY once per second > > for link changes. > > > > 2) The PHY has in interrupt. Link changes will cause the interrupt to > > fire, and the phylib will then read the current state. > > > > For PHYs embedded within a switch driver by mv88e6xxx interrupts > > should always be used. Hi Dave From my Espressobin cat /proc/interrupts ... 44: 0 0 mv88e6xxx-g1 3 Edge mv88e6xxx-g1-atu-prob 46: 0 0 mv88e6xxx-g1 5 Edge mv88e6xxx-g1-vtu-prob 48: 38 24 mv88e6xxx-g1 7 Edge mv88e6xxx-g2 51: 0 1 mv88e6xxx-g2 1 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:11 52: 0 0 mv88e6xxx-g2 2 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:12 53: 38 23 mv88e6xxx-g2 3 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:13 These are PHY interrupts. > I don't think option 2) is implemented. Didn't see any irq code in phy.c. You would not. All the interrupt code is in the PHY core and the PHY driver. drivers/net/dsa/mv88e6xxx/phy.c is just a bunch of helpers which allow the mdio bus driver to access phy registers. The PHY driver itself is drivers/net/phy/marvell.c, and the interrupt handling is spread between that and drivers/net/phy/phy.c > If I remember correctly, one needs to use clause 45 accesses to get at > the PHY registers in the 88E6341. Nope. The PHYs are c22 devices. The SERDES are probably C45, but those are not being used here. Andrew
On 2019-01-22 7:22 p.m., Andrew Lunn wrote: > >From my Espressobin > > cat /proc/interrupts > ... > 44: 0 0 mv88e6xxx-g1 3 Edge mv88e6xxx-g1-atu-prob > 46: 0 0 mv88e6xxx-g1 5 Edge mv88e6xxx-g1-vtu-prob > 48: 38 24 mv88e6xxx-g1 7 Edge mv88e6xxx-g2 > 51: 0 1 mv88e6xxx-g2 1 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:11 > 52: 0 0 mv88e6xxx-g2 2 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:12 > 53: 38 23 mv88e6xxx-g2 3 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:13 > > These are PHY interrupts. > Thanks. That was the clue. I had tried to enable hardware for support for switch interrupts. However, the espressobin connects the switch interrupt output to a southbridge pin that only supports edge interrupts. Link detection works okay once the attached change is removed. I may try and see if moving the interrupt to a northbridge pin works. I have to wonder about the use of edge interrupts. Given that a GPIO interrupt isn't defined for the mv88e6xxx, then the interrupt must be done by polling the global1 control register. This also makes me wonder about this SD interrupt: 43: 0 0 GPIO1 3 Edge d00d0000.sdhci cd Many poeple have trouble with SD cards on reboot. Although the driver doesn't handle AVB interrupts, I started down this path to try get ptp4l working better. I have tweaked the polling but there are still circumstances where timestamps are overwritten (or not written). Dave
On Fri, Jan 25, 2019 at 11:30:54AM -0500, John David Anglin wrote: > This also makes me wonder about this SD interrupt: > 43: 0 0 GPIO1 3 Edge d00d0000.sdhci cd > Many poeple have trouble with SD cards on reboot. [off topic, but a relevant reply to the above] Very likely unrelated to edge interrupts. For a signal that goes low when a card is inserted and high when removed, you really need an edge interrupt to avoid flooding the CPU with a "stuck" interrupt - if you were to use an active low interrupt for card detection, and the signal is pulled low by card insertion, the interrupt remains active until you remove the card! In any case, I've seen the same with some early SolidRun platforms when using high speed modes. Most issues with SD cards at reboot is where the card is left in low voltage/high speed mode, the reboot happens, and then you try to talk to it in low-speed mode (as required by the specs for card detection/ initialisation) and they don't respond. The only way out of that is to reset the card - and the only way to reset the card is to remove power from it. If you're lucky enough to have a "regulator" which can be switched off at reboot, that may not be sufficient - if there's a capacitor on the SD card's supply line downstream from the regulator, the card can be drawing soo little current that the voltage doesn't fall sufficiently to cause the card to power down. If you don't have any means to reset the card like that, then the only workaround is to reduce the maximum speed - and avoid going to 1.8V signalling mode.
On 2019-01-25 11:48 a.m., Russell King - ARM Linux admin wrote: > If you don't have any means to reset the card like that, then the only > workaround is to reduce the maximum speed - and avoid going to 1.8V > signalling mode. There is no power controller on espressobin. The regulator only switches the I/O voltage. The attached change fixed reboot on espressobin with problematic SD card. Thanks, Dave
On 2019-01-22 7:22 p.m., Andrew Lunn wrote: > >From my Espressobin > > cat /proc/interrupts > ... > 44: 0 0 mv88e6xxx-g1 3 Edge mv88e6xxx-g1-atu-prob > 46: 0 0 mv88e6xxx-g1 5 Edge mv88e6xxx-g1-vtu-prob > 48: 38 24 mv88e6xxx-g1 7 Edge mv88e6xxx-g2 > 51: 0 1 mv88e6xxx-g2 1 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:11 > 52: 0 0 mv88e6xxx-g2 2 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:12 > 53: 38 23 mv88e6xxx-g2 3 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:13 > > These are PHY interrupts. If we come back to my trying to use the INTn pin on the esspressobin, I have found that clearing and resetting the interrupt enable bits in the global control register (offset 0x4) restarts link detection when the device is stuck. This suggests that the INTn connection to MPP2_23 is low when the the GIC interrupt is enabled on this pin. Possibly, this is caused by the fact that EEIntEn is set to 1 on reset. INTn then goes low when EEPROM loading is done. Another possibility might be race conditions in processing interrupts. Thoughts? Dave
On Wed, Jan 30, 2019 at 12:08:39PM -0500, John David Anglin wrote: > On 2019-01-22 7:22 p.m., Andrew Lunn wrote: > > >From my Espressobin > > > > cat /proc/interrupts > > ... > > 44: 0 0 mv88e6xxx-g1 3 Edge mv88e6xxx-g1-atu-prob > > 46: 0 0 mv88e6xxx-g1 5 Edge mv88e6xxx-g1-vtu-prob > > 48: 38 24 mv88e6xxx-g1 7 Edge mv88e6xxx-g2 > > 51: 0 1 mv88e6xxx-g2 1 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:11 > > 52: 0 0 mv88e6xxx-g2 2 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:12 > > 53: 38 23 mv88e6xxx-g2 3 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:13 > > > > These are PHY interrupts. > If we come back to my trying to use the INTn pin on the esspressobin, I > have found that clearing and resetting the interrupt > enable bits in the global control register (offset 0x4) restarts link > detection when the device is stuck. This suggests that the > INTn connection to MPP2_23 is low when the the GIC interrupt is enabled > on this pin. Possibly, this is caused by the fact > that EEIntEn is set to 1 on reset. INTn then goes low when EEPROM > loading is done. Another possibility might be race conditions > in processing interrupts. > > Thoughts? Hi David You need active low interrupts. Without it, i think you are always going to have race conditions which will cause interrupts to get stuck/lost. I would suggest you remove the interrupt from your device tree and use the mv88e6xxx polling method. If i remember correctly, it currently polls 10 per second, so PHY link up/down is going to be 5 times faster on average than when phylib is polling the PHY. Andrew
On 2019-01-30 12:28 p.m., Andrew Lunn wrote: > On Wed, Jan 30, 2019 at 12:08:39PM -0500, John David Anglin wrote: >> On 2019-01-22 7:22 p.m., Andrew Lunn wrote: >>> >From my Espressobin >>> >>> cat /proc/interrupts >>> ... >>> 44: 0 0 mv88e6xxx-g1 3 Edge mv88e6xxx-g1-atu-prob >>> 46: 0 0 mv88e6xxx-g1 5 Edge mv88e6xxx-g1-vtu-prob >>> 48: 38 24 mv88e6xxx-g1 7 Edge mv88e6xxx-g2 >>> 51: 0 1 mv88e6xxx-g2 1 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:11 >>> 52: 0 0 mv88e6xxx-g2 2 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:12 >>> 53: 38 23 mv88e6xxx-g2 3 Edge !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:13 >>> >>> These are PHY interrupts. >> If we come back to my trying to use the INTn pin on the esspressobin, I >> have found that clearing and resetting the interrupt >> enable bits in the global control register (offset 0x4) restarts link >> detection when the device is stuck. This suggests that the >> INTn connection to MPP2_23 is low when the the GIC interrupt is enabled >> on this pin. Possibly, this is caused by the fact >> that EEIntEn is set to 1 on reset. INTn then goes low when EEPROM >> loading is done. Another possibility might be race conditions >> in processing interrupts. >> >> Thoughts? > Hi David > > You need active low interrupts. Without it, i think you are always > going to have race conditions which will cause interrupts to get > stuck/lost. > > I would suggest you remove the interrupt from your device tree and use > the mv88e6xxx polling method. If i remember correctly, it currently > polls 10 per second, so PHY link up/down is going to be 5 times faster > on average than when phylib is polling the PHY. Hi Andrew, The main motivation in doing this is to try to enable the AVB interrupt and to improve the PTP support. I agree that polling is perfectly fine for PHY link interrupts. Possibly, ATU and VTU might need faster support but I'm not using that at the moment. I have hacked on the time stamp code quit a bit to try and improve things but there are still issues with lost or overwritten time stamps: Jan 28 11:15:05 localhost kernel: [234850.840883] mv88e6085 d0032004.mdio-mii:01 : timestamp discarded Jan 28 11:15:05 localhost ptp4l: [234852.998] port 3: received SYNC without time stamp I think when PTP packets other than PDELAY are too closely spaced, we have problems accessing the timestamp quick enough. Also, timestamp access is dependent on CPU speed and HZ. It looks like I can easily connect MPP2_23 to MPP1_16 on the edge connector P8. I believe the northbridge pins support level interrupts. In /proc/interrupts, the switch interrupts are shown as edge. The only place that I see where this is potentially set is mv88e6xxx_g2_watchdog_setup() where the call to request_threaded_irq() passes "IRQF_ONESHOT | IRQF_TRIGGER_FALLING". Does this need to change? Dave
> In /proc/interrupts, the switch interrupts are shown as edge. The only > place that I see where this > is potentially set is mv88e6xxx_g2_watchdog_setup() where the call to > request_threaded_irq() passes > "IRQF_ONESHOT | IRQF_TRIGGER_FALLING". Does this need to change? Hi Dave Device tree determines how the interrupt is configured. It is the second part of the interrupts property. Andrew
On 2019-01-30 12:28 p.m., Andrew Lunn wrote: > You need active low interrupts. Without it, i think you are always > going to have race conditions which will cause interrupts to get > stuck/lost. I don't know if this is a hardware limitation or not, but currently the armada 37xx doesn't support level interrupts: [ 4.013280] genirq: Setting trigger mode 8 for irq 44 failed (armada_37xx_irq_set_type+0x0/0x158) [ 4.014075] mv88e6085: probe of d0032004.mdio-mii:01 failed with error -22 The function armada_37xx_irq_set_type() only supports edge interrupts. On the plus side, DTC no longer objects to level interrupts on southbridge. Dave
On Wed, Jan 30, 2019 at 05:24:53PM -0500, John David Anglin wrote: > On 2019-01-30 12:28 p.m., Andrew Lunn wrote: > > You need active low interrupts. Without it, i think you are always > > going to have race conditions which will cause interrupts to get > > stuck/lost. > I don't know if this is a hardware limitation or not, but currently the > armada 37xx doesn't support > level interrupts: Yes, i had a quick look at the pinctrl driver. It only has code for edges. I'd suggest you take a look at the datasheet for the 37xx and check what the hardware actually supports. You might need to extend the driver. Andrew
On 2019-01-30 5:38 p.m., Andrew Lunn wrote: > I'd suggest you take a look at the datasheet for the 37xx and check > what the hardware actually supports. You might need to extend the > driver. I did look and the GIC does support level interrupts. But all the documentation is in generic ARM documents that I don't currently have. I'll see if I can find them tomorrow. Dave
On 2019-01-30 8:27 p.m., John David Anglin wrote: > On 2019-01-30 5:38 p.m., Andrew Lunn wrote: >> I'd suggest you take a look at the datasheet for the 37xx and check >> what the hardware actually supports. You might need to extend the >> driver. > I did look and the GIC does support level interrupts. But all the > documentation is in > generic ARM documents that I don't currently have. I'll see if I can > find them tomorrow. On a closer look at MV-S110897-00C, I see that the north and south bridge GPIO interrupt registers only provide edge polarity control. The GPIO pins don't appear to support level interrupts on 88F37xx. Dave
diff --git a/drivers/net/phy/phylink.c b/drivers/net/phy/phylink.c index 9b8dd0d0ee42..c1ec13b320ee 100644 --- a/drivers/net/phy/phylink.c +++ b/drivers/net/phy/phylink.c @@ -405,6 +405,7 @@ static void phylink_resolve(struct work_struct *w) case MLO_AN_PHY: link_state = pl->phy_state; phylink_resolve_flow(pl, &link_state); + phylink_get_mac_state(pl, &link_state); phylink_mac_config(pl, &link_state); break;