Message ID | 20231001222553.3005147-1-linus.walleij@linaro.org |
---|---|
State | Superseded |
Delegated to: | Hauke Mehrtens |
Headers | show |
Series | [v2] bmips: bcm6368-enetsw: Bump max MTU | expand |
On Mon, Oct 2, 2023 at 12:25 AM Linus Walleij <linus.walleij@linaro.org> wrote: > Test with ping from host: > > PING 192.168.1.220 (192.168.1.220) 1472(1500) bytes of data. > 1480 bytes from 192.168.1.220: icmp_seq=1 ttl=64 time=0.723 ms > 1480 bytes from 192.168.1.220: icmp_seq=2 ttl=64 time=0.562 ms > 1480 bytes from 192.168.1.220: icmp_seq=3 ttl=64 time=0.555 ms > > When this ping reaches the device, the Marvell switch adds 8 bytes of DSA overhead, > and the Broadcom enetsw adds 6 bytes, resulting in exactly 1514 bytes which is what > the switch can handle. No wait this isn't correctly stated. I was too tired. OpenWrt adds a VLAN to each port so we get VLAN tags on all frames. So when I ping with ping -s 1472 -M do 192.168.1.220 (command missing) this first turns into a 1500 byte ICMP package (as seen), when it reaches the Marvell switch it adds VLAN_ETH_HLEN + VLAN_HLEN 1500 + 18 + 4 = 1522 and then 6 more bytes for DSA tag ending up with 1528 bytes. Then the Broadcom switch adds another DSA tag of 8 bytes, resulting in 1536 bytes reaching Linux, the maximum. tcpdump (on the target) gives this confirming the suspicion: 00:05:03.678271 AF Unknown (1429722180), length 1532: 0x0000: 3d93 bcae c56b a83d 8874 02fd 0004 8100 =....k.=.t...... 0x0010: 0000 dada 0000 c000 0fff 0800 4500 05dc ............E... 0x0020: 0000 4000 4001 b146 c0a8 0189 c0a8 0101 ..@.@..F........ 0x0030: 0800 1fd0 0068 001f 7231 1b65 0000 0000 .....h..r1.e.... 0x0040: f6ef 0600 0000 0000 1011 1213 1415 1617 ................ 0x0050: 1819 1a1b 1c1d 1e1f 2021 2223 2425 2627 .........!"#$%&' 0x0060: 2829 2a2b 2c2d 2e2f 3031 3233 3435 3637 ()*+,-./01234567 0x0070: 3839 3a3b 3c3d 3e3f 4041 4243 4445 4647 89:;<=>?@ABCDEFG 3d93 = First four bytes are the last two bytes of the destination ethernet address I don't know why the first four are missing, but it sure explains why the paket is 1532 bytes and not 1536. bcae c56b a83b = source ethernet address 8874 02fd 0004 = Broadcom enetsw DSA tag 8100 0000 = VLAN 802.1Q header dada 0000 c000 0fff 0800 = EDSA tag for the Marvell (outer) switch, 0800 is the ethertype obviously Next follows the contents of the ping packet as it appears if we dump it on the DSA interface such as tcpdump -i lan1 etc, there we get the stripped out packet. At the end 4 bytes of FCS. I will add this to the commit so it's clear how this works... Yours, Linus Walleij
diff --git a/target/linux/bmips/files/drivers/net/ethernet/broadcom/bcm6368-enetsw.c b/target/linux/bmips/files/drivers/net/ethernet/broadcom/bcm6368-enetsw.c index 321e95dbbb3d..01e4723c176d 100644 --- a/target/linux/bmips/files/drivers/net/ethernet/broadcom/bcm6368-enetsw.c +++ b/target/linux/bmips/files/drivers/net/ethernet/broadcom/bcm6368-enetsw.c @@ -22,7 +22,8 @@ #include <linux/reset.h> #include <linux/version.h> -/* MTU */ +/* The max frame size is 1536 including VLAN headers but excluding the 6 byte DSA tag */ +#define ENETSW_MAX_MTU (1536 - VLAN_ETH_HLEN - VLAN_HLEN) #define ENETSW_TAG_SIZE (6 + VLAN_HLEN) #define ENETSW_MTU_OVERHEAD (VLAN_ETH_HLEN + VLAN_HLEN + \ ENETSW_TAG_SIZE) @@ -1067,7 +1068,7 @@ static int bcm6368_enetsw_probe(struct platform_device *pdev) ndev->netdev_ops = &bcm6368_enetsw_ops; ndev->min_mtu = ETH_ZLEN; ndev->mtu = ETH_DATA_LEN + ENETSW_TAG_SIZE; - ndev->max_mtu = ETH_DATA_LEN + ENETSW_TAG_SIZE; + ndev->max_mtu = ENETSW_MAX_MTU; #if LINUX_VERSION_CODE >= KERNEL_VERSION(6,1,0) netif_napi_add(ndev, &priv->napi, bcm6368_enetsw_poll); #else
The max frame size for this ethernet switch is 1536 bytes, excluding the VLAN headers, which is 1536 - VLAN_ETH_HLEN - VLAN_HLEN = 1536 - 18 - 4 = 1514 not ETH_DATA_LEN + ENETSW_TAG_SIZE = 1500 + 4 + 6 = 1510 as it is right now. The available overhead is needed when using the DSA switch with a cascaded Marvell DSA switch, which is something that exist in real products, in this case the Inteno XG6846. Before this patch (on the lan1 DSA port in this case): dsa_slave_change_mtu: master->max_mtu = 9724, dev->max_mtu = 10218, DSA overhead = 8 dsa_slave_change_mtu: master = extsw, dev = lan1 dsa_slave_change_mtu: master->max_mtu = 1510, dev->max_mtu = 9724, DSA overhead = 6 dsa_slave_change_mtu: master = eth0, dev = extsw dsa_slave_change_mtu new_master_mtu 1514 > mtu_limit 1510 mdio_mux-0.1:00: nonfatal error -34 setting MTU to 1500 on port 0 My added debug prints before the nonfatal error: the first switch from the top is the Marvell switch, the second in the bcm6368-enetsw with its 1510 limit. After this patch the error is gone. Test with ping from host: PING 192.168.1.220 (192.168.1.220) 1472(1500) bytes of data. 1480 bytes from 192.168.1.220: icmp_seq=1 ttl=64 time=0.723 ms 1480 bytes from 192.168.1.220: icmp_seq=2 ttl=64 time=0.562 ms 1480 bytes from 192.168.1.220: icmp_seq=3 ttl=64 time=0.555 ms When this ping reaches the device, the Marvell switch adds 8 bytes of DSA overhead, and the Broadcom enetsw adds 6 bytes, resulting in exactly 1514 bytes which is what the switch can handle. Tests with larger MTUs have not been successful so this is likely the actual hardware max MTU. Cc: Álvaro Fernández Rojas <noltari@gmail.com> Cc: Jonas Gorski <jonas.gorski@gmail.com> Signed-off-by: Linus Walleij <linus.walleij@linaro.org> --- ChangeLog v1->v2: - Do some better research after help on IRC, did some ping tests. --- .../files/drivers/net/ethernet/broadcom/bcm6368-enetsw.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)