| Message ID | 20180131031534.25166-3-dja@axtens.net |
|---|---|
| State | Accepted, archived |
| Delegated to: | David Miller |
| Headers | show |
| Series | bnx2x: disable GSO on too-large packets | expand |
> -----Original Message----- > From: Daniel Axtens [mailto:dja@axtens.net] > Sent: Wednesday, January 31, 2018 8:46 AM > To: netdev@vger.kernel.org > Cc: Daniel Axtens <dja@axtens.net>; Eric Dumazet <eric.dumazet@gmail.com>; > Chopra, Manish <Manish.Chopra@cavium.com>; Jason Wang > <jasowang@redhat.com>; Pravin Shelar <pshelar@ovn.org>; Marcelo Ricardo > Leitner <marcelo.leitner@gmail.com> > Subject: [PATCH v4 2/2] bnx2x: disable GSO where gso_size is too big for > hardware > > If a bnx2x card is passed a GSO packet with a gso_size larger than > ~9700 bytes, it will cause a firmware error that will bring the card > down: > > bnx2x: [bnx2x_attn_int_deasserted3:4323(enP24p1s0f0)]MC assert! > bnx2x: [bnx2x_mc_assert:720(enP24p1s0f0)]XSTORM_ASSERT_LIST_INDEX 0x2 > bnx2x: [bnx2x_mc_assert:736(enP24p1s0f0)]XSTORM_ASSERT_INDEX 0x0 = > 0x00000000 0x25e43e47 0x00463e01 0x00010052 > bnx2x: [bnx2x_mc_assert:750(enP24p1s0f0)]Chip Revision: everest3, FW > Version: 7_13_1 ... (dump of values continues) ... > > Detect when the mac length of a GSO packet is greater than the maximum > packet size (9700 bytes) and disable GSO. > > Signed-off-by: Daniel Axtens <dja@axtens.net> > > --- > > v4: Only call the slow check if the gso_size is large. > Eric - I think this is what you had in mind? > Manish - do you think this is an acceptable performance trade-off? > GSO will work for any packet size, and only jumbo frames will > have to do the slower test. > > Again, only build-tested. > --- > drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 18 > ++++++++++++++++++ > 1 file changed, 18 insertions(+) > > diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c > b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c > index 7b08323e3f3d..74fc9af4aadb 100644 > --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c > +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c > @@ -12934,6 +12934,24 @@ static netdev_features_t > bnx2x_features_check(struct sk_buff *skb, > struct net_device *dev, > netdev_features_t features) > { > + /* > + * A skb with gso_size + header length > 9700 will cause a > + * firmware panic. Drop GSO support. > + * > + * Eventually the upper layer should not pass these packets down. > + * > + * For speed, if the gso_size is <= 9000, assume there will > + * not be 700 bytes of headers and pass it through. Only do a > + * full (slow) validation if the gso_size is > 9000. > + * > + * (Due to the way SKB_BY_FRAGS works this will also do a full > + * validation in that case.) > + */ > + if (unlikely(skb_is_gso(skb) && > + (skb_shinfo(skb)->gso_size > 9000) && > + !skb_gso_validate_mac_len(skb, 9700))) > + features &= ~NETIF_F_GSO_MASK; Hi Daniel, Obviously, it could be bad from performance perspective since every gso packet has to do these check. When running iperf/netperf performance benchmark, where GSO is likely to occur. Why do you have to put two checks for skb_is_gso() and gso_size ? Isn't gso_size > anything means GSO skb ? I assume it won't cause disabling the offload if "headers [L2 + L3 + L4] + gso_size" is still <= 9700. ? Thanks, Manish
"Chopra, Manish" <Manish.Chopra@cavium.com> writes: >> -----Original Message----- >> From: Daniel Axtens [mailto:dja@axtens.net] >> Sent: Wednesday, January 31, 2018 8:46 AM >> To: netdev@vger.kernel.org >> Cc: Daniel Axtens <dja@axtens.net>; Eric Dumazet <eric.dumazet@gmail.com>; >> Chopra, Manish <Manish.Chopra@cavium.com>; Jason Wang >> <jasowang@redhat.com>; Pravin Shelar <pshelar@ovn.org>; Marcelo Ricardo >> Leitner <marcelo.leitner@gmail.com> >> Subject: [PATCH v4 2/2] bnx2x: disable GSO where gso_size is too big for >> hardware >> >> If a bnx2x card is passed a GSO packet with a gso_size larger than >> ~9700 bytes, it will cause a firmware error that will bring the card >> down: >> >> bnx2x: [bnx2x_attn_int_deasserted3:4323(enP24p1s0f0)]MC assert! >> bnx2x: [bnx2x_mc_assert:720(enP24p1s0f0)]XSTORM_ASSERT_LIST_INDEX 0x2 >> bnx2x: [bnx2x_mc_assert:736(enP24p1s0f0)]XSTORM_ASSERT_INDEX 0x0 = >> 0x00000000 0x25e43e47 0x00463e01 0x00010052 >> bnx2x: [bnx2x_mc_assert:750(enP24p1s0f0)]Chip Revision: everest3, FW >> Version: 7_13_1 ... (dump of values continues) ... >> >> Detect when the mac length of a GSO packet is greater than the maximum >> packet size (9700 bytes) and disable GSO. >> >> Signed-off-by: Daniel Axtens <dja@axtens.net> >> >> --- >> >> v4: Only call the slow check if the gso_size is large. >> Eric - I think this is what you had in mind? >> Manish - do you think this is an acceptable performance trade-off? >> GSO will work for any packet size, and only jumbo frames will >> have to do the slower test. >> >> Again, only build-tested. >> --- >> drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 18 >> ++++++++++++++++++ >> 1 file changed, 18 insertions(+) >> >> diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c >> b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c >> index 7b08323e3f3d..74fc9af4aadb 100644 >> --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c >> +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c >> @@ -12934,6 +12934,24 @@ static netdev_features_t >> bnx2x_features_check(struct sk_buff *skb, >> struct net_device *dev, >> netdev_features_t features) >> { >> + /* >> + * A skb with gso_size + header length > 9700 will cause a >> + * firmware panic. Drop GSO support. >> + * >> + * Eventually the upper layer should not pass these packets down. >> + * >> + * For speed, if the gso_size is <= 9000, assume there will >> + * not be 700 bytes of headers and pass it through. Only do a >> + * full (slow) validation if the gso_size is > 9000. >> + * >> + * (Due to the way SKB_BY_FRAGS works this will also do a full >> + * validation in that case.) >> + */ >> + if (unlikely(skb_is_gso(skb) && >> + (skb_shinfo(skb)->gso_size > 9000) && >> + !skb_gso_validate_mac_len(skb, 9700))) >> + features &= ~NETIF_F_GSO_MASK; > > Hi Daniel, > > Obviously, it could be bad from performance perspective since every gso packet has to do these check. > When running iperf/netperf performance benchmark, where GSO is likely to occur. > > Why do you have to put two checks for skb_is_gso() and gso_size ? Isn't gso_size > anything means GSO skb ? Yes, the check is redundant. I do it to make my logic clearer. The compiler will optimise it into one check. I compiled with gcc-7.2 and gcc-4.8, both targeting amd64, and in both cases I could only find one relevant cmp: skb_is_gso(): /home/dja/dev/linux/linux/include/linux/skbuff.h:4032 3686: 48 8b 8f d0 00 00 00 mov 0xd0(%rdi),%rcx bnx2x_features_check(): /home/dja/dev/linux/linux/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c:12950 368d: 66 81 7c 01 04 28 23 cmpw $0x2328,0x4(%rcx,%rax,1) 3694: 0f 87 ea 01 00 00 ja 3884 <bnx2x_features_check+0x214> 0x2328 is decimal 9000. > I assume it won't cause disabling the offload if "headers [L2 + L3 + L4] + gso_size" is still <= 9700. ? That is correct. The flow is: If the gso_size is less than or equal to 9000, offload is enabled with no further checks. This is the fast path. If the gso_size is greater than 9000, then the "headers [L2 + L3 + L4] + gso_size" is checked. This is an out-of-line check done once per GSO skb. If headers + gso_size is <= 9700, offload is enabled. If headers + gso_size > 9700, offload is disabled. Regards, Daniel > > Thanks, > Manish
On Wed, 2018-01-31 at 14:15 +1100, Daniel Axtens wrote: > If a bnx2x card is passed a GSO packet with a gso_size larger than > ~9700 bytes, it will cause a firmware error that will bring the card > down: > > bnx2x: [bnx2x_attn_int_deasserted3:4323(enP24p1s0f0)]MC assert! > bnx2x: [bnx2x_mc_assert:720(enP24p1s0f0)]XSTORM_ASSERT_LIST_INDEX 0x2 > bnx2x: [bnx2x_mc_assert:736(enP24p1s0f0)]XSTORM_ASSERT_INDEX 0x0 = 0x00000000 0x25e43e47 0x00463e01 0x00010052 > bnx2x: [bnx2x_mc_assert:750(enP24p1s0f0)]Chip Revision: everest3, FW Version: 7_13_1 > ... (dump of values continues) ... > > Detect when the mac length of a GSO packet is greater than the maximum > packet size (9700 bytes) and disable GSO. > > Signed-off-by: Daniel Axtens <dja@axtens.net> > > --- > > v4: Only call the slow check if the gso_size is large. > Eric - I think this is what you had in mind? > Manish - do you think this is an acceptable performance trade-off? > GSO will work for any packet size, and only jumbo frames will > have to do the slower test. Yes this looks good to me, thanks ! Reviewed-by: Eric Dumazet <edumazet@google.com>
diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c index 7b08323e3f3d..74fc9af4aadb 100644 --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c @@ -12934,6 +12934,24 @@ static netdev_features_t bnx2x_features_check(struct sk_buff *skb, struct net_device *dev, netdev_features_t features) { + /* + * A skb with gso_size + header length > 9700 will cause a + * firmware panic. Drop GSO support. + * + * Eventually the upper layer should not pass these packets down. + * + * For speed, if the gso_size is <= 9000, assume there will + * not be 700 bytes of headers and pass it through. Only do a + * full (slow) validation if the gso_size is > 9000. + * + * (Due to the way SKB_BY_FRAGS works this will also do a full + * validation in that case.) + */ + if (unlikely(skb_is_gso(skb) && + (skb_shinfo(skb)->gso_size > 9000) && + !skb_gso_validate_mac_len(skb, 9700))) + features &= ~NETIF_F_GSO_MASK; + features = vlan_features_check(skb, features); return vxlan_features_check(skb, features); }
If a bnx2x card is passed a GSO packet with a gso_size larger than ~9700 bytes, it will cause a firmware error that will bring the card down: bnx2x: [bnx2x_attn_int_deasserted3:4323(enP24p1s0f0)]MC assert! bnx2x: [bnx2x_mc_assert:720(enP24p1s0f0)]XSTORM_ASSERT_LIST_INDEX 0x2 bnx2x: [bnx2x_mc_assert:736(enP24p1s0f0)]XSTORM_ASSERT_INDEX 0x0 = 0x00000000 0x25e43e47 0x00463e01 0x00010052 bnx2x: [bnx2x_mc_assert:750(enP24p1s0f0)]Chip Revision: everest3, FW Version: 7_13_1 ... (dump of values continues) ... Detect when the mac length of a GSO packet is greater than the maximum packet size (9700 bytes) and disable GSO. Signed-off-by: Daniel Axtens <dja@axtens.net> --- v4: Only call the slow check if the gso_size is large. Eric - I think this is what you had in mind? Manish - do you think this is an acceptable performance trade-off? GSO will work for any packet size, and only jumbo frames will have to do the slower test. Again, only build-tested. --- drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)