| Message ID | 20220610131645.22047-1-zajec5@gmail.com |
|---|---|
| State | Superseded |
| Delegated to: | Rafał Miłecki |
| Headers | show |
| Series | bcm53xx: disable GRO support at kernel level | expand |
On 10.06.2022 15:16, Rafał Miłecki wrote: > This improves NAT masquarade network performance. > > An alternative to kernel change would be runtime setup but that requires > ethtool and identifying relevant network interface and all related > switch ports interfaces. I sent another patch that should supersede this one: [PATCH] bcm53xx: disable GRO by default at kernel level https://patchwork.ozlabs.org/project/openwrt/patch/20220620082120.11714-1-zajec5@gmail.com/ It's probably a better idea to disable GRO *by default* (and allow re-enabling it).
On Mon, Jun 20, 2022 at 2:33 AM Rafał Miłecki <zajec5@gmail.com> wrote: > > On 10.06.2022 15:16, Rafał Miłecki wrote: > > This improves NAT masquarade network performance. > > > > An alternative to kernel change would be runtime setup but that requires > > ethtool and identifying relevant network interface and all related > > switch ports interfaces. > > I sent another patch that should supersede this one: > [PATCH] bcm53xx: disable GRO by default at kernel level > https://patchwork.ozlabs.org/project/openwrt/patch/20220620082120.11714-1-zajec5@gmail.com/ > > It's probably a better idea to disable GRO *by default* (and allow > re-enabling it). Reminds me of this patch: https://github.com/openwrt/openwrt/commit/13e5e473699b92f171205e0f5c57c9ebe7922492 which was subsequently reverted because of slower routing performance Is this patch the same as using napi_gro_receive and passing NETIF_F_GRO ? > > _______________________________________________ > openwrt-devel mailing list > openwrt-devel@lists.openwrt.org > https://lists.openwrt.org/mailman/listinfo/openwrt-devel
On 21.06.2022 01:13, Rosen Penev wrote: > On Mon, Jun 20, 2022 at 2:33 AM Rafał Miłecki <zajec5@gmail.com> wrote: >> >> On 10.06.2022 15:16, Rafał Miłecki wrote: >>> This improves NAT masquarade network performance. >>> >>> An alternative to kernel change would be runtime setup but that requires >>> ethtool and identifying relevant network interface and all related >>> switch ports interfaces. >> >> I sent another patch that should supersede this one: >> [PATCH] bcm53xx: disable GRO by default at kernel level >> https://patchwork.ozlabs.org/project/openwrt/patch/20220620082120.11714-1-zajec5@gmail.com/ >> >> It's probably a better idea to disable GRO *by default* (and allow >> re-enabling it). > Reminds me of this patch: > https://github.com/openwrt/openwrt/commit/13e5e473699b92f171205e0f5c57c9ebe7922492 > which was subsequently reverted because of slower routing performance > > Is this patch the same as using napi_gro_receive and passing NETIF_F_GRO ? Kind of. Reverting above patch removes ar71xx support for GRO totally. It's not the best option as some people / cases / devices may want it. It's much better to just disable it. GRO can be runtime disabled using ethtool like: ethtool -K eth0 gro off ethtool -K lan1 gro off ethtool -K lan2 gro off ethtool -K lan3 gro off ethtool -K lan4 gro off ethtool -K wan gro off (or enabled with "on"). It can also be disabled by default with patch like the one I sent. In that case you can still enable it with: ethtool -K eth0 gro on ethtool -K lan1 gro on ethtool -K lan2 gro on ethtool -K lan3 gro on ethtool -K lan4 gro on ethtool -K wan gro on So for ar71xx I'd suggest to: 1. Re-implement GRO support 2. Disable GRO by default if it worsens performance for most devices
On 23.06.2022 05:49, Rafał Miłecki wrote: > On 21.06.2022 01:13, Rosen Penev wrote: >> On Mon, Jun 20, 2022 at 2:33 AM Rafał Miłecki <zajec5@gmail.com> wrote: >>> >>> On 10.06.2022 15:16, Rafał Miłecki wrote: >>>> This improves NAT masquarade network performance. >>>> >>>> An alternative to kernel change would be runtime setup but that requires >>>> ethtool and identifying relevant network interface and all related >>>> switch ports interfaces. >>> >>> I sent another patch that should supersede this one: >>> [PATCH] bcm53xx: disable GRO by default at kernel level >>> https://patchwork.ozlabs.org/project/openwrt/patch/20220620082120.11714-1-zajec5@gmail.com/ >>> >>> It's probably a better idea to disable GRO *by default* (and allow >>> re-enabling it). >> Reminds me of this patch: >> https://github.com/openwrt/openwrt/commit/13e5e473699b92f171205e0f5c57c9ebe7922492 >> which was subsequently reverted because of slower routing performance >> >> Is this patch the same as using napi_gro_receive and passing NETIF_F_GRO ? > > Kind of. Reverting above patch removes ar71xx support for GRO totally. > It's not the best option as some people / cases / devices may want it. > It's much better to just disable it. > > GRO can be runtime disabled using ethtool like: > ethtool -K eth0 gro off > ethtool -K lan1 gro off > ethtool -K lan2 gro off > ethtool -K lan3 gro off > ethtool -K lan4 gro off > ethtool -K wan gro off > (or enabled with "on"). > > It can also be disabled by default with patch like the one I sent. In > that case you can still enable it with: > ethtool -K eth0 gro on > ethtool -K lan1 gro on > ethtool -K lan2 gro on > ethtool -K lan3 gro on > ethtool -K lan4 gro on > ethtool -K wan gro on I was refering to the [PATCH] bcm53xx: disable GRO by default at kernel level https://patchwork.ozlabs.org/project/openwrt/patch/20220620082120.11714-1-zajec5@gmail.com/ With *this* patch (the one we're commenting on) it isn't possible to enable GRO with ethtool as it's permanently disabled on kernel level. That's why I decided to send patch to disable GRO only in default (initial) setup. > So for ar71xx I'd suggest to: > 1. Re-implement GRO support > 2. Disable GRO by default if it worsens performance for most devices
On Wed, Jun 22, 2022 at 8:49 PM Rafał Miłecki <zajec5@gmail.com> wrote: > > On 21.06.2022 01:13, Rosen Penev wrote: > > On Mon, Jun 20, 2022 at 2:33 AM Rafał Miłecki <zajec5@gmail.com> wrote: > >> > >> On 10.06.2022 15:16, Rafał Miłecki wrote: > >>> This improves NAT masquarade network performance. > >>> > >>> An alternative to kernel change would be runtime setup but that requires > >>> ethtool and identifying relevant network interface and all related > >>> switch ports interfaces. > >> > >> I sent another patch that should supersede this one: > >> [PATCH] bcm53xx: disable GRO by default at kernel level > >> https://patchwork.ozlabs.org/project/openwrt/patch/20220620082120.11714-1-zajec5@gmail.com/ > >> > >> It's probably a better idea to disable GRO *by default* (and allow > >> re-enabling it). > > Reminds me of this patch: > > https://github.com/openwrt/openwrt/commit/13e5e473699b92f171205e0f5c57c9ebe7922492 > > which was subsequently reverted because of slower routing performance > > > > Is this patch the same as using napi_gro_receive and passing NETIF_F_GRO ? > > Kind of. Reverting above patch removes ar71xx support for GRO totally. > It's not the best option as some people / cases / devices may want it. > It's much better to just disable it. I imagine that would be no routing cases. > > GRO can be runtime disabled using ethtool like: > ethtool -K eth0 gro off > ethtool -K lan1 gro off > ethtool -K lan2 gro off > ethtool -K lan3 gro off > ethtool -K lan4 gro off > ethtool -K wan gro off > (or enabled with "on"). > > It can also be disabled by default with patch like the one I sent. In > that case you can still enable it with: > ethtool -K eth0 gro on > ethtool -K lan1 gro on > ethtool -K lan2 gro on > ethtool -K lan3 gro on > ethtool -K lan4 gro on > ethtool -K wan gro on > > > So for ar71xx I'd suggest to: > 1. Re-implement GRO support > 2. Disable GRO by default if it worsens performance for most devices Basically your other patch.
diff --git a/target/linux/bcm53xx/patches-5.10/600-net-disable-GRO-support.patch b/target/linux/bcm53xx/patches-5.10/600-net-disable-GRO-support.patch new file mode 100644 index 0000000000..1024aec7e1 --- /dev/null +++ b/target/linux/bcm53xx/patches-5.10/600-net-disable-GRO-support.patch @@ -0,0 +1,46 @@ +From: =?UTF-8?q?Rafa=C5=82=20Mi=C5=82ecki?= <rafal@milecki.pl> +Date: Fri, 10 Jun 2022 15:08:33 +0200 +Subject: [PATCH] net: disable GRO support +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +In many cases GRO improves network performance however it comes at a +cost of chacksums calculations. In case of slow CPU and missing hardware +csum calculation support GRO can actually decrease network speed. + +On BCM4708 *disabling* GRO results in following NAT masquarade speed +changes: +1. 364 Mb/s → 396 Mb/s (packet steering disabled) +2. 341 Mb/s → 566 Mb/s (packet steering enabled) + +Signed-off-by: Rafał Miłecki <rafal@milecki.pl> +--- + net/8021q/vlan_core.c | 2 ++ + net/ipv4/af_inet.c | 2 ++ + 2 files changed, 4 insertions(+) + +--- a/net/8021q/vlan_core.c ++++ b/net/8021q/vlan_core.c +@@ -544,6 +544,8 @@ static int __init vlan_offload_init(void) + { + unsigned int i; + ++ return -ENOTSUPP; ++ + for (i = 0; i < ARRAY_SIZE(vlan_packet_offloads); i++) + dev_add_offload(&vlan_packet_offloads[i]); + +diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c +index 8267349..6194a14 100644 +--- a/net/ipv4/af_inet.c ++++ b/net/ipv4/af_inet.c +@@ -1913,6 +1913,8 @@ static int __init ipip_offload_init(void) + + static int __init ipv4_offload_init(void) + { ++ return -ENOTSUPP; ++ + /* + * Add offloads + */