[ovs-dev,v4,14/14] userspace: Enable TSO if available.

Message ID	20220701035834.1851648-14-mkp@redhat.com
State	Superseded
Headers	show Return-Path: <ovs-dev-bounces@openvswitch.org> From: Mike Pattrick <mkp@redhat.com> To: dev@openvswitch.org Date: Thu, 30 Jun 2022 23:58:34 -0400 Message-Id: <20220701035834.1851648-14-mkp@redhat.com> In-Reply-To: <20220701035834.1851648-13-mkp@redhat.com> References: <20220701035834.1851648-1-mkp@redhat.com> <20220701035834.1851648-2-mkp@redhat.com> <20220701035834.1851648-3-mkp@redhat.com> <20220701035834.1851648-4-mkp@redhat.com> <20220701035834.1851648-5-mkp@redhat.com> <20220701035834.1851648-6-mkp@redhat.com> <20220701035834.1851648-7-mkp@redhat.com> <20220701035834.1851648-8-mkp@redhat.com> <20220701035834.1851648-9-mkp@redhat.com> <20220701035834.1851648-10-mkp@redhat.com> <20220701035834.1851648-11-mkp@redhat.com> <20220701035834.1851648-12-mkp@redhat.com> <20220701035834.1851648-13-mkp@redhat.com> MIME-Version: 1.0 Cc: Flavio Leitner <fbl@sysclose.org>, maxime.coquelin@redhat.com, david.marchand@redhat.com Subject: [ovs-dev] [PATCH v4 14/14] userspace: Enable TSO if available. Precedence: list Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" <ovs-dev-bounces@openvswitch.org>
Series	[ovs-dev,v4,01/14] dp-packet: Rename flags with CKSUM to CSUM. \| expand [ovs-dev,v4,01/14] dp-packet: Rename flags with CKSUM to CSUM. [ovs-dev,v4,02/14] netdev: Prefix offload flags with NETDEV_OFFLOAD_ [ovs-dev,v4,03/14] dp-packet: Rename dp_packet_hwol to dp_packet_ol. [ovs-dev,v4,04/14] dp-packet: Use p for packet and b for batch. [ovs-dev,v4,05/14] dp-packet: Rename dp_packet_ol_tcp_seg [ovs-dev,v4,06/14] dp-packet: Rename dp_packet_ol l4 functions. [ovs-dev,v4,07/14] dp-packet: Add _ol_ to functions using OL flags. [ovs-dev,v4,08/14] Documentation: Document netdev offload. [ovs-dev,v4,09/14] dpif-netdev: Show netdev offloading flags. [ovs-dev,v4,10/14] userspace: Enable IP checksum offloading by default. [ovs-dev,v4,11/14] userspace: Enable L4 csum offloading by default. [ovs-dev,v4,12/14] userspace: Respect tso/gso segment size. [ovs-dev,v4,13/14] userspace: Add Generic Segmentation Offloading. [ovs-dev,v4,14/14] userspace: Enable TSO if available.

Context	Check	Description
ovsrobot/apply-robot	success	apply and check: success
ovsrobot/github-robot-_Build_and_Test	fail	github build: failed
ovsrobot/intel-ovs-compilation	fail	test: fail

diff --git a/Documentation/topics/userspace-tso.rst b/Documentation/topics/userspace-tso.rst index 33a85965c..0245723af 100644 --- a/Documentation/topics/userspace-tso.rst +++ b/Documentation/topics/userspace-tso.rst @@ -27,8 +27,6 @@ Userspace Datapath - TSO ======================== -**Note:** This feature is considered experimental. - TCP Segmentation Offload (TSO) enables a network stack to delegate segmentation of an oversized TCP segment to the underlying physical NIC. Offload of frame segmentation achieves computational savings in the core, freeing up CPU cycles @@ -48,16 +46,16 @@ refer to the `DPDK documentation`__. __ https://doc.dpdk.org/guides-21.11/nics/overview.html -Enabling TSO -~~~~~~~~~~~~ +Disabling TSO +~~~~~~~~~~~~~ -The TSO support may be enabled via a global config value -``userspace-tso-enable``. Setting this to ``true`` enables TSO support for -all ports.:: +The TSO support is enabled by default it may be disabled via a global config +value ``userspace-tso-enable``. Setting this to ``false`` disabled TSO support +for all ports.:: - $ ovs-vsctl set Open_vSwitch . other_config:userspace-tso-enable=true + $ ovs-vsctl set Open_vSwitch . other_config:userspace-tso-enable=false -The default value is ``false``. +The default value is ``true``. Changing ``userspace-tso-enable`` requires restarting the daemon. @@ -108,7 +106,10 @@ All kernel devices that use the raw socket interface (veth, for example) require the kernel commit 9d2f67e43b73 ("net/packet: fix packet drop as of virtio gso") in order to work properly. This commit was merged in upstream kernel 4.19-rc7, so make sure your kernel is either newer or contains the -backport. +backport. Network topologies that include both TSO and Linux VLAN ports +requires the kernel commit dfed913e8b55 ("net/af_packet: add VLAN support +for AF_PACKET SOCK_RAW GSO") in order to work properly. This commit was +merged in upstream kernel 5.19-rc1. ~~~~~~~~~~~~~~~~~~ Performance Tuning diff --git a/NEWS b/NEWS index 994fdf6a9..e5e2f71c6 100644 --- a/NEWS +++ b/NEWS @@ -43,6 +43,10 @@ Post-v2.17.0 * 'dpif-netdev/subtable-lookup-prio-get' appctl command renamed to 'dpif-netdev/subtable-lookup-info-get' to better reflect its purpose. The old variant is kept for backward compatibility. + * Userspace TSO is now enabled by default, but can still be disabled by + setting other_config:userspace-tso-enable=false. + See Documentation/topics/userspace-tso.rst for more information. + v2.17.0 - 17 Feb 2022 diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c index 7f314b810..1d38d783e 100644 --- a/lib/netdev-linux.c +++ b/lib/netdev-linux.c @@ -542,6 +542,7 @@ static bool netdev_linux_miimon_enabled(void); static void netdev_linux_miimon_run(void); static void netdev_linux_miimon_wait(void); static int netdev_linux_get_mtu__(struct netdev_linux *netdev, int *mtup); +static void netdev_linux_set_ol(struct netdev *netdev); static bool is_tap_netdev(const struct netdev *netdev) @@ -943,11 +944,7 @@ netdev_linux_construct(struct netdev *netdev_) /* The socket interface doesn't offer the option to enable only * csum offloading without TSO. */ if (userspace_tso_enabled()) { - netdev_->ol_flags |= NETDEV_OFFLOAD_TX_TCP_TSO; - netdev_->ol_flags |= NETDEV_OFFLOAD_TX_TCP_CSUM; - netdev_->ol_flags |= NETDEV_OFFLOAD_TX_UDP_CSUM; - netdev_->ol_flags |= NETDEV_OFFLOAD_TX_SCTP_CSUM; - netdev_->ol_flags |= NETDEV_OFFLOAD_TX_IPV4_CSUM; + netdev_linux_set_ol(netdev_); } error = get_flags(&netdev->up, &netdev->ifi_flags); @@ -2348,6 +2345,132 @@ netdev_internal_get_stats(const struct netdev *netdev_, return error; } +static int +netdev_linux_read_stringset_info(struct netdev_linux *netdev, uint32_t *len) +{ + struct { + struct ethtool_sset_info hdr; + uint32_t buf[1]; + } sset_info; + + sset_info.hdr.cmd = ETHTOOL_GSSET_INFO; + sset_info.hdr.reserved = 0; + sset_info.hdr.sset_mask = 1ULL << ETH_SS_FEATURES; + + int error = netdev_linux_do_ethtool(netdev->up.name, + (struct ethtool_cmd *)&sset_info, + ETHTOOL_GSSET_INFO, "ETHTOOL_GSSET_INFO"); + if (error) { + return error; + } + *len = *(uint32_t *) sset_info.hdr.data; + return 0; +} + + +static int +netdev_linux_read_definitions(struct netdev_linux *netdev, + struct ethtool_gstrings **pstrings) +{ + int error = 0; + struct ethtool_gstrings *strings = NULL; + uint32_t len = 0; + + error = netdev_linux_read_stringset_info(netdev, &len); + if (error || !len) { + return error; + } + strings = xcalloc(1, sizeof(*strings) + len * ETH_GSTRING_LEN); + if (!strings) { + return ENOMEM; + } + + strings->cmd = ETHTOOL_GSTRINGS; + strings->string_set = ETH_SS_FEATURES; + strings->len = len; + error = netdev_linux_do_ethtool(netdev->up.name, + (struct ethtool_cmd *) strings, + ETHTOOL_GSTRINGS, "ETHTOOL_GSTRINGS"); + if (error) { + goto out; + } + + for (int i = 0; i < len; i++) { + strings->data[(i + 1) * ETH_GSTRING_LEN - 1] = 0; + } + + *pstrings = strings; + + return 0; +out: + *pstrings = NULL; + free(strings); + return error; +} + +static void +netdev_linux_set_ol(struct netdev *netdev_) +{ + struct netdev_linux *netdev = netdev_linux_cast(netdev_); + struct ethtool_gstrings *names = NULL; + struct ethtool_gfeatures *features = NULL; + int error; + + COVERAGE_INC(netdev_get_ethtool); + + error = netdev_linux_read_definitions(netdev, &names); + if (error) { + return; + } + + features = xmalloc(sizeof *features + + DIV_ROUND_UP(names->len, 32) * + sizeof features->features[0]); + if (!features) { + goto out; + } + + features->cmd = ETHTOOL_GFEATURES; + features->size = DIV_ROUND_UP(names->len, 32); + error = netdev_linux_do_ethtool(netdev_get_name(netdev_), + (struct ethtool_cmd *) features, + ETHTOOL_GFEATURES, "ETHTOOL_GFEATURES"); + + if (error) { + goto out; + } + +#define FEATURE_WORD(blocks, index, field) ((blocks)[(index) / 32U].field) +#define FEATURE_FIELD_FLAG(index) (1U << (index) % 32U) +#define FEATURE_BIT_IS_SET(blocks, index, field) \ + (FEATURE_WORD(blocks, index, field) & FEATURE_FIELD_FLAG(index)) + + netdev->up.ol_flags = 0; + static const struct { + char * string; + int value; + } t_list[] = { + {"tx-checksum-ipv4", NETDEV_OFFLOAD_TX_IPV4_CSUM}, + {"tx-sctp-segmentation", NETDEV_OFFLOAD_TX_SCTP_CSUM}, + {"tx-udp-segmentation", NETDEV_OFFLOAD_TX_UDP_CSUM}, + }; + + for (int i = 0; i < names->len; i++) { + char * name = (char *) names->data + i * ETH_GSTRING_LEN; + for (int j = 0; j < sizeof t_list / sizeof t_list[0]; j++) { + if (strcmp(t_list[j].string, name) == 0) { + if (FEATURE_BIT_IS_SET(features->features, i, active)) { + netdev_->ol_flags |= t_list[j].value; + } + } + } + } + +out: + free(names); + free(features); +} + static void netdev_linux_read_features(struct netdev_linux *netdev) { diff --git a/lib/userspace-tso.c b/lib/userspace-tso.c index f843c2a76..27b9384d7 100644 --- a/lib/userspace-tso.c +++ b/lib/userspace-tso.c @@ -25,17 +25,18 @@ VLOG_DEFINE_THIS_MODULE(userspace_tso); -static bool userspace_tso = false; +static bool userspace_tso = true; void userspace_tso_init(const struct smap *ovs_other_config) { - if (smap_get_bool(ovs_other_config, "userspace-tso-enable", false)) { + if (!smap_get_bool(ovs_other_config, "userspace-tso-enable", true)) { static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; if (ovsthread_once_start(&once)) { - VLOG_INFO("Userspace TCP Segmentation Offloading support enabled"); - userspace_tso = true; + VLOG_INFO("Userspace TCP Segmentation Offloading support " + "disabled"); + userspace_tso = false; ovsthread_once_done(&once); } } diff --git a/tests/ofproto-macros.at b/tests/ofproto-macros.at index b18f0fbc1..e91ef233e 100644 --- a/tests/ofproto-macros.at +++ b/tests/ofproto-macros.at @@ -256,6 +256,7 @@ check_logs () { /ovs_rcu.*blocked [[0-9]]* ms waiting for .* to quiesce/d /Dropped [[0-9]]* log messages/d /setting extended ack support failed/d +/ETHTOOL_GSSET_INFO/d /|WARN|/p /|ERR|/p /|EMER|/p" ${logs} diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index cc1dd77ec..8aacddac9 100644 --- a/vswitchd/vswitch.xml +++ b/vswitchd/vswitch.xml @@ -761,22 +761,15 @@ <column name="other_config" key="userspace-tso-enable" type='{"type": "boolean"}'> - Set this value to <code>true</code> to enable userspace support for - TCP Segmentation Offloading (TSO). When it is enabled, the interfaces - can provide an oversized TCP segment to the datapath and the datapath - will offload the TCP segmentation and checksum calculation to the - interfaces when necessary. + Set this value to <code>false</code> to disable userspace support for + TCP Segmentation Offloading (TSO). When it is disabled, the datapath + will not offload the TCP segmentation and checksum calculation to + the interfaces. - The default value is <code>false</code>. Changing this value requires + The default value is <code>true</code>. Changing this value requires restarting the daemon. - - The feature only works if Open vSwitch is built with DPDK support. - - - The feature is considered experimental. - </column> </group> <group title="Status">

[ovs-dev,v4,14/14] userspace: Enable TSO if available.

Checks

Commit Message

Comments

Patch