From patchwork Fri Apr 12 06:29:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jun Wang X-Patchwork-Id: 1922866 X-Patchwork-Delegate: i.maximets@samsung.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VG6BF0WJZz1yYL for ; Fri, 12 Apr 2024 16:30:12 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id CADF941F10; Fri, 12 Apr 2024 06:30:10 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id pkdt_5HLzz-m; Fri, 12 Apr 2024 06:30:08 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 384DD41EF1 Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 384DD41EF1; Fri, 12 Apr 2024 06:30:08 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id EDCF6C007C; Fri, 12 Apr 2024 06:30:07 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1F1AEC0037 for ; Fri, 12 Apr 2024 06:30:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 1306A41EE5 for ; Fri, 12 Apr 2024 06:30:06 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id FOI9inLKLrUb for ; Fri, 12 Apr 2024 06:30:04 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=1.203.97.240; helo=smtp.cecloud.com; envelope-from=junwang01@cestc.cn; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org 8020041EE1 Authentication-Results: smtp4.osuosl.org; dmarc=none (p=none dis=none) header.from=cestc.cn DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 8020041EE1 Received: from smtp.cecloud.com (unknown [1.203.97.240]) by smtp4.osuosl.org (Postfix) with ESMTP id 8020041EE1 for ; Fri, 12 Apr 2024 06:30:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp.cecloud.com (Postfix) with ESMTP id 3DE87900119 for ; Fri, 12 Apr 2024 14:29:57 +0800 (CST) X-MAIL-GRAY: 0 X-MAIL-DELIVERY: 1 X-ANTISPAM-LEVEL: 2 X-ABS-CHECKED: 0 Received: from izgc79c142y6tt9fty3vzrz.localdomain (unknown [47.122.22.90]) by smtp.cecloud.com (postfix) whith ESMTP id P4022987T281466880127344S1712903389168577_; Fri, 12 Apr 2024 14:29:57 +0800 (CST) X-RL-SENDER: junwang01@cestc.cn X-SENDER: junwang01@cestc.cn X-LOGIN-NAME: junwang01@cestc.cn X-FST-TO: dev@openvswitch.org X-RCPT-COUNT: 2 X-LOCAL-RCPT-COUNT: 0 X-MUTI-DOMAIN-COUNT: 0 X-SENDER-IP: 47.122.22.90 X-ATTACHMENT-NUM: 0 X-UNIQUE-TAG: <98f3f08b3378cef6f3040c0b1f844b0d> X-System-Flag: 0 From: Jun Wang To: dev@openvswitch.org Date: Fri, 12 Apr 2024 14:29:45 +0800 Message-Id: <1712903385-24393-1-git-send-email-junwang01@cestc.cn> X-Mailer: git-send-email 1.8.3.1 Cc: i.maximets@ovn.org Subject: [ovs-dev] [PATCH v1] nedev-dpdk: Fix config with dpdk net_bonding offloads. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" If it's a DPDK net_bonding, it may cause offload-related configurations to take effect, leading to offload failure. /usr/share/openvswitch/scripts/ovs-ctl restart --no-ovs-vswitchd \ --system-id=test ovs-vsctl --no-wait set open . external-ids:ovn-bridge-datapath-type\ =netdev ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem=\ "4096,4096" ovs-vsctl --no-wait set Open_vSwitch . other_config:pmd-cpu-mask=\ 0xff0000 ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-extra=\ "-a 0000:ca:00.0 -a 0000:ca:00.1 --vdev net_bonding2494589023,\ mode=4,member=0000:ca:00.0,member=0000:ca:00.1,xmit_policy=l34,\ lacp_rate=fast" ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer \ -vsyslog:err -vfile:info --mlockall --no-chdir \ --log-file=/var/log/openvswitch/ovs-vswitchd.log \ --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor ovs-vsctl add-br br-tun -- set bridge br-tun datapath_type=netdev ovs-vsctl --may-exist add-port br-tun dpdk_tun_port -- set Interface \ dpdk_tun_port type=dpdk options:dpdk-devargs="net_bonding2494589023" {bus_info="bus_name=vdev", driver_name=net_bonding, if_descr="DPDK 23.11.0 net_bonding", if_type="6",link_speed="20Gbps", max_hash_mac_addrs="0", max_mac_addrs="16", max_rx_pktlen="1618", max_rx_queues="1023", max_tx_queues="1023", max_vfs="0", max_vmdq_pools="0", min_rx_bufsize="0", n_rxq="4", n_txq="9", numa_id="0", port_no="2", rx-steering=rss, rx_csum_offload="false", tx_geneve_tso_offload="false", tx_ip_csum_offload="false", tx_out_ip_csum_offload="false", tx_out_udp_csum_offload="false", tx_sctp_csum_offload="false", tx_tcp_csum_offload="false", tx_tcp_seg_offload="false", tx_udp_csum_offload="false", tx_vxlan_tso_offload="false"} Fixes: 5d11c47d3ebe ("userspace: Enable IP checksum offloading by default.") Signed-off-by: Jun Wang --- lib/netdev-dpdk.c | 75 +++++++++++++++++++++++++++++++++---------------------- 1 file changed, 45 insertions(+), 30 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 2111f77..191c83d 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -1294,15 +1294,10 @@ dpdk_eth_dev_init_rx_metadata(struct netdev_dpdk *dev) dev->rx_metadata_delivery_configured = true; } -static int -dpdk_eth_dev_init(struct netdev_dpdk *dev) - OVS_REQUIRES(dev->mutex) +static void +dpdk_eth_offload_config(struct netdev_dpdk *dev, + struct rte_eth_dev_info *info) { - struct rte_pktmbuf_pool_private *mbp_priv; - struct rte_eth_dev_info info; - struct rte_ether_addr eth_addr; - int diag; - int n_rxq, n_txq; uint32_t rx_chksm_offload_capa = RTE_ETH_RX_OFFLOAD_UDP_CKSUM | RTE_ETH_RX_OFFLOAD_TCP_CKSUM | RTE_ETH_RX_OFFLOAD_IPV4_CKSUM; @@ -1319,16 +1314,14 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) dpdk_eth_dev_init_rx_metadata(dev); } - rte_eth_dev_info_get(dev->port_id, &info); - - if (strstr(info.driver_name, "vf") != NULL) { + if (strstr(info->driver_name, "vf") != NULL) { VLOG_INFO("Virtual function detected, HW_CRC_STRIP will be enabled"); dev->hw_ol_features |= NETDEV_RX_HW_CRC_STRIP; } else { dev->hw_ol_features &= ~NETDEV_RX_HW_CRC_STRIP; } - if ((info.rx_offload_capa & rx_chksm_offload_capa) != + if ((info->rx_offload_capa & rx_chksm_offload_capa) != rx_chksm_offload_capa) { VLOG_WARN("Rx checksum offload is not supported on port " DPDK_PORT_ID_FMT, dev->port_id); @@ -1337,66 +1330,66 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) dev->hw_ol_features |= NETDEV_RX_CHECKSUM_OFFLOAD; } - if (info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_SCATTER) { + if (info->rx_offload_capa & RTE_ETH_RX_OFFLOAD_SCATTER) { dev->hw_ol_features |= NETDEV_RX_HW_SCATTER; } else { /* Do not warn on lack of scatter support */ dev->hw_ol_features &= ~NETDEV_RX_HW_SCATTER; } - if (!strcmp(info.driver_name, "net_tap")) { + if (!strcmp(info->driver_name, "net_tap")) { /* FIXME: L4 checksum offloading is broken in DPDK net/tap driver. * This workaround can be removed once the fix makes it to a DPDK * LTS release used by OVS. */ VLOG_INFO("%s: disabled Tx L4 checksum offloads for a net/tap port.", netdev_get_name(&dev->up)); - info.tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_UDP_CKSUM; - info.tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_TCP_CKSUM; + info->tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_UDP_CKSUM; + info->tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_TCP_CKSUM; } - if (!strcmp(info.driver_name, "net_ice") - || !strcmp(info.driver_name, "net_i40e")) { + if (!strcmp(info->driver_name, "net_ice") + || !strcmp(info->driver_name, "net_i40e")) { /* FIXME: Driver advertises the capability but doesn't seem * to actually support it correctly. Can remove this once * the driver is fixed on DPDK side. */ VLOG_INFO("%s: disabled Tx outer udp checksum offloads for a " "net/ice or net/i40e port.", netdev_get_name(&dev->up)); - info.tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM; - info.tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO; - info.tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO; + info->tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM; + info->tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO; + info->tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO; } - if (info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_IPV4_CKSUM) { + if (info->tx_offload_capa & RTE_ETH_TX_OFFLOAD_IPV4_CKSUM) { dev->hw_ol_features |= NETDEV_TX_IPV4_CKSUM_OFFLOAD; } else { dev->hw_ol_features &= ~NETDEV_TX_IPV4_CKSUM_OFFLOAD; } - if (info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_TCP_CKSUM) { + if (info->tx_offload_capa & RTE_ETH_TX_OFFLOAD_TCP_CKSUM) { dev->hw_ol_features |= NETDEV_TX_TCP_CKSUM_OFFLOAD; } else { dev->hw_ol_features &= ~NETDEV_TX_TCP_CKSUM_OFFLOAD; } - if (info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_UDP_CKSUM) { + if (info->tx_offload_capa & RTE_ETH_TX_OFFLOAD_UDP_CKSUM) { dev->hw_ol_features |= NETDEV_TX_UDP_CKSUM_OFFLOAD; } else { dev->hw_ol_features &= ~NETDEV_TX_UDP_CKSUM_OFFLOAD; } - if (info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_SCTP_CKSUM) { + if (info->tx_offload_capa & RTE_ETH_TX_OFFLOAD_SCTP_CKSUM) { dev->hw_ol_features |= NETDEV_TX_SCTP_CKSUM_OFFLOAD; } else { dev->hw_ol_features &= ~NETDEV_TX_SCTP_CKSUM_OFFLOAD; } - if (info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM) { + if (info->tx_offload_capa & RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM) { dev->hw_ol_features |= NETDEV_TX_OUTER_IP_CKSUM_OFFLOAD; } else { dev->hw_ol_features &= ~NETDEV_TX_OUTER_IP_CKSUM_OFFLOAD; } - if (info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM) { + if (info->tx_offload_capa & RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM) { dev->hw_ol_features |= NETDEV_TX_OUTER_UDP_CKSUM_OFFLOAD; } else { dev->hw_ol_features &= ~NETDEV_TX_OUTER_UDP_CKSUM_OFFLOAD; @@ -1404,21 +1397,21 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) dev->hw_ol_features &= ~NETDEV_TX_TSO_OFFLOAD; if (userspace_tso_enabled()) { - if (info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_TCP_TSO) { + if (info->tx_offload_capa & RTE_ETH_TX_OFFLOAD_TCP_TSO) { dev->hw_ol_features |= NETDEV_TX_TSO_OFFLOAD; } else { VLOG_WARN("%s: Tx TSO offload is not supported.", netdev_get_name(&dev->up)); } - if (info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO) { + if (info->tx_offload_capa & RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO) { dev->hw_ol_features |= NETDEV_TX_VXLAN_TNL_TSO_OFFLOAD; } else { VLOG_WARN("%s: Tx Vxlan tunnel TSO offload is not supported.", netdev_get_name(&dev->up)); } - if (info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO) { + if (info->tx_offload_capa & RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO) { dev->hw_ol_features |= NETDEV_TX_GENEVE_TNL_TSO_OFFLOAD; } else { VLOG_WARN("%s: Tx Geneve tunnel TSO offload is not supported.", @@ -1426,6 +1419,21 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) } } +} + +static int +dpdk_eth_dev_init(struct netdev_dpdk *dev) + OVS_REQUIRES(dev->mutex) +{ + struct rte_pktmbuf_pool_private *mbp_priv; + struct rte_eth_dev_info info; + struct rte_ether_addr eth_addr; + int diag; + int n_rxq, n_txq; + + rte_eth_dev_info_get(dev->port_id, &info); + dpdk_eth_offload_config(dev, &info); + n_rxq = MIN(info.max_rx_queues, dev->up.n_rxq); n_txq = MIN(info.max_tx_queues, dev->up.n_txq); @@ -1439,6 +1447,13 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) return -diag; } + rte_eth_dev_info_get(dev->port_id, &info); + if (!strcmp(info.driver_name, "net_bonding")) { + dpdk_eth_offload_config(dev, &info); + VLOG_INFO("%s: configure offloads for a dpdk net_bonding port.", + netdev_get_name(&dev->up)); + } + diag = rte_eth_dev_start(dev->port_id); if (diag) { VLOG_ERR("Interface %s start error: %s", dev->up.name,