From patchwork Tue May 16 11:59:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Traynor X-Patchwork-Id: 1782000 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iQ5hdugH; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QLFDv0gtNz20dX for ; Tue, 16 May 2023 22:00:47 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 4F08E83C32; Tue, 16 May 2023 12:00:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 4F08E83C32 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iQ5hdugH X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wVQziarU5vij; Tue, 16 May 2023 12:00:44 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 889BD83C39; Tue, 16 May 2023 12:00:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 889BD83C39 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 33E4EC0093; Tue, 16 May 2023 12:00:43 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id A7492C0037 for ; Tue, 16 May 2023 12:00:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id E2A2683F76 for ; Tue, 16 May 2023 12:00:33 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E2A2683F76 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4Tds_EQcqZ49 for ; Tue, 16 May 2023 12:00:33 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3CD4483F75 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3CD4483F75 for ; Tue, 16 May 2023 12:00:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684238432; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r+OZSav2ZnIVjG8n56Qq1Lmo7QJTbJoDRXzgggcaWRw=; b=iQ5hdugH/721Rc+0h4QPg7YN5Ozw30vyKvFg40Qk7MWYGMdtIWT6vv0A+zbZbNB5Akzf6r 2ElJEL7w9NGRQ9uhcTR1CBLrcRcKgMNzHBN1iKl3lOFwck+Fbfk2gN1+YZNH7+sf3k5fQ7 gVu96JLhm+kJb5Xpqgy/IXxZR6jOYgo= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-5-P-Gh7t2ZNEiP3irfxTM6HQ-1; Tue, 16 May 2023 08:00:15 -0400 X-MC-Unique: P-Gh7t2ZNEiP3irfxTM6HQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 60C401825074; Tue, 16 May 2023 12:00:13 +0000 (UTC) Received: from rh.Home (unknown [10.39.192.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id 22F39C16508; Tue, 16 May 2023 12:00:11 +0000 (UTC) From: Kevin Traynor To: dev@openvswitch.org Date: Tue, 16 May 2023 12:59:57 +0100 Message-Id: <20230516115958.287305-2-ktraynor@redhat.com> In-Reply-To: <20230516115958.287305-1-ktraynor@redhat.com> References: <20230516115958.287305-1-ktraynor@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: simon.horman@corigine.com, david.marchand@redhat.com, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v4 1/2] netdev-dpdk: Remove requested descriptors from get_config. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" There is no need to display 'requested_rx/tx_descriptors' and 'configured_rx/tx_descriptors' as they will be the same. It is simpler to just have a single 'n_rxq/txq_desc' value. Suggested-by: Ilya Maximets Reviewed-by: David Marchand Signed-off-by: Kevin Traynor Reviewed-by: Simon Horman --- lib/netdev-dpdk.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index fb0dd43f7..2d9afc493 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -1741,12 +1741,6 @@ netdev_dpdk_get_config(const struct netdev *netdev, struct smap *args) if (dev->type == DPDK_DEV_ETH) { - smap_add_format(args, "requested_rxq_descriptors", "%d", - dev->requested_rxq_size); - smap_add_format(args, "configured_rxq_descriptors", "%d", - dev->rxq_size); - smap_add_format(args, "requested_txq_descriptors", "%d", - dev->requested_txq_size); - smap_add_format(args, "configured_txq_descriptors", "%d", - dev->txq_size); + smap_add_format(args, "n_rxq_desc", "%d", dev->rxq_size); + smap_add_format(args, "n_txq_desc", "%d", dev->txq_size); if (dev->hw_ol_features & NETDEV_RX_CHECKSUM_OFFLOAD) { smap_add(args, "rx_csum_offload", "true"); From patchwork Tue May 16 11:59:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Traynor X-Patchwork-Id: 1782001 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MV5R6+iQ; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QLFFd5xpQz20dX for ; Tue, 16 May 2023 22:01:25 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id A5F3D60FD7; Tue, 16 May 2023 12:01:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org A5F3D60FD7 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MV5R6+iQ X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id luKE8zv7n2QR; Tue, 16 May 2023 12:01:19 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id CC48060E7C; Tue, 16 May 2023 12:01:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org CC48060E7C Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B365DC0037; Tue, 16 May 2023 12:01:18 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id CD7FBC002A for ; Tue, 16 May 2023 12:01:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9561460F53 for ; Tue, 16 May 2023 12:01:14 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 9561460F53 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lQZXEtBeKVbW for ; Tue, 16 May 2023 12:01:10 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org BDC13614EF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id BDC13614EF for ; Tue, 16 May 2023 12:01:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684238468; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h4EexCP9mdKygkrRUBrjKXayNQSxdJQg/TgPz/4JNIU=; b=MV5R6+iQS0SU7ebqr7o3Qs+Wwan8Hj1P/o9Qf0iTXnDdT+17VJE6JFQhRSR6wFnSmgYK4/ H+ncHGoxA7wuF4XN3XTKTzXNdvIRReSgM5Y2gfXN8rmkKy7iz31CEITGO2jdMtPLGAwPqH oN3u5ZHUpgXLAkQ0cb5fElyA0KuVZzI= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-648-qQnmlfRCN3GF646vWZHA9Q-1; Tue, 16 May 2023 08:00:20 -0400 X-MC-Unique: qQnmlfRCN3GF646vWZHA9Q-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 37FCA38184FA; Tue, 16 May 2023 12:00:15 +0000 (UTC) Received: from rh.Home (unknown [10.39.192.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id BE924C1650F; Tue, 16 May 2023 12:00:13 +0000 (UTC) From: Kevin Traynor To: dev@openvswitch.org Date: Tue, 16 May 2023 12:59:58 +0100 Message-Id: <20230516115958.287305-3-ktraynor@redhat.com> In-Reply-To: <20230516115958.287305-1-ktraynor@redhat.com> References: <20230516115958.287305-1-ktraynor@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: simon.horman@corigine.com, david.marchand@redhat.com, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v4 2/2] netdev-dpdk: Check rx/tx descriptor sizes for device. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" By default OVS configures 2048 descriptors for tx and rx queues on DPDK devices. It also allows the user to configure those values. If the values used are not acceptable to the device then queue setup would fail. The device exposes it's max/min/alignment requirements and OVS applies some limits also. Use these to ensure an acceptable value is used for the number of descriptors on a device tx/rx. If the default or user value is not acceptable, adjust to a suitable value and log. Reported-at: https://bugzilla.redhat.com/2119876 Signed-off-by: Kevin Traynor Reviewed-by: David Marchand Reviewed-by: Simon Horman --- lib/netdev-dpdk.c | 61 ++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 50 insertions(+), 11 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 2d9afc493..e5fc1aa30 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -1911,8 +1911,30 @@ dpdk_set_rxq_config(struct netdev_dpdk *dev, const struct smap *args) static void dpdk_process_queue_size(struct netdev *netdev, const struct smap *args, - const char *flag, int default_size, int *new_size) + struct rte_eth_dev_info *info, bool is_rx) { - int queue_size = smap_get_int(args, flag, default_size); + struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); + struct rte_eth_desc_lim *lim; + int default_size, queue_size, cur_size, new_requested_size; + int *cur_requested_size; + bool reconfig = false; + if (is_rx) { + default_size = NIC_PORT_DEFAULT_RXQ_SIZE; + new_requested_size = smap_get_int(args, "n_rxq_desc", default_size); + cur_size = dev->rxq_size; + cur_requested_size = &dev->requested_rxq_size; + lim = info ? &info->rx_desc_lim : NULL; + + } else { + default_size = NIC_PORT_DEFAULT_TXQ_SIZE; + new_requested_size = smap_get_int(args, "n_txq_desc", default_size); + cur_size = dev->txq_size; + cur_requested_size = &dev->requested_txq_size; + lim = info ? &info->tx_desc_lim : NULL; + } + + queue_size = new_requested_size; + + /* Check for OVS limits. */ if (queue_size <= 0 || queue_size > NIC_PORT_MAX_Q_SIZE || !is_pow2(queue_size)) { @@ -1920,7 +1942,24 @@ dpdk_process_queue_size(struct netdev *netdev, const struct smap *args, } - if (queue_size != *new_size) { - *new_size = queue_size; + if (lim) { + /* Check for device limits. */ + if (lim->nb_align) { + queue_size = ROUND_UP(queue_size, lim->nb_align); + } + queue_size = MIN(queue_size, lim->nb_max); + queue_size = MAX(queue_size, lim->nb_min); + } + + *cur_requested_size = queue_size; + + if (cur_size != queue_size) { netdev_request_reconfigure(netdev); + reconfig = true; + } + if (new_requested_size != queue_size) { + VLOG(reconfig ? VLL_INFO : VLL_DBG, + "Interface %s cannot set %s descriptor size to %d. " + "Adjusted to %d.", dev->up.name, is_rx ? "rx": "tx" , + new_requested_size, queue_size); } } @@ -1938,7 +1977,9 @@ netdev_dpdk_set_config(struct netdev *netdev, const struct smap *args, {RTE_ETH_FC_RX_PAUSE, RTE_ETH_FC_FULL } }; + struct rte_eth_dev_info info; const char *new_devargs; const char *vf_mac; int err = 0; + int ret; ovs_mutex_lock(&dpdk_mutex); @@ -1947,11 +1988,4 @@ netdev_dpdk_set_config(struct netdev *netdev, const struct smap *args, dpdk_set_rxq_config(dev, args); - dpdk_process_queue_size(netdev, args, "n_rxq_desc", - NIC_PORT_DEFAULT_RXQ_SIZE, - &dev->requested_rxq_size); - dpdk_process_queue_size(netdev, args, "n_txq_desc", - NIC_PORT_DEFAULT_TXQ_SIZE, - &dev->requested_txq_size); - new_devargs = smap_get(args, "dpdk-devargs"); @@ -2073,4 +2107,9 @@ netdev_dpdk_set_config(struct netdev *netdev, const struct smap *args, } + ret = rte_eth_dev_info_get(dev->port_id, &info); + + dpdk_process_queue_size(netdev, args, !ret ? &info : NULL, true); + dpdk_process_queue_size(netdev, args, !ret ? &info : NULL, false); + out: ovs_mutex_unlock(&dev->mutex);