From patchwork Fri May 26 08:57:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Traynor X-Patchwork-Id: 1786442 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=JcMTLUDR; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QSJj934tpz20Pb for ; Fri, 26 May 2023 18:57:49 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 5750342904; Fri, 26 May 2023 08:57:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 5750342904 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=JcMTLUDR X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y66ZLpdvsH6G; Fri, 26 May 2023 08:57:45 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 9D830428F2; Fri, 26 May 2023 08:57:44 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 9D830428F2 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7C0FFC0036; Fri, 26 May 2023 08:57:44 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3DB28C002A for ; Fri, 26 May 2023 08:57:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id A652442CA6 for ; Fri, 26 May 2023 08:57:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A652442CA6 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=JcMTLUDR X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZDCoAdu3ipfs for ; Fri, 26 May 2023 08:57:39 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 9BD7C42C9D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 9BD7C42C9D for ; Fri, 26 May 2023 08:57:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685091458; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VQIeIG52ukXcEG2GKAi8VLJI5TGnVLYb9Pdq2K2O1Gk=; b=JcMTLUDRe9SE5EpsLOixmb7B5q8rsl8nRpDLrweK033I6I2T9ZmJaCfs0PxttXOaeFK6Z6 TNog+WHbD6SsM+WKuYBfD/r6Abbk4eClfz5GTSqny4WXbrzJXFVzNA0MYXYEHB0uGzancj oUyZkCUw9FGmQd4mHrFB06oroMOjdmE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-60-phSqzRJSNdqlnSBkIj9rEw-1; Fri, 26 May 2023 04:57:34 -0400 X-MC-Unique: phSqzRJSNdqlnSBkIj9rEw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3C059811E85; Fri, 26 May 2023 08:57:34 +0000 (UTC) Received: from rh.Home (unknown [10.39.192.214]) by smtp.corp.redhat.com (Postfix) with ESMTP id 334E62166B2E; Fri, 26 May 2023 08:57:33 +0000 (UTC) From: Kevin Traynor To: dev@openvswitch.org Date: Fri, 26 May 2023 09:57:12 +0100 Message-Id: <20230526085712.411852-3-ktraynor@redhat.com> In-Reply-To: <20230526085712.411852-1-ktraynor@redhat.com> References: <20230526085712.411852-1-ktraynor@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: simon.horman@corigine.com, david.marchand@redhat.com, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 2/2] netdev-dpdk: Check rx/tx descriptor sizes for device. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" By default OVS configures 2048 descriptors for tx and rx queues on DPDK devices. It also allows the user to configure those values. If the values used are not acceptable to the device then queue setup would fail. The device exposes it's max/min/alignment requirements and OVS applies some limits also. Use these to ensure an acceptable value is used for the number of descriptors on a device tx/rx. If the default or user value is not acceptable, adjust to a suitable value and log. Reported-at: https://bugzilla.redhat.com/2119876 Reviewed-by: David Marchand Reviewed-by: Simon Horman Signed-off-by: Kevin Traynor --- lib/netdev-dpdk.c | 60 ++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 49 insertions(+), 11 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 2d9afc493..6bf672d43 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -1911,8 +1911,29 @@ dpdk_set_rxq_config(struct netdev_dpdk *dev, const struct smap *args) static void dpdk_process_queue_size(struct netdev *netdev, const struct smap *args, - const char *flag, int default_size, int *new_size) + struct rte_eth_dev_info *info, bool is_rx) { - int queue_size = smap_get_int(args, flag, default_size); + struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); + struct rte_eth_desc_lim *lim; + int default_size, queue_size, cur_size, new_requested_size; + int *cur_requested_size; + bool reconfig = false; + if (is_rx) { + default_size = NIC_PORT_DEFAULT_RXQ_SIZE; + new_requested_size = smap_get_int(args, "n_rxq_desc", default_size); + cur_size = dev->rxq_size; + cur_requested_size = &dev->requested_rxq_size; + lim = info ? &info->rx_desc_lim : NULL; + } else { + default_size = NIC_PORT_DEFAULT_TXQ_SIZE; + new_requested_size = smap_get_int(args, "n_txq_desc", default_size); + cur_size = dev->txq_size; + cur_requested_size = &dev->requested_txq_size; + lim = info ? &info->tx_desc_lim : NULL; + } + + queue_size = new_requested_size; + + /* Check for OVS limits. */ if (queue_size <= 0 || queue_size > NIC_PORT_MAX_Q_SIZE || !is_pow2(queue_size)) { @@ -1920,7 +1941,24 @@ dpdk_process_queue_size(struct netdev *netdev, const struct smap *args, } - if (queue_size != *new_size) { - *new_size = queue_size; + if (lim) { + /* Check for device limits. */ + if (lim->nb_align) { + queue_size = ROUND_UP(queue_size, lim->nb_align); + } + queue_size = MIN(queue_size, lim->nb_max); + queue_size = MAX(queue_size, lim->nb_min); + } + + *cur_requested_size = queue_size; + + if (cur_size != queue_size) { netdev_request_reconfigure(netdev); + reconfig = true; + } + if (new_requested_size != queue_size) { + VLOG(reconfig ? VLL_INFO : VLL_DBG, + "%s: Unable to set the number of %s descriptors to %d. " + "Adjusted to %d.", netdev_get_name(netdev), + is_rx ? "rx": "tx", new_requested_size, queue_size); } } @@ -1938,7 +1976,9 @@ netdev_dpdk_set_config(struct netdev *netdev, const struct smap *args, {RTE_ETH_FC_RX_PAUSE, RTE_ETH_FC_FULL } }; + struct rte_eth_dev_info info; const char *new_devargs; const char *vf_mac; int err = 0; + int ret; ovs_mutex_lock(&dpdk_mutex); @@ -1947,11 +1987,4 @@ netdev_dpdk_set_config(struct netdev *netdev, const struct smap *args, dpdk_set_rxq_config(dev, args); - dpdk_process_queue_size(netdev, args, "n_rxq_desc", - NIC_PORT_DEFAULT_RXQ_SIZE, - &dev->requested_rxq_size); - dpdk_process_queue_size(netdev, args, "n_txq_desc", - NIC_PORT_DEFAULT_TXQ_SIZE, - &dev->requested_txq_size); - new_devargs = smap_get(args, "dpdk-devargs"); @@ -2009,4 +2042,9 @@ netdev_dpdk_set_config(struct netdev *netdev, const struct smap *args, } + ret = rte_eth_dev_info_get(dev->port_id, &info); + + dpdk_process_queue_size(netdev, args, !ret ? &info : NULL, true); + dpdk_process_queue_size(netdev, args, !ret ? &info : NULL, false); + vf_mac = smap_get(args, "dpdk-vf-mac"); if (vf_mac) {