From patchwork Wed Feb 28 14:20:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mateusz Polchlopek X-Patchwork-Id: 1905836 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=4eOKT+7P; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TlGwP4rcZz23qP for ; Thu, 29 Feb 2024 01:30:13 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id B98D081EC1; Wed, 28 Feb 2024 14:30:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lxKGMWsTPjdD; Wed, 28 Feb 2024 14:30:11 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 0302081DEA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1709130611; bh=MUnS3+dx5Zo87v5YZcwpbWK/doM9ftCHy/30vyJAleg=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=4eOKT+7PGuuyvMXL8/cpFckLTo3yNuXiNCW+C/eWV9M+NzhfES1+Fc0ITsjiZgsiT 55VrfQ4HMVr/dWJOB/zYWhF6cIRRr1PHhpqR3Fn8BJZhA50OYLYlAte2Fxt8Aa/1dr 33EaysiWkSHWMpVO/BqUlnBFSu8ux4jsD1pQ79ISogOdVHTb/4QBgfNl4vqNIPa8JV RLXK00as6GchtJxRFOBZUnx8qhDXnP4lvAxkmnD8SqReGtZu+VzlcvnwehGhIvp2yg H16FDsQ2nbvZ33yI12k7HzvnKQCIvrBB59wtBBZ8WzRlwsEPdC90wL2vCZkbCaswRQ yhlDadD87Myow== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 0302081DEA; Wed, 28 Feb 2024 14:30:11 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id DD5F91BF391 for ; Wed, 28 Feb 2024 14:30:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id CB14560A57 for ; Wed, 28 Feb 2024 14:30:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3aPTJAnwC1FW for ; Wed, 28 Feb 2024 14:30:09 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=198.175.65.14; helo=mgamail.intel.com; envelope-from=mateusz.polchlopek@intel.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org EE9B8605D4 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org EE9B8605D4 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by smtp3.osuosl.org (Postfix) with ESMTPS id EE9B8605D4 for ; Wed, 28 Feb 2024 14:30:08 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6600,9927,10998"; a="7354553" X-IronPort-AV: E=Sophos;i="6.06,190,1705392000"; d="scan'208";a="7354553" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Feb 2024 06:30:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,190,1705392000"; d="scan'208";a="7659023" Received: from irvmail002.ir.intel.com ([10.43.11.120]) by fmviesa006.fm.intel.com with ESMTP; 28 Feb 2024 06:30:04 -0800 Received: from fedora.igk.intel.com (Metan_eth.igk.intel.com [10.123.220.124]) by irvmail002.ir.intel.com (Postfix) with ESMTP id B7EF836342; Wed, 28 Feb 2024 14:30:03 +0000 (GMT) From: Mateusz Polchlopek To: intel-wired-lan@lists.osuosl.org Date: Wed, 28 Feb 2024 09:20:54 -0500 Message-Id: <20240228142054.474626-6-mateusz.polchlopek@intel.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20240228142054.474626-1-mateusz.polchlopek@intel.com> References: <20240228142054.474626-1-mateusz.polchlopek@intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1709130609; x=1740666609; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zLfIHt2dGuwOkbpC/bp3U0IB/u6iLNKIZuLuNDYezO0=; b=XkJ+41BTgUH62F+d0keeqyJiJd8smdQn3DlUOOYoT/ZU9OxnJfkfGPUK FnSMIUe49fvXiYBqeA2+B08tUQsPCIAaHNljkyWS27HT93sTUgFL8LSaR DQAoMkos2uCDwDo/rW3znrLh1xGjKFeErKQcZwiT+Eajloz5z/0nmr9M8 x/VmBrjB0m0x8KEmA/wqaLm4PT0npdrsW2NvyZFsqNw6Ntzh4LVDZg6y4 WSquFwK86BVZT3k2sMQfweaYiR5oQ9BtdkNCKmh8Gr3ITpmfwbIs4aQsn okw6TFhTYfjL75J57oBCp56/vlh9k/5zOZWNIO6riDB570852VpWqMHxm g==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=intel.com X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=XkJ+41BT Subject: [Intel-wired-lan] [PATCH iwl-next v5 5/5] ice: Document tx_scheduling_layers parameter X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: andrew@lunn.ch, jiri@resnulli.us, michal.wilczynski@intel.com, Mateusz Polchlopek , netdev@vger.kernel.org, lukasz.czapnik@intel.com, victor.raj@intel.com, horms@kernel.org, przemyslaw.kitszel@intel.com, kuba@kernel.org Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" From: Michal Wilczynski New driver specific parameter 'tx_scheduling_layers' was introduced. Describe parameter in the documentation. Signed-off-by: Michal Wilczynski Co-developed-by: Mateusz Polchlopek Signed-off-by: Mateusz Polchlopek Acked-by: Jakub Kicinski --- Documentation/networking/devlink/ice.rst | 47 ++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst index efc6be109dc3..c9253e1f61c9 100644 --- a/Documentation/networking/devlink/ice.rst +++ b/Documentation/networking/devlink/ice.rst @@ -36,6 +36,53 @@ Parameters The latter allows for bandwidth higher than external port speed when looping back traffic between VFs. Works with 8x10G and 4x25G cards. + * - ``tx_scheduling_layers`` + - permanent + - The ice hardware uses hierarchical scheduling for Tx with a fixed + number of layers in the scheduling tree. Each of them are decision + points. Root node represents a port, while all the leaves represent + the queues. This way of configuring the Tx scheduler allows features + like DCB or devlink-rate (documented below) to configure how much + bandwidth is given to any given queue or group of queues, enabling + fine-grained control because scheduling parameters can be configured + at any given layer of the tree. + + The default 9-layer tree topology was deemed best for most workloads, + as it gives an optimal ratio of performance to configurability. However, + for some specific cases, this 9-layer topology might not be desired. + One example would be sending traffic to queues that are not a multiple + of 8. Because the maximum radix is limited to 8 in 9-layer topology, + the 9th queue has a different parent than the rest, and it's given + more bandwidth credits. This causes a problem when the system is + sending traffic to 9 queues: + + | tx_queue_0_packets: 24163396 + | tx_queue_1_packets: 24164623 + | tx_queue_2_packets: 24163188 + | tx_queue_3_packets: 24163701 + | tx_queue_4_packets: 24163683 + | tx_queue_5_packets: 24164668 + | tx_queue_6_packets: 23327200 + | tx_queue_7_packets: 24163853 + | tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th + + To address this need, you can switch to a 5-layer topology, which + changes the maximum topology radix to 512. With this enhancement, + the performance characteristic is equal as all queues can be assigned + to the same parent in the tree. The obvious drawback of this solution + is a lower configuration depth of the tree. + + Use the ``tx_scheduling_layer`` parameter with the devlink command + to change the transmit scheduler topology. To use 5-layer topology, + use a value of 5. For example: + $ devlink dev param set pci/0000:16:00.0 name tx_scheduling_layers + value 5 cmode permanent + Use a value of 9 to set it back to the default value. + + You must reboot the system for the selected topology to take effect. + + To verify that value has been set: + $ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers Info versions =============