From patchwork Wed Jul 12 07:02:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adrian Moreno X-Patchwork-Id: 1806638 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=aUNncVCT; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4R17xV5VrSz20ZZ for ; Wed, 12 Jul 2023 17:03:26 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id D6A3C41955; Wed, 12 Jul 2023 07:03:22 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org D6A3C41955 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=aUNncVCT X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hy_Dh9ynJIRs; Wed, 12 Jul 2023 07:03:20 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 2E23B40448; Wed, 12 Jul 2023 07:03:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2E23B40448 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 26403C0DD7; Wed, 12 Jul 2023 07:03:17 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 475A4C0DD7 for ; Wed, 12 Jul 2023 07:03:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 1D49740448 for ; Wed, 12 Jul 2023 07:03:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 1D49740448 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dRTTPD52CP7Y for ; Wed, 12 Jul 2023 07:03:13 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 84FFD402DC Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 84FFD402DC for ; Wed, 12 Jul 2023 07:03:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1689145392; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=urcNNQEmqmeGdFD/XIZI0sxmvOtVSnicjYsCcliBmcY=; b=aUNncVCTeSOuQwSlNnMS+SDVMZn/Clije9tNY4ix9LVjnJw1bjQzrZJqQR3ChpWoiZuDac x7jvxrXfjsuTi0Lj+977w7fwJxyH3zocrk6EMQVRbNIaXDs+w3kFsps3Mi4I6fZyzgTADw tuXu87O5+KBKWb38zlPK8M+WWGpt878= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-645-15Nr_i0SPj-SitRMcZQE7w-1; Wed, 12 Jul 2023 03:03:10 -0400 X-MC-Unique: 15Nr_i0SPj-SitRMcZQE7w-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E4FEE3C0C48B; Wed, 12 Jul 2023 07:03:09 +0000 (UTC) Received: from antares.redhat.com (unknown [10.39.192.238]) by smtp.corp.redhat.com (Postfix) with ESMTP id E993C492B03; Wed, 12 Jul 2023 07:03:08 +0000 (UTC) From: Adrian Moreno To: dev@openvswitch.org Date: Wed, 12 Jul 2023 09:02:48 +0200 Message-ID: <20230712070259.531904-5-amorenoz@redhat.com> In-Reply-To: <20230712070259.531904-1-amorenoz@redhat.com> References: <20230712070259.531904-1-amorenoz@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: simon.horman@corigine.com, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v3 4/7] netdev-linux: use 64-bit rates in htb tc classes X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Currently, htb rates are capped at ~34Gbps because they are internally expressed as 32-bit fields. Move min and max rates to 64-bit fields and use TCA_HTB_RATE64 and TCA_HTB_CEIL64 to configure HTC classes to break this barrier. In order to test this, create a dummy tuntap device and set it's speed to a very high value so we can try adding a QoS queue with big rates. Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=2137619 Signed-off-by: Adrian Moreno Acked-by: Eelco Chaudron --- acinclude.m4 | 10 ++++++++++ lib/netdev-linux.c | 41 ++++++++++++++++++++++++++++++++--------- tests/atlocal.in | 1 + tests/system-traffic.at | 28 ++++++++++++++++++++++++++++ 4 files changed, 71 insertions(+), 9 deletions(-) diff --git a/acinclude.m4 b/acinclude.m4 index 690a13c25..28d028f37 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -218,6 +218,16 @@ AC_DEFUN([OVS_CHECK_LINUX_TC], [ ])], [AC_DEFINE([HAVE_TCA_STATS_PKT64], [1], [Define to 1 if TCA_STATS_PKT64 is available.])]) + + AC_COMPILE_IFELSE([ + AC_LANG_PROGRAM([#include ], [ + int x = TCA_HTB_RATE64; + ])], + [AC_SUBST(HAVE_TCA_HTB_RATE64,yes) + AC_DEFINE([HAVE_TCA_HTB_RATE64], [1], + [Define to 1 if TCA_HTB_RATE64 is available.])], + [AC_SUBST(HAVE_TCA_HTB_RATE64,no)] + ) ]) dnl OVS_CHECK_LINUX_SCTP_CT diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c index 1956624b7..652bc3d5b 100644 --- a/lib/netdev-linux.c +++ b/lib/netdev-linux.c @@ -4601,13 +4601,13 @@ static const struct tc_ops tc_ops_netem = { struct htb { struct tc tc; - unsigned int max_rate; /* In bytes/s. */ + uint64_t max_rate; /* In bytes/s. */ }; struct htb_class { struct tc_queue tc_queue; - unsigned int min_rate; /* In bytes/s. */ - unsigned int max_rate; /* In bytes/s. */ + uint64_t min_rate; /* In bytes/s. */ + uint64_t max_rate; /* In bytes/s. */ unsigned int burst; /* In bytes. */ unsigned int priority; /* Lower values are higher priorities. */ }; @@ -4695,8 +4695,8 @@ htb_setup_class__(struct netdev *netdev, unsigned int handle, if ((class->min_rate / HTB_RATE2QUANTUM) < mtu) { opt.quantum = mtu; } - opt.buffer = tc_calc_buffer(opt.rate.rate, mtu, class->burst); - opt.cbuffer = tc_calc_buffer(opt.ceil.rate, mtu, class->burst); + opt.buffer = tc_calc_buffer(class->min_rate, mtu, class->burst); + opt.cbuffer = tc_calc_buffer(class->max_rate, mtu, class->burst); opt.prio = class->priority; tcmsg = netdev_linux_tc_make_request(netdev, RTM_NEWTCLASS, NLM_F_CREATE, @@ -4709,15 +4709,26 @@ htb_setup_class__(struct netdev *netdev, unsigned int handle, nl_msg_put_string(&request, TCA_KIND, "htb"); opt_offset = nl_msg_start_nested(&request, TCA_OPTIONS); + +#ifdef HAVE_TCA_HTB_RATE64 + if (class->min_rate > UINT32_MAX) { + nl_msg_put_u64(&request, TCA_HTB_RATE64, class->min_rate); + } + if (class->max_rate > UINT32_MAX) { + nl_msg_put_u64(&request, TCA_HTB_CEIL64, class->max_rate); + } +#endif nl_msg_put_unspec(&request, TCA_HTB_PARMS, &opt, sizeof opt); - tc_put_rtab(&request, TCA_HTB_RTAB, &opt.rate, 0); - tc_put_rtab(&request, TCA_HTB_CTAB, &opt.ceil, 0); + + tc_put_rtab(&request, TCA_HTB_RTAB, &opt.rate, class->min_rate); + tc_put_rtab(&request, TCA_HTB_CTAB, &opt.ceil, class->max_rate); nl_msg_end_nested(&request, opt_offset); error = tc_transact(&request, NULL); if (error) { VLOG_WARN_RL(&rl, "failed to replace %s class %u:%u, parent %u:%u, " - "min_rate=%u max_rate=%u burst=%u prio=%u (%s)", + "min_rate=%"PRIu64" max_rate=%"PRIu64" burst=%u prio=%u " + "(%s)", netdev_get_name(netdev), tc_get_major(handle), tc_get_minor(handle), tc_get_major(parent), tc_get_minor(parent), @@ -4737,6 +4748,10 @@ htb_parse_tca_options__(struct nlattr *nl_options, struct htb_class *class) static const struct nl_policy tca_htb_policy[] = { [TCA_HTB_PARMS] = { .type = NL_A_UNSPEC, .optional = false, .min_len = sizeof(struct tc_htb_opt) }, +#ifdef HAVE_TCA_HTB_RATE64 + [TCA_HTB_RATE64] = { .type = NL_A_U64, .optional = true }, + [TCA_HTB_CEIL64] = { .type = NL_A_U64, .optional = true }, +#endif }; struct nlattr *attrs[ARRAY_SIZE(tca_htb_policy)]; @@ -4751,7 +4766,15 @@ htb_parse_tca_options__(struct nlattr *nl_options, struct htb_class *class) htb = nl_attr_get(attrs[TCA_HTB_PARMS]); class->min_rate = htb->rate.rate; class->max_rate = htb->ceil.rate; - class->burst = tc_ticks_to_bytes(htb->rate.rate, htb->buffer); +#ifdef HAVE_TCA_HTB_RATE64 + if (attrs[TCA_HTB_RATE64]) { + class->min_rate = nl_attr_get_u64(attrs[TCA_HTB_RATE64]); + } + if (attrs[TCA_HTB_CEIL64]) { + class->max_rate = nl_attr_get_u64(attrs[TCA_HTB_CEIL64]); + } +#endif + class->burst = tc_ticks_to_bytes(class->min_rate, htb->buffer); class->priority = htb->prio; return 0; } diff --git a/tests/atlocal.in b/tests/atlocal.in index 94b5c4d0b..ffdea5cc0 100644 --- a/tests/atlocal.in +++ b/tests/atlocal.in @@ -7,6 +7,7 @@ HAVE_UNWIND='@HAVE_UNWIND@' EGREP='@EGREP@' PYTHON3='@PYTHON3@' CFLAGS='@CFLAGS@' +HAVE_TCA_HTB_RATE64='@HAVE_TCA_HTB_RATE64@' # PYTHONCOERCECLOCALE=0 disables the Unicode compatibility warning on # stderr that breaks almost any Python3 test (PEP 0538) diff --git a/tests/system-traffic.at b/tests/system-traffic.at index a05ca311c..ce73415e2 100644 --- a/tests/system-traffic.at +++ b/tests/system-traffic.at @@ -2354,6 +2354,34 @@ AT_CHECK([tc class show dev ovs-p1 | grep -q 'class htb .* HTB_CONF']) OVS_TRAFFIC_VSWITCHD_STOP AT_CLEANUP +AT_SETUP([QoS - 64bit]) +AT_SKIP_IF([test $HAVE_TC = no]) +AT_SKIP_IF([test $HAVE_TCA_HTB_RATE64 = no]) +OVS_TRAFFIC_VSWITCHD_START() + +ADD_NAMESPACES(at_ns0, at_ns1) +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24") +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24") + +dnl Configure the QoS with rates that require 64bits, i.e: > 34Gbps. +AT_CHECK([ovs-vsctl set port ovs-p0 qos=@qos -- set port ovs-p1 qos=@qos dnl + -- --id=@qos create qos dnl + type=linux-htb other-config:max-rate=50000000000 queues:0=@queue dnl + -- --id=@queue create queue dnl + other_config:min-rate=40000000000 other_config:max-rate=50000000000 dnl + other_config:burst=5000000], + [ignore], [ignore]) + +OVS_WAIT_UNTIL([tc qdisc show dev ovs-p0 | grep -q htb]) +OVS_WAIT_UNTIL([tc qdisc show dev ovs-p1 | grep -q htb]) + +m4_define([HTB_CONF], [rate 40Gbit ceil 50Gbit burst 620000b cburst 618750b]) +AT_CHECK([tc class show dev ovs-p0 | grep -q 'class htb .* HTB_CONF']) +AT_CHECK([tc class show dev ovs-p1 | grep -q 'class htb .* HTB_CONF']) + +OVS_TRAFFIC_VSWITCHD_STOP +AT_CLEANUP + AT_BANNER([conntrack]) AT_SETUP([conntrack - controller])