From patchwork Wed Jun 9 09:52:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Horman X-Patchwork-Id: 1489771 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.a=rsa-sha256 header.s=20150623 header.b=015GMexn; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4G0Mpq5n2Vz9sCD for ; Wed, 9 Jun 2021 19:52:35 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 1020160A5D; Wed, 9 Jun 2021 09:52:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5R_-W4Sqfj_y; Wed, 9 Jun 2021 09:52:32 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 1E8D9608EB; Wed, 9 Jun 2021 09:52:31 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CDA2AC0011; Wed, 9 Jun 2021 09:52:30 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id CDFB3C0025 for ; Wed, 9 Jun 2021 09:52:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id BDFE383C33 for ; Wed, 9 Jun 2021 09:52:28 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=netronome-com.20150623.gappssmtp.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GVOatxGvK4Sw for ; Wed, 9 Jun 2021 09:52:27 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by smtp1.osuosl.org (Postfix) with ESMTPS id 2355C83C27 for ; Wed, 9 Jun 2021 09:52:27 +0000 (UTC) Received: by mail-ej1-x62a.google.com with SMTP id og14so32206247ejc.5 for ; Wed, 09 Jun 2021 02:52:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pvyb4ENd0/JVWfQ06/SD8tKlJy+F/J/IUNLbuuSmpRc=; b=015GMexnmU2h9zU5ZZ1YGDvDpD7tKUIvt2SBfo20X91V6T5qSQGKgYiZep9eGPNVOv lt6guD+y5tpLwVLubhTw3k71LIMYrVJljrCQx0STt+/hSKh8x7emH3SkFZXn08EfVr0/ /2y9Oo4YnLl8tizKT/YwCNCSQfiXHBrIv/FtHfTl0jZKV9tjf3lb6cWts6E3wBeayEdP qhCmCR2ZeqQN3Vpaj1K6VMntq23iI2fE2kwKDHC5dzORVPVcf2+KGjuvtJYLXTYNGHY0 NvsX5qS2S15toP85iZ2Bieaa8E0V8LG1a49ZaHn2YogMyk/tEhemL6KXpRJDW7lwIpdj WhCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pvyb4ENd0/JVWfQ06/SD8tKlJy+F/J/IUNLbuuSmpRc=; b=kfb4ka5FtLQBYzayMClVCacl0nBlvzr94+e6Cn2AfOmh6vcTps8Dx1mii/iUY6Fo8g Xz8axvR8HGn3V2NwkvDbCx1U+5FrKYtvSNsZzStoxwRiGjfdIe1kPimoPmHguvx/S2qU ccS+5Qj4xsaCatgOcQJGysdSyUonvyT1aqE5Rh5DkP1yBSR5BjHeZVRI/QwwfTOwR983 7m1WT3SclZAvFDK6zOUE04u0Z8uUp6PcYTxgimfi3KUEAPV4goaCM+oSMBNB62feiksh YPgHX7RbOPPKoNSkrSPZ3JTJf44CDqxZEIMAgZ6B17nzNGqi/nL6jsVFinxD1UIPY5ku qEZw== X-Gm-Message-State: AOAM531n2eu6mv47SWqodBbe0xhV3aLansri/Rvq7jNMjQECWekA3lXA 0kJV3mNkERRcDVlHDqaH5UKA/u6jINuP+SnoEduAJ38EnGITThC/TPtmU7WgKTe5BFFXP8EXZyz sv6R7JDEgpTOVkwqrcK16DX8cjwHl3xZBGFV8osKTvyVkytqNX7ViafIvNJKkG++sNmteZU3V2+ Uu X-Google-Smtp-Source: ABdhPJw39c/JMsmdg9I7QulioDNqynA4wAFZxF+49cXtu71WonV61Zi0sXRa/mRZtZmp6Y5x4hJ66g== X-Received: by 2002:a17:906:3845:: with SMTP id w5mr28425227ejc.518.1623232344332; Wed, 09 Jun 2021 02:52:24 -0700 (PDT) Received: from momiji.horms.nl ([2001:982:756:703:d63d:7eff:fe99:ac9d]) by smtp.gmail.com with ESMTPSA id nc26sm840636ejc.106.2021.06.09.02.52.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 02:52:23 -0700 (PDT) From: Simon Horman To: dev@openvswitch.org Date: Wed, 9 Jun 2021 11:52:08 +0200 Message-Id: <20210609095209.6205-2-simon.horman@netronome.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210609095209.6205-1-simon.horman@netronome.com> References: <20210609095209.6205-1-simon.horman@netronome.com> MIME-Version: 1.0 Cc: Baowen Zheng , Yong Xu , oss-drivers@netronome.com, Simon Horman , Tianyu Yuan , Marcelo Ricardo Leitner Subject: [ovs-dev] [PATCH v3 1/2] add port-based ingress policing based packet-per-second rate-limiting X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Yong Xu OVS has support for using policing to enforce a rate limit in kilobits per second. This is configured using OVSDB. f.e. $ ovs-vsctl set interface tap0 ingress_policing_rate=1000 $ ovs-vsctl set interface tap0 ingress_policing_burst=100 This patch adds a related feature, allowing policing to enforce a rate limit in kilo-packets per second. This is also configured using OVSDB. $ ovs-vsctl set interface tap0 ingress_policing_kpkts_rate=1000 $ ovs-vsctl set interface tap0 ingress_policing_kpkts_burst=100 The kilo-bit and kilo-packet rate limits may be used separately or in combination. Add separate action for BPS and PPS in netlink message. Revise code and change action result to pipe to allow traffic pipe into second action. This patch implements the feature for: * OVSDB (northbound API) * TC policer when used both with and without TC offload (kernel API) Signed-off-by: Yong Xu Signed-off-by: Simon Horman --- v2 - correct typo v3 - Remove the for loop in function nl_msg_put_act_police() - Remove unused enum definition for qos type - Define 1 kpkts as 1000 packets rather than 1024 packets - Update the description for the new item in ovsdb - Fix some format warnings according robot's comments --- acinclude.m4 | 6 +- include/linux/pkt_cls.h | 6 +- lib/netdev-dpdk.c | 4 +- lib/netdev-linux-private.h | 4 +- lib/netdev-linux.c | 109 ++++++++++++++++++++++++++----------- lib/netdev-provider.h | 11 ++-- lib/netdev.c | 15 +++-- lib/netdev.h | 3 +- vswitchd/bridge.c | 6 +- vswitchd/vswitch.ovsschema | 10 +++- vswitchd/vswitch.xml | 59 +++++++++++++++++++- 11 files changed, 179 insertions(+), 54 deletions(-) diff --git a/acinclude.m4 b/acinclude.m4 index 15a54d636..18c52f63a 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -209,10 +209,10 @@ dnl Configure Linux tc compat. AC_DEFUN([OVS_CHECK_LINUX_TC], [ AC_COMPILE_IFELSE([ AC_LANG_PROGRAM([#include ], [ - int x = TCA_FLOWER_KEY_CT_FLAGS_REPLY; + int x = TCA_POLICE_PKTRATE64; ])], - [AC_DEFINE([HAVE_TCA_FLOWER_KEY_CT_FLAGS_REPLY], [1], - [Define to 1 if TCA_FLOWER_KEY_CT_FLAGS_REPLY is available.])]) + [AC_DEFINE([HAVE_TCA_POLICE_PKTRATE64], [1], + [Define to 1 if TCA_POLICE_PKTRATE64 is available.])]) AC_CHECK_MEMBERS([struct tcf_t.firstuse], [], [], [#include ]) diff --git a/include/linux/pkt_cls.h b/include/linux/pkt_cls.h index bc51a5767..ba82e690e 100644 --- a/include/linux/pkt_cls.h +++ b/include/linux/pkt_cls.h @@ -1,7 +1,7 @@ #ifndef __LINUX_PKT_CLS_WRAPPER_H #define __LINUX_PKT_CLS_WRAPPER_H 1 -#if defined(__KERNEL__) || defined(HAVE_TCA_FLOWER_KEY_CT_FLAGS_REPLY) +#if defined(__KERNEL__) || defined(HAVE_TCA_POLICE_PKTRATE64) #include_next #else @@ -104,6 +104,10 @@ enum { TCA_POLICE_RESULT, TCA_POLICE_TM, TCA_POLICE_PAD, + TCA_POLICE_RATE64, + TCA_POLICE_PEAKRATE64, + TCA_POLICE_PKTRATE64, + TCA_POLICE_PKTBURST64, __TCA_POLICE_MAX #define TCA_POLICE_RESULT TCA_POLICE_RESULT }; diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 9d8096668..2b255f579 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -3389,7 +3389,9 @@ netdev_dpdk_policer_construct(uint32_t rate, uint32_t burst) static int netdev_dpdk_set_policing(struct netdev* netdev, uint32_t policer_rate, - uint32_t policer_burst) + uint32_t policer_burst, + uint32_t policer_kpkts_rate OVS_UNUSED, + uint32_t policer_kpkts_burst OVS_UNUSED) { struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); struct ingress_policer *policer; diff --git a/lib/netdev-linux-private.h b/lib/netdev-linux-private.h index c7c515f70..deb015bdb 100644 --- a/lib/netdev-linux-private.h +++ b/lib/netdev-linux-private.h @@ -75,8 +75,10 @@ struct netdev_linux { int mtu; unsigned int ifi_flags; long long int carrier_resets; - uint32_t kbits_rate; /* Policing data. */ + uint32_t kbits_rate; /* Policing data - kbps */ uint32_t kbits_burst; + uint32_t kpkts_rate; /* Policing data - kpps */ + uint32_t kpkts_burst; int vport_stats_error; /* Cached error code from vport_get_stats(). 0 or an errno value. */ int netdev_mtu_error; /* Cached error code from SIOCGIFMTU diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c index 19fdd7243..07ece0c7f 100644 --- a/lib/netdev-linux.c +++ b/lib/netdev-linux.c @@ -485,8 +485,10 @@ static struct tcmsg *netdev_linux_tc_make_request(const struct netdev *, int type, unsigned int flags, struct ofpbuf *); -static int tc_add_policer(struct netdev *, - uint32_t kbits_rate, uint32_t kbits_burst); + +static int tc_add_policer(struct netdev *, uint32_t kbits_rate, + uint32_t kbits_burst, uint32_t kpkts_rate, + uint32_t kpkts_burst); static int tc_parse_qdisc(const struct ofpbuf *, const char **kind, struct nlattr **options); @@ -2595,24 +2597,62 @@ tc_matchall_fill_police(uint32_t kbits_rate, uint32_t kbits_burst) } static void -nl_msg_put_act_police(struct ofpbuf *request, struct tc_police police) +nl_msg_act_police_start_nest(struct ofpbuf *request, uint32_t prio, + size_t *offset, size_t *act_offset) { - size_t offset; - + *act_offset = nl_msg_start_nested(request, prio); nl_msg_put_string(request, TCA_ACT_KIND, "police"); - offset = nl_msg_start_nested(request, TCA_ACT_OPTIONS); - nl_msg_put_unspec(request, TCA_POLICE_TBF, &police, sizeof police); - tc_put_rtab(request, TCA_POLICE_RATE, &police.rate); - nl_msg_put_u32(request, TCA_POLICE_RESULT, TC_ACT_UNSPEC); + *offset = nl_msg_start_nested(request, TCA_ACT_OPTIONS); +} + +static void +nl_msg_act_police_end_nest(struct ofpbuf *request, size_t offset, + size_t act_offset) +{ + nl_msg_put_u32(request, TCA_POLICE_RESULT, TC_ACT_PIPE); nl_msg_end_nested(request, offset); + nl_msg_end_nested(request, act_offset); +} + +static void +nl_msg_put_act_police(struct ofpbuf *request, struct tc_police police, + uint32_t kpkts_rate, uint32_t kpkts_burst) +{ + size_t offset, act_offset; + uint32_t prio = 0; + /* used for PPS, set rate as 0 to act as a single action */ + struct tc_police null_police = {0}; + + if (police.rate.rate) { + nl_msg_act_police_start_nest(request, ++prio, &offset, &act_offset); + tc_put_rtab(request, TCA_POLICE_RATE, &police.rate); + nl_msg_put_unspec(request, TCA_POLICE_TBF, &police, sizeof police); + nl_msg_act_police_end_nest(request, offset, act_offset); + } + if (kpkts_rate) { + unsigned int pkt_burst_ticks, pps_rate, size; + nl_msg_act_police_start_nest(request, ++prio, &offset, &act_offset); + pps_rate = kpkts_rate * 1000; + size = MIN(UINT32_MAX / 1000, kpkts_burst) * 1000; + /* Here tc_bytes_to_ticks is used to convert packets rather than bytes + to ticks. */ + pkt_burst_ticks = tc_bytes_to_ticks(pps_rate, size); + nl_msg_put_u64(request, TCA_POLICE_PKTRATE64, (uint64_t) pps_rate); + nl_msg_put_u64(request, TCA_POLICE_PKTBURST64, + (uint64_t) pkt_burst_ticks); + nl_msg_put_unspec(request, TCA_POLICE_TBF, &null_police, + sizeof null_police); + nl_msg_act_police_end_nest(request, offset, act_offset); + } } static int tc_add_matchall_policer(struct netdev *netdev, uint32_t kbits_rate, - uint32_t kbits_burst) + uint32_t kbits_burst, uint32_t kpkts_rate, + uint32_t kpkts_burst) { uint16_t eth_type = (OVS_FORCE uint16_t) htons(ETH_P_ALL); - size_t basic_offset, action_offset, inner_offset; + size_t basic_offset, action_offset; uint16_t prio = TC_RESERVED_PRIORITY_POLICE; int ifindex, err = 0; struct tc_police pol_act; @@ -2636,9 +2676,7 @@ tc_add_matchall_policer(struct netdev *netdev, uint32_t kbits_rate, nl_msg_put_string(&request, TCA_KIND, "matchall"); basic_offset = nl_msg_start_nested(&request, TCA_OPTIONS); action_offset = nl_msg_start_nested(&request, TCA_MATCHALL_ACT); - inner_offset = nl_msg_start_nested(&request, 1); - nl_msg_put_act_police(&request, pol_act); - nl_msg_end_nested(&request, inner_offset); + nl_msg_put_act_police(&request, pol_act, kpkts_rate, kpkts_burst); nl_msg_end_nested(&request, action_offset); nl_msg_end_nested(&request, basic_offset); @@ -2678,8 +2716,9 @@ tc_del_matchall_policer(struct netdev *netdev) /* Attempts to set input rate limiting (policing) policy. Returns 0 if * successful, otherwise a positive errno value. */ static int -netdev_linux_set_policing(struct netdev *netdev_, - uint32_t kbits_rate, uint32_t kbits_burst) +netdev_linux_set_policing(struct netdev *netdev_, uint32_t kbits_rate, + uint32_t kbits_burst, uint32_t kpkts_rate, + uint32_t kpkts_burst) { struct netdev_linux *netdev = netdev_linux_cast(netdev_); const char *netdev_name = netdev_get_name(netdev_); @@ -2690,6 +2729,10 @@ netdev_linux_set_policing(struct netdev *netdev_, : !kbits_burst ? 8000 /* Default to 8000 kbits if 0. */ : kbits_burst); /* Stick with user-specified value. */ + kpkts_burst = (!kpkts_rate ? 0 /* Force to 0 if no rate specified. */ + : !kpkts_burst ? 16 /* Default to 16 kpkts if 0. */ + : kpkts_burst); /* Stick with user-specified value. */ + ovs_mutex_lock(&netdev->mutex); if (netdev_linux_netnsid_is_remote(netdev)) { error = EOPNOTSUPP; @@ -2699,7 +2742,9 @@ netdev_linux_set_policing(struct netdev *netdev_, if (netdev->cache_valid & VALID_POLICING) { error = netdev->netdev_policing_error; if (error || (netdev->kbits_rate == kbits_rate && - netdev->kbits_burst == kbits_burst)) { + netdev->kpkts_rate == kpkts_rate && + netdev->kbits_burst == kbits_burst && + netdev->kpkts_burst == kpkts_burst)) { /* Assume that settings haven't changed since we last set them. */ goto out; } @@ -2711,8 +2756,9 @@ netdev_linux_set_policing(struct netdev *netdev_, /* Use matchall for policing when offloadling ovs with tc-flower. */ if (netdev_is_flow_api_enabled()) { error = tc_del_matchall_policer(netdev_); - if (kbits_rate) { - error = tc_add_matchall_policer(netdev_, kbits_rate, kbits_burst); + if (kbits_rate || kpkts_rate) { + error = tc_add_matchall_policer(netdev_, kbits_rate, kbits_burst, + kpkts_rate, kpkts_burst); } ovs_mutex_unlock(&netdev->mutex); return error; @@ -2731,7 +2777,7 @@ netdev_linux_set_policing(struct netdev *netdev_, goto out; } - if (kbits_rate) { + if (kbits_rate || kpkts_rate) { error = tc_add_del_qdisc(ifindex, true, 0, TC_INGRESS); if (error) { VLOG_WARN_RL(&rl, "%s: adding policing qdisc failed: %s", @@ -2739,7 +2785,8 @@ netdev_linux_set_policing(struct netdev *netdev_, goto out; } - error = tc_add_policer(netdev_, kbits_rate, kbits_burst); + error = tc_add_policer(netdev_, kbits_rate, kbits_burst, + kpkts_rate, kpkts_burst); if (error){ VLOG_WARN_RL(&rl, "%s: adding policing action failed: %s", netdev_name, ovs_strerror(error)); @@ -2749,6 +2796,8 @@ netdev_linux_set_policing(struct netdev *netdev_, netdev->kbits_rate = kbits_rate; netdev->kbits_burst = kbits_burst; + netdev->kpkts_rate = kpkts_rate; + netdev->kpkts_burst = kpkts_burst; out: if (!error || error == ENODEV) { @@ -5525,7 +5574,8 @@ netdev_linux_tc_make_request(const struct netdev *netdev, int type, } /* Adds a policer to 'netdev' with a rate of 'kbits_rate' and a burst size - * of 'kbits_burst'. + * of 'kbits_burst', with a rate of 'kpkts_rate' and a burst size of + * 'kpkts_burst'. * * This function is equivalent to running: * /sbin/tc filter add dev parent ffff: protocol all prio 49 @@ -5538,14 +5588,14 @@ netdev_linux_tc_make_request(const struct netdev *netdev, int type, * Returns 0 if successful, otherwise a positive errno value. */ static int -tc_add_policer(struct netdev *netdev, - uint32_t kbits_rate, uint32_t kbits_burst) +tc_add_policer(struct netdev *netdev, uint32_t kbits_rate, + uint32_t kbits_burst, uint32_t kpkts_rate, + uint32_t kpkts_burst) { + size_t basic_offset, police_offset; struct tc_police tc_police; struct ofpbuf request; struct tcmsg *tcmsg; - size_t basic_offset; - size_t police_offset; int error; int mtu = 65535; @@ -5563,7 +5613,6 @@ tc_add_policer(struct netdev *netdev, * tc's point of view. Whatever. */ tc_police.burst = tc_bytes_to_ticks( tc_police.rate.rate, MIN(UINT32_MAX / 1024, kbits_burst) * 1024 / 8); - tcmsg = netdev_linux_tc_make_request(netdev, RTM_NEWTFILTER, NLM_F_EXCL | NLM_F_CREATE, &request); if (!tcmsg) { @@ -5572,12 +5621,10 @@ tc_add_policer(struct netdev *netdev, tcmsg->tcm_parent = tc_make_handle(0xffff, 0); tcmsg->tcm_info = tc_make_handle(49, (OVS_FORCE uint16_t) htons(ETH_P_ALL)); - nl_msg_put_string(&request, TCA_KIND, "basic"); basic_offset = nl_msg_start_nested(&request, TCA_OPTIONS); - police_offset = nl_msg_start_nested(&request, TCA_BASIC_POLICE); - nl_msg_put_unspec(&request, TCA_POLICE_TBF, &tc_police, sizeof tc_police); - tc_put_rtab(&request, TCA_POLICE_RATE, &tc_police.rate); + police_offset = nl_msg_start_nested(&request, TCA_BASIC_ACT); + nl_msg_put_act_police(&request, tc_police, kpkts_rate, kpkts_burst); nl_msg_end_nested(&request, police_offset); nl_msg_end_nested(&request, basic_offset); diff --git a/lib/netdev-provider.h b/lib/netdev-provider.h index 73dce2fca..b5420947d 100644 --- a/lib/netdev-provider.h +++ b/lib/netdev-provider.h @@ -514,13 +514,16 @@ struct netdev_class { * NETDEV_PT_LEGACY_L2. */ enum netdev_pt_mode (*get_pt_mode)(const struct netdev *netdev); - /* Attempts to set input rate limiting (policing) policy, such that up to - * 'kbits_rate' kbps of traffic is accepted, with a maximum accumulative - * burst size of 'kbits' kb. + /* Attempts to set input rate limiting (policing) policy, such that: + * - up to 'kbits_rate' kbps of traffic is accepted, with a maximum + * accumulative burst size of 'kbits' kb; and + * - up to 'kpkts' kpps of traffic is accepted, with a maximum + * accumulative burst size of 'kpkts' kilo packets. * * This function may be set to null if policing is not supported. */ int (*set_policing)(struct netdev *netdev, unsigned int kbits_rate, - unsigned int kbits_burst); + unsigned int kbits_burst, unsigned int kpkts_rate, + unsigned int kpkts_burst); /* Adds to 'types' all of the forms of QoS supported by 'netdev', or leaves * it empty if 'netdev' does not support QoS. Any names added to 'types' diff --git a/lib/netdev.c b/lib/netdev.c index 91e91955c..8305f6c42 100644 --- a/lib/netdev.c +++ b/lib/netdev.c @@ -1619,17 +1619,20 @@ netdev_get_custom_stats(const struct netdev *netdev, return error; } - -/* Attempts to set input rate limiting (policing) policy, such that up to - * 'kbits_rate' kbps of traffic is accepted, with a maximum accumulative burst - * size of 'kbits' kb. */ +/* Attempts to set input rate limiting (policing) policy, such that: + * - up to 'kbits_rate' kbps of traffic is accepted, with a maximum + * accumulative burst size of 'kbits' kb; and + * - up to 'kpkts' kpps of traffic is accepted, with a maximum + * accumulative burst size of 'kpkts' kilo packets. + */ int netdev_set_policing(struct netdev *netdev, uint32_t kbits_rate, - uint32_t kbits_burst) + uint32_t kbits_burst, uint32_t kpkts_rate, + uint32_t kpkts_burst) { return (netdev->netdev_class->set_policing ? netdev->netdev_class->set_policing(netdev, - kbits_rate, kbits_burst) + kbits_rate, kbits_burst, kpkts_rate, kpkts_burst) : EOPNOTSUPP); } diff --git a/lib/netdev.h b/lib/netdev.h index b705a9e56..acf174927 100644 --- a/lib/netdev.h +++ b/lib/netdev.h @@ -289,7 +289,8 @@ struct netdev_queue_stats { }; int netdev_set_policing(struct netdev *, uint32_t kbits_rate, - uint32_t kbits_burst); + uint32_t kbits_burst, uint32_t kpkts_rate, + uint32_t kpkts_burst); int netdev_get_qos_types(const struct netdev *, struct sset *types); int netdev_get_qos_capabilities(const struct netdev *, diff --git a/vswitchd/bridge.c b/vswitchd/bridge.c index 5ed7e8234..2591e29d8 100644 --- a/vswitchd/bridge.c +++ b/vswitchd/bridge.c @@ -4895,8 +4895,10 @@ iface_configure_qos(struct iface *iface, const struct ovsrec_qos *qos) } netdev_set_policing(iface->netdev, - MIN(UINT32_MAX, iface->cfg->ingress_policing_rate), - MIN(UINT32_MAX, iface->cfg->ingress_policing_burst)); + MIN(UINT32_MAX, iface->cfg->ingress_policing_rate), + MIN(UINT32_MAX, iface->cfg->ingress_policing_burst), + MIN(UINT32_MAX, iface->cfg->ingress_policing_kpkts_rate), + MIN(UINT32_MAX, iface->cfg->ingress_policing_kpkts_burst)); ofpbuf_uninit(&queues_buf); } diff --git a/vswitchd/vswitch.ovsschema b/vswitchd/vswitch.ovsschema index 0666c8c76..4873cfde7 100644 --- a/vswitchd/vswitch.ovsschema +++ b/vswitchd/vswitch.ovsschema @@ -1,6 +1,6 @@ {"name": "Open_vSwitch", - "version": "8.2.0", - "cksum": "1076640191 26427", + "version": "8.3.0", + "cksum": "3781850481 26690", "tables": { "Open_vSwitch": { "columns": { @@ -242,6 +242,12 @@ "ingress_policing_burst": { "type": {"key": {"type": "integer", "minInteger": 0}}}, + "ingress_policing_kpkts_rate": { + "type": {"key": {"type": "integer", + "minInteger": 0}}}, + "ingress_policing_kpkts_burst": { + "type": {"key": {"type": "integer", + "minInteger": 0}}}, "mac_in_use": { "type": {"key": {"type": "string"}, "min": 0, "max": 1}, diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index 4597a215d..3522b2497 100644 --- a/vswitchd/vswitch.xml +++ b/vswitchd/vswitch.xml @@ -3654,8 +3654,18 @@ ovs-vsctl add-port br0 p0 -- set Interface p0 type=patch options:peer=p1 \ table="Queue"/> tables).

- Policing is currently implemented on Linux and OVS with DPDK. Both - implementations use a simple ``token bucket'' approach: + Policing settings can be set with byte rate or packet rate, and they + can be configured together, in which case they take effect together, + that means the smaller speed limit of them is in effect. +

+

+ Currently, byte rate policing is implemented on Linux and OVS with + DPDK, while packet rate policing is only implemented on Linux. Both + Linux and OVS DPDK implementations use a simple ``token bucket'' + approach. +

+

+ Byte rate policing:

  • @@ -3673,6 +3683,26 @@ ovs-vsctl add-port br0 p0 -- set Interface p0 type=patch options:peer=p1 \ rate specified by .
+

+ Packet rate policing: +

+
    +
  • + The size of the bucket corresponds to . Initially the bucket is + full. +
    • +
  • + Whenever a packet is received, it will consume one token from the + current bucket. If the token is available in the bucket, it's + removed and the packet is forwarded. Otherwise, the packet is + dropped. +
    • +
  • + Whenever it is not full, the bucket is refilled with tokens at the + rate specified by . +
    • +

Policing interacts badly with some network protocols, and especially with fragmented IP packets. Suppose that there is enough network @@ -3698,6 +3728,14 @@ ovs-vsctl add-port br0 p0 -- set Interface p0 type=patch options:peer=p1 \

+ +

+ Maximum rate for data received on this interface, in kpps (1 kpps is + 1000 pps). Data received faster than this rate is dropped. Set to + 0 (the default) to disable policing. +

+ +

Maximum burst size for data received on this interface, in kb. The default burst size if set to 0 is 8000 kbit. This value @@ -3712,6 +3750,23 @@ ovs-vsctl add-port br0 p0 -- set Interface p0 type=patch options:peer=p1 \ closer to achieving the full rate.

+ + +

+ Maximum burst size for data received on this interface, in kpkts (1 + kpkts is 1000 packets). The default burst size if set to + 0 is 16 kpkts. This value has no effect if + is 0. +

+

+ Specifying a larger burst size lets the algorithm be more forgiving, + which is important for protocols like TCP that react severely to + dropped packets. Specifying a value that is numerically at least as + large as 80% of helps TCP + come closer to achieving the full rate. +

+ + From patchwork Wed Jun 9 09:52:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Horman X-Patchwork-Id: 1489772 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.a=rsa-sha256 header.s=20150623 header.b=JwP5qUC5; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4G0Mps4fXLz9sCD for ; Wed, 9 Jun 2021 19:52:37 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 86EF283C33; Wed, 9 Jun 2021 09:52:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MSj8v5lufMNE; Wed, 9 Jun 2021 09:52:34 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 6B22B83C48; Wed, 9 Jun 2021 09:52:32 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D52FCC0028; Wed, 9 Jun 2021 09:52:31 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 95891C000E for ; Wed, 9 Jun 2021 09:52:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 8C4CC404F5 for ; Wed, 9 Jun 2021 09:52:29 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=netronome-com.20150623.gappssmtp.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7oIhGRs8waTs for ; Wed, 9 Jun 2021 09:52:28 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) by smtp4.osuosl.org (Postfix) with ESMTPS id 11A7040122 for ; Wed, 9 Jun 2021 09:52:27 +0000 (UTC) Received: by mail-ed1-x52c.google.com with SMTP id dj8so28002860edb.6 for ; Wed, 09 Jun 2021 02:52:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=t12D3WuQ5l97FK24Wi3sVdnEXl3B9EsMKG/V7Fq5bks=; b=JwP5qUC5vU3W1jOyPl/DX3987si79+iCUrOO3KomzZ9ZSolJweDzSY8nyeiuZMzjFz z/tMT8BHUzU9fkDMCcWZRDsVbZVpheehT7ioycj7BF0DlPten2OF7qkSZHILrw8imUmO SfuBBORpRNBUyn9fFMHzzYK63bkJ9hrBM/FmILm9KwFQYE4716HvD3WCqrXFe64NYVw7 s+2g8BX4NF3B7g3HUpToQLHpUmMNql7H/erAUW8yC0RIRIvQm8KEIpaJDDQCEhfI3sWP i0EqAe91SVt6lCRq063Zk8lXO5tcQcsspZTbXwsox/Ob4ujc+vUA3uwkjPHRdoKaq3D6 MZrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t12D3WuQ5l97FK24Wi3sVdnEXl3B9EsMKG/V7Fq5bks=; b=T+VsqV/KiNk1nqLQqo0z+8TbQ1wuTHoCtx1KFTjMh/bR99odfRAcjI2npoGcW/e4DP bTZHnNozYUCnP+P09oAzwsBSNXgRmAe+JxD3vAsC+sxlMivUTt9EJZopbDDMr+awMFXp pOBk8P04GhRvXkJEdTGo10K6A61q8qsEXt6Xj3N6khgkXVHZ+ndmPYTda9ugJo/ZwBea 7XgB0tj3Gg4JffpOflBVNtXtOnL4KGXa3rJSzmc20Zgr+pFGW7qK+10foiLEpa7yYIqs 1UaJXHiqdIyPY7bkrj+2SFZ+XaGngCwkHQluVnm1XxOkSYUsvfdNgKVillkD7hnWxEAO lSUA== X-Gm-Message-State: AOAM532xLwpd0Rda70/uSUXLcnGIOxTYs2XRCVK/EytIpQGXFlnjGEa9 2EuqkrjB8K5fSk/YWnS6IfoLUSyDNVedQqT3OpHeqga/j+oT250+o/ASr7q+q3N3TeclwNv1owE e0pNoz1MiAkuZ6xam9T8R60//2OQbFRwidEaWp/TYWxiGwNxGdkQAejZZ2Hi10FVJume3RQpljT je X-Google-Smtp-Source: ABdhPJzxjiRfuvzzWN+MJIfoYvZuqP4+t9sP3fjQ5wHmDVRX2ZBVCAshQFFvAJR8nVXWzmxOAeuL7Q== X-Received: by 2002:a50:e702:: with SMTP id a2mr30535431edn.3.1623232345621; Wed, 09 Jun 2021 02:52:25 -0700 (PDT) Received: from momiji.horms.nl ([2001:982:756:703:d63d:7eff:fe99:ac9d]) by smtp.gmail.com with ESMTPSA id nc26sm840636ejc.106.2021.06.09.02.52.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Jun 2021 02:52:24 -0700 (PDT) From: Simon Horman To: dev@openvswitch.org Date: Wed, 9 Jun 2021 11:52:09 +0200 Message-Id: <20210609095209.6205-3-simon.horman@netronome.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210609095209.6205-1-simon.horman@netronome.com> References: <20210609095209.6205-1-simon.horman@netronome.com> MIME-Version: 1.0 Cc: Baowen Zheng , Yong Xu , oss-drivers@netronome.com, Simon Horman , Tianyu Yuan , Marcelo Ricardo Leitner Subject: [ovs-dev] [PATCH v3 2/2] add test cases for ingress_policing_kpkts parameters X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Tianyu Yuan Exercise OVS setting of ingress_policing_kpkts parameters using ovs-vsctl and verify that the correct values are stored on OVSDB. Verify the ingress_policing parameters with tc command. Also check offload and non-offload in tc software datapath based on tc filter type (matchall and basic). Skip test of pps if OVS or kernel does not support pps rate limit. Example invocation: make check TESTSUITEFLAGS='-k ingress_policing_kpkts' make check-offloads TESTSUITEFLAGS='-k ingress_policing_kpkts' Signed-off-by: Tianyu Yuan Signed-off-by: Simon Horman --- tests/atlocal.in | 17 +++++++++-- tests/ovs-vsctl.at | 23 +++++++++++++++ tests/system-offloads-traffic.at | 50 ++++++++++++++++++++++++++++++++ 3 files changed, 87 insertions(+), 3 deletions(-) diff --git a/tests/atlocal.in b/tests/atlocal.in index e1b75f5a3..a0ad239ec 100644 --- a/tests/atlocal.in +++ b/tests/atlocal.in @@ -159,9 +159,6 @@ find_command() # Set HAVE_NC find_command nc -# Set HAVE_TC -find_command tc - # Determine correct netcat option to quit on stdin EOF if nc --version 2>&1 | grep -q nmap.org; then # Nmap netcat @@ -171,6 +168,20 @@ else NC_EOF_OPT="-q 1 -w 5" fi +# Set HAVE_TC +find_command tc + +# When HAVE_TC=yes, check if the current tc supports adding pps filter +SUPPORT_TC_INGRESS_PPS="no" +if test $HAVE_TC="yes"; then + ip link add veth0 type veth peer name veth1 + tc qdisc add dev veth0 handle ffff: ingress + if tc filter add dev veth0 parent ffff: u32 match u32 0 0 police pkts_rate 100 pkts_burst 10; then + SUPPORT_TC_INGRESS_PPS="yes" + fi + ip link del veth0 +fi + # Set HAVE_TCPDUMP find_command tcpdump diff --git a/tests/ovs-vsctl.at b/tests/ovs-vsctl.at index dccb11741..d6cd2c084 100644 --- a/tests/ovs-vsctl.at +++ b/tests/ovs-vsctl.at @@ -1687,3 +1687,26 @@ ingress_policing_rate: 100 ]) OVS_VSCTL_CLEANUP AT_CLEANUP + +dnl ---------------------------------------------------------------------- +AT_BANNER([set ingress policing(kpkts) test]) + +AT_SETUP([set ingress_policing_kpkts_rate and ingress_policing_kpkts_burst]) +AT_KEYWORDS([ingress_policing_kpkts]) +OVS_VSCTL_SETUP +AT_CHECK([RUN_OVS_VSCTL_TOGETHER( + [add-br a], + [add-port a a1], + [set interface a1 ingress_policing_kpkts_rate=100], + [set interface a1 ingress_policing_kpkts_burst=10], + [--columns=ingress_policing_kpkts_burst,ingress_policing_kpkts_rate list interface a1])], + [0], + [ + + + +ingress_policing_kpkts_burst: 10 +ingress_policing_kpkts_rate: 100 +]) +OVS_VSCTL_CLEANUP +AT_CLEANUP diff --git a/tests/system-offloads-traffic.at b/tests/system-offloads-traffic.at index c8e4c68fa..be63057bb 100644 --- a/tests/system-offloads-traffic.at +++ b/tests/system-offloads-traffic.at @@ -120,3 +120,53 @@ matchall ]) OVS_TRAFFIC_VSWITCHD_STOP AT_CLEANUP + +AT_SETUP([offloads - set ingress_policing_kpkts_rate and ingress_policing_kpkts_burst - offloads disabled]) +AT_KEYWORDS([ingress_policing_kpkts]) +AT_SKIP_IF([test $SUPPORT_TC_INGRESS_PPS = "no"]) +OVS_TRAFFIC_VSWITCHD_START() +AT_CHECK([ovs-vsctl set Open_vSwitch . other_config:hw-offload=false]) +AT_CHECK([ovs-ofctl add-flow br0 "actions=normal"]) +ADD_NAMESPACES(at_ns0) +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24") +AT_CHECK([ovs-vsctl set interface ovs-p0 ingress_policing_kpkts_rate=100]) +AT_CHECK([ovs-vsctl set interface ovs-p0 ingress_policing_kpkts_burst=10]) +AT_CHECK([ovs-vsctl --columns=other_config list open], [0], [dnl +other_config : {hw-offload="false"} +]) +AT_CHECK([tc -o -s -d filter show dev ovs-p0 ingress | + sed -n 's/.*\(pkts_rate [[0-9]]*[[a-zA-Z]]* pkts_burst [[0-9]]*[[a-zA-Z]]*\).*/\1/; T; p; q'], + [0],[dnl +pkts_rate 100000 pkts_burst 10240 +]) +AT_CHECK([tc -s -d filter show dev ovs-p0 ingress | + sed -n 's/.*\(basic\).*/\1/; T; p; q'], [0], [dnl +basic +]) +OVS_TRAFFIC_VSWITCHD_STOP +AT_CLEANUP + +AT_SETUP([offloads - set ingress_policing_kpkts_rate and ingress_policing_kpkts_burst - offloads enabled]) +AT_KEYWORDS([ingress_policing_kpkts]) +AT_SKIP_IF([test $SUPPORT_TC_INGRESS_PPS = "no"]) +OVS_TRAFFIC_VSWITCHD_START() +AT_CHECK([ovs-vsctl set Open_vSwitch . other_config:hw-offload=true]) +AT_CHECK([ovs-ofctl add-flow br0 "actions=normal"]) +ADD_NAMESPACES(at_ns0) +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24") +AT_CHECK([ovs-vsctl set interface ovs-p0 ingress_policing_kpkts_rate=100]) +AT_CHECK([ovs-vsctl set interface ovs-p0 ingress_policing_kpkts_burst=10]) +AT_CHECK([ovs-vsctl --columns=other_config list open], [0], [dnl +other_config : {hw-offload="true"} +]) +AT_CHECK([tc -o -s -d filter show dev ovs-p0 ingress | + sed -n 's/.*\(pkts_rate [[0-9]]*[[a-zA-Z]]* pkts_burst [[0-9]]*[[a-zA-Z]]*\).*/\1/; T; p; q'], + [0],[dnl +pkts_rate 100000 pkts_burst 10240 +]) +AT_CHECK([tc -s -d filter show dev ovs-p0 ingress | + sed -n 's/.*\(matchall\).*/\1/; T; p; q'], [0], [dnl +matchall +]) +OVS_TRAFFIC_VSWITCHD_STOP +AT_CLEANUP