From patchwork Sun Apr 9 11:15:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: miter X-Patchwork-Id: 1767017 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=outlook.com header.i=@outlook.com header.a=rsa-sha256 header.s=selector1 header.b=WbgBXnPO; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PvTzx6hYFz1yb0 for ; Sun, 9 Apr 2023 21:15:41 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id AD2DA841A6; Sun, 9 Apr 2023 11:15:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org AD2DA841A6 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key, unprotected) header.d=outlook.com header.i=@outlook.com header.a=rsa-sha256 header.s=selector1 header.b=WbgBXnPO X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m_DjowVXDxsg; Sun, 9 Apr 2023 11:15:35 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 35AD38202A; Sun, 9 Apr 2023 11:15:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 35AD38202A Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1D764C008A; Sun, 9 Apr 2023 11:15:33 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1DB79C002A for ; Sun, 9 Apr 2023 11:15:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id EAE66416B6 for ; Sun, 9 Apr 2023 11:15:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org EAE66416B6 Authentication-Results: smtp4.osuosl.org; dkim=pass (2048-bit key, unprotected) header.d=outlook.com header.i=@outlook.com header.a=rsa-sha256 header.s=selector1 header.b=WbgBXnPO X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lZWnWjoWYaEP for ; Sun, 9 Apr 2023 11:15:27 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 457E4416BC Received: from AUS01-SY4-obe.outbound.protection.outlook.com (mail-sy4aus01olkn20822.outbound.protection.outlook.com [IPv6:2a01:111:f403:7005::822]) by smtp4.osuosl.org (Postfix) with ESMTPS id 457E4416BC for ; Sun, 9 Apr 2023 11:15:27 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mBBRR832rnxdG9QXyBmGvSjjPLMdUyJlRxuOMREQtB1z1w9Md20aMOx6YVSuG+PTdKZ+58QC7PB9v6ee6fHe8ZZx0BdIsmwcBRv/Vwlc2kUSSBbHvol+n3YvIwX5zNllhTfgxSM44tUkPBUV9aLU+7JLBXMGnwSQZmujiy8BQhACf0R1fCfZAgKaaUbF0oplWf4YJ2/fwa4AeA817wI/gbRci0kwmAx5inKgYdAjs7EGMgpPmKtxqV4qadMikwLgFI05vq6B/UsU4a+S1beDh06NX5ydAKaHPvS9XWQbod4RsJYj6YHM+OMltgnFqQ1S7m8P4V5VnuE/ghQD28Oq0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CEt0zwssFvhXkJ5/wGacTkIEFRnqaRhnIOp5lwWhabE=; b=kxh9b4UXoTrIzOblXFdRjUhv8SomUmNbBOFu0BxGuMdhd4GwIdYeK7Rw2use47nHf1l6DSKqvWxJ7N5vnnNTko0CgoNskc0J/WLiYhYXdXILyfSf0AnzjRwab2Gw8U7iRmO0OXaMKFnFJ+RYxDJ5d/gDnmRUE/pb5vRZ/2sZCeP5AlEMrHd14zCc/o6bbo8WAtGEu/3u82+DfdtwnW1KK+6Cs8XFtQbQN7g4WutwXBYoLaNO0T+Z2znQMs/En1tUA8SEObIcznlk/ErDUoukOjwnmhcR0wKKt4Llv+Uj8sHpcfi2arNCNvxaOZjsqFxRpVWNP0E8K7I0s9iLZDBmOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CEt0zwssFvhXkJ5/wGacTkIEFRnqaRhnIOp5lwWhabE=; b=WbgBXnPOoRpqUpqaLkW1ql7xgYmbZvTXwjJ8GDaCkqJgU4sPTJe2MeYKvUVOF5vhq0G8Incmg7En+pYKygs2ccdwN99i8wWE4vksDmdUDOKI7zuc3lvS503Bu8UUkejuGFzemuPqZHec+t83Xq0E9ZBMDicsNAxqREObSR5SydifZazNEhft+QlrjSCoO+i5UZgN2Urz/O46ksNgyKL2MlBSDIP1mdj/wI5ZHwRHDezpIZVsto8fHJdqHMOcV9hE6R+CeqAxO5NXqJGrkpdx1mLlQ3RXixMl4JT5LkOA3jnT5S8wlgcmyLuNshhLCJsSWM68a0xNlwF0RKZBj8/taw== Received: from SY4PR01MB8438.ausprd01.prod.outlook.com (2603:10c6:10:19e::10) by MEYPR01MB7358.ausprd01.prod.outlook.com (2603:10c6:220:161::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.38; Sun, 9 Apr 2023 11:15:22 +0000 Received: from SY4PR01MB8438.ausprd01.prod.outlook.com ([fe80::4790:4ece:97e0:4db5]) by SY4PR01MB8438.ausprd01.prod.outlook.com ([fe80::4790:4ece:97e0:4db5%6]) with mapi id 15.20.6277.035; Sun, 9 Apr 2023 11:15:22 +0000 From: miterv@outlook.com To: dev@openvswitch.org, i.maximets@ovn.org, simon.horman@corigine.com Date: Sun, 9 Apr 2023 19:15:08 +0800 Message-ID: X-Mailer: git-send-email 2.31.1 In-Reply-To: <0> References: <0> X-TMN: [ZnpsAvkiO4rv4HP3lemwrcc8Z06zD5wq] X-ClientProxiedBy: SGXP274CA0017.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:b8::29) To SY4PR01MB8438.ausprd01.prod.outlook.com (2603:10c6:10:19e::10) X-Microsoft-Original-Message-ID: <20230409111509.27575-2-miterv@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SY4PR01MB8438:EE_|MEYPR01MB7358:EE_ X-MS-Office365-Filtering-Correlation-Id: bfcc4ec5-8d99-499d-ff51-08db38ebb5a6 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sCwTZlDHlYxA82ldwc+M9C3QK0ot6T1bJujT3DlS1Zm5wdeo71f1412+ll7P9+Z5jtVZ2kGLBdo3bTQTKdiJGJQoAi55xF+TR73le9Bj8lmKH5EGYQwkiSw/tsJOR+D4m7UoXE803M/ljaNncZoRCfdWgQcsZ6NSxpsoRRL98umVXTlPDQlCnmCfjbXxsu6xGpjPFhlfjgt+I718/Jw4InIXo48ghMDhWPJNLPLhoZprGqkVAwaP8tUJLQMCamzg5J64xae0DOnOlrwx23kZF9b5hpzV1a80edHueLx3aBS9dcVF0rn/CXTbvs2PFCmJsi90P3MC/hUkQY9Ew3HAi4uLBm09xQAkkY1IH9+kgvv2PTMv+sYTJJApNDJ3c3B8frmxB+y22HruuamEunJeZOQEk+ussqvWUmlnJcMmg6Mn1WRu2+xxF1wa0NTU1dhGF31UlUP/IxRdVPrdsuNelgl5I3i4htSUfvlAxUmrq9+rQSRC9zgFZmsYaWN5T0vcf5Xlx5sEQu0PowT/ifPheGnxc2biWXw7QQf+z8aBFiJ4eT++bF8zJhZhB0CrGZa0 X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: GCheZDP/Ss31gJJCmEXEhpJ/EKm7ofIduEGk8Ub5B3LLM6A3UxgysYkg0BPSILysp4MzOV2EoKRxCumVAQ01vwNBW2Zq9skrRYBaHeMA5gXAvFcX2AP1ZDV2qj6Ct4L/VBSUAzG9czzJspRqr1xYGgUHElTLWTNGys0LdCjWDJdc+KozgRxr3CxaK+TK1zT1gTS7eKnl6LVCCHMFEedf312yLnxWE4vwNFHvw+4ks7G+8LOBH7SqRLKRRD/qLTeujuQefhP/66IVUPPEc1XpJRijxMlq4lYl3dHQNr+yseP5mB2EpKXhGIUU0r8NF15vcGGTL0gS2cXLsp9nPZ/BVOzR36KQ5bRmsWB+vj8pBy7VCsOWq3E3UtLYmzhDsAXYM28yABT/U8SolWHqMG2eNkQrkn/8ek6riTjb15z3Oau0AJm3PzyJi8YzY7OFHYESzv9D9FrtiZkdbUSBAVbE0DdwJSTVPVKEG051E61piXcTyft+6vwX3763LuSy/VeYnitBSVMRrXSvXWTpxbe51Dr+cPzT/rFYnjQ9MdqlFnZhFXV9JMRe0W6gNFFp8ow1oQxQZdOVs00qk4auvdkWnSAkPYJ/SckMpdvC/mCDOxLJVaqoCiG6oUCaSjrfVQAdeD7EovTdB9Wmb8V3pohPe5DfXC7o8rrOmc2jfnJlJWRdcewnIx+C9MWyH+hNMvHcUu2lR1f/5Hb6hOx0gP/S67LBBH4iZZ9tPu17u6AtmQ/3exO6IRZsSOwKaFI9qnsXBlAq4YygFWWfspyoa1NiYhpzxgFPnsBbP05RVLnPkJNeSSb9oblmbtweXhc/q7gv5Auc7GT/T3+mxmfIwc17MsRusEUaYrGXQmCdMthoznsyY0qp9EM/4d0mv3a0FAqwzLiXyIkf7eNgc3x9ndLkTN4ir0gVnTrrDnurhTddQeC4WsSvgF6A3Vela1wy5w+jzulcJ/lgE3iJ29grtHYbiBpgjtuyCJKSgv7u/TfQynFXvDhYeJgyafGugryWle3zLLav9UHg3klzmSF8on4jHtC+49FylCWpRbVJTeZFxoqNHZ2OevXr1th5ZGRO1tJWFmhQtLdtxN9uYEvTX8FcSmzhNJqtQVpQV79HzjV7G335gIwUzRzrEzc+3TKeVHnhWwNmoPje3HT1+jRVSn+TiS1vfeqscpr2D9mZ3LkPj0+0y8O0wXZ+YUjpyau1xxtaCH55Bzr/ULcnlyB0ZhjtQg== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: bfcc4ec5-8d99-499d-ff51-08db38ebb5a6 X-MS-Exchange-CrossTenant-AuthSource: SY4PR01MB8438.ausprd01.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Apr 2023 11:15:22.2096 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MEYPR01MB7358 Cc: Lin Huang Subject: [ovs-dev] [PATCH v2 1/2] * netdev-dpdk: Add support for userspace port-based egress packet-per-second policing. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Lin Huang OvS has supported packet-per-second policer which can be set at ingress and egress side in kernel datapath. But the userspace datapath dosen't support for ingress and egress packet-per-second policing now. So, this patch add support for userspace egress pps policing by using native ovs token bucket library. Token bucket is accumulated by 'rate' tokens per millisecond and store maxiumim tokens at 'burst' bucket size. One token in the bucket means one packet (1 kpkts * millisecond) which will drop or pass by policer. This patch add new configuration option 'kpkts_rate' and 'kpkts_burst' for egress-policer QoS type which now supports setting packet-per-second limits in addition to the previously configurable byte rate settings. Examples: $ovs-vsctl set port vhost-user0 qos=@newqos -- --id=@newqos create qos type=egress-policer \ other-config:cir=123000 other-config:cbs=123000 other-config:kpkts_rate=123 other-config:kpkts_burst=123 Add some unit tests for egress packet-per-second policing. Signed-off-by: Lin Huang --- NEWS | 7 +++ lib/netdev-dpdk.c | 90 +++++++++++++++++++++++++---- tests/system-dpdk.at | 134 +++++++++++++++++++++++++++++++++++++++++++ vswitchd/vswitch.xml | 10 ++++ 4 files changed, 229 insertions(+), 12 deletions(-) diff --git a/NEWS b/NEWS index b6418c36e..c41c6adf6 100644 --- a/NEWS +++ b/NEWS @@ -23,6 +23,13 @@ Post-v3.1.0 process extra privileges when mapping physical interconnect memory. - SRv6 Tunnel Protocol * Added support for userspace datapath (only). + - Userspace datapath: + * Added new configuration options 'kpkts_rate' and 'kpkts_burst' for + 'egress-policer' to support packet-per-second policing. + Examples: + ovs-vsctl set port vhost-user0 qos=@newqos -- \ + --id=@newqos create qos type=egress-policer \ + other-config:kpkts_rate=123 other-config:kpkts_burst=123 v3.1.0 - 16 Feb 2023 diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index fb0dd43f7..2fd8bc68d 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -19,6 +19,7 @@ #include #include +#include #include #include #include @@ -59,6 +60,7 @@ #include "openvswitch/ofp-parse.h" #include "openvswitch/ofp-print.h" #include "openvswitch/shash.h" +#include "openvswitch/token-bucket.h" #include "openvswitch/vlog.h" #include "ovs-numa.h" #include "ovs-rcu.h" @@ -400,6 +402,11 @@ struct dpdk_tx_queue { ); }; +enum policer_type { + POLICER_BPS = 1 << 0, /* Rate value in bytes/sec. */ + POLICER_PKTPS = 1 << 1, /* Rate value in packet/sec. */ +}; + struct ingress_policer { struct rte_meter_srtcm_params app_srtcm_params; struct rte_meter_srtcm in_policer; @@ -2335,6 +2342,29 @@ srtcm_policer_run_single_packet(struct rte_meter_srtcm *meter, return cnt; } +static int +pkts_policer_run_single_packet(struct token_bucket *tb, struct rte_mbuf **pkts, + int pkt_cnt, bool should_steal) +{ + struct rte_mbuf *pkt; + int cnt = 0; + + for (int i = 0; i < pkt_cnt; i++) { + pkt = pkts[i]; + /* Handle current packet. */ + if (token_bucket_withdraw(tb, 1)) { + if (cnt != i) { + pkts[cnt] = pkt; + } + cnt++; + } else if (should_steal) { + rte_pktmbuf_free(pkt); + } + } + + return cnt; +} + static int ingress_policer_run(struct ingress_policer *policer, struct rte_mbuf **pkts, int pkt_cnt, bool should_steal) @@ -4757,6 +4787,10 @@ netdev_dpdk_queue_dump_done(const struct netdev *netdev OVS_UNUSED, struct egress_policer { struct qos_conf qos_conf; + enum policer_type type; + uint32_t kpkts_rate; + uint32_t kpkts_burst; + struct token_bucket egress_tb; struct rte_meter_srtcm_params app_srtcm_params; struct rte_meter_srtcm egress_meter; struct rte_meter_srtcm_profile egress_prof; @@ -4776,10 +4810,11 @@ static int egress_policer_qos_construct(const struct smap *details, struct qos_conf **conf) { + uint32_t kpkts_burst, kpkts_rate; struct egress_policer *policer; int err = 0; - policer = xmalloc(sizeof *policer); + policer = xcalloc(1, sizeof *policer); qos_conf_init(&policer->qos_conf, &egress_policer_ops); egress_policer_details_to_param(details, &policer->app_srtcm_params); err = rte_meter_srtcm_profile_config(&policer->egress_prof, @@ -4787,15 +4822,32 @@ egress_policer_qos_construct(const struct smap *details, if (!err) { err = rte_meter_srtcm_config(&policer->egress_meter, &policer->egress_prof); + if (!err) { + policer->type |= POLICER_BPS; + } else { + VLOG_ERR("Could not create rte meter for egress policer"); + } } - if (!err) { - *conf = &policer->qos_conf; - } else { - VLOG_ERR("Could not create rte meter for egress policer"); + kpkts_rate = smap_get_uint(details, "kpkts_rate", 0); + kpkts_burst = smap_get_uint(details, "kpkts_burst", 0); + if (kpkts_rate && kpkts_burst) { + /* Rate in kilo-packets/second, bucket 1000 packets. */ + /* msec * kilo-packets/sec = 1 packets. */ + token_bucket_set(&policer->egress_tb, kpkts_rate, kpkts_burst * 1000); + policer->kpkts_burst = kpkts_burst; + policer->kpkts_rate = kpkts_rate; + policer->type |= POLICER_PKTPS; + } + + if (!policer->type) { + /* both bps and kpkts contrsruct failed.*/ free(policer); *conf = NULL; - err = -err; + err = EINVAL; + } else { + err = 0; + *conf = &policer->qos_conf; } return err; @@ -4817,6 +4869,8 @@ egress_policer_qos_get(const struct qos_conf *conf, struct smap *details) smap_add_format(details, "cir", "%"PRIu64, policer->app_srtcm_params.cir); smap_add_format(details, "cbs", "%"PRIu64, policer->app_srtcm_params.cbs); + smap_add_format(details, "kpkts_rate", "%"PRIu32, policer->kpkts_rate); + smap_add_format(details, "kpkts_burst", "%"PRIu32, policer->kpkts_burst); return 0; } @@ -4828,25 +4882,37 @@ egress_policer_qos_is_equal(const struct qos_conf *conf, struct egress_policer *policer = CONTAINER_OF(conf, struct egress_policer, qos_conf); struct rte_meter_srtcm_params params; + uint32_t kpkts_burst, kpkts_rate; egress_policer_details_to_param(details, ¶ms); - return !memcmp(¶ms, &policer->app_srtcm_params, sizeof params); + kpkts_rate = smap_get_uint(details, "kpkts_rate", 0); + kpkts_burst = smap_get_uint(details, "kpkts_burst", 0); + + return (!memcmp(¶ms, &policer->app_srtcm_params, sizeof params)) + && (policer->kpkts_rate == kpkts_rate + && policer->kpkts_burst == kpkts_burst); } static int egress_policer_run(struct qos_conf *conf, struct rte_mbuf **pkts, int pkt_cnt, bool should_steal) { - int cnt = 0; struct egress_policer *policer = CONTAINER_OF(conf, struct egress_policer, qos_conf); - cnt = srtcm_policer_run_single_packet(&policer->egress_meter, - &policer->egress_prof, pkts, - pkt_cnt, should_steal); + if (policer->type & POLICER_BPS) { + pkt_cnt = srtcm_policer_run_single_packet(&policer->egress_meter, + &policer->egress_prof, pkts, + pkt_cnt, should_steal); + } - return cnt; + if (policer->type & POLICER_PKTPS) { + pkt_cnt = pkts_policer_run_single_packet(&policer->egress_tb, pkts, + pkt_cnt, should_steal); + } + + return pkt_cnt; } static const struct dpdk_qos_ops egress_policer_ops = { diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index cb6c6d590..ab527d406 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -570,6 +570,140 @@ dnl -------------------------------------------------------------------------- +dnl -------------------------------------------------------------------------- +dnl QoS (kpkts) create delete phy port +AT_SETUP([OVS-DPDK - QoS (kpkts) create delete phy port]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_PHY_SKIP() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add egress policer +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr]) +OVS_WAIT_UNTIL([ovs-vsctl set port phy0 qos=@newqos -- --id=@newqos create qos type=egress-policer other-config:kpkts_rate=123 other-config:kpkts_burst=246]) +AT_CHECK([ovs-appctl -t ovs-vswitchd qos/show phy0], [], [stdout]) +sleep 2 + +dnl Remove egress policer +AT_CHECK([ovs-vsctl destroy QoS phy0 -- clear Port phy0 qos]) + +dnl Check egress policer was removed correctly +AT_CHECK([ovs-appctl -t ovs-vswitchd qos/show phy0], [], [stdout]) +AT_CHECK([grep -E 'QoS not configured on phy0' stdout], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 phy0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("[SYSTEM_DPDK_ALLOWED_LOGS]") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl QoS (kpkts) create delete vport port +AT_SETUP([OVS-DPDK - QoS (kpkts) create delete vport port]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add egress policer +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], [stderr]) +OVS_WAIT_UNTIL([ovs-vsctl set port dpdkvhostuserclient0 qos=@newqos -- --id=@newqos create qos type=egress-policer other-config:kpkts_rate=123 \ + other-config:kpkts_burst=456]) +AT_CHECK([ovs-appctl -t ovs-vswitchd qos/show dpdkvhostuserclient0], [], [stdout]) +sleep 2 + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Remove egress policer +AT_CHECK([ovs-vsctl destroy QoS dpdkvhostuserclient0 -- clear Port dpdkvhostuserclient0 qos]) + +dnl Check egress policer was removed correctly +AT_CHECK([ovs-appctl -t ovs-vswitchd qos/show dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'QoS not configured on dpdkvhostuserclient0' stdout], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No such file or directory@d +\@Failed to set QoS type egress-policer on port dpdkvhostuserclient0@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl QoS (kpkts) no rate +AT_SETUP([OVS-DPDK - QoS (kpkts) no rate]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add egress policer +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], [stderr]) +OVS_WAIT_UNTIL([ovs-vsctl set port dpdkvhostuserclient0 qos=@newqos -- --id=@newqos create qos type=egress-policer other-config:kpkts_burst=123]) +sleep 2 + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Check egress policer was not created +AT_CHECK([ovs-appctl -t ovs-vswitchd qos/show dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'QoS not configured on dpdkvhostuserclient0' stdout], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No such file or directory@d +\@Could not create rte meter for egress policer@d +\@Failed to set QoS type egress-policer on port dpdkvhostuserclient0@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl QoS (kpkts) no burst +AT_SETUP([OVS-DPDK - QoS (kpkts) no burst]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add egress policer +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], [stderr]) +OVS_WAIT_UNTIL([ovs-vsctl set port dpdkvhostuserclient0 qos=@newqos -- --id=@newqos create qos type=egress-policer other-config:kpkts_rate=123]) +sleep 2 + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No such file or directory@d +\@Could not create rte meter for egress policer@d +\@Failed to set QoS type egress-policer on port dpdkvhostuserclient0@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + dnl -------------------------------------------------------------------------- dnl MTU increase phy port diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index edb5eafa0..93c387e2a 100644 --- a/vswitchd/vswitch.xml +++ b/vswitchd/vswitch.xml @@ -4894,6 +4894,16 @@ ovs-vsctl add-port br0 p0 -- set Interface p0 type=patch options:peer=p1 \ bytes/tokens of the packet. If there are not enough tokens in the cbs bucket the packet might be dropped. + + The Packets Per Second (pps) represents the packet per second rate at + which the token bucket will be updated. + + + The Packets Per Second Burst Size is measured in count and represents a + token bucket. + From patchwork Sun Apr 9 11:15:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: miter X-Patchwork-Id: 1767018 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=outlook.com header.i=@outlook.com header.a=rsa-sha256 header.s=selector1 header.b=tH1KVUH3; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PvTzz1sljz1yXv for ; Sun, 9 Apr 2023 21:15:43 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 1FC8241902; Sun, 9 Apr 2023 11:15:41 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 1FC8241902 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key, unprotected) header.d=outlook.com header.i=@outlook.com header.a=rsa-sha256 header.s=selector1 header.b=tH1KVUH3 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZvEtmUx2OFZu; Sun, 9 Apr 2023 11:15:39 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 2D80241749; Sun, 9 Apr 2023 11:15:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2D80241749 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 05D36C0092; Sun, 9 Apr 2023 11:15:36 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id AC435C0036 for ; Sun, 9 Apr 2023 11:15:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 7B3DA416B6 for ; Sun, 9 Apr 2023 11:15:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 7B3DA416B6 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zYrS1yckUNz9 for ; Sun, 9 Apr 2023 11:15:28 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org D10AF416D8 Received: from AUS01-SY4-obe.outbound.protection.outlook.com (mail-sy4aus01olkn20822.outbound.protection.outlook.com [IPv6:2a01:111:f403:7005::822]) by smtp4.osuosl.org (Postfix) with ESMTPS id D10AF416D8 for ; Sun, 9 Apr 2023 11:15:27 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jAhlQYmOl95D5WcSx0ZG0XxOSHaCYKd8lVfhL0TNW16bLBnMvqC1dMPVj92+sfiQ5HtRnFf7olbpFsb+dwESqwND5rJpZ5l4a7veWLBefHQkwgfliiJVGF0zgJlogHctKisN8MZ+sQuRm0oT8WJ1NGf7Z7TxjO14EyEOKSmkxo/qOhT9oVrGiuPKFCNuRc7tys4FT3PkJm/VkL/ZO+3MHfxWRIr2jInHRlexiLW9SMEMtYOlqHwKZrtBX/BBWpeCCiuq1DZ3xljPGUlfRdBSpq7krL2CXYfQfxv03Jh0MkkF+Gr8zFIJ6FugqDDGCMna1wWDqcj/B2QFeSkjvwi/Vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3EOA9qdn/RdeZYBiif7fIY2sVM06QBYstBkmua2kwgI=; b=dbFbT528HSl1X9En9p0l3OLW90QcGI+YN7U8k54frJD6hpYvzUUTzJ4MAi6wKmxVVQkBlwb8S29go8cP+LEIXKNWBuIUg5JMzgbOU05yBYSvYPONIS2Axl4TCmDHKFwSeUZngofkzz7WcX9KOtD1lp8oMWr30ZKC+iZc/Ii0Op105C6mK0KYA6wyfv/VVdgGTWABqYbYd+hCOT9BAJKC1WMCIQRDLCiSMJzCZL5zNZYQpJaZiuWwA74ePTQHS42Zos23gWrGXGkkz3BNuvwX6tKHfJlGtbcKeUB6/ip8761uraldLp+zQoBZmi+VZsE60kCLKosvY1E5BmhST3oazA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3EOA9qdn/RdeZYBiif7fIY2sVM06QBYstBkmua2kwgI=; b=tH1KVUH3IjE7mF1Nx/1Ujwal7+92moOpU8IYKAVwRZp5xPRB4KC0v5P2aNbX4wsO8Jz4Sz7itcSvIX1SjzxZs5DwYVjYjOVHKpC4NCJ7iuMZzy3xpKef4u/GThDHqUEQYW2xbd4Klm3P1NE0d1POExGou3N2IBGtCDTrYkI+T1B79iw0c/r0OqBcpInjmHK9uAV0f8xQh7tUvryVtwrkXDy6IdaOgZ1dhCwRj19FWe0+JYeGcaEmMIVnHuwLQrrJKgPAHNyrwrgR+VrYgOh919CW+cVHToU0FpA6h6OJcp9dMa5QL9+HAMsknvT2m2x5Tdoy6TsIfgAJ0D4B3Z7fyA== Received: from SY4PR01MB8438.ausprd01.prod.outlook.com (2603:10c6:10:19e::10) by MEYPR01MB7358.ausprd01.prod.outlook.com (2603:10c6:220:161::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.38; Sun, 9 Apr 2023 11:15:23 +0000 Received: from SY4PR01MB8438.ausprd01.prod.outlook.com ([fe80::4790:4ece:97e0:4db5]) by SY4PR01MB8438.ausprd01.prod.outlook.com ([fe80::4790:4ece:97e0:4db5%6]) with mapi id 15.20.6277.035; Sun, 9 Apr 2023 11:15:23 +0000 From: miterv@outlook.com To: dev@openvswitch.org, i.maximets@ovn.org, simon.horman@corigine.com Date: Sun, 9 Apr 2023 19:15:09 +0800 Message-ID: X-Mailer: git-send-email 2.31.1 In-Reply-To: <0> References: <0> X-TMN: [LY+ULWAZUEfNGQ63gy/FUyjEkG+EXmBg] X-ClientProxiedBy: SGXP274CA0017.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:b8::29) To SY4PR01MB8438.ausprd01.prod.outlook.com (2603:10c6:10:19e::10) X-Microsoft-Original-Message-ID: <20230409111509.27575-3-miterv@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SY4PR01MB8438:EE_|MEYPR01MB7358:EE_ X-MS-Office365-Filtering-Correlation-Id: dcfd8f6d-99ba-4588-ed39-08db38ebb67f X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: O7D7bjc16lbZc+YmuIE3mXgWeTYQCORotLBACGlCbVxe8rOK5XgH4JoVx3h2Pf7lXRhYJ+VMdml5+FChHDWDw6io8oVD29V1YPfELJ9vdp3vFMKoMTBjg0bDndX2E+ktlKEE+ZnIb+HRfecC17zVQxUOt18gXmB69ivCLFth0dnggqY+a5Bu95mLQjDf14byAZ6m33Yew74ctZ24WgnE0aA/g0bVZSaDKcihv6oymG3Zqg4vE0fJVfBZoQ8R9Hr2xkQ4b9V1i4a5BlelyvivLi5De83WwdHiMTIXHCs8/6UnIF8gbBIL7TPp+xvNgC/97Zzja5sr/d8vJkiX1958HIcdSJj9EdhGjwI9kOz+R82Gdtoo6VE6GDmTbfHGj8aoVdD/n8G52dHfAef3DzgDYd0BGLKl1GrrBhK+V27uWrqLh9cY+BqYdRGf9pvIpndSNoGNtumMrBU7fQ7bjWox4h6tZo6J8QEAYMWjU++wgRgLJxdIY1167lqYqZkeZq5UObJrmPJehDVuqMYCfQJ1Kg3s70YXGb56x88PL1A9GfrwUMt4eObh+rIxKS3XPrgh X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ngP+a74kK8kwslkOq6VCBVxkxjRksR5hi7wKfl28DqbZ9Rd4P7i0NEyssPVBD7GZZgdZKIgFZtHYSeyGls7SrcYtkB4b2KD1i0z5LTcod82BFq3h93PNFy0LQ90iFnfFw2MxiZmcuLFXK5yGfkHFham5yi9Qczq7I8ERMNibJoQdWlhbGFoLowUf9JqInn0VvpCAZKHC2bSgXRQZuS+402qFnFDN2LH9gtMx61jo0WuREvDbswQEr/ke6x6hdddudKyJbsozaF8DnDVmm25zneED5u4PMO/fhKiqRmXIFGuI+S5urxjC6P6NxQrVNev2fogUtDFTHL+RTMoKGdl5CEVeYnacBlc0rCSgBj1Kmf3gIu7gt3qyZPht7o0cGb7cdbwmO1OqlCVEJtlZLIxdoA3lcFBD1Khm2QF+QCvUQq1JlGkhU2eMFsiNq3UT3YEXI3D8LbOzBuCTlGwZcslvbbZZj6Qo6QgodiCEAdmlNsIffcrL7MTgE73R+GH7WdxDY0LTneRw3HgGB7UgoKU5Agq/1QShoysQzWWYL5OOCbMJt4/dxmR+J6dRJBncsHHJXNXkmHzOKKf2HcKsRNyoqDblscXJWaDz5CLAshZGO3qCftwuPf+xiyZnCxNuwCD63J/aXL5eUd8mloK+IglN/3ia9JmcGbUk+5dc9MCK430lnbRRybYoOL//xkq8N31P9GGHODBtGuh1lcaNO6PP5Dk4kEKI+nYSrG2i+7ujZNsznrojlEQKq/yJEB3kNefRRodla0/+Js0JAbMJfc9EPpdybwXA6Spbg8jQB4wyr7tnbzqCro3GslR+w6idUrMc7nhxFBEw+jBf8Qi6A/9NBWSknH5pS0OaiPVa7moI08YUoxh7IkLdRMJEZ/pLiQ0c0mBCf9KxPPKlLGFRCvJK+CzZ41+7JsdrsNRlw/GIhpmh3mFL/hMlKtZz5iZHXY4cgmklW7BL/z3IKdn2WaTij3mCS6ZsLTkmmNWeDOlWcGNdScrB7D6K90pznmog4dokLdLunT7yxyXY8dqEkzDp2KOEQ/zccxikFAUqot+bEitGWhMU33aDpEOjj0rFO1T4N9Zmy2jWpwt2NcaICFzCcNfZOt2VSXuQzNLB6iZs4IVne0UGDEuz6GhAyXwSm3m+OXQfPQv24TDEsRmqdSFJIaZyy9+4btrX1p38fxyV08XrqbRgFlxgm2oxBTv1ZMmHJDV4HKYwpQAqlKpnW0SFyA== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: dcfd8f6d-99ba-4588-ed39-08db38ebb67f X-MS-Exchange-CrossTenant-AuthSource: SY4PR01MB8438.ausprd01.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Apr 2023 11:15:23.5518 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MEYPR01MB7358 Cc: Lin Huang Subject: [ovs-dev] [PATCH v2 2/2] * netdev-dpdk: Add support for userspace port-based ingress packet-per-second policing. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Lin Huang OvS has supported packet-per-second policer which can be set at ingress and egress side in kernel datapath. But the userspace datapath dosen't support for ingress and egress packet-per-second policing now. So, this patch add support for userspace ingress pps policing by using native ovs token bucket library. Token bucket is accumulated by 'rate' tokens per millisecond and store maxiumim tokens at 'burst' bucket size. One token in the bucket means one packet (1 kpkts * millisecond) which will drop or pass by policer. This patch reuse 'ingress_policing_kpkts_rate' and 'ingress_policing_kpkts_burst' options at interface table. Now userspace ingress policer supports setting packet-per-second limits in addition to the previously configurable byte rate settings. Examples: $ ovs-vsctl set interface dpdk0 ingress_policing_rate=12300 $ ovs-vsctl set interface dpdk0 ingress_policing_burst=12300 $ ovs-vsctl set interface dpdk0 ingress_policing_kpkts_rate=123 $ ovs-vsctl set interface dpdk0 ingress_policing_kpkts_burst=123 Add some unit tests for ingress packet-per-second policing. Signed-off-by: Lin Huang --- NEWS | 4 ++ lib/netdev-dpdk.c | 99 +++++++++++++++++++++------- tests/system-dpdk.at | 152 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 230 insertions(+), 25 deletions(-) diff --git a/NEWS b/NEWS index c41c6adf6..53adbed82 100644 --- a/NEWS +++ b/NEWS @@ -30,6 +30,10 @@ Post-v3.1.0 ovs-vsctl set port vhost-user0 qos=@newqos -- \ --id=@newqos create qos type=egress-policer \ other-config:kpkts_rate=123 other-config:kpkts_burst=123 + * Added support for ingress packet-per-second policing. + Examples: + $ ovs-vsctl set interface dpdk0 ingress_policing_kpkts_rate=123 + $ ovs-vsctl set interface dpdk0 ingress_policing_kpkts_burst=123 v3.1.0 - 16 Feb 2023 diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 2fd8bc68d..ca0f22287 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -411,6 +411,8 @@ struct ingress_policer { struct rte_meter_srtcm_params app_srtcm_params; struct rte_meter_srtcm in_policer; struct rte_meter_srtcm_profile in_prof; + struct token_bucket tb; + enum policer_type type; rte_spinlock_t policer_lock; }; @@ -494,6 +496,9 @@ struct netdev_dpdk { uint32_t policer_rate; uint32_t policer_burst; + uint32_t policer_kpkts_rate; + uint32_t policer_kpkts_burst; + /* Array of vhost rxq states, see vring_state_changed. */ bool *vhost_rxq_enabled; ); @@ -1313,6 +1318,8 @@ common_construct(struct netdev *netdev, dpdk_port_t port_no, ovsrcu_init(&dev->ingress_policer, NULL); dev->policer_rate = 0; dev->policer_burst = 0; + dev->policer_kpkts_rate = 0; + dev->policer_kpkts_burst = 0; netdev->n_rxq = 0; netdev->n_txq = 0; @@ -2372,9 +2379,15 @@ ingress_policer_run(struct ingress_policer *policer, struct rte_mbuf **pkts, int cnt = 0; rte_spinlock_lock(&policer->policer_lock); - cnt = srtcm_policer_run_single_packet(&policer->in_policer, - &policer->in_prof, - pkts, pkt_cnt, should_steal); + if (policer->type & POLICER_BPS) { + cnt = srtcm_policer_run_single_packet(&policer->in_policer, + &policer->in_prof, + pkts, pkt_cnt, should_steal); + } + if (policer->type & POLICER_PKTPS) { + cnt = pkts_policer_run_single_packet(&policer->tb, pkts, pkt_cnt, + should_steal); + } rte_spinlock_unlock(&policer->policer_lock); return cnt; @@ -3536,7 +3549,8 @@ netdev_dpdk_get_features(const struct netdev *netdev, } static struct ingress_policer * -netdev_dpdk_policer_construct(uint32_t rate, uint32_t burst) +netdev_dpdk_policer_construct(uint32_t rate, uint32_t burst, + uint32_t kpkts_rate, uint32_t kpkts_burst) { struct ingress_policer *policer = NULL; uint64_t rate_bytes; @@ -3546,23 +3560,43 @@ netdev_dpdk_policer_construct(uint32_t rate, uint32_t burst) policer = xmalloc(sizeof *policer); rte_spinlock_init(&policer->policer_lock); - /* rte_meter requires bytes so convert kbits rate and burst to bytes. */ - rate_bytes = rate * 1000ULL / 8; - burst_bytes = burst * 1000ULL / 8; + if (rate) { + /* rte_meter requires bytes so convert kbits rate and burst to bytes. */ + rate_bytes = rate * 1000ULL / 8; + burst_bytes = burst * 1000ULL / 8; - policer->app_srtcm_params.cir = rate_bytes; - policer->app_srtcm_params.cbs = burst_bytes; - policer->app_srtcm_params.ebs = 0; - err = rte_meter_srtcm_profile_config(&policer->in_prof, - &policer->app_srtcm_params); - if (!err) { - err = rte_meter_srtcm_config(&policer->in_policer, - &policer->in_prof); + policer->app_srtcm_params.cir = rate_bytes; + policer->app_srtcm_params.cbs = burst_bytes; + policer->app_srtcm_params.ebs = 0; + err = rte_meter_srtcm_profile_config(&policer->in_prof, + &policer->app_srtcm_params); + if (!err) { + err = rte_meter_srtcm_config(&policer->in_policer, + &policer->in_prof); + } + if (err) { + VLOG_ERR("Could not create rte meter for ingress policer"); + } else { + policer->type |= POLICER_BPS; + } } - if (err) { - VLOG_ERR("Could not create rte meter for ingress policer"); + + if (kpkts_rate) { + /* + * By default token is added per millisecond. + * Rate in kilo-packets/second, bucket 1000 packets. + * msec * kilo-packets/sec = 1 packets. + */ + kpkts_burst *= 1000; + + token_bucket_init(&policer->tb, kpkts_rate, kpkts_burst); + policer->type |= POLICER_PKTPS; + } + + if (!policer->type) { + /* Neither kbps nor kpkts policer is configured. */ free(policer); - return NULL; + policer = NULL; } return policer; @@ -3571,8 +3605,8 @@ netdev_dpdk_policer_construct(uint32_t rate, uint32_t burst) static int netdev_dpdk_set_policing(struct netdev* netdev, uint32_t policer_rate, uint32_t policer_burst, - uint32_t policer_kpkts_rate OVS_UNUSED, - uint32_t policer_kpkts_burst OVS_UNUSED) + uint32_t policer_kpkts_rate, + uint32_t policer_kpkts_burst) { struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); struct ingress_policer *policer; @@ -3585,13 +3619,23 @@ netdev_dpdk_set_policing(struct netdev* netdev, uint32_t policer_rate, : !policer_burst ? 8000 : policer_burst); + /* + * Force to 0 if no rate specified, + * default to rate value if burst is 0, + * else stick with user-specified value. + */ + policer_kpkts_burst = (!policer_kpkts_rate ? 0 + : !policer_kpkts_burst ? policer_kpkts_rate + : policer_kpkts_burst); ovs_mutex_lock(&dev->mutex); policer = ovsrcu_get_protected(struct ingress_policer *, - &dev->ingress_policer); + &dev->ingress_policer); if (dev->policer_rate == policer_rate && - dev->policer_burst == policer_burst) { + dev->policer_burst == policer_burst && + dev->policer_kpkts_rate == policer_kpkts_rate && + dev->policer_kpkts_burst == policer_kpkts_burst) { /* Assume that settings haven't changed since we last set them. */ ovs_mutex_unlock(&dev->mutex); return 0; @@ -3602,14 +3646,19 @@ netdev_dpdk_set_policing(struct netdev* netdev, uint32_t policer_rate, ovsrcu_postpone(free, policer); } - if (policer_rate != 0) { - policer = netdev_dpdk_policer_construct(policer_rate, policer_burst); - } else { + if (!policer_rate && !policer_kpkts_rate) { + /* both policers are zero. */ policer = NULL; + } else { + policer = netdev_dpdk_policer_construct(policer_rate, policer_burst, + policer_kpkts_rate, + policer_kpkts_burst); } ovsrcu_set(&dev->ingress_policer, policer); dev->policer_rate = policer_rate; dev->policer_burst = policer_burst; + dev->policer_kpkts_rate = policer_kpkts_rate; + dev->policer_kpkts_burst = policer_kpkts_burst; ovs_mutex_unlock(&dev->mutex); return 0; diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index ab527d406..05ad5adc1 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -426,6 +426,158 @@ AT_CLEANUP dnl -------------------------------------------------------------------------- + +dnl -------------------------------------------------------------------------- +dnl Ingress policing (kpkts) create delete phy port +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) create delete phy port]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_PHY_SKIP() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add policer +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl set interface phy0 ingress_policing_kpkts_rate=10000 ingress_policing_kpkts_burst=1000]) +AT_CHECK([ovs-vsctl show], [], [stdout]) +sleep 2 + +dnl Remove policer +AT_CHECK([ovs-vsctl set interface phy0 ingress_policing_kpkts_rate=0 ingress_policing_kpkts_burst=0]) + +dnl Check policer was removed correctly +AT_CHECK([ovs-vsctl list interface phy0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 0' stdout], [], [stdout]) + +AT_CHECK([ovs-vsctl list interface phy0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 0' stdout], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 phy0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("[SYSTEM_DPDK_ALLOWED_LOGS]") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl Ingress policing (kpkts) create delete vport port +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) create delete vport port]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add ingress policer +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 ingress_policing_kpkts_rate=10000 ingress_policing_kpkts_burst=1000]) +AT_CHECK([ovs-vsctl show], [], [stdout]) +sleep 2 + +dnl Remove ingress policer +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 ingress_policing_kpkts_rate=0 ingress_policing_kpkts_burst=0]) + +dnl Check ingress policer was removed correctly +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 0' stdout], [], [stdout]) + +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 0' stdout], [], [stdout]) + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No such file or directory@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl Ingress policing (kpkts) no policing rate +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) no policing rate]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add ingress policer +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 ingress_policing_kpkts_burst=1000]) +AT_CHECK([ovs-vsctl show], [], [stdout]) +sleep 2 + +dnl Check ingress policer was created correctly +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 1000' stdout], [], [stdout]) + +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 0' stdout], [], [stdout]) + + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No such file or directory@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl Ingress policing (kpkts) no policing burst +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) no policing burst]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add ingress policer +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 ingress_policing_kpkts_rate=10000]) +AT_CHECK([ovs-vsctl show], [], [stdout]) +sleep 2 + +dnl Fail if ingress policer could not be created +AT_FAIL_IF([grep "Could not create rte meter for ingress policer" ovs-vswitchd.log], [], [stdout]) + +dnl Check ingress policer was created correctly +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 0' stdout], [], [stdout]) + +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 10000' stdout], [], [stdout]) + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No such file or directory@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + dnl -------------------------------------------------------------------------- dnl QoS create delete phy port AT_SETUP([OVS-DPDK - QoS create delete phy port])