From patchwork Mon Jun 21 08:45:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Surya Rudra X-Patchwork-Id: 1494949 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=acldigital.com header.i=@acldigital.com header.a=rsa-sha256 header.s=AB254A94-C841-11EA-AD01-099C9C043707 header.b=MJYou6AZ; dkim-atps=neutral Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4G7jp3356Mz9sRN for ; Mon, 21 Jun 2021 18:47:23 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 4F976401D7; Mon, 21 Jun 2021 08:47:21 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UpU0XFhLqpzk; Mon, 21 Jun 2021 08:47:20 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 557D94013A; Mon, 21 Jun 2021 08:47:19 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 17BE6C000E; Mon, 21 Jun 2021 08:47:19 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5D88FC000C for ; Mon, 21 Jun 2021 08:47:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 440BE40323 for ; Mon, 21 Jun 2021 08:47:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=acldigital.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bezDTLK0NfvM for ; Mon, 21 Jun 2021 08:47:16 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from mail.acldigital.com (mail.acldigital.com [182.73.72.45]) by smtp4.osuosl.org (Postfix) with ESMTPS id 8068C40320 for ; Mon, 21 Jun 2021 08:47:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.acldigital.com (Postfix) with ESMTP id C0F872120796 for ; Mon, 21 Jun 2021 14:17:06 +0530 (IST) Received: from mail.acldigital.com ([127.0.0.1]) by localhost (mail.acldigital.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id uYrZwLCqnNKJ; Mon, 21 Jun 2021 14:17:06 +0530 (IST) Received: from localhost (localhost [127.0.0.1]) by mail.acldigital.com (Postfix) with ESMTP id 217D4212074D; Mon, 21 Jun 2021 14:17:06 +0530 (IST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.acldigital.com 217D4212074D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=acldigital.com; s=AB254A94-C841-11EA-AD01-099C9C043707; t=1624265226; bh=68OIlHsEn9di6dsANi/2PFsDPiUMXbJnL3674n7ujv8=; h=From:To:Date:Message-Id:MIME-Version; b=MJYou6AZL5wRZk5phli1So0S6JZY67LY7qH7U+H2di3qNf5Tkx51nvbzkkDhhhDxU D2GQhcm8YNc9HK8BU7OHpbyF/P230EeKM8s8rIm9QK3N+xtQuBQFPz3//Hy5+JpHJZ ZWIp2fjZK4poLd10T/UV7u/bebTTxErCbA0ujHcM1P2Z+X6F3qf8rgTxILUjkQG4al OKbIQ3qM6Pawfy7S1/hJ/FSVZ0nzxMyDJ6491TKS+0zZ+Hx+JwEIjg6TdTgRVFcymS DDe4NY30hQxwqSytWv9uVGSMyDMlTodiyKi+WKAf0GJtL7WJmDa2RZNHEYEOQ8Dchs CnpZOuUqwU3zw== X-Virus-Scanned: amavisd-new at acldigital.com Received: from mail.acldigital.com ([127.0.0.1]) by localhost (mail.acldigital.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id OCKaWgd6Lbm4; Mon, 21 Jun 2021 14:17:06 +0530 (IST) Received: from localhost.localdomain (unknown [10.1.0.4]) by mail.acldigital.com (Postfix) with ESMTPSA id 6FC0B21206F6; Mon, 21 Jun 2021 14:17:05 +0530 (IST) To: ovs-dev@openvswitch.org Date: Mon, 21 Jun 2021 14:15:10 +0530 Message-Id: <20210621084510.1820-1-rudrasurya.r@acldigital.com> X-Mailer: git-send-email 2.27.0.windows.1 MIME-Version: 1.0 Cc: Surya Rudra Subject: [ovs-dev] [PATCH] ofproto-dpif-xlate: Optimize select group performance. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Surya Rudra via dev From: Surya Rudra Reply-To: Surya Rudra Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In a new OVS use case with a 3rd party SDN controller, we observe two new typical patterns that are handled sub-optimally: 1. Degenerate select groups with a single bucket. For such select groups the overhead of computing a dp_hash and recirculating the packet in the datapath to look up the bucket is superfluous. 2. Select groups with 2 equally weighted buckets The minimum lookup table size is set to 16 slots (4 bit hash) even in this scenario where two slots would suffice (1 bit hash). This implies an unnecessary 8-fold increase of needed recirculation megaflow entries in the datapath. In select group use cases where the number of megaflows is already high(i.e.10-100K) an additional factor 8 can push the OVS megaflow cache over the edge. Solution: The patch contains two changes: 1. If the select group has only a single bucket, skip dp_hash calculation an recirculation entirely and execute the single bucket's actions directly. 2. Do not artificially increase the size of the dp_hash lookup table to 16 slots if the bucket configuration allows an accurate representation with 2, 4 or 8 slots. This minimizes the number of recirculated megaflows. Testing: Updated existing unit testcases and added new testcase with one bucket in select group. Signed-off-by: Surya Rudra --- ofproto/ofproto-dpif-xlate.c | 5 +++++ ofproto/ofproto-dpif.c | 3 +-- tests/ofproto-dpif.at | 43 +++++++++++++++++++++++++++--------- 3 files changed, 38 insertions(+), 13 deletions(-) diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c index a6f4ea334..5c41357f1 100644 --- a/ofproto/ofproto-dpif-xlate.c +++ b/ofproto/ofproto-dpif-xlate.c @@ -4612,6 +4612,11 @@ pick_hash_fields_select_group(struct xlate_ctx *ctx, struct group_dpif *group) static struct ofputil_bucket * pick_dp_hash_select_group(struct xlate_ctx *ctx, struct group_dpif *group) { + /* Skip dp_hash recirculation in the special case of a single bucket */ + if (group->up.n_buckets == 1) { + return group_first_live_bucket(ctx, group, 0); + } + uint32_t dp_hash = ctx->xin->flow.dp_hash; /* dp_hash value 0 is special since it means that the dp_hash has not been diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c index 47db9bb57..3bd2136d8 100644 --- a/ofproto/ofproto-dpif.c +++ b/ofproto/ofproto-dpif.c @@ -5080,8 +5080,7 @@ group_setup_dp_hash_table(struct group_dpif *group, size_t max_hash) min_weight, total_weight); uint64_t min_slots = DIV_ROUND_UP(total_weight, min_weight); - uint64_t min_slots2 = ROUND_UP_POW2(min_slots); - uint64_t n_hash = MAX(16, min_slots2); + uint64_t n_hash = ROUND_UP_POW2(min_slots); if (n_hash > MAX_SELECT_GROUP_HASH_VALUES || (max_hash != 0 && n_hash > max_hash)) { VLOG_DBG(" Too many hash values required: %"PRIu64, n_hash); diff --git a/tests/ofproto-dpif.at b/tests/ofproto-dpif.at index 31064ed95..e41c44d57 100644 --- a/tests/ofproto-dpif.at +++ b/tests/ofproto-dpif.at @@ -607,10 +607,31 @@ AT_CHECK([tail -1 stdout], [0], OVS_VSWITCHD_STOP AT_CLEANUP +AT_SETUP([ofproto-dpif - select group with one bucket in action list]) +OVS_VSWITCHD_START +add_of_ports br0 1 10 +AT_CHECK([ovs-ofctl -O OpenFlow12 add-group br0 'group_id=1234,type=select,bucket=weight:10,set_field:192.168.3.90->ip_src,output:10']) +AT_CHECK([ovs-ofctl -O OpenFlow12 add-flow br0 'ip actions=group:1234,output:10']) +AT_CHECK([ovs-appctl ofproto/trace br0 'in_port=1,dl_src=50:54:00:00:00:05,dl_dst=50:54:00:00:00:07,dl_type=0x0800,nw_src=192.168.0.1,nw_dst=192.168.0.2,nw_proto=1,nw_tos=0,nw_ttl=128,icmp_type=8,icmp_code=0'], [0], [stdout]) +AT_CHECK([tail -10 stdout | sed 's/dp_hash=.*\/0xf/dp_hash=0xXXXX\/0xf/'], [0], + [ group:1234 + -> using bucket 0 + bucket 0 + set_field:192.168.3.90->ip_src + output:10 + output:10 + +Final flow: unchanged +Megaflow: recirc_id=0,eth,ip,in_port=1,nw_src=192.168.0.1,nw_frag=no +Datapath actions: set(ipv4(src=192.168.3.90)),10,set(ipv4(src=192.168.0.1)),10 +]) +OVS_VSWITCHD_STOP +AT_CLEANUP + AT_SETUP([ofproto-dpif - group actions have no effect afterwards]) OVS_VSWITCHD_START add_of_ports br0 1 10 -AT_CHECK([ovs-ofctl -O OpenFlow12 add-group br0 'group_id=1234,type=select,bucket=set_field:192.168.3.90->ip_src,output:10']) +AT_CHECK([ovs-ofctl -O OpenFlow12 add-group br0 'group_id=1234,type=select,bucket=weight:10,set_field:192.168.3.90->ip_src,output:10,bucket=set_field:192.168.3.90->ip_src,output:10']) AT_CHECK([ovs-ofctl -O OpenFlow12 add-flow br0 'ip actions=group:1234,output:10']) for d in 0 1 2 3; do @@ -690,10 +711,10 @@ AT_CHECK([grep -A6 "Constructing select group 1234" ovs-vswitchd.log | sed 's/^. ofproto_dpif|DBG|Constructing select group 1234 ofproto_dpif|DBG|No selection method specified. Trying dp_hash. ofproto_dpif|DBG| Minimum weight: 1, total weight: 2 -ofproto_dpif|DBG| Using 16 hash values: -ofproto_dpif|DBG| Bucket 0: weight=1, target=8.00 hits=8 -ofproto_dpif|DBG| Bucket 1: weight=1, target=8.00 hits=8 -ofproto_dpif|DBG|Use dp_hash with 16 hash values using algorithm 1. +ofproto_dpif|DBG| Using 2 hash values: +ofproto_dpif|DBG| Bucket 0: weight=1, target=1.00 hits=1 +ofproto_dpif|DBG| Bucket 1: weight=1, target=1.00 hits=1 +ofproto_dpif|DBG|Use dp_hash with 2 hash values using algorithm 1. ]) AT_CHECK([ovs-ofctl -O OpenFlow12 add-flow br0 'ip actions=write_actions(group:1234)']) @@ -706,7 +727,7 @@ for d in 0 1 2 3; do done done -AT_CHECK([ovs-appctl dpctl/dump-flows | sort | strip_ufid | strip_used | check_dpflow_stats 5 2 dp_hash], [0], [dnl +AT_CHECK([ovs-appctl dpctl/dump-flows | sort | strip_ufid | strip_used | check_dpflow_stats 1 2 dp_hash], [0], [dnl recirc_id(0),in_port(1),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(frag=no), packets:15, bytes:1590, used:0.0s, actions:hash(sym_l4(0)),recirc(0x1) n_flows=ok n_buckets=ok ]) @@ -726,7 +747,7 @@ for d in 0 1 2 3; do AT_CHECK([ovs-appctl netdev-dummy/receive p1 $pkt]) done -AT_CHECK([ovs-appctl dpctl/dump-flows | sort| sed 's/dp_hash(.*\/0xf)/dp_hash(0xXXXX\/0xf)/' | strip_ufid | strip_used], [0], [dnl +AT_CHECK([ovs-appctl dpctl/dump-flows | sort| sed 's/dp_hash(.*\/0x1)/dp_hash(0xXXXX\/0xf)/' | strip_ufid | strip_used], [0], [dnl flow-dump from the main thread: recirc_id(0),in_port(1),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(frag=no), packets:3, bytes:318, used:0.0s, actions:hash(sym_l4(0)),recirc(0x1) recirc_id(0x1),dp_hash(0xXXXX/0xf),in_port(1),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(frag=no), packets:3, bytes:318, used:0.0s, actions:11 @@ -820,10 +841,10 @@ AT_CHECK([grep -A6 "Constructing select group 1234" ovs-vswitchd.log | sed 's/^. ofproto_dpif|DBG|Constructing select group 1234 ofproto_dpif|DBG|Selection method specified: dp_hash. ofproto_dpif|DBG| Minimum weight: 1, total weight: 2 -ofproto_dpif|DBG| Using 16 hash values: -ofproto_dpif|DBG| Bucket 0: weight=1, target=8.00 hits=8 -ofproto_dpif|DBG| Bucket 1: weight=1, target=8.00 hits=8 -ofproto_dpif|DBG|Use dp_hash with 16 hash values using algorithm 0. +ofproto_dpif|DBG| Using 2 hash values: +ofproto_dpif|DBG| Bucket 0: weight=1, target=1.00 hits=1 +ofproto_dpif|DBG| Bucket 1: weight=1, target=1.00 hits=1 +ofproto_dpif|DBG|Use dp_hash with 2 hash values using algorithm 0. ]) # Fall back to legacy hash with zero buckets