From patchwork Thu Sep 9 03:06:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: wenxu X-Patchwork-Id: 1526071 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H4kSZ0RpFz9sQt for ; Thu, 9 Sep 2021 13:07:09 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id EED1783E6B; Thu, 9 Sep 2021 03:07:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rtVNvLx7I6Et; Thu, 9 Sep 2021 03:07:04 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 7630B82F11; Thu, 9 Sep 2021 03:07:03 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 330EBC000F; Thu, 9 Sep 2021 03:07:03 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id D9D08C000D for ; Thu, 9 Sep 2021 03:07:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id C35F4407B1 for ; Thu, 9 Sep 2021 03:07:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aLrPtmQQbpte for ; Thu, 9 Sep 2021 03:07:00 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mail-m2456.qiye.163.com (mail-m2456.qiye.163.com [220.194.24.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 97FF5402B1 for ; Thu, 9 Sep 2021 03:06:59 +0000 (UTC) Received: from localhost.localdomain (unknown [117.50.0.204]) by mail-m2456.qiye.163.com (Hmail) with ESMTPA id D8CC1700248; Thu, 9 Sep 2021 11:06:54 +0800 (CST) From: wenxu@ucloud.cn To: pvalerio@redhat.com, aconole@redhat.com Date: Thu, 9 Sep 2021 11:06:52 +0800 Message-Id: <1631156814-12127-1-git-send-email-wenxu@ucloud.cn> X-Mailer: git-send-email 1.8.3.1 X-HM-Spam-Status: e1kfGhgUHx5ZQUtXWQgPGg8OCBgUHx5ZQUlOS1dZCBgUCR5ZQVlLVUtZV1 kWDxoPAgseWUFZKDYvK1lXWShZQUlCN1dZLVlBSVdZDwkaFQgSH1lBWRoaTh9WS0hNSk1MGEkfGR kYVRkRExYaEhckFA4PWVdZFhoPEhUdFFlBWVVLWQY+ X-HM-Sender-Digest: e1kMHhlZQR0aFwgeV1kSHx4VD1lBWUc6MAw6Pzo4EDNDMxQpGkwxCxg2 TlEKFDBVSlVKTUhKSk5NQ0pOSUhDVTMWGhIXVQweFQMOOw4YFxQOH1UYFUVZV1kSC1lBWUpKTFVO S1VLVUlLT1lXWQgBWUFJT09LNwY+ X-HM-Tid: 0a7bc885c43c8c15kuqtd8cc1700248 Cc: dev@openvswitch.org, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v4 1/3] conntrack: restore the origin sport for each round with new address X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: wenxu It is better to choose the origin select sport as current sport for each port search round with new address. Signed-off-by: wenxu --- lib/conntrack.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/conntrack.c b/lib/conntrack.c index 551c206..00906f8 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -2409,11 +2409,11 @@ nat_get_unique_tuple(struct conntrack *ct, const struct conn *conn, { union ct_addr min_addr = {0}, max_addr = {0}, curr_addr = {0}, guard_addr = {0}; + uint16_t min_sport, max_sport, curr_sport, orig_sport; uint32_t hash = nat_range_hash(conn, ct->hash_basis); bool pat_proto = conn->key.nw_proto == IPPROTO_TCP || conn->key.nw_proto == IPPROTO_UDP; uint16_t min_dport, max_dport, curr_dport; - uint16_t min_sport, max_sport, curr_sport; min_addr = conn->nat_info->min_addr; max_addr = conn->nat_info->max_addr; @@ -2425,7 +2425,7 @@ nat_get_unique_tuple(struct conntrack *ct, const struct conn *conn, * we can stop once we reach it. */ guard_addr = curr_addr; - set_sport_range(conn->nat_info, &conn->key, hash, &curr_sport, + set_sport_range(conn->nat_info, &conn->key, hash, &orig_sport, &min_sport, &max_sport); set_dport_range(conn->nat_info, &conn->key, hash, &curr_dport, &min_dport, &max_dport); @@ -2443,6 +2443,8 @@ another_round: goto next_addr; } + curr_sport = orig_sport; + FOR_EACH_PORT_IN_RANGE(curr_dport, min_dport, max_dport) { nat_conn->rev_key.src.port = htons(curr_dport); FOR_EACH_PORT_IN_RANGE(curr_sport, min_sport, max_sport) { From patchwork Thu Sep 9 03:06:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: wenxu X-Patchwork-Id: 1526072 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H4kSd5Y8Dz9sQt for ; Thu, 9 Sep 2021 13:07:13 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 7F032407EA; Thu, 9 Sep 2021 03:07:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id w51507ouVTPw; Thu, 9 Sep 2021 03:07:08 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id EEEE3402B1; Thu, 9 Sep 2021 03:07:06 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 980C9C0028; Thu, 9 Sep 2021 03:07:04 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id F3162C000D for ; Thu, 9 Sep 2021 03:07:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id CA80B402B1 for ; Thu, 9 Sep 2021 03:07:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yBKuZs3NPzQF for ; Thu, 9 Sep 2021 03:07:01 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mail-m2456.qiye.163.com (mail-m2456.qiye.163.com [220.194.24.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 1450B407AC for ; Thu, 9 Sep 2021 03:07:00 +0000 (UTC) Received: from localhost.localdomain (unknown [117.50.0.204]) by mail-m2456.qiye.163.com (Hmail) with ESMTPA id 3F216700263; Thu, 9 Sep 2021 11:06:55 +0800 (CST) From: wenxu@ucloud.cn To: pvalerio@redhat.com, aconole@redhat.com Date: Thu, 9 Sep 2021 11:06:53 +0800 Message-Id: <1631156814-12127-2-git-send-email-wenxu@ucloud.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1631156814-12127-1-git-send-email-wenxu@ucloud.cn> References: <1631156814-12127-1-git-send-email-wenxu@ucloud.cn> X-HM-Spam-Status: e1kfGhgUHx5ZQUtXWQgPGg8OCBgUHx5ZQUlOS1dZCBgUCR5ZQVlLVUtZV1 kWDxoPAgseWUFZKDYvK1lXWShZQUlCN1dZLVlBSVdZDwkaFQgSH1lBWUNCGk1WHh0fTEpPQ0pDHx 1LVRkRExYaEhckFA4PWVdZFhoPEhUdFFlBWVVLWQY+ X-HM-Sender-Digest: e1kMHhlZQR0aFwgeV1kSHx4VD1lBWUc6Mz46DDo*CDNOPxQjMEw2CxhJ TCIwCx9VSlVKTUhKSk5NQ0pOT0xLVTMWGhIXVQweFQMOOw4YFxQOH1UYFUVZV1kSC1lBWUpKTFVO S1VLVUlLT1lXWQgBWUFKQkNNNwY+ X-HM-Tid: 0a7bc885c5af8c15kuqt3f216700263 Cc: dev@openvswitch.org, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v4 2/3] conntrack: select correct sport range for well-known origin sport X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: wenxu Like the kernel datapath. The sport nat range for well-konwn origin sport should limit in the well-known ports. Signed-off-by: wenxu --- lib/conntrack.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/lib/conntrack.c b/lib/conntrack.c index 00906f8..f95532c 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -2261,8 +2261,16 @@ set_sport_range(struct nat_action_info_t *ni, const struct conn_key *k, if (((ni->nat_action & NAT_ACTION_SNAT_ALL) == NAT_ACTION_SRC) || ((ni->nat_action & NAT_ACTION_DST))) { *curr = ntohs(k->src.port); - *min = MIN_NAT_EPHEMERAL_PORT; - *max = MAX_NAT_EPHEMERAL_PORT; + if (*curr < 512) { + *min = 1; + *max = 511; + } else if (*curr < 1024) { + *min = 600; + *max = 1023; + } else { + *min = MIN_NAT_EPHEMERAL_PORT; + *max = MAX_NAT_EPHEMERAL_PORT; + } } else { *min = ni->min_port; *max = ni->max_port; From patchwork Thu Sep 9 03:06:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: wenxu X-Patchwork-Id: 1526073 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H4kSl0P0Jz9t0k for ; Thu, 9 Sep 2021 13:07:19 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id EC1B84071E; Thu, 9 Sep 2021 03:07:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZBnD259w7QmQ; Thu, 9 Sep 2021 03:07:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id A475640708; Thu, 9 Sep 2021 03:07:05 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id EA6FEC0024; Thu, 9 Sep 2021 03:07:03 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 37F2BC000D for ; Thu, 9 Sep 2021 03:07:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 1B5FA830CD for ; Thu, 9 Sep 2021 03:07:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p2GvVI9BD_-s for ; Thu, 9 Sep 2021 03:07:01 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mail-m2456.qiye.163.com (mail-m2456.qiye.163.com [220.194.24.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3D9CB82F11 for ; Thu, 9 Sep 2021 03:07:01 +0000 (UTC) Received: from localhost.localdomain (unknown [117.50.0.204]) by mail-m2456.qiye.163.com (Hmail) with ESMTPA id 7773670023C; Thu, 9 Sep 2021 11:06:55 +0800 (CST) From: wenxu@ucloud.cn To: pvalerio@redhat.com, aconole@redhat.com Date: Thu, 9 Sep 2021 11:06:54 +0800 Message-Id: <1631156814-12127-3-git-send-email-wenxu@ucloud.cn> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1631156814-12127-1-git-send-email-wenxu@ucloud.cn> References: <1631156814-12127-1-git-send-email-wenxu@ucloud.cn> X-HM-Spam-Status: e1kfGhgUHx5ZQUtXWQgPGg8OCBgUHx5ZQUlOS1dZCBgUCR5ZQVlLVUtZV1 kWDxoPAgseWUFZKDYvK1lXWShZQUlCN1dZLVlBSVdZDwkaFQgSH1lBWUMYQ0lWQxhLTEtDHxkeTx pLVRkRExYaEhckFA4PWVdZFhoPEhUdFFlBWVVLWQY+ X-HM-Sender-Digest: e1kMHhlZQR0aFwgeV1kSHx4VD1lBWUc6MyI6TTo5PDNCPxQxT0JRCx0i Qh8KCR5VSlVKTUhKSk5NQ0pOTExNVTMWGhIXVQweFQMOOw4YFxQOH1UYFUVZV1kSC1lBWUpKTFVO S1VLVUlLT1lXWQgBWUFPT0NLNwY+ X-HM-Tid: 0a7bc885c6948c15kuqt7773670023c Cc: dev@openvswitch.org, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v4 3/3] conntrack: limit port clash resolution attempts X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: wenxu In case almost or all available ports are taken, clash resolution can take a very long time, resulting in pmd hang in conntrack. This can happen when many to-be-natted hosts connect to same destination:port (e.g. a proxy) and all connections pass the same SNAT. Pick a random offset in the acceptable range, then try ever smaller number of adjacent port numbers, until either the limit is reached or a useable port was found. This results in at most 248 attempts (128 + 64 + 32 + 16 + 8, i.e. 4 restarts with new search offset) instead of 64000+. Signed-off-by: wenxu --- lib/conntrack.c | 47 +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 43 insertions(+), 4 deletions(-) diff --git a/lib/conntrack.c b/lib/conntrack.c index f95532c..485b31c 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -2421,7 +2421,11 @@ nat_get_unique_tuple(struct conntrack *ct, const struct conn *conn, uint32_t hash = nat_range_hash(conn, ct->hash_basis); bool pat_proto = conn->key.nw_proto == IPPROTO_TCP || conn->key.nw_proto == IPPROTO_UDP; + unsigned int attempts, max_attempts, min_attempts; uint16_t min_dport, max_dport, curr_dport; + uint16_t range_src, range_dst, range_max; + uint32_t range_addr; + unsigned int i; min_addr = conn->nat_info->min_addr; max_addr = conn->nat_info->max_addr; @@ -2438,6 +2442,19 @@ nat_get_unique_tuple(struct conntrack *ct, const struct conn *conn, set_dport_range(conn->nat_info, &conn->key, hash, &curr_dport, &min_dport, &max_dport); + range_src = max_sport - min_sport + 1; + range_dst = max_dport - min_dport + 1; + range_max = range_src > range_dst ? range_src : range_dst; + range_addr = ntohl(max_addr.ipv4) - ntohl(min_addr.ipv4) + 1; + max_attempts = 128 / range_addr; + if (max_attempts < 1) { + max_attempts = 1; + } + min_attempts = 16 / range_addr; + if (min_attempts < 2) { + min_attempts = 2; + } + another_round: store_addr_to_key(&curr_addr, &nat_conn->rev_key, conn->nat_info->nat_action); @@ -2453,17 +2470,39 @@ another_round: curr_sport = orig_sport; + attempts = range_max; + if (attempts > max_attempts) { + attempts = max_attempts; + } + +another_port_round: + i = 0; FOR_EACH_PORT_IN_RANGE(curr_dport, min_dport, max_dport) { nat_conn->rev_key.src.port = htons(curr_dport); FOR_EACH_PORT_IN_RANGE(curr_sport, min_sport, max_sport) { - nat_conn->rev_key.dst.port = htons(curr_sport); - if (!conn_lookup(ct, &nat_conn->rev_key, - time_msec(), NULL, NULL)) { - return true; + if (i++ < attempts) { + nat_conn->rev_key.dst.port = htons(curr_sport); + if (!conn_lookup(ct, &nat_conn->rev_key, + time_msec(), NULL, NULL)) { + return true; + } + } else { + goto next_attempts; } } } +next_attempts: + if (attempts >= range_max || attempts < min_attempts) { + goto next_addr; + } + + attempts /= 2; + curr_dport = min_dport + (random_uint32() % range_dst); + curr_sport = min_sport + (random_uint32() % range_src); + + goto another_port_round; + /* Check if next IP is in range and respin. Otherwise, notify * exhaustion to the caller. */ next_addr: