From patchwork Sun May 24 21:49:35 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 476002 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id B71141402B9 for ; Mon, 25 May 2015 07:49:54 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b=p/fzcsOs; dkim-atps=neutral Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751353AbbEXVtj (ORCPT ); Sun, 24 May 2015 17:49:39 -0400 Received: from mail-ig0-f181.google.com ([209.85.213.181]:33528 "EHLO mail-ig0-f181.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751244AbbEXVti (ORCPT ); Sun, 24 May 2015 17:49:38 -0400 Received: by igbpi8 with SMTP id pi8so25925996igb.0 for ; Sun, 24 May 2015 14:49:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:cc:date:content-type:mime-version :content-transfer-encoding; bh=EdYxtA0YkyPRFx/kE4PMm1w0IEs69pt/BnA1YK7ZYHs=; b=p/fzcsOsEAsY/fM36tryYXJGEnFBCw+/soiGhZ1xff7wNzjJJPpZ0AV7DgCWlLqh4u 32vx42vF3IBAEmmUmffXu2uYoX4SGfZyGhdUWrWL5ZjilZ45qxH6LhGsScPA05DGtqSc nayBBmBVl3Y5xx6PGkoJV8TcpYiebL7THgSRooVmBY5sGzsxkDLeglZTiCpy4WDm7ScT 9gFJnCikKZNDnmtZhKabDbtScLG8m4XQDHV/oCtlGltydj9/JLfdI8d7gPpdr5RwQe8P uoY4b8mGEp+nr6jt6qaU78kaD0D9QBDoAAKy2P9J9e6Q8UK7QOnoPFupoJn6JIQYoxLR CmQQ== X-Received: by 10.50.79.202 with SMTP id l10mr19626498igx.7.1432504177662; Sun, 24 May 2015 14:49:37 -0700 (PDT) Received: from [172.19.255.13] ([172.19.255.13]) by mx.google.com with ESMTPSA id i4sm4663142igm.2.2015.05.24.14.49.36 (version=TLSv1.2 cipher=AES128-GCM-SHA256 bits=128/128); Sun, 24 May 2015 14:49:36 -0700 (PDT) Message-ID: <1432504175.4060.155.camel@edumazet-glaptop2.roam.corp.google.com> Subject: [PATCH net-next] tcp/dccp: try to not exhaust ip_local_port_range in connect() From: Eric Dumazet To: David Miller Cc: netdev Date: Sun, 24 May 2015 14:49:35 -0700 X-Mailer: Evolution 3.10.4-0ubuntu2 Mime-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Eric Dumazet A long standing problem on busy servers is the tiny available TCP port range (/proc/sys/net/ipv4/ip_local_port_range) and the default sequential allocation of source ports in connect() system call. If a host is having a lot of active TCP sessions, chances are very high that all ports are in use by at least one flow, and subsequent bind(0) attempts fail, or have to scan a big portion of space to find a slot. In this patch, I changed the starting point in __inet_hash_connect() so that we try to favor even [1] ports, leaving odd ports for bind() users. We still perform a sequential search, so there is no guarantee, but if connect() targets are very different, end result is we leave more ports available to bind(), and we spread them all over the range, lowering time for both connect() and bind() to find a slot. This strategy only works well if /proc/sys/net/ipv4/ip_local_port_range is even, ie if start/end values have different parity. Therefore, default /proc/sys/net/ipv4/ip_local_port_range was changed to 32768 - 60999 (instead of 32768 - 61000) There is no change on security aspects here, only some poor hashing schemes could be eventually impacted by this change. [1] : The odd/even property depends on ip_local_port_range values parity Signed-off-by: Eric Dumazet --- Documentation/networking/ip-sysctl.txt | 8 +++++--- net/ipv4/af_inet.c | 2 +- net/ipv4/inet_hashtables.c | 10 ++++++++-- 3 files changed, 14 insertions(+), 6 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/Documentation/networking/ip-sysctl.txt b/Documentation/networking/ip-sysctl.txt index cb083e0d682c6faae13aea21aa1a88868a39c632..5fae7704daab292cf900158666c2d4bb80dd2424 100644 --- a/Documentation/networking/ip-sysctl.txt +++ b/Documentation/networking/ip-sysctl.txt @@ -751,8 +751,10 @@ IP Variables: ip_local_port_range - 2 INTEGERS Defines the local port range that is used by TCP and UDP to choose the local port. The first number is the first, the - second the last local port number. The default values are - 32768 and 61000 respectively. + second the last local port number. + If possible, it is better these numbers have different parity. + (one even and one odd values) + The default values are 32768 and 60999 respectively. ip_local_reserved_ports - list of comma separated ranges Specify the ports which are reserved for known third-party @@ -775,7 +777,7 @@ ip_local_reserved_ports - list of comma separated ranges ip_local_port_range, e.g.: $ cat /proc/sys/net/ipv4/ip_local_port_range - 32000 61000 + 32000 60999 $ cat /proc/sys/net/ipv4/ip_local_reserved_ports 8080,9148 diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c index 235d36afece3b53f5fd5f795f28d15f6f2a79ab6..6ad0f7a711c97b4dabcd328509b9a38ef8a159f5 100644 --- a/net/ipv4/af_inet.c +++ b/net/ipv4/af_inet.c @@ -1595,7 +1595,7 @@ static __net_init int inet_init_net(struct net *net) */ seqlock_init(&net->ipv4.ip_local_ports.lock); net->ipv4.ip_local_ports.range[0] = 32768; - net->ipv4.ip_local_ports.range[1] = 61000; + net->ipv4.ip_local_ports.range[1] = 60999; seqlock_init(&net->ipv4.ping_group_range.lock); /* diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c index 3766bddb3e8a7303123aa7e32507f6f7801c10d5..8c0fc6fbc1afa08baf07ca86e98aa966a3f8e826 100644 --- a/net/ipv4/inet_hashtables.c +++ b/net/ipv4/inet_hashtables.c @@ -501,8 +501,14 @@ int __inet_hash_connect(struct inet_timewait_death_row *death_row, inet_get_local_port_range(net, &low, &high); remaining = (high - low) + 1; + /* By starting with offset being an even number, + * we tend to leave about 50% of ports for other uses, + * like bind(0). + */ + offset &= ~1; + local_bh_disable(); - for (i = 1; i <= remaining; i++) { + for (i = 0; i < remaining; i++) { port = low + (i + offset) % remaining; if (inet_is_local_reserved_port(net, port)) continue; @@ -546,7 +552,7 @@ int __inet_hash_connect(struct inet_timewait_death_row *death_row, return -EADDRNOTAVAIL; ok: - hint += i; + hint += (i + 2) & ~1; /* Head lock still held and bh's disabled */ inet_bind_hash(sk, tb, port);