From patchwork Sat Aug 9 17:15:30 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom Herbert X-Patchwork-Id: 378787 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id F1A471400D6 for ; Sun, 10 Aug 2014 03:15:42 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751951AbaHIRPd (ORCPT ); Sat, 9 Aug 2014 13:15:33 -0400 Received: from mail-oa0-f73.google.com ([209.85.219.73]:39379 "EHLO mail-oa0-f73.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751821AbaHIRPb (ORCPT ); Sat, 9 Aug 2014 13:15:31 -0400 Received: by mail-oa0-f73.google.com with SMTP id g18so1233575oah.0 for ; Sat, 09 Aug 2014 10:15:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=date:from:to:cc:subject:message-id:user-agent:mime-version :content-type; bh=YGX8YEBwqmp448chqgIGDHEnXxS3xZJAdqnHrFI745M=; b=fYwok/TI5NVSIdtjvBpIJ3hmrDwfYRfzZoBKDVOq/GLxhJF9kLn+bMc77IPlYjdl52 KYLcBsxq3N/ChzXC5qaBvhK7vS0T6y3KbvJ9YHHpS6gib85ZFi6BFVePmBtIL7pAxtPu 79GofvzpGzQwOHYWs4VriKCOjCty4lqUe6+zCoXxCsOYXgaSDowhyznJsGKZ5pfQi7UF 7U1HO2sjrfm5Lam+Ld9OJiYKlv50aIQLObXwWKOKaxwE8hr5AW2vh1LbkMquW1DwQERx 20D8YDGdpfdoA31Fu4cVulP/SHP0Umuy+1/3F3YK2/GtRVzcd2Zx3m9LCoOXojflH19W G5og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:user-agent :mime-version:content-type; bh=YGX8YEBwqmp448chqgIGDHEnXxS3xZJAdqnHrFI745M=; b=GDJP8wPhfIJ6ApPAXkVP+qn38aEfwij64OiRY61Gh5mQFivrxpJE5KmCP+u7aZ1Ifv pnhvp6puvOmDZloIO30+xw7G60j6oe9DZCa1aKRMYfPT+1xiEBLQz3xzax+tg1dGR8o2 hdPm1VrEk9t+icsfeN5Kn1pymHQHNHEPZAgwEjajJsMIbSjpaqLEp8X+I/TBSj5LKeZX NQhrfYmDc4TXy2hZamvZChg9m5jTglBDvIpitoL4oI0szIwnW7yZXbpLbegYkVchMEkj zggsgZFOnHZxtfuDYcVbGHiUcOG7MmYQPwZA2ALGzS7iioFZZ64kbLXXeM00Cu109ZGE UUJA== X-Gm-Message-State: ALoCoQn78y2Zu22lpHEgSNdTDF1I+PCh8Q7GXMqvBVVfyNMcQiCZOzw1E1CnQv1p7R4ms+/LlhKc X-Received: by 10.42.133.200 with SMTP id i8mr8001239ict.14.1407604530800; Sat, 09 Aug 2014 10:15:30 -0700 (PDT) Received: from corp2gmr1-2.hot.corp.google.com (corp2gmr1-2.hot.corp.google.com [172.24.189.93]) by gmr-mx.google.com with ESMTPS id l23si680492yhg.1.2014.08.09.10.15.30 for (version=TLSv1.1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 09 Aug 2014 10:15:30 -0700 (PDT) Received: from tomh.mtv.corp.google.com (tomh.mtv.corp.google.com [172.18.117.126]) by corp2gmr1-2.hot.corp.google.com (Postfix) with ESMTP id A03675A4532; Sat, 9 Aug 2014 10:15:30 -0700 (PDT) Received: by tomh.mtv.corp.google.com (Postfix, from userid 60832) id 3D3A4200792; Sat, 9 Aug 2014 10:15:30 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by tomh.mtv.corp.google.com (Postfix) with ESMTP id 2026C200434; Sat, 9 Aug 2014 10:15:30 -0700 (PDT) Date: Sat, 9 Aug 2014 10:15:30 -0700 (PDT) From: Tom Herbert To: davem@davemloft.net, netdev@vger.kernel.org cc: eric.dumazet@gmail.com Subject: [PATCH v2 net-next] udp: clear rps flow table for packets recv on UDP unconnected sockets Message-ID: User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This patch addresses a problem posed by Eric Dumazet in RPS/RFS concerning the interaction between connected sockets and traffic on unnconnected sockets. On a server which has both traffic using connected connected sockets and traffic that is going through unconnected UDP sockets, it is very possible that the connected sockets could heavily populate the RPS flow tables. Packets received on unconnected sockets would then be steered based on unrelated entries in the flow tables which leads to suboptimal steering. This happens even if the connections that populate the table are inactive with no traffic, as long as the connected sockets simply remain open unrelated traffic can be steered using that information. This problem would further be exacerbated if the packets on the unconnected UDP sockets are actually part of long lived flows (which apparently would happen with QUIC in their current implementation). This patch clears the RPS flow hash table for packet recieved on unnconnected UDP sockets. The effect is that the "flows" on unconnected socekts will be steered using RPS. We don't do this for unconnected UDP tunnels (encap_rcv) under the assumption that the flow table entries will be adjusted during processing of inner packets. Tested using super_netperf UDP_RR with 200 flows. Did not see any noticeable regression in writing to flow table for every packet. Before fix: 76.99% CPU utilization 112/158/230 90/95/99% latencies 1.62776e+06 tps After fix: 76.03% CPU utilization 115/162/239 90/95/99% latencies 1.62257e+06 tps Signed-off-by: Tom Herbert --- net/ipv4/udp.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index f57c0e4..7ba764a 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -1444,6 +1444,8 @@ static int __udp_queue_rcv_skb(struct sock *sk, struct sk_buff *skb) if (inet_sk(sk)->inet_daddr) { sock_rps_save_rxhash(sk, skb); sk_mark_napi_id(sk, skb); + } else { + sock_rps_reset_flow_hash(skb->hash); } rc = sock_queue_rcv_skb(sk, skb);