From patchwork Mon Nov 18 11:01:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Abeni X-Patchwork-Id: 1196684 Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="fBPQkH8A"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 47GmJG546rz9s4Y for ; Mon, 18 Nov 2019 22:02:38 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726898AbfKRLCi (ORCPT ); Mon, 18 Nov 2019 06:02:38 -0500 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:32715 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727239AbfKRLCf (ORCPT ); Mon, 18 Nov 2019 06:02:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574074954; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/CPy6WDs/EWc+Bu0cYFEz/Gm3YKQJa2lR6NOi7rwnOo=; b=fBPQkH8Aj5c5DwoQMDGS70SvL23qn8NMmT4SeV/mzrWKc0K58rcvgULbk6Lmcr0ccTSGoS 5WfqQlyoshHQrwN9+9Y/ZKzvDdZe3S1hxlOpkgbwYwlj63fyyACY9cCSDCAbFjkcFTBOn9 f2Bo779DbJ/eIJvhuwQCfJ8UIyhIv+w= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-254-MjdWET-6PgqAalU8f84WYg-1; Mon, 18 Nov 2019 06:02:31 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E855A1852E2F; Mon, 18 Nov 2019 11:02:29 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-117-52.ams2.redhat.com [10.36.117.52]) by smtp.corp.redhat.com (Postfix) with ESMTP id C79B44F9B6; Mon, 18 Nov 2019 11:02:28 +0000 (UTC) From: Paolo Abeni To: netdev@vger.kernel.org Cc: "David S. Miller" , Willem de Bruijn , Edward Cree Subject: [PATCH net-next v2 1/2] ipv6: introduce and uses route look hints for list input Date: Mon, 18 Nov 2019 12:01:29 +0100 Message-Id: <643f2b258e275e915fa96ef0c635f9c5ff804c9d.1574071944.git.pabeni@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: MjdWET-6PgqAalU8f84WYg-1 X-Mimecast-Spam-Score: 0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org When doing RX batch packet processing, we currently always repeat the route lookup for each ingress packet. If policy routing is configured, and IPV6_SUBTREES is disabled at build time, we know that packets with the same destination address will use the same dst. This change tries to avoid per packet route lookup caching the destination address of the latest successful lookup, and reusing it for the next packet when the above conditions are in place. Ingress traffic for most servers should fit. The measured performance delta under UDP flood vs a recvmmsg receiver is as follow: vanilla patched delta Kpps Kpps % 1431 1664 +14 In the worst-case scenario - each packet has a different destination address - the performance delta is within noise range. v1 -> v2: - fix build issue with !CONFIG_IPV6_MULTIPLE_TABLES - fix potential race when fib6_has_custom_rules is set while processing a packet batch Signed-off-by: Paolo Abeni --- net/ipv6/ip6_input.c | 40 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 36 insertions(+), 4 deletions(-) diff --git a/net/ipv6/ip6_input.c b/net/ipv6/ip6_input.c index ef7f707d9ae3..f559ad6b09ef 100644 --- a/net/ipv6/ip6_input.c +++ b/net/ipv6/ip6_input.c @@ -44,10 +44,16 @@ #include #include +struct ip6_route_input_hint { + unsigned long refdst; + struct in6_addr daddr; +}; + INDIRECT_CALLABLE_DECLARE(void udp_v6_early_demux(struct sk_buff *)); INDIRECT_CALLABLE_DECLARE(void tcp_v6_early_demux(struct sk_buff *)); static void ip6_rcv_finish_core(struct net *net, struct sock *sk, - struct sk_buff *skb) + struct sk_buff *skb, + struct ip6_route_input_hint *hint) { void (*edemux)(struct sk_buff *skb); @@ -59,7 +65,13 @@ static void ip6_rcv_finish_core(struct net *net, struct sock *sk, INDIRECT_CALL_2(edemux, tcp_v6_early_demux, udp_v6_early_demux, skb); } - if (!skb_valid_dst(skb)) + + if (skb_valid_dst(skb)) + return; + + if (hint && ipv6_addr_equal(&hint->daddr, &ipv6_hdr(skb)->daddr)) + __skb_dst_copy(skb, hint->refdst); + else ip6_route_input(skb); } @@ -71,7 +83,7 @@ int ip6_rcv_finish(struct net *net, struct sock *sk, struct sk_buff *skb) skb = l3mdev_ip6_rcv(skb); if (!skb) return NET_RX_SUCCESS; - ip6_rcv_finish_core(net, sk, skb); + ip6_rcv_finish_core(net, sk, skb, NULL); return dst_input(skb); } @@ -86,9 +98,20 @@ static void ip6_sublist_rcv_finish(struct list_head *head) } } +static bool ip6_can_cache_route_hint(struct net *net) +{ + return !IS_ENABLED(IPV6_SUBTREES) && +#ifdef CONFIG_IPV6_MULTIPLE_TABLES + !net->ipv6.fib6_has_custom_rules; +#else + 1; +#endif +} + static void ip6_list_rcv_finish(struct net *net, struct sock *sk, struct list_head *head) { + struct ip6_route_input_hint _hint, *hint = NULL; struct dst_entry *curr_dst = NULL; struct sk_buff *skb, *next; struct list_head sublist; @@ -104,9 +127,18 @@ static void ip6_list_rcv_finish(struct net *net, struct sock *sk, skb = l3mdev_ip6_rcv(skb); if (!skb) continue; - ip6_rcv_finish_core(net, sk, skb); + ip6_rcv_finish_core(net, sk, skb, hint); dst = skb_dst(skb); if (curr_dst != dst) { + if (ip6_can_cache_route_hint(net)) { + _hint.refdst = skb->_skb_refdst; + memcpy(&_hint.daddr, &ipv6_hdr(skb)->daddr, + sizeof(_hint.daddr)); + hint = &_hint; + } else { + hint = NULL; + } + /* dispatch old sublist */ if (!list_empty(&sublist)) ip6_sublist_rcv_finish(&sublist);