From patchwork Wed May 16 01:26:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nambiar, Amritha" X-Patchwork-Id: 914109 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40lxhJ67s8z9s1p for ; Wed, 16 May 2018 11:30:24 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752234AbeEPBaT (ORCPT ); Tue, 15 May 2018 21:30:19 -0400 Received: from mga11.intel.com ([192.55.52.93]:22589 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751402AbeEPBaS (ORCPT ); Tue, 15 May 2018 21:30:18 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 May 2018 18:30:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,404,1520924400"; d="scan'208";a="39650135" Received: from anamdev.jf.intel.com ([10.166.28.109]) by fmsmga007.fm.intel.com with ESMTP; 15 May 2018 18:30:17 -0700 Subject: [net-next PATCH v2 2/4] net: Enable Tx queue selection based on Rx queues From: Amritha Nambiar To: netdev@vger.kernel.org, davem@davemloft.net Cc: alexander.h.duyck@intel.com, amritha.nambiar@intel.com, sridhar.samudrala@intel.com, edumazet@google.com, hannes@stressinduktion.org, tom@herbertland.com Date: Tue, 15 May 2018 18:26:49 -0700 Message-ID: <152643400925.4991.5029989601625953592.stgit@anamdev.jf.intel.com> In-Reply-To: <152643356116.4991.7215767041139726872.stgit@anamdev.jf.intel.com> References: <152643356116.4991.7215767041139726872.stgit@anamdev.jf.intel.com> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This patch adds support to pick Tx queue based on the Rx queue map configuration set by the admin through the sysfs attribute for each Tx queue. If the user configuration for receive queue map does not apply, then the Tx queue selection falls back to CPU map based selection and finally to hashing. Signed-off-by: Amritha Nambiar Signed-off-by: Sridhar Samudrala --- include/net/sock.h | 18 ++++++++++++++++++ net/core/dev.c | 36 +++++++++++++++++++++++++++++------- net/core/sock.c | 5 +++++ net/ipv4/tcp_input.c | 7 +++++++ net/ipv4/tcp_ipv4.c | 1 + net/ipv4/tcp_minisocks.c | 1 + 6 files changed, 61 insertions(+), 7 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index 4f7c584..0613f63 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -139,6 +139,8 @@ typedef __u64 __bitwise __addrpair; * @skc_node: main hash linkage for various protocol lookup tables * @skc_nulls_node: main hash linkage for TCP/UDP/UDP-Lite protocol * @skc_tx_queue_mapping: tx queue number for this connection + * @skc_rx_queue_mapping: rx queue number for this connection + * @skc_rx_ifindex: rx ifindex for this connection * @skc_flags: place holder for sk_flags * %SO_LINGER (l_onoff), %SO_BROADCAST, %SO_KEEPALIVE, * %SO_OOBINLINE settings, %SO_TIMESTAMPING settings @@ -215,6 +217,10 @@ struct sock_common { struct hlist_nulls_node skc_nulls_node; }; int skc_tx_queue_mapping; +#ifdef CONFIG_XPS + int skc_rx_queue_mapping; + int skc_rx_ifindex; +#endif union { int skc_incoming_cpu; u32 skc_rcv_wnd; @@ -326,6 +332,10 @@ struct sock { #define sk_nulls_node __sk_common.skc_nulls_node #define sk_refcnt __sk_common.skc_refcnt #define sk_tx_queue_mapping __sk_common.skc_tx_queue_mapping +#ifdef CONFIG_XPS +#define sk_rx_queue_mapping __sk_common.skc_rx_queue_mapping +#define sk_rx_ifindex __sk_common.skc_rx_ifindex +#endif #define sk_dontcopy_begin __sk_common.skc_dontcopy_begin #define sk_dontcopy_end __sk_common.skc_dontcopy_end @@ -1696,6 +1706,14 @@ static inline int sk_tx_queue_get(const struct sock *sk) return sk ? sk->sk_tx_queue_mapping : -1; } +static inline void sk_mark_rx_queue(struct sock *sk, struct sk_buff *skb) +{ +#ifdef CONFIG_XPS + sk->sk_rx_ifindex = skb->skb_iif; + sk->sk_rx_queue_mapping = skb_get_rx_queue(skb); +#endif +} + static inline void sk_set_socket(struct sock *sk, struct socket *sock) { sk_tx_queue_clear(sk); diff --git a/net/core/dev.c b/net/core/dev.c index 7e5dfdb..4030368 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -3458,18 +3458,14 @@ sch_handle_egress(struct sk_buff *skb, int *ret, struct net_device *dev) } #endif /* CONFIG_NET_EGRESS */ -static inline int get_xps_queue(struct net_device *dev, struct sk_buff *skb) -{ #ifdef CONFIG_XPS - struct xps_dev_maps *dev_maps; +static int __get_xps_queue_idx(struct net_device *dev, struct sk_buff *skb, + struct xps_dev_maps *dev_maps, unsigned int tci) +{ struct xps_map *map; int queue_index = -1; - rcu_read_lock(); - dev_maps = rcu_dereference(dev->xps_maps[XPS_MAP_CPUS]); if (dev_maps) { - unsigned int tci = skb->sender_cpu - 1; - if (dev->num_tc) { tci *= dev->num_tc; tci += netdev_get_prio_tc_map(dev, skb->priority); @@ -3486,6 +3482,32 @@ static inline int get_xps_queue(struct net_device *dev, struct sk_buff *skb) queue_index = -1; } } + return queue_index; +} +#endif + +static int get_xps_queue(struct net_device *dev, struct sk_buff *skb) +{ +#ifdef CONFIG_XPS + enum xps_map_type i = XPS_MAP_RXQS; + struct xps_dev_maps *dev_maps; + struct sock *sk = skb->sk; + int queue_index = -1; + unsigned int tci = 0; + + if (sk && sk->sk_rx_queue_mapping <= dev->real_num_rx_queues && + dev->ifindex == sk->sk_rx_ifindex) + tci = sk->sk_rx_queue_mapping; + + rcu_read_lock(); + while (queue_index < 0 && i < __XPS_MAP_MAX) { + if (i == XPS_MAP_CPUS) + tci = skb->sender_cpu - 1; + dev_maps = rcu_dereference(dev->xps_maps[i]); + queue_index = __get_xps_queue_idx(dev, skb, dev_maps, tci); + i++; + } + rcu_read_unlock(); return queue_index; diff --git a/net/core/sock.c b/net/core/sock.c index 042cfc6..73d7fa8 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -2824,6 +2824,11 @@ void sock_init_data(struct socket *sock, struct sock *sk) sk->sk_pacing_rate = ~0U; sk->sk_pacing_shift = 10; sk->sk_incoming_cpu = -1; + +#ifdef CONFIG_XPS + sk->sk_rx_ifindex = -1; + sk->sk_rx_queue_mapping = -1; +#endif /* * Before updating sk_refcnt, we must commit prior changes to memory * (Documentation/RCU/rculist_nulls.txt for details) diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index b188e0d..d33911c 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -78,6 +78,7 @@ #include #include #include +#include int sysctl_tcp_max_orphans __read_mostly = NR_FILE; @@ -5559,6 +5560,11 @@ void tcp_finish_connect(struct sock *sk, struct sk_buff *skb) __tcp_fast_path_on(tp, tp->snd_wnd); else tp->pred_flags = 0; + + if (skb) { + sk_mark_napi_id(sk, skb); + sk_mark_rx_queue(sk, skb); + } } static bool tcp_rcv_fastopen_synack(struct sock *sk, struct sk_buff *synack, @@ -6371,6 +6377,7 @@ int tcp_conn_request(struct request_sock_ops *rsk_ops, tcp_rsk(req)->snt_isn = isn; tcp_rsk(req)->txhash = net_tx_rndhash(); tcp_openreq_init_rwin(req, sk, dst); + sk_mark_rx_queue(req_to_sk(req), skb); if (!want_cookie) { tcp_reqsk_record_syn(sk, req, skb); fastopen_sk = tcp_try_fastopen(sk, skb, req, &foc, dst); diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index caf23de..abdf02e 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -1479,6 +1479,7 @@ int tcp_v4_do_rcv(struct sock *sk, struct sk_buff *skb) sock_rps_save_rxhash(sk, skb); sk_mark_napi_id(sk, skb); + sk_mark_rx_queue(sk, skb); if (dst) { if (inet_sk(sk)->rx_dst_ifindex != skb->skb_iif || !dst->ops->check(dst, 0)) { diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c index f867658..4939c28 100644 --- a/net/ipv4/tcp_minisocks.c +++ b/net/ipv4/tcp_minisocks.c @@ -836,6 +836,7 @@ int tcp_child_process(struct sock *parent, struct sock *child, /* record NAPI ID of child */ sk_mark_napi_id(child, skb); + sk_mark_rx_queue(child, skb); tcp_segs_in(tcp_sk(child), skb); if (!sock_owned_by_user(child)) {