[RFC] net: save RX queue number in sock for dev_pick_tx() use

For the packets sent out from a local server socket, we can use the queue
from which the packets from the client socket are received.

It may help on a TCP or UDP server. Because I don't have a multiqueue NIC,
I don't even test it.

Signed-off-by: Changli Gao <xiaosuo@gmail.com>
---
 include/net/sock.h  |   18 ++++++++++++++++++
 net/core/dev.c      |   25 ++++++++++++++++++-------
 net/ipv4/tcp_ipv4.c |    5 ++++-
 net/ipv4/udp.c      |    4 +++-
 4 files changed, 43 insertions(+), 9 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Le mardi 24 août 2010 à 17:01 +0800, Changli Gao a écrit :
> For the packets sent out from a local server socket, we can use the queue
> from which the packets from the client socket are received.
> 
> It may help on a TCP or UDP server. Because I don't have a multiqueue NIC,
> I don't even test it.
> 

It may help ? or it may not...

> Signed-off-by: Changli Gao <xiaosuo@gmail.com>
> ---
>  include/net/sock.h  |   18 ++++++++++++++++++
>  net/core/dev.c      |   25 ++++++++++++++++++-------
>  net/ipv4/tcp_ipv4.c |    5 ++++-
>  net/ipv4/udp.c      |    4 +++-
>  4 files changed, 43 insertions(+), 9 deletions(-)
> diff --git a/include/net/sock.h b/include/net/sock.h
> index 100e43b..4e5f2f4 100644
> --- a/include/net/sock.h
> +++ b/include/net/sock.h
> @@ -1231,6 +1231,24 @@ static inline int sk_tx_queue_get(const struct sock *sk)
>  	return sk ? sk->sk_tx_queue_mapping : -1;
>  }
>  
> +static inline void sk_rx_queue_save(struct sock *sk, struct sk_buff *skb)
> +{
> +	struct dst_entry *dst;
> +	int rxqueue;
> +
> +	if (!skb_rx_queue_recorded(skb))
> +		return;
> +	rcu_read_lock();
> +	dst = rcu_dereference_check(sk->sk_dst_cache, 1);

rcu_dereference(sk->sk_dst_cache) is fine here.

> +	if (dst && !dst->dev->netdev_ops->ndo_select_queue &&
> +	    dst->dev == skb->dev) {
> +		rxqueue = skb_get_rx_queue(skb);
> +		if (rxqueue != sk_tx_queue_get(sk))
> +			sk_tx_queue_set(sk, rxqueue);
> +	}
> +	rcu_read_unlock();
> +}
> +

ipv6 not handled

Anyway, should we store a queue number, a rxhash, a cpu number, all
values ?

Tom Herbert last patch (XPS) is about selecting a txqueue given a cpu
number at sending time. In its use, no need to add logic at receive time
to remember rxqueue. But logic is wide (for a given NIC)

I do think we should really clarify the needs before writing hundred of
lines of code that we have to maintain for next decade...

For example, storing the cpu number of the cpu that received a UDP/TCP
frame would probably be nice, and using it as a key for xmit path queue
selection would probably be faster...

Probably we want a selectable way... A multi threaded UDP application,
receiving frames on a single socket, might need to select a single
txqueue (socket property), or multiple queues (based on packet content,
or sender cpu, or last receive cpu number, ...)



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Tue, 2010-08-24 at 17:01 +0800, Changli Gao wrote:
> For the packets sent out from a local server socket, we can use the queue
> from which the packets from the client socket are received.
> 
> It may help on a TCP or UDP server. Because I don't have a multiqueue NIC,
> I don't even test it.
[...]

There are some fairly cheap 1G multiqueue NICs around now (and even
several of the ARM embedded Ethernet controllers support multiqueue).

Could you compare this with my patch in
<http://thread.gmane.org/gmane.linux.network/158477>?

Ben.