Patchwork [1/2] ipv4: Invalidate the socket cached route on pmtu events if possible

login
register
mail settings
Submitter Steffen Klassert
Date Jan. 21, 2013, 11:59 a.m.
Message ID <20130121115911.GE30530@secunet.com>
Download mbox | patch
Permalink /patch/214110/
State Accepted
Delegated to: David Miller
Headers show

Comments

Steffen Klassert - Jan. 21, 2013, 11:59 a.m.
The route lookup in ipv4_sk_update_pmtu() might return a route
different from the route we cached at the socket. This is because
standart routes are per cpu, so each cpu has it's own struct rtable.
This means that we do not invalidate the socket cached route if the
NET_RX_SOFTIRQ is not served by the same cpu that the sending socket
uses. As a result, the cached route reused until we disconnect.

With this patch we invalidate the socket cached route if possible.
If the socket is owened by the user, we can't update the cached
route directly. A followup patch will implement socket release
callback functions for datagram sockets to handle this case.

Reported-by: Yurij M. Plotnikov <Yurij.Plotnikov@oktetlabs.ru>
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
---
 net/ipv4/route.c |   42 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 41 insertions(+), 1 deletion(-)
David Miller - Jan. 21, 2013, 7:18 p.m.
From: Steffen Klassert <steffen.klassert@secunet.com>
Date: Mon, 21 Jan 2013 12:59:11 +0100

> The route lookup in ipv4_sk_update_pmtu() might return a route
> different from the route we cached at the socket. This is because
> standart routes are per cpu, so each cpu has it's own struct rtable.
> This means that we do not invalidate the socket cached route if the
> NET_RX_SOFTIRQ is not served by the same cpu that the sending socket
> uses. As a result, the cached route reused until we disconnect.
> 
> With this patch we invalidate the socket cached route if possible.
> If the socket is owened by the user, we can't update the cached
> route directly. A followup patch will implement socket release
> callback functions for datagram sockets to handle this case.
> 
> Reported-by: Yurij M. Plotnikov <Yurij.Plotnikov@oktetlabs.ru>
> Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>

Applied.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Julian Anastasov - Jan. 21, 2013, 9:18 p.m.
Hello,

On Mon, 21 Jan 2013, Steffen Klassert wrote:

> The route lookup in ipv4_sk_update_pmtu() might return a route
> different from the route we cached at the socket. This is because
> standart routes are per cpu, so each cpu has it's own struct rtable.
> This means that we do not invalidate the socket cached route if the
> NET_RX_SOFTIRQ is not served by the same cpu that the sending socket
> uses. As a result, the cached route reused until we disconnect.
> 
> With this patch we invalidate the socket cached route if possible.
> If the socket is owened by the user, we can't update the cached
> route directly. A followup patch will implement socket release
> callback functions for datagram sockets to handle this case.
> 
> Reported-by: Yurij M. Plotnikov <Yurij.Plotnikov@oktetlabs.ru>
> Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
> ---
>  net/ipv4/route.c |   42 +++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 41 insertions(+), 1 deletion(-)
> 
> diff --git a/net/ipv4/route.c b/net/ipv4/route.c
> index 259cbee..132737a 100644
> --- a/net/ipv4/route.c
> +++ b/net/ipv4/route.c
> @@ -965,7 +965,7 @@ void ipv4_update_pmtu(struct sk_buff *skb, struct net *net, u32 mtu,
>  }
>  EXPORT_SYMBOL_GPL(ipv4_update_pmtu);
>  
> -void ipv4_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, u32 mtu)
> +static void __ipv4_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, u32 mtu)
>  {
>  	const struct iphdr *iph = (const struct iphdr *) skb->data;
>  	struct flowi4 fl4;
> @@ -978,6 +978,46 @@ void ipv4_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, u32 mtu)
>  		ip_rt_put(rt);
>  	}
>  }
> +
> +void ipv4_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, u32 mtu)
> +{
> +	const struct iphdr *iph = (const struct iphdr *) skb->data;
> +	struct flowi4 fl4;
> +	struct rtable *rt;
> +	struct dst_entry *dst;
> +
> +	bh_lock_sock(sk);
> +	rt = (struct rtable *) __sk_dst_get(sk);

	I just saw another problem, sorry that
I missed it the first time. Here __sk_dst_get
does not get reference...

> +
> +	if (sock_owned_by_user(sk) || !rt) {
> +		__ipv4_sk_update_pmtu(skb, sk, mtu);
> +		goto out;
> +	}
> +
> +	__build_flow_key(&fl4, sk, iph, 0, 0, 0, 0, 0);
> +
> +	if (!__sk_dst_check(sk, 0)) {
> +		rt = ip_route_output_flow(sock_net(sk), &fl4, sk);
> +		if (IS_ERR(rt))
> +			goto out;

		but here rt->dst comes with reference.

> +	}

	May be here we can use 'else dst_hold(&rt->dst);' ?
It is needed for __sk_dst_set.

> +
> +	__ip_rt_update_pmtu((struct rtable *) rt->dst.path, &fl4, mtu);
> +
> +	dst = dst_check(&rt->dst, 0);
> +	if (!dst) {

		dst_release(&rt->dst);

> +		rt = ip_route_output_flow(sock_net(sk), &fl4, sk);
> +		if (IS_ERR(rt))
> +			goto out;
> +
> +		dst = &rt->dst;

		Remove above line...

> +	}
> +

	and use here __sk_dst_set(sk, &rt->dst) instead:

> +	 __sk_dst_set(sk, dst);

	Another variant is to remember with flag 'new_rt'
that we should call __sk_dst_set, eg. when rt comes from
ip_route_output_flow. By this way we can avoid some of
the dst_hold/dst_release calls if sk_dst_cache is not
changed. IIRC, according to sk_dst_check, dst_check can
not return different dst from the ->check method.

> +
> +out:
> +	bh_unlock_sock(sk);
> +}
>  EXPORT_SYMBOL_GPL(ipv4_sk_update_pmtu);
>  
>  void ipv4_redirect(struct sk_buff *skb, struct net *net,
> -- 
> 1.7.9.5

Regards

--
Julian Anastasov <ja@ssi.bg>
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Steffen Klassert - Jan. 22, 2013, 8:42 a.m.
On Mon, Jan 21, 2013 at 11:18:27PM +0200, Julian Anastasov wrote:
> 
> 	Hello,
> 
> On Mon, 21 Jan 2013, Steffen Klassert wrote:
> 
> > +{
> > +	const struct iphdr *iph = (const struct iphdr *) skb->data;
> > +	struct flowi4 fl4;
> > +	struct rtable *rt;
> > +	struct dst_entry *dst;
> > +
> > +	bh_lock_sock(sk);
> > +	rt = (struct rtable *) __sk_dst_get(sk);
> 
> 	I just saw another problem, sorry that
> I missed it the first time. Here __sk_dst_get
> does not get reference...
> 

Somehow I thought that I can reuse the the refcount
from the socket, but __sk_dst_set() will release it
when we continue to use the same route.

> > +
> > +	if (sock_owned_by_user(sk) || !rt) {
> > +		__ipv4_sk_update_pmtu(skb, sk, mtu);
> > +		goto out;
> > +	}
> > +
> > +	__build_flow_key(&fl4, sk, iph, 0, 0, 0, 0, 0);
> > +
> > +	if (!__sk_dst_check(sk, 0)) {
> > +		rt = ip_route_output_flow(sock_net(sk), &fl4, sk);
> > +		if (IS_ERR(rt))
> > +			goto out;
> 
> 		but here rt->dst comes with reference.
> 
> > +	}
> 
> 	May be here we can use 'else dst_hold(&rt->dst);' ?
> It is needed for __sk_dst_set.
> 
> > +
> > +	__ip_rt_update_pmtu((struct rtable *) rt->dst.path, &fl4, mtu);
> > +
> > +	dst = dst_check(&rt->dst, 0);
> > +	if (!dst) {
> 
> 		dst_release(&rt->dst);
> 
> > +		rt = ip_route_output_flow(sock_net(sk), &fl4, sk);
> > +		if (IS_ERR(rt))
> > +			goto out;
> > +
> > +		dst = &rt->dst;
> 
> 		Remove above line...
> 
> > +	}
> > +
> 
> 	and use here __sk_dst_set(sk, &rt->dst) instead:
> 
> > +	 __sk_dst_set(sk, dst);
> 
> 	Another variant is to remember with flag 'new_rt'
> that we should call __sk_dst_set, eg. when rt comes from
> ip_route_output_flow. By this way we can avoid some of
> the dst_hold/dst_release calls if sk_dst_cache is not
> changed. IIRC, according to sk_dst_check, dst_check can
> not return different dst from the ->check method.
> 

With this variant, we also need to check the new_rt flag
before we call the dst_release() function you mentioned
above, right?

I think we should avoid refcounts whenever we can,
so I prefer this variant. I'll do a patch.

Thanks!

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Julian Anastasov - Jan. 22, 2013, 9:04 a.m.
Hello,

On Tue, 22 Jan 2013, Steffen Klassert wrote:

> > 	Another variant is to remember with flag 'new_rt'
> > that we should call __sk_dst_set, eg. when rt comes from
> > ip_route_output_flow. By this way we can avoid some of
> > the dst_hold/dst_release calls if sk_dst_cache is not
> > changed. IIRC, according to sk_dst_check, dst_check can
> > not return different dst from the ->check method.
> > 
> 
> With this variant, we also need to check the new_rt flag
> before we call the dst_release() function you mentioned
> above, right?

	Yes, also before the dst_release we add
after the first ip_route_output_flow.

> I think we should avoid refcounts whenever we can,
> so I prefer this variant. I'll do a patch.

	Yes, the flag is preferred because we should
keep sk_dst_cache until we decide to replace it with
__sk_dst_set.

Regards

--
Julian Anastasov <ja@ssi.bg>
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index 259cbee..132737a 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -965,7 +965,7 @@  void ipv4_update_pmtu(struct sk_buff *skb, struct net *net, u32 mtu,
 }
 EXPORT_SYMBOL_GPL(ipv4_update_pmtu);
 
-void ipv4_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, u32 mtu)
+static void __ipv4_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, u32 mtu)
 {
 	const struct iphdr *iph = (const struct iphdr *) skb->data;
 	struct flowi4 fl4;
@@ -978,6 +978,46 @@  void ipv4_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, u32 mtu)
 		ip_rt_put(rt);
 	}
 }
+
+void ipv4_sk_update_pmtu(struct sk_buff *skb, struct sock *sk, u32 mtu)
+{
+	const struct iphdr *iph = (const struct iphdr *) skb->data;
+	struct flowi4 fl4;
+	struct rtable *rt;
+	struct dst_entry *dst;
+
+	bh_lock_sock(sk);
+	rt = (struct rtable *) __sk_dst_get(sk);
+
+	if (sock_owned_by_user(sk) || !rt) {
+		__ipv4_sk_update_pmtu(skb, sk, mtu);
+		goto out;
+	}
+
+	__build_flow_key(&fl4, sk, iph, 0, 0, 0, 0, 0);
+
+	if (!__sk_dst_check(sk, 0)) {
+		rt = ip_route_output_flow(sock_net(sk), &fl4, sk);
+		if (IS_ERR(rt))
+			goto out;
+	}
+
+	__ip_rt_update_pmtu((struct rtable *) rt->dst.path, &fl4, mtu);
+
+	dst = dst_check(&rt->dst, 0);
+	if (!dst) {
+		rt = ip_route_output_flow(sock_net(sk), &fl4, sk);
+		if (IS_ERR(rt))
+			goto out;
+
+		dst = &rt->dst;
+	}
+
+	 __sk_dst_set(sk, dst);
+
+out:
+	bh_unlock_sock(sk);
+}
 EXPORT_SYMBOL_GPL(ipv4_sk_update_pmtu);
 
 void ipv4_redirect(struct sk_buff *skb, struct net *net,