[RFC] xfrm: fix pmtu discovery (kill xfrm6_update_pmtu)

On Tue, Jan 29, 2013 at 03:43:25PM +0100, Jiri Bohac wrote:
> Hi,
> 
> there is a problem in the xfrm PMTU discovery. This happens with
> IPv6, I'm not sure if the same applies to IPv4:
> 
> Let's have e.g. an ESP transport-mode policy mode between
> two endpoints: A and B. The ESP-encapsulated packets are sent
> over a router R:
> 	A <----> R <----> B
> and the R <----> B link has a small MTU of 1452.
> 
> R sends an ICMPV6_PKT_TOOBIG to A with MTU==1452.
> This is what then happens on A:
> 
> icmpv6_rcv() -> icmpv6_notify() -> esp6_err() -> ip6_update_pmtu()
> 
> This looks up the non-xfrm dst entry to host B (dst_B) and
> decreases its MTU to 1452
> 
> Next time a large TCP segment (len=1452 bytes including TCP/IP
> headers in this example) from A to B is created:
> 
> tcp_sendmsg() -> ... -> inet6_csk_xmit() -> ... -> xfrm_bundle_ok()
> 
> dst->child and xdst->route now point to the dst_B with MTU==1452
> xdst->route_mtu_cached and xdst->child_mtu_cached are both 1500,
> so the MTU of the xfrm bundle's dst (dst_B_xfrm) is decreased to 
> xfrm_state_mtu(dst_B_xfrm, 1452)==1414.
> 
> When the TCP segment reaches ip6_xmit:
> 	skb->len > dst_mtu(dst_B_xfrm)
> 	1452 > 1414
> This generates an ICMPV6_PKT_TOOBIG to self with MTU==1414.
> This is intended to reach the protocol error handler (decrease
> the MSS in the TCP case):

I think the above is the problem, we should not send packet to
big messages to ourselves. The reduced mtu is because of some
local reason (e.g. IPsec), it is not learned and therefore we
should not update the pmtu value.

You could try the patch below. I'm travelling this week, so I
can't do tests myself before monday.

Subject: [PATCH] ipv6: Don't send packet to big messages to self

Calling icmpv6_send() on a local message size error leads to an
incorrect update of the path mtu in the case when IPsec is used.
So use ipv6_local_error() instead to notify the socket about the
error.

Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
---
 net/ipv6/ip6_output.c |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

On Fri, Feb 01, 2013 at 09:45:15AM +0100, Steffen Klassert wrote:
> On Tue, Jan 29, 2013 at 03:43:25PM +0100, Jiri Bohac wrote:
> > When the TCP segment reaches ip6_xmit:
> > 	skb->len > dst_mtu(dst_B_xfrm)
> > 	1452 > 1414
> > This generates an ICMPV6_PKT_TOOBIG to self with MTU==1414.
> > This is intended to reach the protocol error handler (decrease
> > the MSS in the TCP case):
> 
> I think the above is the problem, we should not send packet to
> big messages to ourselves. The reduced mtu is because of some
> local reason (e.g. IPsec), it is not learned and therefore we
> should not update the pmtu value.
> 
> You could try the patch below. I'm travelling this week, so I
> can't do tests myself before monday.
> 
> Subject: [PATCH] ipv6: Don't send packet to big messages to self
> 
> Calling icmpv6_send() on a local message size error leads to an
> incorrect update of the path mtu in the case when IPsec is used.
> So use ipv6_local_error() instead to notify the socket about the
> error.

It fixes the problem I was trying to fix.
I tested the patch with the PMTU and IPsec parts of the ipv6ready
testsuite and it passed.

I wonder if this is going to do everything that should be done
e.g. in the TCPv6 case.  Now, tcp_v6_err() calls
tcp_v6_mtu_reduced() which syncs the MSS and forces a retransmit.

Looking at the code, I don't think this is going to happen with
this patch, so the MSS update and retransmit will only be done
when the respective TCP timer expires.

So perhaps the individual protocols will need to handle the new
error reporting?


And out of curiosity, do you know the reason why
xfrm[46]_update_pmtu changes the non-xfrm route's MTU?
Is this really intended?

Thanks,

On Fri, Feb 01, 2013 at 06:25:29PM +0100, Jiri Bohac wrote:
> On Fri, Feb 01, 2013 at 09:45:15AM +0100, Steffen Klassert wrote:
> > 
> > Subject: [PATCH] ipv6: Don't send packet to big messages to self
> > 
> > Calling icmpv6_send() on a local message size error leads to an
> > incorrect update of the path mtu in the case when IPsec is used.
> > So use ipv6_local_error() instead to notify the socket about the
> > error.
> 
> It fixes the problem I was trying to fix.
> I tested the patch with the PMTU and IPsec parts of the ipv6ready
> testsuite and it passed.
> 
> I wonder if this is going to do everything that should be done
> e.g. in the TCPv6 case.  Now, tcp_v6_err() calls
> tcp_v6_mtu_reduced() which syncs the MSS and forces a retransmit.
> 
> Looking at the code, I don't think this is going to happen with
> this patch, so the MSS update and retransmit will only be done
> when the respective TCP timer expires.
> 
> So perhaps the individual protocols will need to handle the new
> error reporting?
> 

Well, we might loose the fast path retransmit. We could think
about implementing a fast path for such situations, but that's
another issue. Let's fix the bug first, I'll submit the patch
to fix it.

> 
> And out of curiosity, do you know the reason why
> xfrm[46]_update_pmtu changes the non-xfrm route's MTU?
> Is this really intended?
> 

The pmtu is a learned measure of the route, it depends on the
external environment, not on the question whether we add local
overhead. So I think it was intended to change always the
pmtu of the original route, even if we use IPsec.


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html