diff mbox

ipsec doesn't route TCP with 4.11 kernel

Message ID 20170428071337.GG2649@secunet.com
State RFC, archived
Delegated to: David Miller
Headers show

Commit Message

Steffen Klassert April 28, 2017, 7:13 a.m. UTC
On Thu, Apr 27, 2017 at 06:13:38PM -0400, Don Bowman wrote:
> On 27 April 2017 at 04:42, Steffen Klassert <steffen.klassert@secunet.com>
> wrote:
> > On Wed, Apr 26, 2017 at 10:01:34PM -0700, Cong Wang wrote:
> >> (Cc'ing netdev and IPSec maintainers)
> >>
> >> On Tue, Apr 25, 2017 at 6:08 PM, Don Bowman <db@donbowman.ca> wrote:
> 
> for 'esp' question, i have ' esp = aes256-sha256-modp1536!' is that what
> you mean?
> its nat-aware tunnel [from my desktop pc to my office]
> 
> root@office:~# ip -s x s
> src 172.16.0.8 dst 64.7.137.180
>         proto esp spi 0x0d588366(223904614) reqid 1(0x00000001) mode tunnel
>         replay-window 0 seq 0x00000000 flag af-unspec (0x00100000)
>         auth-trunc hmac(sha256)
> 0x046cafdf19c5d78d1c29165d96a0b9fce1c500029d77be0fe956dce1bf80a86a (256
> bits) 128
>         enc cbc(aes)
> 0x79ff2fbc2178eb468de6ff16612f0603b514a1d1d5f375c67222294463ec7c62 (256
> bits)
>         encap type espinudp sport 4500 dport 4500 addr 0.0.0.0

Ok, this is espinudp. This information was important.

> 
> I'm not sure what you mean the receiving interface, you mean the outer, the
> native interface?
> listening on eno1, link-type EN10MB (Ethernet), capture size 262144 bytes
> 18:11:32.061501 IP 172.16.0.8.3416 > 64.7.137.180.33638:
> truncated-udplength 0
> 18:11:32.788091 IP 64.7.137.180.4500 > 172.16.0.8.4500: NONESP-encap:
> isakmp: child_sa  inf2
> 18:11:32.788354 IP 172.16.0.8.4500 > 64.7.137.180.4500: NONESP-encap:
> isakmp: child_sa  inf2[IR]
> 18:11:33.066830 IP 172.16.0.8.3416 > 64.7.137.180.33638:
> truncated-udplength 0
> 18:11:35.082839 IP 172.16.0.8.3416 > 64.7.137.180.33638:
> truncated-udplength 0

This is not a GRO issue as I thought, the TX side is already broken.

Could you please try the patch below?

Subject: [PATCH] esp4: Fix udpencap for local TCP packets.

Locally generated TCP packets are usually cloned, so we
do skb_cow_data() on this packets. After that we need to
reload the pointer to the esp header. On udpencap this
header has an offset to skb_transport_header, so take this
offset into account.

Fixes: commit cac2661c53f ("esp4: Avoid skb_cow_data whenever possible")
Signed-off-by: Steffen Klassert <steffen.klassert@secunet.com>
---
 net/ipv4/esp4.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

Comments

Eric Dumazet April 28, 2017, 4:46 p.m. UTC | #1
On Fri, 2017-04-28 at 09:13 +0200, Steffen Klassert wrote:
>          encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
> 
> Ok, this is espinudp. This information was important.

> This is not a GRO issue as I thought, the TX side is already broken.
> 
> Could you please try the patch below?
> 
> Subject: [PATCH] esp4: Fix udpencap for local TCP packets.
> 
> Locally generated TCP packets are usually cloned, so we
> do skb_cow_data() on this packets. After that we need to
> reload the pointer to the esp header. On udpencap this
> header has an offset to skb_transport_header, so take this
> offset into account.


It looks like locally generated TCP packets could avoid the
skb_cow_data(), if you were using skb_header_cloned() instead of
skb_cloned()  ?
Don Bowman April 30, 2017, 12:39 a.m. UTC | #2
On 28 April 2017 at 03:13, Steffen Klassert
<steffen.klassert@secunet.com> wrote:
> On Thu, Apr 27, 2017 at 06:13:38PM -0400, Don Bowman wrote:
>> On 27 April 2017 at 04:42, Steffen Klassert <steffen.klassert@secunet.com>
>> wrote:
>> > On Wed, Apr 26, 2017 at 10:01:34PM -0700, Cong Wang wrote:
>> >> (Cc'ing netdev and IPSec maintainers)
>> >>
>> >> On Tue, Apr 25, 2017 at 6:08 PM, Don Bowman <db@donbowman.ca> wrote:
>>

<snip>

confirmed, with this patch in place that the tcp functions properly.
Steffen Klassert May 3, 2017, 8:14 a.m. UTC | #3
On Sat, Apr 29, 2017 at 08:39:34PM -0400, Don Bowman wrote:
> On 28 April 2017 at 03:13, Steffen Klassert
> <steffen.klassert@secunet.com> wrote:
> > On Thu, Apr 27, 2017 at 06:13:38PM -0400, Don Bowman wrote:
> >> On 27 April 2017 at 04:42, Steffen Klassert <steffen.klassert@secunet.com>
> >> wrote:
> >> > On Wed, Apr 26, 2017 at 10:01:34PM -0700, Cong Wang wrote:
> >> >> (Cc'ing netdev and IPSec maintainers)
> >> >>
> >> >> On Tue, Apr 25, 2017 at 6:08 PM, Don Bowman <db@donbowman.ca> wrote:
> >>
> 
> <snip>
> 
> confirmed, with this patch in place that the tcp functions properly.

Thanks for testing!

I'll make sure to get this fix into the mainline soon.
Steffen Klassert May 3, 2017, 8:21 a.m. UTC | #4
On Fri, Apr 28, 2017 at 09:46:42AM -0700, Eric Dumazet wrote:
> On Fri, 2017-04-28 at 09:13 +0200, Steffen Klassert wrote:
> >          encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
> > 
> > Ok, this is espinudp. This information was important.
> 
> > This is not a GRO issue as I thought, the TX side is already broken.
> > 
> > Could you please try the patch below?
> > 
> > Subject: [PATCH] esp4: Fix udpencap for local TCP packets.
> > 
> > Locally generated TCP packets are usually cloned, so we
> > do skb_cow_data() on this packets. After that we need to
> > reload the pointer to the esp header. On udpencap this
> > header has an offset to skb_transport_header, so take this
> > offset into account.
> 
> 
> It looks like locally generated TCP packets could avoid the
> skb_cow_data(), if you were using skb_header_cloned() instead of
> skb_cloned()  ?

Yes, should be possible in the codepath where we do crypto
with separate src and dst buffers. Would require some
rearrangements to make sure we don't do inplace crypto
in this case.

Thanks for the hint!
Don Bowman May 16, 2017, 7:05 p.m. UTC | #5
On 3 May 2017 at 04:14, Steffen Klassert <steffen.klassert@secunet.com> wrote:
> On Sat, Apr 29, 2017 at 08:39:34PM -0400, Don Bowman wrote:
>> On 28 April 2017 at 03:13, Steffen Klassert
>> <steffen.klassert@secunet.com> wrote:
>> > On Thu, Apr 27, 2017 at 06:13:38PM -0400, Don Bowman wrote:
>> >> On 27 April 2017 at 04:42, Steffen Klassert <steffen.klassert@secunet.com>
>> >> wrote:
>> >> > On Wed, Apr 26, 2017 at 10:01:34PM -0700, Cong Wang wrote:
>> >> >> (Cc'ing netdev and IPSec maintainers)
>> >> >>
>> >> >> On Tue, Apr 25, 2017 at 6:08 PM, Don Bowman <db@donbowman.ca> wrote:
>> >>
>>
>> <snip>
>>
>> confirmed, with this patch in place that the tcp functions properly.
>
> Thanks for testing!
>
> I'll make sure to get this fix into the mainline soon.

Thanks. Let me know if there is any more assistance I can provide.
I've been running the patch for 2 weeks now on 3 machines.
Steffen Klassert May 19, 2017, 10:03 a.m. UTC | #6
On Tue, May 16, 2017 at 03:05:40PM -0400, Don Bowman wrote:
> On 3 May 2017 at 04:14, Steffen Klassert <steffen.klassert@secunet.com> wrote:
> > On Sat, Apr 29, 2017 at 08:39:34PM -0400, Don Bowman wrote:
> >> On 28 April 2017 at 03:13, Steffen Klassert
> >> <steffen.klassert@secunet.com> wrote:
> >> > On Thu, Apr 27, 2017 at 06:13:38PM -0400, Don Bowman wrote:
> >> >> On 27 April 2017 at 04:42, Steffen Klassert <steffen.klassert@secunet.com>
> >> >> wrote:
> >> >> > On Wed, Apr 26, 2017 at 10:01:34PM -0700, Cong Wang wrote:
> >> >> >> (Cc'ing netdev and IPSec maintainers)
> >> >> >>
> >> >> >> On Tue, Apr 25, 2017 at 6:08 PM, Don Bowman <db@donbowman.ca> wrote:
> >> >>
> >>
> >> <snip>
> >>
> >> confirmed, with this patch in place that the tcp functions properly.
> >
> > Thanks for testing!
> >
> > I'll make sure to get this fix into the mainline soon.
> 
> Thanks. Let me know if there is any more assistance I can provide.
> I've been running the patch for 2 weeks now on 3 machines.

Thanks for testing!
I plan to push it upstream at the beginning of te next week.
diff mbox

Patch

diff --git a/net/ipv4/esp4.c b/net/ipv4/esp4.c
index b1e2444..ab71fbb 100644
--- a/net/ipv4/esp4.c
+++ b/net/ipv4/esp4.c
@@ -223,6 +223,7 @@  static int esp_output(struct xfrm_state *x, struct sk_buff *skb)
 	int extralen;
 	int tailen;
 	__be64 seqno;
+	int esp_offset = 0;
 	__u8 proto = *skb_mac_header(skb);
 
 	/* skb is pure payload to encrypt */
@@ -288,6 +289,8 @@  static int esp_output(struct xfrm_state *x, struct sk_buff *skb)
 			break;
 		}
 
+		esp_offset = (unsigned char *)esph - (unsigned char *)uh;
+
 		*skb_mac_header(skb) = IPPROTO_UDP;
 	}
 
@@ -397,7 +400,7 @@  static int esp_output(struct xfrm_state *x, struct sk_buff *skb)
 		goto error;
 	nfrags = err;
 	tail = skb_tail_pointer(trailer);
-	esph = ip_esp_hdr(skb);
+	esph = (struct ip_esp_hdr *)(skb_transport_header(skb) + esp_offset);
 
 skip_cow:
 	esp_output_fill_trailer(tail, tfclen, plen, proto);