| Message ID | 20180501200539.GA69207@davejwatson-mba.dhcp.thefacebook.com |
|---|---|
| State | Accepted, archived |
| Delegated to: | David Miller |
| Headers | show |
| Series | [net] net/tls: Don't recursively call push_record during tls_write_space callbacks | expand |
From: Dave Watson <davejwatson@fb.com> Date: Tue, 1 May 2018 13:05:39 -0700 > It is reported that in some cases, write_space may be called in > do_tcp_sendpages, such that we recursively invoke do_tcp_sendpages again: > > [ 660.468802] ? do_tcp_sendpages+0x8d/0x580 > [ 660.468826] ? tls_push_sg+0x74/0x130 [tls] > [ 660.468852] ? tls_push_record+0x24a/0x390 [tls] > [ 660.468880] ? tls_write_space+0x6a/0x80 [tls] > ... > > tls_push_sg already does a loop over all sending sg's, so ignore > any tls_write_space notifications until we are done sending. > We then have to call the previous write_space to wake up > poll() waiters after we are done with the send loop. > > Reported-by: Andre Tomt <andre@tomt.net> > Signed-off-by: Dave Watson <davejwatson@fb.com> Applied.
On 01. mai 2018 22:05, Dave Watson wrote: > It is reported that in some cases, write_space may be called in > do_tcp_sendpages, such that we recursively invoke do_tcp_sendpages again: > > [ 660.468802] ? do_tcp_sendpages+0x8d/0x580 > [ 660.468826] ? tls_push_sg+0x74/0x130 [tls] > [ 660.468852] ? tls_push_record+0x24a/0x390 [tls] > [ 660.468880] ? tls_write_space+0x6a/0x80 [tls] > ... > > tls_push_sg already does a loop over all sending sg's, so ignore > any tls_write_space notifications until we are done sending. > We then have to call the previous write_space to wake up > poll() waiters after we are done with the send loop. > > Reported-by: Andre Tomt <andre@tomt.net> > Signed-off-by: Dave Watson <davejwatson@fb.com> Unfortunately it seems like this patch has a bug, while it fixed the kernel crashing it is causing some connections in my testbed to stall. Making sure ctx->in_tcp_sendpages is also cleared before the return ret within the while(1) loop seems to fix it for me. diff -Naurp a/net/tls/tls_main.c b/net/tls/tls_main.c --- a/net/tls/tls_main.c 2018-05-06 02:23:41.638597066 +0200 +++ b/net/tls/tls_main.c 2018-05-06 01:59:14.378568139 +0200 @@ -135,6 +135,7 @@ retry: offset -= sg->offset; ctx->partially_sent_offset = offset; ctx->partially_sent_record = (void *)sg; + ctx->in_tcp_sendpages = false; return ret; }
diff --git a/include/net/tls.h b/include/net/tls.h index 3da8e13..b400d0bb 100644 --- a/include/net/tls.h +++ b/include/net/tls.h @@ -148,6 +148,7 @@ struct tls_context { struct scatterlist *partially_sent_record; u16 partially_sent_offset; unsigned long flags; + bool in_tcp_sendpages; u16 pending_open_record_frags; int (*push_pending_record)(struct sock *sk, int flags); diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index 0d37997..cc03e00 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -114,6 +114,7 @@ int tls_push_sg(struct sock *sk, size = sg->length - offset; offset += sg->offset; + ctx->in_tcp_sendpages = true; while (1) { if (sg_is_last(sg)) sendpage_flags = flags; @@ -148,6 +149,8 @@ int tls_push_sg(struct sock *sk, } clear_bit(TLS_PENDING_CLOSED_RECORD, &ctx->flags); + ctx->in_tcp_sendpages = false; + ctx->sk_write_space(sk); return 0; } @@ -217,6 +220,10 @@ static void tls_write_space(struct sock *sk) { struct tls_context *ctx = tls_get_ctx(sk); + /* We are already sending pages, ignore notification */ + if (ctx->in_tcp_sendpages) + return; + if (!sk->sk_write_pending && tls_is_pending_closed_record(ctx)) { gfp_t sk_allocation = sk->sk_allocation; int rc;
It is reported that in some cases, write_space may be called in do_tcp_sendpages, such that we recursively invoke do_tcp_sendpages again: [ 660.468802] ? do_tcp_sendpages+0x8d/0x580 [ 660.468826] ? tls_push_sg+0x74/0x130 [tls] [ 660.468852] ? tls_push_record+0x24a/0x390 [tls] [ 660.468880] ? tls_write_space+0x6a/0x80 [tls] ... tls_push_sg already does a loop over all sending sg's, so ignore any tls_write_space notifications until we are done sending. We then have to call the previous write_space to wake up poll() waiters after we are done with the send loop. Reported-by: Andre Tomt <andre@tomt.net> Signed-off-by: Dave Watson <davejwatson@fb.com> --- include/net/tls.h | 1 + net/tls/tls_main.c | 7 +++++++ 2 files changed, 8 insertions(+)