mbox series

[0/4,SRU,X] Multiple TCP Fixups

Message ID 20190829004948.1573-1-tyhicks@canonical.com
Headers show
Series Multiple TCP Fixups | expand

Message

Tyler Hicks Aug. 29, 2019, 12:49 a.m. UTC
This series reverts my backport of a fixup for the CVE-2019-11478 fix
and applies the version of the fixup that the TCP maintainer provided
for the 4.4 linux-stable tree. It also includes another fixup, from
upstream, which addresses some performance issues that were reported to
me. Details can be found here:

 https://databricks.com/blog/2019/08/01/network-performance-regressions-from-tcp-sack-vulnerability-fixes.html

The fix for CVE-2019-15239 is sandwiched in the middle of the series. It
made cherry-picking of the entire series from linux-stable possible but,
more importantly, it fixes a flaw that was caused by a bad backport in
the linux-stable tree.

 https://people.canonical.com/~ubuntu-security/cve/?cve=CVE-2019-11478
 https://people.canonical.com/~ubuntu-security/cve/?cve=CVE-2019-15239

Note that the Ubuntu CVE Tracker entry for CVE-2019-15239 is not fully
updated with breaks-fix commit info as I'm still trying to decide how
best to do that for this somewhat unique CVE that affects linux-stable
but not linux.

I believe that I was able to reproduce the some of the nondeterministic
performance regression that Databricks was seeing using netperf while
running the 4.4.0-159.187-generic. I didn't see this behavior while
testing the 4.4.0-150.176-generic kernel, which is the last published
kernel before CVE-2019-11478 was fixed. I also don't see the behavior
once these patches are applied to the 4.4.0-159.187-generic kernel.

Tyler

Eric Dumazet (2):
  tcp: refine memory limit test in tcp_fragment()
  tcp: be more careful in tcp_fragment()

Soheil Hassas Yeganeh (1):
  tcp: reset sk_send_head in tcp_write_queue_purge

Tyler Hicks (1):
  UBUNTU: SAUCE: Revert "tcp: refine memory limit test in
    tcp_fragment()"

 include/net/tcp.h     | 22 ++++++++++++++++++++--
 net/ipv4/tcp_output.c | 12 ++++++++++--
 2 files changed, 30 insertions(+), 4 deletions(-)

Comments

Thadeu Lima de Souza Cascardo Aug. 30, 2019, 11:43 a.m. UTC | #1
On Thu, Aug 29, 2019 at 12:49:44AM +0000, Tyler Hicks wrote:
> This series reverts my backport of a fixup for the CVE-2019-11478 fix
> and applies the version of the fixup that the TCP maintainer provided
> for the 4.4 linux-stable tree. It also includes another fixup, from
> upstream, which addresses some performance issues that were reported to
> me. Details can be found here:
> 
>  https://databricks.com/blog/2019/08/01/network-performance-regressions-from-tcp-sack-vulnerability-fixes.html
> 
> The fix for CVE-2019-15239 is sandwiched in the middle of the series. It
> made cherry-picking of the entire series from linux-stable possible but,
> more importantly, it fixes a flaw that was caused by a bad backport in
> the linux-stable tree.
> 
>  https://people.canonical.com/~ubuntu-security/cve/?cve=CVE-2019-11478
>  https://people.canonical.com/~ubuntu-security/cve/?cve=CVE-2019-15239


Acked-by: Thadeu Lima de Souza Cascardo <cascardo@canonical.com>

I'm all for making our code more in line with linux-stable. Thanks!

Also, the NULL pointer dereference, which seems to be the point of the
patchset. Yay! \o/

Cheking for the limits using truesize and allowing head and tail to be split
seem safer from the point of view of performance regression, or even possible
regressions with small send buffers. Which was the point of the backport in the
first place, but who knows what other usecases are out there.

Which takes me to the point of the comment. Was this patchset tested with the
example regression we had? The packetdrill test that set the small send buffer
and got stuck on a write? And was it tested against the PoCs for the SACK
attacks?

Thanks.
Cascardo.

> 
> Note that the Ubuntu CVE Tracker entry for CVE-2019-15239 is not fully
> updated with breaks-fix commit info as I'm still trying to decide how
> best to do that for this somewhat unique CVE that affects linux-stable
> but not linux.
> 
> I believe that I was able to reproduce the some of the nondeterministic
> performance regression that Databricks was seeing using netperf while
> running the 4.4.0-159.187-generic. I didn't see this behavior while
> testing the 4.4.0-150.176-generic kernel, which is the last published
> kernel before CVE-2019-11478 was fixed. I also don't see the behavior
> once these patches are applied to the 4.4.0-159.187-generic kernel.
> 
> Tyler
> 
> Eric Dumazet (2):
>   tcp: refine memory limit test in tcp_fragment()
>   tcp: be more careful in tcp_fragment()
> 
> Soheil Hassas Yeganeh (1):
>   tcp: reset sk_send_head in tcp_write_queue_purge
> 
> Tyler Hicks (1):
>   UBUNTU: SAUCE: Revert "tcp: refine memory limit test in
>     tcp_fragment()"
> 
>  include/net/tcp.h     | 22 ++++++++++++++++++++--
>  net/ipv4/tcp_output.c | 12 ++++++++++--
>  2 files changed, 30 insertions(+), 4 deletions(-)
> 
> -- 
> 2.17.1
> 
> 
> -- 
> kernel-team mailing list
> kernel-team@lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/kernel-team
Tyler Hicks Sept. 3, 2019, 5:57 p.m. UTC | #2
On 2019-08-30 08:43:17, Thadeu Lima de Souza Cascardo wrote:
> On Thu, Aug 29, 2019 at 12:49:44AM +0000, Tyler Hicks wrote:
> > This series reverts my backport of a fixup for the CVE-2019-11478 fix
> > and applies the version of the fixup that the TCP maintainer provided
> > for the 4.4 linux-stable tree. It also includes another fixup, from
> > upstream, which addresses some performance issues that were reported to
> > me. Details can be found here:
> > 
> >  https://databricks.com/blog/2019/08/01/network-performance-regressions-from-tcp-sack-vulnerability-fixes.html
> > 
> > The fix for CVE-2019-15239 is sandwiched in the middle of the series. It
> > made cherry-picking of the entire series from linux-stable possible but,
> > more importantly, it fixes a flaw that was caused by a bad backport in
> > the linux-stable tree.
> > 
> >  https://people.canonical.com/~ubuntu-security/cve/?cve=CVE-2019-11478
> >  https://people.canonical.com/~ubuntu-security/cve/?cve=CVE-2019-15239
> 
> 
> Acked-by: Thadeu Lima de Souza Cascardo <cascardo@canonical.com>
> 
> I'm all for making our code more in line with linux-stable. Thanks!
> 
> Also, the NULL pointer dereference, which seems to be the point of the
> patchset. Yay! \o/
> 
> Cheking for the limits using truesize and allowing head and tail to be split
> seem safer from the point of view of performance regression, or even possible
> regressions with small send buffers. Which was the point of the backport in the
> first place, but who knows what other usecases are out there.
> 
> Which takes me to the point of the comment. Was this patchset tested with the
> example regression we had? The packetdrill test that set the small send buffer
> and got stuck on a write? And was it tested against the PoCs for the SACK
> attacks?

My testing for these changes was focused on the fix for the reported
performance regression. I didn't test with the PoC or the packetdrill
test although I wish that I would have now that you mention it...

Tyler

> 
> Thanks.
> Cascardo.
> 
> > 
> > Note that the Ubuntu CVE Tracker entry for CVE-2019-15239 is not fully
> > updated with breaks-fix commit info as I'm still trying to decide how
> > best to do that for this somewhat unique CVE that affects linux-stable
> > but not linux.
> > 
> > I believe that I was able to reproduce the some of the nondeterministic
> > performance regression that Databricks was seeing using netperf while
> > running the 4.4.0-159.187-generic. I didn't see this behavior while
> > testing the 4.4.0-150.176-generic kernel, which is the last published
> > kernel before CVE-2019-11478 was fixed. I also don't see the behavior
> > once these patches are applied to the 4.4.0-159.187-generic kernel.
> > 
> > Tyler
> > 
> > Eric Dumazet (2):
> >   tcp: refine memory limit test in tcp_fragment()
> >   tcp: be more careful in tcp_fragment()
> > 
> > Soheil Hassas Yeganeh (1):
> >   tcp: reset sk_send_head in tcp_write_queue_purge
> > 
> > Tyler Hicks (1):
> >   UBUNTU: SAUCE: Revert "tcp: refine memory limit test in
> >     tcp_fragment()"
> > 
> >  include/net/tcp.h     | 22 ++++++++++++++++++++--
> >  net/ipv4/tcp_output.c | 12 ++++++++++--
> >  2 files changed, 30 insertions(+), 4 deletions(-)
> > 
> > -- 
> > 2.17.1
> > 
> > 
> > -- 
> > kernel-team mailing list
> > kernel-team@lists.ubuntu.com
> > https://lists.ubuntu.com/mailman/listinfo/kernel-team
Tyler Hicks Sept. 3, 2019, 5:59 p.m. UTC | #3
On 2019-08-29 00:49:44, Tyler Hicks wrote:
> This series reverts my backport of a fixup for the CVE-2019-11478 fix
> and applies the version of the fixup that the TCP maintainer provided
> for the 4.4 linux-stable tree. It also includes another fixup, from
> upstream, which addresses some performance issues that were reported to
> me. Details can be found here:
> 
>  https://databricks.com/blog/2019/08/01/network-performance-regressions-from-tcp-sack-vulnerability-fixes.html
> 
> The fix for CVE-2019-15239 is sandwiched in the middle of the series. It
> made cherry-picking of the entire series from linux-stable possible but,
> more importantly, it fixes a flaw that was caused by a bad backport in
> the linux-stable tree.
> 
>  https://people.canonical.com/~ubuntu-security/cve/?cve=CVE-2019-11478
>  https://people.canonical.com/~ubuntu-security/cve/?cve=CVE-2019-15239
> 
> Note that the Ubuntu CVE Tracker entry for CVE-2019-15239 is not fully
> updated with breaks-fix commit info as I'm still trying to decide how
> best to do that for this somewhat unique CVE that affects linux-stable
> but not linux.
> 
> I believe that I was able to reproduce the some of the nondeterministic
> performance regression that Databricks was seeing using netperf while
> running the 4.4.0-159.187-generic. I didn't see this behavior while
> testing the 4.4.0-150.176-generic kernel, which is the last published
> kernel before CVE-2019-11478 was fixed. I also don't see the behavior
> once these patches are applied to the 4.4.0-159.187-generic kernel.

These changes are all in the 4.4.189 upstream linux-stable pull request
that was prepared by Connor. This series sent out by me can be ignored.

Tyler

> 
> Tyler
> 
> Eric Dumazet (2):
>   tcp: refine memory limit test in tcp_fragment()
>   tcp: be more careful in tcp_fragment()
> 
> Soheil Hassas Yeganeh (1):
>   tcp: reset sk_send_head in tcp_write_queue_purge
> 
> Tyler Hicks (1):
>   UBUNTU: SAUCE: Revert "tcp: refine memory limit test in
>     tcp_fragment()"
> 
>  include/net/tcp.h     | 22 ++++++++++++++++++++--
>  net/ipv4/tcp_output.c | 12 ++++++++++--
>  2 files changed, 30 insertions(+), 4 deletions(-)
> 
> -- 
> 2.17.1
> 
> 
> -- 
> kernel-team mailing list
> kernel-team@lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/kernel-team