[net-next,1/5] tcp: fix SO_RCVLOWAT and RCVBUF autotuning

Applications might use SO_RCVLOWAT on TCP socket hoping to receive
one [E]POLLIN event only when a given amount of bytes are ready in socket
receive queue.

Problem is that receive autotuning is not aware of this constraint,
meaning sk_rcvbuf might be too small to allow all bytes to be stored.

Add a new (struct proto_ops)->set_rcvlowat method so that a protocol
can override the default setsockopt(SO_RCVLOWAT) behavior.

Signed-off-by: Eric Dumazet <edumazet@google.com>
---
 include/linux/net.h |  1 +
 include/net/tcp.h   |  1 +
 net/core/sock.c     |  5 ++++-
 net/ipv4/af_inet.c  |  1 +
 net/ipv4/tcp.c      | 21 +++++++++++++++++++++
 net/ipv6/af_inet6.c |  1 +
 6 files changed, 29 insertions(+), 1 deletion(-)

On Mon, Apr 16, 2018 at 10:33:35AM -0700, Eric Dumazet wrote:
> Applications might use SO_RCVLOWAT on TCP socket hoping to receive
> one [E]POLLIN event only when a given amount of bytes are ready in socket
> receive queue.
>
> Problem is that receive autotuning is not aware of this constraint,
> meaning sk_rcvbuf might be too small to allow all bytes to be stored.
>
> Add a new (struct proto_ops)->set_rcvlowat method so that a protocol
> can override the default setsockopt(SO_RCVLOWAT) behavior.
>

...

> +/* Make sure sk_rcvbuf is big enough to satisfy SO_RCVLOWAT hint */
> +int tcp_set_rcvlowat(struct sock *sk, int val)
> +{
> +	sk->sk_rcvlowat = val ? : 1;
> +	if (sk->sk_userlocks & SOCK_RCVBUF_LOCK)
> +		return 0;
> +
> +	/* val comes from user space and might be close to INT_MAX */
> +	val <<= 1;
> +	if (val < 0)
> +		val = INT_MAX;
> +
> +	val = min(val, sock_net(sk)->ipv4.sysctl_tcp_rmem[2]);

Hi Eric,

As val may be changed to a smaller value by the line above, shouldn't
it assign sk->sk_rcvlowat again?  Otherwise it may still be bigger
than sk_rcvbuf.

Say val = 512k, sysctl_tcp_rmem[2] = 256k
val <<= 1 ,  val = 1M
val = min() ,  val = 256k
val > sk_rcvbuf
   sk_rcvbuf = 256k , at most, which is smaller than sk_rcvlowat

Without reassigning the application has to check how big is
tcp_rmem[2] and be sure to not go above /2 of it to not trip on this
again.

Or, as you have added a return value here, it could return -EINVAL in
such cases. Probably better, as then the application will not get a
smaller buffer than wanted later.

> +	if (val > sk->sk_rcvbuf) {
> +		sk->sk_rcvbuf = val;
> +		tcp_sk(sk)->window_clamp = tcp_win_from_space(sk, val);
> +	}
> +	return 0;
> +}
> +EXPORT_SYMBOL(tcp_set_rcvlowat);
> +
...

On Thu, Apr 19, 2018 at 7:02 PM Marcelo Ricardo Leitner <
marcelo.leitner@gmail.com> wrote:

> Hi Eric,

> As val may be changed to a smaller value by the line above, shouldn't
> it assign sk->sk_rcvlowat again?  Otherwise it may still be bigger
> than sk_rcvbuf.

> Say val = 512k, sysctl_tcp_rmem[2] = 256k
> val <<= 1 ,  val = 1M
> val = min() ,  val = 256k
> val > sk_rcvbuf
>     sk_rcvbuf = 256k , at most, which is smaller than sk_rcvlowat

> Without reassigning the application has to check how big is
> tcp_rmem[2] and be sure to not go above /2 of it to not trip on this
> again.

I am not sure about that :

Reporting an error might break existing applications that were not
expecting setsockopt()
to return an error, even if the value was 'probably too big to be okay'


> Or, as you have added a return value here, it could return -EINVAL in
> such cases. Probably better, as then the application will not get a
> smaller buffer than wanted later.

Note that maybe some applications might first set SO_RCVLOWAT, then
SO_RCVBUF,
we do not want to break them.


My patch really covers the case were autotuning should immediately grow the
sk_rcvbuf
for reasonable SO_RCVLOWAT values.

On Fri, Apr 20, 2018 at 02:36:52AM +0000, Eric Dumazet wrote:
> On Thu, Apr 19, 2018 at 7:02 PM Marcelo Ricardo Leitner <
> marcelo.leitner@gmail.com> wrote:
>
> > Hi Eric,
>
> > As val may be changed to a smaller value by the line above, shouldn't
> > it assign sk->sk_rcvlowat again?  Otherwise it may still be bigger
> > than sk_rcvbuf.
>
> > Say val = 512k, sysctl_tcp_rmem[2] = 256k
> > val <<= 1 ,  val = 1M
> > val = min() ,  val = 256k
> > val > sk_rcvbuf
> >     sk_rcvbuf = 256k , at most, which is smaller than sk_rcvlowat
>
> > Without reassigning the application has to check how big is
> > tcp_rmem[2] and be sure to not go above /2 of it to not trip on this
> > again.
>
> I am not sure about that :
>
> Reporting an error might break existing applications that were not
> expecting setsockopt()
> to return an error, even if the value was 'probably too big to be okay'

I would argue that they are already broken but...

>
>
> > Or, as you have added a return value here, it could return -EINVAL in
> > such cases. Probably better, as then the application will not get a
> > smaller buffer than wanted later.
>
> Note that maybe some applications might first set SO_RCVLOWAT, then
> SO_RCVBUF,
> we do not want to break them.

... yeah.. if they do it this way, they work today. Good point.

>
>
> My patch really covers the case were autotuning should immediately grow the
> sk_rcvbuf
> for reasonable SO_RCVLOWAT values.

That's not exactly what the comment above the function says, thus why
my comments.

Thanks.