Patchwork [1/3] net: TCP thin-stream detection

login
register
mail settings
Submitter Andreas Petlund
Date Oct. 27, 2009, 4:31 p.m.
Message ID <4AE72075.4070702@simula.no>
Download mbox | patch
Permalink /patch/37002/
State Changes Requested
Delegated to: David Miller
Headers show

Comments

Andreas Petlund - Oct. 27, 2009, 4:31 p.m.
Inline function to dynamically detect thin streams based on the number of packets in flight. Used to trigger thin-stream mechanisms.


Signed-off-by: Andreas Petlund <apetlund@simula.no>
---
 include/net/tcp.h |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)
William Allen Simpson - Oct. 28, 2009, 3:09 a.m.
Andreas Petlund wrote:
> +/* Determines whether this is a thin stream (which may suffer from
> + * increased latency). Used to trigger latency-reducing mechanisms.
> + */
> +static inline unsigned int tcp_stream_is_thin(const struct tcp_sock *tp)
> +{
> +	return tp->packets_out < 4;
> +}
> +
This bothers me a bit.  Having just looked at your Linux presentation,
and not (yet) read your papers, it seems much of your justification was
with 1 packet per RTT.  Here, you seem to be concentrating on 4, probably
because many implementations quickly ramp up to 4.

But there's a fair amount of experience showing that ramping to 4 is
problematic on congested paths, especially wireless networks.  Fast
retransmit in that case would be disastrous.

Once upon a time, I worked on a fair number of interactive games a decade
or so ago.  And agree that this can be a problem, although I've never
been a fan of turning off the Nagle algorithm.  My solution has always
been a heartbeat, rather than trying to shoehorn this into TCP.

Also, I've not seen any discussion on the end-to-end interest list.

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Andreas Petlund - Oct. 29, 2009, 1:51 p.m.
Den 28. okt. 2009 kl. 04.09 skrev William Allen Simpson:

> Andreas Petlund wrote:
>> +/* Determines whether this is a thin stream (which may suffer from
>> + * increased latency). Used to trigger latency-reducing mechanisms.
>> + */
>> +static inline unsigned int tcp_stream_is_thin(const struct  
>> tcp_sock *tp)
>> +{
>> +	return tp->packets_out < 4;
>> +}
>> +
> This bothers me a bit.  Having just looked at your Linux presentation,
> and not (yet) read your papers, it seems much of your justification  
> was
> with 1 packet per RTT.  Here, you seem to be concentrating on 4,  
> probably
> because many implementations quickly ramp up to 4.
>

The limit of 4 packets in flight is based on the fact that less than 4  
packets in flight makes fast retransmissions impossible, thus limiting  
the retransmit options to timeout-retransmissions. The criterion is  
therefore as conservative as possible while still serving its purpose.  
If further losses occur, the exponential backoff will increase latency  
further. The concept of using this limit is also discussed in the  
Internet draft for Early Retransmit by Allman et al.:
http://www.icir.org/mallman/papers/draft-ietf-tcpm-early-rexmt-01.txt

> But there's a fair amount of experience showing that ramping to 4 is
> problematic on congested paths, especially wireless networks.  Fast
> retransmit in that case would be disastrous.

First, the modifications implemented in the patch is explicitly  
enabled only for applications where the developer knows that streams  
will be thin, thus only a small subset of the streams will apply the  
modifications. Second, experiments we have performed to try to map the  
effect on a congested bottleneck both with and without the  
modifications show that no measurable effect is recorded.

Graphs presenting results from experiments performed to analyse  
latency and fairness issues can be found here:
http://folk.uio.no/apetlund/lktmp/

> Once upon a time, I worked on a fair number of interactive games a  
> decade
> or so ago.  And agree that this can be a problem, although I've never
> been a fan of turning off the Nagle algorithm.  My solution has always
> been a heartbeat, rather than trying to shoehorn this into TCP.
>

The beginning of this patch was an analysis of game traffic from the  
Norwegian game company Funcom. They use TCP for all their MMOGs as  
does, for example, Blizzard for WoW. Our analysis showed that many  
players experienced extreme latencies, and the source of this was  
tracked to the effects that we discuss here. As long as a wide range  
of time-dependent applications choose to use TCP, and we can improve  
conditions for their needs without jeopardising other functionality,  
we think that this will add value to the TCP stack.

> Also, I've not seen any discussion on the end-to-end interest list.

It will be enlightening to have a discussion on end-to-end about this  
topic.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Arnd Hannemann - Oct. 29, 2009, 4:32 p.m.
Andreas Petlund schrieb:
> Den 28. okt. 2009 kl. 04.09 skrev William Allen Simpson:
> 
>> Andreas Petlund wrote:
>>> +/* Determines whether this is a thin stream (which may suffer from
>>> + * increased latency). Used to trigger latency-reducing mechanisms.
>>> + */
>>> +static inline unsigned int tcp_stream_is_thin(const struct  
>>> tcp_sock *tp)
>>> +{
>>> +	return tp->packets_out < 4;
>>> +}
>>> +
>> This bothers me a bit.  Having just looked at your Linux presentation,
>> and not (yet) read your papers, it seems much of your justification  
>> was
>> with 1 packet per RTT.  Here, you seem to be concentrating on 4,  
>> probably
>> because many implementations quickly ramp up to 4.
>>
> 
> The limit of 4 packets in flight is based on the fact that less than 4  
> packets in flight makes fast retransmissions impossible, thus limiting  
> the retransmit options to timeout-retransmissions. The criterion is  

There is Limited Transmit! So this is generally not true.

> therefore as conservative as possible while still serving its purpose.  
> If further losses occur, the exponential backoff will increase latency  
> further. The concept of using this limit is also discussed in the  
> Internet draft for Early Retransmit by Allman et al.:
> http://www.icir.org/mallman/papers/draft-ietf-tcpm-early-rexmt-01.txt

This ID is covering exactly the cases which Limited Transmit does not
cover and works "automagically" without help of application. So why not
just implement this ID?

Best regards,
Arnd
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Ilpo Järvinen - Oct. 29, 2009, 8:26 p.m.
On Thu, 29 Oct 2009, Arnd Hannemann wrote:

> Andreas Petlund schrieb:
> > Den 28. okt. 2009 kl. 04.09 skrev William Allen Simpson:
> > 
> >> Andreas Petlund wrote:
> >>> +/* Determines whether this is a thin stream (which may suffer from
> >>> + * increased latency). Used to trigger latency-reducing mechanisms.
> >>> + */
> >>> +static inline unsigned int tcp_stream_is_thin(const struct  
> >>> tcp_sock *tp)
> >>> +{
> >>> +	return tp->packets_out < 4;
> >>> +}
> >>> +
> >> This bothers me a bit.  Having just looked at your Linux presentation,
> >> and not (yet) read your papers, it seems much of your justification  
> >> was
> >> with 1 packet per RTT.  Here, you seem to be concentrating on 4,  
> >> probably
> >> because many implementations quickly ramp up to 4.
> >>
> > 
> > The limit of 4 packets in flight is based on the fact that less than 4  
> > packets in flight makes fast retransmissions impossible, thus limiting  
> > the retransmit options to timeout-retransmissions. The criterion is  
> 
> There is Limited Transmit! So this is generally not true.
> 
> > therefore as conservative as possible while still serving its purpose.  
> > If further losses occur, the exponential backoff will increase latency  
> > further. The concept of using this limit is also discussed in the  
> > Internet draft for Early Retransmit by Allman et al.:
> > http://www.icir.org/mallman/papers/draft-ietf-tcpm-early-rexmt-01.txt
> 
> This ID is covering exactly the cases which Limited Transmit does not
> cover and works "automagically" without help of application. So why not
> just implement this ID?

I even gave some advise recently to one guy how to polish up the early 
retransmit implementation of his. ...However, I think we haven't heard 
from that since then... I added him as CC if he happens to have it already 
done.

It is actually so that patches 1+3 implement sort of an early retransmit, 
just slightly more aggressive of it than what is given in ID but I find 
the difference in the aggressiveness rather insignificant. ...Whereas the 
RTO stuff is more questionable.

Patch

diff --git a/include/net/tcp.h b/include/net/tcp.h
index 03a49c7..7c4482f 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -800,6 +800,14 @@  static inline bool tcp_in_initial_slowstart(const struct tcp_sock *tp)
 	return tp->snd_ssthresh >= TCP_INFINITE_SSTHRESH;
 }
 
+/* Determines whether this is a thin stream (which may suffer from
+ * increased latency). Used to trigger latency-reducing mechanisms.
+ */
+static inline unsigned int tcp_stream_is_thin(const struct tcp_sock *tp)
+{
+	return tp->packets_out < 4;
+}
+
 /* If cwnd > ssthresh, we may raise ssthresh to be half-way to cwnd.
  * The exception is rate halving phase, when cwnd is decreasing towards
  * ssthresh.