| Message ID | 4B7AAE60.3070305@simula.no |
|---|---|
| State | Changes Requested, archived |
| Delegated to: | David Miller |
| Headers | show |
From: Andreas Petlund <apetlund@simula.no> Date: Tue, 16 Feb 2010 15:40:32 +0100 > > +/* Determines whether this is a thin stream (which may suffer from > + * increased latency). Used to trigger latency-reducing mechanisms.*/ > +static inline unsigned int tcp_stream_is_thin(struct tcp_sock *tp) > +{ > + return tp->packets_out < 4 && !tcp_in_initial_slowstart(tp); > +} > + > /* /proc */ Please format comments: /* Like this. */ or: /* Like this. * And this. */ Thanks. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi! > +After analysing a large number of time-dependent interactive > +applications, we have seen that they often produce thin streams > +and also stay with this traffic pattern throughout its entire > +lifespan. The combination of time-dependency and the fact that the > +streams provoke high latencies when using TCP is unfortunate. > + > +In order to reduce application-layer latency when packets are lost, > +a set of mechanisms has been made, which address these latency issues > +for thin streams. In short, if the kernel detects a thin stream, > +the retransmission mechanisms are modified in the following manner: > + > +1) If the stream is thin, fast retransmit on the first dupACK. > +2) If the stream is thin, do not apply exponential backoff. 2) seems very dangerous/unfair. If network congestion is caused just by thin streams, will the network just fall apart?
Am 21.02.2010 um 11:21 schrieb Pavel Machek: > Hi! > >> +After analysing a large number of time-dependent interactive >> +applications, we have seen that they often produce thin streams >> +and also stay with this traffic pattern throughout its entire >> +lifespan. The combination of time-dependency and the fact that the >> +streams provoke high latencies when using TCP is unfortunate. >> + >> +In order to reduce application-layer latency when packets are lost, >> +a set of mechanisms has been made, which address these latency issues >> +for thin streams. In short, if the kernel detects a thin stream, >> +the retransmission mechanisms are modified in the following manner: >> + >> +1) If the stream is thin, fast retransmit on the first dupACK. >> +2) If the stream is thin, do not apply exponential backoff. > > 2) seems very dangerous/unfair. If network congestion is caused just > by thin streams, will the network just fall apart? and 1) can also be dangerous if we have reordering on the path. I strongly suggest that we discuss Andreas' idea on IETF TCPM *before* we integrate it in the kernel and enable it for everyone Alex, as an netdev reader and TCPM member > > > -- > (english) http://www.livejournal.com/~pavelmachek > (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html > -- > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html // // Dipl.-Inform. Alexander Zimmermann // Department of Computer Science, Informatik 4 // RWTH Aachen University // Ahornstr. 55, 52056 Aachen, Germany // phone: (49-241) 80-21422, fax: (49-241) 80-22220 // email: zimmermann@cs.rwth-aachen.de // web: http://www.umic-mesh.net // -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2010-2-21, at 13:23, Alexander Zimmermann wrote: > I strongly suggest that we discuss Andreas' idea on IETF TCPM *before* > we integrate it in the kernel and enable it for everyone Agree with Alexander. It is not at all clear to me that these changes are safe for non-exerimental use in the wider Internet. Lars
* Pavel Machek | 2010-02-21 11:21:03 [+0100]: >> +1) If the stream is thin, fast retransmit on the first dupACK. >> +2) If the stream is thin, do not apply exponential backoff. > >2) seems very dangerous/unfair. If network congestion is caused just >by thin streams, will the network just fall apart? I question the fairness of this modification! The modification sounds _really_ far-reaching. Did you analyse the behaviour of this modification? What about fairness analysis? Great work is available in the network simulator sector for this kind of analysis (e.g. jain's fairness index). HGN
On Sun, 21 Feb 2010, Pavel Machek wrote: > Hi! > > > +After analysing a large number of time-dependent interactive > > +applications, we have seen that they often produce thin streams > > +and also stay with this traffic pattern throughout its entire > > +lifespan. The combination of time-dependency and the fact that the > > +streams provoke high latencies when using TCP is unfortunate. > > + > > +In order to reduce application-layer latency when packets are lost, > > +a set of mechanisms has been made, which address these latency issues > > +for thin streams. In short, if the kernel detects a thin stream, > > +the retransmission mechanisms are modified in the following manner: > > + > > +1) If the stream is thin, fast retransmit on the first dupACK. > > +2) If the stream is thin, do not apply exponential backoff. > > 2) seems very dangerous/unfair. If network congestion is caused just > by thin streams, will the network just fall apart? ...Network will not fall apart. Two possible cases: a) cwnd > 1 segment, on RTO the window is reduced, thus the sender backs off regardless of exponential backoff. b) All flows have window of 1 already... Well, what do you suggest? I'd say that obviously the network is seriously underprovisioned for the workload and since the bottleneck can only serve some of them anyway dropping the rest, no useless work gets done at the bottleneck. RTT estimator then gets a new samples whenever a flow belongs into the lucky group. In case any unnecessary retransmission happened (if there was very dramatic and sudden increase transient in the workload) they won't happen thereafter (unless we increase the workload toward infinity). ...Thus no problem of "falling apart".
On Sun, 21 Feb 2010, Alexander Zimmermann wrote: > > Am 21.02.2010 um 11:21 schrieb Pavel Machek: > > > Hi! > > > >> +After analysing a large number of time-dependent interactive > >> +applications, we have seen that they often produce thin streams > >> +and also stay with this traffic pattern throughout its entire > >> +lifespan. The combination of time-dependency and the fact that the > >> +streams provoke high latencies when using TCP is unfortunate. > >> + > >> +In order to reduce application-layer latency when packets are lost, > >> +a set of mechanisms has been made, which address these latency issues > >> +for thin streams. In short, if the kernel detects a thin stream, > >> +the retransmission mechanisms are modified in the following manner: > >> + > >> +1) If the stream is thin, fast retransmit on the first dupACK. > >> +2) If the stream is thin, do not apply exponential backoff. > > > > 2) seems very dangerous/unfair. If network congestion is caused just > > by thin streams, will the network just fall apart? > > and 1) can also be dangerous if we have reordering on the path. > > I strongly suggest that we discuss Andreas' idea on IETF TCPM *before* > we integrate it in the kernel and enable it for everyone What difference you see with 1) and early rexmit when cwnd = 2, the latter being afaict "discussed already" on TCPM?
From: Lars Eggert <lars.eggert@nokia.com> Date: Sun, 21 Feb 2010 22:04:46 +0200 > Agree with Alexander. It is not at all clear to me that these > changes are safe for non-exerimental use in the wider Internet. It's off by default and we provide all sorts of experimental congestion control algorithms which can cause just as much if not more trouble than these thin-stream bits. There is no problem integrating these changes. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
If it's off by default there is no issue. On 2010-2-22, at 1:03, David Miller wrote: > From: Lars Eggert <lars.eggert@nokia.com> > Date: Sun, 21 Feb 2010 22:04:46 +0200 > >> Agree with Alexander. It is not at all clear to me that these >> changes are safe for non-exerimental use in the wider Internet. > > It's off by default and we provide all sorts of experimental > congestion control algorithms which can cause just as much > if not more trouble than these thin-stream bits. > > There is no problem integrating these changes.
Lars Eggert <lars.eggert@nokia.com> writes:
> If it's off by default there is no issue.
Still a sufficient large number of people might turn it on
and then the network would melt.
IMHO TCP changes with badly understood congestion behaviour
should not be merged until the necessary analysis/simulation etc.
is done.
-Andi
Am 22.02.2010 um 16:17 schrieb Andi Kleen: > Lars Eggert <lars.eggert@nokia.com> writes: > >> If it's off by default there is no issue. > > Still a sufficient large number of people might turn it on > and then the network would melt. > > IMHO TCP changes with badly understood congestion behaviour > should not be merged until the necessary analysis/simulation etc. > is done. > > -Andi 100% ack. See for example here: http://www.umic-mesh.net/~zimmermann/linux.png It's a screenshot of the ps3mediaserver. Since it's a streaming server, it's very useful to deactivate Nagle :-) Alex > > -- > ak@linux.intel.com -- Speaking for myself only. // // Dipl.-Inform. Alexander Zimmermann // Department of Computer Science, Informatik 4 // RWTH Aachen University // Ahornstr. 55, 52056 Aachen, Germany // phone: (49-241) 80-21422, fax: (49-241) 80-22220 // email: zimmermann@cs.rwth-aachen.de // web: http://www.umic-mesh.net // -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 22. feb. 2010 16:17, Andi Kleen wrote: > Lars Eggert <lars.eggert@nokia.com> writes: > >> If it's off by default there is no issue. > > Still a sufficient large number of people might turn it on > and then the network would melt. > > IMHO TCP changes with badly understood congestion behaviour > should not be merged until the necessary analysis/simulation etc. > is done. > > -Andi Graphs from our experiments can be found here: http://simula.no/research/nd/publications/Simula.nd.477/simula_pdf_file Section 5.2.5 covers experiments on fairness-issues. We have found no adverse effects of the modifications submitted in this patch set (mFR and LT in the plot), but further testing cannot hurt. Best regards, Andreas -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 22. feb. 2010 16:17, Andi Kleen wrote: > Lars Eggert <lars.eggert@nokia.com> writes: > >> If it's off by default there is no issue. > > Still a sufficient large number of people might turn it on > and then the network would melt. > > IMHO TCP changes with badly understood congestion behaviour > should not be merged until the necessary analysis/simulation etc. > is done. > > -Andi > I also encourage you to read the discussions from the first posted version of the patch set, since I here reference a lot of the supporting research and related material from other researchers. http://thread.gmane.org/gmane.linux.network/141394 http://thread.gmane.org/gmane.linux.network/141395 http://thread.gmane.org/gmane.linux.network/141396 http://thread.gmane.org/gmane.linux.network/141397 Cheers, Andreas -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 22 Feb 2010 16:17:33 +0100, Andi Kleen <andi@firstfloor.org> wrote: > Lars Eggert <lars.eggert@nokia.com> writes: > >> If it's off by default there is no issue. > > Still a sufficient large number of people might turn it on > and then the network would melt. Then we should apply the same policy as used for the congestion control algorithms!? Restrict the usage to change the standard behavior via Kconfig (e.g. menuconfig TCP_CONG_ADVANCED) or even more restricted demand for CAP_NET_ADMIN. Hagen -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Andi Kleen <andi@firstfloor.org> Date: Mon, 22 Feb 2010 16:17:33 +0100 > Lars Eggert <lars.eggert@nokia.com> writes: > >> If it's off by default there is no issue. > > Still a sufficient large number of people might turn it on > and then the network would melt. And they can do the same by changing the initial congestion window setting, which we also support. The foundation of your argument is as solid as quicksand. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/Documentation/networking/tcp-thin.txt b/Documentation/networking/tcp-thin.txt new file mode 100644 index 0000000..151e229 --- /dev/null +++ b/Documentation/networking/tcp-thin.txt @@ -0,0 +1,47 @@ +Thin-streams and TCP +==================== +A wide range of Internet-based services that use reliable transport +protocols display what we call thin-stream properties. This means +that the application sends data with such a low rate that the +retransmission mechanisms of the transport protocol are not fully +effective. In time-dependent scenarios (like online games, control +systems, stock trading etc.) where the user experience depends +on the data delivery latency, packet loss can be devastating for +the service quality. Extreme latencies are caused by TCP's +dependency on the arrival of new data from the application to trigger +retransmissions effectively through fast retransmit instead of +waiting for long timeouts. + +After analysing a large number of time-dependent interactive +applications, we have seen that they often produce thin streams +and also stay with this traffic pattern throughout its entire +lifespan. The combination of time-dependency and the fact that the +streams provoke high latencies when using TCP is unfortunate. + +In order to reduce application-layer latency when packets are lost, +a set of mechanisms has been made, which address these latency issues +for thin streams. In short, if the kernel detects a thin stream, +the retransmission mechanisms are modified in the following manner: + +1) If the stream is thin, fast retransmit on the first dupACK. +2) If the stream is thin, do not apply exponential backoff. + +These enhancements are applied only if the stream is detected as +thin. This is accomplished by defining a threshold for the number +of packets in flight. If there are less than 4 packets in flight, +fast retransmissions can not be triggered, and the stream is prone +to experience high retransmission latencies. + +Since these mechanisms are targeted at time-dependent applications, +they must be specifically activated by the application using the +TCP_THIN_LINEAR_TIMEOUTS and TCP_THIN_DUPACK IOCTLS or the +tcp_thin_linear_timeouts and tcp_thin_dupack sysctls. Both +modifications are turned off by default. + +References +========== +More information on the modifications, as well as a wide range of +experimental data can be found here: +"Improving latency for interactive, thin-stream applications over +reliable transport" +http://simula.no/research/nd/publications/Simula.nd.477/simula_pdf_file diff --git a/include/net/tcp.h b/include/net/tcp.h index 87d164b..e5e2056 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -1386,6 +1386,13 @@ static inline void tcp_highest_sack_combine(struct sock *sk, tcp_sk(sk)->highest_sack = new; } +/* Determines whether this is a thin stream (which may suffer from + * increased latency). Used to trigger latency-reducing mechanisms.*/ +static inline unsigned int tcp_stream_is_thin(struct tcp_sock *tp) +{ + return tp->packets_out < 4 && !tcp_in_initial_slowstart(tp); +} + /* /proc */ enum tcp_seq_states { TCP_SEQ_STATE_LISTENING,
Major changes: Added thin-stream info in new file: Documentation/networking/tcp-thin.txt Signed-off-by: Andreas Petlund <apetlund@simula.no> --- Documentation/networking/tcp-thin.txt | 47 +++++++++++++++++++++++++++++++++ include/net/tcp.h | 7 +++++ 2 files changed, 54 insertions(+), 0 deletions(-) create mode 100644 Documentation/networking/tcp-thin.txt