| Message ID | 20090518065629.GA6006@ff.dom.local |
|---|---|
| State | Not Applicable, archived |
| Delegated to: | David Miller |
| Headers | show |
I'm not sure if I'm able to test this patch. What do you mean with "smallest sizes"? Are you talking about packet's size? What kind of sizes? When I feed my bridge with 950Mbits/s of packets with 800 bytes that is close to 150.000pps and CPUs start to get busy. For packets 100 bytes long, 150.000pps would be close to 125Mbits/s and CPUs start to get busy already, so I'm not able to get close to 500Mbits/s. For rates near 125bits/s the bad accuracy is not so expressive. For packets of 100 bytes increasing analyser sent traffic, at some point is not HTB shaping but the CPU that can't process so many packets. I might misunderstood your point. I applied this tc_core.c patch and for packets of 800 bytes it had no effect in HTB accuracy with rates over 500Mbit. Anyway I also test it with packets of 100 bytes, generating 200Mbits, and the result is the same as without this patch: With the patch: class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate 100000Kbit ceil 100000Kbit burst 14087b/8 mpu 0b overhead 0b cburst 14087b/8 mpu 0b overhead 0b level 0 Sent 2187884640 bytes 22790465 pkt (dropped 8624566, overlimits 0 requeues 0) rate 124946Kbit 162691pps backlog 0b 0p requeues 0 lended: 22790465 borrowed: 0 giants: 0 tokens: 180 ctokens: 180 Without the patch: class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate 100000Kbit ceil 100000Kbit burst 14087b/8 mpu 0b overhead 0b cburst 14087b/8 mpu 0b overhead 0b level 0 Sent 1260235680 bytes 13127455 pkt (dropped 4531299, overlimits 0 requeues 0) rate 124575Kbit 162207pps backlog 0b 0p requeues 0 lended: 13127455 borrowed: 0 giants: 0 tokens: 123 ctokens: 123 Thanks Antonio Almeida On Mon, May 18, 2009 at 7:56 AM, Jarek Poplawski <jarkao2@gmail.com> wrote: > Return non-zero tc_calc_xmittime() for rate tables > > While looking at the problem of HTB accuracy for high speed (~500Mbit > rates) I've found that rate tables have cells filled with zeros for > the smallest sizes. It means such packets aren't accounted at all. > Apart from the correctness of such configs, let's make it safe with > rather overaccounting than living it unlimited. > > Reported-by: Antonio Almeida <vexwek@gmail.com> > Signed-off-by: Jarek Poplawski <jarkao2@gmail.com> > --- > > tc/tc_core.c | 4 +++- > 1 files changed, 3 insertions(+), 1 deletions(-) > > diff --git a/tc/tc_core.c b/tc/tc_core.c > index 9a0ff39..14f25bc 100644 > --- a/tc/tc_core.c > +++ b/tc/tc_core.c > @@ -58,7 +58,9 @@ unsigned tc_core_ktime2time(unsigned ktime) > > unsigned tc_calc_xmittime(unsigned rate, unsigned size) > { > - return tc_core_time2tick(TIME_UNITS_PER_SEC*((double)size/rate)); > + unsigned t; > + t = tc_core_time2tick(TIME_UNITS_PER_SEC*((double)size/rate)); > + return t ? : 1; > } > > unsigned tc_calc_xmitsize(unsigned rate, unsigned ticks) > -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
I forgot to tell you that I used tc source code from iproute2-2.6.16. I couldn't use the newest version because I got errors when compiling. Antonio Almeida On Mon, May 18, 2009 at 5:54 PM, Antonio Almeida <vexwek@gmail.com> wrote: > I'm not sure if I'm able to test this patch. What do you mean with > "smallest sizes"? Are you talking about packet's size? What kind of > sizes? > When I feed my bridge with 950Mbits/s of packets with 800 bytes that > is close to 150.000pps and CPUs start to get busy. For packets 100 > bytes long, 150.000pps would be close to 125Mbits/s and CPUs start to > get busy already, so I'm not able to get close to 500Mbits/s. For > rates near 125bits/s the bad accuracy is not so expressive. For > packets of 100 bytes increasing analyser sent traffic, at some point > is not HTB shaping but the CPU that can't process so many packets. I > might misunderstood your point. > > I applied this tc_core.c patch and for packets of 800 bytes it had no > effect in HTB accuracy with rates over 500Mbit. > Anyway I also test it with packets of 100 bytes, generating 200Mbits, > and the result is the same as without this patch: > > With the patch: > class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > 100000Kbit ceil 100000Kbit burst 14087b/8 mpu 0b overhead 0b cburst > 14087b/8 mpu 0b overhead 0b level 0 > Sent 2187884640 bytes 22790465 pkt (dropped 8624566, overlimits 0 requeues 0) > rate 124946Kbit 162691pps backlog 0b 0p requeues 0 > lended: 22790465 borrowed: 0 giants: 0 > tokens: 180 ctokens: 180 > > > Without the patch: > class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > 100000Kbit ceil 100000Kbit burst 14087b/8 mpu 0b overhead 0b cburst > 14087b/8 mpu 0b overhead 0b level 0 > Sent 1260235680 bytes 13127455 pkt (dropped 4531299, overlimits 0 requeues 0) > rate 124575Kbit 162207pps backlog 0b 0p requeues 0 > lended: 13127455 borrowed: 0 giants: 0 > tokens: 123 ctokens: 123 > > > Thanks > Antonio Almeida > > > On Mon, May 18, 2009 at 7:56 AM, Jarek Poplawski <jarkao2@gmail.com> wrote: >> Return non-zero tc_calc_xmittime() for rate tables >> >> While looking at the problem of HTB accuracy for high speed (~500Mbit >> rates) I've found that rate tables have cells filled with zeros for >> the smallest sizes. It means such packets aren't accounted at all. >> Apart from the correctness of such configs, let's make it safe with >> rather overaccounting than living it unlimited. >> >> Reported-by: Antonio Almeida <vexwek@gmail.com> >> Signed-off-by: Jarek Poplawski <jarkao2@gmail.com> >> --- >> >> tc/tc_core.c | 4 +++- >> 1 files changed, 3 insertions(+), 1 deletions(-) >> >> diff --git a/tc/tc_core.c b/tc/tc_core.c >> index 9a0ff39..14f25bc 100644 >> --- a/tc/tc_core.c >> +++ b/tc/tc_core.c >> @@ -58,7 +58,9 @@ unsigned tc_core_ktime2time(unsigned ktime) >> >> unsigned tc_calc_xmittime(unsigned rate, unsigned size) >> { >> - return tc_core_time2tick(TIME_UNITS_PER_SEC*((double)size/rate)); >> + unsigned t; >> + t = tc_core_time2tick(TIME_UNITS_PER_SEC*((double)size/rate)); >> + return t ? : 1; >> } >> >> unsigned tc_calc_xmitsize(unsigned rate, unsigned ticks) >> > -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, May 18, 2009 at 05:54:18PM +0100, Antonio Almeida wrote: > I'm not sure if I'm able to test this patch. What do you mean with > "smallest sizes"? Are you talking about packet's size? What kind of > sizes? > When I feed my bridge with 950Mbits/s of packets with 800 bytes that > is close to 150.000pps and CPUs start to get busy. For packets 100 > bytes long, 150.000pps would be close to 125Mbits/s and CPUs start to > get busy already, so I'm not able to get close to 500Mbits/s. For > rates near 125bits/s the bad accuracy is not so expressive. For > packets of 100 bytes increasing analyser sent traffic, at some point > is not HTB shaping but the CPU that can't process so many packets. I > might misunderstood your point. > > I applied this tc_core.c patch and for packets of 800 bytes it had no > effect in HTB accuracy with rates over 500Mbit. > Anyway I also test it with packets of 100 bytes, generating 200Mbits, > and the result is the same as without this patch: You're right: if there were only 800 byte packets this patch shouldn't matter. It should matter e.g. if these 800 byte were mixed with 100 byte packets, rate 550Mbit, and HZ 1000. Btw. if could you send your .config (gzipped)? I guess, I've to look for some other reason yet. Thanks, Jarek P. > > With the patch: > class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > 100000Kbit ceil 100000Kbit burst 14087b/8 mpu 0b overhead 0b cburst > 14087b/8 mpu 0b overhead 0b level 0 > Sent 2187884640 bytes 22790465 pkt (dropped 8624566, overlimits 0 requeues 0) > rate 124946Kbit 162691pps backlog 0b 0p requeues 0 > lended: 22790465 borrowed: 0 giants: 0 > tokens: 180 ctokens: 180 > > > Without the patch: > class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > 100000Kbit ceil 100000Kbit burst 14087b/8 mpu 0b overhead 0b cburst > 14087b/8 mpu 0b overhead 0b level 0 > Sent 1260235680 bytes 13127455 pkt (dropped 4531299, overlimits 0 requeues 0) > rate 124575Kbit 162207pps backlog 0b 0p requeues 0 > lended: 13127455 borrowed: 0 giants: 0 > tokens: 123 ctokens: 123 > > > Thanks > Antonio Almeida > > > On Mon, May 18, 2009 at 7:56 AM, Jarek Poplawski <jarkao2@gmail.com> wrote: > > Return non-zero tc_calc_xmittime() for rate tables > > > > While looking at the problem of HTB accuracy for high speed (~500Mbit > > rates) I've found that rate tables have cells filled with zeros for > > the smallest sizes. It means such packets aren't accounted at all. > > Apart from the correctness of such configs, let's make it safe with > > rather overaccounting than living it unlimited. > > > > Reported-by: Antonio Almeida <vexwek@gmail.com> > > Signed-off-by: Jarek Poplawski <jarkao2@gmail.com> > > --- > > > > tc/tc_core.c | 4 +++- > > 1 files changed, 3 insertions(+), 1 deletions(-) > > > > diff --git a/tc/tc_core.c b/tc/tc_core.c > > index 9a0ff39..14f25bc 100644 > > --- a/tc/tc_core.c > > +++ b/tc/tc_core.c > > @@ -58,7 +58,9 @@ unsigned tc_core_ktime2time(unsigned ktime) > > > > unsigned tc_calc_xmittime(unsigned rate, unsigned size) > > { > > - return tc_core_time2tick(TIME_UNITS_PER_SEC*((double)size/rate)); > > + unsigned t; > > + t = tc_core_time2tick(TIME_UNITS_PER_SEC*((double)size/rate)); > > + return t ? : 1; > > } > > > > unsigned tc_calc_xmitsize(unsigned rate, unsigned ticks) > > -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Here's my .config Antonio Almeida On Mon, May 18, 2009 at 6:53 PM, Jarek Poplawski <jarkao2@gmail.com> wrote: > On Mon, May 18, 2009 at 05:54:18PM +0100, Antonio Almeida wrote: >> I'm not sure if I'm able to test this patch. What do you mean with >> "smallest sizes"? Are you talking about packet's size? What kind of >> sizes? >> When I feed my bridge with 950Mbits/s of packets with 800 bytes that >> is close to 150.000pps and CPUs start to get busy. For packets 100 >> bytes long, 150.000pps would be close to 125Mbits/s and CPUs start to >> get busy already, so I'm not able to get close to 500Mbits/s. For >> rates near 125bits/s the bad accuracy is not so expressive. For >> packets of 100 bytes increasing analyser sent traffic, at some point >> is not HTB shaping but the CPU that can't process so many packets. I >> might misunderstood your point. >> >> I applied this tc_core.c patch and for packets of 800 bytes it had no >> effect in HTB accuracy with rates over 500Mbit. >> Anyway I also test it with packets of 100 bytes, generating 200Mbits, >> and the result is the same as without this patch: > > You're right: if there were only 800 byte packets this patch shouldn't > matter. It should matter e.g. if these 800 byte were mixed with 100 > byte packets, rate 550Mbit, and HZ 1000. Btw. if could you send your > .config (gzipped)? I guess, I've to look for some other reason yet. > > Thanks, > Jarek P. > >> >> With the patch: >> class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate >> 100000Kbit ceil 100000Kbit burst 14087b/8 mpu 0b overhead 0b cburst >> 14087b/8 mpu 0b overhead 0b level 0 >> Sent 2187884640 bytes 22790465 pkt (dropped 8624566, overlimits 0 requeues 0) >> rate 124946Kbit 162691pps backlog 0b 0p requeues 0 >> lended: 22790465 borrowed: 0 giants: 0 >> tokens: 180 ctokens: 180 >> >> >> Without the patch: >> class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate >> 100000Kbit ceil 100000Kbit burst 14087b/8 mpu 0b overhead 0b cburst >> 14087b/8 mpu 0b overhead 0b level 0 >> Sent 1260235680 bytes 13127455 pkt (dropped 4531299, overlimits 0 requeues 0) >> rate 124575Kbit 162207pps backlog 0b 0p requeues 0 >> lended: 13127455 borrowed: 0 giants: 0 >> tokens: 123 ctokens: 123 >> >> >> Thanks >> Antonio Almeida >> >> >> On Mon, May 18, 2009 at 7:56 AM, Jarek Poplawski <jarkao2@gmail.com> wrote: >> > Return non-zero tc_calc_xmittime() for rate tables >> > >> > While looking at the problem of HTB accuracy for high speed (~500Mbit >> > rates) I've found that rate tables have cells filled with zeros for >> > the smallest sizes. It means such packets aren't accounted at all. >> > Apart from the correctness of such configs, let's make it safe with >> > rather overaccounting than living it unlimited. >> > >> > Reported-by: Antonio Almeida <vexwek@gmail.com> >> > Signed-off-by: Jarek Poplawski <jarkao2@gmail.com> >> > --- >> > >> > tc/tc_core.c | 4 +++- >> > 1 files changed, 3 insertions(+), 1 deletions(-) >> > >> > diff --git a/tc/tc_core.c b/tc/tc_core.c >> > index 9a0ff39..14f25bc 100644 >> > --- a/tc/tc_core.c >> > +++ b/tc/tc_core.c >> > @@ -58,7 +58,9 @@ unsigned tc_core_ktime2time(unsigned ktime) >> > >> > unsigned tc_calc_xmittime(unsigned rate, unsigned size) >> > { >> > - return tc_core_time2tick(TIME_UNITS_PER_SEC*((double)size/rate)); >> > + unsigned t; >> > + t = tc_core_time2tick(TIME_UNITS_PER_SEC*((double)size/rate)); >> > + return t ? : 1; >> > } >> > >> > unsigned tc_calc_xmitsize(unsigned rate, unsigned ticks) >> > >
On Mon, May 18, 2009 at 07:23:14PM +0100, Antonio Almeida wrote:
> Here's my .config
Hmm... And if it's not a big problem I'd also ask you to try this test
with 555000Kbit rate for 850 and 900 byte packets. (It can wait.)
Thanks again,
Jarek P.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Precise measurements: 800 bytes: class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst 70901b/8 mpu 0b overhead 0b level 0 Sent 46793626324 bytes 57771194 pkt (dropped 29920019, overlimits 0 requeues 0) rate 621714Kbit 97631pps backlog 0b 126p requeues 0 lended: 57771068 borrowed: 0 giants: 0 tokens: -8 ctokens: -8 850 bytes: class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst 70901b/8 mpu 0b overhead 0b level 0 Sent 63422144616 bytes 77714246 pkt (dropped 41012275, overlimits 0 requeues 0) rate 600699Kbit 88756pps backlog 0b 127p requeues 0 lended: 77714119 borrowed: 0 giants: 0 tokens: -11 ctokens: -11 900 bytes: class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst 70901b/8 mpu 0b overhead 0b level 0 Sent 76868403562 bytes 92835297 pkt (dropped 48565133, overlimits 0 requeues 0) rate 636195Kbit 88755pps backlog 0b 126p requeues 0 lended: 92835171 borrowed: 0 giants: 0 tokens: -7 ctokens: -7 If you need more values you're free to ask. Antonio Almeida On Mon, May 18, 2009 at 7:32 PM, Jarek Poplawski <jarkao2@gmail.com> wrote: > On Mon, May 18, 2009 at 07:23:14PM +0100, Antonio Almeida wrote: >> Here's my .config > > Hmm... And if it's not a big problem I'd also ask you to try this test > with 555000Kbit rate for 850 and 900 byte packets. (It can wait.) > > Thanks again, > Jarek P. > -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, May 18, 2009 at 07:56:12PM +0100, Antonio Almeida wrote: > Precise measurements: > > 800 bytes: > class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst > 70901b/8 mpu 0b overhead 0b level 0 > Sent 46793626324 bytes 57771194 pkt (dropped 29920019, overlimits 0 requeues 0) > rate 621714Kbit 97631pps backlog 0b 126p requeues 0 > lended: 57771068 borrowed: 0 giants: 0 > tokens: -8 ctokens: -8 > > > 850 bytes: > class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst > 70901b/8 mpu 0b overhead 0b level 0 > Sent 63422144616 bytes 77714246 pkt (dropped 41012275, overlimits 0 requeues 0) > rate 600699Kbit 88756pps backlog 0b 127p requeues 0 > lended: 77714119 borrowed: 0 giants: 0 > tokens: -11 ctokens: -11 > > > 900 bytes: > class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst > 70901b/8 mpu 0b overhead 0b level 0 > Sent 76868403562 bytes 92835297 pkt (dropped 48565133, overlimits 0 requeues 0) > rate 636195Kbit 88755pps backlog 0b 126p requeues 0 > lended: 92835171 borrowed: 0 giants: 0 > tokens: -7 ctokens: -7 > > > If you need more values you're free to ask. Since you're so kind... :-) There is a line in net/sched/sch_htb.c: #define HTB_HYSTERESIS 1 /* whether to use mode hysteresis for speedup */ Could you change 1 to 0, and repeat these tests above after recompiling? More thanks, Jarek P. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Doesn't seem to make any diference seting HTB_HYSTERESIS to 0. Here're the values using #define HTB_HYSTERESIS 0 800 bytes: class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst 70901b/8 mpu 0b overhead 0b level 0 Sent 9773257752 bytes 12277962 pkt (dropped 6292541, overlimits 0 requeues 0) rate 621796Kbit 97644pps backlog 0b 127p requeues 0 lended: 12277835 borrowed: 0 giants: 0 tokens: -7 ctokens: -7 850 bytes: class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst 70901b/8 mpu 0b overhead 0b level 0 Sent 18225005732 bytes 22409017 pkt (dropped 11937269, overlimits 0 requeues 0) rate 600890Kbit 88796pps backlog 0b 43p requeues 0 lended: 22408974 borrowed: 0 giants: 0 tokens: -2 ctokens: -2 900 bytes: class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst 70901b/8 mpu 0b overhead 0b level 0 Sent 29790867368 bytes 35400708 pkt (dropped 18399726, overlimits 0 requeues 0) rate 636361Kbit 88779pps backlog 0b 127p requeues 0 lended: 35400581 borrowed: 0 giants: 0 tokens: -2 ctokens: -2 Antonio Almeida On Mon, May 18, 2009 at 8:05 PM, Jarek Poplawski <jarkao2@gmail.com> wrote: > On Mon, May 18, 2009 at 07:56:12PM +0100, Antonio Almeida wrote: >> Precise measurements: >> >> 800 bytes: >> class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate >> 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst >> 70901b/8 mpu 0b overhead 0b level 0 >> Sent 46793626324 bytes 57771194 pkt (dropped 29920019, overlimits 0 requeues 0) >> rate 621714Kbit 97631pps backlog 0b 126p requeues 0 >> lended: 57771068 borrowed: 0 giants: 0 >> tokens: -8 ctokens: -8 >> >> >> 850 bytes: >> class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate >> 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst >> 70901b/8 mpu 0b overhead 0b level 0 >> Sent 63422144616 bytes 77714246 pkt (dropped 41012275, overlimits 0 requeues 0) >> rate 600699Kbit 88756pps backlog 0b 127p requeues 0 >> lended: 77714119 borrowed: 0 giants: 0 >> tokens: -11 ctokens: -11 >> >> >> 900 bytes: >> class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate >> 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst >> 70901b/8 mpu 0b overhead 0b level 0 >> Sent 76868403562 bytes 92835297 pkt (dropped 48565133, overlimits 0 requeues 0) >> rate 636195Kbit 88755pps backlog 0b 126p requeues 0 >> lended: 92835171 borrowed: 0 giants: 0 >> tokens: -7 ctokens: -7 >> >> >> If you need more values you're free to ask. > > Since you're so kind... :-) There is a line in net/sched/sch_htb.c: > > #define HTB_HYSTERESIS 1 /* whether to use mode hysteresis for speedup */ > > Could you change 1 to 0, and repeat these tests above after recompiling? > > More thanks, > Jarek P. > -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tuesday 19 May 2009 13:55:43 Antonio Almeida wrote: > Doesn't seem to make any diference seting HTB_HYSTERESIS to 0. Here're > the values using #define HTB_HYSTERESIS 0 > > 800 bytes: > class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst > 70901b/8 mpu 0b overhead 0b level 0 > Sent 9773257752 bytes 12277962 pkt (dropped 6292541, overlimits 0 requeues > 0) rate 621796Kbit 97644pps backlog 0b 127p requeues 0 > lended: 12277835 borrowed: 0 giants: 0 > tokens: -7 ctokens: -7 6292541 dropped from 12277962 pkt, means 51% dropped. Maybe something fishy here? Can you try instead of SFQ - BFIFO? For 100ms buffer, 550Mbit/s it will be ~6875000 bytes bfifo. It is by the way too short, IMHO, for this bandwidth, 127 packets is not enough. 127 packets with 800 bytes can buffer 1 second for 812Kbit/s only, and for 550Mbit/s it will buffer data for ~2ms only. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 19, 2009 at 11:55:43AM +0100, Antonio Almeida wrote: > Doesn't seem to make any diference seting HTB_HYSTERESIS to 0. Here're > the values using #define HTB_HYSTERESIS 0 OK, so it looks like some hidden bug yet. Many thanks for now, Jarek P. > > 800 bytes: > class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst > 70901b/8 mpu 0b overhead 0b level 0 > Sent 9773257752 bytes 12277962 pkt (dropped 6292541, overlimits 0 requeues 0) > rate 621796Kbit 97644pps backlog 0b 127p requeues 0 > lended: 12277835 borrowed: 0 giants: 0 > tokens: -7 ctokens: -7 > > 850 bytes: > class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst > 70901b/8 mpu 0b overhead 0b level 0 > Sent 18225005732 bytes 22409017 pkt (dropped 11937269, overlimits 0 requeues 0) > rate 600890Kbit 88796pps backlog 0b 43p requeues 0 > lended: 22408974 borrowed: 0 giants: 0 > tokens: -2 ctokens: -2 > > 900 bytes: > class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst > 70901b/8 mpu 0b overhead 0b level 0 > Sent 29790867368 bytes 35400708 pkt (dropped 18399726, overlimits 0 requeues 0) > rate 636361Kbit 88779pps backlog 0b 127p requeues 0 > lended: 35400581 borrowed: 0 giants: 0 > tokens: -2 ctokens: -2 > > > Antonio Almeida > > > > On Mon, May 18, 2009 at 8:05 PM, Jarek Poplawski <jarkao2@gmail.com> wrote: > > On Mon, May 18, 2009 at 07:56:12PM +0100, Antonio Almeida wrote: > >> Precise measurements: > >> > >> 800 bytes: > >> class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > >> 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst > >> 70901b/8 mpu 0b overhead 0b level 0 > >> Sent 46793626324 bytes 57771194 pkt (dropped 29920019, overlimits 0 requeues 0) > >> rate 621714Kbit 97631pps backlog 0b 126p requeues 0 > >> lended: 57771068 borrowed: 0 giants: 0 > >> tokens: -8 ctokens: -8 > >> > >> > >> 850 bytes: > >> class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > >> 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst > >> 70901b/8 mpu 0b overhead 0b level 0 > >> Sent 63422144616 bytes 77714246 pkt (dropped 41012275, overlimits 0 requeues 0) > >> rate 600699Kbit 88756pps backlog 0b 127p requeues 0 > >> lended: 77714119 borrowed: 0 giants: 0 > >> tokens: -11 ctokens: -11 > >> > >> > >> 900 bytes: > >> class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > >> 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst > >> 70901b/8 mpu 0b overhead 0b level 0 > >> Sent 76868403562 bytes 92835297 pkt (dropped 48565133, overlimits 0 requeues 0) > >> rate 636195Kbit 88755pps backlog 0b 126p requeues 0 > >> lended: 92835171 borrowed: 0 giants: 0 > >> tokens: -7 ctokens: -7 > >> > >> > >> If you need more values you're free to ask. > > > > Since you're so kind... :-) There is a line in net/sched/sch_htb.c: > > > > #define HTB_HYSTERESIS 1 /* whether to use mode hysteresis for speedup */ > > > > Could you change 1 to 0, and repeat these tests above after recompiling? > > > > More thanks, > > Jarek P. > > -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 19, 2009 at 02:04:50PM +0300, Denys Fedoryschenko wrote: > On Tuesday 19 May 2009 13:55:43 Antonio Almeida wrote: > > Doesn't seem to make any diference seting HTB_HYSTERESIS to 0. Here're > > the values using #define HTB_HYSTERESIS 0 > > > > 800 bytes: > > class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate > > 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst > > 70901b/8 mpu 0b overhead 0b level 0 > > Sent 9773257752 bytes 12277962 pkt (dropped 6292541, overlimits 0 requeues > > 0) rate 621796Kbit 97644pps backlog 0b 127p requeues 0 > > lended: 12277835 borrowed: 0 giants: 0 > > tokens: -7 ctokens: -7 > 6292541 dropped from 12277962 pkt, means 51% dropped. Maybe something fishy > here? > > Can you try instead of SFQ - BFIFO? For 100ms buffer, 550Mbit/s it will be > ~6875000 bytes bfifo. > > It is by the way too short, IMHO, for this bandwidth, 127 packets is not > enough. 127 packets with 800 bytes can buffer 1 second for 812Kbit/s only, > and for 550Mbit/s it will buffer data for ~2ms only. > Sure, if the queue is too short we could have a problem with reaching the expected rate; but here it's all backwards - it could actually "help" with the stats. ;-) Jarek P. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tuesday 19 May 2009 14:18:57 Jarek Poplawski wrote: > > Sure, if the queue is too short we could have a problem with reaching > the expected rate; but here it's all backwards - it could actually > "help" with the stats. ;-) > > Jarek P. Well, i had real experience on HTB, when i set too short buffers on my QoS qdiscs, the incoming rate jumped too high than overall. When i set larger buffers (and by the way dropped sfq and use bfifo) - it is dropped. No idea why, bug or specific things in protocols congestion control. Maybe worth to try... -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 19, 2009 at 02:21:28PM +0300, Denys Fedoryschenko wrote: > On Tuesday 19 May 2009 14:18:57 Jarek Poplawski wrote: > > > > Sure, if the queue is too short we could have a problem with reaching > > the expected rate; but here it's all backwards - it could actually > > "help" with the stats. ;-) > > > > Jarek P. > Well, i had real experience on HTB, when i set too short buffers on my QoS > qdiscs, the incoming rate jumped too high than overall. When i set larger > buffers (and by the way dropped sfq and use bfifo) - it is dropped. No idea > why, bug or specific things in protocols congestion control. Maybe worth to > try... > Very strange. Anyway, "overlimits 0" suggests HTB always got packets when it needed... Jarek P. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, 18 May 2009, Jarek Poplawski wrote: > Since you're so kind... :-) There is a line in net/sched/sch_htb.c: > > #define HTB_HYSTERESIS 1 /* whether to use mode hysteresis for speedup */ > > Could you change 1 to 0, and repeat these tests above after recompiling? Notice its runtime adjustable via: /sys/module/sch_htb/parameters/htb_hysteresis Since kernel version v2.6.26. Cheers, Jesper Brouer -- ------------------------------------------------------------------- MSc. Master of Computer Science Dept. of Computer Science, University of Copenhagen Author of http://www.adsl-optimizer.dk ------------------------------------------------------------------- -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
I tested it with BFIFO using limit 6875000. (Analyser keeps sending 950Mbits/s of 800 bytes tcp packets - lots of drops for sure) Backlog is now huge but the throughout stays much higher than the configured ceil. # tc -s -d class ls dev eth1 class htb 1:10 parent 1:2 rate 900000Kbit ceil 900000Kbit burst 113962b/8 mpu 0b overhead 0b cburst 113962b/8 mpu 0b overhead 0b level 5 Sent 9542831672 bytes 11988482 pkt (dropped 0, overlimits 0 requeues 0) rate 621765Kbit 97639pps backlog 0b 0p requeues 0 lended: 0 borrowed: 0 giants: 0 tokens: -186 ctokens: -186 class htb 1:1 root rate 900000Kbit ceil 900000Kbit burst 113962b/8 mpu 0b overhead 0b cburst 113962b/8 mpu 0b overhead 0b level 7 Sent 9542831672 bytes 11988482 pkt (dropped 0, overlimits 0 requeues 0) rate 621765Kbit 97639pps backlog 0b 0p requeues 0 lended: 0 borrowed: 0 giants: 0 tokens: -186 ctokens: -186 class htb 1:2 parent 1:1 rate 900000Kbit ceil 900000Kbit burst 113962b/8 mpu 0b overhead 0b cburst 113962b/8 mpu 0b overhead 0b level 6 Sent 9542831672 bytes 11988482 pkt (dropped 0, overlimits 0 requeues 0) rate 621765Kbit 97639pps backlog 0b 0p requeues 0 lended: 0 borrowed: 0 giants: 0 tokens: -186 ctokens: -186 class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst 70901b/8 mpu 0b overhead 0b level 0 Sent 9549705928 bytes 11997118 pkt (dropped 6092846, overlimits 0 requeues 0) rate 621764Kbit 97639pps backlog 0b 8636p requeues 0 lended: 11988482 borrowed: 0 giants: 0 tokens: -1008 ctokens: -1008 # tc -s -d qdisc ls dev eth1 qdisc htb 1: root r2q 10 default 0 direct_packets_stat 11955 ver 3.17 Sent 9608660872 bytes 12071182 pkt (dropped 6124502, overlimits 18190041 requeues 0) rate 0bit 0pps backlog 0b 8636p requeues 0 qdisc bfifo 108: parent 1:108 limit 6875000b Sent 9599144692 bytes 12059227 pkt (dropped 6124502, overlimits 0 requeues 0) rate 0bit 0pps backlog 6874256b 8636p requeues 0 Antonio Almeida On Tue, May 19, 2009 at 12:28 PM, Jarek Poplawski <jarkao2@gmail.com> wrote: > On Tue, May 19, 2009 at 02:21:28PM +0300, Denys Fedoryschenko wrote: >> On Tuesday 19 May 2009 14:18:57 Jarek Poplawski wrote: >> > >> > Sure, if the queue is too short we could have a problem with reaching >> > the expected rate; but here it's all backwards - it could actually >> > "help" with the stats. ;-) >> > >> > Jarek P. >> Well, i had real experience on HTB, when i set too short buffers on my QoS >> qdiscs, the incoming rate jumped too high than overall. When i set larger >> buffers (and by the way dropped sfq and use bfifo) - it is dropped. No idea >> why, bug or specific things in protocols congestion control. Maybe worth to >> try... >> > > Very strange. Anyway, "overlimits 0" suggests HTB always got packets > when it needed... > > Jarek P. > -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, May 19, 2009 at 03:18:47PM +0200, Jesper Dangaard Brouer wrote: > > On Mon, 18 May 2009, Jarek Poplawski wrote: > >> Since you're so kind... :-) There is a line in net/sched/sch_htb.c: >> >> #define HTB_HYSTERESIS 1 /* whether to use mode hysteresis for speedup */ >> >> Could you change 1 to 0, and repeat these tests above after recompiling? > > Notice its runtime adjustable via: > /sys/module/sch_htb/parameters/htb_hysteresis > > Since kernel version v2.6.26. Yes, this should convince Antonio to try something newer. (Alas it didn't seem to make much difference to his case ;-) Cheers, Jarek P. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, May 18, 2009 at 06:16:26PM +0100, Antonio Almeida wrote: > I forgot to tell you that I used tc source code from iproute2-2.6.16. > I couldn't use the newest version because I got errors when compiling. I still have no clue about the reason, but it would be really nice to do some short test with more current kernel (>= 2.6.27) and iproute2 (to exclude the possibility of some incomaptibility in configs e.g. rate tables passed to htb). Thanks, Jarek P. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, May 21, 2009 at 9:51 AM, Jarek Poplawski wrote: > I still have no clue about the reason, but it would be really nice to > do some short test with more current kernel (>= 2.6.27) and iproute2 > (to exclude the possibility of some incomaptibility in configs e.g. > rate tables passed to htb). I installed kernel 2.6.29 (finaly! wasn't easy... I couldn't set memory split 2G/2G), but the results are the same. I've already applied gen_estimator.c patches (works fine). # tc -s -d class ls dev eth1 | head -24 class htb 1:1 root rate 900000Kbit ceil 900000Kbit burst 113962b/8 mpu 0b overhead 0b cburst 113962b/8 mpu 0b overhead 0b level 7 Sent 119955303928 bytes 150697618 pkt (dropped 0, overlimits 0 requeues 0) rate 621844Kbit 97651pps backlog 0b 0p requeues 0 lended: 0 borrowed: 0 giants: 0 tokens: 402 ctokens: 402 class htb 1:10 parent 1:2 rate 900000Kbit ceil 900000Kbit burst 113962b/8 mpu 0b overhead 0b cburst 113962b/8 mpu 0b overhead 0b level 5 Sent 119955303928 bytes 150697618 pkt (dropped 0, overlimits 0 requeues 0) rate 621844Kbit 97651pps backlog 0b 0p requeues 0 lended: 0 borrowed: 0 giants: 0 tokens: 402 ctokens: 402 class htb 1:108 parent 1:10 leaf 108: prio 7 quantum 1514 rate 555000Kbit ceil 555000Kbit burst 70901b/8 mpu 0b overhead 0b cburst 70901b/8 mpu 0b overhead 0b level 0 Sent 119955366812 bytes 150697697 pkt (dropped 76696483, overlimits 0 requeues 0) rate 621847Kbit 97652pps backlog 0b 79p requeues 0 lended: 150697618 borrowed: 0 giants: 0 tokens: -5 ctokens: -5 class htb 1:2 parent 1:1 rate 900000Kbit ceil 900000Kbit burst 113962b/8 mpu 0b overhead 0b cburst 113962b/8 mpu 0b overhead 0b level 6 Sent 119955303928 bytes 150697618 pkt (dropped 0, overlimits 0 requeues 0) rate 621844Kbit 97651pps backlog 0b 0p requeues 0 lended: 0 borrowed: 0 giants: 0 tokens: 402 ctokens: 402 # cat /sys/module/sch_htb/parameters/htb_hysteresis 0 # ethtool -k eth0 Offload parameters for eth0: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: off udp fragmentation offload: off generic segmentation offload: off # ethtool -k eth1 Offload parameters for eth1: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: off udp fragmentation offload: off generic segmentation offload: off I'm working on a newer iproute2. Antonio Almeida -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, May 22, 2009 at 06:42:16PM +0100, Antonio Almeida wrote: > On Thu, May 21, 2009 at 9:51 AM, Jarek Poplawski wrote: > > I still have no clue about the reason, but it would be really nice to > > do some short test with more current kernel (>= 2.6.27) and iproute2 > > (to exclude the possibility of some incomaptibility in configs e.g. > > rate tables passed to htb). > > I installed kernel 2.6.29 (finaly! wasn't easy... I couldn't set > memory split 2G/2G), > but the results are the same. I've already applied gen_estimator.c > patches (works fine). ... > I'm working on a newer iproute2. Actually, from these two I was more interested in iproute2 more fitting the kernel version. :-((It should be enough to have at least tc compiled properly, I guess.) Btw.: if at any point you think this testing is too disturbing to you etc., feel free to stop this or delay in time as you like. Thanks, Jarek P. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Sat, May 23, 2009 at 8:32 AM, Jarek Poplawski wrote: > Actually, from these two I was more interested in iproute2 more > fitting the kernel version. :-((It should be enough to have at least > tc compiled properly, I guess.) I installed iproute2-ss090115 with the new patch but the results are the same for my test scenery. HTB keeps sending 620Mbit/s when I configure it's ceil to 555Mbit/s, with 800 bytes packets long. > Btw.: if at any point you think this testing is too disturbing to you > etc., feel free to stop this or delay in time as you like. I'm working on this, don't worry. Since I have a traffic generator/analyser, any modification you would make I can test it. You're free to ask. I've been looking inside htb source code. The granularity problem could be in the use qdisc_rate_table or near that. Antonio Almeida -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/tc/tc_core.c b/tc/tc_core.c index 9a0ff39..14f25bc 100644 --- a/tc/tc_core.c +++ b/tc/tc_core.c @@ -58,7 +58,9 @@ unsigned tc_core_ktime2time(unsigned ktime) unsigned tc_calc_xmittime(unsigned rate, unsigned size) { - return tc_core_time2tick(TIME_UNITS_PER_SEC*((double)size/rate)); + unsigned t; + t = tc_core_time2tick(TIME_UNITS_PER_SEC*((double)size/rate)); + return t ? : 1; } unsigned tc_calc_xmitsize(unsigned rate, unsigned ticks)
Return non-zero tc_calc_xmittime() for rate tables While looking at the problem of HTB accuracy for high speed (~500Mbit rates) I've found that rate tables have cells filled with zeros for the smallest sizes. It means such packets aren't accounted at all. Apart from the correctness of such configs, let's make it safe with rather overaccounting than living it unlimited. Reported-by: Antonio Almeida <vexwek@gmail.com> Signed-off-by: Jarek Poplawski <jarkao2@gmail.com> --- tc/tc_core.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html