| Message ID | 20161103135606.28737.67383.stgit@firesoul |
|---|---|
| State | Accepted, archived |
| Delegated to: | David Miller |
| Headers | show |
On Thu, Nov 03, 2016 at 02:56:06PM +0100, Jesper Dangaard Brouer wrote: > The flag IFF_NO_QUEUE marks virtual device drivers that doesn't need a > default qdisc attached, given they will be backed by physical device, > that already have a qdisc attached for pushback. > > It is still supported to attach a qdisc to a IFF_NO_QUEUE device, as > this can be useful for difference policy reasons (e.g. bandwidth > limiting containers). For this to work, the tx_queue_len need to have > a sane value, because some qdiscs inherit/copy the tx_queue_len > (namely, pfifo, bfifo, gred, htb, plug and sfb). > > Commit a813104d9233 ("IFF_NO_QUEUE: Fix for drivers not calling > ether_setup()") caught situations where some drivers didn't initialize > tx_queue_len. The problem with the commit was choosing 1 as the > fallback value. > > A qdisc queue length of 1 causes more harm than good, because it > creates hard to debug situations for userspace. It gives userspace a > false sense of a working config after attaching a qdisc. As low > volume traffic (that doesn't activate the qdisc policy) works, > like ping, while traffic that e.g. needs shaping cannot reach the > configured policy levels, given the queue length is too small. Thanks for fixing this. I've run into this in the exact scenario you describe -- bandwith limiting containers. I'm pretty sure my vote doesn't count, but I'm in favor of this change. -K
On Thu, 3 Nov 2016 13:54:40 -0700 Krister Johansen <kjlx@templeofstupid.com> wrote: > On Thu, Nov 03, 2016 at 02:56:06PM +0100, Jesper Dangaard Brouer wrote: > > The flag IFF_NO_QUEUE marks virtual device drivers that doesn't need a > > default qdisc attached, given they will be backed by physical device, > > that already have a qdisc attached for pushback. > > > > It is still supported to attach a qdisc to a IFF_NO_QUEUE device, as > > this can be useful for difference policy reasons (e.g. bandwidth > > limiting containers). For this to work, the tx_queue_len need to have > > a sane value, because some qdiscs inherit/copy the tx_queue_len > > (namely, pfifo, bfifo, gred, htb, plug and sfb). > > > > Commit a813104d9233 ("IFF_NO_QUEUE: Fix for drivers not calling > > ether_setup()") caught situations where some drivers didn't initialize > > tx_queue_len. The problem with the commit was choosing 1 as the > > fallback value. > > > > A qdisc queue length of 1 causes more harm than good, because it > > creates hard to debug situations for userspace. It gives userspace a > > false sense of a working config after attaching a qdisc. As low > > volume traffic (that doesn't activate the qdisc policy) works, > > like ping, while traffic that e.g. needs shaping cannot reach the > > configured policy levels, given the queue length is too small. > > Thanks for fixing this. I've run into this in the exact scenario you > describe -- bandwith limiting containers. I'm pretty sure my vote > doesn't count, but I'm in favor of this change. Thanks for confirming the problem. You voice is actually very important in matters like this. It is important to know if people were actually hit by this. My own story is that I was hit by this subtle queue length 1 problem approx 11 years ago without noticing. An ISP were doing qdisc shaping (with HTB) on VLAN devices. The original guy who developed the system were fired because Internet customers were not getting the bandwidth they paid for. I were hired to fix the problem, and unknowingly fixed it (and bufferbloat) by using SFQ instead of pfifo_fast as leaf qdisc. I actually didn't realize the root-cause until Oct 2014, see[1]. (I also ended-up fixing other scalability issues in iptables[2])
diff --git a/net/core/dev.c b/net/core/dev.c index f23e28668f32..0260ad314506 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -7651,7 +7651,7 @@ struct net_device *alloc_netdev_mqs(int sizeof_priv, const char *name, if (!dev->tx_queue_len) { dev->priv_flags |= IFF_NO_QUEUE; - dev->tx_queue_len = 1; + dev->tx_queue_len = DEFAULT_TX_QUEUE_LEN; } dev->num_tx_queues = txqs;
The flag IFF_NO_QUEUE marks virtual device drivers that doesn't need a default qdisc attached, given they will be backed by physical device, that already have a qdisc attached for pushback. It is still supported to attach a qdisc to a IFF_NO_QUEUE device, as this can be useful for difference policy reasons (e.g. bandwidth limiting containers). For this to work, the tx_queue_len need to have a sane value, because some qdiscs inherit/copy the tx_queue_len (namely, pfifo, bfifo, gred, htb, plug and sfb). Commit a813104d9233 ("IFF_NO_QUEUE: Fix for drivers not calling ether_setup()") caught situations where some drivers didn't initialize tx_queue_len. The problem with the commit was choosing 1 as the fallback value. A qdisc queue length of 1 causes more harm than good, because it creates hard to debug situations for userspace. It gives userspace a false sense of a working config after attaching a qdisc. As low volume traffic (that doesn't activate the qdisc policy) works, like ping, while traffic that e.g. needs shaping cannot reach the configured policy levels, given the queue length is too small. This patch change the value to DEFAULT_TX_QUEUE_LEN, given other IFF_NO_QUEUE devices (that call ether_setup()) also use this value. Fixes: a813104d9233 ("IFF_NO_QUEUE: Fix for drivers not calling ether_setup()") Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> --- net/core/dev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)