net: orphan queued skbs if device tx can stall

On Tue, Apr 10, 2012 at 12:04:19PM +0200, Eric Dumazet wrote:
> On Tue, 2012-04-10 at 12:31 +0300, Michael S. Tsirkin wrote:
> 
> > True. Still this is the only interface we have for controlling
> > the internal queue length so it seems safe to assume someone
> > is using it for this purpose.
> > 
> 
> So to workaround a problem in tun, you want to hack net/core/dev.c :(

Sorry about being unclear, I'm just saying that your patch assumes
tx_queue_len == 0 since you set it that way at device init but we can't
rely on this as existing users might have changed that value.
One way to fix would be a patch at the bottom: then we
can leave tun to treat tx_queue_len like it always did.

> Packets in qdisc should not be orphaned.
>
> If you think about it, why do we attach skb to socket in the first
> place ?

Good point. The answer is to avoid skb drops for local sockets by
stopping them, right?

> If its not needed for tun, why should it be needed for other devices ?

Maybe needed but it's already broken for tun since the skbs in the
private queue are orphaned by skb_orphan_try?

> If TUN has a problem being stopped forever, maybe it should take
> appropriate action to flush all packets in qdisc queue after a while, as
> this makes no sense to delay packets forever.
> 

Well arbitrary timers aren't a solid protection, right?
We get out of stalling transmitters forever but a bad VM can still
degrade performance significantly for others...


----

We don't want a queue for tun since it can stall forever, but userspace
might tweak it's tx_queue_len as a way to control RX queue depth,
and we don't want to break userspace. Use a private flag to disable queue.

Warning: untested.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

On Tue, 2012-04-10 at 14:25 +0300, Michael S. Tsirkin wrote:
> On Tue, Apr 10, 2012 at 12:04:19PM +0200, Eric Dumazet wrote:
> > On Tue, 2012-04-10 at 12:31 +0300, Michael S. Tsirkin wrote:
> > 
> > > True. Still this is the only interface we have for controlling
> > > the internal queue length so it seems safe to assume someone
> > > is using it for this purpose.
> > > 
> > 
> > So to workaround a problem in tun, you want to hack net/core/dev.c :(
> 
> Sorry about being unclear, I'm just saying that your patch assumes
> tx_queue_len == 0 since you set it that way at device init but we can't
> rely on this as existing users might have changed that value.
> One way to fix would be a patch at the bottom: then we
> can leave tun to treat tx_queue_len like it always did.


> ----
> 
> We don't want a queue for tun since it can stall forever, but userspace
> might tweak it's tx_queue_len as a way to control RX queue depth,
> and we don't want to break userspace. Use a private flag to disable queue.
> 
> Warning: untested.
> 
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> 
> diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
> index 27883d1..644ca53 100644
> --- a/net/sched/sch_generic.c
> +++ b/net/sched/sch_generic.c
> @@ -695,7 +692,7 @@ static void attach_one_default_qdisc(struct net_device *dev,
>  {
>  	struct Qdisc *qdisc = &noqueue_qdisc;
>  
> -	if (dev->tx_queue_len) {
> +	if (dev->tx_queue_len && !(dev->priv_flags & IFF_TX_CAN_STALL)) {
>  		qdisc = qdisc_create_dflt(dev_queue,
>  					  &pfifo_fast_ops, TC_H_ROOT);
>  		if (!qdisc) {


Thing is this function is called before userspace can tweak tx_queue_len

So if you create a vlan device (this sets tx_queue_len to 0), no qdisc
is attached.

If later userspace changes tx_queue_len to this device, qdisc wont
automatically be created/attached.

Really, tx_queue_len is private to net/sched layer, it should not be
used by tun device to control a receive queue limit.

Please try to not hack net/sched or net/core for your needs.

Its not because tun abused tx_queue_len in the past we must keep this
hack forever.

In ethernet drivers, TX ring size is controlled by ethtool -g

Why tun driver would use another way ?



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Tue, Apr 10, 2012 at 01:45:00PM +0200, Eric Dumazet wrote:
> On Tue, 2012-04-10 at 14:25 +0300, Michael S. Tsirkin wrote:
> > On Tue, Apr 10, 2012 at 12:04:19PM +0200, Eric Dumazet wrote:
> > > On Tue, 2012-04-10 at 12:31 +0300, Michael S. Tsirkin wrote:
> > > 
> > > > True. Still this is the only interface we have for controlling
> > > > the internal queue length so it seems safe to assume someone
> > > > is using it for this purpose.
> > > > 
> > > 
> > > So to workaround a problem in tun, you want to hack net/core/dev.c :(
> > 
> > Sorry about being unclear, I'm just saying that your patch assumes
> > tx_queue_len == 0 since you set it that way at device init but we can't
> > rely on this as existing users might have changed that value.
> > One way to fix would be a patch at the bottom: then we
> > can leave tun to treat tx_queue_len like it always did.
> 
> 
> > ----
> > 
> > We don't want a queue for tun since it can stall forever, but userspace
> > might tweak it's tx_queue_len as a way to control RX queue depth,
> > and we don't want to break userspace. Use a private flag to disable queue.
> > 
> > Warning: untested.
> > 
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > 
> > diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
> > index 27883d1..644ca53 100644
> > --- a/net/sched/sch_generic.c
> > +++ b/net/sched/sch_generic.c
> > @@ -695,7 +692,7 @@ static void attach_one_default_qdisc(struct net_device *dev,
> >  {
> >  	struct Qdisc *qdisc = &noqueue_qdisc;
> >  
> > -	if (dev->tx_queue_len) {
> > +	if (dev->tx_queue_len && !(dev->priv_flags & IFF_TX_CAN_STALL)) {
> >  		qdisc = qdisc_create_dflt(dev_queue,
> >  					  &pfifo_fast_ops, TC_H_ROOT);
> >  		if (!qdisc) {
> 
> 
> Thing is this function is called before userspace can tweak tx_queue_len
> 
> So if you create a vlan device (this sets tx_queue_len to 0), no qdisc
> is attached.
>
> If later userspace changes tx_queue_len to this device, qdisc wont
> automatically be created/attached.

True. But there's another place where this can happen - after
dev_change_net_namespace, no?
This calls dev_shutdown.

> Really, tx_queue_len is private to net/sched layer, it should not be
> used by tun device to control a receive queue limit.
> 
> Please try to not hack net/sched or net/core for your needs.
> 
> Its not because tun abused tx_queue_len in the past we must keep this
> hack forever.
> 
> In ethernet drivers, TX ring size is controlled by ethtool -g
> 
> Why tun driver would use another way ?
> 

I think it's a bad interface too but it's in a userspace ABI
now so I suspect we are stuck with it for now. We can try deprecating
but we can't just drop it.