Message ID | 1274454480.2439.418.camel@edumazet-laptop |
---|---|
State | Accepted, archived |
Delegated to: | David Miller |
Headers | show |
Eric Dumazet wrote: > Tests with following script gave a boost from ~50.000 pps to ~600.000 > pps on a dual quad core machine (E5450 @3.00GHz), tg3 driver. > (A single netperf flow can reach ~800.000 pps on this platform) > > for j in `seq 0 3`; do > for i in `seq 0 7`; do > netperf -H 192.168.0.1 -t UDP_STREAM -l 60 -N -T $i -- -m 6 & > done > done Running the same script with your patch my results went from 200Kpps to 1.2Mpps on a dual Xeon 5570. Acked-by: Alexander Duyck <alexander.h.duyck@intel.com>
From: "Duyck, Alexander H" <alexander.h.duyck@intel.com> Date: Fri, 21 May 2010 13:04:20 -0700 > Eric Dumazet wrote: >> Tests with following script gave a boost from ~50.000 pps to ~600.000 >> pps on a dual quad core machine (E5450 @3.00GHz), tg3 driver. >> (A single netperf flow can reach ~800.000 pps on this platform) >> >> for j in `seq 0 3`; do >> for i in `seq 0 7`; do >> netperf -H 192.168.0.1 -t UDP_STREAM -l 60 -N -T $i -- -m 6 & >> done >> done > > Running the same script with your patch my results went from 200Kpps to 1.2Mpps on a dual Xeon 5570. > > Acked-by: Alexander Duyck <alexander.h.duyck@intel.com> Applied, thanks guys. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Le mercredi 02 juin 2010 à 05:10 -0700, David Miller a écrit : > From: "Duyck, Alexander H" <alexander.h.duyck@intel.com> > Date: Fri, 21 May 2010 13:04:20 -0700 > > > Eric Dumazet wrote: > >> Tests with following script gave a boost from ~50.000 pps to ~600.000 > >> pps on a dual quad core machine (E5450 @3.00GHz), tg3 driver. > >> (A single netperf flow can reach ~800.000 pps on this platform) > >> > >> for j in `seq 0 3`; do > >> for i in `seq 0 7`; do > >> netperf -H 192.168.0.1 -t UDP_STREAM -l 60 -N -T $i -- -m 6 & > >> done > >> done > > > > Running the same script with your patch my results went from 200Kpps to 1.2Mpps on a dual Xeon 5570. > > > > Acked-by: Alexander Duyck <alexander.h.duyck@intel.com> > > Applied, thanks guys. Thanks David, I realize you did all the necessary changes after qdisc_run_begin/qdisc_is_running/ integration ;) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h index 03ca5d8..2d55649 100644 --- a/include/net/sch_generic.h +++ b/include/net/sch_generic.h @@ -73,7 +73,8 @@ struct Qdisc { struct sk_buff_head q; struct gnet_stats_basic_packed bstats; struct gnet_stats_queue qstats; - struct rcu_head rcu_head; + struct rcu_head rcu_head; + spinlock_t busylock; }; struct Qdisc_class_ops { diff --git a/net/core/dev.c b/net/core/dev.c index 6c82065..1b313eb 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2040,6 +2040,16 @@ static inline int __dev_xmit_skb(struct sk_buff *skb, struct Qdisc *q, { spinlock_t *root_lock = qdisc_lock(q); int rc; + bool contended = test_bit(__QDISC_STATE_RUNNING, &q->state); + + /* + * Heuristic to force contended enqueues to serialize on a + * separate lock before trying to get qdisc main lock. + * This permits __QDISC_STATE_RUNNING owner to get the lock more often + * and dequeue packets faster. + */ + if (unlikely(contended)) + spin_lock(&q->busylock); spin_lock(root_lock); if (unlikely(test_bit(__QDISC_STATE_DEACTIVATED, &q->state))) { @@ -2055,19 +2065,30 @@ static inline int __dev_xmit_skb(struct sk_buff *skb, struct Qdisc *q, if (!(dev->priv_flags & IFF_XMIT_DST_RELEASE)) skb_dst_force(skb); __qdisc_update_bstats(q, skb->len); - if (sch_direct_xmit(skb, q, dev, txq, root_lock)) + if (sch_direct_xmit(skb, q, dev, txq, root_lock)) { + if (unlikely(contended)) { + spin_unlock(&q->busylock); + contended = false; + } __qdisc_run(q); - else + } else clear_bit(__QDISC_STATE_RUNNING, &q->state); rc = NET_XMIT_SUCCESS; } else { skb_dst_force(skb); rc = qdisc_enqueue_root(skb, q); - qdisc_run(q); + if (!test_and_set_bit(__QDISC_STATE_RUNNING, &q->state)) { + if (unlikely(contended)) { + spin_unlock(&q->busylock); + contended = false; + } + __qdisc_run(q); + } } spin_unlock(root_lock); - + if (unlikely(contended)) + spin_unlock(&q->busylock); return rc; } diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c index a63029e..0a44290 100644 --- a/net/sched/sch_generic.c +++ b/net/sched/sch_generic.c @@ -543,6 +543,7 @@ struct Qdisc *qdisc_alloc(struct netdev_queue *dev_queue, INIT_LIST_HEAD(&sch->list); skb_queue_head_init(&sch->q); + spin_lock_init(&sch->busylock); sch->ops = ops; sch->enqueue = ops->enqueue; sch->dequeue = ops->dequeue;