From patchwork Thu Sep 4 12:55:49 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesper Dangaard Brouer X-Patchwork-Id: 385829 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id CAD8D140092 for ; Thu, 4 Sep 2014 22:57:08 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754401AbaIDM4F (ORCPT ); Thu, 4 Sep 2014 08:56:05 -0400 Received: from mx1.redhat.com ([209.132.183.28]:48107 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754395AbaIDM4C (ORCPT ); Thu, 4 Sep 2014 08:56:02 -0400 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s84CtowY029457 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 4 Sep 2014 08:55:51 -0400 Received: from dragon.localdomain (ovpn-116-45.ams2.redhat.com [10.36.116.45]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s84CtopG008694; Thu, 4 Sep 2014 08:55:50 -0400 Received: from [127.0.0.1] (localhost [IPv6:::1]) by dragon.localdomain (Postfix) with ESMTP id D1087E4072C; Thu, 4 Sep 2014 14:55:49 +0200 (CEST) From: Jesper Dangaard Brouer Subject: [RFC net-next PATCH V2 2/3] qdisc: bulk dequeue support for qdiscs with TCQ_F_ONETXQUEUE To: Jesper Dangaard Brouer , netdev@vger.kernel.org, "David S. Miller" , Tom Herbert , Eric Dumazet , Hannes Frederic Sowa , Florian Westphal , Daniel Borkmann Cc: Jamal Hadi Salim , Alexander Duyck , John Fastabend Date: Thu, 04 Sep 2014 14:55:49 +0200 Message-ID: <20140904125438.4108.38160.stgit@dragon> In-Reply-To: <20140904125247.4108.8132.stgit@dragon> References: <20140904125247.4108.8132.stgit@dragon> User-Agent: StGIT/0.14.3 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Based on DaveM's recent API work on dev_hard_start_xmit(), that allows sending/processing an entire skb list. This patch implements qdisc bulk dequeue, by allowing multiple packets to be dequeued in dequeue_skb(). The optimization principle for this is two fold, (1) to amortize locking cost and (2) avoid expensive tailptr update for notifying HW. (1) Several packets are dequeued while holding the qdisc root_lock, amortizing locking cost over several packet. The dequeued SKB list is processed under the TXQ lock in dev_hard_start_xmit(), thus also amortizing the cost of the TXQ lock. (2) Further more, dev_hard_start_xmit() will utilize the skb->xmit_more API to delay HW tailptr update, which also reduces the cost per packet. One restriction of the new API is that every SKB must belong to the same TXQ. This patch takes the easy way out, by restricting bulk dequeue to qdisc's with the TCQ_F_ONETXQUEUE flag, that specifies the qdisc only have attached a single TXQ. Some detail about the flow; dev_hard_start_xmit() will process the skb list, and transmit packets individually towards the driver (see xmit_one()). In case the driver stops midway in the list, the remaining skb list is returned by dev_hard_start_xmit(). In sch_direct_xmit() this returned list is requeued by dev_requeue_skb(). The patch also tries to limit the amount of bytes dequeued, based on the drivers BQL limits. It also tries to avoid and stop dequeuing when seeing a GSO packet (both real GSO and segmented GSO skb lists). Signed-off-by: Jesper Dangaard Brouer --- V2: - Restruct functions, split out functionality - Use BQL bytelimit to avoid overshooting driver limits, causing too large skb lists to be sitting on the requeue gso_skb. net/sched/sch_generic.c | 67 +++++++++++++++++++++++++++++++++++++++++++++-- 1 files changed, 64 insertions(+), 3 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c index 19696eb..a0c8070 100644 --- a/net/sched/sch_generic.c +++ b/net/sched/sch_generic.c @@ -56,6 +56,67 @@ static inline int dev_requeue_skb(struct sk_buff *skb, struct Qdisc *q) return 0; } +static inline bool qdisc_may_bulk(const struct Qdisc *qdisc, + const struct sk_buff *skb) +{ + return (qdisc->flags & TCQ_F_ONETXQUEUE); +} + +static inline struct sk_buff *qdisc_dequeue_validate(struct Qdisc *qdisc) +{ + struct sk_buff *skb = qdisc->dequeue(qdisc); + + if (skb != NULL) + skb = validate_xmit_skb(skb, qdisc_dev(qdisc)); + + return skb; +} + +static inline struct sk_buff *qdisc_bulk_dequeue_skb(struct Qdisc *q, + struct sk_buff *head) +{ + struct sk_buff *new, *skb = head; + struct netdev_queue *txq = q->dev_queue; + int bytelimit = netdev_tx_avail_queue(txq); + int limit = 5; + + if (bytelimit <= 0) + return head; + + do { + if (skb->next || skb_is_gso(skb)) { + /* Stop processing if the skb is already a skb + * list (e.g a segmented GSO packet) or a real + * GSO packet */ + break; + } + new = qdisc_dequeue_validate(q); + if (new) { + skb->next = new; + skb = new; + bytelimit -= skb->len; + cnt++; + /* One problem here is it is difficult to + * requeue the "new" dequeued skb, e.g. in + * case of GSO, thus a "normal" packet can + * have a GSO packet on its ->next ptr. + * + * Requeue is difficult because if requeuing + * on q->gso_skb, then a second requeue can + * happen from sch_direct_xmit e.g. if driver + * returns NETDEV_TX_BUSY, which would + * overwrite this requeue. + */ + } + } while (new && --limit && (bytelimit > 0)); + skb = head; + + return skb; +} + +/* Note that dequeue_skb can possibly return a SKB list (via skb->next). + * A requeued skb (via q->gso_skb) can also be a SKB list. + */ static inline struct sk_buff *dequeue_skb(struct Qdisc *q) { struct sk_buff *skb = q->gso_skb; @@ -71,9 +132,9 @@ static inline struct sk_buff *dequeue_skb(struct Qdisc *q) skb = NULL; } else { if (!(q->flags & TCQ_F_ONETXQUEUE) || !netif_xmit_frozen_or_stopped(txq)) { - skb = q->dequeue(q); - if (skb) - skb = validate_xmit_skb(skb, qdisc_dev(q)); + skb = qdisc_dequeue_validate(q); + if (skb && qdisc_may_bulk(q, skb)) + skb = qdisc_bulk_dequeue_skb(q, skb); } }