From patchwork Mon Nov 21 08:43:39 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yevgeny Petrilin X-Patchwork-Id: 126740 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id CEB98B720B for ; Mon, 21 Nov 2011 22:19:29 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756117Ab1KULTT (ORCPT ); Mon, 21 Nov 2011 06:19:19 -0500 Received: from eu1sys200aog115.obsmtp.com ([207.126.144.139]:41570 "HELO eu1sys200aog115.obsmtp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1755478Ab1KULTS (ORCPT ); Mon, 21 Nov 2011 06:19:18 -0500 Received: from MTLCAS01.mtl.com ([194.90.237.34]) (using TLSv1) by eu1sys200aob115.postini.com ([207.126.147.11]) with SMTP ID DSNKTsozs6nJRI4Bvt6IwzNzZkistzjJ65c9@postini.com; Mon, 21 Nov 2011 11:19:17 UTC Received: from vnc2.lab.mtl.com (10.4.45.2) by MTLCAS01.mtl.com (10.0.8.70) with Microsoft SMTP Server id 14.1.289.1; Mon, 21 Nov 2011 12:53:01 +0200 Message-ID: <4ECA0F3B.70107@mellanox.co.il> Date: Mon, 21 Nov 2011 10:43:39 +0200 From: Yevgeny Petrilin User-Agent: Thunderbird 2.0.0.17 (X11/20080914) MIME-Version: 1.0 To: CC: , , , , Subject: [PATCH 4/8] net/mlx4_en: using non collapsed CQ on TX X-Originating-IP: [10.4.45.2] Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Yevgeny Petrilin Moving to regular Completion Queue implementation (not collapsed) Completion for each transmitted packet is written to new entry. Signed-off-by: Yevgeny Petrilin --- drivers/net/ethernet/mellanox/mlx4/en_cq.c | 7 +-- drivers/net/ethernet/mellanox/mlx4/en_tx.c | 65 ++++++++++++++-------------- 2 files changed, 35 insertions(+), 37 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_cq.c b/drivers/net/ethernet/mellanox/mlx4/en_cq.c index 227997d..2d1a342 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_cq.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_cq.c @@ -51,10 +51,7 @@ int mlx4_en_create_cq(struct mlx4_en_priv *priv, int err; cq->size = entries; - if (mode == RX) - cq->buf_size = cq->size * sizeof(struct mlx4_cqe); - else - cq->buf_size = sizeof(struct mlx4_cqe); + cq->buf_size = cq->size * sizeof(struct mlx4_cqe); cq->ring = ring; cq->is_tx = mode; @@ -120,7 +117,7 @@ int mlx4_en_activate_cq(struct mlx4_en_priv *priv, struct mlx4_en_cq *cq, cq->size = priv->rx_ring[cq->ring].actual_size; err = mlx4_cq_alloc(mdev->dev, cq->size, &cq->wqres.mtt, &mdev->priv_uar, - cq->wqres.db.dma, &cq->mcq, cq->vector, cq->is_tx); + cq->wqres.db.dma, &cq->mcq, cq->vector, 0); if (err) return err; diff --git a/drivers/net/ethernet/mellanox/mlx4/en_tx.c b/drivers/net/ethernet/mellanox/mlx4/en_tx.c index d901b42..3094f94 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_tx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_tx.c @@ -307,59 +307,60 @@ int mlx4_en_free_tx_buf(struct net_device *dev, struct mlx4_en_tx_ring *ring) return cnt; } - static void mlx4_en_process_tx_cq(struct net_device *dev, struct mlx4_en_cq *cq) { struct mlx4_en_priv *priv = netdev_priv(dev); struct mlx4_cq *mcq = &cq->mcq; struct mlx4_en_tx_ring *ring = &priv->tx_ring[cq->ring]; - struct mlx4_cqe *cqe = cq->buf; + struct mlx4_cqe *cqe; u16 index; - u16 new_index; + u16 new_index, ring_index; u32 txbbs_skipped = 0; - u32 cq_last_sav; - - /* index always points to the first TXBB of the last polled descriptor */ - index = ring->cons & ring->size_mask; - new_index = be16_to_cpu(cqe->wqe_index) & ring->size_mask; - if (index == new_index) - return; + u32 cons_index = mcq->cons_index; + int size = cq->size; + u32 size_mask = ring->size_mask; + struct mlx4_cqe *buf = cq->buf; if (!priv->port_up) return; - /* - * We use a two-stage loop: - * - the first samples the HW-updated CQE - * - the second frees TXBBs until the last sample - * This lets us amortize CQE cache misses, while still polling the CQ - * until is quiescent. - */ - cq_last_sav = mcq->cons_index; - do { + index = cons_index & size_mask; + cqe = &buf[index]; + ring_index = ring->cons & size_mask; + + /* Process all completed CQEs */ + while (XNOR(cqe->owner_sr_opcode & MLX4_CQE_OWNER_MASK, + cons_index & size)) { + /* + * make sure we read the CQE after we read the + * ownership bit + */ + rmb(); + + /* Skip over last polled CQE */ + new_index = be16_to_cpu(cqe->wqe_index) & size_mask; + do { - /* Skip over last polled CQE */ - index = (index + ring->last_nr_txbb) & ring->size_mask; txbbs_skipped += ring->last_nr_txbb; - - /* Poll next CQE */ + ring_index = (ring_index + ring->last_nr_txbb) & size_mask; + /* free next descriptor */ ring->last_nr_txbb = mlx4_en_free_tx_desc( - priv, ring, index, - !!((ring->cons + txbbs_skipped) & - ring->size)); - ++mcq->cons_index; + priv, ring, ring_index, + !!((ring->cons + txbbs_skipped) & + ring->size)); + } while (ring_index != new_index); - } while (index != new_index); + ++cons_index; + index = cons_index & size_mask; + cqe = &buf[index]; + } - new_index = be16_to_cpu(cqe->wqe_index) & ring->size_mask; - } while (index != new_index); - AVG_PERF_COUNTER(priv->pstats.tx_coal_avg, - (u32) (mcq->cons_index - cq_last_sav)); /* * To prevent CQ overflow we first update CQ consumer and only then * the ring consumer. */ + mcq->cons_index = cons_index; mlx4_cq_set_ci(mcq); wmb(); ring->cons += txbbs_skipped;