From patchwork Tue Mar 26 15:20:28 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Henriques X-Patchwork-Id: 231374 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) by ozlabs.org (Postfix) with ESMTP id 447242C00BE for ; Wed, 27 Mar 2013 02:24:37 +1100 (EST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.76) (envelope-from ) id 1UKVjh-0002ZC-3l; Tue, 26 Mar 2013 15:24:29 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtp (Exim 4.76) (envelope-from ) id 1UKVj5-0001eV-Jl for kernel-team@lists.ubuntu.com; Tue, 26 Mar 2013 15:23:51 +0000 Received: from bl20-143-66.dsl.telepac.pt ([2.81.143.66] helo=localhost) by youngberry.canonical.com with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1UKVj5-0004qj-8r; Tue, 26 Mar 2013 15:23:51 +0000 From: Luis Henriques To: linux-kernel@vger.kernel.org, stable@vger.kernel.org, kernel-team@lists.ubuntu.com Subject: [PATCH 129/150] sfc: Work-around flush timeout when flushes have completed Date: Tue, 26 Mar 2013 15:20:28 +0000 Message-Id: <1364311249-14454-130-git-send-email-luis.henriques@canonical.com> X-Mailer: git-send-email 1.8.1.2 In-Reply-To: <1364311249-14454-1-git-send-email-luis.henriques@canonical.com> References: <1364311249-14454-1-git-send-email-luis.henriques@canonical.com> X-Extended-Stable: 3.5 Cc: Ben Hutchings X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.14 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: kernel-team-bounces@lists.ubuntu.com 3.5.7.9 -stable review patch. If anyone has any objections, please let me know. ------------------ From: Daniel Pieczko commit 525d9e824018cd7cc8d8d44832ddcd363abfe6e1 upstream. We sometimes hit a "failed to flush" timeout on some TX queues, but the flushes have completed and the flush completion events seem to go missing. In this case, we can check the TX_DESC_PTR_TBL register and drain the queues if the flushes had finished. [bwh: Minor fixes to coding style] Signed-off-by: Ben Hutchings Signed-off-by: Luis Henriques --- drivers/net/ethernet/sfc/net_driver.h | 1 + drivers/net/ethernet/sfc/nic.c | 56 ++++++++++++++++++++++++++++++++--- 2 files changed, 53 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/sfc/net_driver.h b/drivers/net/ethernet/sfc/net_driver.h index cb7fd7c..8e3e0cf 100644 --- a/drivers/net/ethernet/sfc/net_driver.h +++ b/drivers/net/ethernet/sfc/net_driver.h @@ -196,6 +196,7 @@ struct efx_tx_queue { /* Members shared between paths and sometimes updated */ unsigned int empty_read_count ____cacheline_aligned_in_smp; #define EFX_EMPTY_COUNT_VALID 0x80000000 + atomic_t flush_outstanding; }; /** diff --git a/drivers/net/ethernet/sfc/nic.c b/drivers/net/ethernet/sfc/nic.c index c44a3a0..76f16b7 100644 --- a/drivers/net/ethernet/sfc/nic.c +++ b/drivers/net/ethernet/sfc/nic.c @@ -73,6 +73,8 @@ _EFX_CHANNEL_MAGIC(_EFX_CHANNEL_MAGIC_TX_DRAIN, \ (_tx_queue)->queue) +static void efx_magic_event(struct efx_channel *channel, u32 magic); + /************************************************************************** * * Solarstorm hardware access @@ -492,6 +494,9 @@ static void efx_flush_tx_queue(struct efx_tx_queue *tx_queue) struct efx_nic *efx = tx_queue->efx; efx_oword_t tx_flush_descq; + WARN_ON(atomic_read(&tx_queue->flush_outstanding)); + atomic_set(&tx_queue->flush_outstanding, 1); + EFX_POPULATE_OWORD_2(tx_flush_descq, FRF_AZ_TX_FLUSH_DESCQ_CMD, 1, FRF_AZ_TX_FLUSH_DESCQ, tx_queue->queue); @@ -667,6 +672,47 @@ static bool efx_flush_wake(struct efx_nic *efx) && atomic_read(&efx->rxq_flush_pending) > 0)); } +static bool efx_check_tx_flush_complete(struct efx_nic *efx) +{ + bool i = true; + efx_oword_t txd_ptr_tbl; + struct efx_channel *channel; + struct efx_tx_queue *tx_queue; + + efx_for_each_channel(channel, efx) { + efx_for_each_channel_tx_queue(tx_queue, channel) { + efx_reado_table(efx, &txd_ptr_tbl, + FR_BZ_TX_DESC_PTR_TBL, tx_queue->queue); + if (EFX_OWORD_FIELD(txd_ptr_tbl, + FRF_AZ_TX_DESCQ_FLUSH) || + EFX_OWORD_FIELD(txd_ptr_tbl, + FRF_AZ_TX_DESCQ_EN)) { + netif_dbg(efx, hw, efx->net_dev, + "flush did not complete on TXQ %d\n", + tx_queue->queue); + i = false; + } else if (atomic_cmpxchg(&tx_queue->flush_outstanding, + 1, 0)) { + /* The flush is complete, but we didn't + * receive a flush completion event + */ + netif_dbg(efx, hw, efx->net_dev, + "flush complete on TXQ %d, so drain " + "the queue\n", tx_queue->queue); + /* Don't need to increment drain_pending as it + * has already been incremented for the queues + * which did not drain + */ + efx_magic_event(channel, + EFX_CHANNEL_MAGIC_TX_DRAIN( + tx_queue)); + } + } + } + + return i; +} + /* Flush all the transmit queues, and continue flushing receive queues until * they're all flushed. Wait for the DRAIN events to be recieved so that there * are no more RX and TX events left on any channel. */ @@ -727,7 +773,8 @@ int efx_nic_flush_queues(struct efx_nic *efx) timeout); } - if (atomic_read(&efx->drain_pending)) { + if (atomic_read(&efx->drain_pending) && + !efx_check_tx_flush_complete(efx)) { netif_err(efx, hw, efx->net_dev, "failed to flush %d queues " "(rx %d+%d)\n", atomic_read(&efx->drain_pending), atomic_read(&efx->rxq_flush_outstanding), @@ -1014,9 +1061,10 @@ efx_handle_tx_flush_done(struct efx_nic *efx, efx_qword_t *event) if (qid < EFX_TXQ_TYPES * efx->n_tx_channels) { tx_queue = efx_get_tx_queue(efx, qid / EFX_TXQ_TYPES, qid % EFX_TXQ_TYPES); - - efx_magic_event(tx_queue->channel, - EFX_CHANNEL_MAGIC_TX_DRAIN(tx_queue)); + if (atomic_cmpxchg(&tx_queue->flush_outstanding, 1, 0)) { + efx_magic_event(tx_queue->channel, + EFX_CHANNEL_MAGIC_TX_DRAIN(tx_queue)); + } } }