From patchwork Mon Nov 17 17:18:26 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Duyck X-Patchwork-Id: 411767 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 1FA4B14011B for ; Tue, 18 Nov 2014 04:23:20 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752127AbaKQRWf (ORCPT ); Mon, 17 Nov 2014 12:22:35 -0500 Received: from mx1.redhat.com ([209.132.183.28]:38882 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750978AbaKQRWd (ORCPT ); Mon, 17 Nov 2014 12:22:33 -0500 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id sAHHIYbn029418 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 17 Nov 2014 12:18:34 -0500 Received: from [192.168.1.5] (ovpn-112-76.phx2.redhat.com [10.3.112.76]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id sAHHIQtq031356; Mon, 17 Nov 2014 12:18:28 -0500 Subject: [PATCH 3/4] r8169: Use fast_rmb() and fast_wmb() for DescOwn checks From: Alexander Duyck To: linux-arch@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: mathieu.desnoyers@polymtl.ca, peterz@infradead.org, benh@kernel.crashing.org, heiko.carstens@de.ibm.com, mingo@kernel.org, mikey@neuling.org, linux@arm.linux.org.uk, donald.c.skidmore@intel.com, matthew.vick@intel.com, geert@linux-m68k.org, jeffrey.t.kirsher@intel.com, romieu@fr.zoreil.com, paulmck@linux.vnet.ibm.com, nic_swsd@realtek.com, will.deacon@arm.com, michael@ellerman.id.au, tony.luck@intel.com, torvalds@linux-foundation.org, oleg@redhat.com, schwidefsky@de.ibm.com, fweisbec@gmail.com, davem@davemloft.net Date: Mon, 17 Nov 2014 09:18:26 -0800 Message-ID: <20141117171825.22333.1998.stgit@ahduyck-server> In-Reply-To: <20141117171005.22333.96544.stgit@ahduyck-server> References: <20141117171005.22333.96544.stgit@ahduyck-server> User-Agent: StGit/0.17-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The r8169 use a pair of wmb() calls when setting up the descriptor rings. The first is to synchronize the descriptor data with the descriptor status, and the second is to synchronize the descriptor status with the use of the MMIO doorbell to notify the device that descriptors are ready. This can come at a heavy price on some systems, and is not really necessary on systems such as x86 as a simple barrier() would suffice to order store/store accesses. As such we can replace the first memory barrier with fast_wmb() to reduce the cost for these accesses. In addition the r8169 uses a rmb() to prevent compiler optimization in the cleanup paths, however by moving the barrier down a few lines and replacing with with a fast_rmb() we should be able to use it to guarantee descriptor accesses do not occur until the device has updated the DescOwn bit from its end. One last change I made is to move the update of cur_tx in the xmit path to after the wmb. This way we can guarantee the device and all CPUs should see the DescOwn update before they see the cur_tx value update. Cc: Realtek linux nic maintainers Cc: Francois Romieu Signed-off-by: Alexander Duyck --- drivers/net/ethernet/realtek/r8169.c | 29 +++++++++++++++++++++-------- 1 file changed, 21 insertions(+), 8 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c index cf154f7..be79447 100644 --- a/drivers/net/ethernet/realtek/r8169.c +++ b/drivers/net/ethernet/realtek/r8169.c @@ -6601,6 +6601,9 @@ static inline void rtl8169_mark_to_asic(struct RxDesc *desc, u32 rx_buf_sz) { u32 eor = le32_to_cpu(desc->opts1) & RingEnd; + /* Force memory writes to complete before releasing descriptor */ + fast_wmb(); + desc->opts1 = cpu_to_le32(DescOwn | eor | rx_buf_sz); } @@ -6608,7 +6611,6 @@ static inline void rtl8169_map_to_asic(struct RxDesc *desc, dma_addr_t mapping, u32 rx_buf_sz) { desc->addr = cpu_to_le64(mapping); - wmb(); rtl8169_mark_to_asic(desc, rx_buf_sz); } @@ -7077,16 +7079,18 @@ static netdev_tx_t rtl8169_start_xmit(struct sk_buff *skb, skb_tx_timestamp(skb); - wmb(); + /* Force memory writes to complete before releasing descriptor */ + fast_wmb(); /* Anti gcc 2.95.3 bugware (sic) */ status = opts[0] | len | (RingEnd * !((entry + 1) % NUM_TX_DESC)); txd->opts1 = cpu_to_le32(status); - tp->cur_tx += frags + 1; - + /* Force all memory writes to complete before notifying device */ wmb(); + tp->cur_tx += frags + 1; + RTL_W8(TxPoll, NPQ); mmiowb(); @@ -7185,11 +7189,16 @@ static void rtl_tx(struct net_device *dev, struct rtl8169_private *tp) struct ring_info *tx_skb = tp->tx_skb + entry; u32 status; - rmb(); status = le32_to_cpu(tp->TxDescArray[entry].opts1); if (status & DescOwn) break; + /* This barrier is needed to keep us from reading + * any other fields out of the Tx descriptor until + * we know the status of DescOwn + */ + fast_rmb(); + rtl8169_unmap_tx_skb(&tp->pci_dev->dev, tx_skb, tp->TxDescArray + entry); if (status & LastFrag) { @@ -7284,11 +7293,16 @@ static int rtl_rx(struct net_device *dev, struct rtl8169_private *tp, u32 budget struct RxDesc *desc = tp->RxDescArray + entry; u32 status; - rmb(); status = le32_to_cpu(desc->opts1) & tp->opts1_mask; - if (status & DescOwn) break; + + /* This barrier is needed to keep us from reading + * any other fields out of the Rx descriptor until + * we know the status of DescOwn + */ + fast_rmb(); + if (unlikely(status & RxRES)) { netif_info(tp, rx_err, dev, "Rx ERROR. status = %08x\n", status); @@ -7350,7 +7364,6 @@ process_pkt: } release_descriptor: desc->opts2 = 0; - wmb(); rtl8169_mark_to_asic(desc, rx_buf_sz); }