From patchwork Fri Nov 11 01:31:31 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: david decotigny X-Patchwork-Id: 125064 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 799B71007D1 for ; Fri, 11 Nov 2011 12:32:41 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755742Ab1KKBcR (ORCPT ); Thu, 10 Nov 2011 20:32:17 -0500 Received: from mail-wy0-f202.google.com ([74.125.82.202]:64895 "EHLO mail-wy0-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752807Ab1KKBcQ (ORCPT ); Thu, 10 Nov 2011 20:32:16 -0500 Received: by wyg8 with SMTP id 8so81581wyg.1 for ; Thu, 10 Nov 2011 17:32:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=beta; h=from:to:cc:subject:date:message-id:x-mailer:in-reply-to:references :in-reply-to:references:organization:x-system-of-record; bh=1MXkDZa4npXhCQo2nChEsVqotHhP7CPgc0yIVviKFLk=; b=hYSI2HsdCXaprCeurF9+FwQdTue/YNWaPlkyuYDbzQtsCD+XR1mPL9LE9f6Fpro+2g nXYf5E0EP+vDkaecJn0g== Received: by 10.14.2.15 with SMTP id 15mr614574eee.2.1320975133817; Thu, 10 Nov 2011 17:32:13 -0800 (PST) Received: by 10.14.2.15 with SMTP id 15mr614524eee.2.1320975133309; Thu, 10 Nov 2011 17:32:13 -0800 (PST) Received: from hpza10.eem.corp.google.com ([74.125.121.33]) by gmr-mx.google.com with ESMTPS id d10si5730266eed.3.2011.11.10.17.32.13 (version=TLSv1/SSLv3 cipher=AES128-SHA); Thu, 10 Nov 2011 17:32:13 -0800 (PST) Received: from wpaz24.hot.corp.google.com (wpaz24.hot.corp.google.com [172.24.198.88]) by hpza10.eem.corp.google.com (Postfix) with ESMTPS id 13EB920004E; Thu, 10 Nov 2011 17:32:12 -0800 (PST) Received: from decotigny.mtv.corp.google.com (decotigny.mtv.corp.google.com [172.18.64.159]) by wpaz24.hot.corp.google.com with ESMTP id pAB1W9Va016815; Thu, 10 Nov 2011 17:32:10 -0800 Received: by decotigny.mtv.corp.google.com (Postfix, from userid 128857) id B525525258; Thu, 10 Nov 2011 17:32:07 -0800 (PST) From: David Decotigny To: netdev@vger.kernel.org, linux-kernel@vger.kernel.org Cc: "David S. Miller" , Ian Campbell , Eric Dumazet , Jeff Kirsher , Ben Hutchings , Jiri Pirko , Joe Perches , Szymon Janc , David Decotigny Subject: [PATCH net-next v2 06/10] forcedeth: implement ndo_get_stats64() API Date: Thu, 10 Nov 2011 17:31:31 -0800 Message-Id: X-Mailer: git-send-email 1.7.3.1 In-Reply-To: References: In-Reply-To: References: Organization: Google, Inc. X-System-Of-Record: true Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This commit implements the ndo_get_stats64() API for forcedeth. Since these stats are being updated from different contexts (process and timer), this commit adds protection (locking + atomic variables). Tested: 16-way SMP x86_64 -> RX bytes:7244556582 (7.2 GB) TX bytes:181904254 (181.9 MB) Signed-off-by: David Decotigny --- drivers/net/ethernet/nvidia/forcedeth.c | 155 +++++++++++++++++++++++-------- 1 files changed, 118 insertions(+), 37 deletions(-) diff --git a/drivers/net/ethernet/nvidia/forcedeth.c b/drivers/net/ethernet/nvidia/forcedeth.c index 1a1972b..cabc121 100644 --- a/drivers/net/ethernet/nvidia/forcedeth.c +++ b/drivers/net/ethernet/nvidia/forcedeth.c @@ -693,6 +693,21 @@ struct nv_ethtool_stats { #define NV_DEV_STATISTICS_V2_COUNT (NV_DEV_STATISTICS_V3_COUNT - 3) #define NV_DEV_STATISTICS_V1_COUNT (NV_DEV_STATISTICS_V2_COUNT - 6) +/* driver statistics */ +struct nv_driver_stat { + atomic_t delta; /* increase since last nv_update_stats() */ + u64 total; /* cumulative, requires netdev_priv(dev)->stats_lock */ +}; + +#define NV_DRIVER_STAT_ATOMIC_INC(ptr_stat) /* atomic */ \ + ({ atomic_inc(&(ptr_stat)->delta); }) +#define NV_DRIVER_STAT_ATOMIC_ADD(ptr_stat,increment) /* atomic */ \ + ({ atomic_add((increment), &(ptr_stat)->delta); }) +#define NV_DRIVER_STAT_UPDATE_TOTAL(ptr_stat) /* requires stats_lock */ \ + ({ (ptr_stat)->total += atomic_xchg(&(ptr_stat)->delta, 0); }) +#define NV_DRIVER_STAT_GET_TOTAL(ptr_stat) /* requires stats_lock */ \ + ((ptr_stat)->total) + /* diagnostics */ #define NV_TEST_COUNT_BASE 3 #define NV_TEST_COUNT_EXTENDED 4 @@ -737,6 +752,12 @@ struct nv_skb_map { * - tx setup is lockless: it relies on netif_tx_lock. Actual submission * needs netdev_priv(dev)->lock :-( * - set_multicast_list: preparation lockless, relies on netif_tx_lock. + * + * Stats are protected with stats_lock: + * - updated by nv_do_stats_poll (timer). This is meant to avoid + * integer wraparound in the NIC stats registers, at low frequency + * (0.1 Hz) + * - updated by nv_get_ethtool_stats + nv_get_stats64 */ /* in dev: base, irq */ @@ -746,9 +767,10 @@ struct fe_priv { struct net_device *dev; struct napi_struct napi; - /* General data: - * Locking: spin_lock(&np->lock); */ + /* stats are updated in syscall and timer */ + spinlock_t stats_lock; struct nv_ethtool_stats estats; + int in_shutdown; u32 linkspeed; int duplex; @@ -799,6 +821,10 @@ struct fe_priv { u32 nic_poll_irq; int rx_ring_size; + /* RX software stats */ + struct nv_driver_stat stat_rx_bytes; /* not always available in HW */ + struct nv_driver_stat stat_rx_missed_errors; + /* media detection workaround. * Locking: Within irq hander or disable_irq+spin_lock(&np->lock); */ @@ -821,6 +847,10 @@ struct fe_priv { struct nv_skb_map *tx_end_flip; int tx_stop; + /* TX software stats */ + struct nv_driver_stat stat_tx_packets; /* not always available in HW */ + struct nv_driver_stat stat_tx_dropped; + /* msi/msi-x fields */ u32 msi_flags; struct msix_entry msi_x_entry[NV_MSI_X_MAX_VECTORS]; @@ -1636,11 +1666,19 @@ static void nv_mac_reset(struct net_device *dev) pci_push(base); } -static void nv_get_hw_stats(struct net_device *dev) +/* Caller must appropriately lock netdev_priv(dev)->stats_lock */ +static void nv_update_stats(struct net_device *dev) { struct fe_priv *np = netdev_priv(dev); u8 __iomem *base = get_hwbase(dev); + /* If it happens that this is run in top-half context, then + * replace the spin_lock of stats_lock with + * spin_lock_irqsave() in calling functions. */ + WARN_ONCE(in_irq(), "forcedeth: estats spin_lock(_bh) from top-half"); + assert_spin_locked(&np->stats_lock); + + /* query hardware */ np->estats.tx_bytes += readl(base + NvRegTxCnt); np->estats.tx_zero_rexmt += readl(base + NvRegTxZeroReXmt); np->estats.tx_one_rexmt += readl(base + NvRegTxOneReXmt); @@ -1696,44 +1734,75 @@ static void nv_get_hw_stats(struct net_device *dev) np->estats.tx_multicast += readl(base + NvRegTxMulticast); np->estats.tx_broadcast += readl(base + NvRegTxBroadcast); } + + /* update software stats */ + NV_DRIVER_STAT_UPDATE_TOTAL(&np->stat_rx_bytes); + NV_DRIVER_STAT_UPDATE_TOTAL(&np->stat_rx_missed_errors); + + NV_DRIVER_STAT_UPDATE_TOTAL(&np->stat_tx_packets); + NV_DRIVER_STAT_UPDATE_TOTAL(&np->stat_tx_dropped); } /* - * nv_get_stats: dev->get_stats function + * nv_get_stats64: dev->ndo_get_stats64 function * Get latest stats value from the nic. * Called with read_lock(&dev_base_lock) held for read - * only synchronized against unregister_netdevice. */ -static struct net_device_stats *nv_get_stats(struct net_device *dev) +static struct rtnl_link_stats64* +nv_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *storage) + __acquires(&netdev_priv(dev)->stats_lock) + __releases(&netdev_priv(dev)->stats_lock) { struct fe_priv *np = netdev_priv(dev); /* If the nic supports hw counters then retrieve latest values */ if (np->driver_data & DEV_HAS_STATISTICS_V123) { - /* query hardware */ - nv_get_hw_stats(dev); + spin_lock_bh(&np->stats_lock); - /* copy to net_device stats */ + nv_update_stats(dev); + + /* generic stats */ if (np->driver_data & DEV_HAS_STATISTICS_V23) { /* When HW stats are available for following * stats, we use them. Otherwise they are * updated by software. */ - dev->stats.tx_packets = np->estats.tx_packets; - dev->stats.rx_bytes = np->estats.rx_bytes; + storage->tx_packets = np->estats.tx_packets; + storage->rx_bytes = np->estats.rx_bytes; + } else { + storage->tx_packets = NV_DRIVER_STAT_GET_TOTAL( + &np->stat_tx_packets); + storage->rx_bytes = NV_DRIVER_STAT_GET_TOTAL( + &np->stat_rx_bytes); } - dev->stats.rx_packets = np->estats.rx_packets; - dev->stats.tx_bytes = np->estats.tx_bytes; - dev->stats.tx_fifo_errors = np->estats.tx_fifo_errors; - dev->stats.tx_carrier_errors = np->estats.tx_carrier_errors; - dev->stats.rx_crc_errors = np->estats.rx_crc_errors; - dev->stats.rx_over_errors = np->estats.rx_over_errors; - dev->stats.rx_fifo_errors = np->estats.rx_drop_frame; - dev->stats.rx_errors = np->estats.rx_errors_total; - dev->stats.tx_errors = np->estats.tx_errors_total; - } - - return &dev->stats; + storage->rx_packets = np->estats.rx_packets; + storage->tx_bytes = np->estats.tx_bytes; + storage->rx_errors = np->estats.rx_errors_total; + storage->tx_errors = np->estats.tx_errors_total; + storage->tx_dropped = NV_DRIVER_STAT_GET_TOTAL( + &np->stat_tx_dropped); + + /* meaningful only when NIC supports stats v3 */ + storage->multicast = np->estats.rx_multicast; + + /* detailed rx_errors */ + storage->rx_length_errors = np->estats.rx_length_error; + storage->rx_over_errors = np->estats.rx_over_errors; + storage->rx_crc_errors = np->estats.rx_crc_errors; + storage->rx_frame_errors = np->estats.rx_frame_align_error; + storage->rx_fifo_errors = np->estats.rx_drop_frame; + storage->rx_missed_errors = NV_DRIVER_STAT_GET_TOTAL( + &np->stat_rx_missed_errors); + + /* detailed tx_errors */ + storage->tx_carrier_errors = np->estats.tx_carrier_errors; + storage->tx_fifo_errors = np->estats.tx_fifo_errors; + + spin_unlock_bh(&np->stats_lock); + } + + return storage; } /* @@ -1935,7 +2004,7 @@ static void nv_drain_tx(struct net_device *dev) np->tx_ring.ex[i].buflow = 0; } if (nv_release_txskb(np, &np->tx_skb[i])) - dev->stats.tx_dropped++; + NV_DRIVER_STAT_ATOMIC_INC(&np->stat_tx_dropped); np->tx_skb[i].dma = 0; np->tx_skb[i].dma_len = 0; np->tx_skb[i].dma_single = 0; @@ -2397,7 +2466,7 @@ static int nv_tx_done(struct net_device *dev, int limit) nv_legacybackoff_reseed(dev); } else if (unlikely(!(np->driver_data & DEV_HAS_STATISTICS_V23))) { - dev->stats.tx_packets++; + NV_DRIVER_STAT_ATOMIC_INC(&np->stat_tx_packets); } dev_kfree_skb_any(np->get_tx_ctx->skb); np->get_tx_ctx->skb = NULL; @@ -2411,7 +2480,7 @@ static int nv_tx_done(struct net_device *dev, int limit) nv_legacybackoff_reseed(dev); } else if (unlikely(!(np->driver_data & DEV_HAS_STATISTICS_V23))) { - dev->stats.tx_packets++; + NV_DRIVER_STAT_ATOMIC_INC(&np->stat_tx_packets); } dev_kfree_skb_any(np->get_tx_ctx->skb); np->get_tx_ctx->skb = NULL; @@ -2453,7 +2522,7 @@ static int nv_tx_done_optimized(struct net_device *dev, int limit) } } else if (unlikely(!(np->driver_data & DEV_HAS_STATISTICS_V23))) { - dev->stats.tx_packets++; + NV_DRIVER_STAT_ATOMIC_INC(&np->stat_tx_packets); } dev_kfree_skb_any(np->get_tx_ctx->skb); @@ -2667,7 +2736,7 @@ static int nv_rx_process(struct net_device *dev, int limit) /* the rest are hard errors */ else { if (flags & NV_RX_MISSEDFRAME) - dev->stats.rx_missed_errors++; + NV_DRIVER_STAT_ATOMIC_INC(&np->stat_rx_missed_errors); dev_kfree_skb(skb); goto next_pkt; } @@ -2711,7 +2780,7 @@ static int nv_rx_process(struct net_device *dev, int limit) skb->protocol = eth_type_trans(skb, dev); napi_gro_receive(&np->napi, skb); if (unlikely(!(np->driver_data & DEV_HAS_STATISTICS_V23))) - dev->stats.rx_bytes += len; + NV_DRIVER_STAT_ATOMIC_ADD(&np->stat_rx_bytes, len); next_pkt: if (unlikely(np->get_rx.orig++ == np->last_rx.orig)) np->get_rx.orig = np->first_rx.orig; @@ -2796,7 +2865,7 @@ static int nv_rx_process_optimized(struct net_device *dev, int limit) napi_gro_receive(&np->napi, skb); if (unlikely(!(np->driver_data & DEV_HAS_STATISTICS_V23))) - dev->stats.rx_bytes += len; + NV_DRIVER_STAT_ATOMIC_ADD(&np->stat_rx_bytes, len); } else { dev_kfree_skb(skb); } @@ -3924,11 +3993,18 @@ static void nv_poll_controller(struct net_device *dev) #endif static void nv_do_stats_poll(unsigned long data) + __acquires(&netdev_priv(dev)->stats_lock) + __releases(&netdev_priv(dev)->stats_lock) { struct net_device *dev = (struct net_device *) data; struct fe_priv *np = netdev_priv(dev); - nv_get_hw_stats(dev); + /* If lock is currently taken, the stats are being refreshed + * and hence fresh enough */ + if (spin_trylock(&np->stats_lock)) { + nv_update_stats(dev); + spin_unlock(&np->stats_lock); + } if (!np->in_shutdown) mod_timer(&np->stats_poll, @@ -4573,14 +4649,18 @@ static int nv_get_sset_count(struct net_device *dev, int sset) } } -static void nv_get_ethtool_stats(struct net_device *dev, struct ethtool_stats *estats, u64 *buffer) +static void nv_get_ethtool_stats(struct net_device *dev, + struct ethtool_stats *estats, u64 *buffer) + __acquires(&netdev_priv(dev)->stats_lock) + __releases(&netdev_priv(dev)->stats_lock) { struct fe_priv *np = netdev_priv(dev); - /* update stats */ - nv_get_hw_stats(dev); - - memcpy(buffer, &np->estats, nv_get_sset_count(dev, ETH_SS_STATS)*sizeof(u64)); + spin_lock_bh(&np->stats_lock); + nv_update_stats(dev); + memcpy(buffer, &np->estats, + nv_get_sset_count(dev, ETH_SS_STATS)*sizeof(u64)); + spin_unlock_bh(&np->stats_lock); } static int nv_link_test(struct net_device *dev) @@ -5218,7 +5298,7 @@ static int nv_close(struct net_device *dev) static const struct net_device_ops nv_netdev_ops = { .ndo_open = nv_open, .ndo_stop = nv_close, - .ndo_get_stats = nv_get_stats, + .ndo_get_stats64 = nv_get_stats64, .ndo_start_xmit = nv_start_xmit, .ndo_tx_timeout = nv_tx_timeout, .ndo_change_mtu = nv_change_mtu, @@ -5235,7 +5315,7 @@ static const struct net_device_ops nv_netdev_ops = { static const struct net_device_ops nv_netdev_ops_optimized = { .ndo_open = nv_open, .ndo_stop = nv_close, - .ndo_get_stats = nv_get_stats, + .ndo_get_stats64 = nv_get_stats64, .ndo_start_xmit = nv_start_xmit_optimized, .ndo_tx_timeout = nv_tx_timeout, .ndo_change_mtu = nv_change_mtu, @@ -5274,6 +5354,7 @@ static int __devinit nv_probe(struct pci_dev *pci_dev, const struct pci_device_i np->dev = dev; np->pci_dev = pci_dev; spin_lock_init(&np->lock); + spin_lock_init(&np->stats_lock); SET_NETDEV_DEV(dev, &pci_dev->dev); init_timer(&np->oom_kick);