From patchwork Tue Nov 17 14:53:09 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 38639 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by ozlabs.org (Postfix) with ESMTP id CBE82B7B96 for ; Wed, 18 Nov 2009 01:53:21 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753919AbZKQOxJ (ORCPT ); Tue, 17 Nov 2009 09:53:09 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1753907AbZKQOxJ (ORCPT ); Tue, 17 Nov 2009 09:53:09 -0500 Received: from gw1.cosmosbay.com ([212.99.114.194]:53594 "EHLO gw1.cosmosbay.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750905AbZKQOxI (ORCPT ); Tue, 17 Nov 2009 09:53:08 -0500 Received: from [127.0.0.1] (localhost [127.0.0.1]) by gw1.cosmosbay.com (8.13.7/8.13.7) with ESMTP id nAHErA62006096; Tue, 17 Nov 2009 15:53:10 +0100 Message-ID: <4B02B8D5.8040402@gmail.com> Date: Tue, 17 Nov 2009 15:53:09 +0100 From: Eric Dumazet User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: David Miller CC: netdev@vger.kernel.org Subject: Re: [PATCH 2/2 net-next-2.6] vlan: Precise RX stats accounting References: <4B01558F.5000306@gmail.com> <20091117.002037.229474882.davem@davemloft.net> In-Reply-To: <20091117.002037.229474882.davem@davemloft.net> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.6 (gw1.cosmosbay.com [0.0.0.0]); Tue, 17 Nov 2009 15:53:10 +0100 (CET) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org David Miller a écrit : > > Eric, please make allocation failure cause the vlan_setup() > to fail and propagate -EFAULT back to the caller. > Given that vlan_setup(struct net_device *) is void and cannot return error status, unless using a global variable (ugly), or changing all setup() prototypes (oh well...) One possibility is to perform the allocation in vlan_dev_init() (called from register_netdevice()), and freeing it in vlan_dev_uninit(). Thanks [PATCH net-next-2.6 take2] vlan: Precise RX stats accounting With multi queue devices, its possible that several cpus call vlan RX routines simultaneously for the same vlan device. We update RX stats counter without any locking, so we can get slightly wrong counters. One possible fix is to use percpu counters, to get precise accounting and also get guarantee of no cache line ping pongs between cpus. Note: this adds 16 bytes (32 bytes on 64bit arches) of percpu data per vlan device. Signed-off-by: Eric Dumazet --- net/8021q/vlan.h | 17 ++++++++++++++ net/8021q/vlan_core.c | 12 +++++----- net/8021q/vlan_dev.c | 47 ++++++++++++++++++++++++++++++++++------ 3 files changed, 65 insertions(+), 11 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/8021q/vlan.h b/net/8021q/vlan.h index 68f9290..5685296 100644 --- a/net/8021q/vlan.h +++ b/net/8021q/vlan.h @@ -16,6 +16,21 @@ struct vlan_priority_tci_mapping { struct vlan_priority_tci_mapping *next; }; + +/** + * struct vlan_rx_stats - VLAN percpu rx stats + * @rx_packets: number of received packets + * @rx_bytes: number of received bytes + * @multicast: number of received multicast packets + * @rx_errors: number of errors + */ +struct vlan_rx_stats { + unsigned long rx_packets; + unsigned long rx_bytes; + unsigned long multicast; + unsigned long rx_errors; +}; + /** * struct vlan_dev_info - VLAN private device data * @nr_ingress_mappings: number of ingress priority mappings @@ -29,6 +44,7 @@ struct vlan_priority_tci_mapping { * @dent: proc dir entry * @cnt_inc_headroom_on_tx: statistic - number of skb expansions on TX * @cnt_encap_on_xmit: statistic - number of skb encapsulations on TX + * @vlan_rx_stats: ptr to percpu rx stats */ struct vlan_dev_info { unsigned int nr_ingress_mappings; @@ -45,6 +61,7 @@ struct vlan_dev_info { struct proc_dir_entry *dent; unsigned long cnt_inc_headroom_on_tx; unsigned long cnt_encap_on_xmit; + struct vlan_rx_stats *vlan_rx_stats; }; static inline struct vlan_dev_info *vlan_dev_info(const struct net_device *dev) diff --git a/net/8021q/vlan_core.c b/net/8021q/vlan_core.c index 971d375..e75a2f3 100644 --- a/net/8021q/vlan_core.c +++ b/net/8021q/vlan_core.c @@ -31,7 +31,7 @@ EXPORT_SYMBOL(__vlan_hwaccel_rx); int vlan_hwaccel_do_receive(struct sk_buff *skb) { struct net_device *dev = skb->dev; - struct net_device_stats *stats; + struct vlan_rx_stats *rx_stats; skb->dev = vlan_dev_info(dev)->real_dev; netif_nit_deliver(skb); @@ -40,15 +40,17 @@ int vlan_hwaccel_do_receive(struct sk_buff *skb) skb->priority = vlan_get_ingress_priority(dev, skb->vlan_tci); skb->vlan_tci = 0; - stats = &dev->stats; - stats->rx_packets++; - stats->rx_bytes += skb->len; + rx_stats = per_cpu_ptr(vlan_dev_info(dev)->vlan_rx_stats, + smp_processor_id()); + + rx_stats->rx_packets++; + rx_stats->rx_bytes += skb->len; switch (skb->pkt_type) { case PACKET_BROADCAST: break; case PACKET_MULTICAST: - stats->multicast++; + rx_stats->multicast++; break; case PACKET_OTHERHOST: /* Our lower layer thinks this is not local, let's make sure. diff --git a/net/8021q/vlan_dev.c b/net/8021q/vlan_dev.c index 9159659..de0dc6b 100644 --- a/net/8021q/vlan_dev.c +++ b/net/8021q/vlan_dev.c @@ -140,7 +140,7 @@ int vlan_skb_recv(struct sk_buff *skb, struct net_device *dev, struct packet_type *ptype, struct net_device *orig_dev) { struct vlan_hdr *vhdr; - struct net_device_stats *stats; + struct vlan_rx_stats *rx_stats; u16 vlan_id; u16 vlan_tci; @@ -163,9 +163,10 @@ int vlan_skb_recv(struct sk_buff *skb, struct net_device *dev, goto err_unlock; } - stats = &skb->dev->stats; - stats->rx_packets++; - stats->rx_bytes += skb->len; + rx_stats = per_cpu_ptr(vlan_dev_info(dev)->vlan_rx_stats, + smp_processor_id()); + rx_stats->rx_packets++; + rx_stats->rx_bytes += skb->len; skb_pull_rcsum(skb, VLAN_HLEN); @@ -180,7 +181,7 @@ int vlan_skb_recv(struct sk_buff *skb, struct net_device *dev, break; case PACKET_MULTICAST: - stats->multicast++; + rx_stats->multicast++; break; case PACKET_OTHERHOST: @@ -200,7 +201,7 @@ int vlan_skb_recv(struct sk_buff *skb, struct net_device *dev, skb = vlan_check_reorder_header(skb); if (!skb) { - stats->rx_errors++; + rx_stats->rx_errors++; goto err_unlock; } @@ -731,6 +732,11 @@ static int vlan_dev_init(struct net_device *dev) subclass = 1; vlan_dev_set_lockdep_class(dev, subclass); + + vlan_dev_info(dev)->vlan_rx_stats = alloc_percpu(struct vlan_rx_stats); + if (!vlan_dev_info(dev)->vlan_rx_stats) + return -ENOMEM; + return 0; } @@ -740,6 +746,8 @@ static void vlan_dev_uninit(struct net_device *dev) struct vlan_dev_info *vlan = vlan_dev_info(dev); int i; + free_percpu(vlan->vlan_rx_stats); + vlan->vlan_rx_stats = NULL; for (i = 0; i < ARRAY_SIZE(vlan->egress_priority_map); i++) { while ((pm = vlan->egress_priority_map[i]) != NULL) { vlan->egress_priority_map[i] = pm->next; @@ -775,6 +783,31 @@ static u32 vlan_ethtool_get_flags(struct net_device *dev) return dev_ethtool_get_flags(vlan->real_dev); } +static struct net_device_stats *vlan_dev_get_stats(struct net_device *dev) +{ + struct net_device_stats *stats = &dev->stats; + + dev_txq_stats_fold(dev, stats); + + if (vlan_dev_info(dev)->vlan_rx_stats) { + struct vlan_rx_stats *p, rx = {0}; + int i; + + for_each_possible_cpu(i) { + p = per_cpu_ptr(vlan_dev_info(dev)->vlan_rx_stats, i); + rx.rx_packets += p->rx_packets; + rx.rx_bytes += p->rx_bytes; + rx.rx_errors += p->rx_errors; + rx.multicast += p->multicast; + } + stats->rx_packets = rx.rx_packets; + stats->rx_bytes = rx.rx_bytes; + stats->rx_errors = rx.rx_errors; + stats->multicast = rx.multicast; + } + return stats; +} + static const struct ethtool_ops vlan_ethtool_ops = { .get_settings = vlan_ethtool_get_settings, .get_drvinfo = vlan_ethtool_get_drvinfo, @@ -797,6 +830,7 @@ static const struct net_device_ops vlan_netdev_ops = { .ndo_change_rx_flags = vlan_dev_change_rx_flags, .ndo_do_ioctl = vlan_dev_ioctl, .ndo_neigh_setup = vlan_dev_neigh_setup, + .ndo_get_stats = vlan_dev_get_stats, #if defined(CONFIG_FCOE) || defined(CONFIG_FCOE_MODULE) .ndo_fcoe_ddp_setup = vlan_dev_fcoe_ddp_setup, .ndo_fcoe_ddp_done = vlan_dev_fcoe_ddp_done, @@ -820,6 +854,7 @@ static const struct net_device_ops vlan_netdev_accel_ops = { .ndo_change_rx_flags = vlan_dev_change_rx_flags, .ndo_do_ioctl = vlan_dev_ioctl, .ndo_neigh_setup = vlan_dev_neigh_setup, + .ndo_get_stats = vlan_dev_get_stats, #if defined(CONFIG_FCOE) || defined(CONFIG_FCOE_MODULE) .ndo_fcoe_ddp_setup = vlan_dev_fcoe_ddp_setup, .ndo_fcoe_ddp_done = vlan_dev_fcoe_ddp_done,