From patchwork Wed Jul 9 13:14:06 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 368220 X-Patchwork-Delegate: pablo@netfilter.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 6A10614011B for ; Wed, 9 Jul 2014 23:14:14 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752309AbaGINOK (ORCPT ); Wed, 9 Jul 2014 09:14:10 -0400 Received: from mail-pd0-f180.google.com ([209.85.192.180]:63941 "EHLO mail-pd0-f180.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751268AbaGINOJ (ORCPT ); Wed, 9 Jul 2014 09:14:09 -0400 Received: by mail-pd0-f180.google.com with SMTP id fp1so8999779pdb.39 for ; Wed, 09 Jul 2014 06:14:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:subject:from:to:cc:date:content-type :content-transfer-encoding:mime-version; bh=8bo55R55h5EGEZYpa9XB8Gqam2vpN35FgfThrWngXEE=; b=mbh5GMj/W3PCZcUo2odfqBXiFCmP8QehfarhtmX1VwLAtsp8D6dCORm60akmJ8pWqQ YiutXyKxPnU9zBtvibCRkhYqhooyu3e9oLnGUms0VTQDXgLYRCMnSCEXC8MZXKHGpYX9 t4CTozI5EM0Xi/RnBQGpnvD8r3648didZ35AuJW9EeBIvV8rhG0nVpLLe7YEhGJsytRS AF6AAOq/x9TCXDisDzDxt/mOoUPMMrLlFyYsoo4iL+QbHI99XceeThh8LidjeKbRVZW+ 3KCcumifyNQFIJphwNucFn8Eb71Zc6REBRNpBhMKec6c8nQ+5+pY7XrR/yOCJgxvu4yc BfOQ== X-Received: by 10.70.41.77 with SMTP id d13mr11208005pdl.3.1404911648766; Wed, 09 Jul 2014 06:14:08 -0700 (PDT) Received: from [151.216.17.33] ([151.216.17.33]) by mx.google.com with ESMTPSA id be7sm28471443pdb.37.2014.07.09.06.14.07 for (version=SSLv3 cipher=RC4-SHA bits=128/128); Wed, 09 Jul 2014 06:14:08 -0700 (PDT) Message-ID: <1404911646.3515.26.camel@edumazet-glaptop2.roam.corp.google.com> Subject: [PATCH] netfilter: nf_tables: 64bit stats need some extra synchronization From: Eric Dumazet To: Pablo Neira Ayuso Cc: Patrick McHardy , netfilter-devel@vger.kernel.org, netdev Date: Wed, 09 Jul 2014 15:14:06 +0200 X-Mailer: Evolution 3.2.3-0ubuntu6 Mime-Version: 1.0 Sender: netfilter-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netfilter-devel@vger.kernel.org From: Eric Dumazet Use generic u64_stats_sync infrastructure to get proper 64bit stats, even on 32bit arches, at no extra cost for 64bit arches. Without this fix, 32bit arches can have some wrong counters at the time the carry is propagated into upper word. Signed-off-by: Eric Dumazet --- Note: Compiled only, I haven't tested this on 32bit kernel. include/net/netfilter/nf_tables.h | 6 ++++-- net/netfilter/nf_tables_api.c | 15 +++++++++++---- net/netfilter/nf_tables_core.c | 10 ++++++---- 3 files changed, 21 insertions(+), 10 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/include/net/netfilter/nf_tables.h b/include/net/netfilter/nf_tables.h index 713b0b88bd5a..c4d86198d3d6 100644 --- a/include/net/netfilter/nf_tables.h +++ b/include/net/netfilter/nf_tables.h @@ -6,6 +6,7 @@ #include #include #include +#include #include #define NFT_JUMP_STACK_SIZE 16 @@ -528,8 +529,9 @@ enum nft_chain_type { }; struct nft_stats { - u64 bytes; - u64 pkts; + u64 bytes; + u64 pkts; + struct u64_stats_sync syncp; }; #define NFT_HOOK_OPS_MAX 2 diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c index ab4566cfcbe4..add177afa308 100644 --- a/net/netfilter/nf_tables_api.c +++ b/net/netfilter/nf_tables_api.c @@ -635,13 +635,20 @@ static int nft_dump_stats(struct sk_buff *skb, struct nft_stats __percpu *stats) { struct nft_stats *cpu_stats, total; struct nlattr *nest; + unsigned int seq; + u64 pkts, bytes; int cpu; memset(&total, 0, sizeof(total)); for_each_possible_cpu(cpu) { cpu_stats = per_cpu_ptr(stats, cpu); - total.pkts += cpu_stats->pkts; - total.bytes += cpu_stats->bytes; + do { + seq = u64_stats_fetch_begin_irq(&cpu_stats->syncp); + pkts = cpu_stats->pkts; + bytes = cpu_stats->bytes; + } while (u64_stats_fetch_retry_irq(&cpu_stats->syncp, seq)); + total.pkts += pkts; + total.bytes += bytes; } nest = nla_nest_start(skb, NFTA_CHAIN_COUNTERS); if (nest == NULL) @@ -861,7 +868,7 @@ static struct nft_stats __percpu *nft_stats_alloc(const struct nlattr *attr) if (!tb[NFTA_COUNTER_BYTES] || !tb[NFTA_COUNTER_PACKETS]) return ERR_PTR(-EINVAL); - newstats = alloc_percpu(struct nft_stats); + newstats = netdev_alloc_pcpu_stats(struct nft_stats); if (newstats == NULL) return ERR_PTR(-ENOMEM); @@ -1077,7 +1084,7 @@ static int nf_tables_newchain(struct sock *nlsk, struct sk_buff *skb, } basechain->stats = stats; } else { - stats = alloc_percpu(struct nft_stats); + stats = netdev_alloc_pcpu_stats(struct nft_stats); if (IS_ERR(stats)) { module_put(type->owner); kfree(basechain); diff --git a/net/netfilter/nf_tables_core.c b/net/netfilter/nf_tables_core.c index 345acfb1720b..3b90eb2b2c55 100644 --- a/net/netfilter/nf_tables_core.c +++ b/net/netfilter/nf_tables_core.c @@ -109,7 +109,7 @@ nft_do_chain(struct nft_pktinfo *pkt, const struct nf_hook_ops *ops) struct nft_data data[NFT_REG_MAX + 1]; unsigned int stackptr = 0; struct nft_jumpstack jumpstack[NFT_JUMP_STACK_SIZE]; - struct nft_stats __percpu *stats; + struct nft_stats *stats; int rulenum; /* * Cache cursor to avoid problems in case that the cursor is updated @@ -205,9 +205,11 @@ next_rule: nft_trace_packet(pkt, basechain, -1, NFT_TRACE_POLICY); rcu_read_lock_bh(); - stats = rcu_dereference(nft_base_chain(basechain)->stats); - __this_cpu_inc(stats->pkts); - __this_cpu_add(stats->bytes, pkt->skb->len); + stats = this_cpu_ptr(rcu_dereference(nft_base_chain(basechain)->stats)); + u64_stats_update_begin(&stats->syncp); + stats->pkts++; + stats->bytes += pkt->skb->len; + u64_stats_update_end(&stats->syncp); rcu_read_unlock_bh(); return nft_base_chain(basechain)->policy;