From patchwork Wed Oct 13 18:22:03 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 67721 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 7C109B70D4 for ; Thu, 14 Oct 2010 05:22:56 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752224Ab0JMSWw (ORCPT ); Wed, 13 Oct 2010 14:22:52 -0400 Received: from mail-ww0-f42.google.com ([74.125.82.42]:48666 "EHLO mail-ww0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751363Ab0JMSWv (ORCPT ); Wed, 13 Oct 2010 14:22:51 -0400 Received: by wwi14 with SMTP id 14so175021wwi.1 for ; Wed, 13 Oct 2010 11:22:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:from:to:cc :content-type:date:message-id:mime-version:x-mailer :content-transfer-encoding; bh=zaDwfuNrUl4ciVEiGFKUbFSlvzftxDaVr4nnWtpka/g=; b=aUNHE+UwZxihWAH/LAvMPu5fqCbmCslKboQtbfDfWS37fyjBEJF6woFywx3nHpYjT8 IJxqRz1m7t9I96ipdHhLU+L7wsR42TW1XvLtQRPUSs0JLf7MPteXmE7QFW/78jrjrAtK vE7f6ylJydLC2VpoWQxhOS5vlxZ1DYHdrkxc4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:from:to:cc:content-type:date:message-id:mime-version :x-mailer:content-transfer-encoding; b=k5Gh0Ne9TsbrXDpSj6dBnbViY56PjUcAh+i3qCaAhSBvDNuQztAkiCKaDn4k51zRke Tw07SwCQsurh5PSrVkcitvZSvVpHq524t2uX5fSj6A+wJBDPZIbU0PI2EnzYu26aoGzJ o35Imz17rs6an4oUK0BkG9gc0u3DdJkAujUMM= Received: by 10.227.128.197 with SMTP id l5mr7552944wbs.179.1286994127970; Wed, 13 Oct 2010 11:22:07 -0700 (PDT) Received: from [10.150.51.214] (gw0.net.jmsp.net [212.23.165.14]) by mx.google.com with ESMTPS id b30sm5541084wbb.22.2010.10.13.11.22.06 (version=SSLv3 cipher=RC4-MD5); Wed, 13 Oct 2010 11:22:06 -0700 (PDT) Subject: [PATCH net-next] fib: avoid false sharing on fib_table_hash From: Eric Dumazet To: David Miller Cc: netdev Date: Wed, 13 Oct 2010 20:22:03 +0200 Message-ID: <1286994123.3876.1137.camel@edumazet-laptop> Mime-Version: 1.0 X-Mailer: Evolution 2.30.3 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org While doing profile analysis, I found fib_hash_table was sometime in a cache line shared by a possibly often written kernel structure. (CONFIG_IP_ROUTE_MULTIPATH || !CONFIG_IPV6_MULTIPLE_TABLES) It's hard to detect because not easily reproductible. Make sure we allocate a full cache line to keep this shared in all cpus caches. Signed-off-by: Eric Dumazet --- We probably need a generic kernel function, but it might take time... net/ipv4/fib_frontend.c | 11 +++++------ net/ipv6/ip6_fib.c | 9 ++++++--- 2 files changed, 11 insertions(+), 9 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/ipv4/fib_frontend.c b/net/ipv4/fib_frontend.c index 919f2ad..3df057e 100644 --- a/net/ipv4/fib_frontend.c +++ b/net/ipv4/fib_frontend.c @@ -1016,16 +1016,15 @@ static struct notifier_block fib_netdev_notifier = { static int __net_init ip_fib_net_init(struct net *net) { int err; - unsigned int i; + size_t size = sizeof(struct hlist_head) * FIB_TABLE_HASHSZ; + + /* Avoid false sharing : Use at least a full cache line */ + size = max_t(size_t, size, L1_CACHE_BYTES); - net->ipv4.fib_table_hash = kzalloc( - sizeof(struct hlist_head)*FIB_TABLE_HASHSZ, GFP_KERNEL); + net->ipv4.fib_table_hash = kzalloc(size, GFP_KERNEL); if (net->ipv4.fib_table_hash == NULL) return -ENOMEM; - for (i = 0; i < FIB_TABLE_HASHSZ; i++) - INIT_HLIST_HEAD(&net->ipv4.fib_table_hash[i]); - err = fib4_rules_init(net); if (err < 0) goto fail; diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c index b6a5859..de38211 100644 --- a/net/ipv6/ip6_fib.c +++ b/net/ipv6/ip6_fib.c @@ -1500,15 +1500,18 @@ static void fib6_gc_timer_cb(unsigned long arg) static int __net_init fib6_net_init(struct net *net) { + size_t size = sizeof(struct hlist_head) * FIB6_TABLE_HASHSZ; + setup_timer(&net->ipv6.ip6_fib_timer, fib6_gc_timer_cb, (unsigned long)net); net->ipv6.rt6_stats = kzalloc(sizeof(*net->ipv6.rt6_stats), GFP_KERNEL); if (!net->ipv6.rt6_stats) goto out_timer; - net->ipv6.fib_table_hash = kcalloc(FIB6_TABLE_HASHSZ, - sizeof(*net->ipv6.fib_table_hash), - GFP_KERNEL); + /* Avoid false sharing : Use at least a full cache line */ + size = max_t(size_t, size, L1_CACHE_BYTES); + + net->ipv6.fib_table_hash = kzalloc(size, GFP_KERNEL); if (!net->ipv6.fib_table_hash) goto out_rt6_stats;