From patchwork Thu Dec 11 22:39:18 2008 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 13603 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by ozlabs.org (Postfix) with ESMTP id C6A66DE010 for ; Fri, 12 Dec 2008 09:40:28 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758969AbYLKWkC (ORCPT ); Thu, 11 Dec 2008 17:40:02 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1758682AbYLKWj7 (ORCPT ); Thu, 11 Dec 2008 17:39:59 -0500 Received: from gw1.cosmosbay.com ([86.65.150.130]:35435 "EHLO gw1.cosmosbay.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758691AbYLKWjy (ORCPT ); Thu, 11 Dec 2008 17:39:54 -0500 Received: from [127.0.0.1] (localhost [127.0.0.1]) by gw1.cosmosbay.com (8.13.7/8.13.7) with ESMTP id mBBMdI12013658; Thu, 11 Dec 2008 23:39:18 +0100 Message-ID: <49419696.9080509@cosmosbay.com> Date: Thu, 11 Dec 2008 23:39:18 +0100 From: Eric Dumazet User-Agent: Thunderbird 2.0.0.18 (Windows/20081105) MIME-Version: 1.0 To: Andrew Morton CC: Ingo Molnar , Christoph Hellwig , David Miller , "Rafael J. Wysocki" , linux-kernel@vger.kernel.org, "kernel-testers@vger.kernel.org >> Kernel Testers List" , Mike Galbraith , Peter Zijlstra , Linux Netdev List , Christoph Lameter , linux-fsdevel@vger.kernel.org, Al Viro , "Paul E. McKenney" Subject: [PATCH v3 3/7] fs: Introduce a per_cpu last_ino allocator References: <20081121083044.GL16242@elte.hu> <49267694.1030506@cosmosbay.com> <20081121.010508.40225532.davem@davemloft.net> <4926AEDB.10007@cosmosbay.com> <4926D022.5060008@cosmosbay.com> <20081121152148.GA20388@elte.hu> <4926D39D.9050603@cosmosbay.com> <20081121153453.GA23713@elte.hu> <492DDB6A.8090806@cosmosbay.com> <493100B0.6090104@cosmosbay.com> In-Reply-To: <493100B0.6090104@cosmosbay.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.6 (gw1.cosmosbay.com [0.0.0.0]); Thu, 11 Dec 2008 23:39:19 +0100 (CET) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org new_inode() dirties a contended cache line to get increasing inode numbers. Solve this problem by providing to each cpu a per_cpu variable, feeded by the shared last_ino, but once every 1024 allocations. This reduce contention on the shared last_ino, and give same spreading ino numbers than before. (same wraparound after 2^32 allocations) Signed-off-by: Eric Dumazet Reviewed-by: Paul E. McKenney --- fs/inode.c | 35 ++++++++++++++++++++++++++++++++--- 1 files changed, 32 insertions(+), 3 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/fs/inode.c b/fs/inode.c index f94f889..dc8e72a 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -556,6 +556,36 @@ repeat: return node ? inode : NULL; } +#ifdef CONFIG_SMP +/* + * Each cpu owns a range of 1024 numbers. + * 'shared_last_ino' is dirtied only once out of 1024 allocations, + * to renew the exhausted range. + */ +static DEFINE_PER_CPU(int, last_ino); + +static int last_ino_get(void) +{ + static atomic_t shared_last_ino; + int *p = &get_cpu_var(last_ino); + int res = *p; + + if (unlikely((res & 1023) == 0)) + res = atomic_add_return(1024, &shared_last_ino) - 1024; + + *p = ++res; + put_cpu_var(last_ino); + return res; +} +#else +static int last_ino_get(void) +{ + static int last_ino; + + return ++last_ino; +} +#endif + /** * new_inode - obtain an inode * @sb: superblock @@ -575,7 +605,6 @@ struct inode *new_inode(struct super_block *sb) * error if st_ino won't fit in target struct field. Use 32bit counter * here to attempt to avoid that. */ - static unsigned int last_ino; struct inode * inode; spin_lock_prefetch(&inode_lock); @@ -583,11 +612,11 @@ struct inode *new_inode(struct super_block *sb) inode = alloc_inode(sb); if (inode) { percpu_counter_inc(&nr_inodes); + inode->i_state = 0; + inode->i_ino = last_ino_get(); spin_lock(&inode_lock); list_add(&inode->i_list, &inode_in_use); list_add(&inode->i_sb_list, &sb->s_inodes); - inode->i_ino = ++last_ino; - inode->i_state = 0; spin_unlock(&inode_lock); } return inode;