| Message ID | 1442703892-26692-6-git-send-email-santosh.shilimkar@oracle.com |
|---|---|
| State | Not Applicable, archived |
| Delegated to: | David Miller |
| Headers | show |
From: Santosh Shilimkar > Sent: 20 September 2015 00:05 > Even with per bucket locking scheme, in a massive parallel > system with active rds sockets which could be in excess of multiple > of 10K, rds_bin_lookup() workload is siginificant because of smaller > hashtable size. > > With some tests, it was found that we get modest but still nice > reduction in rds_bind_lookup with bigger bucket. > > Hashtable Baseline(1k) Delta > 2048: 8.28% -2.45% > 4096: 8.28% -4.60% > 8192: 8.28% -6.46% > 16384: 8.28% -6.75% > > Based on the data, we set 8K as the bind hash-table size. Can't you use of on the dynamically sizing hash tables? 8k hash table entries is OTT for a lot of systems. David -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 9/21/2015 1:31 AM, David Laight wrote: > From: Santosh Shilimkar >> Sent: 20 September 2015 00:05 >> Even with per bucket locking scheme, in a massive parallel >> system with active rds sockets which could be in excess of multiple >> of 10K, rds_bin_lookup() workload is significant because of smaller >> hashtable size. >> >> With some tests, it was found that we get modest but still nice >> reduction in rds_bind_lookup with bigger bucket. >> >> Hashtable Baseline(1k) Delta >> 2048: 8.28% -2.45% >> 4096: 8.28% -4.60% >> 8192: 8.28% -6.46% >> 16384: 8.28% -6.75% >> >> Based on the data, we set 8K as the bind hash-table size. > > Can't you use of on the dynamically sizing hash tables? > 8k hash table entries is OTT for a lot of systems. > Do you know an example in Linux kernel uses that ? What I certainly don't want is over-head of re-sizing whenever that happens in running systems running multiple databases. Memory is certainly not an issue on the systems where RDS has been deployed. I certainly don't want to over-use the memory but in the system where RDS being used and also amount of connection it needs to handle, it needs bigger bucket. Regards, Santosh -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Santosh Shilimkar <santosh.shilimkar@oracle.com> Date: Sat, 19 Sep 2015 19:04:42 -0400 > Even with per bucket locking scheme, in a massive parallel > system with active rds sockets which could be in excess of multiple > of 10K, rds_bin_lookup() workload is siginificant because of smaller > hashtable size. > > With some tests, it was found that we get modest but still nice > reduction in rds_bind_lookup with bigger bucket. > > Hashtable Baseline(1k) Delta > 2048: 8.28% -2.45% > 4096: 8.28% -4.60% > 8192: 8.28% -6.46% > 16384: 8.28% -6.75% > > Based on the data, we set 8K as the bind hash-table size. > > Signed-off-by: Santosh Shilimkar <ssantosh@kernel.org> > Signed-off-by: Santosh Shilimkar <santosh.shilimkar@oracle.com> Like others I would strongly prefer that you use a dynamically sized hash table. Eating 8k just because a module just happened to get loaded is really not appropriate. And there are many other places that use such a scheme, one example is the AF_NETLINK socket hash table. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 9/21/2015 4:05 PM, David Miller wrote: > From: Santosh Shilimkar <santosh.shilimkar@oracle.com> > Date: Sat, 19 Sep 2015 19:04:42 -0400 > >> Even with per bucket locking scheme, in a massive parallel >> system with active rds sockets which could be in excess of multiple >> of 10K, rds_bin_lookup() workload is siginificant because of smaller >> hashtable size. >> >> With some tests, it was found that we get modest but still nice >> reduction in rds_bind_lookup with bigger bucket. >> >> Hashtable Baseline(1k) Delta >> 2048: 8.28% -2.45% >> 4096: 8.28% -4.60% >> 8192: 8.28% -6.46% >> 16384: 8.28% -6.75% >> >> Based on the data, we set 8K as the bind hash-table size. >> >> Signed-off-by: Santosh Shilimkar <ssantosh@kernel.org> >> Signed-off-by: Santosh Shilimkar <santosh.shilimkar@oracle.com> > > Like others I would strongly prefer that you use a dynamically sized > hash table. > > Eating 8k just because a module just happened to get loaded is really > not appropriate. > > And there are many other places that use such a scheme, one example is > the AF_NETLINK socket hash table. OK. Thanks for AF_NETLINK pointer. I will look it up. Regards, Santosh -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Dave, On 9/21/2015 4:55 PM, santosh shilimkar wrote: > On 9/21/2015 4:05 PM, David Miller wrote: >> From: Santosh Shilimkar <santosh.shilimkar@oracle.com> >> Date: Sat, 19 Sep 2015 19:04:42 -0400 >> >>> Even with per bucket locking scheme, in a massive parallel >>> system with active rds sockets which could be in excess of multiple >>> of 10K, rds_bin_lookup() workload is siginificant because of smaller >>> hashtable size. >>> >>> With some tests, it was found that we get modest but still nice >>> reduction in rds_bind_lookup with bigger bucket. >>> >>> Hashtable Baseline(1k) Delta >>> 2048: 8.28% -2.45% >>> 4096: 8.28% -4.60% >>> 8192: 8.28% -6.46% >>> 16384: 8.28% -6.75% >>> >>> Based on the data, we set 8K as the bind hash-table size. >>> >>> Signed-off-by: Santosh Shilimkar <ssantosh@kernel.org> >>> Signed-off-by: Santosh Shilimkar <santosh.shilimkar@oracle.com> >> >> Like others I would strongly prefer that you use a dynamically sized >> hash table. >> >> Eating 8k just because a module just happened to get loaded is really >> not appropriate. >> >> And there are many other places that use such a scheme, one example is >> the AF_NETLINK socket hash table. > > OK. Thanks for AF_NETLINK pointer. I will look it up. > I will follow your advice on resizable hash table usage. It seems to be neat and fits well. But I want to make sure that the implementation works for all the workloads so it will take some time. Hopefully I can get that ready with testing for 4.5. So for now,lets just drop the $subject patch from this series. Do you want me to resend the series with the $subject patch dropped, or you can apply rest of the series except this one. Let me know. Thanks !! Regards, Santosh -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/net/rds/bind.c b/net/rds/bind.c index bc6b93e..fb2d545 100644 --- a/net/rds/bind.c +++ b/net/rds/bind.c @@ -43,7 +43,7 @@ struct bind_bucket { struct hlist_head head; }; -#define BIND_HASH_SIZE 1024 +#define BIND_HASH_SIZE 8192 static struct bind_bucket bind_hash_table[BIND_HASH_SIZE]; static struct bind_bucket *hash_to_bucket(__be32 addr, __be16 port)