rib_trie / Fix inflate_threshold_root. Now=15 size=11 bits

Robert Olsson wrote, On 06/26/2009 11:19 AM:

> Jarek Poplawski writes:
> 
>  > >> oprofile: using NMI interrupt.
>  > >> Fix inflate_threshold_root. Now=15 size=11 bits
>  > >> Fix inflate_threshold_root. Now=15 size=11 bits
>  > >> Fix inflate_threshold_root. Now=15 size=11 bits
>  > >> Fix inflate_threshold_root. Now=15 size=11 bits
>  > >> Fix inflate_threshold_root. Now=15 size=11 bits
>  > >> Fix inflate_threshold_root. Now=15 size=11 bits
> 
>  > On the other hand, even if there is no problem with memory, it seems
>  > because of hitting max_resize the threshold should be changed, e.g.
>  > by reverting the patch below.
> 
>  You seem to have some temporary memory problem. So the printout might be
>  a bit misleading in this case. We really like to keep the root node as big 
>  as we can to keep the tree as flat as possible for performance reasons.
>  (We're even more motivated now when we can disable the route cache)
> 
>  So I'll guess the next insert/delete inflates the root node to be within
>  the interval. So I'll assume this just a temporary failure?
> 
>  I would be nice to have *threshholds* settable by /proc or /sys. I would
>  use this in the other direction to trade memory for even faster lookups. 
>  
>  But maybe experts memory allocation has some good suggestions.

Robert, you and Eric pointed at memory problems, so I thought I missed
something. But after the second look I see "skipped node resize" should
show this, but it's always zero on these reports. So, isn't it possible
the current inflate_threshold_root is simply unreachable with some
conditions, at least within 10 loops?

Then these settable thresholds might be more useful here than memory
fixes, but here is some idea to try handle this automatically within
some limits. The patch below increases inflate_threshold_root (only)
up to ~50% of its initial value if needed, and should be able to go
back sometimes.

Pawel and Jorge, could you try this? (It applies to 2.6.29 too, with
some offsets.)

Thanks,
Jarek P.
---

 net/ipv4/fib_trie.c |   23 ++++++++++++++++-------
 1 files changed, 16 insertions(+), 7 deletions(-)

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Jarek Poplawski writes:

 > Robert, you and Eric pointed at memory problems, so I thought I missed
 > something. But after the second look I see "skipped node resize" should
 > show this, but it's always zero on these reports. So, isn't it possible
 > the current inflate_threshold_root is simply unreachable with some
 > conditions, at least within 10 loops?
 > 
 > Then these settable thresholds might be more useful here than memory
 > fixes, but here is some idea to try handle this automatically within
 > some limits. The patch below increases inflate_threshold_root (only)
 > up to ~50% of its initial value if needed, and should be able to go
 > back sometimes.

 Yes we keep the old tnode size and the convergence interval was some
 of the concerns. That why this checks was added. Still we want to
 inflate the root node to a very max. 

 So this approach with halving or doubling tnodes towards the root
 node was the suggest by Dyntree paper. I asked Stefan (one of the
 authors) if we could get safe and very offensive settings. But 
 from what I understood there was no easy way to calculate this. 
 So any bright ideas in this area are very welcome. But we should
 also monitor size of root and average tree depth so we don't
 take an to defensive approach just to solve this case.

 The memory patches and "manual RCU" are also interesting to address
 the case with PREEMTP's.

 Inserts and deletes are also very fast due to the flat tree so I think
 we can "slow down" this a bit if need to be safe with all PREEMPT's.

 Thanks for giving this area energy.

 Cheers
					--ro


 > Pawel and Jorge, could you try this? (It applies to 2.6.29 too, with
 > some offsets.)
 > 
 > Thanks,
 > Jarek P.
 > ---
 > 
 >  net/ipv4/fib_trie.c |   23 ++++++++++++++++-------
 >  1 files changed, 16 insertions(+), 7 deletions(-)
 > 
 > diff --git a/net/ipv4/fib_trie.c b/net/ipv4/fib_trie.c
 > index 012cf5a..1dc1bb4 100644
 > --- a/net/ipv4/fib_trie.c
 > +++ b/net/ipv4/fib_trie.c
 > @@ -318,6 +318,7 @@ static const int halve_threshold = 25;
 >  static const int inflate_threshold = 50;
 >  static const int halve_threshold_root = 8;
 >  static const int inflate_threshold_root = 15;
 > +static int inflate_threshold_root_fix;
 >  
 >  
 >  static void __alias_free_mem(struct rcu_head *head)
 > @@ -602,7 +603,8 @@ static struct node *resize(struct trie *t, struct tnode *tn)
 >  	/* Keep root node larger  */
 >  
 >  	if (!tn->parent)
 > -		inflate_threshold_use = inflate_threshold_root;
 > +		inflate_threshold_use = inflate_threshold_root +
 > +					inflate_threshold_root_fix;
 >  	else
 >  		inflate_threshold_use = inflate_threshold;
 >  
 > @@ -626,15 +628,22 @@ static struct node *resize(struct trie *t, struct tnode *tn)
 >  	}
 >  
 >  	if (max_resize < 0) {
 > -		if (!tn->parent)
 > -			pr_warning("Fix inflate_threshold_root."
 > -				   " Now=%d size=%d bits\n",
 > -				   inflate_threshold_root, tn->bits);
 > -		else
 > +		if (!tn->parent) {
 > +			if (inflate_threshold_root_fix * 2 <
 > +			    inflate_threshold_root)
 > +				inflate_threshold_root_fix++;
 > +			else
 > +				pr_warning("Fix inflate_threshold_root."
 > +					   " Now=%d size=%d bits fix=%d\n",
 > +					   inflate_threshold_root, tn->bits,
 > +					   inflate_threshold_root_fix);
 > +		} else {
 >  			pr_warning("Fix inflate_threshold."
 >  				   " Now=%d size=%d bits\n",
 >  				   inflate_threshold, tn->bits);
 > -	}
 > +		}
 > +	} else if (max_resize < 5 && !tn->parent && inflate_threshold_root_fix)
 > +		inflate_threshold_root_fix--;
 >  
 >  	check_tnode(tn);
 >  
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Sun, Jun 28, 2009 at 01:04:51PM +0200, Robert Olsson wrote:
...
>  Yes we keep the old tnode size and the convergence interval was some
>  of the concerns. That why this checks was added. Still we want to
>  inflate the root node to a very max. 
> 
>  So this approach with halving or doubling tnodes towards the root
>  node was the suggest by Dyntree paper. I asked Stefan (one of the
>  authors) if we could get safe and very offensive settings. But 
>  from what I understood there was no easy way to calculate this. 
>  So any bright ideas in this area are very welcome. But we should
>  also monitor size of root and average tree depth so we don't
>  take an to defensive approach just to solve this case.

Yes, but with this offensive approach it seems the current level of
warnings could be too alarming. Btw., because of a design flaw in my
current patch this _fix variable, which should be logically per trie/
table, can be reset by changes of other tables now, but I think it
all could be fine tuned more in the future. Of course if there are
people interested in testing/reporting this more.

Thanks,
Jarek P.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html