[net] net: reduce RECURSION_LIMIT to 8

When RECURSION_LIMIT was first introduced, Eric proposed a limit of 3.
This limit was later raised to 10 by DaveM. Nowadays it is observed that
configuraion errors in openvswitch cause the STACK_END_MAGIC to be
overwritten shortly after 9 recursion.

This patch tries to be conservative and reduces the limit to 8 without
further measurements. It seems ovs uses the stack more than other parts
of the networking stack - I couldn't bring the system down with a non-ovs
tunneling setup.

Cc: David S. Miller <davem@davemloft.net>
Cc: Eric Dumazet <edumazet@google.com>
Signed-off-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
---
I don't do crazy run-time estimation of the stack size for one recursion
and try automatically to come up with a limit per arch or kconfig
settings, as I assume that all systems should behave the same regarding
the recursion maximum. All configurations should run on all kinds of
systems. I consider 8 recursions to be plenty enough for the time being.

 net/core/dev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

From: Hannes Frederic Sowa <hannes@stressinduktion.org>
Date: Thu,  7 Jan 2016 18:40:53 +0100

> This patch tries to be conservative and reduces the limit to 8
> without further measurements. It seems ovs uses the stack more than
> other parts of the networking stack - I couldn't bring the system
> down with a non-ovs tunneling setup.

Can we figure out why OVS sucks so much wrt. stack usage instead?

I'd rather not paper over something like this, especially when it's
OVS which I'm completely not in the mood to specially cater for in
any way, shape, or form.

On Thu, Jan 7, 2016 at 9:40 AM, Hannes Frederic Sowa
<hannes@stressinduktion.org> wrote:
> When RECURSION_LIMIT was first introduced, Eric proposed a limit of 3.
> This limit was later raised to 10 by DaveM. Nowadays it is observed that
> configuraion errors in openvswitch cause the STACK_END_MAGIC to be
> overwritten shortly after 9 recursion.
>
Major user of stack space in OVS is sw_flow_key in
ovs_vport_receive(). With recent features like IPv6 tunnel support we
have increased the size of the flow-key which could have caused the
stack overflow sooner.
One way to avoid using stack in subsequent recursive call is to use
per-cpu storage for the sw_flow_key object. This is already done for
OVS recursive actions, so we can expand on that facility.


> This patch tries to be conservative and reduces the limit to 8 without
> further measurements. It seems ovs uses the stack more than other parts
> of the networking stack - I couldn't bring the system down with a non-ovs
> tunneling setup.
>
> Cc: David S. Miller <davem@davemloft.net>
> Cc: Eric Dumazet <edumazet@google.com>
> Signed-off-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
> ---
> I don't do crazy run-time estimation of the stack size for one recursion
> and try automatically to come up with a limit per arch or kconfig
> settings, as I assume that all systems should behave the same regarding
> the recursion maximum. All configurations should run on all kinds of
> systems. I consider 8 recursions to be plenty enough for the time being.
>
>  net/core/dev.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/net/core/dev.c b/net/core/dev.c
> index ae00b894e67555..d93da7df84325d 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -2941,7 +2941,7 @@ static void skb_update_prio(struct sk_buff *skb)
>  DEFINE_PER_CPU(int, xmit_recursion);
>  EXPORT_SYMBOL(xmit_recursion);
>
> -#define RECURSION_LIMIT 10
> +#define RECURSION_LIMIT 8
>
>  /**
>   *     dev_loopback_xmit - loop back @skb
> --
> 2.5.0
>

On 11.01.2016 07:38, pravin shelar wrote:
> On Thu, Jan 7, 2016 at 9:40 AM, Hannes Frederic Sowa
> <hannes@stressinduktion.org> wrote:
>> When RECURSION_LIMIT was first introduced, Eric proposed a limit of 3.
>> This limit was later raised to 10 by DaveM. Nowadays it is observed that
>> configuraion errors in openvswitch cause the STACK_END_MAGIC to be
>> overwritten shortly after 9 recursion.
>>
> Major user of stack space in OVS is sw_flow_key in
> ovs_vport_receive(). With recent features like IPv6 tunnel support we
> have increased the size of the flow-key which could have caused the
> stack overflow sooner.
> One way to avoid using stack in subsequent recursive call is to use
> per-cpu storage for the sw_flow_key object. This is already done for
> OVS recursive actions, so we can expand on that facility.

Hmmm. This already came up. I think the difficulty is that 
ovs_vport_receive can be called from actions again with skb_cloned skb 
before the original's skb callstack is actually finished. Data in the 
percpu area would be overwritten while still being used. It would need 
some more logic IMHO.

What are recursive actions in ovs? I couldn't find any use of pcpu data 
in there? Thanks! :)

We could as an intermediate step add a recursion counter to openvswitch 
and limit call chains to depth 5, what do you think?

Bye,
Hannes

On Mon, Jan 11, 2016 at 4:24 AM, Hannes Frederic Sowa
<hannes@stressinduktion.org> wrote:
> On 11.01.2016 07:38, pravin shelar wrote:
>>
>> On Thu, Jan 7, 2016 at 9:40 AM, Hannes Frederic Sowa
>> <hannes@stressinduktion.org> wrote:
>>>
>>> When RECURSION_LIMIT was first introduced, Eric proposed a limit of 3.
>>> This limit was later raised to 10 by DaveM. Nowadays it is observed that
>>> configuraion errors in openvswitch cause the STACK_END_MAGIC to be
>>> overwritten shortly after 9 recursion.
>>>
>> Major user of stack space in OVS is sw_flow_key in
>> ovs_vport_receive(). With recent features like IPv6 tunnel support we
>> have increased the size of the flow-key which could have caused the
>> stack overflow sooner.
>> One way to avoid using stack in subsequent recursive call is to use
>> per-cpu storage for the sw_flow_key object. This is already done for
>> OVS recursive actions, so we can expand on that facility.
>
>
> Hmmm. This already came up. I think the difficulty is that ovs_vport_receive
> can be called from actions again with skb_cloned skb before the original's
> skb callstack is actually finished. Data in the percpu area would be
> overwritten while still being used. It would need some more logic IMHO.
>
You can have stack of flow-keys and allocate a flow-key for each recursive call.

> What are recursive actions in ovs? I couldn't find any use of pcpu data in
> there? Thanks! :)
>
There are couple of recursive actions in OVS, e.g.
OVS_ACTION_ATTR_RECIRC. But it is implemented by using per-cpu
flow-key stack to avoid recursive function call.