[net,v2] driver: vrf: Fix one possible use-after-free issue

From: Gao Feng <gfree.wind@vip.163.com>

The current codes only deal with the case that the skb is dropped, it
may meet one use-after-free issue when NF_HOOK returns 0 that means
the skb is stolen by one netfilter rule or hook.

When one netfilter rule or hook stoles the skb and return NF_STOLEN,
it means the skb is taken by the rule, and other modules should not
touch this skb ever. Maybe the skb is queued or freed directly by the
rule.

Now uses the nf_hook instead of NF_HOOK to get the result of netfilter,
and check the return value of nf_hook. Only when its value equals 1, it
means the skb could go ahead. Or reset the skb as NULL.

BTW, because vrf_rcv_finish is empty function, so needn't invoke it
even though nf_hook returns 1. But we need to modify vrf_rcv_finish
to deal with the NF_STOLEN case.

There are two cases when skb is stolen.
1. The skb is stolen and freed directly.
   There is nothing we need to do, and vrf_rcv_finish isn't invoked.
2. The skb is queued and reinjected again.
   The vrf_rcv_finish would be invoked as okfn, so need to free the
   skb in it.

Signed-off-by: Gao Feng <gfree.wind@vip.163.com>
---
 v2: Free the skb in vrf_rcv_finish, per Florian
 v1: initial version

 drivers/net/vrf.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

On 5/9/17 3:27 AM, gfree.wind@vip.163.com wrote:
> diff --git a/drivers/net/vrf.c b/drivers/net/vrf.c
> index ceda586..db88249 100644
> --- a/drivers/net/vrf.c
> +++ b/drivers/net/vrf.c
> @@ -989,6 +989,7 @@ static u32 vrf_fib_table(const struct net_device *dev)
>  
>  static int vrf_rcv_finish(struct net *net, struct sock *sk, struct sk_buff *skb)
>  {
> +	kfree_skb(skb);
>  	return 0;
>  }
>  
> @@ -998,7 +999,7 @@ static struct sk_buff *vrf_rcv_nfhook(u8 pf, unsigned int hook,
>  {
>  	struct net *net = dev_net(dev);
>  
> -	if (NF_HOOK(pf, hook, net, NULL, skb, dev, NULL, vrf_rcv_finish) < 0)
> +	if (nf_hook(pf, hook, net, NULL, skb, dev, NULL, vrf_rcv_finish) != 1)
>  		skb = NULL;    /* kfree_skb(skb) handled by nf code */
>  
>  	return skb;
> 

I'm clearly misunderstanding something ...

With the current code:
- nf_hook returns 1, NF_HOOK invokes vrf_rcv_finish as the okfn, it
returns 0, skb passes on.

- nf_hook returns 0, vrf_rcv_finish has been called by the nf_hook tree,
vrf_rcv_finish returns 0, skb passes on

- nf_hook returns < 0,  vrf_rcv_finish is not called, skb is freed by
netfilter code, vrf_rcv_nfhook returns NULL

What am I missing?

With the above, if nf_hook returns 1, vrf_rcv_finish is not called.

David Ahern <dsahern@gmail.com> wrote:
> On 5/9/17 3:27 AM, gfree.wind@vip.163.com wrote:
> > diff --git a/drivers/net/vrf.c b/drivers/net/vrf.c
> > index ceda586..db88249 100644
> > --- a/drivers/net/vrf.c
> > +++ b/drivers/net/vrf.c
> > @@ -989,6 +989,7 @@ static u32 vrf_fib_table(const struct net_device *dev)
> >  
> >  static int vrf_rcv_finish(struct net *net, struct sock *sk, struct sk_buff *skb)
> >  {
> > +	kfree_skb(skb);
> >  	return 0;
> >  }
> >  
> > @@ -998,7 +999,7 @@ static struct sk_buff *vrf_rcv_nfhook(u8 pf, unsigned int hook,
> >  {
> >  	struct net *net = dev_net(dev);
> >  
> > -	if (NF_HOOK(pf, hook, net, NULL, skb, dev, NULL, vrf_rcv_finish) < 0)
> > +	if (nf_hook(pf, hook, net, NULL, skb, dev, NULL, vrf_rcv_finish) != 1)
> >  		skb = NULL;    /* kfree_skb(skb) handled by nf code */
> >  
> >  	return skb;
> > 
> 
> I'm clearly misunderstanding something ...
> 
> With the current code:
> - nf_hook returns 1, NF_HOOK invokes vrf_rcv_finish as the okfn, it
> returns 0, skb passes on.
> 
> - nf_hook returns 0, vrf_rcv_finish has been called by the nf_hook tree,
> vrf_rcv_finish returns 0, skb passes on

Yes, thats a bug. The skb could be have been queued to userspace, or
stolen by a hook.

It is illegal to use the skb after NF_HOOK() no matter the return value.
The okfn has to do further processing.

If nf_hook() is used instead, only a return value of 1 means the skb is
still valid.

In < 0 case it was free'd, in 0 case its in unknown state (usually queued
or free'd).

As for the patch, it avoids skb leak on userspace reinject but nfqueue
still won't work as no reinject is possible (vrf_rcv_finish is a sink that
doesn't do anyting).

Hope this clarifies things.

From: gfree.wind@vip.163.com
Date: Tue,  9 May 2017 18:27:33 +0800

> @@ -989,6 +989,7 @@ static u32 vrf_fib_table(const struct net_device *dev)
>  
>  static int vrf_rcv_finish(struct net *net, struct sock *sk, struct sk_buff *skb)
>  {
> +	kfree_skb(skb);
>  	return 0;
>  }
>  
> @@ -998,7 +999,7 @@ static struct sk_buff *vrf_rcv_nfhook(u8 pf, unsigned int hook,
>  {
>  	struct net *net = dev_net(dev);
>  
> -	if (NF_HOOK(pf, hook, net, NULL, skb, dev, NULL, vrf_rcv_finish) < 0)
> +	if (nf_hook(pf, hook, net, NULL, skb, dev, NULL, vrf_rcv_finish) != 1)
>  		skb = NULL;    /* kfree_skb(skb) handled by nf code */
>  
>  	return skb;

Indeed, this fixes the immediate problem with NF_STOLEN.

Making NF_STOLEN fully functional is another story, we'd need to stack
this all together properly:

static int __ip_rcv_finish(struct net *net, struct sock *sk, struct sk_buff *skb)
{
 ...
}

static int ip_rcv_finish(struct net *net, struct sock *sk, struct sk_buff *skb)
{
	return l3mdev_ip_rcv(skb, __ip_rcv_finish);
}
...
static inline
struct sk_buff *l3mdev_ip_rcv(struct sk_buff *skb,
			      int (*okfn)(struct net *, struct sock *, struct sk_buff *))
{
	return l3mdev_l3_rcv(skb, okfn, AF_INET);
}

etc. but that's going to really add a kink to the receive path,
microbenchmark wise.

At 2017-05-10 02:37:36, "David Miller" <davem@davemloft.net> wrote:
>From: gfree.wind@vip.163.com
>Date: Tue,  9 May 2017 18:27:33 +0800
>
>> @@ -989,6 +989,7 @@ static u32 vrf_fib_table(const struct net_device *dev)
>>  
>>  static int vrf_rcv_finish(struct net *net, struct sock *sk, struct sk_buff *skb)
>>  {
>> +	kfree_skb(skb);
>>  	return 0;
>>  }
>>  
>> @@ -998,7 +999,7 @@ static struct sk_buff *vrf_rcv_nfhook(u8 pf, unsigned int hook,
>>  {
>>  	struct net *net = dev_net(dev);
>>  
>> -	if (NF_HOOK(pf, hook, net, NULL, skb, dev, NULL, vrf_rcv_finish) < 0)
>> +	if (nf_hook(pf, hook, net, NULL, skb, dev, NULL, vrf_rcv_finish) != 1)
>>  		skb = NULL;    /* kfree_skb(skb) handled by nf code */
>>  
>>  	return skb;
>
>Indeed, this fixes the immediate problem with NF_STOLEN.
>
>Making NF_STOLEN fully functional is another story, we'd need to stack
>this all together properly:

Yes, this fix just make the vrf wouldn't cause panic which is caused by use-after-free skb.
It doesn't work as real NF_QUEUE when reinject the skb. 

>
>static int __ip_rcv_finish(struct net *net, struct sock *sk, struct sk_buff *skb)
>{
> ...
>}
>
>static int ip_rcv_finish(struct net *net, struct sock *sk, struct sk_buff *skb)
>{
>	return l3mdev_ip_rcv(skb, __ip_rcv_finish);
>}
>...
>static inline
>struct sk_buff *l3mdev_ip_rcv(struct sk_buff *skb,
>			      int (*okfn)(struct net *, struct sock *, struct sk_buff *))
>{
>	return l3mdev_l3_rcv(skb, okfn, AF_INET);
>}
>
>etc. but that's going to really add a kink to the receive path,
>microbenchmark wise.

It is a solution to make NF_STOLEN fully function, but I haven't environment to test the benchmark.

Best Regards
Feng

From: gfree.wind@vip.163.com
Date: Tue,  9 May 2017 18:27:33 +0800

> From: Gao Feng <gfree.wind@vip.163.com>
> 
> The current codes only deal with the case that the skb is dropped, it
> may meet one use-after-free issue when NF_HOOK returns 0 that means
> the skb is stolen by one netfilter rule or hook.
> 
> When one netfilter rule or hook stoles the skb and return NF_STOLEN,
> it means the skb is taken by the rule, and other modules should not
> touch this skb ever. Maybe the skb is queued or freed directly by the
> rule.
> 
> Now uses the nf_hook instead of NF_HOOK to get the result of netfilter,
> and check the return value of nf_hook. Only when its value equals 1, it
> means the skb could go ahead. Or reset the skb as NULL.
> 
> BTW, because vrf_rcv_finish is empty function, so needn't invoke it
> even though nf_hook returns 1. But we need to modify vrf_rcv_finish
> to deal with the NF_STOLEN case.
> 
> There are two cases when skb is stolen.
> 1. The skb is stolen and freed directly.
>    There is nothing we need to do, and vrf_rcv_finish isn't invoked.
> 2. The skb is queued and reinjected again.
>    The vrf_rcv_finish would be invoked as okfn, so need to free the
>    skb in it.
> 
> Signed-off-by: Gao Feng <gfree.wind@vip.163.com>

Applied and queued up for -stable, thanks.