diff mbox

[Bug,#16626] Machine hangs with EIP at skb_copy_and_csum_dev

Message ID 20100904203429.GA4891@del.dom.local
State Accepted, archived
Delegated to: David Miller
Headers show

Commit Message

Jarek Poplawski Sept. 4, 2010, 8:34 p.m. UTC
Eric Dumazet wrote, On 09/01/2010 12:50 PM:

> [PATCH] gro: fix different skb headrooms
> 
> packets entering GRO might have different headrooms, even for a given
> flow (because of implementation details in drivers, like copybreak).
> We cant force drivers to deliver packets with a fixed headroom.
> 
> 1) fix skb_segment()
> 
> skb_segment() makes the false assumption headrooms of fragments are same
> than the head. When CHECKSUM_PARTIAL is used, this can give csum_start
> errors, and crash later in skb_copy_and_csum_dev()
> 
> 2) allocate a minimal skb for head of frag_list
> 
> skb_gro_receive() uses netdev_alloc_skb(headroom + skb_gro_offset(p)) to
> allocate a fresh skb. This adds NET_SKB_PAD to a padding already
> provided by netdevice, depending on various things, like copybreak.
> 
> Use alloc_skb() to allocate an exact padding, to reduce cache line
> needs:
> NET_SKB_PAD + NET_IP_ALIGN
> 
> bugzilla : https://bugzilla.kernel.org/show_bug.cgi?id=16626
> 
> Many thanks to Plamen Petrov, testing many debugging patches !
> With help of Jarek Poplawski.
> 
> Reported-by: Plamen Petrov <pvp-lsts@fs.uni-ruse.bg>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
> CC: Jarek Poplawski <jarkao2@gmail.com>
> ---
> patch against linux-2.6 current tree
> 
>  net/core/skbuff.c |    8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/net/core/skbuff.c b/net/core/skbuff.c
...
> @@ -2702,8 +2706,8 @@ int skb_gro_receive(struct sk_buff **head, struct sk_buff *skb)
>  	} else if (skb_gro_len(p) != pinfo->gso_size)
>  		return -E2BIG;
>  
> -	headroom = skb_headroom(p);
> -	nskb = netdev_alloc_skb(p->dev, headroom + skb_gro_offset(p));
> +	headroom = NET_SKB_PAD + NET_IP_ALIGN;
> +	nskb = alloc_skb(headroom + skb_gro_offset(p), GFP_ATOMIC);
>  	if (unlikely(!nskb))
>  		return -ENOMEM;

Hi again,

Just had a second look, and unless I miss something...

Plamen, could you test this patch, too? (Without removing the previous
one.)

Thanks,
Jarek P.

------------------->

[PATCH] gro: Re-fix different skb headrooms

The patch: "gro: fix different skb headrooms" in its part:
"2) allocate a minimal skb for head of frag_list" is buggy. The copied
skb has p->data set at the ip header at the moment, and skb_gro_offset
is the length of ip + tcp headers. So, after the change the length of
mac header is skipped. Later skb_set_mac_header() sets it into the
NET_SKB_PAD area (if it's long enough) and ip header is misaligned at
NET_SKB_PAD + NET_IP_ALIGN offset. There is no reason to assume the
original skb was wrongly allocated, so let's copy it as it was.

bugzilla : https://bugzilla.kernel.org/show_bug.cgi?id=16626
fixes commit: 3d3be4333fdf6faa080947b331a6a19bce1a4f57

Reported-by: Plamen Petrov <pvp-lsts@fs.uni-ruse.bg>
Signed-off-by: Jarek Poplawski <jarkao2@gmail.com>
CC: Eric Dumazet <eric.dumazet@gmail.com>
---

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Eric Dumazet Sept. 5, 2010, 7:49 a.m. UTC | #1
Le samedi 04 septembre 2010 à 22:34 +0200, Jarek Poplawski a écrit :

> Hi again,
> 
> Just had a second look, and unless I miss something...
> 
> Plamen, could you test this patch, too? (Without removing the previous
> one.)
> 
> Thanks,
> Jarek P.
> 
> ------------------->
> 
> [PATCH] gro: Re-fix different skb headrooms
> 
> The patch: "gro: fix different skb headrooms" in its part:
> "2) allocate a minimal skb for head of frag_list" is buggy. The copied
> skb has p->data set at the ip header at the moment, and skb_gro_offset
> is the length of ip + tcp headers. So, after the change the length of
> mac header is skipped. Later skb_set_mac_header() sets it into the
> NET_SKB_PAD area (if it's long enough) and ip header is misaligned at
> NET_SKB_PAD + NET_IP_ALIGN offset. There is no reason to assume the
> original skb was wrongly allocated, so let's copy it as it was.
> 
> bugzilla : https://bugzilla.kernel.org/show_bug.cgi?id=16626
> fixes commit: 3d3be4333fdf6faa080947b331a6a19bce1a4f57
> 
> Reported-by: Plamen Petrov <pvp-lsts@fs.uni-ruse.bg>
> Signed-off-by: Jarek Poplawski <jarkao2@gmail.com>
> CC: Eric Dumazet <eric.dumazet@gmail.com>
> ---
> 
> diff --git a/net/core/skbuff.c b/net/core/skbuff.c
> index 26396ff..c83b421 100644
> --- a/net/core/skbuff.c
> +++ b/net/core/skbuff.c
> @@ -2706,7 +2706,7 @@ int skb_gro_receive(struct sk_buff **head, struct sk_buff *skb)
>  	} else if (skb_gro_len(p) != pinfo->gso_size)
>  		return -E2BIG;
>  
> -	headroom = NET_SKB_PAD + NET_IP_ALIGN;
> +	headroom = skb_headroom(p);
>  	nskb = alloc_skb(headroom + skb_gro_offset(p), GFP_ATOMIC);
>  	if (unlikely(!nskb))
>  		return -ENOMEM;

You are right, thanks for reviewing this patch again :)

By the way, NET_IP_ALIGN is now 0 on x86, so technically speaking, your
patch un-aligns IP header on x86, but thats OK, since other arches might
want it being aligned, while x86 doesnt care.

Acked-by: Eric Dumazet <eric.dumazet@gmail.com>



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Plamen Petrov Sept. 8, 2010, 4:57 a.m. UTC | #2
На 05.9.2010 г. 10:49, Eric Dumazet написа:
> Le samedi 04 septembre 2010 à 22:34 +0200, Jarek Poplawski a écrit :
>
>> Hi again,
>>
>> Just had a second look, and unless I miss something...
>>
>> Plamen, could you test this patch, too? (Without removing the previous
>> one.)
>>
>> Thanks,
>> Jarek P.
>>
>> ------------------->
>>
>> [PATCH] gro: Re-fix different skb headrooms
>>
>> The patch: "gro: fix different skb headrooms" in its part:
>> "2) allocate a minimal skb for head of frag_list" is buggy. The copied
>> skb has p->data set at the ip header at the moment, and skb_gro_offset
>> is the length of ip + tcp headers. So, after the change the length of
>> mac header is skipped. Later skb_set_mac_header() sets it into the
>> NET_SKB_PAD area (if it's long enough) and ip header is misaligned at
>> NET_SKB_PAD + NET_IP_ALIGN offset. There is no reason to assume the
>> original skb was wrongly allocated, so let's copy it as it was.
>>
>> bugzilla : https://bugzilla.kernel.org/show_bug.cgi?id=16626
>> fixes commit: 3d3be4333fdf6faa080947b331a6a19bce1a4f57
>>
>> Reported-by: Plamen Petrov<pvp-lsts@fs.uni-ruse.bg>
>> Signed-off-by: Jarek Poplawski<jarkao2@gmail.com>
>> CC: Eric Dumazet<eric.dumazet@gmail.com>
>> ---
>>
>> diff --git a/net/core/skbuff.c b/net/core/skbuff.c
>> index 26396ff..c83b421 100644
>> --- a/net/core/skbuff.c
>> +++ b/net/core/skbuff.c
>> @@ -2706,7 +2706,7 @@ int skb_gro_receive(struct sk_buff **head, struct sk_buff *skb)
>>   	} else if (skb_gro_len(p) != pinfo->gso_size)
>>   		return -E2BIG;
>>
>> -	headroom = NET_SKB_PAD + NET_IP_ALIGN;
>> +	headroom = skb_headroom(p);
>>   	nskb = alloc_skb(headroom + skb_gro_offset(p), GFP_ATOMIC);
>>   	if (unlikely(!nskb))
>>   		return -ENOMEM;
>
> You are right, thanks for reviewing this patch again :)
>
> By the way, NET_IP_ALIGN is now 0 on x86, so technically speaking, your
> patch un-aligns IP header on x86, but thats OK, since other arches might
> want it being aligned, while x86 doesnt care.
>
> Acked-by: Eric Dumazet<eric.dumazet@gmail.com>
>
>
>

Well, now that I'm back at work, I'm glad to report
that I tested both variants of the patch, and as Eric
points out - it works both ways.

So, which ever fits you guys better.

Thanks a lot!
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Jarek Poplawski Sept. 8, 2010, 6:20 a.m. UTC | #3
On Wed, Sep 08, 2010 at 07:57:31AM +0300, Plamen Petrov wrote:
> ???? 05.9.2010 ??. 10:49, Eric Dumazet ????????????:
>> Le samedi 04 septembre 2010 ?? 22:34 +0200, Jarek Poplawski a écrit :
>>
>>> Hi again,
>>>
>>> Just had a second look, and unless I miss something...
>>>
>>> Plamen, could you test this patch, too? (Without removing the previous
>>> one.)
>>>
>>> Thanks,
>>> Jarek P.
>>>
>>> ------------------->
>>>
>>> [PATCH] gro: Re-fix different skb headrooms
>>>
>>> The patch: "gro: fix different skb headrooms" in its part:
>>> "2) allocate a minimal skb for head of frag_list" is buggy. The copied
>>> skb has p->data set at the ip header at the moment, and skb_gro_offset
>>> is the length of ip + tcp headers. So, after the change the length of
>>> mac header is skipped. Later skb_set_mac_header() sets it into the
>>> NET_SKB_PAD area (if it's long enough) and ip header is misaligned at
>>> NET_SKB_PAD + NET_IP_ALIGN offset. There is no reason to assume the
>>> original skb was wrongly allocated, so let's copy it as it was.
>>>
>>> bugzilla : https://bugzilla.kernel.org/show_bug.cgi?id=16626
>>> fixes commit: 3d3be4333fdf6faa080947b331a6a19bce1a4f57
>>>
>>> Reported-by: Plamen Petrov<pvp-lsts@fs.uni-ruse.bg>
>>> Signed-off-by: Jarek Poplawski<jarkao2@gmail.com>
>>> CC: Eric Dumazet<eric.dumazet@gmail.com>
>>> ---
>>>
>>> diff --git a/net/core/skbuff.c b/net/core/skbuff.c
>>> index 26396ff..c83b421 100644
>>> --- a/net/core/skbuff.c
>>> +++ b/net/core/skbuff.c
>>> @@ -2706,7 +2706,7 @@ int skb_gro_receive(struct sk_buff **head, struct sk_buff *skb)
>>>   	} else if (skb_gro_len(p) != pinfo->gso_size)
>>>   		return -E2BIG;
>>>
>>> -	headroom = NET_SKB_PAD + NET_IP_ALIGN;
>>> +	headroom = skb_headroom(p);
>>>   	nskb = alloc_skb(headroom + skb_gro_offset(p), GFP_ATOMIC);
>>>   	if (unlikely(!nskb))
>>>   		return -ENOMEM;
>>
>> You are right, thanks for reviewing this patch again :)
>>
>> By the way, NET_IP_ALIGN is now 0 on x86, so technically speaking, your
>> patch un-aligns IP header on x86, but thats OK, since other arches might
>> want it being aligned, while x86 doesnt care.
>>
>> Acked-by: Eric Dumazet<eric.dumazet@gmail.com>
>>
>>
>>
>
> Well, now that I'm back at work, I'm glad to report
> that I tested both variants of the patch, and as Eric
> points out - it works both ways.
>
> So, which ever fits you guys better.

We need both of them. I hope David could add this too:

Tested-by: Plamen Petrov <pvp-lsts@fs.uni-ruse.bg>

Thanks a lot everybody!
Jarek P.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
David Miller Sept. 8, 2010, 5:32 p.m. UTC | #4
From: Jarek Poplawski <jarkao2@gmail.com>
Date: Wed, 8 Sep 2010 06:20:04 +0000

> We need both of them. I hope David could add this too:
> 
> Tested-by: Plamen Petrov <pvp-lsts@fs.uni-ruse.bg>

Done, and applied, thanks :-)
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Rafael J. Wysocki Sept. 8, 2010, 8:21 p.m. UTC | #5
On Wednesday, September 08, 2010, David Miller wrote:
> From: Jarek Poplawski <jarkao2@gmail.com>
> Date: Wed, 8 Sep 2010 06:20:04 +0000
> 
> > We need both of them. I hope David could add this too:
> > 
> > Tested-by: Plamen Petrov <pvp-lsts@fs.uni-ruse.bg>
> 
> Done, and applied, thanks :-)

Please kindly let me know when Linus gets them, so that I can close the bug.

Rafael
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Plamen Petrov Sept. 12, 2010, 6:55 a.m. UTC | #6
На 08.9.2010 г. 23:21, Rafael J. Wysocki написа:
> On Wednesday, September 08, 2010, David Miller wrote:
>> From: Jarek Poplawski<jarkao2@gmail.com>
>> Date: Wed, 8 Sep 2010 06:20:04 +0000
>>
>>> We need both of them. I hope David could add this too:
>>>
>>> Tested-by: Plamen Petrov<pvp-lsts@fs.uni-ruse.bg>
>>
>> Done, and applied, thanks :-)
>
> Please kindly let me know when Linus gets them, so that I can close the bug.
>
> Rafael

Now that both commits that fix my problems are in Linus' tree, the
bug can be closed, but these fixes should go in 2.6.35.y, too.
So, CCing -stable.

Fix 1:
> commit	3d3be4333fdf6faa080947b331a6a19bce1a4f57
>
> gro: fix different skb headrooms
>
> Packets entering GRO might have different headrooms, even for a given
> flow (because of implementation details in drivers, like copybreak).
> We cant force drivers to deliver packets with a fixed headroom.
>
> 1) fix skb_segment()
>
> skb_segment() makes the false assumption headrooms of fragments are same
> than the head. When CHECKSUM_PARTIAL is used, this can give csum_start
> errors, and crash later in skb_copy_and_csum_dev()
>
> 2) allocate a minimal skb for head of frag_list
>
> skb_gro_receive() uses netdev_alloc_skb(headroom + skb_gro_offset(p)) to
> allocate a fresh skb. This adds NET_SKB_PAD to a padding already
> provided by netdevice, depending on various things, like copybreak.
>
> Use alloc_skb() to allocate an exact padding, to reduce cache line
> needs:
> NET_SKB_PAD + NET_IP_ALIGN
>
> bugzilla : https://bugzilla.kernel.org/show_bug.cgi?id=16626
>
> Many thanks to Plamen Petrov, testing many debugging patches !
> With help of Jarek Poplawski.
>
> Reported-by: Plamen Petrov <pvp-lsts@fs.uni-ruse.bg>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
> CC: Jarek Poplawski <jarkao2@gmail.com>
> Signed-off-by: David S. Miller <davem@davemloft.net>

Fix 2:
> commit	64289c8e6851bca0e589e064c9a5c9fbd6ae5dd4
>
> gro: Re-fix different skb headrooms
>
> The patch: "gro: fix different skb headrooms" in its part:
> "2) allocate a minimal skb for head of frag_list" is buggy. The copied
> skb has p->data set at the ip header at the moment, and skb_gro_offset
> is the length of ip + tcp headers. So, after the change the length of
> mac header is skipped. Later skb_set_mac_header() sets it into the
> NET_SKB_PAD area (if it's long enough) and ip header is misaligned at
> NET_SKB_PAD + NET_IP_ALIGN offset. There is no reason to assume the
> original skb was wrongly allocated, so let's copy it as it was.
>
> bugzilla : https://bugzilla.kernel.org/show_bug.cgi?id=16626
> fixes commit: 3d3be4333fdf6faa080947b331a6a19bce1a4f57
>
> Reported-by: Plamen Petrov <pvp-lsts@fs.uni-ruse.bg>
> Signed-off-by: Jarek Poplawski <jarkao2@gmail.com>
> CC: Eric Dumazet <eric.dumazet@gmail.com>
> Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
> Tested-by: Plamen Petrov <pvp-lsts@fs.uni-ruse.bg>
> Signed-off-by: David S. Miller <davem@davemloft.net>

Thanks,
Plamen
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
David Miller Sept. 12, 2010, 3:55 p.m. UTC | #7
From: Plamen Petrov <pvp-lsts@fs.uni-ruse.bg>
Date: Sun, 12 Sep 2010 09:55:19 +0300

> Now that both commits that fix my problems are in Linus' tree, the
> bug can be closed, but these fixes should go in 2.6.35.y, too.
> So, CCing -stable.

You don't need to do this, I will submit whatever is appropriate for
networking to -stable as needed.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Rafael J. Wysocki Sept. 12, 2010, 5:28 p.m. UTC | #8
On Sunday, September 12, 2010, Plamen Petrov wrote:
> На 08.9.2010 г. 23:21, Rafael J. Wysocki написа:
> > On Wednesday, September 08, 2010, David Miller wrote:
> >> From: Jarek Poplawski<jarkao2@gmail.com>
> >> Date: Wed, 8 Sep 2010 06:20:04 +0000
> >>
> >>> We need both of them. I hope David could add this too:
> >>>
> >>> Tested-by: Plamen Petrov<pvp-lsts@fs.uni-ruse.bg>
> >>
> >> Done, and applied, thanks :-)
> >
> > Please kindly let me know when Linus gets them, so that I can close the bug.
> >
> > Rafael
> 
> Now that both commits that fix my problems are in Linus' tree, the
> bug can be closed, but these fixes should go in 2.6.35.y, too.
> So, CCing -stable.

Thanks, bug closed.

Rafael
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Plamen Petrov Sept. 13, 2010, 6:49 a.m. UTC | #9
На 12.9.2010 г. 18:55, David Miller написа:
> From: Plamen Petrov<pvp-lsts@fs.uni-ruse.bg>
> Date: Sun, 12 Sep 2010 09:55:19 +0300
>
>> Now that both commits that fix my problems are in Linus' tree, the
>> bug can be closed, but these fixes should go in 2.6.35.y, too.
>> So, CCing -stable.
>
> You don't need to do this, I will submit whatever is appropriate for
> networking to -stable as needed.

Sorry! I didn't know it will be taken care of.

Thanks!
Plamen
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 26396ff..c83b421 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -2706,7 +2706,7 @@  int skb_gro_receive(struct sk_buff **head, struct sk_buff *skb)
 	} else if (skb_gro_len(p) != pinfo->gso_size)
 		return -E2BIG;
 
-	headroom = NET_SKB_PAD + NET_IP_ALIGN;
+	headroom = skb_headroom(p);
 	nskb = alloc_skb(headroom + skb_gro_offset(p), GFP_ATOMIC);
 	if (unlikely(!nskb))
 		return -ENOMEM;