Patchwork Hight speed data sending from custom IP out of kernel

login
register
mail settings
Submitter Michal Simek
Date April 25, 2011, 11:18 a.m.
Message ID <4DB55876.2010008@monstr.eu>
Download mbox | patch
Permalink /patch/92743/
State RFC
Delegated to: David Miller
Headers show

Comments

Michal Simek - April 25, 2011, 11:18 a.m.
Hi,

Eric Dumazet wrote:
> Le jeudi 21 avril 2011 à 10:02 +0200, Michal Simek a écrit :
> 
>> Thanks for that. I am looking at pktgen. On UDP my system is able to send full 
>> bandwidth on 100Mbit/s ethernet and 220Mbit/s on 1G/s.
>> I will let you know when I have any useful resutls.
> 
> 220Mbits/s in pktgen or an application ?
> - how many packets per second ? (or packet size ?)
> 
> pktgen has the "clone_skb 100" thing that avoid skb_alloc()/skb_free()
> overhead, and permits to really test driver performance.
> 
> It also bypass qdisc management.
> 

I have reused the part of code from pktgen and I have found that I am missing 
some IDs that's why I have done one simple patch(below) in pktgen which is 
update IP ID field to find out if all packets are sent or not. As you suggest I 
am also missing some IDs here.
My question is if I can use any mechanism to ensure to sending all IDs?

The next my question about packet fragments. Is it possible to setup IP 
fragments from higher level? I do it on low level as pktgen and I have change 
page address to memory which I need to send but it in under UDP.

The point is to create packet with frags > 1 where the first fragment is IP/TCP 
header and the second fragments contains pointer to data which are prepared in 
the memory and will be copied directly by network driver. I am doing the same 
hacked code from pktgen. Is it possible to do it on higher level?

Thanks,
Michal


For 2.6.37.6
Eric Dumazet - April 25, 2011, 12:14 p.m.
Le lundi 25 avril 2011 à 13:18 +0200, Michal Simek a écrit :
> Hi,
> 
> Eric Dumazet wrote:
> > Le jeudi 21 avril 2011 à 10:02 +0200, Michal Simek a écrit :
> > 
> >> Thanks for that. I am looking at pktgen. On UDP my system is able to send full 
> >> bandwidth on 100Mbit/s ethernet and 220Mbit/s on 1G/s.
> >> I will let you know when I have any useful resutls.
> > 
> > 220Mbits/s in pktgen or an application ?
> > - how many packets per second ? (or packet size ?)
> > 
> > pktgen has the "clone_skb 100" thing that avoid skb_alloc()/skb_free()
> > overhead, and permits to really test driver performance.
> > 
> > It also bypass qdisc management.
> > 
> 
> I have reused the part of code from pktgen and I have found that I am missing 
> some IDs that's why I have done one simple patch(below) in pktgen which is 
> update IP ID field to find out if all packets are sent or not. As you suggest I 
> am also missing some IDs here.
> My question is if I can use any mechanism to ensure to sending all IDs?
> 
> The next my question about packet fragments. Is it possible to setup IP 
> fragments from higher level? I do it on low level as pktgen and I have change 
> page address to memory which I need to send but it in under UDP.
> 
> The point is to create packet with frags > 1 where the first fragment is IP/TCP 
> header and the second fragments contains pointer to data which are prepared in 
> the memory and will be copied directly by network driver. I am doing the same 
> hacked code from pktgen. Is it possible to do it on higher level?
> 

sendfile() is mostly doing this.

> Thanks,
> Michal
> 
> 
> For 2.6.37.6
> diff --git a/net/core/pktgen.c b/net/core/pktgen.c
> index 33bc382..3429eb3 100644
> --- a/net/core/pktgen.c
> +++ b/net/core/pktgen.c
> @@ -3500,6 +3500,13 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
>                  pkt_dev->last_pkt_size = pkt_dev->skb->len;
>                  pkt_dev->allocated_skbs++;
>                  pkt_dev->clone_count = 0;       /* reset counter */
> +       } else {
> +               struct iphdr *iph;
> +               iph = ip_hdr(pkt_dev->skb);
> +               iph->id = htons(pkt_dev->ip_id);
> +               pkt_dev->ip_id++;
> +               iph->check = 0;
> +               iph->check = ip_fast_csum((void *)iph, iph->ihl);
>          }
> 
>          if (pkt_dev->delay && pkt_dev->last_ok)
> 
> 
> 


Well, you cant do that in pktgen, since you're changing previous packet
content (it might still be in device TX queue, not yet sent, or being
sent right now)

Now, if all you want to do is send many packets from pktgen (with only
ID changing), you could add a fast path to not rebuild from scratch new
packets.



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Michal Simek - April 25, 2011, 12:18 p.m.
Eric Dumazet wrote:
> Le lundi 25 avril 2011 à 13:18 +0200, Michal Simek a écrit :
>> Hi,
>>
>> Eric Dumazet wrote:
>>> Le jeudi 21 avril 2011 à 10:02 +0200, Michal Simek a écrit :
>>>
>>>> Thanks for that. I am looking at pktgen. On UDP my system is able to send full 
>>>> bandwidth on 100Mbit/s ethernet and 220Mbit/s on 1G/s.
>>>> I will let you know when I have any useful resutls.
>>> 220Mbits/s in pktgen or an application ?
>>> - how many packets per second ? (or packet size ?)
>>>
>>> pktgen has the "clone_skb 100" thing that avoid skb_alloc()/skb_free()
>>> overhead, and permits to really test driver performance.
>>>
>>> It also bypass qdisc management.
>>>
>> I have reused the part of code from pktgen and I have found that I am missing 
>> some IDs that's why I have done one simple patch(below) in pktgen which is 
>> update IP ID field to find out if all packets are sent or not. As you suggest I 
>> am also missing some IDs here.
>> My question is if I can use any mechanism to ensure to sending all IDs?
>>
>> The next my question about packet fragments. Is it possible to setup IP 
>> fragments from higher level? I do it on low level as pktgen and I have change 
>> page address to memory which I need to send but it in under UDP.
>>
>> The point is to create packet with frags > 1 where the first fragment is IP/TCP 
>> header and the second fragments contains pointer to data which are prepared in 
>> the memory and will be copied directly by network driver. I am doing the same 
>> hacked code from pktgen. Is it possible to do it on higher level?
>>
> 
> sendfile() is mostly doing this.

will look, thanks.

> 
>> Thanks,
>> Michal
>>
>>
>> For 2.6.37.6
>> diff --git a/net/core/pktgen.c b/net/core/pktgen.c
>> index 33bc382..3429eb3 100644
>> --- a/net/core/pktgen.c
>> +++ b/net/core/pktgen.c
>> @@ -3500,6 +3500,13 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
>>                  pkt_dev->last_pkt_size = pkt_dev->skb->len;
>>                  pkt_dev->allocated_skbs++;
>>                  pkt_dev->clone_count = 0;       /* reset counter */
>> +       } else {
>> +               struct iphdr *iph;
>> +               iph = ip_hdr(pkt_dev->skb);
>> +               iph->id = htons(pkt_dev->ip_id);
>> +               pkt_dev->ip_id++;
>> +               iph->check = 0;
>> +               iph->check = ip_fast_csum((void *)iph, iph->ihl);
>>          }
>>
>>          if (pkt_dev->delay && pkt_dev->last_ok)
>>
>>
>>
> 
> 
> Well, you cant do that in pktgen, since you're changing previous packet
> content (it might still be in device TX queue, not yet sent, or being
> sent right now)

It is likely happening.

> 
> Now, if all you want to do is send many packets from pktgen (with only
> ID changing), you could add a fast path to not rebuild from scratch new
> packets.

What do you mean?

Thanks,
Michal
Eric Dumazet - April 25, 2011, 12:27 p.m.
Le lundi 25 avril 2011 à 14:18 +0200, Michal Simek a écrit :

> > 
> > Now, if all you want to do is send many packets from pktgen (with only
> > ID changing), you could add a fast path to not rebuild from scratch new
> > packets.
> 
> What do you mean?


If you know your device has X slots in its TX ring buffer, you would
have to maintain at least X+1 skbs in pktgen to make sure you reuse an
skb while its previous logical content was sent on wire.

Then you are free to only change iph->id and iph->check very fast.




--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Dumazet - April 25, 2011, 12:30 p.m.
Le lundi 25 avril 2011 à 14:27 +0200, Eric Dumazet a écrit :
> Le lundi 25 avril 2011 à 14:18 +0200, Michal Simek a écrit :
> 
> > > 
> > > Now, if all you want to do is send many packets from pktgen (with only
> > > ID changing), you could add a fast path to not rebuild from scratch new
> > > packets.
> > 
> > What do you mean?
> 
> 
> If you know your device has X slots in its TX ring buffer, you would
> have to maintain at least X+1 skbs in pktgen to make sure you reuse an
> skb while its previous logical content was sent on wire.
> 
> Then you are free to only change iph->id and iph->check very fast.
> 

Checking skb->users would also be a good way to know if TX completion
released skb reference. If your module owns the last reference, it can
do a recycle.




--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Michal Simek - April 25, 2011, 12:48 p.m.
Eric Dumazet wrote:
> Le lundi 25 avril 2011 à 14:27 +0200, Eric Dumazet a écrit :
>> Le lundi 25 avril 2011 à 14:18 +0200, Michal Simek a écrit :
>>
>>>> Now, if all you want to do is send many packets from pktgen (with only
>>>> ID changing), you could add a fast path to not rebuild from scratch new
>>>> packets.
>>> What do you mean?
>>
>> If you know your device has X slots in its TX ring buffer, you would
>> have to maintain at least X+1 skbs in pktgen to make sure you reuse an
>> skb while its previous logical content was sent on wire.
>>
>> Then you are free to only change iph->id and iph->check very fast.
>>

got it. There is an option to setup number of BDs in the driver where I need to 
use half skb because of nr_frags=2 where two BDs are used.

> 
> Checking skb->users would also be a good way to know if TX completion
> released skb reference. If your module owns the last reference, it can
> do a recycle.

Ok. I see that dev_kfree_skb(consume_skb).

Thanks will try,
Michal

Patch

diff --git a/net/core/pktgen.c b/net/core/pktgen.c
index 33bc382..3429eb3 100644
--- a/net/core/pktgen.c
+++ b/net/core/pktgen.c
@@ -3500,6 +3500,13 @@  static void pktgen_xmit(struct pktgen_dev *pkt_dev)
                 pkt_dev->last_pkt_size = pkt_dev->skb->len;
                 pkt_dev->allocated_skbs++;
                 pkt_dev->clone_count = 0;       /* reset counter */
+       } else {
+               struct iphdr *iph;
+               iph = ip_hdr(pkt_dev->skb);
+               iph->id = htons(pkt_dev->ip_id);
+               pkt_dev->ip_id++;
+               iph->check = 0;
+               iph->check = ip_fast_csum((void *)iph, iph->ihl);
         }

         if (pkt_dev->delay && pkt_dev->last_ok)