| Message ID | 20180828124435.30578-1-bjorn.topel@gmail.com |
|---|---|
| Headers | show |
| Series | AF_XDP zero-copy support for i40e | expand |
Den tis 28 aug. 2018 kl 14:47 skrev Björn Töpel <bjorn.topel@gmail.com>: > > From: Björn Töpel <bjorn.topel@intel.com> > > This patch set introduces zero-copy AF_XDP support for Intel's i40e > driver. In the first preparatory patch we also add support for > XDP_REDIRECT for zero-copy allocated frames so that XDP programs can > redirect them. This was a ToDo from the first AF_XDP zero-copy patch > set from early June. Special thanks to Alex Duyck and Jesper Dangaard > Brouer for reviewing earlier versions of this patch set. > > The i40e zero-copy code is located in its own file i40e_xsk.[ch]. Note > that in the interest of time, to get an AF_XDP zero-copy implementation > out there for people to try, some code paths have been copied from the > XDP path to the zero-copy path. It is out goal to merge the two paths > in later patch sets. > > In contrast to the implementation from beginning of June, this patch > set does not require any extra HW queues for AF_XDP zero-copy > TX. Instead, the XDP TX HW queue is used for both XDP_REDIRECT and > AF_XDP zero-copy TX. > > Jeff, given that most of changes are in i40e, it is up to you how you > would like to route these patches. The set is tagged bpf-next, but > if taking it via the Intel driver tree is easier, let us know. > > We have run some benchmarks on a dual socket system with two Broadwell > E5 2660 @ 2.0 GHz with hyperthreading turned off. Each socket has 14 > cores which gives a total of 28, but only two cores are used in these > experiments. One for TR/RX and one for the user space application. The > memory is DDR4 @ 2133 MT/s (1067 MHz) and the size of each DIMM is > 8192MB and with 8 of those DIMMs in the system we have 64 GB of total > memory. The compiler used is gcc (Ubuntu 7.3.0-16ubuntu3) 7.3.0. The > NIC is Intel I40E 40Gbit/s using the i40e driver. > > Below are the results in Mpps of the I40E NIC benchmark runs for 64 > and 1500 byte packets, generated by a commercial packet generator HW > outputing packets at full 40 Gbit/s line rate. The results are with > retpoline and all other spectre and meltdown fixes, so these results > are not comparable to the ones from the zero-copy patch set in June. > > AF_XDP performance 64 byte packets. > Benchmark XDP_SKB XDP_DRV XDP_DRV with zerocopy > rxdrop 2.6 8.2 15.0 > txpush 2.2 - 21.9 > l2fwd 1.7 2.3 11.3 > > AF_XDP performance 1500 byte packets: > Benchmark XDP_SKB XDP_DRV XDP_DRV with zerocopy > rxdrop 2.0 3.3 3.3 > l2fwd 1.3 1.7 3.1 > > XDP performance on our system as a base line: > > 64 byte packets: > XDP stats CPU pps issue-pps > XDP-RX CPU 16 18.4M 0 > > 1500 byte packets: > XDP stats CPU pps issue-pps > XDP-RX CPU 16 3.3M 0 > > The structure of the patch set is as follows: > > Patch 1: Add support for XDP_REDIRECT of zero-copy allocated frames > Patches 2-4: Preparatory patches to common xsk and net code > Patches 5-7: Preparatory patches to i40e driver code for RX > Patch 8: i40e zero-copy support for RX > Patch 9: Preparatory patch to i40e driver code for TX > Patch 10: i40e zero-copy support for TX > Patch 11: Add flags to sample application to force zero-copy/copy mode > > We based this patch set on bpf-next commit 050cdc6c9501 ("Merge > git://git.kernel.org/pub/scm/linux/kernel/git/davem/net") > > > Magnus & Björn > > Björn Töpel (8): > xdp: implement convert_to_xdp_frame for MEM_TYPE_ZERO_COPY > xdp: export xdp_rxq_info_unreg_mem_model > xsk: expose xdp_umem_get_{data,dma} to drivers > i40e: added queue pair disable/enable functions > i40e: refactor Rx path for re-use > i40e: move common Rx functions to i40e_txrx_common.h > i40e: add AF_XDP zero-copy Rx support > samples/bpf: add -c/--copy -z/--zero-copy flags to xdpsock > > Magnus Karlsson (3): > net: add napi_if_scheduled_mark_missed > i40e: move common Tx functions to i40e_txrx_common.h > i40e: add AF_XDP zero-copy Tx support > > drivers/net/ethernet/intel/i40e/Makefile | 3 +- > drivers/net/ethernet/intel/i40e/i40e.h | 19 + > drivers/net/ethernet/intel/i40e/i40e_main.c | 307 ++++++- > drivers/net/ethernet/intel/i40e/i40e_txrx.c | 182 ++-- > drivers/net/ethernet/intel/i40e/i40e_txrx.h | 20 +- > .../ethernet/intel/i40e/i40e_txrx_common.h | 90 ++ > drivers/net/ethernet/intel/i40e/i40e_xsk.c | 834 ++++++++++++++++++ > drivers/net/ethernet/intel/i40e/i40e_xsk.h | 25 + > include/linux/netdevice.h | 26 + > include/net/xdp.h | 6 +- > include/net/xdp_sock.h | 43 + > net/core/xdp.c | 54 +- > net/xdp/xdp_umem.h | 10 - > samples/bpf/xdpsock_user.c | 12 +- > 14 files changed, 1523 insertions(+), 108 deletions(-) > create mode 100644 drivers/net/ethernet/intel/i40e/i40e_txrx_common.h > create mode 100644 drivers/net/ethernet/intel/i40e/i40e_xsk.c > create mode 100644 drivers/net/ethernet/intel/i40e/i40e_xsk.h > > -- > 2.17.1 > I was too quick on the trigger. Adding intel-wired-lan and JeffK. @Jeff Apologies. :-( Björn
On 08/28/2018 02:44 PM, Björn Töpel wrote: > From: Björn Töpel <bjorn.topel@intel.com> > > This patch set introduces zero-copy AF_XDP support for Intel's i40e > driver. In the first preparatory patch we also add support for > XDP_REDIRECT for zero-copy allocated frames so that XDP programs can > redirect them. This was a ToDo from the first AF_XDP zero-copy patch > set from early June. Special thanks to Alex Duyck and Jesper Dangaard > Brouer for reviewing earlier versions of this patch set. > > The i40e zero-copy code is located in its own file i40e_xsk.[ch]. Note > that in the interest of time, to get an AF_XDP zero-copy implementation > out there for people to try, some code paths have been copied from the > XDP path to the zero-copy path. It is out goal to merge the two paths > in later patch sets. > > In contrast to the implementation from beginning of June, this patch > set does not require any extra HW queues for AF_XDP zero-copy > TX. Instead, the XDP TX HW queue is used for both XDP_REDIRECT and > AF_XDP zero-copy TX. > > Jeff, given that most of changes are in i40e, it is up to you how you > would like to route these patches. The set is tagged bpf-next, but > if taking it via the Intel driver tree is easier, let us know. > > We have run some benchmarks on a dual socket system with two Broadwell > E5 2660 @ 2.0 GHz with hyperthreading turned off. Each socket has 14 > cores which gives a total of 28, but only two cores are used in these > experiments. One for TR/RX and one for the user space application. The > memory is DDR4 @ 2133 MT/s (1067 MHz) and the size of each DIMM is > 8192MB and with 8 of those DIMMs in the system we have 64 GB of total > memory. The compiler used is gcc (Ubuntu 7.3.0-16ubuntu3) 7.3.0. The > NIC is Intel I40E 40Gbit/s using the i40e driver. > > Below are the results in Mpps of the I40E NIC benchmark runs for 64 > and 1500 byte packets, generated by a commercial packet generator HW > outputing packets at full 40 Gbit/s line rate. The results are with > retpoline and all other spectre and meltdown fixes, so these results > are not comparable to the ones from the zero-copy patch set in June. > > AF_XDP performance 64 byte packets. > Benchmark XDP_SKB XDP_DRV XDP_DRV with zerocopy > rxdrop 2.6 8.2 15.0 > txpush 2.2 - 21.9 > l2fwd 1.7 2.3 11.3 > > AF_XDP performance 1500 byte packets: > Benchmark XDP_SKB XDP_DRV XDP_DRV with zerocopy > rxdrop 2.0 3.3 3.3 > l2fwd 1.3 1.7 3.1 > > XDP performance on our system as a base line: > > 64 byte packets: > XDP stats CPU pps issue-pps > XDP-RX CPU 16 18.4M 0 > > 1500 byte packets: > XDP stats CPU pps issue-pps > XDP-RX CPU 16 3.3M 0 > > The structure of the patch set is as follows: > > Patch 1: Add support for XDP_REDIRECT of zero-copy allocated frames > Patches 2-4: Preparatory patches to common xsk and net code > Patches 5-7: Preparatory patches to i40e driver code for RX > Patch 8: i40e zero-copy support for RX > Patch 9: Preparatory patch to i40e driver code for TX > Patch 10: i40e zero-copy support for TX > Patch 11: Add flags to sample application to force zero-copy/copy mode > > We based this patch set on bpf-next commit 050cdc6c9501 ("Merge > git://git.kernel.org/pub/scm/linux/kernel/git/davem/net") > > > Magnus & Björn > > Björn Töpel (8): > xdp: implement convert_to_xdp_frame for MEM_TYPE_ZERO_COPY > xdp: export xdp_rxq_info_unreg_mem_model > xsk: expose xdp_umem_get_{data,dma} to drivers > i40e: added queue pair disable/enable functions > i40e: refactor Rx path for re-use > i40e: move common Rx functions to i40e_txrx_common.h > i40e: add AF_XDP zero-copy Rx support > samples/bpf: add -c/--copy -z/--zero-copy flags to xdpsock > > Magnus Karlsson (3): > net: add napi_if_scheduled_mark_missed > i40e: move common Tx functions to i40e_txrx_common.h > i40e: add AF_XDP zero-copy Tx support Thanks for working on this, LGTM! Are you also planning to get ixgbe out after that? For the series: Acked-by: Daniel Borkmann <daniel@iogearbox.net> Thanks, Daniel
On Tue, Aug 28, 2018 at 02:44:24PM +0200, Björn Töpel wrote: > From: Björn Töpel <bjorn.topel@intel.com> > > This patch set introduces zero-copy AF_XDP support for Intel's i40e > driver. In the first preparatory patch we also add support for > XDP_REDIRECT for zero-copy allocated frames so that XDP programs can > redirect them. This was a ToDo from the first AF_XDP zero-copy patch > set from early June. Special thanks to Alex Duyck and Jesper Dangaard > Brouer for reviewing earlier versions of this patch set. > > The i40e zero-copy code is located in its own file i40e_xsk.[ch]. Note > that in the interest of time, to get an AF_XDP zero-copy implementation > out there for people to try, some code paths have been copied from the > XDP path to the zero-copy path. It is out goal to merge the two paths > in later patch sets. > > In contrast to the implementation from beginning of June, this patch > set does not require any extra HW queues for AF_XDP zero-copy > TX. Instead, the XDP TX HW queue is used for both XDP_REDIRECT and > AF_XDP zero-copy TX. > > Jeff, given that most of changes are in i40e, it is up to you how you > would like to route these patches. The set is tagged bpf-next, but > if taking it via the Intel driver tree is easier, let us know. > > We have run some benchmarks on a dual socket system with two Broadwell > E5 2660 @ 2.0 GHz with hyperthreading turned off. Each socket has 14 > cores which gives a total of 28, but only two cores are used in these > experiments. One for TR/RX and one for the user space application. The > memory is DDR4 @ 2133 MT/s (1067 MHz) and the size of each DIMM is > 8192MB and with 8 of those DIMMs in the system we have 64 GB of total > memory. The compiler used is gcc (Ubuntu 7.3.0-16ubuntu3) 7.3.0. The > NIC is Intel I40E 40Gbit/s using the i40e driver. > > Below are the results in Mpps of the I40E NIC benchmark runs for 64 > and 1500 byte packets, generated by a commercial packet generator HW > outputing packets at full 40 Gbit/s line rate. The results are with > retpoline and all other spectre and meltdown fixes, so these results > are not comparable to the ones from the zero-copy patch set in June. > > AF_XDP performance 64 byte packets. > Benchmark XDP_SKB XDP_DRV XDP_DRV with zerocopy > rxdrop 2.6 8.2 15.0 > txpush 2.2 - 21.9 > l2fwd 1.7 2.3 11.3 > > AF_XDP performance 1500 byte packets: > Benchmark XDP_SKB XDP_DRV XDP_DRV with zerocopy > rxdrop 2.0 3.3 3.3 > l2fwd 1.3 1.7 3.1 > > XDP performance on our system as a base line: > > 64 byte packets: > XDP stats CPU pps issue-pps > XDP-RX CPU 16 18.4M 0 > > 1500 byte packets: > XDP stats CPU pps issue-pps > XDP-RX CPU 16 3.3M 0 > > The structure of the patch set is as follows: > > Patch 1: Add support for XDP_REDIRECT of zero-copy allocated frames > Patches 2-4: Preparatory patches to common xsk and net code > Patches 5-7: Preparatory patches to i40e driver code for RX > Patch 8: i40e zero-copy support for RX > Patch 9: Preparatory patch to i40e driver code for TX > Patch 10: i40e zero-copy support for TX > Patch 11: Add flags to sample application to force zero-copy/copy mode > > We based this patch set on bpf-next commit 050cdc6c9501 ("Merge > git://git.kernel.org/pub/scm/linux/kernel/git/davem/net") > > > Magnus & Björn > > Björn Töpel (8): > xdp: implement convert_to_xdp_frame for MEM_TYPE_ZERO_COPY > xdp: export xdp_rxq_info_unreg_mem_model > xsk: expose xdp_umem_get_{data,dma} to drivers > i40e: added queue pair disable/enable functions > i40e: refactor Rx path for re-use > i40e: move common Rx functions to i40e_txrx_common.h > i40e: add AF_XDP zero-copy Rx support > samples/bpf: add -c/--copy -z/--zero-copy flags to xdpsock > > Magnus Karlsson (3): > net: add napi_if_scheduled_mark_missed > i40e: move common Tx functions to i40e_txrx_common.h > i40e: add AF_XDP zero-copy Tx support Applied to bpf-next. Thanks a lot to the authors, code reviewers and testers. I hope all the issues brought up during code review can be quickly addressed in the follow up patches and zerocopy feature in i40e and other drivers will be mainline ready for the next merge window.
> Thanks for working on this, LGTM! Are you also planning to get ixgbe > out after that? > I currently don't have i40e nic to test, so I'm also looking forward to the ixgbe patch! Thank you William
Den ons 29 aug. 2018 kl 18:12 skrev Daniel Borkmann <daniel@iogearbox.net>: > > On 08/28/2018 02:44 PM, Björn Töpel wrote: > > From: Björn Töpel <bjorn.topel@intel.com> > > > > This patch set introduces zero-copy AF_XDP support for Intel's i40e > > driver. In the first preparatory patch we also add support for > > XDP_REDIRECT for zero-copy allocated frames so that XDP programs can > > redirect them. This was a ToDo from the first AF_XDP zero-copy patch > > set from early June. Special thanks to Alex Duyck and Jesper Dangaard > > Brouer for reviewing earlier versions of this patch set. > > > > The i40e zero-copy code is located in its own file i40e_xsk.[ch]. Note > > that in the interest of time, to get an AF_XDP zero-copy implementation > > out there for people to try, some code paths have been copied from the > > XDP path to the zero-copy path. It is out goal to merge the two paths > > in later patch sets. > > > > In contrast to the implementation from beginning of June, this patch > > set does not require any extra HW queues for AF_XDP zero-copy > > TX. Instead, the XDP TX HW queue is used for both XDP_REDIRECT and > > AF_XDP zero-copy TX. > > > > Jeff, given that most of changes are in i40e, it is up to you how you > > would like to route these patches. The set is tagged bpf-next, but > > if taking it via the Intel driver tree is easier, let us know. > > > > We have run some benchmarks on a dual socket system with two Broadwell > > E5 2660 @ 2.0 GHz with hyperthreading turned off. Each socket has 14 > > cores which gives a total of 28, but only two cores are used in these > > experiments. One for TR/RX and one for the user space application. The > > memory is DDR4 @ 2133 MT/s (1067 MHz) and the size of each DIMM is > > 8192MB and with 8 of those DIMMs in the system we have 64 GB of total > > memory. The compiler used is gcc (Ubuntu 7.3.0-16ubuntu3) 7.3.0. The > > NIC is Intel I40E 40Gbit/s using the i40e driver. > > > > Below are the results in Mpps of the I40E NIC benchmark runs for 64 > > and 1500 byte packets, generated by a commercial packet generator HW > > outputing packets at full 40 Gbit/s line rate. The results are with > > retpoline and all other spectre and meltdown fixes, so these results > > are not comparable to the ones from the zero-copy patch set in June. > > > > AF_XDP performance 64 byte packets. > > Benchmark XDP_SKB XDP_DRV XDP_DRV with zerocopy > > rxdrop 2.6 8.2 15.0 > > txpush 2.2 - 21.9 > > l2fwd 1.7 2.3 11.3 > > > > AF_XDP performance 1500 byte packets: > > Benchmark XDP_SKB XDP_DRV XDP_DRV with zerocopy > > rxdrop 2.0 3.3 3.3 > > l2fwd 1.3 1.7 3.1 > > > > XDP performance on our system as a base line: > > > > 64 byte packets: > > XDP stats CPU pps issue-pps > > XDP-RX CPU 16 18.4M 0 > > > > 1500 byte packets: > > XDP stats CPU pps issue-pps > > XDP-RX CPU 16 3.3M 0 > > > > The structure of the patch set is as follows: > > > > Patch 1: Add support for XDP_REDIRECT of zero-copy allocated frames > > Patches 2-4: Preparatory patches to common xsk and net code > > Patches 5-7: Preparatory patches to i40e driver code for RX > > Patch 8: i40e zero-copy support for RX > > Patch 9: Preparatory patch to i40e driver code for TX > > Patch 10: i40e zero-copy support for TX > > Patch 11: Add flags to sample application to force zero-copy/copy mode > > > > We based this patch set on bpf-next commit 050cdc6c9501 ("Merge > > git://git.kernel.org/pub/scm/linux/kernel/git/davem/net") > > > > > > Magnus & Björn > > > > Björn Töpel (8): > > xdp: implement convert_to_xdp_frame for MEM_TYPE_ZERO_COPY > > xdp: export xdp_rxq_info_unreg_mem_model > > xsk: expose xdp_umem_get_{data,dma} to drivers > > i40e: added queue pair disable/enable functions > > i40e: refactor Rx path for re-use > > i40e: move common Rx functions to i40e_txrx_common.h > > i40e: add AF_XDP zero-copy Rx support > > samples/bpf: add -c/--copy -z/--zero-copy flags to xdpsock > > > > Magnus Karlsson (3): > > net: add napi_if_scheduled_mark_missed > > i40e: move common Tx functions to i40e_txrx_common.h > > i40e: add AF_XDP zero-copy Tx support > > Thanks for working on this, LGTM! Are you also planning to get ixgbe > out after that? > Yes, the plan is to get ixgbe out as the next driver, but we'll focus on the existing i40e issues first. Thanks, Björn > For the series: > > Acked-by: Daniel Borkmann <daniel@iogearbox.net> > > Thanks, > Daniel
From: Björn Töpel <bjorn.topel@intel.com> This patch set introduces zero-copy AF_XDP support for Intel's i40e driver. In the first preparatory patch we also add support for XDP_REDIRECT for zero-copy allocated frames so that XDP programs can redirect them. This was a ToDo from the first AF_XDP zero-copy patch set from early June. Special thanks to Alex Duyck and Jesper Dangaard Brouer for reviewing earlier versions of this patch set. The i40e zero-copy code is located in its own file i40e_xsk.[ch]. Note that in the interest of time, to get an AF_XDP zero-copy implementation out there for people to try, some code paths have been copied from the XDP path to the zero-copy path. It is out goal to merge the two paths in later patch sets. In contrast to the implementation from beginning of June, this patch set does not require any extra HW queues for AF_XDP zero-copy TX. Instead, the XDP TX HW queue is used for both XDP_REDIRECT and AF_XDP zero-copy TX. Jeff, given that most of changes are in i40e, it is up to you how you would like to route these patches. The set is tagged bpf-next, but if taking it via the Intel driver tree is easier, let us know. We have run some benchmarks on a dual socket system with two Broadwell E5 2660 @ 2.0 GHz with hyperthreading turned off. Each socket has 14 cores which gives a total of 28, but only two cores are used in these experiments. One for TR/RX and one for the user space application. The memory is DDR4 @ 2133 MT/s (1067 MHz) and the size of each DIMM is 8192MB and with 8 of those DIMMs in the system we have 64 GB of total memory. The compiler used is gcc (Ubuntu 7.3.0-16ubuntu3) 7.3.0. The NIC is Intel I40E 40Gbit/s using the i40e driver. Below are the results in Mpps of the I40E NIC benchmark runs for 64 and 1500 byte packets, generated by a commercial packet generator HW outputing packets at full 40 Gbit/s line rate. The results are with retpoline and all other spectre and meltdown fixes, so these results are not comparable to the ones from the zero-copy patch set in June. AF_XDP performance 64 byte packets. Benchmark XDP_SKB XDP_DRV XDP_DRV with zerocopy rxdrop 2.6 8.2 15.0 txpush 2.2 - 21.9 l2fwd 1.7 2.3 11.3 AF_XDP performance 1500 byte packets: Benchmark XDP_SKB XDP_DRV XDP_DRV with zerocopy rxdrop 2.0 3.3 3.3 l2fwd 1.3 1.7 3.1 XDP performance on our system as a base line: 64 byte packets: XDP stats CPU pps issue-pps XDP-RX CPU 16 18.4M 0 1500 byte packets: XDP stats CPU pps issue-pps XDP-RX CPU 16 3.3M 0 The structure of the patch set is as follows: Patch 1: Add support for XDP_REDIRECT of zero-copy allocated frames Patches 2-4: Preparatory patches to common xsk and net code Patches 5-7: Preparatory patches to i40e driver code for RX Patch 8: i40e zero-copy support for RX Patch 9: Preparatory patch to i40e driver code for TX Patch 10: i40e zero-copy support for TX Patch 11: Add flags to sample application to force zero-copy/copy mode We based this patch set on bpf-next commit 050cdc6c9501 ("Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net") Magnus & Björn Björn Töpel (8): xdp: implement convert_to_xdp_frame for MEM_TYPE_ZERO_COPY xdp: export xdp_rxq_info_unreg_mem_model xsk: expose xdp_umem_get_{data,dma} to drivers i40e: added queue pair disable/enable functions i40e: refactor Rx path for re-use i40e: move common Rx functions to i40e_txrx_common.h i40e: add AF_XDP zero-copy Rx support samples/bpf: add -c/--copy -z/--zero-copy flags to xdpsock Magnus Karlsson (3): net: add napi_if_scheduled_mark_missed i40e: move common Tx functions to i40e_txrx_common.h i40e: add AF_XDP zero-copy Tx support drivers/net/ethernet/intel/i40e/Makefile | 3 +- drivers/net/ethernet/intel/i40e/i40e.h | 19 + drivers/net/ethernet/intel/i40e/i40e_main.c | 307 ++++++- drivers/net/ethernet/intel/i40e/i40e_txrx.c | 182 ++-- drivers/net/ethernet/intel/i40e/i40e_txrx.h | 20 +- .../ethernet/intel/i40e/i40e_txrx_common.h | 90 ++ drivers/net/ethernet/intel/i40e/i40e_xsk.c | 834 ++++++++++++++++++ drivers/net/ethernet/intel/i40e/i40e_xsk.h | 25 + include/linux/netdevice.h | 26 + include/net/xdp.h | 6 +- include/net/xdp_sock.h | 43 + net/core/xdp.c | 54 +- net/xdp/xdp_umem.h | 10 - samples/bpf/xdpsock_user.c | 12 +- 14 files changed, 1523 insertions(+), 108 deletions(-) create mode 100644 drivers/net/ethernet/intel/i40e/i40e_txrx_common.h create mode 100644 drivers/net/ethernet/intel/i40e/i40e_xsk.c create mode 100644 drivers/net/ethernet/intel/i40e/i40e_xsk.h