Message ID | cover.1522136948.git.shahafs@mellanox.com |
---|---|
Headers | show |
Series | OVS-DPDK flow offload with rte_flow | expand |
Tuesday, March 27, 2018 10:55 AM, Shahaf Shuler: Hi, Any comments on this version? > > Hi, > > Here is a joint work from Mellanox and Napatech, to enable the flow hw > offload with the DPDK generic flow interface (rte_flow). > > The basic idea is to associate the flow with a mark id (a unit32_t number). > Later, we then get the flow directly from the mark id, which could bypass > some heavy CPU operations, including but not limiting to mini flow extract, > emc lookup, dpcls lookup, etc. > > The association is done with CMAP in patch 1. The CPU workload bypassing is > done in patch 2. The flow offload is done in patch 3, which mainly does two > things: > > - translate the ovs match to DPDK rte flow patterns > - bind those patterns with a RSS + MARK action. > > Patch 5 makes the offload work happen in another thread, for leaving the > datapath as light as possible. > > A PHY-PHY forwarding with 1000 mega flows (udp,tp_src=1000-1999) and 1 > million streams (tp_src=1000-1999, tp_dst=2000-2999) show more than 260% > performance boost. > > Note that it's disabled by default, which can be enabled by: > > $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true > > v8: - enhanced documentation with more details on supported protocols > - fixed VLOG to start with capital letter > - fixed compilation issues > - fixed coding style > - addressed the rest of Ian's comments > > v7: - fixed wrong hash for mark_to_flow that has been refactored in v6 > - set the rss_conf for rss action to NULL, to workaround a mlx5 change > in DPDK v17.11. Note that it will obey the rss settings OVS-DPDK has > set in the beginning. Thus, nothing should be effected. > > v6: - fixed a sparse warning > - added documentation > - used hash_int to compute mark to flow hash > - added more comments > - added lock for pot lookup > - rebased on top of the latest code > > v5: - fixed an issue that it took too long if we do flow add/remove > repeatedly. > - removed an unused mutex lock > - turned most of the log level to DBG > - rebased on top of the latest code > > v4: - use RSS action instead of QUEUE action with MARK > - make it work with multiple queue (see patch 1) > - rebased on top of latest code > > v3: - The mark and id association is done with array instead of CMAP. > - Added a thread to do hw offload operations > - Removed macros completely > - dropped the patch to set FDIR_CONF, which is a workround some > Intel NICs. > - Added a debug patch to show all flow patterns we have created. > - Misc fixes > > v2: - workaround the queue action issue > - fixed the tcp_flags being skipped issue, which also fixed the > build warnings > - fixed l2 patterns for Intel nic > - Converted some macros to functions > - did not hardcode the max number of flow/action > - rebased on top of the latest code > > Thanks. > > --- > > Finn Christensen (1): > netdev-dpdk: implement flow offload with rte flow > > Yuanhan Liu (5): > dpif-netdev: associate flow with a mark id > dpif-netdev: retrieve flow directly from the flow mark > netdev-dpdk: add debug for rte flow patterns > dpif-netdev: do hw flow offload in a thread > Documentation: document ovs-dpdk flow offload > > Documentation/howto/dpdk.rst | 22 ++ > NEWS | 3 +- > lib/dp-packet.h | 13 + > lib/dpif-netdev.c | 497 ++++++++++++++++++++++++- > lib/flow.c | 155 ++++++-- > lib/flow.h | 1 + > lib/netdev-dpdk.c | 740 > +++++++++++++++++++++++++++++++++++++- > lib/netdev.h | 6 + > 8 files changed, 1397 insertions(+), 40 deletions(-) > > -- > 2.7.4 > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fma > il.openvswitch.org%2Fmailman%2Flistinfo%2Fovs- > dev&data=02%7C01%7Cshahafs%40mellanox.com%7C06b32d22c2e34de31af > 508d593b82bd2%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C1%7C6365 > 77341596013800&sdata=FrFTVCkg6HzG%2FMaFYw9zsytUuQPXMzyr8z893Qp > zYbc%3D&reserved=0
> Tuesday, March 27, 2018 10:55 AM, Shahaf Shuler: > > Hi, > > Any comments on this version? I should have some time to look at this today. I would also echo the request for anyone else in the community interested to review. Ian > > > > > Hi, > > > > Here is a joint work from Mellanox and Napatech, to enable the flow hw > > offload with the DPDK generic flow interface (rte_flow). > > > > The basic idea is to associate the flow with a mark id (a unit32_t > number). > > Later, we then get the flow directly from the mark id, which could > > bypass some heavy CPU operations, including but not limiting to mini > > flow extract, emc lookup, dpcls lookup, etc. > > > > The association is done with CMAP in patch 1. The CPU workload > > bypassing is done in patch 2. The flow offload is done in patch 3, > > which mainly does two > > things: > > > > - translate the ovs match to DPDK rte flow patterns > > - bind those patterns with a RSS + MARK action. > > > > Patch 5 makes the offload work happen in another thread, for leaving > > the datapath as light as possible. > > > > A PHY-PHY forwarding with 1000 mega flows (udp,tp_src=1000-1999) and 1 > > million streams (tp_src=1000-1999, tp_dst=2000-2999) show more than > > 260% performance boost. > > > > Note that it's disabled by default, which can be enabled by: > > > > $ ovs-vsctl set Open_vSwitch . other_config:hw-offload=true > > > > v8: - enhanced documentation with more details on supported protocols > > - fixed VLOG to start with capital letter > > - fixed compilation issues > > - fixed coding style > > - addressed the rest of Ian's comments > > > > v7: - fixed wrong hash for mark_to_flow that has been refactored in v6 > > - set the rss_conf for rss action to NULL, to workaround a mlx5 > change > > in DPDK v17.11. Note that it will obey the rss settings OVS-DPDK > has > > set in the beginning. Thus, nothing should be effected. > > > > v6: - fixed a sparse warning > > - added documentation > > - used hash_int to compute mark to flow hash > > - added more comments > > - added lock for pot lookup > > - rebased on top of the latest code > > > > v5: - fixed an issue that it took too long if we do flow add/remove > > repeatedly. > > - removed an unused mutex lock > > - turned most of the log level to DBG > > - rebased on top of the latest code > > > > v4: - use RSS action instead of QUEUE action with MARK > > - make it work with multiple queue (see patch 1) > > - rebased on top of latest code > > > > v3: - The mark and id association is done with array instead of CMAP. > > - Added a thread to do hw offload operations > > - Removed macros completely > > - dropped the patch to set FDIR_CONF, which is a workround some > > Intel NICs. > > - Added a debug patch to show all flow patterns we have created. > > - Misc fixes > > > > v2: - workaround the queue action issue > > - fixed the tcp_flags being skipped issue, which also fixed the > > build warnings > > - fixed l2 patterns for Intel nic > > - Converted some macros to functions > > - did not hardcode the max number of flow/action > > - rebased on top of the latest code > > > > Thanks. > > > > --- > > > > Finn Christensen (1): > > netdev-dpdk: implement flow offload with rte flow > > > > Yuanhan Liu (5): > > dpif-netdev: associate flow with a mark id > > dpif-netdev: retrieve flow directly from the flow mark > > netdev-dpdk: add debug for rte flow patterns > > dpif-netdev: do hw flow offload in a thread > > Documentation: document ovs-dpdk flow offload > > > > Documentation/howto/dpdk.rst | 22 ++ > > NEWS | 3 +- > > lib/dp-packet.h | 13 + > > lib/dpif-netdev.c | 497 ++++++++++++++++++++++++- > > lib/flow.c | 155 ++++++-- > > lib/flow.h | 1 + > > lib/netdev-dpdk.c | 740 > > +++++++++++++++++++++++++++++++++++++- > > lib/netdev.h | 6 + > > 8 files changed, 1397 insertions(+), 40 deletions(-) > > > > -- > > 2.7.4 > > > > _______________________________________________ > > dev mailing list > > dev@openvswitch.org > > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fma > > il.openvswitch.org%2Fmailman%2Flistinfo%2Fovs- > > dev&data=02%7C01%7Cshahafs%40mellanox.com%7C06b32d22c2e34de31af > > 508d593b82bd2%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C1%7C6365 > > 77341596013800&sdata=FrFTVCkg6HzG%2FMaFYw9zsytUuQPXMzyr8z893Qp > > zYbc%3D&reserved=0