| Message ID | 1585172682-14891-1-git-send-email-u9012063@gmail.com |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [ovs-dev,PATCHv4] netdev-afxdp: Add interrupt mode netdev class. | expand |
On 3/25/20 10:44 PM, William Tu wrote: > The patch adds a new netdev class 'afxdp-nonpmd' to enable afxdp > interrupt mode. This is similar to 'type=afxdp', except that the > is_pmd field is set to false. As a result, the packet processing > is handled by main thread, not pmd thread. This avoids burning > the CPU to always 100% when there is no traffic. > > Tested-at: https://travis-ci.org/github/williamtu/ovs-travis/builds/666980138 > Signed-off-by: William Tu <u9012063@gmail.com> > --- > v4: > - Previously crash and fix it with qid = qid % netdev_n_txq(netdev) > Now remove it because Ilya's fix: > "dpif-netdev: Force port reconfiguration to change dynamic_txqs." Thanks! This version looks mostly fine. One small comment inline. Best regards, Ilya Maximets. > --- > NEWS | 3 +++ > lib/netdev-linux.c | 20 ++++++++++++++++++++ > lib/netdev-provider.h | 1 + > lib/netdev.c | 1 + > tests/system-afxdp.at | 23 +++++++++++++++++++++++ > 5 files changed, 48 insertions(+) > > diff --git a/NEWS b/NEWS > index 32ca2e0c6642..2ca8f6266145 100644 > --- a/NEWS > +++ b/NEWS > @@ -10,6 +10,9 @@ Post-v2.13.0 > * Deprecated DPDK ring ports (dpdkr) are no longer supported. > - Linux datapath: > * Support for kernel versions up to 5.5.x. > + - AF_XDP: > + * New netdev class 'afxdp-nonpmd' for netdev-afxdp to save CPU cycles > + by enabling interrupt mode. > > > v2.13.0 - 14 Feb 2020 > diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c > index c6e46f1885aa..bcc28869516e 100644 > --- a/lib/netdev-linux.c > +++ b/lib/netdev-linux.c > @@ -3624,6 +3624,26 @@ const struct netdev_class netdev_afxdp_class = { > .rxq_destruct = netdev_afxdp_rxq_destruct, > .rxq_recv = netdev_afxdp_rxq_recv, > }; > + > +const struct netdev_class netdev_afxdp_nonpmd_class = { > + NETDEV_LINUX_CLASS_COMMON, > + .type = "afxdp-nonpmd", > + .is_pmd = false, > + .init = netdev_afxdp_init, > + .construct = netdev_afxdp_construct, > + .destruct = netdev_afxdp_destruct, > + .get_stats = netdev_afxdp_get_stats, > + .get_custom_stats = netdev_afxdp_get_custom_stats, > + .get_status = netdev_linux_get_status, > + .set_config = netdev_afxdp_set_config, > + .get_config = netdev_afxdp_get_config, > + .reconfigure = netdev_afxdp_reconfigure, > + .get_numa_id = netdev_linux_get_numa_id, > + .send = netdev_afxdp_batch_send, > + .rxq_construct = netdev_afxdp_rxq_construct, > + .rxq_destruct = netdev_afxdp_rxq_destruct, > + .rxq_recv = netdev_afxdp_rxq_recv, Above looks like almost same as for "pmd" version. Could you, please, make a macro like NETDEV_AFXDP_CLASS_COMMON or something like that to avoid duplication? > +}; > #endif > > > diff --git a/lib/netdev-provider.h b/lib/netdev-provider.h > index 6f509424bc81..d9503adb0fb6 100644 > --- a/lib/netdev-provider.h > +++ b/lib/netdev-provider.h > @@ -850,6 +850,7 @@ extern const struct netdev_class netdev_tap_class; > > #ifdef HAVE_AF_XDP > extern const struct netdev_class netdev_afxdp_class; > +extern const struct netdev_class netdev_afxdp_nonpmd_class; > #endif > #ifdef __cplusplus > } > diff --git a/lib/netdev.c b/lib/netdev.c > index 8c44eee8e98a..90962eec66cf 100644 > --- a/lib/netdev.c > +++ b/lib/netdev.c > @@ -154,6 +154,7 @@ netdev_initialize(void) > netdev_register_flow_api_provider(&netdev_offload_tc); > #ifdef HAVE_AF_XDP > netdev_register_provider(&netdev_afxdp_class); > + netdev_register_provider(&netdev_afxdp_nonpmd_class); > #endif > #endif > #if defined(__FreeBSD__) || defined(__NetBSD__) > diff --git a/tests/system-afxdp.at b/tests/system-afxdp.at > index e4451624f882..0d09906fb6c8 100644 > --- a/tests/system-afxdp.at > +++ b/tests/system-afxdp.at > @@ -22,3 +22,26 @@ AT_CHECK([grep "ovs-p0: could not set configuration" ovs-vswitchd.log | wc -l], > OVS_TRAFFIC_VSWITCHD_STOP(["/ovs-p0: Too big 'n_rxq'/d > /ovs-p0: could not set configuration/d"]) > AT_CLEANUP > + > + > +AT_SETUP([AF_XDP - ping between pmd and non-pmd ports]) > +AT_KEYWORDS([afxdp nonpmd]) > +OVS_TRAFFIC_VSWITCHD_START() > + > +AT_CHECK([ovs-ofctl add-flow br0 "actions=normal"]) > + > +ADD_NAMESPACES(at_ns0, at_ns1) > +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24") > +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24") > + > +AT_CHECK([ovs-vsctl del-port ovs-p0]) > +AT_CHECK([ovs-vsctl add-port br0 ovs-p0 -- \ > + set interface ovs-p0 type=afxdp-nonpmd options:n_rxq=1], > + [0], [], [stderr]) > + > +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 | FORMAT_PING], [0], [dnl > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > +]) > + > +OVS_TRAFFIC_VSWITCHD_STOP > +AT_CLEANUP >
On Mon, Apr 13, 2020 at 3:20 PM Ilya Maximets <i.maximets@ovn.org> wrote: > > On 3/25/20 10:44 PM, William Tu wrote: > > The patch adds a new netdev class 'afxdp-nonpmd' to enable afxdp > > interrupt mode. This is similar to 'type=afxdp', except that the > > is_pmd field is set to false. As a result, the packet processing > > is handled by main thread, not pmd thread. This avoids burning > > the CPU to always 100% when there is no traffic. > > > > Tested-at: https://travis-ci.org/github/williamtu/ovs-travis/builds/666980138 > > Signed-off-by: William Tu <u9012063@gmail.com> > > --- > > v4: > > - Previously crash and fix it with qid = qid % netdev_n_txq(netdev) > > Now remove it because Ilya's fix: > > "dpif-netdev: Force port reconfiguration to change dynamic_txqs." > > Thanks! This version looks mostly fine. One small comment inline. > > Best regards, Ilya Maximets. > > > --- > > NEWS | 3 +++ > > lib/netdev-linux.c | 20 ++++++++++++++++++++ > > lib/netdev-provider.h | 1 + > > lib/netdev.c | 1 + > > tests/system-afxdp.at | 23 +++++++++++++++++++++++ > > 5 files changed, 48 insertions(+) > > > > diff --git a/NEWS b/NEWS > > index 32ca2e0c6642..2ca8f6266145 100644 > > --- a/NEWS > > +++ b/NEWS > > @@ -10,6 +10,9 @@ Post-v2.13.0 > > * Deprecated DPDK ring ports (dpdkr) are no longer supported. > > - Linux datapath: > > * Support for kernel versions up to 5.5.x. > > + - AF_XDP: > > + * New netdev class 'afxdp-nonpmd' for netdev-afxdp to save CPU cycles > > + by enabling interrupt mode. > > > > > > v2.13.0 - 14 Feb 2020 > > diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c > > index c6e46f1885aa..bcc28869516e 100644 > > --- a/lib/netdev-linux.c > > +++ b/lib/netdev-linux.c > > @@ -3624,6 +3624,26 @@ const struct netdev_class netdev_afxdp_class = { > > .rxq_destruct = netdev_afxdp_rxq_destruct, > > .rxq_recv = netdev_afxdp_rxq_recv, > > }; > > + > > +const struct netdev_class netdev_afxdp_nonpmd_class = { > > + NETDEV_LINUX_CLASS_COMMON, > > + .type = "afxdp-nonpmd", > > + .is_pmd = false, > > + .init = netdev_afxdp_init, > > + .construct = netdev_afxdp_construct, > > + .destruct = netdev_afxdp_destruct, > > + .get_stats = netdev_afxdp_get_stats, > > + .get_custom_stats = netdev_afxdp_get_custom_stats, > > + .get_status = netdev_linux_get_status, > > + .set_config = netdev_afxdp_set_config, > > + .get_config = netdev_afxdp_get_config, > > + .reconfigure = netdev_afxdp_reconfigure, > > + .get_numa_id = netdev_linux_get_numa_id, > > + .send = netdev_afxdp_batch_send, > > + .rxq_construct = netdev_afxdp_rxq_construct, > > + .rxq_destruct = netdev_afxdp_rxq_destruct, > > + .rxq_recv = netdev_afxdp_rxq_recv, > > Above looks like almost same as for "pmd" version. Could you, please, > make a macro like NETDEV_AFXDP_CLASS_COMMON or something like that to > avoid duplication? > OK! Thanks for the feedback. William
diff --git a/NEWS b/NEWS index 32ca2e0c6642..2ca8f6266145 100644 --- a/NEWS +++ b/NEWS @@ -10,6 +10,9 @@ Post-v2.13.0 * Deprecated DPDK ring ports (dpdkr) are no longer supported. - Linux datapath: * Support for kernel versions up to 5.5.x. + - AF_XDP: + * New netdev class 'afxdp-nonpmd' for netdev-afxdp to save CPU cycles + by enabling interrupt mode. v2.13.0 - 14 Feb 2020 diff --git a/lib/netdev-linux.c b/lib/netdev-linux.c index c6e46f1885aa..bcc28869516e 100644 --- a/lib/netdev-linux.c +++ b/lib/netdev-linux.c @@ -3624,6 +3624,26 @@ const struct netdev_class netdev_afxdp_class = { .rxq_destruct = netdev_afxdp_rxq_destruct, .rxq_recv = netdev_afxdp_rxq_recv, }; + +const struct netdev_class netdev_afxdp_nonpmd_class = { + NETDEV_LINUX_CLASS_COMMON, + .type = "afxdp-nonpmd", + .is_pmd = false, + .init = netdev_afxdp_init, + .construct = netdev_afxdp_construct, + .destruct = netdev_afxdp_destruct, + .get_stats = netdev_afxdp_get_stats, + .get_custom_stats = netdev_afxdp_get_custom_stats, + .get_status = netdev_linux_get_status, + .set_config = netdev_afxdp_set_config, + .get_config = netdev_afxdp_get_config, + .reconfigure = netdev_afxdp_reconfigure, + .get_numa_id = netdev_linux_get_numa_id, + .send = netdev_afxdp_batch_send, + .rxq_construct = netdev_afxdp_rxq_construct, + .rxq_destruct = netdev_afxdp_rxq_destruct, + .rxq_recv = netdev_afxdp_rxq_recv, +}; #endif diff --git a/lib/netdev-provider.h b/lib/netdev-provider.h index 6f509424bc81..d9503adb0fb6 100644 --- a/lib/netdev-provider.h +++ b/lib/netdev-provider.h @@ -850,6 +850,7 @@ extern const struct netdev_class netdev_tap_class; #ifdef HAVE_AF_XDP extern const struct netdev_class netdev_afxdp_class; +extern const struct netdev_class netdev_afxdp_nonpmd_class; #endif #ifdef __cplusplus } diff --git a/lib/netdev.c b/lib/netdev.c index 8c44eee8e98a..90962eec66cf 100644 --- a/lib/netdev.c +++ b/lib/netdev.c @@ -154,6 +154,7 @@ netdev_initialize(void) netdev_register_flow_api_provider(&netdev_offload_tc); #ifdef HAVE_AF_XDP netdev_register_provider(&netdev_afxdp_class); + netdev_register_provider(&netdev_afxdp_nonpmd_class); #endif #endif #if defined(__FreeBSD__) || defined(__NetBSD__) diff --git a/tests/system-afxdp.at b/tests/system-afxdp.at index e4451624f882..0d09906fb6c8 100644 --- a/tests/system-afxdp.at +++ b/tests/system-afxdp.at @@ -22,3 +22,26 @@ AT_CHECK([grep "ovs-p0: could not set configuration" ovs-vswitchd.log | wc -l], OVS_TRAFFIC_VSWITCHD_STOP(["/ovs-p0: Too big 'n_rxq'/d /ovs-p0: could not set configuration/d"]) AT_CLEANUP + + +AT_SETUP([AF_XDP - ping between pmd and non-pmd ports]) +AT_KEYWORDS([afxdp nonpmd]) +OVS_TRAFFIC_VSWITCHD_START() + +AT_CHECK([ovs-ofctl add-flow br0 "actions=normal"]) + +ADD_NAMESPACES(at_ns0, at_ns1) +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24") +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24") + +AT_CHECK([ovs-vsctl del-port ovs-p0]) +AT_CHECK([ovs-vsctl add-port br0 ovs-p0 -- \ + set interface ovs-p0 type=afxdp-nonpmd options:n_rxq=1], + [0], [], [stderr]) + +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -w 2 10.1.1.2 | FORMAT_PING], [0], [dnl +3 packets transmitted, 3 received, 0% packet loss, time 0ms +]) + +OVS_TRAFFIC_VSWITCHD_STOP +AT_CLEANUP
The patch adds a new netdev class 'afxdp-nonpmd' to enable afxdp interrupt mode. This is similar to 'type=afxdp', except that the is_pmd field is set to false. As a result, the packet processing is handled by main thread, not pmd thread. This avoids burning the CPU to always 100% when there is no traffic. Tested-at: https://travis-ci.org/github/williamtu/ovs-travis/builds/666980138 Signed-off-by: William Tu <u9012063@gmail.com> --- v4: - Previously crash and fix it with qid = qid % netdev_n_txq(netdev) Now remove it because Ilya's fix: "dpif-netdev: Force port reconfiguration to change dynamic_txqs." --- NEWS | 3 +++ lib/netdev-linux.c | 20 ++++++++++++++++++++ lib/netdev-provider.h | 1 + lib/netdev.c | 1 + tests/system-afxdp.at | 23 +++++++++++++++++++++++ 5 files changed, 48 insertions(+)