Message ID | SY4PR01MB84389B059A1EC5FCE3AEE74ECDE2A@SY4PR01MB8438.ausprd01.prod.outlook.com |
---|---|
State | Changes Requested |
Headers | show |
Series | netdev-dpdk: Add support for userspace port-based packet-per-second policing. | expand |
Context | Check | Description |
---|---|---|
ovsrobot/apply-robot | warning | apply and check: warning |
ovsrobot/github-robot-_Build_and_Test | success | github build: passed |
On 26 Aug 2023, at 8:01, miterv@outlook.com wrote: > From: Lin Huang <linhuang@ruijie.com.cn> > > OvS has supported packet-per-second policer which can be set at > ingress > and egress side in kernel datapath. But the userspace datapath dosen't > support for ingress and egress packet-per-second policing now. > > So, this patch add support for userspace ingress pps policing by using > native ovs token bucket library. Token bucket is accumulated by 'rate' > tokens per millisecond and store maxiumim tokens at 'burst' bucket > size. > One token in the bucket means one packet (1 kpkts * millisecond) which > will drop or pass by policer. > > This patch reuse 'ingress_policing_kpkts_rate' and > 'ingress_policing_kpkts_burst' options at interface table. Now > userspace > ingress policer supports setting packet-per-second limits in addition > to > the previously configurable byte rate settings. > > Examples: > $ ovs-vsctl set interface dpdk0 ingress_policing_rate=12300 > $ ovs-vsctl set interface dpdk0 ingress_policing_burst=12300 > $ ovs-vsctl set interface dpdk0 ingress_policing_kpkts_rate=123 > $ ovs-vsctl set interface dpdk0 ingress_policing_kpkts_burst=123 > > Add some unit tests for ingress packet-per-second policing. > > Signed-off-by: Lin Huang <linhuang@ruijie.com.cn> > --- > Documentation/topics/dpdk/qos.rst | 21 ++ > NEWS | 2 + > lib/netdev-dpdk.c | 91 +++++++- > tests/system-dpdk.at | 337 > ++++++++++++++++++++++++++++++ > 4 files changed, 443 insertions(+), 8 deletions(-) > > diff --git a/Documentation/topics/dpdk/qos.rst > b/Documentation/topics/dpdk/qos.rst > index 37f482cc5..3c5d14e4c 100644 > --- a/Documentation/topics/dpdk/qos.rst > +++ b/Documentation/topics/dpdk/qos.rst > @@ -120,6 +120,9 @@ Refer to ``vswitch.xml`` for more details on > egress policer. > Rate Limiting (Ingress Policing) > -------------------------------- > > +Bytes Per Second Policer > +~~~~~~~~~~~~~~~~~~~~~~~~ > + > Assuming you have a :doc:`vhost-user port <vhost-user>` receiving > traffic > consisting of packets of size 64 bytes, the following command would > limit the > reception rate of the port to ~1,000,000 packets per second:: > @@ -135,6 +138,24 @@ To clear the ingress policer configuration from > the port:: > > $ ovs-vsctl set interface vhost-user0 ingress_policing_rate=0 > > +Packets Per Second Policer > +~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +Assuming you have a :doc:`vhost-user port <vhost-user>` receiving > traffic, > +the following command would limit the reception rate of the port to > ~1,000,000 > +packets per second:: > + > + $ ovs-vsctl set interface dpdk0 ingress_policing_kpkts_rate=1000 > \ > + ingress_policing_kpkts_burst=1000` > + > +To examine the ingress policer configuration of the port:: > + > + $ ovs-vsctl list interface vhost-user0 > + > +To clear the ingress policer configuration from the port:: > + > + $ ovs-vsctl set interface vhost-user0 ingress_policing_rate=0 Cut/paste error should be ingress_policing_kpkts_rate=0 > + > Refer to ``vswitch.xml`` for more details on ingress policer. > > Flow Control > diff --git a/NEWS b/NEWS > index 430c3daaf..39195768a 100644 > --- a/NEWS > +++ b/NEWS > @@ -64,6 +64,8 @@ v3.2.0 - 17 Aug 2023 > max sleep configuration of PMD thread cores. > * Removed experimental tag from PMD load based sleeping. > * Added new Qos type 'pkts-policer' to support kilo > packet-per-second policing. > + * Added support for ingress kilo packet-per-second policing > configured by > + ingress_policing_kpkts_rate/burst options. > - Linux TC offload: > * Add support for offloading VXLAN tunnels with the GBP > extensions. > - Python > diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c > index c6a26dc7e..06bac720a 100644 > --- a/lib/netdev-dpdk.c > +++ b/lib/netdev-dpdk.c > @@ -406,10 +406,17 @@ struct dpdk_tx_queue { > ); > }; > > +enum policer_type { > + POLICER_BPS = 1 << 0, /* Rate value in bytes/sec. */ > + POLICER_PKTPS = 1 << 1, /* Rate value in packet/sec. */ > +}; > + > struct ingress_policer { > struct rte_meter_srtcm_params app_srtcm_params; > struct rte_meter_srtcm in_policer; > struct rte_meter_srtcm_profile in_prof; > + struct token_bucket tb; > + enum policer_type type; > rte_spinlock_t policer_lock; > }; > > @@ -516,6 +523,9 @@ struct netdev_dpdk { > uint32_t policer_rate; > uint32_t policer_burst; > > + uint32_t policer_kpkts_rate; > + uint32_t policer_kpkts_burst; > + > /* Array of vhost rxq states, see vring_state_changed. */ > bool *vhost_rxq_enabled; > > @@ -615,6 +625,14 @@ is_dpdk_class(const struct netdev_class *class) > || class->destruct == netdev_dpdk_vhost_destruct; > } > > +static int > +kpkts_policer_run_single_packet(struct token_bucket *tb, struct > rte_mbuf **pkts, For function (forward) declatation we leave out the name for structures. So this would become (same below): kpkts_policer_run_single_packet(struct token_bucket *, struct rte_mbuf **, > + int pkt_cnt, bool should_steal); > + > +static int > +kpkts_policer_profile_config(struct token_bucket *tb, > + uint32_t kpkts_rate, uint32_t > kpkts_burst); > + > /* DPDK NIC drivers allocate RX buffers at a particular granularity, > typically > * aligned at 1k or less. If a declared mbuf size is not a multiple > of this > * value, insufficient buffers are allocated to accomodate the packet > in its > @@ -1462,6 +1480,8 @@ common_construct(struct netdev *netdev, > dpdk_port_t port_no, > ovsrcu_init(&dev->ingress_policer, NULL); > dev->policer_rate = 0; > dev->policer_burst = 0; > + dev->policer_kpkts_rate = 0; > + dev->policer_kpkts_burst = 0; > > netdev->n_rxq = 0; > netdev->n_txq = 0; > @@ -2581,9 +2601,17 @@ ingress_policer_run(struct ingress_policer > *policer, struct rte_mbuf **pkts, > int cnt = 0; > > rte_spinlock_lock(&policer->policer_lock); > - cnt = srtcm_policer_run_single_packet(&policer->in_policer, > - &policer->in_prof, > - pkts, pkt_cnt, > should_steal); > + if (policer->type & POLICER_BPS) { > + cnt = srtcm_policer_run_single_packet(&policer->in_policer, > + &policer->in_prof, > + pkts, pkt_cnt, > should_steal); > + } > + > + /* bps and pps rate limits not allowed to configure at the same > time. */ If this is true it shouw be "} else if if (policer->type & POLICER_PKTPS) {" > + if (policer->type & POLICER_PKTPS) { > + cnt = kpkts_policer_run_single_packet(&policer->tb, pkts, > pkt_cnt, > + should_steal); > + } > rte_spinlock_unlock(&policer->policer_lock); > > return cnt; > @@ -3809,7 +3837,7 @@ netdev_dpdk_policer_construct(uint32_t rate, > uint32_t burst) > uint64_t burst_bytes; > int err = 0; > > - policer = xmalloc(sizeof *policer); > + policer = xzalloc(sizeof *policer); > rte_spinlock_init(&policer->policer_lock); > > /* rte_meter requires bytes so convert kbits rate and burst to > bytes. */ > @@ -3831,18 +3859,48 @@ netdev_dpdk_policer_construct(uint32_t rate, > uint32_t burst) > return NULL; > } > > + policer->type |= POLICER_BPS; > + > + return policer; > +} > + > +static struct ingress_policer * > +netdev_dpdk_kpkts_policer_construct(uint32_t kpkts_rate, uint32_t > kpkts_burst) > +{ > + struct ingress_policer *policer; > + int err; > + > + policer = xzalloc(sizeof *policer); > + rte_spinlock_init(&policer->policer_lock); > + > + err = kpkts_policer_profile_config(&policer->tb, kpkts_rate, > kpkts_burst); > + if (err) { > + VLOG_ERR("Could not create token bucket for ingress > policer"); > + free(policer); > + return NULL; > + } > + > + policer->type |= POLICER_PKTPS; > + > return policer; > } > > static int > netdev_dpdk_set_policing(struct netdev* netdev, uint32_t > policer_rate, > uint32_t policer_burst, > - uint32_t policer_kpkts_rate OVS_UNUSED, > - uint32_t policer_kpkts_burst OVS_UNUSED) > + uint32_t policer_kpkts_rate, > + uint32_t policer_kpkts_burst) > { > struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); > struct ingress_policer *policer; > > + if (policer_rate && policer_kpkts_rate) { > + VLOG_WARN("packet-per-second and byte-per-second rate limits > not" > + " allowed to configure at the same time."); What about (notice the missing . at the end): packet-per-second and byte-per-second rate limits can't be configured at the same time > + > + return -EINVAL; > + } > + > /* Force to 0 if no rate specified, > * default to 8000 kbits if burst is 0, > * else stick with user-specified value. > @@ -3851,13 +3909,24 @@ netdev_dpdk_set_policing(struct netdev* > netdev, uint32_t policer_rate, > : !policer_burst ? 8000 > : policer_burst); > > + /* > + * Force to 0 if no rate specified, > + * default to rate value if burst is 0, > + * else stick with user-specified value. > + */ > + policer_kpkts_burst = (!policer_kpkts_rate ? 0 > + : !policer_kpkts_burst ? > policer_kpkts_rate > + : policer_kpkts_burst); See same comments in other patch. Also for the ingress rate limiter this is what is in the doc(xml): 3989 Maximum burst size for data received on this interface, in kpkts (1 3990 kpkts is 1000 packets). The default burst size if set to 3991 <code>0</code> is 16 kpkts. This value has no effect if > + > ovs_mutex_lock(&dev->mutex); > > policer = ovsrcu_get_protected(struct ingress_policer *, > - &dev->ingress_policer); > + &dev->ingress_policer); > > if (dev->policer_rate == policer_rate && > - dev->policer_burst == policer_burst) { > + dev->policer_burst == policer_burst && > + dev->policer_kpkts_rate == policer_kpkts_rate && > + dev->policer_kpkts_burst == policer_kpkts_burst) { > /* Assume that settings haven't changed since we last set > them. */ > ovs_mutex_unlock(&dev->mutex); > return 0; > @@ -3870,12 +3939,18 @@ netdev_dpdk_set_policing(struct netdev* > netdev, uint32_t policer_rate, > > if (policer_rate != 0) { > policer = netdev_dpdk_policer_construct(policer_rate, > policer_burst); > + } else if (policer_kpkts_rate != 0) { > + policer = > netdev_dpdk_kpkts_policer_construct(policer_kpkts_rate, > + > policer_kpkts_burst); Should we add a warning here to the user when both are configured, so he is aware of the invalid configuration and that we are using the byte policer? > } else { > policer = NULL; > } > + > ovsrcu_set(&dev->ingress_policer, policer); > dev->policer_rate = policer_rate; > dev->policer_burst = policer_burst; > + dev->policer_kpkts_rate = policer_kpkts_rate; > + dev->policer_kpkts_burst = policer_kpkts_burst; > ovs_mutex_unlock(&dev->mutex); > > return 0; > diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at > index 8b80a31e6..3680d4d40 100644 > --- a/tests/system-dpdk.at > +++ b/tests/system-dpdk.at > @@ -426,6 +426,343 @@ AT_CLEANUP > dnl > -------------------------------------------------------------------------- > > > +dnl > -------------------------------------------------------------------------- Did not add specific comments below, but please go over this taking the comments on patch 3 into account, as most of them apply here! > +dnl Ingress policing (kpkts) create delete vport port > +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) create delete vport > port]) > +AT_KEYWORDS([dpdk]) > + > +OVS_DPDK_PRE_CHECK() > +OVS_DPDK_START() > + > +dnl Add userspace bridge and attach it to OVS and add ingress policer > +AT_CHECK([ovs-appctl vlog/set netdev_dpdk:dbg], [], [stdout]) > +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 > datapath_type=netdev]) > +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set > Interface dpdkvhostuserclient0 type=dpdkvhostuserclient > options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], > [stderr]) > +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 > ingress_policing_kpkts_rate=10000 ingress_policing_kpkts_burst=10000]) > +AT_CHECK([ovs-vsctl show], [], [stdout]) > +sleep 2 > + > +dnl Remove ingress policer > +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 > ingress_policing_kpkts_rate=0 ingress_policing_kpkts_burst=0]) > + > +dnl Fail if ingress policer could not be created > +AT_FAIL_IF([grep "Could not create token bucket for ingress policer" > ovs-vswitchd.log], [], [stdout]) > + > +dnl Check ingress policer was removed correctly > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 0' stdout], [], > [stdout]) > + > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 0' stdout], [], > [stdout]) > + > +dnl Parse log file > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) > vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in > 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) > reconnecting..." ovs-vswitchd.log], [], [stdout]) > + > +dnl Clean up > +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], > [stdout], [stderr]) > +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ > +\@Could not create rte meter for ingress policer@d > +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No > such file or directory@d > +])") > +AT_CLEANUP > +dnl > -------------------------------------------------------------------------- > + > + > + > +dnl > -------------------------------------------------------------------------- > +dnl Ingress policing (kpkts) no policing rate > +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) no policing rate]) > +AT_KEYWORDS([dpdk]) > + > +OVS_DPDK_PRE_CHECK() > +OVS_DPDK_START() > + > +dnl Add userspace bridge and attach it to OVS and add ingress policer > +AT_CHECK([ovs-appctl vlog/set netdev_dpdk:dbg], [], [stdout]) > +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 > datapath_type=netdev]) > +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set > Interface dpdkvhostuserclient0 type=dpdkvhostuserclient > options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], > [stderr]) > +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 > ingress_policing_kpkts_burst=1000]) > +AT_CHECK([ovs-vsctl show], [], [stdout]) > +sleep 2 > + > +dnl check ingress policer not be created > +AT_FAIL_IF([grep "Could not create token bucket for ingress policer" > ovs-vswitchd.log], [], [stdout]) > + > +dnl Check ingress policer was created correctly > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 1000' stdout], [], > [stdout]) > + > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 0' stdout], [], > [stdout]) > + > + > +dnl Parse log file > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) > vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in > 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) > reconnecting..." ovs-vswitchd.log], [], [stdout]) > + > +dnl Clean up > +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], > [stdout], [stderr]) > +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ > +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No > such file or directory@d > +])") > +AT_CLEANUP > +dnl > -------------------------------------------------------------------------- > + > + > + > +dnl > -------------------------------------------------------------------------- > +dnl Ingress policing (kpkts) no policing burst > +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) no policing burst]) > +AT_KEYWORDS([dpdk]) > + > +OVS_DPDK_PRE_CHECK() > +OVS_DPDK_START() > + > +dnl Add userspace bridge and attach it to OVS and add ingress policer > +AT_CHECK([ovs-appctl vlog/set netdev_dpdk:dbg], [], [stdout]) > +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 > datapath_type=netdev]) > +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set > Interface dpdkvhostuserclient0 type=dpdkvhostuserclient > options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], > [stderr]) > +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 > ingress_policing_kpkts_rate=10000]) > +AT_CHECK([ovs-vsctl show], [], [stdout]) > +sleep 2 > + > +dnl check ingress policer not be created > +AT_FAIL_IF([grep "Could not create token bucket for ingress policer" > ovs-vswitchd.log], [], [stdout]) > + > +dnl Check ingress policer was created correctly > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 0' stdout], [], > [stdout]) > + > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 10000' stdout], [], > [stdout]) > + > +dnl Parse log file > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) > vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in > 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) > reconnecting..." ovs-vswitchd.log], [], [stdout]) > + > +dnl Clean up > +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], > [stdout], [stderr]) > +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ > +\@Could not create rte meter for ingress policer@d > +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No > such file or directory@d > +])") > +AT_CLEANUP > +dnl > -------------------------------------------------------------------------- > + > + > + > +dnl > -------------------------------------------------------------------------- > +dnl Ingress policing (kpkts) max policing rate > +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) max policing rate]) > +AT_KEYWORDS([dpdk]) > + > +OVS_DPDK_PRE_CHECK() > +OVS_DPDK_START() > + > +dnl Add userspace bridge and attach it to OVS and add ingress policer > +AT_CHECK([ovs-appctl vlog/set netdev_dpdk:dbg], [], [stdout]) > +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 > datapath_type=netdev]) > +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set > Interface dpdkvhostuserclient0 type=dpdkvhostuserclient > options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], > [stderr]) > +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 > ingress_policing_kpkts_rate=42949671]) > +AT_CHECK([ovs-vsctl show], [], [stdout]) > +sleep 2 > + > +dnl Check ingress policer was created correctly > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 0' stdout], [], > [stdout]) > + > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 42949671' stdout], > [], [stdout]) > + > +dnl Parse log file > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) > vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in > 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) > reconnecting..." ovs-vswitchd.log], [], [stdout]) > + > +dnl Clean up > +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], > [stdout], [stderr]) > +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ > +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No > such file or directory@d > +\@Could not create token bucket for ingress policer@d > +])") > +AT_CLEANUP > +dnl > -------------------------------------------------------------------------- > + > + > + > +dnl > -------------------------------------------------------------------------- > +dnl Ingress policing (kpkts) max policing burst > +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) max policing burst]) > +AT_KEYWORDS([dpdk]) > + > +OVS_DPDK_PRE_CHECK() > +OVS_DPDK_START() > + > +dnl Add userspace bridge and attach it to OVS and add ingress policer > +AT_CHECK([ovs-appctl vlog/set netdev_dpdk:dbg], [], [stdout]) > +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 > datapath_type=netdev]) > +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set > Interface dpdkvhostuserclient0 type=dpdkvhostuserclient > options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], > [stderr]) > +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 > ingress_policing_kpkts_burst=42949671]) > +AT_CHECK([ovs-vsctl show], [], [stdout]) > +sleep 2 > + > +dnl check ingress policer not be created > +AT_FAIL_IF([grep "Could not create token bucket for ingress policer" > ovs-vswitchd.log], [], [stdout]) > + > +dnl Check ingress policer was created correctly > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 42949671' stdout], > [], [stdout]) > + > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 0' stdout], [], > [stdout]) > + > +dnl Parse log file > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) > vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in > 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) > reconnecting..." ovs-vswitchd.log], [], [stdout]) > + > +dnl Clean up > +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], > [stdout], [stderr]) > +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ > +\@Could not create token bucket for ingress policer@d > +\@Could not create rte meter for ingress policer@d > +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No > such file or directory@d > +])") > +AT_CLEANUP > +dnl > -------------------------------------------------------------------------- > + > + > + > +dnl > -------------------------------------------------------------------------- > +dnl Ingress policing (kpkts) set both bps and pps policing > configurations > +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) both policing > configurations]) > +AT_KEYWORDS([dpdk]) > + > +OVS_DPDK_PRE_CHECK() > +OVS_DPDK_START() > + > +dnl Add userspace bridge and attach it to OVS and add ingress policer > +AT_CHECK([ovs-appctl vlog/set netdev_dpdk:dbg], [], [stdout]) > +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 > datapath_type=netdev]) > +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set > Interface dpdkvhostuserclient0 type=dpdkvhostuserclient > options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], > [stderr]) > +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 > ingress_policing_rate=1234 ingress_policing_burst=1234 > ingress_policing_kpkts_rate=1234 ingress_policing_kpkts_burst=1234]) > +AT_CHECK([ovs-vsctl show], [], [stdout]) > +sleep 2 > + > +dnl check ingress policer not be created > +AT_CHECK([grep "packet-per-second and byte-per-second" > ovs-vswitchd.log | sed 's/^.*|WARN|//'], [0], [stdout]) > + > +dnl Check ingress policer was created correctly > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 1234' stdout], [], > [stdout]) > + > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 1234' stdout], [], > [stdout]) > + > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_rate: 1234' stdout], [], > [stdout]) > + > +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], > [stdout]) > +AT_CHECK([grep -E 'ingress_policing_burst: 1234' stdout], [], > [stdout]) > + > +dnl check ingress policer not be created > +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 > ingress_policing_rate=0 ingress_policing_burst=1234 > ingress_policing_kpkts_rate=1234 ingress_policing_kpkts_burst=1234]) > +AT_FAIL_IF([grep "Could not create token bucket for ingress policer" > ovs-vswitchd.log], [], [stdout]) > + > +dnl Parse log file > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) > vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in > 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) > reconnecting..." ovs-vswitchd.log], [], [stdout]) > + > +dnl Clean up > +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], > [stdout], [stderr]) > +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ > +\@Could not create token bucket for ingress policer@d > +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No > such file or directory@d > +\@packet-per-second and byte-per-second@d > +])") > +AT_CLEANUP > +dnl > -------------------------------------------------------------------------- > + > + > + > +dnl > -------------------------------------------------------------------------- > +dnl Ingress policing (kpkts) police > +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) police]) > +AT_KEYWORDS([dpdk]) > + > +OVS_DPDK_PRE_CHECK() > +AT_SKIP_IF([! which dpdk-testpmd >/dev/null 2>/dev/null]) > +OVS_DPDK_START([--no-pci]) > + > +dnl Find number of sockets > +AT_CHECK([lscpu], [], [stdout]) > +AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while > (c++<$(3)) {printf "512,"}; print "512"}' > NUMA_NODE]) > + > +dnl Add userspace bridge and attach it to OVS > +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 > datapath_type=netdev]) > + > +dnl Parse log file > +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuser0 -- set Interface > dpdkvhostuser0 type=dpdkvhostuser], [], [stdout], [stderr]) > +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuser1 -- set Interface > dpdkvhostuser1 type=dpdkvhostuser], [], [stdout], [stderr]) > +AT_CHECK([ovs-vsctl show], [], [stdout]) > + > +dnl Parse log file > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostuser0) vhost-user > server: socket created" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "Socket $OVS_RUNDIR/dpdkvhostuser0 created for > vhost-user port dpdkvhostuser0" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostuser0) binding > succeeded" ovs-vswitchd.log], [], [stdout]) > + > +dnl Parse log file > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostuser1) vhost-user > server: socket created" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "Socket $OVS_RUNDIR/dpdkvhostuser1 created for > vhost-user port dpdkvhostuser1" ovs-vswitchd.log], [], [stdout]) > +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostuser1) binding > succeeded" ovs-vswitchd.log], [], [stdout]) > + > +dnl Configure the police for interface. > +AT_CHECK([ovs-vsctl set interface dpdkvhostuser1 > ingress_policing_kpkts_rate=1 ingress_policing_kpkts_burst=1]) > + > +dnl add flows, only send packets from dpdkvhostuser1 to > dpdkvhostuser0. > +AT_DATA([flows.txt], [dnl > +priority=100,in_port=dpdkvhostuser1,ip,actions=dpdkvhostuser0 > +]) > + > +AT_CHECK([ovs-ofctl del-flows br10]) > +AT_CHECK([ovs-ofctl add-flows br10 flows.txt]) > + > +dnl Execute testpmd in background > +on_exit "pkill -f -x -9 'tail -f /dev/null'" > +tail -f /dev/null | dpdk-testpmd --socket-mem="$(cat NUMA_NODE)" > --no-pci\ > + --vdev="net_virtio_user0,path=$OVS_RUNDIR/dpdkvhostuser0" > \ > + --vdev="net_virtio_user1,path=$OVS_RUNDIR/dpdkvhostuser1" > \ > + --single-file-segments -- --forward-mode=flowgen -a > > $OVS_RUNDIR/testpmd-dpdkvhostuser.log 2>&1 & > + > +dnl sent packet 10 second. > +AT_CHECK([sleep 10]) > + > +dnl Clean up the testpmd now > +pkill -f -x -9 'tail -f /dev/null' > + > +dnl ---------------------- Forward statistics for port 0 > ---------------------- > +dnl RX-packets: 9911 RX-dropped: 0 RX-total: > 9911 > +dnl TX-packets: 15937632 TX-dropped: 226661984 TX-total: > 242599616 > +dnl > ---------------------------------------------------------------------------- > +port0_rx_packets=`cat testpmd-dpdkvhostuser.log | grep "Forward > statistics for port 0" -A 1 | grep "RX-packets:" | awk '{print $2}'` > +echo "port0_rx_packets=$port0_rx_packets" > + > +AT_CHECK([test $port0_rx_packets -lt 10500]) > +AT_CHECK([test $port0_rx_packets -gt 9500]) > + > +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ > +\@dpdkvhostuser ports are considered deprecated; please migrate to > dpdkvhostuserclient ports.@d > +])") > +AT_CLEANUP > +dnl > -------------------------------------------------------------------------- > + > + > + > dnl > -------------------------------------------------------------------------- > dnl QoS create delete phy port > AT_SETUP([OVS-DPDK - QoS create delete phy port]) > -- > 2.39.3
diff --git a/Documentation/topics/dpdk/qos.rst b/Documentation/topics/dpdk/qos.rst index 37f482cc5..3c5d14e4c 100644 --- a/Documentation/topics/dpdk/qos.rst +++ b/Documentation/topics/dpdk/qos.rst @@ -120,6 +120,9 @@ Refer to ``vswitch.xml`` for more details on egress policer. Rate Limiting (Ingress Policing) -------------------------------- +Bytes Per Second Policer +~~~~~~~~~~~~~~~~~~~~~~~~ + Assuming you have a :doc:`vhost-user port <vhost-user>` receiving traffic consisting of packets of size 64 bytes, the following command would limit the reception rate of the port to ~1,000,000 packets per second:: @@ -135,6 +138,24 @@ To clear the ingress policer configuration from the port:: $ ovs-vsctl set interface vhost-user0 ingress_policing_rate=0 +Packets Per Second Policer +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Assuming you have a :doc:`vhost-user port <vhost-user>` receiving traffic, +the following command would limit the reception rate of the port to ~1,000,000 +packets per second:: + + $ ovs-vsctl set interface dpdk0 ingress_policing_kpkts_rate=1000 \ + ingress_policing_kpkts_burst=1000` + +To examine the ingress policer configuration of the port:: + + $ ovs-vsctl list interface vhost-user0 + +To clear the ingress policer configuration from the port:: + + $ ovs-vsctl set interface vhost-user0 ingress_policing_rate=0 + Refer to ``vswitch.xml`` for more details on ingress policer. Flow Control diff --git a/NEWS b/NEWS index 430c3daaf..39195768a 100644 --- a/NEWS +++ b/NEWS @@ -64,6 +64,8 @@ v3.2.0 - 17 Aug 2023 max sleep configuration of PMD thread cores. * Removed experimental tag from PMD load based sleeping. * Added new Qos type 'pkts-policer' to support kilo packet-per-second policing. + * Added support for ingress kilo packet-per-second policing configured by + ingress_policing_kpkts_rate/burst options. - Linux TC offload: * Add support for offloading VXLAN tunnels with the GBP extensions. - Python diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index c6a26dc7e..06bac720a 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -406,10 +406,17 @@ struct dpdk_tx_queue { ); }; +enum policer_type { + POLICER_BPS = 1 << 0, /* Rate value in bytes/sec. */ + POLICER_PKTPS = 1 << 1, /* Rate value in packet/sec. */ +}; + struct ingress_policer { struct rte_meter_srtcm_params app_srtcm_params; struct rte_meter_srtcm in_policer; struct rte_meter_srtcm_profile in_prof; + struct token_bucket tb; + enum policer_type type; rte_spinlock_t policer_lock; }; @@ -516,6 +523,9 @@ struct netdev_dpdk { uint32_t policer_rate; uint32_t policer_burst; + uint32_t policer_kpkts_rate; + uint32_t policer_kpkts_burst; + /* Array of vhost rxq states, see vring_state_changed. */ bool *vhost_rxq_enabled; @@ -615,6 +625,14 @@ is_dpdk_class(const struct netdev_class *class) || class->destruct == netdev_dpdk_vhost_destruct; } +static int +kpkts_policer_run_single_packet(struct token_bucket *tb, struct rte_mbuf **pkts, + int pkt_cnt, bool should_steal); + +static int +kpkts_policer_profile_config(struct token_bucket *tb, + uint32_t kpkts_rate, uint32_t kpkts_burst); + /* DPDK NIC drivers allocate RX buffers at a particular granularity, typically * aligned at 1k or less. If a declared mbuf size is not a multiple of this * value, insufficient buffers are allocated to accomodate the packet in its @@ -1462,6 +1480,8 @@ common_construct(struct netdev *netdev, dpdk_port_t port_no, ovsrcu_init(&dev->ingress_policer, NULL); dev->policer_rate = 0; dev->policer_burst = 0; + dev->policer_kpkts_rate = 0; + dev->policer_kpkts_burst = 0; netdev->n_rxq = 0; netdev->n_txq = 0; @@ -2581,9 +2601,17 @@ ingress_policer_run(struct ingress_policer *policer, struct rte_mbuf **pkts, int cnt = 0; rte_spinlock_lock(&policer->policer_lock); - cnt = srtcm_policer_run_single_packet(&policer->in_policer, - &policer->in_prof, - pkts, pkt_cnt, should_steal); + if (policer->type & POLICER_BPS) { + cnt = srtcm_policer_run_single_packet(&policer->in_policer, + &policer->in_prof, + pkts, pkt_cnt, should_steal); + } + + /* bps and pps rate limits not allowed to configure at the same time. */ + if (policer->type & POLICER_PKTPS) { + cnt = kpkts_policer_run_single_packet(&policer->tb, pkts, pkt_cnt, + should_steal); + } rte_spinlock_unlock(&policer->policer_lock); return cnt; @@ -3809,7 +3837,7 @@ netdev_dpdk_policer_construct(uint32_t rate, uint32_t burst) uint64_t burst_bytes; int err = 0; - policer = xmalloc(sizeof *policer); + policer = xzalloc(sizeof *policer); rte_spinlock_init(&policer->policer_lock); /* rte_meter requires bytes so convert kbits rate and burst to bytes. */ @@ -3831,18 +3859,48 @@ netdev_dpdk_policer_construct(uint32_t rate, uint32_t burst) return NULL; } + policer->type |= POLICER_BPS; + + return policer; +} + +static struct ingress_policer * +netdev_dpdk_kpkts_policer_construct(uint32_t kpkts_rate, uint32_t kpkts_burst) +{ + struct ingress_policer *policer; + int err; + + policer = xzalloc(sizeof *policer); + rte_spinlock_init(&policer->policer_lock); + + err = kpkts_policer_profile_config(&policer->tb, kpkts_rate, kpkts_burst); + if (err) { + VLOG_ERR("Could not create token bucket for ingress policer"); + free(policer); + return NULL; + } + + policer->type |= POLICER_PKTPS; + return policer; } static int netdev_dpdk_set_policing(struct netdev* netdev, uint32_t policer_rate, uint32_t policer_burst, - uint32_t policer_kpkts_rate OVS_UNUSED, - uint32_t policer_kpkts_burst OVS_UNUSED) + uint32_t policer_kpkts_rate, + uint32_t policer_kpkts_burst) { struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); struct ingress_policer *policer; + if (policer_rate && policer_kpkts_rate) { + VLOG_WARN("packet-per-second and byte-per-second rate limits not" + " allowed to configure at the same time."); + + return -EINVAL; + } + /* Force to 0 if no rate specified, * default to 8000 kbits if burst is 0, * else stick with user-specified value. @@ -3851,13 +3909,24 @@ netdev_dpdk_set_policing(struct netdev* netdev, uint32_t policer_rate, : !policer_burst ? 8000 : policer_burst); + /* + * Force to 0 if no rate specified, + * default to rate value if burst is 0, + * else stick with user-specified value. + */ + policer_kpkts_burst = (!policer_kpkts_rate ? 0 + : !policer_kpkts_burst ? policer_kpkts_rate + : policer_kpkts_burst); + ovs_mutex_lock(&dev->mutex); policer = ovsrcu_get_protected(struct ingress_policer *, - &dev->ingress_policer); + &dev->ingress_policer); if (dev->policer_rate == policer_rate && - dev->policer_burst == policer_burst) { + dev->policer_burst == policer_burst && + dev->policer_kpkts_rate == policer_kpkts_rate && + dev->policer_kpkts_burst == policer_kpkts_burst) { /* Assume that settings haven't changed since we last set them. */ ovs_mutex_unlock(&dev->mutex); return 0; @@ -3870,12 +3939,18 @@ netdev_dpdk_set_policing(struct netdev* netdev, uint32_t policer_rate, if (policer_rate != 0) { policer = netdev_dpdk_policer_construct(policer_rate, policer_burst); + } else if (policer_kpkts_rate != 0) { + policer = netdev_dpdk_kpkts_policer_construct(policer_kpkts_rate, + policer_kpkts_burst); } else { policer = NULL; } + ovsrcu_set(&dev->ingress_policer, policer); dev->policer_rate = policer_rate; dev->policer_burst = policer_burst; + dev->policer_kpkts_rate = policer_kpkts_rate; + dev->policer_kpkts_burst = policer_kpkts_burst; ovs_mutex_unlock(&dev->mutex); return 0; diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index 8b80a31e6..3680d4d40 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -426,6 +426,343 @@ AT_CLEANUP dnl -------------------------------------------------------------------------- +dnl -------------------------------------------------------------------------- +dnl Ingress policing (kpkts) create delete vport port +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) create delete vport port]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add ingress policer +AT_CHECK([ovs-appctl vlog/set netdev_dpdk:dbg], [], [stdout]) +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 ingress_policing_kpkts_rate=10000 ingress_policing_kpkts_burst=10000]) +AT_CHECK([ovs-vsctl show], [], [stdout]) +sleep 2 + +dnl Remove ingress policer +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 ingress_policing_kpkts_rate=0 ingress_policing_kpkts_burst=0]) + +dnl Fail if ingress policer could not be created +AT_FAIL_IF([grep "Could not create token bucket for ingress policer" ovs-vswitchd.log], [], [stdout]) + +dnl Check ingress policer was removed correctly +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 0' stdout], [], [stdout]) + +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 0' stdout], [], [stdout]) + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@Could not create rte meter for ingress policer@d +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No such file or directory@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl Ingress policing (kpkts) no policing rate +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) no policing rate]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add ingress policer +AT_CHECK([ovs-appctl vlog/set netdev_dpdk:dbg], [], [stdout]) +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 ingress_policing_kpkts_burst=1000]) +AT_CHECK([ovs-vsctl show], [], [stdout]) +sleep 2 + +dnl check ingress policer not be created +AT_FAIL_IF([grep "Could not create token bucket for ingress policer" ovs-vswitchd.log], [], [stdout]) + +dnl Check ingress policer was created correctly +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 1000' stdout], [], [stdout]) + +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 0' stdout], [], [stdout]) + + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No such file or directory@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl Ingress policing (kpkts) no policing burst +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) no policing burst]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add ingress policer +AT_CHECK([ovs-appctl vlog/set netdev_dpdk:dbg], [], [stdout]) +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 ingress_policing_kpkts_rate=10000]) +AT_CHECK([ovs-vsctl show], [], [stdout]) +sleep 2 + +dnl check ingress policer not be created +AT_FAIL_IF([grep "Could not create token bucket for ingress policer" ovs-vswitchd.log], [], [stdout]) + +dnl Check ingress policer was created correctly +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 0' stdout], [], [stdout]) + +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 10000' stdout], [], [stdout]) + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@Could not create rte meter for ingress policer@d +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No such file or directory@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl Ingress policing (kpkts) max policing rate +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) max policing rate]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add ingress policer +AT_CHECK([ovs-appctl vlog/set netdev_dpdk:dbg], [], [stdout]) +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 ingress_policing_kpkts_rate=42949671]) +AT_CHECK([ovs-vsctl show], [], [stdout]) +sleep 2 + +dnl Check ingress policer was created correctly +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 0' stdout], [], [stdout]) + +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 42949671' stdout], [], [stdout]) + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No such file or directory@d +\@Could not create token bucket for ingress policer@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl Ingress policing (kpkts) max policing burst +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) max policing burst]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add ingress policer +AT_CHECK([ovs-appctl vlog/set netdev_dpdk:dbg], [], [stdout]) +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 ingress_policing_kpkts_burst=42949671]) +AT_CHECK([ovs-vsctl show], [], [stdout]) +sleep 2 + +dnl check ingress policer not be created +AT_FAIL_IF([grep "Could not create token bucket for ingress policer" ovs-vswitchd.log], [], [stdout]) + +dnl Check ingress policer was created correctly +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 42949671' stdout], [], [stdout]) + +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 0' stdout], [], [stdout]) + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@Could not create token bucket for ingress policer@d +\@Could not create rte meter for ingress policer@d +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No such file or directory@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl Ingress policing (kpkts) set both bps and pps policing configurations +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) both policing configurations]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +OVS_DPDK_START() + +dnl Add userspace bridge and attach it to OVS and add ingress policer +AT_CHECK([ovs-appctl vlog/set netdev_dpdk:dbg], [], [stdout]) +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuserclient0 -- set Interface dpdkvhostuserclient0 type=dpdkvhostuserclient options:vhost-server-path=$OVS_RUNDIR/dpdkvhostclient0], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 ingress_policing_rate=1234 ingress_policing_burst=1234 ingress_policing_kpkts_rate=1234 ingress_policing_kpkts_burst=1234]) +AT_CHECK([ovs-vsctl show], [], [stdout]) +sleep 2 + +dnl check ingress policer not be created +AT_CHECK([grep "packet-per-second and byte-per-second" ovs-vswitchd.log | sed 's/^.*|WARN|//'], [0], [stdout]) + +dnl Check ingress policer was created correctly +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_burst: 1234' stdout], [], [stdout]) + +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_kpkts_rate: 1234' stdout], [], [stdout]) + +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_rate: 1234' stdout], [], [stdout]) + +AT_CHECK([ovs-vsctl list interface dpdkvhostuserclient0], [], [stdout]) +AT_CHECK([grep -E 'ingress_policing_burst: 1234' stdout], [], [stdout]) + +dnl check ingress policer not be created +AT_CHECK([ovs-vsctl set interface dpdkvhostuserclient0 ingress_policing_rate=0 ingress_policing_burst=1234 ingress_policing_kpkts_rate=1234 ingress_policing_kpkts_burst=1234]) +AT_FAIL_IF([grep "Could not create token bucket for ingress policer" ovs-vswitchd.log], [], [stdout]) + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) vhost-user client: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "vHost User device 'dpdkvhostuserclient0' created in 'client' mode, using client socket" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) reconnecting..." ovs-vswitchd.log], [], [stdout]) + +dnl Clean up +AT_CHECK([ovs-vsctl del-port br10 dpdkvhostuserclient0], [], [stdout], [stderr]) +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@Could not create token bucket for ingress policer@d +\@VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostclient0) failed to connect: No such file or directory@d +\@packet-per-second and byte-per-second@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + +dnl -------------------------------------------------------------------------- +dnl Ingress policing (kpkts) police +AT_SETUP([OVS-DPDK - Ingress policing (kpkts) police]) +AT_KEYWORDS([dpdk]) + +OVS_DPDK_PRE_CHECK() +AT_SKIP_IF([! which dpdk-testpmd >/dev/null 2>/dev/null]) +OVS_DPDK_START([--no-pci]) + +dnl Find number of sockets +AT_CHECK([lscpu], [], [stdout]) +AT_CHECK([cat stdout | grep "NUMA node(s)" | awk '{c=1; while (c++<$(3)) {printf "512,"}; print "512"}' > NUMA_NODE]) + +dnl Add userspace bridge and attach it to OVS +AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) + +dnl Parse log file +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuser0 -- set Interface dpdkvhostuser0 type=dpdkvhostuser], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl add-port br10 dpdkvhostuser1 -- set Interface dpdkvhostuser1 type=dpdkvhostuser], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl show], [], [stdout]) + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostuser0) vhost-user server: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "Socket $OVS_RUNDIR/dpdkvhostuser0 created for vhost-user port dpdkvhostuser0" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostuser0) binding succeeded" ovs-vswitchd.log], [], [stdout]) + +dnl Parse log file +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostuser1) vhost-user server: socket created" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "Socket $OVS_RUNDIR/dpdkvhostuser1 created for vhost-user port dpdkvhostuser1" ovs-vswitchd.log], [], [stdout]) +AT_CHECK([grep "VHOST_CONFIG: ($OVS_RUNDIR/dpdkvhostuser1) binding succeeded" ovs-vswitchd.log], [], [stdout]) + +dnl Configure the police for interface. +AT_CHECK([ovs-vsctl set interface dpdkvhostuser1 ingress_policing_kpkts_rate=1 ingress_policing_kpkts_burst=1]) + +dnl add flows, only send packets from dpdkvhostuser1 to dpdkvhostuser0. +AT_DATA([flows.txt], [dnl +priority=100,in_port=dpdkvhostuser1,ip,actions=dpdkvhostuser0 +]) + +AT_CHECK([ovs-ofctl del-flows br10]) +AT_CHECK([ovs-ofctl add-flows br10 flows.txt]) + +dnl Execute testpmd in background +on_exit "pkill -f -x -9 'tail -f /dev/null'" +tail -f /dev/null | dpdk-testpmd --socket-mem="$(cat NUMA_NODE)" --no-pci\ + --vdev="net_virtio_user0,path=$OVS_RUNDIR/dpdkvhostuser0" \ + --vdev="net_virtio_user1,path=$OVS_RUNDIR/dpdkvhostuser1" \ + --single-file-segments -- --forward-mode=flowgen -a > $OVS_RUNDIR/testpmd-dpdkvhostuser.log 2>&1 & + +dnl sent packet 10 second. +AT_CHECK([sleep 10]) + +dnl Clean up the testpmd now +pkill -f -x -9 'tail -f /dev/null' + +dnl ---------------------- Forward statistics for port 0 ---------------------- +dnl RX-packets: 9911 RX-dropped: 0 RX-total: 9911 +dnl TX-packets: 15937632 TX-dropped: 226661984 TX-total: 242599616 +dnl ---------------------------------------------------------------------------- +port0_rx_packets=`cat testpmd-dpdkvhostuser.log | grep "Forward statistics for port 0" -A 1 | grep "RX-packets:" | awk '{print $2}'` +echo "port0_rx_packets=$port0_rx_packets" + +AT_CHECK([test $port0_rx_packets -lt 10500]) +AT_CHECK([test $port0_rx_packets -gt 9500]) + +OVS_VSWITCHD_STOP("m4_join([], [SYSTEM_DPDK_ALLOWED_LOGS], [ +\@dpdkvhostuser ports are considered deprecated; please migrate to dpdkvhostuserclient ports.@d +])") +AT_CLEANUP +dnl -------------------------------------------------------------------------- + + + dnl -------------------------------------------------------------------------- dnl QoS create delete phy port AT_SETUP([OVS-DPDK - QoS create delete phy port])