diff mbox

[ovs-dev,v2] netdev-dpdk: Add Flow Control support.

Message ID 1469719812-14572-1-git-send-email-sugesh.chandran@intel.com
State Accepted
Delegated to: Daniele Di Proietto
Headers show

Commit Message

Chandran, Sugesh July 28, 2016, 3:30 p.m. UTC 
Add support for flow-control(mac control frame) to DPDK enabled physical
port types. By default, the flow-control is OFF on both rx and tx side.
The flow control can be enabled/disabled either when adding a port to OVS
or at run time.

For eg:
To enable flow control support at tx side while adding a port, add the
'tx-flow-ctrl' option to the 'ovs-vsctl add-port' command-line as below.

 'ovs-vsctl add-port br0 dpdk0 -- \
  set Interface dpdk0 type=dpdk options:tx-flow-ctrl=true'

Similarly to enable rx flow control,
 'ovs-vsctl add-port br0 dpdk0 -- \
  set Interface dpdk0 type=dpdk options:rx-flow-ctrl=true'

And to enable the flow control auto-negotiation,
 'ovs-vsctl add-port br0 dpdk0 -- \
  set Interface dpdk0 type=dpdk options:flow-ctrl-autoneg=true'

To turn ON the tx flow control at run time(After the port is being added
to OVS), the command-line input will be,
 'ovs-vsctl set Interface dpdk0 options:tx-flow-ctrl=true'

The flow control parameters can be turned off by setting 'false' to the
respective parameter. To dsiable the flow control at tx side,
 'ovs-vsctl set Interface dpdk0 options:tx-flow-ctrl=false'

Signed-off-by: Sugesh Chandran <sugesh.chandran@intel.com>
---
 INSTALL.DPDK-ADVANCED.md | 39 +++++++++++++++++++++++++++++++++++++--
 lib/netdev-dpdk.c        | 36 ++++++++++++++++++++++++++++++++++++
 vswitchd/vswitch.xml     | 23 +++++++++++++++++++++++
 3 files changed, 96 insertions(+), 2 deletions(-)
 mode change 100644 => 100755 INSTALL.DPDK-ADVANCED.md

Comments

Bodireddy, Bhanuprakash July 28, 2016, 3:53 p.m. UTC | #1 
>-----Original Message-----
>From: Chandran, Sugesh
>Sent: Thursday, July 28, 2016 4:30 PM
>To: diproiettod@ovn.org; Bodireddy, Bhanuprakash
><bhanuprakash.bodireddy@intel.com>; dev@openvswitch.org
>Cc: Chandran, Sugesh <sugesh.chandran@intel.com>
>Subject: [PATCH v2] netdev-dpdk: Add Flow Control support.
>
>Add support for flow-control(mac control frame) to DPDK enabled physical
>port types. By default, the flow-control is OFF on both rx and tx side.
>The flow control can be enabled/disabled either when adding a port to OVS or
>at run time.
>
>For eg:
>To enable flow control support at tx side while adding a port, add the 'tx-flow-
>ctrl' option to the 'ovs-vsctl add-port' command-line as below.
>
> 'ovs-vsctl add-port br0 dpdk0 -- \
>  set Interface dpdk0 type=dpdk options:tx-flow-ctrl=true'
>
>Similarly to enable rx flow control,
> 'ovs-vsctl add-port br0 dpdk0 -- \
>  set Interface dpdk0 type=dpdk options:rx-flow-ctrl=true'
>
>And to enable the flow control auto-negotiation,  'ovs-vsctl add-port br0
>dpdk0 -- \
>  set Interface dpdk0 type=dpdk options:flow-ctrl-autoneg=true'
>
>To turn ON the tx flow control at run time(After the port is being added to
>OVS), the command-line input will be,  'ovs-vsctl set Interface dpdk0
>options:tx-flow-ctrl=true'
>
>The flow control parameters can be turned off by setting 'false' to the
>respective parameter. To dsiable the flow control at tx side,  'ovs-vsctl set
>Interface dpdk0 options:tx-flow-ctrl=false'
>
>Signed-off-by: Sugesh Chandran <sugesh.chandran@intel.com>

LGTM, I tested it and can apply the rx flow control setting even when the interface is transmitting.
Acked-by: Bhanuprakash Bodireddy <Bhanuprakash.bodireddy@intel.com>

Regards,
Bhanu Prakash.
Chandran, Sugesh July 28, 2016, 8:36 p.m. UTC | #2 
Thank you Bhanu for reviewing and testing it.


Regards
_Sugesh


> -----Original Message-----
> From: Bodireddy, Bhanuprakash
> Sent: Thursday, July 28, 2016 4:54 PM
> To: Chandran, Sugesh <sugesh.chandran@intel.com>; diproiettod@ovn.org;
> dev@openvswitch.org
> Subject: RE: [PATCH v2] netdev-dpdk: Add Flow Control support.
> 
> >-----Original Message-----
> >From: Chandran, Sugesh
> >Sent: Thursday, July 28, 2016 4:30 PM
> >To: diproiettod@ovn.org; Bodireddy, Bhanuprakash
> ><bhanuprakash.bodireddy@intel.com>; dev@openvswitch.org
> >Cc: Chandran, Sugesh <sugesh.chandran@intel.com>
> >Subject: [PATCH v2] netdev-dpdk: Add Flow Control support.
> >
> >Add support for flow-control(mac control frame) to DPDK enabled physical
> >port types. By default, the flow-control is OFF on both rx and tx side.
> >The flow control can be enabled/disabled either when adding a port to OVS
> or
> >at run time.
> >
> >For eg:
> >To enable flow control support at tx side while adding a port, add the 'tx-
> flow-
> >ctrl' option to the 'ovs-vsctl add-port' command-line as below.
> >
> > 'ovs-vsctl add-port br0 dpdk0 -- \
> >  set Interface dpdk0 type=dpdk options:tx-flow-ctrl=true'
> >
> >Similarly to enable rx flow control,
> > 'ovs-vsctl add-port br0 dpdk0 -- \
> >  set Interface dpdk0 type=dpdk options:rx-flow-ctrl=true'
> >
> >And to enable the flow control auto-negotiation,  'ovs-vsctl add-port br0
> >dpdk0 -- \
> >  set Interface dpdk0 type=dpdk options:flow-ctrl-autoneg=true'
> >
> >To turn ON the tx flow control at run time(After the port is being added to
> >OVS), the command-line input will be,  'ovs-vsctl set Interface dpdk0
> >options:tx-flow-ctrl=true'
> >
> >The flow control parameters can be turned off by setting 'false' to the
> >respective parameter. To dsiable the flow control at tx side,  'ovs-vsctl set
> >Interface dpdk0 options:tx-flow-ctrl=false'
> >
> >Signed-off-by: Sugesh Chandran <sugesh.chandran@intel.com>
> 
> LGTM, I tested it and can apply the rx flow control setting even when the
> interface is transmitting.
> Acked-by: Bhanuprakash Bodireddy <Bhanuprakash.bodireddy@intel.com>
> 
> Regards,
> Bhanu Prakash.
Daniele Di Proietto July 30, 2016, 1:01 a.m. UTC | #3 
I changed the fc_conf initialization to use memset, because clang was
complaining about the initializer.

I moved the description in vswitch.xml to avoid nesting inside the "Common
Columns" group.

I changed slightly the wording in vswitch.xml, in case we want to implement
this even for non DPDK devices.

I added your name to AUTHORS

and pushed this to master

Thanks!

Daniele


2016-07-28 13:36 GMT-07:00 Chandran, Sugesh <sugesh.chandran@intel.com>:

> Thank you Bhanu for reviewing and testing it.
>
>
> Regards
> _Sugesh
>
>
> > -----Original Message-----
> > From: Bodireddy, Bhanuprakash
> > Sent: Thursday, July 28, 2016 4:54 PM
> > To: Chandran, Sugesh <sugesh.chandran@intel.com>; diproiettod@ovn.org;
> > dev@openvswitch.org
> > Subject: RE: [PATCH v2] netdev-dpdk: Add Flow Control support.
> >
> > >-----Original Message-----
> > >From: Chandran, Sugesh
> > >Sent: Thursday, July 28, 2016 4:30 PM
> > >To: diproiettod@ovn.org; Bodireddy, Bhanuprakash
> > ><bhanuprakash.bodireddy@intel.com>; dev@openvswitch.org
> > >Cc: Chandran, Sugesh <sugesh.chandran@intel.com>
> > >Subject: [PATCH v2] netdev-dpdk: Add Flow Control support.
> > >
> > >Add support for flow-control(mac control frame) to DPDK enabled physical
> > >port types. By default, the flow-control is OFF on both rx and tx side.
> > >The flow control can be enabled/disabled either when adding a port to
> OVS
> > or
> > >at run time.
> > >
> > >For eg:
> > >To enable flow control support at tx side while adding a port, add the
> 'tx-
> > flow-
> > >ctrl' option to the 'ovs-vsctl add-port' command-line as below.
> > >
> > > 'ovs-vsctl add-port br0 dpdk0 -- \
> > >  set Interface dpdk0 type=dpdk options:tx-flow-ctrl=true'
> > >
> > >Similarly to enable rx flow control,
> > > 'ovs-vsctl add-port br0 dpdk0 -- \
> > >  set Interface dpdk0 type=dpdk options:rx-flow-ctrl=true'
> > >
> > >And to enable the flow control auto-negotiation,  'ovs-vsctl add-port
> br0
> > >dpdk0 -- \
> > >  set Interface dpdk0 type=dpdk options:flow-ctrl-autoneg=true'
> > >
> > >To turn ON the tx flow control at run time(After the port is being
> added to
> > >OVS), the command-line input will be,  'ovs-vsctl set Interface dpdk0
> > >options:tx-flow-ctrl=true'
> > >
> > >The flow control parameters can be turned off by setting 'false' to the
> > >respective parameter. To dsiable the flow control at tx side,
> 'ovs-vsctl set
> > >Interface dpdk0 options:tx-flow-ctrl=false'
> > >
> > >Signed-off-by: Sugesh Chandran <sugesh.chandran@intel.com>
> >
> > LGTM, I tested it and can apply the rx flow control setting even when the
> > interface is transmitting.
> > Acked-by: Bhanuprakash Bodireddy <Bhanuprakash.bodireddy@intel.com>
> >
> > Regards,
> > Bhanu Prakash.
>
diff mbox

Patch

diff --git a/INSTALL.DPDK-ADVANCED.md b/INSTALL.DPDK-ADVANCED.md
old mode 100644
new mode 100755
index 9ae536d..191e69e
--- a/INSTALL.DPDK-ADVANCED.md
+++ b/INSTALL.DPDK-ADVANCED.md
@@ -11,7 +11,8 @@  OVS DPDK ADVANCED INSTALL GUIDE
 6. [Vhost Walkthrough](#vhost)
 7. [QOS](#qos)
 8. [Rate Limiting](#rl)
-9. [Vsperf](#vsperf)
+9. [Flow Control](#fc)
+10. [Vsperf](#vsperf)
 
 ## <a name="overview"></a> 1. Overview
 
@@ -827,7 +828,41 @@  To clear the ingress policer configuration from the port use the following:
 
 For more details regarding ingress-policer see the vswitch.xml.
 
-## <a name="vsperf"></a> 9. Vsperf
+## <a name="fc"></a> 9. Flow control.
+Flow control can be enabled only on DPDK physical ports.
+To enable flow control support at tx side while adding a port, add the
+'tx-flow-ctrl' option to the 'ovs-vsctl add-port' as in the eg: below.
+
+```
+ovs-vsctl add-port br0 dpdk0 -- \
+set Interface dpdk0 type=dpdk options:tx-flow-ctrl=true
+```
+
+Similarly to enable rx flow control,
+
+```
+ovs-vsctl add-port br0 dpdk0 -- \
+set Interface dpdk0 type=dpdk options:rx-flow-ctrl=true
+```
+
+And to enable the flow control auto-negotiation,
+
+```
+ovs-vsctl add-port br0 dpdk0 -- \
+set Interface dpdk0 type=dpdk options:flow-ctrl-autoneg=true
+```
+
+To turn ON the tx flow control at run time(After the port is being added
+to OVS), the command-line input will be,
+
+`ovs-vsctl set Interface dpdk0 options:tx-flow-ctrl=true`
+
+The flow control parameters can be turned off by setting 'false' to the
+respective parameter. To disable the flow control at tx side,
+
+`ovs-vsctl set Interface dpdk0 options:tx-flow-ctrl=false`
+
+## <a name="vsperf"></a> 10. Vsperf
 
 Vsperf project goal is to develop vSwitch test framework that can be used to
 validate the suitability of different vSwitch implementations in a Telco deployment
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
index c208f32..dd1f7b0 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -375,6 +375,9 @@  struct netdev_dpdk {
     OVSRCU_TYPE(struct ingress_policer *) ingress_policer;
     uint32_t policer_rate;
     uint32_t policer_burst;
+
+    /* DPDK-ETH Flow control */
+    struct rte_eth_fc_conf fc_conf;
 };
 
 struct netdev_rxq_dpdk {
@@ -628,6 +631,13 @@  dpdk_eth_dev_queue_setup(struct netdev_dpdk *dev, int n_rxq, int n_txq)
     return diag;
 }
 
+static void
+dpdk_eth_flow_ctrl_setup(struct netdev_dpdk *dev) OVS_REQUIRES(dev->mutex)
+{
+    if (rte_eth_dev_flow_ctrl_set(dev->port_id, &dev->fc_conf)) {
+        VLOG_WARN("Failed to enable flow control on device %d", dev->port_id);
+    }
+}
 
 static int
 dpdk_eth_dev_init(struct netdev_dpdk *dev) OVS_REQUIRES(dpdk_mutex)
@@ -676,6 +686,14 @@  dpdk_eth_dev_init(struct netdev_dpdk *dev) OVS_REQUIRES(dpdk_mutex)
     dev->buf_size = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM;
 
     dev->flags = NETDEV_UP | NETDEV_PROMISC;
+
+    /* Get the Flow control configuration for DPDK-ETH */
+    diag  = rte_eth_dev_flow_ctrl_get(dev->port_id, &dev->fc_conf);
+    if (diag) {
+        VLOG_DBG("cannot get flow control parameters on port=%d, err=%d",
+                 dev->port_id, diag);
+    }
+
     return 0;
 }
 
@@ -765,6 +783,8 @@  netdev_dpdk_init(struct netdev *netdev, unsigned int port_no,
     dev->requested_n_rxq = netdev->n_rxq;
     dev->requested_n_txq = netdev->n_txq;
 
+    /* Initialize the flow control to NULL */
+    dev->fc_conf = (struct rte_eth_fc_conf){ 0 };
     if (type == DPDK_DEV_ETH) {
         err = dpdk_eth_dev_init(dev);
         if (err) {
@@ -983,6 +1003,22 @@  netdev_dpdk_set_config(struct netdev *netdev, const struct smap *args)
         dev->requested_n_rxq = new_n_rxq;
         netdev_request_reconfigure(netdev);
     }
+
+    /* Flow control configuration for DPDK Ethernet ports. */
+    if (dev->type == DPDK_DEV_ETH) {
+        bool rx_fc_en = false;
+        bool tx_fc_en = false;
+        enum rte_eth_fc_mode fc_mode_set[2][2] =
+                                           {{RTE_FC_NONE, RTE_FC_TX_PAUSE},
+                                            {RTE_FC_RX_PAUSE, RTE_FC_FULL}
+                                           };
+        rx_fc_en = smap_get_bool(args, "rx-flow-ctrl", false);
+        tx_fc_en = smap_get_bool(args, "tx-flow-ctrl", false);
+        dev->fc_conf.autoneg = smap_get_bool(args, "flow-ctrl-autoneg", false);
+        dev->fc_conf.mode = fc_mode_set[tx_fc_en][rx_fc_en];
+
+        dpdk_eth_flow_ctrl_setup(dev);
+    }
     ovs_mutex_unlock(&dev->mutex);
 
     return 0;
diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml
index f0e1381..5cb51ce 100644
--- a/vswitchd/vswitch.xml
+++ b/vswitchd/vswitch.xml
@@ -3150,6 +3150,29 @@ 
       The overall purpose of these columns is described under <code>Common
       Columns</code> at the beginning of this document.
 
+    <group title="DPDK Flow control Configuration">
+      <p>
+        Ethernet flow control defined in IEEE 802.1Qbb provides link level flow
+        control using MAC pause frames.
+      </p>
+
+      <column name="options" key="rx-flow-ctrl" type='{"type": "boolean"}'>
+        Set to <code>true</code> to enable Rx flow control on DPDK
+        physical ports. By default, Rx flow control is disabled.
+      </column>
+
+      <column name="options" key="tx-flow-ctrl" type='{"type": "boolean"}'>
+        Set to <code>true</code> to enable Tx flow control on DPDK physical
+        ports. By default, Tx flow control is disabled.
+      </column>
+
+      <column name="options" key="flow-ctrl-autoneg"
+              type='{"type": "boolean"}'>
+        Set to <code>true</code> to enable flow control auto negotiation on
+        DPDK physical ports. By default, auto-neg is disabled.
+      </column>
+    </group>
+
       <column name="other_config"/>
       <column name="external_ids"/>
     </group>