| Message ID | 20180212181306.6674-4-stephen@that.guru |
|---|---|
| State | Changes Requested |
| Delegated to: | Ian Stokes |
| Headers | show |
| Series | Split up the DPDK howto | expand |
> The "vdev", "hotplugging", and "Rx checksum offload" sections only apply > to 'dpdk' ports and are too detailed to include in a high-level howto. Should flow control be in here too? AFAIK it's phy port only. > Move them, reworking some aspects of this in the process. > It may not be obvious to users that these are relevant to phy only and as such are found under Documentation/topics/dpdk/phy.rst. We should be making it clear where these topics can be found to user at a higher level. Have you considered a high level documentation map, possibly in Documentation/howto/dpdk.rst showing where feature docs can be found? More comments below. > Signed-off-by: Stephen Finucane <stephen@that.guru> > --- > Documentation/howto/dpdk.rst | 93 +++------------------------------- > ----- > Documentation/topics/dpdk/phy.rst | 91 > ++++++++++++++++++++++++++++++++++++++ > 2 files changed, 97 insertions(+), 87 deletions(-) > > diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst > index c2324118d..4d993a0eb 100644 > --- a/Documentation/howto/dpdk.rst > +++ b/Documentation/howto/dpdk.rst > @@ -57,8 +57,12 @@ usage is suggested:: > $ ovs-vsctl add-port br0 dpdk-p1 -- set Interface dpdk-p1 type=dpdk \ > options:dpdk-devargs="class=eth,mac=00:11:22:33:44:55:02" > > -Note: such syntax won't support hotplug. The hotplug is supposed to work > with -future DPDK release, v18.05. > +.. important:: > + > + Hotplugging physical interfaces is not supported using the above > syntax. > + This is expected to change with the release of DPDK v18.05. For > information > + on hotplugging physical interfaces, you should instead refer to > + :ref:`port-hotplug`. > > After the DPDK ports get added to switch, a polling thread continuously > polls DPDK devices and consumes 100% of the core, as can be checked from > ``top`` and @@ -236,16 +240,6 @@ largest frame size supported by Fortville > NIC using the DPDK i40e driver, but larger frames and other DPDK NIC > drivers may be supported. These cases are common for use cases involving > East-West traffic only. > > -Rx Checksum Offload > -------------------- > - > -By default, DPDK physical ports are enabled with Rx checksum offload. > - > -Rx checksum offload can offer performance improvement only for tunneling > -traffic in OVS-DPDK because the checksum validation of tunnel packets is > -offloaded to the NIC. Also enabling Rx checksum may slightly reduce the - > performance of non-tunnel traffic, specifically for smaller size packet. > - > .. _extended-statistics: > > Extended & Custom Statistics > @@ -278,81 +272,6 @@ Note about "Extended Statistics": vHost ports > supports only partial statistics. RX packet size based counter are only > supported and doesn't include TX packet size counters. > > -.. _port-hotplug: > - > -Port Hotplug > ------------- > - > -OVS supports port hotplugging, allowing the use of ports that were not > bound -to DPDK when vswitchd was started. > -In order to attach a port, it has to be bound to DPDK using the - > ``dpdk_nic_bind.py`` script:: > - > - $ $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio 0000:01:00.0 > - > -Then it can be attached to OVS:: > - > - $ ovs-vsctl add-port br0 dpdkx -- set Interface dpdkx type=dpdk \ > - options:dpdk-devargs=0000:01:00.0 > - > -Detaching will be performed while processing del-port command:: > - > - $ ovs-vsctl del-port dpdkx > - > -Sometimes, the del-port command may not detach the device. > -Detaching can be confirmed by the appearance of an INFO log. > -For example:: > - > - INFO|Device '0000:04:00.1' has been detached > - > -If the log is not seen, then the port can be detached using:: > - > -$ ovs-appctl netdev-dpdk/detach 0000:01:00.0 > - > -Detaching can be confirmed by console output:: > - > - Device '0000:04:00.1' has been detached > - > -.. warning:: > - Detaching should not be done if a device is known to be non- > detachable, as > - this may cause the device to behave improperly when added back with > - add-port. The Chelsio Terminator adapters which use the cxgbe driver > seem > - to be an example of this behavior; check the driver documentation if > this > - is suspected. > - > -This feature does not work with some NICs. > -For more information please refer to the `DPDK Port Hotplug Framework - > <http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html#hotplug > >`__. > - > -.. _vdev-support: > - > -Vdev Support > ------------- > - > -DPDK provides drivers for both physical and virtual devices. Physical > DPDK -devices are added to OVS by specifying a valid PCI address in 'dpdk- > devargs'. > -Virtual DPDK devices which do not have PCI addresses can be added using a > -different format for 'dpdk-devargs'. > - > -Typically, the format expected is 'eth_<driver_name><x>' where 'x' is a - > unique identifier of your choice for the given port. > - > -For example to add a dpdk port that uses the 'null' DPDK PMD driver:: > - > - $ ovs-vsctl add-port br0 null0 -- set Interface null0 type=dpdk \ > - options:dpdk-devargs=eth_null0 > - > -Similarly, to add a dpdk port that uses the 'af_packet' DPDK PMD driver:: > - > - $ ovs-vsctl add-port br0 myeth0 -- set Interface myeth0 type=dpdk > \ > - options:dpdk-devargs=eth_af_packet0,iface=eth0 > - > -More information on the different types of virtual DPDK PMDs can be found > in -the `DPDK documentation - > <http://dpdk.org/doc/guides/nics/overview.html>`__. > - > -Note: Not all DPDK virtual PMD drivers have been tested and verified to > work. > - > EMC Insertion Probability > ------------------------- > By default 1 in every 100 flows are inserted into the Exact Match Cache > (EMC). > diff --git a/Documentation/topics/dpdk/phy.rst > b/Documentation/topics/dpdk/phy.rst > index 222fa3e9f..507dac869 100644 > --- a/Documentation/topics/dpdk/phy.rst > +++ b/Documentation/topics/dpdk/phy.rst > @@ -119,3 +119,94 @@ utilize is a requirement in order to deliver the > high-performance possible with the DPDK datapath. It is possible to > configure multiple Rx queues for ``dpdk`` ports, thus ensuring this is > not a bottleneck for performance. For information on configuring PMD > threads, refer to :doc:`pmd`. > + > +Rx Checksum Offload > +------------------- > + > +By default, DPDK physical ports are enabled with Rx checksum offload. > + > +Rx checksum offload can offer performance improvement only for > +tunneling traffic in OVS-DPDK because the checksum validation of tunnel > +packets is offloaded to the NIC. Also enabling Rx checksum may slightly > +reduce the performance of non-tunnel traffic, specifically for smaller > size packet. > + > +.. _port-hotplug: > + > +Hotplugging > +----------- > + > +OVS supports port hotplugging, allowing the use of physical ports that > +were not bound to DPDK when ovs-vswitchd was started. > + > +.. warning:: > + > + This feature is not compatible with all NICs. Refer to vendor > documentation > + for more information. > + > +.. important:: > + > + Ports must be bound to DPDK. Refer to :ref:`dpdk-binding-nics` for > more > + information. > + > +To *hotplug* a port, simply add it like any other port:: > + > + $ ovs-vsctl add-port br0 dpdkx -- set Interface dpdkx type=dpdk \ > + options:dpdk-devargs=0000:01:00.0 > + > +Ports can be detached using the ``del-port`` command:: > + > + $ ovs-vsctl del-port dpdkx > + > +This should both delete the port and detach the device. If successful, > +you should see an ``INFO`` log. For example:: > + > + INFO|Device '0000:04:00.1' has been detached > + > +If the log is not seen then the port can be detached like so:: > + > + $ ovs-appctl netdev-dpdk/detach 0000:01:00.0 > + > +.. warning:: > + > + Detaching should not be done if a device is known to be non- > detachable, as > + this may cause the device to behave improperly when added back with > + add-port. The Chelsio Terminator adapters which use the cxgbe driver > seem > + to be an example of this behavior; check the driver documentation if > this > + is suspected. > + > +For more information please refer to the `DPDK Port Hotplug Framework`__. > + > +__ > +http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html#hotpl > +ug > + > +.. _vdev-support: > + > +vdev Support > +------------ It seems strange to have vdevs part of the phy document. Consider moving them to their own document. I think this would make sense as different vdevs have different requirements and behaviors. For example if crypto vdevs are introduced they will have associated SW library requirements that have no relation to phy devices. Ian > + > +DPDK provides drivers for both physical and virtual devices. Physical > +DPDK devices are added to OVS by specifying a valid PCI address in > ``dpdk-devargs``. > +Virtual DPDK devices which do not have PCI addresses can be added using > +a different format for ``dpdk-devargs``. > + > +.. note:: > + > + Not all DPDK virtual PMD drivers have been tested and verified to > work. > + > +To add a virtual ``dpdk`` devices, the ``dpdk-devargs`` argument should > +be of the format ``eth_<driver_name><x>``, where ``x``' is a unique > +identifier of your choice for the given port. For example to add a > +``dpdk`` port that uses the ``null`` DPDK PMD driver, run:: > + > + $ ovs-vsctl add-port br0 null0 -- set Interface null0 type=dpdk \ > + options:dpdk-devargs=eth_null0 > + > +Similarly, to add a ``dpdk`` port that uses the ``af_packet`` DPDK PMD > +driver, > +run:: > + > + $ ovs-vsctl add-port br0 myeth0 -- set Interface myeth0 type=dpdk \ > + options:dpdk-devargs=eth_af_packet0,iface=eth0 > + > +More information on the different types of virtual DPDK PMDs can be > +found in the `DPDK documentation`__. > + > +__ http://dpdk.org/doc/guides/nics/overview.html > -- > 2.14.3 > > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
On Mon, 2018-04-09 at 15:16 +0000, Stokes, Ian wrote: > > The "vdev", "hotplugging", and "Rx checksum offload" sections only apply > > to 'dpdk' ports and are too detailed to include in a high-level howto. > > Should flow control be in here too? AFAIK it's phy port only. Indeed it should. Done. > > Move them, reworking some aspects of this in the process. > > It may not be obvious to users that these are relevant to phy only > and as such are found under Documentation/topics/dpdk/phy.rst. > > We should be making it clear where these topics can be found to user > at a higher level. > > Have you considered a high level documentation map, possibly in > Documentation/howto/dpdk.rst showing where feature docs can be found? I've added a further reading section to the howto guide which should address some of these issues. A more substantial follow up to come on the cover letter. > More comments below. > > > > Signed-off-by: Stephen Finucane <stephen@that.guru> > > --- > > Documentation/howto/dpdk.rst | 93 +++------------------------------- > > ----- > > Documentation/topics/dpdk/phy.rst | 91 > > ++++++++++++++++++++++++++++++++++++++ > > 2 files changed, 97 insertions(+), 87 deletions(-) > > > > diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst > > index c2324118d..4d993a0eb 100644 > > --- a/Documentation/howto/dpdk.rst > > +++ b/Documentation/howto/dpdk.rst > > @@ -57,8 +57,12 @@ usage is suggested:: > > $ ovs-vsctl add-port br0 dpdk-p1 -- set Interface dpdk-p1 type=dpdk \ > > options:dpdk-devargs="class=eth,mac=00:11:22:33:44:55:02" > > > > -Note: such syntax won't support hotplug. The hotplug is supposed to work > > with -future DPDK release, v18.05. > > +.. important:: > > + > > + Hotplugging physical interfaces is not supported using the above > > syntax. > > + This is expected to change with the release of DPDK v18.05. For > > information > > + on hotplugging physical interfaces, you should instead refer to > > + :ref:`port-hotplug`. > > > > After the DPDK ports get added to switch, a polling thread continuously > > polls DPDK devices and consumes 100% of the core, as can be checked from > > ``top`` and @@ -236,16 +240,6 @@ largest frame size supported by Fortville > > NIC using the DPDK i40e driver, but larger frames and other DPDK NIC > > drivers may be supported. These cases are common for use cases involving > > East-West traffic only. > > > > -Rx Checksum Offload > > -------------------- > > - > > -By default, DPDK physical ports are enabled with Rx checksum offload. > > - > > -Rx checksum offload can offer performance improvement only for tunneling > > -traffic in OVS-DPDK because the checksum validation of tunnel packets is > > -offloaded to the NIC. Also enabling Rx checksum may slightly reduce the - > > performance of non-tunnel traffic, specifically for smaller size packet. > > - > > .. _extended-statistics: > > > > Extended & Custom Statistics > > @@ -278,81 +272,6 @@ Note about "Extended Statistics": vHost ports > > supports only partial statistics. RX packet size based counter are only > > supported and doesn't include TX packet size counters. > > > > -.. _port-hotplug: > > - > > -Port Hotplug > > ------------- > > - > > -OVS supports port hotplugging, allowing the use of ports that were not > > bound -to DPDK when vswitchd was started. > > -In order to attach a port, it has to be bound to DPDK using the - > > ``dpdk_nic_bind.py`` script:: > > - > > - $ $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio 0000:01:00.0 > > - > > -Then it can be attached to OVS:: > > - > > - $ ovs-vsctl add-port br0 dpdkx -- set Interface dpdkx type=dpdk \ > > - options:dpdk-devargs=0000:01:00.0 > > - > > -Detaching will be performed while processing del-port command:: > > - > > - $ ovs-vsctl del-port dpdkx > > - > > -Sometimes, the del-port command may not detach the device. > > -Detaching can be confirmed by the appearance of an INFO log. > > -For example:: > > - > > - INFO|Device '0000:04:00.1' has been detached > > - > > -If the log is not seen, then the port can be detached using:: > > - > > -$ ovs-appctl netdev-dpdk/detach 0000:01:00.0 > > - > > -Detaching can be confirmed by console output:: > > - > > - Device '0000:04:00.1' has been detached > > - > > -.. warning:: > > - Detaching should not be done if a device is known to be non- > > detachable, as > > - this may cause the device to behave improperly when added back with > > - add-port. The Chelsio Terminator adapters which use the cxgbe driver > > seem > > - to be an example of this behavior; check the driver documentation if > > this > > - is suspected. > > - > > -This feature does not work with some NICs. > > -For more information please refer to the `DPDK Port Hotplug Framework - > > <http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html#hotplug > > > `__. > > > > - > > -.. _vdev-support: > > - > > -Vdev Support > > ------------- > > - > > -DPDK provides drivers for both physical and virtual devices. Physical > > DPDK -devices are added to OVS by specifying a valid PCI address in 'dpdk- > > devargs'. > > -Virtual DPDK devices which do not have PCI addresses can be added using a > > -different format for 'dpdk-devargs'. > > - > > -Typically, the format expected is 'eth_<driver_name><x>' where 'x' is a - > > unique identifier of your choice for the given port. > > - > > -For example to add a dpdk port that uses the 'null' DPDK PMD driver:: > > - > > - $ ovs-vsctl add-port br0 null0 -- set Interface null0 type=dpdk \ > > - options:dpdk-devargs=eth_null0 > > - > > -Similarly, to add a dpdk port that uses the 'af_packet' DPDK PMD driver:: > > - > > - $ ovs-vsctl add-port br0 myeth0 -- set Interface myeth0 type=dpdk > > \ > > - options:dpdk-devargs=eth_af_packet0,iface=eth0 > > - > > -More information on the different types of virtual DPDK PMDs can be found > > in -the `DPDK documentation - > > <http://dpdk.org/doc/guides/nics/overview.html>`__. > > - > > -Note: Not all DPDK virtual PMD drivers have been tested and verified to > > work. > > - > > EMC Insertion Probability > > ------------------------- > > By default 1 in every 100 flows are inserted into the Exact Match Cache > > (EMC). > > diff --git a/Documentation/topics/dpdk/phy.rst > > b/Documentation/topics/dpdk/phy.rst > > index 222fa3e9f..507dac869 100644 > > --- a/Documentation/topics/dpdk/phy.rst > > +++ b/Documentation/topics/dpdk/phy.rst > > @@ -119,3 +119,94 @@ utilize is a requirement in order to deliver the > > high-performance possible with the DPDK datapath. It is possible to > > configure multiple Rx queues for ``dpdk`` ports, thus ensuring this is > > not a bottleneck for performance. For information on configuring PMD > > threads, refer to :doc:`pmd`. > > + > > +Rx Checksum Offload > > +------------------- > > + > > +By default, DPDK physical ports are enabled with Rx checksum offload. > > + > > +Rx checksum offload can offer performance improvement only for > > +tunneling traffic in OVS-DPDK because the checksum validation of tunnel > > +packets is offloaded to the NIC. Also enabling Rx checksum may slightly > > +reduce the performance of non-tunnel traffic, specifically for smaller > > size packet. > > + > > +.. _port-hotplug: > > + > > +Hotplugging > > +----------- > > + > > +OVS supports port hotplugging, allowing the use of physical ports that > > +were not bound to DPDK when ovs-vswitchd was started. > > + > > +.. warning:: > > + > > + This feature is not compatible with all NICs. Refer to vendor > > documentation > > + for more information. > > + > > +.. important:: > > + > > + Ports must be bound to DPDK. Refer to :ref:`dpdk-binding-nics` for > > more > > + information. > > + > > +To *hotplug* a port, simply add it like any other port:: > > + > > + $ ovs-vsctl add-port br0 dpdkx -- set Interface dpdkx type=dpdk \ > > + options:dpdk-devargs=0000:01:00.0 > > + > > +Ports can be detached using the ``del-port`` command:: > > + > > + $ ovs-vsctl del-port dpdkx > > + > > +This should both delete the port and detach the device. If successful, > > +you should see an ``INFO`` log. For example:: > > + > > + INFO|Device '0000:04:00.1' has been detached > > + > > +If the log is not seen then the port can be detached like so:: > > + > > + $ ovs-appctl netdev-dpdk/detach 0000:01:00.0 > > + > > +.. warning:: > > + > > + Detaching should not be done if a device is known to be non- > > detachable, as > > + this may cause the device to behave improperly when added back with > > + add-port. The Chelsio Terminator adapters which use the cxgbe driver > > seem > > + to be an example of this behavior; check the driver documentation if > > this > > + is suspected. > > + > > +For more information please refer to the `DPDK Port Hotplug Framework`__. > > + > > +__ > > +http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html#hotpl > > +ug > > + > > +.. _vdev-support: > > + > > +vdev Support > > +------------ > > It seems strange to have vdevs part of the phy document. Consider > moving them to their own document. > > I think this would make sense as different vdevs have different > requirements and behaviors. For example if crypto vdevs are > introduced they will have associated SW library requirements that > have no relation to phy devices. Agreed. I've done this now (it's a separate patch). Stephen > Ian > > > + > > +DPDK provides drivers for both physical and virtual devices. Physical > > +DPDK devices are added to OVS by specifying a valid PCI address in > > ``dpdk-devargs``. > > +Virtual DPDK devices which do not have PCI addresses can be added using > > +a different format for ``dpdk-devargs``. > > + > > +.. note:: > > + > > + Not all DPDK virtual PMD drivers have been tested and verified to > > work. > > + > > +To add a virtual ``dpdk`` devices, the ``dpdk-devargs`` argument should > > +be of the format ``eth_<driver_name><x>``, where ``x``' is a unique > > +identifier of your choice for the given port. For example to add a > > +``dpdk`` port that uses the ``null`` DPDK PMD driver, run:: > > + > > + $ ovs-vsctl add-port br0 null0 -- set Interface null0 type=dpdk \ > > + options:dpdk-devargs=eth_null0 > > + > > +Similarly, to add a ``dpdk`` port that uses the ``af_packet`` DPDK PMD > > +driver, > > +run:: > > + > > + $ ovs-vsctl add-port br0 myeth0 -- set Interface myeth0 type=dpdk \ > > + options:dpdk-devargs=eth_af_packet0,iface=eth0 > > + > > +More information on the different types of virtual DPDK PMDs can be > > +found in the `DPDK documentation`__. > > + > > +__ http://dpdk.org/doc/guides/nics/overview.html > > -- > > 2.14.3 > > > > _______________________________________________ > > dev mailing list > > dev@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> On Mon, 2018-04-09 at 15:16 +0000, Stokes, Ian wrote: > > > The "vdev", "hotplugging", and "Rx checksum offload" sections only > > > apply to 'dpdk' ports and are too detailed to include in a high-level > howto. > > > > Should flow control be in here too? AFAIK it's phy port only. > > Indeed it should. Done. > > > > Move them, reworking some aspects of this in the process. > > > > It may not be obvious to users that these are relevant to phy only and > > as such are found under Documentation/topics/dpdk/phy.rst. > > > > We should be making it clear where these topics can be found to user > > at a higher level. > > > > Have you considered a high level documentation map, possibly in > > Documentation/howto/dpdk.rst showing where feature docs can be found? > > I've added a further reading section to the howto guide which should > address some of these issues. A more substantial follow up to come on the > cover letter. Ok, will review in the v2. > > > More comments below. > > > > > > > Signed-off-by: Stephen Finucane <stephen@that.guru> > > > --- > > > Documentation/howto/dpdk.rst | 93 +++--------------------------- > ---- > > > ----- > > > Documentation/topics/dpdk/phy.rst | 91 > > > ++++++++++++++++++++++++++++++++++++++ > > > 2 files changed, 97 insertions(+), 87 deletions(-) > > > > > > diff --git a/Documentation/howto/dpdk.rst > > > b/Documentation/howto/dpdk.rst index c2324118d..4d993a0eb 100644 > > > --- a/Documentation/howto/dpdk.rst > > > +++ b/Documentation/howto/dpdk.rst > > > @@ -57,8 +57,12 @@ usage is suggested:: > > > $ ovs-vsctl add-port br0 dpdk-p1 -- set Interface dpdk-p1 > type=dpdk \ > > > options:dpdk-devargs="class=eth,mac=00:11:22:33:44:55:02" > > > > > > -Note: such syntax won't support hotplug. The hotplug is supposed to > > > work with -future DPDK release, v18.05. > > > +.. important:: > > > + > > > + Hotplugging physical interfaces is not supported using the > > > + above > > > syntax. > > > + This is expected to change with the release of DPDK v18.05. For > > > information > > > + on hotplugging physical interfaces, you should instead refer to > > > + :ref:`port-hotplug`. > > > > > > After the DPDK ports get added to switch, a polling thread > > > continuously polls DPDK devices and consumes 100% of the core, as > > > can be checked from ``top`` and @@ -236,16 +240,6 @@ largest frame > > > size supported by Fortville NIC using the DPDK i40e driver, but > > > larger frames and other DPDK NIC drivers may be supported. These > > > cases are common for use cases involving East-West traffic only. > > > > > > -Rx Checksum Offload > > > -------------------- > > > - > > > -By default, DPDK physical ports are enabled with Rx checksum offload. > > > - > > > -Rx checksum offload can offer performance improvement only for > > > tunneling -traffic in OVS-DPDK because the checksum validation of > > > tunnel packets is -offloaded to the NIC. Also enabling Rx checksum > > > may slightly reduce the - performance of non-tunnel traffic, > specifically for smaller size packet. > > > - > > > .. _extended-statistics: > > > > > > Extended & Custom Statistics > > > @@ -278,81 +272,6 @@ Note about "Extended Statistics": vHost ports > > > supports only partial statistics. RX packet size based counter are > > > only supported and doesn't include TX packet size counters. > > > > > > -.. _port-hotplug: > > > - > > > -Port Hotplug > > > ------------- > > > - > > > -OVS supports port hotplugging, allowing the use of ports that were > > > not bound -to DPDK when vswitchd was started. > > > -In order to attach a port, it has to be bound to DPDK using the - > > > ``dpdk_nic_bind.py`` script:: > > > - > > > - $ $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio 0000:01:00.0 > > > - > > > -Then it can be attached to OVS:: > > > - > > > - $ ovs-vsctl add-port br0 dpdkx -- set Interface dpdkx type=dpdk \ > > > - options:dpdk-devargs=0000:01:00.0 > > > - > > > -Detaching will be performed while processing del-port command:: > > > - > > > - $ ovs-vsctl del-port dpdkx > > > - > > > -Sometimes, the del-port command may not detach the device. > > > -Detaching can be confirmed by the appearance of an INFO log. > > > -For example:: > > > - > > > - INFO|Device '0000:04:00.1' has been detached > > > - > > > -If the log is not seen, then the port can be detached using:: > > > - > > > -$ ovs-appctl netdev-dpdk/detach 0000:01:00.0 > > > - > > > -Detaching can be confirmed by console output:: > > > - > > > - Device '0000:04:00.1' has been detached > > > - > > > -.. warning:: > > > - Detaching should not be done if a device is known to be non- > > > detachable, as > > > - this may cause the device to behave improperly when added back > with > > > - add-port. The Chelsio Terminator adapters which use the cxgbe > driver > > > seem > > > - to be an example of this behavior; check the driver documentation > if > > > this > > > - is suspected. > > > - > > > -This feature does not work with some NICs. > > > -For more information please refer to the `DPDK Port Hotplug > > > Framework - > > > <http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html#h > > > otplug > > > > `__. > > > > > > - > > > -.. _vdev-support: > > > - > > > -Vdev Support > > > ------------- > > > - > > > -DPDK provides drivers for both physical and virtual devices. > > > Physical DPDK -devices are added to OVS by specifying a valid PCI > > > address in 'dpdk- devargs'. > > > -Virtual DPDK devices which do not have PCI addresses can be added > > > using a -different format for 'dpdk-devargs'. > > > - > > > -Typically, the format expected is 'eth_<driver_name><x>' where 'x' > > > is a - unique identifier of your choice for the given port. > > > - > > > -For example to add a dpdk port that uses the 'null' DPDK PMD driver:: > > > - > > > - $ ovs-vsctl add-port br0 null0 -- set Interface null0 > type=dpdk \ > > > - options:dpdk-devargs=eth_null0 > > > - > > > -Similarly, to add a dpdk port that uses the 'af_packet' DPDK PMD > driver:: > > > - > > > - $ ovs-vsctl add-port br0 myeth0 -- set Interface myeth0 > type=dpdk > > > \ > > > - options:dpdk-devargs=eth_af_packet0,iface=eth0 > > > - > > > -More information on the different types of virtual DPDK PMDs can be > > > found in -the `DPDK documentation - > > > <http://dpdk.org/doc/guides/nics/overview.html>`__. > > > - > > > -Note: Not all DPDK virtual PMD drivers have been tested and > > > verified to work. > > > - > > > EMC Insertion Probability > > > ------------------------- > > > By default 1 in every 100 flows are inserted into the Exact Match > > > Cache (EMC). > > > diff --git a/Documentation/topics/dpdk/phy.rst > > > b/Documentation/topics/dpdk/phy.rst > > > index 222fa3e9f..507dac869 100644 > > > --- a/Documentation/topics/dpdk/phy.rst > > > +++ b/Documentation/topics/dpdk/phy.rst > > > @@ -119,3 +119,94 @@ utilize is a requirement in order to deliver > > > the high-performance possible with the DPDK datapath. It is > > > possible to configure multiple Rx queues for ``dpdk`` ports, thus > > > ensuring this is not a bottleneck for performance. For information > > > on configuring PMD threads, refer to :doc:`pmd`. > > > + > > > +Rx Checksum Offload > > > +------------------- > > > + > > > +By default, DPDK physical ports are enabled with Rx checksum offload. > > > + > > > +Rx checksum offload can offer performance improvement only for > > > +tunneling traffic in OVS-DPDK because the checksum validation of > > > +tunnel packets is offloaded to the NIC. Also enabling Rx checksum > > > +may slightly reduce the performance of non-tunnel traffic, > > > +specifically for smaller > > > size packet. > > > + > > > +.. _port-hotplug: > > > + > > > +Hotplugging > > > +----------- > > > + > > > +OVS supports port hotplugging, allowing the use of physical ports > > > +that were not bound to DPDK when ovs-vswitchd was started. > > > + > > > +.. warning:: > > > + > > > + This feature is not compatible with all NICs. Refer to vendor > > > documentation > > > + for more information. > > > + > > > +.. important:: > > > + > > > + Ports must be bound to DPDK. Refer to :ref:`dpdk-binding-nics` > > > + for > > > more > > > + information. > > > + > > > +To *hotplug* a port, simply add it like any other port:: > > > + > > > + $ ovs-vsctl add-port br0 dpdkx -- set Interface dpdkx type=dpdk \ > > > + options:dpdk-devargs=0000:01:00.0 > > > + > > > +Ports can be detached using the ``del-port`` command:: > > > + > > > + $ ovs-vsctl del-port dpdkx > > > + > > > +This should both delete the port and detach the device. If > > > +successful, you should see an ``INFO`` log. For example:: > > > + > > > + INFO|Device '0000:04:00.1' has been detached > > > + > > > +If the log is not seen then the port can be detached like so:: > > > + > > > + $ ovs-appctl netdev-dpdk/detach 0000:01:00.0 > > > + > > > +.. warning:: > > > + > > > + Detaching should not be done if a device is known to be non- > > > detachable, as > > > + this may cause the device to behave improperly when added back > with > > > + add-port. The Chelsio Terminator adapters which use the cxgbe > > > + driver > > > seem > > > + to be an example of this behavior; check the driver > > > + documentation if > > > this > > > + is suspected. > > > + > > > +For more information please refer to the `DPDK Port Hotplug > Framework`__. > > > + > > > +__ > > > +http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html#h > > > +otpl > > > +ug > > > + > > > +.. _vdev-support: > > > + > > > +vdev Support > > > +------------ > > > > It seems strange to have vdevs part of the phy document. Consider > > moving them to their own document. > > > > I think this would make sense as different vdevs have different > > requirements and behaviors. For example if crypto vdevs are introduced > > they will have associated SW library requirements that have no > > relation to phy devices. > > Agreed. I've done this now (it's a separate patch). > > Stephen > > > Ian > > > > > + > > > +DPDK provides drivers for both physical and virtual devices. > > > +Physical DPDK devices are added to OVS by specifying a valid PCI > > > +address in > > > ``dpdk-devargs``. > > > +Virtual DPDK devices which do not have PCI addresses can be added > > > +using a different format for ``dpdk-devargs``. > > > + > > > +.. note:: > > > + > > > + Not all DPDK virtual PMD drivers have been tested and verified > > > + to > > > work. > > > + > > > +To add a virtual ``dpdk`` devices, the ``dpdk-devargs`` argument > > > +should be of the format ``eth_<driver_name><x>``, where ``x``' is a > > > +unique identifier of your choice for the given port. For example to > > > +add a ``dpdk`` port that uses the ``null`` DPDK PMD driver, run:: > > > + > > > + $ ovs-vsctl add-port br0 null0 -- set Interface null0 type=dpdk \ > > > + options:dpdk-devargs=eth_null0 > > > + > > > +Similarly, to add a ``dpdk`` port that uses the ``af_packet`` DPDK > > > +PMD driver, > > > +run:: > > > + > > > + $ ovs-vsctl add-port br0 myeth0 -- set Interface myeth0 type=dpdk > \ > > > + options:dpdk-devargs=eth_af_packet0,iface=eth0 > > > + > > > +More information on the different types of virtual DPDK PMDs can be > > > +found in the `DPDK documentation`__. > > > + > > > +__ http://dpdk.org/doc/guides/nics/overview.html > > > -- > > > 2.14.3 > > > > > > _______________________________________________ > > > dev mailing list > > > dev@openvswitch.org > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index c2324118d..4d993a0eb 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -57,8 +57,12 @@ usage is suggested:: $ ovs-vsctl add-port br0 dpdk-p1 -- set Interface dpdk-p1 type=dpdk \ options:dpdk-devargs="class=eth,mac=00:11:22:33:44:55:02" -Note: such syntax won't support hotplug. The hotplug is supposed to work with -future DPDK release, v18.05. +.. important:: + + Hotplugging physical interfaces is not supported using the above syntax. + This is expected to change with the release of DPDK v18.05. For information + on hotplugging physical interfaces, you should instead refer to + :ref:`port-hotplug`. After the DPDK ports get added to switch, a polling thread continuously polls DPDK devices and consumes 100% of the core, as can be checked from ``top`` and @@ -236,16 +240,6 @@ largest frame size supported by Fortville NIC using the DPDK i40e driver, but larger frames and other DPDK NIC drivers may be supported. These cases are common for use cases involving East-West traffic only. -Rx Checksum Offload -------------------- - -By default, DPDK physical ports are enabled with Rx checksum offload. - -Rx checksum offload can offer performance improvement only for tunneling -traffic in OVS-DPDK because the checksum validation of tunnel packets is -offloaded to the NIC. Also enabling Rx checksum may slightly reduce the -performance of non-tunnel traffic, specifically for smaller size packet. - .. _extended-statistics: Extended & Custom Statistics @@ -278,81 +272,6 @@ Note about "Extended Statistics": vHost ports supports only partial statistics. RX packet size based counter are only supported and doesn't include TX packet size counters. -.. _port-hotplug: - -Port Hotplug ------------- - -OVS supports port hotplugging, allowing the use of ports that were not bound -to DPDK when vswitchd was started. -In order to attach a port, it has to be bound to DPDK using the -``dpdk_nic_bind.py`` script:: - - $ $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio 0000:01:00.0 - -Then it can be attached to OVS:: - - $ ovs-vsctl add-port br0 dpdkx -- set Interface dpdkx type=dpdk \ - options:dpdk-devargs=0000:01:00.0 - -Detaching will be performed while processing del-port command:: - - $ ovs-vsctl del-port dpdkx - -Sometimes, the del-port command may not detach the device. -Detaching can be confirmed by the appearance of an INFO log. -For example:: - - INFO|Device '0000:04:00.1' has been detached - -If the log is not seen, then the port can be detached using:: - -$ ovs-appctl netdev-dpdk/detach 0000:01:00.0 - -Detaching can be confirmed by console output:: - - Device '0000:04:00.1' has been detached - -.. warning:: - Detaching should not be done if a device is known to be non-detachable, as - this may cause the device to behave improperly when added back with - add-port. The Chelsio Terminator adapters which use the cxgbe driver seem - to be an example of this behavior; check the driver documentation if this - is suspected. - -This feature does not work with some NICs. -For more information please refer to the `DPDK Port Hotplug Framework -<http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html#hotplug>`__. - -.. _vdev-support: - -Vdev Support ------------- - -DPDK provides drivers for both physical and virtual devices. Physical DPDK -devices are added to OVS by specifying a valid PCI address in 'dpdk-devargs'. -Virtual DPDK devices which do not have PCI addresses can be added using a -different format for 'dpdk-devargs'. - -Typically, the format expected is 'eth_<driver_name><x>' where 'x' is a -unique identifier of your choice for the given port. - -For example to add a dpdk port that uses the 'null' DPDK PMD driver:: - - $ ovs-vsctl add-port br0 null0 -- set Interface null0 type=dpdk \ - options:dpdk-devargs=eth_null0 - -Similarly, to add a dpdk port that uses the 'af_packet' DPDK PMD driver:: - - $ ovs-vsctl add-port br0 myeth0 -- set Interface myeth0 type=dpdk \ - options:dpdk-devargs=eth_af_packet0,iface=eth0 - -More information on the different types of virtual DPDK PMDs can be found in -the `DPDK documentation -<http://dpdk.org/doc/guides/nics/overview.html>`__. - -Note: Not all DPDK virtual PMD drivers have been tested and verified to work. - EMC Insertion Probability ------------------------- By default 1 in every 100 flows are inserted into the Exact Match Cache (EMC). diff --git a/Documentation/topics/dpdk/phy.rst b/Documentation/topics/dpdk/phy.rst index 222fa3e9f..507dac869 100644 --- a/Documentation/topics/dpdk/phy.rst +++ b/Documentation/topics/dpdk/phy.rst @@ -119,3 +119,94 @@ utilize is a requirement in order to deliver the high-performance possible with the DPDK datapath. It is possible to configure multiple Rx queues for ``dpdk`` ports, thus ensuring this is not a bottleneck for performance. For information on configuring PMD threads, refer to :doc:`pmd`. + +Rx Checksum Offload +------------------- + +By default, DPDK physical ports are enabled with Rx checksum offload. + +Rx checksum offload can offer performance improvement only for tunneling +traffic in OVS-DPDK because the checksum validation of tunnel packets is +offloaded to the NIC. Also enabling Rx checksum may slightly reduce the +performance of non-tunnel traffic, specifically for smaller size packet. + +.. _port-hotplug: + +Hotplugging +----------- + +OVS supports port hotplugging, allowing the use of physical ports that were not +bound to DPDK when ovs-vswitchd was started. + +.. warning:: + + This feature is not compatible with all NICs. Refer to vendor documentation + for more information. + +.. important:: + + Ports must be bound to DPDK. Refer to :ref:`dpdk-binding-nics` for more + information. + +To *hotplug* a port, simply add it like any other port:: + + $ ovs-vsctl add-port br0 dpdkx -- set Interface dpdkx type=dpdk \ + options:dpdk-devargs=0000:01:00.0 + +Ports can be detached using the ``del-port`` command:: + + $ ovs-vsctl del-port dpdkx + +This should both delete the port and detach the device. If successful, you +should see an ``INFO`` log. For example:: + + INFO|Device '0000:04:00.1' has been detached + +If the log is not seen then the port can be detached like so:: + + $ ovs-appctl netdev-dpdk/detach 0000:01:00.0 + +.. warning:: + + Detaching should not be done if a device is known to be non-detachable, as + this may cause the device to behave improperly when added back with + add-port. The Chelsio Terminator adapters which use the cxgbe driver seem + to be an example of this behavior; check the driver documentation if this + is suspected. + +For more information please refer to the `DPDK Port Hotplug Framework`__. + +__ http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html#hotplug + +.. _vdev-support: + +vdev Support +------------ + +DPDK provides drivers for both physical and virtual devices. Physical DPDK +devices are added to OVS by specifying a valid PCI address in ``dpdk-devargs``. +Virtual DPDK devices which do not have PCI addresses can be added using a +different format for ``dpdk-devargs``. + +.. note:: + + Not all DPDK virtual PMD drivers have been tested and verified to work. + +To add a virtual ``dpdk`` devices, the ``dpdk-devargs`` argument should be of +the format ``eth_<driver_name><x>``, where ``x``' is a unique identifier of +your choice for the given port. For example to add a ``dpdk`` port that uses +the ``null`` DPDK PMD driver, run:: + + $ ovs-vsctl add-port br0 null0 -- set Interface null0 type=dpdk \ + options:dpdk-devargs=eth_null0 + +Similarly, to add a ``dpdk`` port that uses the ``af_packet`` DPDK PMD driver, +run:: + + $ ovs-vsctl add-port br0 myeth0 -- set Interface myeth0 type=dpdk \ + options:dpdk-devargs=eth_af_packet0,iface=eth0 + +More information on the different types of virtual DPDK PMDs can be found in +the `DPDK documentation`__. + +__ http://dpdk.org/doc/guides/nics/overview.html
The "vdev", "hotplugging", and "Rx checksum offload" sections only apply to 'dpdk' ports and are too detailed to include in a high-level howto. Move them, reworking some aspects of this in the process. Signed-off-by: Stephen Finucane <stephen@that.guru> --- Documentation/howto/dpdk.rst | 93 +++------------------------------------ Documentation/topics/dpdk/phy.rst | 91 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 97 insertions(+), 87 deletions(-)