[ovs-dev,v2,1/9] doc: Add an overview of the 'dpdk' port

These ports are used to allow ingress/egress from the host and are
therefore _reasonably_ important. However, there is no clear overview of
what these ports actually are or why things are done the way they are.
Start closing this gap by providing a standalone example of using these
ports along with a little more detailed overview of the binding process.

There is additional cleanup to be done for the DPDK howto, but that will
be done separately.

We enable the TODO directive so we can actually start calling out some
TODOs.

Signed-off-by: Stephen Finucane <stephen@that.guru>
---
v2:
- Add TODO for multiple ports sharing the same bus slot function
  (ian.stokes)
---
 Documentation/conf.py               |   2 +-
 Documentation/topics/dpdk/index.rst |   1 +
 Documentation/topics/dpdk/phy.rst   | 115 ++++++++++++++++++++++++++++++++++++
 3 files changed, 117 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/topics/dpdk/phy.rst

> These ports are used to allow ingress/egress from the host and are
> therefore _reasonably_ important. However, there is no clear overview of
> what these ports actually are or why things are done the way they are.
> Start closing this gap by providing a standalone example of using these
> ports along with a little more detailed overview of the binding process.
> 
> There is additional cleanup to be done for the DPDK howto, but that will
> be done separately.

This patch breaks compilation with the following

The following files are in git but not the distribution:
Documentation/topics/dpdk/phy.rst

Need to add it to Documentation/automake.at

> 
> We enable the TODO directive so we can actually start calling out some
> TODOs.
> 
> Signed-off-by: Stephen Finucane <stephen@that.guru>
> ---
> v2:
> - Add TODO for multiple ports sharing the same bus slot function
>   (ian.stokes)
> ---
>  Documentation/conf.py               |   2 +-
>  Documentation/topics/dpdk/index.rst |   1 +
>  Documentation/topics/dpdk/phy.rst   | 115
> ++++++++++++++++++++++++++++++++++++
>  3 files changed, 117 insertions(+), 1 deletion(-)  create mode 100644
> Documentation/topics/dpdk/phy.rst
> 
> diff --git a/Documentation/conf.py b/Documentation/conf.py index
> 6ab144c5d..babda21de 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -32,7 +32,7 @@ needs_sphinx = '1.1'
>  # Add any Sphinx extension module names here, as strings. They can be  #
> extensions coming with Sphinx (named 'sphinx.ext.*') or your custom  #
> ones.
> -extensions = []
> +extensions = ['sphinx.ext.todo']
> 
>  # Add any paths that contain templates here, relative to this directory.
>  templates_path = ['_templates']
> diff --git a/Documentation/topics/dpdk/index.rst
> b/Documentation/topics/dpdk/index.rst
> index da148b323..5f836a6e9 100644
> --- a/Documentation/topics/dpdk/index.rst
> +++ b/Documentation/topics/dpdk/index.rst
> @@ -28,5 +28,6 @@ The DPDK Datapath
>  .. toctree::
>     :maxdepth: 2
> 
> +   phy
>     vhost-user
>     ring
> diff --git a/Documentation/topics/dpdk/phy.rst
> b/Documentation/topics/dpdk/phy.rst
> new file mode 100644
> index 000000000..a3f8b475c
> --- /dev/null
> +++ b/Documentation/topics/dpdk/phy.rst
> @@ -0,0 +1,115 @@
> +..
> +      Copyright 2018, Red Hat, Inc.
> +
> +      Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> +      not use this file except in compliance with the License. You may
> obtain
> +      a copy of the License at
> +
> +          http://www.apache.org/licenses/LICENSE-2.0
> +
> +      Unless required by applicable law or agreed to in writing, software
> +      distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> +      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> +      License for the specific language governing permissions and
> limitations
> +      under the License.
> +
> +      Convention for heading levels in Open vSwitch documentation:
> +
> +      =======  Heading 0 (reserved for the title in a document)
> +      -------  Heading 1
> +      ~~~~~~~  Heading 2
> +      +++++++  Heading 3
> +      '''''''  Heading 4
> +
> +      Avoid deeper levels because they do not render well.
> +
> +===================
> +DPDK Physical Ports
> +===================
> +
> +The netdev datapath allows attaching of DPDK-backed physical interfaces
> +in order to provide high-performance ingress/egress from the host.
> +
> +.. versionchanged:: 2.7.0
> +
> +   Before Open vSwitch 2.7.0, it was necessary to prefix port names with
> a
> +   ``dpdk`` prefix. Starting with 2.7.0, this is no longer necessary.
> +
> +.. todo::
> +
> +   Add an example for multiple ports share the same bus slot function.
> +
> +Quick Example
> +-------------
> +
> +This example demonstrates how to bind two ``dpdk`` ports, bound to
> +physical interfaces identified by hardware IDs ``0000:01:00.0`` and
> +``0000:01:00.1``, to an existing bridge called ``br0``::
> +
> +    $ ovs-vsctl add-port br0 dpdk-p0 \
> +       -- set Interface dpdk-p0 type=dpdk options:dpdk-
> devargs=0000:01:00.0
> +    $ ovs-vsctl add-port br0 dpdk-p1 \
> +       -- set Interface dpdk-p1 type=dpdk
> + options:dpdk-devargs=0000:01:00.1
> +
> +For the above example to work, the two physical interfaces must be
> +bound to the DPDK poll-mode drivers in userspace rather than the
> +traditional kernel drivers. See the `binding NIC drivers <dpdk-binding-
> nics>` section for details.
> +
> +.. _dpdk-binding-nics:
> +
> +Binding NIC Drivers
> +-------------------
> +
> +DPDK operates entirely in userspace and, as a result, requires use of
> +its own poll-mode drivers in user space for physical interfaces and a
> +passthrough-style driver for the devices in kernel space.
> +
> +There are two different tools for binding drivers: :command:`driverctl`
> +which is a generic tool for persistently configuring alternative device
> +drivers, and :command:`dpdk-devbind` which is a DPDK-specific tool and
> +whose changes do not persist across reboots. In addition, there are two
> +options available for this kernel space driver - VFIO (Virtual Function
> +I/O) and UIO (Userspace I/O) - along with a number of drivers for each
> +option. We will demonstrate examples of both tools and will use the
> +``vfio-pci`` driver, which is the more secure, robust driver of those
> +available. More information can be found in the `DPDK documentation
> <dpdk-drivers>`__.
> +
> +To list devices using :command:`driverctl`, run::
> +
> +    $ driverctl -v list-devices | grep -i net
> +    0000:07:00.0 igb (I350 Gigabit Network Connection (Ethernet Server
> Adapter I350-T2))
> +    0000:07:00.1 igb (I350 Gigabit Network Connection (Ethernet Server
> + Adapter I350-T2))
> +
> +You can then bind one or more of these devices using the same tool::
> +
> +    $ driverctl set-override 0000:07:00.0 vfio-pci
> +
> +Alternatively, to list devices using :command:`dpdk-devbind`, run::
> +
> +    $ dpdk-devbind --status
> +    Network devices using DPDK-compatible driver
> +    ============================================
> +    <none>
> +
> +    Network devices using kernel driver
> +    ===================================
> +    0000:07:00.0 'I350 Gigabit Network Connection 1521' if=enp7s0f0
> drv=igb unused=igb_uio
> +    0000:07:00.1 'I350 Gigabit Network Connection 1521' if=enp7s0f1
> + drv=igb unused=igb_uio

Just a note, these exceed the 79 character limit but I think for the sake of readability they are better kept as are.

Ian
> +
> +    Other Network devices
> +    =====================
> +    ...
> +
> +Once again, you can then bind one or more of these devices using the
> +same
> +tool::
> +
> +    $ dpdk-devbind --bind=vfio-pci 0000:07:00.0
> +
> +.. versionchanged:: 2.6.0
> +
> +   Open vSwitch 2.6.0 added support for DPDK 16.07, which in turn renamed
> the
> +   former ``dpdk_nic_bind`` tool to ``dpdk-devbind``.
> +
> +For more information, refer to the `DPDK documentation <dpdk-drivers>`__.
> +
> +.. _dpdk-drivers:
> +http://dpdk.org/doc/guides/linux_gsg/linux_drivers.html
> --
> 2.14.3

On Wed, 2018-04-18 at 15:30 +0000, Stokes, Ian wrote:
> > These ports are used to allow ingress/egress from the host and are
> > therefore _reasonably_ important. However, there is no clear overview of
> > what these ports actually are or why things are done the way they are.
> > Start closing this gap by providing a standalone example of using these
> > ports along with a little more detailed overview of the binding process.
> > 
> > There is additional cleanup to be done for the DPDK howto, but that will
> > be done separately.
> 
> This patch breaks compilation with the following
> 
> The following files are in git but not the distribution:
> Documentation/topics/dpdk/phy.rst
> 
> Need to add it to Documentation/automake.at

Whoops. I was building this with 'sphinx-build' directly to avoid
installing all the other build dependencies. Fixed now.

> > 
> > We enable the TODO directive so we can actually start calling out some
> > TODOs.
> > 
> > Signed-off-by: Stephen Finucane <stephen@that.guru>
> > ---
> > v2:
> > - Add TODO for multiple ports sharing the same bus slot function
> >   (ian.stokes)
> > ---
> >  Documentation/conf.py               |   2 +-
> >  Documentation/topics/dpdk/index.rst |   1 +
> >  Documentation/topics/dpdk/phy.rst   | 115
> > ++++++++++++++++++++++++++++++++++++
> >  3 files changed, 117 insertions(+), 1 deletion(-)  create mode 100644
> > Documentation/topics/dpdk/phy.rst
> > 
> > diff --git a/Documentation/conf.py b/Documentation/conf.py index
> > 6ab144c5d..babda21de 100644
> > --- a/Documentation/conf.py
> > +++ b/Documentation/conf.py
> > @@ -32,7 +32,7 @@ needs_sphinx = '1.1'
> >  # Add any Sphinx extension module names here, as strings. They can be  #
> > extensions coming with Sphinx (named 'sphinx.ext.*') or your custom  #
> > ones.
> > -extensions = []
> > +extensions = ['sphinx.ext.todo']
> > 
> >  # Add any paths that contain templates here, relative to this directory.
> >  templates_path = ['_templates']
> > diff --git a/Documentation/topics/dpdk/index.rst
> > b/Documentation/topics/dpdk/index.rst
> > index da148b323..5f836a6e9 100644
> > --- a/Documentation/topics/dpdk/index.rst
> > +++ b/Documentation/topics/dpdk/index.rst
> > @@ -28,5 +28,6 @@ The DPDK Datapath
> >  .. toctree::
> >     :maxdepth: 2
> > 
> > +   phy
> >     vhost-user
> >     ring
> > diff --git a/Documentation/topics/dpdk/phy.rst
> > b/Documentation/topics/dpdk/phy.rst
> > new file mode 100644
> > index 000000000..a3f8b475c
> > --- /dev/null
> > +++ b/Documentation/topics/dpdk/phy.rst
> > @@ -0,0 +1,115 @@
> > +..
> > +      Copyright 2018, Red Hat, Inc.
> > +
> > +      Licensed under the Apache License, Version 2.0 (the "License"); you
> > may
> > +      not use this file except in compliance with the License. You may
> > obtain
> > +      a copy of the License at
> > +
> > +          http://www.apache.org/licenses/LICENSE-2.0
> > +
> > +      Unless required by applicable law or agreed to in writing, software
> > +      distributed under the License is distributed on an "AS IS" BASIS,
> > WITHOUT
> > +      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> > See the
> > +      License for the specific language governing permissions and
> > limitations
> > +      under the License.
> > +
> > +      Convention for heading levels in Open vSwitch documentation:
> > +
> > +      =======  Heading 0 (reserved for the title in a document)
> > +      -------  Heading 1
> > +      ~~~~~~~  Heading 2
> > +      +++++++  Heading 3
> > +      '''''''  Heading 4
> > +
> > +      Avoid deeper levels because they do not render well.
> > +
> > +===================
> > +DPDK Physical Ports
> > +===================
> > +
> > +The netdev datapath allows attaching of DPDK-backed physical interfaces
> > +in order to provide high-performance ingress/egress from the host.
> > +
> > +.. versionchanged:: 2.7.0
> > +
> > +   Before Open vSwitch 2.7.0, it was necessary to prefix port names with
> > a
> > +   ``dpdk`` prefix. Starting with 2.7.0, this is no longer necessary.
> > +
> > +.. todo::
> > +
> > +   Add an example for multiple ports share the same bus slot function.
> > +
> > +Quick Example
> > +-------------
> > +
> > +This example demonstrates how to bind two ``dpdk`` ports, bound to
> > +physical interfaces identified by hardware IDs ``0000:01:00.0`` and
> > +``0000:01:00.1``, to an existing bridge called ``br0``::
> > +
> > +    $ ovs-vsctl add-port br0 dpdk-p0 \
> > +       -- set Interface dpdk-p0 type=dpdk options:dpdk-
> > devargs=0000:01:00.0
> > +    $ ovs-vsctl add-port br0 dpdk-p1 \
> > +       -- set Interface dpdk-p1 type=dpdk
> > + options:dpdk-devargs=0000:01:00.1
> > +
> > +For the above example to work, the two physical interfaces must be
> > +bound to the DPDK poll-mode drivers in userspace rather than the
> > +traditional kernel drivers. See the `binding NIC drivers <dpdk-binding-
> > nics>` section for details.
> > +
> > +.. _dpdk-binding-nics:
> > +
> > +Binding NIC Drivers
> > +-------------------
> > +
> > +DPDK operates entirely in userspace and, as a result, requires use of
> > +its own poll-mode drivers in user space for physical interfaces and a
> > +passthrough-style driver for the devices in kernel space.
> > +
> > +There are two different tools for binding drivers: :command:`driverctl`
> > +which is a generic tool for persistently configuring alternative device
> > +drivers, and :command:`dpdk-devbind` which is a DPDK-specific tool and
> > +whose changes do not persist across reboots. In addition, there are two
> > +options available for this kernel space driver - VFIO (Virtual Function
> > +I/O) and UIO (Userspace I/O) - along with a number of drivers for each
> > +option. We will demonstrate examples of both tools and will use the
> > +``vfio-pci`` driver, which is the more secure, robust driver of those
> > +available. More information can be found in the `DPDK documentation
> > <dpdk-drivers>`__.
> > +
> > +To list devices using :command:`driverctl`, run::
> > +
> > +    $ driverctl -v list-devices | grep -i net
> > +    0000:07:00.0 igb (I350 Gigabit Network Connection (Ethernet Server
> > Adapter I350-T2))
> > +    0000:07:00.1 igb (I350 Gigabit Network Connection (Ethernet Server
> > + Adapter I350-T2))
> > +
> > +You can then bind one or more of these devices using the same tool::
> > +
> > +    $ driverctl set-override 0000:07:00.0 vfio-pci
> > +
> > +Alternatively, to list devices using :command:`dpdk-devbind`, run::
> > +
> > +    $ dpdk-devbind --status
> > +    Network devices using DPDK-compatible driver
> > +    ============================================
> > +    <none>
> > +
> > +    Network devices using kernel driver
> > +    ===================================
> > +    0000:07:00.0 'I350 Gigabit Network Connection 1521' if=enp7s0f0
> > drv=igb unused=igb_uio
> > +    0000:07:00.1 'I350 Gigabit Network Connection 1521' if=enp7s0f1
> > + drv=igb unused=igb_uio
> 
> Just a note, these exceed the 79 character limit but I think for the
> sake of readability they are better kept as are.

Yes, the 79 character limit shouldn't apply to literal blocks that
can't be wrapped (generally output). If that's not called out, it
should be :)

Stephen

> Ian
> > +
> > +    Other Network devices
> > +    =====================
> > +    ...
> > +
> > +Once again, you can then bind one or more of these devices using the
> > +same
> > +tool::
> > +
> > +    $ dpdk-devbind --bind=vfio-pci 0000:07:00.0
> > +
> > +.. versionchanged:: 2.6.0
> > +
> > +   Open vSwitch 2.6.0 added support for DPDK 16.07, which in turn renamed
> > the
> > +   former ``dpdk_nic_bind`` tool to ``dpdk-devbind``.
> > +
> > +For more information, refer to the `DPDK documentation <dpdk-drivers>`__.
> > +
> > +.. _dpdk-drivers:
> > +http://dpdk.org/doc/guides/linux_gsg/linux_drivers.html
> > --
> > 2.14.3
> 
>