From patchwork Mon Apr 16 14:30:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 898655 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="UezOi5Up"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40PrWh21zbz9rx7 for ; Tue, 17 Apr 2018 00:35:12 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 561A0ED6; Mon, 16 Apr 2018 14:30:50 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id A66E1EA5 for ; Mon, 16 Apr 2018 14:30:47 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from anteater.ash.relay.mailchannels.net (anteater.ash.relay.mailchannels.net [23.83.222.3]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id A35CC681 for ; Mon, 16 Apr 2018 14:30:44 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 76919680BCB; Mon, 16 Apr 2018 14:30:37 +0000 (UTC) Received: from one.mxroute.com (unknown [100.96.13.37]) (Authenticated sender: 5xi41l16bi) by relay.mailchannels.net (Postfix) with ESMTPA id E2CAF680981; Mon, 16 Apr 2018 14:30:36 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from one.mxroute.com (one-outgoing.mxroute.com [172.19.42.253]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.14.1); Mon, 16 Apr 2018 14:30:37 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: 5xi41l16bi|x-authuser|stephen@that.guru X-MailChannels-Auth-Id: 5xi41l16bi X-Cooperative-Hook: 60bbcc9e3100b3d6_1523889037348_2631579356 X-MC-Loop-Signature: 1523889037348:2219410218 X-MC-Ingress-Time: 1523889037347 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=vf6XmJevFMAMUgCEIh2C83MBWIeIt9szPqM3ufcYbHY=; b=UezOi5UpWLDEWJaaj00I5FVSmU miZ5WeKTdc3S9beIE963hfOFIg9MMfC9rKOcE8GQe7X3ke+pI6/SvPPIb0SBOYKzrjmJr96Tc0GfK eeImz6fnzKv57htIklSmD8MVfrLso/TLfyRB6ACakCostgVUj8+7KsaK0SfdNTq7I8dmVHUfQwiuI awaYoTE2XQSPXKT7FbjcmSTtizLlsP+sH3QoKx1IBLvCh8WiH0I8v0OCIQHBqj0qssq5IbkWe3SeK MKHUtL/ZTjnNTuZt3Pd/gMNskkX2mgfCXJhYLLMip5fj50nrIyJ+mliPm2wkRHMhwwMd9APDM1ACN seGNub2g==; From: Stephen Finucane To: dev@openvswitch.org Date: Mon, 16 Apr 2018 15:30:18 +0100 Message-Id: <20180416143026.24561-2-stephen@that.guru> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180416143026.24561-1-stephen@that.guru> References: <20180416143026.24561-1-stephen@that.guru> X-AuthUser: stephen@that.guru X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v2 1/9] doc: Add an overview of the 'dpdk' port X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org These ports are used to allow ingress/egress from the host and are therefore _reasonably_ important. However, there is no clear overview of what these ports actually are or why things are done the way they are. Start closing this gap by providing a standalone example of using these ports along with a little more detailed overview of the binding process. There is additional cleanup to be done for the DPDK howto, but that will be done separately. We enable the TODO directive so we can actually start calling out some TODOs. Signed-off-by: Stephen Finucane --- v2: - Add TODO for multiple ports sharing the same bus slot function (ian.stokes) --- Documentation/conf.py | 2 +- Documentation/topics/dpdk/index.rst | 1 + Documentation/topics/dpdk/phy.rst | 115 ++++++++++++++++++++++++++++++++++++ 3 files changed, 117 insertions(+), 1 deletion(-) create mode 100644 Documentation/topics/dpdk/phy.rst diff --git a/Documentation/conf.py b/Documentation/conf.py index 6ab144c5d..babda21de 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -32,7 +32,7 @@ needs_sphinx = '1.1' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions = [] +extensions = ['sphinx.ext.todo'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] diff --git a/Documentation/topics/dpdk/index.rst b/Documentation/topics/dpdk/index.rst index da148b323..5f836a6e9 100644 --- a/Documentation/topics/dpdk/index.rst +++ b/Documentation/topics/dpdk/index.rst @@ -28,5 +28,6 @@ The DPDK Datapath .. toctree:: :maxdepth: 2 + phy vhost-user ring diff --git a/Documentation/topics/dpdk/phy.rst b/Documentation/topics/dpdk/phy.rst new file mode 100644 index 000000000..a3f8b475c --- /dev/null +++ b/Documentation/topics/dpdk/phy.rst @@ -0,0 +1,115 @@ +.. + Copyright 2018, Red Hat, Inc. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +=================== +DPDK Physical Ports +=================== + +The netdev datapath allows attaching of DPDK-backed physical interfaces in +order to provide high-performance ingress/egress from the host. + +.. versionchanged:: 2.7.0 + + Before Open vSwitch 2.7.0, it was necessary to prefix port names with a + ``dpdk`` prefix. Starting with 2.7.0, this is no longer necessary. + +.. todo:: + + Add an example for multiple ports share the same bus slot function. + +Quick Example +------------- + +This example demonstrates how to bind two ``dpdk`` ports, bound to physical +interfaces identified by hardware IDs ``0000:01:00.0`` and ``0000:01:00.1``, to +an existing bridge called ``br0``:: + + $ ovs-vsctl add-port br0 dpdk-p0 \ + -- set Interface dpdk-p0 type=dpdk options:dpdk-devargs=0000:01:00.0 + $ ovs-vsctl add-port br0 dpdk-p1 \ + -- set Interface dpdk-p1 type=dpdk options:dpdk-devargs=0000:01:00.1 + +For the above example to work, the two physical interfaces must be bound to +the DPDK poll-mode drivers in userspace rather than the traditional kernel +drivers. See the `binding NIC drivers ` section for details. + +.. _dpdk-binding-nics: + +Binding NIC Drivers +------------------- + +DPDK operates entirely in userspace and, as a result, requires use of its own +poll-mode drivers in user space for physical interfaces and a passthrough-style +driver for the devices in kernel space. + +There are two different tools for binding drivers: :command:`driverctl` which +is a generic tool for persistently configuring alternative device drivers, and +:command:`dpdk-devbind` which is a DPDK-specific tool and whose changes do not +persist across reboots. In addition, there are two options available for this +kernel space driver - VFIO (Virtual Function I/O) and UIO (Userspace I/O) - +along with a number of drivers for each option. We will demonstrate examples of +both tools and will use the ``vfio-pci`` driver, which is the more secure, +robust driver of those available. More information can be found in the `DPDK +documentation `__. + +To list devices using :command:`driverctl`, run:: + + $ driverctl -v list-devices | grep -i net + 0000:07:00.0 igb (I350 Gigabit Network Connection (Ethernet Server Adapter I350-T2)) + 0000:07:00.1 igb (I350 Gigabit Network Connection (Ethernet Server Adapter I350-T2)) + +You can then bind one or more of these devices using the same tool:: + + $ driverctl set-override 0000:07:00.0 vfio-pci + +Alternatively, to list devices using :command:`dpdk-devbind`, run:: + + $ dpdk-devbind --status + Network devices using DPDK-compatible driver + ============================================ + + + Network devices using kernel driver + =================================== + 0000:07:00.0 'I350 Gigabit Network Connection 1521' if=enp7s0f0 drv=igb unused=igb_uio + 0000:07:00.1 'I350 Gigabit Network Connection 1521' if=enp7s0f1 drv=igb unused=igb_uio + + Other Network devices + ===================== + ... + +Once again, you can then bind one or more of these devices using the same +tool:: + + $ dpdk-devbind --bind=vfio-pci 0000:07:00.0 + +.. versionchanged:: 2.6.0 + + Open vSwitch 2.6.0 added support for DPDK 16.07, which in turn renamed the + former ``dpdk_nic_bind`` tool to ``dpdk-devbind``. + +For more information, refer to the `DPDK documentation `__. + +.. _dpdk-drivers: http://dpdk.org/doc/guides/linux_gsg/linux_drivers.html