From patchwork Mon Apr 16 14:30:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 898655 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="UezOi5Up"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40PrWh21zbz9rx7 for ; Tue, 17 Apr 2018 00:35:12 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 561A0ED6; Mon, 16 Apr 2018 14:30:50 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id A66E1EA5 for ; Mon, 16 Apr 2018 14:30:47 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from anteater.ash.relay.mailchannels.net (anteater.ash.relay.mailchannels.net [23.83.222.3]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id A35CC681 for ; Mon, 16 Apr 2018 14:30:44 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 76919680BCB; Mon, 16 Apr 2018 14:30:37 +0000 (UTC) Received: from one.mxroute.com (unknown [100.96.13.37]) (Authenticated sender: 5xi41l16bi) by relay.mailchannels.net (Postfix) with ESMTPA id E2CAF680981; Mon, 16 Apr 2018 14:30:36 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from one.mxroute.com (one-outgoing.mxroute.com [172.19.42.253]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.14.1); Mon, 16 Apr 2018 14:30:37 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: 5xi41l16bi|x-authuser|stephen@that.guru X-MailChannels-Auth-Id: 5xi41l16bi X-Cooperative-Hook: 60bbcc9e3100b3d6_1523889037348_2631579356 X-MC-Loop-Signature: 1523889037348:2219410218 X-MC-Ingress-Time: 1523889037347 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=vf6XmJevFMAMUgCEIh2C83MBWIeIt9szPqM3ufcYbHY=; b=UezOi5UpWLDEWJaaj00I5FVSmU miZ5WeKTdc3S9beIE963hfOFIg9MMfC9rKOcE8GQe7X3ke+pI6/SvPPIb0SBOYKzrjmJr96Tc0GfK eeImz6fnzKv57htIklSmD8MVfrLso/TLfyRB6ACakCostgVUj8+7KsaK0SfdNTq7I8dmVHUfQwiuI awaYoTE2XQSPXKT7FbjcmSTtizLlsP+sH3QoKx1IBLvCh8WiH0I8v0OCIQHBqj0qssq5IbkWe3SeK MKHUtL/ZTjnNTuZt3Pd/gMNskkX2mgfCXJhYLLMip5fj50nrIyJ+mliPm2wkRHMhwwMd9APDM1ACN seGNub2g==; From: Stephen Finucane To: dev@openvswitch.org Date: Mon, 16 Apr 2018 15:30:18 +0100 Message-Id: <20180416143026.24561-2-stephen@that.guru> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180416143026.24561-1-stephen@that.guru> References: <20180416143026.24561-1-stephen@that.guru> X-AuthUser: stephen@that.guru X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v2 1/9] doc: Add an overview of the 'dpdk' port X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org These ports are used to allow ingress/egress from the host and are therefore _reasonably_ important. However, there is no clear overview of what these ports actually are or why things are done the way they are. Start closing this gap by providing a standalone example of using these ports along with a little more detailed overview of the binding process. There is additional cleanup to be done for the DPDK howto, but that will be done separately. We enable the TODO directive so we can actually start calling out some TODOs. Signed-off-by: Stephen Finucane --- v2: - Add TODO for multiple ports sharing the same bus slot function (ian.stokes) --- Documentation/conf.py | 2 +- Documentation/topics/dpdk/index.rst | 1 + Documentation/topics/dpdk/phy.rst | 115 ++++++++++++++++++++++++++++++++++++ 3 files changed, 117 insertions(+), 1 deletion(-) create mode 100644 Documentation/topics/dpdk/phy.rst diff --git a/Documentation/conf.py b/Documentation/conf.py index 6ab144c5d..babda21de 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -32,7 +32,7 @@ needs_sphinx = '1.1' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions = [] +extensions = ['sphinx.ext.todo'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] diff --git a/Documentation/topics/dpdk/index.rst b/Documentation/topics/dpdk/index.rst index da148b323..5f836a6e9 100644 --- a/Documentation/topics/dpdk/index.rst +++ b/Documentation/topics/dpdk/index.rst @@ -28,5 +28,6 @@ The DPDK Datapath .. toctree:: :maxdepth: 2 + phy vhost-user ring diff --git a/Documentation/topics/dpdk/phy.rst b/Documentation/topics/dpdk/phy.rst new file mode 100644 index 000000000..a3f8b475c --- /dev/null +++ b/Documentation/topics/dpdk/phy.rst @@ -0,0 +1,115 @@ +.. + Copyright 2018, Red Hat, Inc. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +=================== +DPDK Physical Ports +=================== + +The netdev datapath allows attaching of DPDK-backed physical interfaces in +order to provide high-performance ingress/egress from the host. + +.. versionchanged:: 2.7.0 + + Before Open vSwitch 2.7.0, it was necessary to prefix port names with a + ``dpdk`` prefix. Starting with 2.7.0, this is no longer necessary. + +.. todo:: + + Add an example for multiple ports share the same bus slot function. + +Quick Example +------------- + +This example demonstrates how to bind two ``dpdk`` ports, bound to physical +interfaces identified by hardware IDs ``0000:01:00.0`` and ``0000:01:00.1``, to +an existing bridge called ``br0``:: + + $ ovs-vsctl add-port br0 dpdk-p0 \ + -- set Interface dpdk-p0 type=dpdk options:dpdk-devargs=0000:01:00.0 + $ ovs-vsctl add-port br0 dpdk-p1 \ + -- set Interface dpdk-p1 type=dpdk options:dpdk-devargs=0000:01:00.1 + +For the above example to work, the two physical interfaces must be bound to +the DPDK poll-mode drivers in userspace rather than the traditional kernel +drivers. See the `binding NIC drivers ` section for details. + +.. _dpdk-binding-nics: + +Binding NIC Drivers +------------------- + +DPDK operates entirely in userspace and, as a result, requires use of its own +poll-mode drivers in user space for physical interfaces and a passthrough-style +driver for the devices in kernel space. + +There are two different tools for binding drivers: :command:`driverctl` which +is a generic tool for persistently configuring alternative device drivers, and +:command:`dpdk-devbind` which is a DPDK-specific tool and whose changes do not +persist across reboots. In addition, there are two options available for this +kernel space driver - VFIO (Virtual Function I/O) and UIO (Userspace I/O) - +along with a number of drivers for each option. We will demonstrate examples of +both tools and will use the ``vfio-pci`` driver, which is the more secure, +robust driver of those available. More information can be found in the `DPDK +documentation `__. + +To list devices using :command:`driverctl`, run:: + + $ driverctl -v list-devices | grep -i net + 0000:07:00.0 igb (I350 Gigabit Network Connection (Ethernet Server Adapter I350-T2)) + 0000:07:00.1 igb (I350 Gigabit Network Connection (Ethernet Server Adapter I350-T2)) + +You can then bind one or more of these devices using the same tool:: + + $ driverctl set-override 0000:07:00.0 vfio-pci + +Alternatively, to list devices using :command:`dpdk-devbind`, run:: + + $ dpdk-devbind --status + Network devices using DPDK-compatible driver + ============================================ + + + Network devices using kernel driver + =================================== + 0000:07:00.0 'I350 Gigabit Network Connection 1521' if=enp7s0f0 drv=igb unused=igb_uio + 0000:07:00.1 'I350 Gigabit Network Connection 1521' if=enp7s0f1 drv=igb unused=igb_uio + + Other Network devices + ===================== + ... + +Once again, you can then bind one or more of these devices using the same +tool:: + + $ dpdk-devbind --bind=vfio-pci 0000:07:00.0 + +.. versionchanged:: 2.6.0 + + Open vSwitch 2.6.0 added support for DPDK 16.07, which in turn renamed the + former ``dpdk_nic_bind`` tool to ``dpdk-devbind``. + +For more information, refer to the `DPDK documentation `__. + +.. _dpdk-drivers: http://dpdk.org/doc/guides/linux_gsg/linux_drivers.html From patchwork Mon Apr 16 14:30:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 898654 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="nFwJVojp"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40PrVm1TmCz9rx7 for ; Tue, 17 Apr 2018 00:34:24 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 1FB56EBA; Mon, 16 Apr 2018 14:30:49 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 60A42E81 for ; Mon, 16 Apr 2018 14:30:47 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from glass.birch.relay.mailchannels.net (glass.birch.relay.mailchannels.net [23.83.209.70]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B355B682 for ; Mon, 16 Apr 2018 14:30:45 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 1ECFE6813B4; Mon, 16 Apr 2018 14:30:39 +0000 (UTC) Received: from one.mxroute.com (unknown [100.96.20.22]) (Authenticated sender: 5xi41l16bi) by relay.mailchannels.net (Postfix) with ESMTPA id 77BE768131F; Mon, 16 Apr 2018 14:30:38 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from one.mxroute.com (one-outgoing.mxroute.com [172.19.42.59]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.14.1); Mon, 16 Apr 2018 14:30:39 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: 5xi41l16bi|x-authuser|stephen@that.guru X-MailChannels-Auth-Id: 5xi41l16bi X-Rock-Power: 6ae43fb140215078_1523889039005_3115375063 X-MC-Loop-Signature: 1523889039005:981229200 X-MC-Ingress-Time: 1523889039004 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Ec/FDWQZbYVUK/x+sr0nTvbfv1B6ZORr03zEsp+M39c=; b=nFwJVojp6gEU8LEBpelqTlGEUU 11z6hD5WRk0cHZANuHiURy5+xmS7jEODuzFk6cnbCsCuX5ArG9/nGK8hD2d6p2pqj3+xFeyWOZ/Zs CD9ReltqE+ixsvKlez/ohdgh8mNtEUorbMFDJJ+nchV0j6qu084OWjCuTuRylF2e3hil73qihLqqg 34Y57dYUKKZ01W7wGxJ9yO7M9oVUwqEeLpAHOjvnGq8FORp5W+XMfBsxW8JU+FWaB3HfdbxzAicZJ iGpNQjhJ2maE877CK2XqI/rzM2Jm3MJPqHv4DAqB+NOxyilvwr4KVyJQ1SOA7NPJ6Hz9nAESoQ813 oBe8yOZw==; From: Stephen Finucane To: dev@openvswitch.org Date: Mon, 16 Apr 2018 15:30:19 +0100 Message-Id: <20180416143026.24561-3-stephen@that.guru> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180416143026.24561-1-stephen@that.guru> References: <20180416143026.24561-1-stephen@that.guru> X-AuthUser: stephen@that.guru X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v2 2/9] doc: Add "PMD" topic document X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This continues the breakup of the huge DPDK "howto" into smaller components. There are a couple of related changes included, such as using "Rx queue" instead of "rxq" and noting how Tx queues cannot be configured. Signed-off-by: Stephen Finucane --- v2: - Add cross-references from 'pmd' doc to 'vhost-user' and 'phy' docs - Add 'versionchanged' warning about automatic assignment of Rx queues - Add a 'todo' to describe Tx queue behavior --- Documentation/howto/dpdk.rst | 86 ----------------- Documentation/topics/dpdk/index.rst | 1 + Documentation/topics/dpdk/phy.rst | 12 +++ Documentation/topics/dpdk/pmd.rst | 156 +++++++++++++++++++++++++++++++ Documentation/topics/dpdk/vhost-user.rst | 17 ++-- 5 files changed, 177 insertions(+), 95 deletions(-) create mode 100644 Documentation/topics/dpdk/pmd.rst diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index 79b626c76..388728363 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -81,92 +81,6 @@ To stop ovs-vswitchd & delete bridge, run:: $ ovs-appctl -t ovsdb-server exit $ ovs-vsctl del-br br0 -PMD Thread Statistics ---------------------- - -To show current stats:: - - $ ovs-appctl dpif-netdev/pmd-stats-show - -To clear previous stats:: - - $ ovs-appctl dpif-netdev/pmd-stats-clear - -Port/RXQ Assigment to PMD Threads ---------------------------------- - -To show port/rxq assignment:: - - $ ovs-appctl dpif-netdev/pmd-rxq-show - -To change default rxq assignment to pmd threads, rxqs may be manually pinned to -desired cores using:: - - $ ovs-vsctl set Interface \ - other_config:pmd-rxq-affinity= - -where: - -- ```` is a CSV list of ``:`` values - -For example:: - - $ ovs-vsctl set interface dpdk-p0 options:n_rxq=4 \ - other_config:pmd-rxq-affinity="0:3,1:7,3:8" - -This will ensure: - -- Queue #0 pinned to core 3 -- Queue #1 pinned to core 7 -- Queue #2 not pinned -- Queue #3 pinned to core 8 - -After that PMD threads on cores where RX queues was pinned will become -``isolated``. This means that this thread will poll only pinned RX queues. - -.. warning:: - If there are no ``non-isolated`` PMD threads, ``non-pinned`` RX queues will - not be polled. Also, if provided ``core_id`` is not available (ex. this - ``core_id`` not in ``pmd-cpu-mask``), RX queue will not be polled by any PMD - thread. - -If pmd-rxq-affinity is not set for rxqs, they will be assigned to pmds (cores) -automatically. The processing cycles that have been stored for each rxq -will be used where known to assign rxqs to pmd based on a round robin of the -sorted rxqs. - -For example, in the case where here there are 5 rxqs and 3 cores (e.g. 3,7,8) -available, and the measured usage of core cycles per rxq over the last -interval is seen to be: - -- Queue #0: 30% -- Queue #1: 80% -- Queue #3: 60% -- Queue #4: 70% -- Queue #5: 10% - -The rxqs will be assigned to cores 3,7,8 in the following order: - -Core 3: Q1 (80%) | -Core 7: Q4 (70%) | Q5 (10%) -core 8: Q3 (60%) | Q0 (30%) - -To see the current measured usage history of pmd core cycles for each rxq:: - - $ ovs-appctl dpif-netdev/pmd-rxq-show - -.. note:: - - A history of one minute is recorded and shown for each rxq to allow for - traffic pattern spikes. An rxq's pmd core cycles usage changes due to traffic - pattern or reconfig changes will take one minute before they are fully - reflected in the stats. - -Rxq to pmds assignment takes place whenever there are configuration changes -or can be triggered by using:: - - $ ovs-appctl dpif-netdev/pmd-rxq-rebalance - QoS --- diff --git a/Documentation/topics/dpdk/index.rst b/Documentation/topics/dpdk/index.rst index 5f836a6e9..dfde88377 100644 --- a/Documentation/topics/dpdk/index.rst +++ b/Documentation/topics/dpdk/index.rst @@ -31,3 +31,4 @@ The DPDK Datapath phy vhost-user ring + pmd diff --git a/Documentation/topics/dpdk/phy.rst b/Documentation/topics/dpdk/phy.rst index a3f8b475c..ad191dad0 100644 --- a/Documentation/topics/dpdk/phy.rst +++ b/Documentation/topics/dpdk/phy.rst @@ -113,3 +113,15 @@ tool:: For more information, refer to the `DPDK documentation `__. .. _dpdk-drivers: http://dpdk.org/doc/guides/linux_gsg/linux_drivers.html + +.. _dpdk-phy-multiqueue: + +Multiqueue +---------- + +Poll Mode Driver (PMD) threads are the threads that do the heavy lifting for +the DPDK datapath. Correct configuration of PMD threads and the Rx queues they +utilize is a requirement in order to deliver the high-performance possible with +DPDK acceleration. It is possible to configure multiple Rx queues for ``dpdk`` +ports, thus ensuring this is not a bottleneck for performance. For information +on configuring PMD threads, refer to :doc:`pmd`. diff --git a/Documentation/topics/dpdk/pmd.rst b/Documentation/topics/dpdk/pmd.rst new file mode 100644 index 000000000..1be25ade0 --- /dev/null +++ b/Documentation/topics/dpdk/pmd.rst @@ -0,0 +1,156 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +=========== +PMD Threads +=========== + +Poll Mode Driver (PMD) threads are the threads that do the heavy lifting for +the DPDK datapath and perform tasks such as continuous polling of input ports +for packets, classifying packets once received, and executing actions on the +packets once they are classified. + +PMD threads utilize Receive (Rx) and Transmit (Tx) queues, commonly known as +*rxq*\s and *txq*\s. While Tx queue configuration happens automatically, Rx +queues can be configured by the user. This can happen in one of two ways: + +- For physical interfaces, configuration is done using the + :program:`ovs-appctl` utility. + +- For virtual interfaces, configuration is done using the :program:`ovs-appctl` + utility, but this configuration must be reflected in the guest configuration + (e.g. QEMU command line arguments). + +The :program:`ovs-appctl` utility also provides a number of commands for +querying PMD threads and their respective queues. This, and all of the above, +is discussed here. + +.. todo:: + + Add an overview of Tx queues including numbers created, how they relate to + PMD threads, etc. + +PMD Thread Statistics +--------------------- + +To show current stats:: + + $ ovs-appctl dpif-netdev/pmd-stats-show + +To clear previous stats:: + + $ ovs-appctl dpif-netdev/pmd-stats-clear + +Port/Rx Queue Assigment to PMD Threads +-------------------------------------- + +.. todo:: + + This needs a more detailed overview of *why* this should be done, along with + the impact on things like NUMA affinity. + +Correct configuration of PMD threads and the Rx queues they utilize is a +requirement in order to achieve maximum performance. This is particularly true +for enabling things like multiqueue for :ref:`physical ` +and :ref:`vhost-user ` interfaces. + +To show port/Rx queue assignment:: + + $ ovs-appctl dpif-netdev/pmd-rxq-show + +Rx queues may be manually pinned to cores. This will change the default Rx +queue assignment to PMD threads:: + + $ ovs-vsctl set Interface \ + other_config:pmd-rxq-affinity= + +where: + +- ```` is a CSV list of ``:`` values + +For example:: + + $ ovs-vsctl set interface dpdk-p0 options:n_rxq=4 \ + other_config:pmd-rxq-affinity="0:3,1:7,3:8" + +This will ensure there are *4* Rx queues and that these queues are configured +like so: + +- Queue #0 pinned to core 3 +- Queue #1 pinned to core 7 +- Queue #2 not pinned +- Queue #3 pinned to core 8 + +PMD threads on cores where Rx queues are *pinned* will become *isolated*. This +means that this thread will only poll the *pinned* Rx queues. + +.. warning:: + + If there are no *non-isolated* PMD threads, *non-pinned* RX queues will not + be polled. Also, if the provided ```` is not available (e.g. the + ```` is not in ``pmd-cpu-mask``), the RX queue will not be polled + by any PMD thread. + +If ``pmd-rxq-affinity`` is not set for Rx queues, they will be assigned to PMDs +(cores) automatically. Where known, the processing cycles that have been stored +for each Rx queue will be used to assign Rx queue to PMDs based on a round +robin of the sorted Rx queues. For example, take the following example, where +there are five Rx queues and three cores - 3, 7, and 8 - available and the +measured usage of core cycles per Rx queue over the last interval is seen to +be: + +- Queue #0: 30% +- Queue #1: 80% +- Queue #3: 60% +- Queue #4: 70% +- Queue #5: 10% + +The Rx queues will be assigned to the cores in the following order:: + + Core 3: Q1 (80%) | + Core 7: Q4 (70%) | Q5 (10%) + Core 8: Q3 (60%) | Q0 (30%) + +To see the current measured usage history of PMD core cycles for each Rx +queue:: + + $ ovs-appctl dpif-netdev/pmd-rxq-show + +.. note:: + + A history of one minute is recorded and shown for each Rx queue to allow for + traffic pattern spikes. Any changes in the Rx queue's PMD core cycles usage, + due to traffic pattern or reconfig changes, will take one minute to be fully + reflected in the stats. + +Rx queue to PMD assignment takes place whenever there are configuration changes +or can be triggered by using:: + + $ ovs-appctl dpif-netdev/pmd-rxq-rebalance + +.. versionchanged:: 2.8.0 + + Automatic assignment of Rx queues to PMDs and the two related commands, + ``pmd-rxq-show`` and ``pmd-rxq-rebalance``, were added in OVS 2.8.0. Prior + to this, behavior was round-robin and processing cycles were not taken into + consideration. Tracking for stats was not available. diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst index ca8a3289f..6f794f296 100644 --- a/Documentation/topics/dpdk/vhost-user.rst +++ b/Documentation/topics/dpdk/vhost-user.rst @@ -130,11 +130,10 @@ an additional set of parameters:: -netdev type=vhost-user,id=mynet2,chardev=char2,vhostforce -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2 -In addition, QEMU must allocate the VM's memory on hugetlbfs. vhost-user -ports access a virtio-net device's virtual rings and packet buffers mapping the -VM's physical memory on hugetlbfs. To enable vhost-user ports to map the VM's -memory into their process address space, pass the following parameters to -QEMU:: +In addition, QEMU must allocate the VM's memory on hugetlbfs. vhost-user ports +access a virtio-net device's virtual rings and packet buffers mapping the VM's +physical memory on hugetlbfs. To enable vhost-user ports to map the VM's memory +into their process address space, pass the following parameters to QEMU:: -object memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on -numa node,memdev=mem -mem-prealloc @@ -154,18 +153,18 @@ where: The number of vectors, which is ``$q`` * 2 + 2 The vhost-user interface will be automatically reconfigured with required -number of rx and tx queues after connection of virtio device. Manual +number of Rx and Tx queues after connection of virtio device. Manual configuration of ``n_rxq`` is not supported because OVS will work properly only if ``n_rxq`` will match number of queues configured in QEMU. -A least 2 PMDs should be configured for the vswitch when using multiqueue. +A least two PMDs should be configured for the vswitch when using multiqueue. Using a single PMD will cause traffic to be enqueued to the same vhost queue rather than being distributed among different vhost queues for a vhost-user interface. If traffic destined for a VM configured with multiqueue arrives to the vswitch -via a physical DPDK port, then the number of rxqs should also be set to at -least 2 for that physical DPDK port. This is required to increase the +via a physical DPDK port, then the number of Rx queues should also be set to at +least two for that physical DPDK port. This is required to increase the probability that a different PMD will handle the multiqueue transmission to the guest using a different vhost queue. From patchwork Mon Apr 16 14:30:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 898652 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="G72eg6FS"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40PrSc1xzTz9s1l for ; Tue, 17 Apr 2018 00:32:32 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id E0A13E95; Mon, 16 Apr 2018 14:30:45 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 50EDCDCF for ; Mon, 16 Apr 2018 14:30:43 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from catfish.maple.relay.mailchannels.net (catfish.maple.relay.mailchannels.net [23.83.214.32]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 29156682 for ; Mon, 16 Apr 2018 14:30:41 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 12FA7121461; Mon, 16 Apr 2018 14:30:39 +0000 (UTC) Received: from one.mxroute.com (unknown [100.96.14.60]) (Authenticated sender: 5xi41l16bi) by relay.mailchannels.net (Postfix) with ESMTPA id 5C748121883; Mon, 16 Apr 2018 14:30:38 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from one.mxroute.com (one-outgoing.mxroute.com [172.18.56.151]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.14.1); Mon, 16 Apr 2018 14:30:39 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: 5xi41l16bi|x-authuser|stephen@that.guru X-MailChannels-Auth-Id: 5xi41l16bi X-Arithmetic-Bitter: 49a3bdd8073f2789_1523889038686_4085092112 X-MC-Loop-Signature: 1523889038685:1889046088 X-MC-Ingress-Time: 1523889038685 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=UmLk6nwv3lYHcpZazOdxLCSYGViXJcKL4R54vRFlAWY=; b=G72eg6FSl3Pv9jnaI2TpF3FAer NFAwi+dv/32cSQwipVCxdhEBVFAmuuJvy1s0YRjnFoofrFyxf6Kjit3ahVsDakDp0Evvw3x56QSLA 6WWW0YwYFrZIHGgTu6UUtPLaI0BwS3TA1KoxU2bp9D1KCcMTgwLflQNPrcarMHInQ6YO3FmpOP97g SJ62F4FQwg/pRZwD7+WTsxI868IDfoGp9X9cobruqvxM6EjITP9tLph7x4A1VQVRkUKkwLwai0bUe Wcp0QmXJK5gkJdUS0mku23O8TzBbn8ZVwaPWJJAN2J12fg0Im4jTrg/RRA5kCRmf6MAwhM7M64SFM bfgrCy5Q==; From: Stephen Finucane To: dev@openvswitch.org Date: Mon, 16 Apr 2018 15:30:20 +0100 Message-Id: <20180416143026.24561-4-stephen@that.guru> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180416143026.24561-1-stephen@that.guru> References: <20180416143026.24561-1-stephen@that.guru> X-AuthUser: stephen@that.guru X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v2 3/9] doc: Move additional sections to "physical ports" doc X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org The "hotplugging", "flow control", and "Rx checksum offload" sections only apply to 'dpdk' ports and are too detailed to include in a high-level howto. Move them, reworking some aspects of this in the process. Signed-off-by: Stephen Finucane --- v2: - Split out vdev into its own document (and patch) - Add cross-references from DPDK howto guide to detailed topic guides - Add flow control --- Documentation/howto/dpdk.rst | 103 ++++++------------------------------ Documentation/topics/dpdk/index.rst | 11 ++-- Documentation/topics/dpdk/phy.rst | 86 ++++++++++++++++++++++++++++++ 3 files changed, 110 insertions(+), 90 deletions(-) diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index 388728363..531612880 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -57,8 +57,12 @@ usage is suggested:: $ ovs-vsctl add-port br0 dpdk-p1 -- set Interface dpdk-p1 type=dpdk \ options:dpdk-devargs="class=eth,mac=00:11:22:33:44:55:02" -Note: such syntax won't support hotplug. The hotplug is supposed to work with -future DPDK release, v18.05. +.. important:: + + Hotplugging physical interfaces is not supported using the above syntax. + This is expected to change with the release of DPDK v18.05. For information + on hotplugging physical interfaces, you should instead refer to + :ref:`port-hotplug`. After the DPDK ports get added to switch, a polling thread continuously polls DPDK devices and consumes 100% of the core, as can be checked from ``top`` and @@ -123,34 +127,6 @@ To clear the ingress policer configuration from the port:: Refer to vswitch.xml for more details on ingress-policer. -Flow Control ------------- - -Flow control can be enabled only on DPDK physical ports. To enable flow control -support at tx side while adding a port, run:: - - $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \ - options:dpdk-devargs=0000:01:00.0 options:tx-flow-ctrl=true - -Similarly, to enable rx flow control, run:: - - $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \ - options:dpdk-devargs=0000:01:00.0 options:rx-flow-ctrl=true - -To enable flow control auto-negotiation, run:: - - $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \ - options:dpdk-devargs=0000:01:00.0 options:flow-ctrl-autoneg=true - -To turn ON the tx flow control at run time for an existing port, run:: - - $ ovs-vsctl set Interface dpdk-p0 options:tx-flow-ctrl=true - -The flow control parameters can be turned off by setting ``false`` to the -respective parameter. To disable the flow control at tx side, run:: - - $ ovs-vsctl set Interface dpdk-p0 options:tx-flow-ctrl=false - pdump ----- @@ -236,16 +212,6 @@ largest frame size supported by Fortville NIC using the DPDK i40e driver, but larger frames and other DPDK NIC drivers may be supported. These cases are common for use cases involving East-West traffic only. -Rx Checksum Offload -------------------- - -By default, DPDK physical ports are enabled with Rx checksum offload. - -Rx checksum offload can offer performance improvement only for tunneling -traffic in OVS-DPDK because the checksum validation of tunnel packets is -offloaded to the NIC. Also enabling Rx checksum may slightly reduce the -performance of non-tunnel traffic, specifically for smaller size packet. - .. _extended-statistics: Extended & Custom Statistics @@ -278,52 +244,6 @@ Note about "Extended Statistics": vHost ports supports only partial statistics. RX packet size based counter are only supported and doesn't include TX packet size counters. -.. _port-hotplug: - -Port Hotplug ------------- - -OVS supports port hotplugging, allowing the use of ports that were not bound -to DPDK when vswitchd was started. -In order to attach a port, it has to be bound to DPDK using the -``dpdk_nic_bind.py`` script:: - - $ $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio 0000:01:00.0 - -Then it can be attached to OVS:: - - $ ovs-vsctl add-port br0 dpdkx -- set Interface dpdkx type=dpdk \ - options:dpdk-devargs=0000:01:00.0 - -Detaching will be performed while processing del-port command:: - - $ ovs-vsctl del-port dpdkx - -Sometimes, the del-port command may not detach the device. -Detaching can be confirmed by the appearance of an INFO log. -For example:: - - INFO|Device '0000:04:00.1' has been detached - -If the log is not seen, then the port can be detached using:: - -$ ovs-appctl netdev-dpdk/detach 0000:01:00.0 - -Detaching can be confirmed by console output:: - - Device '0000:04:00.1' has been detached - -.. warning:: - Detaching should not be done if a device is known to be non-detachable, as - this may cause the device to behave improperly when added back with - add-port. The Chelsio Terminator adapters which use the cxgbe driver seem - to be an example of this behavior; check the driver documentation if this - is suspected. - -This feature does not work with some NICs. -For more information please refer to the `DPDK Port Hotplug Framework -`__. - .. _vdev-support: Vdev Support @@ -653,3 +573,14 @@ devices to bridge ``br0``. Once complete, follow the below steps: Check traffic on multiple queues:: $ cat /proc/interrupts | grep virtio + +Further Reading +--------------- + +More detailed information can be found in the :doc:`DPDK topics section +` of the documentation. These guides are listed below. + +.. NOTE(stephenfin): Remember to keep this in sync with topics/dpdk/index + +.. include:: ../topics/dpdk/index.rst + :start-line: 30 diff --git a/Documentation/topics/dpdk/index.rst b/Documentation/topics/dpdk/index.rst index dfde88377..4b4dc119a 100644 --- a/Documentation/topics/dpdk/index.rst +++ b/Documentation/topics/dpdk/index.rst @@ -25,10 +25,13 @@ The DPDK Datapath ================= +.. NOTE(stephenfin): Part of this doc is included in 'howto/dpdk'. Update that +.. if you modify this document + .. toctree:: :maxdepth: 2 - phy - vhost-user - ring - pmd + /topics/dpdk/phy + /topics/dpdk/vhost-user + /topics/dpdk/ring + /topics/dpdk/pmd diff --git a/Documentation/topics/dpdk/phy.rst b/Documentation/topics/dpdk/phy.rst index ad191dad0..929f394ea 100644 --- a/Documentation/topics/dpdk/phy.rst +++ b/Documentation/topics/dpdk/phy.rst @@ -125,3 +125,89 @@ utilize is a requirement in order to deliver the high-performance possible with DPDK acceleration. It is possible to configure multiple Rx queues for ``dpdk`` ports, thus ensuring this is not a bottleneck for performance. For information on configuring PMD threads, refer to :doc:`pmd`. + +Flow Control +------------ + +Flow control can be enabled only on DPDK physical ports. To enable flow control +support at Tx side while adding a port, run:: + + $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \ + options:dpdk-devargs=0000:01:00.0 options:tx-flow-ctrl=true + +Similarly, to enable Rx flow control, run:: + + $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \ + options:dpdk-devargs=0000:01:00.0 options:rx-flow-ctrl=true + +To enable flow control auto-negotiation, run:: + + $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \ + options:dpdk-devargs=0000:01:00.0 options:flow-ctrl-autoneg=true + +To turn on the Tx flow control at run time for an existing port, run:: + + $ ovs-vsctl set Interface dpdk-p0 options:tx-flow-ctrl=true + +The flow control parameters can be turned off by setting ``false`` to the +respective parameter. To disable the flow control at Tx side, run:: + + $ ovs-vsctl set Interface dpdk-p0 options:tx-flow-ctrl=false + +Rx Checksum Offload +------------------- + +By default, DPDK physical ports are enabled with Rx checksum offload. + +Rx checksum offload can offer performance improvement only for tunneling +traffic in OVS-DPDK because the checksum validation of tunnel packets is +offloaded to the NIC. Also enabling Rx checksum may slightly reduce the +performance of non-tunnel traffic, specifically for smaller size packet. + +.. _port-hotplug: + +Hotplugging +----------- + +OVS supports port hotplugging, allowing the use of physical ports that were not +bound to DPDK when ovs-vswitchd was started. + +.. warning:: + + This feature is not compatible with all NICs. Refer to vendor documentation + for more information. + +.. important:: + + Ports must be bound to DPDK. Refer to :ref:`dpdk-binding-nics` for more + information. + +To *hotplug* a port, simply add it like any other port:: + + $ ovs-vsctl add-port br0 dpdkx -- set Interface dpdkx type=dpdk \ + options:dpdk-devargs=0000:01:00.0 + +Ports can be detached using the ``del-port`` command:: + + $ ovs-vsctl del-port dpdkx + +This should both delete the port and detach the device. If successful, you +should see an ``INFO`` log. For example:: + + INFO|Device '0000:04:00.1' has been detached + +If the log is not seen then the port can be detached like so:: + + $ ovs-appctl netdev-dpdk/detach 0000:01:00.0 + +.. warning:: + + Detaching should not be done if a device is known to be non-detachable, as + this may cause the device to behave improperly when added back with + add-port. The Chelsio Terminator adapters which use the cxgbe driver seem + to be an example of this behavior; check the driver documentation if this + is suspected. + +For more information please refer to the `DPDK Port Hotplug Framework`__. + +__ http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html#hotplug From patchwork Mon Apr 16 14:30:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 898650 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="dWzXUQIZ"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40PrQZ4rLPz9s1l for ; Tue, 17 Apr 2018 00:30:46 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 130F6E0B; Mon, 16 Apr 2018 14:30:44 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 8D89DD6F for ; Mon, 16 Apr 2018 14:30:42 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from catfish.maple.relay.mailchannels.net (catfish.maple.relay.mailchannels.net [23.83.214.32]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 3CE7767B for ; Mon, 16 Apr 2018 14:30:41 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 72BD93E23FD; Mon, 16 Apr 2018 14:30:40 +0000 (UTC) Received: from one.mxroute.com (unknown [100.96.11.60]) (Authenticated sender: 5xi41l16bi) by relay.mailchannels.net (Postfix) with ESMTPA id A104D3E2200; Mon, 16 Apr 2018 14:30:39 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from one.mxroute.com (one-outgoing.mxroute.com [172.18.58.245]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.14.1); Mon, 16 Apr 2018 14:30:40 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: 5xi41l16bi|x-authuser|stephen@that.guru X-MailChannels-Auth-Id: 5xi41l16bi X-Cellar-Industry: 372854ac538966df_1523889040023_3179597741 X-MC-Loop-Signature: 1523889040022:4101370727 X-MC-Ingress-Time: 1523889040022 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=4ZmauqsLm/gKUJPO53/Ed88DLX4YZIUM7yIuEVxqLLQ=; b=dWzXUQIZImKfnrWx0IG4Vbfzik LYSugGsvytPSM3hSmpc/50ls1hZts2LoZPXQniqMerM9oO73NpZr7RrR/H3B1bNqYtZFRIr1lhIMT Ap4X7cphR8OI6O0K8vPo+1qfsiEskBVcwmhCBuYKEq6nIJ2uE+szrMqSAhgjoDcWEbGsvQ8mqmcXA /DfTPvev+pG7sgi+MyhpjsiI8+Sk0MEI/e4rmUylwwdo6PlKhJr31GlqrezNYePVQtmeVPwtLixzx gpk72wypk6qaSq9qbZKUHefeXbVRqAZHcs7v0n762j3KOuZyakXXRx1L6TrBN5GCcKmdvqiQrHO8Z 8wJAx/4Q==; From: Stephen Finucane To: dev@openvswitch.org Date: Mon, 16 Apr 2018 15:30:21 +0100 Message-Id: <20180416143026.24561-5-stephen@that.guru> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180416143026.24561-1-stephen@that.guru> References: <20180416143026.24561-1-stephen@that.guru> X-AuthUser: stephen@that.guru X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v2 4/9] doc: Add "vdev" topic document X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org These are separate things from physical, ring and vhost-user interfaces and deserve their own documents. A couple of small typos are fixed along the way. Signed-off-by: Stephen Finucane --- Documentation/howto/dpdk.rst | 29 ------------------ Documentation/topics/dpdk/index.rst | 1 + Documentation/topics/dpdk/vdev.rst | 59 +++++++++++++++++++++++++++++++++++++ 3 files changed, 60 insertions(+), 29 deletions(-) create mode 100644 Documentation/topics/dpdk/vdev.rst diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index 531612880..9cb76b7e1 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -244,35 +244,6 @@ Note about "Extended Statistics": vHost ports supports only partial statistics. RX packet size based counter are only supported and doesn't include TX packet size counters. -.. _vdev-support: - -Vdev Support ------------- - -DPDK provides drivers for both physical and virtual devices. Physical DPDK -devices are added to OVS by specifying a valid PCI address in 'dpdk-devargs'. -Virtual DPDK devices which do not have PCI addresses can be added using a -different format for 'dpdk-devargs'. - -Typically, the format expected is 'eth_' where 'x' is a -unique identifier of your choice for the given port. - -For example to add a dpdk port that uses the 'null' DPDK PMD driver:: - - $ ovs-vsctl add-port br0 null0 -- set Interface null0 type=dpdk \ - options:dpdk-devargs=eth_null0 - -Similarly, to add a dpdk port that uses the 'af_packet' DPDK PMD driver:: - - $ ovs-vsctl add-port br0 myeth0 -- set Interface myeth0 type=dpdk \ - options:dpdk-devargs=eth_af_packet0,iface=eth0 - -More information on the different types of virtual DPDK PMDs can be found in -the `DPDK documentation -`__. - -Note: Not all DPDK virtual PMD drivers have been tested and verified to work. - EMC Insertion Probability ------------------------- By default 1 in every 100 flows are inserted into the Exact Match Cache (EMC). diff --git a/Documentation/topics/dpdk/index.rst b/Documentation/topics/dpdk/index.rst index 4b4dc119a..c1e6ea78c 100644 --- a/Documentation/topics/dpdk/index.rst +++ b/Documentation/topics/dpdk/index.rst @@ -34,4 +34,5 @@ The DPDK Datapath /topics/dpdk/phy /topics/dpdk/vhost-user /topics/dpdk/ring + /topics/dpdk/vdev /topics/dpdk/pmd diff --git a/Documentation/topics/dpdk/vdev.rst b/Documentation/topics/dpdk/vdev.rst new file mode 100644 index 000000000..1b00ac573 --- /dev/null +++ b/Documentation/topics/dpdk/vdev.rst @@ -0,0 +1,59 @@ +.. + Copyright 2018, Red Hat, Inc. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +==================== +DPDK Virtual Devices +==================== + +DPDK provides drivers for both physical and virtual devices. Physical DPDK +devices are added to OVS by specifying a valid PCI address in ``dpdk-devargs``. +Virtual DPDK devices which do not have PCI addresses can be added using a +different format for ``dpdk-devargs``. + +.. note:: + + Not all DPDK virtual PMD drivers have been tested and verified to work. + +Quick Example +------------- + +To add a virtual ``dpdk`` devices, the ``dpdk-devargs`` argument should be of +the format ``eth_``, where ``x``' is a unique identifier of +your choice for the given port. For example to add a ``dpdk`` port that uses +the ``null`` DPDK PMD driver, run:: + + $ ovs-vsctl add-port br0 null0 -- set Interface null0 type=dpdk \ + options:dpdk-devargs=eth_null0 + +Similarly, to add a ``dpdk`` port that uses the ``af_packet`` DPDK PMD driver, +run:: + + $ ovs-vsctl add-port br0 myeth0 -- set Interface myeth0 type=dpdk \ + options:dpdk-devargs=eth_af_packet0,iface=eth0 + +More information on the different types of virtual DPDK PMDs can be found in +the `DPDK documentation`__. + +__ http://dpdk.org/doc/guides/nics/overview.html From patchwork Mon Apr 16 14:30:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 898651 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="HetCAoyq"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40PrRV69d5z9s1l for ; Tue, 17 Apr 2018 00:31:34 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 08976E76; Mon, 16 Apr 2018 14:30:45 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id D6BCCD86 for ; Mon, 16 Apr 2018 14:30:42 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from glass.birch.relay.mailchannels.net (glass.birch.relay.mailchannels.net [23.83.209.70]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 81D79680 for ; Mon, 16 Apr 2018 14:30:41 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id A4A5B4071A; Mon, 16 Apr 2018 14:30:40 +0000 (UTC) Received: from one.mxroute.com (unknown [100.96.12.37]) (Authenticated sender: 5xi41l16bi) by relay.mailchannels.net (Postfix) with ESMTPA id 20C4A41C01; Mon, 16 Apr 2018 14:30:40 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from one.mxroute.com (one-outgoing.mxroute.com [172.19.36.130]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.14.1); Mon, 16 Apr 2018 14:30:40 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: 5xi41l16bi|x-authuser|stephen@that.guru X-MailChannels-Auth-Id: 5xi41l16bi X-Unite-Irritate: 636f1ed65af1f9a7_1523889040465_2366022619 X-MC-Loop-Signature: 1523889040465:1684535447 X-MC-Ingress-Time: 1523889040465 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=6Ff/Q5uBoiOhNSDf+ZuMLLhwO1gxNlaDaQNrHvUCr7k=; b=HetCAoyqPPDB5U4+7T/mC4VyHy DynxZQSWIpKumcCMi4tmiizCAkpJwf/ot1KDH9ZoWj9pOzq7H43HqLxCk76J7ZZdwf1ZKwt/C+lBn XyG8nLcyHGBSbBoxu1tm6LCTW8tl8ict848X14JGW2ZVzhKm6Hs1qkLZNyLFfEj41LE9BEbxVbXR+ SR2GJMskMwB2n6L3vekyCAGWUUZv19heWg3fNdYOaacqe/IefQsMvilpP+ga8421J6I0LSI0I/cHl Aao3YBQJ310GiJIjKakabTU4SQJgPG4C35E3aSwVVbChOSGxJzFMtcbzku8/RGOdvSXoKmcZwd39p otujJIcA==; From: Stephen Finucane To: dev@openvswitch.org Date: Mon, 16 Apr 2018 15:30:22 +0100 Message-Id: <20180416143026.24561-6-stephen@that.guru> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180416143026.24561-1-stephen@that.guru> References: <20180416143026.24561-1-stephen@that.guru> X-AuthUser: stephen@that.guru X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v2 5/9] doc: Move "QoS" guide to its own document X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Again, this stuff is too detailed for a high-level howto. Signed-off-by: Stephen Finucane --- v2: - Keep flow control section in 'phy' document --- Documentation/howto/dpdk.rst | 42 -------------------- Documentation/topics/dpdk/index.rst | 1 + Documentation/topics/dpdk/phy.rst | 2 + Documentation/topics/dpdk/qos.rst | 76 +++++++++++++++++++++++++++++++++++++ 4 files changed, 79 insertions(+), 42 deletions(-) create mode 100644 Documentation/topics/dpdk/qos.rst diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index 9cb76b7e1..3e04d8627 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -85,48 +85,6 @@ To stop ovs-vswitchd & delete bridge, run:: $ ovs-appctl -t ovsdb-server exit $ ovs-vsctl del-br br0 -QoS ---- - -Assuming you have a vhost-user port transmitting traffic consisting of packets -of size 64 bytes, the following command would limit the egress transmission -rate of the port to ~1,000,000 packets per second:: - - $ ovs-vsctl set port vhost-user0 qos=@newqos -- \ - --id=@newqos create qos type=egress-policer other-config:cir=46000000 \ - other-config:cbs=2048` - -To examine the QoS configuration of the port, run:: - - $ ovs-appctl -t ovs-vswitchd qos/show vhost-user0 - -To clear the QoS configuration from the port and ovsdb, run:: - - $ ovs-vsctl destroy QoS vhost-user0 -- clear Port vhost-user0 qos - -Refer to vswitch.xml for more details on egress-policer. - -Rate Limiting --------------- - -Here is an example on Ingress Policing usage. Assuming you have a vhost-user -port receiving traffic consisting of packets of size 64 bytes, the following -command would limit the reception rate of the port to ~1,000,000 packets per -second:: - - $ ovs-vsctl set interface vhost-user0 ingress_policing_rate=368000 \ - ingress_policing_burst=1000` - -To examine the ingress policer configuration of the port:: - - $ ovs-vsctl list interface vhost-user0 - -To clear the ingress policer configuration from the port:: - - $ ovs-vsctl set interface vhost-user0 ingress_policing_rate=0 - -Refer to vswitch.xml for more details on ingress-policer. - pdump ----- diff --git a/Documentation/topics/dpdk/index.rst b/Documentation/topics/dpdk/index.rst index c1e6ea78c..d3c1e2099 100644 --- a/Documentation/topics/dpdk/index.rst +++ b/Documentation/topics/dpdk/index.rst @@ -36,3 +36,4 @@ The DPDK Datapath /topics/dpdk/ring /topics/dpdk/vdev /topics/dpdk/pmd + /topics/dpdk/qos diff --git a/Documentation/topics/dpdk/phy.rst b/Documentation/topics/dpdk/phy.rst index 929f394ea..581984b63 100644 --- a/Documentation/topics/dpdk/phy.rst +++ b/Documentation/topics/dpdk/phy.rst @@ -126,6 +126,8 @@ DPDK acceleration. It is possible to configure multiple Rx queues for ``dpdk`` ports, thus ensuring this is not a bottleneck for performance. For information on configuring PMD threads, refer to :doc:`pmd`. +.. _dpdk-phy-flow-control: + Flow Control ------------ diff --git a/Documentation/topics/dpdk/qos.rst b/Documentation/topics/dpdk/qos.rst new file mode 100644 index 000000000..b42140515 --- /dev/null +++ b/Documentation/topics/dpdk/qos.rst @@ -0,0 +1,76 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +======================== +Quality of Service (QoS) +======================== + +It is possible to apply both ingress and egress limiting when using the DPDK +datapath. These are referred to as *QoS* and *Rate Limiting*, respectively. + +QoS (Egress Policing) +--------------------- + +Assuming you have a :doc:`vhost-user port ` transmitting traffic +consisting of packets of size 64 bytes, the following command would limit the +egress transmission rate of the port to ~1,000,000 packets per second:: + + $ ovs-vsctl set port vhost-user0 qos=@newqos -- \ + --id=@newqos create qos type=egress-policer other-config:cir=46000000 \ + other-config:cbs=2048` + +To examine the QoS configuration of the port, run:: + + $ ovs-appctl -t ovs-vswitchd qos/show vhost-user0 + +To clear the QoS configuration from the port and ovsdb, run:: + + $ ovs-vsctl destroy QoS vhost-user0 -- clear Port vhost-user0 qos + +Refer to ``vswitch.xml`` for more details on egress policer. + +Rate Limiting (Ingress Policing) +-------------------------------- + +Assuming you have a :doc:`vhost-user port ` receiving traffic +consisting of packets of size 64 bytes, the following command would limit the +reception rate of the port to ~1,000,000 packets per second:: + + $ ovs-vsctl set interface vhost-user0 ingress_policing_rate=368000 \ + ingress_policing_burst=1000` + +To examine the ingress policer configuration of the port:: + + $ ovs-vsctl list interface vhost-user0 + +To clear the ingress policer configuration from the port:: + + $ ovs-vsctl set interface vhost-user0 ingress_policing_rate=0 + +Refer to ``vswitch.xml`` for more details on ingress policer. + +Flow Control +------------ + +Flow control is available for :doc:`DPDK physical ports `. For more +information, refer to :ref:`dpdk-phy-flow-control`. From patchwork Mon Apr 16 14:30:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 898659 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="oy6D0ha5"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40Prb04zyMz9s1l for ; Tue, 17 Apr 2018 00:38:04 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id AA469EF7; Mon, 16 Apr 2018 14:30:55 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 7E460EA4 for ; Mon, 16 Apr 2018 14:30:52 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from anteater.ash.relay.mailchannels.net (anteater.ash.relay.mailchannels.net [23.83.222.3]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 5A28A477 for ; Mon, 16 Apr 2018 14:30:49 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 9C426280946; Mon, 16 Apr 2018 14:30:43 +0000 (UTC) Received: from one.mxroute.com (unknown [100.96.13.37]) (Authenticated sender: 5xi41l16bi) by relay.mailchannels.net (Postfix) with ESMTPA id 0CDCC280A05; Mon, 16 Apr 2018 14:30:42 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from one.mxroute.com (one-outgoing.mxroute.com [172.19.61.140]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.14.1); Mon, 16 Apr 2018 14:30:43 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: 5xi41l16bi|x-authuser|stephen@that.guru X-MailChannels-Auth-Id: 5xi41l16bi X-Chief-Print: 4198c83575087578_1523889043536_398652349 X-MC-Loop-Signature: 1523889043536:2138522491 X-MC-Ingress-Time: 1523889043535 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=MDJfFCuXuGR1dW0k9p7o9GJnn8SGWwwtWjg6KMADC+E=; b=oy6D0ha58d5VwLJ9onRzINCAfw hIJHkeiSu8u5z4O0ATZL5vUwBuUqba5AwpY+L+sI//M9odInat/ec6VtVlH/IdpxbLrBORfiqeuGU uCrW562sEE8sY0SjrvvmdpOwBSz8r+RNH+F7WkjBdobMOQ9/J/G7b2gvwiV8n6PUMeSL3GydDWdnE cMHm3JloKf54HLm91QlUegSa6aZVG+o1iGjqpl2a9Ly+EncDoPNtWjLU0UfWRBp66y+o8I4+o7ytl 0ZMGQEAwaMAKGrHGU2yM3VVhI5Q6reVjBMmma0O1iObn5jIEDbiGUplr1tejLTJSmGV50E5XfeTR7 DefvCuOw==; From: Stephen Finucane To: dev@openvswitch.org Date: Mon, 16 Apr 2018 15:30:23 +0100 Message-Id: <20180416143026.24561-7-stephen@that.guru> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180416143026.24561-1-stephen@that.guru> References: <20180416143026.24561-1-stephen@that.guru> X-AuthUser: stephen@that.guru X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v2 6/9] doc: Add "bridge" topic document X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This details configuration steps that apply to the entire bridge, rather than individual ports. Signed-off-by: Stephen Finucane --- v2: - Cross-reference this document from all interface documents --- Documentation/howto/dpdk.rst | 60 ------------------ Documentation/topics/dpdk/bridge.rst | 104 +++++++++++++++++++++++++++++++ Documentation/topics/dpdk/index.rst | 1 + Documentation/topics/dpdk/phy.rst | 5 ++ Documentation/topics/dpdk/ring.rst | 5 ++ Documentation/topics/dpdk/vdev.rst | 5 ++ Documentation/topics/dpdk/vhost-user.rst | 5 ++ 7 files changed, 125 insertions(+), 60 deletions(-) create mode 100644 Documentation/topics/dpdk/bridge.rst diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index 3e04d8627..b3cba19b2 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -170,66 +170,6 @@ largest frame size supported by Fortville NIC using the DPDK i40e driver, but larger frames and other DPDK NIC drivers may be supported. These cases are common for use cases involving East-West traffic only. -.. _extended-statistics: - -Extended & Custom Statistics ----------------------------- - -DPDK Extended Statistics API allows PMD to expose a unique set of statistics. -The Extended Statistics are implemented and supported only for DPDK physical -and vHost ports. Custom statistics are a dynamic set of counters which can -vary depending on the driver. Those statistics are implemented -for DPDK physical ports and contain all "dropped", "error" and "management" -counters from XSTATS. XSTATS counters list can be found here: -`__. - -To enable statistics, you have to enable OpenFlow 1.4 support for OVS. -Configure bridge br0 to support OpenFlow version 1.4:: - - $ ovs-vsctl set bridge br0 datapath_type=netdev \ - protocols=OpenFlow10,OpenFlow11,OpenFlow12,OpenFlow13,OpenFlow14 - -Check the OVSDB protocols column in the bridge table if OpenFlow 1.4 support -is enabled for OVS:: - - $ ovsdb-client dump Bridge protocols - -Query the port statistics by explicitly specifying -O OpenFlow14 option:: - - $ ovs-ofctl -O OpenFlow14 dump-ports br0 - -Note about "Extended Statistics": vHost ports supports only partial -statistics. RX packet size based counter are only supported and -doesn't include TX packet size counters. - -EMC Insertion Probability -------------------------- -By default 1 in every 100 flows are inserted into the Exact Match Cache (EMC). -It is possible to change this insertion probability by setting the -``emc-insert-inv-prob`` option:: - - $ ovs-vsctl --no-wait set Open_vSwitch . other_config:emc-insert-inv-prob=N - -where: - -``N`` - is a positive integer representing the inverse probability of insertion ie. - on average 1 in every N packets with a unique flow will generate an EMC - insertion. - -If ``N`` is set to 1, an insertion will be performed for every flow. If set to -0, no insertions will be performed and the EMC will effectively be disabled. - -With default ``N`` set to 100, higher megaflow hits will occur initially -as observed with pmd stats:: - - $ ovs-appctl dpif-netdev/pmd-stats-show - -For certain traffic profiles with many parallel flows, it's recommended to set -``N`` to '0' to achieve higher forwarding performance. - -For more information on the EMC refer to :doc:`/intro/install/dpdk` . - .. _dpdk-ovs-in-guest: OVS with DPDK Inside VMs diff --git a/Documentation/topics/dpdk/bridge.rst b/Documentation/topics/dpdk/bridge.rst new file mode 100644 index 000000000..63f8a62de --- /dev/null +++ b/Documentation/topics/dpdk/bridge.rst @@ -0,0 +1,104 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +============ +DPDK Bridges +============ + +The DPDK datapath requires specially configured bridge(s) in order to utilize +DPDK-backed :doc:`physical ` and `virtual ` ports. + +Quick Example +------------- + +This example demonstrates how to add a bridge using the DPDK datapath:: + + $ ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev + +This assumes Open vSwitch has been built with DPDK support. Refer to +:doc:`/intro/install/dpdk` for more information. + +.. _extended-statistics: + +Extended & Custom Statistics +---------------------------- + +The DPDK Extended Statistics API allows PMDs to expose a unique set of +statistics. The Extended Statistics are implemented and supported only for +DPDK physical and vHost ports. Custom statistics are a dynamic set of counters +which can vary depending on the driver. Those statistics are implemented for +DPDK physical ports and contain all "dropped", "error" and "management" +counters from ``XSTATS``. A list of all ``XSTATS`` counters can be found +`here`__. + +__ https://wiki.opnfv.org/display/fastpath/Collectd+Metrics+and+Events + +.. note:: + + vHost ports only support RX packet size-based counters. TX packet size + counters are not available. + +To enable statistics, you have to enable OpenFlow 1.4 support for OVS. To +configure a bridge, ``br0``, to support OpenFlow version 1.4, run:: + + $ ovs-vsctl set bridge br0 datapath_type=netdev \ + protocols=OpenFlow10,OpenFlow11,OpenFlow12,OpenFlow13,OpenFlow14 + +Once configured, check the OVSDB protocols column in the bridge table to ensure +OpenFlow 1.4 support is enabled:: + + $ ovsdb-client dump Bridge protocols + +You can also query the port statistics by explicitly specifying the ``-O +OpenFlow14`` option:: + + $ ovs-ofctl -O OpenFlow14 dump-ports br0 + +EMC Insertion Probability +------------------------- + +By default 1 in every 100 flows is inserted into the Exact Match Cache (EMC). +It is possible to change this insertion probability by setting the +``emc-insert-inv-prob`` option:: + + $ ovs-vsctl --no-wait set Open_vSwitch . other_config:emc-insert-inv-prob=N + +where: + +``N`` + A positive integer representing the inverse probability of insertion, i.e. on + average 1 in every ``N`` packets with a unique flow will generate an EMC + insertion. + +If ``N`` is set to 1, an insertion will be performed for every flow. If set to +0, no insertions will be performed and the EMC will effectively be disabled. + +With default ``N`` set to 100, higher megaflow hits will occur initially as +observed with pmd stats:: + + $ ovs-appctl dpif-netdev/pmd-stats-show + +For certain traffic profiles with many parallel flows, it's recommended to set +``N`` to '0' to achieve higher forwarding performance. + +For more information on the EMC refer to :doc:`/intro/install/dpdk` . diff --git a/Documentation/topics/dpdk/index.rst b/Documentation/topics/dpdk/index.rst index d3c1e2099..fd7cf38b5 100644 --- a/Documentation/topics/dpdk/index.rst +++ b/Documentation/topics/dpdk/index.rst @@ -31,6 +31,7 @@ The DPDK Datapath .. toctree:: :maxdepth: 2 + /topics/dpdk/bridge /topics/dpdk/phy /topics/dpdk/vhost-user /topics/dpdk/ring diff --git a/Documentation/topics/dpdk/phy.rst b/Documentation/topics/dpdk/phy.rst index 581984b63..c0931b2c4 100644 --- a/Documentation/topics/dpdk/phy.rst +++ b/Documentation/topics/dpdk/phy.rst @@ -30,6 +30,11 @@ DPDK Physical Ports The netdev datapath allows attaching of DPDK-backed physical interfaces in order to provide high-performance ingress/egress from the host. +.. important:: + + To use any DPDK-backed interface, you must ensure your bridge is configured + correctly. For more information, refer to :doc:`bridge`. + .. versionchanged:: 2.7.0 Before Open vSwitch 2.7.0, it was necessary to prefix port names with a diff --git a/Documentation/topics/dpdk/ring.rst b/Documentation/topics/dpdk/ring.rst index 8e8fb6088..9ef1dc3a5 100644 --- a/Documentation/topics/dpdk/ring.rst +++ b/Documentation/topics/dpdk/ring.rst @@ -36,6 +36,11 @@ The DPDK datapath provides DPDK-backed ring ports that are implemented using DPDK's ``librte_ring`` library. For more information on this library, refer to the `DPDK documentation`_. +.. important:: + + To use any DPDK-backed interface, you must ensure your bridge is configured + correctly. For more information, refer to :doc:`bridge`. + Quick Example ------------- diff --git a/Documentation/topics/dpdk/vdev.rst b/Documentation/topics/dpdk/vdev.rst index 1b00ac573..19b41a328 100644 --- a/Documentation/topics/dpdk/vdev.rst +++ b/Documentation/topics/dpdk/vdev.rst @@ -32,6 +32,11 @@ devices are added to OVS by specifying a valid PCI address in ``dpdk-devargs``. Virtual DPDK devices which do not have PCI addresses can be added using a different format for ``dpdk-devargs``. +.. important:: + + To use any DPDK-backed interface, you must ensure your bridge is configured + correctly. For more information, refer to :doc:`bridge`. + .. note:: Not all DPDK virtual PMD drivers have been tested and verified to work. diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst index 6f794f296..168d5c5ce 100644 --- a/Documentation/topics/dpdk/vhost-user.rst +++ b/Documentation/topics/dpdk/vhost-user.rst @@ -29,6 +29,11 @@ The DPDK datapath provides DPDK-backed vHost user ports as a primary way to interact with guests. For more information on vHost User, refer to the `QEMU documentation`_ on same. +.. important:: + + To use any DPDK-backed interface, you must ensure your bridge is configured + correctly. For more information, refer to :doc:`bridge`. + Quick Example ------------- From patchwork Mon Apr 16 14:30:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 898653 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="BL1raSEB"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40PrTh2Svpz9rx7 for ; Tue, 17 Apr 2018 00:33:28 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id D66F6E9D; Mon, 16 Apr 2018 14:30:46 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id CB06AE92 for ; Mon, 16 Apr 2018 14:30:45 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from catfish.maple.relay.mailchannels.net (catfish.maple.relay.mailchannels.net [23.83.214.32]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 9157A625 for ; Mon, 16 Apr 2018 14:30:44 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id D0EBD5C2673; Mon, 16 Apr 2018 14:30:43 +0000 (UTC) Received: from one.mxroute.com (unknown [100.96.32.20]) (Authenticated sender: 5xi41l16bi) by relay.mailchannels.net (Postfix) with ESMTPA id 364735C2638; Mon, 16 Apr 2018 14:30:43 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from one.mxroute.com (one-outgoing.mxroute.com [172.18.36.240]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.14.1); Mon, 16 Apr 2018 14:30:43 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: 5xi41l16bi|x-authuser|stephen@that.guru X-MailChannels-Auth-Id: 5xi41l16bi X-Belong-Eight: 134d810c15300ae0_1523889043610_578555940 X-MC-Loop-Signature: 1523889043610:1492006210 X-MC-Ingress-Time: 1523889043609 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=CHnt9UbSw91DG3dmNWX52kXN3uFMcStLz7Nmqcj3QA8=; b=BL1raSEBIJOe6dTP9yR8nbi5C8 GJg3X8dDlvAVaBXMBZn6MyQS6oIwj9um+rAy2yaTS/b0Z2LPDlxTKEPuK+OYQyQOtr2qAlGPe5mys WPJPMrfFOIxGsEbMEo8yV72SG5ZB+M3XUT7XrfBaXEoxONQCSIBdXrQOQqcZ57cPiYjnRWRbhyaFf rNtjrC/JWfAR5EmX69GaL84aHDdHUIWToloLSrIbgbhGBTaMknY0h506zSEpWmBe2Zl3wYX0fNdXZ b3NelXMctFv/JnOu0ECuXmo7vVoCfyhUIihb5JauAghm3D1YIOMarXoPEfgsm35fju3uZaA1xDD8V pgyyCLng==; From: Stephen Finucane To: dev@openvswitch.org Date: Mon, 16 Apr 2018 15:30:24 +0100 Message-Id: <20180416143026.24561-8-stephen@that.guru> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180416143026.24561-1-stephen@that.guru> References: <20180416143026.24561-1-stephen@that.guru> X-AuthUser: stephen@that.guru X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v2 7/9] doc: Move "pdump" guide to its own document X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Yet another section that's far too detailed for someone getting started with DPDK in OVS. Split it out. Signed-off-by: Stephen Finucane --- Documentation/howto/dpdk.rst | 39 ---------------------- Documentation/topics/dpdk/index.rst | 1 + Documentation/topics/dpdk/pdump.rst | 65 +++++++++++++++++++++++++++++++++++++ 3 files changed, 66 insertions(+), 39 deletions(-) create mode 100644 Documentation/topics/dpdk/pdump.rst diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index b3cba19b2..1a0fb6df1 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -85,45 +85,6 @@ To stop ovs-vswitchd & delete bridge, run:: $ ovs-appctl -t ovsdb-server exit $ ovs-vsctl del-br br0 -pdump ------ - -pdump allows you to listen on DPDK ports and view the traffic that is passing -on them. To use this utility, one must have libpcap installed on the system. -Furthermore, DPDK must be built with ``CONFIG_RTE_LIBRTE_PDUMP=y`` and -``CONFIG_RTE_LIBRTE_PMD_PCAP=y``. - -.. warning:: - A performance decrease is expected when using a monitoring application like - the DPDK pdump app. - -To use pdump, simply launch OVS as usual, then navigate to the ``app/pdump`` -directory in DPDK, ``make`` the application and run like so:: - - $ sudo ./build/app/dpdk-pdump -- \ - --pdump port=0,queue=0,rx-dev=/tmp/pkts.pcap \ - --server-socket-path=/usr/local/var/run/openvswitch - -The above command captures traffic received on queue 0 of port 0 and stores it -in ``/tmp/pkts.pcap``. Other combinations of port numbers, queues numbers and -pcap locations are of course also available to use. For example, to capture all -packets that traverse port 0 in a single pcap file:: - - $ sudo ./build/app/dpdk-pdump -- \ - --pdump 'port=0,queue=*,rx-dev=/tmp/pkts.pcap,tx-dev=/tmp/pkts.pcap' \ - --server-socket-path=/usr/local/var/run/openvswitch - -``server-socket-path`` must be set to the value of ``ovs_rundir()`` which -typically resolves to ``/usr/local/var/run/openvswitch``. - -Many tools are available to view the contents of the pcap file. Once example is -tcpdump. Issue the following command to view the contents of ``pkts.pcap``:: - - $ tcpdump -r pkts.pcap - -More information on the pdump app and its usage can be found in the `DPDK docs -`__. - Jumbo Frames ------------ diff --git a/Documentation/topics/dpdk/index.rst b/Documentation/topics/dpdk/index.rst index fd7cf38b5..d083d929e 100644 --- a/Documentation/topics/dpdk/index.rst +++ b/Documentation/topics/dpdk/index.rst @@ -38,3 +38,4 @@ The DPDK Datapath /topics/dpdk/vdev /topics/dpdk/pmd /topics/dpdk/qos + /topics/dpdk/pdump diff --git a/Documentation/topics/dpdk/pdump.rst b/Documentation/topics/dpdk/pdump.rst new file mode 100644 index 000000000..f1a408989 --- /dev/null +++ b/Documentation/topics/dpdk/pdump.rst @@ -0,0 +1,65 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +===== +pdump +===== + +pdump allows you to listen on DPDK ports and view the traffic that is passing +on them. To use this utility, one must have libpcap installed on the system. +Furthermore, DPDK must be built with ``CONFIG_RTE_LIBRTE_PDUMP=y`` and +``CONFIG_RTE_LIBRTE_PMD_PCAP=y``. + +.. warning:: + + A performance decrease is expected when using a monitoring application like + the DPDK pdump app. + +To use pdump, simply launch OVS as usual, then navigate to the ``app/pdump`` +directory in DPDK, ``make`` the application and run like so:: + + $ sudo ./build/app/dpdk-pdump -- \ + --pdump port=0,queue=0,rx-dev=/tmp/pkts.pcap \ + --server-socket-path=/usr/local/var/run/openvswitch + +The above command captures traffic received on queue 0 of port 0 and stores it +in ``/tmp/pkts.pcap``. Other combinations of port numbers, queues numbers and +pcap locations are of course also available to use. For example, to capture all +packets that traverse port 0 in a single pcap file:: + + $ sudo ./build/app/dpdk-pdump -- \ + --pdump 'port=0,queue=*,rx-dev=/tmp/pkts.pcap,tx-dev=/tmp/pkts.pcap' \ + --server-socket-path=/usr/local/var/run/openvswitch + +``server-socket-path`` must be set to the value of ``ovs_rundir()`` which +typically resolves to ``/usr/local/var/run/openvswitch``. + +Many tools are available to view the contents of the pcap file. Once example is +tcpdump. Issue the following command to view the contents of ``pkts.pcap``:: + + $ tcpdump -r pkts.pcap + +More information on the pdump app and its usage can be found in the `DPDK +documentation`__. + +__ http://dpdk.org/doc/guides/tools/pdump.html From patchwork Mon Apr 16 14:30:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 898657 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="fyXiPUXr"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40PrYW2PKhz9s1R for ; Tue, 17 Apr 2018 00:36:47 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id D9DA6EEC; Mon, 16 Apr 2018 14:30:53 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id E7E29EB7 for ; Mon, 16 Apr 2018 14:30:49 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from glass.birch.relay.mailchannels.net (glass.birch.relay.mailchannels.net [23.83.209.70]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 00AD6473 for ; Mon, 16 Apr 2018 14:30:48 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 58400280C8A; Mon, 16 Apr 2018 14:30:43 +0000 (UTC) Received: from one.mxroute.com (unknown [100.96.13.37]) (Authenticated sender: 5xi41l16bi) by relay.mailchannels.net (Postfix) with ESMTPA id DB3CF280A3D; Mon, 16 Apr 2018 14:30:42 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from one.mxroute.com (one-outgoing.mxroute.com [172.19.51.240]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.14.1); Mon, 16 Apr 2018 14:30:43 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: 5xi41l16bi|x-authuser|stephen@that.guru X-MailChannels-Auth-Id: 5xi41l16bi X-Bored-Harbor: 69508a272ff69c81_1523889043224_1869664863 X-MC-Loop-Signature: 1523889043224:1455241161 X-MC-Ingress-Time: 1523889043222 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=TLI+pdwRxsST4C8ff8atVQKbws1hG9n/BI1ud1PxXCY=; b=fyXiPUXrOPjoa95x3A8GpyRu4z kfFIXfq+jpwDZ4BHAOXjR+jrmsAiLEfPk8JD74GAB72RJM9tFzgWfZDH/NW3BeM6+YuPEpg/xWNnt Q3xOiAw3YNabyrWCQz8NSsRktPN06Fx6WTx/WLcWjomMAlJWF6STuf4gwXEvk4n+Ak2NBpDf9PGk8 4Lgyrp+5y3QxbMoc5TRLxe69L6wKRDvotStxOjFPPz1Piaz7XN+uOE1T89GfiAV8OF0ktchCTmE4m WMoUi2RhheoXAjZUNjgPiTcxttXMOI57mEtlK4NQpS1DDV3rpHQmUUaV12GrlyguI90xvKlzJCo2o SpzKcVfQ==; From: Stephen Finucane To: dev@openvswitch.org Date: Mon, 16 Apr 2018 15:30:25 +0100 Message-Id: <20180416143026.24561-9-stephen@that.guru> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180416143026.24561-1-stephen@that.guru> References: <20180416143026.24561-1-stephen@that.guru> X-AuthUser: stephen@that.guru X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v2 8/9] doc: Add "jumbo frames" topic document X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org We include refrences from the physical and vhost-user interface guides. Signed-off-by: Stephen Finucane --- v2: - Don't split the document into multiple docs --- Documentation/howto/dpdk.rst | 52 ++-------------------- Documentation/topics/dpdk/index.rst | 1 + Documentation/topics/dpdk/jumbo-frames.rst | 71 ++++++++++++++++++++++++++++++ Documentation/topics/dpdk/phy.rst | 6 +++ Documentation/topics/dpdk/vhost-user.rst | 6 +++ 5 files changed, 87 insertions(+), 49 deletions(-) create mode 100644 Documentation/topics/dpdk/jumbo-frames.rst diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index 1a0fb6df1..e65e7dfaa 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -48,9 +48,9 @@ number of dpdk devices found in the log file:: $ ovs-vsctl add-port br0 dpdk-p1 -- set Interface dpdk-p1 type=dpdk \ options:dpdk-devargs=0000:01:00.1 -Some NICs (i.e. Mellanox ConnectX-3) have only one PCI address associated -with multiple ports. Using a PCI device like above won't work. Instead, below -usage is suggested:: +Some NICs (i.e. Mellanox ConnectX-3) have only one PCI address associated with +multiple ports. Using a PCI device like above won't work. Instead, below usage +is suggested:: $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \ options:dpdk-devargs="class=eth,mac=00:11:22:33:44:55:01" @@ -85,52 +85,6 @@ To stop ovs-vswitchd & delete bridge, run:: $ ovs-appctl -t ovsdb-server exit $ ovs-vsctl del-br br0 -Jumbo Frames ------------- - -By default, DPDK ports are configured with standard Ethernet MTU (1500B). To -enable Jumbo Frames support for a DPDK port, change the Interface's -``mtu_request`` attribute to a sufficiently large value. For example, to add a -DPDK Phy port with MTU of 9000:: - - $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \ - options:dpdk-devargs=0000:01:00.0 mtu_request=9000 - -Similarly, to change the MTU of an existing port to 6200:: - - $ ovs-vsctl set Interface dpdk-p0 mtu_request=6200 - -Some additional configuration is needed to take advantage of jumbo frames with -vHost ports: - -1. *mergeable buffers* must be enabled for vHost ports, as demonstrated in the - QEMU command line snippet below:: - - -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ - -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=on - -2. Where virtio devices are bound to the Linux kernel driver in a guest - environment (i.e. interfaces are not bound to an in-guest DPDK driver), the - MTU of those logical network interfaces must also be increased to a - sufficiently large value. This avoids segmentation of Jumbo Frames received - in the guest. Note that 'MTU' refers to the length of the IP packet only, - and not that of the entire frame. - - To calculate the exact MTU of a standard IPv4 frame, subtract the L2 header - and CRC lengths (i.e. 18B) from the max supported frame size. So, to set - the MTU for a 9018B Jumbo Frame:: - - $ ip link set eth1 mtu 9000 - -When Jumbo Frames are enabled, the size of a DPDK port's mbuf segments are -increased, such that a full Jumbo Frame of a specific size may be accommodated -within a single mbuf segment. - -Jumbo frame support has been validated against 9728B frames, which is the -largest frame size supported by Fortville NIC using the DPDK i40e driver, but -larger frames and other DPDK NIC drivers may be supported. These cases are -common for use cases involving East-West traffic only. - .. _dpdk-ovs-in-guest: OVS with DPDK Inside VMs diff --git a/Documentation/topics/dpdk/index.rst b/Documentation/topics/dpdk/index.rst index d083d929e..181f61abb 100644 --- a/Documentation/topics/dpdk/index.rst +++ b/Documentation/topics/dpdk/index.rst @@ -39,3 +39,4 @@ The DPDK Datapath /topics/dpdk/pmd /topics/dpdk/qos /topics/dpdk/pdump + /topics/dpdk/jumbo-frames diff --git a/Documentation/topics/dpdk/jumbo-frames.rst b/Documentation/topics/dpdk/jumbo-frames.rst new file mode 100644 index 000000000..692b291ac --- /dev/null +++ b/Documentation/topics/dpdk/jumbo-frames.rst @@ -0,0 +1,71 @@ +.. + Copyright 2018, Red Hat, Inc. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +============ +Jumbo Frames +============ + +By default, DPDK ports are configured with standard Ethernet MTU (1500B). To +enable Jumbo Frames support for a DPDK port, change the Interface's +``mtu_request`` attribute to a sufficiently large value. For example, to add a +:doc:`DPDK physical port ` with an MTU of 9000, run:: + + $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \ + options:dpdk-devargs=0000:01:00.0 mtu_request=9000 + +Similarly, to change the MTU of an existing port to 6200, run:: + + $ ovs-vsctl set Interface dpdk-p0 mtu_request=6200 + +Some additional configuration is needed to take advantage of jumbo frames with +:doc:`vHost User ports `: + +- *Mergeable buffers* must be enabled for vHost User ports, as demonstrated in + the QEMU command line snippet below:: + + -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ + -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=on + +- Where virtio devices are bound to the Linux kernel driver in a guest + environment (i.e. interfaces are not bound to an in-guest DPDK driver), the + MTU of those logical network interfaces must also be increased to a + sufficiently large value. This avoids segmentation of Jumbo Frames received + in the guest. Note that 'MTU' refers to the length of the IP packet only, and + not that of the entire frame. + + To calculate the exact MTU of a standard IPv4 frame, subtract the L2 header + and CRC lengths (i.e. 18B) from the max supported frame size. So, to set the + MTU for a 9018B Jumbo Frame:: + + $ ip link set eth1 mtu 9000 + +When Jumbo Frames are enabled, the size of a DPDK port's mbuf segments are +increased, such that a full Jumbo Frame of a specific size may be accommodated +within a single mbuf segment. + +Jumbo frame support has been validated against 9728B frames, which is the +largest frame size supported by Fortville NIC using the DPDK i40e driver, but +larger frames and other DPDK NIC drivers may be supported. These cases are +common for use cases involving East-West traffic only. diff --git a/Documentation/topics/dpdk/phy.rst b/Documentation/topics/dpdk/phy.rst index c0931b2c4..477b291e4 100644 --- a/Documentation/topics/dpdk/phy.rst +++ b/Documentation/topics/dpdk/phy.rst @@ -218,3 +218,9 @@ If the log is not seen then the port can be detached like so:: For more information please refer to the `DPDK Port Hotplug Framework`__. __ http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html#hotplug + +Jumbo Frames +------------ + +DPDK physical ports can be configured to use Jumbo Frames. For more +information, refer to :doc:`jumbo-frames`. diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst index 168d5c5ce..67eb77d43 100644 --- a/Documentation/topics/dpdk/vhost-user.rst +++ b/Documentation/topics/dpdk/vhost-user.rst @@ -444,6 +444,12 @@ Sample XML .. _QEMU documentation: http://git.qemu-project.org/?p=qemu.git;a=blob;f=docs/specs/vhost-user.txt;h=7890d7169;hb=HEAD +Jumbo Frames +------------ + +DPDK vHost User ports can be configured to use Jumbo Frames. For more +information, refer to :doc:`jumbo-frames`. + vhost-user Dequeue Zero Copy (experimental) ------------------------------------------- From patchwork Mon Apr 16 14:30:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 898658 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="WO/JwOW3"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40PrZH1yLrz9s1R for ; Tue, 17 Apr 2018 00:37:27 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id C0541EEE; Mon, 16 Apr 2018 14:30:54 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 0C917EDC for ; Mon, 16 Apr 2018 14:30:51 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from anteater.ash.relay.mailchannels.net (anteater.ash.relay.mailchannels.net [23.83.222.3]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 70C5A625 for ; Mon, 16 Apr 2018 14:30:48 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 1FDC45C2A49; Mon, 16 Apr 2018 14:30:45 +0000 (UTC) Received: from one.mxroute.com (unknown [100.96.32.20]) (Authenticated sender: 5xi41l16bi) by relay.mailchannels.net (Postfix) with ESMTPA id 7A3D65C24C4; Mon, 16 Apr 2018 14:30:44 +0000 (UTC) X-Sender-Id: 5xi41l16bi|x-authuser|stephen@that.guru Received: from one.mxroute.com (one-outgoing.mxroute.com [172.18.52.53]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.14.1); Mon, 16 Apr 2018 14:30:45 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: 5xi41l16bi|x-authuser|stephen@that.guru X-MailChannels-Auth-Id: 5xi41l16bi X-Cellar-Keen: 7e36d6744dadd9c8_1523889044819_1636466637 X-MC-Loop-Signature: 1523889044818:3310205581 X-MC-Ingress-Time: 1523889044818 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Czy2m3JQKeGAsrFe704mm8+lX3kaGx6tTeTVCprcwzk=; b=WO/JwOW34bj88TZNPapROaeYy6 mZrYraFiokYY2a/5S2UhatX4ICsmy/6gVrEgjwxbZAb8eULjowG6VGfTpJd+OBILKbeZbriZNnO3+ Wdi/ftqM69u8AWMqgo2vRS1PXsjP0Q7YNp1nxeaOFyV2BNqiDIp7/xMtnupc6vt/kC48MKdHJbx9q y2nsfUGyBh8Ua0WgnLhJSHGTe0TyS7/g/D1pNFW5fK0BFwT6p8QXyWJMm/GD2nuaheyvLFaeap0JD 0DwiK1WVw5xdZGSzMZmsRwpGkubuJZKCa82Ka/2Dol9pA7tUY/9iX9chZVha3iQMVLbfoBANFAl86 qxNaFTMQ==; From: Stephen Finucane To: dev@openvswitch.org Date: Mon, 16 Apr 2018 15:30:26 +0100 Message-Id: <20180416143026.24561-10-stephen@that.guru> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180416143026.24561-1-stephen@that.guru> References: <20180416143026.24561-1-stephen@that.guru> X-AuthUser: stephen@that.guru X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v2 9/9] doc: Final cleanup of the DPDK documents X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This concludes the cleanup by fixing some nits and adding some additional cross-references. Signed-off-by: Stephen Finucane --- v2: - Add changes to DPDK topic index doc --- Documentation/howto/dpdk.rst | 51 ++++++++++++++++--------------------- Documentation/topics/dpdk/index.rst | 6 +++++ 2 files changed, 28 insertions(+), 29 deletions(-) diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index e65e7dfaa..199b8be65 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -25,23 +25,26 @@ Using Open vSwitch with DPDK ============================ -This document describes how to use Open vSwitch with DPDK datapath. +This document describes how to use Open vSwitch with DPDK datapath. For more +detailed information, refer to the various :doc:`DPDK topic guides +`. .. important:: Using the DPDK datapath requires building OVS with DPDK support. Refer to :doc:`/intro/install/dpdk` for more information. -Ports and Bridges ------------------ +Overview +-------- -ovs-vsctl can be used to set up bridges and other Open vSwitch features. -Bridges should be created with a ``datapath_type=netdev``:: +:program:`ovs-vsctl` can be used to set up bridges and other Open vSwitch +features. Bridges should be created with a ``datapath_type=netdev``:: $ ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev -ovs-vsctl can also be used to add DPDK devices. ovs-vswitchd should print the -number of dpdk devices found in the log file:: +:program:`ovs-vsctl` can also be used to add DPDK devices. +:program:`ovs-vswitchd` should print the number of ``dpdk`` devices found in +the log file:: $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \ options:dpdk-devargs=0000:01:00.0 @@ -59,14 +62,13 @@ is suggested:: .. important:: - Hotplugging physical interfaces is not supported using the above syntax. - This is expected to change with the release of DPDK v18.05. For information - on hotplugging physical interfaces, you should instead refer to - :ref:`port-hotplug`. + Hotplugging physical interfaces is not supported for these devices. This + is expected to change with the release of DPDK v18.05. For information on + hotplugging physical interfaces, refer to :ref:`port-hotplug`. After the DPDK ports get added to switch, a polling thread continuously polls -DPDK devices and consumes 100% of the core, as can be checked from ``top`` and -``ps`` commands:: +DPDK devices and consumes 100% of the core, as can be checked from +:command:`top` and :command:`ps` commands:: $ top -H $ ps -eLo pid,psr,comm | grep pmd @@ -79,7 +81,7 @@ set. For example:: -- set Interface p0 type=dpdk options:dpdk-devargs=0000:01:00.0 \ -- set Interface p1 type=dpdk options:dpdk-devargs=0000:01:00.1 -To stop ovs-vswitchd & delete bridge, run:: +To stop :program:`ovs-vswitchd` and delete the bridge, run:: $ ovs-appctl -t ovs-vswitchd exit $ ovs-appctl -t ovsdb-server exit @@ -137,13 +139,16 @@ Add test flows to forward packets between DPDK port 0 and port 1:: Transmit traffic into either port. You should see it returned via the other. +More information on the ``dpdk`` ports can be found in :doc:`/topics/dpdk/phy`. + .. _dpdk-vhost-loopback: PHY-VM-PHY (vHost Loopback) --------------------------- Add a userspace bridge, two ``dpdk`` (PHY) ports, and two ``dpdkvhostuser`` -ports:: +ports. It is assumed that the physical ports are already bound to DPDK, as +described in :ref:`dpdk-binding-nics`:: # Add userspace bridge $ ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev @@ -221,19 +226,7 @@ described in :ref:`dpdk-testpmd`. Once compiled, run the application:: $ set fwd mac retry $ start -When you finish testing, bind the vNICs back to kernel:: - - $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 - $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 - -.. note:: - - Valid PCI IDs must be passed in above example. The PCI IDs can be retrieved - like so:: - - $ $DPDK_DIR/usertools/dpdk-devbind.py --status - -More information on the dpdkvhostuser ports can be found in +More information on the ``dpdkvhostuser`` ports can be found in :doc:`/topics/dpdk/vhost-user`. PHY-VM-PHY (vHost Loopback) (Kernel Forwarding) @@ -277,7 +270,7 @@ devices to bridge ``br0``. Once complete, follow the below steps: $ ovs-vsctl set Interface phy0 options:n_rxq=2 $ ovs-vsctl set Interface phy1 options:n_rxq=2 -2. Instantiate Guest VM using QEMU cmdline +2. Instantiate Guest VM using QEMU command line We must configure with appropriate software versions to ensure this feature is supported. diff --git a/Documentation/topics/dpdk/index.rst b/Documentation/topics/dpdk/index.rst index 181f61abb..79d3af379 100644 --- a/Documentation/topics/dpdk/index.rst +++ b/Documentation/topics/dpdk/index.rst @@ -30,12 +30,18 @@ The DPDK Datapath .. toctree:: :maxdepth: 2 + :caption: Bridges and Ports /topics/dpdk/bridge /topics/dpdk/phy /topics/dpdk/vhost-user /topics/dpdk/ring /topics/dpdk/vdev + +.. toctree:: + :maxdepth: 2 + :caption: Advanced Configuration + /topics/dpdk/pmd /topics/dpdk/qos /topics/dpdk/pdump