From patchwork Fri Jun 30 14:39:52 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Weglicki, MichalX" X-Patchwork-Id: 782886 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3wzfM46bVJz9sNX for ; Sat, 1 Jul 2017 00:40:00 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 3A75989F; Fri, 30 Jun 2017 14:39:59 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 8492989E for ; Fri, 30 Jun 2017 14:39:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 5B0A7125 for ; Fri, 30 Jun 2017 14:39:56 +0000 (UTC) Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 Jun 2017 07:39:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.40,286,1496127600"; d="scan'208";a="120877111" Received: from sivswdev02.ir.intel.com (HELO localhost.localdomain) ([10.237.217.46]) by fmsmga005.fm.intel.com with ESMTP; 30 Jun 2017 07:39:54 -0700 From: mweglicx To: dev@openvswitch.org Date: Fri, 30 Jun 2017 15:39:52 +0100 Message-Id: <1498833592-30452-1-git-send-email-michalx.weglicki@intel.com> X-Mailer: git-send-email 1.7.0.7 X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH] Update relevant artifacts to add support for DPDK 17.05.1. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Upgrading to DPDK 17.05.1 stable release adds new significant features relevant to OVS, including, but not limited to: - tun/tap PMD, - VFIO hotplug support, - Generic flow API. Following changes are applied: - netdev-dpdk: Changes required by DPDK API modifications. - doc: Because of DPDK API changes, backward compatibility with previous DPDK releases will be broken, thus all relevant documentation entries are updated. - .travis: DPDK version change from 16.11.1 to 17.05.1. - rhel/openvswitch-fedora.spec.in: DPDK version change from 16.11 to 17.05.1 Signed-off-by: Michal Weglicki Acked-by: Mark Kavanagh --- .travis/linux-build.sh | 2 +- Documentation/intro/install/dpdk.rst | 8 +- Documentation/topics/dpdk/vhost-user.rst | 8 +- NEWS | 1 + lib/netdev-dpdk.c | 144 +++++++++++++++++++------------ rhel/openvswitch-fedora.spec.in | 2 +- tests/dpdk/ring_client.c | 6 +- 7 files changed, 105 insertions(+), 66 deletions(-) diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh index 8750d68..efccdf1 100755 --- a/.travis/linux-build.sh +++ b/.travis/linux-build.sh @@ -80,7 +80,7 @@ fi if [ "$DPDK" ]; then if [ -z "$DPDK_VER" ]; then - DPDK_VER="16.11.1" + DPDK_VER="17.05.1" fi install_dpdk $DPDK_VER if [ "$CC" = "clang" ]; then diff --git a/Documentation/intro/install/dpdk.rst b/Documentation/intro/install/dpdk.rst index e83f852..4a178f3 100644 --- a/Documentation/intro/install/dpdk.rst +++ b/Documentation/intro/install/dpdk.rst @@ -40,7 +40,7 @@ Build requirements In addition to the requirements described in :doc:`general`, building Open vSwitch with DPDK will require the following: -- DPDK 16.11 +- DPDK 17.05.1 - A `DPDK supported NIC`_ @@ -69,9 +69,9 @@ Install DPDK #. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``:: $ cd /usr/src/ - $ wget http://fast.dpdk.org/rel/dpdk-16.11.1.tar.xz - $ tar xf dpdk-16.11.1.tar.xz - $ export DPDK_DIR=/usr/src/dpdk-stable-16.11.1 + $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz + $ tar xf dpdk-17.05.1.tar.xz + $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.1 $ cd $DPDK_DIR #. (Optional) Configure DPDK as a shared library diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst index 6b0bf9b..9f11ea1 100644 --- a/Documentation/topics/dpdk/vhost-user.rst +++ b/Documentation/topics/dpdk/vhost-user.rst @@ -292,9 +292,9 @@ To begin, instantiate a guest as described in :ref:`dpdk-vhost-user` or DPDK sources to VM and build DPDK:: $ cd /root/dpdk/ - $ wget http://fast.dpdk.org/rel/dpdk-16.11.1.tar.xz - $ tar xf dpdk-16.11.1.tar.xz - $ export DPDK_DIR=/root/dpdk/dpdk-stable-16.11.1 + $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz + $ tar xf dpdk-17.05.1.tar.xz + $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.1 $ export DPDK_TARGET=x86_64-native-linuxapp-gcc $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET $ cd $DPDK_DIR @@ -378,7 +378,7 @@ Sample XML - + diff --git a/NEWS b/NEWS index 4ea7e62..8c1331f 100644 --- a/NEWS +++ b/NEWS @@ -20,6 +20,7 @@ Post-v2.7.0 still can be configured via extra arguments for DPDK EAL. * dpdkvhostuser ports are marked as deprecated. They will be removed in an upcoming release. + * Support for DPDK v17.05.1. - IPFIX now provides additional counters: * Total counters since metering process startup. * Per-flow TCP flag counters. diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index bba4de3..580a279 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -22,6 +22,9 @@ #include #include #include +#include +#include +#include #include #include @@ -31,7 +34,7 @@ #include #include #include -#include +#include #include "dirs.h" #include "dp-packet.h" @@ -56,6 +59,8 @@ #include "timeval.h" #include "unixctl.h" +enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM}; + VLOG_DEFINE_THIS_MODULE(netdev_dpdk); static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20); @@ -171,6 +176,21 @@ static const struct rte_eth_conf port_conf = { }, }; +/* + * These callbacks allow virtio-net devices to be added to vhost ports when + * configuration has been fully completed. + */ +static int new_device(int vid); +static void destroy_device(int vid); +static int vring_state_changed(int vid, uint16_t queue_id, int enable); +static const struct vhost_device_ops virtio_net_device_ops = +{ + .new_device = new_device, + .destroy_device = destroy_device, + .vring_state_changed = vring_state_changed, + .features_changed = NULL +}; + enum { DPDK_RING_SIZE = 256 }; BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE)); enum { DRAIN_TSC = 200000ULL }; @@ -412,8 +432,8 @@ struct netdev_rxq_dpdk { dpdk_port_t port_id; }; -static int netdev_dpdk_class_init(void); -static int netdev_dpdk_vhost_class_init(void); +static void netdev_dpdk_destruct(struct netdev *netdev); +static void netdev_dpdk_vhost_destruct(struct netdev *netdev); int netdev_dpdk_get_vid(const struct netdev_dpdk *dev); @@ -423,8 +443,8 @@ netdev_dpdk_get_ingress_policer(const struct netdev_dpdk *dev); static bool is_dpdk_class(const struct netdev_class *class) { - return class->init == netdev_dpdk_class_init - || class->init == netdev_dpdk_vhost_class_init; + return class->destruct == netdev_dpdk_destruct + || class->destruct == netdev_dpdk_vhost_destruct; } /* DPDK NIC drivers allocate RX buffers at a particular granularity, typically @@ -959,13 +979,45 @@ netdev_dpdk_vhost_construct(struct netdev *netdev) if (err) { VLOG_ERR("vhost-user socket device setup failure for socket %s\n", dev->vhost_id); + goto out; } else { fatal_signal_add_file_to_unlink(dev->vhost_id); VLOG_INFO("Socket %s created for vhost-user port %s\n", dev->vhost_id, name); } + + err = rte_vhost_driver_callback_register(dev->vhost_id, + &virtio_net_device_ops); + if (err) { + VLOG_ERR("rte_vhost_driver_callback_register failed for vhost user " + "port: %s\n", name); + goto out; + } + + err = rte_vhost_driver_disable_features(dev->vhost_id, + 1ULL << VIRTIO_NET_F_HOST_TSO4 + | 1ULL << VIRTIO_NET_F_HOST_TSO6 + | 1ULL << VIRTIO_NET_F_CSUM); + if (err) { + VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user " + "port: %s\n", name); + goto out; + } + + err = rte_vhost_driver_start(dev->vhost_id); + if (err) { + VLOG_ERR("rte_vhost_driver_start failed for vhost user " + "port: %s\n", name); + goto out; + } + err = vhost_common_construct(netdev); + if (err) { + VLOG_ERR("vhost_common_construct failed for vhost user " + "port: %s\n", name); + } +out: ovs_mutex_unlock(&dpdk_mutex); VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; " "please migrate to dpdkvhostuserclient ports."); @@ -979,6 +1031,10 @@ netdev_dpdk_vhost_client_construct(struct netdev *netdev) ovs_mutex_lock(&dpdk_mutex); err = vhost_common_construct(netdev); + if (err) { + VLOG_ERR("vhost_common_construct failed for vhost user client" + "port: %s\n", netdev->name); + } ovs_mutex_unlock(&dpdk_mutex); return err; } @@ -2479,12 +2535,9 @@ static void set_irq_status(int vid) { uint32_t i; - uint64_t idx; - for (i = 0; i < rte_vhost_get_queue_num(vid); i++) { - idx = i * VIRTIO_QNUM; - rte_vhost_enable_guest_notification(vid, idx + VIRTIO_RXQ, 0); - rte_vhost_enable_guest_notification(vid, idx + VIRTIO_TXQ, 0); + for (i = 0; i < rte_vhost_get_vring_num(vid); i++) { + rte_vhost_enable_guest_notification(vid, i, 0); } } @@ -2547,7 +2600,7 @@ new_device(int vid) LIST_FOR_EACH(dev, list_node, &dpdk_list) { ovs_mutex_lock(&dev->mutex); if (strncmp(ifname, dev->vhost_id, IF_NAME_SZ) == 0) { - uint32_t qp_num = rte_vhost_get_queue_num(vid); + uint32_t qp_num = rte_vhost_get_vring_num(vid)/VIRTIO_QNUM; /* Get NUMA information */ newnode = rte_vhost_get_numa_node(vid); @@ -2714,27 +2767,6 @@ netdev_dpdk_get_ingress_policer(const struct netdev_dpdk *dev) return ovsrcu_get(struct ingress_policer *, &dev->ingress_policer); } -/* - * These callbacks allow virtio-net devices to be added to vhost ports when - * configuration has been fully complete. - */ -static const struct virtio_net_device_ops virtio_net_device_ops = -{ - .new_device = new_device, - .destroy_device = destroy_device, - .vring_state_changed = vring_state_changed -}; - -static void * -start_vhost_loop(void *dummy OVS_UNUSED) -{ - pthread_detach(pthread_self()); - /* Put the vhost thread into quiescent state. */ - ovsrcu_quiesce_start(); - rte_vhost_driver_session_start(); - return NULL; -} - static int netdev_dpdk_class_init(void) { @@ -2754,25 +2786,6 @@ netdev_dpdk_class_init(void) return 0; } -static int -netdev_dpdk_vhost_class_init(void) -{ - static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; - - /* This function can be called for different classes. The initialization - * needs to be done only once */ - if (ovsthread_once_start(&once)) { - rte_vhost_driver_callback_register(&virtio_net_device_ops); - rte_vhost_feature_disable(1ULL << VIRTIO_NET_F_HOST_TSO4 - | 1ULL << VIRTIO_NET_F_HOST_TSO6 - | 1ULL << VIRTIO_NET_F_CSUM); - ovs_thread_create("vhost_thread", start_vhost_loop, NULL); - - ovsthread_once_done(&once); - } - - return 0; -} /* Client Rings */ @@ -3227,6 +3240,31 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev) "using client socket '%s'", dev->up.name, dev->vhost_id); } + + err = rte_vhost_driver_callback_register(dev->vhost_id, + &virtio_net_device_ops); + if (err) { + VLOG_ERR("rte_vhost_driver_callback_register failed for " + "vhost user client port: %s\n", dev->up.name); + goto unlock; + } + + err = rte_vhost_driver_disable_features(dev->vhost_id, + 1ULL << VIRTIO_NET_F_HOST_TSO4 + | 1ULL << VIRTIO_NET_F_HOST_TSO6 + | 1ULL << VIRTIO_NET_F_CSUM); + if (err) { + VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user " + "client port: %s\n", dev->up.name); + goto unlock; + } + + err = rte_vhost_driver_start(dev->vhost_id); + if (err) { + VLOG_ERR("rte_vhost_driver_start failed for vhost user " + "client port: %s\n", dev->up.name); + goto unlock; + } } err = dpdk_vhost_reconfigure_helper(dev); @@ -3346,7 +3384,7 @@ static const struct netdev_class dpdk_ring_class = static const struct netdev_class dpdk_vhost_class = NETDEV_DPDK_CLASS( "dpdkvhostuser", - netdev_dpdk_vhost_class_init, + NULL, netdev_dpdk_vhost_construct, netdev_dpdk_vhost_destruct, NULL, @@ -3361,7 +3399,7 @@ static const struct netdev_class dpdk_vhost_class = static const struct netdev_class dpdk_vhost_client_class = NETDEV_DPDK_CLASS( "dpdkvhostuserclient", - netdev_dpdk_vhost_class_init, + NULL, netdev_dpdk_vhost_client_construct, netdev_dpdk_vhost_destruct, netdev_dpdk_vhost_client_set_config, diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch-fedora.spec.in index f822ad3..1c0255c 100644 --- a/rhel/openvswitch-fedora.spec.in +++ b/rhel/openvswitch-fedora.spec.in @@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel %endif %if %{with dpdk} BuildRequires: libpcap-devel numactl-devel -BuildRequires: dpdk-devel >= 16.11 +BuildRequires: dpdk-devel >= 17.05.1 Provides: %{name}-dpdk = %{version}-%{release} %endif diff --git a/tests/dpdk/ring_client.c b/tests/dpdk/ring_client.c index b153713..8cc3fb5 100644 --- a/tests/dpdk/ring_client.c +++ b/tests/dpdk/ring_client.c @@ -185,15 +185,15 @@ main(int argc, char *argv[]) /* Try dequeuing max possible packets first, if that fails, get the * most we can. Loop body should only execute once, maximum. */ - while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, rx_pkts) != 0) && - rx_pkts > 0) { + while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, + rx_pkts, NULL) != 0) && rx_pkts > 0) { rx_pkts = (uint16_t)RTE_MIN(rte_ring_count(rx_ring), PKT_READ_SIZE); } if (rx_pkts > 0) { /* blocking enqueue */ do { - rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts); + rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts, NULL); } while (rslt == -ENOBUFS); } }