| Message ID | 1500986075-50620-1-git-send-email-michalx.weglicki@intel.com |
|---|---|
| State | Accepted |
| Headers | show |
On 07/25/2017 01:34 PM, Michal Weglicki wrote: > Upgrading to DPDK 17.05.1 stable release adds new > significant features relevant to OVS, including, > but not limited to: > - tun/tap PMD, > - VFIO hotplug support, > - Generic flow API. > > Following changes are applied: > - netdev-dpdk: Changes required by DPDK API modifications. > - doc: Because of DPDK API changes, backward compatibility > with previous DPDK releases will be broken, thus all > relevant documentation entries are updated. > - .travis: DPDK version change from 16.11.1 to 17.05.1. > - rhel/openvswitch-fedora.spec.in: DPDK version change > from 16.11 to 17.05.1 > > v1->v2: Patch rebase. > v2->v3: Fixed wrong formating after v2 patch rebase. > v3->v4: Minor documentation changes. > > Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> > Reviewed-by: Aaron Conole <aconole@redhat.com> > --- Thanks Michal. I tried a few things out and working ok for me. Acked-by: Kevin Traynor <ktraynor@redhat.com>
Hi Aaron You had mentioned that you would be testing with the patch as well. Were you able to try it ? Thanks Darrell -----Original Message----- From: <ovs-dev-bounces@openvswitch.org> on behalf of Kevin Traynor <ktraynor@redhat.com> Organization: Red Hat Date: Tuesday, July 25, 2017 at 9:35 AM To: Michal Weglicki <michalx.weglicki@intel.com>, "dev@openvswitch.org" <dev@openvswitch.org> Subject: Re: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for DPDK 17.05.1. On 07/25/2017 01:34 PM, Michal Weglicki wrote: > Upgrading to DPDK 17.05.1 stable release adds new > significant features relevant to OVS, including, > but not limited to: > - tun/tap PMD, > - VFIO hotplug support, > - Generic flow API. > > Following changes are applied: > - netdev-dpdk: Changes required by DPDK API modifications. > - doc: Because of DPDK API changes, backward compatibility > with previous DPDK releases will be broken, thus all > relevant documentation entries are updated. > - .travis: DPDK version change from 16.11.1 to 17.05.1. > - rhel/openvswitch-fedora.spec.in: DPDK version change > from 16.11 to 17.05.1 > > v1->v2: Patch rebase. > v2->v3: Fixed wrong formating after v2 patch rebase. > v3->v4: Minor documentation changes. > > Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> > Reviewed-by: Aaron Conole <aconole@redhat.com> > --- Thanks Michal. I tried a few things out and working ok for me. Acked-by: Kevin Traynor <ktraynor@redhat.com> _______________________________________________ dev mailing list dev@openvswitch.org https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=ekKh60DEjO7sJHJqmg8W8agp2GxZlvAzz5bWGN3ux4k&s=AZOPEg-J3gwSlEr1nLn9aCrmRt_LBA0YK88h9QS1_-g&e=
>-----Original Message----- >From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev-bounces@openvswitch.org] >On Behalf Of Michal Weglicki >Sent: Tuesday, July 25, 2017 1:35 PM >To: dev@openvswitch.org >Subject: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for >DPDK 17.05.1. > >Upgrading to DPDK 17.05.1 stable release adds new >significant features relevant to OVS, including, >but not limited to: >- tun/tap PMD, >- VFIO hotplug support, >- Generic flow API. > >Following changes are applied: >- netdev-dpdk: Changes required by DPDK API modifications. >- doc: Because of DPDK API changes, backward compatibility > with previous DPDK releases will be broken, thus all > relevant documentation entries are updated. >- .travis: DPDK version change from 16.11.1 to 17.05.1. >- rhel/openvswitch-fedora.spec.in: DPDK version change > from 16.11 to 17.05.1 > >v1->v2: Patch rebase. >v2->v3: Fixed wrong formating after v2 patch rebase. >v3->v4: Minor documentation changes. > >Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> >Reviewed-by: Aaron Conole <aconole@redhat.com> Hi Michal, Is there a reason why you haven't added my acked-by to this latest patch? https://mail.openvswitch.org/pipermail/ovs-dev/2017-July/334893.html Thanks, Mark >--- > .travis/linux-build.sh | 2 +- > Documentation/faq/releases.rst | 1 + > Documentation/howto/dpdk.rst | 6 +- > Documentation/intro/install/dpdk.rst | 14 +-- > Documentation/topics/dpdk/vhost-user.rst | 12 +-- > NEWS | 1 + > lib/netdev-dpdk.c | 144 +++++++++++++++++++----------- >- > rhel/openvswitch-fedora.spec.in | 2 +- > tests/dpdk/ring_client.c | 6 +- > 9 files changed, 114 insertions(+), 74 deletions(-) > >diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh >index f66b534..efccdf1 100755 >--- a/.travis/linux-build.sh >+++ b/.travis/linux-build.sh >@@ -80,7 +80,7 @@ fi > > if [ "$DPDK" ]; then > if [ -z "$DPDK_VER" ]; then >- DPDK_VER="16.11.2" >+ DPDK_VER="17.05.1" > fi > install_dpdk $DPDK_VER > if [ "$CC" = "clang" ]; then >diff --git a/Documentation/faq/releases.rst b/Documentation/faq/releases.rst >index 707834b..2ecc24c 100644 >--- a/Documentation/faq/releases.rst >+++ b/Documentation/faq/releases.rst >@@ -161,6 +161,7 @@ Q: What DPDK version does each Open vSwitch release work >with? > 2.5.x 2.2 > 2.6.x 16.07.2 > 2.7.x 16.11.2 >+ 2.8.x 17.05.1 > ============ ======= > > Q: I get an error like this when I configure Open vSwitch: >diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst >index af01d3e..1756c8c 100644 >--- a/Documentation/howto/dpdk.rst >+++ b/Documentation/howto/dpdk.rst >@@ -528,15 +528,15 @@ described in :ref:`dpdk-testpmd`. Once compiled, run the >application:: > > When you finish testing, bind the vNICs back to kernel:: > >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 > > .. note:: > > Valid PCI IDs must be passed in above example. The PCI IDs can be retrieved > like so:: > >- $ $DPDK_DIR/tools/dpdk-devbind.py --status >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status > > More information on the dpdkvhostuser ports can be found in > :doc:`/topics/dpdk/vhost-user`. >diff --git a/Documentation/intro/install/dpdk.rst >b/Documentation/intro/install/dpdk.rst >index a05aa1a..20baa63 100644 >--- a/Documentation/intro/install/dpdk.rst >+++ b/Documentation/intro/install/dpdk.rst >@@ -40,7 +40,7 @@ Build requirements > In addition to the requirements described in :doc:`general`, building Open > vSwitch with DPDK will require the following: > >-- DPDK 16.11 >+- DPDK 17.05.1 > > - A `DPDK supported NIC`_ > >@@ -69,9 +69,9 @@ Install DPDK > #. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``:: > > $ cd /usr/src/ >- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz >- $ tar xf dpdk-16.11.2.tar.xz >- $ export DPDK_DIR=/usr/src/dpdk-stable-16.11.2 >+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz >+ $ tar xf dpdk-17.05.1.tar.xz >+ $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.1 > $ cd $DPDK_DIR > > #. (Optional) Configure DPDK as a shared library >@@ -187,8 +187,8 @@ to the VFIO driver:: > $ modprobe vfio-pci > $ /usr/bin/chmod a+x /dev/vfio > $ /usr/bin/chmod 0666 /dev/vfio/* >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1 >- $ $DPDK_DIR/tools/dpdk-devbind.py --status >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=vfio-pci eth1 >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status > > Setup OVS > ~~~~~~~~~ >@@ -569,7 +569,7 @@ Limitations > The latest list of validated firmware versions can be found in the `DPDK > release notes`_. > >-.. _DPDK release notes: >http://dpdk.org/doc/guides/rel_notes/release_16_11.html >+.. _DPDK release notes: >http://dpdk.org/doc/guides/rel_notes/release_17_05.html > > Reporting Bugs > -------------- >diff --git a/Documentation/topics/dpdk/vhost-user.rst >b/Documentation/topics/dpdk/vhost-user.rst >index e76da5f..a3d5de3 100644 >--- a/Documentation/topics/dpdk/vhost-user.rst >+++ b/Documentation/topics/dpdk/vhost-user.rst >@@ -292,9 +292,9 @@ To begin, instantiate a guest as described in :ref:`dpdk- >vhost-user` or > DPDK sources to VM and build DPDK:: > > $ cd /root/dpdk/ >- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz >- $ tar xf dpdk-16.11.2.tar.xz >- $ export DPDK_DIR=/root/dpdk/dpdk-stable-16.11.2 >+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz >+ $ tar xf dpdk-17.05.1.tar.xz >+ $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.1 > $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > $ cd $DPDK_DIR >@@ -314,8 +314,8 @@ Setup huge pages and DPDK devices using UIO:: > $ mount -t hugetlbfs hugetlbfs /dev/hugepages # only if not already >mounted > $ modprobe uio > $ insmod $DPDK_BUILD/kmod/igb_uio.ko >- $ $DPDK_DIR/tools/dpdk-devbind.py --status >- $ $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status >+ $ $DPDK_DIR/usertools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 > > .. note:: > >@@ -378,7 +378,7 @@ Sample XML > </disk> > <disk type='dir' device='disk'> > <driver name='qemu' type='fat'/> >- <source dir='/usr/src/dpdk-stable-16.11.2'/> >+ <source dir='/usr/src/dpdk-stable-17.05.1'/> > <target dev='vdb' bus='virtio'/> > <readonly/> > </disk> >diff --git a/NEWS b/NEWS >index d61fc5f..3f58e8b 100644 >--- a/NEWS >+++ b/NEWS >@@ -21,6 +21,7 @@ Post-v2.7.0 > still can be configured via extra arguments for DPDK EAL. > * dpdkvhostuser ports are marked as deprecated. They will be removed > in an upcoming release. >+ * Support for DPDK v17.05.1. > - IPFIX now provides additional counters: > * Total counters since metering process startup. > * Per-flow TCP flag counters. >diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c >index ea17b97..5e767e1 100644 >--- a/lib/netdev-dpdk.c >+++ b/lib/netdev-dpdk.c >@@ -22,6 +22,9 @@ > #include <stdlib.h> > #include <errno.h> > #include <unistd.h> >+#include <linux/virtio_net.h> >+#include <sys/socket.h> >+#include <linux/if.h> > > #include <rte_config.h> > #include <rte_cycles.h> >@@ -31,7 +34,7 @@ > #include <rte_malloc.h> > #include <rte_mbuf.h> > #include <rte_meter.h> >-#include <rte_virtio_net.h> >+#include <rte_vhost.h> > > #include "dirs.h" > #include "dp-packet.h" >@@ -56,6 +59,8 @@ > #include "timeval.h" > #include "unixctl.h" > >+enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM}; >+ > VLOG_DEFINE_THIS_MODULE(netdev_dpdk); > static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20); > >@@ -171,6 +176,21 @@ static const struct rte_eth_conf port_conf = { > }, > }; > >+/* >+ * These callbacks allow virtio-net devices to be added to vhost ports when >+ * configuration has been fully completed. >+ */ >+static int new_device(int vid); >+static void destroy_device(int vid); >+static int vring_state_changed(int vid, uint16_t queue_id, int enable); >+static const struct vhost_device_ops virtio_net_device_ops = >+{ >+ .new_device = new_device, >+ .destroy_device = destroy_device, >+ .vring_state_changed = vring_state_changed, >+ .features_changed = NULL >+}; >+ > enum { DPDK_RING_SIZE = 256 }; > BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE)); > enum { DRAIN_TSC = 200000ULL }; >@@ -412,8 +432,8 @@ struct netdev_rxq_dpdk { > dpdk_port_t port_id; > }; > >-static int netdev_dpdk_class_init(void); >-static int netdev_dpdk_vhost_class_init(void); >+static void netdev_dpdk_destruct(struct netdev *netdev); >+static void netdev_dpdk_vhost_destruct(struct netdev *netdev); > > int netdev_dpdk_get_vid(const struct netdev_dpdk *dev); > >@@ -423,8 +443,8 @@ netdev_dpdk_get_ingress_policer(const struct netdev_dpdk >*dev); > static bool > is_dpdk_class(const struct netdev_class *class) > { >- return class->init == netdev_dpdk_class_init >- || class->init == netdev_dpdk_vhost_class_init; >+ return class->destruct == netdev_dpdk_destruct >+ || class->destruct == netdev_dpdk_vhost_destruct; > } > > /* DPDK NIC drivers allocate RX buffers at a particular granularity, >typically >@@ -948,13 +968,45 @@ netdev_dpdk_vhost_construct(struct netdev *netdev) > if (err) { > VLOG_ERR("vhost-user socket device setup failure for socket %s\n", > dev->vhost_id); >+ goto out; > } else { > fatal_signal_add_file_to_unlink(dev->vhost_id); > VLOG_INFO("Socket %s created for vhost-user port %s\n", > dev->vhost_id, name); > } >+ >+ err = rte_vhost_driver_callback_register(dev->vhost_id, >+ &virtio_net_device_ops); >+ if (err) { >+ VLOG_ERR("rte_vhost_driver_callback_register failed for vhost user " >+ "port: %s\n", name); >+ goto out; >+ } >+ >+ err = rte_vhost_driver_disable_features(dev->vhost_id, >+ 1ULL << VIRTIO_NET_F_HOST_TSO4 >+ | 1ULL << VIRTIO_NET_F_HOST_TSO6 >+ | 1ULL << VIRTIO_NET_F_CSUM); >+ if (err) { >+ VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user " >+ "port: %s\n", name); >+ goto out; >+ } >+ >+ err = rte_vhost_driver_start(dev->vhost_id); >+ if (err) { >+ VLOG_ERR("rte_vhost_driver_start failed for vhost user " >+ "port: %s\n", name); >+ goto out; >+ } >+ > err = vhost_common_construct(netdev); >+ if (err) { >+ VLOG_ERR("vhost_common_construct failed for vhost user " >+ "port: %s\n", name); >+ } > >+out: > ovs_mutex_unlock(&dpdk_mutex); > VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; " > "please migrate to dpdkvhostuserclient ports."); >@@ -968,6 +1020,10 @@ netdev_dpdk_vhost_client_construct(struct netdev >*netdev) > > ovs_mutex_lock(&dpdk_mutex); > err = vhost_common_construct(netdev); >+ if (err) { >+ VLOG_ERR("vhost_common_construct failed for vhost user client" >+ "port: %s\n", netdev->name); >+ } > ovs_mutex_unlock(&dpdk_mutex); > return err; > } >@@ -2456,12 +2512,9 @@ static void > set_irq_status(int vid) > { > uint32_t i; >- uint64_t idx; > >- for (i = 0; i < rte_vhost_get_queue_num(vid); i++) { >- idx = i * VIRTIO_QNUM; >- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_RXQ, 0); >- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_TXQ, 0); >+ for (i = 0; i < rte_vhost_get_vring_num(vid); i++) { >+ rte_vhost_enable_guest_notification(vid, i, 0); > } > } > >@@ -2524,7 +2577,7 @@ new_device(int vid) > LIST_FOR_EACH(dev, list_node, &dpdk_list) { > ovs_mutex_lock(&dev->mutex); > if (strncmp(ifname, dev->vhost_id, IF_NAME_SZ) == 0) { >- uint32_t qp_num = rte_vhost_get_queue_num(vid); >+ uint32_t qp_num = rte_vhost_get_vring_num(vid)/VIRTIO_QNUM; > > /* Get NUMA information */ > newnode = rte_vhost_get_numa_node(vid); >@@ -2691,27 +2744,6 @@ netdev_dpdk_get_ingress_policer(const struct >netdev_dpdk *dev) > return ovsrcu_get(struct ingress_policer *, &dev->ingress_policer); > } > >-/* >- * These callbacks allow virtio-net devices to be added to vhost ports when >- * configuration has been fully complete. >- */ >-static const struct virtio_net_device_ops virtio_net_device_ops = >-{ >- .new_device = new_device, >- .destroy_device = destroy_device, >- .vring_state_changed = vring_state_changed >-}; >- >-static void * >-start_vhost_loop(void *dummy OVS_UNUSED) >-{ >- pthread_detach(pthread_self()); >- /* Put the vhost thread into quiescent state. */ >- ovsrcu_quiesce_start(); >- rte_vhost_driver_session_start(); >- return NULL; >-} >- > static int > netdev_dpdk_class_init(void) > { >@@ -2731,25 +2763,6 @@ netdev_dpdk_class_init(void) > return 0; > } > >-static int >-netdev_dpdk_vhost_class_init(void) >-{ >- static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; >- >- /* This function can be called for different classes. The initialization >- * needs to be done only once */ >- if (ovsthread_once_start(&once)) { >- rte_vhost_driver_callback_register(&virtio_net_device_ops); >- rte_vhost_feature_disable(1ULL << VIRTIO_NET_F_HOST_TSO4 >- | 1ULL << VIRTIO_NET_F_HOST_TSO6 >- | 1ULL << VIRTIO_NET_F_CSUM); >- ovs_thread_create("vhost_thread", start_vhost_loop, NULL); >- >- ovsthread_once_done(&once); >- } >- >- return 0; >-} > > /* Client Rings */ > >@@ -3204,6 +3217,31 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev >*netdev) > "using client socket '%s'", > dev->up.name, dev->vhost_id); > } >+ >+ err = rte_vhost_driver_callback_register(dev->vhost_id, >+ &virtio_net_device_ops); >+ if (err) { >+ VLOG_ERR("rte_vhost_driver_callback_register failed for " >+ "vhost user client port: %s\n", dev->up.name); >+ goto unlock; >+ } >+ >+ err = rte_vhost_driver_disable_features(dev->vhost_id, >+ 1ULL << VIRTIO_NET_F_HOST_TSO4 >+ | 1ULL << VIRTIO_NET_F_HOST_TSO6 >+ | 1ULL << VIRTIO_NET_F_CSUM); >+ if (err) { >+ VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user >" >+ "client port: %s\n", dev->up.name); >+ goto unlock; >+ } >+ >+ err = rte_vhost_driver_start(dev->vhost_id); >+ if (err) { >+ VLOG_ERR("rte_vhost_driver_start failed for vhost user " >+ "client port: %s\n", dev->up.name); >+ goto unlock; >+ } > } > > err = dpdk_vhost_reconfigure_helper(dev); >@@ -3324,7 +3362,7 @@ static const struct netdev_class dpdk_ring_class = > static const struct netdev_class dpdk_vhost_class = > NETDEV_DPDK_CLASS( > "dpdkvhostuser", >- netdev_dpdk_vhost_class_init, >+ NULL, > netdev_dpdk_vhost_construct, > netdev_dpdk_vhost_destruct, > NULL, >@@ -3339,7 +3377,7 @@ static const struct netdev_class dpdk_vhost_class = > static const struct netdev_class dpdk_vhost_client_class = > NETDEV_DPDK_CLASS( > "dpdkvhostuserclient", >- netdev_dpdk_vhost_class_init, >+ NULL, > netdev_dpdk_vhost_client_construct, > netdev_dpdk_vhost_destruct, > netdev_dpdk_vhost_client_set_config, >diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch-fedora.spec.in >index 3a045d3..2bb7102 100644 >--- a/rhel/openvswitch-fedora.spec.in >+++ b/rhel/openvswitch-fedora.spec.in >@@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel > %endif > %if %{with dpdk} > BuildRequires: libpcap-devel numactl-devel >-BuildRequires: dpdk-devel >= 16.11 >+BuildRequires: dpdk-devel >= 17.05.1 > Provides: %{name}-dpdk = %{version}-%{release} > %endif > >diff --git a/tests/dpdk/ring_client.c b/tests/dpdk/ring_client.c >index b153713..8cc3fb5 100644 >--- a/tests/dpdk/ring_client.c >+++ b/tests/dpdk/ring_client.c >@@ -185,15 +185,15 @@ main(int argc, char *argv[]) > /* Try dequeuing max possible packets first, if that fails, get the > * most we can. Loop body should only execute once, maximum. > */ >- while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, rx_pkts) != 0) >&& >- rx_pkts > 0) { >+ while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, >+ rx_pkts, NULL) != 0) && rx_pkts > 0) { > rx_pkts = (uint16_t)RTE_MIN(rte_ring_count(rx_ring), >PKT_READ_SIZE); > } > > if (rx_pkts > 0) { > /* blocking enqueue */ > do { >- rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts); >+ rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts, NULL); > } while (rslt == -ENOBUFS); > } > } >-- >1.8.3.1 > >_______________________________________________ >dev mailing list >dev@openvswitch.org >https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Hey Mark, Sorry, I had some problems with rebase, and somehow I just simply forgot t to do so. Do you want me to re-apply this patch as v5? > -----Original Message----- > From: Kavanagh, Mark B > Sent: Wednesday, July 26, 2017 10:04 AM > To: Weglicki, MichalX <michalx.weglicki@intel.com>; dev@openvswitch.org > Subject: RE: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for DPDK 17.05.1. > > > > >-----Original Message----- > >From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev-bounces@openvswitch.org] > >On Behalf Of Michal Weglicki > >Sent: Tuesday, July 25, 2017 1:35 PM > >To: dev@openvswitch.org > >Subject: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for > >DPDK 17.05.1. > > > >Upgrading to DPDK 17.05.1 stable release adds new > >significant features relevant to OVS, including, > >but not limited to: > >- tun/tap PMD, > >- VFIO hotplug support, > >- Generic flow API. > > > >Following changes are applied: > >- netdev-dpdk: Changes required by DPDK API modifications. > >- doc: Because of DPDK API changes, backward compatibility > > with previous DPDK releases will be broken, thus all > > relevant documentation entries are updated. > >- .travis: DPDK version change from 16.11.1 to 17.05.1. > >- rhel/openvswitch-fedora.spec.in: DPDK version change > > from 16.11 to 17.05.1 > > > >v1->v2: Patch rebase. > >v2->v3: Fixed wrong formating after v2 patch rebase. > >v3->v4: Minor documentation changes. > > > >Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> > >Reviewed-by: Aaron Conole <aconole@redhat.com> > > > Hi Michal, > > Is there a reason why you haven't added my acked-by to this latest patch? https://mail.openvswitch.org/pipermail/ovs-dev/2017- > July/334893.html > > Thanks, > Mark > > >--- > > .travis/linux-build.sh | 2 +- > > Documentation/faq/releases.rst | 1 + > > Documentation/howto/dpdk.rst | 6 +- > > Documentation/intro/install/dpdk.rst | 14 +-- > > Documentation/topics/dpdk/vhost-user.rst | 12 +-- > > NEWS | 1 + > > lib/netdev-dpdk.c | 144 +++++++++++++++++++----------- > >- > > rhel/openvswitch-fedora.spec.in | 2 +- > > tests/dpdk/ring_client.c | 6 +- > > 9 files changed, 114 insertions(+), 74 deletions(-) > > > >diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh > >index f66b534..efccdf1 100755 > >--- a/.travis/linux-build.sh > >+++ b/.travis/linux-build.sh > >@@ -80,7 +80,7 @@ fi > > > > if [ "$DPDK" ]; then > > if [ -z "$DPDK_VER" ]; then > >- DPDK_VER="16.11.2" > >+ DPDK_VER="17.05.1" > > fi > > install_dpdk $DPDK_VER > > if [ "$CC" = "clang" ]; then > >diff --git a/Documentation/faq/releases.rst b/Documentation/faq/releases.rst > >index 707834b..2ecc24c 100644 > >--- a/Documentation/faq/releases.rst > >+++ b/Documentation/faq/releases.rst > >@@ -161,6 +161,7 @@ Q: What DPDK version does each Open vSwitch release work > >with? > > 2.5.x 2.2 > > 2.6.x 16.07.2 > > 2.7.x 16.11.2 > >+ 2.8.x 17.05.1 > > ============ ======= > > > > Q: I get an error like this when I configure Open vSwitch: > >diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst > >index af01d3e..1756c8c 100644 > >--- a/Documentation/howto/dpdk.rst > >+++ b/Documentation/howto/dpdk.rst > >@@ -528,15 +528,15 @@ described in :ref:`dpdk-testpmd`. Once compiled, run the > >application:: > > > > When you finish testing, bind the vNICs back to kernel:: > > > >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 > >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 > >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 > >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 > > > > .. note:: > > > > Valid PCI IDs must be passed in above example. The PCI IDs can be retrieved > > like so:: > > > >- $ $DPDK_DIR/tools/dpdk-devbind.py --status > >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status > > > > More information on the dpdkvhostuser ports can be found in > > :doc:`/topics/dpdk/vhost-user`. > >diff --git a/Documentation/intro/install/dpdk.rst > >b/Documentation/intro/install/dpdk.rst > >index a05aa1a..20baa63 100644 > >--- a/Documentation/intro/install/dpdk.rst > >+++ b/Documentation/intro/install/dpdk.rst > >@@ -40,7 +40,7 @@ Build requirements > > In addition to the requirements described in :doc:`general`, building Open > > vSwitch with DPDK will require the following: > > > >-- DPDK 16.11 > >+- DPDK 17.05.1 > > > > - A `DPDK supported NIC`_ > > > >@@ -69,9 +69,9 @@ Install DPDK > > #. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``:: > > > > $ cd /usr/src/ > >- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz > >- $ tar xf dpdk-16.11.2.tar.xz > >- $ export DPDK_DIR=/usr/src/dpdk-stable-16.11.2 > >+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz > >+ $ tar xf dpdk-17.05.1.tar.xz > >+ $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.1 > > $ cd $DPDK_DIR > > > > #. (Optional) Configure DPDK as a shared library > >@@ -187,8 +187,8 @@ to the VFIO driver:: > > $ modprobe vfio-pci > > $ /usr/bin/chmod a+x /dev/vfio > > $ /usr/bin/chmod 0666 /dev/vfio/* > >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1 > >- $ $DPDK_DIR/tools/dpdk-devbind.py --status > >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=vfio-pci eth1 > >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status > > > > Setup OVS > > ~~~~~~~~~ > >@@ -569,7 +569,7 @@ Limitations > > The latest list of validated firmware versions can be found in the `DPDK > > release notes`_. > > > >-.. _DPDK release notes: > >http://dpdk.org/doc/guides/rel_notes/release_16_11.html > >+.. _DPDK release notes: > >http://dpdk.org/doc/guides/rel_notes/release_17_05.html > > > > Reporting Bugs > > -------------- > >diff --git a/Documentation/topics/dpdk/vhost-user.rst > >b/Documentation/topics/dpdk/vhost-user.rst > >index e76da5f..a3d5de3 100644 > >--- a/Documentation/topics/dpdk/vhost-user.rst > >+++ b/Documentation/topics/dpdk/vhost-user.rst > >@@ -292,9 +292,9 @@ To begin, instantiate a guest as described in :ref:`dpdk- > >vhost-user` or > > DPDK sources to VM and build DPDK:: > > > > $ cd /root/dpdk/ > >- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz > >- $ tar xf dpdk-16.11.2.tar.xz > >- $ export DPDK_DIR=/root/dpdk/dpdk-stable-16.11.2 > >+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz > >+ $ tar xf dpdk-17.05.1.tar.xz > >+ $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.1 > > $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > > $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > > $ cd $DPDK_DIR > >@@ -314,8 +314,8 @@ Setup huge pages and DPDK devices using UIO:: > > $ mount -t hugetlbfs hugetlbfs /dev/hugepages # only if not already > >mounted > > $ modprobe uio > > $ insmod $DPDK_BUILD/kmod/igb_uio.ko > >- $ $DPDK_DIR/tools/dpdk-devbind.py --status > >- $ $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 > >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status > >+ $ $DPDK_DIR/usertools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 > > > > .. note:: > > > >@@ -378,7 +378,7 @@ Sample XML > > </disk> > > <disk type='dir' device='disk'> > > <driver name='qemu' type='fat'/> > >- <source dir='/usr/src/dpdk-stable-16.11.2'/> > >+ <source dir='/usr/src/dpdk-stable-17.05.1'/> > > <target dev='vdb' bus='virtio'/> > > <readonly/> > > </disk> > >diff --git a/NEWS b/NEWS > >index d61fc5f..3f58e8b 100644 > >--- a/NEWS > >+++ b/NEWS > >@@ -21,6 +21,7 @@ Post-v2.7.0 > > still can be configured via extra arguments for DPDK EAL. > > * dpdkvhostuser ports are marked as deprecated. They will be removed > > in an upcoming release. > >+ * Support for DPDK v17.05.1. > > - IPFIX now provides additional counters: > > * Total counters since metering process startup. > > * Per-flow TCP flag counters. > >diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c > >index ea17b97..5e767e1 100644 > >--- a/lib/netdev-dpdk.c > >+++ b/lib/netdev-dpdk.c > >@@ -22,6 +22,9 @@ > > #include <stdlib.h> > > #include <errno.h> > > #include <unistd.h> > >+#include <linux/virtio_net.h> > >+#include <sys/socket.h> > >+#include <linux/if.h> > > > > #include <rte_config.h> > > #include <rte_cycles.h> > >@@ -31,7 +34,7 @@ > > #include <rte_malloc.h> > > #include <rte_mbuf.h> > > #include <rte_meter.h> > >-#include <rte_virtio_net.h> > >+#include <rte_vhost.h> > > > > #include "dirs.h" > > #include "dp-packet.h" > >@@ -56,6 +59,8 @@ > > #include "timeval.h" > > #include "unixctl.h" > > > >+enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM}; > >+ > > VLOG_DEFINE_THIS_MODULE(netdev_dpdk); > > static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20); > > > >@@ -171,6 +176,21 @@ static const struct rte_eth_conf port_conf = { > > }, > > }; > > > >+/* > >+ * These callbacks allow virtio-net devices to be added to vhost ports when > >+ * configuration has been fully completed. > >+ */ > >+static int new_device(int vid); > >+static void destroy_device(int vid); > >+static int vring_state_changed(int vid, uint16_t queue_id, int enable); > >+static const struct vhost_device_ops virtio_net_device_ops = > >+{ > >+ .new_device = new_device, > >+ .destroy_device = destroy_device, > >+ .vring_state_changed = vring_state_changed, > >+ .features_changed = NULL > >+}; > >+ > > enum { DPDK_RING_SIZE = 256 }; > > BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE)); > > enum { DRAIN_TSC = 200000ULL }; > >@@ -412,8 +432,8 @@ struct netdev_rxq_dpdk { > > dpdk_port_t port_id; > > }; > > > >-static int netdev_dpdk_class_init(void); > >-static int netdev_dpdk_vhost_class_init(void); > >+static void netdev_dpdk_destruct(struct netdev *netdev); > >+static void netdev_dpdk_vhost_destruct(struct netdev *netdev); > > > > int netdev_dpdk_get_vid(const struct netdev_dpdk *dev); > > > >@@ -423,8 +443,8 @@ netdev_dpdk_get_ingress_policer(const struct netdev_dpdk > >*dev); > > static bool > > is_dpdk_class(const struct netdev_class *class) > > { > >- return class->init == netdev_dpdk_class_init > >- || class->init == netdev_dpdk_vhost_class_init; > >+ return class->destruct == netdev_dpdk_destruct > >+ || class->destruct == netdev_dpdk_vhost_destruct; > > } > > > > /* DPDK NIC drivers allocate RX buffers at a particular granularity, > >typically > >@@ -948,13 +968,45 @@ netdev_dpdk_vhost_construct(struct netdev *netdev) > > if (err) { > > VLOG_ERR("vhost-user socket device setup failure for socket %s\n", > > dev->vhost_id); > >+ goto out; > > } else { > > fatal_signal_add_file_to_unlink(dev->vhost_id); > > VLOG_INFO("Socket %s created for vhost-user port %s\n", > > dev->vhost_id, name); > > } > >+ > >+ err = rte_vhost_driver_callback_register(dev->vhost_id, > >+ &virtio_net_device_ops); > >+ if (err) { > >+ VLOG_ERR("rte_vhost_driver_callback_register failed for vhost user " > >+ "port: %s\n", name); > >+ goto out; > >+ } > >+ > >+ err = rte_vhost_driver_disable_features(dev->vhost_id, > >+ 1ULL << VIRTIO_NET_F_HOST_TSO4 > >+ | 1ULL << VIRTIO_NET_F_HOST_TSO6 > >+ | 1ULL << VIRTIO_NET_F_CSUM); > >+ if (err) { > >+ VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user " > >+ "port: %s\n", name); > >+ goto out; > >+ } > >+ > >+ err = rte_vhost_driver_start(dev->vhost_id); > >+ if (err) { > >+ VLOG_ERR("rte_vhost_driver_start failed for vhost user " > >+ "port: %s\n", name); > >+ goto out; > >+ } > >+ > > err = vhost_common_construct(netdev); > >+ if (err) { > >+ VLOG_ERR("vhost_common_construct failed for vhost user " > >+ "port: %s\n", name); > >+ } > > > >+out: > > ovs_mutex_unlock(&dpdk_mutex); > > VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; " > > "please migrate to dpdkvhostuserclient ports."); > >@@ -968,6 +1020,10 @@ netdev_dpdk_vhost_client_construct(struct netdev > >*netdev) > > > > ovs_mutex_lock(&dpdk_mutex); > > err = vhost_common_construct(netdev); > >+ if (err) { > >+ VLOG_ERR("vhost_common_construct failed for vhost user client" > >+ "port: %s\n", netdev->name); > >+ } > > ovs_mutex_unlock(&dpdk_mutex); > > return err; > > } > >@@ -2456,12 +2512,9 @@ static void > > set_irq_status(int vid) > > { > > uint32_t i; > >- uint64_t idx; > > > >- for (i = 0; i < rte_vhost_get_queue_num(vid); i++) { > >- idx = i * VIRTIO_QNUM; > >- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_RXQ, 0); > >- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_TXQ, 0); > >+ for (i = 0; i < rte_vhost_get_vring_num(vid); i++) { > >+ rte_vhost_enable_guest_notification(vid, i, 0); > > } > > } > > > >@@ -2524,7 +2577,7 @@ new_device(int vid) > > LIST_FOR_EACH(dev, list_node, &dpdk_list) { > > ovs_mutex_lock(&dev->mutex); > > if (strncmp(ifname, dev->vhost_id, IF_NAME_SZ) == 0) { > >- uint32_t qp_num = rte_vhost_get_queue_num(vid); > >+ uint32_t qp_num = rte_vhost_get_vring_num(vid)/VIRTIO_QNUM; > > > > /* Get NUMA information */ > > newnode = rte_vhost_get_numa_node(vid); > >@@ -2691,27 +2744,6 @@ netdev_dpdk_get_ingress_policer(const struct > >netdev_dpdk *dev) > > return ovsrcu_get(struct ingress_policer *, &dev->ingress_policer); > > } > > > >-/* > >- * These callbacks allow virtio-net devices to be added to vhost ports when > >- * configuration has been fully complete. > >- */ > >-static const struct virtio_net_device_ops virtio_net_device_ops = > >-{ > >- .new_device = new_device, > >- .destroy_device = destroy_device, > >- .vring_state_changed = vring_state_changed > >-}; > >- > >-static void * > >-start_vhost_loop(void *dummy OVS_UNUSED) > >-{ > >- pthread_detach(pthread_self()); > >- /* Put the vhost thread into quiescent state. */ > >- ovsrcu_quiesce_start(); > >- rte_vhost_driver_session_start(); > >- return NULL; > >-} > >- > > static int > > netdev_dpdk_class_init(void) > > { > >@@ -2731,25 +2763,6 @@ netdev_dpdk_class_init(void) > > return 0; > > } > > > >-static int > >-netdev_dpdk_vhost_class_init(void) > >-{ > >- static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; > >- > >- /* This function can be called for different classes. The initialization > >- * needs to be done only once */ > >- if (ovsthread_once_start(&once)) { > >- rte_vhost_driver_callback_register(&virtio_net_device_ops); > >- rte_vhost_feature_disable(1ULL << VIRTIO_NET_F_HOST_TSO4 > >- | 1ULL << VIRTIO_NET_F_HOST_TSO6 > >- | 1ULL << VIRTIO_NET_F_CSUM); > >- ovs_thread_create("vhost_thread", start_vhost_loop, NULL); > >- > >- ovsthread_once_done(&once); > >- } > >- > >- return 0; > >-} > > > > /* Client Rings */ > > > >@@ -3204,6 +3217,31 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev > >*netdev) > > "using client socket '%s'", > > dev->up.name, dev->vhost_id); > > } > >+ > >+ err = rte_vhost_driver_callback_register(dev->vhost_id, > >+ &virtio_net_device_ops); > >+ if (err) { > >+ VLOG_ERR("rte_vhost_driver_callback_register failed for " > >+ "vhost user client port: %s\n", dev->up.name); > >+ goto unlock; > >+ } > >+ > >+ err = rte_vhost_driver_disable_features(dev->vhost_id, > >+ 1ULL << VIRTIO_NET_F_HOST_TSO4 > >+ | 1ULL << VIRTIO_NET_F_HOST_TSO6 > >+ | 1ULL << VIRTIO_NET_F_CSUM); > >+ if (err) { > >+ VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user > >" > >+ "client port: %s\n", dev->up.name); > >+ goto unlock; > >+ } > >+ > >+ err = rte_vhost_driver_start(dev->vhost_id); > >+ if (err) { > >+ VLOG_ERR("rte_vhost_driver_start failed for vhost user " > >+ "client port: %s\n", dev->up.name); > >+ goto unlock; > >+ } > > } > > > > err = dpdk_vhost_reconfigure_helper(dev); > >@@ -3324,7 +3362,7 @@ static const struct netdev_class dpdk_ring_class = > > static const struct netdev_class dpdk_vhost_class = > > NETDEV_DPDK_CLASS( > > "dpdkvhostuser", > >- netdev_dpdk_vhost_class_init, > >+ NULL, > > netdev_dpdk_vhost_construct, > > netdev_dpdk_vhost_destruct, > > NULL, > >@@ -3339,7 +3377,7 @@ static const struct netdev_class dpdk_vhost_class = > > static const struct netdev_class dpdk_vhost_client_class = > > NETDEV_DPDK_CLASS( > > "dpdkvhostuserclient", > >- netdev_dpdk_vhost_class_init, > >+ NULL, > > netdev_dpdk_vhost_client_construct, > > netdev_dpdk_vhost_destruct, > > netdev_dpdk_vhost_client_set_config, > >diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch-fedora.spec.in > >index 3a045d3..2bb7102 100644 > >--- a/rhel/openvswitch-fedora.spec.in > >+++ b/rhel/openvswitch-fedora.spec.in > >@@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel > > %endif > > %if %{with dpdk} > > BuildRequires: libpcap-devel numactl-devel > >-BuildRequires: dpdk-devel >= 16.11 > >+BuildRequires: dpdk-devel >= 17.05.1 > > Provides: %{name}-dpdk = %{version}-%{release} > > %endif > > > >diff --git a/tests/dpdk/ring_client.c b/tests/dpdk/ring_client.c > >index b153713..8cc3fb5 100644 > >--- a/tests/dpdk/ring_client.c > >+++ b/tests/dpdk/ring_client.c > >@@ -185,15 +185,15 @@ main(int argc, char *argv[]) > > /* Try dequeuing max possible packets first, if that fails, get the > > * most we can. Loop body should only execute once, maximum. > > */ > >- while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, rx_pkts) != 0) > >&& > >- rx_pkts > 0) { > >+ while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, > >+ rx_pkts, NULL) != 0) && rx_pkts > 0) { > > rx_pkts = (uint16_t)RTE_MIN(rte_ring_count(rx_ring), > >PKT_READ_SIZE); > > } > > > > if (rx_pkts > 0) { > > /* blocking enqueue */ > > do { > >- rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts); > >+ rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts, NULL); > > } while (rslt == -ENOBUFS); > > } > > } > >-- > >1.8.3.1 > > > >_______________________________________________ > >dev mailing list > >dev@openvswitch.org > >https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>From: Weglicki, MichalX >Sent: Wednesday, July 26, 2017 10:52 AM >To: Kavanagh, Mark B <mark.b.kavanagh@intel.com>; dev@openvswitch.org >Subject: RE: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for >DPDK 17.05.1. > >Hey Mark, > >Sorry, I had some problems with rebase, and somehow I just simply forgot t to >do so. Do you want me to re-apply this patch as v5? No need Michal - hopefully the maintainer will just roll it in when applying to master. Thanks, Mark > >> -----Original Message----- >> From: Kavanagh, Mark B >> Sent: Wednesday, July 26, 2017 10:04 AM >> To: Weglicki, MichalX <michalx.weglicki@intel.com>; dev@openvswitch.org >> Subject: RE: [ovs-dev] [PATCH v4] Update relevant artifacts to add support >for DPDK 17.05.1. >> >> >> >> >-----Original Message----- >> >From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev- >bounces@openvswitch.org] >> >On Behalf Of Michal Weglicki >> >Sent: Tuesday, July 25, 2017 1:35 PM >> >To: dev@openvswitch.org >> >Subject: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for >> >DPDK 17.05.1. >> > >> >Upgrading to DPDK 17.05.1 stable release adds new >> >significant features relevant to OVS, including, >> >but not limited to: >> >- tun/tap PMD, >> >- VFIO hotplug support, >> >- Generic flow API. >> > >> >Following changes are applied: >> >- netdev-dpdk: Changes required by DPDK API modifications. >> >- doc: Because of DPDK API changes, backward compatibility >> > with previous DPDK releases will be broken, thus all >> > relevant documentation entries are updated. >> >- .travis: DPDK version change from 16.11.1 to 17.05.1. >> >- rhel/openvswitch-fedora.spec.in: DPDK version change >> > from 16.11 to 17.05.1 >> > >> >v1->v2: Patch rebase. >> >v2->v3: Fixed wrong formating after v2 patch rebase. >> >v3->v4: Minor documentation changes. >> > >> >Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> >> >Reviewed-by: Aaron Conole <aconole@redhat.com> >> >> >> Hi Michal, >> >> Is there a reason why you haven't added my acked-by to this latest patch? >https://mail.openvswitch.org/pipermail/ovs-dev/2017- >> July/334893.html >> >> Thanks, >> Mark >> >> >--- >> > .travis/linux-build.sh | 2 +- >> > Documentation/faq/releases.rst | 1 + >> > Documentation/howto/dpdk.rst | 6 +- >> > Documentation/intro/install/dpdk.rst | 14 +-- >> > Documentation/topics/dpdk/vhost-user.rst | 12 +-- >> > NEWS | 1 + >> > lib/netdev-dpdk.c | 144 +++++++++++++++++++-------- >--- >> >- >> > rhel/openvswitch-fedora.spec.in | 2 +- >> > tests/dpdk/ring_client.c | 6 +- >> > 9 files changed, 114 insertions(+), 74 deletions(-) >> > >> >diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh >> >index f66b534..efccdf1 100755 >> >--- a/.travis/linux-build.sh >> >+++ b/.travis/linux-build.sh >> >@@ -80,7 +80,7 @@ fi >> > >> > if [ "$DPDK" ]; then >> > if [ -z "$DPDK_VER" ]; then >> >- DPDK_VER="16.11.2" >> >+ DPDK_VER="17.05.1" >> > fi >> > install_dpdk $DPDK_VER >> > if [ "$CC" = "clang" ]; then >> >diff --git a/Documentation/faq/releases.rst >b/Documentation/faq/releases.rst >> >index 707834b..2ecc24c 100644 >> >--- a/Documentation/faq/releases.rst >> >+++ b/Documentation/faq/releases.rst >> >@@ -161,6 +161,7 @@ Q: What DPDK version does each Open vSwitch release >work >> >with? >> > 2.5.x 2.2 >> > 2.6.x 16.07.2 >> > 2.7.x 16.11.2 >> >+ 2.8.x 17.05.1 >> > ============ ======= >> > >> > Q: I get an error like this when I configure Open vSwitch: >> >diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst >> >index af01d3e..1756c8c 100644 >> >--- a/Documentation/howto/dpdk.rst >> >+++ b/Documentation/howto/dpdk.rst >> >@@ -528,15 +528,15 @@ described in :ref:`dpdk-testpmd`. Once compiled, run >the >> >application:: >> > >> > When you finish testing, bind the vNICs back to kernel:: >> > >> >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 >> >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 >> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 >> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 >> > >> > .. note:: >> > >> > Valid PCI IDs must be passed in above example. The PCI IDs can be >retrieved >> > like so:: >> > >> >- $ $DPDK_DIR/tools/dpdk-devbind.py --status >> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status >> > >> > More information on the dpdkvhostuser ports can be found in >> > :doc:`/topics/dpdk/vhost-user`. >> >diff --git a/Documentation/intro/install/dpdk.rst >> >b/Documentation/intro/install/dpdk.rst >> >index a05aa1a..20baa63 100644 >> >--- a/Documentation/intro/install/dpdk.rst >> >+++ b/Documentation/intro/install/dpdk.rst >> >@@ -40,7 +40,7 @@ Build requirements >> > In addition to the requirements described in :doc:`general`, building Open >> > vSwitch with DPDK will require the following: >> > >> >-- DPDK 16.11 >> >+- DPDK 17.05.1 >> > >> > - A `DPDK supported NIC`_ >> > >> >@@ -69,9 +69,9 @@ Install DPDK >> > #. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``:: >> > >> > $ cd /usr/src/ >> >- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz >> >- $ tar xf dpdk-16.11.2.tar.xz >> >- $ export DPDK_DIR=/usr/src/dpdk-stable-16.11.2 >> >+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz >> >+ $ tar xf dpdk-17.05.1.tar.xz >> >+ $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.1 >> > $ cd $DPDK_DIR >> > >> > #. (Optional) Configure DPDK as a shared library >> >@@ -187,8 +187,8 @@ to the VFIO driver:: >> > $ modprobe vfio-pci >> > $ /usr/bin/chmod a+x /dev/vfio >> > $ /usr/bin/chmod 0666 /dev/vfio/* >> >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1 >> >- $ $DPDK_DIR/tools/dpdk-devbind.py --status >> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=vfio-pci eth1 >> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status >> > >> > Setup OVS >> > ~~~~~~~~~ >> >@@ -569,7 +569,7 @@ Limitations >> > The latest list of validated firmware versions can be found in the `DPDK >> > release notes`_. >> > >> >-.. _DPDK release notes: >> >http://dpdk.org/doc/guides/rel_notes/release_16_11.html >> >+.. _DPDK release notes: >> >http://dpdk.org/doc/guides/rel_notes/release_17_05.html >> > >> > Reporting Bugs >> > -------------- >> >diff --git a/Documentation/topics/dpdk/vhost-user.rst >> >b/Documentation/topics/dpdk/vhost-user.rst >> >index e76da5f..a3d5de3 100644 >> >--- a/Documentation/topics/dpdk/vhost-user.rst >> >+++ b/Documentation/topics/dpdk/vhost-user.rst >> >@@ -292,9 +292,9 @@ To begin, instantiate a guest as described in >:ref:`dpdk- >> >vhost-user` or >> > DPDK sources to VM and build DPDK:: >> > >> > $ cd /root/dpdk/ >> >- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz >> >- $ tar xf dpdk-16.11.2.tar.xz >> >- $ export DPDK_DIR=/root/dpdk/dpdk-stable-16.11.2 >> >+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz >> >+ $ tar xf dpdk-17.05.1.tar.xz >> >+ $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.1 >> > $ export DPDK_TARGET=x86_64-native-linuxapp-gcc >> > $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET >> > $ cd $DPDK_DIR >> >@@ -314,8 +314,8 @@ Setup huge pages and DPDK devices using UIO:: >> > $ mount -t hugetlbfs hugetlbfs /dev/hugepages # only if not already >> >mounted >> > $ modprobe uio >> > $ insmod $DPDK_BUILD/kmod/igb_uio.ko >> >- $ $DPDK_DIR/tools/dpdk-devbind.py --status >> >- $ $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 >> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status >> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 >> > >> > .. note:: >> > >> >@@ -378,7 +378,7 @@ Sample XML >> > </disk> >> > <disk type='dir' device='disk'> >> > <driver name='qemu' type='fat'/> >> >- <source dir='/usr/src/dpdk-stable-16.11.2'/> >> >+ <source dir='/usr/src/dpdk-stable-17.05.1'/> >> > <target dev='vdb' bus='virtio'/> >> > <readonly/> >> > </disk> >> >diff --git a/NEWS b/NEWS >> >index d61fc5f..3f58e8b 100644 >> >--- a/NEWS >> >+++ b/NEWS >> >@@ -21,6 +21,7 @@ Post-v2.7.0 >> > still can be configured via extra arguments for DPDK EAL. >> > * dpdkvhostuser ports are marked as deprecated. They will be removed >> > in an upcoming release. >> >+ * Support for DPDK v17.05.1. >> > - IPFIX now provides additional counters: >> > * Total counters since metering process startup. >> > * Per-flow TCP flag counters. >> >diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c >> >index ea17b97..5e767e1 100644 >> >--- a/lib/netdev-dpdk.c >> >+++ b/lib/netdev-dpdk.c >> >@@ -22,6 +22,9 @@ >> > #include <stdlib.h> >> > #include <errno.h> >> > #include <unistd.h> >> >+#include <linux/virtio_net.h> >> >+#include <sys/socket.h> >> >+#include <linux/if.h> >> > >> > #include <rte_config.h> >> > #include <rte_cycles.h> >> >@@ -31,7 +34,7 @@ >> > #include <rte_malloc.h> >> > #include <rte_mbuf.h> >> > #include <rte_meter.h> >> >-#include <rte_virtio_net.h> >> >+#include <rte_vhost.h> >> > >> > #include "dirs.h" >> > #include "dp-packet.h" >> >@@ -56,6 +59,8 @@ >> > #include "timeval.h" >> > #include "unixctl.h" >> > >> >+enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM}; >> >+ >> > VLOG_DEFINE_THIS_MODULE(netdev_dpdk); >> > static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20); >> > >> >@@ -171,6 +176,21 @@ static const struct rte_eth_conf port_conf = { >> > }, >> > }; >> > >> >+/* >> >+ * These callbacks allow virtio-net devices to be added to vhost ports >when >> >+ * configuration has been fully completed. >> >+ */ >> >+static int new_device(int vid); >> >+static void destroy_device(int vid); >> >+static int vring_state_changed(int vid, uint16_t queue_id, int enable); >> >+static const struct vhost_device_ops virtio_net_device_ops = >> >+{ >> >+ .new_device = new_device, >> >+ .destroy_device = destroy_device, >> >+ .vring_state_changed = vring_state_changed, >> >+ .features_changed = NULL >> >+}; >> >+ >> > enum { DPDK_RING_SIZE = 256 }; >> > BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE)); >> > enum { DRAIN_TSC = 200000ULL }; >> >@@ -412,8 +432,8 @@ struct netdev_rxq_dpdk { >> > dpdk_port_t port_id; >> > }; >> > >> >-static int netdev_dpdk_class_init(void); >> >-static int netdev_dpdk_vhost_class_init(void); >> >+static void netdev_dpdk_destruct(struct netdev *netdev); >> >+static void netdev_dpdk_vhost_destruct(struct netdev *netdev); >> > >> > int netdev_dpdk_get_vid(const struct netdev_dpdk *dev); >> > >> >@@ -423,8 +443,8 @@ netdev_dpdk_get_ingress_policer(const struct >netdev_dpdk >> >*dev); >> > static bool >> > is_dpdk_class(const struct netdev_class *class) >> > { >> >- return class->init == netdev_dpdk_class_init >> >- || class->init == netdev_dpdk_vhost_class_init; >> >+ return class->destruct == netdev_dpdk_destruct >> >+ || class->destruct == netdev_dpdk_vhost_destruct; >> > } >> > >> > /* DPDK NIC drivers allocate RX buffers at a particular granularity, >> >typically >> >@@ -948,13 +968,45 @@ netdev_dpdk_vhost_construct(struct netdev *netdev) >> > if (err) { >> > VLOG_ERR("vhost-user socket device setup failure for socket %s\n", >> > dev->vhost_id); >> >+ goto out; >> > } else { >> > fatal_signal_add_file_to_unlink(dev->vhost_id); >> > VLOG_INFO("Socket %s created for vhost-user port %s\n", >> > dev->vhost_id, name); >> > } >> >+ >> >+ err = rte_vhost_driver_callback_register(dev->vhost_id, >> >+ &virtio_net_device_ops); >> >+ if (err) { >> >+ VLOG_ERR("rte_vhost_driver_callback_register failed for vhost user >" >> >+ "port: %s\n", name); >> >+ goto out; >> >+ } >> >+ >> >+ err = rte_vhost_driver_disable_features(dev->vhost_id, >> >+ 1ULL << VIRTIO_NET_F_HOST_TSO4 >> >+ | 1ULL << VIRTIO_NET_F_HOST_TSO6 >> >+ | 1ULL << VIRTIO_NET_F_CSUM); >> >+ if (err) { >> >+ VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user >" >> >+ "port: %s\n", name); >> >+ goto out; >> >+ } >> >+ >> >+ err = rte_vhost_driver_start(dev->vhost_id); >> >+ if (err) { >> >+ VLOG_ERR("rte_vhost_driver_start failed for vhost user " >> >+ "port: %s\n", name); >> >+ goto out; >> >+ } >> >+ >> > err = vhost_common_construct(netdev); >> >+ if (err) { >> >+ VLOG_ERR("vhost_common_construct failed for vhost user " >> >+ "port: %s\n", name); >> >+ } >> > >> >+out: >> > ovs_mutex_unlock(&dpdk_mutex); >> > VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; " >> > "please migrate to dpdkvhostuserclient ports."); >> >@@ -968,6 +1020,10 @@ netdev_dpdk_vhost_client_construct(struct netdev >> >*netdev) >> > >> > ovs_mutex_lock(&dpdk_mutex); >> > err = vhost_common_construct(netdev); >> >+ if (err) { >> >+ VLOG_ERR("vhost_common_construct failed for vhost user client" >> >+ "port: %s\n", netdev->name); >> >+ } >> > ovs_mutex_unlock(&dpdk_mutex); >> > return err; >> > } >> >@@ -2456,12 +2512,9 @@ static void >> > set_irq_status(int vid) >> > { >> > uint32_t i; >> >- uint64_t idx; >> > >> >- for (i = 0; i < rte_vhost_get_queue_num(vid); i++) { >> >- idx = i * VIRTIO_QNUM; >> >- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_RXQ, 0); >> >- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_TXQ, 0); >> >+ for (i = 0; i < rte_vhost_get_vring_num(vid); i++) { >> >+ rte_vhost_enable_guest_notification(vid, i, 0); >> > } >> > } >> > >> >@@ -2524,7 +2577,7 @@ new_device(int vid) >> > LIST_FOR_EACH(dev, list_node, &dpdk_list) { >> > ovs_mutex_lock(&dev->mutex); >> > if (strncmp(ifname, dev->vhost_id, IF_NAME_SZ) == 0) { >> >- uint32_t qp_num = rte_vhost_get_queue_num(vid); >> >+ uint32_t qp_num = rte_vhost_get_vring_num(vid)/VIRTIO_QNUM; >> > >> > /* Get NUMA information */ >> > newnode = rte_vhost_get_numa_node(vid); >> >@@ -2691,27 +2744,6 @@ netdev_dpdk_get_ingress_policer(const struct >> >netdev_dpdk *dev) >> > return ovsrcu_get(struct ingress_policer *, &dev->ingress_policer); >> > } >> > >> >-/* >> >- * These callbacks allow virtio-net devices to be added to vhost ports >when >> >- * configuration has been fully complete. >> >- */ >> >-static const struct virtio_net_device_ops virtio_net_device_ops = >> >-{ >> >- .new_device = new_device, >> >- .destroy_device = destroy_device, >> >- .vring_state_changed = vring_state_changed >> >-}; >> >- >> >-static void * >> >-start_vhost_loop(void *dummy OVS_UNUSED) >> >-{ >> >- pthread_detach(pthread_self()); >> >- /* Put the vhost thread into quiescent state. */ >> >- ovsrcu_quiesce_start(); >> >- rte_vhost_driver_session_start(); >> >- return NULL; >> >-} >> >- >> > static int >> > netdev_dpdk_class_init(void) >> > { >> >@@ -2731,25 +2763,6 @@ netdev_dpdk_class_init(void) >> > return 0; >> > } >> > >> >-static int >> >-netdev_dpdk_vhost_class_init(void) >> >-{ >> >- static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; >> >- >> >- /* This function can be called for different classes. The >initialization >> >- * needs to be done only once */ >> >- if (ovsthread_once_start(&once)) { >> >- rte_vhost_driver_callback_register(&virtio_net_device_ops); >> >- rte_vhost_feature_disable(1ULL << VIRTIO_NET_F_HOST_TSO4 >> >- | 1ULL << VIRTIO_NET_F_HOST_TSO6 >> >- | 1ULL << VIRTIO_NET_F_CSUM); >> >- ovs_thread_create("vhost_thread", start_vhost_loop, NULL); >> >- >> >- ovsthread_once_done(&once); >> >- } >> >- >> >- return 0; >> >-} >> > >> > /* Client Rings */ >> > >> >@@ -3204,6 +3217,31 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev >> >*netdev) >> > "using client socket '%s'", >> > dev->up.name, dev->vhost_id); >> > } >> >+ >> >+ err = rte_vhost_driver_callback_register(dev->vhost_id, >> >+ &virtio_net_device_ops); >> >+ if (err) { >> >+ VLOG_ERR("rte_vhost_driver_callback_register failed for " >> >+ "vhost user client port: %s\n", dev->up.name); >> >+ goto unlock; >> >+ } >> >+ >> >+ err = rte_vhost_driver_disable_features(dev->vhost_id, >> >+ 1ULL << VIRTIO_NET_F_HOST_TSO4 >> >+ | 1ULL << VIRTIO_NET_F_HOST_TSO6 >> >+ | 1ULL << VIRTIO_NET_F_CSUM); >> >+ if (err) { >> >+ VLOG_ERR("rte_vhost_driver_disable_features failed for vhost >user >> >" >> >+ "client port: %s\n", dev->up.name); >> >+ goto unlock; >> >+ } >> >+ >> >+ err = rte_vhost_driver_start(dev->vhost_id); >> >+ if (err) { >> >+ VLOG_ERR("rte_vhost_driver_start failed for vhost user " >> >+ "client port: %s\n", dev->up.name); >> >+ goto unlock; >> >+ } >> > } >> > >> > err = dpdk_vhost_reconfigure_helper(dev); >> >@@ -3324,7 +3362,7 @@ static const struct netdev_class dpdk_ring_class = >> > static const struct netdev_class dpdk_vhost_class = >> > NETDEV_DPDK_CLASS( >> > "dpdkvhostuser", >> >- netdev_dpdk_vhost_class_init, >> >+ NULL, >> > netdev_dpdk_vhost_construct, >> > netdev_dpdk_vhost_destruct, >> > NULL, >> >@@ -3339,7 +3377,7 @@ static const struct netdev_class dpdk_vhost_class = >> > static const struct netdev_class dpdk_vhost_client_class = >> > NETDEV_DPDK_CLASS( >> > "dpdkvhostuserclient", >> >- netdev_dpdk_vhost_class_init, >> >+ NULL, >> > netdev_dpdk_vhost_client_construct, >> > netdev_dpdk_vhost_destruct, >> > netdev_dpdk_vhost_client_set_config, >> >diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch- >fedora.spec.in >> >index 3a045d3..2bb7102 100644 >> >--- a/rhel/openvswitch-fedora.spec.in >> >+++ b/rhel/openvswitch-fedora.spec.in >> >@@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel >> > %endif >> > %if %{with dpdk} >> > BuildRequires: libpcap-devel numactl-devel >> >-BuildRequires: dpdk-devel >= 16.11 >> >+BuildRequires: dpdk-devel >= 17.05.1 >> > Provides: %{name}-dpdk = %{version}-%{release} >> > %endif >> > >> >diff --git a/tests/dpdk/ring_client.c b/tests/dpdk/ring_client.c >> >index b153713..8cc3fb5 100644 >> >--- a/tests/dpdk/ring_client.c >> >+++ b/tests/dpdk/ring_client.c >> >@@ -185,15 +185,15 @@ main(int argc, char *argv[]) >> > /* Try dequeuing max possible packets first, if that fails, get >the >> > * most we can. Loop body should only execute once, maximum. >> > */ >> >- while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, rx_pkts) != >0) >> >&& >> >- rx_pkts > 0) { >> >+ while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, >> >+ rx_pkts, NULL) != 0) && rx_pkts > 0) { >> > rx_pkts = (uint16_t)RTE_MIN(rte_ring_count(rx_ring), >> >PKT_READ_SIZE); >> > } >> > >> > if (rx_pkts > 0) { >> > /* blocking enqueue */ >> > do { >> >- rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts); >> >+ rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts, >NULL); >> > } while (rslt == -ENOBUFS); >> > } >> > } >> >-- >> >1.8.3.1 >> > >> >_______________________________________________ >> >dev mailing list >> >dev@openvswitch.org >> >https://mail.openvswitch.org/mailman/listinfo/ovs-dev
"Kavanagh, Mark B" <mark.b.kavanagh@intel.com> writes: >>From: Weglicki, MichalX >>Sent: Wednesday, July 26, 2017 10:52 AM >>To: Kavanagh, Mark B <mark.b.kavanagh@intel.com>; dev@openvswitch.org >>Subject: RE: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for >>DPDK 17.05.1. >> >>Hey Mark, >> >>Sorry, I had some problems with rebase, and somehow I just simply forgot t to >>do so. Do you want me to re-apply this patch as v5? > > No need Michal - hopefully the maintainer will just roll it in when > applying to master. If you reply with your Acked-by tag, the patchwork instance will re-apply it and the maintainer will not have to remember. > Thanks, > Mark > >> >>> -----Original Message----- >>> From: Kavanagh, Mark B >>> Sent: Wednesday, July 26, 2017 10:04 AM >>> To: Weglicki, MichalX <michalx.weglicki@intel.com>; dev@openvswitch.org >>> Subject: RE: [ovs-dev] [PATCH v4] Update relevant artifacts to add support >>for DPDK 17.05.1. >>> >>> >>> >>> >-----Original Message----- >>> >From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev- >>bounces@openvswitch.org] >>> >On Behalf Of Michal Weglicki >>> >Sent: Tuesday, July 25, 2017 1:35 PM >>> >To: dev@openvswitch.org >>> >Subject: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for >>> >DPDK 17.05.1. >>> > >>> >Upgrading to DPDK 17.05.1 stable release adds new >>> >significant features relevant to OVS, including, >>> >but not limited to: >>> >- tun/tap PMD, >>> >- VFIO hotplug support, >>> >- Generic flow API. >>> > >>> >Following changes are applied: >>> >- netdev-dpdk: Changes required by DPDK API modifications. >>> >- doc: Because of DPDK API changes, backward compatibility >>> > with previous DPDK releases will be broken, thus all >>> > relevant documentation entries are updated. >>> >- .travis: DPDK version change from 16.11.1 to 17.05.1. >>> >- rhel/openvswitch-fedora.spec.in: DPDK version change >>> > from 16.11 to 17.05.1 >>> > >>> >v1->v2: Patch rebase. >>> >v2->v3: Fixed wrong formating after v2 patch rebase. >>> >v3->v4: Minor documentation changes. >>> > >>> >Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> >>> >Reviewed-by: Aaron Conole <aconole@redhat.com> >>> >>> >>> Hi Michal, >>> >>> Is there a reason why you haven't added my acked-by to this latest patch? >>https://mail.openvswitch.org/pipermail/ovs-dev/2017- >>> July/334893.html >>> >>> Thanks, >>> Mark >>> >>> >--- >>> > .travis/linux-build.sh | 2 +- >>> > Documentation/faq/releases.rst | 1 + >>> > Documentation/howto/dpdk.rst | 6 +- >>> > Documentation/intro/install/dpdk.rst | 14 +-- >>> > Documentation/topics/dpdk/vhost-user.rst | 12 +-- >>> > NEWS | 1 + >>> > lib/netdev-dpdk.c | 144 +++++++++++++++++++-------- >>--- >>> >- >>> > rhel/openvswitch-fedora.spec.in | 2 +- >>> > tests/dpdk/ring_client.c | 6 +- >>> > 9 files changed, 114 insertions(+), 74 deletions(-) >>> > >>> >diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh >>> >index f66b534..efccdf1 100755 >>> >--- a/.travis/linux-build.sh >>> >+++ b/.travis/linux-build.sh >>> >@@ -80,7 +80,7 @@ fi >>> > >>> > if [ "$DPDK" ]; then >>> > if [ -z "$DPDK_VER" ]; then >>> >- DPDK_VER="16.11.2" >>> >+ DPDK_VER="17.05.1" >>> > fi >>> > install_dpdk $DPDK_VER >>> > if [ "$CC" = "clang" ]; then >>> >diff --git a/Documentation/faq/releases.rst >>b/Documentation/faq/releases.rst >>> >index 707834b..2ecc24c 100644 >>> >--- a/Documentation/faq/releases.rst >>> >+++ b/Documentation/faq/releases.rst >>> >@@ -161,6 +161,7 @@ Q: What DPDK version does each Open vSwitch release >>work >>> >with? >>> > 2.5.x 2.2 >>> > 2.6.x 16.07.2 >>> > 2.7.x 16.11.2 >>> >+ 2.8.x 17.05.1 >>> > ============ ======= >>> > >>> > Q: I get an error like this when I configure Open vSwitch: >>> >diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst >>> >index af01d3e..1756c8c 100644 >>> >--- a/Documentation/howto/dpdk.rst >>> >+++ b/Documentation/howto/dpdk.rst >>> >@@ -528,15 +528,15 @@ described in :ref:`dpdk-testpmd`. Once compiled, run >>the >>> >application:: >>> > >>> > When you finish testing, bind the vNICs back to kernel:: >>> > >>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 >>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 >>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 >>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 >>> > >>> > .. note:: >>> > >>> > Valid PCI IDs must be passed in above example. The PCI IDs can be >>retrieved >>> > like so:: >>> > >>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --status >>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status >>> > >>> > More information on the dpdkvhostuser ports can be found in >>> > :doc:`/topics/dpdk/vhost-user`. >>> >diff --git a/Documentation/intro/install/dpdk.rst >>> >b/Documentation/intro/install/dpdk.rst >>> >index a05aa1a..20baa63 100644 >>> >--- a/Documentation/intro/install/dpdk.rst >>> >+++ b/Documentation/intro/install/dpdk.rst >>> >@@ -40,7 +40,7 @@ Build requirements >>> > In addition to the requirements described in :doc:`general`, building Open >>> > vSwitch with DPDK will require the following: >>> > >>> >-- DPDK 16.11 >>> >+- DPDK 17.05.1 >>> > >>> > - A `DPDK supported NIC`_ >>> > >>> >@@ -69,9 +69,9 @@ Install DPDK >>> > #. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``:: >>> > >>> > $ cd /usr/src/ >>> >- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz >>> >- $ tar xf dpdk-16.11.2.tar.xz >>> >- $ export DPDK_DIR=/usr/src/dpdk-stable-16.11.2 >>> >+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz >>> >+ $ tar xf dpdk-17.05.1.tar.xz >>> >+ $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.1 >>> > $ cd $DPDK_DIR >>> > >>> > #. (Optional) Configure DPDK as a shared library >>> >@@ -187,8 +187,8 @@ to the VFIO driver:: >>> > $ modprobe vfio-pci >>> > $ /usr/bin/chmod a+x /dev/vfio >>> > $ /usr/bin/chmod 0666 /dev/vfio/* >>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1 >>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --status >>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=vfio-pci eth1 >>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status >>> > >>> > Setup OVS >>> > ~~~~~~~~~ >>> >@@ -569,7 +569,7 @@ Limitations >>> > The latest list of validated firmware versions can be found in the `DPDK >>> > release notes`_. >>> > >>> >-.. _DPDK release notes: >>> >http://dpdk.org/doc/guides/rel_notes/release_16_11.html >>> >+.. _DPDK release notes: >>> >http://dpdk.org/doc/guides/rel_notes/release_17_05.html >>> > >>> > Reporting Bugs >>> > -------------- >>> >diff --git a/Documentation/topics/dpdk/vhost-user.rst >>> >b/Documentation/topics/dpdk/vhost-user.rst >>> >index e76da5f..a3d5de3 100644 >>> >--- a/Documentation/topics/dpdk/vhost-user.rst >>> >+++ b/Documentation/topics/dpdk/vhost-user.rst >>> >@@ -292,9 +292,9 @@ To begin, instantiate a guest as described in >>:ref:`dpdk- >>> >vhost-user` or >>> > DPDK sources to VM and build DPDK:: >>> > >>> > $ cd /root/dpdk/ >>> >- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz >>> >- $ tar xf dpdk-16.11.2.tar.xz >>> >- $ export DPDK_DIR=/root/dpdk/dpdk-stable-16.11.2 >>> >+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz >>> >+ $ tar xf dpdk-17.05.1.tar.xz >>> >+ $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.1 >>> > $ export DPDK_TARGET=x86_64-native-linuxapp-gcc >>> > $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET >>> > $ cd $DPDK_DIR >>> >@@ -314,8 +314,8 @@ Setup huge pages and DPDK devices using UIO:: >>> > $ mount -t hugetlbfs hugetlbfs /dev/hugepages # only if not already >>> >mounted >>> > $ modprobe uio >>> > $ insmod $DPDK_BUILD/kmod/igb_uio.ko >>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --status >>> >- $ $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 >>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status >>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 >>> > >>> > .. note:: >>> > >>> >@@ -378,7 +378,7 @@ Sample XML >>> > </disk> >>> > <disk type='dir' device='disk'> >>> > <driver name='qemu' type='fat'/> >>> >- <source dir='/usr/src/dpdk-stable-16.11.2'/> >>> >+ <source dir='/usr/src/dpdk-stable-17.05.1'/> >>> > <target dev='vdb' bus='virtio'/> >>> > <readonly/> >>> > </disk> >>> >diff --git a/NEWS b/NEWS >>> >index d61fc5f..3f58e8b 100644 >>> >--- a/NEWS >>> >+++ b/NEWS >>> >@@ -21,6 +21,7 @@ Post-v2.7.0 >>> > still can be configured via extra arguments for DPDK EAL. >>> > * dpdkvhostuser ports are marked as deprecated. They will be removed >>> > in an upcoming release. >>> >+ * Support for DPDK v17.05.1. >>> > - IPFIX now provides additional counters: >>> > * Total counters since metering process startup. >>> > * Per-flow TCP flag counters. >>> >diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c >>> >index ea17b97..5e767e1 100644 >>> >--- a/lib/netdev-dpdk.c >>> >+++ b/lib/netdev-dpdk.c >>> >@@ -22,6 +22,9 @@ >>> > #include <stdlib.h> >>> > #include <errno.h> >>> > #include <unistd.h> >>> >+#include <linux/virtio_net.h> >>> >+#include <sys/socket.h> >>> >+#include <linux/if.h> >>> > >>> > #include <rte_config.h> >>> > #include <rte_cycles.h> >>> >@@ -31,7 +34,7 @@ >>> > #include <rte_malloc.h> >>> > #include <rte_mbuf.h> >>> > #include <rte_meter.h> >>> >-#include <rte_virtio_net.h> >>> >+#include <rte_vhost.h> >>> > >>> > #include "dirs.h" >>> > #include "dp-packet.h" >>> >@@ -56,6 +59,8 @@ >>> > #include "timeval.h" >>> > #include "unixctl.h" >>> > >>> >+enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM}; >>> >+ >>> > VLOG_DEFINE_THIS_MODULE(netdev_dpdk); >>> > static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20); >>> > >>> >@@ -171,6 +176,21 @@ static const struct rte_eth_conf port_conf = { >>> > }, >>> > }; >>> > >>> >+/* >>> >+ * These callbacks allow virtio-net devices to be added to vhost ports >>when >>> >+ * configuration has been fully completed. >>> >+ */ >>> >+static int new_device(int vid); >>> >+static void destroy_device(int vid); >>> >+static int vring_state_changed(int vid, uint16_t queue_id, int enable); >>> >+static const struct vhost_device_ops virtio_net_device_ops = >>> >+{ >>> >+ .new_device = new_device, >>> >+ .destroy_device = destroy_device, >>> >+ .vring_state_changed = vring_state_changed, >>> >+ .features_changed = NULL >>> >+}; >>> >+ >>> > enum { DPDK_RING_SIZE = 256 }; >>> > BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE)); >>> > enum { DRAIN_TSC = 200000ULL }; >>> >@@ -412,8 +432,8 @@ struct netdev_rxq_dpdk { >>> > dpdk_port_t port_id; >>> > }; >>> > >>> >-static int netdev_dpdk_class_init(void); >>> >-static int netdev_dpdk_vhost_class_init(void); >>> >+static void netdev_dpdk_destruct(struct netdev *netdev); >>> >+static void netdev_dpdk_vhost_destruct(struct netdev *netdev); >>> > >>> > int netdev_dpdk_get_vid(const struct netdev_dpdk *dev); >>> > >>> >@@ -423,8 +443,8 @@ netdev_dpdk_get_ingress_policer(const struct >>netdev_dpdk >>> >*dev); >>> > static bool >>> > is_dpdk_class(const struct netdev_class *class) >>> > { >>> >- return class->init == netdev_dpdk_class_init >>> >- || class->init == netdev_dpdk_vhost_class_init; >>> >+ return class->destruct == netdev_dpdk_destruct >>> >+ || class->destruct == netdev_dpdk_vhost_destruct; >>> > } >>> > >>> > /* DPDK NIC drivers allocate RX buffers at a particular granularity, >>> >typically >>> >@@ -948,13 +968,45 @@ netdev_dpdk_vhost_construct(struct netdev *netdev) >>> > if (err) { >>> > VLOG_ERR("vhost-user socket device setup failure for socket %s\n", >>> > dev->vhost_id); >>> >+ goto out; >>> > } else { >>> > fatal_signal_add_file_to_unlink(dev->vhost_id); >>> > VLOG_INFO("Socket %s created for vhost-user port %s\n", >>> > dev->vhost_id, name); >>> > } >>> >+ >>> >+ err = rte_vhost_driver_callback_register(dev->vhost_id, >>> >+ &virtio_net_device_ops); >>> >+ if (err) { >>> >+ VLOG_ERR("rte_vhost_driver_callback_register failed for vhost user >>" >>> >+ "port: %s\n", name); >>> >+ goto out; >>> >+ } >>> >+ >>> >+ err = rte_vhost_driver_disable_features(dev->vhost_id, >>> >+ 1ULL << VIRTIO_NET_F_HOST_TSO4 >>> >+ | 1ULL << VIRTIO_NET_F_HOST_TSO6 >>> >+ | 1ULL << VIRTIO_NET_F_CSUM); >>> >+ if (err) { >>> >+ VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user >>" >>> >+ "port: %s\n", name); >>> >+ goto out; >>> >+ } >>> >+ >>> >+ err = rte_vhost_driver_start(dev->vhost_id); >>> >+ if (err) { >>> >+ VLOG_ERR("rte_vhost_driver_start failed for vhost user " >>> >+ "port: %s\n", name); >>> >+ goto out; >>> >+ } >>> >+ >>> > err = vhost_common_construct(netdev); >>> >+ if (err) { >>> >+ VLOG_ERR("vhost_common_construct failed for vhost user " >>> >+ "port: %s\n", name); >>> >+ } >>> > >>> >+out: >>> > ovs_mutex_unlock(&dpdk_mutex); >>> > VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; " >>> > "please migrate to dpdkvhostuserclient ports."); >>> >@@ -968,6 +1020,10 @@ netdev_dpdk_vhost_client_construct(struct netdev >>> >*netdev) >>> > >>> > ovs_mutex_lock(&dpdk_mutex); >>> > err = vhost_common_construct(netdev); >>> >+ if (err) { >>> >+ VLOG_ERR("vhost_common_construct failed for vhost user client" >>> >+ "port: %s\n", netdev->name); >>> >+ } >>> > ovs_mutex_unlock(&dpdk_mutex); >>> > return err; >>> > } >>> >@@ -2456,12 +2512,9 @@ static void >>> > set_irq_status(int vid) >>> > { >>> > uint32_t i; >>> >- uint64_t idx; >>> > >>> >- for (i = 0; i < rte_vhost_get_queue_num(vid); i++) { >>> >- idx = i * VIRTIO_QNUM; >>> >- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_RXQ, 0); >>> >- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_TXQ, 0); >>> >+ for (i = 0; i < rte_vhost_get_vring_num(vid); i++) { >>> >+ rte_vhost_enable_guest_notification(vid, i, 0); >>> > } >>> > } >>> > >>> >@@ -2524,7 +2577,7 @@ new_device(int vid) >>> > LIST_FOR_EACH(dev, list_node, &dpdk_list) { >>> > ovs_mutex_lock(&dev->mutex); >>> > if (strncmp(ifname, dev->vhost_id, IF_NAME_SZ) == 0) { >>> >- uint32_t qp_num = rte_vhost_get_queue_num(vid); >>> >+ uint32_t qp_num = rte_vhost_get_vring_num(vid)/VIRTIO_QNUM; >>> > >>> > /* Get NUMA information */ >>> > newnode = rte_vhost_get_numa_node(vid); >>> >@@ -2691,27 +2744,6 @@ netdev_dpdk_get_ingress_policer(const struct >>> >netdev_dpdk *dev) >>> > return ovsrcu_get(struct ingress_policer *, &dev->ingress_policer); >>> > } >>> > >>> >-/* >>> >- * These callbacks allow virtio-net devices to be added to vhost ports >>when >>> >- * configuration has been fully complete. >>> >- */ >>> >-static const struct virtio_net_device_ops virtio_net_device_ops = >>> >-{ >>> >- .new_device = new_device, >>> >- .destroy_device = destroy_device, >>> >- .vring_state_changed = vring_state_changed >>> >-}; >>> >- >>> >-static void * >>> >-start_vhost_loop(void *dummy OVS_UNUSED) >>> >-{ >>> >- pthread_detach(pthread_self()); >>> >- /* Put the vhost thread into quiescent state. */ >>> >- ovsrcu_quiesce_start(); >>> >- rte_vhost_driver_session_start(); >>> >- return NULL; >>> >-} >>> >- >>> > static int >>> > netdev_dpdk_class_init(void) >>> > { >>> >@@ -2731,25 +2763,6 @@ netdev_dpdk_class_init(void) >>> > return 0; >>> > } >>> > >>> >-static int >>> >-netdev_dpdk_vhost_class_init(void) >>> >-{ >>> >- static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; >>> >- >>> >- /* This function can be called for different classes. The >>initialization >>> >- * needs to be done only once */ >>> >- if (ovsthread_once_start(&once)) { >>> >- rte_vhost_driver_callback_register(&virtio_net_device_ops); >>> >- rte_vhost_feature_disable(1ULL << VIRTIO_NET_F_HOST_TSO4 >>> >- | 1ULL << VIRTIO_NET_F_HOST_TSO6 >>> >- | 1ULL << VIRTIO_NET_F_CSUM); >>> >- ovs_thread_create("vhost_thread", start_vhost_loop, NULL); >>> >- >>> >- ovsthread_once_done(&once); >>> >- } >>> >- >>> >- return 0; >>> >-} >>> > >>> > /* Client Rings */ >>> > >>> >@@ -3204,6 +3217,31 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev >>> >*netdev) >>> > "using client socket '%s'", >>> > dev->up.name, dev->vhost_id); >>> > } >>> >+ >>> >+ err = rte_vhost_driver_callback_register(dev->vhost_id, >>> >+ &virtio_net_device_ops); >>> >+ if (err) { >>> >+ VLOG_ERR("rte_vhost_driver_callback_register failed for " >>> >+ "vhost user client port: %s\n", dev->up.name); >>> >+ goto unlock; >>> >+ } >>> >+ >>> >+ err = rte_vhost_driver_disable_features(dev->vhost_id, >>> >+ 1ULL << VIRTIO_NET_F_HOST_TSO4 >>> >+ | 1ULL << VIRTIO_NET_F_HOST_TSO6 >>> >+ | 1ULL << VIRTIO_NET_F_CSUM); >>> >+ if (err) { >>> >+ VLOG_ERR("rte_vhost_driver_disable_features failed for vhost >>user >>> >" >>> >+ "client port: %s\n", dev->up.name); >>> >+ goto unlock; >>> >+ } >>> >+ >>> >+ err = rte_vhost_driver_start(dev->vhost_id); >>> >+ if (err) { >>> >+ VLOG_ERR("rte_vhost_driver_start failed for vhost user " >>> >+ "client port: %s\n", dev->up.name); >>> >+ goto unlock; >>> >+ } >>> > } >>> > >>> > err = dpdk_vhost_reconfigure_helper(dev); >>> >@@ -3324,7 +3362,7 @@ static const struct netdev_class dpdk_ring_class = >>> > static const struct netdev_class dpdk_vhost_class = >>> > NETDEV_DPDK_CLASS( >>> > "dpdkvhostuser", >>> >- netdev_dpdk_vhost_class_init, >>> >+ NULL, >>> > netdev_dpdk_vhost_construct, >>> > netdev_dpdk_vhost_destruct, >>> > NULL, >>> >@@ -3339,7 +3377,7 @@ static const struct netdev_class dpdk_vhost_class = >>> > static const struct netdev_class dpdk_vhost_client_class = >>> > NETDEV_DPDK_CLASS( >>> > "dpdkvhostuserclient", >>> >- netdev_dpdk_vhost_class_init, >>> >+ NULL, >>> > netdev_dpdk_vhost_client_construct, >>> > netdev_dpdk_vhost_destruct, >>> > netdev_dpdk_vhost_client_set_config, >>> >diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch- >>fedora.spec.in >>> >index 3a045d3..2bb7102 100644 >>> >--- a/rhel/openvswitch-fedora.spec.in >>> >+++ b/rhel/openvswitch-fedora.spec.in >>> >@@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel >>> > %endif >>> > %if %{with dpdk} >>> > BuildRequires: libpcap-devel numactl-devel >>> >-BuildRequires: dpdk-devel >= 16.11 >>> >+BuildRequires: dpdk-devel >= 17.05.1 >>> > Provides: %{name}-dpdk = %{version}-%{release} >>> > %endif >>> > >>> >diff --git a/tests/dpdk/ring_client.c b/tests/dpdk/ring_client.c >>> >index b153713..8cc3fb5 100644 >>> >--- a/tests/dpdk/ring_client.c >>> >+++ b/tests/dpdk/ring_client.c >>> >@@ -185,15 +185,15 @@ main(int argc, char *argv[]) >>> > /* Try dequeuing max possible packets first, if that fails, get >>the >>> > * most we can. Loop body should only execute once, maximum. >>> > */ >>> >- while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, rx_pkts) != >>0) >>> >&& >>> >- rx_pkts > 0) { >>> >+ while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, >>> >+ rx_pkts, NULL) != 0) && rx_pkts > 0) { >>> > rx_pkts = (uint16_t)RTE_MIN(rte_ring_count(rx_ring), >>> >PKT_READ_SIZE); >>> > } >>> > >>> > if (rx_pkts > 0) { >>> > /* blocking enqueue */ >>> > do { >>> >- rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts); >>> >+ rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts, >>NULL); >>> > } while (rslt == -ENOBUFS); >>> > } >>> > } >>> >-- >>> >1.8.3.1 >>> > >>> >_______________________________________________ >>> >dev mailing list >>> >dev@openvswitch.org >>> >https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>From: Aaron Conole [mailto:aconole@redhat.com] >Sent: Wednesday, July 26, 2017 3:25 PM >To: Kavanagh, Mark B <mark.b.kavanagh@intel.com> >Cc: Weglicki, MichalX <michalx.weglicki@intel.com>; dev@openvswitch.org >Subject: Re: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for >DPDK 17.05.1. > >"Kavanagh, Mark B" <mark.b.kavanagh@intel.com> writes: > >>>From: Weglicki, MichalX >>>Sent: Wednesday, July 26, 2017 10:52 AM >>>To: Kavanagh, Mark B <mark.b.kavanagh@intel.com>; dev@openvswitch.org >>>Subject: RE: [ovs-dev] [PATCH v4] Update relevant artifacts to add support >for >>>DPDK 17.05.1. >>> >>>Hey Mark, >>> >>>Sorry, I had some problems with rebase, and somehow I just simply forgot t >to >>>do so. Do you want me to re-apply this patch as v5? >> >> No need Michal - hopefully the maintainer will just roll it in when >> applying to master. > >If you reply with your Acked-by tag, the patchwork instance will >re-apply it and the maintainer will not have to remember. Thanks for the tip Aaron - I wasn't aware of that! Acked-by: Mark Kavanagh <mark.b.kavanagh@intel.com> -Mark > >> Thanks, >> Mark >> >>> >>>> -----Original Message----- >>>> From: Kavanagh, Mark B >>>> Sent: Wednesday, July 26, 2017 10:04 AM >>>> To: Weglicki, MichalX <michalx.weglicki@intel.com>; dev@openvswitch.org >>>> Subject: RE: [ovs-dev] [PATCH v4] Update relevant artifacts to add support >>>for DPDK 17.05.1. >>>> >>>> >>>> >>>> >-----Original Message----- >>>> >From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev- >>>bounces@openvswitch.org] >>>> >On Behalf Of Michal Weglicki >>>> >Sent: Tuesday, July 25, 2017 1:35 PM >>>> >To: dev@openvswitch.org >>>> >Subject: [ovs-dev] [PATCH v4] Update relevant artifacts to add support >for >>>> >DPDK 17.05.1. >>>> > >>>> >Upgrading to DPDK 17.05.1 stable release adds new >>>> >significant features relevant to OVS, including, >>>> >but not limited to: >>>> >- tun/tap PMD, >>>> >- VFIO hotplug support, >>>> >- Generic flow API. >>>> > >>>> >Following changes are applied: >>>> >- netdev-dpdk: Changes required by DPDK API modifications. >>>> >- doc: Because of DPDK API changes, backward compatibility >>>> > with previous DPDK releases will be broken, thus all >>>> > relevant documentation entries are updated. >>>> >- .travis: DPDK version change from 16.11.1 to 17.05.1. >>>> >- rhel/openvswitch-fedora.spec.in: DPDK version change >>>> > from 16.11 to 17.05.1 >>>> > >>>> >v1->v2: Patch rebase. >>>> >v2->v3: Fixed wrong formating after v2 patch rebase. >>>> >v3->v4: Minor documentation changes. >>>> > >>>> >Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> >>>> >Reviewed-by: Aaron Conole <aconole@redhat.com> >>>> >>>> >>>> Hi Michal, >>>> >>>> Is there a reason why you haven't added my acked-by to this latest patch? >>>https://mail.openvswitch.org/pipermail/ovs-dev/2017- >>>> July/334893.html >>>> >>>> Thanks, >>>> Mark >>>> >>>> >--- >>>> > .travis/linux-build.sh | 2 +- >>>> > Documentation/faq/releases.rst | 1 + >>>> > Documentation/howto/dpdk.rst | 6 +- >>>> > Documentation/intro/install/dpdk.rst | 14 +-- >>>> > Documentation/topics/dpdk/vhost-user.rst | 12 +-- >>>> > NEWS | 1 + >>>> > lib/netdev-dpdk.c | 144 +++++++++++++++++++------ >-- >>>--- >>>> >- >>>> > rhel/openvswitch-fedora.spec.in | 2 +- >>>> > tests/dpdk/ring_client.c | 6 +- >>>> > 9 files changed, 114 insertions(+), 74 deletions(-) >>>> > >>>> >diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh >>>> >index f66b534..efccdf1 100755 >>>> >--- a/.travis/linux-build.sh >>>> >+++ b/.travis/linux-build.sh >>>> >@@ -80,7 +80,7 @@ fi >>>> > >>>> > if [ "$DPDK" ]; then >>>> > if [ -z "$DPDK_VER" ]; then >>>> >- DPDK_VER="16.11.2" >>>> >+ DPDK_VER="17.05.1" >>>> > fi >>>> > install_dpdk $DPDK_VER >>>> > if [ "$CC" = "clang" ]; then >>>> >diff --git a/Documentation/faq/releases.rst >>>b/Documentation/faq/releases.rst >>>> >index 707834b..2ecc24c 100644 >>>> >--- a/Documentation/faq/releases.rst >>>> >+++ b/Documentation/faq/releases.rst >>>> >@@ -161,6 +161,7 @@ Q: What DPDK version does each Open vSwitch release >>>work >>>> >with? >>>> > 2.5.x 2.2 >>>> > 2.6.x 16.07.2 >>>> > 2.7.x 16.11.2 >>>> >+ 2.8.x 17.05.1 >>>> > ============ ======= >>>> > >>>> > Q: I get an error like this when I configure Open vSwitch: >>>> >diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst >>>> >index af01d3e..1756c8c 100644 >>>> >--- a/Documentation/howto/dpdk.rst >>>> >+++ b/Documentation/howto/dpdk.rst >>>> >@@ -528,15 +528,15 @@ described in :ref:`dpdk-testpmd`. Once compiled, >run >>>the >>>> >application:: >>>> > >>>> > When you finish testing, bind the vNICs back to kernel:: >>>> > >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 >>>> > >>>> > .. note:: >>>> > >>>> > Valid PCI IDs must be passed in above example. The PCI IDs can be >>>retrieved >>>> > like so:: >>>> > >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --status >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status >>>> > >>>> > More information on the dpdkvhostuser ports can be found in >>>> > :doc:`/topics/dpdk/vhost-user`. >>>> >diff --git a/Documentation/intro/install/dpdk.rst >>>> >b/Documentation/intro/install/dpdk.rst >>>> >index a05aa1a..20baa63 100644 >>>> >--- a/Documentation/intro/install/dpdk.rst >>>> >+++ b/Documentation/intro/install/dpdk.rst >>>> >@@ -40,7 +40,7 @@ Build requirements >>>> > In addition to the requirements described in :doc:`general`, building >Open >>>> > vSwitch with DPDK will require the following: >>>> > >>>> >-- DPDK 16.11 >>>> >+- DPDK 17.05.1 >>>> > >>>> > - A `DPDK supported NIC`_ >>>> > >>>> >@@ -69,9 +69,9 @@ Install DPDK >>>> > #. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``:: >>>> > >>>> > $ cd /usr/src/ >>>> >- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz >>>> >- $ tar xf dpdk-16.11.2.tar.xz >>>> >- $ export DPDK_DIR=/usr/src/dpdk-stable-16.11.2 >>>> >+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz >>>> >+ $ tar xf dpdk-17.05.1.tar.xz >>>> >+ $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.1 >>>> > $ cd $DPDK_DIR >>>> > >>>> > #. (Optional) Configure DPDK as a shared library >>>> >@@ -187,8 +187,8 @@ to the VFIO driver:: >>>> > $ modprobe vfio-pci >>>> > $ /usr/bin/chmod a+x /dev/vfio >>>> > $ /usr/bin/chmod 0666 /dev/vfio/* >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1 >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --status >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=vfio-pci eth1 >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status >>>> > >>>> > Setup OVS >>>> > ~~~~~~~~~ >>>> >@@ -569,7 +569,7 @@ Limitations >>>> > The latest list of validated firmware versions can be found in the >`DPDK >>>> > release notes`_. >>>> > >>>> >-.. _DPDK release notes: >>>> >http://dpdk.org/doc/guides/rel_notes/release_16_11.html >>>> >+.. _DPDK release notes: >>>> >http://dpdk.org/doc/guides/rel_notes/release_17_05.html >>>> > >>>> > Reporting Bugs >>>> > -------------- >>>> >diff --git a/Documentation/topics/dpdk/vhost-user.rst >>>> >b/Documentation/topics/dpdk/vhost-user.rst >>>> >index e76da5f..a3d5de3 100644 >>>> >--- a/Documentation/topics/dpdk/vhost-user.rst >>>> >+++ b/Documentation/topics/dpdk/vhost-user.rst >>>> >@@ -292,9 +292,9 @@ To begin, instantiate a guest as described in >>>:ref:`dpdk- >>>> >vhost-user` or >>>> > DPDK sources to VM and build DPDK:: >>>> > >>>> > $ cd /root/dpdk/ >>>> >- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz >>>> >- $ tar xf dpdk-16.11.2.tar.xz >>>> >- $ export DPDK_DIR=/root/dpdk/dpdk-stable-16.11.2 >>>> >+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz >>>> >+ $ tar xf dpdk-17.05.1.tar.xz >>>> >+ $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.1 >>>> > $ export DPDK_TARGET=x86_64-native-linuxapp-gcc >>>> > $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET >>>> > $ cd $DPDK_DIR >>>> >@@ -314,8 +314,8 @@ Setup huge pages and DPDK devices using UIO:: >>>> > $ mount -t hugetlbfs hugetlbfs /dev/hugepages # only if not already >>>> >mounted >>>> > $ modprobe uio >>>> > $ insmod $DPDK_BUILD/kmod/igb_uio.ko >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --status >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 >>>> > >>>> > .. note:: >>>> > >>>> >@@ -378,7 +378,7 @@ Sample XML >>>> > </disk> >>>> > <disk type='dir' device='disk'> >>>> > <driver name='qemu' type='fat'/> >>>> >- <source dir='/usr/src/dpdk-stable-16.11.2'/> >>>> >+ <source dir='/usr/src/dpdk-stable-17.05.1'/> >>>> > <target dev='vdb' bus='virtio'/> >>>> > <readonly/> >>>> > </disk> >>>> >diff --git a/NEWS b/NEWS >>>> >index d61fc5f..3f58e8b 100644 >>>> >--- a/NEWS >>>> >+++ b/NEWS >>>> >@@ -21,6 +21,7 @@ Post-v2.7.0 >>>> > still can be configured via extra arguments for DPDK EAL. >>>> > * dpdkvhostuser ports are marked as deprecated. They will be >removed >>>> > in an upcoming release. >>>> >+ * Support for DPDK v17.05.1. >>>> > - IPFIX now provides additional counters: >>>> > * Total counters since metering process startup. >>>> > * Per-flow TCP flag counters. >>>> >diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c >>>> >index ea17b97..5e767e1 100644 >>>> >--- a/lib/netdev-dpdk.c >>>> >+++ b/lib/netdev-dpdk.c >>>> >@@ -22,6 +22,9 @@ >>>> > #include <stdlib.h> >>>> > #include <errno.h> >>>> > #include <unistd.h> >>>> >+#include <linux/virtio_net.h> >>>> >+#include <sys/socket.h> >>>> >+#include <linux/if.h> >>>> > >>>> > #include <rte_config.h> >>>> > #include <rte_cycles.h> >>>> >@@ -31,7 +34,7 @@ >>>> > #include <rte_malloc.h> >>>> > #include <rte_mbuf.h> >>>> > #include <rte_meter.h> >>>> >-#include <rte_virtio_net.h> >>>> >+#include <rte_vhost.h> >>>> > >>>> > #include "dirs.h" >>>> > #include "dp-packet.h" >>>> >@@ -56,6 +59,8 @@ >>>> > #include "timeval.h" >>>> > #include "unixctl.h" >>>> > >>>> >+enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM}; >>>> >+ >>>> > VLOG_DEFINE_THIS_MODULE(netdev_dpdk); >>>> > static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20); >>>> > >>>> >@@ -171,6 +176,21 @@ static const struct rte_eth_conf port_conf = { >>>> > }, >>>> > }; >>>> > >>>> >+/* >>>> >+ * These callbacks allow virtio-net devices to be added to vhost ports >>>when >>>> >+ * configuration has been fully completed. >>>> >+ */ >>>> >+static int new_device(int vid); >>>> >+static void destroy_device(int vid); >>>> >+static int vring_state_changed(int vid, uint16_t queue_id, int enable); >>>> >+static const struct vhost_device_ops virtio_net_device_ops = >>>> >+{ >>>> >+ .new_device = new_device, >>>> >+ .destroy_device = destroy_device, >>>> >+ .vring_state_changed = vring_state_changed, >>>> >+ .features_changed = NULL >>>> >+}; >>>> >+ >>>> > enum { DPDK_RING_SIZE = 256 }; >>>> > BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE)); >>>> > enum { DRAIN_TSC = 200000ULL }; >>>> >@@ -412,8 +432,8 @@ struct netdev_rxq_dpdk { >>>> > dpdk_port_t port_id; >>>> > }; >>>> > >>>> >-static int netdev_dpdk_class_init(void); >>>> >-static int netdev_dpdk_vhost_class_init(void); >>>> >+static void netdev_dpdk_destruct(struct netdev *netdev); >>>> >+static void netdev_dpdk_vhost_destruct(struct netdev *netdev); >>>> > >>>> > int netdev_dpdk_get_vid(const struct netdev_dpdk *dev); >>>> > >>>> >@@ -423,8 +443,8 @@ netdev_dpdk_get_ingress_policer(const struct >>>netdev_dpdk >>>> >*dev); >>>> > static bool >>>> > is_dpdk_class(const struct netdev_class *class) >>>> > { >>>> >- return class->init == netdev_dpdk_class_init >>>> >- || class->init == netdev_dpdk_vhost_class_init; >>>> >+ return class->destruct == netdev_dpdk_destruct >>>> >+ || class->destruct == netdev_dpdk_vhost_destruct; >>>> > } >>>> > >>>> > /* DPDK NIC drivers allocate RX buffers at a particular granularity, >>>> >typically >>>> >@@ -948,13 +968,45 @@ netdev_dpdk_vhost_construct(struct netdev *netdev) >>>> > if (err) { >>>> > VLOG_ERR("vhost-user socket device setup failure for socket >%s\n", >>>> > dev->vhost_id); >>>> >+ goto out; >>>> > } else { >>>> > fatal_signal_add_file_to_unlink(dev->vhost_id); >>>> > VLOG_INFO("Socket %s created for vhost-user port %s\n", >>>> > dev->vhost_id, name); >>>> > } >>>> >+ >>>> >+ err = rte_vhost_driver_callback_register(dev->vhost_id, >>>> >+ &virtio_net_device_ops); >>>> >+ if (err) { >>>> >+ VLOG_ERR("rte_vhost_driver_callback_register failed for vhost >user >>>" >>>> >+ "port: %s\n", name); >>>> >+ goto out; >>>> >+ } >>>> >+ >>>> >+ err = rte_vhost_driver_disable_features(dev->vhost_id, >>>> >+ 1ULL << VIRTIO_NET_F_HOST_TSO4 >>>> >+ | 1ULL << VIRTIO_NET_F_HOST_TSO6 >>>> >+ | 1ULL << VIRTIO_NET_F_CSUM); >>>> >+ if (err) { >>>> >+ VLOG_ERR("rte_vhost_driver_disable_features failed for vhost >user >>>" >>>> >+ "port: %s\n", name); >>>> >+ goto out; >>>> >+ } >>>> >+ >>>> >+ err = rte_vhost_driver_start(dev->vhost_id); >>>> >+ if (err) { >>>> >+ VLOG_ERR("rte_vhost_driver_start failed for vhost user " >>>> >+ "port: %s\n", name); >>>> >+ goto out; >>>> >+ } >>>> >+ >>>> > err = vhost_common_construct(netdev); >>>> >+ if (err) { >>>> >+ VLOG_ERR("vhost_common_construct failed for vhost user " >>>> >+ "port: %s\n", name); >>>> >+ } >>>> > >>>> >+out: >>>> > ovs_mutex_unlock(&dpdk_mutex); >>>> > VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; " >>>> > "please migrate to dpdkvhostuserclient ports."); >>>> >@@ -968,6 +1020,10 @@ netdev_dpdk_vhost_client_construct(struct netdev >>>> >*netdev) >>>> > >>>> > ovs_mutex_lock(&dpdk_mutex); >>>> > err = vhost_common_construct(netdev); >>>> >+ if (err) { >>>> >+ VLOG_ERR("vhost_common_construct failed for vhost user client" >>>> >+ "port: %s\n", netdev->name); >>>> >+ } >>>> > ovs_mutex_unlock(&dpdk_mutex); >>>> > return err; >>>> > } >>>> >@@ -2456,12 +2512,9 @@ static void >>>> > set_irq_status(int vid) >>>> > { >>>> > uint32_t i; >>>> >- uint64_t idx; >>>> > >>>> >- for (i = 0; i < rte_vhost_get_queue_num(vid); i++) { >>>> >- idx = i * VIRTIO_QNUM; >>>> >- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_RXQ, 0); >>>> >- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_TXQ, 0); >>>> >+ for (i = 0; i < rte_vhost_get_vring_num(vid); i++) { >>>> >+ rte_vhost_enable_guest_notification(vid, i, 0); >>>> > } >>>> > } >>>> > >>>> >@@ -2524,7 +2577,7 @@ new_device(int vid) >>>> > LIST_FOR_EACH(dev, list_node, &dpdk_list) { >>>> > ovs_mutex_lock(&dev->mutex); >>>> > if (strncmp(ifname, dev->vhost_id, IF_NAME_SZ) == 0) { >>>> >- uint32_t qp_num = rte_vhost_get_queue_num(vid); >>>> >+ uint32_t qp_num = rte_vhost_get_vring_num(vid)/VIRTIO_QNUM; >>>> > >>>> > /* Get NUMA information */ >>>> > newnode = rte_vhost_get_numa_node(vid); >>>> >@@ -2691,27 +2744,6 @@ netdev_dpdk_get_ingress_policer(const struct >>>> >netdev_dpdk *dev) >>>> > return ovsrcu_get(struct ingress_policer *, &dev->ingress_policer); >>>> > } >>>> > >>>> >-/* >>>> >- * These callbacks allow virtio-net devices to be added to vhost ports >>>when >>>> >- * configuration has been fully complete. >>>> >- */ >>>> >-static const struct virtio_net_device_ops virtio_net_device_ops = >>>> >-{ >>>> >- .new_device = new_device, >>>> >- .destroy_device = destroy_device, >>>> >- .vring_state_changed = vring_state_changed >>>> >-}; >>>> >- >>>> >-static void * >>>> >-start_vhost_loop(void *dummy OVS_UNUSED) >>>> >-{ >>>> >- pthread_detach(pthread_self()); >>>> >- /* Put the vhost thread into quiescent state. */ >>>> >- ovsrcu_quiesce_start(); >>>> >- rte_vhost_driver_session_start(); >>>> >- return NULL; >>>> >-} >>>> >- >>>> > static int >>>> > netdev_dpdk_class_init(void) >>>> > { >>>> >@@ -2731,25 +2763,6 @@ netdev_dpdk_class_init(void) >>>> > return 0; >>>> > } >>>> > >>>> >-static int >>>> >-netdev_dpdk_vhost_class_init(void) >>>> >-{ >>>> >- static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; >>>> >- >>>> >- /* This function can be called for different classes. The >>>initialization >>>> >- * needs to be done only once */ >>>> >- if (ovsthread_once_start(&once)) { >>>> >- rte_vhost_driver_callback_register(&virtio_net_device_ops); >>>> >- rte_vhost_feature_disable(1ULL << VIRTIO_NET_F_HOST_TSO4 >>>> >- | 1ULL << VIRTIO_NET_F_HOST_TSO6 >>>> >- | 1ULL << VIRTIO_NET_F_CSUM); >>>> >- ovs_thread_create("vhost_thread", start_vhost_loop, NULL); >>>> >- >>>> >- ovsthread_once_done(&once); >>>> >- } >>>> >- >>>> >- return 0; >>>> >-} >>>> > >>>> > /* Client Rings */ >>>> > >>>> >@@ -3204,6 +3217,31 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev >>>> >*netdev) >>>> > "using client socket '%s'", >>>> > dev->up.name, dev->vhost_id); >>>> > } >>>> >+ >>>> >+ err = rte_vhost_driver_callback_register(dev->vhost_id, >>>> >+ >&virtio_net_device_ops); >>>> >+ if (err) { >>>> >+ VLOG_ERR("rte_vhost_driver_callback_register failed for " >>>> >+ "vhost user client port: %s\n", dev->up.name); >>>> >+ goto unlock; >>>> >+ } >>>> >+ >>>> >+ err = rte_vhost_driver_disable_features(dev->vhost_id, >>>> >+ 1ULL << VIRTIO_NET_F_HOST_TSO4 >>>> >+ | 1ULL << VIRTIO_NET_F_HOST_TSO6 >>>> >+ | 1ULL << VIRTIO_NET_F_CSUM); >>>> >+ if (err) { >>>> >+ VLOG_ERR("rte_vhost_driver_disable_features failed for vhost >>>user >>>> >" >>>> >+ "client port: %s\n", dev->up.name); >>>> >+ goto unlock; >>>> >+ } >>>> >+ >>>> >+ err = rte_vhost_driver_start(dev->vhost_id); >>>> >+ if (err) { >>>> >+ VLOG_ERR("rte_vhost_driver_start failed for vhost user " >>>> >+ "client port: %s\n", dev->up.name); >>>> >+ goto unlock; >>>> >+ } >>>> > } >>>> > >>>> > err = dpdk_vhost_reconfigure_helper(dev); >>>> >@@ -3324,7 +3362,7 @@ static const struct netdev_class dpdk_ring_class = >>>> > static const struct netdev_class dpdk_vhost_class = >>>> > NETDEV_DPDK_CLASS( >>>> > "dpdkvhostuser", >>>> >- netdev_dpdk_vhost_class_init, >>>> >+ NULL, >>>> > netdev_dpdk_vhost_construct, >>>> > netdev_dpdk_vhost_destruct, >>>> > NULL, >>>> >@@ -3339,7 +3377,7 @@ static const struct netdev_class dpdk_vhost_class = >>>> > static const struct netdev_class dpdk_vhost_client_class = >>>> > NETDEV_DPDK_CLASS( >>>> > "dpdkvhostuserclient", >>>> >- netdev_dpdk_vhost_class_init, >>>> >+ NULL, >>>> > netdev_dpdk_vhost_client_construct, >>>> > netdev_dpdk_vhost_destruct, >>>> > netdev_dpdk_vhost_client_set_config, >>>> >diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch- >>>fedora.spec.in >>>> >index 3a045d3..2bb7102 100644 >>>> >--- a/rhel/openvswitch-fedora.spec.in >>>> >+++ b/rhel/openvswitch-fedora.spec.in >>>> >@@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel >>>> > %endif >>>> > %if %{with dpdk} >>>> > BuildRequires: libpcap-devel numactl-devel >>>> >-BuildRequires: dpdk-devel >= 16.11 >>>> >+BuildRequires: dpdk-devel >= 17.05.1 >>>> > Provides: %{name}-dpdk = %{version}-%{release} >>>> > %endif >>>> > >>>> >diff --git a/tests/dpdk/ring_client.c b/tests/dpdk/ring_client.c >>>> >index b153713..8cc3fb5 100644 >>>> >--- a/tests/dpdk/ring_client.c >>>> >+++ b/tests/dpdk/ring_client.c >>>> >@@ -185,15 +185,15 @@ main(int argc, char *argv[]) >>>> > /* Try dequeuing max possible packets first, if that fails, get >>>the >>>> > * most we can. Loop body should only execute once, maximum. >>>> > */ >>>> >- while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, rx_pkts) != >>>0) >>>> >&& >>>> >- rx_pkts > 0) { >>>> >+ while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, >>>> >+ rx_pkts, NULL) != 0) && rx_pkts > 0) { >>>> > rx_pkts = (uint16_t)RTE_MIN(rte_ring_count(rx_ring), >>>> >PKT_READ_SIZE); >>>> > } >>>> > >>>> > if (rx_pkts > 0) { >>>> > /* blocking enqueue */ >>>> > do { >>>> >- rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts); >>>> >+ rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts, >>>NULL); >>>> > } while (rslt == -ENOBUFS); >>>> > } >>>> > } >>>> >-- >>>> >1.8.3.1 >>>> > >>>> >_______________________________________________ >>>> >dev mailing list >>>> >dev@openvswitch.org >>>> >https://mail.openvswitch.org/mailman/listinfo/ovs-dev >> _______________________________________________ >> dev mailing list >> dev@openvswitch.org >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> -----Original Message----- > From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev- > bounces@openvswitch.org] On Behalf Of Kavanagh, Mark B > Sent: Wednesday, July 26, 2017 4:03 PM > To: Aaron Conole <aconole@redhat.com> > Cc: dev@openvswitch.org > Subject: Re: [ovs-dev] [PATCH v4] Update relevant artifacts to add support > for DPDK 17.05.1. > > >From: Aaron Conole [mailto:aconole@redhat.com] > >Sent: Wednesday, July 26, 2017 3:25 PM > >To: Kavanagh, Mark B <mark.b.kavanagh@intel.com> > >Cc: Weglicki, MichalX <michalx.weglicki@intel.com>; dev@openvswitch.org > >Subject: Re: [ovs-dev] [PATCH v4] Update relevant artifacts to add > >support for DPDK 17.05.1. > > > >"Kavanagh, Mark B" <mark.b.kavanagh@intel.com> writes: > > > >>>From: Weglicki, MichalX > >>>Sent: Wednesday, July 26, 2017 10:52 AM > >>>To: Kavanagh, Mark B <mark.b.kavanagh@intel.com>; dev@openvswitch.org > >>>Subject: RE: [ovs-dev] [PATCH v4] Update relevant artifacts to add > >>>support > >for > >>>DPDK 17.05.1. > >>> > >>>Hey Mark, > >>> > >>>Sorry, I had some problems with rebase, and somehow I just simply > >>>forgot t > >to > >>>do so. Do you want me to re-apply this patch as v5? > >> > >> No need Michal - hopefully the maintainer will just roll it in when > >> applying to master. > > > >If you reply with your Acked-by tag, the patchwork instance will > >re-apply it and the maintainer will not have to remember. > > Thanks for the tip Aaron - I wasn't aware of that! > > Acked-by: Mark Kavanagh <mark.b.kavanagh@intel.com> > > -Mark Adding tested by tag. Tested-by: Ian Stokes <ian.stokes@intel.com> > > > > >> Thanks, > >> Mark > >> > >>> > >>>> -----Original Message----- > >>>> From: Kavanagh, Mark B > >>>> Sent: Wednesday, July 26, 2017 10:04 AM > >>>> To: Weglicki, MichalX <michalx.weglicki@intel.com>; > >>>> dev@openvswitch.org > >>>> Subject: RE: [ovs-dev] [PATCH v4] Update relevant artifacts to add > >>>> support > >>>for DPDK 17.05.1. > >>>> > >>>> > >>>> > >>>> >-----Original Message----- > >>>> >From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev- > >>>bounces@openvswitch.org] > >>>> >On Behalf Of Michal Weglicki > >>>> >Sent: Tuesday, July 25, 2017 1:35 PM > >>>> >To: dev@openvswitch.org > >>>> >Subject: [ovs-dev] [PATCH v4] Update relevant artifacts to add > >>>> >support > >for > >>>> >DPDK 17.05.1. > >>>> > > >>>> >Upgrading to DPDK 17.05.1 stable release adds new significant > >>>> >features relevant to OVS, including, but not limited to: > >>>> >- tun/tap PMD, > >>>> >- VFIO hotplug support, > >>>> >- Generic flow API. > >>>> > > >>>> >Following changes are applied: > >>>> >- netdev-dpdk: Changes required by DPDK API modifications. > >>>> >- doc: Because of DPDK API changes, backward compatibility > >>>> > with previous DPDK releases will be broken, thus all > >>>> > relevant documentation entries are updated. > >>>> >- .travis: DPDK version change from 16.11.1 to 17.05.1. > >>>> >- rhel/openvswitch-fedora.spec.in: DPDK version change > >>>> > from 16.11 to 17.05.1 > >>>> > > >>>> >v1->v2: Patch rebase. > >>>> >v2->v3: Fixed wrong formating after v2 patch rebase. > >>>> >v3->v4: Minor documentation changes. > >>>> > > >>>> >Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> > >>>> >Reviewed-by: Aaron Conole <aconole@redhat.com> > >>>> > >>>> > >>>> Hi Michal, > >>>> > >>>> Is there a reason why you haven't added my acked-by to this latest > patch? > >>>https://mail.openvswitch.org/pipermail/ovs-dev/2017- > >>>> July/334893.html > >>>> > >>>> Thanks, > >>>> Mark > >>>> > >>>> >--- > >>>> > .travis/linux-build.sh | 2 +- > >>>> > Documentation/faq/releases.rst | 1 + > >>>> > Documentation/howto/dpdk.rst | 6 +- > >>>> > Documentation/intro/install/dpdk.rst | 14 +-- > >>>> > Documentation/topics/dpdk/vhost-user.rst | 12 +-- > >>>> > NEWS | 1 + > >>>> > lib/netdev-dpdk.c | 144 +++++++++++++++++++- > ----- > >-- > >>>--- > >>>> >- > >>>> > rhel/openvswitch-fedora.spec.in | 2 +- > >>>> > tests/dpdk/ring_client.c | 6 +- > >>>> > 9 files changed, 114 insertions(+), 74 deletions(-) > >>>> > > >>>> >diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh index > >>>> >f66b534..efccdf1 100755 > >>>> >--- a/.travis/linux-build.sh > >>>> >+++ b/.travis/linux-build.sh > >>>> >@@ -80,7 +80,7 @@ fi > >>>> > > >>>> > if [ "$DPDK" ]; then > >>>> > if [ -z "$DPDK_VER" ]; then > >>>> >- DPDK_VER="16.11.2" > >>>> >+ DPDK_VER="17.05.1" > >>>> > fi > >>>> > install_dpdk $DPDK_VER > >>>> > if [ "$CC" = "clang" ]; then > >>>> >diff --git a/Documentation/faq/releases.rst > >>>b/Documentation/faq/releases.rst > >>>> >index 707834b..2ecc24c 100644 > >>>> >--- a/Documentation/faq/releases.rst > >>>> >+++ b/Documentation/faq/releases.rst > >>>> >@@ -161,6 +161,7 @@ Q: What DPDK version does each Open vSwitch > >>>> >release > >>>work > >>>> >with? > >>>> > 2.5.x 2.2 > >>>> > 2.6.x 16.07.2 > >>>> > 2.7.x 16.11.2 > >>>> >+ 2.8.x 17.05.1 > >>>> > ============ ======= > >>>> > > >>>> > Q: I get an error like this when I configure Open vSwitch: > >>>> >diff --git a/Documentation/howto/dpdk.rst > >>>> >b/Documentation/howto/dpdk.rst index af01d3e..1756c8c 100644 > >>>> >--- a/Documentation/howto/dpdk.rst > >>>> >+++ b/Documentation/howto/dpdk.rst > >>>> >@@ -528,15 +528,15 @@ described in :ref:`dpdk-testpmd`. Once > >>>> >compiled, > >run > >>>the > >>>> >application:: > >>>> > > >>>> > When you finish testing, bind the vNICs back to kernel:: > >>>> > > >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci > 0000:00:03.0 > >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci > 0000:00:04.0 > >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci > 0000:00:03.0 > >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci > >>>> >+ 0000:00:04.0 > >>>> > > >>>> > .. note:: > >>>> > > >>>> > Valid PCI IDs must be passed in above example. The PCI IDs can > >>>> > be > >>>retrieved > >>>> > like so:: > >>>> > > >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --status > >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status > >>>> > > >>>> > More information on the dpdkvhostuser ports can be found in > >>>> >:doc:`/topics/dpdk/vhost-user`. > >>>> >diff --git a/Documentation/intro/install/dpdk.rst > >>>> >b/Documentation/intro/install/dpdk.rst > >>>> >index a05aa1a..20baa63 100644 > >>>> >--- a/Documentation/intro/install/dpdk.rst > >>>> >+++ b/Documentation/intro/install/dpdk.rst > >>>> >@@ -40,7 +40,7 @@ Build requirements In addition to the > >>>> >requirements described in :doc:`general`, building > >Open > >>>> > vSwitch with DPDK will require the following: > >>>> > > >>>> >-- DPDK 16.11 > >>>> >+- DPDK 17.05.1 > >>>> > > >>>> > - A `DPDK supported NIC`_ > >>>> > > >>>> >@@ -69,9 +69,9 @@ Install DPDK > >>>> > #. Download the `DPDK sources`_, extract the file and set > ``DPDK_DIR``:: > >>>> > > >>>> > $ cd /usr/src/ > >>>> >- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz > >>>> >- $ tar xf dpdk-16.11.2.tar.xz > >>>> >- $ export DPDK_DIR=/usr/src/dpdk-stable-16.11.2 > >>>> >+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz > >>>> >+ $ tar xf dpdk-17.05.1.tar.xz > >>>> >+ $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.1 > >>>> > $ cd $DPDK_DIR > >>>> > > >>>> > #. (Optional) Configure DPDK as a shared library @@ -187,8 +187,8 > >>>> >@@ to the VFIO driver:: > >>>> > $ modprobe vfio-pci > >>>> > $ /usr/bin/chmod a+x /dev/vfio > >>>> > $ /usr/bin/chmod 0666 /dev/vfio/* > >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1 > >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --status > >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=vfio-pci eth1 > >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status > >>>> > > >>>> > Setup OVS > >>>> > ~~~~~~~~~ > >>>> >@@ -569,7 +569,7 @@ Limitations > >>>> > The latest list of validated firmware versions can be found in > >>>> >the > >`DPDK > >>>> > release notes`_. > >>>> > > >>>> >-.. _DPDK release notes: > >>>> >http://dpdk.org/doc/guides/rel_notes/release_16_11.html > >>>> >+.. _DPDK release notes: > >>>> >http://dpdk.org/doc/guides/rel_notes/release_17_05.html > >>>> > > >>>> > Reporting Bugs > >>>> > -------------- > >>>> >diff --git a/Documentation/topics/dpdk/vhost-user.rst > >>>> >b/Documentation/topics/dpdk/vhost-user.rst > >>>> >index e76da5f..a3d5de3 100644 > >>>> >--- a/Documentation/topics/dpdk/vhost-user.rst > >>>> >+++ b/Documentation/topics/dpdk/vhost-user.rst > >>>> >@@ -292,9 +292,9 @@ To begin, instantiate a guest as described in > >>>:ref:`dpdk- > >>>> >vhost-user` or > >>>> > DPDK sources to VM and build DPDK:: > >>>> > > >>>> > $ cd /root/dpdk/ > >>>> >- $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz > >>>> >- $ tar xf dpdk-16.11.2.tar.xz > >>>> >- $ export DPDK_DIR=/root/dpdk/dpdk-stable-16.11.2 > >>>> >+ $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz > >>>> >+ $ tar xf dpdk-17.05.1.tar.xz > >>>> >+ $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.1 > >>>> > $ export DPDK_TARGET=x86_64-native-linuxapp-gcc > >>>> > $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET > >>>> > $ cd $DPDK_DIR > >>>> >@@ -314,8 +314,8 @@ Setup huge pages and DPDK devices using UIO:: > >>>> > $ mount -t hugetlbfs hugetlbfs /dev/hugepages # only if not > >>>> >already mounted > >>>> > $ modprobe uio > >>>> > $ insmod $DPDK_BUILD/kmod/igb_uio.ko > >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py --status > >>>> >- $ $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 > >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py --status > >>>> >+ $ $DPDK_DIR/usertools/dpdk-devbind.py -b igb_uio 00:03.0 > >>>> >+ 00:04.0 > >>>> > > >>>> > .. note:: > >>>> > > >>>> >@@ -378,7 +378,7 @@ Sample XML > >>>> > </disk> > >>>> > <disk type='dir' device='disk'> > >>>> > <driver name='qemu' type='fat'/> > >>>> >- <source dir='/usr/src/dpdk-stable-16.11.2'/> > >>>> >+ <source dir='/usr/src/dpdk-stable-17.05.1'/> > >>>> > <target dev='vdb' bus='virtio'/> > >>>> > <readonly/> > >>>> > </disk> > >>>> >diff --git a/NEWS b/NEWS > >>>> >index d61fc5f..3f58e8b 100644 > >>>> >--- a/NEWS > >>>> >+++ b/NEWS > >>>> >@@ -21,6 +21,7 @@ Post-v2.7.0 > >>>> > still can be configured via extra arguments for DPDK EAL. > >>>> > * dpdkvhostuser ports are marked as deprecated. They will > >>>> >be > >removed > >>>> > in an upcoming release. > >>>> >+ * Support for DPDK v17.05.1. > >>>> > - IPFIX now provides additional counters: > >>>> > * Total counters since metering process startup. > >>>> > * Per-flow TCP flag counters. > >>>> >diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index > >>>> >ea17b97..5e767e1 100644 > >>>> >--- a/lib/netdev-dpdk.c > >>>> >+++ b/lib/netdev-dpdk.c > >>>> >@@ -22,6 +22,9 @@ > >>>> > #include <stdlib.h> > >>>> > #include <errno.h> > >>>> > #include <unistd.h> > >>>> >+#include <linux/virtio_net.h> > >>>> >+#include <sys/socket.h> > >>>> >+#include <linux/if.h> > >>>> > > >>>> > #include <rte_config.h> > >>>> > #include <rte_cycles.h> > >>>> >@@ -31,7 +34,7 @@ > >>>> > #include <rte_malloc.h> > >>>> > #include <rte_mbuf.h> > >>>> > #include <rte_meter.h> > >>>> >-#include <rte_virtio_net.h> > >>>> >+#include <rte_vhost.h> > >>>> > > >>>> > #include "dirs.h" > >>>> > #include "dp-packet.h" > >>>> >@@ -56,6 +59,8 @@ > >>>> > #include "timeval.h" > >>>> > #include "unixctl.h" > >>>> > > >>>> >+enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM}; > >>>> >+ > >>>> > VLOG_DEFINE_THIS_MODULE(netdev_dpdk); > >>>> > static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20); > >>>> > > >>>> >@@ -171,6 +176,21 @@ static const struct rte_eth_conf port_conf = { > >>>> > }, > >>>> > }; > >>>> > > >>>> >+/* > >>>> >+ * These callbacks allow virtio-net devices to be added to vhost > >>>> >+ports > >>>when > >>>> >+ * configuration has been fully completed. > >>>> >+ */ > >>>> >+static int new_device(int vid); > >>>> >+static void destroy_device(int vid); static int > >>>> >+vring_state_changed(int vid, uint16_t queue_id, int enable); > >>>> >+static const struct vhost_device_ops virtio_net_device_ops = { > >>>> >+ .new_device = new_device, > >>>> >+ .destroy_device = destroy_device, > >>>> >+ .vring_state_changed = vring_state_changed, > >>>> >+ .features_changed = NULL > >>>> >+}; > >>>> >+ > >>>> > enum { DPDK_RING_SIZE = 256 }; > >>>> > BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE)); > >>>> > enum { DRAIN_TSC = 200000ULL }; > >>>> >@@ -412,8 +432,8 @@ struct netdev_rxq_dpdk { > >>>> > dpdk_port_t port_id; > >>>> > }; > >>>> > > >>>> >-static int netdev_dpdk_class_init(void); -static int > >>>> >netdev_dpdk_vhost_class_init(void); > >>>> >+static void netdev_dpdk_destruct(struct netdev *netdev); static > >>>> >+void netdev_dpdk_vhost_destruct(struct netdev *netdev); > >>>> > > >>>> > int netdev_dpdk_get_vid(const struct netdev_dpdk *dev); > >>>> > > >>>> >@@ -423,8 +443,8 @@ netdev_dpdk_get_ingress_policer(const struct > >>>netdev_dpdk > >>>> >*dev); > >>>> > static bool > >>>> > is_dpdk_class(const struct netdev_class *class) { > >>>> >- return class->init == netdev_dpdk_class_init > >>>> >- || class->init == netdev_dpdk_vhost_class_init; > >>>> >+ return class->destruct == netdev_dpdk_destruct > >>>> >+ || class->destruct == netdev_dpdk_vhost_destruct; > >>>> > } > >>>> > > >>>> > /* DPDK NIC drivers allocate RX buffers at a particular > >>>> >granularity, typically @@ -948,13 +968,45 @@ > >>>> >netdev_dpdk_vhost_construct(struct netdev *netdev) > >>>> > if (err) { > >>>> > VLOG_ERR("vhost-user socket device setup failure for > >>>> >socket > >%s\n", > >>>> > dev->vhost_id); > >>>> >+ goto out; > >>>> > } else { > >>>> > fatal_signal_add_file_to_unlink(dev->vhost_id); > >>>> > VLOG_INFO("Socket %s created for vhost-user port %s\n", > >>>> > dev->vhost_id, name); > >>>> > } > >>>> >+ > >>>> >+ err = rte_vhost_driver_callback_register(dev->vhost_id, > >>>> >+ > &virtio_net_device_ops); > >>>> >+ if (err) { > >>>> >+ VLOG_ERR("rte_vhost_driver_callback_register failed for > >>>> >+ vhost > >user > >>>" > >>>> >+ "port: %s\n", name); > >>>> >+ goto out; > >>>> >+ } > >>>> >+ > >>>> >+ err = rte_vhost_driver_disable_features(dev->vhost_id, > >>>> >+ 1ULL << VIRTIO_NET_F_HOST_TSO4 > >>>> >+ | 1ULL << VIRTIO_NET_F_HOST_TSO6 > >>>> >+ | 1ULL << VIRTIO_NET_F_CSUM); > >>>> >+ if (err) { > >>>> >+ VLOG_ERR("rte_vhost_driver_disable_features failed for > >>>> >+ vhost > >user > >>>" > >>>> >+ "port: %s\n", name); > >>>> >+ goto out; > >>>> >+ } > >>>> >+ > >>>> >+ err = rte_vhost_driver_start(dev->vhost_id); > >>>> >+ if (err) { > >>>> >+ VLOG_ERR("rte_vhost_driver_start failed for vhost user " > >>>> >+ "port: %s\n", name); > >>>> >+ goto out; > >>>> >+ } > >>>> >+ > >>>> > err = vhost_common_construct(netdev); > >>>> >+ if (err) { > >>>> >+ VLOG_ERR("vhost_common_construct failed for vhost user " > >>>> >+ "port: %s\n", name); > >>>> >+ } > >>>> > > >>>> >+out: > >>>> > ovs_mutex_unlock(&dpdk_mutex); > >>>> > VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; > " > >>>> > "please migrate to dpdkvhostuserclient > >>>> >ports."); @@ -968,6 +1020,10 @@ > >>>> >netdev_dpdk_vhost_client_construct(struct netdev > >>>> >*netdev) > >>>> > > >>>> > ovs_mutex_lock(&dpdk_mutex); > >>>> > err = vhost_common_construct(netdev); > >>>> >+ if (err) { > >>>> >+ VLOG_ERR("vhost_common_construct failed for vhost user > client" > >>>> >+ "port: %s\n", netdev->name); > >>>> >+ } > >>>> > ovs_mutex_unlock(&dpdk_mutex); > >>>> > return err; > >>>> > } > >>>> >@@ -2456,12 +2512,9 @@ static void set_irq_status(int vid) { > >>>> > uint32_t i; > >>>> >- uint64_t idx; > >>>> > > >>>> >- for (i = 0; i < rte_vhost_get_queue_num(vid); i++) { > >>>> >- idx = i * VIRTIO_QNUM; > >>>> >- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_RXQ, > 0); > >>>> >- rte_vhost_enable_guest_notification(vid, idx + VIRTIO_TXQ, > 0); > >>>> >+ for (i = 0; i < rte_vhost_get_vring_num(vid); i++) { > >>>> >+ rte_vhost_enable_guest_notification(vid, i, 0); > >>>> > } > >>>> > } > >>>> > > >>>> >@@ -2524,7 +2577,7 @@ new_device(int vid) > >>>> > LIST_FOR_EACH(dev, list_node, &dpdk_list) { > >>>> > ovs_mutex_lock(&dev->mutex); > >>>> > if (strncmp(ifname, dev->vhost_id, IF_NAME_SZ) == 0) { > >>>> >- uint32_t qp_num = rte_vhost_get_queue_num(vid); > >>>> >+ uint32_t qp_num = > >>>> >+ rte_vhost_get_vring_num(vid)/VIRTIO_QNUM; > >>>> > > >>>> > /* Get NUMA information */ > >>>> > newnode = rte_vhost_get_numa_node(vid); @@ -2691,27 > >>>> >+2744,6 @@ netdev_dpdk_get_ingress_policer(const struct > >>>> >netdev_dpdk *dev) > >>>> > return ovsrcu_get(struct ingress_policer *, > >>>> >&dev->ingress_policer); } > >>>> > > >>>> >-/* > >>>> >- * These callbacks allow virtio-net devices to be added to vhost > >>>> >ports > >>>when > >>>> >- * configuration has been fully complete. > >>>> >- */ > >>>> >-static const struct virtio_net_device_ops virtio_net_device_ops = > >>>> >-{ > >>>> >- .new_device = new_device, > >>>> >- .destroy_device = destroy_device, > >>>> >- .vring_state_changed = vring_state_changed > >>>> >-}; > >>>> >- > >>>> >-static void * > >>>> >-start_vhost_loop(void *dummy OVS_UNUSED) -{ > >>>> >- pthread_detach(pthread_self()); > >>>> >- /* Put the vhost thread into quiescent state. */ > >>>> >- ovsrcu_quiesce_start(); > >>>> >- rte_vhost_driver_session_start(); > >>>> >- return NULL; > >>>> >-} > >>>> >- > >>>> > static int > >>>> > netdev_dpdk_class_init(void) > >>>> > { > >>>> >@@ -2731,25 +2763,6 @@ netdev_dpdk_class_init(void) > >>>> > return 0; > >>>> > } > >>>> > > >>>> >-static int > >>>> >-netdev_dpdk_vhost_class_init(void) > >>>> >-{ > >>>> >- static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; > >>>> >- > >>>> >- /* This function can be called for different classes. The > >>>initialization > >>>> >- * needs to be done only once */ > >>>> >- if (ovsthread_once_start(&once)) { > >>>> >- rte_vhost_driver_callback_register(&virtio_net_device_ops); > >>>> >- rte_vhost_feature_disable(1ULL << VIRTIO_NET_F_HOST_TSO4 > >>>> >- | 1ULL << VIRTIO_NET_F_HOST_TSO6 > >>>> >- | 1ULL << VIRTIO_NET_F_CSUM); > >>>> >- ovs_thread_create("vhost_thread", start_vhost_loop, NULL); > >>>> >- > >>>> >- ovsthread_once_done(&once); > >>>> >- } > >>>> >- > >>>> >- return 0; > >>>> >-} > >>>> > > >>>> > /* Client Rings */ > >>>> > > >>>> >@@ -3204,6 +3217,31 @@ netdev_dpdk_vhost_client_reconfigure(struct > >>>> >netdev > >>>> >*netdev) > >>>> > "using client socket '%s'", > >>>> > dev->up.name, dev->vhost_id); > >>>> > } > >>>> >+ > >>>> >+ err = rte_vhost_driver_callback_register(dev->vhost_id, > >>>> >+ > >&virtio_net_device_ops); > >>>> >+ if (err) { > >>>> >+ VLOG_ERR("rte_vhost_driver_callback_register failed for > " > >>>> >+ "vhost user client port: %s\n", dev->up.name); > >>>> >+ goto unlock; > >>>> >+ } > >>>> >+ > >>>> >+ err = rte_vhost_driver_disable_features(dev->vhost_id, > >>>> >+ 1ULL << VIRTIO_NET_F_HOST_TSO4 > >>>> >+ | 1ULL << > VIRTIO_NET_F_HOST_TSO6 > >>>> >+ | 1ULL << VIRTIO_NET_F_CSUM); > >>>> >+ if (err) { > >>>> >+ VLOG_ERR("rte_vhost_driver_disable_features failed > >>>> >+ for vhost > >>>user > >>>> >" > >>>> >+ "client port: %s\n", dev->up.name); > >>>> >+ goto unlock; > >>>> >+ } > >>>> >+ > >>>> >+ err = rte_vhost_driver_start(dev->vhost_id); > >>>> >+ if (err) { > >>>> >+ VLOG_ERR("rte_vhost_driver_start failed for vhost user > " > >>>> >+ "client port: %s\n", dev->up.name); > >>>> >+ goto unlock; > >>>> >+ } > >>>> > } > >>>> > > >>>> > err = dpdk_vhost_reconfigure_helper(dev); > >>>> >@@ -3324,7 +3362,7 @@ static const struct netdev_class > >>>> >dpdk_ring_class = static const struct netdev_class dpdk_vhost_class > = > >>>> > NETDEV_DPDK_CLASS( > >>>> > "dpdkvhostuser", > >>>> >- netdev_dpdk_vhost_class_init, > >>>> >+ NULL, > >>>> > netdev_dpdk_vhost_construct, > >>>> > netdev_dpdk_vhost_destruct, > >>>> > NULL, > >>>> >@@ -3339,7 +3377,7 @@ static const struct netdev_class > >>>> >dpdk_vhost_class = static const struct netdev_class > dpdk_vhost_client_class = > >>>> > NETDEV_DPDK_CLASS( > >>>> > "dpdkvhostuserclient", > >>>> >- netdev_dpdk_vhost_class_init, > >>>> >+ NULL, > >>>> > netdev_dpdk_vhost_client_construct, > >>>> > netdev_dpdk_vhost_destruct, > >>>> > netdev_dpdk_vhost_client_set_config, > >>>> >diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch- > >>>fedora.spec.in > >>>> >index 3a045d3..2bb7102 100644 > >>>> >--- a/rhel/openvswitch-fedora.spec.in > >>>> >+++ b/rhel/openvswitch-fedora.spec.in > >>>> >@@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel %endif > >>>> >%if %{with dpdk} > >>>> > BuildRequires: libpcap-devel numactl-devel > >>>> >-BuildRequires: dpdk-devel >= 16.11 > >>>> >+BuildRequires: dpdk-devel >= 17.05.1 > >>>> > Provides: %{name}-dpdk = %{version}-%{release} %endif > >>>> > > >>>> >diff --git a/tests/dpdk/ring_client.c b/tests/dpdk/ring_client.c > >>>> >index b153713..8cc3fb5 100644 > >>>> >--- a/tests/dpdk/ring_client.c > >>>> >+++ b/tests/dpdk/ring_client.c > >>>> >@@ -185,15 +185,15 @@ main(int argc, char *argv[]) > >>>> > /* Try dequeuing max possible packets first, if that > >>>> >fails, get > >>>the > >>>> > * most we can. Loop body should only execute once, > maximum. > >>>> > */ > >>>> >- while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, > rx_pkts) != > >>>0) > >>>> >&& > >>>> >- rx_pkts > 0) { > >>>> >+ while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, > >>>> >+ rx_pkts, NULL) != 0) && rx_pkts > 0) { > >>>> > rx_pkts = (uint16_t)RTE_MIN(rte_ring_count(rx_ring), > >>>> >PKT_READ_SIZE); > >>>> > } > >>>> > > >>>> > if (rx_pkts > 0) { > >>>> > /* blocking enqueue */ > >>>> > do { > >>>> >- rslt = rte_ring_enqueue_bulk(tx_ring, pkts, > rx_pkts); > >>>> >+ rslt = rte_ring_enqueue_bulk(tx_ring, pkts, > >>>> >+ rx_pkts, > >>>NULL); > >>>> > } while (rslt == -ENOBUFS); > >>>> > } > >>>> > } > >>>> >-- > >>>> >1.8.3.1 > >>>> > > >>>> >_______________________________________________ > >>>> >dev mailing list > >>>> >dev@openvswitch.org > >>>> >https://mail.openvswitch.org/mailman/listinfo/ovs-dev > >> _______________________________________________ > >> dev mailing list > >> dev@openvswitch.org > >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Darrell Ball <dball@vmware.com> writes: > Hi Aaron > > You had mentioned that you would be testing with the patch as well. > Were you able to try it ? I did some limited testing, with single queue and VM-to-VM traffic only (no dpdk physical ports). That seemed to work, but I trust Ian's and Kevin's testing more. Acked-by: Aaron Conole <aconole@redhat.com> > Thanks Darrell > > > > -----Original Message----- > From: <ovs-dev-bounces@openvswitch.org> on behalf of Kevin Traynor > <ktraynor@redhat.com> > Organization: Red Hat > Date: Tuesday, July 25, 2017 at 9:35 AM > To: Michal Weglicki <michalx.weglicki@intel.com>, > "dev@openvswitch.org" <dev@openvswitch.org> > Subject: Re: [ovs-dev] [PATCH v4] Update relevant artifacts to add > support for DPDK 17.05.1. > > On 07/25/2017 01:34 PM, Michal Weglicki wrote: > > Upgrading to DPDK 17.05.1 stable release adds new > > significant features relevant to OVS, including, > > but not limited to: > > - tun/tap PMD, > > - VFIO hotplug support, > > - Generic flow API. > > > > Following changes are applied: > > - netdev-dpdk: Changes required by DPDK API modifications. > > - doc: Because of DPDK API changes, backward compatibility > > with previous DPDK releases will be broken, thus all > > relevant documentation entries are updated. > > - .travis: DPDK version change from 16.11.1 to 17.05.1. > > - rhel/openvswitch-fedora.spec.in: DPDK version change > > from 16.11 to 17.05.1 > > > > v1->v2: Patch rebase. > > v2->v3: Fixed wrong formating after v2 patch rebase. > > v3->v4: Minor documentation changes. > > > > Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> > > Reviewed-by: Aaron Conole <aconole@redhat.com> > > --- > > Thanks Michal. I tried a few things out and working ok for me. > > Acked-by: Kevin Traynor <ktraynor@redhat.com> > _______________________________________________ > dev mailing list > dev@openvswitch.org > https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=ekKh60DEjO7sJHJqmg8W8agp2GxZlvAzz5bWGN3ux4k&s=AZOPEg-J3gwSlEr1nLn9aCrmRt_LBA0YK88h9QS1_-g&e= >
-----Original Message----- From: <ovs-dev-bounces@openvswitch.org> on behalf of Michal Weglicki <michalx.weglicki@intel.com> Date: Tuesday, July 25, 2017 at 5:34 AM To: "dev@openvswitch.org" <dev@openvswitch.org> Subject: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for DPDK 17.05.1. Upgrading to DPDK 17.05.1 stable release adds new significant features relevant to OVS, including, but not limited to: - tun/tap PMD, - VFIO hotplug support, - Generic flow API. Following changes are applied: - netdev-dpdk: Changes required by DPDK API modifications. - doc: Because of DPDK API changes, backward compatibility with previous DPDK releases will be broken, thus all relevant documentation entries are updated. - .travis: DPDK version change from 16.11.1 to 17.05.1. - rhel/openvswitch-fedora.spec.in: DPDK version change from 16.11 to 17.05.1 v1->v2: Patch rebase. v2->v3: Fixed wrong formating after v2 patch rebase. v3->v4: Minor documentation changes. Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> Reviewed-by: Aaron Conole <aconole@redhat.com> --- .travis/linux-build.sh | 2 +- Documentation/faq/releases.rst | 1 + Documentation/howto/dpdk.rst | 6 +- Documentation/intro/install/dpdk.rst | 14 +-- Documentation/topics/dpdk/vhost-user.rst | 12 +-- NEWS | 1 + lib/netdev-dpdk.c | 144 +++++++++++++++++++------------ rhel/openvswitch-fedora.spec.in | 2 +- tests/dpdk/ring_client.c | 6 +- 9 files changed, 114 insertions(+), 74 deletions(-) diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh index f66b534..efccdf1 100755 --- a/.travis/linux-build.sh +++ b/.travis/linux-build.sh @@ -80,7 +80,7 @@ fi if [ "$DPDK" ]; then if [ -z "$DPDK_VER" ]; then - DPDK_VER="16.11.2" + DPDK_VER="17.05.1" fi install_dpdk $DPDK_VER if [ "$CC" = "clang" ]; then diff --git a/Documentation/faq/releases.rst b/Documentation/faq/releases.rst index 707834b..2ecc24c 100644 --- a/Documentation/faq/releases.rst +++ b/Documentation/faq/releases.rst @@ -161,6 +161,7 @@ Q: What DPDK version does each Open vSwitch release work with? 2.5.x 2.2 2.6.x 16.07.2 2.7.x 16.11.2 + 2.8.x 17.05.1 ============ ======= Q: I get an error like this when I configure Open vSwitch: diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index af01d3e..1756c8c 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -528,15 +528,15 @@ described in :ref:`dpdk-testpmd`. Once compiled, run the application:: When you finish testing, bind the vNICs back to kernel:: - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 + $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 + $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 .. note:: Valid PCI IDs must be passed in above example. The PCI IDs can be retrieved like so:: - $ $DPDK_DIR/tools/dpdk-devbind.py --status + $ $DPDK_DIR/usertools/dpdk-devbind.py --status More information on the dpdkvhostuser ports can be found in :doc:`/topics/dpdk/vhost-user`. [Darrell] While dealing with something else, I noticed that this file has an outdated reference, as of OVS 2.8/DPDK 17.05.1, to VFIO not supporting hotplug. Specifically, the text “This feature is not supported with VFIO” under “Port Hotplug” It will be removed unless someone objects. ////////////////// diff --git a/Documentation/intro/install/dpdk.rst b/Documentation/intro/install/dpdk.rst index a05aa1a..20baa63 100644 --- a/Documentation/intro/install/dpdk.rst +++ b/Documentation/intro/install/dpdk.rst @@ -40,7 +40,7 @@ Build requirements In addition to the requirements described in :doc:`general`, building Open vSwitch with DPDK will require the following: -- DPDK 16.11 +- DPDK 17.05.1 - A `DPDK supported NIC`_ @@ -69,9 +69,9 @@ Install DPDK #. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``:: $ cd /usr/src/ - $ wget https://urldefense.proofpoint.com/v2/url?u=http-3A__fast.dpdk.org_rel_dpdk-2D16.11.2.tar.xz&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=1y2ctE1whdC2J1_P88hUEbLvRvKEC-C8osVeldZgdj4&s=MP_pr-aH-lrKfYN45gHDyPQ6aLBg5TGAW1UPDDHkAN4&e= - $ tar xf dpdk-16.11.2.tar.xz - $ export DPDK_DIR=/usr/src/dpdk-stable-16.11.2 + $ wget https://urldefense.proofpoint.com/v2/url?u=http-3A__fast.dpdk.org_rel_dpdk-2D17.05.1.tar.xz&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=1y2ctE1whdC2J1_P88hUEbLvRvKEC-C8osVeldZgdj4&s=f_ilq_XNBOLxaVzU1zrEm3tSh6IUZ3RVzXBuLrVpE6Y&e= + $ tar xf dpdk-17.05.1.tar.xz + $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.1 $ cd $DPDK_DIR #. (Optional) Configure DPDK as a shared library @@ -187,8 +187,8 @@ to the VFIO driver:: $ modprobe vfio-pci $ /usr/bin/chmod a+x /dev/vfio $ /usr/bin/chmod 0666 /dev/vfio/* - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1 - $ $DPDK_DIR/tools/dpdk-devbind.py --status + $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=vfio-pci eth1 + $ $DPDK_DIR/usertools/dpdk-devbind.py --status Setup OVS ~~~~~~~~~ @@ -569,7 +569,7 @@ Limitations The latest list of validated firmware versions can be found in the `DPDK release notes`_. -.. _DPDK release notes: https://urldefense.proofpoint.com/v2/url?u=http-3A__dpdk.org_doc_guides_rel-5Fnotes_release-5F16-5F11.html&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=1y2ctE1whdC2J1_P88hUEbLvRvKEC-C8osVeldZgdj4&s=uw1pYUVE7Qnihq-7TAAFR9r9HMSdL2yzngjzsGsKoJU&e= +.. _DPDK release notes: https://urldefense.proofpoint.com/v2/url?u=http-3A__dpdk.org_doc_guides_rel-5Fnotes_release-5F17-5F05.html&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=1y2ctE1whdC2J1_P88hUEbLvRvKEC-C8osVeldZgdj4&s=OhIAq2Irg3Wp6bSBdFXP7izvHmzjJNN7zaMYYDat5t0&e= Reporting Bugs -------------- diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst index e76da5f..a3d5de3 100644 --- a/Documentation/topics/dpdk/vhost-user.rst +++ b/Documentation/topics/dpdk/vhost-user.rst @@ -292,9 +292,9 @@ To begin, instantiate a guest as described in :ref:`dpdk-vhost-user` or DPDK sources to VM and build DPDK:: $ cd /root/dpdk/ - $ wget https://urldefense.proofpoint.com/v2/url?u=http-3A__fast.dpdk.org_rel_dpdk-2D16.11.2.tar.xz&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=1y2ctE1whdC2J1_P88hUEbLvRvKEC-C8osVeldZgdj4&s=MP_pr-aH-lrKfYN45gHDyPQ6aLBg5TGAW1UPDDHkAN4&e= - $ tar xf dpdk-16.11.2.tar.xz - $ export DPDK_DIR=/root/dpdk/dpdk-stable-16.11.2 + $ wget https://urldefense.proofpoint.com/v2/url?u=http-3A__fast.dpdk.org_rel_dpdk-2D17.05.1.tar.xz&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=1y2ctE1whdC2J1_P88hUEbLvRvKEC-C8osVeldZgdj4&s=f_ilq_XNBOLxaVzU1zrEm3tSh6IUZ3RVzXBuLrVpE6Y&e= + $ tar xf dpdk-17.05.1.tar.xz + $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.1 $ export DPDK_TARGET=x86_64-native-linuxapp-gcc $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET $ cd $DPDK_DIR @@ -314,8 +314,8 @@ Setup huge pages and DPDK devices using UIO:: $ mount -t hugetlbfs hugetlbfs /dev/hugepages # only if not already mounted $ modprobe uio $ insmod $DPDK_BUILD/kmod/igb_uio.ko - $ $DPDK_DIR/tools/dpdk-devbind.py --status - $ $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 + $ $DPDK_DIR/usertools/dpdk-devbind.py --status + $ $DPDK_DIR/usertools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 .. note:: @@ -378,7 +378,7 @@ Sample XML </disk> <disk type='dir' device='disk'> <driver name='qemu' type='fat'/> - <source dir='/usr/src/dpdk-stable-16.11.2'/> + <source dir='/usr/src/dpdk-stable-17.05.1'/> <target dev='vdb' bus='virtio'/> <readonly/> </disk> diff --git a/NEWS b/NEWS index d61fc5f..3f58e8b 100644 --- a/NEWS +++ b/NEWS @@ -21,6 +21,7 @@ Post-v2.7.0 still can be configured via extra arguments for DPDK EAL. * dpdkvhostuser ports are marked as deprecated. They will be removed in an upcoming release. + * Support for DPDK v17.05.1. - IPFIX now provides additional counters: * Total counters since metering process startup. * Per-flow TCP flag counters. diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index ea17b97..5e767e1 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -22,6 +22,9 @@ #include <stdlib.h> #include <errno.h> #include <unistd.h> +#include <linux/virtio_net.h> +#include <sys/socket.h> +#include <linux/if.h> #include <rte_config.h> #include <rte_cycles.h> @@ -31,7 +34,7 @@ #include <rte_malloc.h> #include <rte_mbuf.h> #include <rte_meter.h> -#include <rte_virtio_net.h> +#include <rte_vhost.h> #include "dirs.h" #include "dp-packet.h" @@ -56,6 +59,8 @@ #include "timeval.h" #include "unixctl.h" +enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM}; + VLOG_DEFINE_THIS_MODULE(netdev_dpdk); static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20); @@ -171,6 +176,21 @@ static const struct rte_eth_conf port_conf = { }, }; +/* + * These callbacks allow virtio-net devices to be added to vhost ports when + * configuration has been fully completed. + */ +static int new_device(int vid); +static void destroy_device(int vid); +static int vring_state_changed(int vid, uint16_t queue_id, int enable); +static const struct vhost_device_ops virtio_net_device_ops = +{ + .new_device = new_device, + .destroy_device = destroy_device, + .vring_state_changed = vring_state_changed, + .features_changed = NULL +}; + enum { DPDK_RING_SIZE = 256 }; BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE)); enum { DRAIN_TSC = 200000ULL }; @@ -412,8 +432,8 @@ struct netdev_rxq_dpdk { dpdk_port_t port_id; }; -static int netdev_dpdk_class_init(void); -static int netdev_dpdk_vhost_class_init(void); +static void netdev_dpdk_destruct(struct netdev *netdev); +static void netdev_dpdk_vhost_destruct(struct netdev *netdev); int netdev_dpdk_get_vid(const struct netdev_dpdk *dev); @@ -423,8 +443,8 @@ netdev_dpdk_get_ingress_policer(const struct netdev_dpdk *dev); static bool is_dpdk_class(const struct netdev_class *class) { - return class->init == netdev_dpdk_class_init - || class->init == netdev_dpdk_vhost_class_init; + return class->destruct == netdev_dpdk_destruct + || class->destruct == netdev_dpdk_vhost_destruct; } /* DPDK NIC drivers allocate RX buffers at a particular granularity, typically @@ -948,13 +968,45 @@ netdev_dpdk_vhost_construct(struct netdev *netdev) if (err) { VLOG_ERR("vhost-user socket device setup failure for socket %s\n", dev->vhost_id); + goto out; } else { fatal_signal_add_file_to_unlink(dev->vhost_id); VLOG_INFO("Socket %s created for vhost-user port %s\n", dev->vhost_id, name); } + + err = rte_vhost_driver_callback_register(dev->vhost_id, + &virtio_net_device_ops); + if (err) { + VLOG_ERR("rte_vhost_driver_callback_register failed for vhost user " + "port: %s\n", name); + goto out; + } + + err = rte_vhost_driver_disable_features(dev->vhost_id, + 1ULL << VIRTIO_NET_F_HOST_TSO4 + | 1ULL << VIRTIO_NET_F_HOST_TSO6 + | 1ULL << VIRTIO_NET_F_CSUM); + if (err) { + VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user " + "port: %s\n", name); + goto out; + } + + err = rte_vhost_driver_start(dev->vhost_id); + if (err) { + VLOG_ERR("rte_vhost_driver_start failed for vhost user " + "port: %s\n", name); + goto out; + } + err = vhost_common_construct(netdev); + if (err) { + VLOG_ERR("vhost_common_construct failed for vhost user " + "port: %s\n", name); + } +out: ovs_mutex_unlock(&dpdk_mutex); VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; " "please migrate to dpdkvhostuserclient ports."); @@ -968,6 +1020,10 @@ netdev_dpdk_vhost_client_construct(struct netdev *netdev) ovs_mutex_lock(&dpdk_mutex); err = vhost_common_construct(netdev); + if (err) { + VLOG_ERR("vhost_common_construct failed for vhost user client" + "port: %s\n", netdev->name); + } ovs_mutex_unlock(&dpdk_mutex); return err; } @@ -2456,12 +2512,9 @@ static void set_irq_status(int vid) { uint32_t i; - uint64_t idx; - for (i = 0; i < rte_vhost_get_queue_num(vid); i++) { - idx = i * VIRTIO_QNUM; - rte_vhost_enable_guest_notification(vid, idx + VIRTIO_RXQ, 0); - rte_vhost_enable_guest_notification(vid, idx + VIRTIO_TXQ, 0); + for (i = 0; i < rte_vhost_get_vring_num(vid); i++) { + rte_vhost_enable_guest_notification(vid, i, 0); } } @@ -2524,7 +2577,7 @@ new_device(int vid) LIST_FOR_EACH(dev, list_node, &dpdk_list) { ovs_mutex_lock(&dev->mutex); if (strncmp(ifname, dev->vhost_id, IF_NAME_SZ) == 0) { - uint32_t qp_num = rte_vhost_get_queue_num(vid); + uint32_t qp_num = rte_vhost_get_vring_num(vid)/VIRTIO_QNUM; /* Get NUMA information */ newnode = rte_vhost_get_numa_node(vid); @@ -2691,27 +2744,6 @@ netdev_dpdk_get_ingress_policer(const struct netdev_dpdk *dev) return ovsrcu_get(struct ingress_policer *, &dev->ingress_policer); } -/* - * These callbacks allow virtio-net devices to be added to vhost ports when - * configuration has been fully complete. - */ -static const struct virtio_net_device_ops virtio_net_device_ops = -{ - .new_device = new_device, - .destroy_device = destroy_device, - .vring_state_changed = vring_state_changed -}; - -static void * -start_vhost_loop(void *dummy OVS_UNUSED) -{ - pthread_detach(pthread_self()); - /* Put the vhost thread into quiescent state. */ - ovsrcu_quiesce_start(); - rte_vhost_driver_session_start(); - return NULL; -} - static int netdev_dpdk_class_init(void) { @@ -2731,25 +2763,6 @@ netdev_dpdk_class_init(void) return 0; } -static int -netdev_dpdk_vhost_class_init(void) -{ - static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; - - /* This function can be called for different classes. The initialization - * needs to be done only once */ - if (ovsthread_once_start(&once)) { - rte_vhost_driver_callback_register(&virtio_net_device_ops); - rte_vhost_feature_disable(1ULL << VIRTIO_NET_F_HOST_TSO4 - | 1ULL << VIRTIO_NET_F_HOST_TSO6 - | 1ULL << VIRTIO_NET_F_CSUM); - ovs_thread_create("vhost_thread", start_vhost_loop, NULL); - - ovsthread_once_done(&once); - } - - return 0; -} /* Client Rings */ @@ -3204,6 +3217,31 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev) "using client socket '%s'", dev->up.name, dev->vhost_id); } + + err = rte_vhost_driver_callback_register(dev->vhost_id, + &virtio_net_device_ops); + if (err) { + VLOG_ERR("rte_vhost_driver_callback_register failed for " + "vhost user client port: %s\n", dev->up.name); + goto unlock; + } + + err = rte_vhost_driver_disable_features(dev->vhost_id, + 1ULL << VIRTIO_NET_F_HOST_TSO4 + | 1ULL << VIRTIO_NET_F_HOST_TSO6 + | 1ULL << VIRTIO_NET_F_CSUM); + if (err) { + VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user " + "client port: %s\n", dev->up.name); + goto unlock; + } + + err = rte_vhost_driver_start(dev->vhost_id); + if (err) { + VLOG_ERR("rte_vhost_driver_start failed for vhost user " + "client port: %s\n", dev->up.name); + goto unlock; + } } err = dpdk_vhost_reconfigure_helper(dev); @@ -3324,7 +3362,7 @@ static const struct netdev_class dpdk_ring_class = static const struct netdev_class dpdk_vhost_class = NETDEV_DPDK_CLASS( "dpdkvhostuser", - netdev_dpdk_vhost_class_init, + NULL, netdev_dpdk_vhost_construct, netdev_dpdk_vhost_destruct, NULL, @@ -3339,7 +3377,7 @@ static const struct netdev_class dpdk_vhost_class = static const struct netdev_class dpdk_vhost_client_class = NETDEV_DPDK_CLASS( "dpdkvhostuserclient", - netdev_dpdk_vhost_class_init, + NULL, netdev_dpdk_vhost_client_construct, netdev_dpdk_vhost_destruct, netdev_dpdk_vhost_client_set_config, diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch-fedora.spec.in index 3a045d3..2bb7102 100644 --- a/rhel/openvswitch-fedora.spec.in +++ b/rhel/openvswitch-fedora.spec.in @@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel %endif %if %{with dpdk} BuildRequires: libpcap-devel numactl-devel -BuildRequires: dpdk-devel >= 16.11 +BuildRequires: dpdk-devel >= 17.05.1 Provides: %{name}-dpdk = %{version}-%{release} %endif diff --git a/tests/dpdk/ring_client.c b/tests/dpdk/ring_client.c index b153713..8cc3fb5 100644 --- a/tests/dpdk/ring_client.c +++ b/tests/dpdk/ring_client.c @@ -185,15 +185,15 @@ main(int argc, char *argv[]) /* Try dequeuing max possible packets first, if that fails, get the * most we can. Loop body should only execute once, maximum. */ - while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, rx_pkts) != 0) && - rx_pkts > 0) { + while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, + rx_pkts, NULL) != 0) && rx_pkts > 0) { rx_pkts = (uint16_t)RTE_MIN(rte_ring_count(rx_ring), PKT_READ_SIZE); } if (rx_pkts > 0) { /* blocking enqueue */ do { - rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts); + rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts, NULL); } while (rslt == -ENOBUFS); } } -- 1.8.3.1 _______________________________________________ dev mailing list dev@openvswitch.org https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddev&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=1y2ctE1whdC2J1_P88hUEbLvRvKEC-C8osVeldZgdj4&s=NRh9tIHfSupitemjr2ORp8ymj_X_AUj4NAoJwv9LL_s&e=
Darrell Ball <dball@vmware.com> writes: > -----Original Message----- > From: <ovs-dev-bounces@openvswitch.org> on behalf of Michal Weglicki <michalx.weglicki@intel.com> > Date: Tuesday, July 25, 2017 at 5:34 AM > To: "dev@openvswitch.org" <dev@openvswitch.org> > Subject: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for DPDK 17.05.1. > > Upgrading to DPDK 17.05.1 stable release adds new > significant features relevant to OVS, including, > but not limited to: > - tun/tap PMD, > - VFIO hotplug support, > - Generic flow API. > > Following changes are applied: > - netdev-dpdk: Changes required by DPDK API modifications. > - doc: Because of DPDK API changes, backward compatibility > with previous DPDK releases will be broken, thus all > relevant documentation entries are updated. > - .travis: DPDK version change from 16.11.1 to 17.05.1. > - rhel/openvswitch-fedora.spec.in: DPDK version change > from 16.11 to 17.05.1 > > v1->v2: Patch rebase. > v2->v3: Fixed wrong formating after v2 patch rebase. > v3->v4: Minor documentation changes. > > Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> > Reviewed-by: Aaron Conole <aconole@redhat.com> > --- > .travis/linux-build.sh | 2 +- > Documentation/faq/releases.rst | 1 + > Documentation/howto/dpdk.rst | 6 +- > Documentation/intro/install/dpdk.rst | 14 +-- > Documentation/topics/dpdk/vhost-user.rst | 12 +-- > NEWS | 1 + > lib/netdev-dpdk.c | 144 +++++++++++++++++++------------ > rhel/openvswitch-fedora.spec.in | 2 +- > tests/dpdk/ring_client.c | 6 +- > 9 files changed, 114 insertions(+), 74 deletions(-) > > diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh > index f66b534..efccdf1 100755 > --- a/.travis/linux-build.sh > +++ b/.travis/linux-build.sh > @@ -80,7 +80,7 @@ fi > > if [ "$DPDK" ]; then > if [ -z "$DPDK_VER" ]; then > - DPDK_VER="16.11.2" > + DPDK_VER="17.05.1" > fi > install_dpdk $DPDK_VER > if [ "$CC" = "clang" ]; then > diff --git a/Documentation/faq/releases.rst b/Documentation/faq/releases.rst > index 707834b..2ecc24c 100644 > --- a/Documentation/faq/releases.rst > +++ b/Documentation/faq/releases.rst > @@ -161,6 +161,7 @@ Q: What DPDK version does each Open vSwitch release work with? > 2.5.x 2.2 > 2.6.x 16.07.2 > 2.7.x 16.11.2 > + 2.8.x 17.05.1 > ============ ======= > > Q: I get an error like this when I configure Open vSwitch: > diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst > index af01d3e..1756c8c 100644 > --- a/Documentation/howto/dpdk.rst > +++ b/Documentation/howto/dpdk.rst > @@ -528,15 +528,15 @@ described in :ref:`dpdk-testpmd`. Once compiled, run the application:: > > When you finish testing, bind the vNICs back to kernel:: > > - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 > - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 > + $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 > + $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 > > .. note:: > > Valid PCI IDs must be passed in above example. The PCI IDs can be retrieved > like so:: > > - $ $DPDK_DIR/tools/dpdk-devbind.py --status > + $ $DPDK_DIR/usertools/dpdk-devbind.py --status > > More information on the dpdkvhostuser ports can be found in > :doc:`/topics/dpdk/vhost-user`. > > [Darrell] > While dealing with something else, I noticed that this file has an outdated reference, > as of OVS 2.8/DPDK 17.05.1, to VFIO not supporting hotplug. Specifically, the text > > “This feature is not supported with VFIO” under “Port Hotplug” > > It will be removed unless someone objects. > > ////////////////// That makes sense. OTOH, how is it going to work as far as applying / pulling / etc. goes? I know that the DPDK related work is planning on going with a pull-request model. Does that mean this patch is applied to your branch and you will submit a pull request with it at some point? How about work that builds on this patch? As an example, https://mail.openvswitch.org/pipermail/ovs-dev/2017-July/335250.html I'd like to submit my latest work for this series, but not sure where or how. -Aaron
-----Original Message----- From: Aaron Conole <aconole@redhat.com> Date: Sunday, July 30, 2017 at 8:22 AM To: Darrell Ball <dball@vmware.com> Cc: Michal Weglicki <michalx.weglicki@intel.com>, "dev@openvswitch.org" <dev@openvswitch.org> Subject: Re: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for DPDK 17.05.1. Darrell Ball <dball@vmware.com> writes: > -----Original Message----- > From: <ovs-dev-bounces@openvswitch.org> on behalf of Michal Weglicki <michalx.weglicki@intel.com> > Date: Tuesday, July 25, 2017 at 5:34 AM > To: "dev@openvswitch.org" <dev@openvswitch.org> > Subject: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for DPDK 17.05.1. > > Upgrading to DPDK 17.05.1 stable release adds new > significant features relevant to OVS, including, > but not limited to: > - tun/tap PMD, > - VFIO hotplug support, > - Generic flow API. > > Following changes are applied: > - netdev-dpdk: Changes required by DPDK API modifications. > - doc: Because of DPDK API changes, backward compatibility > with previous DPDK releases will be broken, thus all > relevant documentation entries are updated. > - .travis: DPDK version change from 16.11.1 to 17.05.1. > - rhel/openvswitch-fedora.spec.in: DPDK version change > from 16.11 to 17.05.1 > > v1->v2: Patch rebase. > v2->v3: Fixed wrong formating after v2 patch rebase. > v3->v4: Minor documentation changes. > > Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> > Reviewed-by: Aaron Conole <aconole@redhat.com> > --- > .travis/linux-build.sh | 2 +- > Documentation/faq/releases.rst | 1 + > Documentation/howto/dpdk.rst | 6 +- > Documentation/intro/install/dpdk.rst | 14 +-- > Documentation/topics/dpdk/vhost-user.rst | 12 +-- > NEWS | 1 + > lib/netdev-dpdk.c | 144 +++++++++++++++++++------------ > rhel/openvswitch-fedora.spec.in | 2 +- > tests/dpdk/ring_client.c | 6 +- > 9 files changed, 114 insertions(+), 74 deletions(-) > > diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh > index f66b534..efccdf1 100755 > --- a/.travis/linux-build.sh > +++ b/.travis/linux-build.sh > @@ -80,7 +80,7 @@ fi > > if [ "$DPDK" ]; then > if [ -z "$DPDK_VER" ]; then > - DPDK_VER="16.11.2" > + DPDK_VER="17.05.1" > fi > install_dpdk $DPDK_VER > if [ "$CC" = "clang" ]; then > diff --git a/Documentation/faq/releases.rst b/Documentation/faq/releases.rst > index 707834b..2ecc24c 100644 > --- a/Documentation/faq/releases.rst > +++ b/Documentation/faq/releases.rst > @@ -161,6 +161,7 @@ Q: What DPDK version does each Open vSwitch release work with? > 2.5.x 2.2 > 2.6.x 16.07.2 > 2.7.x 16.11.2 > + 2.8.x 17.05.1 > ============ ======= > > Q: I get an error like this when I configure Open vSwitch: > diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst > index af01d3e..1756c8c 100644 > --- a/Documentation/howto/dpdk.rst > +++ b/Documentation/howto/dpdk.rst > @@ -528,15 +528,15 @@ described in :ref:`dpdk-testpmd`. Once compiled, run the application:: > > When you finish testing, bind the vNICs back to kernel:: > > - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 > - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 > + $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 > + $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 > > .. note:: > > Valid PCI IDs must be passed in above example. The PCI IDs can be retrieved > like so:: > > - $ $DPDK_DIR/tools/dpdk-devbind.py --status > + $ $DPDK_DIR/usertools/dpdk-devbind.py --status > > More information on the dpdkvhostuser ports can be found in > :doc:`/topics/dpdk/vhost-user`. > > [Darrell] > While dealing with something else, I noticed that this file has an outdated reference, > as of OVS 2.8/DPDK 17.05.1, to VFIO not supporting hotplug. Specifically, the text > > “This feature is not supported with VFIO” under “Port Hotplug” > > It will be removed unless someone objects. > > ////////////////// That makes sense. OTOH, how is it going to work as far as applying / pulling / etc. goes? I know that the DPDK related work is planning on going with a pull-request model. Does that mean this patch is applied to your branch and you will submit a pull request with it at some point? [Darrell] Right. How about work that builds on this patch? As an example, https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_pipermail_ovs-2Ddev_2017-2DJuly_335250.html&d=DwIFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=FSHC_5hvqniBmseIL_5bT8jWrzB0GdkTg4pt_u9a5MY&s=ovw7pxgYKy81nrb2hiiBmebXdzFPy3OzgptTCBKUtlI&e= I'd like to submit my latest work for this series, but not sure where or how. [Darrell] I have followed your series; since it is related to packaging and infra, it is outside the scope of dpdk core changes. Your series should go thru. the usual path. The code for this patch (i.e. Update relevant artifacts to add support for DPDK 17.05.1.) has converged (tested by several people) and is not changing, so your series can simply say it is based on this patch. We have identified patches in the past as depending on another patch, which is not yet applied. In this case, it is more simple, since the code for this patch (i.e. Update relevant artifacts to add support for DPDK 17.05.1.) will not change. If there is anything going forward, it will be a separate bug fix. -Aaron
On 07/25/2017 02:34 PM, Michal Weglicki wrote: > Upgrading to DPDK 17.05.1 stable release adds new > significant features relevant to OVS, including, > but not limited to: > - tun/tap PMD, > - VFIO hotplug support, > - Generic flow API. > > Following changes are applied: > - netdev-dpdk: Changes required by DPDK API modifications. > - doc: Because of DPDK API changes, backward compatibility > with previous DPDK releases will be broken, thus all > relevant documentation entries are updated. > - .travis: DPDK version change from 16.11.1 to 17.05.1. > - rhel/openvswitch-fedora.spec.in: DPDK version change > from 16.11 to 17.05.1 > > v1->v2: Patch rebase. > v2->v3: Fixed wrong formating after v2 patch rebase. > v3->v4: Minor documentation changes. > > Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> > Reviewed-by: Aaron Conole <aconole@redhat.com> > --- > .travis/linux-build.sh | 2 +- > Documentation/faq/releases.rst | 1 + > Documentation/howto/dpdk.rst | 6 +- > Documentation/intro/install/dpdk.rst | 14 +-- > Documentation/topics/dpdk/vhost-user.rst | 12 +-- > NEWS | 1 + > lib/netdev-dpdk.c | 144 +++++++++++++++++++------------ > rhel/openvswitch-fedora.spec.in | 2 +- > tests/dpdk/ring_client.c | 6 +- > 9 files changed, 114 insertions(+), 74 deletions(-) > [...] > diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch-fedora.spec.in > index 3a045d3..2bb7102 100644 > --- a/rhel/openvswitch-fedora.spec.in > +++ b/rhel/openvswitch-fedora.spec.in > @@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel > %endif > %if %{with dpdk} > BuildRequires: libpcap-devel numactl-devel > -BuildRequires: dpdk-devel >= 16.11 > +BuildRequires: dpdk-devel >= 17.05.1 > Provides: %{name}-dpdk = %{version}-%{release} > %endif > Hi, on Fedora there is no dpdk-devel 17.05.1 package, the last rawhide version is dpdk-devel 17.05 (and usually Fedora doesn't package minor releases of dpdk). I tried to build openvswitch with the dpdk 17.05 package from rawhide. It build and it works (tested with dpdkvhostuserclient) so, unless you need a specific fix from 17.05.1, I suggest to change the BuildRequires to 17.05 instead of 17.05.1 or the package cannot be built on Fedora, unless you are using an additional repository (copr).
> -----Original Message----- > From: ovs-dev-bounces@openvswitch.org [mailto:ovs-dev- > bounces@openvswitch.org] On Behalf Of Timothy M. Redaelli > Sent: Tuesday, August 1, 2017 11:37 AM > To: Weglicki, MichalX <michalx.weglicki@intel.com>; dev@openvswitch.org > Subject: Re: [ovs-dev] [PATCH v4] Update relevant artifacts to add support for > DPDK 17.05.1. > > On 07/25/2017 02:34 PM, Michal Weglicki wrote: > > Upgrading to DPDK 17.05.1 stable release adds new > > significant features relevant to OVS, including, > > but not limited to: > > - tun/tap PMD, > > - VFIO hotplug support, > > - Generic flow API. > > > > Following changes are applied: > > - netdev-dpdk: Changes required by DPDK API modifications. > > - doc: Because of DPDK API changes, backward compatibility > > with previous DPDK releases will be broken, thus all > > relevant documentation entries are updated. > > - .travis: DPDK version change from 16.11.1 to 17.05.1. > > - rhel/openvswitch-fedora.spec.in: DPDK version change > > from 16.11 to 17.05.1 > > > > v1->v2: Patch rebase. > > v2->v3: Fixed wrong formating after v2 patch rebase. > > v3->v4: Minor documentation changes. > > > > Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com> > > Reviewed-by: Aaron Conole <aconole@redhat.com> > > --- > > .travis/linux-build.sh | 2 +- > > Documentation/faq/releases.rst | 1 + > > Documentation/howto/dpdk.rst | 6 +- > > Documentation/intro/install/dpdk.rst | 14 +-- > > Documentation/topics/dpdk/vhost-user.rst | 12 +-- > > NEWS | 1 + > > lib/netdev-dpdk.c | 144 +++++++++++++++++++------------ > > rhel/openvswitch-fedora.spec.in | 2 +- > > tests/dpdk/ring_client.c | 6 +- > > 9 files changed, 114 insertions(+), 74 deletions(-) > > > [...] > > diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch-fedora.spec.in > > index 3a045d3..2bb7102 100644 > > --- a/rhel/openvswitch-fedora.spec.in > > +++ b/rhel/openvswitch-fedora.spec.in > > @@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel > > %endif > > %if %{with dpdk} > > BuildRequires: libpcap-devel numactl-devel > > -BuildRequires: dpdk-devel >= 16.11 > > +BuildRequires: dpdk-devel >= 17.05.1 > > Provides: %{name}-dpdk = %{version}-%{release} > > %endif > > > > > Hi, > on Fedora there is no dpdk-devel 17.05.1 package, the last rawhide > version is dpdk-devel 17.05 (and usually Fedora doesn't package minor > releases of dpdk). > > I tried to build openvswitch with the dpdk 17.05 package from rawhide. > It build and it works (tested with dpdkvhostuserclient) so, unless you > need a specific fix from 17.05.1, I suggest to change the BuildRequires > to 17.05 instead of 17.05.1 or the package cannot be built on Fedora, > unless you are using an additional repository (copr). Hi, If I'm not mistaken we moved to 17.05.1 version, because it includes fix for 'vhost: fix crash on NUMA', which was causing ovs-vswitchd to terminate. More details here: http://dpdk.org/dev/patchwork/patch/25012/ Regards, Przemek -------------------------------------------------------------- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh index f66b534..efccdf1 100755 --- a/.travis/linux-build.sh +++ b/.travis/linux-build.sh @@ -80,7 +80,7 @@ fi if [ "$DPDK" ]; then if [ -z "$DPDK_VER" ]; then - DPDK_VER="16.11.2" + DPDK_VER="17.05.1" fi install_dpdk $DPDK_VER if [ "$CC" = "clang" ]; then diff --git a/Documentation/faq/releases.rst b/Documentation/faq/releases.rst index 707834b..2ecc24c 100644 --- a/Documentation/faq/releases.rst +++ b/Documentation/faq/releases.rst @@ -161,6 +161,7 @@ Q: What DPDK version does each Open vSwitch release work with? 2.5.x 2.2 2.6.x 16.07.2 2.7.x 16.11.2 + 2.8.x 17.05.1 ============ ======= Q: I get an error like this when I configure Open vSwitch: diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index af01d3e..1756c8c 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -528,15 +528,15 @@ described in :ref:`dpdk-testpmd`. Once compiled, run the application:: When you finish testing, bind the vNICs back to kernel:: - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 + $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 + $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 .. note:: Valid PCI IDs must be passed in above example. The PCI IDs can be retrieved like so:: - $ $DPDK_DIR/tools/dpdk-devbind.py --status + $ $DPDK_DIR/usertools/dpdk-devbind.py --status More information on the dpdkvhostuser ports can be found in :doc:`/topics/dpdk/vhost-user`. diff --git a/Documentation/intro/install/dpdk.rst b/Documentation/intro/install/dpdk.rst index a05aa1a..20baa63 100644 --- a/Documentation/intro/install/dpdk.rst +++ b/Documentation/intro/install/dpdk.rst @@ -40,7 +40,7 @@ Build requirements In addition to the requirements described in :doc:`general`, building Open vSwitch with DPDK will require the following: -- DPDK 16.11 +- DPDK 17.05.1 - A `DPDK supported NIC`_ @@ -69,9 +69,9 @@ Install DPDK #. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``:: $ cd /usr/src/ - $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz - $ tar xf dpdk-16.11.2.tar.xz - $ export DPDK_DIR=/usr/src/dpdk-stable-16.11.2 + $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz + $ tar xf dpdk-17.05.1.tar.xz + $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.1 $ cd $DPDK_DIR #. (Optional) Configure DPDK as a shared library @@ -187,8 +187,8 @@ to the VFIO driver:: $ modprobe vfio-pci $ /usr/bin/chmod a+x /dev/vfio $ /usr/bin/chmod 0666 /dev/vfio/* - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1 - $ $DPDK_DIR/tools/dpdk-devbind.py --status + $ $DPDK_DIR/usertools/dpdk-devbind.py --bind=vfio-pci eth1 + $ $DPDK_DIR/usertools/dpdk-devbind.py --status Setup OVS ~~~~~~~~~ @@ -569,7 +569,7 @@ Limitations The latest list of validated firmware versions can be found in the `DPDK release notes`_. -.. _DPDK release notes: http://dpdk.org/doc/guides/rel_notes/release_16_11.html +.. _DPDK release notes: http://dpdk.org/doc/guides/rel_notes/release_17_05.html Reporting Bugs -------------- diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst index e76da5f..a3d5de3 100644 --- a/Documentation/topics/dpdk/vhost-user.rst +++ b/Documentation/topics/dpdk/vhost-user.rst @@ -292,9 +292,9 @@ To begin, instantiate a guest as described in :ref:`dpdk-vhost-user` or DPDK sources to VM and build DPDK:: $ cd /root/dpdk/ - $ wget http://fast.dpdk.org/rel/dpdk-16.11.2.tar.xz - $ tar xf dpdk-16.11.2.tar.xz - $ export DPDK_DIR=/root/dpdk/dpdk-stable-16.11.2 + $ wget http://fast.dpdk.org/rel/dpdk-17.05.1.tar.xz + $ tar xf dpdk-17.05.1.tar.xz + $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.1 $ export DPDK_TARGET=x86_64-native-linuxapp-gcc $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET $ cd $DPDK_DIR @@ -314,8 +314,8 @@ Setup huge pages and DPDK devices using UIO:: $ mount -t hugetlbfs hugetlbfs /dev/hugepages # only if not already mounted $ modprobe uio $ insmod $DPDK_BUILD/kmod/igb_uio.ko - $ $DPDK_DIR/tools/dpdk-devbind.py --status - $ $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 + $ $DPDK_DIR/usertools/dpdk-devbind.py --status + $ $DPDK_DIR/usertools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 .. note:: @@ -378,7 +378,7 @@ Sample XML </disk> <disk type='dir' device='disk'> <driver name='qemu' type='fat'/> - <source dir='/usr/src/dpdk-stable-16.11.2'/> + <source dir='/usr/src/dpdk-stable-17.05.1'/> <target dev='vdb' bus='virtio'/> <readonly/> </disk> diff --git a/NEWS b/NEWS index d61fc5f..3f58e8b 100644 --- a/NEWS +++ b/NEWS @@ -21,6 +21,7 @@ Post-v2.7.0 still can be configured via extra arguments for DPDK EAL. * dpdkvhostuser ports are marked as deprecated. They will be removed in an upcoming release. + * Support for DPDK v17.05.1. - IPFIX now provides additional counters: * Total counters since metering process startup. * Per-flow TCP flag counters. diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index ea17b97..5e767e1 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -22,6 +22,9 @@ #include <stdlib.h> #include <errno.h> #include <unistd.h> +#include <linux/virtio_net.h> +#include <sys/socket.h> +#include <linux/if.h> #include <rte_config.h> #include <rte_cycles.h> @@ -31,7 +34,7 @@ #include <rte_malloc.h> #include <rte_mbuf.h> #include <rte_meter.h> -#include <rte_virtio_net.h> +#include <rte_vhost.h> #include "dirs.h" #include "dp-packet.h" @@ -56,6 +59,8 @@ #include "timeval.h" #include "unixctl.h" +enum {VIRTIO_RXQ, VIRTIO_TXQ, VIRTIO_QNUM}; + VLOG_DEFINE_THIS_MODULE(netdev_dpdk); static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20); @@ -171,6 +176,21 @@ static const struct rte_eth_conf port_conf = { }, }; +/* + * These callbacks allow virtio-net devices to be added to vhost ports when + * configuration has been fully completed. + */ +static int new_device(int vid); +static void destroy_device(int vid); +static int vring_state_changed(int vid, uint16_t queue_id, int enable); +static const struct vhost_device_ops virtio_net_device_ops = +{ + .new_device = new_device, + .destroy_device = destroy_device, + .vring_state_changed = vring_state_changed, + .features_changed = NULL +}; + enum { DPDK_RING_SIZE = 256 }; BUILD_ASSERT_DECL(IS_POW2(DPDK_RING_SIZE)); enum { DRAIN_TSC = 200000ULL }; @@ -412,8 +432,8 @@ struct netdev_rxq_dpdk { dpdk_port_t port_id; }; -static int netdev_dpdk_class_init(void); -static int netdev_dpdk_vhost_class_init(void); +static void netdev_dpdk_destruct(struct netdev *netdev); +static void netdev_dpdk_vhost_destruct(struct netdev *netdev); int netdev_dpdk_get_vid(const struct netdev_dpdk *dev); @@ -423,8 +443,8 @@ netdev_dpdk_get_ingress_policer(const struct netdev_dpdk *dev); static bool is_dpdk_class(const struct netdev_class *class) { - return class->init == netdev_dpdk_class_init - || class->init == netdev_dpdk_vhost_class_init; + return class->destruct == netdev_dpdk_destruct + || class->destruct == netdev_dpdk_vhost_destruct; } /* DPDK NIC drivers allocate RX buffers at a particular granularity, typically @@ -948,13 +968,45 @@ netdev_dpdk_vhost_construct(struct netdev *netdev) if (err) { VLOG_ERR("vhost-user socket device setup failure for socket %s\n", dev->vhost_id); + goto out; } else { fatal_signal_add_file_to_unlink(dev->vhost_id); VLOG_INFO("Socket %s created for vhost-user port %s\n", dev->vhost_id, name); } + + err = rte_vhost_driver_callback_register(dev->vhost_id, + &virtio_net_device_ops); + if (err) { + VLOG_ERR("rte_vhost_driver_callback_register failed for vhost user " + "port: %s\n", name); + goto out; + } + + err = rte_vhost_driver_disable_features(dev->vhost_id, + 1ULL << VIRTIO_NET_F_HOST_TSO4 + | 1ULL << VIRTIO_NET_F_HOST_TSO6 + | 1ULL << VIRTIO_NET_F_CSUM); + if (err) { + VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user " + "port: %s\n", name); + goto out; + } + + err = rte_vhost_driver_start(dev->vhost_id); + if (err) { + VLOG_ERR("rte_vhost_driver_start failed for vhost user " + "port: %s\n", name); + goto out; + } + err = vhost_common_construct(netdev); + if (err) { + VLOG_ERR("vhost_common_construct failed for vhost user " + "port: %s\n", name); + } +out: ovs_mutex_unlock(&dpdk_mutex); VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; " "please migrate to dpdkvhostuserclient ports."); @@ -968,6 +1020,10 @@ netdev_dpdk_vhost_client_construct(struct netdev *netdev) ovs_mutex_lock(&dpdk_mutex); err = vhost_common_construct(netdev); + if (err) { + VLOG_ERR("vhost_common_construct failed for vhost user client" + "port: %s\n", netdev->name); + } ovs_mutex_unlock(&dpdk_mutex); return err; } @@ -2456,12 +2512,9 @@ static void set_irq_status(int vid) { uint32_t i; - uint64_t idx; - for (i = 0; i < rte_vhost_get_queue_num(vid); i++) { - idx = i * VIRTIO_QNUM; - rte_vhost_enable_guest_notification(vid, idx + VIRTIO_RXQ, 0); - rte_vhost_enable_guest_notification(vid, idx + VIRTIO_TXQ, 0); + for (i = 0; i < rte_vhost_get_vring_num(vid); i++) { + rte_vhost_enable_guest_notification(vid, i, 0); } } @@ -2524,7 +2577,7 @@ new_device(int vid) LIST_FOR_EACH(dev, list_node, &dpdk_list) { ovs_mutex_lock(&dev->mutex); if (strncmp(ifname, dev->vhost_id, IF_NAME_SZ) == 0) { - uint32_t qp_num = rte_vhost_get_queue_num(vid); + uint32_t qp_num = rte_vhost_get_vring_num(vid)/VIRTIO_QNUM; /* Get NUMA information */ newnode = rte_vhost_get_numa_node(vid); @@ -2691,27 +2744,6 @@ netdev_dpdk_get_ingress_policer(const struct netdev_dpdk *dev) return ovsrcu_get(struct ingress_policer *, &dev->ingress_policer); } -/* - * These callbacks allow virtio-net devices to be added to vhost ports when - * configuration has been fully complete. - */ -static const struct virtio_net_device_ops virtio_net_device_ops = -{ - .new_device = new_device, - .destroy_device = destroy_device, - .vring_state_changed = vring_state_changed -}; - -static void * -start_vhost_loop(void *dummy OVS_UNUSED) -{ - pthread_detach(pthread_self()); - /* Put the vhost thread into quiescent state. */ - ovsrcu_quiesce_start(); - rte_vhost_driver_session_start(); - return NULL; -} - static int netdev_dpdk_class_init(void) { @@ -2731,25 +2763,6 @@ netdev_dpdk_class_init(void) return 0; } -static int -netdev_dpdk_vhost_class_init(void) -{ - static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; - - /* This function can be called for different classes. The initialization - * needs to be done only once */ - if (ovsthread_once_start(&once)) { - rte_vhost_driver_callback_register(&virtio_net_device_ops); - rte_vhost_feature_disable(1ULL << VIRTIO_NET_F_HOST_TSO4 - | 1ULL << VIRTIO_NET_F_HOST_TSO6 - | 1ULL << VIRTIO_NET_F_CSUM); - ovs_thread_create("vhost_thread", start_vhost_loop, NULL); - - ovsthread_once_done(&once); - } - - return 0; -} /* Client Rings */ @@ -3204,6 +3217,31 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev) "using client socket '%s'", dev->up.name, dev->vhost_id); } + + err = rte_vhost_driver_callback_register(dev->vhost_id, + &virtio_net_device_ops); + if (err) { + VLOG_ERR("rte_vhost_driver_callback_register failed for " + "vhost user client port: %s\n", dev->up.name); + goto unlock; + } + + err = rte_vhost_driver_disable_features(dev->vhost_id, + 1ULL << VIRTIO_NET_F_HOST_TSO4 + | 1ULL << VIRTIO_NET_F_HOST_TSO6 + | 1ULL << VIRTIO_NET_F_CSUM); + if (err) { + VLOG_ERR("rte_vhost_driver_disable_features failed for vhost user " + "client port: %s\n", dev->up.name); + goto unlock; + } + + err = rte_vhost_driver_start(dev->vhost_id); + if (err) { + VLOG_ERR("rte_vhost_driver_start failed for vhost user " + "client port: %s\n", dev->up.name); + goto unlock; + } } err = dpdk_vhost_reconfigure_helper(dev); @@ -3324,7 +3362,7 @@ static const struct netdev_class dpdk_ring_class = static const struct netdev_class dpdk_vhost_class = NETDEV_DPDK_CLASS( "dpdkvhostuser", - netdev_dpdk_vhost_class_init, + NULL, netdev_dpdk_vhost_construct, netdev_dpdk_vhost_destruct, NULL, @@ -3339,7 +3377,7 @@ static const struct netdev_class dpdk_vhost_class = static const struct netdev_class dpdk_vhost_client_class = NETDEV_DPDK_CLASS( "dpdkvhostuserclient", - netdev_dpdk_vhost_class_init, + NULL, netdev_dpdk_vhost_client_construct, netdev_dpdk_vhost_destruct, netdev_dpdk_vhost_client_set_config, diff --git a/rhel/openvswitch-fedora.spec.in b/rhel/openvswitch-fedora.spec.in index 3a045d3..2bb7102 100644 --- a/rhel/openvswitch-fedora.spec.in +++ b/rhel/openvswitch-fedora.spec.in @@ -84,7 +84,7 @@ BuildRequires: libcap-ng libcap-ng-devel %endif %if %{with dpdk} BuildRequires: libpcap-devel numactl-devel -BuildRequires: dpdk-devel >= 16.11 +BuildRequires: dpdk-devel >= 17.05.1 Provides: %{name}-dpdk = %{version}-%{release} %endif diff --git a/tests/dpdk/ring_client.c b/tests/dpdk/ring_client.c index b153713..8cc3fb5 100644 --- a/tests/dpdk/ring_client.c +++ b/tests/dpdk/ring_client.c @@ -185,15 +185,15 @@ main(int argc, char *argv[]) /* Try dequeuing max possible packets first, if that fails, get the * most we can. Loop body should only execute once, maximum. */ - while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, rx_pkts) != 0) && - rx_pkts > 0) { + while (unlikely(rte_ring_dequeue_bulk(rx_ring, pkts, + rx_pkts, NULL) != 0) && rx_pkts > 0) { rx_pkts = (uint16_t)RTE_MIN(rte_ring_count(rx_ring), PKT_READ_SIZE); } if (rx_pkts > 0) { /* blocking enqueue */ do { - rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts); + rslt = rte_ring_enqueue_bulk(tx_ring, pkts, rx_pkts, NULL); } while (rslt == -ENOBUFS); } }