mbox series

[ovs-dev,RFC,dpdk-latest,v4,0/1] netdev-dpdk: Enable vhost async API's.

Message ID 20220321173640.326795-1-sunil.pai.g@intel.com
Headers show
Series netdev-dpdk: Enable vhost async API's. | expand

Message

Pai G, Sunil March 21, 2022, 5:36 p.m. UTC 
This series brings in the new asynchronous vHost API's in DPDK to OVS.
With the asynchronous framework, vHost-user can offload the memory copy operation
to DPDK DMA-dev based enabled devices without blocking the CPU.

The intention of this patch is solely to communicate the design of the feature,
evaluate the code impact and to get early feedback from the community.

Usage:
To enable DMA offload for vhost path, one can issue the following command:
ovs-vsctl --no-wait set Open_vSwitch . other_config:vhost-async-support=true

Issuing this requires restarting the vswitchd daemon.

Please note that, its required to have DMA devices visible to DPDK before OVS launch either
by binding them to userspace driver vfio-pci or through other facilities as mentioned in the driver documentation.

The current assignment model is 1 DMA device per data plane thread.
If no DMA devices are found for that thread, it will fall back to perform CPU copy.

To check if async support is enabled, one can check for the following message in the ovs-vswitchd.log
dpdk|INFO|Async support enabled for vhost-user-client.

To verify if DMA devices are assigned to the data plane thread, one can check the ovs-vswitchd.log for messages like:
netdev_dpdk(pmd-c01/id:7)|INFO|DMA device with dev id: 0 assigned to pmd for vhost async copy offload.


This patch was tested on :
OVS : branch  : dpdk-latest
with
DPDK : branch : main,
commit id : 80c5b4d635 ,  version: 22.03
+ https://patches.dpdk.org/project/dpdk/list/?series=22093  : v3 RFC of : vhost: support async dequeue data path
+ https://patches.dpdk.org/project/dpdk/list/?series=22119  : v2 RFC of : vhost: support clear in-flight packets for async dequeue.
+ https://patches.dpdk.org/project/dpdk/patch/20220216070417.9597-2-xuan.ding@intel.com/ :  v1 RFC of : vhost: add unsafe API to check inflight packets

Please note:
	- All the DPDK API's (both vhost async and DMADEV) utilized in the patch are experimental.
	- Its required to have atleast one vhost Rxq on every single core since currently even the vhost Tx packets are polled for completions from vhost Rx context.

Future work:
	- Compatibility with PMD auto load balancing and solve requirement of vhost-rxq on every PMD-thread


V3-> V4:
-----------
	- Based on community feedback, use lockless ring(rte_ring) to avoid vhost tx contention among data plane threads.
	- Drain packets from Rxq before the pmd reload.
	- Clear packets from a Rxq/txq on queue disable.
	- ovs-appctl netdev-dpdk/get-dmadev-info : to get the DMA stats and info.


V2-> V3:
----------
	- Update code according to new architecture with dmadev datapath in vhost library.
		○ The dmadev datapath is now moved to the vhost library and OVS is now only responsible to configure/use the DMA device.
        ○ This allows OVS the flexibility to decide on the allocation scheme of DMA devices.




Sunil Pai G (1):
  netdev-dpdk: Enable vhost async API's in OvS.

 lib/dpdk-stub.c   |   6 +
 lib/dpdk.c        |  33 ++
 lib/dpdk.h        |   1 +
 lib/dpif-netdev.c |   7 +
 lib/netdev-dpdk.c | 795 ++++++++++++++++++++++++++++++++++++++++++++--
 lib/netdev-dpdk.h |  14 +
 6 files changed, 829 insertions(+), 27 deletions(-)