| Message ID | 20250109122914.3534966-1-maxime.coquelin@redhat.com |
|---|---|
| State | Accepted |
| Commit | a24413cd3e36190ae4690315ceb24dfe6a37af1e |
| Delegated to: | Kevin Traynor |
| Headers | show |
| Series | [ovs-dev,v6] netdev-dpdk: Set Vhost port maximum number of queue pairs. | expand |
| Context | Check | Description |
|---|---|---|
| ovsrobot/apply-robot | success | apply and check: success |
| ovsrobot/github-robot-_Build_and_Test | success | github build: passed |
On 09/01/2025 12:29, Maxime Coquelin wrote: > This patch uses the new rte_vhost_driver_set_max_queue_num > API to set the maximum number of queue pairs supported by > the Vhost-user port. > > This is required for VDUSE which needs to specify the > maximum number of queue pairs at creation time. Without it > 128 queue pairs metadata would be allocated. > > To configure it, a new 'vhost-max-queue-pairs' option is > introduced. > > Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com> > Acked-by: Eelco Chaudron <echaudro@redhat.com> > --- > Documentation/topics/dpdk/vhost-user.rst | 15 +++++++++++ > lib/netdev-dpdk.c | 34 ++++++++++++++++++++++++ > vswitchd/vswitch.xml | 13 +++++++++ > 3 files changed, 62 insertions(+) > LGTM. There was a few nits on comment style from Ilya that I fixed on applying locally [1]. If no further comments, I will push tomorrow. Acked-by: Kevin Traynor <ktraynor@redhat.com> [1] https://github.com/kevintraynor/ovs/commit/a8ae0fce70f88d3d8a47225450ae29cb086fcf38
On 1/9/25 3:27 PM, Kevin Traynor wrote: > On 09/01/2025 12:29, Maxime Coquelin wrote: >> This patch uses the new rte_vhost_driver_set_max_queue_num >> API to set the maximum number of queue pairs supported by >> the Vhost-user port. >> >> This is required for VDUSE which needs to specify the >> maximum number of queue pairs at creation time. Without it >> 128 queue pairs metadata would be allocated. >> >> To configure it, a new 'vhost-max-queue-pairs' option is >> introduced. >> >> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com> >> Acked-by: Eelco Chaudron <echaudro@redhat.com> >> --- >> Documentation/topics/dpdk/vhost-user.rst | 15 +++++++++++ >> lib/netdev-dpdk.c | 34 ++++++++++++++++++++++++ >> vswitchd/vswitch.xml | 13 +++++++++ >> 3 files changed, 62 insertions(+) >> > > LGTM. There was a few nits on comment style from Ilya that I fixed on > applying locally [1]. If no further comments, I will push tomorrow. > > Acked-by: Kevin Traynor <ktraynor@redhat.com> I missed the other comments, thanks for handling them. I sent the DPDK patch to only emit the log with VDUSE backends: https://patches.dpdk.org/project/dpdk/patch/20250109143130.3696613-1-maxime.coquelin@redhat.com/ > [1] > https://github.com/kevintraynor/ovs/commit/a8ae0fce70f88d3d8a47225450ae29cb086fcf38 > >
On 09/01/2025 14:33, Maxime Coquelin wrote: > > > On 1/9/25 3:27 PM, Kevin Traynor wrote: >> On 09/01/2025 12:29, Maxime Coquelin wrote: >>> This patch uses the new rte_vhost_driver_set_max_queue_num >>> API to set the maximum number of queue pairs supported by >>> the Vhost-user port. >>> >>> This is required for VDUSE which needs to specify the >>> maximum number of queue pairs at creation time. Without it >>> 128 queue pairs metadata would be allocated. >>> >>> To configure it, a new 'vhost-max-queue-pairs' option is >>> introduced. >>> >>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com> >>> Acked-by: Eelco Chaudron <echaudro@redhat.com> >>> --- >>> Documentation/topics/dpdk/vhost-user.rst | 15 +++++++++++ >>> lib/netdev-dpdk.c | 34 ++++++++++++++++++++++++ >>> vswitchd/vswitch.xml | 13 +++++++++ >>> 3 files changed, 62 insertions(+) >>> >> >> LGTM. There was a few nits on comment style from Ilya that I fixed on >> applying locally [1]. If no further comments, I will push tomorrow. >> >> Acked-by: Kevin Traynor <ktraynor@redhat.com> > > I missed the other comments, thanks for handling them. > > I sent the DPDK patch to only emit the log with VDUSE backends: > https://patches.dpdk.org/project/dpdk/patch/20250109143130.3696613-1-maxime.coquelin@redhat.com/ > >> [1] >> https://github.com/kevintraynor/ovs/commit/a8ae0fce70f88d3d8a47225450ae29cb086fcf38 >> >> > Thanks Maxime, Eelco and Ilya. Pushed to main branch.
diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst index 3c02738cf..20e8eb245 100644 --- a/Documentation/topics/dpdk/vhost-user.rst +++ b/Documentation/topics/dpdk/vhost-user.rst @@ -375,6 +375,21 @@ Tx retries max can be set for vhost-user-client ports:: Configurable vhost tx retries are not supported with vhost-user ports. +vhost-user-client max queue pairs config +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For vhost-user-client interfaces using the VDUSE backend, the maximum number of +queue pairs the Virtio device will support can be set at port creation time. If +not set, the default value is 1 queue pair. This value is ignored for +Vhost-user backends. + +Maximum number of queue pairs can be set for vhost-user-client-ports:: + + $ ovs-vsctl add-port br0 vduse0 \ + -- set Interface vduse0 type=dpdkvhostuserclient \ + options:vhost-server-path=/dev/vduse/vduse0 \ + options:vhost-max-queue-pairs=4 + .. _dpdk-testpmd: DPDK in the Guest diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 449c660a7..d38c9d266 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -19,6 +19,7 @@ #include <errno.h> #include <signal.h> +#include <stdint.h> #include <stdlib.h> #include <string.h> #include <unistd.h> @@ -153,6 +154,11 @@ typedef uint16_t dpdk_port_t; /* Legacy default value for vhost tx retries. */ #define VHOST_ENQ_RETRY_DEF 8 +/* VDUSE-only, ignore for Vhost-user */ +#define VHOST_MAX_QUEUE_PAIRS_MIN 1 +#define VHOST_MAX_QUEUE_PAIRS_DEF VHOST_MAX_QUEUE_PAIRS_MIN +#define VHOST_MAX_QUEUE_PAIRS_MAX 128 + #define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ) /* List of required flags advertised by the hardware that will be used @@ -554,6 +560,9 @@ struct netdev_dpdk { /* Socket ID detected when vHost device is brought up */ int requested_socket_id; + /* Ignored by DPDK for Vhost-user backends, only for VDUSE */ + uint8_t vhost_max_queue_pairs; + /* Denotes whether vHost port is client/server mode */ uint64_t vhost_driver_flags; @@ -1606,6 +1615,8 @@ vhost_common_construct(struct netdev *netdev) atomic_init(&dev->vhost_tx_retries_max, VHOST_ENQ_RETRY_DEF); + dev->vhost_max_queue_pairs = VHOST_MAX_QUEUE_PAIRS_DEF; + return common_construct(netdev, DPDK_ETH_PORT_ID_INVALID, DPDK_DEV_VHOST, socket_id); } @@ -2488,6 +2499,7 @@ netdev_dpdk_vhost_client_set_config(struct netdev *netdev, struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); const char *path; int max_tx_retries, cur_max_tx_retries; + uint32_t max_queue_pairs; ovs_mutex_lock(&dev->mutex); if (!(dev->vhost_driver_flags & RTE_VHOST_USER_CLIENT)) { @@ -2495,6 +2507,15 @@ netdev_dpdk_vhost_client_set_config(struct netdev *netdev, if (!nullable_string_is_equal(path, dev->vhost_id)) { free(dev->vhost_id); dev->vhost_id = nullable_xstrdup(path); + + max_queue_pairs = smap_get_int(args, "vhost-max-queue-pairs", + VHOST_MAX_QUEUE_PAIRS_DEF); + if (max_queue_pairs < VHOST_MAX_QUEUE_PAIRS_MIN + || max_queue_pairs > VHOST_MAX_QUEUE_PAIRS_MAX) { + max_queue_pairs = VHOST_MAX_QUEUE_PAIRS_DEF; + } + dev->vhost_max_queue_pairs = max_queue_pairs; + netdev_request_reconfigure(netdev); } } @@ -2511,6 +2532,7 @@ netdev_dpdk_vhost_client_set_config(struct netdev *netdev, VLOG_INFO("Max Tx retries for vhost device '%s' set to %d", netdev_get_name(netdev), max_tx_retries); } + ovs_mutex_unlock(&dev->mutex); return 0; @@ -6397,6 +6419,18 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev) goto unlock; } + /* Setting max queue pairs is only useful and effective with VDUSE. */ + if (strncmp(dev->vhost_id, "/dev/vduse/", 11) == 0) { + uint32_t max_qp = dev->vhost_max_queue_pairs; + + err = rte_vhost_driver_set_max_queue_num(dev->vhost_id, max_qp); + if (err) { + VLOG_ERR("rte_vhost_driver_set_max_queue_num failed for " + "vhost-user client port: %s\n", dev->up.name); + goto unlock; + } + } + err = rte_vhost_driver_start(dev->vhost_id); if (err) { VLOG_ERR("rte_vhost_driver_start failed for vhost user " diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index 275bcbec0..2cb503c37 100644 --- a/vswitchd/vswitch.xml +++ b/vswitchd/vswitch.xml @@ -3540,6 +3540,19 @@ ovs-vsctl add-port br0 p0 -- set Interface p0 type=patch options:peer=p1 \ </p> </column> + <column name="options" key="vhost-max-queue-pairs" + type='{"type": "integer", "minInteger" : 1, "maxInteger": 128}'> + <p> + The value specifies the maximum number of queue pairs supported by + a vHost device. This is ignored for Vhost-user backends, only VDUSE + is supported. + Only supported by dpdkvhostuserclient interfaces. + </p> + <p> + Default value is 1. + </p> + </column> + <column name="options" key="tx-retries-max" type='{"type": "integer", "minInteger": 0, "maxInteger": 32}'> <p>