| Message ID | 1468592131-8790-1-git-send-email-mauricio.vasquezbernal@studenti.polito.it |
|---|---|
| State | Superseded |
| Delegated to: | Daniele Di Proietto |
| Headers | show |
Hi Mauricio, Could you please rebase this patch? It doesn't apply anymore. I will review ASAP. Thanks! Flavio On Fri, Jul 15, 2016 at 04:15:31PM +0200, Mauricio Vasquez B wrote: > In order to use dpdk ports in ovs they have to be bound to a DPDK > compatible driver before ovs is started. > > This patch adds the possibility to hotplug (or hot-unplug) a device > after ovs has been started. The implementation adds two appctl commands: > netdev-dpdk/port-attach and netdev-dpdk/port-detach > > After the user attaches a new device, it has to be added to a bridge > using the add-port command, similarly, before detaching a device, > it has to be removed using the del-port command. > > Signed-off-by: Mauricio Vasquez B <mauricio.vasquezbernal@studenti.polito.it> > --- > v7: > - rebase to master > v6: > - add explicit comment about supporting VFIO > - rebase to master > v5: > - use two appctl commands instead of a single one > - rebase to master > v4: > - fix typo in commit message > - remove unnecessary whitespace change in INSTALL.DPDK.md > v3: > - create dpdk_port_attach and dpdk_port_detach functions > - modify mutex locking order > v2: > - use rte_eth_dev_is_valid_port() to check if a port is valid > INSTALL.DPDK-ADVANCED.md | 25 ++++++++++++ > NEWS | 1 + > lib/netdev-dpdk.c | 101 +++++++++++++++++++++++++++++++++++++++++++---- > 3 files changed, 120 insertions(+), 7 deletions(-) >
Hi Flavio, I was thinking that instead of having a separate appctl we could integrate the attach into netdev_dpdk_construct() while changing the naming convention, as discussed here: http://openvswitch.org/pipermail/dev/2016-August/078113.html What do you think? Thanks, Daniele 2016-10-26 11:22 GMT-07:00 Flavio Leitner <fbl@sysclose.org>: > > Hi Mauricio, > > Could you please rebase this patch? It doesn't apply anymore. > I will review ASAP. > Thanks! > Flavio > > On Fri, Jul 15, 2016 at 04:15:31PM +0200, Mauricio Vasquez B wrote: > > In order to use dpdk ports in ovs they have to be bound to a DPDK > > compatible driver before ovs is started. > > > > This patch adds the possibility to hotplug (or hot-unplug) a device > > after ovs has been started. The implementation adds two appctl commands: > > netdev-dpdk/port-attach and netdev-dpdk/port-detach > > > > After the user attaches a new device, it has to be added to a bridge > > using the add-port command, similarly, before detaching a device, > > it has to be removed using the del-port command. > > > > Signed-off-by: Mauricio Vasquez B <mauricio.vasquezbernal@ > studenti.polito.it> > > --- > > v7: > > - rebase to master > > v6: > > - add explicit comment about supporting VFIO > > - rebase to master > > v5: > > - use two appctl commands instead of a single one > > - rebase to master > > v4: > > - fix typo in commit message > > - remove unnecessary whitespace change in INSTALL.DPDK.md > > v3: > > - create dpdk_port_attach and dpdk_port_detach functions > > - modify mutex locking order > > v2: > > - use rte_eth_dev_is_valid_port() to check if a port is valid > > INSTALL.DPDK-ADVANCED.md | 25 ++++++++++++ > > NEWS | 1 + > > lib/netdev-dpdk.c | 101 ++++++++++++++++++++++++++++++ > +++++++++++++---- > > 3 files changed, 120 insertions(+), 7 deletions(-) > > > > _______________________________________________ > dev mailing list > dev@openvswitch.org > http://openvswitch.org/mailman/listinfo/dev >
Hi Flavio, Daniele I think that the idea of integrating the patch in the naming convention is good. I only have two comments: - I would keep it limited to physical devices for the moment, maybe in a future we can think about supporting other device creation arguments. - How is the detach suppose to work?, should it be an appctl to detach or should we implement some logic two automatically detach the port when it is removed from ovs? Thanks, Mauricio On Wed, Oct 26, 2016 at 8:36 PM, Daniele Di Proietto <diproiettod@ovn.org> wrote: > Hi Flavio, > > I was thinking that instead of having a separate appctl we could integrate > the attach into netdev_dpdk_construct() while changing the naming > convention, as discussed here: > > http://openvswitch.org/pipermail/dev/2016-August/078113.html > > What do you think? > > Thanks, > > Daniele > > 2016-10-26 11:22 GMT-07:00 Flavio Leitner <fbl@sysclose.org>: > >> >> Hi Mauricio, >> >> Could you please rebase this patch? It doesn't apply anymore. >> I will review ASAP. >> Thanks! >> Flavio >> >> On Fri, Jul 15, 2016 at 04:15:31PM +0200, Mauricio Vasquez B wrote: >> > In order to use dpdk ports in ovs they have to be bound to a DPDK >> > compatible driver before ovs is started. >> > >> > This patch adds the possibility to hotplug (or hot-unplug) a device >> > after ovs has been started. The implementation adds two appctl commands: >> > netdev-dpdk/port-attach and netdev-dpdk/port-detach >> > >> > After the user attaches a new device, it has to be added to a bridge >> > using the add-port command, similarly, before detaching a device, >> > it has to be removed using the del-port command. >> > >> > Signed-off-by: Mauricio Vasquez B <mauricio.vasquezbernal@studen >> ti.polito.it> >> > --- >> > v7: >> > - rebase to master >> > v6: >> > - add explicit comment about supporting VFIO >> > - rebase to master >> > v5: >> > - use two appctl commands instead of a single one >> > - rebase to master >> > v4: >> > - fix typo in commit message >> > - remove unnecessary whitespace change in INSTALL.DPDK.md >> > v3: >> > - create dpdk_port_attach and dpdk_port_detach functions >> > - modify mutex locking order >> > v2: >> > - use rte_eth_dev_is_valid_port() to check if a port is valid >> > INSTALL.DPDK-ADVANCED.md | 25 ++++++++++++ >> > NEWS | 1 + >> > lib/netdev-dpdk.c | 101 ++++++++++++++++++++++++++++++ >> +++++++++++++---- >> > 3 files changed, 120 insertions(+), 7 deletions(-) >> > >> >> _______________________________________________ >> dev mailing list >> dev@openvswitch.org >> http://openvswitch.org/mailman/listinfo/dev >> > >
diff --git a/INSTALL.DPDK-ADVANCED.md b/INSTALL.DPDK-ADVANCED.md index 9ae536d..61b4e82 100644 --- a/INSTALL.DPDK-ADVANCED.md +++ b/INSTALL.DPDK-ADVANCED.md @@ -12,6 +12,7 @@ OVS DPDK ADVANCED INSTALL GUIDE 7. [QOS](#qos) 8. [Rate Limiting](#rl) 9. [Vsperf](#vsperf) +10. [Port Hotplug](#hotplug) ## <a name="overview"></a> 1. Overview @@ -835,6 +836,29 @@ environment. More information can be found in below link. https://wiki.opnfv.org/display/vsperf/VSperf+Home +## <a name="hotplug"></a> 10. Port Hotplug + +OvS supports port hotplugging, it allows to use ports that were not bound +to DPDK when vswitchd was started. +In order to attach a port, it has to be bound to DPDK using the +dpdk_nic_bind.py script: + +`$DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio 0000:01:00.0` + +Then it can be attached to OVS: + +`ovs-appctl netdev-dpdk/port-attach 0000:01:00.0` + +At this point, the user can create a ovs port using the add-port command. + +It is also possible to detach a port from ovs, the user has to remove the +port using the del-port command, then it can be detached using: + +`ovs-appctl netdev-dpdk/port-detach dpdk0` + +This feature is not supported with VFIO and could not work with some NICs, +please refer to the [DPDK Port Hotplug Framework] in order to get more +information. Bug Reporting: -------------- @@ -850,3 +874,4 @@ Please report problems to bugs@openvswitch.org. [Guest VM using libvirt]: INSTALL.DPDK.md#ovstc [INSTALL DPDK]: INSTALL.DPDK.md#build [INSTALL OVS]: INSTALL.DPDK.md#build +[DPDK Port Hotplug Framework]: http://dpdk.org/doc/guides/prog_guide/port_hotplug_framework.html diff --git a/NEWS b/NEWS index 6496dc1..9064225 100644 --- a/NEWS +++ b/NEWS @@ -58,6 +58,7 @@ Post-v2.5.0 * PMD threads servicing vHost User ports can now come from the NUMA node that device memory is located on if CONFIG_RTE_LIBRTE_VHOST_NUMA is enabled in DPDK. + * Port Hotplug is now supported. - Increase number of registers to 16. - ovs-benchmark: This utility has been removed due to lack of use and bitrot. diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 85b18fd..3fab52c 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -644,7 +644,7 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) OVS_REQUIRES(dpdk_mutex) int diag; int n_rxq, n_txq; - if (dev->port_id < 0 || dev->port_id >= rte_eth_dev_count()) { + if (!rte_eth_dev_is_valid_port(dev->port_id)) { return ENODEV; } @@ -2172,6 +2172,83 @@ netdev_dpdk_set_admin_state(struct unixctl_conn *conn, int argc, unixctl_command_reply(conn, "OK"); } +static void +netdev_dpdk_port_attach(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[], void *aux OVS_UNUSED) +{ + int ret; + char response[128]; + uint8_t port_id; + + ovs_mutex_lock(&dpdk_mutex); + + ret = rte_eth_dev_attach(argv[1], &port_id); + if (ret < 0) { + snprintf(response, sizeof(response), + "Error attaching device '%s'", argv[1]); + ovs_mutex_unlock(&dpdk_mutex); + unixctl_command_reply_error(conn, response); + return; + } + + snprintf(response, sizeof(response), + "Device '%s' has been attached as 'dpdk%d'", argv[1], port_id); + + ovs_mutex_unlock(&dpdk_mutex); + unixctl_command_reply(conn, response); +} + +static void +netdev_dpdk_port_detach(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[], void *aux OVS_UNUSED) +{ + int ret; + char response[128]; + unsigned int parsed_port; + uint8_t port_id; + char devname[RTE_ETH_NAME_MAX_LEN]; + + ovs_mutex_lock(&dpdk_mutex); + + ret = dpdk_dev_parse_name(argv[1], "dpdk", &parsed_port); + if (ret) { + snprintf(response, sizeof(response), + "'%s' is not a valid port", argv[1]); + goto error; + } + + port_id = parsed_port; + + struct netdev *netdev = netdev_from_name(argv[1]); + if (netdev) { + netdev_close(netdev); + snprintf(response, sizeof(response), + "Port '%s' is being used. Remove it before detaching", + argv[1]); + goto error; + } + + rte_eth_dev_close(port_id); + + ret = rte_eth_dev_detach(port_id, devname); + if (ret < 0) { + snprintf(response, sizeof(response), + "Port '%s' can not be detached", argv[1]); + goto error; + } + + snprintf(response, sizeof(response), + "Port '%s' has been detached", argv[1]); + + ovs_mutex_unlock(&dpdk_mutex); + unixctl_command_reply(conn, response); + return; + +error: + ovs_mutex_unlock(&dpdk_mutex); + unixctl_command_reply_error(conn, response); +} + /* * Set virtqueue flags so that we do not receive interrupts. */ @@ -2457,6 +2534,15 @@ dpdk_common_init(void) "[netdev] up|down", 1, 2, netdev_dpdk_set_admin_state, NULL); + unixctl_command_register("netdev-dpdk/port-attach", + "pci address of device", 1, 1, + netdev_dpdk_port_attach, NULL); + + unixctl_command_register("netdev-dpdk/port-detach", + "port", 1, 1, + netdev_dpdk_port_detach, NULL); + + ovs_thread_create("dpdk_watchdog", dpdk_watchdog, NULL); } /* Client Rings */ @@ -2467,7 +2553,7 @@ dpdk_ring_create(const char dev_name[], unsigned int port_no, { struct dpdk_ring *ivshmem; char ring_name[RTE_RING_NAMESIZE]; - int err; + int err, port_id; ivshmem = dpdk_rte_mzalloc(sizeof *ivshmem); if (ivshmem == NULL) { @@ -2501,19 +2587,20 @@ dpdk_ring_create(const char dev_name[], unsigned int port_no, return ENOMEM; } - err = rte_eth_from_rings(dev_name, &ivshmem->cring_rx, 1, - &ivshmem->cring_tx, 1, SOCKET0); + port_id = rte_eth_from_rings(dev_name, &ivshmem->cring_rx, 1, + &ivshmem->cring_tx, 1, SOCKET0); - if (err < 0) { + if (port_id < 0) { rte_free(ivshmem); return ENODEV; } ivshmem->user_port_id = port_no; - ivshmem->eth_port_id = rte_eth_dev_count() - 1; + ivshmem->eth_port_id = port_id; + *eth_port_id = port_id; + ovs_list_push_back(&dpdk_ring_list, &ivshmem->list_node); - *eth_port_id = ivshmem->eth_port_id; return 0; }
In order to use dpdk ports in ovs they have to be bound to a DPDK compatible driver before ovs is started. This patch adds the possibility to hotplug (or hot-unplug) a device after ovs has been started. The implementation adds two appctl commands: netdev-dpdk/port-attach and netdev-dpdk/port-detach After the user attaches a new device, it has to be added to a bridge using the add-port command, similarly, before detaching a device, it has to be removed using the del-port command. Signed-off-by: Mauricio Vasquez B <mauricio.vasquezbernal@studenti.polito.it> --- v7: - rebase to master v6: - add explicit comment about supporting VFIO - rebase to master v5: - use two appctl commands instead of a single one - rebase to master v4: - fix typo in commit message - remove unnecessary whitespace change in INSTALL.DPDK.md v3: - create dpdk_port_attach and dpdk_port_detach functions - modify mutex locking order v2: - use rte_eth_dev_is_valid_port() to check if a port is valid INSTALL.DPDK-ADVANCED.md | 25 ++++++++++++ NEWS | 1 + lib/netdev-dpdk.c | 101 +++++++++++++++++++++++++++++++++++++++++++---- 3 files changed, 120 insertions(+), 7 deletions(-)