From patchwork Fri Jul 29 13:29:24 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ciara Loftus X-Patchwork-Id: 654119 X-Patchwork-Delegate: diproiettod@vmware.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from archives.nicira.com (archives.nicira.com [96.126.127.54]) by ozlabs.org (Postfix) with ESMTP id 3s19B737NNz9ssM for ; Fri, 29 Jul 2016 23:51:27 +1000 (AEST) Received: from archives.nicira.com (localhost [127.0.0.1]) by archives.nicira.com (Postfix) with ESMTP id A5DC61151D; Fri, 29 Jul 2016 06:51:26 -0700 (PDT) X-Original-To: dev@openvswitch.org Delivered-To: dev@openvswitch.org Received: from mx1e3.cudamail.com (mx1.cudamail.com [69.90.118.67]) by archives.nicira.com (Postfix) with ESMTPS id BC7BB11518 for ; Fri, 29 Jul 2016 06:51:25 -0700 (PDT) Received: from bar5.cudamail.com (localhost [127.0.0.1]) by mx1e3.cudamail.com (Postfix) with ESMTPS id 594F64201C6 for ; Fri, 29 Jul 2016 07:51:25 -0600 (MDT) X-ASG-Debug-ID: 1469798966-09eadd7ae83e5320001-byXFYA Received: from mx1-pf2.cudamail.com ([192.168.24.2]) by bar5.cudamail.com with ESMTP id i8glKkhRVAhFQH4n (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 29 Jul 2016 07:29:26 -0600 (MDT) X-Barracuda-Envelope-From: cloftus@ecsmtp.ir.intel.com X-Barracuda-RBL-Trusted-Forwarder: 192.168.24.2 Received: from unknown (HELO mga14.intel.com) (192.55.52.115) by mx1-pf2.cudamail.com with SMTP; 29 Jul 2016 13:29:26 -0000 Received-SPF: none (mx1-pf2.cudamail.com: domain at ecsmtp.ir.intel.com does not designate permitted sender hosts) X-Barracuda-Apparent-Source-IP: 192.55.52.115 X-Barracuda-RBL-IP: 192.55.52.115 Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga103.fm.intel.com with ESMTP; 29 Jul 2016 06:29:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.28,439,1464678000"; d="scan'208";a="741492989" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by FMSMGA003.fm.intel.com with ESMTP; 29 Jul 2016 06:29:25 -0700 Received: from sivswdev01.ir.intel.com (sivswdev01.ir.intel.com [10.237.217.45]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id u6TDTOJS010168 for ; Fri, 29 Jul 2016 14:29:24 +0100 Received: from sivswdev01.ir.intel.com (localhost [127.0.0.1]) by sivswdev01.ir.intel.com with ESMTP id u6TDTOAc002310 for ; Fri, 29 Jul 2016 14:29:24 +0100 Received: (from cloftus@localhost) by sivswdev01.ir.intel.com with id u6TDTOgY002305 for dev@openvswitch.org; Fri, 29 Jul 2016 14:29:24 +0100 X-CudaMail-Envelope-Sender: cloftus@ecsmtp.ir.intel.com From: Ciara Loftus To: dev@openvswitch.org X-CudaMail-MID: CM-E2-728014770 X-CudaMail-DTE: 072916 X-CudaMail-Originating-IP: 192.55.52.115 Date: Fri, 29 Jul 2016 14:29:24 +0100 X-ASG-Orig-Subj: [##CM-E2-728014770##][PATCH RFC v2 1/1] netdev-dpdk: vHost client mode and reconnect Message-Id: <1469798964-2261-2-git-send-email-ciara.loftus@intel.com> X-Mailer: git-send-email 1.7.4.1 In-Reply-To: <1469798964-2261-1-git-send-email-ciara.loftus@intel.com> References: <1469798964-2261-1-git-send-email-ciara.loftus@intel.com> X-GBUdb-Analysis: 0, 192.55.52.115, Ugly c=0 p=-1 Source Normal X-MessageSniffer-Rules: 0-0-0-14764-c X-Barracuda-Connect: UNKNOWN[192.168.24.2] X-Barracuda-Start-Time: 1469798966 X-Barracuda-Encrypted: DHE-RSA-AES256-SHA X-Barracuda-URL: https://web.cudamail.com:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at cudamail.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.60 X-Barracuda-Spam-Status: No, SCORE=0.60 using global scores of TAG_LEVEL=3.5 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=4.0 tests=BSF_SC5_MJ1963, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.31604 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 Subject: [ovs-dev] [PATCH RFC v2 1/1] netdev-dpdk: vHost client mode and reconnect X-BeenThere: dev@openvswitch.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dev-bounces@openvswitch.org Sender: "dev" A new other_config DB option has been added called 'vhost-driver-mode'. By default this is set to 'server' which is the mode of operation OVS with DPDK has used up until this point - whereby OVS creates and manages vHost user sockets. If set to 'client', OVS will act as the vHost client and connect to sockets created and managed by QEMU which acts as the server. This mode allows for reconnect capability, which allows vHost ports to resume normal connectivity in event of switch reset. QEMU v2.7.0+ is required when using OVS in client mode and QEMU in server mode. Signed-off-by: Ciara Loftus --- INSTALL.DPDK-ADVANCED.md | 27 +++++++++++++++++++++++++++ NEWS | 1 + lib/netdev-dpdk.c | 27 +++++++++++++++++++++------ vswitchd/vswitch.xml | 13 +++++++++++++ 4 files changed, 62 insertions(+), 6 deletions(-) diff --git a/INSTALL.DPDK-ADVANCED.md b/INSTALL.DPDK-ADVANCED.md index ca9e9bb..29eb78f 100644 --- a/INSTALL.DPDK-ADVANCED.md +++ b/INSTALL.DPDK-ADVANCED.md @@ -483,6 +483,33 @@ For users wanting to do packet forwarding using kernel stack below are the steps where `-L`: Changes the numbers of channels of the specified network device and `combined`: Changes the number of multi-purpose channels. + 4. Enable OVS vHost client-mode & vHost reconnect (OPTIONAL) + + By default, OVS DPDK acts as the vHost socket server and QEMU the + client. In QEMU v2.7 the option is available for QEMU to act as the + server. In order for this to work, OVS DPDK must be switched to 'client' + mode. This is possible by setting the 'vhost-driver-mode' DB entry to + 'client' like so: + + ``` + ovs-vsctl set Open_vSwitch . other_config:vhost-driver-mode="client" + ``` + + This must be done before the switch is launched. It cannot sucessfully + be changed after switch has launched. + + One must also append ',server' to the 'chardev' arguments on the QEMU + command line, to instruct QEMU to use vHost server mode, like so: + + ```` + -chardev socket,id=char0,path=/usr/local/var/run/openvswitch/vhost0,server + ```` + + One benefit of using this mode is the ability for vHost ports to + 'reconnect' in event of the switch crashing or being brought down. Once + it is brought back up, the vHost ports will reconnect automatically and + normal service will resume. + - VM Configuration with libvirt * change the user/group, access control policty and restart libvirtd. diff --git a/NEWS b/NEWS index 248d856..59f2dfb 100644 --- a/NEWS +++ b/NEWS @@ -67,6 +67,7 @@ Post-v2.5.0 * vHost PMD integration brings vhost-user ports under control of the rte_ether DPDK API. * Support for DPDK 16.07 + * OVS client mode for vHost and vHost reconnect (Requires QEMU 2.7) - Increase number of registers to 16. - ovs-benchmark: This utility has been removed due to lack of use and bitrot. diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 07e6045..dabd631 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -137,6 +137,7 @@ BUILD_ASSERT_DECL((MAX_NB_MBUF / ROUND_DOWN_POW2(MAX_NB_MBUF/MIN_NB_MBUF)) * yet mapped to another queue. */ static char *vhost_sock_dir = NULL; /* Location of vhost-user sockets */ +static int client_mode = 0; /* Denote whether client/server mode */ #define VHOST_ENQ_RETRY_NUM 8 #define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ) @@ -972,9 +973,10 @@ netdev_dpdk_vhost_construct(struct netdev *netdev) err = ENODEV; goto out; } else { - devargs = xasprintf("eth_vhost%u,iface=%s,queues=%i", + devargs = xasprintf("eth_vhost%u,iface=%s,queues=%i,client=%i", driver_id, dev->vhost_id, - MIN(OVS_VHOST_MAX_QUEUE_NUM, RTE_MAX_QUEUES_PER_PORT)); + MIN(OVS_VHOST_MAX_QUEUE_NUM, RTE_MAX_QUEUES_PER_PORT), + client_mode); err = rte_eth_dev_attach(devargs, &port_no); } @@ -982,9 +984,11 @@ netdev_dpdk_vhost_construct(struct netdev *netdev) VLOG_ERR("Failed to attach vhost-user device %s to DPDK", dev->vhost_id); } else { - fatal_signal_add_file_to_unlink(dev->vhost_id); - VLOG_INFO("Socket %s created for vhost-user port %s\n", - dev->vhost_id, name); + if (!client_mode) { + fatal_signal_add_file_to_unlink(dev->vhost_id); + VLOG_INFO("Socket %s created for vhost-user port %s\n", + dev->vhost_id, name); + } dev->vhost_pmd_id = driver_id; set_vhost_drv_id(driver_id, 1); err = netdev_dpdk_init(netdev, port_no, DPDK_DEV_VHOST); @@ -1053,7 +1057,8 @@ netdev_dpdk_vhost_destruct(struct netdev *netdev) if (rte_eth_dev_detach(dev->port_id, dev->vhost_id)) { VLOG_ERR("Error removing vhost device %s", dev->vhost_id); - } else { + } else if (!client_mode) { + /* OVS server mode - OVS may delete the socket */ fatal_signal_remove_file_to_unlink(dev->vhost_id); } set_vhost_drv_id(dev->vhost_pmd_id, 0); @@ -2861,6 +2866,7 @@ dpdk_init__(const struct smap *ovs_other_config) int err = 0; cpu_set_t cpuset; char *sock_dir_subcomponent; + const char *val; if (!smap_get_bool(ovs_other_config, "dpdk-init", false)) { VLOG_INFO("DPDK Disabled - to change this requires a restart.\n"); @@ -2891,6 +2897,15 @@ dpdk_init__(const struct smap *ovs_other_config) } else { vhost_sock_dir = sock_dir_subcomponent; } + val = smap_get(ovs_other_config, "vhost-driver-mode"); + if (val != NULL && strncmp(val, "client", strlen(val)) == 0) { + client_mode = 1; + VLOG_INFO("OVS client mode (QEMU server mode) selected for vHost"); + } else { + /* default case */ + client_mode = 0; + VLOG_INFO("OVS server mode (QEMU client mode) selected for vHost"); + } argv = grow_argv(&argv, 0, 1); argc = 1; diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index 03f3638..e7954c3 100644 --- a/vswitchd/vswitch.xml +++ b/vswitchd/vswitch.xml @@ -299,6 +299,19 @@

+ +

+ Specifies which mode OVS will use for vHost. In 'server' mode, OVS + creates and destroys the vHost User sockets. In 'client' mode, OVS + attaches to sockets created by QEMU. +

+

+ Defaults to 'server' mode. Changing this value requires restarting + the daemon. +

+ +