From patchwork Thu May 3 19:08:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aaron Conole X-Patchwork-Id: 908183 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40cPn63Wkmz9s2t for ; Fri, 4 May 2018 05:08:26 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id A81ECC04; Thu, 3 May 2018 19:08:10 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 4908C95E for ; Thu, 3 May 2018 19:08:08 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id C7FC04FA for ; Thu, 3 May 2018 19:08:07 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 05437F95CD; Thu, 3 May 2018 19:08:07 +0000 (UTC) Received: from dhcp-25.97.bos.redhat.com (ovpn-122-198.rdu2.redhat.com [10.10.122.198]) by smtp.corp.redhat.com (Postfix) with ESMTP id 25F29215CDA7; Thu, 3 May 2018 19:08:06 +0000 (UTC) From: Aaron Conole To: dev@openvswitch.org Date: Thu, 3 May 2018 15:08:00 -0400 Message-Id: <20180503190801.20925-2-aconole@redhat.com> In-Reply-To: <20180503190801.20925-1-aconole@redhat.com> References: <20180503190801.20925-1-aconole@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Thu, 03 May 2018 19:08:07 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Thu, 03 May 2018 19:08:07 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'aconole@redhat.com' RCPT:'' X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Flavio Leitner , Ilya Maximets Subject: [ovs-dev] [PATCH 1/2] dpdk: allow init to fail X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org It's possible for dpdk initialization to fail either due to an internal error or an invalid configuration. When that happens, it's rather impolite to immediately abort without any details. With this change, a failed dpdk initialization attempt will continue to trigger a SIGABRT. However, the failure details will be logged, and a user or administrator may have more information to correct the issue. A restart of OvS would still be required to re-attempt initialization. The refactor to propagate the init error will be used in an upcoming commit. Signed-off-by: Aaron Conole Acked-by: Kevin Traynor --- lib/dpdk.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/lib/dpdk.c b/lib/dpdk.c index 00dd97470..8bb686c43 100644 --- a/lib/dpdk.c +++ b/lib/dpdk.c @@ -22,6 +22,7 @@ #include #include +#include #include #include #include @@ -306,7 +307,7 @@ static cookie_io_functions_t dpdk_log_func = { .write = dpdk_log_write, }; -static void +static bool dpdk_init__(const struct smap *ovs_other_config) { char **argv = NULL, **argv_to_release = NULL; @@ -422,9 +423,6 @@ dpdk_init__(const struct smap *ovs_other_config) /* Make sure things are initialized ... */ result = rte_eal_init(argc, argv); - if (result < 0) { - ovs_abort(result, "Cannot init EAL"); - } argv_release(argv, argv_to_release, argc); /* Set the main thread affinity back to pre rte_eal_init() value */ @@ -436,6 +434,11 @@ dpdk_init__(const struct smap *ovs_other_config) } } + if (result < 0) { + VLOG_EMER("Unable to initialize DPDK: %s", ovs_strerror(rte_errno)); + return false; + } + rte_memzone_dump(stdout); /* We are called from the main thread here */ @@ -459,6 +462,7 @@ dpdk_init__(const struct smap *ovs_other_config) /* Finally, register the dpdk classes */ netdev_dpdk_register(); + return true; } void @@ -476,10 +480,15 @@ dpdk_init(const struct smap *ovs_other_config) if (ovsthread_once_start(&once_enable)) { VLOG_INFO("Using %s", rte_version()); VLOG_INFO("DPDK Enabled - initializing..."); - dpdk_init__(ovs_other_config); - enabled = true; - VLOG_INFO("DPDK Enabled - initialized"); + enabled = dpdk_init__(ovs_other_config); + if (enabled) { + VLOG_INFO("DPDK Enabled - initialized"); + } else { + ovs_abort(rte_errno, "Cannot init EAL"); + } ovsthread_once_done(&once_enable); + } else { + VLOG_ERR_ONCE("DPDK Initialization Failed."); } } else { VLOG_INFO_ONCE("DPDK Disabled - Use other_config:dpdk-init to enable"); From patchwork Thu May 3 19:08:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aaron Conole X-Patchwork-Id: 908184 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40cPnv0cPcz9s3G for ; Fri, 4 May 2018 05:09:07 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id AADF9C19; Thu, 3 May 2018 19:08:11 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 279B5BF8 for ; Thu, 3 May 2018 19:08:10 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 39A1E299 for ; Thu, 3 May 2018 19:08:09 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 72003402610C; Thu, 3 May 2018 19:08:08 +0000 (UTC) Received: from dhcp-25.97.bos.redhat.com (ovpn-122-198.rdu2.redhat.com [10.10.122.198]) by smtp.corp.redhat.com (Postfix) with ESMTP id 25560215CDA7; Thu, 3 May 2018 19:08:07 +0000 (UTC) From: Aaron Conole To: dev@openvswitch.org Date: Thu, 3 May 2018 15:08:01 -0400 Message-Id: <20180503190801.20925-3-aconole@redhat.com> In-Reply-To: <20180503190801.20925-1-aconole@redhat.com> References: <20180503190801.20925-1-aconole@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 03 May 2018 19:08:08 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 03 May 2018 19:08:08 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'aconole@redhat.com' RCPT:'' X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Flavio Leitner , Ilya Maximets Subject: [ovs-dev] [PATCH 2/2] dpdk: reflect status and version in the database X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org The normal way of retrieving the running DPDK status involves parsing log files and issuing various incantations of ovs-vsctl and ovs-appctl commands to determine whether the rte_eal_init successfully started. This commit adds two new records to reflect the dpdk version, and the dpdk initialization status. To support this, the other_config:dpdk-init configuration block supports the 'true' and 'try' keywords now, instead of just 'true'. Signed-off-by: Aaron Conole Acked-by: Kevin Traynor --- Documentation/faq/configuration.rst | 8 +++++--- Documentation/intro/install/dpdk.rst | 27 ++++++++++++++++++++++++--- lib/dpdk-stub.c | 10 ++++++++++ lib/dpdk.c | 21 +++++++++++++++++++-- lib/dpdk.h | 3 ++- vswitchd/bridge.c | 5 +++++ vswitchd/vswitch.ovsschema | 11 ++++++++--- vswitchd/vswitch.xml | 11 +++++++++++ 8 files changed, 84 insertions(+), 12 deletions(-) diff --git a/Documentation/faq/configuration.rst b/Documentation/faq/configuration.rst index 1c93a55cc..0213c58dd 100644 --- a/Documentation/faq/configuration.rst +++ b/Documentation/faq/configuration.rst @@ -102,9 +102,11 @@ Q: How do I configure a DPDK port as an access port? A: Firstly, you must have a DPDK-enabled version of Open vSwitch. - If your version is DPDK-enabled it will support the other-config:dpdk-init - configuration in the database and will display lines with "EAL:..." during - startup when other_config:dpdk-init is set to 'true'. + If your version is DPDK-enabled it may support the dpdk_version and + dpdk_initialized keys in the configuration database. Earlier versions + of Open vSwitch only supported the other-config:dpdk-init key in the + configuration in the database. All versions will display lines with + "EAL:..." during startup when other_config:dpdk-init is set to 'true'. Secondly, when adding a DPDK port, unlike a system port, the type for the interface and valid dpdk-devargs must be specified. For example:: diff --git a/Documentation/intro/install/dpdk.rst b/Documentation/intro/install/dpdk.rst index fea48908d..8fb0c4163 100644 --- a/Documentation/intro/install/dpdk.rst +++ b/Documentation/intro/install/dpdk.rst @@ -208,7 +208,8 @@ Open vSwitch should be started as described in :doc:`general` with the exception of ovs-vswitchd, which requires some special configuration to enable DPDK functionality. DPDK configuration arguments can be passed to ovs-vswitchd via the ``other_config`` column of the ``Open_vSwitch`` table. At a minimum, -the ``dpdk-init`` option must be set to ``true``. For example:: +the ``dpdk-init`` option must be set to either ``true`` or ``try``. +For example:: $ export PATH=$PATH:/usr/local/share/openvswitch/scripts $ export DB_SOCK=/usr/local/var/run/openvswitch/db.sock @@ -219,8 +220,12 @@ There are many other configuration options, the most important of which are listed below. Defaults will be provided for all values not explicitly set. ``dpdk-init`` - Specifies whether OVS should initialize and support DPDK ports. This is a - boolean, and defaults to false. + Specifies whether OVS should initialize and support DPDK ports. This field + can either be ``true`` or ``try``. + A value of ``true`` will cause the ovs-vswitchd process to abort on + initialization failure. + A value of ``try`` will imply that the ovs-vswitchd process should + continue running even if the EAL initialization fails. ``dpdk-lcore-mask`` Specifies the CPU cores on which dpdk lcore threads should be spawned and @@ -257,6 +262,22 @@ See the section ``Performance Tuning`` for important DPDK customizations. Validating ---------- +DPDK support can be confirmed by validating the ``dpdk_initialized`` boolean +value from the ovsdb. A value of ``true`` means that the DPDK EAL +initialization succeeded:: + + $ ovs-vsctl get Open_vSwitch . dpdk_initialized + true + +Additionally, the library version linked to ovs-vswitchd can be confirmed +with either the ovs-vswitchd logs, or by running either of the commands:: + + $ ovs-vswitchd --version + ovs-vswitchd (Open vSwitch) 2.9.0 + DPDK 17.11.0 + $ ovs-vsctl get Open_vSwitch . dpdk_version + "DPDK 17.11.0" + At this point you can use ovs-vsctl to set up bridges and other Open vSwitch features. Seeing as we've configured the DPDK datapath, we will use DPDK-type ports. For example, to create a userspace bridge named ``br0`` and add two diff --git a/lib/dpdk-stub.c b/lib/dpdk-stub.c index 041cd0cbb..1df1c5848 100644 --- a/lib/dpdk-stub.c +++ b/lib/dpdk-stub.c @@ -21,6 +21,7 @@ #include "smap.h" #include "ovs-thread.h" #include "openvswitch/vlog.h" +#include "vswitch-idl.h" VLOG_DEFINE_THIS_MODULE(dpdk); @@ -59,3 +60,12 @@ void print_dpdk_version(void) { } + +void +dpdk_status(const struct ovsrec_open_vswitch *cfg) +{ + if (cfg) { + ovsrec_open_vswitch_set_dpdk_initialized(cfg, false); + ovsrec_open_vswitch_set_dpdk_version(cfg, "none"); + } +} diff --git a/lib/dpdk.c b/lib/dpdk.c index 8bb686c43..09afd8cc2 100644 --- a/lib/dpdk.c +++ b/lib/dpdk.c @@ -37,6 +37,7 @@ #include "openvswitch/dynamic-string.h" #include "openvswitch/vlog.h" #include "smap.h" +#include "vswitch-idl.h" VLOG_DEFINE_THIS_MODULE(dpdk); @@ -44,6 +45,8 @@ static FILE *log_stream = NULL; /* Stream for DPDK log redirection */ static char *vhost_sock_dir = NULL; /* Location of vhost-user sockets */ static bool vhost_iommu_enabled = false; /* Status of vHost IOMMU support */ +static bool dpdk_initialized = false; /* Indicates successful initialization + * of DPDK. */ static int process_vhost_flags(char *flag, const char *default_val, int size, @@ -474,7 +477,11 @@ dpdk_init(const struct smap *ovs_other_config) return; } - if (smap_get_bool(ovs_other_config, "dpdk-init", false)) { + const char *dpdk_init_val = smap_get_def(ovs_other_config, "dpdk-init", + "false"); + + bool try_only = !strcmp(dpdk_init_val, "try"); + if (!strcmp(dpdk_init_val, "true") || try_only) { static struct ovsthread_once once_enable = OVSTHREAD_ONCE_INITIALIZER; if (ovsthread_once_start(&once_enable)) { @@ -483,7 +490,7 @@ dpdk_init(const struct smap *ovs_other_config) enabled = dpdk_init__(ovs_other_config); if (enabled) { VLOG_INFO("DPDK Enabled - initialized"); - } else { + } else if (!try_only) { ovs_abort(rte_errno, "Cannot init EAL"); } ovsthread_once_done(&once_enable); @@ -493,6 +500,7 @@ dpdk_init(const struct smap *ovs_other_config) } else { VLOG_INFO_ONCE("DPDK Disabled - Use other_config:dpdk-init to enable"); } + dpdk_initialized = enabled; } const char * @@ -520,3 +528,12 @@ print_dpdk_version(void) { puts(rte_version()); } + +void +dpdk_status(const struct ovsrec_open_vswitch *cfg) +{ + if (cfg) { + ovsrec_open_vswitch_set_dpdk_initialized(cfg, dpdk_initialized); + ovsrec_open_vswitch_set_dpdk_version(cfg, rte_version()); + } +} diff --git a/lib/dpdk.h b/lib/dpdk.h index b04153591..efdaa637c 100644 --- a/lib/dpdk.h +++ b/lib/dpdk.h @@ -33,11 +33,12 @@ #endif /* DPDK_NETDEV */ struct smap; +struct ovsrec_open_vswitch; void dpdk_init(const struct smap *ovs_other_config); void dpdk_set_lcore_id(unsigned cpu); const char *dpdk_get_vhost_sock_dir(void); bool dpdk_vhost_iommu_enabled(void); void print_dpdk_version(void); - +void dpdk_status(const struct ovsrec_open_vswitch *); #endif /* dpdk.h */ diff --git a/vswitchd/bridge.c b/vswitchd/bridge.c index d90997e3a..ef04b015f 100644 --- a/vswitchd/bridge.c +++ b/vswitchd/bridge.c @@ -407,6 +407,8 @@ bridge_init(const char *remote) ovsdb_idl_omit(idl, &ovsrec_open_vswitch_col_db_version); ovsdb_idl_omit(idl, &ovsrec_open_vswitch_col_system_type); ovsdb_idl_omit(idl, &ovsrec_open_vswitch_col_system_version); + ovsdb_idl_omit_alert(idl, &ovsrec_open_vswitch_col_dpdk_version); + ovsdb_idl_omit_alert(idl, &ovsrec_open_vswitch_col_dpdk_initialized); ovsdb_idl_omit_alert(idl, &ovsrec_bridge_col_datapath_id); ovsdb_idl_omit_alert(idl, &ovsrec_bridge_col_datapath_version); @@ -2836,10 +2838,13 @@ run_status_update(void) * previous one is not done. */ seq = seq_read(connectivity_seq_get()); if (seq != connectivity_seqno || status_txn_try_again) { + const struct ovsrec_open_vswitch *cfg = + ovsrec_open_vswitch_first(idl); struct bridge *br; connectivity_seqno = seq; status_txn = ovsdb_idl_txn_create(idl); + dpdk_status(cfg); HMAP_FOR_EACH (br, node, &all_bridges) { struct port *port; diff --git a/vswitchd/vswitch.ovsschema b/vswitchd/vswitch.ovsschema index 90e50b626..80f17e89b 100644 --- a/vswitchd/vswitch.ovsschema +++ b/vswitchd/vswitch.ovsschema @@ -1,6 +1,6 @@ {"name": "Open_vSwitch", - "version": "7.15.1", - "cksum": "3682332033 23608", + "version": "7.16.0", + "cksum": "2403910601 23776", "tables": { "Open_vSwitch": { "columns": { @@ -47,7 +47,12 @@ "min": 0, "max": "unlimited"}}, "iface_types": { "type": {"key": {"type": "string"}, - "min": 0, "max": "unlimited"}}}, + "min": 0, "max": "unlimited"}}, + "dpdk_initialized": { + "type": "boolean"}, + "dpdk_version": { + "type": {"key": {"type": "string"}, + "min": 0, "max": 1}}}, "isRoot": true, "maxRows": 1}, "Bridge": { diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index 9c2a8263e..37c7f4f80 100644 --- a/vswitchd/vswitch.xml +++ b/vswitchd/vswitch.xml @@ -466,6 +466,11 @@ configuration changes. + + True if is set to + true and the DPDK library is successfully initialized. + +

The statistics column contains key-value pairs that @@ -649,6 +654,12 @@

+ +

+ The version of the linked DPDK library. +

+ +