From patchwork Wed Apr 18 18:30:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aaron Conole X-Patchwork-Id: 900299 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40R9gw355pz9s4c for ; Thu, 19 Apr 2018 04:31:56 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 42D5413D6C; Wed, 18 Apr 2018 18:30:33 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 326A413D4E for ; Wed, 18 Apr 2018 18:30:29 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id BFF2E690 for ; Wed, 18 Apr 2018 18:30:26 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 107654022909; Wed, 18 Apr 2018 18:30:26 +0000 (UTC) Received: from dhcp-25.97.bos.redhat.com (unknown [10.18.25.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id AEBED2026DFD; Wed, 18 Apr 2018 18:30:25 +0000 (UTC) From: Aaron Conole To: dev@openvswitch.org Date: Wed, 18 Apr 2018 14:30:22 -0400 Message-Id: <20180418183023.2627-2-aconole@redhat.com> In-Reply-To: <20180418183023.2627-1-aconole@redhat.com> References: <20180418183023.2627-1-aconole@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 18 Apr 2018 18:30:26 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Wed, 18 Apr 2018 18:30:26 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'aconole@redhat.com' RCPT:'' X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Flavio Leitner , Ilya Maximets Subject: [ovs-dev] [RFC v2 1/2] dpdk: allow init to fail X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org It's possible for dpdk initialization to fail either due to an internal error or an invalid configuration. When that happens, it's rather impolite to immediately abort without any details. With this change, a failed dpdk initialization attempt will continue to trigger a SIGABRT. However, the failure details will be logged, and a user or administrator may have more information to correct the issue. A restart of OvS would still be required to re-attempt initialization. The refactor to propagate the init error will be used in an upcoming commit. Signed-off-by: Aaron Conole --- lib/dpdk.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/lib/dpdk.c b/lib/dpdk.c index 00dd97470..641474cde 100644 --- a/lib/dpdk.c +++ b/lib/dpdk.c @@ -22,6 +22,7 @@ #include #include +#include #include #include #include @@ -306,7 +307,7 @@ static cookie_io_functions_t dpdk_log_func = { .write = dpdk_log_write, }; -static void +static bool dpdk_init__(const struct smap *ovs_other_config) { char **argv = NULL, **argv_to_release = NULL; @@ -422,10 +423,11 @@ dpdk_init__(const struct smap *ovs_other_config) /* Make sure things are initialized ... */ result = rte_eal_init(argc, argv); + argv_release(argv, argv_to_release, argc); if (result < 0) { - ovs_abort(result, "Cannot init EAL"); + VLOG_EMER("Unable to initialize DPDK: %s", ovs_strerror(rte_errno)); + return false; } - argv_release(argv, argv_to_release, argc); /* Set the main thread affinity back to pre rte_eal_init() value */ if (auto_determine && !err) { @@ -459,6 +461,7 @@ dpdk_init__(const struct smap *ovs_other_config) /* Finally, register the dpdk classes */ netdev_dpdk_register(); + return true; } void @@ -476,10 +479,15 @@ dpdk_init(const struct smap *ovs_other_config) if (ovsthread_once_start(&once_enable)) { VLOG_INFO("Using %s", rte_version()); VLOG_INFO("DPDK Enabled - initializing..."); - dpdk_init__(ovs_other_config); - enabled = true; - VLOG_INFO("DPDK Enabled - initialized"); + enabled = dpdk_init__(ovs_other_config); + if (enabled) { + VLOG_INFO("DPDK Enabled - initialized"); + } else { + ovs_abort(rte_errno, "Cannot init EAL"); + } ovsthread_once_done(&once_enable); + } else { + VLOG_ERR_ONCE("DPDK Initialization Failed."); } } else { VLOG_INFO_ONCE("DPDK Disabled - Use other_config:dpdk-init to enable"); From patchwork Wed Apr 18 18:30:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aaron Conole X-Patchwork-Id: 900298 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40R9g94VDXz9s4l for ; Thu, 19 Apr 2018 04:31:17 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 28C5713D62; Wed, 18 Apr 2018 18:30:32 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 4268213D51 for ; Wed, 18 Apr 2018 18:30:29 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 46A3E4DA for ; Wed, 18 Apr 2018 18:30:27 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7201F406E8C3; Wed, 18 Apr 2018 18:30:26 +0000 (UTC) Received: from dhcp-25.97.bos.redhat.com (unknown [10.18.25.61]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1C8892023238; Wed, 18 Apr 2018 18:30:26 +0000 (UTC) From: Aaron Conole To: dev@openvswitch.org Date: Wed, 18 Apr 2018 14:30:23 -0400 Message-Id: <20180418183023.2627-3-aconole@redhat.com> In-Reply-To: <20180418183023.2627-1-aconole@redhat.com> References: <20180418183023.2627-1-aconole@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 18 Apr 2018 18:30:26 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 18 Apr 2018 18:30:26 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'aconole@redhat.com' RCPT:'' X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Flavio Leitner , Ilya Maximets Subject: [ovs-dev] [RFC v2 2/2] dpdk: reflect status and version in the database X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org The normal way of retrieving the running DPDK status involves parsing log files and issuing various incantations of ovs-vsctl and ovs-appctl commands to determine whether the rte_eal_init successfully started. This commit adds two new records to reflect the dpdk version, and the dpdk initialization status. To support this, the other_config:dpdk-init configuration block supports the 'true' and 'try' keywords now, instead of just 'true'. Signed-off-by: Aaron Conole --- Documentation/faq/configuration.rst | 8 +++++--- Documentation/intro/install/dpdk.rst | 27 ++++++++++++++++++++++++--- lib/dpdk-stub.c | 10 ++++++++++ lib/dpdk.c | 21 +++++++++++++++++++-- lib/dpdk.h | 3 ++- vswitchd/bridge.c | 5 +++++ vswitchd/vswitch.ovsschema | 11 ++++++++--- vswitchd/vswitch.xml | 11 +++++++++++ 8 files changed, 84 insertions(+), 12 deletions(-) diff --git a/Documentation/faq/configuration.rst b/Documentation/faq/configuration.rst index 1c93a55cc..0213c58dd 100644 --- a/Documentation/faq/configuration.rst +++ b/Documentation/faq/configuration.rst @@ -102,9 +102,11 @@ Q: How do I configure a DPDK port as an access port? A: Firstly, you must have a DPDK-enabled version of Open vSwitch. - If your version is DPDK-enabled it will support the other-config:dpdk-init - configuration in the database and will display lines with "EAL:..." during - startup when other_config:dpdk-init is set to 'true'. + If your version is DPDK-enabled it may support the dpdk_version and + dpdk_initialized keys in the configuration database. Earlier versions + of Open vSwitch only supported the other-config:dpdk-init key in the + configuration in the database. All versions will display lines with + "EAL:..." during startup when other_config:dpdk-init is set to 'true'. Secondly, when adding a DPDK port, unlike a system port, the type for the interface and valid dpdk-devargs must be specified. For example:: diff --git a/Documentation/intro/install/dpdk.rst b/Documentation/intro/install/dpdk.rst index fea48908d..23fabeffd 100644 --- a/Documentation/intro/install/dpdk.rst +++ b/Documentation/intro/install/dpdk.rst @@ -208,7 +208,8 @@ Open vSwitch should be started as described in :doc:`general` with the exception of ovs-vswitchd, which requires some special configuration to enable DPDK functionality. DPDK configuration arguments can be passed to ovs-vswitchd via the ``other_config`` column of the ``Open_vSwitch`` table. At a minimum, -the ``dpdk-init`` option must be set to ``true``. For example:: +the ``dpdk-init`` option must be set to eithr ``true`` or ``try``. +For example:: $ export PATH=$PATH:/usr/local/share/openvswitch/scripts $ export DB_SOCK=/usr/local/var/run/openvswitch/db.sock @@ -219,8 +220,12 @@ There are many other configuration options, the most important of which are listed below. Defaults will be provided for all values not explicitly set. ``dpdk-init`` - Specifies whether OVS should initialize and support DPDK ports. This is a - boolean, and defaults to false. + Specifies whether OVS should initialize and support DPDK ports. This field + can either be ``true`` or ``try``. + A value of ``true`` will cause the ovs-vswitchd process to abort on + initialization failure. + A value of ``try`` will imply that the ovs-vswitchd process should + continue running even if the EAL initialization fails. ``dpdk-lcore-mask`` Specifies the CPU cores on which dpdk lcore threads should be spawned and @@ -257,6 +262,22 @@ See the section ``Performance Tuning`` for important DPDK customizations. Validating ---------- +DPDK support can be confirmed by validating the ``dpdk_initialized`` boolean +value from the ovsdb. A value of ``true`` means that the DPDK EAL +initialization succeeded:: + + $ ovs-vsctl get Open_vSwitch . dpdk_initialized + true + +Additionally, the library version linked to ovs-vswitchd can be confirmed +with either the ovs-vswitchd logs, or by running either of the commands:: + + $ ovs-vswitchd --version + ovs-vswitchd (Open vSwitch) 2.9.0 + DPDK 17.11.0 + $ ovs-vsctl get Open_vSwitch . dpdk_version + "DPDK 17.11.0" + At this point you can use ovs-vsctl to set up bridges and other Open vSwitch features. Seeing as we've configured the DPDK datapath, we will use DPDK-type ports. For example, to create a userspace bridge named ``br0`` and add two diff --git a/lib/dpdk-stub.c b/lib/dpdk-stub.c index 041cd0cbb..1df1c5848 100644 --- a/lib/dpdk-stub.c +++ b/lib/dpdk-stub.c @@ -21,6 +21,7 @@ #include "smap.h" #include "ovs-thread.h" #include "openvswitch/vlog.h" +#include "vswitch-idl.h" VLOG_DEFINE_THIS_MODULE(dpdk); @@ -59,3 +60,12 @@ void print_dpdk_version(void) { } + +void +dpdk_status(const struct ovsrec_open_vswitch *cfg) +{ + if (cfg) { + ovsrec_open_vswitch_set_dpdk_initialized(cfg, false); + ovsrec_open_vswitch_set_dpdk_version(cfg, "none"); + } +} diff --git a/lib/dpdk.c b/lib/dpdk.c index 641474cde..37c1eac90 100644 --- a/lib/dpdk.c +++ b/lib/dpdk.c @@ -37,6 +37,7 @@ #include "openvswitch/dynamic-string.h" #include "openvswitch/vlog.h" #include "smap.h" +#include "vswitch-idl.h" VLOG_DEFINE_THIS_MODULE(dpdk); @@ -44,6 +45,8 @@ static FILE *log_stream = NULL; /* Stream for DPDK log redirection */ static char *vhost_sock_dir = NULL; /* Location of vhost-user sockets */ static bool vhost_iommu_enabled = false; /* Status of vHost IOMMU support */ +static bool dpdk_initialized = false; /* Indicates successful initialization + * of DPDK. */ static int process_vhost_flags(char *flag, const char *default_val, int size, @@ -473,7 +476,11 @@ dpdk_init(const struct smap *ovs_other_config) return; } - if (smap_get_bool(ovs_other_config, "dpdk-init", false)) { + const char *dpdk_init_val = smap_get_def(ovs_other_config, "dpdk-init", + "false"); + + bool should_try = !strcmp(dpdk_init_val, "try"); + if (!strcmp(dpdk_init_val, "true") || should_try) { static struct ovsthread_once once_enable = OVSTHREAD_ONCE_INITIALIZER; if (ovsthread_once_start(&once_enable)) { @@ -482,7 +489,7 @@ dpdk_init(const struct smap *ovs_other_config) enabled = dpdk_init__(ovs_other_config); if (enabled) { VLOG_INFO("DPDK Enabled - initialized"); - } else { + } else if (!should_try) { ovs_abort(rte_errno, "Cannot init EAL"); } ovsthread_once_done(&once_enable); @@ -492,6 +499,7 @@ dpdk_init(const struct smap *ovs_other_config) } else { VLOG_INFO_ONCE("DPDK Disabled - Use other_config:dpdk-init to enable"); } + dpdk_initialized = enabled; } const char * @@ -519,3 +527,12 @@ print_dpdk_version(void) { puts(rte_version()); } + +void +dpdk_status(const struct ovsrec_open_vswitch *cfg) +{ + if (cfg) { + ovsrec_open_vswitch_set_dpdk_initialized(cfg, dpdk_initialized); + ovsrec_open_vswitch_set_dpdk_version(cfg, rte_version()); + } +} diff --git a/lib/dpdk.h b/lib/dpdk.h index b04153591..efdaa637c 100644 --- a/lib/dpdk.h +++ b/lib/dpdk.h @@ -33,11 +33,12 @@ #endif /* DPDK_NETDEV */ struct smap; +struct ovsrec_open_vswitch; void dpdk_init(const struct smap *ovs_other_config); void dpdk_set_lcore_id(unsigned cpu); const char *dpdk_get_vhost_sock_dir(void); bool dpdk_vhost_iommu_enabled(void); void print_dpdk_version(void); - +void dpdk_status(const struct ovsrec_open_vswitch *); #endif /* dpdk.h */ diff --git a/vswitchd/bridge.c b/vswitchd/bridge.c index d90997e3a..ef04b015f 100644 --- a/vswitchd/bridge.c +++ b/vswitchd/bridge.c @@ -407,6 +407,8 @@ bridge_init(const char *remote) ovsdb_idl_omit(idl, &ovsrec_open_vswitch_col_db_version); ovsdb_idl_omit(idl, &ovsrec_open_vswitch_col_system_type); ovsdb_idl_omit(idl, &ovsrec_open_vswitch_col_system_version); + ovsdb_idl_omit_alert(idl, &ovsrec_open_vswitch_col_dpdk_version); + ovsdb_idl_omit_alert(idl, &ovsrec_open_vswitch_col_dpdk_initialized); ovsdb_idl_omit_alert(idl, &ovsrec_bridge_col_datapath_id); ovsdb_idl_omit_alert(idl, &ovsrec_bridge_col_datapath_version); @@ -2836,10 +2838,13 @@ run_status_update(void) * previous one is not done. */ seq = seq_read(connectivity_seq_get()); if (seq != connectivity_seqno || status_txn_try_again) { + const struct ovsrec_open_vswitch *cfg = + ovsrec_open_vswitch_first(idl); struct bridge *br; connectivity_seqno = seq; status_txn = ovsdb_idl_txn_create(idl); + dpdk_status(cfg); HMAP_FOR_EACH (br, node, &all_bridges) { struct port *port; diff --git a/vswitchd/vswitch.ovsschema b/vswitchd/vswitch.ovsschema index 90e50b626..80f17e89b 100644 --- a/vswitchd/vswitch.ovsschema +++ b/vswitchd/vswitch.ovsschema @@ -1,6 +1,6 @@ {"name": "Open_vSwitch", - "version": "7.15.1", - "cksum": "3682332033 23608", + "version": "7.16.0", + "cksum": "2403910601 23776", "tables": { "Open_vSwitch": { "columns": { @@ -47,7 +47,12 @@ "min": 0, "max": "unlimited"}}, "iface_types": { "type": {"key": {"type": "string"}, - "min": 0, "max": "unlimited"}}}, + "min": 0, "max": "unlimited"}}, + "dpdk_initialized": { + "type": "boolean"}, + "dpdk_version": { + "type": {"key": {"type": "string"}, + "min": 0, "max": 1}}}, "isRoot": true, "maxRows": 1}, "Bridge": { diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index 9c2a8263e..37c7f4f80 100644 --- a/vswitchd/vswitch.xml +++ b/vswitchd/vswitch.xml @@ -466,6 +466,11 @@ configuration changes. + + True if is set to + true and the DPDK library is successfully initialized. + +

The statistics column contains key-value pairs that @@ -649,6 +654,12 @@

+ +

+ The version of the linked DPDK library. +

+ +