From patchwork Fri Dec 8 12:04:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 846194 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ytWY73lH2z9s82 for ; Fri, 8 Dec 2017 23:16:27 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 43176CC9; Fri, 8 Dec 2017 12:15:45 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id D82D1CB7 for ; Fri, 8 Dec 2017 12:15:43 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id F0597189 for ; Fri, 8 Dec 2017 12:15:42 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Dec 2017 04:15:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,377,1508828400"; d="scan'208";a="185249709" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by fmsmga006.fm.intel.com with ESMTP; 08 Dec 2017 04:15:42 -0800 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Fri, 8 Dec 2017 12:04:20 +0000 Message-Id: <1512734667-23202-2-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v6 1/8] Keepalive: Add initial keepalive configuration. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This commit introduces the keepalive configuration by adding 'keepalive' module and also helper and initialization functions that will be invoked by later commits. This commit adds new ovsdb column "keepalive" that shows the status of the datapath threads. This is implemented for DPDK datapath and only status of PMD threads is reported. Signed-off-by: Bhanuprakash Bodireddy Tested-by: Antonio Fischetti Acked-by: Antonio Fischetti --- lib/automake.mk | 2 + lib/keepalive.c | 147 +++++++++++++++++++++++++++++++++++++++++++++ lib/keepalive.h | 86 ++++++++++++++++++++++++++ vswitchd/bridge.c | 3 + vswitchd/vswitch.ovsschema | 8 ++- vswitchd/vswitch.xml | 49 +++++++++++++++ 6 files changed, 293 insertions(+), 2 deletions(-) create mode 100644 lib/keepalive.c create mode 100644 lib/keepalive.h diff --git a/lib/automake.mk b/lib/automake.mk index effe5b5..91d65be 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -110,6 +110,8 @@ lib_libopenvswitch_la_SOURCES = \ lib/json.c \ lib/jsonrpc.c \ lib/jsonrpc.h \ + lib/keepalive.c \ + lib/keepalive.h \ lib/lacp.c \ lib/lacp.h \ lib/latch.h \ diff --git a/lib/keepalive.c b/lib/keepalive.c new file mode 100644 index 0000000..ca8dccb --- /dev/null +++ b/lib/keepalive.c @@ -0,0 +1,147 @@ +/* + * Copyright (c) 2017 Intel, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include "keepalive.h" +#include "lib/vswitch-idl.h" +#include "openvswitch/vlog.h" +#include "seq.h" +#include "timeval.h" + +VLOG_DEFINE_THIS_MODULE(keepalive); + +static bool keepalive_enable = false; /* Keepalive disabled by default. */ +static uint32_t keepalive_timer_interval; /* keepalive timer interval. */ +static struct keepalive_info ka_info; + +/* Returns true if keepalive is enabled, false otherwise. */ +bool +ka_is_enabled(void) +{ + return keepalive_enable; +} + +/* Finds the thread by 'tid' in 'process_list' map and update + * the thread state and last_seen_time stamp. This is invoked + * periodically(based on keepalive-interval) as part of callback + * function in the context of keepalive thread. + */ +static void +ka_set_thread_state_ts(pid_t tid, enum keepalive_state state, + uint64_t last_alive) +{ + struct ka_process_info *pinfo; + + ovs_mutex_lock(&ka_info.proclist_mutex); + HMAP_FOR_EACH_WITH_HASH (pinfo, node, hash_int(tid, 0), + &ka_info.process_list) { + if (pinfo->tid == tid) { + pinfo->state = state; + pinfo->last_seen_time = last_alive; + } + } + ovs_mutex_unlock(&ka_info.proclist_mutex); +} + +/* Retrieve and return the keepalive timer interval from OVSDB. */ +static uint32_t +ka_get_timer_interval(const struct smap *ovs_other_config) +{ + uint32_t ka_interval; + + /* Timer granularity in milliseconds + * Defaults to OVS_KEEPALIVE_TIMEOUT(ms) if not set */ + ka_interval = smap_get_int(ovs_other_config, "keepalive-interval", + OVS_KEEPALIVE_DEFAULT_TIMEOUT); + + VLOG_INFO("Keepalive timer interval set to %"PRIu32" (ms)\n", ka_interval); + return ka_interval; +} + +/* Invoke periodically to update the status and last seen timestamp + * of the thread in to 'process_list' map. Runs in the context of + * keepalive thread. + */ +static void +ka_update_thread_state(pid_t tid, const enum keepalive_state state, + uint64_t last_alive) +{ + switch (state) { + case KA_STATE_ALIVE: + case KA_STATE_MISSING: + ka_set_thread_state_ts(tid, KA_STATE_ALIVE, last_alive); + break; + case KA_STATE_UNUSED: + case KA_STATE_SLEEP: + case KA_STATE_DEAD: + case KA_STATE_GONE: + ka_set_thread_state_ts(tid, state, last_alive); + break; + default: + OVS_NOT_REACHED(); + } +} + +/* Register relay callback function. */ +static void +ka_register_relay_cb(ka_relay_cb cb, void *aux) +{ + ka_info.relay_cb = cb; + ka_info.relay_cb_data = aux; +} + +void +ka_init(const struct smap *ovs_other_config) +{ + if (smap_get_bool(ovs_other_config, "enable-keepalive", false)) { + static struct ovsthread_once once_enable = OVSTHREAD_ONCE_INITIALIZER; + + if (ovsthread_once_start(&once_enable)) { + keepalive_enable = true; + VLOG_INFO("OvS Keepalive enabled."); + + keepalive_timer_interval = + ka_get_timer_interval(ovs_other_config); + + ka_register_relay_cb(ka_update_thread_state, NULL); + ovs_mutex_init(&ka_info.proclist_mutex); + hmap_init(&ka_info.process_list); + + ka_info.init_time = time_wall_msec(); + + ovsthread_once_done(&once_enable); + } + } +} + +void +ka_destroy(void) +{ + if (!ka_is_enabled()) { + return; + } + + ovs_mutex_lock(&ka_info.proclist_mutex); + struct ka_process_info *pinfo; + HMAP_FOR_EACH_POP (pinfo, node, &ka_info.process_list) { + free(pinfo); + } + ovs_mutex_unlock(&ka_info.proclist_mutex); + + hmap_destroy(&ka_info.process_list); + ovs_mutex_destroy(&ka_info.proclist_mutex); +} diff --git a/lib/keepalive.h b/lib/keepalive.h new file mode 100644 index 0000000..a738daa --- /dev/null +++ b/lib/keepalive.h @@ -0,0 +1,86 @@ +/* + * Copyright (c) 2017 Intel, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef KEEPALIVE_H +#define KEEPALIVE_H + +#include +#include "openvswitch/hmap.h" +#include "ovs-thread.h" + +/* Default timeout set to 1000ms */ +#define OVS_KEEPALIVE_DEFAULT_TIMEOUT 1000 + +struct smap; + +/* + * Keepalive states with description + * + * KA_STATE_UNUSED - Not registered to KA framework. + * KA_STATE_ALIVE - Thread is alive. + * KA_STATE_MISSING - Thread missed first heartbeat. + * KA_STATE_DEAD - Thread missed two heartbeats. + * KA_STATE_GONE - Thread missed two or more heartbeats + * and is completely 'burried'. + * KA_STATE_SLEEP - Thread is sleeping. + * + */ +enum keepalive_state { + KA_STATE_UNUSED, + KA_STATE_ALIVE, + KA_STATE_DEAD, + KA_STATE_GONE, + KA_STATE_MISSING, + KA_STATE_SLEEP, +}; + +struct ka_process_info { + /* Thread id of the process, retrieved using ovs_gettid(). */ + pid_t tid; + + /* Core id the thread was last scheduled. */ + int core_id; + + /* Last seen thread state. */ + enum keepalive_state state; + + /* Last seen timestamp of the thread. */ + uint64_t last_seen_time; + struct hmap_node node; +}; + +typedef void (*ka_relay_cb)(int, enum keepalive_state, uint64_t); + +struct keepalive_info { + /* Mutex for 'process_list'. */ + struct ovs_mutex proclist_mutex; + + /* List of process/threads monitored by KA framework. */ + struct hmap process_list OVS_GUARDED; + + /* Keepalive initialization time. */ + uint64_t init_time; + + /* keepalive relay handler. */ + ka_relay_cb relay_cb; + void *relay_cb_data; +}; + +bool ka_is_enabled(void); +void ka_init(const struct smap *); +void ka_destroy(void); + +#endif /* keepalive.h */ diff --git a/vswitchd/bridge.c b/vswitchd/bridge.c index 630c6fa..f70407f 100644 --- a/vswitchd/bridge.c +++ b/vswitchd/bridge.c @@ -34,6 +34,7 @@ #include "hmapx.h" #include "if-notifier.h" #include "jsonrpc.h" +#include "keepalive.h" #include "lacp.h" #include "mac-learning.h" #include "mcast-snooping.h" @@ -506,6 +507,7 @@ bridge_exit(bool delete_datapath) bridge_destroy(br, delete_datapath); } ovsdb_idl_destroy(idl); + ka_destroy(); } /* Looks at the list of managers in 'ovs_cfg' and extracts their remote IP @@ -2959,6 +2961,7 @@ bridge_run(void) if (cfg) { netdev_set_flow_api_enabled(&cfg->other_config); dpdk_init(&cfg->other_config); + ka_init(&cfg->other_config); } /* Initialize the ofproto library. This only needs to run once, but diff --git a/vswitchd/vswitch.ovsschema b/vswitchd/vswitch.ovsschema index 90e50b6..c56a64c 100644 --- a/vswitchd/vswitch.ovsschema +++ b/vswitchd/vswitch.ovsschema @@ -1,6 +1,6 @@ {"name": "Open_vSwitch", - "version": "7.15.1", - "cksum": "3682332033 23608", + "version": "7.16.0", + "cksum": "3631938350 23762", "tables": { "Open_vSwitch": { "columns": { @@ -30,6 +30,10 @@ "type": {"key": "string", "value": "string", "min": 0, "max": "unlimited"}, "ephemeral": true}, + "keepalive": { + "type": {"key": "string", "value": "string", "min": 0, + "max": "unlimited"}, + "ephemeral": true}, "ovs_version": { "type": {"key": {"type": "string"}, "min": 0, "max": 1}}, diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index c145e1a..512292a 100644 --- a/vswitchd/vswitch.xml +++ b/vswitchd/vswitch.xml @@ -568,6 +568,55 @@

+ + +

+ The keepalive column contains key-value pairs that + report health of datapath threads in Open vSwitch. These are updated + periodically (based on the keepalive-interval). +

+ + + Keepalive is disabled by default to avoid overhead in the common + case when heartbeat monitoring is not useful. Set this value to + true to enable keepalive + column or to false to explicitly disable it. + + + +

+ Specifies the keepalive interval value in milliseconds. +

+

+ If not specified, this will be set to 1000 milliseconds (default + value). Changing this value requires restarting the daemon. +

+ + + +

+ One such key-value pair, with ID replaced by the + PMD thread, will exist for each active PMD thread. The value is a + comma-separated list of PMD thread status, core number and the + last seen timestamp of PMD thread. In respective order, these + values are: +

+ +
    +
  1. Status of PMD thread. Valid status include ALIVE, MISSING, + DEAD, GONE, SLEEP.
    2. +
  2. Core id the PMD thread was scheduled.
    3. +
  3. Last seen timestamp(epoch) of the PMD thread.
    4. +
+ +

+ This is only valid for OvS-DPDK Datapath and PMD threads status + is implemented currently. +

+ + From patchwork Fri Dec 8 12:04:21 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 846195 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ytWYr68DPz9s83 for ; Fri, 8 Dec 2017 23:17:04 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 3B4DACF5; Fri, 8 Dec 2017 12:15:47 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id E466DCBD for ; Fri, 8 Dec 2017 12:15:44 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id E34C2189 for ; Fri, 8 Dec 2017 12:15:43 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Dec 2017 04:15:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,377,1508828400"; d="scan'208";a="185249727" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by fmsmga006.fm.intel.com with ESMTP; 08 Dec 2017 04:15:43 -0800 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Fri, 8 Dec 2017 12:04:21 +0000 Message-Id: <1512734667-23202-3-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v6 2/8] dpif-netdev: Register packet processing cores to KA framework. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This commit registers the packet processing PMD threads to keepalive framework. Only PMDs that have rxqs mapped will be registered and actively monitored by KA framework. This commit spawns a keepalive thread that will dispatch heartbeats to PMD threads. The pmd threads respond to heartbeats by marking themselves alive. As long as PMD responds to heartbeats it is considered 'healthy'. Signed-off-by: Bhanuprakash Bodireddy Tested-by: Antonio Fischetti Acked-by: Antonio Fischetti --- lib/dpif-netdev.c | 79 ++++++++++++++++++++++ lib/keepalive.c | 194 ++++++++++++++++++++++++++++++++++++++++++++++++++++-- lib/keepalive.h | 20 ++++++ lib/ovs-thread.c | 6 ++ lib/ovs-thread.h | 1 + lib/util.c | 22 +++++++ lib/util.h | 1 + 7 files changed, 318 insertions(+), 5 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 07f6113..c978a76 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -49,6 +49,7 @@ #include "flow.h" #include "hmapx.h" #include "id-pool.h" +#include "keepalive.h" #include "latch.h" #include "netdev.h" #include "netdev-vport.h" @@ -592,6 +593,7 @@ struct dp_netdev_pmd_thread { atomic_bool reload; /* Do we need to reload ports? */ pthread_t thread; unsigned core_id; /* CPU core id of this pmd thread. */ + pid_t tid; /* PMD thread tid. */ int numa_id; /* numa node id of this pmd thread. */ bool isolated; @@ -1018,6 +1020,72 @@ sorted_poll_thread_list(struct dp_netdev *dp, *n = k; } +static void * +ovs_keepalive(void *f_ OVS_UNUSED) +{ + pthread_detach(pthread_self()); + + for (;;) { + uint64_t interval; + + interval = get_ka_interval(); + xnanosleep(interval * 1000 * 1000); + } + + return NULL; +} + +/* Kickstart 'ovs_keepalive' thread. */ +static void +ka_thread_start(struct dp_netdev *dp) +{ + static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; + + if (ovsthread_once_start(&once)) { + ovs_thread_create("ovs_keepalive", ovs_keepalive, dp); + + ovsthread_once_done(&once); + } +} + +/* Register the datapath threads. This gets invoked on every datapath + * reconfiguration. The pmd thread[s] having rxq[s] mapped will be + * registered to KA framework. + */ +static void +ka_register_datapath_threads(struct dp_netdev *dp) +{ + if (!ka_is_enabled()) { + return; + } + + ka_thread_start(dp); + + ka_reload_datapath_threads_begin(); + + struct dp_netdev_pmd_thread *pmd; + CMAP_FOR_EACH (pmd, node, &dp->poll_threads) { + /* Register only PMD threads. */ + if (pmd->core_id != NON_PMD_CORE_ID) { + /* Skip PMD thread with no rxqs mapping. */ + if (OVS_UNLIKELY(!hmap_count(&pmd->poll_list))) { + /* Rxq mapping changes due to datapath reconfiguration. + * If no rxqs mapped to PMD now due to reconfiguration, + * unregister the pmd thread. */ + ka_unregister_thread(pmd->tid); + continue; + } + + ka_register_thread(pmd->tid); + VLOG_INFO("Registered PMD thread [%d] on Core[%d] to KA framework", + pmd->tid, pmd->core_id); + } + } + ka_cache_registered_threads(); + + ka_reload_datapath_threads_end(); +} + static void dpif_netdev_pmd_rebalance(struct unixctl_conn *conn, int argc, const char *argv[], void *aux OVS_UNUSED) @@ -3819,6 +3887,9 @@ reconfigure_datapath(struct dp_netdev *dp) /* Reload affected pmd threads. */ reload_affected_pmds(dp); + + /* Register datapath threads to KA monitoring. */ + ka_register_datapath_threads(dp); } /* Returns true if one of the netdevs in 'dp' requires a reconfiguration */ @@ -4023,6 +4094,8 @@ pmd_thread_main(void *f_) /* Stores the pmd thread's 'pmd' to 'per_pmd_key'. */ ovsthread_setspecific(pmd->dp->per_pmd_key, pmd); + /* Stores tid in to 'pmd->tid'. */ + ovsthread_set_tid(&pmd->tid); ovs_numa_thread_setaffinity_core(pmd->core_id); dpdk_set_lcore_id(pmd->core_id); poll_cnt = pmd_load_queues_and_ports(pmd, &poll_list); @@ -4056,6 +4129,9 @@ reload: : PMD_CYCLES_IDLE); } + /* Mark PMD thread alive. */ + ka_mark_pmd_thread_alive(pmd->tid); + if (lc++ > 1024) { bool reload; @@ -4089,6 +4165,9 @@ reload: } emc_cache_uninit(&pmd->flow_cache); + + ka_unregister_thread(pmd->tid); + free(poll_list); pmd_free_cached_ports(pmd); return NULL; diff --git a/lib/keepalive.c b/lib/keepalive.c index ca8dccb..b04877f 100644 --- a/lib/keepalive.c +++ b/lib/keepalive.c @@ -19,6 +19,7 @@ #include "keepalive.h" #include "lib/vswitch-idl.h" #include "openvswitch/vlog.h" +#include "process.h" #include "seq.h" #include "timeval.h" @@ -28,11 +29,19 @@ static bool keepalive_enable = false; /* Keepalive disabled by default. */ static uint32_t keepalive_timer_interval; /* keepalive timer interval. */ static struct keepalive_info ka_info; -/* Returns true if keepalive is enabled, false otherwise. */ -bool -ka_is_enabled(void) +/* Returns true if state update is allowed, false otherwise. */ +static bool +ka_can_update_state(void) { - return keepalive_enable; + bool reload_inprogress; + bool ka_enable; + + atomic_read_relaxed(&ka_info.reload_threads, &reload_inprogress); + ka_enable = ka_is_enabled(); + + /* Return true if KA is enabled and 'cached_process_list' map reload + * is completed. */ + return ka_enable && !reload_inprogress; } /* Finds the thread by 'tid' in 'process_list' map and update @@ -49,7 +58,7 @@ ka_set_thread_state_ts(pid_t tid, enum keepalive_state state, ovs_mutex_lock(&ka_info.proclist_mutex); HMAP_FOR_EACH_WITH_HASH (pinfo, node, hash_int(tid, 0), &ka_info.process_list) { - if (pinfo->tid == tid) { + if (OVS_LIKELY(pinfo->tid == tid)) { pinfo->state = state; pinfo->last_seen_time = last_alive; } @@ -104,6 +113,177 @@ ka_register_relay_cb(ka_relay_cb cb, void *aux) ka_info.relay_cb_data = aux; } +/* Returns true if keepalive is enabled, false otherwise. */ +bool +ka_is_enabled(void) +{ + return keepalive_enable; +} + +/* Return the Keepalive timer interval. */ +uint32_t +get_ka_interval(void) +{ + return keepalive_timer_interval; +} + +/* 'cached_process_list' map reload in progress. + * + * Should be called before the 'ka_info.cached_process_list' + * is populated from 'ka_info.process_list'. This way the pmd + * doesn't heartbeat while the reload is in progress. */ +void +ka_reload_datapath_threads_begin(void) +{ + atomic_store_relaxed(&ka_info.reload_threads, true); +} + +/* 'cached_process_list' map reload finished. + * + * Should be called after the 'ka_info.cached_process_list' + * is populated from 'ka_info.process_list'. This way the pmd + * can restart heartbeat when the reload is finished. */ +void +ka_reload_datapath_threads_end(void) +{ + atomic_store_relaxed(&ka_info.reload_threads, false); +} + +/* Register thread to KA framework. */ +void +ka_register_thread(pid_t tid) +{ + if (ka_is_enabled()) { + struct ka_process_info *ka_pinfo; + int core_id = -1; + char proc_name[18] = "UNDEFINED"; + + struct process_info pinfo; + int success = get_process_info(tid, &pinfo); + if (success) { + core_id = pinfo.core_id; + ovs_strlcpy(proc_name, pinfo.name, sizeof proc_name); + } + + uint32_t hash = hash_int(tid, 0); + ovs_mutex_lock(&ka_info.proclist_mutex); + HMAP_FOR_EACH_WITH_HASH (ka_pinfo, node, + hash, &ka_info.process_list) { + /* Thread is already registered. */ + if (ka_pinfo->tid == tid) { + goto out; + } + } + + ka_pinfo = xmalloc(sizeof *ka_pinfo); + ka_pinfo->tid = tid; + ka_pinfo->core_id = core_id; + ovs_strlcpy(ka_pinfo->name, proc_name, sizeof ka_pinfo->name); + + hmap_insert(&ka_info.process_list, &ka_pinfo->node, hash); + + ka_pinfo->state = KA_STATE_ALIVE; + ka_pinfo->last_seen_time = time_wall_msec(); + ka_info.thread_cnt++; /* Increment count of registered threads. */ +out: + ovs_mutex_unlock(&ka_info.proclist_mutex); + } +} + +/* Unregister thread from KA framework. */ +void +ka_unregister_thread(pid_t tid) +{ + if (ka_is_enabled()) { + struct ka_process_info *ka_pinfo; + + ovs_mutex_lock(&ka_info.proclist_mutex); + HMAP_FOR_EACH_WITH_HASH (ka_pinfo, node, hash_int(tid, 0), + &ka_info.process_list) { + /* If thread is registered, remove it from the list */ + if (ka_pinfo->tid == tid) { + hmap_remove(&ka_info.process_list, &ka_pinfo->node); + free(ka_pinfo); + + ka_pinfo->state = KA_STATE_UNUSED; + ka_info.thread_cnt--; /* Decrement thread count. */ + break; + } + } + ovs_mutex_unlock(&ka_info.proclist_mutex); + } +} + +/* Free the 'ka_info.cached_process_list' list. */ +void +ka_free_cached_threads(void) +{ + struct ka_process_info *pinfo_cached; + /* Free threads in the cached list. */ + HMAP_FOR_EACH_POP (pinfo_cached, node, &ka_info.cached_process_list) { + free(pinfo_cached); + } + hmap_shrink(&ka_info.cached_process_list); +} + +/* Cache the list of registered threads from 'ka_info.process_list' + * map into 'ka_info.cached_process_list. + * + * 'cached_process_list' map is an exact copy of 'process_list' that will + * be updated by 'pmd' and 'ovs_keepalive' threads as part of heartbeat + * mechanism. This cached copy is created so that the heartbeats can be + * performed with out acquiring locks. + * + * On datapath reconfiguration, both the 'process_list' and the cached copy + * 'cached_process_list' is updated after setting 'reload_threads' to 'true' + * so that pmd doesn't heartbeat while the maps are updated. + * + */ +void +ka_cache_registered_threads(void) +{ + struct ka_process_info *pinfo, *next, *pinfo_cached; + + ka_free_cached_threads(); + + HMAP_FOR_EACH_SAFE (pinfo, next, node, &ka_info.process_list) { + pinfo_cached = xmemdup(pinfo, sizeof *pinfo_cached); + hmap_insert(&ka_info.cached_process_list, &pinfo_cached->node, + hash_int(pinfo->tid,0)); + } +} + +/* Mark packet processing thread alive. */ +void +ka_mark_pmd_thread_alive(int tid) +{ + if (ka_can_update_state()) { + struct ka_process_info *pinfo; + HMAP_FOR_EACH_WITH_HASH (pinfo, node, hash_int(tid, 0), + &ka_info.cached_process_list) { + if (OVS_LIKELY(pinfo->tid == tid)) { + pinfo->state = KA_STATE_ALIVE; + } + } + } +} + +/* Mark packet processing thread as sleeping. */ +void +ka_mark_pmd_thread_sleep(int tid) +{ + if (ka_can_update_state()) { + struct ka_process_info *pinfo; + + HMAP_FOR_EACH_WITH_HASH (pinfo, node, hash_int(tid, 0), + &ka_info.cached_process_list) { + if (pinfo->tid == tid) { + pinfo->state = KA_STATE_SLEEP; + } + } + } +} + void ka_init(const struct smap *ovs_other_config) { @@ -120,6 +300,7 @@ ka_init(const struct smap *ovs_other_config) ka_register_relay_cb(ka_update_thread_state, NULL); ovs_mutex_init(&ka_info.proclist_mutex); hmap_init(&ka_info.process_list); + hmap_init(&ka_info.cached_process_list); ka_info.init_time = time_wall_msec(); @@ -143,5 +324,8 @@ ka_destroy(void) ovs_mutex_unlock(&ka_info.proclist_mutex); hmap_destroy(&ka_info.process_list); + + ka_free_cached_threads(); + hmap_destroy(&ka_info.cached_process_list); ovs_mutex_destroy(&ka_info.proclist_mutex); } diff --git a/lib/keepalive.h b/lib/keepalive.h index a738daa..7674ea3 100644 --- a/lib/keepalive.h +++ b/lib/keepalive.h @@ -48,6 +48,9 @@ enum keepalive_state { }; struct ka_process_info { + /* Process name. */ + char name[16]; + /* Thread id of the process, retrieved using ovs_gettid(). */ pid_t tid; @@ -71,15 +74,32 @@ struct keepalive_info { /* List of process/threads monitored by KA framework. */ struct hmap process_list OVS_GUARDED; + /* cached copy of 'process_list' list. */ + struct hmap cached_process_list; + + /* count of threads registered to KA framework. */ + uint32_t thread_cnt; + /* Keepalive initialization time. */ uint64_t init_time; /* keepalive relay handler. */ ka_relay_cb relay_cb; void *relay_cb_data; + + atomic_bool reload_threads; /* Reload threads in to cached list. */ }; bool ka_is_enabled(void); +uint32_t get_ka_interval(void); +void ka_reload_datapath_threads_begin(void); +void ka_reload_datapath_threads_end(void); +void ka_register_thread(pid_t); +void ka_unregister_thread(pid_t); +void ka_free_cached_threads(void); +void ka_cache_registered_threads(void); +void ka_mark_pmd_thread_alive(int); +void ka_mark_pmd_thread_sleep(int); void ka_init(const struct smap *); void ka_destroy(void); diff --git a/lib/ovs-thread.c b/lib/ovs-thread.c index f8bc06d..ae8e450 100644 --- a/lib/ovs-thread.c +++ b/lib/ovs-thread.c @@ -597,6 +597,12 @@ thread_is_pmd(void) return !strncmp(name, "pmd", 3); } +void +ovsthread_set_tid(pid_t *tid) +{ + *tid = ovs_get_tid(); +} + /* ovsthread_key. */ diff --git a/lib/ovs-thread.h b/lib/ovs-thread.h index 55e51a4..cfb4b04 100644 --- a/lib/ovs-thread.h +++ b/lib/ovs-thread.h @@ -524,5 +524,6 @@ bool may_fork(void); int count_cpu_cores(void); bool thread_is_pmd(void); +void ovsthread_set_tid(pid_t *); #endif /* ovs-thread.h */ diff --git a/lib/util.c b/lib/util.c index 2965656..927929b 100644 --- a/lib/util.c +++ b/lib/util.c @@ -26,6 +26,12 @@ #include #include #include +#ifdef __linux__ +#include +#endif +#ifdef __FreeBSD__ +#include +#endif #include #include "bitmap.h" #include "byte-order.h" @@ -575,6 +581,22 @@ get_page_size(void) return cached; } +/* Returns the tid of the calling thread if supported, -EINVAL otherwise. */ +pid_t +ovs_get_tid(void) +{ +#ifdef __linux__ + return syscall(SYS_gettid); +#elif defined(__FreeBSD__) || defined(__NetBSD__) + long tid; + thr_self(&tid); + return (pid_t)tid; +#endif + + VLOG_ERR("ovs_get_tid(): unsupported."); + return -EINVAL; +} + /* Returns the time at which the system booted, as the number of milliseconds * since the epoch, or 0 if the time of boot cannot be determined. */ long long int diff --git a/lib/util.h b/lib/util.h index b01f421..259346d 100644 --- a/lib/util.h +++ b/lib/util.h @@ -156,6 +156,7 @@ void free_cacheline(void *); void ovs_strlcpy(char *dst, const char *src, size_t size); void ovs_strzcpy(char *dst, const char *src, size_t size); +pid_t ovs_get_tid(void); /* The C standards say that neither the 'dst' nor 'src' argument to * memcpy() may be null, even if 'n' is zero. This wrapper tolerates From patchwork Fri Dec 8 12:04:22 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 846196 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ytWZX0h2kz9s82 for ; Fri, 8 Dec 2017 23:17:39 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 43F06CEC; Fri, 8 Dec 2017 12:15:48 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 5C32ACE5 for ; Fri, 8 Dec 2017 12:15:45 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id F2407189 for ; Fri, 8 Dec 2017 12:15:44 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Dec 2017 04:15:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,377,1508828400"; d="scan'208";a="185249744" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by fmsmga006.fm.intel.com with ESMTP; 08 Dec 2017 04:15:43 -0800 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Fri, 8 Dec 2017 12:04:22 +0000 Message-Id: <1512734667-23202-4-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v6 3/8] dpif-netdev: Enable heartbeats for DPDK datapath. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This commit adds heartbeat mechanism support for DPDK datapath. Heartbeats are sent to registered PMD threads at predefined intervals (as set in ovsdb with 'keepalive-interval'). The heartbeats are only enabled when there is atleast one port added to the bridge and with active PMD thread polling the port. Signed-off-by: Bhanuprakash Bodireddy Tested-by: Antonio Fischetti Acked-by: Antonio Fischetti --- lib/dpif-netdev.c | 14 +++++++++++++- lib/keepalive.c | 42 ++++++++++++++++++++++++++++++++++++++++++ lib/keepalive.h | 1 + 3 files changed, 56 insertions(+), 1 deletion(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index c978a76..9021906 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -1021,14 +1021,26 @@ sorted_poll_thread_list(struct dp_netdev *dp, } static void * -ovs_keepalive(void *f_ OVS_UNUSED) +ovs_keepalive(void *f_) { + struct dp_netdev *dp = f_; + pthread_detach(pthread_self()); for (;;) { + bool hb_enable; + int n_pmds; uint64_t interval; interval = get_ka_interval(); + n_pmds = cmap_count(&dp->poll_threads) - 1; + hb_enable = (n_pmds > 0) ? true : false; + + /* Dispatch heartbeats only if pmd[s] exist. */ + if (hb_enable) { + dispatch_heartbeats(); + } + xnanosleep(interval * 1000 * 1000); } diff --git a/lib/keepalive.c b/lib/keepalive.c index b04877f..0e4b2b6 100644 --- a/lib/keepalive.c +++ b/lib/keepalive.c @@ -284,6 +284,48 @@ ka_mark_pmd_thread_sleep(int tid) } } +/* Dispatch heartbeats from 'ovs_keepalive' thread. */ +void +dispatch_heartbeats(void) +{ + struct ka_process_info *pinfo, *pinfo_next; + + /* Iterates over the list of processes in 'cached_process_list' map. */ + HMAP_FOR_EACH_SAFE (pinfo, pinfo_next, node, + &ka_info.cached_process_list) { + if (pinfo->state == KA_STATE_UNUSED) { + continue; + } + + switch (pinfo->state) { + case KA_STATE_UNUSED: + break; + case KA_STATE_ALIVE: + pinfo->state = KA_STATE_MISSING; + pinfo->last_seen_time = time_wall_msec(); + break; + case KA_STATE_MISSING: + pinfo->state = KA_STATE_DEAD; + break; + case KA_STATE_DEAD: + pinfo->state = KA_STATE_GONE; + break; + case KA_STATE_GONE: + break; + case KA_STATE_SLEEP: + pinfo->state = KA_STATE_SLEEP; + pinfo->last_seen_time = time_wall_msec(); + break; + default: + OVS_NOT_REACHED(); + } + + /* Invoke 'ka_update_thread_state' cb function to update state info + * in to 'ka_info.process_list' map. */ + ka_info.relay_cb(pinfo->tid, pinfo->state, pinfo->last_seen_time); + } +} + void ka_init(const struct smap *ovs_other_config) { diff --git a/lib/keepalive.h b/lib/keepalive.h index 7674ea3..cbc2387 100644 --- a/lib/keepalive.h +++ b/lib/keepalive.h @@ -100,6 +100,7 @@ void ka_free_cached_threads(void); void ka_cache_registered_threads(void); void ka_mark_pmd_thread_alive(int); void ka_mark_pmd_thread_sleep(int); +void dispatch_heartbeats(void); void ka_init(const struct smap *); void ka_destroy(void); From patchwork Fri Dec 8 12:04:23 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 846197 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ytWbC3Cs7z9s82 for ; Fri, 8 Dec 2017 23:18:15 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 8571FCFC; Fri, 8 Dec 2017 12:15:49 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id AE4B3CE6 for ; Fri, 8 Dec 2017 12:15:46 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 3D06B189 for ; Fri, 8 Dec 2017 12:15:46 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Dec 2017 04:15:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,377,1508828400"; d="scan'208";a="185249749" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by fmsmga006.fm.intel.com with ESMTP; 08 Dec 2017 04:15:45 -0800 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Fri, 8 Dec 2017 12:04:23 +0000 Message-Id: <1512734667-23202-5-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v6 4/8] keepalive: Retrieve PMD status periodically. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This commit implements APIs to retrieve the PMD thread status and return the status in the below format for each PMD thread. Format: pmdid="status,core id,last_seen_timestamp(epoch)" eg: pmd62="ALIVE,2,1503333332575" pmd63="GONE,3,1503333332525" The status is periodically retrieved by keepalive thread and stored in keepalive_stats struc which later shall be retrieved by vswitchd thread. In case of four PMD threads the status is as below: "pmd62"="ALIVE,0,1503333332575" "pmd63"="ALIVE,1,1503333332575" "pmd64"="ALIVE,2,1503333332575" "pmd65"="ALIVE,3,1503333332575" Signed-off-by: Bhanuprakash Bodireddy Tested-by: Antonio Fischetti Acked-by: Antonio Fischetti --- lib/dpif-netdev.c | 1 + lib/keepalive.c | 63 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ lib/keepalive.h | 1 + 3 files changed, 65 insertions(+) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 9021906..e9fa3c1 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -1039,6 +1039,7 @@ ovs_keepalive(void *f_) /* Dispatch heartbeats only if pmd[s] exist. */ if (hb_enable) { dispatch_heartbeats(); + get_ka_stats(); } xnanosleep(interval * 1000 * 1000); diff --git a/lib/keepalive.c b/lib/keepalive.c index 0e4b2b6..7d3dbad 100644 --- a/lib/keepalive.c +++ b/lib/keepalive.c @@ -19,6 +19,7 @@ #include "keepalive.h" #include "lib/vswitch-idl.h" #include "openvswitch/vlog.h" +#include "ovs-thread.h" #include "process.h" #include "seq.h" #include "timeval.h" @@ -29,6 +30,9 @@ static bool keepalive_enable = false; /* Keepalive disabled by default. */ static uint32_t keepalive_timer_interval; /* keepalive timer interval. */ static struct keepalive_info ka_info; +static struct ovs_mutex mutex = OVS_MUTEX_INITIALIZER; +static struct smap *keepalive_stats OVS_GUARDED_BY(mutex); + /* Returns true if state update is allowed, false otherwise. */ static bool ka_can_update_state(void) @@ -284,6 +288,65 @@ ka_mark_pmd_thread_sleep(int tid) } } +static void +get_pmd_status(struct smap *ka_pmd_stats) + OVS_REQUIRES(ka_info.proclist_mutex) +{ + struct ka_process_info *pinfo, *pinfo_next; + HMAP_FOR_EACH_SAFE (pinfo, pinfo_next, node, &ka_info.process_list) { + char *state = NULL; + if (pinfo->state == KA_STATE_UNUSED) { + continue; + } + + switch (pinfo->state) { + case KA_STATE_ALIVE: + state = "ALIVE"; + break; + case KA_STATE_MISSING: + state = "MISSING"; + break; + case KA_STATE_DEAD: + state = "DEAD"; + break; + case KA_STATE_GONE: + state = "GONE"; + break; + case KA_STATE_SLEEP: + state = "SLEEP"; + break; + case KA_STATE_UNUSED: + break; + default: + OVS_NOT_REACHED(); + } + + smap_add_format(ka_pmd_stats, pinfo->name, "%s,%d,%ld", + state, pinfo->core_id, pinfo->last_seen_time); + } +} + +void +get_ka_stats(void) +{ + struct smap *ka_pmd_stats; + ka_pmd_stats = xmalloc(sizeof *ka_pmd_stats); + smap_init(ka_pmd_stats); + + ovs_mutex_lock(&ka_info.proclist_mutex); + get_pmd_status(ka_pmd_stats); + ovs_mutex_unlock(&ka_info.proclist_mutex); + + ovs_mutex_lock(&mutex); + if (keepalive_stats) { + smap_destroy(keepalive_stats); + free(keepalive_stats); + keepalive_stats = NULL; + } + keepalive_stats = ka_pmd_stats; + ovs_mutex_unlock(&mutex); +} + /* Dispatch heartbeats from 'ovs_keepalive' thread. */ void dispatch_heartbeats(void) diff --git a/lib/keepalive.h b/lib/keepalive.h index cbc2387..2bae8f1 100644 --- a/lib/keepalive.h +++ b/lib/keepalive.h @@ -100,6 +100,7 @@ void ka_free_cached_threads(void); void ka_cache_registered_threads(void); void ka_mark_pmd_thread_alive(int); void ka_mark_pmd_thread_sleep(int); +void get_ka_stats(void); void dispatch_heartbeats(void); void ka_init(const struct smap *); void ka_destroy(void); From patchwork Fri Dec 8 12:04:24 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 846198 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ytWbm1JMZz9s82 for ; Fri, 8 Dec 2017 23:18:44 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 84429CFA; Fri, 8 Dec 2017 12:15:50 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 6B6FECF8 for ; Fri, 8 Dec 2017 12:15:47 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 0C9D5189 for ; Fri, 8 Dec 2017 12:15:47 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Dec 2017 04:15:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,377,1508828400"; d="scan'208";a="185249752" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by fmsmga006.fm.intel.com with ESMTP; 08 Dec 2017 04:15:46 -0800 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Fri, 8 Dec 2017 12:04:24 +0000 Message-Id: <1512734667-23202-6-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v6 5/8] bridge: Update keepalive status in OVSDB. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This commit allows vswitchd thread to update the OVSDB with the status of all registered PMD threads. The status can be monitored using ovsdb-client and the sample output is below. $ ovsdb-client monitor Open_vSwitch Open_vSwitch keepalive row action keepalive 7b746190-ee71-4dcc-becf-f8cb9c7cb909 old { "pmd62"="ALIVE,0,9226457935188922" "pmd63"="ALIVE,1,1503333678618" "pmd64"="ALIVE,2,1503333678618" "pmd65"="ALIVE,3,1503333678618"} new { "pmd62"="ALIVE,0,9226460230167364" "pmd63"="ALIVE,1,1503333679619" "pmd64"="ALIVE,2,1503333679619" "pmd65"="ALIVE,3,1503333679619""} Signed-off-by: Bhanuprakash Bodireddy Tested-by: Antonio Fischetti Acked-by: Antonio Fischetti --- lib/keepalive.c | 15 +++++++++++++++ lib/keepalive.h | 1 + vswitchd/bridge.c | 26 ++++++++++++++++++++++++++ 3 files changed, 42 insertions(+) diff --git a/lib/keepalive.c b/lib/keepalive.c index 7d3dbad..14ac093 100644 --- a/lib/keepalive.c +++ b/lib/keepalive.c @@ -347,6 +347,21 @@ get_ka_stats(void) ovs_mutex_unlock(&mutex); } +struct smap * +ka_stats_run(void) +{ + struct smap *ka_stats = NULL; + + ovs_mutex_lock(&mutex); + if (keepalive_stats) { + ka_stats = keepalive_stats; + keepalive_stats = NULL; + } + ovs_mutex_unlock(&mutex); + + return ka_stats; +} + /* Dispatch heartbeats from 'ovs_keepalive' thread. */ void dispatch_heartbeats(void) diff --git a/lib/keepalive.h b/lib/keepalive.h index 2bae8f1..e84646a 100644 --- a/lib/keepalive.h +++ b/lib/keepalive.h @@ -101,6 +101,7 @@ void ka_cache_registered_threads(void); void ka_mark_pmd_thread_alive(int); void ka_mark_pmd_thread_sleep(int); void get_ka_stats(void); +struct smap *ka_stats_run(void); void dispatch_heartbeats(void); void ka_init(const struct smap *); void ka_destroy(void); diff --git a/vswitchd/bridge.c b/vswitchd/bridge.c index f70407f..55c925e 100644 --- a/vswitchd/bridge.c +++ b/vswitchd/bridge.c @@ -286,6 +286,7 @@ static bool port_is_synthetic(const struct port *); static void reconfigure_system_stats(const struct ovsrec_open_vswitch *); static void run_system_stats(void); +static void run_keepalive_stats(void); static void bridge_configure_mirrors(struct bridge *); static struct mirror *mirror_create(struct bridge *, @@ -403,6 +404,7 @@ bridge_init(const char *remote) ovsdb_idl_omit_alert(idl, &ovsrec_open_vswitch_col_cur_cfg); ovsdb_idl_omit_alert(idl, &ovsrec_open_vswitch_col_statistics); + ovsdb_idl_omit_alert(idl, &ovsrec_open_vswitch_col_keepalive); ovsdb_idl_omit_alert(idl, &ovsrec_open_vswitch_col_datapath_types); ovsdb_idl_omit_alert(idl, &ovsrec_open_vswitch_col_iface_types); ovsdb_idl_omit(idl, &ovsrec_open_vswitch_col_external_ids); @@ -2686,6 +2688,29 @@ run_system_stats(void) } } +void +run_keepalive_stats(void) +{ + struct smap *ka_stats; + const struct ovsrec_open_vswitch *cfg = ovsrec_open_vswitch_first(idl); + + ka_stats = ka_stats_run(); + if (ka_stats && cfg) { + struct ovsdb_idl_txn *txn; + struct ovsdb_datum datum; + + txn = ovsdb_idl_txn_create(idl); + ovsdb_datum_from_smap(&datum, ka_stats); + smap_destroy(ka_stats); + ovsdb_idl_txn_write(&cfg->header_, &ovsrec_open_vswitch_col_keepalive, + &datum); + ovsdb_idl_txn_commit(txn); + ovsdb_idl_txn_destroy(txn); + + free(ka_stats); + } +} + static const char * ofp12_controller_role_to_str(enum ofp12_controller_role role) { @@ -3039,6 +3064,7 @@ bridge_run(void) run_stats_update(); run_status_update(); run_system_stats(); + run_keepalive_stats(); } void From patchwork Fri Dec 8 12:04:25 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 846199 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ytWc64Prwz9s83 for ; Fri, 8 Dec 2017 23:19:02 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 6AAC3D19; Fri, 8 Dec 2017 12:15:51 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id BB69ECF1 for ; Fri, 8 Dec 2017 12:15:48 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 2251E189 for ; Fri, 8 Dec 2017 12:15:48 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Dec 2017 04:15:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,377,1508828400"; d="scan'208";a="185249757" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by fmsmga006.fm.intel.com with ESMTP; 08 Dec 2017 04:15:47 -0800 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Fri, 8 Dec 2017 12:04:25 +0000 Message-Id: <1512734667-23202-7-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v6 6/8] keepalive: Add support to query keepalive status and statistics. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org This commit adds support to query keepalive status and statistics. $ ovs-appctl keepalive/status keepAlive Status: Enabled $ ovs-appctl keepalive/pmd-health-show Keepalive status keepalive status : Enabled keepalive interva l: 1000 ms keepalive init time: 21 Aug 2017 16:20:31 PMD threads : 4 PMD CORE STATE LAST SEEN TIMESTAMP(UTC) pmd62 0 ALIVE 21 Aug 2017 16:29:31 pmd63 1 ALIVE 21 Aug 2017 16:29:31 pmd64 2 ALIVE 21 Aug 2017 16:29:31 pmd65 3 GONE 21 Aug 2017 16:26:31 Signed-off-by: Bhanuprakash Bodireddy Tested-by: Antonio Fischetti Acked-by: Antonio Fischetti --- lib/keepalive.c | 101 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) diff --git a/lib/keepalive.c b/lib/keepalive.c index 14ac093..75c0884 100644 --- a/lib/keepalive.c +++ b/lib/keepalive.c @@ -18,11 +18,13 @@ #include "keepalive.h" #include "lib/vswitch-idl.h" +#include "openvswitch/dynamic-string.h" #include "openvswitch/vlog.h" #include "ovs-thread.h" #include "process.h" #include "seq.h" #include "timeval.h" +#include "unixctl.h" VLOG_DEFINE_THIS_MODULE(keepalive); @@ -362,6 +364,99 @@ ka_stats_run(void) return ka_stats; } +static void +ka_unixctl_status(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED) +{ + struct ds ds = DS_EMPTY_INITIALIZER; + + ds_put_format(&ds, "keepAlive Status: %s", + ka_is_enabled() ? "Enabled" : "Disabled"); + + unixctl_command_reply(conn, ds_cstr(&ds)); + ds_destroy(&ds); +} + +static void +ka_unixctl_pmd_health_show(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *ka_info_) +{ + struct ds ds = DS_EMPTY_INITIALIZER; + ds_put_format(&ds, + "\n\t\tKeepalive status\n\n"); + + ds_put_format(&ds, "keepalive status : %s\n", + ka_is_enabled() ? "Enabled" : "Disabled"); + + if (!ka_is_enabled()) { + goto out; + } + + ds_put_format(&ds, "keepalive interval : %"PRIu32" ms\n", + get_ka_interval()); + + char *utc = xastrftime_msec("%d %b %Y %H:%M:%S", + ka_info.init_time, true); + ds_put_format(&ds, "keepalive init time: %s \n", utc); + + struct keepalive_info *ka_info = (struct keepalive_info *)ka_info_; + if (OVS_UNLIKELY(!ka_info)) { + goto out; + } + + ds_put_format(&ds, "PMD threads : %"PRIu32" \n", + ka_info->thread_cnt); + ds_put_format(&ds, + "\n PMD\tCORE\tSTATE\tLAST SEEN TIMESTAMP(UTC)\n"); + + struct ka_process_info *pinfo, *pinfo_next; + + ovs_mutex_lock(&ka_info->proclist_mutex); + HMAP_FOR_EACH_SAFE (pinfo, pinfo_next, node, &ka_info->process_list) { + char *state = NULL; + + if (pinfo->state == KA_STATE_UNUSED) { + continue; + } + + switch (pinfo->state) { + case KA_STATE_ALIVE: + state = "ALIVE"; + break; + case KA_STATE_MISSING: + state = "MISSING"; + break; + case KA_STATE_DEAD: + state = "DEAD"; + break; + case KA_STATE_GONE: + state = "GONE"; + break; + case KA_STATE_SLEEP: + state = "SLEEP"; + break; + case KA_STATE_UNUSED: + break; + default: + OVS_NOT_REACHED(); + } + + utc = xastrftime_msec("%d %b %Y %H:%M:%S", + pinfo->last_seen_time, true); + + ds_put_format(&ds, "%s\t%2d\t%s\t%s\n", + pinfo->name, pinfo->core_id, state, utc); + + free(utc); + } + ovs_mutex_unlock(&ka_info->proclist_mutex); + + ds_put_format(&ds, "\n"); +out: + unixctl_command_reply(conn, ds_cstr(&ds)); + ds_destroy(&ds); +} + /* Dispatch heartbeats from 'ovs_keepalive' thread. */ void dispatch_heartbeats(void) @@ -424,6 +519,12 @@ ka_init(const struct smap *ovs_other_config) ka_info.init_time = time_wall_msec(); + unixctl_command_register("keepalive/status", "", 0, 0, + ka_unixctl_status, NULL); + + unixctl_command_register("keepalive/pmd-health-show", "", 0, 0, + ka_unixctl_pmd_health_show, &ka_info); + ovsthread_once_done(&once_enable); } } From patchwork Fri Dec 8 12:04:26 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 846200 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ytWcw1C88z9s82 for ; Fri, 8 Dec 2017 23:19:44 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 6A52BD06; Fri, 8 Dec 2017 12:15:52 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id A7F9DD05 for ; Fri, 8 Dec 2017 12:15:49 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 0630C189 for ; Fri, 8 Dec 2017 12:15:48 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Dec 2017 04:15:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,377,1508828400"; d="scan'208";a="185249762" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by fmsmga006.fm.intel.com with ESMTP; 08 Dec 2017 04:15:48 -0800 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Fri, 8 Dec 2017 12:04:26 +0000 Message-Id: <1512734667-23202-8-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v6 7/8] Documentation: Update DPDK doc with Keepalive feature. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Keepalive feature is aimed at achieving Fastpath Service Assurance in OVS-DPDK deployments. It adds support for monitoring the packet processing threads by dispatching heartbeats at regular intervals. The implementation uses OvSDB for reporting the health of the PMD threads. Any external monitoring application can query the OvSDB for status at regular intervals (or) subscribe to OvSDB updates. keepalive feature can be enabled through below OVSDB settings. enable-keepalive=true - Keepalive feature is disabled by default and should be enabled at startup before ovs-vswitchd daemon is started. keepalive-interval="5000" - Timer interval in milliseconds for monitoring the packet processing cores. When KA is enabled, 'ovs-keepalive' thread shall be spawned that wakes up at regular intervals to update the timestamp and status of pmd threads in process map. This information shall be read by vswitchd thread and written in to 'keepalive' column of Open_vSwitch table in OVSDB. An external monitoring framework like collectd with ovs events support can read (or) subscribe to the datapath status changes in ovsdb. When the state is updated, the collectd shall be notified and will eventually relay the status to ceilometer service running in the controller. Below is the high level overview of deployment model. Compute Node Controller Compute Node Collectd <----------> Ceilometer <--------> Collectd OvS DPDK OvS DPDK +-----+ | VM | +--+--+ \---+---/ | +--+---+ +------------+----------+ +------+-------+ | OVS |-----> | ovsevents plugin | --> | collectd | +--+---+ +------------+----------+ +------+-------+ +------+-----+ +---------------+------------+ | | Ceilometer | <-- | collectd ceilometer plugin | <--- +------+-----+ +---------------+------------+ Performance impact: No noticeable performance or latency impact is observed with KA feature enabled. Signed-off-by: Bhanuprakash Bodireddy Acked-by: Antonio Fischetti --- Documentation/howto/dpdk.rst | 112 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 112 insertions(+) diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index d123819..e7a2b27 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -439,6 +439,118 @@ For certain traffic profiles with many parallel flows, it's recommended to set For more information on the EMC refer to :doc:`/intro/install/dpdk` . +.. _dpdk_keepalive: + +Keepalive +--------- + +OvS Keepalive(KA) feature is disabled by default. To enable KA feature:: + + $ ovs-vsctl --no-wait set Open_vSwitch . other_config:enable-keepalive=true + +The KA feature can't be enabled at run time and should be done at startup +before ovs-vswitchd daemon is started. + +The default timer interval for monitoring packet processing threads is 1000ms. +To set a different timer value, run:: + + $ ovs-vsctl --no-wait set Open_vSwitch . \ + other_config:keepalive-interval="5000" + +The events comprise of thread states and the last seen timestamps. The events +are written in to process map periodically by keepalive thread. + +The events in the process map are retrieved by main(vswitchd) thread and +updated in to keepalive column of Open_vSwitch table in OVSDB. Any external +monitoring application can read the status from OVSDB at intervals or subscribe +to the updates so that they get notified when the changes happen on OvSDB. + +To monitor the datapath status using ovsdb-client, run:: + + $ ovsdb-client monitor Open_vSwitch + $ ovsdb-client monitor Open_vSwitch Open_vSwitch keepalive + +The datapath thread states are explained below:: + + KA_STATE_UNUSED - Not registered to KA framework. + KA_STATE_ALIVE - Thread alive. + KA_STATE_MISSING - Thread missed first heartbeat. + KA_STATE_DEAD - Thread missed two heartbeats. + KA_STATE_GONE - Thread missed two or more heartbeats and burried. + KA_STATE_SLEEP - Thread is sleeping. + +To query the datapath status, run:: + + $ ovs-appctl keepalive/pmd-health-show + +`collectd `__ has built-in support for DPDK and provides +a `ovs_events` and `ovs_stats` plugin that can be enabled to relay the datapath +status and the PMD status to OpenStack service `Ceilometer +`__. + +To install and configure `collectd`, run:: + + # Clone collectd from Git repository + $ git clone https://github.com/collectd/collectd.git + + # configure and install collectd + $ cd collectd + $ ./build.sh + $ ./configure --enable-syslog --enable-logfile --with-libdpdk=/usr + $ make + $ make install + +`collectd` is installed in ``/opt/collectd`` by default. Edit the configuration +file in ``/opt/collectd/etc/collectd.conf`` to enable logfile, dpdkevents +and csv plugin:: + + LoadPlugin logfile + + LogLevel debug + File "/var/log/collectd/collectd.log" + Timestamp true + PrintSeverity false + + + + LogLevel info + + +Enable `ovs_events` plugin and update the plugindetails as below:: + + LoadPlugin ovs_events + + + Port "6640" + Address "127.0.0.1" + Socket "/usr/local/var/run/openvswitch/db.sock" + SendNotification true + DispatchValues false + + +Enable `ovs_stats` plugin and update the plugindetails as below:: + + LoadPlugin ovs_stats + + + Port "6640" + Address "127.0.0.1" + Socket "/usr/local/var/run/openvswitch/db.sock" + Bridges "br0" + + +Enable ``csv`` plugin as below:: + + LoadPlugin csv + + + DataDir "/var/log/collectd/csv" + StoreRates false + + +With csv plugin enabled, *meter* or *gauge* file is created and timestamp +and thread status gets updated which are sent to ceilometer service. + .. _dpdk-ovs-in-guest: OVS with DPDK Inside VMs From patchwork Fri Dec 8 12:04:27 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bodireddy, Bhanuprakash" X-Patchwork-Id: 846201 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ytWdg1WNqz9s82 for ; Fri, 8 Dec 2017 23:20:23 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 98891D23; Fri, 8 Dec 2017 12:15:53 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 50C84CFA for ; Fri, 8 Dec 2017 12:15:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 161EB1A4 for ; Fri, 8 Dec 2017 12:15:50 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Dec 2017 04:15:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,377,1508828400"; d="scan'208";a="185249765" Received: from silpixa00393942.ir.intel.com (HELO silpixa00393942.ger.corp.intel.com) ([10.237.223.42]) by fmsmga006.fm.intel.com with ESMTP; 08 Dec 2017 04:15:49 -0800 From: Bhanuprakash Bodireddy To: dev@openvswitch.org Date: Fri, 8 Dec 2017 12:04:27 +0000 Message-Id: <1512734667-23202-9-git-send-email-bhanuprakash.bodireddy@intel.com> X-Mailer: git-send-email 2.4.11 In-Reply-To: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> References: <1512734667-23202-1-git-send-email-bhanuprakash.bodireddy@intel.com> X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v6 8/8] NEWS: Add keepalive support information in NEWS. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Signed-off-by: Bhanuprakash Bodireddy Acked-by: Antonio Fischetti --- NEWS | 2 ++ 1 file changed, 2 insertions(+) diff --git a/NEWS b/NEWS index 188a075..6fa69ed 100644 --- a/NEWS +++ b/NEWS @@ -15,6 +15,8 @@ Post-v2.8.0 * Add support for compiling OVS with the latest Linux 4.13 kernel - "flush-conntrack" in ovs-dpctl and ovs-appctl now accept a 5-tuple to delete a specific connection tracking entry. + - Userspace Datapath: + * Added Keepalive support for userspace datapath. v2.8.0 - 31 Aug 2017 --------------------