From patchwork Mon Feb 17 05:53:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1238976 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=ovn.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48LY8J6m3Pz9sPJ for ; Mon, 17 Feb 2020 16:54:06 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id D92B885C92; Mon, 17 Feb 2020 05:54:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4G+fRfRO96R3; Mon, 17 Feb 2020 05:54:02 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 3372C85BC9; Mon, 17 Feb 2020 05:54:02 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0F3E6C08A0; Mon, 17 Feb 2020 05:54:02 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 96320C013E for ; Mon, 17 Feb 2020 05:54:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 7F0E585BCE for ; Mon, 17 Feb 2020 05:54:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7w3cMEsIb+fI for ; Mon, 17 Feb 2020 05:53:59 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by whitealder.osuosl.org (Postfix) with ESMTPS id A1E5585BC9 for ; Mon, 17 Feb 2020 05:53:58 +0000 (UTC) X-Originating-IP: 115.99.247.96 Received: from nummac.local (unknown [115.99.247.96]) (Authenticated sender: numans@ovn.org) by relay3-d.mail.gandi.net (Postfix) with ESMTPSA id DEA6D60004; Mon, 17 Feb 2020 05:53:54 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 17 Feb 2020 11:23:45 +0530 Message-Id: <20200217055345.1153487-1-numans@ovn.org> X-Mailer: git-send-email 2.24.1 MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn] ovn-ctl: Provide the option to configure inactive probe from standby to active. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Recently ovsdb-server supported an unixctl command - ovsdb-server/set-active-ovsdb-server-probe-interval to configure inactive probe interval from standby connection to the active. This patch provides the option to configure this from ovn-ctl and the pacemaker OVN OCF script. Signed-off-by: Numan Siddique Acked-by: Han Zhou --- utilities/ovn-ctl | 14 +++++++++++--- utilities/ovn-ctl.8.xml | 8 ++++++-- utilities/ovndb-servers.ocf | 23 +++++++++++++++++++---- 3 files changed, 36 insertions(+), 9 deletions(-) diff --git a/utilities/ovn-ctl b/utilities/ovn-ctl index c7cb42bc1..2a337ae27 100755 --- a/utilities/ovn-ctl +++ b/utilities/ovn-ctl @@ -82,7 +82,8 @@ demote_xx_ovsdb () { local sync_from_proto=$2 local sync_from_port=$3 local active_conf_file=$4 - local ctl_file=$5 + local inactive_probe_to_active=$5 + local ctl_file=$6 if test ! -z "$sync_from_addr"; then echo "$sync_from_proto:$sync_from_addr:$sync_from_port" > $active_conf_file @@ -91,6 +92,7 @@ demote_xx_ovsdb () { if test -e $active_conf_file; then ovn-appctl -t $OVN_RUNDIR/$ctl_file ovsdb-server/set-active-ovsdb-server `cat $active_conf_file` ovn-appctl -t $OVN_RUNDIR/$ctl_file ovsdb-server/connect-active-ovsdb-server + ovn-appctl -t $OVN_RUNDIR/$ctl_file ovsdb-server/set-active-ovsdb-server-probe-interval $inactive_probe_to_active else echo >&2 "$0: active server details not set" exit 1 @@ -99,12 +101,14 @@ demote_xx_ovsdb () { demote_ovnnb() { demote_xx_ovsdb $DB_NB_SYNC_FROM_ADDR $DB_NB_SYNC_FROM_PROTO \ - $DB_NB_SYNC_FROM_PORT $ovnnb_active_conf_file ovnnb_db.ctl + $DB_NB_SYNC_FROM_PORT $ovnnb_active_conf_file \ + $DB_NB_PROBE_INTERVAL_TO_ACTIVE ovnnb_db.ctl } demote_ovnsb() { demote_xx_ovsdb $DB_SB_SYNC_FROM_ADDR $DB_SB_SYNC_FROM_PROTO \ - $DB_SB_SYNC_FROM_PORT $ovnsb_active_conf_file ovnsb_db.ctl + $DB_SB_SYNC_FROM_PORT $ovnsb_active_conf_file \ + $DB_SB_PROBE_INTERVAL_TO_ACTIVE ovnsb_db.ctl } demote_ic_nb() { @@ -642,6 +646,7 @@ set_defaults () { DB_NB_SYNC_FROM_PROTO=tcp DB_NB_SYNC_FROM_ADDR= DB_NB_SYNC_FROM_PORT=6641 + DB_NB_PROBE_INTERVAL_TO_ACTIVE=60000 DB_SB_SOCK=$OVN_RUNDIR/ovnsb_db.sock DB_SB_PID=$OVN_RUNDIR/ovnsb_db.pid @@ -652,6 +657,7 @@ set_defaults () { DB_SB_SYNC_FROM_PROTO=tcp DB_SB_SYNC_FROM_ADDR= DB_SB_SYNC_FROM_PORT=6642 + DB_SB_PROBE_INTERVAL_TO_ACTIVE=60000 DB_IC_NB_SOCK=$OVN_RUNDIR/ovn_ic_nb_db.sock DB_IC_NB_PID=$OVN_RUNDIR/ovn_ic_nb_db.pid @@ -923,10 +929,12 @@ File location options: --db-nb-sync-from-port=PORT OVN Northbound active db tcp port (default: $DB_NB_SYNC_FROM_PORT) --db-nb-sync-from-proto=PROTO OVN Northbound active db transport (default: $DB_NB_SYNC_FROM_PROTO) --db-nb-create-insecure-remote=yes|no Create ptcp OVN Northbound remote (default: $DB_NB_CREATE_INSECURE_REMOTE) + --db-nb-probe-interval-to-active Active probe interval from standby to active ovsdb-server remote (default: $DB_NB_PROBE_INTERVAL_TO_ACTIVE) --db-sb-sync-from-addr=ADDR OVN Southbound active db tcp address (default: $DB_SB_SYNC_FROM_ADDR) --db-sb-sync-from-port=ADDR OVN Southbound active db tcp port (default: $DB_SB_SYNC_FROM_PORT) --db-sb-sync-from-proto=PROTO OVN Southbound active db transport (default: $DB_SB_SYNC_FROM_PROTO) --db-sb-create-insecure-remote=yes|no Create ptcp OVN Southbound remote (default: $DB_SB_CREATE_INSECURE_REMOTE) + --db-sb-probe-interval-to-active Active probe interval from standby to active ovsdb-server remote (default: $DB_SB_PROBE_INTERVAL_TO_ACTIVE) --db-nb-cluster-local-addr=ADDR OVN_Northbound cluster local address \ (default: $DB_NB_CLUSTER_LOCAL_ADDR) --db-nb-cluster-local-port=PORT OVN_Northbound cluster local tcp port \ diff --git a/utilities/ovn-ctl.8.xml b/utilities/ovn-ctl.8.xml index 816701379..f5b7f7aeb 100644 --- a/utilities/ovn-ctl.8.xml +++ b/utilities/ovn-ctl.8.xml @@ -150,6 +150,10 @@

--db-ic-sb-cluster-remote-port=PORT NUMBER

--db-ic-sb-cluster-remote-proto=PROTO (tcp/ssl)

+

Probe interval options

+

--db-nb-probe-interval-to-active=Time in milliseconds

+

--db-sb-probe-interval-to-active=Time in milliseconds

+

Configuration files

Following are the optional configuration files. If present, it should be located in the etc dir

@@ -241,8 +245,8 @@

Promote and demote ovsdb servers

# ovn-ctl promote_ovnnb

# ovn-ctl promote_ovnsb

-

# ovn-ctl --db-nb-sync-from-addr=x.x.x.x --db-nb-sync-from-port=6641 demote_ovnnb

-

# ovn-ctl --db-sb-sync-from-addr=x.x.x.x --db-sb-sync-from-port=6642 demote_ovnsb

+

# ovn-ctl --db-nb-sync-from-addr=x.x.x.x --db-nb-sync-from-port=6641 --db-nb-probe-interval-to-active=60000 demote_ovnnb

+

# ovn-ctl --db-sb-sync-from-addr=x.x.x.x --db-sb-sync-from-port=6642 --db-sb-probe-interval-to-active=60000 demote_ovnsb

Creating a clustered db on 3 nodes with IPs x.x.x.x, y.y.y.y and z.z.z.z

Starting OVN ovsdb servers and ovn-northd on the node with IP x.x.x.x

diff --git a/utilities/ovndb-servers.ocf b/utilities/ovndb-servers.ocf index 42e0412ad..56c2bc322 100755 --- a/utilities/ovndb-servers.ocf +++ b/utilities/ovndb-servers.ocf @@ -9,6 +9,7 @@ : ${SB_MASTER_PROTO_DEFAULT="tcp"} : ${MANAGE_NORTHD_DEFAULT="no"} : ${INACTIVE_PROBE_DEFAULT="5000"} +: ${INACTIVE_PROBE_TO_MASTER_DEFAULT="60000"} : ${LISTEN_ON_MASTER_IP_ONLY_DEFAULT="yes"} : ${NB_SSL_KEY_DEFAULT="/etc/openvswitch/ovnnb-privkey.pem"} : ${NB_SSL_CERT_DEFAULT="/etc/openvswitch/ovnnb-cert.pem"} @@ -27,6 +28,7 @@ SB_MASTER_PORT=${OCF_RESKEY_sb_master_port:-${SB_MASTER_PORT_DEFAULT}} SB_MASTER_PROTO=${OCF_RESKEY_sb_master_protocol:-${SB_MASTER_PROTO_DEFAULT}} MANAGE_NORTHD=${OCF_RESKEY_manage_northd:-${MANAGE_NORTHD_DEFAULT}} INACTIVE_PROBE=${OCF_RESKEY_inactive_probe_interval:-${INACTIVE_PROBE_DEFAULT}} +INACTIVE_PROBE_TO_MASTER=${OCF_RESKEY_inactive_probe_interval_to_master:-${INACTIVE_PROBE_TO_MASTER_DEFAULT}} NB_PRIVKEY=${OCF_RESKEY_ovn_nb_db_privkey:-${NB_SSL_KEY_DEFAULT}} NB_CERT=${OCF_RESKEY_ovn_nb_db_cert:-${NB_SSL_CERT_DEFAULT}} NB_CACERT=${OCF_RESKEY_ovn_nb_db_cacert:-${NB_SSL_CACERT_DEFAULT}} @@ -135,6 +137,15 @@ ovsdb_server_metadata() { + + + Inactive probe interval to use for the connection from standby + ovsdb-server to master ovsdb-server. + + Set inactive probe interval to master + + + If set to yes, the OVNDBs will listen on master IP. Otherwise, it will @@ -266,10 +277,12 @@ inactivity_probe=$INACTIVE_PROBE -- set SB_Global . connections=@conn_uuid ocf_log debug "ovndb_server: Connecting to the new master ${OCF_RESKEY_CRM_meta_notify_promote_uname}" ${OVN_CTL} demote_ovnnb --db-nb-sync-from-addr=${MASTER_IP} \ --db-nb-sync-from-port=${NB_MASTER_PORT} \ - --db-nb-sync-from-proto=${NB_MASTER_PROTO} + --db-nb-sync-from-proto=${NB_MASTER_PROTO} \ + --db-nb-probe-interval-to-active=${INACTIVE_PROBE_TO_MASTER} ${OVN_CTL} demote_ovnsb --db-sb-sync-from-addr=${MASTER_IP} \ --db-sb-sync-from-port=${SB_MASTER_PORT} \ - --db-sb-sync-from-proto=${SB_MASTER_PROTO} + --db-sb-sync-from-proto=${SB_MASTER_PROTO} \ + --db-sb-probe-interval-to-active=${INACTIVE_PROBE_TO_MASTER} fi } @@ -596,10 +609,12 @@ ovsdb_server_demote() { # being demoted. Sync to the surviving one ${OVN_CTL} demote_ovnnb --db-nb-sync-from-addr=${MASTER_IP} \ --db-nb-sync-from-port=${NB_MASTER_PORT} \ - --db-nb-sync-from-proto=${NB_MASTER_PROTO} + --db-nb-sync-from-proto=${NB_MASTER_PROTO} \ + --db-nb-probe-interval-to-active=${INACTIVE_PROBE_TO_MASTER} ${OVN_CTL} demote_ovnsb --db-sb-sync-from-addr=${MASTER_IP} \ --db-sb-sync-from-port=${SB_MASTER_PORT} \ - --db-sb-sync-from-proto=${SB_MASTER_PROTO} + --db-sb-sync-from-proto=${SB_MASTER_PROTO} \ + --db-sb-probe-interval-to-active=${INACTIVE_PROBE_TO_MASTER} else # For completeness, should never be called