From patchwork Tue May 8 05:07:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: aginwala aginwala X-Patchwork-Id: 915890 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Y1CiMo0M"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40n8xP3B9wz9s2k for ; Fri, 18 May 2018 11:00:57 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id B1A90EF4; Fri, 18 May 2018 01:00:54 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 7EDC6E96 for ; Fri, 18 May 2018 01:00:53 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pg0-f45.google.com (mail-pg0-f45.google.com [74.125.83.45]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 17759196 for ; Fri, 18 May 2018 01:00:53 +0000 (UTC) Received: by mail-pg0-f45.google.com with SMTP id n9-v6so2552505pgq.5 for ; Thu, 17 May 2018 18:00:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=r/muBEocXHd/rK8p+svgdbGeHd00Yz2HXl8znXkhPRA=; b=Y1CiMo0MPo3b3Ym5g1h06MJ5tdM4PyMOzc3erbRq9Z1mB+1FaamFTPOAVkcT7aQZun ysQfUh8EnmMneJjvIJGba5RxbmRLUSe6QWuJSj6ugM6u6gaFYJ9gkxrc0FxP2U6ZGtc7 EoopE2Of0NJ7YrpuNGrULyDssd2qhJdg3+wGIzsyv4udiLkDPydmXdZzLzYjHy70ACin rksza/LCVtOnqO6RflQeJgUVRzWJL5bDZ6ygMjz5CBDntbqNwrQPkb/7nAYIxnKyR5MX z3RIZTAkSaiL/bdd145DeT0436Eh/D+eHnI1XBYCSUJ4Gd/AuBJiID8VDpHTla0JSTq+ RYAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=r/muBEocXHd/rK8p+svgdbGeHd00Yz2HXl8znXkhPRA=; b=PD3iz/RiIqOqtCnHRXdL3OpvLttz1gR0kv9KM8N3xiutTKPmuIgA4ojq/Na24zj9kO FCew0eJZLpedFYazU5AEyHrf66tIKZlYjrYdBFrVl5T0zya3hJjhJ195jnoZWNOxrtHP 0gGRZvagpT5t4nGJ8yrC/hvQKMa/gyS6mk6bZg+8XpqBaqkr8WvSBtmx+lZf5ChrW8q1 /x98Z/Is1CgE3LQI/HmvQ5CtdudzC0phRUz1v+kyFIcZON0vWmK4JeOvYAaqNz9/22TU DtCBCmQPmQl3cpFczi52U1RI/+A+N3EwGq+Kf3qh9B2VVlNjUZjD1bp8R1SXZ2jGzkTL fKjg== X-Gm-Message-State: ALKqPwcyR1+obDZHG6CRcJQwwZskoQxb0ZgslMg7oyIaCK+cVR6psvGV agXK3egURIyhxF0PxUarq16n5g== X-Google-Smtp-Source: AB8JxZqOOa8PCiLw6WPfT9lo1oEWY/LhXbGM+d8mcxe4jk0i9BucXB8IN8axQ1qQLkzXLMsv0m982Q== X-Received: by 2002:a65:6510:: with SMTP id x16-v6mr5836192pgv.204.1526605252404; Thu, 17 May 2018 18:00:52 -0700 (PDT) Received: from ubuntu.corp.ebay.com ([216.113.160.70]) by smtp.gmail.com with ESMTPSA id 131-v6sm9743197pfa.128.2018.05.17.18.00.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 17 May 2018 18:00:51 -0700 (PDT) From: aginwala X-Google-Original-From: aginwala To: dev@openvswitch.org Date: Mon, 7 May 2018 22:07:20 -0700 Message-Id: <1525756040-10610-1-git-send-email-aginwala@ebay.com> X-Mailer: git-send-email 1.9.1 X-Spam-Status: No, score=1.4 required=5.0 tests=BAYES_00, DATE_IN_PAST_96_XX, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=no version=3.3.1 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: aginwala Subject: [ovs-dev] [PATCH v1] ovndb-servers.ocf: add LB support for managing ovsdb cluster X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org using pacemaker so that controllers can be placed in different fault domains. Signed-off-by: aginwala Signed-off-by: aginwala > Signed-off-by: aginwala > --- ovn/utilities/ovndb-servers.ocf | 63 +++++++++++++++++++++++++++++++++-------- 1 file changed, 51 insertions(+), 12 deletions(-) diff --git a/ovn/utilities/ovndb-servers.ocf b/ovn/utilities/ovndb-servers.ocf index 164b6bc..85a5d92 100755 --- a/ovn/utilities/ovndb-servers.ocf +++ b/ovn/utilities/ovndb-servers.ocf @@ -9,6 +9,7 @@ : ${SB_MASTER_PROTO_DEFAULT="tcp"} : ${MANAGE_NORTHD_DEFAULT="no"} : ${INACTIVE_PROBE_DEFAULT="5000"} +: ${LISTEN_ON_MASTER_IP_ONLY_DEFAULT="yes"} CRM_MASTER="${HA_SBIN_DIR}/crm_master -l reboot" CRM_ATTR_REPL_INFO="${HA_SBIN_DIR}/crm_attribute --type crm_config --name OVN_REPL_INFO -s ovn_ovsdb_master_server" @@ -21,6 +22,10 @@ SB_MASTER_PROTO=${OCF_RESKEY_sb_master_protocol:-${SB_MASTER_PROTO_DEFAULT}} MANAGE_NORTHD=${OCF_RESKEY_manage_northd:-${MANAGE_NORTHD_DEFAULT}} INACTIVE_PROBE=${OCF_RESKEY_inactive_probe_interval:-${INACTIVE_PROBE_DEFAULT}} +# In order for pacemaker to work with LB, we can keep LISTEN_ON_MASTER_IP_ONLY +# to false and pass LB vip IP while creating pcs resource. +LISTEN_ON_MASTER_IP_ONLY=${OCF_RESKEY_listen_on_master_ip:-${LISTEN_ON_MASTER_IP_ONLY_DEFAULT}} + # Invalid IP address is an address that can never exist in the network, as # mentioned in rfc-5737. The ovsdb servers connects to this IP address till # a master is promoted and the IPAddr2 resource is started. @@ -157,20 +162,29 @@ ovsdb_server_notify() { ${OVN_CTL} --ovn-manage-ovsdb=no start_northd fi + if [ "x${LISTEN_ON_MASTER_IP_ONLY}" = xno ]; then + nb_target="" + sb_target="" + else + nb_target="p${NB_MASTER_PROTO}\:${NB_MASTER_PORT}\:${MASTER_IP}" + sb_target="p${SB_MASTER_PROTO}\:${SB_MASTER_PORT}\:${MASTER_IP}" + fi + conn=`ovn-nbctl get NB_global . connections` if [ "$conn" == "[]" ] then - ovn-nbctl -- --id=@conn_uuid create Connection \ -target="p${NB_MASTER_PROTO}\:${NB_MASTER_PORT}\:${MASTER_IP}" \ + ovn-nbctl -- --id=@conn_uuid create Connection target=$nb_target \ inactivity_probe=$INACTIVE_PROBE -- set NB_Global . connections=@conn_uuid + fi conn=`ovn-sbctl get SB_global . connections` if [ "$conn" == "[]" ] then - ovn-sbctl -- --id=@conn_uuid create Connection \ -target="p${SB_MASTER_PROTO}\:${SB_MASTER_PORT}\:${MASTER_IP}" \ + + ovn-sbctl -- --id=@conn_uuid create Connection target=$sb_target \ inactivity_probe=$INACTIVE_PROBE -- set SB_Global . connections=@conn_uuid + fi else @@ -295,15 +309,15 @@ ovsdb_server_start() { set ${OVN_CTL} - set $@ --db-nb-addr=${MASTER_IP} --db-nb-port=${NB_MASTER_PORT} - set $@ --db-sb-addr=${MASTER_IP} --db-sb-port=${SB_MASTER_PORT} + # For LB vip to talk to master pool member on a specific tcp port, we need + # to listen on 0.0.0.0.instead of master_ip + if [ "x${LISTEN_ON_MASTER_IP_ONLY}" = xno ]; then + set $@ --db-nb-port=${NB_MASTER_PORT} + set $@ --db-sb-port=${SB_MASTER_PORT} - if [ "x${NB_MASTER_PROTO}" = xtcp ]; then - set $@ --db-nb-create-insecure-remote=yes - fi - - if [ "x${SB_MASTER_PROTO}" = xtcp ]; then - set $@ --db-sb-create-insecure-remote=yes + else + set $@ --db-nb-addr=${MASTER_IP} --db-nb-port=${NB_MASTER_PORT} + set $@ --db-sb-addr=${MASTER_IP} --db-sb-port=${SB_MASTER_PORT} fi if [ "x${present_master}" = x ]; then @@ -313,9 +327,29 @@ ovsdb_server_start() { # Force all copies to come up as slaves by pointing them into # space and let pacemaker pick one to promote: # + if [ "x${NB_MASTER_PROTO}" = xtcp ]; then + set $@ --db-nb-create-insecure-remote=yes + fi + + if [ "x${SB_MASTER_PROTO}" = xtcp ]; then + set $@ --db-sb-create-insecure-remote=yes + fi set $@ --db-nb-sync-from-addr=${INVALID_IP_ADDRESS} --db-sb-sync-from-addr=${INVALID_IP_ADDRESS} elif [ ${present_master} != ${host_name} ]; then + if [ "x${LISTEN_ON_MASTER_IP_ONLY}" = xno ]; then + # TODO for using LB vip, need to test for ssl. + set $@ --db-nb-create-insecure-remote=no + set $@ --db-sb-create-insecure-remote=no + else + if [ "x${NB_MASTER_PROTO}" = xtcp ]; then + set $@ --db-nb-create-insecure-remote=yes + fi + + if [ "x${SB_MASTER_PROTO}" = xtcp ]; then + set $@ --db-sb-create-insecure-remote=yes + fi + fi # An existing master is active, connect to it set $@ --db-nb-sync-from-addr=${MASTER_IP} --db-sb-sync-from-addr=${MASTER_IP} set $@ --db-nb-sync-from-port=${NB_MASTER_PORT} @@ -416,6 +450,11 @@ ovsdb_server_promote() { ;; esac + if [ "x${LISTEN_ON_MASTER_IP_ONLY}" = xno ]; then + # Restart ovs so that new master can listen on tcp port + ${OVN_CTL} stop_ovsdb + ovsdb_server_start + fi ${OVN_CTL} promote_ovnnb ${OVN_CTL} promote_ovnsb