From patchwork Thu Feb 28 17:15:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1049634 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="A/a3ZE43"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 449KFJ2rrpz9s4V for ; Fri, 1 Mar 2019 04:25:24 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id D26BAAB87; Thu, 28 Feb 2019 17:23:08 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 29FE6AAF8 for ; Thu, 28 Feb 2019 17:15:40 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B4D9182F for ; Thu, 28 Feb 2019 17:15:39 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id m1so10014831pgq.8 for ; Thu, 28 Feb 2019 09:15:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=zYd+KyQrDMg1fq7WlA7SFG6MWEJMWtY9zXDB7DeE6sM=; b=A/a3ZE43FoOwKtcrScqez18JhIQGK7gfSdJaAfKwlRDi1vzLGjxyLxGOwfN/qKJT7b DaMHj9ecCmATS/mDnOZoLjPfGYUAcrnskMLZiaS+y9TgpS7h+KWr+Hkm8U7YwCsW1Nkb +hl8CE8uM8TlV1G7Jedw30Kcn6gB+gWo3HtaAzP62w8OOYYjoZNIe1BSPtOAH51iTvJh YEStgMzQh6Kf98Wab00liU8IgbHRbhC8HbblVI/Yzyj1wJgU6tOLTEBa2AhEXCQbQ55G jwpEtSTSPDpW2uNXs4guET1MGUA3t2uq06UcvqFrLWsi7VgpJwOhrO4lATFKf/yqFNe6 rOpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=zYd+KyQrDMg1fq7WlA7SFG6MWEJMWtY9zXDB7DeE6sM=; b=inJUD3nqLtWrLJhdvyDMse7RT8BIqSqpNj+l2At+3Ckrl/45AeUq95R5xIH5vZJQ1S f26BMCLNX7vG9mmnnsfvDUqHRE3oncjbX4vytbmGezw28pHMrltqo52pSOV/GhhENv1/ T/ozrqK2LqeXN2A0RquOJR9JnyEwZ0fWgDn7NO8SBKbVbkeEUSZ5tQzPJjLzSGN5jfHK YnS6RKuRHzMU5pEVYLEw5gASRoVIx0yKbhKviFWabwTP/NpoMg3LMRzI54xyugSmROcm ANiL09KK0o3+RHjf5n0H1FBW0WP19quIbNJ0cz0YBbrIgZO/P6X413N2MrS0X3diAo69 nGDA== X-Gm-Message-State: APjAAAVZBZU1hSl5EImHrNKpsOahoC0ThGWBP0k/p6aFO+mXv8ZbGKPZ xhH2Y7swZekzT9g63mKM3dbyCWaP X-Google-Smtp-Source: APXvYqyfBzoV0ufloDyrKFUIctBJvHeOKiqj6Yj5guanIPPk6e6LPau5y/1z80gluhAztNmho83wLA== X-Received: by 2002:a65:4844:: with SMTP id i4mr154146pgs.347.1551374138940; Thu, 28 Feb 2019 09:15:38 -0800 (PST) Received: from localhost.localdomain.localdomain (c-76-21-108-74.hsd1.ca.comcast.net. [76.21.108.74]) by smtp.gmail.com with ESMTPSA id a4sm36972372pga.52.2019.02.28.09.15.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Feb 2019 09:15:38 -0800 (PST) From: Han Zhou X-Google-Original-From: Han Zhou To: dev@openvswitch.org Date: Thu, 28 Feb 2019 09:15:20 -0800 Message-Id: <1551374120-44287-6-git-send-email-hzhou8@ebay.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1551374120-44287-1-git-send-email-hzhou8@ebay.com> References: <1551374120-44287-1-git-send-email-hzhou8@ebay.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v4 5/5] ovsdb-idl.c: Fast resync from server when connection reset. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Han Zhou Use monitor_cond_since to request changes after last version of local data when connection to server is reset, without clearing the local data. It falls back to clearing and repopulating all the data when the requested id cannot be fulfilled by the server. Test result at ovn-scale-test environment using clustered mode: - 1K HVs (ovsdb clients) - 10K lports Without the patch it took 30+ min for the SB ovsdb-server to calm down and HVs to stablize the connectin and finish syncing data. With the patch there were no noticible CPU spike of SB ovsdb-server, and all HVs were in sync with SB within 1 min, which is the probe interval set in this test (so it took at most 1 min for HVs to notice the TCP connection reset and reconnect and resync finished immediately after that). Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-September/047457.html Signed-off-by: Han Zhou --- lib/ovsdb-idl.c | 34 ++++++++++++++++++++++++++++++---- 1 file changed, 30 insertions(+), 4 deletions(-) diff --git a/lib/ovsdb-idl.c b/lib/ovsdb-idl.c index 5d86046..8cfb201 100644 --- a/lib/ovsdb-idl.c +++ b/lib/ovsdb-idl.c @@ -216,6 +216,9 @@ struct ovsdb_idl_db { bool has_lock; /* Has db server told us we have the lock? */ bool is_lock_contended; /* Has db server told us we can't get lock? */ struct json *lock_request_id; /* JSON-RPC ID of in-flight lock request. */ + + /* Last db txn id, used for fast resync through monitor_cond_since */ + struct uuid last_id; }; static void ovsdb_idl_db_track_clear(struct ovsdb_idl_db *); @@ -2070,9 +2073,8 @@ ovsdb_idl_send_monitor_request(struct ovsdb_idl *idl, struct ovsdb_idl_db *db, break; case OVSDB_IDL_MM_MONITOR_COND_SINCE: method = "monitor_cond_since"; - struct uuid last_id = UUID_ZERO; struct json *json_last_id = json_string_create_nocopy( - xasprintf(UUID_FMT, UUID_ARGS(&last_id))); + xasprintf(UUID_FMT, UUID_ARGS(&db->last_id))); json_array_add(params, json_last_id); break; default: @@ -2100,6 +2102,7 @@ ovsdb_idl_db_parse_monitor_reply(struct ovsdb_idl_db *db, { db->change_seqno++; const struct json *table_updates = result; + bool clear_db = true; if (method == OVSDB_IDL_MM_MONITOR_COND_SINCE) { if (result->type != JSON_ARRAY || result->array.n != 3) { struct ovsdb_error *error = ovsdb_syntax_error(result, NULL, @@ -2109,10 +2112,25 @@ ovsdb_idl_db_parse_monitor_reply(struct ovsdb_idl_db *db, return; } - table_updates = result->array.elems[2]; + bool found = json_boolean(result->array.elems[0]); + if (found) { + clear_db = false; + } + const char *last_id = json_string(result->array.elems[1]); + if (!uuid_from_string(&db->last_id, last_id)) { + struct ovsdb_error *error = ovsdb_syntax_error(result, NULL, + "Last-id %s is not in UUID format.", + last_id); + log_parse_update_error(error); + return; + } + + table_updates = result->array.elems[2]; + } + if (clear_db) { + ovsdb_idl_db_clear(db); } - ovsdb_idl_db_clear(db); ovsdb_idl_db_parse_update(db, table_updates, method); } @@ -2155,6 +2173,14 @@ ovsdb_idl_db_parse_update_rpc(struct ovsdb_idl_db *db, struct json *table_updates = params->array.elems[1]; if (!strcmp(msg->method, "update3")) { table_updates = params->array.elems[2]; + const char *last_id = json_string(params->array.elems[1]); + if (!uuid_from_string(&db->last_id, last_id)) { + struct ovsdb_error *error = ovsdb_syntax_error(params, NULL, + "Last-id %s is not in UUID format.", + last_id); + log_parse_update_error(error); + return false; + } } ovsdb_idl_db_parse_update(db, table_updates, mm); return true;