From patchwork Thu Feb 28 17:15:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1049632 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ciEGF2SL"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 449KCx4kMvz9s4V for ; Fri, 1 Mar 2019 04:24:13 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 28573AB7B; Thu, 28 Feb 2019 17:23:07 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 38283AAF8 for ; Thu, 28 Feb 2019 17:15:38 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 5A65782F for ; Thu, 28 Feb 2019 17:15:36 +0000 (UTC) Received: by mail-pf1-f195.google.com with SMTP id g6so10008496pfh.13 for ; Thu, 28 Feb 2019 09:15:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dKDbMUikwPsOb+ph2gxTtuys+UB8v+NFczPPriWGHkU=; b=ciEGF2SLU3QzbsYK3e8LBGz9OPjvQr5I23WXw2nAYEA392rM1xLhQmMH2P17VCdWSp lQD4uKvoMj0VwMPUfnRXL3xs0JaxrKxsjJxA92DF5WwqTrWUKxaNcfMQWI4LUqZm20Mu r6ibIx2gW/ZVjJgYUfPLeBz3IqbVp5A3xCllakcqSTsQz5P4q/4/TBFoVWov6H0I2wG+ bp5pIHjau7INlbKGVB2ndMRyM38uXXbz/sb5I195BjhP7RP62hF1SRELr4InE2HP/98F J441O7NRIBDDvqT4KIH8zpUkXgTSarmw1CbRU3TWSbmRmYs2BY3SjtWzi7VcgpfTu98p NNMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dKDbMUikwPsOb+ph2gxTtuys+UB8v+NFczPPriWGHkU=; b=kreEMHiI0U8vGp5/S2N6v25OcTG+pPI7lFGiUPj502/T6zLWrKRGOTtgm19xnNdxDa 3VWIudJLPlgBxQZRImDvb40dRkg5NT83sRxk6suMn0X1mhDIdz5maTODRcdq4PvKSRnx NCmIXjx2S7JqZLKY6qYMiAoAO2XiXwix0e7VxrONfZYufelToAvIO3xdJfFKb7eO2sz2 TVu9Y2l7cd4ny++75ipBEAgtOmtFlpBi9WLwZWQs/HTTD7c2Ed8xHPp1kN+P/EYwRE6s qU0XQeEe8sAMB5Rq4ZTZ72LwCrLMncXr3fe1xdrIT+jKkdhKSJ/7FkNR/mURZI5wZEsZ lyzw== X-Gm-Message-State: APjAAAWIoZFPY6lNTs+Hoe9Kr0utKGAQcll7fUqYTjKctnS+u5OYiASp 06qUHyGwWgMl+G7ahJM3K8VlOGow X-Google-Smtp-Source: APXvYqxXrdXqpedrJJV0A3DrTTLW9UnlYqosKhMUcFzvDxTTvpDGpoyiypAkmZ/LHjhrvNbqhKR/9w== X-Received: by 2002:a63:5b43:: with SMTP id l3mr201139pgm.298.1551374135329; Thu, 28 Feb 2019 09:15:35 -0800 (PST) Received: from localhost.localdomain.localdomain (c-76-21-108-74.hsd1.ca.comcast.net. [76.21.108.74]) by smtp.gmail.com with ESMTPSA id a4sm36972372pga.52.2019.02.28.09.15.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Feb 2019 09:15:34 -0800 (PST) From: Han Zhou X-Google-Original-From: Han Zhou To: dev@openvswitch.org Date: Thu, 28 Feb 2019 09:15:16 -0800 Message-Id: <1551374120-44287-2-git-send-email-hzhou8@ebay.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1551374120-44287-1-git-send-email-hzhou8@ebay.com> References: <1551374120-44287-1-git-send-email-hzhou8@ebay.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v4 1/5] ovsdb-monitor: Refactor ovsdb monitor implementation. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Han Zhou Current ovsdb monitor maintains its own transaction version through an incremental integer and use it to identify changes starting from different version, and also use it to figure out if each set of changes should be flushed. In particular, it uses number 0 to represent that the change set contains all data for initial client population. It is a smart way but it prevents further extension of the monitoring mechanism to support future use case for clients to request changes starting from a given history point. This patch refactors the structures so that change sets are referenced directly through the pointer. It uses additional members such as init_change_set, new_change_set to indicates the specific change set explicitely, instead of through calculated version numbers based on implicite rules. At the same time, this patch provides better encapsulation for change set (composed of data in a list of tables), while still allowing traversing across change sets for a given table. Signed-off-by: Han Zhou --- ovsdb/jsonrpc-server.c | 18 +-- ovsdb/monitor.c | 417 ++++++++++++++++++++++++++----------------------- ovsdb/monitor.h | 10 +- 3 files changed, 236 insertions(+), 209 deletions(-) diff --git a/ovsdb/jsonrpc-server.c b/ovsdb/jsonrpc-server.c index 77f15d9..f9b7c27 100644 --- a/ovsdb/jsonrpc-server.c +++ b/ovsdb/jsonrpc-server.c @@ -1216,8 +1216,7 @@ struct ovsdb_jsonrpc_monitor { struct ovsdb *db; struct json *monitor_id; struct ovsdb_monitor *dbmon; - uint64_t unflushed; /* The first transaction that has not been - flushed to the jsonrpc remote client. */ + struct ovsdb_monitor_change_set *change_set; enum ovsdb_monitor_version version; struct ovsdb_monitor_session_condition *condition;/* Session's condition */ }; @@ -1389,7 +1388,6 @@ ovsdb_jsonrpc_monitor_create(struct ovsdb_jsonrpc_session *s, struct ovsdb *db, if (version == OVSDB_MONITOR_V2) { m->condition = ovsdb_monitor_session_condition_create(); } - m->unflushed = 0; m->version = version; hmap_insert(&s->monitors, &m->node, json_hash(monitor_id, 0)); m->monitor_id = json_clone(monitor_id); @@ -1436,7 +1434,7 @@ ovsdb_jsonrpc_monitor_create(struct ovsdb_jsonrpc_session *s, struct ovsdb *db, dbmon = ovsdb_monitor_add(m->dbmon); if (dbmon != m->dbmon) { /* Found an exisiting dbmon, reuse the current one. */ - ovsdb_monitor_remove_jsonrpc_monitor(m->dbmon, m, m->unflushed); + ovsdb_monitor_remove_jsonrpc_monitor(m->dbmon, m, NULL); ovsdb_monitor_add_jsonrpc_monitor(dbmon, m); m->dbmon = dbmon; } @@ -1446,7 +1444,7 @@ ovsdb_jsonrpc_monitor_create(struct ovsdb_jsonrpc_session *s, struct ovsdb *db, ovsdb_monitor_condition_bind(m->dbmon, m->condition); } - ovsdb_monitor_get_initial(m->dbmon); + ovsdb_monitor_get_initial(m->dbmon, &m->change_set); json = ovsdb_jsonrpc_monitor_compose_update(m, true); json = json ? json : json_object_create(); return jsonrpc_create_reply(json, request_id); @@ -1578,7 +1576,7 @@ ovsdb_jsonrpc_monitor_cond_change(struct ovsdb_jsonrpc_session *s, struct json *update_json; update_json = ovsdb_monitor_get_update(m->dbmon, false, true, - m->condition, m->version, &m->unflushed); + m->condition, m->version, &m->change_set); if (update_json) { struct jsonrpc_msg *msg; struct json *p; @@ -1648,12 +1646,12 @@ ovsdb_jsonrpc_monitor_compose_update(struct ovsdb_jsonrpc_monitor *m, bool initial) { - if (!ovsdb_monitor_needs_flush(m->dbmon, m->unflushed)) { + if (!ovsdb_monitor_needs_flush(m->dbmon, m->change_set)) { return NULL; } return ovsdb_monitor_get_update(m->dbmon, initial, false, - m->condition, m->version, &m->unflushed); + m->condition, m->version, &m->change_set); } static bool @@ -1662,7 +1660,7 @@ ovsdb_jsonrpc_monitor_needs_flush(struct ovsdb_jsonrpc_session *s) struct ovsdb_jsonrpc_monitor *m; HMAP_FOR_EACH (m, node, &s->monitors) { - if (ovsdb_monitor_needs_flush(m->dbmon, m->unflushed)) { + if (ovsdb_monitor_needs_flush(m->dbmon, m->change_set)) { return true; } } @@ -1686,7 +1684,7 @@ ovsdb_jsonrpc_monitor_destroy(struct ovsdb_jsonrpc_monitor *m, json_destroy(m->monitor_id); hmap_remove(&m->session->monitors, &m->node); - ovsdb_monitor_remove_jsonrpc_monitor(m->dbmon, m, m->unflushed); + ovsdb_monitor_remove_jsonrpc_monitor(m->dbmon, m, m->change_set); ovsdb_monitor_session_condition_destroy(m->condition); free(m); } diff --git a/ovsdb/monitor.c b/ovsdb/monitor.c index 1175e28..03be5dc 100644 --- a/ovsdb/monitor.c +++ b/ovsdb/monitor.c @@ -69,17 +69,31 @@ struct ovsdb_monitor { struct shash tables; /* Holds "struct ovsdb_monitor_table"s. */ struct ovs_list jsonrpc_monitors; /* Contains "jsonrpc_monitor_node"s. */ struct ovsdb *db; - uint64_t n_transactions; /* Count number of committed transactions. */ - struct hmap_node hmap_node; /* Elements within ovsdb_monitors. */ - struct hmap json_cache; /* Contains "ovsdb_monitor_json_cache_node"s.*/ + + /* Contains "ovsdb_monitor_change_set". Each change set contains changes + * from some start point up to the latest committed transaction. There can + * be different change sets for the same struct ovsdb_monitor because there + * are different clients pending on changes starting from different points. + * The different change sets are maintained as a list. */ + struct ovs_list change_sets; + + /* The new change set that is to be populated for future transactions. */ + struct ovsdb_monitor_change_set *new_change_set; + + /* The change set that starts from the first transaction of the DB, which + * is used for populating the initial data for new clients. */ + struct ovsdb_monitor_change_set *init_change_set; + + struct hmap_node hmap_node; /* Elements within ovsdb_monitors. */ + struct hmap json_cache; /* Contains "ovsdb_monitor_json_cache_node"s.*/ }; -/* A json object of updates between 'from_txn' and 'dbmon->n_transactions' - * inclusive. */ +/* A json object of updates for the ovsdb_monitor_change_set and the given + * monitor version. */ struct ovsdb_monitor_json_cache_node { struct hmap_node hmap_node; /* Elements in json cache. */ enum ovsdb_monitor_version version; - uint64_t from_txn; + struct uuid change_set_uuid; struct json *json; /* Null, or a cloned of json */ }; @@ -97,29 +111,48 @@ struct ovsdb_monitor_column { /* A row that has changed in a monitored table. */ struct ovsdb_monitor_row { - struct hmap_node hmap_node; /* In ovsdb_jsonrpc_monitor_table.changes. */ + struct hmap_node hmap_node; /* In ovsdb_monitor_change_set_for_table. */ struct uuid uuid; /* UUID of row that changed. */ struct ovsdb_datum *old; /* Old data, NULL for an inserted row. */ struct ovsdb_datum *new; /* New data, NULL for a deleted row. */ }; -/* Contains 'struct ovsdb_monitor_row's for rows that have been - * updated but not yet flushed to all the jsonrpc connection. - * - * 'n_refs' represent the number of jsonrpc connections that have - * not received updates. Generate the update for the last jsonprc - * connection will also destroy the whole "struct ovsdb_monitor_changes" - * object. +/* Contains a set of changes that are not yet flushed to all the jsonrpc + * connections. * - * 'transaction' stores the first update's transaction id. - * */ -struct ovsdb_monitor_changes { - struct hmap_node hmap_node; /* Element in ovsdb_monitor_tables' changes - hmap. */ + * 'n_refs' represent the number of jsonrpc connections that depend on this + * change set (have not received updates). Generate the update for the last + * jsonprc connection will also destroy the whole "struct + * ovsdb_monitor_change_set" object. + */ +struct ovsdb_monitor_change_set { + /* Element in change_sets of ovsdb_monitor. */ + struct ovs_list list_node; + + /* Internally generated uuid that identifies this data structure. */ + struct uuid uuid; + + /* Contains struct ovsdb_monitor_change_set_for_table. */ + struct ovs_list change_set_for_tables; + + int n_refs; +}; + +/* Contains 'struct ovsdb_monitor_row's for rows in a specific table + * of struct ovsdb_monitor_change_set. It can also be searched from + * member 'change_sets' of struct ovsdb_monitor_table. */ +struct ovsdb_monitor_change_set_for_table { + /* Element in ovsdb_monitor_tables' change_sets list. */ + struct ovs_list list_in_mt; + + /* Element in ovsdb_monitor_change_sets' change_set_for_tables list. */ + struct ovs_list list_in_change_set; + struct ovsdb_monitor_table *mt; + struct ovsdb_monitor_change_set *mcs; + + /* Contains struct ovsdb_monitor_row. */ struct hmap rows; - int n_refs; - uint64_t transaction; /* Save the mt->n_columns that is used when creating the changes. * It can be different from the current mt->n_columns because @@ -147,8 +180,8 @@ struct ovsdb_monitor_table { * ovsdb_monitor_row. It is used for condition evaluation. */ unsigned int *columns_index_map; - /* Contains 'ovsdb_monitor_changes' indexed by 'transaction'. */ - struct hmap changes; + /* Contains 'ovsdb_monitor_change_set_for_table'. */ + struct ovs_list change_sets; }; enum ovsdb_monitor_row_type { @@ -166,36 +199,30 @@ typedef struct json * size_t n_columns); static void ovsdb_monitor_destroy(struct ovsdb_monitor *); -static struct ovsdb_monitor_changes * ovsdb_monitor_table_add_changes( - struct ovsdb_monitor_table *, uint64_t next_txn); -static struct ovsdb_monitor_changes *ovsdb_monitor_table_find_changes( - struct ovsdb_monitor_table *, uint64_t unflushed); -static void ovsdb_monitor_changes_destroy( - struct ovsdb_monitor_changes *); -static void ovsdb_monitor_table_track_changes(struct ovsdb_monitor_table *, - uint64_t unflushed); +static struct ovsdb_monitor_change_set * ovsdb_monitor_add_change_set( + struct ovsdb_monitor *, bool init_only); +static void ovsdb_monitor_change_set_destroy( + struct ovsdb_monitor_change_set *); +static void ovsdb_monitor_track_new_change_set(struct ovsdb_monitor *); static uint32_t -json_cache_hash(enum ovsdb_monitor_version version, uint64_t from_txn) +json_cache_hash(enum ovsdb_monitor_version version, + struct ovsdb_monitor_change_set *change_set) { - uint32_t hash; - - hash = hash_uint64(version); - hash = hash_uint64_basis(from_txn, hash); - - return hash; + return hash_uint64_basis(version, uuid_hash(&change_set->uuid)); } static struct ovsdb_monitor_json_cache_node * ovsdb_monitor_json_cache_search(const struct ovsdb_monitor *dbmon, enum ovsdb_monitor_version version, - uint64_t from_txn) + struct ovsdb_monitor_change_set *change_set) { struct ovsdb_monitor_json_cache_node *node; - uint32_t hash = json_cache_hash(version, from_txn); + uint32_t hash = json_cache_hash(version, change_set); HMAP_FOR_EACH_WITH_HASH(node, hmap_node, hash, &dbmon->json_cache) { - if (node->from_txn == from_txn && node->version == version) { + if (uuid_equals(&node->change_set_uuid, &change_set->uuid) && + node->version == version) { return node; } } @@ -206,15 +233,16 @@ ovsdb_monitor_json_cache_search(const struct ovsdb_monitor *dbmon, static void ovsdb_monitor_json_cache_insert(struct ovsdb_monitor *dbmon, enum ovsdb_monitor_version version, - uint64_t from_txn, struct json *json) + struct ovsdb_monitor_change_set *change_set, + struct json *json) { struct ovsdb_monitor_json_cache_node *node; - uint32_t hash = json_cache_hash(version, from_txn); + uint32_t hash = json_cache_hash(version, change_set); node = xmalloc(sizeof *node); node->version = version; - node->from_txn = from_txn; + node->change_set_uuid = change_set->uuid; node->json = json ? json_clone(json) : NULL; hmap_insert(&dbmon->json_cache, &node->hmap_node, hash); @@ -231,6 +259,23 @@ ovsdb_monitor_json_cache_flush(struct ovsdb_monitor *dbmon) } } +/* Free all versions of json cache for a given change_set.*/ +static void +ovsdb_monitor_json_cache_destroy(struct ovsdb_monitor *dbmon, + struct ovsdb_monitor_change_set *change_set) +{ + enum ovsdb_monitor_version v; + for (v = OVSDB_MONITOR_V1; v < OVSDB_MONITOR_VERSION_MAX; v++) { + struct ovsdb_monitor_json_cache_node *node + = ovsdb_monitor_json_cache_search(dbmon, v, change_set); + if (node) { + hmap_remove(&dbmon->json_cache, &node->hmap_node); + json_destroy(node->json); + free(node); + } + } +} + static int compare_ovsdb_monitor_column(const void *a_, const void *b_) { @@ -248,8 +293,9 @@ compare_ovsdb_monitor_column(const void *a_, const void *b_) /* Finds and returns the ovsdb_monitor_row in 'mt->changes->rows' for the * given 'uuid', or NULL if there is no such row. */ static struct ovsdb_monitor_row * -ovsdb_monitor_changes_row_find(const struct ovsdb_monitor_changes *changes, - const struct uuid *uuid) +ovsdb_monitor_changes_row_find( + const struct ovsdb_monitor_change_set_for_table *changes, + const struct uuid *uuid) { struct ovsdb_monitor_row *row; @@ -386,7 +432,7 @@ ovsdb_monitor_create(struct ovsdb *db, ovs_list_push_back(&db->monitors, &dbmon->list_node); ovs_list_init(&dbmon->jsonrpc_monitors); dbmon->db = db; - dbmon->n_transactions = 0; + ovs_list_init(&dbmon->change_sets); shash_init(&dbmon->tables); hmap_node_nullify(&dbmon->hmap_node); hmap_init(&dbmon->json_cache); @@ -406,7 +452,7 @@ ovsdb_monitor_add_table(struct ovsdb_monitor *m, mt = xzalloc(sizeof *mt); mt->table = table; shash_add(&m->tables, table->schema->name, mt); - hmap_init(&mt->changes); + ovs_list_init(&mt->change_sets); mt->columns_index_map = xmalloc(sizeof *mt->columns_index_map * n_columns); for (i = 0; i < n_columns; i++) { @@ -492,82 +538,85 @@ ovsdb_monitor_table_exists(struct ovsdb_monitor *m, return shash_find_data(&m->tables, table->schema->name); } -static struct ovsdb_monitor_changes * -ovsdb_monitor_table_add_changes(struct ovsdb_monitor_table *mt, - uint64_t next_txn) +static struct ovsdb_monitor_change_set * +ovsdb_monitor_add_change_set(struct ovsdb_monitor *dbmon, + bool init_only) { - struct ovsdb_monitor_changes *changes; - - changes = xzalloc(sizeof *changes); - - changes->transaction = next_txn; - changes->mt = mt; - changes->n_refs = 1; - changes->n_columns = mt->n_columns; - hmap_init(&changes->rows); - hmap_insert(&mt->changes, &changes->hmap_node, hash_uint64(next_txn)); + struct ovsdb_monitor_change_set *change_set = xzalloc(sizeof *change_set); + change_set->uuid = uuid_random(); + ovs_list_push_back(&(dbmon->change_sets), &change_set->list_node); + ovs_list_init(&change_set->change_set_for_tables); + change_set->n_refs = 1; - return changes; -}; - -static struct ovsdb_monitor_changes * -ovsdb_monitor_table_find_changes(struct ovsdb_monitor_table *mt, - uint64_t transaction) -{ - struct ovsdb_monitor_changes *changes; - size_t hash = hash_uint64(transaction); - - HMAP_FOR_EACH_WITH_HASH(changes, hmap_node, hash, &mt->changes) { - if (changes->transaction == transaction) { - return changes; + struct shash_node *node; + SHASH_FOR_EACH (node, &dbmon->tables) { + struct ovsdb_monitor_table *mt = node->data; + if (!init_only || (mt->select & OJMS_INITIAL)) { + struct ovsdb_monitor_change_set_for_table *mcst = + xzalloc(sizeof *mcst); + mcst->mt = mt; + mcst->n_columns = mt->n_columns; + mcst->mcs = change_set; + hmap_init(&mcst->rows); + ovs_list_push_back(&mt->change_sets, &mcst->list_in_mt); + ovs_list_push_back(&change_set->change_set_for_tables, + &mcst->list_in_change_set); } } - return NULL; -} + return change_set; +}; -/* Stop currently tracking changes to table 'mt' since 'transaction'. */ static void -ovsdb_monitor_table_untrack_changes(struct ovsdb_monitor_table *mt, - uint64_t transaction) +ovsdb_monitor_untrack_change_set(struct ovsdb_monitor *dbmon, + struct ovsdb_monitor_change_set *mcs) { - struct ovsdb_monitor_changes *changes = - ovsdb_monitor_table_find_changes(mt, transaction); - if (changes) { - if (--changes->n_refs == 0) { - hmap_remove(&mt->changes, &changes->hmap_node); - ovsdb_monitor_changes_destroy(changes); + ovs_assert(mcs); + if (--mcs->n_refs == 0) { + if (mcs == dbmon->init_change_set) { + dbmon->init_change_set = NULL; + } else if (mcs == dbmon->new_change_set) { + dbmon->new_change_set = NULL; } + ovsdb_monitor_json_cache_destroy(dbmon, mcs); + ovsdb_monitor_change_set_destroy(mcs); } } -/* Start tracking changes to table 'mt' begins from 'transaction' inclusive. - */ static void -ovsdb_monitor_table_track_changes(struct ovsdb_monitor_table *mt, - uint64_t transaction) +ovsdb_monitor_track_new_change_set(struct ovsdb_monitor *dbmon) { - struct ovsdb_monitor_changes *changes; + struct ovsdb_monitor_change_set *change_set = dbmon->new_change_set; - changes = ovsdb_monitor_table_find_changes(mt, transaction); - if (changes) { - changes->n_refs++; + if (change_set) { + change_set->n_refs++; } else { - ovsdb_monitor_table_add_changes(mt, transaction); + change_set = ovsdb_monitor_add_change_set(dbmon, false); + dbmon->new_change_set = change_set; } } static void -ovsdb_monitor_changes_destroy(struct ovsdb_monitor_changes *changes) +ovsdb_monitor_change_set_destroy(struct ovsdb_monitor_change_set *mcs) { - struct ovsdb_monitor_row *row, *next; + ovs_list_remove(&mcs->list_node); - HMAP_FOR_EACH_SAFE (row, next, hmap_node, &changes->rows) { - hmap_remove(&changes->rows, &row->hmap_node); - ovsdb_monitor_row_destroy(changes->mt, row, changes->n_columns); + struct ovsdb_monitor_change_set_for_table *mcst, *next_mcst; + LIST_FOR_EACH_SAFE (mcst, next_mcst, list_in_change_set, + &mcs->change_set_for_tables) { + ovs_list_remove(&mcst->list_in_change_set); + ovs_list_remove(&mcst->list_in_mt); + + struct ovsdb_monitor_row *row, *next; + HMAP_FOR_EACH_SAFE (row, next, hmap_node, &mcst->rows) { + hmap_remove(&mcst->rows, &row->hmap_node); + ovsdb_monitor_row_destroy(mcst->mt, row, mcst->n_columns); + } + hmap_destroy(&mcst->rows); + + free(mcst); } - hmap_destroy(&changes->rows); - free(changes); + free(mcs); } static enum ovsdb_monitor_selection @@ -1025,31 +1074,25 @@ ovsdb_monitor_add_json_row(struct json **json, const char *table_name, static struct json* ovsdb_monitor_compose_update( struct ovsdb_monitor *dbmon, - bool initial, uint64_t transaction, + bool initial, struct ovsdb_monitor_change_set *mcs, const struct ovsdb_monitor_session_condition *condition, compose_row_update_cb_func row_update) { - struct shash_node *node; struct json *json; size_t max_columns = ovsdb_monitor_max_columns(dbmon); unsigned long int *changed = xmalloc(bitmap_n_bytes(max_columns)); json = NULL; - SHASH_FOR_EACH (node, &dbmon->tables) { - struct ovsdb_monitor_table *mt = node->data; + struct ovsdb_monitor_change_set_for_table *mcst; + LIST_FOR_EACH (mcst, list_in_change_set, &mcs->change_set_for_tables) { struct ovsdb_monitor_row *row, *next; - struct ovsdb_monitor_changes *changes; struct json *table_json = NULL; + struct ovsdb_monitor_table *mt = mcst->mt; - changes = ovsdb_monitor_table_find_changes(mt, transaction); - if (!changes) { - continue; - } - - HMAP_FOR_EACH_SAFE (row, next, hmap_node, &changes->rows) { + HMAP_FOR_EACH_SAFE (row, next, hmap_node, &mcst->rows) { struct json *row_json; row_json = (*row_update)(mt, condition, OVSDB_MONITOR_ROW, row, - initial, changed, changes->n_columns); + initial, changed, mcst->n_columns); if (row_json) { ovsdb_monitor_add_json_row(&json, mt->table->schema->name, &table_json, row_json, @@ -1109,8 +1152,9 @@ ovsdb_monitor_compose_cond_change_update( } /* Returns JSON for a object (as described in RFC 7047) - * for all the outstanding changes within 'monitor' that starts from - * '*unflushed'. + * for all the outstanding changes in dbmon that are tracked by the change set + * *p_mcs. + * * If cond_updated is true all rows in the db that match conditions will be * sent. * @@ -1123,39 +1167,37 @@ ovsdb_monitor_get_update( bool initial, bool cond_updated, struct ovsdb_monitor_session_condition *condition, enum ovsdb_monitor_version version, - uint64_t *unflushed_) + struct ovsdb_monitor_change_set **p_mcs) { struct ovsdb_monitor_json_cache_node *cache_node = NULL; - struct shash_node *node; struct json *json; - const uint64_t unflushed = *unflushed_; - const uint64_t next_unflushed = dbmon->n_transactions + 1; + struct ovsdb_monitor_change_set *mcs = *p_mcs; - ovs_assert(cond_updated ? unflushed == next_unflushed : true); + ovs_assert(cond_updated ? mcs == dbmon->new_change_set : true); /* Return a clone of cached json if one exists. Otherwise, * generate a new one and add it to the cache. */ if (!condition || (!condition->conditional && !cond_updated)) { cache_node = ovsdb_monitor_json_cache_search(dbmon, version, - unflushed); + mcs); } if (cache_node) { json = cache_node->json ? json_clone(cache_node->json) : NULL; } else { if (version == OVSDB_MONITOR_V1) { json = - ovsdb_monitor_compose_update(dbmon, initial, unflushed, + ovsdb_monitor_compose_update(dbmon, initial, mcs, condition, ovsdb_monitor_compose_row_update); } else { ovs_assert(version == OVSDB_MONITOR_V2); if (!cond_updated) { - json = ovsdb_monitor_compose_update(dbmon, initial, unflushed, + json = ovsdb_monitor_compose_update(dbmon, initial, mcs, condition, ovsdb_monitor_compose_row_update2); if (!condition || !condition->conditional) { - ovsdb_monitor_json_cache_insert(dbmon, version, unflushed, + ovsdb_monitor_json_cache_insert(dbmon, version, mcs, json); } } else { @@ -1167,24 +1209,20 @@ ovsdb_monitor_get_update( } } - /* Maintain transaction id of 'changes'. */ - SHASH_FOR_EACH (node, &dbmon->tables) { - struct ovsdb_monitor_table *mt = node->data; - - ovsdb_monitor_table_untrack_changes(mt, unflushed); - ovsdb_monitor_table_track_changes(mt, next_unflushed); - } - *unflushed_ = next_unflushed; + /* Maintain tracking change set. */ + ovsdb_monitor_untrack_change_set(dbmon, mcs); + ovsdb_monitor_track_new_change_set(dbmon); + *p_mcs = dbmon->new_change_set; return json; } bool ovsdb_monitor_needs_flush(struct ovsdb_monitor *dbmon, - uint64_t next_transaction) + struct ovsdb_monitor_change_set *change_set) { - ovs_assert(next_transaction <= dbmon->n_transactions + 1); - return (next_transaction <= dbmon->n_transactions); + ovs_assert(change_set); + return (change_set != dbmon->new_change_set); } void @@ -1243,18 +1281,18 @@ static void ovsdb_monitor_changes_update(const struct ovsdb_row *old, const struct ovsdb_row *new, const struct ovsdb_monitor_table *mt, - struct ovsdb_monitor_changes *changes) + struct ovsdb_monitor_change_set_for_table *mcst) { const struct uuid *uuid = ovsdb_row_get_uuid(new ? new : old); struct ovsdb_monitor_row *change; - change = ovsdb_monitor_changes_row_find(changes, uuid); + change = ovsdb_monitor_changes_row_find(mcst, uuid); if (!change) { change = xzalloc(sizeof *change); - hmap_insert(&changes->rows, &change->hmap_node, uuid_hash(uuid)); + hmap_insert(&mcst->rows, &change->hmap_node, uuid_hash(uuid)); change->uuid = *uuid; - change->old = clone_monitor_row_data(mt, old, changes->n_columns); - change->new = clone_monitor_row_data(mt, new, changes->n_columns); + change->old = clone_monitor_row_data(mt, old, mcst->n_columns); + change->new = clone_monitor_row_data(mt, new, mcst->n_columns); } else { if (new) { if (!change->new) { @@ -1293,19 +1331,17 @@ ovsdb_monitor_changes_update(const struct ovsdb_row *old, * replication, the row carries the same UUID as the row * just deleted. */ - change->new = clone_monitor_row_data(mt, new, - changes->n_columns); + change->new = clone_monitor_row_data(mt, new, mcst->n_columns); } else { - update_monitor_row_data(mt, new, change->new, - changes->n_columns); + update_monitor_row_data(mt, new, change->new, mcst->n_columns); } } else { - free_monitor_row_data(mt, change->new, changes->n_columns); + free_monitor_row_data(mt, change->new, mcst->n_columns); change->new = NULL; if (!change->old) { /* This row was added then deleted. Forget about it. */ - hmap_remove(&changes->rows, &change->hmap_node); + hmap_remove(&mcst->rows, &change->hmap_node); free(change); } } @@ -1363,7 +1399,7 @@ ovsdb_monitor_change_cb(const struct ovsdb_row *old, const struct ovsdb_monitor *m = aux->monitor; struct ovsdb_table *table = new ? new->table : old->table; struct ovsdb_monitor_table *mt; - struct ovsdb_monitor_changes *changes; + struct ovsdb_monitor_change_set_for_table *mcst; if (!aux->mt || table != aux->mt->table) { aux->mt = shash_find_data(&m->tables, table->schema->name); @@ -1380,9 +1416,9 @@ ovsdb_monitor_change_cb(const struct ovsdb_row *old, enum ovsdb_monitor_changes_efficacy efficacy = ovsdb_monitor_changes_classify(type, mt, changed); - HMAP_FOR_EACH(changes, hmap_node, &mt->changes) { - if (efficacy > OVSDB_CHANGES_NO_EFFECT) { - ovsdb_monitor_changes_update(old, new, mt, changes); + if (efficacy > OVSDB_CHANGES_NO_EFFECT) { + LIST_FOR_EACH (mcst, list_in_mt, &mt->change_sets) { + ovsdb_monitor_changes_update(old, new, mt, mcst); } } if (aux->efficacy < efficacy) { @@ -1393,34 +1429,35 @@ ovsdb_monitor_change_cb(const struct ovsdb_row *old, } void -ovsdb_monitor_get_initial(const struct ovsdb_monitor *dbmon) +ovsdb_monitor_get_initial(struct ovsdb_monitor *dbmon, + struct ovsdb_monitor_change_set **p_mcs) { - struct shash_node *node; - - SHASH_FOR_EACH (node, &dbmon->tables) { - struct ovsdb_monitor_table *mt = node->data; - - if (mt->select & OJMS_INITIAL) { - struct ovsdb_row *row; - struct ovsdb_monitor_changes *changes; - - changes = ovsdb_monitor_table_find_changes(mt, 0); - if (!changes) { - changes = ovsdb_monitor_table_add_changes(mt, 0); - HMAP_FOR_EACH (row, hmap_node, &mt->table->rows) { - ovsdb_monitor_changes_update(NULL, row, mt, changes); + if (!dbmon->init_change_set) { + struct ovsdb_monitor_change_set *change_set = + ovsdb_monitor_add_change_set(dbmon, true); + dbmon->init_change_set = change_set; + + struct ovsdb_monitor_change_set_for_table *mcst; + LIST_FOR_EACH (mcst, list_in_change_set, + &change_set->change_set_for_tables) { + if (mcst->mt->select & OJMS_INITIAL) { + struct ovsdb_row *row; + HMAP_FOR_EACH (row, hmap_node, &mcst->mt->table->rows) { + ovsdb_monitor_changes_update(NULL, row, mcst->mt, mcst); } - } else { - changes->n_refs++; } } + } else { + dbmon->init_change_set->n_refs++; } + + *p_mcs = dbmon->init_change_set; } void ovsdb_monitor_remove_jsonrpc_monitor(struct ovsdb_monitor *dbmon, struct ovsdb_jsonrpc_monitor *jsonrpc_monitor, - uint64_t unflushed) + struct ovsdb_monitor_change_set *change_set) { struct jsonrpc_monitor_node *jm; @@ -1433,10 +1470,8 @@ ovsdb_monitor_remove_jsonrpc_monitor(struct ovsdb_monitor *dbmon, LIST_FOR_EACH(jm, node, &dbmon->jsonrpc_monitors) { if (jm->jsonrpc_monitor == jsonrpc_monitor) { /* Release the tracked changes. */ - struct shash_node *node; - SHASH_FOR_EACH (node, &dbmon->tables) { - struct ovsdb_monitor_table *mt = node->data; - ovsdb_monitor_table_untrack_changes(mt, unflushed); + if (change_set) { + ovsdb_monitor_untrack_change_set(dbmon, change_set); } ovs_list_remove(&jm->node); free(jm); @@ -1567,15 +1602,14 @@ ovsdb_monitor_destroy(struct ovsdb_monitor *dbmon) ovsdb_monitor_json_cache_flush(dbmon); hmap_destroy(&dbmon->json_cache); + struct ovsdb_monitor_change_set *cs, *cs_next; + LIST_FOR_EACH_SAFE (cs, cs_next, list_node, &dbmon->change_sets) { + ovsdb_monitor_change_set_destroy(cs); + } + SHASH_FOR_EACH (node, &dbmon->tables) { struct ovsdb_monitor_table *mt = node->data; - struct ovsdb_monitor_changes *changes, *next; - - HMAP_FOR_EACH_SAFE (changes, next, hmap_node, &mt->changes) { - hmap_remove(&mt->changes, &changes->hmap_node); - ovsdb_monitor_changes_destroy(changes); - } - hmap_destroy(&mt->changes); + ovs_assert(ovs_list_is_empty(&mt->change_sets)); free(mt->columns); free(mt->columns_index_map); free(mt); @@ -1590,24 +1624,17 @@ ovsdb_monitor_commit(struct ovsdb_monitor *m, const struct ovsdb_txn *txn) struct ovsdb_monitor_aux aux; ovsdb_monitor_init_aux(&aux, m); - /* Update ovsdb_monitor's transaction number for - * each transaction, before calling ovsdb_monitor_change_cb(). */ - m->n_transactions++; ovsdb_txn_for_each_change(txn, ovsdb_monitor_change_cb, &aux); - switch(aux.efficacy) { - case OVSDB_CHANGES_NO_EFFECT: - /* The transaction is ignored by the monitor. - * Roll back the 'n_transactions' as if the transaction - * has never happened. */ - m->n_transactions--; - break; - case OVSDB_CHANGES_REQUIRE_INTERNAL_UPDATE: - /* Nothing. */ - break; - case OVSDB_CHANGES_REQUIRE_EXTERNAL_UPDATE: - ovsdb_monitor_json_cache_flush(m); - break; + if (aux.efficacy > OVSDB_CHANGES_NO_EFFECT) { + /* The transaction is has impact to the monitor. + * Reset new_change_set, so that a new change set will be + * created for future trackings. */ + m->new_change_set = NULL; + + if (aux.efficacy == OVSDB_CHANGES_REQUIRE_EXTERNAL_UPDATE) { + ovsdb_monitor_json_cache_flush(m); + } } } diff --git a/ovsdb/monitor.h b/ovsdb/monitor.h index 9bc0613..112c672 100644 --- a/ovsdb/monitor.h +++ b/ovsdb/monitor.h @@ -56,9 +56,10 @@ struct ovsdb_monitor *ovsdb_monitor_add(struct ovsdb_monitor *); void ovsdb_monitor_add_jsonrpc_monitor(struct ovsdb_monitor *, struct ovsdb_jsonrpc_monitor *); +struct ovsdb_monitor_change_set; void ovsdb_monitor_remove_jsonrpc_monitor(struct ovsdb_monitor *, struct ovsdb_jsonrpc_monitor *, - uint64_t unflushed); + struct ovsdb_monitor_change_set *); void ovsdb_monitor_add_table(struct ovsdb_monitor *, const struct ovsdb_table *); @@ -77,16 +78,17 @@ struct json *ovsdb_monitor_get_update(struct ovsdb_monitor *, bool cond_updated, struct ovsdb_monitor_session_condition *, enum ovsdb_monitor_version, - uint64_t *unflushed_transaction); + struct ovsdb_monitor_change_set **p_mcs); void ovsdb_monitor_table_add_select(struct ovsdb_monitor *, const struct ovsdb_table *, enum ovsdb_monitor_selection); bool ovsdb_monitor_needs_flush(struct ovsdb_monitor *, - uint64_t next_transaction); + struct ovsdb_monitor_change_set *); -void ovsdb_monitor_get_initial(const struct ovsdb_monitor *); +void ovsdb_monitor_get_initial(struct ovsdb_monitor *, + struct ovsdb_monitor_change_set **); void ovsdb_monitor_get_memory_usage(struct simap *); From patchwork Thu Feb 28 17:15:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1049631 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="TKrOELwC"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 449KCK0kpKz9s4V for ; Fri, 1 Mar 2019 04:23:41 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 4DA83AB75; Thu, 28 Feb 2019 17:23:06 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id A3109AAF8 for ; Thu, 28 Feb 2019 17:15:37 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id E9182806 for ; Thu, 28 Feb 2019 17:15:36 +0000 (UTC) Received: by mail-pg1-f176.google.com with SMTP id h11so10039819pgl.0 for ; Thu, 28 Feb 2019 09:15:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=G2fP9PCHbFkSLFTORs9fTCGmL7Wli93Ib1qW1s17/pc=; b=TKrOELwCNbVd3TkNaMHcPO8xUu6fxN9ZhhivNXIo8scmLZFtOrmFspBrK7Wlk7PE07 eyL/RmmHurTLBWo9T2fBMSOIsqaG3LqDmG4oYQyywlYmEUAbK58kRf2BCkDrjPnbUftJ 16Dbcdtd6BfVBQIE7a8J+3MEAivFcyZWLEBwubeq8Q6C43rWqGH678Lx0CHvuCyys6WK 8pJY62XqyF+oh9SDdQkzE0rU64draBrY9c0B6GTIyFoV97XD4FVfxZePp4pUfQWLIpER BtHi2GajTydoLGZfgg/SHuc53eKG4C469jayVmJv6+6Rt3eQVmcPcelBmwcl0rBIfHGH pyAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=G2fP9PCHbFkSLFTORs9fTCGmL7Wli93Ib1qW1s17/pc=; b=fMae12Fi+gs+x3vPBzNhDrKuC/DaEXe0oOibYBuXpOMWhECbxb7fM3jk/R76CgBR24 51g7PnZjdCB51HJS4sOFsHCtpX83TvGj9JeDB3lchFLO2jPfYaz0G6/9tpkrJwybAry7 0Dw50hMk2RM8y6cQHdU2xt77cPdWiTM/rs9KeLrUD6VuI5Pb/0gM0e8csG6WDhjvlwUJ zHtMlf4Hmvb6z3d4KVefoKN6ALB0z+XxWsNk+g/oXdUBhn4lcfxPvBF5JmuYn2KRn4Gm 7s6NfDNTR9mGcmdid4C95zZVgbuOG9OpsHnzOT7wrehpxx0ZgICgP7iJN5x8K6VC1G8c eGww== X-Gm-Message-State: APjAAAXlXfGuhpWpNr5iERufrBGNXNDW/kgbHsTf4dRtTJluMdG/NcTQ wsf+hhUWEqVZv6BHteHeCoZ3gMPN X-Google-Smtp-Source: APXvYqy83eA0x+qlUbFVTfSo4Qtt2cpUqEdrzYEfy6ZWO0mfWV3NidSSBAAKAVq4vd1GfO/6NY2gng== X-Received: by 2002:a63:c449:: with SMTP id m9mr184797pgg.41.1551374136219; Thu, 28 Feb 2019 09:15:36 -0800 (PST) Received: from localhost.localdomain.localdomain (c-76-21-108-74.hsd1.ca.comcast.net. [76.21.108.74]) by smtp.gmail.com with ESMTPSA id a4sm36972372pga.52.2019.02.28.09.15.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Feb 2019 09:15:35 -0800 (PST) From: Han Zhou X-Google-Original-From: Han Zhou To: dev@openvswitch.org Date: Thu, 28 Feb 2019 09:15:17 -0800 Message-Id: <1551374120-44287-3-git-send-email-hzhou8@ebay.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1551374120-44287-1-git-send-email-hzhou8@ebay.com> References: <1551374120-44287-1-git-send-email-hzhou8@ebay.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v4 2/5] ovsdb-server: Transaction history tracking. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Han Zhou Maintaining last N (n = 100) transactions in memory, which will be used for future patches for generating monitor data from any point in this N transactions. Signed-off-by: Han Zhou --- ovsdb/ovsdb-server.c | 9 ++++ ovsdb/ovsdb.c | 6 +++ ovsdb/ovsdb.h | 10 +++++ ovsdb/transaction.c | 121 ++++++++++++++++++++++++++++++++++++++++++++++++++- ovsdb/transaction.h | 4 ++ 5 files changed, 149 insertions(+), 1 deletion(-) diff --git a/ovsdb/ovsdb-server.c b/ovsdb/ovsdb-server.c index 65a47a4..4e97de8 100644 --- a/ovsdb/ovsdb-server.c +++ b/ovsdb/ovsdb-server.c @@ -219,6 +219,7 @@ main_loop(struct server_config *config, struct shash_node *next; SHASH_FOR_EACH_SAFE (node, next, all_dbs) { struct db *db = node->data; + ovsdb_txn_history_run(db->db); if (ovsdb_trigger_run(db->db, time_msec())) { /* The message below is currently the only reason to disconnect * all clients. */ @@ -568,6 +569,7 @@ parse_txn(struct server_config *config, struct db *db, error = ovsdb_file_txn_from_json(db->db, txn_json, false, &txn); if (!error) { + ovsdb_txn_set_txnid(txnid, txn); log_and_free_error(ovsdb_txn_replay_commit(txn)); } if (!error && !uuid_is_zero(txnid)) { @@ -658,6 +660,11 @@ open_db(struct server_config *config, const char *filename) db->db = ovsdb_create(schema, storage); ovsdb_jsonrpc_server_add_db(config->jsonrpc, db->db); + /* Enable txn history for clustered mode. It is not enabled for other mode + * for now, since txn id is available for clustered mode only. */ + if (ovsdb_storage_is_clustered(storage)) { + ovsdb_txn_history_init(db->db); + } read_db(config, db); error = (db->db->name[0] == '_' @@ -695,6 +702,8 @@ add_server_db(struct server_config *config) json_destroy(schema_json); struct db *db = xzalloc(sizeof *db); + /* We don't need txn_history for server_db. */ + db->filename = xstrdup(""); db->db = ovsdb_create(schema, ovsdb_storage_create_unbacked()); bool ok OVS_UNUSED = ovsdb_jsonrpc_server_add_db(config->jsonrpc, db->db); diff --git a/ovsdb/ovsdb.c b/ovsdb/ovsdb.c index da8db82..cfc96b3 100644 --- a/ovsdb/ovsdb.c +++ b/ovsdb/ovsdb.c @@ -455,6 +455,9 @@ ovsdb_destroy(struct ovsdb *db) /* Remove all the monitors. */ ovsdb_monitors_remove(db); + /* Destroy txn history. */ + ovsdb_txn_history_destroy(db); + /* The caller must ensure that no triggers remain. */ ovs_assert(ovs_list_is_empty(&db->triggers)); @@ -535,6 +538,9 @@ ovsdb_replace(struct ovsdb *dst, struct ovsdb *src) ovsdb_trigger_prereplace_db(trigger); } + /* Destroy txn history. */ + ovsdb_txn_history_destroy(dst); + struct ovsdb_schema *tmp_schema = dst->schema; dst->schema = src->schema; src->schema = tmp_schema; diff --git a/ovsdb/ovsdb.h b/ovsdb/ovsdb.h index d96b1c2..32e5333 100644 --- a/ovsdb/ovsdb.h +++ b/ovsdb/ovsdb.h @@ -67,6 +67,11 @@ bool ovsdb_parse_version(const char *, struct ovsdb_version *); bool ovsdb_is_valid_version(const char *); /* Database. */ +struct ovsdb_txn_history_node { + struct ovs_list node; /* Element in struct ovsdb's txn_history list */ + struct ovsdb_txn *txn; +}; + struct ovsdb { char *name; struct ovsdb_schema *schema; @@ -80,6 +85,11 @@ struct ovsdb { bool run_triggers; struct ovsdb_table *rbac_role; + + /* History trasanctions for incremental monitor transfer. */ + bool need_txn_history; /* Need to maintain history of transactions. */ + unsigned int n_txn_history; /* Current number of history transactions. */ + struct ovs_list txn_history; /* Contains "struct ovsdb_txn_history_node. */ }; struct ovsdb *ovsdb_create(struct ovsdb_schema *, struct ovsdb_storage *); diff --git a/ovsdb/transaction.c b/ovsdb/transaction.c index 5a43132..148d108 100644 --- a/ovsdb/transaction.c +++ b/ovsdb/transaction.c @@ -40,6 +40,7 @@ struct ovsdb_txn { struct ovsdb *db; struct ovs_list txn_tables; /* Contains "struct ovsdb_txn_table"s. */ struct ds comment; + struct uuid txnid; /* For clustered mode only. It is the eid. */ }; /* A table modified by a transaction. */ @@ -106,13 +107,19 @@ static unsigned int serial; struct ovsdb_txn * ovsdb_txn_create(struct ovsdb *db) { - struct ovsdb_txn *txn = xmalloc(sizeof *txn); + struct ovsdb_txn *txn = xzalloc(sizeof *txn); txn->db = db; ovs_list_init(&txn->txn_tables); ds_init(&txn->comment); return txn; } +void +ovsdb_txn_set_txnid(const struct uuid *txnid, struct ovsdb_txn *txn) +{ + txn->txnid = *txnid; +} + static void ovsdb_txn_free(struct ovsdb_txn *txn) { @@ -881,11 +888,79 @@ ovsdb_txn_precommit(struct ovsdb_txn *txn) return error; } +static struct ovsdb_txn* +ovsdb_txn_clone(const struct ovsdb_txn *txn) +{ + struct ovsdb_txn *txn_cloned = xzalloc(sizeof *txn_cloned); + ovs_list_init(&txn_cloned->txn_tables); + txn_cloned->txnid = txn->txnid; + + struct ovsdb_txn_table *t; + LIST_FOR_EACH (t, node, &txn->txn_tables) { + struct ovsdb_txn_table *t_cloned = xmalloc(sizeof *t_cloned); + ovs_list_push_back(&txn_cloned->txn_tables, &t_cloned->node); + hmap_init(&t_cloned->txn_rows); + + struct ovsdb_txn_row *r; + HMAP_FOR_EACH (r, hmap_node, &t->txn_rows) { + size_t n_columns = shash_count(&t->table->schema->columns); + struct ovsdb_txn_row *r_cloned = + xzalloc(offsetof(struct ovsdb_txn_row, changed) + + bitmap_n_bytes(n_columns)); + + r_cloned->uuid = r->uuid; + r_cloned->table = r->table; + r_cloned->old = r->old ? ovsdb_row_clone(r->old) : NULL; + r_cloned->new = r->new ? ovsdb_row_clone(r->new) : NULL; + memcpy(&r_cloned->changed, &r->changed, bitmap_n_bytes(n_columns)); + hmap_insert(&t_cloned->txn_rows, &r_cloned->hmap_node, + uuid_hash(&r_cloned->uuid)); + } + } + return txn_cloned; +} + +static void +ovsdb_txn_destroy_cloned(struct ovsdb_txn *txn) +{ + ovs_assert(!txn->db); + struct ovsdb_txn_table *t, *next_txn_table; + LIST_FOR_EACH_SAFE (t, next_txn_table, node, &txn->txn_tables) { + struct ovsdb_txn_row *r, *next_txn_row; + HMAP_FOR_EACH_SAFE (r, next_txn_row, hmap_node, &t->txn_rows) { + if (r->old) { + ovsdb_row_destroy(r->old); + } + if (r->new) { + ovsdb_row_destroy(r->new); + } + hmap_remove(&t->txn_rows, &r->hmap_node); + free(r); + } + hmap_destroy(&t->txn_rows); + ovs_list_remove(&t->node); + free(t); + } + free(txn); +} + +static void +ovsdb_txn_add_to_history(struct ovsdb_txn *txn) +{ + if (txn->db->need_txn_history) { + struct ovsdb_txn_history_node *node = xzalloc(sizeof *node); + node->txn = ovsdb_txn_clone(txn); + ovs_list_push_back(&txn->db->txn_history, &node->node); + txn->db->n_txn_history++; + } +} + /* Finalize commit. */ void ovsdb_txn_complete(struct ovsdb_txn *txn) { if (!ovsdb_txn_is_empty(txn)) { + txn->db->run_triggers = true; ovsdb_monitors_commit(txn->db, txn); ovsdb_error_assert(for_each_txn_row(txn, ovsdb_txn_update_weak_refs)); @@ -906,6 +981,7 @@ ovsdb_txn_replay_commit(struct ovsdb_txn *txn) if (error) { ovsdb_txn_abort(txn); } else { + ovsdb_txn_add_to_history(txn); ovsdb_txn_complete(txn); } return error; @@ -1304,3 +1380,46 @@ for_each_txn_row(struct ovsdb_txn *txn, return NULL; } + +void +ovsdb_txn_history_run(struct ovsdb *db) +{ + if (!db->need_txn_history) { + return; + } + /* Remove old histories to limit the size of the history */ + while (db->n_txn_history > 100) { + struct ovsdb_txn_history_node *txn_h_node = CONTAINER_OF( + ovs_list_pop_front(&db->txn_history), + struct ovsdb_txn_history_node, node); + + ovsdb_txn_destroy_cloned(txn_h_node->txn); + free(txn_h_node); + db->n_txn_history--; + } +} + +void +ovsdb_txn_history_init(struct ovsdb *db) +{ + db->need_txn_history = true; + db->n_txn_history = 0; + ovs_list_init(&db->txn_history); +} + +void +ovsdb_txn_history_destroy(struct ovsdb *db) +{ + + if (!db->need_txn_history) { + return; + } + + struct ovsdb_txn_history_node *txn_h_node, *next; + LIST_FOR_EACH_SAFE (txn_h_node, next, node, &db->txn_history) { + ovs_list_remove(&txn_h_node->node); + ovsdb_txn_destroy_cloned(txn_h_node->txn); + free(txn_h_node); + } + db->n_txn_history = 0; +} diff --git a/ovsdb/transaction.h b/ovsdb/transaction.h index 32384fc..5e62ef0 100644 --- a/ovsdb/transaction.h +++ b/ovsdb/transaction.h @@ -25,6 +25,7 @@ struct ovsdb_table; struct uuid; struct ovsdb_txn *ovsdb_txn_create(struct ovsdb *); +void ovsdb_txn_set_txnid(const struct uuid *, struct ovsdb_txn *); void ovsdb_txn_abort(struct ovsdb_txn *); struct ovsdb_error *ovsdb_txn_replay_commit(struct ovsdb_txn *) @@ -59,5 +60,8 @@ void ovsdb_txn_for_each_change(const struct ovsdb_txn *, void ovsdb_txn_add_comment(struct ovsdb_txn *, const char *); const char *ovsdb_txn_get_comment(const struct ovsdb_txn *); +void ovsdb_txn_history_run(struct ovsdb *); +void ovsdb_txn_history_init(struct ovsdb *); +void ovsdb_txn_history_destroy(struct ovsdb *); #endif /* ovsdb/transaction.h */ From patchwork Thu Feb 28 17:15:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1049635 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="T3ne1t1G"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 449KFp1cvXz9s70 for ; Fri, 1 Mar 2019 04:25:50 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 96697AB8D; Thu, 28 Feb 2019 17:23:09 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 44F3CAAF8 for ; Thu, 28 Feb 2019 17:15:41 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 90EAB78C for ; Thu, 28 Feb 2019 17:15:38 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id i19so10040460pfd.0 for ; Thu, 28 Feb 2019 09:15:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=hQWH+CTXm8oyA/qNtRiApmEYzrp9ApqQyGVvXzSc8t8=; b=T3ne1t1GJxNFHV9Z/MwrvAJj7XYvy8uEBas9ccpHYF6W+inGACV4PdXKzo2QaQ6BU2 BDMR+X2SBYWxRjg0suDx6SPDNHDODbmn1cKO1AMM1qM7o4G55h+E0annK7BJ20GBYRoc 5PuRaipKmgbGwfTnqgn8fhB5QfP6LwO2qOs5aX5ykB1RLp3ra8KAvEsBlBwzeqxv4FV0 PzLwlgV6l9Vl+khsjravwdCb7UvcK0iCDLKzQ+b3JuCLpGbX+YL0ffL4UX8K4ftIXvrF LsPp4Nj7FHoEdzaxNBKKzrxWFbjxNhJidEfLXJlhxqI+xm2NOVOvWteU8BNXkDAbZKBE gGRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=hQWH+CTXm8oyA/qNtRiApmEYzrp9ApqQyGVvXzSc8t8=; b=dqYnFVvYQbRDdZ/9eqDAYEey8sSpZHE+kg91aLD/q6xHU3vYPFgNuZ09Y7uJJFk3WL EgaXtCUNFWNOy8fxJpERapG/NiPY4TApfPcPXACF0ozpcfJYwe3b/lIVIBc82Oyv23H4 2b2gZImD/thd0CnnvbzShIfExXd7DSM+QQtHZkG2wAtyJ/eV8M33TE+521ehiaLGXSRe NsI5MXI3OppQrZp7ypDF2POjMWTuCTgxgrbJlRcHePWbTx3FH8uOJLnMrjo0Q6LDX52n Gba6rKO2s62Azy7SzjROY7dKlP29T+ydI+PJjW4IRgOVs1YaeBHZ4rWIYbGpfeIJAoU+ e7Lg== X-Gm-Message-State: APjAAAUj9A0oFWp1JaUQ9KUQ/FvmTz8E/EUvGWfyw7agal+ReXTWyF5R F5vp7TIzgumXFG6HQtk26ChXXPuB X-Google-Smtp-Source: APXvYqwGBf7e6twakru19ZAmIlGhHqCNln0DIqj8OBVcp2joO6E4cNaXQVlg5LqMsgeO0ffE2iHvMQ== X-Received: by 2002:a63:43:: with SMTP id 64mr230985pga.64.1551374137169; Thu, 28 Feb 2019 09:15:37 -0800 (PST) Received: from localhost.localdomain.localdomain (c-76-21-108-74.hsd1.ca.comcast.net. [76.21.108.74]) by smtp.gmail.com with ESMTPSA id a4sm36972372pga.52.2019.02.28.09.15.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Feb 2019 09:15:36 -0800 (PST) From: Han Zhou X-Google-Original-From: Han Zhou To: dev@openvswitch.org Date: Thu, 28 Feb 2019 09:15:18 -0800 Message-Id: <1551374120-44287-4-git-send-email-hzhou8@ebay.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1551374120-44287-1-git-send-email-hzhou8@ebay.com> References: <1551374120-44287-1-git-send-email-hzhou8@ebay.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, WEIRD_QUOTING autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v4 3/5] ovsdb-monitor: Support monitor_cond_since. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Han Zhou Support the new monitor method monitor_cond_since so that a client can request monitoring start from a specific point instead of always from beginning. This will reduce the cost at scenarios when server is restarted/failed-over but client still has all existing data. In these scenarios only new changes (and in most cases no change) needed to be transfered to client. When ovsdb-server restarted, history transactions are read from disk file; when ovsdb-server failed over, history transactions exists already in the memory of the new server. There are situations that the requested transaction may not be found. For example, a transaction that is too old and has been discarded from the maintained history list in memory, or the transactions on disk has been compacted during ovsdb compaction. In those situations the server fall backs to transfer all data start from begining. For more details of the protocol change, see Documentation/ref/ovsdb-server.7.rst. This change includes both server side and ovsdb-client side changes with the new protocol. IDLs using this capability will be added in future patches. Now the feature takes effect only for cluster mode of ovsdb-server, because cluster mode is the only mode that supports unique transcation uuid today. For other modes, the monitor_cond_since always fall back to transfer all data with found = false. Support for those modes can be added in the future. Signed-off-by: Han Zhou --- Documentation/ref/ovsdb-server.7.rst | 78 ++++++++- ovsdb/jsonrpc-server.c | 85 ++++++++-- ovsdb/monitor.c | 121 +++++++++++++- ovsdb/monitor.h | 6 + ovsdb/ovsdb-client.1.in | 28 ++-- ovsdb/ovsdb-client.c | 104 +++++++++++- ovsdb/transaction.c | 6 + ovsdb/transaction.h | 1 + tests/ovsdb-monitor.at | 301 ++++++++++++++++++++++++++++++++++- 9 files changed, 690 insertions(+), 40 deletions(-) diff --git a/Documentation/ref/ovsdb-server.7.rst b/Documentation/ref/ovsdb-server.7.rst index 14c7da8..4bbe232 100644 --- a/Documentation/ref/ovsdb-server.7.rst +++ b/Documentation/ref/ovsdb-server.7.rst @@ -364,7 +364,79 @@ Initial views of rows are not presented in update2 notifications, but in the response object to the ``monitor_cond`` request. The formatting of the object, however, is the same in either case. -4.1.15 Get Server ID +4.1.15 Monitor_cond_since +------------------------- + +A new monitor method added in Open vSwitch version 2.12. The +``monitor_cond_since`` request enables a client to request changes that +happened after a specific transaction id. A client can use this feature to +request only latest changes after a server connection reset instead of +re-transfer all data from the server again. + +The ``monitor_cond`` method described in Section 4.1.12 also applies to +``monitor_cond_since``, with the following exceptions: + +* RPC request method becomes ``monitor_cond_since``. + +* Reply result includes extra parameters. + +* Subsequent changes are sent to the client using the ``update3`` monitor + notification, described in Section 4.1.16 + +The request object has the following members:: + + "method": "monitor_cond_since" + "params": [, , , ] + "id": + +The parameter is the transaction id that identifies the latest +data the client already has, and it requests server to send changes AFTER this +transaction (exclusive). + +All other parameters are the same as ``monitor_cond`` method. + +The response object has the following members:: + + "result": [, , ] + "error": null + "id": same "id" as request + +The is a boolean value that tells if the requested by +client is found in server's history or not. If true, the changes after that +version up to current is sent. Otherwise, all data is sent. + +The is the transaction id that identifies the latest transaction +included in the changes in of this response, so that client +can keep tracking. If there is no change involved in this response, it is the +same as the in the request if is true, or zero uuid if + is false. If the server does not support transaction uuid, it will +be zero uuid as well. + +All other parameters are the same as in response object of ``monitor_cond`` +method. + +Like in ``monitor_cond``, subsequent changes that match conditions in + are automatically sent to the client, but using +``update3`` monitor notification (see Section 4.1.16), instead of ``update2``. + +4.1.16 Update3 notification +--------------------------- + +The ``update3`` notification is sent by the server to the client to report +changes in tables that are being monitored following a ``monitor_cond_since`` +request as described above. The notification has the following members:: + + "method": "update3" + "params": [, , ] + "id": null + +The is the same as described in the response object of +``monitor_cond_since``. + +All other parameters are the same as in ``update2`` monitor notification (see +Section 4.1.14). + +4.1.17 Get Server ID -------------------- A new RPC method added in Open vSwitch version 2.7. The request contains the @@ -384,7 +456,7 @@ The response object contains the following members:: running OVSDB server process. A fresh UUID is generated when the process restarts. -4.1.16 Database Change Awareness +4.1.18 Database Change Awareness -------------------------------- RFC 7047 does not provide a way for a client to find out about some kinds of @@ -414,7 +486,7 @@ The reply is always the same:: "error": null "id": same "id" as request -4.1.17 Schema Conversion +4.1.19 Schema Conversion ------------------------ Open vSwitch 2.9 adds a new JSON-RPC request to convert an online database from diff --git a/ovsdb/jsonrpc-server.c b/ovsdb/jsonrpc-server.c index f9b7c27..4dda63a 100644 --- a/ovsdb/jsonrpc-server.c +++ b/ovsdb/jsonrpc-server.c @@ -986,13 +986,19 @@ ovsdb_jsonrpc_session_got_request(struct ovsdb_jsonrpc_session *s, ovsdb_jsonrpc_trigger_create(s, db, request); } } else if (!strcmp(request->method, "monitor") || - (monitor_cond_enable__ && !strcmp(request->method, - "monitor_cond"))) { + (monitor_cond_enable__ && + (!strcmp(request->method, "monitor_cond") || + !strcmp(request->method, "monitor_cond_since")))) { struct ovsdb *db = ovsdb_jsonrpc_lookup_db(s, request, &reply); if (!reply) { - int l = strlen(request->method) - strlen("monitor"); - enum ovsdb_monitor_version version = l ? OVSDB_MONITOR_V2 - : OVSDB_MONITOR_V1; + enum ovsdb_monitor_version version; + if (!strcmp(request->method, "monitor")) { + version = OVSDB_MONITOR_V1; + } else if (!strcmp(request->method, "monitor_cond")) { + version = OVSDB_MONITOR_V2; + } else { + version = OVSDB_MONITOR_V3; + } reply = ovsdb_jsonrpc_monitor_create(s, db, request->params, version, request->id); } @@ -1364,7 +1370,8 @@ ovsdb_jsonrpc_monitor_create(struct ovsdb_jsonrpc_session *s, struct ovsdb *db, struct shash_node *node; struct json *json; - if (json_array(params)->n != 3) { + if ((version == OVSDB_MONITOR_V2 && json_array(params)->n != 3) || + (version == OVSDB_MONITOR_V3 && json_array(params)->n != 4)) { error = ovsdb_syntax_error(params, NULL, "invalid parameters"); goto error; } @@ -1385,7 +1392,7 @@ ovsdb_jsonrpc_monitor_create(struct ovsdb_jsonrpc_session *s, struct ovsdb *db, m->session = s; m->db = db; m->dbmon = ovsdb_monitor_create(db, m); - if (version == OVSDB_MONITOR_V2) { + if (version == OVSDB_MONITOR_V2 || version == OVSDB_MONITOR_V3) { m->condition = ovsdb_monitor_session_condition_create(); } m->version = version; @@ -1444,9 +1451,42 @@ ovsdb_jsonrpc_monitor_create(struct ovsdb_jsonrpc_session *s, struct ovsdb *db, ovsdb_monitor_condition_bind(m->dbmon, m->condition); } - ovsdb_monitor_get_initial(m->dbmon, &m->change_set); - json = ovsdb_jsonrpc_monitor_compose_update(m, true); + bool initial = false; + if (version == OVSDB_MONITOR_V3) { + struct json *last_id = params->array.elems[3]; + if (last_id->type != JSON_STRING) { + error = ovsdb_syntax_error(last_id, NULL, + "last-txn-id must be string"); + goto error; + } + struct uuid txn_uuid; + if (!uuid_from_string(&txn_uuid, last_id->string)) { + error = ovsdb_syntax_error(last_id, NULL, + "last-txn-id must be UUID format."); + goto error; + } + if (!uuid_is_zero(&txn_uuid)) { + ovsdb_monitor_get_changes_after(&txn_uuid, m->dbmon, + &m->change_set); + } + } + if (!m->change_set) { + ovsdb_monitor_get_initial(m->dbmon, &m->change_set); + initial = true; + } + json = ovsdb_jsonrpc_monitor_compose_update(m, initial); json = json ? json : json_object_create(); + + if (m->version == OVSDB_MONITOR_V3) { + struct json *json_last_id = json_string_create_nocopy( + xasprintf(UUID_FMT, + UUID_ARGS(ovsdb_monitor_get_last_txnid( + m->dbmon)))); + + struct json *json_found = json_boolean_create(!initial); + json = json_array_create_3(json_found, json_last_id, json); + } + return jsonrpc_create_reply(json, request_id); error: @@ -1580,8 +1620,17 @@ ovsdb_jsonrpc_monitor_cond_change(struct ovsdb_jsonrpc_session *s, if (update_json) { struct jsonrpc_msg *msg; struct json *p; - - p = json_array_create_2(json_clone(m->monitor_id), update_json); + if (m->version == OVSDB_MONITOR_V3) { + struct json *json_last_id = json_string_create_nocopy( + xasprintf(UUID_FMT, + UUID_ARGS(ovsdb_monitor_get_last_txnid( + m->dbmon)))); + + p = json_array_create_3(json_clone(m->monitor_id), json_last_id, + update_json); + } else { + p = json_array_create_2(json_clone(m->monitor_id), update_json); + } msg = ovsdb_jsonrpc_create_notify(m, p); jsonrpc_session_send(s->js, msg); } @@ -1702,6 +1751,9 @@ ovsdb_jsonrpc_create_notify(const struct ovsdb_jsonrpc_monitor *m, case OVSDB_MONITOR_V2: method = "update2"; break; + case OVSDB_MONITOR_V3: + method = "update3"; + break; case OVSDB_MONITOR_VERSION_MAX: default: OVS_NOT_REACHED(); @@ -1728,8 +1780,17 @@ ovsdb_jsonrpc_monitor_flush_all(struct ovsdb_jsonrpc_session *s) if (json) { struct jsonrpc_msg *msg; struct json *params; + if (m->version == OVSDB_MONITOR_V3) { + struct json *json_last_id = json_string_create_nocopy( + xasprintf(UUID_FMT, + UUID_ARGS(ovsdb_monitor_get_last_txnid( + m->dbmon)))); + params = json_array_create_3(json_clone(m->monitor_id), + json_last_id, json); + } else { + params = json_array_create_2(json_clone(m->monitor_id), json); + } - params = json_array_create_2(json_clone(m->monitor_id), json); msg = ovsdb_jsonrpc_create_notify(m, params); jsonrpc_session_send(s->js, msg); } diff --git a/ovsdb/monitor.c b/ovsdb/monitor.c index 03be5dc..1c66b42 100644 --- a/ovsdb/monitor.c +++ b/ovsdb/monitor.c @@ -136,6 +136,9 @@ struct ovsdb_monitor_change_set { struct ovs_list change_set_for_tables; int n_refs; + + /* The previous txn id before this change set's start point. */ + struct uuid prev_txn; }; /* Contains 'struct ovsdb_monitor_row's for rows in a specific table @@ -200,7 +203,9 @@ typedef struct json * static void ovsdb_monitor_destroy(struct ovsdb_monitor *); static struct ovsdb_monitor_change_set * ovsdb_monitor_add_change_set( - struct ovsdb_monitor *, bool init_only); + struct ovsdb_monitor *, bool init_only, const struct uuid *prev_txn); +static struct ovsdb_monitor_change_set * ovsdb_monitor_find_change_set( + const struct ovsdb_monitor *, const struct uuid *prev_txn); static void ovsdb_monitor_change_set_destroy( struct ovsdb_monitor_change_set *); static void ovsdb_monitor_track_new_change_set(struct ovsdb_monitor *); @@ -540,13 +545,14 @@ ovsdb_monitor_table_exists(struct ovsdb_monitor *m, static struct ovsdb_monitor_change_set * ovsdb_monitor_add_change_set(struct ovsdb_monitor *dbmon, - bool init_only) + bool init_only, const struct uuid *prev_txn) { struct ovsdb_monitor_change_set *change_set = xzalloc(sizeof *change_set); change_set->uuid = uuid_random(); ovs_list_push_back(&(dbmon->change_sets), &change_set->list_node); ovs_list_init(&change_set->change_set_for_tables); change_set->n_refs = 1; + change_set->prev_txn = prev_txn ? *prev_txn : UUID_ZERO; struct shash_node *node; SHASH_FOR_EACH (node, &dbmon->tables) { @@ -567,6 +573,33 @@ ovsdb_monitor_add_change_set(struct ovsdb_monitor *dbmon, return change_set; }; +static struct ovsdb_monitor_change_set * +ovsdb_monitor_find_change_set(const struct ovsdb_monitor *dbmon, + const struct uuid *prev_txn) +{ + struct ovsdb_monitor_change_set *cs; + LIST_FOR_EACH (cs, list_node, &dbmon->change_sets) { + if (uuid_equals(&cs->prev_txn, prev_txn)) { + /* Check n_columns for each table in dbmon, in case it is changed + * after the change set is populated. */ + bool n_col_is_equal = true; + struct ovsdb_monitor_change_set_for_table *mcst; + LIST_FOR_EACH (mcst, list_in_change_set, + &cs->change_set_for_tables) { + struct ovsdb_monitor_table *mt = mcst->mt; + if (mt->n_columns != mcst->n_columns) { + n_col_is_equal = false; + break; + } + } + if (n_col_is_equal) { + return cs; + } + } + } + return NULL; +} + static void ovsdb_monitor_untrack_change_set(struct ovsdb_monitor *dbmon, struct ovsdb_monitor_change_set *mcs) @@ -591,7 +624,8 @@ ovsdb_monitor_track_new_change_set(struct ovsdb_monitor *dbmon) if (change_set) { change_set->n_refs++; } else { - change_set = ovsdb_monitor_add_change_set(dbmon, false); + change_set = ovsdb_monitor_add_change_set(dbmon, false, + ovsdb_monitor_get_last_txnid(dbmon)); dbmon->new_change_set = change_set; } } @@ -1190,12 +1224,13 @@ ovsdb_monitor_get_update( condition, ovsdb_monitor_compose_row_update); } else { - ovs_assert(version == OVSDB_MONITOR_V2); + ovs_assert(version == OVSDB_MONITOR_V2 || + version == OVSDB_MONITOR_V3); + if (!cond_updated) { json = ovsdb_monitor_compose_update(dbmon, initial, mcs, condition, ovsdb_monitor_compose_row_update2); - if (!condition || !condition->conditional) { ovsdb_monitor_json_cache_insert(dbmon, version, mcs, json); @@ -1434,7 +1469,7 @@ ovsdb_monitor_get_initial(struct ovsdb_monitor *dbmon, { if (!dbmon->init_change_set) { struct ovsdb_monitor_change_set *change_set = - ovsdb_monitor_add_change_set(dbmon, true); + ovsdb_monitor_add_change_set(dbmon, true, NULL); dbmon->init_change_set = change_set; struct ovsdb_monitor_change_set_for_table *mcst; @@ -1454,6 +1489,68 @@ ovsdb_monitor_get_initial(struct ovsdb_monitor *dbmon, *p_mcs = dbmon->init_change_set; } +static bool +ovsdb_monitor_history_change_cb(const struct ovsdb_row *old, + const struct ovsdb_row *new, + const unsigned long int *changed, + void *aux) +{ + struct ovsdb_monitor_change_set *change_set = aux; + struct ovsdb_table *table = new ? new->table : old->table; + struct ovsdb_monitor_change_set_for_table *mcst; + + enum ovsdb_monitor_selection type = + ovsdb_monitor_row_update_type(false, old, new); + LIST_FOR_EACH (mcst, list_in_change_set, + &change_set->change_set_for_tables) { + if (mcst->mt->table == table) { + enum ovsdb_monitor_changes_efficacy efficacy = + ovsdb_monitor_changes_classify(type, mcst->mt, changed); + if (efficacy > OVSDB_CHANGES_NO_EFFECT) { + ovsdb_monitor_changes_update(old, new, mcst->mt, mcst); + } + return true; + } + } + return false; +} + +void +ovsdb_monitor_get_changes_after(const struct uuid *txn_uuid, + struct ovsdb_monitor *dbmon, + struct ovsdb_monitor_change_set **p_mcs) +{ + ovs_assert(*p_mcs == NULL); + ovs_assert(!uuid_is_zero(txn_uuid)); + struct ovsdb_monitor_change_set *change_set = + ovsdb_monitor_find_change_set(dbmon, txn_uuid); + if (change_set) { + change_set->n_refs++; + *p_mcs = change_set; + return; + } + + struct ovsdb_txn_history_node *h_node; + bool found = false; + LIST_FOR_EACH (h_node, node, &dbmon->db->txn_history) { + struct ovsdb_txn *txn = h_node->txn; + if (!found) { + /* find the txn with last_id in history */ + if (uuid_equals(ovsdb_txn_get_txnid(txn), txn_uuid)) { + found = true; + change_set = ovsdb_monitor_add_change_set(dbmon, false, + txn_uuid); + } + } else { + /* Already found. Add changes in each follow up transaction to + * the new change_set. */ + ovsdb_txn_for_each_change(txn, ovsdb_monitor_history_change_cb, + change_set); + } + } + *p_mcs = change_set; +} + void ovsdb_monitor_remove_jsonrpc_monitor(struct ovsdb_monitor *dbmon, struct ovsdb_jsonrpc_monitor *jsonrpc_monitor, @@ -1692,3 +1789,15 @@ ovsdb_monitor_prereplace_db(struct ovsdb *db) } } } + +const struct uuid * +ovsdb_monitor_get_last_txnid(struct ovsdb_monitor *dbmon) { + static struct uuid dummy = { .parts = { 0, 0, 0, 0 } }; + if (dbmon->db->n_txn_history) { + struct ovsdb_txn_history_node *thn = CONTAINER_OF( + ovs_list_back(&dbmon->db->txn_history), + struct ovsdb_txn_history_node, node); + return ovsdb_txn_get_txnid(thn->txn); + } + return &dummy; +} diff --git a/ovsdb/monitor.h b/ovsdb/monitor.h index 112c672..1ac9aaf 100644 --- a/ovsdb/monitor.h +++ b/ovsdb/monitor.h @@ -39,6 +39,8 @@ enum ovsdb_monitor_version { OVSDB_MONITOR_V1, /* RFC 7047 "monitor" method. */ OVSDB_MONITOR_V2, /* Extension to RFC 7047, see ovsdb-server man page for details. */ + OVSDB_MONITOR_V3, /* Extension to V2, see ovsdb-server man + page for details. */ /* Last entry. */ OVSDB_MONITOR_VERSION_MAX @@ -80,6 +82,8 @@ struct json *ovsdb_monitor_get_update(struct ovsdb_monitor *, enum ovsdb_monitor_version, struct ovsdb_monitor_change_set **p_mcs); +const struct uuid *ovsdb_monitor_get_last_txnid(struct ovsdb_monitor *); + void ovsdb_monitor_table_add_select(struct ovsdb_monitor *, const struct ovsdb_table *, enum ovsdb_monitor_selection); @@ -89,6 +93,8 @@ bool ovsdb_monitor_needs_flush(struct ovsdb_monitor *, void ovsdb_monitor_get_initial(struct ovsdb_monitor *, struct ovsdb_monitor_change_set **); +void ovsdb_monitor_get_changes_after(const struct uuid *txn_uuid, + struct ovsdb_monitor *, struct ovsdb_monitor_change_set **); void ovsdb_monitor_get_memory_usage(struct simap *); diff --git a/ovsdb/ovsdb-client.1.in b/ovsdb/ovsdb-client.1.in index 46a3e38..284433b 100644 --- a/ovsdb/ovsdb-client.1.in +++ b/ovsdb/ovsdb-client.1.in @@ -44,6 +44,9 @@ ovsdb\-client \- command-line interface to \fBovsdb-server\fR(1) \fBovsdb\-client\fR [\fIoptions\fR] \fBmonitor\-cond\fR [\fIserver\fR] [\fIdatabase\fR] \fIconditions \fItable\fR [\fIcolumn\fR[\fB,\fIcolumn\fR]...]... .br +\fBovsdb\-client\fR [\fIoptions\fR] \fBmonitor\-cond\-since\fR [\fIserver\fR] [\fIdatabase\fR] +[\fIlast-id\fR] \fIconditions \fItable\fR [\fIcolumn\fR[\fB,\fIcolumn\fR]...]... +.br \fBovsdb\-client \fR[\fIoptions\fR] \fBwait\fR \fR[\fIserver\fR] \fIdatabase\fR \fIstate\fR .IP "Testing Commands:" \fBovsdb\-client\fR [\fIoptions\fR] \fBlock\fR [\fIserver\fR] \fIlock\fR @@ -243,6 +246,7 @@ with an error or succeed with surprising results. . .IP "\fBmonitor\fR [\fIserver\fR] [\fIdatabase\fR] \fItable\fR [\fIcolumn\fR[\fB,\fIcolumn\fR]...]..." .IQ "\fBmonitor\-cond\fR [\fIserver\fR] [\fIdatabase\fR] \fIconditions\fR \fItable\fR [\fIcolumn\fR[\fB,\fIcolumn\fR]...]..." +.IQ "\fBmonitor\-cond\-since\fR [\fIserver\fR] [\fIdatabase\fR] [\fIlast-id\fR] \fIconditions\fR \fItable\fR [\fIcolumn\fR[\fB,\fIcolumn\fR]...]..." Connects to \fIserver\fR and monitors the contents of rows that match conditions in \fItable\fR in \fIdatabase\fR. By default, the initial contents of \fItable\fR are printed, followed by each change as it occurs. If conditions empty, @@ -269,13 +273,14 @@ line. with the following change: A condition can be either a 3-element JSON array as deescribed in the RFC or a boolean value.. .IP -If \fB\-\-detach\fR is used with \fBmonitor\fR or \fBmonitor\-cond\fR, then -\fBovsdb\-client\fR detaches after it has successfully received and -printed the initial contents of \fItable\fR. +If \fB\-\-detach\fR is used with \fBmonitor\fR, \fBmonitor\-cond\fR or +\fBmonitor\-cond\-since\fR, then \fBovsdb\-client\fR detaches after it has +successfully received and printed the initial contents of \fItable\fR. .IP The \fBmonitor\fR command uses RFC 7047 "monitor" method to open a monitor -session with the server. The \fBmonitor\-cond\fR command uses RFC 7047 -extension "monitor_cond" method. See \fBovsdb\-server\fR(1) for details. +session with the server. The \fBmonitor\-cond\fR and \fBmonitor\-cond\-since\fR +commandls uses RFC 7047 extension "monitor_cond" and "monitor_cond_since" +methods. See \fBovsdb\-server\fR(1) for details. .IP "\fBmonitor\fI \fR[\fIserver\fR] \fR[\fIdatabase\fR] \fBALL\fR" Connects to \fIserver\fR and monitors the contents of all tables in \fIdatabase\fR. Prints initial values and all kinds of changes to all @@ -340,10 +345,10 @@ The following options controlling output formatting: .so lib/table.man . .IP "\fB\-\-timestamp\fR" -For the \fBmonitor\fR and \fBmonitor\-cond\fR commands, add a timestamp to each -table update. Most output formats add the timestamp on a line of its own -just above the table. The JSON output format puts the timestamp in a -member of the top-level JSON object named \fBtime\fR. +For the \fBmonitor\fR, \fBmonitor\-cond\fR and \fBmonitor\-cond\-since\fR +commands, add a timestamp to each table update. Most output formats add the +timestamp on a line of its own just above the table. The JSON output format +puts the timestamp in a member of the top-level JSON object named \fBtime\fR. . .IP "\fB\-t\fR" .IQ "\fB\-\-timeout=\fIsecs\fR" @@ -352,8 +357,9 @@ seconds. If the timeout expires, \fBovsdb\-client\fR will exit with a \fBSIGALRM\fR signal. . .SS "Daemon Options" -The daemon options apply only to the \fBmonitor\fR and \fBmonitor\-cond\fR commands. -With any other command, they have no effect. +The daemon options apply only to the \fBmonitor\fR, \fBmonitor\-cond\fR and +\fBmonitor\-cond\-since\fR commands. With any other command, they have no +effect. .ds DD .so lib/daemon.man .SS "Logging Options" diff --git a/ovsdb/ovsdb-client.c b/ovsdb/ovsdb-client.c index 0215357..9ae15e5 100644 --- a/ovsdb/ovsdb-client.c +++ b/ovsdb/ovsdb-client.c @@ -430,6 +430,13 @@ usage(void) " DATABASE on SERVER.\n" " COLUMNs may include !initial, !insert, !delete, !modify\n" " to avoid seeing the specified kinds of changes.\n" + "\n monitor-cond-since [SERVER] [DATABASE] [LASTID] CONDITION TABLE [COLUMN,...]...\n" + " monitor contents that match CONDITION of COLUMNs in TABLE in\n" + " DATABASE on SERVER, since change after LASTID.\n" + " LASTID specifies transaction ID after which the monitoring\n" + " starts, which works only for cluster mode. If ignored, it\n" + " defaults to an all-zero uuid.\n" + " Other arguments are the same as in monitor-cond.\n" "\n convert [SERVER] SCHEMA\n" " convert database on SERVER named in SCHEMA to SCHEMA.\n" "\n needs-conversion [SERVER] SCHEMA\n" @@ -1127,6 +1134,35 @@ monitor2_print(struct json *table_updates2, } static void +monitor3_print(struct json *result, + const struct monitored_table *mts, size_t n_mts) +{ + if (result->type != JSON_ARRAY) { + ovs_error(0, " is not array"); + } + + if (result->array.n != 3) { + ovs_error(0, " should have 3 elements, but has %"PRIuSIZE".", + result->array.n); + } + + bool found = json_boolean(result->array.elems[0]); + const char *last_id = json_string(result->array.elems[1]); + printf("found: %s, last_id: %s\n", found ? "true" : "false", last_id); + + struct json *table_updates2 = result->array.elems[2]; + monitor2_print(table_updates2, mts, n_mts); +} + +static void +monitor3_notify_print(const char *last_id, struct json *table_updates2, + const struct monitored_table *mts, size_t n_mts) +{ + printf("\nlast_id: %s", last_id); + monitor2_print(table_updates2, mts, n_mts); +} + +static void add_column(const char *server, const struct ovsdb_column *column, struct ovsdb_column_set *columns, struct json *columns_json) { @@ -1333,7 +1369,8 @@ destroy_monitored_table(struct monitored_table *mts, size_t n) static void do_monitor__(struct jsonrpc *rpc, const char *database, enum ovsdb_monitor_version version, - int argc, char *argv[], struct json *condition) + int argc, char *argv[], struct json *condition, + const struct uuid *last_id) { const char *server = jsonrpc_get_name(rpc); const char *table_name = argv[0]; @@ -1411,8 +1448,24 @@ do_monitor__(struct jsonrpc *rpc, const char *database, monitor = json_array_create_3(json_string_create(database), json_null_create(), monitor_requests); - const char *method = version == OVSDB_MONITOR_V2 ? "monitor_cond" - : "monitor"; + const char *method; + switch (version) { + case OVSDB_MONITOR_V1: + method = "monitor"; + break; + case OVSDB_MONITOR_V2: + method = "monitor_cond"; + break; + case OVSDB_MONITOR_V3: + method = "monitor_cond_since"; + struct json *json_last_id = json_string_create_nocopy( + xasprintf(UUID_FMT, UUID_ARGS(last_id))); + json_array_add(monitor, json_last_id); + break; + case OVSDB_MONITOR_VERSION_MAX: + default: + OVS_NOT_REACHED(); + } struct jsonrpc_msg *request; request = jsonrpc_create_request(method, monitor, NULL); @@ -1444,6 +1497,9 @@ do_monitor__(struct jsonrpc *rpc, const char *database, case OVSDB_MONITOR_V2: monitor2_print(msg->result, mts, n_mts); break; + case OVSDB_MONITOR_V3: + monitor3_print(msg->result, mts, n_mts); + break; case OVSDB_MONITOR_VERSION_MAX: default: OVS_NOT_REACHED(); @@ -1470,6 +1526,17 @@ do_monitor__(struct jsonrpc *rpc, const char *database, fflush(stdout); } } else if (msg->type == JSONRPC_NOTIFY + && version == OVSDB_MONITOR_V3 + && !strcmp(msg->method, "update3")) { + struct json *params = msg->params; + if (params->type == JSON_ARRAY + && params->array.n == 3 + && params->array.elems[0]->type == JSON_NULL) { + monitor3_notify_print(json_string(params->array.elems[1]), + params->array.elems[2], mts, n_mts); + fflush(stdout); + } + } else if (msg->type == JSONRPC_NOTIFY && !strcmp(msg->method, "monitor_canceled")) { ovs_fatal(0, "%s: %s database was removed", server, database); @@ -1500,12 +1567,13 @@ static void do_monitor(struct jsonrpc *rpc, const char *database, int argc, char *argv[]) { - do_monitor__(rpc, database, OVSDB_MONITOR_V1, argc, argv, NULL); + do_monitor__(rpc, database, OVSDB_MONITOR_V1, argc, argv, NULL, NULL); } static void -do_monitor_cond(struct jsonrpc *rpc, const char *database, - int argc, char *argv[]) +do_monitor_cond__(struct jsonrpc *rpc, const char *database, + enum ovsdb_monitor_version version, + struct uuid *last_id, int argc, char *argv[]) { struct ovsdb_condition cnd; struct json *condition = NULL; @@ -1524,10 +1592,31 @@ do_monitor_cond(struct jsonrpc *rpc, const char *database, check_ovsdb_error(ovsdb_condition_from_json(table, condition, NULL, &cnd)); ovsdb_condition_destroy(&cnd); - do_monitor__(rpc, database, OVSDB_MONITOR_V2, --argc, ++argv, condition); + do_monitor__(rpc, database, version, --argc, ++argv, condition, + last_id); ovsdb_schema_destroy(schema); } +static void +do_monitor_cond(struct jsonrpc *rpc, const char *database, + int argc, char *argv[]) +{ + do_monitor_cond__(rpc, database, OVSDB_MONITOR_V2, NULL, argc, argv); +} + +static void +do_monitor_cond_since(struct jsonrpc *rpc, const char *database, + int argc, char *argv[]) +{ + ovs_assert(argc > 1); + struct uuid last_id; + if (uuid_from_string(&last_id, argv[0])) { + argc--; + argv++; + } + do_monitor_cond__(rpc, database, OVSDB_MONITOR_V3, &last_id, argc, argv); +} + static bool is_database_clustered(struct jsonrpc *rpc, const char *database) { @@ -2409,6 +2498,7 @@ static const struct ovsdb_client_command all_commands[] = { { "query", NEED_NONE, 1, 2, do_query }, { "monitor", NEED_DATABASE, 1, INT_MAX, do_monitor }, { "monitor-cond", NEED_DATABASE, 2, 3, do_monitor_cond }, + { "monitor-cond-since", NEED_DATABASE, 2, 4, do_monitor_cond_since }, { "wait", NEED_NONE, 2, 3, do_wait }, { "convert", NEED_NONE, 1, 2, do_convert }, { "needs-conversion", NEED_NONE, 1, 2, do_needs_conversion }, diff --git a/ovsdb/transaction.c b/ovsdb/transaction.c index 148d108..9fc1fd7 100644 --- a/ovsdb/transaction.c +++ b/ovsdb/transaction.c @@ -120,6 +120,12 @@ ovsdb_txn_set_txnid(const struct uuid *txnid, struct ovsdb_txn *txn) txn->txnid = *txnid; } +const struct uuid * +ovsdb_txn_get_txnid(const struct ovsdb_txn *txn) +{ + return &txn->txnid; +} + static void ovsdb_txn_free(struct ovsdb_txn *txn) { diff --git a/ovsdb/transaction.h b/ovsdb/transaction.h index 5e62ef0..c819373 100644 --- a/ovsdb/transaction.h +++ b/ovsdb/transaction.h @@ -26,6 +26,7 @@ struct uuid; struct ovsdb_txn *ovsdb_txn_create(struct ovsdb *); void ovsdb_txn_set_txnid(const struct uuid *, struct ovsdb_txn *); +const struct uuid *ovsdb_txn_get_txnid(const struct ovsdb_txn *); void ovsdb_txn_abort(struct ovsdb_txn *); struct ovsdb_error *ovsdb_txn_replay_commit(struct ovsdb_txn *) diff --git a/tests/ovsdb-monitor.at b/tests/ovsdb-monitor.at index f6e21d0..84aa208 100644 --- a/tests/ovsdb-monitor.at +++ b/tests/ovsdb-monitor.at @@ -595,8 +595,8 @@ AT_SETUP(monitor-cond-change with many sessions pending) AT_KEYWORDS([ovsdb server monitor monitor-cond negative]) ordinal_schema > schema AT_CHECK([ovsdb-tool create db schema], [0], [stdout], [ignore]) - AT_CAPTURE_FILE([ovsdb-server-log]) + AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --remote=punix:socket --log-file="`pwd`"/ovsdb-server-log db >/dev/null 2>&1]) on_exit 'kill `cat ovsdb-server.pid`' for txn in m4_foreach([txn], [[[["ordinals", @@ -657,3 +657,302 @@ row,action,name <1>,insert,"""one""" ]], [ignore]) AT_CLEANUP + + +# Test monitor-cond-since with zero uuid, which shouldn't +# be found in server and server should send all rows +# as initial. +AT_SETUP([monitor-cond-since not found]) +AT_KEYWORDS([ovsdb server monitor monitor-cond-since positive]) +ordinal_schema > schema +AT_CHECK([ovsdb-tool create-cluster db schema unix:db.raft], [0], [stdout], [ignore]) +AT_CAPTURE_FILE([ovsdb-server-log]) +AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --remote=punix:socket --log-file="`pwd`"/ovsdb-server-log db >/dev/null 2>&1]) +on_exit 'kill `cat ovsdb-server.pid`' +for txn in m4_foreach([txn], [[[["ordinals", + {"op": "insert", + "table": "ordinals", + "row": {"number": 0, "name": "zero"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 1, "name": "one"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 2, "name": "two"}}]]]], ['txn' ]); do + AT_CHECK([ovsdb-client transact unix:socket "$txn"], [0], [ignore], [ignore]) +done + +# Omitting the last_id parameter in ovsdb-client monitor-cond-since command +# will by default using all zero uuid, which doesn't exist in any history txn. +AT_CHECK([ovsdb-client -vjsonrpc --pidfile --detach --no-chdir -d json monitor-cond-since --format=csv unix:socket ordinals '[[["name","==","one"],["name","==","ten"]]]' ordinals > output], + [0], [ignore], [ignore]) +on_exit 'kill `cat ovsdb-client.pid`' +for txn in m4_foreach([txn], [[[["ordinals", + {"op": "insert", + "table": "ordinals", + "row": {"number": 10, "name": "ten"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 11, "name": "eleven"}}]]]], ['txn' ]); do + AT_CHECK([ovsdb-client transact unix:socket "$txn"], [0], + [ignore], [ignore], [kill `cat server-pid client-pid`]) +done +AT_CHECK([ovsdb-client transact unix:socket '[["ordinals"]]'], [0], + [ignore], [ignore]) +AT_CHECK([ovs-appctl -t ovsdb-server -e exit], [0], [ignore], [ignore]) +OVS_WAIT_UNTIL([test ! -e ovsdb-server.pid && test ! -e ovsdb-client.pid]) +AT_CHECK([$PYTHON $srcdir/ovsdb-monitor-sort.py < output | uuidfilt], [0], + [[found: false, last_id: <0> +row,action,name,number,_version +<1>,initial,"""one""",1,"[""uuid"",""<2>""]" + +last_id: <3> +row,action,name,number,_version +<4>,insert,"""ten""",10,"[""uuid"",""<5>""]" +]], [ignore]) +AT_CLEANUP + + +# Test monitor-cond-since in ovsdb server restart scenario. +# ovsdb-client should receive only new changes after the +# specific transaction id. +AT_SETUP([monitor-cond-since db restart]) +AT_KEYWORDS([ovsdb server monitor monitor-cond-since positive]) +ordinal_schema > schema +AT_CHECK([ovsdb-tool create-cluster db schema unix:db.raft], [0], [stdout], [ignore]) +AT_CAPTURE_FILE([ovsdb-server-log]) +AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --remote=punix:socket --log-file="`pwd`"/ovsdb-server-log db >/dev/null 2>&1]) +on_exit 'kill `cat ovsdb-server.pid`' +for txn in m4_foreach([txn], [[[["ordinals", + {"op": "insert", + "table": "ordinals", + "row": {"number": 0, "name": "zero"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 1, "name": "one"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 2, "name": "two"}}]]]], ['txn' ]); do + AT_CHECK([ovsdb-client transact unix:socket "$txn"], [0], [ignore], [ignore]) +done + +AT_CHECK([ovsdb-client -vjsonrpc --pidfile --detach --no-chdir -d json monitor-cond-since --format=csv unix:socket ordinals '[[["name","==","one"],["name","==","ten"]]]' ordinals > output], + [0], [ignore], [ignore]) +on_exit 'kill `cat ovsdb-client.pid`' +OVS_WAIT_UNTIL([grep last_id output]) + +kill `cat ovsdb-client.pid` +kill `cat ovsdb-server.pid` +OVS_WAIT_UNTIL([test ! -e ovsdb-server.pid && test ! -e ovsdb-client.pid]) + +# Remember the last_id, which will be used for monitor-cond-since later. +last_id=`grep last_id output | awk '{print $4}'` + +AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --remote=punix:socket --log-file="`pwd`"/ovsdb-server-log db >/dev/null 2>&1]) + +# Some new changes made to db after restarting the server. +for txn in m4_foreach([txn], [[[["ordinals", + {"op": "insert", + "table": "ordinals", + "row": {"number": 10, "name": "ten"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 11, "name": "eleven"}}]]]], ['txn' ]); do + AT_CHECK([ovsdb-client transact unix:socket "$txn"], [0], + [ignore], [ignore], [kill `cat server-pid client-pid`]) +done + +# Use last_id to monitor and get only the new changes. +AT_CHECK([ovsdb-client -vjsonrpc --pidfile --detach --no-chdir -d json monitor-cond-since --format=csv unix:socket ordinals $last_id '[[["name","==","one"],["name","==","ten"]]]' ordinals > output], + [0], [ignore], [ignore]) + +AT_CHECK([ovsdb-client transact unix:socket '[["ordinals"]]'], [0], + [ignore], [ignore]) +AT_CHECK([ovs-appctl -t ovsdb-server -e exit], [0], [ignore], [ignore]) +OVS_WAIT_UNTIL([test ! -e ovsdb-server.pid && test ! -e ovsdb-client.pid]) +AT_CHECK([$PYTHON $srcdir/ovsdb-monitor-sort.py < output | uuidfilt], [0], + [[found: true, last_id: <0> +row,action,name,number,_version +<1>,insert,"""ten""",10,"[""uuid"",""<2>""]" +]], [ignore]) +AT_CLEANUP + + +# Test monitor-cond-since with last_id found in server +# but there is no new change after that transaction. +AT_SETUP([monitor-cond-since found but no new rows]) +AT_KEYWORDS([ovsdb server monitor monitor-cond-since positive]) +ordinal_schema > schema +AT_CHECK([ovsdb-tool create-cluster db schema unix:db.raft], [0], [stdout], [ignore]) +AT_CAPTURE_FILE([ovsdb-server-log]) +AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --remote=punix:socket --log-file="`pwd`"/ovsdb-server-log db >/dev/null 2>&1]) +on_exit 'kill `cat ovsdb-server.pid`' +for txn in m4_foreach([txn], [[[["ordinals", + {"op": "insert", + "table": "ordinals", + "row": {"number": 0, "name": "zero"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 1, "name": "one"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 2, "name": "two"}}]]]], ['txn' ]); do + AT_CHECK([ovsdb-client transact unix:socket "$txn"], [0], [ignore], [ignore]) +done +AT_CHECK([ovsdb-client -vjsonrpc --pidfile --detach --no-chdir -d json monitor-cond-since --format=csv unix:socket ordinals '[[["name","==","one"],["name","==","ten"]]]' ordinals > output], + [0], [ignore], [ignore]) +on_exit 'kill `cat ovsdb-client.pid`' +OVS_WAIT_UNTIL([grep last_id output]) + +kill `cat ovsdb-client.pid` +OVS_WAIT_UNTIL([test ! -e ovsdb-client.pid]) +last_id=`grep last_id output | awk '{print $4}'` +AT_CHECK([ovsdb-client -vjsonrpc --pidfile --detach --no-chdir -d json monitor-cond-since --format=csv unix:socket ordinals $last_id '[[["name","==","one"],["name","==","ten"]]]' ordinals > output], + [0], [ignore], [ignore]) + +AT_CHECK([ovsdb-client transact unix:socket '[["ordinals"]]'], [0], + [ignore], [ignore]) +AT_CHECK([ovs-appctl -t ovsdb-server -e exit], [0], [ignore], [ignore]) +OVS_WAIT_UNTIL([test ! -e ovsdb-server.pid && test ! -e ovsdb-client.pid]) +AT_CHECK([$PYTHON $srcdir/ovsdb-monitor-sort.py < output | uuidfilt], [0], + [[found: true, last_id: <0> +]], [ignore]) +AT_CLEANUP + + +# Test monitor-cond-since against empty DB +AT_SETUP([monitor-cond-since empty db]) +AT_KEYWORDS([ovsdb server monitor monitor-cond-since positive]) +ordinal_schema > schema +AT_CHECK([ovsdb-tool create-cluster db schema unix:db.raft], [0], [stdout], [ignore]) +AT_CAPTURE_FILE([ovsdb-server-log]) +AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --remote=punix:socket --log-file="`pwd`"/ovsdb-server-log db >/dev/null 2>&1]) +on_exit 'kill `cat ovsdb-server.pid`' +AT_CHECK([ovsdb-client -vjsonrpc --pidfile --detach --no-chdir -d json monitor-cond-since --format=csv unix:socket ordinals '[[["name","==","one"],["name","==","ten"]]]' ordinals > output], + [0], [ignore], [ignore]) +on_exit 'kill `cat ovsdb-client.pid`' +OVS_WAIT_UNTIL([grep last_id output]) + +AT_CHECK([ovsdb-client transact unix:socket '[["ordinals"]]'], [0], + [ignore], [ignore]) +AT_CHECK([ovs-appctl -t ovsdb-server -e exit], [0], [ignore], [ignore]) +OVS_WAIT_UNTIL([test ! -e ovsdb-server.pid && test ! -e ovsdb-client.pid]) +AT_CHECK([$PYTHON $srcdir/ovsdb-monitor-sort.py < output | uuidfilt], [0], + [[found: false, last_id: <0> +]], [ignore]) +AT_CLEANUP + + +# Test monitor-cond-since with cond-change followed. +AT_SETUP([monitor-cond-since condition change]) +AT_KEYWORDS([ovsdb server monitor monitor-cond-since positive]) +ordinal_schema > schema +AT_CHECK([ovsdb-tool create-cluster db schema unix:db.raft], [0], [stdout], [ignore]) +AT_CAPTURE_FILE([ovsdb-server-log]) +AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --remote=punix:socket --log-file="`pwd`"/ovsdb-server-log db >/dev/null 2>&1]) +on_exit 'kill `cat ovsdb-server.pid`' +for txn in m4_foreach([txn], [[[["ordinals", + {"op": "insert", + "table": "ordinals", + "row": {"number": 0, "name": "zero"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 1, "name": "one"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 2, "name": "two"}}]]]], ['txn' ]); do + AT_CHECK([ovsdb-client transact unix:socket "$txn"], [0], [ignore], [ignore]) +done +AT_CHECK([ovsdb-client -vjsonrpc --pidfile --detach --no-chdir -d json monitor-cond-since --format=csv unix:socket ordinals '[[]]' ordinals > output], [0], [ignore], [ignore]) +on_exit 'kill `cat ovsdb-client.pid`' +for cond in m4_foreach([cond], + [[[[["name","==","one"],["name","==","two"]]]], + [[[["name","==","one"]]]], + [[[false]]], + [[[true]]]], ['cond' ]); do + AT_CHECK([ovs-appctl -t ovsdb-client ovsdb-client/cond_change ordinals "$cond"], [0], [ignore], [ignore]) +done +AT_CHECK([ovsdb-client transact unix:socket '[["ordinals"]]'], [0], + [ignore], [ignore]) +AT_CHECK([ovs-appctl -t ovsdb-server -e exit], [0], [ignore], [ignore]) +OVS_WAIT_UNTIL([test ! -e ovsdb-server.pid && test ! -e ovsdb-client.pid]) +AT_CHECK([$PYTHON $srcdir/ovsdb-monitor-sort.py < output | uuidfilt], [0], + [[found: false, last_id: <0> +row,action,name,number,_version +<1>,initial,"""one""",1,"[""uuid"",""<2>""]" +<3>,initial,"""two""",2,"[""uuid"",""<4>""]" +<5>,initial,"""zero""",,"[""uuid"",""<6>""]" + +last_id: <0> +row,action,name,number,_version +<5>,delete,,, + +last_id: <0> +row,action,name,number,_version +<3>,delete,,, + +last_id: <0> +row,action,name,number,_version +<1>,delete,,, + +last_id: <0> +row,action,name,number,_version +<1>,insert,"""one""",1,"[""uuid"",""<2>""]" +<3>,insert,"""two""",2,"[""uuid"",""<4>""]" +<5>,insert,"""zero""",,"[""uuid"",""<6>""]" +]], [ignore]) +AT_CLEANUP + + +# Test monitor-cond-since with non-cluster mode server +AT_SETUP([monitor-cond-since non-cluster]) +AT_KEYWORDS([ovsdb server monitor monitor-cond-since positive]) +ordinal_schema > schema +AT_CHECK([ovsdb-tool create db schema], [0], [stdout], [ignore]) +AT_CAPTURE_FILE([ovsdb-server-log]) +AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --remote=punix:socket --log-file="`pwd`"/ovsdb-server-log db >/dev/null 2>&1]) +on_exit 'kill `cat ovsdb-server.pid`' +for txn in m4_foreach([txn], [[[["ordinals", + {"op": "insert", + "table": "ordinals", + "row": {"number": 0, "name": "zero"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 1, "name": "one"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 2, "name": "two"}}]]]], ['txn' ]); do + AT_CHECK([ovsdb-client transact unix:socket "$txn"], [0], [ignore], [ignore]) +done + +AT_CHECK([ovsdb-client -vjsonrpc --pidfile --detach --no-chdir -d json monitor-cond-since --format=csv unix:socket ordinals '[[["name","==","one"],["name","==","ten"]]]' ordinals > output], + [0], [ignore], [ignore]) +on_exit 'kill `cat ovsdb-client.pid`' +for txn in m4_foreach([txn], [[[["ordinals", + {"op": "insert", + "table": "ordinals", + "row": {"number": 10, "name": "ten"}}, + {"op": "insert", + "table": "ordinals", + "row": {"number": 11, "name": "eleven"}}]]]], ['txn' ]); do + AT_CHECK([ovsdb-client transact unix:socket "$txn"], [0], + [ignore], [ignore], [kill `cat server-pid client-pid`]) +done +AT_CHECK([ovsdb-client transact unix:socket '[["ordinals"]]'], [0], + [ignore], [ignore]) +AT_CHECK([ovs-appctl -t ovsdb-server -e exit], [0], [ignore], [ignore]) +OVS_WAIT_UNTIL([test ! -e ovsdb-server.pid && test ! -e ovsdb-client.pid]) + +# Transaction shouldn't be found, and last_id returned should always +# be the same (all zero uuid) +AT_CHECK([$PYTHON $srcdir/ovsdb-monitor-sort.py < output | uuidfilt], [0], + [[found: false, last_id: <0> +row,action,name,number,_version +<1>,initial,"""one""",1,"[""uuid"",""<2>""]" + +last_id: <0> +row,action,name,number,_version +<3>,insert,"""ten""",10,"[""uuid"",""<4>""]" +]], [ignore]) +AT_CLEANUP + From patchwork Thu Feb 28 17:15:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1049633 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="arDCsoMf"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 449KDh2qwvz9s4V for ; Fri, 1 Mar 2019 04:24:52 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 0B277AB80; Thu, 28 Feb 2019 17:23:08 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id DC208AAF8 for ; Thu, 28 Feb 2019 17:15:39 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id D045F806 for ; Thu, 28 Feb 2019 17:15:38 +0000 (UTC) Received: by mail-pl1-f195.google.com with SMTP id d15so10072994plr.1 for ; Thu, 28 Feb 2019 09:15:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=v4FqeuY5TjkXg+yZHYPg1xN/vf3hI/sZTN4A9Zsz6JM=; b=arDCsoMfEAPm2LhdNqHhKYxeztNbLsCvjpD7fad1BTqbwbSX/MmAOkBkqBwTXYrIM6 SWn7uW/PN9pZ1edvqX2Dl8KVSg0/xhMCer/CYSlhdDJuRtfH7BMyzym8VMr1vJGmM0+j 0/WZwKKtIDGQ1OeUrLVSG1Zc067SKU/1YNG0rs25Y88R/oRwOSXgXVtjpDNW62JEVe1w iA9FCHRrZhPY7/RQma8GqJ1iHHfXdSrc3R9b/qFkZZKF7IM/L4jxe8/KJ7FNSzE4yftI VTgv/8Ut86rxXnbkEgRFnrD6qmzVDKwX2VTR94dB1nlnM+pWSHyIUzHc0XvVDHoITKRU ZJsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=v4FqeuY5TjkXg+yZHYPg1xN/vf3hI/sZTN4A9Zsz6JM=; b=kPNGJD3teWVhmgS0jlmCbMzyUnk4AxKImVn5fOinj+1CuHiH8wYBeA8AurdhT1b8HD qM6AhVWmaOq3OE0GvvMwDU9JWrQLJiLKLflxI/lLdRsMrIOfXUtZNYcY+jZm9YY9q/ma G76bgHYAZtRvy8gu4Tj2w3F6pOeq6126ct9QmSJmSc4cyU7WmYjdcju4oDI+IpB7uSZz THWjB6/IxhHunsLsMy0CZ0v5dt49gDJae3w2vlGkRqv5DftMrMU/w2qhx+IWcTXCldYW r0/ou0ercURwlSqjzv9lmVVeovT27EYxU9l94e9TVxjmjDHzFPBbdTb7gnLRvcAh9Xh2 UprQ== X-Gm-Message-State: APjAAAViVq5w5TvjMqM/FPY3PYI/j1rtA3i5B/xJGpPHJvTCGTmR+6Ay Kqre3rsSMvfmVxlqgSf6wznuwnRp X-Google-Smtp-Source: APXvYqy731anBm7AIURWcRJU7DuppY64WsODZLX90GGl/kjlLOp+a3Z0/bySrak2sex3f6rvHGG17g== X-Received: by 2002:a17:902:28e9:: with SMTP id f96mr400212plb.169.1551374137943; Thu, 28 Feb 2019 09:15:37 -0800 (PST) Received: from localhost.localdomain.localdomain (c-76-21-108-74.hsd1.ca.comcast.net. [76.21.108.74]) by smtp.gmail.com with ESMTPSA id a4sm36972372pga.52.2019.02.28.09.15.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Feb 2019 09:15:37 -0800 (PST) From: Han Zhou X-Google-Original-From: Han Zhou To: dev@openvswitch.org Date: Thu, 28 Feb 2019 09:15:19 -0800 Message-Id: <1551374120-44287-5-git-send-email-hzhou8@ebay.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1551374120-44287-1-git-send-email-hzhou8@ebay.com> References: <1551374120-44287-1-git-send-email-hzhou8@ebay.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v4 4/5] ovsdb-idl.c: Support monitor_cond_since method in C IDL. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Han Zhou Use monitor_cond_since in C IDL. If it is not supported by server, fall back to old method (monitor_cond, or monitor). Signed-off-by: Han Zhou --- lib/ovsdb-idl.c | 203 ++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 155 insertions(+), 48 deletions(-) diff --git a/lib/ovsdb-idl.c b/lib/ovsdb-idl.c index a7274de..5d86046 100644 --- a/lib/ovsdb-idl.c +++ b/lib/ovsdb-idl.c @@ -106,10 +106,10 @@ struct ovsdb_idl_arc { * connection. The next connection attempt has a chance at \ * picking a connected server. \ * \ - * * Otherwise, sends a "monitor_cond" request for the IDL \ + * * Otherwise, sends a "monitor_cond_since" request for the IDL \ * database whose details are informed by the schema \ * (obtained from the row), and transitions to \ - * IDL_S_DATA_MONITOR_COND_REQUESTED. \ + * IDL_S_DATA_MONITOR_COND_SINCE_REQUESTED. \ * \ * - If the reply indicates success, but the Database table does \ * not have a row for the IDL database, transitions to \ @@ -125,6 +125,13 @@ struct ovsdb_idl_arc { * transitions to IDL_S_DATA_MONITOR_COND_REQUESTED. */ \ OVSDB_IDL_STATE(DATA_SCHEMA_REQUESTED) \ \ + /* Waits for "monitor_cond_since" reply. If successful, replaces \ + * the IDL contents by the data carried in the reply and \ + * transitions to IDL_S_MONITORING. On failure, sends a \ + * "monitor_cond" request and transitions to \ + * IDL_S_DATA_MONITOR_COND_REQUESTED. */ \ + OVSDB_IDL_STATE(DATA_MONITOR_COND_SINCE_REQUESTED) \ + \ /* Waits for "monitor_cond" reply. If successful, replaces the \ * IDL contents by the data carried in the reply and transitions \ * to IDL_S_MONITORING. On failure, sends a "monitor" request \ @@ -136,9 +143,9 @@ struct ovsdb_idl_arc { * IDL_S_MONITORING. On failure, transitions to IDL_S_ERROR. */ \ OVSDB_IDL_STATE(DATA_MONITOR_REQUESTED) \ \ - /* State that processes "update" or "update2" notifications for \ - * the main database (and the Database table in _Server if \ - * available). \ + /* State that processes "update", "update2" or "update3" \ + * notifications for the main database (and the Database table \ + * in _Server if available). \ * \ * If we're monitoring the Database table and we get notified \ * that the IDL database has been deleted, we close the \ @@ -167,10 +174,18 @@ enum ovsdb_idl_state { static const char *ovsdb_idl_state_to_string(enum ovsdb_idl_state); +enum ovsdb_idl_monitor_method { + OVSDB_IDL_MM_MONITOR, + OVSDB_IDL_MM_MONITOR_COND, + OVSDB_IDL_MM_MONITOR_COND_SINCE +}; + enum ovsdb_idl_monitoring { OVSDB_IDL_NOT_MONITORING, /* Database is not being monitored. */ OVSDB_IDL_MONITORING, /* Database has "monitor" outstanding. */ OVSDB_IDL_MONITORING_COND, /* Database has "monitor_cond" outstanding. */ + OVSDB_IDL_MONITORING_COND_SINCE, /* Database has "monitor_cond_since" + outstanding. */ }; struct ovsdb_idl_db { @@ -220,7 +235,7 @@ static void ovsdb_idl_send_db_change_aware(struct ovsdb_idl *); static bool ovsdb_idl_check_server_db(struct ovsdb_idl *); static void ovsdb_idl_send_monitor_request(struct ovsdb_idl *, struct ovsdb_idl_db *, - bool use_monitor_cond); + enum ovsdb_idl_monitor_method); struct ovsdb_idl { struct ovsdb_idl_db server; @@ -287,8 +302,8 @@ static struct vlog_rate_limit other_rl = VLOG_RATE_LIMIT_INIT(1, 5); static void ovsdb_idl_clear(struct ovsdb_idl *); static void ovsdb_idl_db_parse_monitor_reply(struct ovsdb_idl_db *, - const struct json *result, - bool is_monitor_cond); + const struct json *result, + enum ovsdb_idl_monitor_method method); static bool ovsdb_idl_db_parse_update_rpc(struct ovsdb_idl_db *, const struct jsonrpc_msg *); static bool ovsdb_idl_handle_monitor_canceled(struct ovsdb_idl *, @@ -296,7 +311,7 @@ static bool ovsdb_idl_handle_monitor_canceled(struct ovsdb_idl *, const struct jsonrpc_msg *); static void ovsdb_idl_db_parse_update(struct ovsdb_idl_db *, const struct json *table_updates, - bool is_monitor_cond); + enum ovsdb_idl_monitor_method method); static bool ovsdb_idl_process_update(struct ovsdb_idl_table *, const struct uuid *, const struct json *old, @@ -674,7 +689,8 @@ ovsdb_idl_process_response(struct ovsdb_idl *idl, struct jsonrpc_msg *msg) if (!ok && idl->state != IDL_S_SERVER_SCHEMA_REQUESTED && idl->state != IDL_S_SERVER_MONITOR_COND_REQUESTED - && idl->state != IDL_S_DATA_MONITOR_COND_REQUESTED) { + && idl->state != IDL_S_DATA_MONITOR_COND_REQUESTED + && idl->state != IDL_S_DATA_MONITOR_COND_SINCE_REQUESTED) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 5); char *s = jsonrpc_msg_to_string(msg); VLOG_INFO_RL(&rl, "%s: received unexpected %s response in " @@ -692,7 +708,8 @@ ovsdb_idl_process_response(struct ovsdb_idl *idl, struct jsonrpc_msg *msg) if (ok) { json_destroy(idl->server.schema); idl->server.schema = json_clone(msg->result); - ovsdb_idl_send_monitor_request(idl, &idl->server, true); + ovsdb_idl_send_monitor_request(idl, &idl->server, + OVSDB_IDL_MM_MONITOR_COND); ovsdb_idl_transition(idl, IDL_S_SERVER_MONITOR_COND_REQUESTED); } else { ovsdb_idl_send_schema_request(idl, &idl->data); @@ -703,7 +720,8 @@ ovsdb_idl_process_response(struct ovsdb_idl *idl, struct jsonrpc_msg *msg) case IDL_S_SERVER_MONITOR_COND_REQUESTED: if (ok) { idl->server.monitoring = OVSDB_IDL_MONITORING_COND; - ovsdb_idl_db_parse_monitor_reply(&idl->server, msg->result, true); + ovsdb_idl_db_parse_monitor_reply(&idl->server, msg->result, + OVSDB_IDL_MM_MONITOR_COND); if (ovsdb_idl_check_server_db(idl)) { ovsdb_idl_send_db_change_aware(idl); } @@ -716,29 +734,48 @@ ovsdb_idl_process_response(struct ovsdb_idl *idl, struct jsonrpc_msg *msg) case IDL_S_DATA_SCHEMA_REQUESTED: json_destroy(idl->data.schema); idl->data.schema = json_clone(msg->result); - ovsdb_idl_send_monitor_request(idl, &idl->data, true); + ovsdb_idl_send_monitor_request(idl, &idl->data, + OVSDB_IDL_MM_MONITOR_COND); ovsdb_idl_transition(idl, IDL_S_DATA_MONITOR_COND_REQUESTED); break; + case IDL_S_DATA_MONITOR_COND_SINCE_REQUESTED: + if (!ok) { + /* "monitor_cond_since" not supported. Try "monitor_cond". */ + ovsdb_idl_send_monitor_request(idl, &idl->data, + OVSDB_IDL_MM_MONITOR_COND); + ovsdb_idl_transition(idl, IDL_S_DATA_MONITOR_COND_REQUESTED); + } else { + idl->data.monitoring = OVSDB_IDL_MONITORING_COND_SINCE; + ovsdb_idl_transition(idl, IDL_S_MONITORING); + ovsdb_idl_db_parse_monitor_reply(&idl->data, msg->result, + OVSDB_IDL_MM_MONITOR_COND_SINCE); + } + break; + case IDL_S_DATA_MONITOR_COND_REQUESTED: if (!ok) { /* "monitor_cond" not supported. Try "monitor". */ - ovsdb_idl_send_monitor_request(idl, &idl->data, false); + ovsdb_idl_send_monitor_request(idl, &idl->data, + OVSDB_IDL_MM_MONITOR); ovsdb_idl_transition(idl, IDL_S_DATA_MONITOR_REQUESTED); } else { idl->data.monitoring = OVSDB_IDL_MONITORING_COND; ovsdb_idl_transition(idl, IDL_S_MONITORING); - ovsdb_idl_db_parse_monitor_reply(&idl->data, msg->result, true); + ovsdb_idl_db_parse_monitor_reply(&idl->data, msg->result, + OVSDB_IDL_MM_MONITOR_COND); } break; case IDL_S_DATA_MONITOR_REQUESTED: idl->data.monitoring = OVSDB_IDL_MONITORING; ovsdb_idl_transition(idl, IDL_S_MONITORING); - ovsdb_idl_db_parse_monitor_reply(&idl->data, msg->result, false); + ovsdb_idl_db_parse_monitor_reply(&idl->data, msg->result, + OVSDB_IDL_MM_MONITOR); idl->data.change_seqno++; ovsdb_idl_clear(idl); - ovsdb_idl_db_parse_update(&idl->data, msg->result, false); + ovsdb_idl_db_parse_update(&idl->data, msg->result, + OVSDB_IDL_MM_MONITOR); break; case IDL_S_MONITORING: @@ -1564,7 +1601,7 @@ ovsdb_idl_send_cond_change(struct ovsdb_idl *idl) * conditional monitoring update request that we have not heard * from the server yet. Don't generate another request in this case. */ if (!jsonrpc_session_is_connected(idl->session) - || idl->data.monitoring != OVSDB_IDL_MONITORING_COND + || idl->data.monitoring == OVSDB_IDL_MONITORING || idl->request_id) { return; } @@ -1860,8 +1897,9 @@ ovsdb_idl_check_server_db(struct ovsdb_idl *idl) if (idl->state == IDL_S_SERVER_MONITOR_COND_REQUESTED) { json_destroy(idl->data.schema); idl->data.schema = json_from_string(database->schema); - ovsdb_idl_send_monitor_request(idl, &idl->data, true); - ovsdb_idl_transition(idl, IDL_S_DATA_MONITOR_COND_REQUESTED); + ovsdb_idl_send_monitor_request(idl, &idl->data, + OVSDB_IDL_MM_MONITOR_COND_SINCE); + ovsdb_idl_transition(idl, IDL_S_DATA_MONITOR_COND_SINCE_REQUESTED); } return true; } @@ -1951,7 +1989,7 @@ parse_schema(const struct json *schema_json) static void ovsdb_idl_send_monitor_request(struct ovsdb_idl *idl, struct ovsdb_idl_db *db, - bool use_monitor_cond) + enum ovsdb_idl_monitor_method monitor_method) { struct shash *schema = parse_schema(db->schema); struct json *monitor_requests = json_object_create(); @@ -2003,7 +2041,9 @@ ovsdb_idl_send_monitor_request(struct ovsdb_idl *idl, struct ovsdb_idl_db *db, json_object_put(monitor_request, "columns", columns); const struct ovsdb_idl_condition *cond = &table->condition; - if (use_monitor_cond && !ovsdb_idl_condition_is_true(cond)) { + if ((monitor_method == OVSDB_IDL_MM_MONITOR_COND || + monitor_method == OVSDB_IDL_MM_MONITOR_COND_SINCE) && + !ovsdb_idl_condition_is_true(cond)) { json_object_put(monitor_request, "where", ovsdb_idl_condition_to_json(cond)); table->cond_changed = false; @@ -2016,13 +2056,30 @@ ovsdb_idl_send_monitor_request(struct ovsdb_idl *idl, struct ovsdb_idl_db *db, db->cond_changed = false; - ovsdb_idl_send_request( - idl, - jsonrpc_create_request( - use_monitor_cond ? "monitor_cond" : "monitor", - json_array_create_3(json_string_create(db->class_->database), - json_clone(db->monitor_id), monitor_requests), - NULL)); + struct json *params = json_array_create_3( + json_string_create(db->class_->database), + json_clone(db->monitor_id), + monitor_requests); + const char *method; + switch (monitor_method) { + case OVSDB_IDL_MM_MONITOR: + method = "monitor"; + break; + case OVSDB_IDL_MM_MONITOR_COND: + method = "monitor_cond"; + break; + case OVSDB_IDL_MM_MONITOR_COND_SINCE: + method = "monitor_cond_since"; + struct uuid last_id = UUID_ZERO; + struct json *json_last_id = json_string_create_nocopy( + xasprintf(UUID_FMT, UUID_ARGS(&last_id))); + json_array_add(params, json_last_id); + break; + default: + OVS_NOT_REACHED(); + } + + ovsdb_idl_send_request(idl, jsonrpc_create_request(method, params, NULL)); } static void @@ -2039,30 +2096,68 @@ log_parse_update_error(struct ovsdb_error *error) static void ovsdb_idl_db_parse_monitor_reply(struct ovsdb_idl_db *db, const struct json *result, - bool is_monitor_cond) + enum ovsdb_idl_monitor_method method) { db->change_seqno++; + const struct json *table_updates = result; + if (method == OVSDB_IDL_MM_MONITOR_COND_SINCE) { + if (result->type != JSON_ARRAY || result->array.n != 3) { + struct ovsdb_error *error = ovsdb_syntax_error(result, NULL, + "Response of monitor_cond_since must " + "be an array with 3 elements."); + log_parse_update_error(error); + return; + } + + table_updates = result->array.elems[2]; + + } ovsdb_idl_db_clear(db); - ovsdb_idl_db_parse_update(db, result, is_monitor_cond); + ovsdb_idl_db_parse_update(db, table_updates, method); } static bool ovsdb_idl_db_parse_update_rpc(struct ovsdb_idl_db *db, const struct jsonrpc_msg *msg) { - if (msg->type == JSONRPC_NOTIFY) { - bool is_update = !strcmp(msg->method, "update"); - bool is_update2 = !strcmp(msg->method, "update2"); - if ((is_update || is_update2) - && msg->params->type == JSON_ARRAY - && msg->params->array.n == 2 - && json_equal(msg->params->array.elems[0], db->monitor_id)) { - ovsdb_idl_db_parse_update(db, msg->params->array.elems[1], - is_update2); - return true; - } + if (msg->type != JSONRPC_NOTIFY) { + return false; } - return false; + + enum ovsdb_idl_monitor_method mm; + uint8_t n; + if (!strcmp(msg->method, "update")) { + mm = OVSDB_IDL_MM_MONITOR; + n = 2; + } else if (!strcmp(msg->method, "update2")) { + mm = OVSDB_IDL_MM_MONITOR_COND; + n = 2; + } else if (!strcmp(msg->method, "update3")) { + mm = OVSDB_IDL_MM_MONITOR_COND_SINCE; + n = 3; + } else { + return false; + } + + struct json *params = msg->params; + if (params->type != JSON_ARRAY || params->array.n != n) { + struct ovsdb_error *error = ovsdb_syntax_error(params, NULL, + "%s must be an array with %u elements.", + msg->method, n); + log_parse_update_error(error); + return false; + } + + if (!json_equal(params->array.elems[0], db->monitor_id)) { + return false; + } + + struct json *table_updates = params->array.elems[1]; + if (!strcmp(msg->method, "update3")) { + table_updates = params->array.elems[2]; + } + ovsdb_idl_db_parse_update(db, table_updates, mm); + return true; } static bool @@ -2103,10 +2198,21 @@ ovsdb_idl_handle_monitor_canceled(struct ovsdb_idl *idl, static struct ovsdb_error * ovsdb_idl_db_parse_update__(struct ovsdb_idl_db *db, const struct json *table_updates, - bool is_monitor_cond) + enum ovsdb_idl_monitor_method method) { const struct shash_node *tables_node; - const char *version_suffix = is_monitor_cond ? "2" : ""; + const char *version_suffix; + switch (method) { + case OVSDB_IDL_MM_MONITOR: + version_suffix = ""; + break; + case OVSDB_IDL_MM_MONITOR_COND: + case OVSDB_IDL_MM_MONITOR_COND_SINCE: + version_suffix = "2"; + break; + default: + OVS_NOT_REACHED(); + } if (table_updates->type != JSON_OBJECT) { return ovsdb_syntax_error(table_updates, NULL, @@ -2155,7 +2261,8 @@ ovsdb_idl_db_parse_update__(struct ovsdb_idl_db *db, version_suffix, table_node->name); } - if (is_monitor_cond) { + if (method == OVSDB_IDL_MM_MONITOR_COND || + method == OVSDB_IDL_MM_MONITOR_COND_SINCE) { const char *ops[] = {"modify", "insert", "delete", "initial"}; const char *operation; const struct json *row; @@ -2216,10 +2323,10 @@ ovsdb_idl_db_parse_update__(struct ovsdb_idl_db *db, static void ovsdb_idl_db_parse_update(struct ovsdb_idl_db *db, const struct json *table_updates, - bool is_monitor_cond) + enum ovsdb_idl_monitor_method method) { struct ovsdb_error *error = ovsdb_idl_db_parse_update__(db, table_updates, - is_monitor_cond); + method); if (error) { log_parse_update_error(error); } From patchwork Thu Feb 28 17:15:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1049634 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="A/a3ZE43"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 449KFJ2rrpz9s4V for ; Fri, 1 Mar 2019 04:25:24 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id D26BAAB87; Thu, 28 Feb 2019 17:23:08 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 29FE6AAF8 for ; Thu, 28 Feb 2019 17:15:40 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B4D9182F for ; Thu, 28 Feb 2019 17:15:39 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id m1so10014831pgq.8 for ; Thu, 28 Feb 2019 09:15:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=zYd+KyQrDMg1fq7WlA7SFG6MWEJMWtY9zXDB7DeE6sM=; b=A/a3ZE43FoOwKtcrScqez18JhIQGK7gfSdJaAfKwlRDi1vzLGjxyLxGOwfN/qKJT7b DaMHj9ecCmATS/mDnOZoLjPfGYUAcrnskMLZiaS+y9TgpS7h+KWr+Hkm8U7YwCsW1Nkb +hl8CE8uM8TlV1G7Jedw30Kcn6gB+gWo3HtaAzP62w8OOYYjoZNIe1BSPtOAH51iTvJh YEStgMzQh6Kf98Wab00liU8IgbHRbhC8HbblVI/Yzyj1wJgU6tOLTEBa2AhEXCQbQ55G jwpEtSTSPDpW2uNXs4guET1MGUA3t2uq06UcvqFrLWsi7VgpJwOhrO4lATFKf/yqFNe6 rOpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=zYd+KyQrDMg1fq7WlA7SFG6MWEJMWtY9zXDB7DeE6sM=; b=inJUD3nqLtWrLJhdvyDMse7RT8BIqSqpNj+l2At+3Ckrl/45AeUq95R5xIH5vZJQ1S f26BMCLNX7vG9mmnnsfvDUqHRE3oncjbX4vytbmGezw28pHMrltqo52pSOV/GhhENv1/ T/ozrqK2LqeXN2A0RquOJR9JnyEwZ0fWgDn7NO8SBKbVbkeEUSZ5tQzPJjLzSGN5jfHK YnS6RKuRHzMU5pEVYLEw5gASRoVIx0yKbhKviFWabwTP/NpoMg3LMRzI54xyugSmROcm ANiL09KK0o3+RHjf5n0H1FBW0WP19quIbNJ0cz0YBbrIgZO/P6X413N2MrS0X3diAo69 nGDA== X-Gm-Message-State: APjAAAVZBZU1hSl5EImHrNKpsOahoC0ThGWBP0k/p6aFO+mXv8ZbGKPZ xhH2Y7swZekzT9g63mKM3dbyCWaP X-Google-Smtp-Source: APXvYqyfBzoV0ufloDyrKFUIctBJvHeOKiqj6Yj5guanIPPk6e6LPau5y/1z80gluhAztNmho83wLA== X-Received: by 2002:a65:4844:: with SMTP id i4mr154146pgs.347.1551374138940; Thu, 28 Feb 2019 09:15:38 -0800 (PST) Received: from localhost.localdomain.localdomain (c-76-21-108-74.hsd1.ca.comcast.net. [76.21.108.74]) by smtp.gmail.com with ESMTPSA id a4sm36972372pga.52.2019.02.28.09.15.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Feb 2019 09:15:38 -0800 (PST) From: Han Zhou X-Google-Original-From: Han Zhou To: dev@openvswitch.org Date: Thu, 28 Feb 2019 09:15:20 -0800 Message-Id: <1551374120-44287-6-git-send-email-hzhou8@ebay.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1551374120-44287-1-git-send-email-hzhou8@ebay.com> References: <1551374120-44287-1-git-send-email-hzhou8@ebay.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v4 5/5] ovsdb-idl.c: Fast resync from server when connection reset. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Han Zhou Use monitor_cond_since to request changes after last version of local data when connection to server is reset, without clearing the local data. It falls back to clearing and repopulating all the data when the requested id cannot be fulfilled by the server. Test result at ovn-scale-test environment using clustered mode: - 1K HVs (ovsdb clients) - 10K lports Without the patch it took 30+ min for the SB ovsdb-server to calm down and HVs to stablize the connectin and finish syncing data. With the patch there were no noticible CPU spike of SB ovsdb-server, and all HVs were in sync with SB within 1 min, which is the probe interval set in this test (so it took at most 1 min for HVs to notice the TCP connection reset and reconnect and resync finished immediately after that). Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-September/047457.html Signed-off-by: Han Zhou --- lib/ovsdb-idl.c | 34 ++++++++++++++++++++++++++++++---- 1 file changed, 30 insertions(+), 4 deletions(-) diff --git a/lib/ovsdb-idl.c b/lib/ovsdb-idl.c index 5d86046..8cfb201 100644 --- a/lib/ovsdb-idl.c +++ b/lib/ovsdb-idl.c @@ -216,6 +216,9 @@ struct ovsdb_idl_db { bool has_lock; /* Has db server told us we have the lock? */ bool is_lock_contended; /* Has db server told us we can't get lock? */ struct json *lock_request_id; /* JSON-RPC ID of in-flight lock request. */ + + /* Last db txn id, used for fast resync through monitor_cond_since */ + struct uuid last_id; }; static void ovsdb_idl_db_track_clear(struct ovsdb_idl_db *); @@ -2070,9 +2073,8 @@ ovsdb_idl_send_monitor_request(struct ovsdb_idl *idl, struct ovsdb_idl_db *db, break; case OVSDB_IDL_MM_MONITOR_COND_SINCE: method = "monitor_cond_since"; - struct uuid last_id = UUID_ZERO; struct json *json_last_id = json_string_create_nocopy( - xasprintf(UUID_FMT, UUID_ARGS(&last_id))); + xasprintf(UUID_FMT, UUID_ARGS(&db->last_id))); json_array_add(params, json_last_id); break; default: @@ -2100,6 +2102,7 @@ ovsdb_idl_db_parse_monitor_reply(struct ovsdb_idl_db *db, { db->change_seqno++; const struct json *table_updates = result; + bool clear_db = true; if (method == OVSDB_IDL_MM_MONITOR_COND_SINCE) { if (result->type != JSON_ARRAY || result->array.n != 3) { struct ovsdb_error *error = ovsdb_syntax_error(result, NULL, @@ -2109,10 +2112,25 @@ ovsdb_idl_db_parse_monitor_reply(struct ovsdb_idl_db *db, return; } - table_updates = result->array.elems[2]; + bool found = json_boolean(result->array.elems[0]); + if (found) { + clear_db = false; + } + const char *last_id = json_string(result->array.elems[1]); + if (!uuid_from_string(&db->last_id, last_id)) { + struct ovsdb_error *error = ovsdb_syntax_error(result, NULL, + "Last-id %s is not in UUID format.", + last_id); + log_parse_update_error(error); + return; + } + + table_updates = result->array.elems[2]; + } + if (clear_db) { + ovsdb_idl_db_clear(db); } - ovsdb_idl_db_clear(db); ovsdb_idl_db_parse_update(db, table_updates, method); } @@ -2155,6 +2173,14 @@ ovsdb_idl_db_parse_update_rpc(struct ovsdb_idl_db *db, struct json *table_updates = params->array.elems[1]; if (!strcmp(msg->method, "update3")) { table_updates = params->array.elems[2]; + const char *last_id = json_string(params->array.elems[1]); + if (!uuid_from_string(&db->last_id, last_id)) { + struct ovsdb_error *error = ovsdb_syntax_error(params, NULL, + "Last-id %s is not in UUID format.", + last_id); + log_parse_update_error(error); + return false; + } } ovsdb_idl_db_parse_update(db, table_updates, mm); return true;