From patchwork Thu Jan 24 23:11:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1030710 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="FOzqJ9jz"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43lydL0Wvwz9s9h for ; Fri, 25 Jan 2019 10:13:42 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 928DBD9E; Thu, 24 Jan 2019 23:12:04 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id B37C0D8E for ; Thu, 24 Jan 2019 23:12:01 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id BA99E852 for ; Thu, 24 Jan 2019 23:12:00 +0000 (UTC) Received: by mail-wr1-f48.google.com with SMTP id t6so8288783wrr.12 for ; Thu, 24 Jan 2019 15:12:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=MrHnTkpy7Br/gKbgOfbpvkj/1O5aX5G7y1PbGqObJUw=; b=FOzqJ9jzf4JsgBwXex0960S0jpY5M8XCsWtdvJ9DoSqgWprZGQIL//OroCa6VQJljb KD0rgsGsKWXUiPeF2ccwFBx+iC8SN196wjsi+oVLNQgk7tmafmZJ/dcqCBVoJv/DQuNR Nt3pANEngPPApQHMEpYNDpiqCH5EYGIEwNWPdoznnyY/N5FXcN/gE+oZj24BG/5rC5pC s5fjVEK9JszPk8Vs4gTlmUYADfsNSTj55LyDAAQAUt+RK56HrMN9qVVnPaDxLY3h8pMn VUMycoxgUSiZcpTpsscdgDbrKrgkDjENyJbDEyyU+IU9gADWaoUuuJb7KlgJjIey/zHl Th6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=MrHnTkpy7Br/gKbgOfbpvkj/1O5aX5G7y1PbGqObJUw=; b=p2+zRzOyadCvWEinbq8d1a//Du5WpUkPlS1zZtCAfy84X6qtKes+7M+VTxkSSDXMir y01X6FJxKQAhEf7M782NeSzkvffZEQbrIiw5UKLTdkf4cwVPgNWRun6Z9f8s0CIbgJgX O3++CRH60jizSM+CdJi6cvVdwPyn0/YPkDQS3lHlZ4qL2DbT2AKvncu5VJEoDTCJYKPB uibEtZO29R37dzPiG59YZRw9/vSe74dNgL6y4tvpY148qYOftNuT4SJniopR0ebX1uug l96/0u3h4PP5f1wF/wef7goh1nttCYVnPRnzqnLQrTxivdFnji+H8/HAoGVGZPRTB54c ZR+Q== X-Gm-Message-State: AJcUukcTkICDQRxEP8rY8IwSGuwUOTW9OVTb43JpXKr4BN+uPkwqD+BR ByZErULRpEyoEq7Mp0+l2QKoxB4v X-Google-Smtp-Source: ALg8bN4HXuCYkJkHWv2QmnT2y36+wVrR+jsjlfmqKoVNMAfilGTnsnqqk/y1qzt5ulSUqSCAFQowLg== X-Received: by 2002:adf:b243:: with SMTP id y3mr9252708wra.184.1548371519121; Thu, 24 Jan 2019 15:11:59 -0800 (PST) Received: from localhost.localdomain.localdomain ([216.113.160.71]) by smtp.gmail.com with ESMTPSA id h62sm46719335wmf.11.2019.01.24.15.11.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 24 Jan 2019 15:11:58 -0800 (PST) From: Han Zhou X-Google-Original-From: Han Zhou To: dev@openvswitch.org Date: Thu, 24 Jan 2019 15:11:18 -0800 Message-Id: <1548371481-109219-5-git-send-email-hzhou8@ebay.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1548371481-109219-1-git-send-email-hzhou8@ebay.com> References: <1548371481-109219-1-git-send-email-hzhou8@ebay.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [RFC 4/7] ovsdb-server: Transaction history tracking. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Han Zhou Maintaining last N (n = 100) transactions in memory, which will be used for future patches for generating monitor data from any point in this N transactions. Signed-off-by: Han Zhou --- ovsdb/ovsdb-server.c | 8 +++++ ovsdb/ovsdb.h | 10 ++++++ ovsdb/transaction.c | 97 +++++++++++++++++++++++++++++++++++++++++++++++++++- ovsdb/transaction.h | 2 ++ 4 files changed, 116 insertions(+), 1 deletion(-) diff --git a/ovsdb/ovsdb-server.c b/ovsdb/ovsdb-server.c index 65a47a4..2e3a84a 100644 --- a/ovsdb/ovsdb-server.c +++ b/ovsdb/ovsdb-server.c @@ -219,6 +219,7 @@ main_loop(struct server_config *config, struct shash_node *next; SHASH_FOR_EACH_SAFE (node, next, all_dbs) { struct db *db = node->data; + ovsdb_txn_history_run(db->db); if (ovsdb_trigger_run(db->db, time_msec())) { /* The message below is currently the only reason to disconnect * all clients. */ @@ -568,6 +569,7 @@ parse_txn(struct server_config *config, struct db *db, error = ovsdb_file_txn_from_json(db->db, txn_json, false, &txn); if (!error) { + ovsdb_txn_set_txnid(txnid, txn); log_and_free_error(ovsdb_txn_replay_commit(txn)); } if (!error && !uuid_is_zero(txnid)) { @@ -658,6 +660,10 @@ open_db(struct server_config *config, const char *filename) db->db = ovsdb_create(schema, storage); ovsdb_jsonrpc_server_add_db(config->jsonrpc, db->db); + db->db->need_txn_history = true; + db->db->n_txn_history = 0; + ovs_list_init(&db->db->txn_history); + read_db(config, db); error = (db->db->name[0] == '_' @@ -695,6 +701,8 @@ add_server_db(struct server_config *config) json_destroy(schema_json); struct db *db = xzalloc(sizeof *db); + /* We don't need txn_history for server_db. */ + db->filename = xstrdup(""); db->db = ovsdb_create(schema, ovsdb_storage_create_unbacked()); bool ok OVS_UNUSED = ovsdb_jsonrpc_server_add_db(config->jsonrpc, db->db); diff --git a/ovsdb/ovsdb.h b/ovsdb/ovsdb.h index d96b1c2..32e5333 100644 --- a/ovsdb/ovsdb.h +++ b/ovsdb/ovsdb.h @@ -67,6 +67,11 @@ bool ovsdb_parse_version(const char *, struct ovsdb_version *); bool ovsdb_is_valid_version(const char *); /* Database. */ +struct ovsdb_txn_history_node { + struct ovs_list node; /* Element in struct ovsdb's txn_history list */ + struct ovsdb_txn *txn; +}; + struct ovsdb { char *name; struct ovsdb_schema *schema; @@ -80,6 +85,11 @@ struct ovsdb { bool run_triggers; struct ovsdb_table *rbac_role; + + /* History trasanctions for incremental monitor transfer. */ + bool need_txn_history; /* Need to maintain history of transactions. */ + unsigned int n_txn_history; /* Current number of history transactions. */ + struct ovs_list txn_history; /* Contains "struct ovsdb_txn_history_node. */ }; struct ovsdb *ovsdb_create(struct ovsdb_schema *, struct ovsdb_storage *); diff --git a/ovsdb/transaction.c b/ovsdb/transaction.c index 5a43132..18b646b 100644 --- a/ovsdb/transaction.c +++ b/ovsdb/transaction.c @@ -40,6 +40,7 @@ struct ovsdb_txn { struct ovsdb *db; struct ovs_list txn_tables; /* Contains "struct ovsdb_txn_table"s. */ struct ds comment; + struct uuid txnid; /* For clustered mode only. It is the eid. */ }; /* A table modified by a transaction. */ @@ -106,13 +107,19 @@ static unsigned int serial; struct ovsdb_txn * ovsdb_txn_create(struct ovsdb *db) { - struct ovsdb_txn *txn = xmalloc(sizeof *txn); + struct ovsdb_txn *txn = xzalloc(sizeof *txn); txn->db = db; ovs_list_init(&txn->txn_tables); ds_init(&txn->comment); return txn; } +void +ovsdb_txn_set_txnid(const struct uuid *txnid, struct ovsdb_txn *txn) +{ + txn->txnid = *txnid; +} + static void ovsdb_txn_free(struct ovsdb_txn *txn) { @@ -881,11 +888,79 @@ ovsdb_txn_precommit(struct ovsdb_txn *txn) return error; } +static struct ovsdb_txn* +ovsdb_txn_clone(const struct ovsdb_txn *txn) +{ + struct ovsdb_txn *txn_cloned = xzalloc(sizeof *txn_cloned); + ovs_list_init(&txn_cloned->txn_tables); + txn_cloned->txnid = txn->txnid; + + struct ovsdb_txn_table *t; + LIST_FOR_EACH (t, node, &txn->txn_tables) { + struct ovsdb_txn_table *t_cloned = xmalloc(sizeof *t_cloned); + ovs_list_push_back(&txn_cloned->txn_tables, &t_cloned->node); + hmap_init(&t_cloned->txn_rows); + + struct ovsdb_txn_row *r; + HMAP_FOR_EACH (r, hmap_node, &t->txn_rows) { + size_t n_columns = shash_count(&t->table->schema->columns); + struct ovsdb_txn_row *r_cloned = + xzalloc(offsetof(struct ovsdb_txn_row, changed) + + bitmap_n_bytes(n_columns)); + + r_cloned->uuid = r->uuid; + r_cloned->table = r->table; + r_cloned->old = r->old ? ovsdb_row_clone(r->old) : NULL; + r_cloned->new = r->new ? ovsdb_row_clone(r->new) : NULL; + memcpy(&r_cloned->changed, &r->changed, bitmap_n_bytes(n_columns)); + hmap_insert(&t_cloned->txn_rows, &r_cloned->hmap_node, + uuid_hash(&r_cloned->uuid)); + } + } + return txn_cloned; +} + +static void +ovsdb_txn_destroy_cloned(struct ovsdb_txn *txn) +{ + ovs_assert(!txn->db); + struct ovsdb_txn_table *t, *next_txn_table; + LIST_FOR_EACH_SAFE (t, next_txn_table, node, &txn->txn_tables) { + struct ovsdb_txn_row *r, *next_txn_row; + HMAP_FOR_EACH_SAFE (r, next_txn_row, hmap_node, &t->txn_rows) { + if (r->old) { + ovsdb_row_destroy(r->old); + } + if (r->new) { + ovsdb_row_destroy(r->new); + } + hmap_remove(&t->txn_rows, &r->hmap_node); + free(r); + } + hmap_destroy(&t->txn_rows); + ovs_list_remove(&t->node); + free(t); + } + free(txn); +} + +static void +ovsdb_txn_add_to_history(struct ovsdb_txn *txn) +{ + if (txn->db->need_txn_history) { + struct ovsdb_txn_history_node *node = xzalloc(sizeof *node); + node->txn = ovsdb_txn_clone(txn); + ovs_list_push_back(&txn->db->txn_history, &node->node); + txn->db->n_txn_history++; + } +} + /* Finalize commit. */ void ovsdb_txn_complete(struct ovsdb_txn *txn) { if (!ovsdb_txn_is_empty(txn)) { + txn->db->run_triggers = true; ovsdb_monitors_commit(txn->db, txn); ovsdb_error_assert(for_each_txn_row(txn, ovsdb_txn_update_weak_refs)); @@ -906,6 +981,7 @@ ovsdb_txn_replay_commit(struct ovsdb_txn *txn) if (error) { ovsdb_txn_abort(txn); } else { + ovsdb_txn_add_to_history(txn); ovsdb_txn_complete(txn); } return error; @@ -1304,3 +1380,22 @@ for_each_txn_row(struct ovsdb_txn *txn, return NULL; } + +void +ovsdb_txn_history_run(struct ovsdb *db) +{ + if (!db->need_txn_history) { + return; + } + /* Remove old histories to limit the size of the history */ + while (db->n_txn_history > 100) { + struct ovsdb_txn_history_node *txn_h_node = CONTAINER_OF( + ovs_list_pop_front(&db->txn_history), + struct ovsdb_txn_history_node, node); + + ovsdb_txn_destroy_cloned(txn_h_node->txn); + free(txn_h_node); + db->n_txn_history--; + } + /* XXX final release of all history when process exits */ +} diff --git a/ovsdb/transaction.h b/ovsdb/transaction.h index 32384fc..2316226 100644 --- a/ovsdb/transaction.h +++ b/ovsdb/transaction.h @@ -25,6 +25,7 @@ struct ovsdb_table; struct uuid; struct ovsdb_txn *ovsdb_txn_create(struct ovsdb *); +void ovsdb_txn_set_txnid(const struct uuid *, struct ovsdb_txn *); void ovsdb_txn_abort(struct ovsdb_txn *); struct ovsdb_error *ovsdb_txn_replay_commit(struct ovsdb_txn *) @@ -59,5 +60,6 @@ void ovsdb_txn_for_each_change(const struct ovsdb_txn *, void ovsdb_txn_add_comment(struct ovsdb_txn *, const char *); const char *ovsdb_txn_get_comment(const struct ovsdb_txn *); +void ovsdb_txn_history_run(struct ovsdb *); #endif /* ovsdb/transaction.h */