diff mbox

[ovs-dev,4/4] ovsdb: Add unixctl commands for OVSDB replication

Message ID 1466811845-32387-4-git-send-email-blp@ovn.org
State Changes Requested
Headers show

Commit Message

Ben Pfaff June 24, 2016, 11:44 p.m. UTC
From: Mario Cabrera <mario.cabrera@hpe.com>

Set and get the server to replicate from:

	ovsdb-server/set-remote-ovsdb-server {server}
	ovsdb-server/get-remote-ovsdb-server

Set and get the replicated table blacklist:

	ovsdb-server/set-sync-excluded-tables {DB:table,...}
	ovsdb-server/get-sync-excluded-tables

Connect to the configured server and start replication:

	ovsdb-server/connect-remote-ovsdb-server

Disconnect from the remote server and stop replication, without dropping
the replicated data:

	ovsdb-server/disconnect-remote-ovsdb-server

Signed-off-by: Mario Cabrera <mario.cabrera@hpe.com>
---
 ovsdb/ovsdb-server.1.in |  21 ++++++
 ovsdb/ovsdb-server.c    |  97 +++++++++++++++++++++++++
 ovsdb/replication.c     |  13 +++-
 ovsdb/replication.h     |   3 +
 tests/ovsdb-server.at   | 188 ++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 321 insertions(+), 1 deletion(-)

Comments

Ben Pfaff June 25, 2016, 12:16 a.m. UTC | #1
On Fri, Jun 24, 2016 at 04:44:05PM -0700, Ben Pfaff wrote:
> From: Mario Cabrera <mario.cabrera@hpe.com>
> 
> Set and get the server to replicate from:
> 
> 	ovsdb-server/set-remote-ovsdb-server {server}
> 	ovsdb-server/get-remote-ovsdb-server
> 
> Set and get the replicated table blacklist:
> 
> 	ovsdb-server/set-sync-excluded-tables {DB:table,...}
> 	ovsdb-server/get-sync-excluded-tables
> 
> Connect to the configured server and start replication:
> 
> 	ovsdb-server/connect-remote-ovsdb-server
> 
> Disconnect from the remote server and stop replication, without dropping
> the replicated data:
> 
> 	ovsdb-server/disconnect-remote-ovsdb-server
> 
> Signed-off-by: Mario Cabrera <mario.cabrera@hpe.com>

When I apply this, the new unit test fails.  Does it pass for you?
Cabrera Vega, Mario Alberto June 27, 2016, 11:59 p.m. UTC | #2
Yes, all the unit tests pass for me. This patch adds 6 unit tests,
one for each unixctl command. Which one is failing for you?

-Mario

-----Original Message-----
From: Ben Pfaff [mailto:blp@ovn.org] 
Sent: Friday, June 24, 2016 6:17 PM
To: dev@openvswitch.org
Cc: Cabrera Vega, Mario Alberto <mario.cabrera@hpe.com>
Subject: Re: [PATCH 4/4] ovsdb: Add unixctl commands for OVSDB replication

On Fri, Jun 24, 2016 at 04:44:05PM -0700, Ben Pfaff wrote:
> From: Mario Cabrera <mario.cabrera@hpe.com>
> 
> Set and get the server to replicate from:
> 
> 	ovsdb-server/set-remote-ovsdb-server {server}
> 	ovsdb-server/get-remote-ovsdb-server
> 
> Set and get the replicated table blacklist:
> 
> 	ovsdb-server/set-sync-excluded-tables {DB:table,...}
> 	ovsdb-server/get-sync-excluded-tables
> 
> Connect to the configured server and start replication:
> 
> 	ovsdb-server/connect-remote-ovsdb-server
> 
> Disconnect from the remote server and stop replication, without 
> dropping the replicated data:
> 
> 	ovsdb-server/disconnect-remote-ovsdb-server
> 
> Signed-off-by: Mario Cabrera <mario.cabrera@hpe.com>

When I apply this, the new unit test fails.  Does it pass for you?
Andy Zhou July 19, 2016, 10:57 p.m. UTC | #3
On Fri, Jun 24, 2016 at 4:44 PM, Ben Pfaff <blp@ovn.org> wrote:

> From: Mario Cabrera <mario.cabrera@hpe.com>
>
> Set and get the server to replicate from:
>
>         ovsdb-server/set-remote-ovsdb-server {server}
>         ovsdb-server/get-remote-ovsdb-server
>
> Set and get the replicated table blacklist:
>
>         ovsdb-server/set-sync-excluded-tables {DB:table,...}
>         ovsdb-server/get-sync-excluded-tables
>
> Connect to the configured server and start replication:
>
>         ovsdb-server/connect-remote-ovsdb-server
>
> Disconnect from the remote server and stop replication, without dropping
> the replicated data:
>
>         ovsdb-server/disconnect-remote-ovsdb-server
>
> Signed-off-by: Mario Cabrera <mario.cabrera@hpe.com>


 Acked-by: Andy Zhou <azhou@ovn.org>

I was able to fix this patch up and rebased it to master. All tests passed.
I will apply it to master in a short while.
diff mbox

Patch

diff --git a/ovsdb/ovsdb-server.1.in b/ovsdb/ovsdb-server.1.in
index 4bbbace..f21b734 100644
--- a/ovsdb/ovsdb-server.1.in
+++ b/ovsdb/ovsdb-server.1.in
@@ -188,6 +188,27 @@  again (with \fBovsdb\-server/add\-db\fR).
 Outputs a list of the currently configured databases added either through
 the command line or through the \fBovsdb\-server/add\-db\fR command.
 .
+.IP "\fBovsdb\-server/set\-remote\-ovsdb\-server \fIserver"
+Sets  the remote \fIserver\fR from which \fBovsdb\-server\fR connects through
+\fBovsdb\-server/connect\-remote\-ovsdb\-server\fR.
+.
+.IP "\fBovsdb\-server/get\-remote\-ovsdb\-server"
+Gets the remote server from which \fBovsdb\-server\fR is currently synchronizing
+its databases.
+.
+.IP "\fBovsdb\-server/connect\-remote\-ovsdb\-server"
+Causes \fBovsdb\-server\fR to synchronize its databases with the server
+specified by \fBovsdb\-server/set\-remote\-ovsdb\-server\fR.
+.
+.IP "\fBovsdb\-server/disconnect\-remote\-ovsdb\-server"
+Causes \fBovsdb\-server\fR to  stop  synchronizing  its  databases with a remote server.
+.
+.IP "\fBovsdb\-server/set\-sync\-excluded\-tables \fIdb\fB:\fItable\fR[\fB,\fIdb\fB:\fItable\fR]..."
+Sets the \fItable\fR whitin \fIdb\fR that will be excluded from synchronization.
+.
+.IP "\fBovsdb\-server/get\-sync\-excluded\-tables"
+Gets  the  tables  that are currently excluded from synchronization.
+.
 .so lib/vlog-unixctl.man
 .so lib/memory-unixctl.man
 .so lib/coverage-unixctl.man
diff --git a/ovsdb/ovsdb-server.c b/ovsdb/ovsdb-server.c
index 1b9de19..c06e8fd 100644
--- a/ovsdb/ovsdb-server.c
+++ b/ovsdb/ovsdb-server.c
@@ -77,6 +77,12 @@  static unixctl_cb_func ovsdb_server_reconnect;
 static unixctl_cb_func ovsdb_server_perf_counters_clear;
 static unixctl_cb_func ovsdb_server_perf_counters_show;
 static unixctl_cb_func ovsdb_server_disable_monitor2;
+static unixctl_cb_func ovsdb_server_set_remote_ovsdb_server;
+static unixctl_cb_func ovsdb_server_get_remote_ovsdb_server;
+static unixctl_cb_func ovsdb_server_connect_remote_ovsdb_server;
+static unixctl_cb_func ovsdb_server_disconnect_remote_ovsdb_server;
+static unixctl_cb_func ovsdb_server_set_sync_excluded_tables;
+static unixctl_cb_func ovsdb_server_get_sync_excluded_tables;
 
 struct server_config {
     struct sset *remotes;
@@ -334,6 +340,19 @@  main(int argc, char *argv[])
     unixctl_command_register("ovsdb-server/perf-counters-clear", "", 0, 0,
                              ovsdb_server_perf_counters_clear, NULL);
 
+    unixctl_command_register("ovsdb-server/set-remote-ovsdb-server", "", 0, 1,
+                              ovsdb_server_set_remote_ovsdb_server, NULL);
+    unixctl_command_register("ovsdb-server/get-remote-ovsdb-server", "", 0, 0,
+                              ovsdb_server_get_remote_ovsdb_server, NULL);
+    unixctl_command_register("ovsdb-server/connect-remote-ovsdb-server", "", 0, 0,
+                              ovsdb_server_connect_remote_ovsdb_server, NULL);
+    unixctl_command_register("ovsdb-server/disconnect-remote-ovsdb-server", "", 0, 0,
+                              ovsdb_server_disconnect_remote_ovsdb_server, NULL);
+    unixctl_command_register("ovsdb-server/set-sync-excluded-tables", "", 0, 1,
+                              ovsdb_server_set_sync_excluded_tables, NULL);
+    unixctl_command_register("ovsdb-server/get-sync-excluded-tables", "", 0, 0,
+                              ovsdb_server_get_sync_excluded_tables, NULL);
+
     /* Simulate the behavior of OVS release prior to version 2.5 that
      * does not support the monitor2 method.  */
     unixctl_command_register("ovsdb-server/disable-monitor2", "", 0, 0,
@@ -1019,6 +1038,84 @@  report_error_if_changed(char *error, char **last_errorp)
 }
 
 static void
+ovsdb_server_set_remote_ovsdb_server(struct unixctl_conn *conn,
+                                     int argc OVS_UNUSED, const char *argv[],
+                                     void *arg_ OVS_UNUSED)
+{
+    set_remote_ovsdb_server(argv[1]);
+    connect_to_remote_server = false;
+    unixctl_command_reply(conn, NULL);
+}
+
+static void
+ovsdb_server_get_remote_ovsdb_server(struct unixctl_conn *conn,
+                                     int argc OVS_UNUSED,
+                                     const char *argv[] OVS_UNUSED,
+                                     void *arg_ OVS_UNUSED)
+{
+    struct ds s;
+    ds_init(&s);
+
+    ds_put_format(&s, "%s\n", get_remote_ovsdb_server());
+
+    unixctl_command_reply(conn, ds_cstr(&s));
+    ds_destroy(&s);
+}
+
+static void
+ovsdb_server_connect_remote_ovsdb_server(struct unixctl_conn *conn,
+                                         int argc OVS_UNUSED,
+                                         const char *argv[] OVS_UNUSED,
+                                         void *arg_ OVS_UNUSED)
+{
+    if (!connect_to_remote_server) {
+	    replication_init();
+        connect_to_remote_server = true;
+    }
+    unixctl_command_reply(conn, NULL);
+}
+
+static void
+ovsdb_server_disconnect_remote_ovsdb_server(struct unixctl_conn *conn,
+                                            int argc OVS_UNUSED,
+                                            const char *argv[] OVS_UNUSED,
+                                            void *arg_ OVS_UNUSED)
+{
+    disconnect_remote_server();
+    connect_to_remote_server = false;
+    unixctl_command_reply(conn, NULL);
+}
+
+static void
+ovsdb_server_set_sync_excluded_tables(struct unixctl_conn *conn,
+                                      int argc OVS_UNUSED,
+                                      const char *argv[],
+                                      void *arg_ OVS_UNUSED)
+{
+    set_tables_blacklist(argv[1]);
+    unixctl_command_reply(conn, NULL);
+}
+
+static void
+ovsdb_server_get_sync_excluded_tables(struct unixctl_conn *conn,
+                                 int argc OVS_UNUSED,
+                                 const char *argv[] OVS_UNUSED,
+                                 void *arg_ OVS_UNUSED)
+{
+    struct ds s;
+    const char *table_name;
+    struct sset table_blacklist = get_tables_blacklist();
+
+    ds_init(&s);
+
+    SSET_FOR_EACH(table_name, &table_blacklist) {
+        ds_put_format(&s, "%s\n", table_name);
+    }
+
+    unixctl_command_reply(conn, ds_cstr(&s));
+}
+
+static void
 ovsdb_server_exit(struct unixctl_conn *conn, int argc OVS_UNUSED,
                   const char *argv[] OVS_UNUSED,
                   void *exiting_)
diff --git a/ovsdb/replication.c b/ovsdb/replication.c
index 4ef9f4f..ca6ce10 100644
--- a/ovsdb/replication.c
+++ b/ovsdb/replication.c
@@ -38,7 +38,6 @@  static struct sset monitored_tables = SSET_INITIALIZER(&monitored_tables);
 static struct sset tables_blacklist = SSET_INITIALIZER(&tables_blacklist);
 static bool reset_dbs = true;
 
-void replication_init(void);
 static struct jsonrpc *open_jsonrpc(const char *server);
 static struct ovsdb_error *check_jsonrpc_error(int error,
                                                struct jsonrpc_msg **reply_);
@@ -118,6 +117,12 @@  set_remote_ovsdb_server(const char *remote_server)
     remote_ovsdb_server = remote_server ? strdup(remote_server) : NULL;
 }
 
+const char *
+get_remote_ovsdb_server(void)
+{
+    return remote_ovsdb_server;
+}
+
 void
 set_tables_blacklist(const char *blacklist)
 {
@@ -137,6 +142,12 @@  set_tables_blacklist(const char *blacklist)
     }
 }
 
+struct sset
+get_tables_blacklist(void)
+{
+    return tables_blacklist;
+}
+
 void
 disconnect_remote_server(void)
 {
diff --git a/ovsdb/replication.h b/ovsdb/replication.h
index 74acdba..012ca09 100644
--- a/ovsdb/replication.h
+++ b/ovsdb/replication.h
@@ -30,9 +30,12 @@  struct db {
     struct ovsdb_txn *txn;
 };
 
+void replication_init(void);
 void replication_run(struct shash *dbs);
 void set_remote_ovsdb_server(const char *remote_server);
+const char *get_remote_ovsdb_server(void);
 void set_tables_blacklist(const char *blacklist);
+struct sset get_tables_blacklist(void);
 void disconnect_remote_server(void);
 const struct db *find_db(const struct shash *all_dbs, const char *db_name);
 void replication_usage(void);
diff --git a/tests/ovsdb-server.at b/tests/ovsdb-server.at
index 299e537..88eb01a 100644
--- a/tests/ovsdb-server.at
+++ b/tests/ovsdb-server.at
@@ -1079,3 +1079,191 @@  m4_define([OVSDB_CHECK_REPLICATION],
    AT_CLEANUP])
 
 REPLICATION_EXAMPLES
+
+AT_BANNER([OVSDB -- ovsdb-server replication runtime management commands])
+
+#ovsdb-server/get-remote-ovsdb-server command
+AT_SETUP([ovsdb-server/get-remote-ovsdb-server])
+AT_KEYWORDS([ovsdb server replication get-remote])
+ordinal_schema > schema
+AT_CHECK([ovsdb-tool create db schema], [0], [ignore], [ignore])
+on_exit 'kill `cat *.pid`'
+AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --sync-from=tcp:127.0.0.1:9999 db])
+
+AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/get-remote-ovsdb-server],
+  [0], [tcp:127.0.0.1:9999
+])
+AT_CLEANUP
+
+#*ovsdb-server/set-remote-ovsdb-server command
+AT_SETUP([ovsdb-server/set-remote-ovsdb-server])
+AT_KEYWORDS([ovsdb server replication set-remote])
+ordinal_schema > schema
+AT_CHECK([ovsdb-tool create db schema], [0], [ignore], [ignore])
+on_exit 'kill `cat *.pid`'
+AT_CHECK([ovsdb-server --detach --no-chdir --pidfile db])
+
+AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/set-remote-ovsdb-server tcp:127.0.0.1:9999])
+AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/get-remote-ovsdb-server],
+  [0], [tcp:127.0.0.1:9999
+])
+AT_CLEANUP
+
+#ovsdb-server/get-sync-excluded-tables command
+AT_SETUP([ovsdb-server/get-sync-excluded-tables])
+AT_KEYWORDS([ovsdb server replication get-excluded-tables])
+ordinal_schema > schema
+AT_CHECK([ovsdb-tool create db schema], [0], [ignore], [ignore])
+on_exit 'kill `cat *.pid`'
+AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --sync-exclude-tables=mydb:db1,mydb:db2 db])
+
+AT_CHECK([ovs-appctl -t ovsdb-server ovsdb-server/get-sync-excluded-tables],
+  [0], [mydb:db2
+mydb:db1
+])
+AT_CLEANUP
+
+#ovsdb-server/set-sync-excluded-tables command
+AT_SETUP([ovsdb-server/set-sync-excluded-tables])
+AT_KEYWORDS([ovsdb server replication set-excluded-tables])
+replication_schema > schema
+AT_CHECK([ovsdb-tool create db1 schema], [0], [stdout], [ignore])
+AT_CHECK([ovsdb-tool create db2 schema], [0], [stdout], [ignore])
+
+AT_CHECK([ovsdb-server --detach --no-chdir --log-file=ovsdb-server1.log --pidfile="`pwd`"/pid --remote=ptcp:0:127.0.0.1 --unixctl="`pwd`"/unixctl db1], [0], [ignore], [ignore])
+PARSE_LISTENING_PORT([ovsdb-server1.log], [TCP_PORT1])
+
+AT_CHECK([ovsdb-server --detach --no-chdir --log-file=ovsdb-server2.log --pidfile="`pwd`"/pid2 --remote=ptcp:0:127.0.0.1 --unixctl="`pwd`"/unixctl2 --sync-from=tcp:127.0.0.1:$TCP_PORT1	 db2], [0], [ignore], [ignore])
+PARSE_LISTENING_PORT([ovsdb-server2.log], [TCP_PORT2])
+
+AT_CHECK([ovs-appctl -t "`pwd`"/unixctl2 ovsdb-server/set-sync-excluded-tables mydb:b], [0], [ignore], [ignore], [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+
+AT_CHECK([ovsdb-client transact tcp:127.0.0.1:$TCP_PORT1 \
+ '[["mydb",
+    {"op": "insert",
+      "table": "a",
+      "row": {"number": 0, "name": "zero"}},
+    {"op": "insert",
+      "table": "b",
+      "row": {"number": 1, "name": "one"}}]]'], [0], [stdout], [ignore],
+ [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+sleep 2
+
+AT_CHECK([ovsdb-client dump tcp:127.0.0.1:$TCP_PORT1], [0], [stdout], [ignore],
+ [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+cat stdout >> dump1
+AT_CHECK([ovsdb-client dump tcp:127.0.0.1:$TCP_PORT2], [0], [stdout], [ignore],
+ [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+cat stdout >> dump2
+
+AT_CHECK([diff dump1 dump2], [1], [stdout], [ignore],
+        [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+cat stdout >> output
+
+AT_CHECK([${PERL} $srcdir/uuidfilt.pl output], [0], [7,9c7,8
+< _uuid                                name number
+< ------------------------------------ ---- ------
+< <0> one  1     @&t@
+---
+> _uuid name number
+> ----- ---- ------
+], [ignore], [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+
+OVSDB_SERVER_SHUTDOWN
+OVSDB_SERVER_SHUTDOWN2
+AT_CLEANUP
+
+#ovsdb-server/connect-remote-ovsdb-server
+AT_SETUP([ovsdb-server/connect-remote-server])
+AT_KEYWORDS([ovsdb server replication connect-remote-server])
+replication_schema > schema
+AT_CHECK([ovsdb-tool create db1 schema], [0], [stdout], [ignore])
+AT_CHECK([ovsdb-tool create db2 schema], [0], [stdout], [ignore])
+
+AT_CHECK([ovsdb-server --detach --no-chdir --log-file=ovsdb-server1.log --pidfile="`pwd`"/pid --remote=ptcp:0:127.0.0.1 --unixctl="`pwd`"/unixctl db1], [0], [ignore], [ignore])
+PARSE_LISTENING_PORT([ovsdb-server1.log], [TCP_PORT1])
+
+AT_CHECK([ovsdb-server --detach --no-chdir --log-file=ovsdb-server2.log --pidfile="`pwd`"/pid2 --remote=ptcp:0:127.0.0.1 --unixctl="`pwd`"/unixctl2 db2], [0], [ignore], [ignore])
+PARSE_LISTENING_PORT([ovsdb-server2.log], [TCP_PORT2])
+
+AT_CHECK([ovs-appctl -t "`pwd`"/unixctl2 ovsdb-server/set-remote-ovsdb-server tcp:127.0.0.1:$TCP_PORT1], [0], [ignore], [ignore], [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+
+AT_CHECK([ovs-appctl -t "`pwd`"/unixctl2 ovsdb-server/connect-remote-ovsdb-server], [0], [ignore], [ignore],
+        [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+
+AT_CHECK([ovsdb-client transact tcp:127.0.0.1:$TCP_PORT1 \
+ '[["mydb",
+  {"op": "insert",
+   "table": "a",
+   "row": {},
+   "uuid-name": "0"}]]'], [0], [stdout], [ignore],
+ [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+sleep 2
+
+AT_CHECK([ovsdb-client dump tcp:127.0.0.1:$TCP_PORT1], [0], [stdout], [ignore],
+ [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+cat stdout >> dump1
+AT_CHECK([ovsdb-client dump tcp:127.0.0.1:$TCP_PORT2], [0], [stdout], [ignore],
+ [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+cat stdout >> dump2
+
+AT_CHECK([diff dump1 dump2], [0], [], [ignore], [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+OVSDB_SERVER_SHUTDOWN
+OVSDB_SERVER_SHUTDOWN2
+AT_CLEANUP
+
+#ovsdb-server/disconnect-remote-server command
+AT_SETUP([ovsbd-server/disconnect-remote-server])
+AT_KEYWORDS([ovsdb server replication disconnect-remote-server])
+replication_schema > schema
+AT_CHECK([ovsdb-tool create db1 schema], [0], [stdout], [ignore])
+AT_CHECK([ovsdb-tool create db2 schema], [0], [stdout], [ignore])
+
+AT_CHECK([ovsdb-server --detach --no-chdir --log-file=ovsdb-server1.log --pidfile="`pwd`"/pid --remote=ptcp:0:127.0.0.1 --unixctl="`pwd`"/unixctl db1], [0], [ignore], [ignore])
+PARSE_LISTENING_PORT([ovsdb-server1.log], [TCP_PORT1])
+
+AT_CHECK([ovsdb-server --detach --no-chdir --log-file=ovsdb-server2.log --pidfile="`pwd`"/pid2 --remote=ptcp:0:127.0.0.1 --unixctl="`pwd`"/unixctl2 --sync-from=tcp:127.0.0.1:$TCP_PORT1	 db2], [0], [ignore], [ignore])
+PARSE_LISTENING_PORT([ovsdb-server2.log], [TCP_PORT2])
+
+AT_CHECK([ovsdb-client transact tcp:127.0.0.1:$TCP_PORT1 \
+'[["mydb",
+  {"op": "insert",
+   "table": "a",
+   "row": {"number": 0, "name": "zero"}}]]'], [0], [stdout], [ignore],
+[test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+sleep 2
+
+AT_CHECK([ovs-appctl -t "`pwd`"/unixctl2 ovsdb-server/disconnect-remote-ovsdb-server], [0], [ignore], [ignore],
+        [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+
+AT_CHECK([ovsdb-client transact tcp:127.0.0.1:$TCP_PORT1 \
+'[["mydb",
+  {"op": "insert",
+   "table": "b",
+   "row": {"number": 1, "name": "one"}}]]'], [0], [stdout], [ignore],
+[test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+sleep 2
+
+AT_CHECK([ovsdb-client dump tcp:127.0.0.1:$TCP_PORT1], [0], [stdout], [ignore],
+ [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+cat stdout >> dump1
+AT_CHECK([ovsdb-client dump tcp:127.0.0.1:$TCP_PORT2], [0], [stdout], [ignore],
+ [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+cat stdout >> dump2
+
+AT_CHECK([diff dump1 dump2], [1], [stdout], [ignore],
+        [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+cat stdout >> output
+
+AT_CHECK([${PERL} $srcdir/uuidfilt.pl output], [0], [7,9c7,8
+< _uuid                                name number
+< ------------------------------------ ---- ------
+< <0> one  1     @&t@
+---
+> _uuid name number
+> ----- ---- ------
+], [ignore], [test ! -e pid || kill `cat pid`; test ! -e pid2 || kill `cat pid2`])
+
+OVSDB_SERVER_SHUTDOWN
+OVSDB_SERVER_SHUTDOWN2
+AT_CLEANUP