[ovs-dev,v9] ovsdb-tool: Convert clustered db to standalone db.
diff mbox series

Message ID 20190830011824.71334-1-amginwal@gmail.com
State New
Headers show
Series
  • [ovs-dev,v9] ovsdb-tool: Convert clustered db to standalone db.
Related show

Commit Message

aginwala aginwala Aug. 30, 2019, 1:18 a.m. UTC
From: Aliasgar Ginwala <aginwala@ebay.com>

Add support in ovsdb-tool for migrating clustered dbs to standalone dbs.
E.g. usage to migrate nb/sb db to standalone db from raft:
ovsdb-tool cluster-to-standalone ovnnb_db.db ovnnb_db_cluster.db

Signed-off-by: Aliasgar Ginwala <aginwala@ebay.com>
---
 Documentation/ref/ovsdb.7.rst |   3 +
 NEWS                          |   3 +
 ovsdb/ovsdb-tool.1.in         |   8 +++
 ovsdb/ovsdb-tool.c            | 101 +++++++++++++++++++++++++++++++++-
 tests/ovsdb-tool.at           |  43 +++++++++++++++
 5 files changed, 157 insertions(+), 1 deletion(-)

Comments

Han Zhou Aug. 30, 2019, 5:29 a.m. UTC | #1
Thanks for the update.

On Thu, Aug 29, 2019 at 6:20 PM <amginwal@gmail.com> wrote:
>
> From: Aliasgar Ginwala <aginwala@ebay.com>
>
> Add support in ovsdb-tool for migrating clustered dbs to standalone dbs.
> E.g. usage to migrate nb/sb db to standalone db from raft:
> ovsdb-tool cluster-to-standalone ovnnb_db.db ovnnb_db_cluster.db
>
> Signed-off-by: Aliasgar Ginwala <aginwala@ebay.com>
> ---
>  Documentation/ref/ovsdb.7.rst |   3 +
>  NEWS                          |   3 +
>  ovsdb/ovsdb-tool.1.in         |   8 +++
>  ovsdb/ovsdb-tool.c            | 101 +++++++++++++++++++++++++++++++++-
>  tests/ovsdb-tool.at           |  43 +++++++++++++++
>  5 files changed, 157 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/ref/ovsdb.7.rst b/Documentation/ref/ovsdb.7.rst
> index cd1c63d64..b12d8066c 100644
> --- a/Documentation/ref/ovsdb.7.rst
> +++ b/Documentation/ref/ovsdb.7.rst
> @@ -514,6 +514,9 @@ standalone database from the contents of a running
clustered database.
>  When the cluster is down and cannot be revived, ``ovsdb-client backup``
will
>  not work.
>
> +Use ``ovsdb-tool cluster-to-standalone`` to convert clustered database to
> +standalone database when the cluster is down and cannot be revived.
> +
>  Upgrading or Downgrading a Database
>  -----------------------------------
>
> diff --git a/NEWS b/NEWS
> index c5caa13d6..a02f9f1a6 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -49,6 +49,9 @@ v2.12.0 - xx xxx xxxx
>         quickly after a brief disconnection, saving bandwidth and CPU
time.
>         See section 4.1.15 of ovsdb-server(7) for details of related OVSDB
>         protocol extension.
> +     * Support to convert from cluster database to standalone database
is now
> +       available when clustered is down and cannot be revived using
ovsdb-tool
> +       . Check "Database Migration Commands" in ovsdb-tool man section.
>     - OVN:
>       * IPAM/MACAM:
>         - select IPAM mac_prefix in a random manner if not provided by
the user
> diff --git a/ovsdb/ovsdb-tool.1.in b/ovsdb/ovsdb-tool.1.in
> index ec85e14c4..31a918d90 100644
> --- a/ovsdb/ovsdb-tool.1.in
> +++ b/ovsdb/ovsdb-tool.1.in
> @@ -147,6 +147,14 @@ avoid this possibility, specify
\fB\-\-cid=\fIuuid\fR, where
>  \fIuuid\fR is the cluster ID of the cluster to join, as printed by
>  \fBovsdb\-tool get\-cid\fR.
>  .
> +.SS "Database Migration Commands"
> +This commands will convert cluster database to standalone database.
> +.
> +.IP "\fBcluster\-to\-standalone\fI db clusterdb"
> +Use this command to convert to standalone database from clustered
database
> +when the cluster is down and cannot be revived. It creates new standalone
> +\fIdb\fR file from the given cluster \fIdb\fR file.
> +.
>  .SS "Version Management Commands"
>  .so ovsdb/ovsdb-schemas.man
>  .PP
> diff --git a/ovsdb/ovsdb-tool.c b/ovsdb/ovsdb-tool.c
> index 438f97590..3bbf4c8bc 100644
> --- a/ovsdb/ovsdb-tool.c
> +++ b/ovsdb/ovsdb-tool.c
> @@ -173,6 +173,9 @@ usage(void)
>             "  compare-versions A OP B  compare OVSDB schema version
numbers\n"
>             "  query [DB] TRNS         execute read-only transaction on
DB\n"
>             "  transact [DB] TRNS      execute read/write transaction on
DB\n"
> +           "  cluster-to-standalone DB DB    Convert clustered DB to\n"
> +           "      standalone DB when cluster is down and cannot be\n"
> +           "        revived\n"
>             "  [-m]... show-log [DB]   print DB's log entries\n"
>             "The default DB is %s.\n"
>             "The default SCHEMA is %s.\n",
> @@ -942,6 +945,55 @@ print_raft_record(const struct raft_record *r,
>      }
>  }
>
> +static void
> +raft_header_to_standalone_log(const struct raft_header *h,
> +                              struct ovsdb_log *db_log_data)
> +{
> +    if (h->snap_index) {
> +        if (!h->snap.data || json_array(h->snap.data)->n != 2) {
> +        ovs_fatal(0, "Incorrect raft header data array length");
> +        }
> +
> +        struct json *schema_json = json_array(h->snap.data)->elems[0];
> +        if (schema_json->type != JSON_NULL) {
> +            struct ovsdb_schema *schema;
> +            check_ovsdb_error(ovsdb_schema_from_json(schema_json,
&schema));
> +            ovsdb_schema_destroy(schema);
> +            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> +                                                       schema_json));
> +        }
> +
> +        struct json *data_json = json_array(h->snap.data)->elems[1];
> +        if (!data_json || data_json->type != JSON_OBJECT) {
> +            ovs_fatal(0, "Invalid raft header data");
> +        }
> +        if (data_json->type != JSON_NULL) {
> +            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> +                                                       data_json));
> +        }
> +    }
> +}
> +
> +static void
> +raft_record_to_standalone_log(const struct raft_record *r,
> +                              struct ovsdb_log *db_log_data)
> +{
> +    if (r->type == RAFT_REC_ENTRY) {
> +        if (!r->entry.data) {
> +            return;
> +        }
> +        if (json_array(r->entry.data)->n != 2) {
> +            ovs_fatal(0, "Incorrect raft record array length");
> +        }
> +
> +        struct json *data_json = json_array(r->entry.data)->elems[1];
> +        if (data_json->type != JSON_NULL) {
> +            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> +                                                       data_json));
> +        }
> +    }
> +}
> +
>  static void
>  do_show_log_cluster(struct ovsdb_log *log)
>  {
> @@ -1511,6 +1563,51 @@ do_compare_versions(struct ovs_cmdl_context *ctx)
>      exit(result ? 0 : 2);
>  }
>
> +static void
> +do_convert_to_standalone(struct ovsdb_log *log, struct ovsdb_log
*db_log_data)
> +{
> +    for (unsigned int i = 0; ; i++) {
> +        struct json *json;
> +        check_ovsdb_error(ovsdb_log_read(log, &json));
> +        if (!json) {
> +            break;
> +        }
> +
> +        if (i == 0) {
> +            struct raft_header h;
> +            check_ovsdb_error(raft_header_from_json(&h, json));
> +            raft_header_to_standalone_log(&h, db_log_data);
> +            raft_header_uninit(&h);
> +        } else {
> +            struct raft_record r;
> +            check_ovsdb_error(raft_record_from_json(&r, json));
> +            raft_record_to_standalone_log(&r, db_log_data);
> +            raft_record_uninit(&r);
> +        }
> +    }
> +}
> +
> +static void
> +do_cluster_standalone(struct ovs_cmdl_context *ctx)
> +{
> +    const char *db_file_name = ctx->argv[1];
> +    const char *cluster_db_file_name = ctx->argv[2];
> +    struct ovsdb_log *log;
> +    struct ovsdb_log *db_log_data;
> +
> +    check_ovsdb_error(ovsdb_log_open(cluster_db_file_name,
> +                                     OVSDB_MAGIC"|"RAFT_MAGIC,
> +                                     OVSDB_LOG_READ_ONLY, -1, &log));
> +    check_ovsdb_error(ovsdb_log_open(db_file_name, OVSDB_MAGIC,
> +                                     OVSDB_LOG_CREATE_EXCL, -1,
&db_log_data));
> +    if (strcmp(ovsdb_log_get_magic(log), RAFT_MAGIC) != 0) {
> +        ovs_fatal(0, "Database is not clustered db.\n");
> +    }
> +    do_convert_to_standalone(log, db_log_data);
> +    check_ovsdb_error(ovsdb_log_commit_block(db_log_data));
> +    ovsdb_log_close(db_log_data);
> +    ovsdb_log_close(log);
> +}
>
>  static void
>  do_help(struct ovs_cmdl_context *ctx OVS_UNUSED)
> @@ -1550,7 +1647,9 @@ static const struct ovs_cmdl_command all_commands[]
= {
>      { "compare-versions", "a op b", 3, 3, do_compare_versions, OVS_RO },
>      { "help", NULL, 0, INT_MAX, do_help, OVS_RO },
>      { "list-commands", NULL, 0, INT_MAX, do_list_commands, OVS_RO },
> -    { NULL, NULL, 0, 0, NULL, OVS_RO },
> +    { "cluster-to-standalone", "db clusterdb", 2, 2,
> +    do_cluster_standalone, OVS_RW },
> +    { NULL, NULL, 2, 2, NULL, OVS_RO },
>  };
>
>  static const struct ovs_cmdl_command *get_all_commands(void)
> diff --git a/tests/ovsdb-tool.at b/tests/ovsdb-tool.at
> index 69c5d6afa..953b1ed8f 100644
> --- a/tests/ovsdb-tool.at
> +++ b/tests/ovsdb-tool.at
> @@ -459,3 +459,46 @@ OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>  # Make sure that the clustered data matched the standalone data.
>  AT_CHECK([cat dump2], [0], [expout])
>  AT_CLEANUP
> +
> +AT_SETUP([ovsdb-tool convert-to-standalone])
> +AT_KEYWORDS([ovsdb file positive])
> +ordinal_schema > schema
> +AT_CHECK([ovsdb-tool create-cluster db schema unix:s1.raft], [0],
[stdout], [ignore])
> +AT_CHECK([ovsdb-server --detach --no-chdir --pidfile
--remote=punix:socket --log-file db >/dev/null 2>&1])
> +for txn in m4_foreach([txn], [[[["ordinals",
> +      {"op": "insert",
> +       "table": "ordinals",
> +       "row": {"number": 0, "name": "zero"}},
> +      {"op": "insert",
> +       "table": "ordinals",
> +       "row": {"number": 1, "name": "one"}},
> +      {"op": "insert",
> +       "table": "ordinals",
> +       "row": {"number": 2, "name": "two"}}]]]], ['txn' ]); do
> +AT_CHECK([ovsdb-client transact unix:socket "$txn"], [0], [ignore],
[ignore])
> +done
> +AT_CHECK([ovsdb-client transact unix:socket '[["ordinals"]]'], [0],
> +         [ignore], [ignore])
> +AT_CHECK([ovsdb-client dump unix:socket > clusterdump])
> +AT_CHECK([ovs-appctl -t ovsdb-server -e exit], [0], [ignore], [ignore])
> +
> +# Convert to standalone database from clustered database.
> +AT_CHECK(ovsdb-tool cluster-to-standalone db1 db)
> +
> +# Check its standalone db
> +AT_CHECK([ovsdb-tool db-is-standalone db1])
> +
> +# Dump the standalone db data.
> +AT_CHECK([ovsdb-server -vconsole:off -vfile -vvlog:off --detach
--no-chdir --pidfile --log-file --remote=punix:db.sock db1])
> +AT_CHECK([ovsdb_client_wait ordinals connected])
> +AT_CHECK([ovsdb-client dump  > standalonedump])
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +# Make sure both standalone and cluster db data matches.
> +AT_CHECK([grep "zero" expout], [0], [ignore])
> +AT_CHECK([grep "zero" standalonedump], [0], [ignore])
> +AT_CHECK([grep "one" clusterdump], [0], [ignore])
> +AT_CHECK([grep "one" standalonedump], [0], [ignore])
> +AT_CHECK([grep "two" clusterdump], [0], [ignore])
> +AT_CHECK([grep "two" standalonedump], [0], [ignore])

Can just compare by:
AT_CHECK([diff standalonedump clusterdump])

Alternatively, if the dumped cluster file name was: expout, then you can do:
AT_CHECK([cat standalonedump], [0], [expout])

This would be a minor fix. So:
Acked-by: Han Zhou <hzhou8@ebay.com>
aginwala Aug. 30, 2019, 3:29 p.m. UTC | #2
Thanks Han. Addressed the minor comment in v10.

On Thu, Aug 29, 2019 at 10:31 PM Han Zhou <zhouhan@gmail.com> wrote:

> Thanks for the update.
>
> On Thu, Aug 29, 2019 at 6:20 PM <amginwal@gmail.com> wrote:
> >
> > From: Aliasgar Ginwala <aginwala@ebay.com>
> >
> > Add support in ovsdb-tool for migrating clustered dbs to standalone dbs.
> > E.g. usage to migrate nb/sb db to standalone db from raft:
> > ovsdb-tool cluster-to-standalone ovnnb_db.db ovnnb_db_cluster.db
> >
> > Signed-off-by: Aliasgar Ginwala <aginwala@ebay.com>
> > ---
> >  Documentation/ref/ovsdb.7.rst |   3 +
> >  NEWS                          |   3 +
> >  ovsdb/ovsdb-tool.1.in         |   8 +++
> >  ovsdb/ovsdb-tool.c            | 101 +++++++++++++++++++++++++++++++++-
> >  tests/ovsdb-tool.at           |  43 +++++++++++++++
> >  5 files changed, 157 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/ref/ovsdb.7.rst
> b/Documentation/ref/ovsdb.7.rst
> > index cd1c63d64..b12d8066c 100644
> > --- a/Documentation/ref/ovsdb.7.rst
> > +++ b/Documentation/ref/ovsdb.7.rst
> > @@ -514,6 +514,9 @@ standalone database from the contents of a running
> clustered database.
> >  When the cluster is down and cannot be revived, ``ovsdb-client backup``
> will
> >  not work.
> >
> > +Use ``ovsdb-tool cluster-to-standalone`` to convert clustered database
> to
> > +standalone database when the cluster is down and cannot be revived.
> > +
> >  Upgrading or Downgrading a Database
> >  -----------------------------------
> >
> > diff --git a/NEWS b/NEWS
> > index c5caa13d6..a02f9f1a6 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -49,6 +49,9 @@ v2.12.0 - xx xxx xxxx
> >         quickly after a brief disconnection, saving bandwidth and CPU
> time.
> >         See section 4.1.15 of ovsdb-server(7) for details of related
> OVSDB
> >         protocol extension.
> > +     * Support to convert from cluster database to standalone database
> is now
> > +       available when clustered is down and cannot be revived using
> ovsdb-tool
> > +       . Check "Database Migration Commands" in ovsdb-tool man section.
> >     - OVN:
> >       * IPAM/MACAM:
> >         - select IPAM mac_prefix in a random manner if not provided by
> the user
> > diff --git a/ovsdb/ovsdb-tool.1.in b/ovsdb/ovsdb-tool.1.in
> > index ec85e14c4..31a918d90 100644
> > --- a/ovsdb/ovsdb-tool.1.in
> > +++ b/ovsdb/ovsdb-tool.1.in
> > @@ -147,6 +147,14 @@ avoid this possibility, specify
> \fB\-\-cid=\fIuuid\fR, where
> >  \fIuuid\fR is the cluster ID of the cluster to join, as printed by
> >  \fBovsdb\-tool get\-cid\fR.
> >  .
> > +.SS "Database Migration Commands"
> > +This commands will convert cluster database to standalone database.
> > +.
> > +.IP "\fBcluster\-to\-standalone\fI db clusterdb"
> > +Use this command to convert to standalone database from clustered
> database
> > +when the cluster is down and cannot be revived. It creates new
> standalone
> > +\fIdb\fR file from the given cluster \fIdb\fR file.
> > +.
> >  .SS "Version Management Commands"
> >  .so ovsdb/ovsdb-schemas.man
> >  .PP
> > diff --git a/ovsdb/ovsdb-tool.c b/ovsdb/ovsdb-tool.c
> > index 438f97590..3bbf4c8bc 100644
> > --- a/ovsdb/ovsdb-tool.c
> > +++ b/ovsdb/ovsdb-tool.c
> > @@ -173,6 +173,9 @@ usage(void)
> >             "  compare-versions A OP B  compare OVSDB schema version
> numbers\n"
> >             "  query [DB] TRNS         execute read-only transaction on
> DB\n"
> >             "  transact [DB] TRNS      execute read/write transaction on
> DB\n"
> > +           "  cluster-to-standalone DB DB    Convert clustered DB to\n"
> > +           "      standalone DB when cluster is down and cannot be\n"
> > +           "        revived\n"
> >             "  [-m]... show-log [DB]   print DB's log entries\n"
> >             "The default DB is %s.\n"
> >             "The default SCHEMA is %s.\n",
> > @@ -942,6 +945,55 @@ print_raft_record(const struct raft_record *r,
> >      }
> >  }
> >
> > +static void
> > +raft_header_to_standalone_log(const struct raft_header *h,
> > +                              struct ovsdb_log *db_log_data)
> > +{
> > +    if (h->snap_index) {
> > +        if (!h->snap.data || json_array(h->snap.data)->n != 2) {
> > +        ovs_fatal(0, "Incorrect raft header data array length");
> > +        }
> > +
> > +        struct json *schema_json = json_array(h->snap.data)->elems[0];
> > +        if (schema_json->type != JSON_NULL) {
> > +            struct ovsdb_schema *schema;
> > +            check_ovsdb_error(ovsdb_schema_from_json(schema_json,
> &schema));
> > +            ovsdb_schema_destroy(schema);
> > +            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> > +                                                       schema_json));
> > +        }
> > +
> > +        struct json *data_json = json_array(h->snap.data)->elems[1];
> > +        if (!data_json || data_json->type != JSON_OBJECT) {
> > +            ovs_fatal(0, "Invalid raft header data");
> > +        }
> > +        if (data_json->type != JSON_NULL) {
> > +            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> > +                                                       data_json));
> > +        }
> > +    }
> > +}
> > +
> > +static void
> > +raft_record_to_standalone_log(const struct raft_record *r,
> > +                              struct ovsdb_log *db_log_data)
> > +{
> > +    if (r->type == RAFT_REC_ENTRY) {
> > +        if (!r->entry.data) {
> > +            return;
> > +        }
> > +        if (json_array(r->entry.data)->n != 2) {
> > +            ovs_fatal(0, "Incorrect raft record array length");
> > +        }
> > +
> > +        struct json *data_json = json_array(r->entry.data)->elems[1];
> > +        if (data_json->type != JSON_NULL) {
> > +            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> > +                                                       data_json));
> > +        }
> > +    }
> > +}
> > +
> >  static void
> >  do_show_log_cluster(struct ovsdb_log *log)
> >  {
> > @@ -1511,6 +1563,51 @@ do_compare_versions(struct ovs_cmdl_context *ctx)
> >      exit(result ? 0 : 2);
> >  }
> >
> > +static void
> > +do_convert_to_standalone(struct ovsdb_log *log, struct ovsdb_log
> *db_log_data)
> > +{
> > +    for (unsigned int i = 0; ; i++) {
> > +        struct json *json;
> > +        check_ovsdb_error(ovsdb_log_read(log, &json));
> > +        if (!json) {
> > +            break;
> > +        }
> > +
> > +        if (i == 0) {
> > +            struct raft_header h;
> > +            check_ovsdb_error(raft_header_from_json(&h, json));
> > +            raft_header_to_standalone_log(&h, db_log_data);
> > +            raft_header_uninit(&h);
> > +        } else {
> > +            struct raft_record r;
> > +            check_ovsdb_error(raft_record_from_json(&r, json));
> > +            raft_record_to_standalone_log(&r, db_log_data);
> > +            raft_record_uninit(&r);
> > +        }
> > +    }
> > +}
> > +
> > +static void
> > +do_cluster_standalone(struct ovs_cmdl_context *ctx)
> > +{
> > +    const char *db_file_name = ctx->argv[1];
> > +    const char *cluster_db_file_name = ctx->argv[2];
> > +    struct ovsdb_log *log;
> > +    struct ovsdb_log *db_log_data;
> > +
> > +    check_ovsdb_error(ovsdb_log_open(cluster_db_file_name,
> > +                                     OVSDB_MAGIC"|"RAFT_MAGIC,
> > +                                     OVSDB_LOG_READ_ONLY, -1, &log));
> > +    check_ovsdb_error(ovsdb_log_open(db_file_name, OVSDB_MAGIC,
> > +                                     OVSDB_LOG_CREATE_EXCL, -1,
> &db_log_data));
> > +    if (strcmp(ovsdb_log_get_magic(log), RAFT_MAGIC) != 0) {
> > +        ovs_fatal(0, "Database is not clustered db.\n");
> > +    }
> > +    do_convert_to_standalone(log, db_log_data);
> > +    check_ovsdb_error(ovsdb_log_commit_block(db_log_data));
> > +    ovsdb_log_close(db_log_data);
> > +    ovsdb_log_close(log);
> > +}
> >
> >  static void
> >  do_help(struct ovs_cmdl_context *ctx OVS_UNUSED)
> > @@ -1550,7 +1647,9 @@ static const struct ovs_cmdl_command all_commands[]
> = {
> >      { "compare-versions", "a op b", 3, 3, do_compare_versions, OVS_RO },
> >      { "help", NULL, 0, INT_MAX, do_help, OVS_RO },
> >      { "list-commands", NULL, 0, INT_MAX, do_list_commands, OVS_RO },
> > -    { NULL, NULL, 0, 0, NULL, OVS_RO },
> > +    { "cluster-to-standalone", "db clusterdb", 2, 2,
> > +    do_cluster_standalone, OVS_RW },
> > +    { NULL, NULL, 2, 2, NULL, OVS_RO },
> >  };
> >
> >  static const struct ovs_cmdl_command *get_all_commands(void)
> > diff --git a/tests/ovsdb-tool.at b/tests/ovsdb-tool.at
> > index 69c5d6afa..953b1ed8f 100644
> > --- a/tests/ovsdb-tool.at
> > +++ b/tests/ovsdb-tool.at
> > @@ -459,3 +459,46 @@ OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> >  # Make sure that the clustered data matched the standalone data.
> >  AT_CHECK([cat dump2], [0], [expout])
> >  AT_CLEANUP
> > +
> > +AT_SETUP([ovsdb-tool convert-to-standalone])
> > +AT_KEYWORDS([ovsdb file positive])
> > +ordinal_schema > schema
> > +AT_CHECK([ovsdb-tool create-cluster db schema unix:s1.raft], [0],
> [stdout], [ignore])
> > +AT_CHECK([ovsdb-server --detach --no-chdir --pidfile
> --remote=punix:socket --log-file db >/dev/null 2>&1])
> > +for txn in m4_foreach([txn], [[[["ordinals",
> > +      {"op": "insert",
> > +       "table": "ordinals",
> > +       "row": {"number": 0, "name": "zero"}},
> > +      {"op": "insert",
> > +       "table": "ordinals",
> > +       "row": {"number": 1, "name": "one"}},
> > +      {"op": "insert",
> > +       "table": "ordinals",
> > +       "row": {"number": 2, "name": "two"}}]]]], ['txn' ]); do
> > +AT_CHECK([ovsdb-client transact unix:socket "$txn"], [0], [ignore],
> [ignore])
> > +done
> > +AT_CHECK([ovsdb-client transact unix:socket '[["ordinals"]]'], [0],
> > +         [ignore], [ignore])
> > +AT_CHECK([ovsdb-client dump unix:socket > clusterdump])
> > +AT_CHECK([ovs-appctl -t ovsdb-server -e exit], [0], [ignore], [ignore])
> > +
> > +# Convert to standalone database from clustered database.
> > +AT_CHECK(ovsdb-tool cluster-to-standalone db1 db)
> > +
> > +# Check its standalone db
> > +AT_CHECK([ovsdb-tool db-is-standalone db1])
> > +
> > +# Dump the standalone db data.
> > +AT_CHECK([ovsdb-server -vconsole:off -vfile -vvlog:off --detach
> --no-chdir --pidfile --log-file --remote=punix:db.sock db1])
> > +AT_CHECK([ovsdb_client_wait ordinals connected])
> > +AT_CHECK([ovsdb-client dump  > standalonedump])
> > +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> > +
> > +# Make sure both standalone and cluster db data matches.
> > +AT_CHECK([grep "zero" expout], [0], [ignore])
> > +AT_CHECK([grep "zero" standalonedump], [0], [ignore])
> > +AT_CHECK([grep "one" clusterdump], [0], [ignore])
> > +AT_CHECK([grep "one" standalonedump], [0], [ignore])
> > +AT_CHECK([grep "two" clusterdump], [0], [ignore])
> > +AT_CHECK([grep "two" standalonedump], [0], [ignore])
>
> Can just compare by:
> AT_CHECK([diff standalonedump clusterdump])
>
> Alternatively, if the dumped cluster file name was: expout, then you can
> do:
> AT_CHECK([cat standalonedump], [0], [expout])
>
> This would be a minor fix. So:
> Acked-by: Han Zhou <hzhou8@ebay.com>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>

Patch
diff mbox series

diff --git a/Documentation/ref/ovsdb.7.rst b/Documentation/ref/ovsdb.7.rst
index cd1c63d64..b12d8066c 100644
--- a/Documentation/ref/ovsdb.7.rst
+++ b/Documentation/ref/ovsdb.7.rst
@@ -514,6 +514,9 @@  standalone database from the contents of a running clustered database.
 When the cluster is down and cannot be revived, ``ovsdb-client backup`` will
 not work.
 
+Use ``ovsdb-tool cluster-to-standalone`` to convert clustered database to
+standalone database when the cluster is down and cannot be revived.
+
 Upgrading or Downgrading a Database
 -----------------------------------
 
diff --git a/NEWS b/NEWS
index c5caa13d6..a02f9f1a6 100644
--- a/NEWS
+++ b/NEWS
@@ -49,6 +49,9 @@  v2.12.0 - xx xxx xxxx
        quickly after a brief disconnection, saving bandwidth and CPU time.
        See section 4.1.15 of ovsdb-server(7) for details of related OVSDB
        protocol extension.
+     * Support to convert from cluster database to standalone database is now
+       available when clustered is down and cannot be revived using ovsdb-tool
+       . Check "Database Migration Commands" in ovsdb-tool man section.
    - OVN:
      * IPAM/MACAM:
        - select IPAM mac_prefix in a random manner if not provided by the user
diff --git a/ovsdb/ovsdb-tool.1.in b/ovsdb/ovsdb-tool.1.in
index ec85e14c4..31a918d90 100644
--- a/ovsdb/ovsdb-tool.1.in
+++ b/ovsdb/ovsdb-tool.1.in
@@ -147,6 +147,14 @@  avoid this possibility, specify \fB\-\-cid=\fIuuid\fR, where
 \fIuuid\fR is the cluster ID of the cluster to join, as printed by
 \fBovsdb\-tool get\-cid\fR.
 .
+.SS "Database Migration Commands"
+This commands will convert cluster database to standalone database.
+.
+.IP "\fBcluster\-to\-standalone\fI db clusterdb"
+Use this command to convert to standalone database from clustered database
+when the cluster is down and cannot be revived. It creates new standalone
+\fIdb\fR file from the given cluster \fIdb\fR file.
+.
 .SS "Version Management Commands"
 .so ovsdb/ovsdb-schemas.man
 .PP
diff --git a/ovsdb/ovsdb-tool.c b/ovsdb/ovsdb-tool.c
index 438f97590..3bbf4c8bc 100644
--- a/ovsdb/ovsdb-tool.c
+++ b/ovsdb/ovsdb-tool.c
@@ -173,6 +173,9 @@  usage(void)
            "  compare-versions A OP B  compare OVSDB schema version numbers\n"
            "  query [DB] TRNS         execute read-only transaction on DB\n"
            "  transact [DB] TRNS      execute read/write transaction on DB\n"
+           "  cluster-to-standalone DB DB    Convert clustered DB to\n"
+           "      standalone DB when cluster is down and cannot be\n"
+           "        revived\n"
            "  [-m]... show-log [DB]   print DB's log entries\n"
            "The default DB is %s.\n"
            "The default SCHEMA is %s.\n",
@@ -942,6 +945,55 @@  print_raft_record(const struct raft_record *r,
     }
 }
 
+static void
+raft_header_to_standalone_log(const struct raft_header *h,
+                              struct ovsdb_log *db_log_data)
+{
+    if (h->snap_index) {
+        if (!h->snap.data || json_array(h->snap.data)->n != 2) {
+        ovs_fatal(0, "Incorrect raft header data array length");
+        }
+
+        struct json *schema_json = json_array(h->snap.data)->elems[0];
+        if (schema_json->type != JSON_NULL) {
+            struct ovsdb_schema *schema;
+            check_ovsdb_error(ovsdb_schema_from_json(schema_json, &schema));
+            ovsdb_schema_destroy(schema);
+            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
+                                                       schema_json));
+        }
+
+        struct json *data_json = json_array(h->snap.data)->elems[1];
+        if (!data_json || data_json->type != JSON_OBJECT) {
+            ovs_fatal(0, "Invalid raft header data");
+        }
+        if (data_json->type != JSON_NULL) {
+            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
+                                                       data_json));
+        }
+    }
+}
+
+static void
+raft_record_to_standalone_log(const struct raft_record *r,
+                              struct ovsdb_log *db_log_data)
+{
+    if (r->type == RAFT_REC_ENTRY) {
+        if (!r->entry.data) {
+            return;
+        }
+        if (json_array(r->entry.data)->n != 2) {
+            ovs_fatal(0, "Incorrect raft record array length");
+        }
+
+        struct json *data_json = json_array(r->entry.data)->elems[1];
+        if (data_json->type != JSON_NULL) {
+            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
+                                                       data_json));
+        }
+    }
+}
+
 static void
 do_show_log_cluster(struct ovsdb_log *log)
 {
@@ -1511,6 +1563,51 @@  do_compare_versions(struct ovs_cmdl_context *ctx)
     exit(result ? 0 : 2);
 }
 
+static void
+do_convert_to_standalone(struct ovsdb_log *log, struct ovsdb_log *db_log_data)
+{
+    for (unsigned int i = 0; ; i++) {
+        struct json *json;
+        check_ovsdb_error(ovsdb_log_read(log, &json));
+        if (!json) {
+            break;
+        }
+
+        if (i == 0) {
+            struct raft_header h;
+            check_ovsdb_error(raft_header_from_json(&h, json));
+            raft_header_to_standalone_log(&h, db_log_data);
+            raft_header_uninit(&h);
+        } else {
+            struct raft_record r;
+            check_ovsdb_error(raft_record_from_json(&r, json));
+            raft_record_to_standalone_log(&r, db_log_data);
+            raft_record_uninit(&r);
+        }
+    }
+}
+
+static void
+do_cluster_standalone(struct ovs_cmdl_context *ctx)
+{
+    const char *db_file_name = ctx->argv[1];
+    const char *cluster_db_file_name = ctx->argv[2];
+    struct ovsdb_log *log;
+    struct ovsdb_log *db_log_data;
+
+    check_ovsdb_error(ovsdb_log_open(cluster_db_file_name,
+                                     OVSDB_MAGIC"|"RAFT_MAGIC,
+                                     OVSDB_LOG_READ_ONLY, -1, &log));
+    check_ovsdb_error(ovsdb_log_open(db_file_name, OVSDB_MAGIC,
+                                     OVSDB_LOG_CREATE_EXCL, -1, &db_log_data));
+    if (strcmp(ovsdb_log_get_magic(log), RAFT_MAGIC) != 0) {
+        ovs_fatal(0, "Database is not clustered db.\n");
+    }
+    do_convert_to_standalone(log, db_log_data);
+    check_ovsdb_error(ovsdb_log_commit_block(db_log_data));
+    ovsdb_log_close(db_log_data);
+    ovsdb_log_close(log);
+}
 
 static void
 do_help(struct ovs_cmdl_context *ctx OVS_UNUSED)
@@ -1550,7 +1647,9 @@  static const struct ovs_cmdl_command all_commands[] = {
     { "compare-versions", "a op b", 3, 3, do_compare_versions, OVS_RO },
     { "help", NULL, 0, INT_MAX, do_help, OVS_RO },
     { "list-commands", NULL, 0, INT_MAX, do_list_commands, OVS_RO },
-    { NULL, NULL, 0, 0, NULL, OVS_RO },
+    { "cluster-to-standalone", "db clusterdb", 2, 2,
+    do_cluster_standalone, OVS_RW },
+    { NULL, NULL, 2, 2, NULL, OVS_RO },
 };
 
 static const struct ovs_cmdl_command *get_all_commands(void)
diff --git a/tests/ovsdb-tool.at b/tests/ovsdb-tool.at
index 69c5d6afa..953b1ed8f 100644
--- a/tests/ovsdb-tool.at
+++ b/tests/ovsdb-tool.at
@@ -459,3 +459,46 @@  OVS_APP_EXIT_AND_WAIT([ovsdb-server])
 # Make sure that the clustered data matched the standalone data.
 AT_CHECK([cat dump2], [0], [expout])
 AT_CLEANUP
+
+AT_SETUP([ovsdb-tool convert-to-standalone])
+AT_KEYWORDS([ovsdb file positive])
+ordinal_schema > schema
+AT_CHECK([ovsdb-tool create-cluster db schema unix:s1.raft], [0], [stdout], [ignore])
+AT_CHECK([ovsdb-server --detach --no-chdir --pidfile --remote=punix:socket --log-file db >/dev/null 2>&1])
+for txn in m4_foreach([txn], [[[["ordinals",
+      {"op": "insert",
+       "table": "ordinals",
+       "row": {"number": 0, "name": "zero"}},
+      {"op": "insert",
+       "table": "ordinals",
+       "row": {"number": 1, "name": "one"}},
+      {"op": "insert",
+       "table": "ordinals",
+       "row": {"number": 2, "name": "two"}}]]]], ['txn' ]); do
+AT_CHECK([ovsdb-client transact unix:socket "$txn"], [0], [ignore], [ignore])
+done
+AT_CHECK([ovsdb-client transact unix:socket '[["ordinals"]]'], [0],
+         [ignore], [ignore])
+AT_CHECK([ovsdb-client dump unix:socket > clusterdump])
+AT_CHECK([ovs-appctl -t ovsdb-server -e exit], [0], [ignore], [ignore])
+
+# Convert to standalone database from clustered database.
+AT_CHECK(ovsdb-tool cluster-to-standalone db1 db)
+
+# Check its standalone db
+AT_CHECK([ovsdb-tool db-is-standalone db1])
+
+# Dump the standalone db data.
+AT_CHECK([ovsdb-server -vconsole:off -vfile -vvlog:off --detach --no-chdir --pidfile --log-file --remote=punix:db.sock db1])
+AT_CHECK([ovsdb_client_wait ordinals connected])
+AT_CHECK([ovsdb-client dump  > standalonedump])
+OVS_APP_EXIT_AND_WAIT([ovsdb-server])
+
+# Make sure both standalone and cluster db data matches.
+AT_CHECK([grep "zero" clusterdump], [0], [ignore])
+AT_CHECK([grep "zero" standalonedump], [0], [ignore])
+AT_CHECK([grep "one" clusterdump], [0], [ignore])
+AT_CHECK([grep "one" standalonedump], [0], [ignore])
+AT_CHECK([grep "two" clusterdump], [0], [ignore])
+AT_CHECK([grep "two" standalonedump], [0], [ignore])
+AT_CLEANUP