[ovs-dev,v6] ovsdb-tool: Convert clustered db to standalone db.
diff mbox series

Message ID 20190829201303.29742-1-amginwal@gmail.com
State New
Headers show
Series
  • [ovs-dev,v6] ovsdb-tool: Convert clustered db to standalone db.
Related show

Commit Message

aginwala aginwala Aug. 29, 2019, 8:13 p.m. UTC
From: Aliasgar Ginwala <aginwala@ebay.com>

Add support in ovsdb-tool for migrating clustered dbs to standalone dbs.
E.g. usage to migrate nb/sb db to standalone db from raft:
ovsdb-tool cluster-to-standalone ovnnb_db.db ovnnb_db_cluster.db

Signed-off-by: Aliasgar Ginwala <aginwala@ebay.com>
---
 Documentation/ref/ovsdb.7.rst |   3 +
 NEWS                          |   3 +
 ovsdb/ovsdb-tool.1.in         |   8 +++
 ovsdb/ovsdb-tool.c            | 101 +++++++++++++++++++++++++++++++++-
 tests/ovsdb-tool.at           |  46 ++++++++++++++++
 5 files changed, 160 insertions(+), 1 deletion(-)

Comments

Han Zhou Aug. 29, 2019, 8:59 p.m. UTC | #1
On Thu, Aug 29, 2019 at 1:14 PM <amginwal@gmail.com> wrote:
>
> From: Aliasgar Ginwala <aginwala@ebay.com>
>
> Add support in ovsdb-tool for migrating clustered dbs to standalone dbs.
> E.g. usage to migrate nb/sb db to standalone db from raft:
> ovsdb-tool cluster-to-standalone ovnnb_db.db ovnnb_db_cluster.db
>
> Signed-off-by: Aliasgar Ginwala <aginwala@ebay.com>
> ---
>  Documentation/ref/ovsdb.7.rst |   3 +
>  NEWS                          |   3 +
>  ovsdb/ovsdb-tool.1.in         |   8 +++
>  ovsdb/ovsdb-tool.c            | 101 +++++++++++++++++++++++++++++++++-
>  tests/ovsdb-tool.at           |  46 ++++++++++++++++
>  5 files changed, 160 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/ref/ovsdb.7.rst b/Documentation/ref/ovsdb.7.rst
> index cd1c63d64..b12d8066c 100644
> --- a/Documentation/ref/ovsdb.7.rst
> +++ b/Documentation/ref/ovsdb.7.rst
> @@ -514,6 +514,9 @@ standalone database from the contents of a running
clustered database.
>  When the cluster is down and cannot be revived, ``ovsdb-client backup``
will
>  not work.
>
> +Use ``ovsdb-tool cluster-to-standalone`` to convert clustered database to
> +standalone database when the cluster is down and cannot be revived.
> +
>  Upgrading or Downgrading a Database
>  -----------------------------------
>
> diff --git a/NEWS b/NEWS
> index c5caa13d6..a02f9f1a6 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -49,6 +49,9 @@ v2.12.0 - xx xxx xxxx
>         quickly after a brief disconnection, saving bandwidth and CPU
time.
>         See section 4.1.15 of ovsdb-server(7) for details of related OVSDB
>         protocol extension.
> +     * Support to convert from cluster database to standalone database
is now
> +       available when clustered is down and cannot be revived using
ovsdb-tool
> +       . Check "Database Migration Commands" in ovsdb-tool man section.
>     - OVN:
>       * IPAM/MACAM:
>         - select IPAM mac_prefix in a random manner if not provided by
the user
> diff --git a/ovsdb/ovsdb-tool.1.in b/ovsdb/ovsdb-tool.1.in
> index ec85e14c4..31a918d90 100644
> --- a/ovsdb/ovsdb-tool.1.in
> +++ b/ovsdb/ovsdb-tool.1.in
> @@ -147,6 +147,14 @@ avoid this possibility, specify
\fB\-\-cid=\fIuuid\fR, where
>  \fIuuid\fR is the cluster ID of the cluster to join, as printed by
>  \fBovsdb\-tool get\-cid\fR.
>  .
> +.SS "Database Migration Commands"
> +This commands will convert cluster database to standalone database.
> +.
> +.IP "\fBcluster\-to\-standalone\fI db clusterdb"
> +Use this command to convert to standalone database from clustered
database
> +when the cluster is down and cannot be revived. It creates new standalone
> +\fIdb\fR file from the given cluster \fIdb\fR file.
> +.
>  .SS "Version Management Commands"
>  .so ovsdb/ovsdb-schemas.man
>  .PP
> diff --git a/ovsdb/ovsdb-tool.c b/ovsdb/ovsdb-tool.c
> index 438f97590..3bbf4c8bc 100644
> --- a/ovsdb/ovsdb-tool.c
> +++ b/ovsdb/ovsdb-tool.c
> @@ -173,6 +173,9 @@ usage(void)
>             "  compare-versions A OP B  compare OVSDB schema version
numbers\n"
>             "  query [DB] TRNS         execute read-only transaction on
DB\n"
>             "  transact [DB] TRNS      execute read/write transaction on
DB\n"
> +           "  cluster-to-standalone DB DB    Convert clustered DB to\n"
> +           "      standalone DB when cluster is down and cannot be\n"
> +           "        revived\n"
>             "  [-m]... show-log [DB]   print DB's log entries\n"
>             "The default DB is %s.\n"
>             "The default SCHEMA is %s.\n",
> @@ -942,6 +945,55 @@ print_raft_record(const struct raft_record *r,
>      }
>  }
>
> +static void
> +raft_header_to_standalone_log(const struct raft_header *h,
> +                              struct ovsdb_log *db_log_data)
> +{
> +    if (h->snap_index) {
> +        if (!h->snap.data || json_array(h->snap.data)->n != 2) {
> +        ovs_fatal(0, "Incorrect raft header data array length");
> +        }
> +
> +        struct json *schema_json = json_array(h->snap.data)->elems[0];
> +        if (schema_json->type != JSON_NULL) {
> +            struct ovsdb_schema *schema;
> +            check_ovsdb_error(ovsdb_schema_from_json(schema_json,
&schema));
> +            ovsdb_schema_destroy(schema);
> +            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> +                                                       schema_json));
> +        }
> +
> +        struct json *data_json = json_array(h->snap.data)->elems[1];
> +        if (!data_json || data_json->type != JSON_OBJECT) {
> +            ovs_fatal(0, "Invalid raft header data");
> +        }
> +        if (data_json->type != JSON_NULL) {
> +            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> +                                                       data_json));
> +        }
> +    }
> +}
> +
> +static void
> +raft_record_to_standalone_log(const struct raft_record *r,
> +                              struct ovsdb_log *db_log_data)
> +{
> +    if (r->type == RAFT_REC_ENTRY) {
> +        if (!r->entry.data) {
> +            return;
> +        }
> +        if (json_array(r->entry.data)->n != 2) {
> +            ovs_fatal(0, "Incorrect raft record array length");
> +        }
> +
> +        struct json *data_json = json_array(r->entry.data)->elems[1];
> +        if (data_json->type != JSON_NULL) {
> +            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> +                                                       data_json));
> +        }
> +    }
> +}
> +
>  static void
>  do_show_log_cluster(struct ovsdb_log *log)
>  {
> @@ -1511,6 +1563,51 @@ do_compare_versions(struct ovs_cmdl_context *ctx)
>      exit(result ? 0 : 2);
>  }
>
> +static void
> +do_convert_to_standalone(struct ovsdb_log *log, struct ovsdb_log
*db_log_data)
> +{
> +    for (unsigned int i = 0; ; i++) {
> +        struct json *json;
> +        check_ovsdb_error(ovsdb_log_read(log, &json));
> +        if (!json) {
> +            break;
> +        }
> +
> +        if (i == 0) {
> +            struct raft_header h;
> +            check_ovsdb_error(raft_header_from_json(&h, json));
> +            raft_header_to_standalone_log(&h, db_log_data);
> +            raft_header_uninit(&h);
> +        } else {
> +            struct raft_record r;
> +            check_ovsdb_error(raft_record_from_json(&r, json));
> +            raft_record_to_standalone_log(&r, db_log_data);
> +            raft_record_uninit(&r);
> +        }
> +    }
> +}
> +
> +static void
> +do_cluster_standalone(struct ovs_cmdl_context *ctx)
> +{
> +    const char *db_file_name = ctx->argv[1];
> +    const char *cluster_db_file_name = ctx->argv[2];
> +    struct ovsdb_log *log;
> +    struct ovsdb_log *db_log_data;
> +
> +    check_ovsdb_error(ovsdb_log_open(cluster_db_file_name,
> +                                     OVSDB_MAGIC"|"RAFT_MAGIC,
> +                                     OVSDB_LOG_READ_ONLY, -1, &log));
> +    check_ovsdb_error(ovsdb_log_open(db_file_name, OVSDB_MAGIC,
> +                                     OVSDB_LOG_CREATE_EXCL, -1,
&db_log_data));
> +    if (strcmp(ovsdb_log_get_magic(log), RAFT_MAGIC) != 0) {
> +        ovs_fatal(0, "Database is not clustered db.\n");
> +    }
> +    do_convert_to_standalone(log, db_log_data);
> +    check_ovsdb_error(ovsdb_log_commit_block(db_log_data));
> +    ovsdb_log_close(db_log_data);
> +    ovsdb_log_close(log);
> +}
>
>  static void
>  do_help(struct ovs_cmdl_context *ctx OVS_UNUSED)
> @@ -1550,7 +1647,9 @@ static const struct ovs_cmdl_command all_commands[]
= {
>      { "compare-versions", "a op b", 3, 3, do_compare_versions, OVS_RO },
>      { "help", NULL, 0, INT_MAX, do_help, OVS_RO },
>      { "list-commands", NULL, 0, INT_MAX, do_list_commands, OVS_RO },
> -    { NULL, NULL, 0, 0, NULL, OVS_RO },
> +    { "cluster-to-standalone", "db clusterdb", 2, 2,
> +    do_cluster_standalone, OVS_RW },
> +    { NULL, NULL, 2, 2, NULL, OVS_RO },
>  };
>
>  static const struct ovs_cmdl_command *get_all_commands(void)
> diff --git a/tests/ovsdb-tool.at b/tests/ovsdb-tool.at
> index 69c5d6afa..cf98e4890 100644
> --- a/tests/ovsdb-tool.at
> +++ b/tests/ovsdb-tool.at
> @@ -459,3 +459,49 @@ OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>  # Make sure that the clustered data matched the standalone data.
>  AT_CHECK([cat dump2], [0], [expout])
>  AT_CLEANUP
> +
> +AT_SETUP([ovsdb-tool convert-to-standalone])
> +AT_KEYWORDS([ovsdb file positive])
> +
> +# Create a standalone database and put some data in it.
> +ordinal_schema > schema
> +ovsdb-tool create db1 schema
> +AT_CHECK(
> +  [[for pair in 'zero 0' 'one 1' 'two 2' 'three 3' 'four 4' 'five 5'; do
> +      set -- $pair
> +      ovsdb-tool transact db1 '
> +        ["ordinals",
> +         {"op": "insert",
> +          "table": "ordinals",
> +          "row": {"name": "'$1'", "number": '$2'}},
> +         {"op": "comment",
> +          "comment": "add row for '"$pair"'"}]'
> +    done | uuidfilt]], [0],
> +[[[{"uuid":["uuid","<0>"]},{}]
> +[{"uuid":["uuid","<1>"]},{}]
> +[{"uuid":["uuid","<2>"]},{}]
> +[{"uuid":["uuid","<3>"]},{}]
> +[{"uuid":["uuid","<4>"]},{}]
> +[{"uuid":["uuid","<5>"]},{}]
> +]], [ignore])
> +
> +# Dump the data.
> +AT_CHECK([ovsdb-server -vfile -vvlog:off --detach --no-chdir --pidfile
--log-file --remote=punix:db.sock db1])
> +AT_CHECK([ovsdb-client dump > expout])
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +# Create a clustered database from the standalone one.
> +ovsdb-tool create-cluster db2 db1 unix:s1.raft

It would be better to execute some transactions in clustered mode before
the conversion test, so that the file will have not only header with
snapshot but also body records, so that the raft record conversion
functions are covered.

> +
> +# Convert to standalone database from clustered database.
> +ovsdb-tool cluster-to-standalone db3 db2
> +

Shall we check if db3 is standalone format by: ovsdb-tool db-is-standalone
db3
Otherwise, would the test pass even if we didn't do conversion at all?

> +# Dump the data.
> +AT_CHECK([ovsdb-server -vconsole:off -vfile -vvlog:off --detach
--no-chdir --pidfile --log-file --remote=punix:db.sock db3])
> +AT_CHECK([ovsdb_client_wait ordinals connected])
> +AT_CHECK([ovsdb-client dump > dump3])
> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> +
> +# Make sure both standalone db data matches.
> +AT_CHECK([cat dump3], [0], [expout])
> +AT_CLEANUP
> --
> 2.20.1 (Apple Git-117)
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
aginwala Aug. 29, 2019, 9:49 p.m. UTC | #2
On Thu, Aug 29, 2019 at 2:01 PM Han Zhou <zhouhan@gmail.com> wrote:

> On Thu, Aug 29, 2019 at 1:14 PM <amginwal@gmail.com> wrote:
> >
> > From: Aliasgar Ginwala <aginwala@ebay.com>
> >
> > Add support in ovsdb-tool for migrating clustered dbs to standalone dbs.
> > E.g. usage to migrate nb/sb db to standalone db from raft:
> > ovsdb-tool cluster-to-standalone ovnnb_db.db ovnnb_db_cluster.db
> >
> > Signed-off-by: Aliasgar Ginwala <aginwala@ebay.com>
> > ---
> >  Documentation/ref/ovsdb.7.rst |   3 +
> >  NEWS                          |   3 +
> >  ovsdb/ovsdb-tool.1.in         |   8 +++
> >  ovsdb/ovsdb-tool.c            | 101 +++++++++++++++++++++++++++++++++-
> >  tests/ovsdb-tool.at           |  46 ++++++++++++++++
> >  5 files changed, 160 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/ref/ovsdb.7.rst
> b/Documentation/ref/ovsdb.7.rst
> > index cd1c63d64..b12d8066c 100644
> > --- a/Documentation/ref/ovsdb.7.rst
> > +++ b/Documentation/ref/ovsdb.7.rst
> > @@ -514,6 +514,9 @@ standalone database from the contents of a running
> clustered database.
> >  When the cluster is down and cannot be revived, ``ovsdb-client backup``
> will
> >  not work.
> >
> > +Use ``ovsdb-tool cluster-to-standalone`` to convert clustered database
> to
> > +standalone database when the cluster is down and cannot be revived.
> > +
> >  Upgrading or Downgrading a Database
> >  -----------------------------------
> >
> > diff --git a/NEWS b/NEWS
> > index c5caa13d6..a02f9f1a6 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -49,6 +49,9 @@ v2.12.0 - xx xxx xxxx
> >         quickly after a brief disconnection, saving bandwidth and CPU
> time.
> >         See section 4.1.15 of ovsdb-server(7) for details of related
> OVSDB
> >         protocol extension.
> > +     * Support to convert from cluster database to standalone database
> is now
> > +       available when clustered is down and cannot be revived using
> ovsdb-tool
> > +       . Check "Database Migration Commands" in ovsdb-tool man section.
> >     - OVN:
> >       * IPAM/MACAM:
> >         - select IPAM mac_prefix in a random manner if not provided by
> the user
> > diff --git a/ovsdb/ovsdb-tool.1.in b/ovsdb/ovsdb-tool.1.in
> > index ec85e14c4..31a918d90 100644
> > --- a/ovsdb/ovsdb-tool.1.in
> > +++ b/ovsdb/ovsdb-tool.1.in
> > @@ -147,6 +147,14 @@ avoid this possibility, specify
> \fB\-\-cid=\fIuuid\fR, where
> >  \fIuuid\fR is the cluster ID of the cluster to join, as printed by
> >  \fBovsdb\-tool get\-cid\fR.
> >  .
> > +.SS "Database Migration Commands"
> > +This commands will convert cluster database to standalone database.
> > +.
> > +.IP "\fBcluster\-to\-standalone\fI db clusterdb"
> > +Use this command to convert to standalone database from clustered
> database
> > +when the cluster is down and cannot be revived. It creates new
> standalone
> > +\fIdb\fR file from the given cluster \fIdb\fR file.
> > +.
> >  .SS "Version Management Commands"
> >  .so ovsdb/ovsdb-schemas.man
> >  .PP
> > diff --git a/ovsdb/ovsdb-tool.c b/ovsdb/ovsdb-tool.c
> > index 438f97590..3bbf4c8bc 100644
> > --- a/ovsdb/ovsdb-tool.c
> > +++ b/ovsdb/ovsdb-tool.c
> > @@ -173,6 +173,9 @@ usage(void)
> >             "  compare-versions A OP B  compare OVSDB schema version
> numbers\n"
> >             "  query [DB] TRNS         execute read-only transaction on
> DB\n"
> >             "  transact [DB] TRNS      execute read/write transaction on
> DB\n"
> > +           "  cluster-to-standalone DB DB    Convert clustered DB to\n"
> > +           "      standalone DB when cluster is down and cannot be\n"
> > +           "        revived\n"
> >             "  [-m]... show-log [DB]   print DB's log entries\n"
> >             "The default DB is %s.\n"
> >             "The default SCHEMA is %s.\n",
> > @@ -942,6 +945,55 @@ print_raft_record(const struct raft_record *r,
> >      }
> >  }
> >
> > +static void
> > +raft_header_to_standalone_log(const struct raft_header *h,
> > +                              struct ovsdb_log *db_log_data)
> > +{
> > +    if (h->snap_index) {
> > +        if (!h->snap.data || json_array(h->snap.data)->n != 2) {
> > +        ovs_fatal(0, "Incorrect raft header data array length");
> > +        }
> > +
> > +        struct json *schema_json = json_array(h->snap.data)->elems[0];
> > +        if (schema_json->type != JSON_NULL) {
> > +            struct ovsdb_schema *schema;
> > +            check_ovsdb_error(ovsdb_schema_from_json(schema_json,
> &schema));
> > +            ovsdb_schema_destroy(schema);
> > +            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> > +                                                       schema_json));
> > +        }
> > +
> > +        struct json *data_json = json_array(h->snap.data)->elems[1];
> > +        if (!data_json || data_json->type != JSON_OBJECT) {
> > +            ovs_fatal(0, "Invalid raft header data");
> > +        }
> > +        if (data_json->type != JSON_NULL) {
> > +            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> > +                                                       data_json));
> > +        }
> > +    }
> > +}
> > +
> > +static void
> > +raft_record_to_standalone_log(const struct raft_record *r,
> > +                              struct ovsdb_log *db_log_data)
> > +{
> > +    if (r->type == RAFT_REC_ENTRY) {
> > +        if (!r->entry.data) {
> > +            return;
> > +        }
> > +        if (json_array(r->entry.data)->n != 2) {
> > +            ovs_fatal(0, "Incorrect raft record array length");
> > +        }
> > +
> > +        struct json *data_json = json_array(r->entry.data)->elems[1];
> > +        if (data_json->type != JSON_NULL) {
> > +            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
> > +                                                       data_json));
> > +        }
> > +    }
> > +}
> > +
> >  static void
> >  do_show_log_cluster(struct ovsdb_log *log)
> >  {
> > @@ -1511,6 +1563,51 @@ do_compare_versions(struct ovs_cmdl_context *ctx)
> >      exit(result ? 0 : 2);
> >  }
> >
> > +static void
> > +do_convert_to_standalone(struct ovsdb_log *log, struct ovsdb_log
> *db_log_data)
> > +{
> > +    for (unsigned int i = 0; ; i++) {
> > +        struct json *json;
> > +        check_ovsdb_error(ovsdb_log_read(log, &json));
> > +        if (!json) {
> > +            break;
> > +        }
> > +
> > +        if (i == 0) {
> > +            struct raft_header h;
> > +            check_ovsdb_error(raft_header_from_json(&h, json));
> > +            raft_header_to_standalone_log(&h, db_log_data);
> > +            raft_header_uninit(&h);
> > +        } else {
> > +            struct raft_record r;
> > +            check_ovsdb_error(raft_record_from_json(&r, json));
> > +            raft_record_to_standalone_log(&r, db_log_data);
> > +            raft_record_uninit(&r);
> > +        }
> > +    }
> > +}
> > +
> > +static void
> > +do_cluster_standalone(struct ovs_cmdl_context *ctx)
> > +{
> > +    const char *db_file_name = ctx->argv[1];
> > +    const char *cluster_db_file_name = ctx->argv[2];
> > +    struct ovsdb_log *log;
> > +    struct ovsdb_log *db_log_data;
> > +
> > +    check_ovsdb_error(ovsdb_log_open(cluster_db_file_name,
> > +                                     OVSDB_MAGIC"|"RAFT_MAGIC,
> > +                                     OVSDB_LOG_READ_ONLY, -1, &log));
> > +    check_ovsdb_error(ovsdb_log_open(db_file_name, OVSDB_MAGIC,
> > +                                     OVSDB_LOG_CREATE_EXCL, -1,
> &db_log_data));
> > +    if (strcmp(ovsdb_log_get_magic(log), RAFT_MAGIC) != 0) {
> > +        ovs_fatal(0, "Database is not clustered db.\n");
> > +    }
> > +    do_convert_to_standalone(log, db_log_data);
> > +    check_ovsdb_error(ovsdb_log_commit_block(db_log_data));
> > +    ovsdb_log_close(db_log_data);
> > +    ovsdb_log_close(log);
> > +}
> >
> >  static void
> >  do_help(struct ovs_cmdl_context *ctx OVS_UNUSED)
> > @@ -1550,7 +1647,9 @@ static const struct ovs_cmdl_command all_commands[]
> = {
> >      { "compare-versions", "a op b", 3, 3, do_compare_versions, OVS_RO },
> >      { "help", NULL, 0, INT_MAX, do_help, OVS_RO },
> >      { "list-commands", NULL, 0, INT_MAX, do_list_commands, OVS_RO },
> > -    { NULL, NULL, 0, 0, NULL, OVS_RO },
> > +    { "cluster-to-standalone", "db clusterdb", 2, 2,
> > +    do_cluster_standalone, OVS_RW },
> > +    { NULL, NULL, 2, 2, NULL, OVS_RO },
> >  };
> >
> >  static const struct ovs_cmdl_command *get_all_commands(void)
> > diff --git a/tests/ovsdb-tool.at b/tests/ovsdb-tool.at
> > index 69c5d6afa..cf98e4890 100644
> > --- a/tests/ovsdb-tool.at
> > +++ b/tests/ovsdb-tool.at
> > @@ -459,3 +459,49 @@ OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> >  # Make sure that the clustered data matched the standalone data.
> >  AT_CHECK([cat dump2], [0], [expout])
> >  AT_CLEANUP
> > +
> > +AT_SETUP([ovsdb-tool convert-to-standalone])
> > +AT_KEYWORDS([ovsdb file positive])
> > +
> > +# Create a standalone database and put some data in it.
> > +ordinal_schema > schema
> > +ovsdb-tool create db1 schema
> > +AT_CHECK(
> > +  [[for pair in 'zero 0' 'one 1' 'two 2' 'three 3' 'four 4' 'five 5'; do
> > +      set -- $pair
> > +      ovsdb-tool transact db1 '
> > +        ["ordinals",
> > +         {"op": "insert",
> > +          "table": "ordinals",
> > +          "row": {"name": "'$1'", "number": '$2'}},
> > +         {"op": "comment",
> > +          "comment": "add row for '"$pair"'"}]'
> > +    done | uuidfilt]], [0],
> > +[[[{"uuid":["uuid","<0>"]},{}]
> > +[{"uuid":["uuid","<1>"]},{}]
> > +[{"uuid":["uuid","<2>"]},{}]
> > +[{"uuid":["uuid","<3>"]},{}]
> > +[{"uuid":["uuid","<4>"]},{}]
> > +[{"uuid":["uuid","<5>"]},{}]
> > +]], [ignore])
> > +
> > +# Dump the data.
> > +AT_CHECK([ovsdb-server -vfile -vvlog:off --detach --no-chdir --pidfile
> --log-file --remote=punix:db.sock db1])
> > +AT_CHECK([ovsdb-client dump > expout])
> > +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> > +
> > +# Create a clustered database from the standalone one.
> > +ovsdb-tool create-cluster db2 db1 unix:s1.raft
>
> It would be better to execute some transactions in clustered mode before
> the conversion test, so that the file will have not only header with
> snapshot but also body records, so that the raft record conversion
> functions are covered.
>
Added one more test case to cover the same. PTAL @
https://patchwork.ozlabs.org/patch/1155523/

>
> > +
> > +# Convert to standalone database from clustered database.
> > +ovsdb-tool cluster-to-standalone db3 db2
> > +
>
> Shall we check if db3 is standalone format by: ovsdb-tool db-is-standalone
> db3
> Otherwise, would the test pass even if we didn't do conversion at all?
>
No. However its not a harm to add db-is-standalone check and hence added
that too.  PTAL @ https://patchwork.ozlabs.org/patch/1155523/

>
> > +# Dump the data.
> > +AT_CHECK([ovsdb-server -vconsole:off -vfile -vvlog:off --detach
> --no-chdir --pidfile --log-file --remote=punix:db.sock db3])
> > +AT_CHECK([ovsdb_client_wait ordinals connected])
> > +AT_CHECK([ovsdb-client dump > dump3])
> > +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> > +
> > +# Make sure both standalone db data matches.
> > +AT_CHECK([cat dump3], [0], [expout])
> > +AT_CLEANUP
> > --
> > 2.20.1 (Apple Git-117)
> >
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>

Patch
diff mbox series

diff --git a/Documentation/ref/ovsdb.7.rst b/Documentation/ref/ovsdb.7.rst
index cd1c63d64..b12d8066c 100644
--- a/Documentation/ref/ovsdb.7.rst
+++ b/Documentation/ref/ovsdb.7.rst
@@ -514,6 +514,9 @@  standalone database from the contents of a running clustered database.
 When the cluster is down and cannot be revived, ``ovsdb-client backup`` will
 not work.
 
+Use ``ovsdb-tool cluster-to-standalone`` to convert clustered database to
+standalone database when the cluster is down and cannot be revived.
+
 Upgrading or Downgrading a Database
 -----------------------------------
 
diff --git a/NEWS b/NEWS
index c5caa13d6..a02f9f1a6 100644
--- a/NEWS
+++ b/NEWS
@@ -49,6 +49,9 @@  v2.12.0 - xx xxx xxxx
        quickly after a brief disconnection, saving bandwidth and CPU time.
        See section 4.1.15 of ovsdb-server(7) for details of related OVSDB
        protocol extension.
+     * Support to convert from cluster database to standalone database is now
+       available when clustered is down and cannot be revived using ovsdb-tool
+       . Check "Database Migration Commands" in ovsdb-tool man section.
    - OVN:
      * IPAM/MACAM:
        - select IPAM mac_prefix in a random manner if not provided by the user
diff --git a/ovsdb/ovsdb-tool.1.in b/ovsdb/ovsdb-tool.1.in
index ec85e14c4..31a918d90 100644
--- a/ovsdb/ovsdb-tool.1.in
+++ b/ovsdb/ovsdb-tool.1.in
@@ -147,6 +147,14 @@  avoid this possibility, specify \fB\-\-cid=\fIuuid\fR, where
 \fIuuid\fR is the cluster ID of the cluster to join, as printed by
 \fBovsdb\-tool get\-cid\fR.
 .
+.SS "Database Migration Commands"
+This commands will convert cluster database to standalone database.
+.
+.IP "\fBcluster\-to\-standalone\fI db clusterdb"
+Use this command to convert to standalone database from clustered database
+when the cluster is down and cannot be revived. It creates new standalone
+\fIdb\fR file from the given cluster \fIdb\fR file.
+.
 .SS "Version Management Commands"
 .so ovsdb/ovsdb-schemas.man
 .PP
diff --git a/ovsdb/ovsdb-tool.c b/ovsdb/ovsdb-tool.c
index 438f97590..3bbf4c8bc 100644
--- a/ovsdb/ovsdb-tool.c
+++ b/ovsdb/ovsdb-tool.c
@@ -173,6 +173,9 @@  usage(void)
            "  compare-versions A OP B  compare OVSDB schema version numbers\n"
            "  query [DB] TRNS         execute read-only transaction on DB\n"
            "  transact [DB] TRNS      execute read/write transaction on DB\n"
+           "  cluster-to-standalone DB DB    Convert clustered DB to\n"
+           "      standalone DB when cluster is down and cannot be\n"
+           "        revived\n"
            "  [-m]... show-log [DB]   print DB's log entries\n"
            "The default DB is %s.\n"
            "The default SCHEMA is %s.\n",
@@ -942,6 +945,55 @@  print_raft_record(const struct raft_record *r,
     }
 }
 
+static void
+raft_header_to_standalone_log(const struct raft_header *h,
+                              struct ovsdb_log *db_log_data)
+{
+    if (h->snap_index) {
+        if (!h->snap.data || json_array(h->snap.data)->n != 2) {
+        ovs_fatal(0, "Incorrect raft header data array length");
+        }
+
+        struct json *schema_json = json_array(h->snap.data)->elems[0];
+        if (schema_json->type != JSON_NULL) {
+            struct ovsdb_schema *schema;
+            check_ovsdb_error(ovsdb_schema_from_json(schema_json, &schema));
+            ovsdb_schema_destroy(schema);
+            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
+                                                       schema_json));
+        }
+
+        struct json *data_json = json_array(h->snap.data)->elems[1];
+        if (!data_json || data_json->type != JSON_OBJECT) {
+            ovs_fatal(0, "Invalid raft header data");
+        }
+        if (data_json->type != JSON_NULL) {
+            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
+                                                       data_json));
+        }
+    }
+}
+
+static void
+raft_record_to_standalone_log(const struct raft_record *r,
+                              struct ovsdb_log *db_log_data)
+{
+    if (r->type == RAFT_REC_ENTRY) {
+        if (!r->entry.data) {
+            return;
+        }
+        if (json_array(r->entry.data)->n != 2) {
+            ovs_fatal(0, "Incorrect raft record array length");
+        }
+
+        struct json *data_json = json_array(r->entry.data)->elems[1];
+        if (data_json->type != JSON_NULL) {
+            check_ovsdb_error(ovsdb_log_write_and_free(db_log_data,
+                                                       data_json));
+        }
+    }
+}
+
 static void
 do_show_log_cluster(struct ovsdb_log *log)
 {
@@ -1511,6 +1563,51 @@  do_compare_versions(struct ovs_cmdl_context *ctx)
     exit(result ? 0 : 2);
 }
 
+static void
+do_convert_to_standalone(struct ovsdb_log *log, struct ovsdb_log *db_log_data)
+{
+    for (unsigned int i = 0; ; i++) {
+        struct json *json;
+        check_ovsdb_error(ovsdb_log_read(log, &json));
+        if (!json) {
+            break;
+        }
+
+        if (i == 0) {
+            struct raft_header h;
+            check_ovsdb_error(raft_header_from_json(&h, json));
+            raft_header_to_standalone_log(&h, db_log_data);
+            raft_header_uninit(&h);
+        } else {
+            struct raft_record r;
+            check_ovsdb_error(raft_record_from_json(&r, json));
+            raft_record_to_standalone_log(&r, db_log_data);
+            raft_record_uninit(&r);
+        }
+    }
+}
+
+static void
+do_cluster_standalone(struct ovs_cmdl_context *ctx)
+{
+    const char *db_file_name = ctx->argv[1];
+    const char *cluster_db_file_name = ctx->argv[2];
+    struct ovsdb_log *log;
+    struct ovsdb_log *db_log_data;
+
+    check_ovsdb_error(ovsdb_log_open(cluster_db_file_name,
+                                     OVSDB_MAGIC"|"RAFT_MAGIC,
+                                     OVSDB_LOG_READ_ONLY, -1, &log));
+    check_ovsdb_error(ovsdb_log_open(db_file_name, OVSDB_MAGIC,
+                                     OVSDB_LOG_CREATE_EXCL, -1, &db_log_data));
+    if (strcmp(ovsdb_log_get_magic(log), RAFT_MAGIC) != 0) {
+        ovs_fatal(0, "Database is not clustered db.\n");
+    }
+    do_convert_to_standalone(log, db_log_data);
+    check_ovsdb_error(ovsdb_log_commit_block(db_log_data));
+    ovsdb_log_close(db_log_data);
+    ovsdb_log_close(log);
+}
 
 static void
 do_help(struct ovs_cmdl_context *ctx OVS_UNUSED)
@@ -1550,7 +1647,9 @@  static const struct ovs_cmdl_command all_commands[] = {
     { "compare-versions", "a op b", 3, 3, do_compare_versions, OVS_RO },
     { "help", NULL, 0, INT_MAX, do_help, OVS_RO },
     { "list-commands", NULL, 0, INT_MAX, do_list_commands, OVS_RO },
-    { NULL, NULL, 0, 0, NULL, OVS_RO },
+    { "cluster-to-standalone", "db clusterdb", 2, 2,
+    do_cluster_standalone, OVS_RW },
+    { NULL, NULL, 2, 2, NULL, OVS_RO },
 };
 
 static const struct ovs_cmdl_command *get_all_commands(void)
diff --git a/tests/ovsdb-tool.at b/tests/ovsdb-tool.at
index 69c5d6afa..cf98e4890 100644
--- a/tests/ovsdb-tool.at
+++ b/tests/ovsdb-tool.at
@@ -459,3 +459,49 @@  OVS_APP_EXIT_AND_WAIT([ovsdb-server])
 # Make sure that the clustered data matched the standalone data.
 AT_CHECK([cat dump2], [0], [expout])
 AT_CLEANUP
+
+AT_SETUP([ovsdb-tool convert-to-standalone])
+AT_KEYWORDS([ovsdb file positive])
+
+# Create a standalone database and put some data in it.
+ordinal_schema > schema
+ovsdb-tool create db1 schema
+AT_CHECK(
+  [[for pair in 'zero 0' 'one 1' 'two 2' 'three 3' 'four 4' 'five 5'; do
+      set -- $pair
+      ovsdb-tool transact db1 '
+        ["ordinals",
+         {"op": "insert",
+          "table": "ordinals",
+          "row": {"name": "'$1'", "number": '$2'}},
+         {"op": "comment",
+          "comment": "add row for '"$pair"'"}]'
+    done | uuidfilt]], [0],
+[[[{"uuid":["uuid","<0>"]},{}]
+[{"uuid":["uuid","<1>"]},{}]
+[{"uuid":["uuid","<2>"]},{}]
+[{"uuid":["uuid","<3>"]},{}]
+[{"uuid":["uuid","<4>"]},{}]
+[{"uuid":["uuid","<5>"]},{}]
+]], [ignore])
+
+# Dump the data.
+AT_CHECK([ovsdb-server -vfile -vvlog:off --detach --no-chdir --pidfile --log-file --remote=punix:db.sock db1])
+AT_CHECK([ovsdb-client dump > expout])
+OVS_APP_EXIT_AND_WAIT([ovsdb-server])
+
+# Create a clustered database from the standalone one.
+ovsdb-tool create-cluster db2 db1 unix:s1.raft
+
+# Convert to standalone database from clustered database.
+ovsdb-tool cluster-to-standalone db3 db2
+
+# Dump the data.
+AT_CHECK([ovsdb-server -vconsole:off -vfile -vvlog:off --detach --no-chdir --pidfile --log-file --remote=punix:db.sock db3])
+AT_CHECK([ovsdb_client_wait ordinals connected])
+AT_CHECK([ovsdb-client dump > dump3])
+OVS_APP_EXIT_AND_WAIT([ovsdb-server])
+
+# Make sure both standalone db data matches.
+AT_CHECK([cat dump3], [0], [expout])
+AT_CLEANUP