From patchwork Fri Aug 11 14:19:08 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Markus Armbruster X-Patchwork-Id: 800588 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xTS9C1Bgsz9sRq for ; Sat, 12 Aug 2017 00:30:55 +1000 (AEST) Received: from localhost ([::1]:44705 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dgAxg-0005vj-S2 for incoming@patchwork.ozlabs.org; Fri, 11 Aug 2017 10:30:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32806) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dgAmR-0004YE-NE for qemu-devel@nongnu.org; Fri, 11 Aug 2017 10:19:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dgAmN-0000GK-Fw for qemu-devel@nongnu.org; Fri, 11 Aug 2017 10:19:15 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51862) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dgAmN-0000FV-6m for qemu-devel@nongnu.org; Fri, 11 Aug 2017 10:19:11 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 44D4EB82A9 for ; Fri, 11 Aug 2017 14:19:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 44D4EB82A9 Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx01.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=armbru@redhat.com Received: from blackfin.pond.sub.org (ovpn-116-90.ams2.redhat.com [10.36.116.90]) by smtp.corp.redhat.com (Postfix) with ESMTPS id DCD2D8FB19; Fri, 11 Aug 2017 14:19:09 +0000 (UTC) Received: by blackfin.pond.sub.org (Postfix, from userid 1000) id 6FDA51138646; Fri, 11 Aug 2017 16:19:08 +0200 (CEST) From: Markus Armbruster To: qemu-devel@nongnu.org Date: Fri, 11 Aug 2017 16:19:08 +0200 Message-Id: <1502461148-10154-1-git-send-email-armbru@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Fri, 11 Aug 2017 14:19:10 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2] tests/qmp-test: Add generic, basic test of query commands X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: crosa@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" A command is a query if it has no side effect and yields a result. Such commands are typically named query-FOO, but there are exceptions. The basic idea is to find candidates with query-qmp-schema, filter out the ones that aren't queries with an explicit blacklist, and test the remaining ones against a QEMU with no special arguments. The current blacklist is just add-fd. The test can't do queries with arguments, because it knows nothing about the arguments. No coverage for query-cpu-model-baseline, query-cpu-model-comparison and query-cpu-model-expansion, because query-rocker, query-rocker-ports, query-rocker-of-dpa-flows and query-rocker-of-dpa-groups. Most tested commands are expected to succeed. The test does not check the return value then. query-balloon and query-vm-generation-id are expected to fail because they need a virtio-balloon / vmgenid device to succeed, and this test is too dumb to set one up. Could be addressed later. query-acpi-ospm-status and query-hotpluggable-cpus are expected to fail because they require features provided only by special machine types, and this test is too dumb to set that up. Could also be addressed later. Several commands may either be functional or stubs that always fail, depending on build configuration. Ideally, the stubs shouldn't be in query-qmp-schema, but that requires QAPI schema compile-time configuration, which we don't have, yet. Until we do, we need to figure out whether a command is a stub. When we have a suitable CONFIG_FOO preprocessor symbol is available, use that. Else, simply blacklist the command for now. We get basic test coverage for the following commands, except as noted: qom-list-types query-acpi-ospm-status (expected to fail) query-balloon (expected to fail) query-block query-block-jobs query-blockstats query-chardev query-chardev-backends query-command-line-options query-commands query-cpu-definitions (blacklisted for now) query-cpus query-dump query-dump-guest-memory-capability query-events query-fdsets query-gic-capabilities (blacklisted for now) query-hotpluggable-cpus (expected to fail) query-iothreads query-kvm query-machines query-memdev query-memory-devices query-mice query-migrate query-migrate-cache-size query-migrate-capabilities query-migrate-parameters query-name query-named-block-nodes query-pci (blacklisted for now) query-qmp-schema query-rx-filter query-spice query-status query-target query-tpm query-tpm-models query-tpm-types query-uuid query-version query-vm-generation-id (expected to fail) query-vnc query-vnc-servers query-xen-replication-status Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake --- v2: * Make the test pass for all targets (note to self: do not post patches after dark) * Tighten the test for query-spice, query-xen-replication-status: expect failure only when they are stubs * Don't blacklist query-acpi-ospm-status and query-hotpluggable-cpus; they fail reliably * R-by dropped tests/qmp-test.c | 182 ++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 181 insertions(+), 1 deletion(-) diff --git a/tests/qmp-test.c b/tests/qmp-test.c index 5d0260b..e5fb17c 100644 --- a/tests/qmp-test.c +++ b/tests/qmp-test.c @@ -15,6 +15,7 @@ #include "qapi-visit.h" #include "qapi/error.h" #include "qapi/qobject-input-visitor.h" +#include "qapi/util.h" #include "qapi/visitor.h" const char common_args[] = "-nodefaults -machine none"; @@ -129,11 +130,190 @@ static void test_qmp_protocol(void) qtest_end(); } +static int query_error_class(const char *cmd) +{ + static struct { + const char *cmd; + int err_class; + } fails[] = { + /* Success depends on build configuration: */ +#ifndef CONFIG_SPICE + "query-spice", ERROR_CLASS_COMMAND_NOT_FOUND, +#endif +#ifndef CONFIG_VNC + "query-vnc", ERROR_CLASS_GENERIC_ERROR }, + "query-vnc-servers", ERROR_CLASS_GENERIC_ERROR }, +#endif +#ifndef CONFIG_REPLICATION + "query-xen-replication-spice", ERROR_CLASS_COMMAND_NOT_FOUND, +#endif + /* Likewise, and require special QEMU command-line arguments: */ + { "query-acpi-ospm-status", ERROR_CLASS_GENERIC_ERROR }, + { "query-balloon", ERROR_CLASS_DEVICE_NOT_ACTIVE }, + { "query-hotpluggable-cpus", ERROR_CLASS_GENERIC_ERROR }, + { "query-vm-generation-id", ERROR_CLASS_GENERIC_ERROR }, + { NULL, -1 } + }; + int i; + + for (i = 0; fails[i].cmd; i++) { + if (!strcmp(cmd, fails[i].cmd)) { + return fails[i].err_class; + } + } + return -1; +} + +static void test_query(const void *data) +{ + const char *cmd = data; + int expected_error_class = query_error_class(cmd); + QDict *resp, *error; + const char *error_class; + + qtest_start(common_args); + + resp = qmp("{ 'execute': %s }", cmd); + error = qdict_get_qdict(resp, "error"); + error_class = error ? qdict_get_str(error, "class") : NULL; + + if (expected_error_class < 0) { + g_assert(qdict_haskey(resp, "return")); + } else { + g_assert(error); + g_assert_cmpint(qapi_enum_parse(QapiErrorClass_lookup, error_class, + QAPI_ERROR_CLASS__MAX, -1, + &error_abort), + ==, expected_error_class); + } + QDECREF(resp); + + qtest_end(); +} + +static bool query_is_blacklisted(const char *cmd) +{ + const char *blacklist[] = { + /* Not actually queries: */ + "add-fd", + /* Success depends on target arch: */ + "query-cpu-definitions", /* arm, i386, ppc, s390x */ + "query-gic-capabilities", /* arm */ + /* Success depends on target-specific build configuration: */ + "query-pci", /* CONFIG_PCI */ + NULL + }; + int i; + + for (i = 0; blacklist[i]; i++) { + if (!strcmp(cmd, blacklist[i])) { + return true; + } + } + return false; +} + +typedef struct { + SchemaInfoList *list; + GHashTable *hash; +} QmpSchema; + +static void qmp_schema_init(QmpSchema *schema) +{ + QDict *resp; + Visitor *qiv; + SchemaInfoList *tail; + + qtest_start(common_args); + resp = qmp("{ 'execute': 'query-qmp-schema' }"); + + qiv = qobject_input_visitor_new(qdict_get(resp, "return")); + visit_type_SchemaInfoList(qiv, NULL, &schema->list, &error_abort); + visit_free(qiv); + + QDECREF(resp); + qtest_end(); + + schema->hash = g_hash_table_new(g_str_hash, g_str_equal); + + /* Build @schema: hash table mapping entity name to SchemaInfo */ + for (tail = schema->list; tail; tail = tail->next) { + g_hash_table_insert(schema->hash, tail->value->name, tail->value); + } +} + +static SchemaInfo *qmp_schema_lookup(QmpSchema *schema, const char *name) +{ + return g_hash_table_lookup(schema->hash, name); +} + +static void qmp_schema_cleanup(QmpSchema *schema) +{ + qapi_free_SchemaInfoList(schema->list); + g_hash_table_destroy(schema->hash); +} + +static bool object_type_has_mandatory_members(SchemaInfo *type) +{ + SchemaInfoObjectMemberList *tail; + + g_assert(type->meta_type == SCHEMA_META_TYPE_OBJECT); + + for (tail = type->u.object.members; tail; tail = tail->next) { + if (!tail->value->has_q_default) { + return true; + } + } + + return false; +} + +static void add_query_tests(QmpSchema *schema) +{ + SchemaInfoList *tail; + SchemaInfo *si, *arg_type, *ret_type; + const char *test_name; + + /* Test the query-like commands */ + for (tail = schema->list; tail; tail = tail->next) { + si = tail->value; + if (si->meta_type != SCHEMA_META_TYPE_COMMAND) { + continue; + } + + if (query_is_blacklisted(si->name)) { + continue; + } + + arg_type = qmp_schema_lookup(schema, si->u.command.arg_type); + if (object_type_has_mandatory_members(arg_type)) { + continue; + } + + ret_type = qmp_schema_lookup(schema, si->u.command.ret_type); + if (ret_type->meta_type == SCHEMA_META_TYPE_OBJECT + && !ret_type->u.object.members) { + continue; + } + + test_name = g_strdup_printf("qmp/%s", si->name); + qtest_add_data_func(test_name, si->name, test_query); + } +} + int main(int argc, char *argv[]) { + QmpSchema schema; + int ret; + g_test_init(&argc, &argv, NULL); qtest_add_func("qmp/protocol", test_qmp_protocol); + qmp_schema_init(&schema); + add_query_tests(&schema); - return g_test_run(); + ret = g_test_run(); + + qmp_schema_cleanup(&schema); + return ret; }