diff mbox

[v5,13/28] qapi: Add some expr tests

Message ID 1427227433-5030-14-git-send-email-eblake@redhat.com
State New
Headers show

Commit Message

Eric Blake March 24, 2015, 8:03 p.m. UTC
Demonstrate that the qapi generator doesn't deal well with
expressions that aren't up to par. Later patches will improve
the expected results as the generator is made stricter.  Only
one of the added tests actually behaves sanely at rejecting
obvious problems.

Note that in some cases, we reject bad QAPI merely because our
pseudo-JSON parser does not yet know how to parse numbers.  This
series does not address that, but when a later series adds support
for numeric defaults of integer fields, the testsuite will ensure
that we don't lose the error (and hopefully that the error
message quality is improved).

Signed-off-by: Eric Blake <eblake@redhat.com>
---
 tests/Makefile                          | 6 ++++--
 tests/qapi-schema/bad-base.err          | 0
 tests/qapi-schema/bad-base.exit         | 1 +
 tests/qapi-schema/bad-base.json         | 3 +++
 tests/qapi-schema/bad-base.out          | 4 ++++
 tests/qapi-schema/bad-ident.err         | 0
 tests/qapi-schema/bad-ident.exit        | 1 +
 tests/qapi-schema/bad-ident.json        | 3 +++
 tests/qapi-schema/bad-ident.out         | 3 +++
 tests/qapi-schema/bad-type-bool.err     | 1 +
 tests/qapi-schema/bad-type-bool.exit    | 1 +
 tests/qapi-schema/bad-type-bool.json    | 3 +++
 tests/qapi-schema/bad-type-bool.out     | 0
 tests/qapi-schema/bad-type-dict.err     | 0
 tests/qapi-schema/bad-type-dict.exit    | 1 +
 tests/qapi-schema/bad-type-dict.json    | 2 ++
 tests/qapi-schema/bad-type-dict.out     | 3 +++
 tests/qapi-schema/bad-type-int.err      | 1 +
 tests/qapi-schema/bad-type-int.exit     | 1 +
 tests/qapi-schema/bad-type-int.json     | 3 +++
 tests/qapi-schema/bad-type-int.out      | 0
 tests/qapi-schema/double-data.err       | 1 +
 tests/qapi-schema/double-data.exit      | 1 +
 tests/qapi-schema/double-data.json      | 2 ++
 tests/qapi-schema/double-data.out       | 0
 tests/qapi-schema/double-type.err       | 0
 tests/qapi-schema/double-type.exit      | 1 +
 tests/qapi-schema/double-type.json      | 2 ++
 tests/qapi-schema/double-type.out       | 3 +++
 tests/qapi-schema/event-case.err        | 0
 tests/qapi-schema/event-case.exit       | 1 +
 tests/qapi-schema/event-case.json       | 2 ++
 tests/qapi-schema/event-case.out        | 3 +++
 tests/qapi-schema/missing-type.err      | 0
 tests/qapi-schema/missing-type.exit     | 1 +
 tests/qapi-schema/missing-type.json     | 2 ++
 tests/qapi-schema/missing-type.out      | 3 +++
 tests/qapi-schema/unknown-expr-key.err  | 0
 tests/qapi-schema/unknown-expr-key.exit | 1 +
 tests/qapi-schema/unknown-expr-key.json | 2 ++
 tests/qapi-schema/unknown-expr-key.out  | 3 +++
 41 files changed, 63 insertions(+), 2 deletions(-)
 create mode 100644 tests/qapi-schema/bad-base.err
 create mode 100644 tests/qapi-schema/bad-base.exit
 create mode 100644 tests/qapi-schema/bad-base.json
 create mode 100644 tests/qapi-schema/bad-base.out
 create mode 100644 tests/qapi-schema/bad-ident.err
 create mode 100644 tests/qapi-schema/bad-ident.exit
 create mode 100644 tests/qapi-schema/bad-ident.json
 create mode 100644 tests/qapi-schema/bad-ident.out
 create mode 100644 tests/qapi-schema/bad-type-bool.err
 create mode 100644 tests/qapi-schema/bad-type-bool.exit
 create mode 100644 tests/qapi-schema/bad-type-bool.json
 create mode 100644 tests/qapi-schema/bad-type-bool.out
 create mode 100644 tests/qapi-schema/bad-type-dict.err
 create mode 100644 tests/qapi-schema/bad-type-dict.exit
 create mode 100644 tests/qapi-schema/bad-type-dict.json
 create mode 100644 tests/qapi-schema/bad-type-dict.out
 create mode 100644 tests/qapi-schema/bad-type-int.err
 create mode 100644 tests/qapi-schema/bad-type-int.exit
 create mode 100644 tests/qapi-schema/bad-type-int.json
 create mode 100644 tests/qapi-schema/bad-type-int.out
 create mode 100644 tests/qapi-schema/double-data.err
 create mode 100644 tests/qapi-schema/double-data.exit
 create mode 100644 tests/qapi-schema/double-data.json
 create mode 100644 tests/qapi-schema/double-data.out
 create mode 100644 tests/qapi-schema/double-type.err
 create mode 100644 tests/qapi-schema/double-type.exit
 create mode 100644 tests/qapi-schema/double-type.json
 create mode 100644 tests/qapi-schema/double-type.out
 create mode 100644 tests/qapi-schema/event-case.err
 create mode 100644 tests/qapi-schema/event-case.exit
 create mode 100644 tests/qapi-schema/event-case.json
 create mode 100644 tests/qapi-schema/event-case.out
 create mode 100644 tests/qapi-schema/missing-type.err
 create mode 100644 tests/qapi-schema/missing-type.exit
 create mode 100644 tests/qapi-schema/missing-type.json
 create mode 100644 tests/qapi-schema/missing-type.out
 create mode 100644 tests/qapi-schema/unknown-expr-key.err
 create mode 100644 tests/qapi-schema/unknown-expr-key.exit
 create mode 100644 tests/qapi-schema/unknown-expr-key.json
 create mode 100644 tests/qapi-schema/unknown-expr-key.out

Comments

Markus Armbruster March 26, 2015, 3:55 p.m. UTC | #1
Eric Blake <eblake@redhat.com> writes:

> Demonstrate that the qapi generator doesn't deal well with
> expressions that aren't up to par. Later patches will improve
> the expected results as the generator is made stricter.  Only
> one of the added tests actually behaves sanely at rejecting
> obvious problems.
>
> Note that in some cases, we reject bad QAPI merely because our
> pseudo-JSON parser does not yet know how to parse numbers.  This
> series does not address that, but when a later series adds support
> for numeric defaults of integer fields, the testsuite will ensure
> that we don't lose the error (and hopefully that the error
> message quality is improved).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
[...]
> new file mode 100644
> index 0000000..e69de29
> diff --git a/tests/qapi-schema/event-case.exit b/tests/qapi-schema/event-case.exit
> new file mode 100644
> index 0000000..573541a
> --- /dev/null
> +++ b/tests/qapi-schema/event-case.exit
> @@ -0,0 +1 @@
> +0
> diff --git a/tests/qapi-schema/event-case.json b/tests/qapi-schema/event-case.json
> new file mode 100644
> index 0000000..52dfc3a
> --- /dev/null
> +++ b/tests/qapi-schema/event-case.json
> @@ -0,0 +1,2 @@
> +# FIXME: we should reject an event name that is not all caps
> +{ 'event': 'oops' }

qapi-code-gen.txt documents the naming conventions:

    Types, commands, and events share a common namespace.  Therefore,
    generally speaking, type definitions should always use CamelCase for
    user-defined type names, while built-in types are lowercase. Type
    definitions should not end in 'Kind', as this namespace is used for
    creating implicit C enums for visiting union types.  Command names,
    and field names within a type, should be all lower case with words
    separated by a hyphen.  However, some existing older commands and
    complex types use underscore; when extending such expressions,
    consistency is preferred over blindly avoiding underscore.  Event
    names should be ALL_CAPS with words separated by underscore.  The
    special string '**' appears for some commands that manually perform
    their own type checking rather than relying on the type-safe code
    produced by the qapi code generators.

We should either enforce the conventions consistently, or not at all.

Enforcing them makes certain kinds of name clashes in generated C
impossible.  If we don't enforce them, we should catch the clashes.

Since I haven't read to the end of your series, I have to ask: do you
intend to enforce them?

> diff --git a/tests/qapi-schema/event-case.out b/tests/qapi-schema/event-case.out
> new file mode 100644
> index 0000000..3764bc7
> --- /dev/null
> +++ b/tests/qapi-schema/event-case.out
> @@ -0,0 +1,3 @@
> +[OrderedDict([('event', 'oops')])]
> +[]
> +[]
[...]
Eric Blake March 26, 2015, 7:02 p.m. UTC | #2
On 03/26/2015 09:55 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
> 
>> Demonstrate that the qapi generator doesn't deal well with
>> expressions that aren't up to par. Later patches will improve
>> the expected results as the generator is made stricter.  Only
>> one of the added tests actually behaves sanely at rejecting
>> obvious problems.
>>
> 
> qapi-code-gen.txt documents the naming conventions:
> 
>     Types, commands, and events share a common namespace.  Therefore,
>     generally speaking, type definitions should always use CamelCase for
>     user-defined type names, while built-in types are lowercase. Type
>     definitions should not end in 'Kind', as this namespace is used for
>     creating implicit C enums for visiting union types.  Command names,
>     and field names within a type, should be all lower case with words
>     separated by a hyphen.  However, some existing older commands and
>     complex types use underscore; when extending such expressions,
>     consistency is preferred over blindly avoiding underscore.  Event
>     names should be ALL_CAPS with words separated by underscore.  The
>     special string '**' appears for some commands that manually perform
>     their own type checking rather than relying on the type-safe code
>     produced by the qapi code generators.
> 
> We should either enforce the conventions consistently, or not at all.
> 
> Enforcing them makes certain kinds of name clashes in generated C
> impossible.  If we don't enforce them, we should catch the clashes.
> 
> Since I haven't read to the end of your series, I have to ask: do you
> intend to enforce them?

I added tests to enforce it for event names, but did not enforce things
for command names or complex type members.  I guess that can be added on
top, if desired.

So, did this patch get R-b?
Markus Armbruster March 27, 2015, 12:38 p.m. UTC | #3
Eric Blake <eblake@redhat.com> writes:

> On 03/26/2015 09:55 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>> 
>>> Demonstrate that the qapi generator doesn't deal well with
>>> expressions that aren't up to par. Later patches will improve
>>> the expected results as the generator is made stricter.  Only
>>> one of the added tests actually behaves sanely at rejecting
>>> obvious problems.
>>>
>> 
>> qapi-code-gen.txt documents the naming conventions:
>> 
>>     Types, commands, and events share a common namespace.  Therefore,
>>     generally speaking, type definitions should always use CamelCase for
>>     user-defined type names, while built-in types are lowercase. Type
>>     definitions should not end in 'Kind', as this namespace is used for
>>     creating implicit C enums for visiting union types.  Command names,
>>     and field names within a type, should be all lower case with words
>>     separated by a hyphen.  However, some existing older commands and
>>     complex types use underscore; when extending such expressions,
>>     consistency is preferred over blindly avoiding underscore.  Event
>>     names should be ALL_CAPS with words separated by underscore.  The
>>     special string '**' appears for some commands that manually perform
>>     their own type checking rather than relying on the type-safe code
>>     produced by the qapi code generators.
>> 
>> We should either enforce the conventions consistently, or not at all.
>> 
>> Enforcing them makes certain kinds of name clashes in generated C
>> impossible.  If we don't enforce them, we should catch the clashes.
>> 
>> Since I haven't read to the end of your series, I have to ask: do you
>> intend to enforce them?
>
> I added tests to enforce it for event names, but did not enforce things
> for command names or complex type members.  I guess that can be added on
> top, if desired.
>
> So, did this patch get R-b?

I'd rather not enforce naming conventions just for events.

If we want to enforce them, let's do it consistently, and in a separate
series that includes this patch.  Okay?
Eric Blake March 27, 2015, 7:39 p.m. UTC | #4
On 03/27/2015 06:38 AM, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
> 
>> On 03/26/2015 09:55 AM, Markus Armbruster wrote:
>>> Eric Blake <eblake@redhat.com> writes:
>>>
>>>> Demonstrate that the qapi generator doesn't deal well with
>>>> expressions that aren't up to par. Later patches will improve
>>>> the expected results as the generator is made stricter.  Only
>>>> one of the added tests actually behaves sanely at rejecting
>>>> obvious problems.
>>>>
>>>
>>> qapi-code-gen.txt documents the naming conventions:
>>>
>>>     Types, commands, and events share a common namespace.  Therefore,
>>>     generally speaking, type definitions should always use CamelCase for
>>>     user-defined type names, while built-in types are lowercase. Type
>>>     definitions should not end in 'Kind', as this namespace is used for
>>>     creating implicit C enums for visiting union types.  Command names,
>>>     and field names within a type, should be all lower case with words
>>>     separated by a hyphen.  However, some existing older commands and
>>>     complex types use underscore; when extending such expressions,
>>>     consistency is preferred over blindly avoiding underscore.  Event
>>>     names should be ALL_CAPS with words separated by underscore.  The
>>>     special string '**' appears for some commands that manually perform
>>>     their own type checking rather than relying on the type-safe code
>>>     produced by the qapi code generators.
>>>
>>> We should either enforce the conventions consistently, or not at all.
>>>
>>> Enforcing them makes certain kinds of name clashes in generated C
>>> impossible.  If we don't enforce them, we should catch the clashes.
>>>
>>> Since I haven't read to the end of your series, I have to ask: do you
>>> intend to enforce them?
>>
>> I added tests to enforce it for event names, but did not enforce things
>> for command names or complex type members.  I guess that can be added on
>> top, if desired.
>>
>> So, did this patch get R-b?
> 
> I'd rather not enforce naming conventions just for events.
> 
> If we want to enforce them, let's do it consistently, and in a separate
> series that includes this patch.  Okay?

Sounds like I need a v6 respin then, where I drop my patch that attempts
to enforce all-caps event naming but did not enforce type or command
naming; but I will keep everything else (enforcing that names are valid
C identifiers + '-' and '.' (which both get flattened to '_').
Markus Armbruster March 29, 2015, 8:27 a.m. UTC | #5
Eric Blake <eblake@redhat.com> writes:

> On 03/27/2015 06:38 AM, Markus Armbruster wrote:
>> Eric Blake <eblake@redhat.com> writes:
>> 
>>> On 03/26/2015 09:55 AM, Markus Armbruster wrote:
>>>> Eric Blake <eblake@redhat.com> writes:
>>>>
>>>>> Demonstrate that the qapi generator doesn't deal well with
>>>>> expressions that aren't up to par. Later patches will improve
>>>>> the expected results as the generator is made stricter.  Only
>>>>> one of the added tests actually behaves sanely at rejecting
>>>>> obvious problems.
>>>>>
>>>>
>>>> qapi-code-gen.txt documents the naming conventions:
>>>>
>>>>     Types, commands, and events share a common namespace.  Therefore,
>>>>     generally speaking, type definitions should always use CamelCase for
>>>>     user-defined type names, while built-in types are lowercase. Type
>>>>     definitions should not end in 'Kind', as this namespace is used for
>>>>     creating implicit C enums for visiting union types.  Command names,
>>>>     and field names within a type, should be all lower case with words
>>>>     separated by a hyphen.  However, some existing older commands and
>>>>     complex types use underscore; when extending such expressions,
>>>>     consistency is preferred over blindly avoiding underscore.  Event
>>>>     names should be ALL_CAPS with words separated by underscore.  The
>>>>     special string '**' appears for some commands that manually perform
>>>>     their own type checking rather than relying on the type-safe code
>>>>     produced by the qapi code generators.
>>>>
>>>> We should either enforce the conventions consistently, or not at all.
>>>>
>>>> Enforcing them makes certain kinds of name clashes in generated C
>>>> impossible.  If we don't enforce them, we should catch the clashes.
>>>>
>>>> Since I haven't read to the end of your series, I have to ask: do you
>>>> intend to enforce them?
>>>
>>> I added tests to enforce it for event names, but did not enforce things
>>> for command names or complex type members.  I guess that can be added on
>>> top, if desired.
>>>
>>> So, did this patch get R-b?
>> 
>> I'd rather not enforce naming conventions just for events.
>> 
>> If we want to enforce them, let's do it consistently, and in a separate
>> series that includes this patch.  Okay?
>
> Sounds like I need a v6 respin then, where I drop my patch that attempts
> to enforce all-caps event naming but did not enforce type or command
> naming; but I will keep everything else (enforcing that names are valid
> C identifiers + '-' and '.' (which both get flattened to '_').

Sounds good!
diff mbox

Patch

diff --git a/tests/Makefile b/tests/Makefile
index f8bc2a8..cf9c42b 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -210,7 +210,9 @@  check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
 	comments.json empty.json enum-empty.json enum-missing-data.json \
 	enum-wrong-data.json enum-int-member.json enum-dict-member.json \
 	enum-clash-member.json enum-max-member.json enum-union-clash.json \
-	funny-char.json indented-expr.json \
+	funny-char.json indented-expr.json missing-type.json bad-ident.json \
+	double-type.json bad-base.json bad-type-bool.json bad-type-int.json \
+	bad-type-dict.json double-data.json unknown-expr-key.json \
 	missing-colon.json missing-comma-list.json \
 	missing-comma-object.json non-objects.json \
 	qapi-schema-test.json quoted-structural-chars.json \
@@ -229,7 +231,7 @@  check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
 	include-simple.json include-relpath.json include-format-err.json \
 	include-non-file.json include-no-file.json include-before-err.json \
 	include-nested-err.json include-self-cycle.json include-cycle.json \
-	include-repetition.json event-nest-struct.json)
+	include-repetition.json event-nest-struct.json event-case.json)

 GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h \
 		     tests/test-qmp-commands.h tests/test-qapi-event.h
diff --git a/tests/qapi-schema/bad-base.err b/tests/qapi-schema/bad-base.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/bad-base.exit b/tests/qapi-schema/bad-base.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/bad-base.exit
@@ -0,0 +1 @@ 
+0
diff --git a/tests/qapi-schema/bad-base.json b/tests/qapi-schema/bad-base.json
new file mode 100644
index 0000000..de964a0
--- /dev/null
+++ b/tests/qapi-schema/bad-base.json
@@ -0,0 +1,3 @@ 
+# FIXME: we should reject a base that is not a struct
+{ 'union': 'Union', 'data': { 'a': 'int', 'b': 'str' } }
+{ 'type': 'MyType', 'base': 'Union', 'data': { 'c': 'int' } }
diff --git a/tests/qapi-schema/bad-base.out b/tests/qapi-schema/bad-base.out
new file mode 100644
index 0000000..91d12fc
--- /dev/null
+++ b/tests/qapi-schema/bad-base.out
@@ -0,0 +1,4 @@ 
+[OrderedDict([('union', 'Union'), ('data', OrderedDict([('a', 'int'), ('b', 'str')]))]),
+ OrderedDict([('type', 'MyType'), ('base', 'Union'), ('data', OrderedDict([('c', 'int')]))])]
+[{'enum_name': 'UnionKind', 'enum_values': None}]
+[OrderedDict([('type', 'MyType'), ('base', 'Union'), ('data', OrderedDict([('c', 'int')]))])]
diff --git a/tests/qapi-schema/bad-ident.err b/tests/qapi-schema/bad-ident.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/bad-ident.exit b/tests/qapi-schema/bad-ident.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/bad-ident.exit
@@ -0,0 +1 @@ 
+0
diff --git a/tests/qapi-schema/bad-ident.json b/tests/qapi-schema/bad-ident.json
new file mode 100644
index 0000000..66333a7
--- /dev/null
+++ b/tests/qapi-schema/bad-ident.json
@@ -0,0 +1,3 @@ 
+# FIXME: we should reject creating a type name with bad name
+{ 'type': '*oops', 'data': { 'i': 'int' } }
+
diff --git a/tests/qapi-schema/bad-ident.out b/tests/qapi-schema/bad-ident.out
new file mode 100644
index 0000000..165e346
--- /dev/null
+++ b/tests/qapi-schema/bad-ident.out
@@ -0,0 +1,3 @@ 
+[OrderedDict([('type', '*oops'), ('data', OrderedDict([('i', 'int')]))])]
+[]
+[OrderedDict([('type', '*oops'), ('data', OrderedDict([('i', 'int')]))])]
diff --git a/tests/qapi-schema/bad-type-bool.err b/tests/qapi-schema/bad-type-bool.err
new file mode 100644
index 0000000..badb7c2
--- /dev/null
+++ b/tests/qapi-schema/bad-type-bool.err
@@ -0,0 +1 @@ 
+tests/qapi-schema/bad-type-bool.json:3:11: Stray "t"
diff --git a/tests/qapi-schema/bad-type-bool.exit b/tests/qapi-schema/bad-type-bool.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/bad-type-bool.exit
@@ -0,0 +1 @@ 
+1
diff --git a/tests/qapi-schema/bad-type-bool.json b/tests/qapi-schema/bad-type-bool.json
new file mode 100644
index 0000000..22d6369
--- /dev/null
+++ b/tests/qapi-schema/bad-type-bool.json
@@ -0,0 +1,3 @@ 
+# we reject an expression with a metatype that is not a string
+# FIXME: once the parser understands bool inputs, improve the error message
+{ 'type': true, 'data': { } }
diff --git a/tests/qapi-schema/bad-type-bool.out b/tests/qapi-schema/bad-type-bool.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/bad-type-dict.err b/tests/qapi-schema/bad-type-dict.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/bad-type-dict.exit b/tests/qapi-schema/bad-type-dict.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/bad-type-dict.exit
@@ -0,0 +1 @@ 
+0
diff --git a/tests/qapi-schema/bad-type-dict.json b/tests/qapi-schema/bad-type-dict.json
new file mode 100644
index 0000000..3c392a7
--- /dev/null
+++ b/tests/qapi-schema/bad-type-dict.json
@@ -0,0 +1,2 @@ 
+# FIXME: we should reject an expression with a metatype that is not a string
+{ 'command': { } }
diff --git a/tests/qapi-schema/bad-type-dict.out b/tests/qapi-schema/bad-type-dict.out
new file mode 100644
index 0000000..c62f1ed
--- /dev/null
+++ b/tests/qapi-schema/bad-type-dict.out
@@ -0,0 +1,3 @@ 
+[OrderedDict([('command', OrderedDict())])]
+[]
+[]
diff --git a/tests/qapi-schema/bad-type-int.err b/tests/qapi-schema/bad-type-int.err
new file mode 100644
index 0000000..9808550
--- /dev/null
+++ b/tests/qapi-schema/bad-type-int.err
@@ -0,0 +1 @@ 
+tests/qapi-schema/bad-type-int.json:3:11: Stray "1"
diff --git a/tests/qapi-schema/bad-type-int.exit b/tests/qapi-schema/bad-type-int.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/bad-type-int.exit
@@ -0,0 +1 @@ 
+1
diff --git a/tests/qapi-schema/bad-type-int.json b/tests/qapi-schema/bad-type-int.json
new file mode 100644
index 0000000..398879d
--- /dev/null
+++ b/tests/qapi-schema/bad-type-int.json
@@ -0,0 +1,3 @@ 
+# we reject an expression with a metatype that is not a string
+# FIXME: once the parser understands integer inputs, improve the error message
+{ 'type': 1, 'data': { } }
diff --git a/tests/qapi-schema/bad-type-int.out b/tests/qapi-schema/bad-type-int.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/double-data.err b/tests/qapi-schema/double-data.err
new file mode 100644
index 0000000..6f1a67b
--- /dev/null
+++ b/tests/qapi-schema/double-data.err
@@ -0,0 +1 @@ 
+tests/qapi-schema/double-data.json:2:39: Duplicate key "data"
diff --git a/tests/qapi-schema/double-data.exit b/tests/qapi-schema/double-data.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/double-data.exit
@@ -0,0 +1 @@ 
+1
diff --git a/tests/qapi-schema/double-data.json b/tests/qapi-schema/double-data.json
new file mode 100644
index 0000000..a94b7df
--- /dev/null
+++ b/tests/qapi-schema/double-data.json
@@ -0,0 +1,2 @@ 
+# we reject an expression with duplicate top-level keys
+{ 'type': 'bar', 'data': { }, 'data': { 'string': 'str'} }
diff --git a/tests/qapi-schema/double-data.out b/tests/qapi-schema/double-data.out
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/double-type.err b/tests/qapi-schema/double-type.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/double-type.exit b/tests/qapi-schema/double-type.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/double-type.exit
@@ -0,0 +1 @@ 
+0
diff --git a/tests/qapi-schema/double-type.json b/tests/qapi-schema/double-type.json
new file mode 100644
index 0000000..6ca96b9
--- /dev/null
+++ b/tests/qapi-schema/double-type.json
@@ -0,0 +1,2 @@ 
+# FIXME: we should reject an expression with ambiguous metatype
+{ 'command': 'foo', 'type': 'bar', 'data': { } }
diff --git a/tests/qapi-schema/double-type.out b/tests/qapi-schema/double-type.out
new file mode 100644
index 0000000..3e244f5
--- /dev/null
+++ b/tests/qapi-schema/double-type.out
@@ -0,0 +1,3 @@ 
+[OrderedDict([('command', 'foo'), ('type', 'bar'), ('data', OrderedDict())])]
+[]
+[OrderedDict([('command', 'foo'), ('type', 'bar'), ('data', OrderedDict())])]
diff --git a/tests/qapi-schema/event-case.err b/tests/qapi-schema/event-case.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/event-case.exit b/tests/qapi-schema/event-case.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/event-case.exit
@@ -0,0 +1 @@ 
+0
diff --git a/tests/qapi-schema/event-case.json b/tests/qapi-schema/event-case.json
new file mode 100644
index 0000000..52dfc3a
--- /dev/null
+++ b/tests/qapi-schema/event-case.json
@@ -0,0 +1,2 @@ 
+# FIXME: we should reject an event name that is not all caps
+{ 'event': 'oops' }
diff --git a/tests/qapi-schema/event-case.out b/tests/qapi-schema/event-case.out
new file mode 100644
index 0000000..3764bc7
--- /dev/null
+++ b/tests/qapi-schema/event-case.out
@@ -0,0 +1,3 @@ 
+[OrderedDict([('event', 'oops')])]
+[]
+[]
diff --git a/tests/qapi-schema/missing-type.err b/tests/qapi-schema/missing-type.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/missing-type.exit b/tests/qapi-schema/missing-type.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/missing-type.exit
@@ -0,0 +1 @@ 
+0
diff --git a/tests/qapi-schema/missing-type.json b/tests/qapi-schema/missing-type.json
new file mode 100644
index 0000000..1696f5c
--- /dev/null
+++ b/tests/qapi-schema/missing-type.json
@@ -0,0 +1,2 @@ 
+# FIXME: we should reject an expression with missing metatype
+{ 'data': { } }
diff --git a/tests/qapi-schema/missing-type.out b/tests/qapi-schema/missing-type.out
new file mode 100644
index 0000000..67fd4fa
--- /dev/null
+++ b/tests/qapi-schema/missing-type.out
@@ -0,0 +1,3 @@ 
+[OrderedDict([('data', OrderedDict())])]
+[]
+[]
diff --git a/tests/qapi-schema/unknown-expr-key.err b/tests/qapi-schema/unknown-expr-key.err
new file mode 100644
index 0000000..e69de29
diff --git a/tests/qapi-schema/unknown-expr-key.exit b/tests/qapi-schema/unknown-expr-key.exit
new file mode 100644
index 0000000..573541a
--- /dev/null
+++ b/tests/qapi-schema/unknown-expr-key.exit
@@ -0,0 +1 @@ 
+0
diff --git a/tests/qapi-schema/unknown-expr-key.json b/tests/qapi-schema/unknown-expr-key.json
new file mode 100644
index 0000000..1e9282d
--- /dev/null
+++ b/tests/qapi-schema/unknown-expr-key.json
@@ -0,0 +1,2 @@ 
+# FIXME: we should reject an expression with unknown top-level keys
+{ 'type': 'bar', 'data': { 'string': 'str'}, 'bogus': { } }
diff --git a/tests/qapi-schema/unknown-expr-key.out b/tests/qapi-schema/unknown-expr-key.out
new file mode 100644
index 0000000..c93f020
--- /dev/null
+++ b/tests/qapi-schema/unknown-expr-key.out
@@ -0,0 +1,3 @@ 
+[OrderedDict([('type', 'bar'), ('data', OrderedDict([('string', 'str')])), ('bogus', OrderedDict())])]
+[]
+[OrderedDict([('type', 'bar'), ('data', OrderedDict([('string', 'str')])), ('bogus', OrderedDict())])]