diff mbox series

[2/3] tests: add oob-test for qapi-schema

Message ID 20180321065506.21091-3-peterx@redhat.com
State New
Headers show
Series tests: trivial enhancements for OOB | expand

Commit Message

Peter Xu March 21, 2018, 6:55 a.m. UTC
It simply tests the new OOB capability, and make sure the QAPISchema can
parse it correctly.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 tests/Makefile.include          | 1 +
 tests/qapi-schema/oob-test.err  | 0
 tests/qapi-schema/oob-test.exit | 1 +
 tests/qapi-schema/oob-test.json | 2 ++
 tests/qapi-schema/oob-test.out  | 6 ++++++
 5 files changed, 10 insertions(+)
 create mode 100644 tests/qapi-schema/oob-test.err
 create mode 100644 tests/qapi-schema/oob-test.exit
 create mode 100644 tests/qapi-schema/oob-test.json
 create mode 100644 tests/qapi-schema/oob-test.out

Comments

Eric Blake March 21, 2018, 12:52 p.m. UTC | #1
On 03/21/2018 01:55 AM, Peter Xu wrote:
> It simply tests the new OOB capability, and make sure the QAPISchema can
> parse it correctly.

We also want negative tests that cover any new error messages in the 
qapi generator (such as 'allow-oob':'bad' diagnosing a non-bool, or 
'allow-oob':false giving an error message that false is already the 
default such that only 'allow-oob':true makes sense).

Also, it's often easier to merge the positive test into the giant 
existing qapi-schema-test.json rather than creating a new positive test.

> +++ b/tests/qapi-schema/oob-test.out
> @@ -0,0 +1,6 @@
> +object q_empty
> +enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
> +    prefix QTYPE
> +module oob-test.json
> +command an-oob-command None -> None
> +   gen=True success_response=True boxed=False oob=True

At any rate, the positive test addition is good. I may beat you to 
submitting a v2 patch that covers the error messages that I'm thinking of.
Peter Xu March 22, 2018, 3:43 a.m. UTC | #2
On Wed, Mar 21, 2018 at 07:52:06AM -0500, Eric Blake wrote:
> On 03/21/2018 01:55 AM, Peter Xu wrote:
> > It simply tests the new OOB capability, and make sure the QAPISchema can
> > parse it correctly.
> 
> We also want negative tests that cover any new error messages in the qapi
> generator (such as 'allow-oob':'bad' diagnosing a non-bool, or
> 'allow-oob':false giving an error message that false is already the default
> such that only 'allow-oob':true makes sense).
> 
> Also, it's often easier to merge the positive test into the giant existing
> qapi-schema-test.json rather than creating a new positive test.

It seems that for one QAPI schema negative test only the first error
will be reported, then the script halts (so the 2nd negative test in
the same .json won't be reported).  To make it simple - I'll put the
positive test into qapi-schema-test.json, and add one negative test in
oob-test.json to check again strings (though in the code I'll only
allow 'false').

(Actually I'll need one liner change to check that value when parsing
 since it was not checked before...)

> 
> > +++ b/tests/qapi-schema/oob-test.out
> > @@ -0,0 +1,6 @@
> > +object q_empty
> > +enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
> > +    prefix QTYPE
> > +module oob-test.json
> > +command an-oob-command None -> None
> > +   gen=True success_response=True boxed=False oob=True
> 
> At any rate, the positive test addition is good. I may beat you to
> submitting a v2 patch that covers the error messages that I'm thinking of.

Will post another version (and I'll see whether I should queue more to
fix existing reported OOB problems).  Thanks,
diff mbox series

Patch

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 0b277036df..059523e2d1 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -523,6 +523,7 @@  qapi-schema += missing-comma-object.json
 qapi-schema += missing-type.json
 qapi-schema += nested-struct-data.json
 qapi-schema += non-objects.json
+qapi-schema += oob-test.json
 qapi-schema += pragma-doc-required-crap.json
 qapi-schema += pragma-extra-junk.json
 qapi-schema += pragma-name-case-whitelist-crap.json
diff --git a/tests/qapi-schema/oob-test.err b/tests/qapi-schema/oob-test.err
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/qapi-schema/oob-test.exit b/tests/qapi-schema/oob-test.exit
new file mode 100644
index 0000000000..573541ac97
--- /dev/null
+++ b/tests/qapi-schema/oob-test.exit
@@ -0,0 +1 @@ 
+0
diff --git a/tests/qapi-schema/oob-test.json b/tests/qapi-schema/oob-test.json
new file mode 100644
index 0000000000..096c34b2fb
--- /dev/null
+++ b/tests/qapi-schema/oob-test.json
@@ -0,0 +1,2 @@ 
+# Some Out-Of-Band specific tests
+{ 'command': 'an-oob-command', 'allow-oob': true }
diff --git a/tests/qapi-schema/oob-test.out b/tests/qapi-schema/oob-test.out
new file mode 100644
index 0000000000..ce5a130dcf
--- /dev/null
+++ b/tests/qapi-schema/oob-test.out
@@ -0,0 +1,6 @@ 
+object q_empty
+enum QType ['none', 'qnull', 'qnum', 'qstring', 'qdict', 'qlist', 'qbool']
+    prefix QTYPE
+module oob-test.json
+command an-oob-command None -> None
+   gen=True success_response=True boxed=False oob=True