Patchwork [10/22] qapi: qapi.py, make json parser more robust

login
register
mail settings
Submitter Michael Roth
Date July 24, 2012, 5:20 p.m.
Message ID <1343150454-4677-11-git-send-email-mdroth@linux.vnet.ibm.com>
Download mbox | patch
Permalink /patch/172992/
State New
Headers show

Comments

Michael Roth - July 24, 2012, 5:20 p.m.
Currently the QAPI JSON parser expects a very particular style of code
indentation, the major one being that terminating curly/square brackets are
not on placed on a seperate line. This is incompatible with most
pretty-print formats, so make it a little more robust by supporting
these cases.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
 scripts/qapi.py |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Anthony Liguori - July 24, 2012, 10:23 p.m.
Michael Roth <mdroth@linux.vnet.ibm.com> writes:

> Currently the QAPI JSON parser expects a very particular style of code
> indentation, the major one being that terminating curly/square brackets are
> not on placed on a seperate line. This is incompatible with most
> pretty-print formats, so make it a little more robust by supporting
> these cases.

The code was supposed to just expect that all subsequent lines are
indented.  This was to simplify the parser.  Can you provide an
problematic example?

Regards,

Anthony Liguori
[...]
Michael Roth - July 24, 2012, 10:59 p.m.
On Tue, Jul 24, 2012 at 05:23:33PM -0500, Anthony Liguori wrote:
> Michael Roth <mdroth@linux.vnet.ibm.com> writes:
> 
> > Currently the QAPI JSON parser expects a very particular style of code
> > indentation, the major one being that terminating curly/square brackets are
> > not on placed on a seperate line. This is incompatible with most
> > pretty-print formats, so make it a little more robust by supporting
> > these cases.
> 
> The code was supposed to just expect that all subsequent lines are
> indented.  This was to simplify the parser.  Can you provide an
> problematic example?

The main one was cases like this:

    mdroth@loki:~/w/qemu3.git$ cat example.json 
    {
        'a': 'test'
    }
    mdroth@loki:~/w/qemu3.git$ python scripts/qapi-types.py -o /tmp <example.json 
    Traceback (most recent call last):
      File "scripts/qapi-types.py", line 260, in <module>
        exprs = parse_schema(sys.stdin)
      File "/home/mdroth/dev/kvm/qemu3.git/scripts/qapi.py", line 76, in parse_schema
        expr_eval = evaluate(expr)
      File "/home/mdroth/dev/kvm/qemu3.git/scripts/qapi.py", line 62, in evaluate
        return parse(map(lambda x: x, tokenize(string)))[0]
      File "/home/mdroth/dev/kvm/qemu3.git/scripts/qapi.py", line 42, in parse
        if tokens[0] == ',':
    IndexError: list index out of range

I had it in my queue because it was what was preventing me from pretty-printing
the QIDL-generated schemas in V1, since python's pretty print stuff generally
puts the opening/closing braces at the same indent level. But now we store them
in the binary, and don't pretty-print those, so I don't think this is an issue
WRT this series anymore. Might still make sense to add it for future use-cases
though.

> 
> Regards,
> 
> Anthony Liguori
> [...]
>

Patch

diff --git a/scripts/qapi.py b/scripts/qapi.py
index 39bb74e..e0c694d 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -70,7 +70,7 @@  def parse_schema(fp):
         if line.startswith('#') or line == '\n':
             continue
 
-        if line.startswith(' '):
+        if line[0] in ['}', ']', ' ', '\t']:
             expr += line
         elif expr:
             expr_eval = evaluate(expr)