| Message ID | 20140228155321.28087.44860.stgit@fimbulvetr.bsc.es |
|---|---|
| State | New |
| Headers | show |
Lluís Vilanova <vilanova@ac.upc.edu> writes: > Adds the "include(...)" primitive to the syntax of QAPI schema files. > > Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu> > --- > docs/qapi-code-gen.txt | 8 ++++++++ > scripts/qapi.py | 36 ++++++++++++++++++++++++++++++++++-- > 2 files changed, 42 insertions(+), 2 deletions(-) > > diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt > index 2e9f036..e007807 100644 > --- a/docs/qapi-code-gen.txt > +++ b/docs/qapi-code-gen.txt > @@ -40,6 +40,14 @@ enumeration types and union types. > Generally speaking, types definitions should always use CamelCase for the type > names. Command names should be all lower case with words separated by a hyphen. > > +The QAPI schema definitions can be modularized using the 'include' directive: > + > + include("sub-system/qapi.json") And now it isn't JSON anymore. To keep it JSON, use syntax like { "include": "sub-system/qapi.json" } If you absolutely must make it non-JSON, you better rename the .json files. Hmm, we already are non-JSON, because we use ' instead of " for no sane reason. Our JSON parser accepts ' as an extension, to save us quoting in C strings. That reason doesn't apply to .json files. > + > +All paths are interpreted as relative to the initial input file passed to the > +QAPI parsing scripts. Really? Consider foo.json includes lib/a.json, which wants to include lib/b.json. foo.json: include("lib/a.json") lib/a.json: include("lib/b.json") # relative to foo.json's directory Now throw in bar/bar.json including lib/a.json: bar/bar.json: include("../lib/a.json") lib/a.json: include("lib/b.json") # relative to bar/ -> ENOENT Make it relative to the file with the include directive. > + > + > === Complex types === > > A complex type is a dictionary containing a single key whose value is a [...] Are you aware of Wenchao Xia's "[PATCH V8 00/10] qapi script: support enum as discriminator and better enum name"? I'm afraid there's a (semantic?) conflict. With include files, "[PATCH V8 03/10] qapi script: remember line number in schema parsing" needs to remember the source file, too. Wenchao's series is likely go in first. Perhaps you want to base on it now.
Markus Armbruster writes: > Lluís Vilanova <vilanova@ac.upc.edu> writes: >> Adds the "include(...)" primitive to the syntax of QAPI schema files. >> >> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu> >> --- >> docs/qapi-code-gen.txt | 8 ++++++++ >> scripts/qapi.py | 36 ++++++++++++++++++++++++++++++++++-- >> 2 files changed, 42 insertions(+), 2 deletions(-) >> >> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt >> index 2e9f036..e007807 100644 >> --- a/docs/qapi-code-gen.txt >> +++ b/docs/qapi-code-gen.txt >> @@ -40,6 +40,14 @@ enumeration types and union types. >> Generally speaking, types definitions should always use CamelCase for the type >> names. Command names should be all lower case with words separated by a hyphen. >> >> +The QAPI schema definitions can be modularized using the 'include' directive: >> + >> + include("sub-system/qapi.json") > And now it isn't JSON anymore. > To keep it JSON, use syntax like > { "include": "sub-system/qapi.json" } > If you absolutely must make it non-JSON, you better rename the .json > files. > Hmm, we already are non-JSON, because we use ' instead of " for no sane > reason. > Our JSON parser accepts ' as an extension, to save us quoting in C > strings. That reason doesn't apply to .json files. Is it a problem if they are not pure JSON? In the end, they are parsed by qapi.py (which already knows about file syntax), and having a separate syntax for includes makes it somewhat easier to spot when that happens. >> + >> +All paths are interpreted as relative to the initial input file passed to the >> +QAPI parsing scripts. > Really? > Consider foo.json includes lib/a.json, which wants to include > lib/b.json. > foo.json: include("lib/a.json") > lib/a.json: include("lib/b.json") # relative to foo.json's directory > Now throw in bar/bar.json including lib/a.json: > bar/bar.json: include("../lib/a.json") > lib/a.json: include("lib/b.json") # relative to bar/ -> ENOENT > Make it relative to the file with the include directive. Right, sorry. The documentation was not updated after removing the explicit include directory argument. What I'm not sure though is what would be a better option, having current-file-relative includes, or resuscitating the old explicit include argument. >> + >> + >> === Complex types === >> >> A complex type is a dictionary containing a single key whose value is a > [...] > Are you aware of Wenchao Xia's "[PATCH V8 00/10] qapi script: support > enum as discriminator and better enum name"? I'm afraid there's a > (semantic?) conflict. With include files, "[PATCH V8 03/10] qapi > script: remember line number in schema parsing" needs to remember the > source file, too. > Wenchao's series is likely go in first. Perhaps you want to base on it > now. I was not aware of that. Since I'm managing multiple series on a single branch with stgit (and extracting that is somewhat tedious), I will redo this series once Xia's is merged upstream. Is the merge going to happen anytime soon? Thanks, Lluis
Lluís Vilanova <vilanova@ac.upc.edu> writes: > Markus Armbruster writes: > >> Lluís Vilanova <vilanova@ac.upc.edu> writes: >>> Adds the "include(...)" primitive to the syntax of QAPI schema files. >>> >>> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu> >>> --- >>> docs/qapi-code-gen.txt | 8 ++++++++ >>> scripts/qapi.py | 36 ++++++++++++++++++++++++++++++++++-- >>> 2 files changed, 42 insertions(+), 2 deletions(-) >>> >>> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt >>> index 2e9f036..e007807 100644 >>> --- a/docs/qapi-code-gen.txt >>> +++ b/docs/qapi-code-gen.txt >>> @@ -40,6 +40,14 @@ enumeration types and union types. >>> Generally speaking, types definitions should always use CamelCase >>> for the type >>> names. Command names should be all lower case with words separated >>> by a hyphen. >>> >>> +The QAPI schema definitions can be modularized using the 'include' >>> directive: >>> + >>> + include("sub-system/qapi.json") > >> And now it isn't JSON anymore. > >> To keep it JSON, use syntax like > >> { "include": "sub-system/qapi.json" } > >> If you absolutely must make it non-JSON, you better rename the .json >> files. > >> Hmm, we already are non-JSON, because we use ' instead of " for no sane >> reason. > >> Our JSON parser accepts ' as an extension, to save us quoting in C >> strings. That reason doesn't apply to .json files. > > Is it a problem if they are not pure JSON? In the end, they are parsed by > qapi.py (which already knows about file syntax), and having a separate syntax > for includes makes it somewhat easier to spot when that happens. I don't particularly care whether schema syntax is pure JSON, some bastardized variation of JSON, or something else entirely. But as long as we advertize schema files it as .json, they better contain JSON. If they contain something else, they should be called something else. >>> + >>> +All paths are interpreted as relative to the initial input file >>> passed to the >>> +QAPI parsing scripts. > >> Really? > >> Consider foo.json includes lib/a.json, which wants to include >> lib/b.json. > >> foo.json: include("lib/a.json") >> lib/a.json: include("lib/b.json") # relative to foo.json's directory > >> Now throw in bar/bar.json including lib/a.json: > >> bar/bar.json: include("../lib/a.json") >> lib/a.json: include("lib/b.json") # relative to bar/ -> ENOENT > >> Make it relative to the file with the include directive. > > Right, sorry. The documentation was not updated after removing the explicit > include directory argument. What I'm not sure though is what would be a better > option, having current-file-relative includes, or resuscitating the old explicit > include argument. Include relative to the including file is simple. If we needed not-so-simple semantics, I suspect we'd be better off piping through cpp. >>> + >>> + >>> === Complex types === >>> >>> A complex type is a dictionary containing a single key whose value is a >> [...] > >> Are you aware of Wenchao Xia's "[PATCH V8 00/10] qapi script: support >> enum as discriminator and better enum name"? I'm afraid there's a >> (semantic?) conflict. With include files, "[PATCH V8 03/10] qapi >> script: remember line number in schema parsing" needs to remember the >> source file, too. > >> Wenchao's series is likely go in first. Perhaps you want to base on it >> now. > > I was not aware of that. Since I'm managing multiple series on a single branch > with stgit (and extracting that is somewhat tedious), I will redo this series > once Xia's is merged upstream. Is the merge going to happen anytime soon? I reviewed v7, and it looked almost ready to me. If v8 is ready, it'll hopefully get merged fairly quickly (days, not weeks).
Markus Armbruster writes: > Lluís Vilanova <vilanova@ac.upc.edu> writes: >> Markus Armbruster writes: >> >>> Lluís Vilanova <vilanova@ac.upc.edu> writes: >>>> Adds the "include(...)" primitive to the syntax of QAPI schema files. >>>> >>>> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu> >>>> --- >>>> docs/qapi-code-gen.txt | 8 ++++++++ >>>> scripts/qapi.py | 36 ++++++++++++++++++++++++++++++++++-- >>>> 2 files changed, 42 insertions(+), 2 deletions(-) >>>> >>>> diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt >>>> index 2e9f036..e007807 100644 >>>> --- a/docs/qapi-code-gen.txt >>>> +++ b/docs/qapi-code-gen.txt >>>> @@ -40,6 +40,14 @@ enumeration types and union types. >>>> Generally speaking, types definitions should always use CamelCase >>>> for the type >>>> names. Command names should be all lower case with words separated >>>> by a hyphen. >>>> >>>> +The QAPI schema definitions can be modularized using the 'include' >>>> directive: >>>> + >>>> + include("sub-system/qapi.json") >> >>> And now it isn't JSON anymore. >> >>> To keep it JSON, use syntax like >> >>> { "include": "sub-system/qapi.json" } >> >>> If you absolutely must make it non-JSON, you better rename the .json >>> files. >> >>> Hmm, we already are non-JSON, because we use ' instead of " for no sane >>> reason. >> >>> Our JSON parser accepts ' as an extension, to save us quoting in C >>> strings. That reason doesn't apply to .json files. >> >> Is it a problem if they are not pure JSON? In the end, they are parsed by >> qapi.py (which already knows about file syntax), and having a separate syntax >> for includes makes it somewhat easier to spot when that happens. > I don't particularly care whether schema syntax is pure JSON, some > bastardized variation of JSON, or something else entirely. But as long > as we advertize schema files it as .json, they better contain JSON. If > they contain something else, they should be called something else. Both are simple enough to implement. Is there any consensus on what's preferred? >>>> + >>>> +All paths are interpreted as relative to the initial input file >>>> passed to the >>>> +QAPI parsing scripts. >> >>> Really? >> >>> Consider foo.json includes lib/a.json, which wants to include >>> lib/b.json. >> >>> foo.json: include("lib/a.json") >>> lib/a.json: include("lib/b.json") # relative to foo.json's directory >> >>> Now throw in bar/bar.json including lib/a.json: >> >>> bar/bar.json: include("../lib/a.json") >>> lib/a.json: include("lib/b.json") # relative to bar/ -> ENOENT >> >>> Make it relative to the file with the include directive. >> >> Right, sorry. The documentation was not updated after removing the explicit >> include directory argument. What I'm not sure though is what would be a better >> option, having current-file-relative includes, or resuscitating the old explicit >> include argument. > Include relative to the including file is simple. If we needed > not-so-simple semantics, I suspect we'd be better off piping through > cpp. I tried, but it chokes on the comment syntax. >>>> + >>>> + >>>> === Complex types === >>>> >>>> A complex type is a dictionary containing a single key whose value is a >>> [...] >> >>> Are you aware of Wenchao Xia's "[PATCH V8 00/10] qapi script: support >>> enum as discriminator and better enum name"? I'm afraid there's a >>> (semantic?) conflict. With include files, "[PATCH V8 03/10] qapi >>> script: remember line number in schema parsing" needs to remember the >>> source file, too. >> >>> Wenchao's series is likely go in first. Perhaps you want to base on it >>> now. >> >> I was not aware of that. Since I'm managing multiple series on a single branch >> with stgit (and extracting that is somewhat tedious), I will redo this series >> once Xia's is merged upstream. Is the merge going to happen anytime soon? > I reviewed v7, and it looked almost ready to me. If v8 is ready, it'll > hopefully get merged fairly quickly (days, not weeks). Excellent. Lluis
On 03/03/2014 08:27 AM, Markus Armbruster wrote: >>>> +The QAPI schema definitions can be modularized using the 'include' >>>> directive: >>>> + >>>> + include("sub-system/qapi.json") >> >>> And now it isn't JSON anymore. >> >>> To keep it JSON, use syntax like >> >>> { "include": "sub-system/qapi.json" } I actually think this looks nicer - makes the file more consistent. >> >>> If you absolutely must make it non-JSON, you better rename the .json >>> files. >> >>> Hmm, we already are non-JSON, because we use ' instead of " for no sane >>> reason. A weak argument: ' is easier than " to type (at least on US keyboards - no shift key required). Another weak argument: using ' in the qapi files vs. " in actual QMP makes it easy to interleave discussions about semantics vs. examples of those semantics in use (you can see whether a code snippet is talking about qapi or wire format based on what quoting it used) Our files are already non-JSON due to comments (JSON has no notion of # introducing a comment to ignore text to the next newline). But both our use of comments and our use of ' instead of " can be remedied in a one-pass sed script to get a true JSON output if such is needed, at least as long as we don't need to quote any " characters in the schema. Therefore, I agree that making the include syntax closer to true JSON is desirable, whether or not we also decide to use " in the files to begin with. I don't see any way around the fact that JSON doesn't define comments, vs. our absolute need for comments in our schema files, though. >> >>> Our JSON parser accepts ' as an extension, to save us quoting in C >>> strings. That reason doesn't apply to .json files. >> >> Is it a problem if they are not pure JSON? In the end, they are parsed by >> qapi.py (which already knows about file syntax), and having a separate syntax >> for includes makes it somewhat easier to spot when that happens. > > I don't particularly care whether schema syntax is pure JSON, some > bastardized variation of JSON, or something else entirely. But as long > as we advertize schema files it as .json, they better contain JSON. If > they contain something else, they should be called something else. Maybe .qapi? But the name qapi-schema.qapi sounds redundant...
Eric Blake <eblake@redhat.com> writes: > On 03/03/2014 08:27 AM, Markus Armbruster wrote: > >>>>> +The QAPI schema definitions can be modularized using the 'include' >>>>> directive: >>>>> + >>>>> + include("sub-system/qapi.json") >>> >>>> And now it isn't JSON anymore. >>> >>>> To keep it JSON, use syntax like >>> >>>> { "include": "sub-system/qapi.json" } > > I actually think this looks nicer - makes the file more consistent. I suspect qapi.py would look nicer, too :) >>>> If you absolutely must make it non-JSON, you better rename the .json >>>> files. >>> >>>> Hmm, we already are non-JSON, because we use ' instead of " for no sane >>>> reason. > > A weak argument: ' is easier than " to type (at least on US keyboards - > no shift key required). > > Another weak argument: using ' in the qapi files vs. " in actual QMP > makes it easy to interleave discussions about semantics vs. examples of > those semantics in use (you can see whether a code snippet is talking > about qapi or wire format based on what quoting it used) > > Our files are already non-JSON due to comments (JSON has no notion of # > introducing a comment to ignore text to the next newline). But both our > use of comments and our use of ' instead of " can be remedied in a > one-pass sed script to get a true JSON output if such is needed, at > least as long as we don't need to quote any " characters in the schema. > > Therefore, I agree that making the include syntax closer to true JSON is > desirable, whether or not we also decide to use " in the files to begin > with. I don't see any way around the fact that JSON doesn't define > comments, vs. our absolute need for comments in our schema files, though. We certainly can't do without comments. JSON is designed for easy data exchange, but we use it as programming language syntax. Its restrictions make sense for easy data exchange, but hurt our use. We're not the first ones experiencing that pain: http://json5.org/ No idea how much momentum this JSON5 thingy has... >>>> Our JSON parser accepts ' as an extension, to save us quoting in C >>>> strings. That reason doesn't apply to .json files. >>> >>> Is it a problem if they are not pure JSON? In the end, they are parsed by >>> qapi.py (which already knows about file syntax), and having a separate syntax >>> for includes makes it somewhat easier to spot when that happens. >> >> I don't particularly care whether schema syntax is pure JSON, some >> bastardized variation of JSON, or something else entirely. But as long >> as we advertize schema files it as .json, they better contain JSON. If >> they contain something else, they should be called something else. > > Maybe .qapi? But the name qapi-schema.qapi sounds redundant... schema.qapi? Switch to JSON5 and call it qapi-schema.json5?
The Tuesday 04 Mar 2014 à 09:02:57 (+0100), Markus Armbruster wrote : > Eric Blake <eblake@redhat.com> writes: > > > On 03/03/2014 08:27 AM, Markus Armbruster wrote: > > > >>>>> +The QAPI schema definitions can be modularized using the 'include' > >>>>> directive: > >>>>> + > >>>>> + include("sub-system/qapi.json") > >>> > >>>> And now it isn't JSON anymore. > >>> > >>>> To keep it JSON, use syntax like > >>> > >>>> { "include": "sub-system/qapi.json" } > > > > I actually think this looks nicer - makes the file more consistent. > > I suspect qapi.py would look nicer, too :) > > >>>> If you absolutely must make it non-JSON, you better rename the .json > >>>> files. > >>> > >>>> Hmm, we already are non-JSON, because we use ' instead of " for no sane > >>>> reason. > > > > A weak argument: ' is easier than " to type (at least on US keyboards - > > no shift key required). > > > > Another weak argument: using ' in the qapi files vs. " in actual QMP > > makes it easy to interleave discussions about semantics vs. examples of > > those semantics in use (you can see whether a code snippet is talking > > about qapi or wire format based on what quoting it used) > > > > Our files are already non-JSON due to comments (JSON has no notion of # > > introducing a comment to ignore text to the next newline). But both our > > use of comments and our use of ' instead of " can be remedied in a > > one-pass sed script to get a true JSON output if such is needed, at > > least as long as we don't need to quote any " characters in the schema. > > > > Therefore, I agree that making the include syntax closer to true JSON is > > desirable, whether or not we also decide to use " in the files to begin > > with. I don't see any way around the fact that JSON doesn't define > > comments, vs. our absolute need for comments in our schema files, though. > > We certainly can't do without comments. > > JSON is designed for easy data exchange, but we use it as programming > language syntax. Its restrictions make sense for easy data exchange, > but hurt our use. We're not the first ones experiencing that pain: > http://json5.org/ > > No idea how much momentum this JSON5 thingy has... > > >>>> Our JSON parser accepts ' as an extension, to save us quoting in C > >>>> strings. That reason doesn't apply to .json files. > >>> > >>> Is it a problem if they are not pure JSON? In the end, they are parsed by > >>> qapi.py (which already knows about file syntax), and having a separate syntax > >>> for includes makes it somewhat easier to spot when that happens. > >> > >> I don't particularly care whether schema syntax is pure JSON, some > >> bastardized variation of JSON, or something else entirely. But as long > >> as we advertize schema files it as .json, they better contain JSON. If > >> they contain something else, they should be called something else. > > > > Maybe .qapi? But the name qapi-schema.qapi sounds redundant... > > schema.qapi? > > Switch to JSON5 and call it qapi-schema.json5? > Hmm don't we want something that python and other language know how to parse out of the box ? Or will we write yet another delicate work of art to parse it ? Best regards Benoît
On 03/13/2014 09:33 AM, Benoît Canet wrote: >> We certainly can't do without comments. >> >> JSON is designed for easy data exchange, but we use it as programming >> language syntax. Its restrictions make sense for easy data exchange, >> but hurt our use. We're not the first ones experiencing that pain: >> http://json5.org/ >> >> No idea how much momentum this JSON5 thingy has... If we 's,#,//,', our comments magically fall in line with JSON5 syntax; everything else in our files is already compliant with JSON5. >> >> Switch to JSON5 and call it qapi-schema.json5? This actually seems like a rather nice idea - but due to our choice of comments, it means rewriting the bulk of the file and tweaking our parser. >> > > Hmm don't we want something that python and other language know how to parse out > of the box ? Or will we write yet another delicate work of art to parse it ? Our existing parser would only need to learn a new comment syntax to parse the subset of JSON5 that we currently actually use. Parsing FULL JSON5 would mean also learning about trailing commas, unquoted names in name:value pairs, multiline strings, and alternative numeric representations. But a point made on the JSON5 page is that ES5 JavaScript already parses JSON5, just as it already parses original JSON.
Eric Blake writes: > On 03/13/2014 09:33 AM, Benoît Canet wrote: >>> We certainly can't do without comments. >>> >>> JSON is designed for easy data exchange, but we use it as programming >>> language syntax. Its restrictions make sense for easy data exchange, >>> but hurt our use. We're not the first ones experiencing that pain: >>> http://json5.org/ >>> >>> No idea how much momentum this JSON5 thingy has... > If we 's,#,//,', our comments magically fall in line with JSON5 syntax; > everything else in our files is already compliant with JSON5. >>> >>> Switch to JSON5 and call it qapi-schema.json5? > This actually seems like a rather nice idea - but due to our choice of > comments, it means rewriting the bulk of the file and tweaking our parser. >>> >> >> Hmm don't we want something that python and other language know how to parse out >> of the box ? Or will we write yet another delicate work of art to parse it ? > Our existing parser would only need to learn a new comment syntax to > parse the subset of JSON5 that we currently actually use. Parsing FULL > JSON5 would mean also learning about trailing commas, unquoted names in > name:value pairs, multiline strings, and alternative numeric > representations. But a point made on the JSON5 page is that ES5 > JavaScript already parses JSON5, just as it already parses original JSON. Another option is to bump QEMU requirements to python 2.6 or later. Then we can use the json parser that comes with python. A simple pre-processing could eliminate the comments before passing them to the json package for loading into python structures. The commands/enums/etc should also be elements of a list for it to work (that must be either changed on the qapi files, or hackishly "injected" before parsing). Lluis
The Thursday 13 Mar 2014 à 19:05:12 (+0100), Lluís Vilanova wrote : > Eric Blake writes: > > > On 03/13/2014 09:33 AM, Benoît Canet wrote: > >>> We certainly can't do without comments. > >>> > >>> JSON is designed for easy data exchange, but we use it as programming > >>> language syntax. Its restrictions make sense for easy data exchange, > >>> but hurt our use. We're not the first ones experiencing that pain: > >>> http://json5.org/ > >>> > >>> No idea how much momentum this JSON5 thingy has... > > > If we 's,#,//,', our comments magically fall in line with JSON5 syntax; > > everything else in our files is already compliant with JSON5. > > >>> > >>> Switch to JSON5 and call it qapi-schema.json5? > > > This actually seems like a rather nice idea - but due to our choice of > > comments, it means rewriting the bulk of the file and tweaking our parser. > > >>> > >> > >> Hmm don't we want something that python and other language know how to parse out > >> of the box ? Or will we write yet another delicate work of art to parse it ? > > > Our existing parser would only need to learn a new comment syntax to > > parse the subset of JSON5 that we currently actually use. Parsing FULL > > JSON5 would mean also learning about trailing commas, unquoted names in > > name:value pairs, multiline strings, and alternative numeric > > representations. But a point made on the JSON5 page is that ES5 > > JavaScript already parses JSON5, just as it already parses original JSON. > > Another option is to bump QEMU requirements to python 2.6 or later. Then we can > use the json parser that comes with python. A simple pre-processing could > eliminate the comments before passing them to the json package for loading into > python structures. The commands/enums/etc should also be elements of a list for > it to work (that must be either changed on the qapi files, or hackishly > "injected" before parsing). > > > Lluis I have an use case for this series. Lluis: Do you plan to respin this series ? Or should I do it ? Best regards Benoît > > -- > "And it's much the same thing with knowledge, for whenever you learn > something new, the whole world becomes that much richer." > -- The Princess of Pure Reason, as told by Norton Juster in The Phantom > Tollbooth >
Benoît Canet writes: > The Thursday 13 Mar 2014 à 19:05:12 (+0100), Lluís Vilanova wrote : >> Eric Blake writes: >> >> > On 03/13/2014 09:33 AM, Benoît Canet wrote: >> >>> We certainly can't do without comments. >> >>> >> >>> JSON is designed for easy data exchange, but we use it as programming >> >>> language syntax. Its restrictions make sense for easy data exchange, >> >>> but hurt our use. We're not the first ones experiencing that pain: >> >>> http://json5.org/ >> >>> >> >>> No idea how much momentum this JSON5 thingy has... >> >> > If we 's,#,//,', our comments magically fall in line with JSON5 syntax; >> > everything else in our files is already compliant with JSON5. >> >> >>> >> >>> Switch to JSON5 and call it qapi-schema.json5? >> >> > This actually seems like a rather nice idea - but due to our choice of >> > comments, it means rewriting the bulk of the file and tweaking our parser. >> >> >>> >> >> >> >> Hmm don't we want something that python and other language know how to parse out >> >> of the box ? Or will we write yet another delicate work of art to parse it ? >> >> > Our existing parser would only need to learn a new comment syntax to >> > parse the subset of JSON5 that we currently actually use. Parsing FULL >> > JSON5 would mean also learning about trailing commas, unquoted names in >> > name:value pairs, multiline strings, and alternative numeric >> > representations. But a point made on the JSON5 page is that ES5 >> > JavaScript already parses JSON5, just as it already parses original JSON. >> >> Another option is to bump QEMU requirements to python 2.6 or later. Then we can >> use the json parser that comes with python. A simple pre-processing could >> eliminate the comments before passing them to the json package for loading into >> python structures. The commands/enums/etc should also be elements of a list for >> it to work (that must be either changed on the qapi files, or hackishly >> "injected" before parsing). >> >> >> Lluis > I have an use case for this series. > Lluis: Do you plan to respin this series ? Or should I do it ? I was waiting for some other series to get merged, since they conflict. But I still did not change the "include" syntax. I will probably not be able to get to this until May 1st. Lluis
The Friday 14 Mar 2014 à 21:24:38 (+0100), Lluís Vilanova wrote : > Benoît Canet writes: > > > The Thursday 13 Mar 2014 à 19:05:12 (+0100), Lluís Vilanova wrote : > >> Eric Blake writes: > >> > >> > On 03/13/2014 09:33 AM, Benoît Canet wrote: > >> >>> We certainly can't do without comments. > >> >>> > >> >>> JSON is designed for easy data exchange, but we use it as programming > >> >>> language syntax. Its restrictions make sense for easy data exchange, > >> >>> but hurt our use. We're not the first ones experiencing that pain: > >> >>> http://json5.org/ > >> >>> > >> >>> No idea how much momentum this JSON5 thingy has... > >> > >> > If we 's,#,//,', our comments magically fall in line with JSON5 syntax; > >> > everything else in our files is already compliant with JSON5. > >> > >> >>> > >> >>> Switch to JSON5 and call it qapi-schema.json5? > >> > >> > This actually seems like a rather nice idea - but due to our choice of > >> > comments, it means rewriting the bulk of the file and tweaking our parser. > >> > >> >>> > >> >> > >> >> Hmm don't we want something that python and other language know how to parse out > >> >> of the box ? Or will we write yet another delicate work of art to parse it ? > >> > >> > Our existing parser would only need to learn a new comment syntax to > >> > parse the subset of JSON5 that we currently actually use. Parsing FULL > >> > JSON5 would mean also learning about trailing commas, unquoted names in > >> > name:value pairs, multiline strings, and alternative numeric > >> > representations. But a point made on the JSON5 page is that ES5 > >> > JavaScript already parses JSON5, just as it already parses original JSON. > >> > >> Another option is to bump QEMU requirements to python 2.6 or later. Then we can > >> use the json parser that comes with python. A simple pre-processing could > >> eliminate the comments before passing them to the json package for loading into > >> python structures. The commands/enums/etc should also be elements of a list for > >> it to work (that must be either changed on the qapi files, or hackishly > >> "injected" before parsing). > >> > >> > >> Lluis > > > I have an use case for this series. > > > Lluis: Do you plan to respin this series ? Or should I do it ? > > I was waiting for some other series to get merged, since they conflict. But I > still did not change the "include" syntax. > > I will probably not be able to get to this until May 1st. Ok, I think I need the include feature more badly than you I will do it. Best regards Benoît > > > Lluis > > > -- > "And it's much the same thing with knowledge, for whenever you learn > something new, the whole world becomes that much richer." > -- The Princess of Pure Reason, as told by Norton Juster in The Phantom > Tollbooth
The Thursday 13 Mar 2014 à 09:54:15 (-0600), Eric Blake wrote : > On 03/13/2014 09:33 AM, Benoît Canet wrote: > > >> We certainly can't do without comments. > >> > >> JSON is designed for easy data exchange, but we use it as programming > >> language syntax. Its restrictions make sense for easy data exchange, > >> but hurt our use. We're not the first ones experiencing that pain: > >> http://json5.org/ > >> > >> No idea how much momentum this JSON5 thingy has... > > If we 's,#,//,', our comments magically fall in line with JSON5 syntax; > everything else in our files is already compliant with JSON5. Not really qapi-schema.json is missing comas between types to be a valid json file. The fragment: { 'type' : 'InputBtnEvent', 'data' : { 'button' : 'InputButton', 'down' : 'bool' } } { 'type' : 'InputMoveEvent', 'data' : { 'axis' : 'InputAxis', 'value' : 'int' } } Should be: [ { 'type' : 'InputBtnEvent', 'data' : { 'button' : 'InputButton', 'down' : 'bool' } }, { 'type' : 'InputMoveEvent', 'data' : { 'axis' : 'InputAxis', 'value' : 'int' } } ] to hope being a valid json file. Best regards Benoît > > >> > >> Switch to JSON5 and call it qapi-schema.json5? > > This actually seems like a rather nice idea - but due to our choice of > comments, it means rewriting the bulk of the file and tweaking our parser. > > >> > > > > Hmm don't we want something that python and other language know how to parse out > > of the box ? Or will we write yet another delicate work of art to parse it ? > > Our existing parser would only need to learn a new comment syntax to > parse the subset of JSON5 that we currently actually use. Parsing FULL > JSON5 would mean also learning about trailing commas, unquoted names in > name:value pairs, multiline strings, and alternative numeric > representations. But a point made on the JSON5 page is that ES5 > JavaScript already parses JSON5, just as it already parses original JSON. > > -- > Eric Blake eblake redhat com +1-919-301-3266 > Libvirt virtualization library http://libvirt.org >
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt index 2e9f036..e007807 100644 --- a/docs/qapi-code-gen.txt +++ b/docs/qapi-code-gen.txt @@ -40,6 +40,14 @@ enumeration types and union types. Generally speaking, types definitions should always use CamelCase for the type names. Command names should be all lower case with words separated by a hyphen. +The QAPI schema definitions can be modularized using the 'include' directive: + + include("sub-system/qapi.json") + +All paths are interpreted as relative to the initial input file passed to the +QAPI parsing scripts. + + === Complex types === A complex type is a dictionary containing a single key whose value is a diff --git a/scripts/qapi.py b/scripts/qapi.py index 59c2b9b..eddbf25 100644 --- a/scripts/qapi.py +++ b/scripts/qapi.py @@ -11,10 +11,14 @@ # This work is licensed under the terms of the GNU GPLv2. # See the COPYING.LIB file in the top-level directory. +import os +import re from ordereddict import OrderedDict import os import sys +include_cre = re.compile("include\(\"([^\"]*)\"\)") + builtin_types = [ 'str', 'int', 'number', 'bool', 'int8', 'int16', 'int32', 'int64', @@ -57,12 +61,16 @@ class QAPISchemaError(Exception): class QAPISchema: - def __init__(self, fp, error_base=None): + def __init__(self, fp, input_dir, error_base=None, included=[]): self.fp = fp if error_base is None: self.error_base = os.getcwd() else: self.error_base = error_base + input_path = os.path.abspath(fp.name) + input_path = os.path.relpath(input_path, self.error_base) + self.included = included + [input_path] + self.input_dir = input_dir self.src = fp.read() if self.src == '' or self.src[-1] != '\n': self.src += '\n' @@ -107,6 +115,29 @@ class QAPISchema: if self.cursor == len(self.src): self.tok = None return + elif self.tok == 'i': + include_src = self.src[self.cursor-1:] + include_match = include_cre.match(include_src) + if include_match is not None: + include_path = os.path.join(self.input_dir, + include_match.group(1)) + include_path = os.path.abspath(include_path) + if not os.path.isfile(include_path): + raise QAPISchemaError( + self, + 'Non-existing included file "%s"' % + include_match.group(1)) + include_path_rel = os.path.relpath(include_path, + self.error_base) + if include_path_rel in self.included: + raise QAPISchemaError(self, "Infinite inclusion loop: %s" + % " -> ".join(self.included + + [include_path_rel])) + include_schema = QAPISchema(open(include_path, "r"), + self.input_dir, self.error_base, + self.included) + self.exprs += include_schema.exprs + self.cursor += include_match.span()[1] - 1 elif not self.tok.isspace(): raise QAPISchemaError(self, 'Stray "%s"' % self.tok) @@ -166,8 +197,9 @@ class QAPISchema: return expr def parse_schema(input_path, error_base=None): + input_dir = os.path.dirname(input_path) try: - schema = QAPISchema(open(input_path, "r"), error_base) + schema = QAPISchema(open(input_path, "r"), input_dir, error_base) except QAPISchemaError, e: print >>sys.stderr, e exit(1)
Adds the "include(...)" primitive to the syntax of QAPI schema files. Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu> --- docs/qapi-code-gen.txt | 8 ++++++++ scripts/qapi.py | 36 ++++++++++++++++++++++++++++++++++-- 2 files changed, 42 insertions(+), 2 deletions(-)