Patchwork [V5,03/28] qapi script: add event support

login
register
mail settings
Submitter Wenchao Xia
Date May 1, 2014, 4:26 a.m.
Message ID <1398918422-3019-4-git-send-email-wenchaoqemu@gmail.com>
Download mbox | patch
Permalink /patch/344398/
State New
Headers show

Comments

Wenchao Xia - May 1, 2014, 4:26 a.m.
qapi-event.py will parse the schema and generate qapi-event.c, then
the API in qapi-event.c can be used to handle event in qemu code.
All API have prefix "qapi_event".

The script mainly includes two parts: generate API for each event
define, generate an enum type for all defined events.

Since in some cases the real emit behavior may change, for example,
qemu-img would not send a event, a callback layer is used to
control the behavior. As a result, the stubs at compile time
can be saved, the binding of block layer code and monitor code
will become looser.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
---
 Makefile                                 |    9 +-
 Makefile.objs                            |    2 +-
 docs/qapi-code-gen.txt                   |   18 ++
 scripts/qapi-event.py                    |  366 ++++++++++++++++++++++++++++++
 scripts/qapi.py                          |   12 +
 tests/Makefile                           |    3 +-
 tests/qapi-schema/event-nest-struct.err  |    1 +
 tests/qapi-schema/event-nest-struct.exit |    1 +
 tests/qapi-schema/event-nest-struct.json |    2 +
 9 files changed, 409 insertions(+), 5 deletions(-)
 create mode 100644 scripts/qapi-event.py
 create mode 100644 tests/qapi-schema/event-nest-struct.err
 create mode 100644 tests/qapi-schema/event-nest-struct.exit
 create mode 100644 tests/qapi-schema/event-nest-struct.json
 create mode 100644 tests/qapi-schema/event-nest-struct.out

diff --git a/tests/qapi-schema/event-nest-struct.out b/tests/qapi-schema/event-nest-struct.out
new file mode 100644
index 0000000..e69de29
Eric Blake - May 1, 2014, 10:05 p.m.
On 04/30/2014 10:26 PM, Wenchao Xia wrote:
> qapi-event.py will parse the schema and generate qapi-event.c, then
> the API in qapi-event.c can be used to handle event in qemu code.
> All API have prefix "qapi_event".
> 
> The script mainly includes two parts: generate API for each event
> define, generate an enum type for all defined events.
> 
> Since in some cases the real emit behavior may change, for example,
> qemu-img would not send a event, a callback layer is used to
> control the behavior. As a result, the stubs at compile time
> can be saved, the binding of block layer code and monitor code
> will become looser.
> 
> Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
> ---

> @@ -252,6 +252,9 @@ $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
>  qapi-visit.c qapi-visit.h :\
>  $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
>  	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o "." -b < $<, "  GEN   $@")
> +qapi-event.c qapi-event.h :\
> +$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-event.py $(qapi-py)
> +	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py $(gen-out-type) -o "." -b < $<, "  GEN   $@")

Long lines; I think you should rebase atop Lluís series.


> +
> +    /* Fake visit, as if all member are under a structure */
> +    visit_start_struct(v, NULL, "", "%(event_name)s", 0, &local_err);
> +    if (error_is_set(&local_err)) {
> +        goto clean;
> +    }

s/error_is_set(&local_err)/local_err/ here and elsewhere.  We are
getting rid of error_is_set.

> +
> +# Start the real job
> +
> +try:
> +    opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:o:",
> +                                   ["source", "header", "builtins", "prefix=",
> +                                    "output-dir="])

Another reason to rebase atop Lluís work: this should take the input
file name as a parameter rather than reading stdin.
Markus Armbruster - May 2, 2014, 7:54 a.m.
Eric Blake <eblake@redhat.com> writes:

> On 04/30/2014 10:26 PM, Wenchao Xia wrote:
>> qapi-event.py will parse the schema and generate qapi-event.c, then
>> the API in qapi-event.c can be used to handle event in qemu code.
>> All API have prefix "qapi_event".
>> 
>> The script mainly includes two parts: generate API for each event
>> define, generate an enum type for all defined events.
>> 
>> Since in some cases the real emit behavior may change, for example,
>> qemu-img would not send a event, a callback layer is used to
>> control the behavior. As a result, the stubs at compile time
>> can be saved, the binding of block layer code and monitor code
>> will become looser.
>> 
>> Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
>> ---
>
>> @@ -252,6 +252,9 @@ $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
>>  qapi-visit.c qapi-visit.h :\
>>  $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
>>  	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o "." -b < $<, "  GEN   $@")
>> +qapi-event.c qapi-event.h :\
>> +$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-event.py $(qapi-py)
>> +	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py $(gen-out-type) -o "." -b < $<, "  GEN   $@")
>
> Long lines; I think you should rebase atop Lluís series.

This one: "qapi: Allow modularization of QAPI schema files".  v9 looked
committable to me, but Lluís decided to address minor review comments
right away.  I haven't reviewed his v10, yet, but I expect it to be the
final one.

>> +
>> +    /* Fake visit, as if all member are under a structure */
>> +    visit_start_struct(v, NULL, "", "%(event_name)s", 0, &local_err);
>> +    if (error_is_set(&local_err)) {
>> +        goto clean;
>> +    }
>
> s/error_is_set(&local_err)/local_err/ here and elsewhere.  We are
> getting rid of error_is_set.

Yes, please!

>> +
>> +# Start the real job
>> +
>> +try:
>> +    opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:o:",
>> + ["source", "header", "builtins", "prefix=",
>> +                                    "output-dir="])
>
> Another reason to rebase atop Lluís work: this should take the input
> file name as a parameter rather than reading stdin.

Patch

diff --git a/Makefile b/Makefile
index 423e373..ed5b612 100644
--- a/Makefile
+++ b/Makefile
@@ -45,8 +45,8 @@  endif
 endif
 
 GENERATED_HEADERS = config-host.h qemu-options.def
-GENERATED_HEADERS += qmp-commands.h qapi-types.h qapi-visit.h
-GENERATED_SOURCES += qmp-marshal.c qapi-types.c qapi-visit.c
+GENERATED_HEADERS += qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h
+GENERATED_SOURCES += qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c
 
 GENERATED_HEADERS += trace/generated-events.h
 GENERATED_SOURCES += trace/generated-events.c
@@ -208,7 +208,7 @@  Makefile: $(version-obj-y) $(version-lobj-y)
 # Build libraries
 
 libqemustub.a: $(stub-obj-y)
-libqemuutil.a: $(util-obj-y) qapi-types.o qapi-visit.o
+libqemuutil.a: $(util-obj-y) qapi-types.o qapi-visit.o qapi-event.o
 
 block-modules = $(foreach o,$(block-obj-m),"$(basename $(subst /,-,$o))",) NULL
 util/module.o-cflags = -D'CONFIG_BLOCK_MODULES=$(block-modules)'
@@ -252,6 +252,9 @@  $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
 qapi-visit.c qapi-visit.h :\
 $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o "." -b < $<, "  GEN   $@")
+qapi-event.c qapi-event.h :\
+$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-event.py $(qapi-py)
+	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py $(gen-out-type) -o "." -b < $<, "  GEN   $@")
 qmp-commands.h qmp-marshal.c :\
 $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -m -o "." < $<, "  GEN   $@")
diff --git a/Makefile.objs b/Makefile.objs
index a6e0e2a..93697a1 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -12,7 +12,7 @@  block-obj-y += main-loop.o iohandler.o qemu-timer.o
 block-obj-$(CONFIG_POSIX) += aio-posix.o
 block-obj-$(CONFIG_WIN32) += aio-win32.o
 block-obj-y += block/
-block-obj-y += qapi-types.o qapi-visit.o
+block-obj-y += qapi-types.o qapi-visit.o qapi-event.o
 block-obj-y += qemu-io-cmds.o
 
 block-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o
diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt
index d78921f..91e4e13 100644
--- a/docs/qapi-code-gen.txt
+++ b/docs/qapi-code-gen.txt
@@ -180,6 +180,24 @@  An example command is:
    'data': { 'arg1': 'str', '*arg2': 'str' },
    'returns': 'str' }
 
+=== Events ===
+
+Events are defined with key word 'event'.  When 'data' is also specified,
+additional info will be carried on.  Finally there will be C API generated
+in qapi-event.h, and when called by QEMU code, a message with timestamp will
+be emitted on the wire.  If timestamp is -1, it means failure to retrieve host
+time.
+
+An example event is:
+
+{ 'event': 'EVENT_C',
+  'data': { '*a': 'int', 'b': 'str' } }
+
+Resulting in this JSON object:
+
+{ "event": "EVENT_C",
+  "data": { "b": "test string" },
+  "timestamp": { "seconds": 1267020223, "microseconds": 435656 } }
 
 == Code generation ==
 
diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py
new file mode 100644
index 0000000..501baff
--- /dev/null
+++ b/scripts/qapi-event.py
@@ -0,0 +1,366 @@ 
+#
+# QAPI event generator
+#
+# Copyright (c) 2014 Wenchao Xia
+#
+# Authors:
+#  Wenchao Xia <wenchaoqemu@gmail.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2.
+# See the COPYING file in the top-level directory.
+
+from ordereddict import OrderedDict
+from qapi import *
+import sys
+import os
+import getopt
+import errno
+
+def _generate_event_api_name(event_name, params):
+    api_name = "void qapi_event_send_%s(" % c_fun(event_name).lower();
+    l = len(api_name)
+
+    if params:
+        for argname, argentry, optional, structured in parse_args(params):
+            if optional:
+                api_name += "bool has_%s,\n" % c_var(argname)
+                api_name += "".ljust(l)
+
+            if argentry == "str":
+                api_name += "const "
+            api_name += "%s %s,\n" % (c_type(argentry), c_var(argname))
+            api_name += "".ljust(l)
+
+    api_name += "Error **errp)"
+    return api_name;
+
+
+# Following are the core functions that generate C APIs to emit event.
+
+def generate_event_declaration(api_name):
+    return mcgen('''
+
+%(api_name)s;
+''',
+                 api_name = api_name)
+
+def generate_event_implement(api_name, event_name, params):
+    # step 1: declare and variables
+    ret = mcgen("""
+
+%(api_name)s
+{
+    QDict *qmp;
+    Error *local_err = NULL;
+    QMPEventFuncEmit emit;
+""",
+                api_name = api_name)
+
+    if params:
+        ret += mcgen("""
+    QmpOutputVisitor *qov;
+    Visitor *v;
+    QObject *obj;
+
+""")
+
+    # step 2: check emit function, create a dict
+    ret += mcgen("""
+    emit = qmp_event_get_func_emit();
+    if (!emit) {
+        return;
+    }
+
+    qmp = qmp_event_build_dict("%(event_name)s");
+
+""",
+                 event_name = event_name)
+
+    # step 3: visit the params if params != None
+    if params:
+        ret += mcgen("""
+    qov = qmp_output_visitor_new();
+    g_assert(qov);
+
+    v = qmp_output_get_visitor(qov);
+    g_assert(v);
+
+    /* Fake visit, as if all member are under a structure */
+    visit_start_struct(v, NULL, "", "%(event_name)s", 0, &local_err);
+    if (error_is_set(&local_err)) {
+        goto clean;
+    }
+
+""",
+                event_name = event_name)
+
+        for argname, argentry, optional, structured in parse_args(params):
+            if optional:
+                ret += mcgen("""
+    if (has_%(var)s) {
+""",
+                             var = c_var(argname))
+                push_indent()
+
+            if argentry == "str":
+                var_type = "(char **)"
+            else:
+                var_type = ""
+
+            ret += mcgen("""
+    visit_type_%(type)s(v, %(var_type)s&%(var)s, "%(name)s", &local_err);
+    if (error_is_set(&local_err)) {
+        goto clean;
+    }
+""",
+                         var_type = var_type,
+                         var = c_var(argname),
+                         type = type_name(argentry),
+                         name = argname)
+
+            if optional:
+                pop_indent()
+                ret += mcgen("""
+    }
+""")
+
+        ret += mcgen("""
+
+    visit_end_struct(v, &local_err);
+    if (error_is_set(&local_err)) {
+        goto clean;
+    }
+
+    obj = qmp_output_get_qobject(qov);
+    g_assert(obj != NULL);
+
+    qdict_put_obj(qmp, "data", obj);
+""")
+
+    # step 4: call qmp event api
+    ret += mcgen("""
+    emit(%(event_enum_value)s, qmp, &local_err);
+
+""",
+                 event_enum_value = event_enum_value)
+
+    # step 5: clean up
+    if params:
+        ret += mcgen("""
+ clean:
+    qmp_output_visitor_cleanup(qov);
+""")
+    ret += mcgen("""
+    error_propagate(errp, local_err);
+    QDECREF(qmp);
+}
+""")
+
+    return ret
+
+
+# Following are the functions that generate an enum type for all defined
+# events, similar to qapi-types.py. Here we already have enum name and
+# values which were generated before and recorded in event_enum_*. It also
+# works around the issue that "import qapi-types" can't work.
+
+def generate_event_enum_decl(event_enum_name, event_enum_values):
+    lookup_decl = mcgen('''
+
+extern const char *%(event_enum_name)s_lookup[];
+''',
+                        event_enum_name = event_enum_name)
+
+    enum_decl = mcgen('''
+typedef enum %(event_enum_name)s
+{
+''',
+                      event_enum_name = event_enum_name)
+
+    # append automatically generated _MAX value
+    enum_max_value = generate_enum_full_value(event_enum_name, "MAX")
+    enum_values = event_enum_values + [ enum_max_value ]
+
+    i = 0
+    for value in enum_values:
+        enum_decl += mcgen('''
+    %(value)s = %(i)d,
+''',
+                     value = value,
+                     i = i)
+        i += 1
+
+    enum_decl += mcgen('''
+} %(event_enum_name)s;
+''',
+                       event_enum_name = event_enum_name)
+
+    return lookup_decl + enum_decl
+
+def generate_event_enum_lookup(event_enum_name, event_enum_strings):
+    ret = mcgen('''
+
+const char *%(event_enum_name)s_lookup[] = {
+''',
+                event_enum_name = event_enum_name)
+
+    i = 0
+    for string in event_enum_strings:
+        ret += mcgen('''
+    "%(string)s",
+''',
+                     string = string)
+
+    ret += mcgen('''
+    NULL,
+};
+''')
+    return ret
+
+
+# Start the real job
+
+try:
+    opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:o:",
+                                   ["source", "header", "builtins", "prefix=",
+                                    "output-dir="])
+except getopt.GetoptError, err:
+    print str(err)
+    sys.exit(1)
+
+output_dir = ""
+prefix = ""
+c_file = 'qapi-event.c'
+h_file = 'qapi-event.h'
+
+do_c = False
+do_h = False
+do_builtins = False
+
+for o, a in opts:
+    if o in ("-p", "--prefix"):
+        prefix = a
+    elif o in ("-o", "--output-dir"):
+        output_dir = a + "/"
+    elif o in ("-c", "--source"):
+        do_c = True
+    elif o in ("-h", "--header"):
+        do_h = True
+    elif o in ("-b", "--builtins"):
+        do_builtins = True
+
+if not do_c and not do_h:
+    do_c = True
+    do_h = True
+
+c_file = output_dir + prefix + c_file
+h_file = output_dir + prefix + h_file
+
+try:
+    os.makedirs(output_dir)
+except os.error, e:
+    if e.errno != errno.EEXIST:
+        raise
+
+def maybe_open(really, name, opt):
+    if really:
+        return open(name, opt)
+    else:
+        import StringIO
+        return StringIO.StringIO()
+
+fdef = maybe_open(do_c, c_file, 'w')
+fdecl = maybe_open(do_h, h_file, 'w')
+
+fdef.write(mcgen('''
+/* THIS FILE IS AUTOMATICALLY GENERATED, DO NOT MODIFY */
+
+/*
+ * schema-defined QAPI event functions
+ *
+ * Copyright (c) 2014 Wenchao Xia
+ *
+ * Authors:
+ *  Wenchao Xia   <wenchaoqemu@gmail.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "%(header)s"
+#include "%(prefix)sqapi-visit.h"
+#include "qapi/qmp-output-visitor.h"
+#include "qapi/qmp-event.h"
+
+''',
+                 prefix=prefix, header=basename(h_file)))
+
+fdecl.write(mcgen('''
+/* THIS FILE IS AUTOMATICALLY GENERATED, DO NOT MODIFY */
+
+/*
+ * schema-defined QAPI event functions
+ *
+ * Copyright (c) 2014 Wenchao Xia
+ *
+ * Authors:
+ *  Wenchao Xia  <wenchaoqemu@gmail.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef %(guard)s
+#define %(guard)s
+
+#include "qapi/error.h"
+#include "qapi/qmp/qdict.h"
+#include "%(prefix)sqapi-types.h"
+
+''',
+                  prefix=prefix, guard=guardname(h_file)))
+
+exprs = parse_schema(sys.stdin)
+
+event_enum_name = prefix.upper().replace('-', '_') + "QAPIEvent"
+event_enum_values = []
+event_enum_strings = []
+
+for expr in exprs:
+    if expr.has_key('event'):
+        event_name = expr['event']
+        params = expr.get('data')
+        if params and len(params) == 0:
+            params = None
+
+        api_name = _generate_event_api_name(event_name, params)
+        ret = generate_event_declaration(api_name)
+        fdecl.write(ret)
+
+        # We need an enum value per event
+        event_enum_value = generate_enum_full_value(event_enum_name,
+                                                    event_name)
+        ret = generate_event_implement(api_name, event_name, params)
+        fdef.write(ret)
+
+        # Record it, and generate enum later
+        event_enum_values.append(event_enum_value)
+        event_enum_strings.append(event_name)
+
+ret = generate_event_enum_decl(event_enum_name, event_enum_values)
+fdecl.write(ret)
+ret = generate_event_enum_lookup(event_enum_name, event_enum_strings)
+fdef.write(ret)
+
+fdecl.write('''
+#endif
+''')
+
+fdecl.flush()
+fdecl.close()
+
+fdef.flush()
+fdef.close()
diff --git a/scripts/qapi.py b/scripts/qapi.py
index b474c39..b2941d1 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -199,6 +199,16 @@  def discriminator_find_enum_define(expr):
 
     return find_enum(discriminator_type)
 
+def check_event(expr, expr_info):
+    params = expr.get('data')
+    if params:
+        for argname, argentry, optional, structured in parse_args(params):
+            if structured:
+                raise QAPIExprError(expr_info,
+                                    "Nested structure define in event is not "
+                                    "supported now, event '%s', argname '%s'"
+                                    % (expr['event'], argname))
+
 def check_union(expr, expr_info):
     name = expr['union']
     base = expr.get('base')
@@ -262,6 +272,8 @@  def check_exprs(schema):
         expr = expr_elem['expr']
         if expr.has_key('union'):
             check_union(expr, expr_elem['info'])
+        if expr.has_key('event'):
+            check_event(expr, expr_elem['info'])
 
 def parse_schema(fp):
     try:
diff --git a/tests/Makefile b/tests/Makefile
index 88f7105..e477f14 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -175,7 +175,8 @@  check-qapi-schema-y := $(addprefix tests/qapi-schema/, \
         duplicate-key.json union-invalid-base.json flat-union-no-base.json \
         flat-union-invalid-discriminator.json \
         flat-union-invalid-branch-key.json flat-union-reverse-define.json \
-        flat-union-string-discriminator.json)
+        flat-union-string-discriminator.json \
+        event-nest-struct.json)
 
 GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h tests/test-qmp-commands.h
 
diff --git a/tests/qapi-schema/event-nest-struct.err b/tests/qapi-schema/event-nest-struct.err
new file mode 100644
index 0000000..6c10a02
--- /dev/null
+++ b/tests/qapi-schema/event-nest-struct.err
@@ -0,0 +1 @@ 
+<stdin>:1: Nested structure define in event is not supported now, event 'EVENT_A', argname 'a'
diff --git a/tests/qapi-schema/event-nest-struct.exit b/tests/qapi-schema/event-nest-struct.exit
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/tests/qapi-schema/event-nest-struct.exit
@@ -0,0 +1 @@ 
+1
diff --git a/tests/qapi-schema/event-nest-struct.json b/tests/qapi-schema/event-nest-struct.json
new file mode 100644
index 0000000..ee6f3ec
--- /dev/null
+++ b/tests/qapi-schema/event-nest-struct.json
@@ -0,0 +1,2 @@ 
+{ 'event': 'EVENT_A',
+  'data': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' } }