From patchwork Thu May 1 04:26:37 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wenchao Xia X-Patchwork-Id: 344398 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 94674140115 for ; Thu, 1 May 2014 14:28:09 +1000 (EST) Received: from localhost ([::1]:60293 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WfibP-0000dO-E5 for incoming@patchwork.ozlabs.org; Thu, 01 May 2014 00:28:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49163) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wfiao-00080U-VT for qemu-devel@nongnu.org; Thu, 01 May 2014 00:27:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Wfiak-0002En-9p for qemu-devel@nongnu.org; Thu, 01 May 2014 00:27:30 -0400 Received: from mail-ie0-x22a.google.com ([2607:f8b0:4001:c03::22a]:48186) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wfiak-0002Ej-23 for qemu-devel@nongnu.org; Thu, 01 May 2014 00:27:26 -0400 Received: by mail-ie0-f170.google.com with SMTP id rd18so3034935iec.15 for ; Wed, 30 Apr 2014 21:27:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=d6GPS4kiKyr7p7Q4w0A+85pmOUQcPIWK+JX+eYBJxKI=; b=MCiebu77xQOdH9FWY9nWrhYSeUKashvJ1YAb74SAtakbGdtJmPuoMTohLXMY9K/P8C zByfRf7CZYfJYs67gBIIsbDC3WJA803S7Nm3AGIuB7AIrueoPFmKi3L7DJlXXetsjWhm 2b8gr7fev4UZDVvMKcj17wO1OD4ukvWOH3un+snU/r/CSPrZfcPHy1wz1UegPvwWnMI/ E9vodVlTJKsr+Y187ZFjESdSXBTnteZAVW9kLU/ztv2vBO5InUI8HsIkwyQ4YjSrnAlX gqDUgaOkCYU23L40Bzau7mo/Z5K9o5BgjAmWLD4yqmigkT3eTnmeSA5ejhso8YcYZ0I5 y4FQ== X-Received: by 10.50.49.44 with SMTP id r12mr425319ign.41.1398918445598; Wed, 30 Apr 2014 21:27:25 -0700 (PDT) Received: from localhost.localdomain.localdomain ([118.250.93.201]) by mx.google.com with ESMTPSA id ii7sm4626797igb.17.2014.04.30.21.27.21 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 30 Apr 2014 21:27:25 -0700 (PDT) From: Wenchao Xia To: qemu-devel@nongnu.org Date: Wed, 30 Apr 2014 21:26:37 -0700 Message-Id: <1398918422-3019-4-git-send-email-wenchaoqemu@gmail.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1398918422-3019-1-git-send-email-wenchaoqemu@gmail.com> References: <1398918422-3019-1-git-send-email-wenchaoqemu@gmail.com> X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2607:f8b0:4001:c03::22a Cc: mdroth@linux.vnet.ibm.com, armbru@redhat.com, Wenchao Xia , lcapitulino@redhat.com Subject: [Qemu-devel] [PATCH V5 03/28] qapi script: add event support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org qapi-event.py will parse the schema and generate qapi-event.c, then the API in qapi-event.c can be used to handle event in qemu code. All API have prefix "qapi_event". The script mainly includes two parts: generate API for each event define, generate an enum type for all defined events. Since in some cases the real emit behavior may change, for example, qemu-img would not send a event, a callback layer is used to control the behavior. As a result, the stubs at compile time can be saved, the binding of block layer code and monitor code will become looser. Signed-off-by: Wenchao Xia --- Makefile | 9 +- Makefile.objs | 2 +- docs/qapi-code-gen.txt | 18 ++ scripts/qapi-event.py | 366 ++++++++++++++++++++++++++++++ scripts/qapi.py | 12 + tests/Makefile | 3 +- tests/qapi-schema/event-nest-struct.err | 1 + tests/qapi-schema/event-nest-struct.exit | 1 + tests/qapi-schema/event-nest-struct.json | 2 + 9 files changed, 409 insertions(+), 5 deletions(-) create mode 100644 scripts/qapi-event.py create mode 100644 tests/qapi-schema/event-nest-struct.err create mode 100644 tests/qapi-schema/event-nest-struct.exit create mode 100644 tests/qapi-schema/event-nest-struct.json create mode 100644 tests/qapi-schema/event-nest-struct.out diff --git a/tests/qapi-schema/event-nest-struct.out b/tests/qapi-schema/event-nest-struct.out new file mode 100644 index 0000000..e69de29 diff --git a/Makefile b/Makefile index 423e373..ed5b612 100644 --- a/Makefile +++ b/Makefile @@ -45,8 +45,8 @@ endif endif GENERATED_HEADERS = config-host.h qemu-options.def -GENERATED_HEADERS += qmp-commands.h qapi-types.h qapi-visit.h -GENERATED_SOURCES += qmp-marshal.c qapi-types.c qapi-visit.c +GENERATED_HEADERS += qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h +GENERATED_SOURCES += qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c GENERATED_HEADERS += trace/generated-events.h GENERATED_SOURCES += trace/generated-events.c @@ -208,7 +208,7 @@ Makefile: $(version-obj-y) $(version-lobj-y) # Build libraries libqemustub.a: $(stub-obj-y) -libqemuutil.a: $(util-obj-y) qapi-types.o qapi-visit.o +libqemuutil.a: $(util-obj-y) qapi-types.o qapi-visit.o qapi-event.o block-modules = $(foreach o,$(block-obj-m),"$(basename $(subst /,-,$o))",) NULL util/module.o-cflags = -D'CONFIG_BLOCK_MODULES=$(block-modules)' @@ -252,6 +252,9 @@ $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py) qapi-visit.c qapi-visit.h :\ $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-visit.py $(qapi-py) $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-visit.py $(gen-out-type) -o "." -b < $<, " GEN $@") +qapi-event.c qapi-event.h :\ +$(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-event.py $(qapi-py) + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-event.py $(gen-out-type) -o "." -b < $<, " GEN $@") qmp-commands.h qmp-marshal.c :\ $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py) $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-commands.py $(gen-out-type) -m -o "." < $<, " GEN $@") diff --git a/Makefile.objs b/Makefile.objs index a6e0e2a..93697a1 100644 --- a/Makefile.objs +++ b/Makefile.objs @@ -12,7 +12,7 @@ block-obj-y += main-loop.o iohandler.o qemu-timer.o block-obj-$(CONFIG_POSIX) += aio-posix.o block-obj-$(CONFIG_WIN32) += aio-win32.o block-obj-y += block/ -block-obj-y += qapi-types.o qapi-visit.o +block-obj-y += qapi-types.o qapi-visit.o qapi-event.o block-obj-y += qemu-io-cmds.o block-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o diff --git a/docs/qapi-code-gen.txt b/docs/qapi-code-gen.txt index d78921f..91e4e13 100644 --- a/docs/qapi-code-gen.txt +++ b/docs/qapi-code-gen.txt @@ -180,6 +180,24 @@ An example command is: 'data': { 'arg1': 'str', '*arg2': 'str' }, 'returns': 'str' } +=== Events === + +Events are defined with key word 'event'. When 'data' is also specified, +additional info will be carried on. Finally there will be C API generated +in qapi-event.h, and when called by QEMU code, a message with timestamp will +be emitted on the wire. If timestamp is -1, it means failure to retrieve host +time. + +An example event is: + +{ 'event': 'EVENT_C', + 'data': { '*a': 'int', 'b': 'str' } } + +Resulting in this JSON object: + +{ "event": "EVENT_C", + "data": { "b": "test string" }, + "timestamp": { "seconds": 1267020223, "microseconds": 435656 } } == Code generation == diff --git a/scripts/qapi-event.py b/scripts/qapi-event.py new file mode 100644 index 0000000..501baff --- /dev/null +++ b/scripts/qapi-event.py @@ -0,0 +1,366 @@ +# +# QAPI event generator +# +# Copyright (c) 2014 Wenchao Xia +# +# Authors: +# Wenchao Xia +# +# This work is licensed under the terms of the GNU GPL, version 2. +# See the COPYING file in the top-level directory. + +from ordereddict import OrderedDict +from qapi import * +import sys +import os +import getopt +import errno + +def _generate_event_api_name(event_name, params): + api_name = "void qapi_event_send_%s(" % c_fun(event_name).lower(); + l = len(api_name) + + if params: + for argname, argentry, optional, structured in parse_args(params): + if optional: + api_name += "bool has_%s,\n" % c_var(argname) + api_name += "".ljust(l) + + if argentry == "str": + api_name += "const " + api_name += "%s %s,\n" % (c_type(argentry), c_var(argname)) + api_name += "".ljust(l) + + api_name += "Error **errp)" + return api_name; + + +# Following are the core functions that generate C APIs to emit event. + +def generate_event_declaration(api_name): + return mcgen(''' + +%(api_name)s; +''', + api_name = api_name) + +def generate_event_implement(api_name, event_name, params): + # step 1: declare and variables + ret = mcgen(""" + +%(api_name)s +{ + QDict *qmp; + Error *local_err = NULL; + QMPEventFuncEmit emit; +""", + api_name = api_name) + + if params: + ret += mcgen(""" + QmpOutputVisitor *qov; + Visitor *v; + QObject *obj; + +""") + + # step 2: check emit function, create a dict + ret += mcgen(""" + emit = qmp_event_get_func_emit(); + if (!emit) { + return; + } + + qmp = qmp_event_build_dict("%(event_name)s"); + +""", + event_name = event_name) + + # step 3: visit the params if params != None + if params: + ret += mcgen(""" + qov = qmp_output_visitor_new(); + g_assert(qov); + + v = qmp_output_get_visitor(qov); + g_assert(v); + + /* Fake visit, as if all member are under a structure */ + visit_start_struct(v, NULL, "", "%(event_name)s", 0, &local_err); + if (error_is_set(&local_err)) { + goto clean; + } + +""", + event_name = event_name) + + for argname, argentry, optional, structured in parse_args(params): + if optional: + ret += mcgen(""" + if (has_%(var)s) { +""", + var = c_var(argname)) + push_indent() + + if argentry == "str": + var_type = "(char **)" + else: + var_type = "" + + ret += mcgen(""" + visit_type_%(type)s(v, %(var_type)s&%(var)s, "%(name)s", &local_err); + if (error_is_set(&local_err)) { + goto clean; + } +""", + var_type = var_type, + var = c_var(argname), + type = type_name(argentry), + name = argname) + + if optional: + pop_indent() + ret += mcgen(""" + } +""") + + ret += mcgen(""" + + visit_end_struct(v, &local_err); + if (error_is_set(&local_err)) { + goto clean; + } + + obj = qmp_output_get_qobject(qov); + g_assert(obj != NULL); + + qdict_put_obj(qmp, "data", obj); +""") + + # step 4: call qmp event api + ret += mcgen(""" + emit(%(event_enum_value)s, qmp, &local_err); + +""", + event_enum_value = event_enum_value) + + # step 5: clean up + if params: + ret += mcgen(""" + clean: + qmp_output_visitor_cleanup(qov); +""") + ret += mcgen(""" + error_propagate(errp, local_err); + QDECREF(qmp); +} +""") + + return ret + + +# Following are the functions that generate an enum type for all defined +# events, similar to qapi-types.py. Here we already have enum name and +# values which were generated before and recorded in event_enum_*. It also +# works around the issue that "import qapi-types" can't work. + +def generate_event_enum_decl(event_enum_name, event_enum_values): + lookup_decl = mcgen(''' + +extern const char *%(event_enum_name)s_lookup[]; +''', + event_enum_name = event_enum_name) + + enum_decl = mcgen(''' +typedef enum %(event_enum_name)s +{ +''', + event_enum_name = event_enum_name) + + # append automatically generated _MAX value + enum_max_value = generate_enum_full_value(event_enum_name, "MAX") + enum_values = event_enum_values + [ enum_max_value ] + + i = 0 + for value in enum_values: + enum_decl += mcgen(''' + %(value)s = %(i)d, +''', + value = value, + i = i) + i += 1 + + enum_decl += mcgen(''' +} %(event_enum_name)s; +''', + event_enum_name = event_enum_name) + + return lookup_decl + enum_decl + +def generate_event_enum_lookup(event_enum_name, event_enum_strings): + ret = mcgen(''' + +const char *%(event_enum_name)s_lookup[] = { +''', + event_enum_name = event_enum_name) + + i = 0 + for string in event_enum_strings: + ret += mcgen(''' + "%(string)s", +''', + string = string) + + ret += mcgen(''' + NULL, +}; +''') + return ret + + +# Start the real job + +try: + opts, args = getopt.gnu_getopt(sys.argv[1:], "chbp:o:", + ["source", "header", "builtins", "prefix=", + "output-dir="]) +except getopt.GetoptError, err: + print str(err) + sys.exit(1) + +output_dir = "" +prefix = "" +c_file = 'qapi-event.c' +h_file = 'qapi-event.h' + +do_c = False +do_h = False +do_builtins = False + +for o, a in opts: + if o in ("-p", "--prefix"): + prefix = a + elif o in ("-o", "--output-dir"): + output_dir = a + "/" + elif o in ("-c", "--source"): + do_c = True + elif o in ("-h", "--header"): + do_h = True + elif o in ("-b", "--builtins"): + do_builtins = True + +if not do_c and not do_h: + do_c = True + do_h = True + +c_file = output_dir + prefix + c_file +h_file = output_dir + prefix + h_file + +try: + os.makedirs(output_dir) +except os.error, e: + if e.errno != errno.EEXIST: + raise + +def maybe_open(really, name, opt): + if really: + return open(name, opt) + else: + import StringIO + return StringIO.StringIO() + +fdef = maybe_open(do_c, c_file, 'w') +fdecl = maybe_open(do_h, h_file, 'w') + +fdef.write(mcgen(''' +/* THIS FILE IS AUTOMATICALLY GENERATED, DO NOT MODIFY */ + +/* + * schema-defined QAPI event functions + * + * Copyright (c) 2014 Wenchao Xia + * + * Authors: + * Wenchao Xia + * + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. + * See the COPYING.LIB file in the top-level directory. + * + */ + +#include "qemu-common.h" +#include "%(header)s" +#include "%(prefix)sqapi-visit.h" +#include "qapi/qmp-output-visitor.h" +#include "qapi/qmp-event.h" + +''', + prefix=prefix, header=basename(h_file))) + +fdecl.write(mcgen(''' +/* THIS FILE IS AUTOMATICALLY GENERATED, DO NOT MODIFY */ + +/* + * schema-defined QAPI event functions + * + * Copyright (c) 2014 Wenchao Xia + * + * Authors: + * Wenchao Xia + * + * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. + * See the COPYING.LIB file in the top-level directory. + * + */ + +#ifndef %(guard)s +#define %(guard)s + +#include "qapi/error.h" +#include "qapi/qmp/qdict.h" +#include "%(prefix)sqapi-types.h" + +''', + prefix=prefix, guard=guardname(h_file))) + +exprs = parse_schema(sys.stdin) + +event_enum_name = prefix.upper().replace('-', '_') + "QAPIEvent" +event_enum_values = [] +event_enum_strings = [] + +for expr in exprs: + if expr.has_key('event'): + event_name = expr['event'] + params = expr.get('data') + if params and len(params) == 0: + params = None + + api_name = _generate_event_api_name(event_name, params) + ret = generate_event_declaration(api_name) + fdecl.write(ret) + + # We need an enum value per event + event_enum_value = generate_enum_full_value(event_enum_name, + event_name) + ret = generate_event_implement(api_name, event_name, params) + fdef.write(ret) + + # Record it, and generate enum later + event_enum_values.append(event_enum_value) + event_enum_strings.append(event_name) + +ret = generate_event_enum_decl(event_enum_name, event_enum_values) +fdecl.write(ret) +ret = generate_event_enum_lookup(event_enum_name, event_enum_strings) +fdef.write(ret) + +fdecl.write(''' +#endif +''') + +fdecl.flush() +fdecl.close() + +fdef.flush() +fdef.close() diff --git a/scripts/qapi.py b/scripts/qapi.py index b474c39..b2941d1 100644 --- a/scripts/qapi.py +++ b/scripts/qapi.py @@ -199,6 +199,16 @@ def discriminator_find_enum_define(expr): return find_enum(discriminator_type) +def check_event(expr, expr_info): + params = expr.get('data') + if params: + for argname, argentry, optional, structured in parse_args(params): + if structured: + raise QAPIExprError(expr_info, + "Nested structure define in event is not " + "supported now, event '%s', argname '%s'" + % (expr['event'], argname)) + def check_union(expr, expr_info): name = expr['union'] base = expr.get('base') @@ -262,6 +272,8 @@ def check_exprs(schema): expr = expr_elem['expr'] if expr.has_key('union'): check_union(expr, expr_elem['info']) + if expr.has_key('event'): + check_event(expr, expr_elem['info']) def parse_schema(fp): try: diff --git a/tests/Makefile b/tests/Makefile index 88f7105..e477f14 100644 --- a/tests/Makefile +++ b/tests/Makefile @@ -175,7 +175,8 @@ check-qapi-schema-y := $(addprefix tests/qapi-schema/, \ duplicate-key.json union-invalid-base.json flat-union-no-base.json \ flat-union-invalid-discriminator.json \ flat-union-invalid-branch-key.json flat-union-reverse-define.json \ - flat-union-string-discriminator.json) + flat-union-string-discriminator.json \ + event-nest-struct.json) GENERATED_HEADERS += tests/test-qapi-types.h tests/test-qapi-visit.h tests/test-qmp-commands.h diff --git a/tests/qapi-schema/event-nest-struct.err b/tests/qapi-schema/event-nest-struct.err new file mode 100644 index 0000000..6c10a02 --- /dev/null +++ b/tests/qapi-schema/event-nest-struct.err @@ -0,0 +1 @@ +:1: Nested structure define in event is not supported now, event 'EVENT_A', argname 'a' diff --git a/tests/qapi-schema/event-nest-struct.exit b/tests/qapi-schema/event-nest-struct.exit new file mode 100644 index 0000000..d00491f --- /dev/null +++ b/tests/qapi-schema/event-nest-struct.exit @@ -0,0 +1 @@ +1 diff --git a/tests/qapi-schema/event-nest-struct.json b/tests/qapi-schema/event-nest-struct.json new file mode 100644 index 0000000..ee6f3ec --- /dev/null +++ b/tests/qapi-schema/event-nest-struct.json @@ -0,0 +1,2 @@ +{ 'event': 'EVENT_A', + 'data': { 'a' : { 'string' : 'str', 'integer': 'int' }, 'b' : 'str' } }