From patchwork Mon Nov 23 17:44:18 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 547699 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id E89FE14030F for ; Tue, 24 Nov 2015 04:47:07 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b=LPxXG0J/; dkim-atps=neutral Received: from localhost ([::1]:33818 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a0vCk-0002df-5j for incoming@patchwork.ozlabs.org; Mon, 23 Nov 2015 12:47:06 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51475) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a0vAF-0006O3-2o for qemu-devel@nongnu.org; Mon, 23 Nov 2015 12:44:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a0vAD-0000SE-Iw for qemu-devel@nongnu.org; Mon, 23 Nov 2015 12:44:31 -0500 Received: from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]:34188) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a0vAD-0000S8-9c for qemu-devel@nongnu.org; Mon, 23 Nov 2015 12:44:29 -0500 Received: by wmvv187 with SMTP id v187so172749196wmv.1 for ; Mon, 23 Nov 2015 09:44:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=5r0CxHdG6i2Anbc6slaPUna9SFMCVmxdjFslvbEeYag=; b=LPxXG0J/zkD8KRC9Baw+UxoFPGKLvSFfy+zVitGgA5kmlGVRT1SZ/KpNGbcJRDXSpg 2QCGLimtqOvb2ZFfQrWu1W8ldjLSk4KnRYZaEpwrCsHjiu5PkBhlS0gvX5cTZjSmXVmR 3b4we/ifml9SyEOt6zxoDy0oaEc/7LDmC3ktcVrettIVYpaEEMtQAVhvnIOK9QxUBZ2s 6vbBXq2+G04zcSksmrUbhNtvUFRAWbep6ktLiDAxBurZbpAfyqo+zBakF0lCRCcbtC41 oD9pHxFFgBTWDH8ASFPB1jwdPTBsTa/GgsvZRtTkHdog7usIAo2auZJOtHspJdXR8oqC 0/JQ== X-Received: by 10.194.82.229 with SMTP id l5mr36093133wjy.140.1448300668803; Mon, 23 Nov 2015 09:44:28 -0800 (PST) Received: from donizetti.redhat.com (94-39-152-11.adsl-ull.clienti.tiscali.it. [94.39.152.11]) by smtp.gmail.com with ESMTPSA id q6sm14381872wmd.8.2015.11.23.09.44.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Nov 2015 09:44:28 -0800 (PST) From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Mon, 23 Nov 2015 18:44:18 +0100 Message-Id: <1448300659-23559-4-git-send-email-pbonzini@redhat.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1448300659-23559-1-git-send-email-pbonzini@redhat.com> References: <1448300659-23559-1-git-send-email-pbonzini@redhat.com> X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2a00:1450:400c:c09::231 Cc: armbru@redhat.com Subject: [Qemu-devel] [PATCH v2 3/4] qjson: store tokens in a GQueue X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Even though we still have the "streamer" concept, the tokens can now be deleted as they are read. While doing so convert from QList to GQueue, since the next step will make tokens not a QObject and we will have to do the conversion anyway. Signed-off-by: Paolo Bonzini --- include/qapi/qmp/json-parser.h | 4 +-- include/qapi/qmp/json-streamer.h | 8 ++--- monitor.c | 2 +- qga/main.c | 2 +- qobject/json-parser.c | 65 +++++++++++++--------------------------- qobject/json-streamer.c | 25 +++++++++------- qobject/qjson.c | 2 +- tests/libqtest.c | 2 +- 8 files changed, 45 insertions(+), 65 deletions(-) diff --git a/include/qapi/qmp/json-parser.h b/include/qapi/qmp/json-parser.h index 44d88f3..fea89f8 100644 --- a/include/qapi/qmp/json-parser.h +++ b/include/qapi/qmp/json-parser.h @@ -18,7 +18,7 @@ #include "qapi/qmp/qlist.h" #include "qapi/error.h" -QObject *json_parser_parse(QList *tokens, va_list *ap); -QObject *json_parser_parse_err(QList *tokens, va_list *ap, Error **errp); +QObject *json_parser_parse(GQueue *tokens, va_list *ap); +QObject *json_parser_parse_err(GQueue *tokens, va_list *ap, Error **errp); #endif diff --git a/include/qapi/qmp/json-streamer.h b/include/qapi/qmp/json-streamer.h index e901144..e9f2937 100644 --- a/include/qapi/qmp/json-streamer.h +++ b/include/qapi/qmp/json-streamer.h @@ -15,21 +15,21 @@ #define QEMU_JSON_STREAMER_H #include -#include "qapi/qmp/qlist.h" +#include "glib-compat.h" #include "qapi/qmp/json-lexer.h" typedef struct JSONMessageParser { - void (*emit)(struct JSONMessageParser *parser, QList *tokens); + void (*emit)(struct JSONMessageParser *parser, GQueue *tokens); JSONLexer lexer; int brace_count; int bracket_count; - QList *tokens; + GQueue *tokens; uint64_t token_size; } JSONMessageParser; void json_message_parser_init(JSONMessageParser *parser, - void (*func)(JSONMessageParser *, QList *)); + void (*func)(JSONMessageParser *, GQueue *)); int json_message_parser_feed(JSONMessageParser *parser, const char *buffer, size_t size); diff --git a/monitor.c b/monitor.c index e4cf34e..781c6f3 100644 --- a/monitor.c +++ b/monitor.c @@ -3849,7 +3849,7 @@ static QDict *qmp_check_input_obj(QObject *input_obj, Error **errp) return input_dict; } -static void handle_qmp_command(JSONMessageParser *parser, QList *tokens) +static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens) { Error *local_err = NULL; QObject *obj, *data; diff --git a/qga/main.c b/qga/main.c index d2a0ffc..f83a97d 100644 --- a/qga/main.c +++ b/qga/main.c @@ -570,7 +570,7 @@ static void process_command(GAState *s, QDict *req) } /* handle requests/control events coming in over the channel */ -static void process_event(JSONMessageParser *parser, QList *tokens) +static void process_event(JSONMessageParser *parser, GQueue *tokens) { GAState *s = container_of(parser, GAState, parser); QDict *qdict; diff --git a/qobject/json-parser.c b/qobject/json-parser.c index 7a287ea..07d9654 100644 --- a/qobject/json-parser.c +++ b/qobject/json-parser.c @@ -26,11 +26,8 @@ typedef struct JSONParserContext { Error *err; - struct { - QObject **buf; - size_t pos; - size_t count; - } tokens; + QObject *current; + GQueue *buf; } JSONParserContext; #define BUG_ON(cond) assert(!(cond)) @@ -274,56 +271,34 @@ out: return NULL; } +/* Note: unless the token object returned by parser_context_peek_token + * or parser_context_pop_token is explicitly incref'd, it will be + * deleted as soon as parser_context_pop_token is called again. + */ static QObject *parser_context_pop_token(JSONParserContext *ctxt) { - QObject *token; - g_assert(ctxt->tokens.pos < ctxt->tokens.count); - token = ctxt->tokens.buf[ctxt->tokens.pos]; - ctxt->tokens.pos++; - return token; + qobject_decref(ctxt->current); + assert(!g_queue_is_empty(ctxt->buf)); + ctxt->current = g_queue_pop_head(ctxt->buf); + return ctxt->current; } -/* Note: parser_context_{peek|pop}_token do not increment the - * token object's refcount. In both cases the references will continue - * to be tracked and cleaned up in parser_context_free(), so do not - * attempt to free the token object. - */ static QObject *parser_context_peek_token(JSONParserContext *ctxt) { - QObject *token; - g_assert(ctxt->tokens.pos < ctxt->tokens.count); - token = ctxt->tokens.buf[ctxt->tokens.pos]; - return token; -} - -static void tokens_append_from_iter(QObject *obj, void *opaque) -{ - JSONParserContext *ctxt = opaque; - g_assert(ctxt->tokens.pos < ctxt->tokens.count); - ctxt->tokens.buf[ctxt->tokens.pos++] = obj; - qobject_incref(obj); + assert(!g_queue_is_empty(ctxt->buf)); + return g_queue_peek_head(ctxt->buf); } -static JSONParserContext *parser_context_new(QList *tokens) +static JSONParserContext *parser_context_new(GQueue *tokens) { JSONParserContext *ctxt; - size_t count; if (!tokens) { return NULL; } - count = qlist_size(tokens); - if (count == 0) { - return NULL; - } - ctxt = g_malloc0(sizeof(JSONParserContext)); - ctxt->tokens.pos = 0; - ctxt->tokens.count = count; - ctxt->tokens.buf = g_malloc(count * sizeof(QObject *)); - qlist_iter(tokens, tokens_append_from_iter, ctxt); - ctxt->tokens.pos = 0; + ctxt->buf = tokens; return ctxt; } @@ -331,12 +306,12 @@ static JSONParserContext *parser_context_new(QList *tokens) /* to support error propagation, ctxt->err must be freed separately */ static void parser_context_free(JSONParserContext *ctxt) { - int i; if (ctxt) { - for (i = 0; i < ctxt->tokens.count; i++) { - qobject_decref(ctxt->tokens.buf[i]); + while (!g_queue_is_empty(ctxt->buf)) { + parser_context_pop_token(ctxt); } - g_free(ctxt->tokens.buf); + qobject_decref(ctxt->current); + g_queue_free(ctxt->buf); g_free(ctxt); } } @@ -673,12 +648,12 @@ static QObject *parse_value(JSONParserContext *ctxt, va_list *ap) return obj; } -QObject *json_parser_parse(QList *tokens, va_list *ap) +QObject *json_parser_parse(GQueue *tokens, va_list *ap) { return json_parser_parse_err(tokens, ap, NULL); } -QObject *json_parser_parse_err(QList *tokens, va_list *ap, Error **errp) +QObject *json_parser_parse_err(GQueue *tokens, va_list *ap, Error **errp) { JSONParserContext *ctxt = parser_context_new(tokens); QObject *result; diff --git a/qobject/json-streamer.c b/qobject/json-streamer.c index f240501..d2af38f 100644 --- a/qobject/json-streamer.c +++ b/qobject/json-streamer.c @@ -22,6 +22,14 @@ #define MAX_TOKEN_SIZE (64ULL << 20) #define MAX_NESTING (1ULL << 10) +static void json_message_free_tokens(JSONMessageParser *parser) +{ + if (parser->tokens) { + g_queue_free(parser->tokens); + parser->tokens = NULL; + } +} + static void json_message_process_token(JSONLexer *lexer, GString *input, JSONTokenType type, int x, int y) { JSONMessageParser *parser = container_of(lexer, JSONMessageParser, lexer); @@ -54,7 +62,7 @@ static void json_message_process_token(JSONLexer *lexer, GString *input, JSONTok parser->token_size += input->len; - qlist_append(parser->tokens, dict); + g_queue_push_tail(parser->tokens, dict); if (type == JSON_ERROR) { goto out_emit_bad; @@ -78,27 +86,24 @@ out_emit_bad: /* clear out token list and tell the parser to emit and error * indication by passing it a NULL list */ - QDECREF(parser->tokens); - parser->tokens = NULL; + json_message_free_tokens(parser); out_emit: /* send current list of tokens to parser and reset tokenizer */ parser->brace_count = 0; parser->bracket_count = 0; + /* parser->emit takes ownership of parser->tokens. */ parser->emit(parser, parser->tokens); - if (parser->tokens) { - QDECREF(parser->tokens); - } - parser->tokens = qlist_new(); + parser->tokens = g_queue_new(); parser->token_size = 0; } void json_message_parser_init(JSONMessageParser *parser, - void (*func)(JSONMessageParser *, QList *)) + void (*func)(JSONMessageParser *, GQueue *)) { parser->emit = func; parser->brace_count = 0; parser->bracket_count = 0; - parser->tokens = qlist_new(); + parser->tokens = g_queue_new(); parser->token_size = 0; json_lexer_init(&parser->lexer, json_message_process_token); @@ -118,5 +123,5 @@ int json_message_parser_flush(JSONMessageParser *parser) void json_message_parser_destroy(JSONMessageParser *parser) { json_lexer_destroy(&parser->lexer); - QDECREF(parser->tokens); + json_message_free_tokens(parser); } diff --git a/qobject/qjson.c b/qobject/qjson.c index 33f8ef5..a3e6a7c 100644 --- a/qobject/qjson.c +++ b/qobject/qjson.c @@ -28,7 +28,7 @@ typedef struct JSONParsingState QObject *result; } JSONParsingState; -static void parse_json(JSONMessageParser *parser, QList *tokens) +static void parse_json(JSONMessageParser *parser, GQueue *tokens) { JSONParsingState *s = container_of(parser, JSONParsingState, parser); s->result = json_parser_parse(tokens, s->ap); diff --git a/tests/libqtest.c b/tests/libqtest.c index f6f3d7a..9753161 100644 --- a/tests/libqtest.c +++ b/tests/libqtest.c @@ -351,7 +351,7 @@ typedef struct { QDict *response; } QMPResponseParser; -static void qmp_response(JSONMessageParser *parser, QList *tokens) +static void qmp_response(JSONMessageParser *parser, GQueue *tokens) { QMPResponseParser *qmp = container_of(parser, QMPResponseParser, parser); QObject *obj;