From patchwork Fri Jun 17 12:19:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victor Toso X-Patchwork-Id: 1644834 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ATuZXhyA; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LPdbh17VNz9s75 for ; Fri, 17 Jun 2022 22:27:44 +1000 (AEST) Received: from localhost ([::1]:46464 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2B4d-0000RG-Tc for incoming@patchwork.ozlabs.org; Fri, 17 Jun 2022 08:27:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47218) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2AxK-0001si-2Z for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:20:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44106) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Ax2-00050P-D6 for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:20:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655468387; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vakw/pNnC3CkaE/+zF+/jES8tUNyN0HAQuQGUIPmDq0=; b=ATuZXhyAaElM38ZDe/fdXKcGz7mcrNBkbK8RIX3fojcrmaa0mdocjlwvi5UqmiCQOODGx5 8sB7hVmkp0c7FgJhzlFCAOwG0e2PAkn3vK11C92JdyI7IEqvoOrgkWFd5tKjdAKctKQAJo 5Ui2JGOkNk1pMKDVVyEwxtnfqw8kFiQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-286-njIXm4cTP_eJ9TaSBzjMRQ-1; Fri, 17 Jun 2022 08:19:39 -0400 X-MC-Unique: njIXm4cTP_eJ9TaSBzjMRQ-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D3A77801756 for ; Fri, 17 Jun 2022 12:19:37 +0000 (UTC) Received: from tapioca.redhat.com (unknown [10.40.192.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3422440334E; Fri, 17 Jun 2022 12:19:36 +0000 (UTC) From: Victor Toso To: qemu-devel@nongnu.org Cc: Eric Blake , Markus Armbruster , John Snow , Andrea Bolognani , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [RFC PATCH v2 1/8] qapi: golang: Generate qapi's enum types in Go Date: Fri, 17 Jun 2022 14:19:25 +0200 Message-Id: <20220617121932.249381-2-victortoso@redhat.com> In-Reply-To: <20220617121932.249381-1-victortoso@redhat.com> References: <20220617121932.249381-1-victortoso@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Received-SPF: pass client-ip=170.10.133.124; envelope-from=victortoso@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch handles QAPI enum types and generates its equivalent in Go. Basically, Enums are being handled as strings in Golang. 1. For each QAPI enum, we will define a string type in Go to be the assigned type of this specific enum. 2. Naming: CamelCase will be used in any identifier that we want to export [0], which is everything. [0] https://go.dev/ref/spec#Exported_identifiers Example: qapi: | { 'enum': 'DisplayProtocol', | 'data': [ 'vnc', 'spice' ] } go: | type DisplayProtocol string | | const ( | DisplayProtocolVnc DisplayProtocol = "vnc" | DisplayProtocolSpice DisplayProtocol = "spice" | ) Signed-off-by: Victor Toso --- scripts/qapi/golang.py | 134 +++++++++++++++++++++++++++++++++++++++++ scripts/qapi/main.py | 2 + 2 files changed, 136 insertions(+) create mode 100644 scripts/qapi/golang.py diff --git a/scripts/qapi/golang.py b/scripts/qapi/golang.py new file mode 100644 index 0000000000..f2776520a1 --- /dev/null +++ b/scripts/qapi/golang.py @@ -0,0 +1,134 @@ +""" +Golang QAPI generator +""" +# Copyright (c) 2022 Red Hat Inc. +# +# Authors: +# Victor Toso +# +# This work is licensed under the terms of the GNU GPL, version 2. +# See the COPYING file in the top-level directory. + +# Just for type hint on self +from __future__ import annotations + +import os +from typing import List, Optional + +from .schema import ( + QAPISchema, + QAPISchemaType, + QAPISchemaVisitor, + QAPISchemaEnumMember, + QAPISchemaFeature, + QAPISchemaIfCond, + QAPISchemaObjectType, + QAPISchemaObjectTypeMember, + QAPISchemaVariants, +) +from .source import QAPISourceInfo + + +class QAPISchemaGenGolangVisitor(QAPISchemaVisitor): + + def __init__(self, prefix: str): + super().__init__() + self.target = {name: "" for name in ["enum"]} + self.schema = None + self.golang_package_name = "qapi" + + def visit_begin(self, schema): + self.schema = schema + + # Every Go file needs to reference its package name + for target in self.target: + self.target[target] = f"package {self.golang_package_name}\n" + + def visit_end(self): + self.schema = None + + def visit_object_type(self: QAPISchemaGenGolangVisitor, + name: str, + info: Optional[QAPISourceInfo], + ifcond: QAPISchemaIfCond, + features: List[QAPISchemaFeature], + base: Optional[QAPISchemaObjectType], + members: List[QAPISchemaObjectTypeMember], + variants: Optional[QAPISchemaVariants] + ) -> None: + pass + + def visit_alternate_type(self: QAPISchemaGenGolangVisitor, + name: str, + info: Optional[QAPISourceInfo], + ifcond: QAPISchemaIfCond, + features: List[QAPISchemaFeature], + variants: QAPISchemaVariants + ) -> None: + pass + + def visit_enum_type(self: QAPISchemaGenGolangVisitor, + name: str, + info: Optional[QAPISourceInfo], + ifcond: QAPISchemaIfCond, + features: List[QAPISchemaFeature], + members: List[QAPISchemaEnumMember], + prefix: Optional[str] + ) -> None: + + value = qapi_to_field_name_enum(members[0].name) + fields = "" + for member in members: + value = qapi_to_field_name_enum(member.name) + fields += f'''\t{name}{value} {name} = "{member.name}"\n''' + + self.target["enum"] += f''' +type {name} string +const ( +{fields[:-1]} +) +''' + + def visit_array_type(self, name, info, ifcond, element_type): + pass + + def visit_command(self, + name: str, + info: Optional[QAPISourceInfo], + ifcond: QAPISchemaIfCond, + features: List[QAPISchemaFeature], + arg_type: Optional[QAPISchemaObjectType], + ret_type: Optional[QAPISchemaType], + gen: bool, + success_response: bool, + boxed: bool, + allow_oob: bool, + allow_preconfig: bool, + coroutine: bool) -> None: + pass + + def visit_event(self, name, info, ifcond, features, arg_type, boxed): + pass + + def write(self, output_dir: str) -> None: + for module_name, content in self.target.items(): + go_module = module_name + "s.go" + go_dir = "go" + pathname = os.path.join(output_dir, go_dir, go_module) + odir = os.path.dirname(pathname) + os.makedirs(odir, exist_ok=True) + + with open(pathname, "w") as outfile: + outfile.write(content) + + +def gen_golang(schema: QAPISchema, + output_dir: str, + prefix: str) -> None: + vis = QAPISchemaGenGolangVisitor(prefix) + schema.visit(vis) + vis.write(output_dir) + + +def qapi_to_field_name_enum(name: str) -> str: + return name.title().replace("-", "") diff --git a/scripts/qapi/main.py b/scripts/qapi/main.py index fc216a53d3..661fb1e091 100644 --- a/scripts/qapi/main.py +++ b/scripts/qapi/main.py @@ -15,6 +15,7 @@ from .common import must_match from .error import QAPIError from .events import gen_events +from .golang import gen_golang from .introspect import gen_introspect from .schema import QAPISchema from .types import gen_types @@ -54,6 +55,7 @@ def generate(schema_file: str, gen_events(schema, output_dir, prefix) gen_introspect(schema, output_dir, prefix, unmask) + gen_golang(schema, output_dir, prefix) def main() -> int: """ From patchwork Fri Jun 17 12:19:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victor Toso X-Patchwork-Id: 1644832 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Wrbn59C9; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LPdYt4Y72z9s75 for ; Fri, 17 Jun 2022 22:26:09 +1000 (AEST) Received: from localhost ([::1]:44304 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2B37-0007PT-B1 for incoming@patchwork.ozlabs.org; Fri, 17 Jun 2022 08:26:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46966) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Awy-0001Ou-4w for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:29969) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Aww-0004ym-7F for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655468381; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jkZTj+cIpLYJ1gsONDWzgyizG72ZOWTMjnB8hK2Ha8s=; b=Wrbn59C91IDTiMy4BpVSWiBaElca95UULpWGeAtd6MKPS2ntff8b6fmf+bx+NUjw7gx956 ZdaAjcmcsosaHfNoJx/NP5fxpLiEKzHAGNRy7QLWywwIsG3pQkNLPx29j7lM7qcAnOhgAm BR7snEX8Km+3fphFqPxywkcOXX7Z0+w= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-609-j9F2u_t5PGWc2vfpiwaxng-1; Fri, 17 Jun 2022 08:19:40 -0400 X-MC-Unique: j9F2u_t5PGWc2vfpiwaxng-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C73433C0D85D for ; Fri, 17 Jun 2022 12:19:39 +0000 (UTC) Received: from tapioca.redhat.com (unknown [10.40.192.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3FF7B40334E; Fri, 17 Jun 2022 12:19:38 +0000 (UTC) From: Victor Toso To: qemu-devel@nongnu.org Cc: Eric Blake , Markus Armbruster , John Snow , Andrea Bolognani , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [RFC PATCH v2 2/8] qapi: golang: Generate qapi's alternate types in Go Date: Fri, 17 Jun 2022 14:19:26 +0200 Message-Id: <20220617121932.249381-3-victortoso@redhat.com> In-Reply-To: <20220617121932.249381-1-victortoso@redhat.com> References: <20220617121932.249381-1-victortoso@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Received-SPF: pass client-ip=170.10.129.124; envelope-from=victortoso@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch handles QAPI alternate types and generates data structures in Go that handles it. At this moment, there are 5 alternates in qemu/qapi, they are: * BlockDirtyBitmapMergeSource * Qcow2OverlapChecks * BlockdevRef * BlockdevRefOrNull * StrOrNull Alternate types are similar to Union but without a discriminator that can be used to identify the underlying value on the wire. It is needed to infer it. In Go, all the types are mapped as optional fields and Marshal and Unmarshal methods will be handling the data checks. Example: qapi: | { 'alternate': 'BlockdevRef', | 'data': { 'definition': 'BlockdevOptions', | 'reference': 'str' } } go: | type BlockdevRef struct { | Definition *BlockdevOptions | Reference *string | } usage: | input := `{"driver":"qcow2","data-file":"/some/place/my-image"}` | k := BlockdevRef{} | err := json.Unmarshal([]byte(input), &k) | if err != nil { | panic(err) | } | // *k.Definition.Qcow2.DataFile.Reference == "/some/place/my-image" Signed-off-by: Victor Toso --- scripts/qapi/golang.py | 119 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 117 insertions(+), 2 deletions(-) diff --git a/scripts/qapi/golang.py b/scripts/qapi/golang.py index f2776520a1..37d7c062c9 100644 --- a/scripts/qapi/golang.py +++ b/scripts/qapi/golang.py @@ -29,11 +29,32 @@ from .source import QAPISourceInfo +TEMPLATE_HELPER = ''' +// Alias for go version lower than 1.18 +type Any = interface{} + +// Creates a decoder that errors on unknown Fields +// Returns true if successfully decoded @from string @into type +// Returns false without error is failed with "unknown field" +// Returns false with error is a different error was found +func StrictDecode(into interface{}, from []byte) error { + dec := json.NewDecoder(strings.NewReader(string(from))) + dec.DisallowUnknownFields() + + if err := dec.Decode(into); err != nil { + return err + } + return nil +} +''' + + class QAPISchemaGenGolangVisitor(QAPISchemaVisitor): def __init__(self, prefix: str): super().__init__() - self.target = {name: "" for name in ["enum"]} + self.target = {name: "" for name in ["alternate", "enum", "helper"]} + self.objects_seen = {} self.schema = None self.golang_package_name = "qapi" @@ -44,6 +65,8 @@ def visit_begin(self, schema): for target in self.target: self.target[target] = f"package {self.golang_package_name}\n" + self.target["helper"] += TEMPLATE_HELPER + def visit_end(self): self.schema = None @@ -65,7 +88,69 @@ def visit_alternate_type(self: QAPISchemaGenGolangVisitor, features: List[QAPISchemaFeature], variants: QAPISchemaVariants ) -> None: - pass + assert name not in self.objects_seen + self.objects_seen[name] = True + + marshal_return_default = f'nil, errors.New("{name} has empty fields")' + marshal_check_fields = "" + unmarshal_check_fields = "" + variant_fields = "" + + # We need to check if the Alternate type supports NULL as that + # means that JSON to Go would allow all fields to be empty. + # Alternate that don't support NULL, would fail to convert + # to JSON if all fields were empty. + return_on_null = f"errors.New(`null not supported for {name}`)" + + # Assembly the fields and all the checks for Marshal and + # Unmarshal methods + for var in variants.variants: + # Nothing to generate on null types. We update some + # variables to handle json-null on marshalling methods. + if var.type.name == "null": + marshal_return_default = '[]byte("null"), nil' + return_on_null = "nil" + continue + + var_name = qapi_to_field_name(var.name) + var_type = qapi_schema_type_to_go_type(var.type.name) + variant_fields += f"\t{var_name} *{var_type}\n" + + if len(marshal_check_fields) > 0: + marshal_check_fields += "} else " + + marshal_check_fields += f'''if s.{var_name} != nil {{ + return json.Marshal(s.{var_name}) + ''' + + unmarshal_check_fields += f'''// Check for {var_type} + {{ + s.{var_name} = new({var_type}) + if err := StrictDecode(s.{var_name}, data); err == nil {{ + return nil + }} + s.{var_name} = nil + }} +''' + + marshal_check_fields += "}" + + self.target["alternate"] += generate_struct_type(name, variant_fields) + self.target["alternate"] += f''' +func (s {name}) MarshalJSON() ([]byte, error) {{ + {marshal_check_fields} + return {marshal_return_default} +}} + +func (s *{name}) UnmarshalJSON(data []byte) error {{ + // Check for json-null first + if string(data) == "null" {{ + return {return_on_null} + }} + {unmarshal_check_fields} + return errors.New(fmt.Sprintf("Can't convert to {name}: %s", string(data))) +}} +''' def visit_enum_type(self: QAPISchemaGenGolangVisitor, name: str, @@ -130,5 +215,35 @@ def gen_golang(schema: QAPISchema, vis.write(output_dir) +# Helper function for boxed or self contained structures. +def generate_struct_type(type_name, args="") -> str: + args = args if len(args) == 0 else f"\n{args}\n" + return f''' +type {type_name} struct {{{args}}} +''' + + +def qapi_schema_type_to_go_type(type: str) -> str: + schema_types_to_go = { + 'str': 'string', 'null': 'nil', 'bool': 'bool', 'number': + 'float64', 'size': 'uint64', 'int': 'int64', 'int8': 'int8', + 'int16': 'int16', 'int32': 'int32', 'int64': 'int64', 'uint8': + 'uint8', 'uint16': 'uint16', 'uint32': 'uint32', 'uint64': + 'uint64', 'any': 'Any', 'QType': 'QType', + } + + prefix = "" + if type.endswith("List"): + prefix = "[]" + type = type[:-4] + + type = schema_types_to_go.get(type, type) + return prefix + type + + def qapi_to_field_name_enum(name: str) -> str: return name.title().replace("-", "") + + +def qapi_to_field_name(name: str) -> str: + return name.title().replace("_", "").replace("-", "") From patchwork Fri Jun 17 12:19:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victor Toso X-Patchwork-Id: 1644838 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=G+tiHRIS; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LPdg223TPz9s75 for ; Fri, 17 Jun 2022 22:30:38 +1000 (AEST) Received: from localhost ([::1]:52008 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2B7T-0004FM-P6 for incoming@patchwork.ozlabs.org; Fri, 17 Jun 2022 08:30:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46994) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Ax0-0001Um-HO for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:49725) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Awy-0004zB-K9 for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655468383; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UxLiIhoCRhDUEQB4LloOvnDaqMfE1O9VhJfYCIPTDo8=; b=G+tiHRISFAnzy4F/+fGUP8ST47KuSLD9DCID/Ue4ZodGor8qJl9kJnGLdbYFi5gMRYx7R8 t+yup+L1/B0eDpYqiO593Zx/eGC965Hk3kvvea/hvLb5Zy+blTYE6J0rVItWNjmLf00TVj gu43ZmcGCIJLose6GzkfcJq5tLr9Rgk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-19-8aETMbMJOzW40n0xVbxtbw-1; Fri, 17 Jun 2022 08:19:42 -0400 X-MC-Unique: 8aETMbMJOzW40n0xVbxtbw-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 10474811E76 for ; Fri, 17 Jun 2022 12:19:42 +0000 (UTC) Received: from tapioca.redhat.com (unknown [10.40.192.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 337CA40334E; Fri, 17 Jun 2022 12:19:40 +0000 (UTC) From: Victor Toso To: qemu-devel@nongnu.org Cc: Eric Blake , Markus Armbruster , John Snow , Andrea Bolognani , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [RFC PATCH v2 3/8] qapi: golang: Generate qapi's struct types in Go Date: Fri, 17 Jun 2022 14:19:27 +0200 Message-Id: <20220617121932.249381-4-victortoso@redhat.com> In-Reply-To: <20220617121932.249381-1-victortoso@redhat.com> References: <20220617121932.249381-1-victortoso@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Received-SPF: pass client-ip=170.10.129.124; envelope-from=victortoso@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch handles QAPI struct types and generates the equivalent types in Go. At the time of this writing, it generates 388 structures. The highlights of this implementation are: 1. Generating an Go struct that requires a @base type, the @base type fields are copied over to the Go struct. The advantage of this approach is to not have embed structs in any of the QAPI types. The downside are some generated Types that are likely useless now, like InetSocketAddressBase from InetSocketAddress. 2. About the Go struct's fields: i) They can be either by Value or Reference. ii) Every field that is marked as optional in the QAPI specification are translated to Reference fields in its Go structure. This design decision is the most straightforward way to check if a given field was set or not. iii) Mandatory fields are always by Value with the exception of QAPI arrays, which are handled by Reference (to a block of memory) by Go. iv) All the fields are named with Uppercase due Golang's export convention. v) In order to avoid any kind of issues when encoding ordecoding, to or from JSON, we mark all fields with its @name and, when it is optional, member, with @omitempty Example: qapi: | { 'struct': 'BlockdevCreateOptionsFile', | 'data': { 'filename': 'str', | 'size': 'size', | '*preallocation': 'PreallocMode', | '*nocow': 'bool', | '*extent-size-hint': 'size'} } go: | type BlockdevCreateOptionsFile struct { | Filename string `json:"filename"` | Size uint64 `json:"size"` | Preallocation *PreallocMode `json:"preallocation,omitempty"` | Nocow *bool `json:"nocow,omitempty"` | ExtentSizeHint *uint64 `json:"extent-size-hint,omitempty"` | } Signed-off-by: Victor Toso --- scripts/qapi/golang.py | 117 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 115 insertions(+), 2 deletions(-) diff --git a/scripts/qapi/golang.py b/scripts/qapi/golang.py index 37d7c062c9..1ab0c0bb46 100644 --- a/scripts/qapi/golang.py +++ b/scripts/qapi/golang.py @@ -53,7 +53,7 @@ class QAPISchemaGenGolangVisitor(QAPISchemaVisitor): def __init__(self, prefix: str): super().__init__() - self.target = {name: "" for name in ["alternate", "enum", "helper"]} + self.target = {name: "" for name in ["alternate", "enum", "helper", "struct"]} self.objects_seen = {} self.schema = None self.golang_package_name = "qapi" @@ -79,7 +79,37 @@ def visit_object_type(self: QAPISchemaGenGolangVisitor, members: List[QAPISchemaObjectTypeMember], variants: Optional[QAPISchemaVariants] ) -> None: - pass + # Do not handle anything besides structs + if (name == self.schema.the_empty_object_type.name or + not isinstance(name, str) or + info.defn_meta not in ["struct"]): + return + + # Safety checks. + assert name not in self.objects_seen + self.objects_seen[name] = True + + # visit all inner objects as well, they are not going to be + # called by python's generator. + if variants: + for var in variants.variants: + assert isinstance(var.type, QAPISchemaObjectType) + self.visit_object_type(self, + var.type.name, + var.type.info, + var.type.ifcond, + var.type.base, + var.type.local_members, + var.type.variants) + + # Save generated Go code to be written later + self.target[info.defn_meta] += qapi_to_golang_struct(name, + info, + ifcond, + features, + base, + members, + variants) def visit_alternate_type(self: QAPISchemaGenGolangVisitor, name: str, @@ -223,6 +253,72 @@ def generate_struct_type(type_name, args="") -> str: ''' +# Helper function that is used for most of QAPI types +def qapi_to_golang_struct(name: str, + info: Optional[QAPISourceInfo], + ifcond: QAPISchemaIfCond, + features: List[QAPISchemaFeature], + base: Optional[QAPISchemaObjectType], + members: List[QAPISchemaObjectTypeMember], + variants: Optional[QAPISchemaVariants]) -> str: + + type_name = qapi_to_go_type_name(name, info.defn_meta) + + fields = "" + + if base: + base_fields = "" + for lm in base.local_members: + # We do not include the Union's discriminator + # into the generated Go struct as the selection + # of what variant was set is enough on Go side. + if variants and variants.tag_member.name == lm.name: + continue + + field = qapi_to_field_name(lm.name) + member_type = qapi_schema_type_to_go_type(lm.type.name) + + isptr = "*" if lm.optional and member_type[0] not in "*[" else "" + optional = ",omitempty" if lm.optional else "" + fieldtag = f'`json:"{lm.name}{optional}"`' + + base_fields += f"\t{field} {isptr}{member_type}{fieldtag}\n" + + if len(base_fields) > 0: + fields += f"\t// Base fields\n\t{base_fields}\n" + + if members: + for memb in members: + field = qapi_to_field_name(memb.name) + member_type = qapi_schema_type_to_go_type(memb.type.name) + + isptr = "*" if memb.optional and member_type[0] not in "*[" else "" + optional = ",omitempty" if memb.optional else "" + fieldtag = f'`json:"{memb.name}{optional}"`' + + fields += f"\t{field} {isptr}{member_type}{fieldtag}\n" + + fields += "\n" + + if variants: + fields += "\t// Variants fields\n" + for var in variants.variants: + if var.type.is_implicit(): + continue + + field = qapi_to_field_name(var.name) + member_type = qapi_schema_type_to_go_type(var.type.name) + # Variant's are handled in the Marshal/Unmarshal methods + fieldtag = '`json:"-"`' + fields += f"\t{field} *{member_type}{fieldtag}\n" + member_type = qapi_schema_type_to_go_type(var.type.name) + # Variant's are handled in the Marshal/Unmarshal methods + fieldtag = '`json:"-"`' + fields += f"\t{field} *{member_type}{fieldtag}\n" + + return generate_struct_type(type_name, fields) + + def qapi_schema_type_to_go_type(type: str) -> str: schema_types_to_go = { 'str': 'string', 'null': 'nil', 'bool': 'bool', 'number': @@ -247,3 +343,20 @@ def qapi_to_field_name_enum(name: str) -> str: def qapi_to_field_name(name: str) -> str: return name.title().replace("_", "").replace("-", "") + + +def qapi_to_go_type_name(name: str, meta: str) -> str: + if name.startswith("q_obj_"): + name = name[6:] + + # We want to keep CamelCase for Golang types. We want to avoid removing + # already set CameCase names while fixing uppercase ones, eg: + # 1) q_obj_SocketAddress_base -> SocketAddressBase + # 2) q_obj_WATCHDOG-arg -> WatchdogArg + words = [word for word in name.replace("_", "-").split("-")] + name = words[0] + if name.islower() or name.isupper(): + name = name.title() + + name += ''.join(word.title() for word in words[1:]) + return name From patchwork Fri Jun 17 12:19:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victor Toso X-Patchwork-Id: 1644833 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Dm0bt9ff; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LPdYz2dFxz9s75 for ; Fri, 17 Jun 2022 22:26:15 +1000 (AEST) Received: from localhost ([::1]:44812 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2B3F-0007jE-3X for incoming@patchwork.ozlabs.org; Fri, 17 Jun 2022 08:26:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47038) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Ax2-0001Z8-EV for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:34198) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Ax0-0004zm-DU for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655468385; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bWBkF88ZLS2dEY/eZXueOzV74Qv41we61gYIwdDw3kw=; b=Dm0bt9ffv1xshnBt8uGWskQbmNCKckeqLQkmle/rFHaIui3L5sRoH8Lv6nAgEw3ENiK4VV zvb1PvGFyadF6O64XfYLT9UQQSbaCsl7bxMd5dX+jYrSYrgew7p5L9dPDLvAFjY71+hfYH jLXDDmn7SQi8f/UHcRz01RDzgnCp/iA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-633-oOFd9xP6OLiKf63y5FtvuQ-1; Fri, 17 Jun 2022 08:19:44 -0400 X-MC-Unique: oOFd9xP6OLiKf63y5FtvuQ-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 353C41C01B23 for ; Fri, 17 Jun 2022 12:19:44 +0000 (UTC) Received: from tapioca.redhat.com (unknown [10.40.192.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7067140334E; Fri, 17 Jun 2022 12:19:42 +0000 (UTC) From: Victor Toso To: qemu-devel@nongnu.org Cc: Eric Blake , Markus Armbruster , John Snow , Andrea Bolognani , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [RFC PATCH v2 4/8] qapi: golang: Generate qapi's union types in Go Date: Fri, 17 Jun 2022 14:19:28 +0200 Message-Id: <20220617121932.249381-5-victortoso@redhat.com> In-Reply-To: <20220617121932.249381-1-victortoso@redhat.com> References: <20220617121932.249381-1-victortoso@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Received-SPF: pass client-ip=170.10.129.124; envelope-from=victortoso@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch handles QAPI union types and generates the equivalent data structures and methods in Go to handle it. At the moment of this writing, it generates 38 structures. The QAPI union type has two types of fields: The @base and the @variants members. The @base fields can be considered common members for the union while only one field maximum is set for the @variants. In the QAPI specification, it defines a @discriminator field, which is an Enum type. The purpose of the @discriminator is to identify which @variant type is being used. The @discriminator is not used in the generated union Go structs as it suffices to check which one of the @variants fields were set. The union types implement the Marshaler and Unmarshaler interfaces to seamless decode from JSON objects to Golang structs and vice versa. qapi: | { 'union': 'SetPasswordOptions', | 'base': { 'protocol': 'DisplayProtocol', | 'password': 'str', | '*connected': 'SetPasswordAction' }, | 'discriminator': 'protocol', | 'data': { 'vnc': 'SetPasswordOptionsVnc' } } go: | type SetPasswordOptions struct { | // Base fields | Password string `json:"password"` | Connected *SetPasswordAction `json:"connected,omitempty"` | | // Variants fields | Vnc *SetPasswordOptionsVnc `json:"-"` | } Signed-off-by: Victor Toso --- scripts/qapi/golang.py | 112 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 105 insertions(+), 7 deletions(-) diff --git a/scripts/qapi/golang.py b/scripts/qapi/golang.py index 1ab0c0bb46..6c6a5cea97 100644 --- a/scripts/qapi/golang.py +++ b/scripts/qapi/golang.py @@ -53,7 +53,8 @@ class QAPISchemaGenGolangVisitor(QAPISchemaVisitor): def __init__(self, prefix: str): super().__init__() - self.target = {name: "" for name in ["alternate", "enum", "helper", "struct"]} + self.target = {name: "" for name in ["alternate", "enum", "helper", "struct", + "union"]} self.objects_seen = {} self.schema = None self.golang_package_name = "qapi" @@ -79,10 +80,14 @@ def visit_object_type(self: QAPISchemaGenGolangVisitor, members: List[QAPISchemaObjectTypeMember], variants: Optional[QAPISchemaVariants] ) -> None: - # Do not handle anything besides structs + # Do not handle anything besides struct and unions. if (name == self.schema.the_empty_object_type.name or not isinstance(name, str) or - info.defn_meta not in ["struct"]): + info.defn_meta not in ["struct", "union"]): + return + + # Base structs are embed + if qapi_name_is_base(name): return # Safety checks. @@ -110,6 +115,10 @@ def visit_object_type(self: QAPISchemaGenGolangVisitor, base, members, variants) + if info.defn_meta == "union": + self.target[info.defn_meta] += qapi_to_golang_methods_union(name, + info, + variants) def visit_alternate_type(self: QAPISchemaGenGolangVisitor, name: str, @@ -311,14 +320,99 @@ def qapi_to_golang_struct(name: str, # Variant's are handled in the Marshal/Unmarshal methods fieldtag = '`json:"-"`' fields += f"\t{field} *{member_type}{fieldtag}\n" - member_type = qapi_schema_type_to_go_type(var.type.name) - # Variant's are handled in the Marshal/Unmarshal methods - fieldtag = '`json:"-"`' - fields += f"\t{field} *{member_type}{fieldtag}\n" return generate_struct_type(type_name, fields) +def qapi_to_golang_methods_union(name: str, + info: Optional[QAPISourceInfo], + variants: Optional[QAPISchemaVariants] + ) -> str: + + type_name = qapi_to_go_type_name(name, info.defn_meta) + + driverCases = "" + checkFields = "" + if variants: + for var in variants.variants: + if var.type.is_implicit(): + continue + + field = qapi_to_field_name(var.name) + member_type = qapi_schema_type_to_go_type(var.type.name) + + if len(checkFields) > 0: + checkFields += "\t} else " + checkFields += f'''if s.{field} != nil {{ + driver = "{var.name}" + payload, err = json.Marshal(s.{field}) +''' + # for Unmarshal method + driverCases += f''' + case "{var.name}": + s.{field} = new({member_type}) + if err := json.Unmarshal(data, s.{field}); err != nil {{ + s.{field} = nil + return err + }}''' + + checkFields += "}" + + return f''' +func (s {type_name}) MarshalJSON() ([]byte, error) {{ + type Alias {type_name} + alias := Alias(s) + base, err := json.Marshal(alias) + if err != nil {{ + return nil, err + }} + + driver := "" + payload := []byte{{}} + {checkFields} + + if err != nil {{ + return nil, err + }} + + if len(base) == len("{{}}") {{ + base = []byte("") + }} else {{ + base = append([]byte{{','}}, base[1:len(base)-1]...) + }} + + if len(payload) == 0 || len(payload) == len("{{}}") {{ + payload = []byte("") + }} else {{ + payload = append([]byte{{','}}, payload[1:len(payload)-1]...) + }} + + result := fmt.Sprintf(`{{"{variants.tag_member.name}":"%s"%s%s}}`, driver, base, payload) + return []byte(result), nil +}} + +func (s *{type_name}) UnmarshalJSON(data []byte) error {{ + type Alias {type_name} + peek := struct {{ + Alias + Driver string `json:"{variants.tag_member.name}"` + }}{{}} + + + if err := json.Unmarshal(data, &peek); err != nil {{ + return err + }} + *s = {type_name}(peek.Alias) + + switch peek.Driver {{ + {driverCases} + }} + // Unrecognizer drivers are silently ignored. + return nil +}} +''' + + def qapi_schema_type_to_go_type(type: str) -> str: schema_types_to_go = { 'str': 'string', 'null': 'nil', 'bool': 'bool', 'number': @@ -345,6 +439,10 @@ def qapi_to_field_name(name: str) -> str: return name.title().replace("_", "").replace("-", "") +def qapi_name_is_base(name: str) -> bool: + return name.startswith("q_obj_") and name.endswith("-base") + + def qapi_to_go_type_name(name: str, meta: str) -> str: if name.startswith("q_obj_"): name = name[6:] From patchwork Fri Jun 17 12:19:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victor Toso X-Patchwork-Id: 1644830 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=PSVfqLfq; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LPdVL0ZDdz9s75 for ; Fri, 17 Jun 2022 22:23:06 +1000 (AEST) Received: from localhost ([::1]:37810 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2B0A-0002jE-CZ for incoming@patchwork.ozlabs.org; Fri, 17 Jun 2022 08:23:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47096) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Ax7-0001fG-7r for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:23129) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Ax3-00050S-1R for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655468387; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QpPwu1g9dMEHFOL4cfYX89f8RiOgjrXW5hdE6Qec2w8=; b=PSVfqLfqhQ+bDVgxMxtQAlBNUcQbNAkFliTJIIiXwWX+bUUvW9Ml1VUOfxeT0HWv3Z1yxm 1aRgb0nI7HgeeZzXRpDK/Bz5UMXorpfXBODnNJUtCTgcGFL3jmTCRr1Vz9frHUtT0FAUma jCHe3jcGBlfFYVvJQRSB1vd0n/q5tPQ= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-377-DgLDvEf6PbOp-ox69ATzdg-1; Fri, 17 Jun 2022 08:19:46 -0400 X-MC-Unique: DgLDvEf6PbOp-ox69ATzdg-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4059C3C0D860 for ; Fri, 17 Jun 2022 12:19:46 +0000 (UTC) Received: from tapioca.redhat.com (unknown [10.40.192.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9232A4328C9; Fri, 17 Jun 2022 12:19:44 +0000 (UTC) From: Victor Toso To: qemu-devel@nongnu.org Cc: Eric Blake , Markus Armbruster , John Snow , Andrea Bolognani , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [RFC PATCH v2 5/8] qapi: golang: Generate qapi's event types in Go Date: Fri, 17 Jun 2022 14:19:29 +0200 Message-Id: <20220617121932.249381-6-victortoso@redhat.com> In-Reply-To: <20220617121932.249381-1-victortoso@redhat.com> References: <20220617121932.249381-1-victortoso@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Received-SPF: pass client-ip=170.10.129.124; envelope-from=victortoso@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch handles QAPI event types and generates data structures in Go that handles it. We also define a Event interface and two helper functions MarshalEvent and UnmarshalEvent. At the moment of this writing, this patch generates 51 structures (50 events) Example: qapi: | { 'event': 'MEMORY_DEVICE_SIZE_CHANGE', | 'data': { '*id': 'str', 'size': 'size', 'qom-path' : 'str'} } go: | type MemoryDeviceSizeChangeEvent struct { | EventTimestamp Timestamp `json:"-"` | Id *string `json:"id,omitempty"` | Size uint64 `json:"size"` | QomPath string `json:"qom-path"` | } usage: | input := `{"event":"MEMORY_DEVICE_SIZE_CHANGE",` + | `"timestamp":{"seconds":1588168529,"microseconds":201316},` + | `"data":{"id":"vm0","size":1073741824,"qom-path":"/machine/unattached/device[2]"}}` | e, err := UnmarshalEvent([]byte(input) | if err != nil { | panic(err) | } | if e.GetName() == `MEMORY_DEVICE_SIZE_CHANGE` { | m := e.(*MemoryDeviceSizeChangeEvent) | // m.QomPath == "/machine/unattached/device[2]" | } Signed-off-by: Victor Toso --- scripts/qapi/golang.py | 120 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 118 insertions(+), 2 deletions(-) diff --git a/scripts/qapi/golang.py b/scripts/qapi/golang.py index 6c6a5cea97..b2e08cebdf 100644 --- a/scripts/qapi/golang.py +++ b/scripts/qapi/golang.py @@ -28,7 +28,66 @@ ) from .source import QAPISourceInfo +# Only variable is @unm_cases to handle +# all events's names and associated types. +TEMPLATE_EVENT = ''' +type Timestamp struct {{ + Seconds int64 `json:"seconds"` + Microseconds int64 `json:"microseconds"` +}} + +type Event interface {{ + GetName() string + GetTimestamp() Timestamp +}} +func MarshalEvent(e Event) ([]byte, error) {{ + baseStruct := struct {{ + Name string `json:"event"` + EventTimestamp Timestamp `json:"timestamp"` + }}{{ + Name: e.GetName(), + EventTimestamp: e.GetTimestamp(), + }} + base, err := json.Marshal(baseStruct) + if err != nil {{ + return []byte{{}}, err + }} + + dataStruct := struct {{ + Payload Event `json:"data"` + }}{{ + Payload: e, + }} + data, err := json.Marshal(dataStruct) + if err != nil {{ + return []byte{{}}, err + }} + + if len(data) == len(`{{"data":{{}}}}`) {{ + return base, nil + }} + + // Combines Event's base and data in a single JSON object + result := fmt.Sprintf("%s,%s", base[:len(base)-1], data[1:]) + return []byte(result), nil +}} + +func UnmarshalEvent(data []byte) (Event, error) {{ + base := struct {{ + Name string `json:"event"` + EventTimestamp Timestamp `json:"timestamp"` + }}{{}} + if err := json.Unmarshal(data, &base); err != nil {{ + return nil, errors.New(fmt.Sprintf("Failed to decode event: %s", string(data))) + }} + + switch base.Name {{ + {unm_cases} + }} + return nil, errors.New("Failed to recognize event") +}} +''' TEMPLATE_HELPER = ''' // Alias for go version lower than 1.18 type Any = interface{} @@ -53,10 +112,12 @@ class QAPISchemaGenGolangVisitor(QAPISchemaVisitor): def __init__(self, prefix: str): super().__init__() - self.target = {name: "" for name in ["alternate", "enum", "helper", "struct", + self.target = {name: "" for name in ["alternate", "enum", + "event", "helper", "struct", "union"]} self.objects_seen = {} self.schema = None + self.events = {} self.golang_package_name = "qapi" def visit_begin(self, schema): @@ -71,6 +132,24 @@ def visit_begin(self, schema): def visit_end(self): self.schema = None + unm_cases = "" + for name in sorted(self.events): + case_type = self.events[name] + unm_cases += f''' + case "{name}": + event := struct {{ + Data {case_type} `json:"data"` + }}{{}} + + if err := json.Unmarshal(data, &event); err != nil {{ + return nil, errors.New(fmt.Sprintf("Failed to unmarshal: %s", string(data))) + }} + event.Data.EventTimestamp = base.EventTimestamp + return &event.Data, nil +''' + self.target["event"] += TEMPLATE_EVENT.format(unm_cases=unm_cases) + + def visit_object_type(self: QAPISchemaGenGolangVisitor, name: str, info: Optional[QAPISourceInfo], @@ -232,7 +311,37 @@ def visit_command(self, pass def visit_event(self, name, info, ifcond, features, arg_type, boxed): - pass + assert name == info.defn_name + type_name = qapi_to_go_type_name(name, info.defn_meta) + self.events[name] = type_name + + self_contained = True + if arg_type and arg_type.name.startswith("q_obj"): + self_contained = False + + content = "" + if self_contained: + content = generate_struct_type(type_name, '''EventTimestamp Timestamp `json:"-"`''') + else: + assert isinstance(arg_type, QAPISchemaObjectType) + content = qapi_to_golang_struct(name, + arg_type.info, + arg_type.ifcond, + arg_type.features, + arg_type.base, + arg_type.members, + arg_type.variants) + + methods = f''' +func (s *{type_name}) GetName() string {{ + return "{name}" +}} + +func (s *{type_name}) GetTimestamp() Timestamp {{ + return s.EventTimestamp +}} +''' + self.target["event"] += content + methods def write(self, output_dir: str) -> None: for module_name, content in self.target.items(): @@ -274,6 +383,8 @@ def qapi_to_golang_struct(name: str, type_name = qapi_to_go_type_name(name, info.defn_meta) fields = "" + if info.defn_meta == "event": + fields += '''\tEventTimestamp Timestamp `json:"-"`\n''' if base: base_fields = "" @@ -457,4 +568,9 @@ def qapi_to_go_type_name(name: str, meta: str) -> str: name = name.title() name += ''.join(word.title() for word in words[1:]) + + if meta in ["event"]: + name = name[:-3] if name.endswith("Arg") else name + name += meta.title().replace(" ", "") + return name From patchwork Fri Jun 17 12:19:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victor Toso X-Patchwork-Id: 1644840 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ViIar478; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LPdnD5VBYz9s75 for ; Fri, 17 Jun 2022 22:36:00 +1000 (AEST) Received: from localhost ([::1]:59452 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2BCe-00010m-QV for incoming@patchwork.ozlabs.org; Fri, 17 Jun 2022 08:35:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47134) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Ax7-0001hs-Eq for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:58911) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Ax4-00050l-NW for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655468390; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ya/yFxVJyK8+F3qyfRthaOOUOhY7r2uw8VIya/1UxQA=; b=ViIar478482QqOgg1OgzIqskku67JPjcLL3Ka+MHMKmiAk77b6UjeK4iSUWGYt1SlLDYe1 gqzEXQQcBP/8pfPk1sCCGagRmcHulb2DpKS7cwQ3pXorZIS4XTwHy77VGrSt1rTOvG+CzW 04vR0tTad01cRmKP14lYBfUvHRvrNF4= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-189-bQw6AqnrNSOAubOputJQIQ-1; Fri, 17 Jun 2022 08:19:48 -0400 X-MC-Unique: bQw6AqnrNSOAubOputJQIQ-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7E0973C0D85D for ; Fri, 17 Jun 2022 12:19:48 +0000 (UTC) Received: from tapioca.redhat.com (unknown [10.40.192.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id BAB5040334E; Fri, 17 Jun 2022 12:19:46 +0000 (UTC) From: Victor Toso To: qemu-devel@nongnu.org Cc: Eric Blake , Markus Armbruster , John Snow , Andrea Bolognani , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [RFC PATCH v2 6/8] qapi: golang: Generate qapi's command types in Go Date: Fri, 17 Jun 2022 14:19:30 +0200 Message-Id: <20220617121932.249381-7-victortoso@redhat.com> In-Reply-To: <20220617121932.249381-1-victortoso@redhat.com> References: <20220617121932.249381-1-victortoso@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Received-SPF: pass client-ip=170.10.129.124; envelope-from=victortoso@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch handles QAPI command types and generates data structures in Go that decodes from QMP JSON Object to Go data structure and vice versa. Simlar to Event, this patch adds a Command interface and two helper functions MarshalCommand and UnmarshalCommand. At the time of this writing, it generates 209 structures. Example: qapi: | { 'command': 'set_password', | 'boxed': true, | 'data': 'SetPasswordOptions' } go: | type SetPasswordCommand struct { | SetPasswordOptions | CommandId string `json:"-"` | } usage: | input := `{"execute":"set_password",` + | `"arguments":{"protocol":"vnc","password":"secret"}}` | c, err := UnmarshalCommand([]byte(input)) | if err != nil { | panic(err) | } | if c.GetName() == `set_password` { | m := c.(*SetPasswordCommand) | // m.Password == "secret" | } Signed-off-by: Victor Toso --- scripts/qapi/golang.py | 123 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 120 insertions(+), 3 deletions(-) diff --git a/scripts/qapi/golang.py b/scripts/qapi/golang.py index b2e08cebdf..123179cced 100644 --- a/scripts/qapi/golang.py +++ b/scripts/qapi/golang.py @@ -88,6 +88,63 @@ return nil, errors.New("Failed to recognize event") }} ''' + +# Only variable is @unm_cases to handle all command's names and associated types. +TEMPLATE_COMMAND = ''' +type Command interface {{ + GetId() string + GetName() string + GetReturnType() CommandReturn +}} + +func MarshalCommand(c Command) ([]byte, error) {{ + baseStruct := struct {{ + CommandId string `json:"id,omitempty"` + Name string `json:"execute"` + }}{{ + CommandId: c.GetId(), + Name: c.GetName(), + }} + base, err := json.Marshal(baseStruct) + if err != nil {{ + return []byte{{}}, err + }} + + argsStruct := struct {{ + Args Command `json:"arguments,omitempty"` + }}{{ + Args: c, + }} + args, err := json.Marshal(argsStruct) + if err != nil {{ + return []byte{{}}, err + }} + + if len(args) == len(`{{"arguments":{{}}}}`) {{ + return base, nil + }} + + // Combines Event's base and data in a single JSON object + result := fmt.Sprintf("%s,%s", base[:len(base)-1], args[1:]) + return []byte(result), nil +}} + +func UnmarshalCommand(data []byte) (Command, error) {{ + base := struct {{ + CommandId string `json:"id,omitempty"` + Name string `json:"execute"` + }}{{}} + if err := json.Unmarshal(data, &base); err != nil {{ + return nil, errors.New(fmt.Sprintf("Failed to decode command: %s", string(data))) + }} + + switch base.Name {{ + {unm_cases} + }} + return nil, errors.New("Failed to recognize command") +}} +''' + TEMPLATE_HELPER = ''' // Alias for go version lower than 1.18 type Any = interface{} @@ -112,12 +169,13 @@ class QAPISchemaGenGolangVisitor(QAPISchemaVisitor): def __init__(self, prefix: str): super().__init__() - self.target = {name: "" for name in ["alternate", "enum", + self.target = {name: "" for name in ["alternate", "command", "enum", "event", "helper", "struct", "union"]} self.objects_seen = {} self.schema = None self.events = {} + self.commands = {} self.golang_package_name = "qapi" def visit_begin(self, schema): @@ -149,6 +207,23 @@ def visit_end(self): ''' self.target["event"] += TEMPLATE_EVENT.format(unm_cases=unm_cases) + unm_cases = "" + for name in sorted(self.commands): + case_type = self.commands[name] + unm_cases += f''' + case "{name}": + command := struct {{ + Args {case_type} `json:"arguments"` + }}{{}} + + if err := json.Unmarshal(data, &command); err != nil {{ + return nil, errors.New(fmt.Sprintf("Failed to unmarshal: %s", string(data))) + }} + command.Args.CommandId = base.CommandId + return &command.Args, nil +''' + self.target["command"] += TEMPLATE_COMMAND.format(unm_cases=unm_cases) + def visit_object_type(self: QAPISchemaGenGolangVisitor, name: str, @@ -308,7 +383,47 @@ def visit_command(self, allow_oob: bool, allow_preconfig: bool, coroutine: bool) -> None: - pass + # Safety check + assert name == info.defn_name + + type_name = qapi_to_go_type_name(name, info.defn_meta) + self.commands[name] = type_name + command_ret = "" + init_ret_type_name = f'''EmptyCommandReturn {{ Name: "{name}" }}''' + + self_contained = True + if arg_type and arg_type.name.startswith("q_obj"): + self_contained = False + + content = "" + if boxed or self_contained: + args = "" if not arg_type else "\n" + arg_type.name + args += '''\n\tCommandId string `json:"-"`''' + content = generate_struct_type(type_name, args) + else: + assert isinstance(arg_type, QAPISchemaObjectType) + content = qapi_to_golang_struct(name, + arg_type.info, + arg_type.ifcond, + arg_type.features, + arg_type.base, + arg_type.members, + arg_type.variants) + + methods = f''' +func (c *{type_name}) GetName() string {{ + return "{name}" +}} + +func (s *{type_name}) GetId() string {{ + return s.CommandId +}} + +func (s *{type_name}) GetReturnType() CommandReturn {{ + return &{init_ret_type_name} +}} +''' + self.target["command"] += content + methods def visit_event(self, name, info, ifcond, features, arg_type, boxed): assert name == info.defn_name @@ -385,6 +500,8 @@ def qapi_to_golang_struct(name: str, fields = "" if info.defn_meta == "event": fields += '''\tEventTimestamp Timestamp `json:"-"`\n''' + elif info.defn_meta == "command": + fields += '''\tCommandId string `json:"-"`\n''' if base: base_fields = "" @@ -569,7 +686,7 @@ def qapi_to_go_type_name(name: str, meta: str) -> str: name += ''.join(word.title() for word in words[1:]) - if meta in ["event"]: + if meta in ["event", "command"]: name = name[:-3] if name.endswith("Arg") else name name += meta.title().replace(" ", "") From patchwork Fri Jun 17 12:19:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victor Toso X-Patchwork-Id: 1644842 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=I/bbjFZO; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LPdsH3QM5z9s75 for ; Fri, 17 Jun 2022 22:39:31 +1000 (AEST) Received: from localhost ([::1]:37524 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2BG4-0005XO-RY for incoming@patchwork.ozlabs.org; Fri, 17 Jun 2022 08:39:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47190) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2AxA-0001p1-UX for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:57 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:36375) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Ax6-000512-Hr for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655468392; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JZ0vOcnsF21RFn7nIjZPVMrS8wCUCtbccTUvlstCT7Y=; b=I/bbjFZOzLC/gqTynwOv8BMXnYsOhyp3guLMzk2u2XtJeF/aLD4uEWlJ+MwWcvWAApRDtf EDLBBxCV1tH0HMKB9edR27QZ6oPoKNtNKEp82CtgKWkqD5CGB0yA+pm8VO3gnD412+VZeJ SxpF6IIwXik+y38gqAluYecozGZ0M20= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-630-FxUZXtkaPA6tcTgSG_PYPw-1; Fri, 17 Jun 2022 08:19:50 -0400 X-MC-Unique: FxUZXtkaPA6tcTgSG_PYPw-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 89AC71C01B2B for ; Fri, 17 Jun 2022 12:19:50 +0000 (UTC) Received: from tapioca.redhat.com (unknown [10.40.192.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id DE770415F5E; Fri, 17 Jun 2022 12:19:48 +0000 (UTC) From: Victor Toso To: qemu-devel@nongnu.org Cc: Eric Blake , Markus Armbruster , John Snow , Andrea Bolognani , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [RFC PATCH v2 7/8] qapi: golang: Add CommandResult type to Go Date: Fri, 17 Jun 2022 14:19:31 +0200 Message-Id: <20220617121932.249381-8-victortoso@redhat.com> In-Reply-To: <20220617121932.249381-1-victortoso@redhat.com> References: <20220617121932.249381-1-victortoso@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Received-SPF: pass client-ip=170.10.133.124; envelope-from=victortoso@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch adds a struct type in Go that will handle return values for QAPI's command types. The return value of a Command is, encouraged to be, QAPI's complex types or an Array of those. Every Command has a underlying CommandResult. The EmptyCommandReturn is for those that don't expect any data e.g: `{ "return": {} }`. All CommandReturn types implement the CommandResult interface. Example: qapi: | { 'command': 'query-sev', 'returns': 'SevInfo', | 'if': 'TARGET_I386' } go: | type QuerySevCommandReturn struct { | CommandId string `json:"id,omitempty"` | Result *SevInfo `json:"return"` | Error *QapiError `json:"error,omitempty"` | } usage: | // One can use QuerySevCommandReturn directly or | // command's interface GetReturnType() instead. | | input := `{ "return": { "enabled": true, "api-major" : 0,` + | `"api-minor" : 0, "build-id" : 0,` + | `"policy" : 0, "state" : "running",` + | `"handle" : 1 } } ` | ret := QuerySevCommandReturn{} | err := json.Unmarshal([]byte(input), &ret) | if ret.Error != nil { | // Handle command failure {"error": { ...}} | } else if ret.Result != nil { | // ret.Result.Enable == true | } Signed-off-by: Victor Toso --- scripts/qapi/golang.py | 73 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 70 insertions(+), 3 deletions(-) diff --git a/scripts/qapi/golang.py b/scripts/qapi/golang.py index 123179cced..ab91cf124f 100644 --- a/scripts/qapi/golang.py +++ b/scripts/qapi/golang.py @@ -89,7 +89,8 @@ }} ''' -# Only variable is @unm_cases to handle all command's names and associated types. +# Only variable is @unm_cases to handle +# all command's names and associated types. TEMPLATE_COMMAND = ''' type Command interface {{ GetId() string @@ -145,10 +146,49 @@ }} ''' +TEMPLATE_COMMAND_RETURN = ''' +type CommandReturn interface { + GetId() string + GetCommandName() string + GetError() error +} + +type EmptyCommandReturn struct { + CommandId string `json:"id,omitempty"` + Error *QapiError `json:"error,omitempty"` + Name string `json:"-"` +} + +func (r EmptyCommandReturn) MarshalJSON() ([]byte, error) { + return []byte(`{"return":{}}`), nil +} + +func (r *EmptyCommandReturn) GetId() string { + return r.CommandId +} + +func (r *EmptyCommandReturn) GetCommandName() string { + return r.Name +} + +func (r *EmptyCommandReturn) GetError() error { + return r.Error +} +''' + TEMPLATE_HELPER = ''' // Alias for go version lower than 1.18 type Any = interface{} +type QapiError struct { + Class string `json:"class"` + Description string `json:"desc"` +} + +func (err *QapiError) Error() string { + return fmt.Sprintf("%s: %s", err.Class, err.Description) +} + // Creates a decoder that errors on unknown Fields // Returns true if successfully decoded @from string @into type // Returns false without error is failed with "unknown field" @@ -176,6 +216,7 @@ def __init__(self, prefix: str): self.schema = None self.events = {} self.commands = {} + self.command_results = {} self.golang_package_name = "qapi" def visit_begin(self, schema): @@ -224,6 +265,7 @@ def visit_end(self): ''' self.target["command"] += TEMPLATE_COMMAND.format(unm_cases=unm_cases) + self.target["command"] += TEMPLATE_COMMAND_RETURN def visit_object_type(self: QAPISchemaGenGolangVisitor, name: str, @@ -390,6 +432,31 @@ def visit_command(self, self.commands[name] = type_name command_ret = "" init_ret_type_name = f'''EmptyCommandReturn {{ Name: "{name}" }}''' + if ret_type: + cmd_ret_name = qapi_to_go_type_name(name, "command return") + ret_type_name = qapi_schema_type_to_go_type(ret_type.name) + init_ret_type_name = f'''{cmd_ret_name}{{}}''' + isptr = "*" if ret_type_name[0] not in "*[" else "" + self.command_results[name] = ret_type_name + command_ret = f''' +type {cmd_ret_name} struct {{ + CommandId string `json:"id,omitempty"` + Result {isptr}{ret_type_name} `json:"return"` + Error *QapiError `json:"error,omitempty"` +}} + +func (r *{cmd_ret_name}) GetCommandName() string {{ + return "{name}" +}} + +func (r *{cmd_ret_name}) GetId() string {{ + return r.CommandId +}} + +func (r *{cmd_ret_name}) GetError() error {{ + return r.Error +}} +''' self_contained = True if arg_type and arg_type.name.startswith("q_obj"): @@ -423,7 +490,7 @@ def visit_command(self, return &{init_ret_type_name} }} ''' - self.target["command"] += content + methods + self.target["command"] += content + methods + command_ret def visit_event(self, name, info, ifcond, features, arg_type, boxed): assert name == info.defn_name @@ -686,7 +753,7 @@ def qapi_to_go_type_name(name: str, meta: str) -> str: name += ''.join(word.title() for word in words[1:]) - if meta in ["event", "command"]: + if meta in ["event", "command", "command return"]: name = name[:-3] if name.endswith("Arg") else name name += meta.title().replace(" ", "") From patchwork Fri Jun 17 12:19:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victor Toso X-Patchwork-Id: 1644839 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=S57QUulb; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LPdgm2pGBz9s75 for ; Fri, 17 Jun 2022 22:31:15 +1000 (AEST) Received: from localhost ([::1]:52356 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2B82-0004TQ-BI for incoming@patchwork.ozlabs.org; Fri, 17 Jun 2022 08:31:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47188) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2AxA-0001ou-PG for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:57 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:53593) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2Ax9-00051S-2B for qemu-devel@nongnu.org; Fri, 17 Jun 2022 08:19:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655468394; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uSesMzZSrgz2MqihrHWKAVFwsYioX5GmOjoRDVbqhvk=; b=S57QUulb5zyhTSAgulIS/ooOgMCh1N43r/nfZJJ8bSyw8cajAij+iXuhv0xhEFyxcRH6uH DutzsZLT4YLQNhLNyZewPVFrJfKxMq1ttN6ijBAIrMmqLZqSADPcxlACCRv/IQ2CBOrcWQ +Bz00I1yqgpu8xayZNnv9AB3wCSPAVA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-511-6q8t4YW7PJ-1JJwpRrg3gA-1; Fri, 17 Jun 2022 08:19:53 -0400 X-MC-Unique: 6q8t4YW7PJ-1JJwpRrg3gA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B1C7D3804528 for ; Fri, 17 Jun 2022 12:19:52 +0000 (UTC) Received: from tapioca.redhat.com (unknown [10.40.192.236]) by smtp.corp.redhat.com (Postfix) with ESMTP id EA48140334E; Fri, 17 Jun 2022 12:19:50 +0000 (UTC) From: Victor Toso To: qemu-devel@nongnu.org Cc: Eric Blake , Markus Armbruster , John Snow , Andrea Bolognani , =?utf-8?q?Daniel_P_=2E_Berrang=C3=A9?= Subject: [RFC PATCH v2 8/8] qapi: golang: document skip function visit_array_types Date: Fri, 17 Jun 2022 14:19:32 +0200 Message-Id: <20220617121932.249381-9-victortoso@redhat.com> In-Reply-To: <20220617121932.249381-1-victortoso@redhat.com> References: <20220617121932.249381-1-victortoso@redhat.com> MIME-Version: 1.0 Content-type: text/plain X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Received-SPF: pass client-ip=170.10.129.124; envelope-from=victortoso@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Victor Toso --- scripts/qapi/golang.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/scripts/qapi/golang.py b/scripts/qapi/golang.py index ab91cf124f..f37014f52b 100644 --- a/scripts/qapi/golang.py +++ b/scripts/qapi/golang.py @@ -410,7 +410,12 @@ def visit_enum_type(self: QAPISchemaGenGolangVisitor, ''' def visit_array_type(self, name, info, ifcond, element_type): - pass + # TLDR: We don't need to any extra boilerplate in Go to handle Arrays. + # + # This function is implemented just to be sure that: + # 1. Every array type ends with List + # 2. Every array type's element is the array type without 'List' + assert name.endswith("List") and name[:-4] == element_type.name def visit_command(self, name: str,