From patchwork Tue Sep 14 14:19:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 1527972 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=gzz/4aQw; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H85Ck08Zlz9sXk for ; Wed, 15 Sep 2021 00:22:42 +1000 (AEST) Received: from localhost ([::1]:40030 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQ9KZ-00009w-PC for incoming@patchwork.ozlabs.org; Tue, 14 Sep 2021 10:22:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33832) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQ9JG-0007QU-55 for qemu-devel@nongnu.org; Tue, 14 Sep 2021 10:21:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:27359) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQ9JC-000335-Pw for qemu-devel@nongnu.org; Tue, 14 Sep 2021 10:21:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631629274; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=4ulHhpKDegj6iAYcvY7KZF8pg/KtdcW4oTK1Usa8Kg4=; b=gzz/4aQwLykxlvtTlNRBquVWTYflWxGk0P1ZbI6M/DJeKmayVAzOSFgchIVXicyaesD1xw eCKStJqo0zsbZMqV4DymjFHulaazQ/SZxNAk4fSJ3wc5jm9s0spSAhMFnRFOXqIF16CCl+ ox8OASbJbW6Dew+1Z8524pbgJWHNoic= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-37-811swv3jMJy1esG27-Wdkw-1; Tue, 14 Sep 2021 10:21:11 -0400 X-MC-Unique: 811swv3jMJy1esG27-Wdkw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 526AC1084684; Tue, 14 Sep 2021 14:21:07 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.39.193.47]) by smtp.corp.redhat.com (Postfix) with ESMTP id C23645D9DC; Tue, 14 Sep 2021 14:20:43 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Subject: [PATCH v2 00/53] monitor: explicitly permit QMP commands to be added for all use cases Date: Tue, 14 Sep 2021 15:19:49 +0100 Message-Id: <20210914142042.1655100-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=berrange@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.398, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Chris Wulff , David Hildenbrand , Bin Meng , Mark Cave-Ayland , Yuval Shaia , Laurent Vivier , Max Filippov , Taylor Simpson , Alistair Francis , Gerd Hoffmann , "Edgar E. Iglesias" , Eric Blake , Marek Vasut , Yoshinori Sato , Markus Armbruster , Halil Pasic , Christian Borntraeger , Palmer Dabbelt , Artyom Tarasenko , Laurent Vivier , Thomas Huth , Eduardo Habkost , Richard Henderson , Greg Kurz , "Dr. David Alan Gilbert" , qemu-s390x@nongnu.org, qemu-arm@nongnu.org, Michael Rolnik , Peter Xu , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Stafford Horne , David Gibson , qemu-riscv@nongnu.org, Bastian Koppelmann , Cornelia Huck , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , qemu-ppc@nongnu.org, Paolo Bonzini , Aleksandar Rikalo , Aurelien Jarno Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Previous postings: v1: https://lists.gnu.org/archive/html/qemu-devel/2021-09/msg02295.html We are still adding HMP commands without any QMP counterparts. This is done because there are a reasonable number of scenarios where the cost of designing a QAPI data type for the command is not justified. This has the downside, however, that we will never be able to fully isolate the monitor code from the remainder of QEMU internals. It is desirable to be able to get to a point where subsystems in QEMU are exclusively implemented using QAPI types and never need to have any knowledge of the monitor. The way to get there is to stop adding commands to HMP only. All commands must be implemented using QMP and any HMP equivalent be a shim around the QMP implemetation. We don't want to compromise our supportability of QMP long term though. This series proposes that we relax our requirements around fine grained QAPI data design, but with the caveat that any command taking this design approach is mandated to use the 'x-' name prefix. This tradeoff should be suitable for any commands we have been adding exclusively to HMP in recent times, and thus mean we have mandate QMP support for all new commands going forward. The series then converts the following HMP commands to be QMP shims. info opcount info jit info tlb info irq info lapic info cmma info skeys info ramblock info rdma info usb info numa info profile info roms info registers After doing this conversion - All except 1 usage of qemu_fprintf is eliminated - 50% of calls to qemu_printf are eliminated - 75 calls to monitor_printf are eliminated Ultimately it should be possible to entirely eliminate qemu_fprintf and qemu_printf, and confine monitor_printf calls exclusively to the top level HMP command handlers. A full conversion would also enable HMP to be emulated entirely outside QEMU. This could be interesting if we introduce a new QEMU system emulator binary which is legacy free and 100% controlled via QMP, as it would let us provide HMP backcompat around it without the burden of HMP being integrated directly. There are still 48 HMP commands with no QMP counterpart after this series. - A few are not relevant to port as they directly reflect HMP functionality (help, info history). - A few are sort of available in QMP but look quite different (drive_add vs blockdev_add) - A few are complicated. "info usbhost" is a dynamically loaded HMP command inside a loadable module and we don't have a way to dynamically register QMP handlers at runtime. - Most are just tedious gruntwork. Changed in v2: - Improved documentation in response to feedback - Finished "info registers" conversion on all targets - Got a bit carried away and converted many many more commands Daniel P. Berrangé (53): docs/devel: rename file for writing monitor commands docs/devel: tweak headings in monitor command docs docs/devel: document expectations for QAPI data modelling for QMP docs/devel: add example of command returning unstructured text docs/devel: document expectations for HMP commands in the future hw/core: introduce 'format_state' callback to replace 'dump_state' target/alpha: convert to use format_state instead of dump_state target/arm: convert to use format_state instead of dump_state target/avr: convert to use format_state instead of dump_state target/cris: convert to use format_state instead of dump_state target/hexagon: delete unused hexagon_debug() method target/hexagon: convert to use format_state instead of dump_state target/hppa: convert to use format_state instead of dump_state target/i386: convert to use format_state instead of dump_state target/m68k: convert to use format_state instead of dump_state target/microblaze: convert to use format_state instead of dump_state target/mips: convert to use format_state instead of dump_state target/nios2: convert to use format_state instead of dump_state target/openrisc: convert to use format_state instead of dump_state target/ppc: convert to use format_state instead of dump_state target/riscv: convert to use format_state instead of dump_state target/rx: convert to use format_state instead of dump_state target/s390x: convert to use format_state instead of dump_state target/sh: convert to use format_state instead of dump_state target/sparc: convert to use format_state instead of dump_state target/tricore: convert to use format_state instead of dump_state target/xtensa: convert to use format_state instead of dump_state monitor: remove 'info ioapic' HMP command qapi: introduce x-query-registers QMP command qapi: introduce x-query-roms QMP command qapi: introduce x-query-profile QMP command qapi: introduce x-query-numa QMP command qapi: introduce x-query-usb QMP command qapi: introduce x-query-rdma QMP command qapi: introduce x-query-ramblock QMP command qapi: introduce x-query-skeys QMP command qapi: introduce x-query-cmma QMP command qapi: introduce x-query-lapic QMP command qapi: introduce x-query-irq QMP command hw/core: drop "dump_state" callback from CPU targets hw/core: drop support for NULL pointer for FILE * in cpu_dump_state hw/core: introduce a 'format_tlb' callback target/i386: convert to use format_tlb callback target/m68k: convert to use format_tlb callback target/nios2: convert to use format_tlb callback target/ppc: convert to use format_tlb callback target/sh4: convert to use format_tlb callback target/sparc: convert to use format_tlb callback target/xtensa: convert to use format_tlb callback monitor: merge duplicate "info tlb" handlers qapi: introduce x-query-tlb QMP command qapi: introduce x-query-jit QMP command qapi: introduce x-query-opcount QMP command accel/tcg/cpu-exec.c | 56 +- accel/tcg/hmp.c | 24 +- accel/tcg/translate-all.c | 84 +-- docs/devel/index.rst | 2 +- ...mands.rst => writing-monitor-commands.rst} | 136 ++++- hmp-commands-info.hx | 18 - hw/core/cpu-common.c | 27 +- hw/core/loader.c | 55 +- hw/core/machine-hmp-cmds.c | 33 +- hw/core/machine-qmp-cmds.c | 94 ++++ hw/rdma/rdma_rm.c | 104 ++-- hw/rdma/rdma_rm.h | 2 +- hw/rdma/vmw/pvrdma_main.c | 31 +- hw/s390x/s390-skeys.c | 37 +- hw/s390x/s390-stattrib.c | 58 ++- hw/usb/bus.c | 38 +- include/exec/cpu-all.h | 6 +- include/exec/ramlist.h | 2 +- include/hw/core/cpu.h | 36 +- include/hw/rdma/rdma.h | 2 +- include/monitor/hmp-target.h | 2 - include/tcg/tcg.h | 4 +- monitor/hmp-cmds.c | 81 +-- monitor/misc.c | 68 ++- monitor/qmp-cmds.c | 127 +++++ qapi/common.json | 11 + qapi/machine-target.json | 58 +++ qapi/machine.json | 140 +++++ softmmu/physmem.c | 19 +- stubs/usb-dev-stub.c | 8 + target/alpha/cpu.c | 2 +- target/alpha/cpu.h | 2 +- target/alpha/helper.c | 28 +- target/arm/cpu.c | 152 +++--- target/avr/cpu.c | 57 +- target/cris/cpu.c | 2 +- target/cris/cpu.h | 2 +- target/cris/translate.c | 33 +- target/hexagon/cpu.c | 70 ++- target/hexagon/internal.h | 1 - target/hppa/cpu.c | 2 +- target/hppa/cpu.h | 2 +- target/hppa/helper.c | 25 +- target/i386/cpu-dump.c | 489 ++++++++++-------- target/i386/cpu.c | 5 +- target/i386/cpu.h | 7 +- target/i386/monitor.c | 142 ++--- target/m68k/cpu.c | 5 +- target/m68k/cpu.h | 5 +- target/m68k/helper.c | 132 ++--- target/m68k/monitor.c | 14 +- target/m68k/translate.c | 92 ++-- target/microblaze/cpu.c | 2 +- target/microblaze/cpu.h | 2 +- target/microblaze/translate.c | 45 +- target/mips/cpu.c | 85 +-- target/nios2/cpu.c | 5 +- target/nios2/cpu.h | 4 +- target/nios2/mmu.c | 37 +- target/nios2/monitor.c | 7 - target/nios2/translate.c | 20 +- target/openrisc/cpu.c | 2 +- target/openrisc/cpu.h | 2 +- target/openrisc/translate.c | 8 +- target/ppc/cpu.h | 5 +- target/ppc/cpu_init.c | 215 ++++---- target/ppc/mmu-hash64.c | 8 +- target/ppc/mmu-hash64.h | 2 +- target/ppc/mmu_common.c | 167 +++--- target/ppc/monitor.c | 11 - target/riscv/cpu.c | 105 ++-- target/rx/cpu.c | 2 +- target/rx/cpu.h | 2 +- target/rx/translate.c | 14 +- target/s390x/cpu-dump.c | 43 +- target/s390x/cpu.c | 2 +- target/s390x/s390x-internal.h | 2 +- target/sh4/cpu.c | 5 +- target/sh4/cpu.h | 3 +- target/sh4/monitor.c | 41 +- target/sh4/translate.c | 36 +- target/sparc/cpu.c | 86 +-- target/sparc/cpu.h | 3 +- target/sparc/mmu_helper.c | 43 +- target/sparc/monitor.c | 12 - target/tricore/cpu.c | 2 +- target/tricore/cpu.h | 2 +- target/tricore/translate.c | 24 +- target/xtensa/cpu.c | 2 +- target/xtensa/cpu.h | 4 +- target/xtensa/mmu_helper.c | 126 +++-- target/xtensa/monitor.c | 11 - target/xtensa/translate.c | 45 +- tcg/tcg.c | 98 ++-- tests/qtest/qmp-cmd-test.c | 8 + 95 files changed, 2429 insertions(+), 1551 deletions(-) rename docs/devel/{writing-qmp-commands.rst => writing-monitor-commands.rst} (78%)