| Message ID | 20190423212246.3542-1-ehabkost@redhat.com |
|---|---|
| Headers | show |
| Series | Export machine type deprecation info through QMP | expand |
Patchew URL: https://patchew.org/QEMU/20190423212246.3542-1-ehabkost@redhat.com/ Hi, This series seems to have some coding style problems. See output below for more information: Type: series Message-id: 20190423212246.3542-1-ehabkost@redhat.com Subject: [Qemu-devel] [PATCH 0/3] Export machine type deprecation info through QMP === TEST SCRIPT BEGIN === #!/bin/bash git rev-parse base > /dev/null || exit 0 git config --local diff.renamelimit 0 git config --local diff.renames True git config --local diff.algorithm histogram ./scripts/checkpatch.pl --mailback base.. === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 From https://github.com/patchew-project/qemu t [tag update] patchew/20190416125744.27770-1-peter.maydell@linaro.org -> patchew/20190416125744.27770-1-peter.maydell@linaro.org t [tag update] patchew/20190418113110.160664-1-borntraeger@de.ibm.com -> patchew/20190418113110.160664-1-borntraeger@de.ibm.com t [tag update] patchew/20190420073442.7488-1-richard.henderson@linaro.org -> patchew/20190420073442.7488-1-richard.henderson@linaro.org t [tag update] patchew/20190422195020.1494-1-philmd@redhat.com -> patchew/20190422195020.1494-1-philmd@redhat.com t [tag update] patchew/20190422221714.7630-1-jsnow@redhat.com -> patchew/20190422221714.7630-1-jsnow@redhat.com t [tag update] patchew/20190423160608.7519-1-thuth@redhat.com -> patchew/20190423160608.7519-1-thuth@redhat.com * [new tag] patchew/20190423212246.3542-1-ehabkost@redhat.com -> patchew/20190423212246.3542-1-ehabkost@redhat.com Switched to a new branch 'test' 29e2716a80 qmp: Add deprecation information to query-machines 34cb9a3b70 machine: Use SupportStatusInfo for deprecation info b44b7a4718 qapi: SupportStatusInfo struct === OUTPUT BEGIN === 1/3 Checking commit b44b7a47188e (qapi: SupportStatusInfo struct) 2/3 Checking commit 34cb9a3b7083 (machine: Use SupportStatusInfo for deprecation info) 3/3 Checking commit 29e2716a80c8 (qmp: Add deprecation information to query-machines) WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #37: new file mode 100644 ERROR: line over 90 characters #65: FILE: tests/acceptance/query_machines.py:24: + "Deprecated machine (%s) must have status-message or suggested-alternative" % (machine['name'])) ERROR: line over 90 characters #68: FILE: tests/acceptance/query_machines.py:27: + "suggested-alternative of %s must point to a valid machine type" % (machine['name'])) total: 2 errors, 1 warnings, 55 lines checked Patch 3/3 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. === OUTPUT END === Test command exited with code: 1 The full log is available at http://patchew.org/logs/20190423212246.3542-1-ehabkost@redhat.com/testing.checkpatch/?type=message. --- Email generated automatically by Patchew [https://patchew.org/]. Please send your feedback to patchew-devel@redhat.com
On 23/04/2019 23.22, Eduardo Habkost wrote: > This series adds machine type deprecation information to the > output of the `query-machines` QMP command. With this, libvirt > and management software will be able to show this information to > users and/or suggest changes to VM configuration to avoid > deprecated machine types. > > Eduardo Habkost (3): > qapi: SupportStatusInfo struct > machine: Use SupportStatusInfo for deprecation info > qmp: Add deprecation information to query-machines > > qapi/common.json | 24 ++++++++++++++++++++++++ > qapi/misc.json | 5 ++++- > include/hw/boards.h | 7 ++++--- > hw/i386/pc_piix.c | 4 +++- > hw/ppc/prep.c | 4 +++- > vl.c | 19 +++++++++++++++---- > tests/acceptance/query_machines.py | 27 +++++++++++++++++++++++++++ > 7 files changed, 80 insertions(+), 10 deletions(-) > create mode 100644 tests/acceptance/query_machines.py Good idea, but some questions come to my mind: - What about devices? IIRC Gerd wrote a patch series last year that does something similar for devices... It would be good to synchronize the work, so that we do not have two completely interfaces between devices and machines here in the end... - Is deprecation as a status enough, or do we want to carry more information here? E.g. is the machine maintained or orphan? Is it stable or rather experimental? And didn't Gerd have also some patches for this last year? ... yes, I think it was this series here: http://lists.gnu.org/archive/html/qemu-ppc/2018-11/msg00039.html ... actually, I like that idea with QemuSupportState... maybe you could base your work on that series instead? Thomas
On 4/23/19 11:22 PM, Eduardo Habkost wrote: > This series adds machine type deprecation information to the > output of the `query-machines` QMP command. With this, libvirt > and management software will be able to show this information to > users and/or suggest changes to VM configuration to avoid > deprecated machine types. > > Eduardo Habkost (3): > qapi: SupportStatusInfo struct > machine: Use SupportStatusInfo for deprecation info > qmp: Add deprecation information to query-machines > > qapi/common.json | 24 ++++++++++++++++++++++++ > qapi/misc.json | 5 ++++- > include/hw/boards.h | 7 ++++--- > hw/i386/pc_piix.c | 4 +++- > hw/ppc/prep.c | 4 +++- > vl.c | 19 +++++++++++++++---- > tests/acceptance/query_machines.py | 27 +++++++++++++++++++++++++++ > 7 files changed, 80 insertions(+), 10 deletions(-) > create mode 100644 tests/acceptance/query_machines.py > ACK. Now we need to think how to deal with this in libvirt. Whether it's sufficient to expose this in capabilities XML or libvirt needs to do something more. Michal
On Wed, Apr 24, 2019 at 09:56:53AM +0200, Thomas Huth wrote: > On 23/04/2019 23.22, Eduardo Habkost wrote: > > This series adds machine type deprecation information to the > > output of the `query-machines` QMP command. With this, libvirt > > and management software will be able to show this information to > > users and/or suggest changes to VM configuration to avoid > > deprecated machine types. > > > > Eduardo Habkost (3): > > qapi: SupportStatusInfo struct > > machine: Use SupportStatusInfo for deprecation info > > qmp: Add deprecation information to query-machines > > > > qapi/common.json | 24 ++++++++++++++++++++++++ > > qapi/misc.json | 5 ++++- > > include/hw/boards.h | 7 ++++--- > > hw/i386/pc_piix.c | 4 +++- > > hw/ppc/prep.c | 4 +++- > > vl.c | 19 +++++++++++++++---- > > tests/acceptance/query_machines.py | 27 +++++++++++++++++++++++++++ > > 7 files changed, 80 insertions(+), 10 deletions(-) > > create mode 100644 tests/acceptance/query_machines.py > > Good idea, but some questions come to my mind: > > - What about devices? IIRC Gerd wrote a patch series last year that does > something similar for devices... It would be good to synchronize the > work, so that we do not have two completely interfaces between devices > and machines here in the end... Might as well add CPU models to that too as there's plenty of awful CPUs no one should use with KVM. The tricky thing is they are fine / sensible with TCG still. > > - Is deprecation as a status enough, or do we want to carry more > information here? E.g. is the machine maintained or orphan? Is it > stable or rather experimental? And didn't Gerd have also some > patches for this last year? ... yes, I think it was this series here: > http://lists.gnu.org/archive/html/qemu-ppc/2018-11/msg00039.html > ... actually, I like that idea with QemuSupportState... maybe you > could base your work on that series instead? Regards, Daniel
On Wed, Apr 24, 2019 at 09:56:53AM +0200, Thomas Huth wrote: > On 23/04/2019 23.22, Eduardo Habkost wrote: > > This series adds machine type deprecation information to the > > output of the `query-machines` QMP command. With this, libvirt > > and management software will be able to show this information to > > users and/or suggest changes to VM configuration to avoid > > deprecated machine types. > > > > Eduardo Habkost (3): > > qapi: SupportStatusInfo struct > > machine: Use SupportStatusInfo for deprecation info > > qmp: Add deprecation information to query-machines > > > > qapi/common.json | 24 ++++++++++++++++++++++++ > > qapi/misc.json | 5 ++++- > > include/hw/boards.h | 7 ++++--- > > hw/i386/pc_piix.c | 4 +++- > > hw/ppc/prep.c | 4 +++- > > vl.c | 19 +++++++++++++++---- > > tests/acceptance/query_machines.py | 27 +++++++++++++++++++++++++++ > > 7 files changed, 80 insertions(+), 10 deletions(-) > > create mode 100644 tests/acceptance/query_machines.py > > Good idea, but some questions come to my mind: > > - What about devices? IIRC Gerd wrote a patch series last year that does > something similar for devices... It would be good to synchronize the > work, so that we do not have two completely interfaces between devices > and machines here in the end... My plan is to support this on devices, too. I even had a version where documentation of SupportStatusInfo mentioned device types, but I decided to leave that out until we actually implement a device deprecation info API. > > - Is deprecation as a status enough, or do we want to carry more > information here? E.g. is the machine maintained or orphan? Is it > stable or rather experimental? And didn't Gerd have also some > patches for this last year? ... yes, I think it was this series here: > http://lists.gnu.org/archive/html/qemu-ppc/2018-11/msg00039.html > ... actually, I like that idea with QemuSupportState... maybe you > could base your work on that series instead? We might want to carry more information eventually. The possibility of extending the data later is the main reason I called the struct SupportStatusInfo and not just DeprecationInfo. But I'd really like us to export the existing information first, before extending the data. Having important data available through stderr and not QMP is a bug. Tracking additional data may be desirable, but it would be an additional feature.
On Wed, Apr 24, 2019 at 09:31:13AM +0100, Daniel P. Berrangé wrote: > On Wed, Apr 24, 2019 at 09:56:53AM +0200, Thomas Huth wrote: > > On 23/04/2019 23.22, Eduardo Habkost wrote: > > > This series adds machine type deprecation information to the > > > output of the `query-machines` QMP command. With this, libvirt > > > and management software will be able to show this information to > > > users and/or suggest changes to VM configuration to avoid > > > deprecated machine types. > > > > > > Eduardo Habkost (3): > > > qapi: SupportStatusInfo struct > > > machine: Use SupportStatusInfo for deprecation info > > > qmp: Add deprecation information to query-machines > > > > > > qapi/common.json | 24 ++++++++++++++++++++++++ > > > qapi/misc.json | 5 ++++- > > > include/hw/boards.h | 7 ++++--- > > > hw/i386/pc_piix.c | 4 +++- > > > hw/ppc/prep.c | 4 +++- > > > vl.c | 19 +++++++++++++++---- > > > tests/acceptance/query_machines.py | 27 +++++++++++++++++++++++++++ > > > 7 files changed, 80 insertions(+), 10 deletions(-) > > > create mode 100644 tests/acceptance/query_machines.py > > > > Good idea, but some questions come to my mind: > > > > - What about devices? IIRC Gerd wrote a patch series last year that does > > something similar for devices... It would be good to synchronize the > > work, so that we do not have two completely interfaces between devices > > and machines here in the end... > > Might as well add CPU models to that too as there's plenty of awful > CPUs no one should use with KVM. The tricky thing is they are fine > / sensible with TCG still. Right. We also have devices that are supported in some machines but not in others. We will probably need a separate API that takes a (device type, machine type, accelerator) tuple as input.
On 24/04/2019 20.10, Eduardo Habkost wrote: > On Wed, Apr 24, 2019 at 09:56:53AM +0200, Thomas Huth wrote: >> On 23/04/2019 23.22, Eduardo Habkost wrote: >>> This series adds machine type deprecation information to the >>> output of the `query-machines` QMP command. With this, libvirt >>> and management software will be able to show this information to >>> users and/or suggest changes to VM configuration to avoid >>> deprecated machine types. >>> >>> Eduardo Habkost (3): >>> qapi: SupportStatusInfo struct >>> machine: Use SupportStatusInfo for deprecation info >>> qmp: Add deprecation information to query-machines >>> >>> qapi/common.json | 24 ++++++++++++++++++++++++ >>> qapi/misc.json | 5 ++++- >>> include/hw/boards.h | 7 ++++--- >>> hw/i386/pc_piix.c | 4 +++- >>> hw/ppc/prep.c | 4 +++- >>> vl.c | 19 +++++++++++++++---- >>> tests/acceptance/query_machines.py | 27 +++++++++++++++++++++++++++ >>> 7 files changed, 80 insertions(+), 10 deletions(-) >>> create mode 100644 tests/acceptance/query_machines.py >> >> Good idea, but some questions come to my mind: >> >> - What about devices? IIRC Gerd wrote a patch series last year that does >> something similar for devices... It would be good to synchronize the >> work, so that we do not have two completely interfaces between devices >> and machines here in the end... > > My plan is to support this on devices, too. I even had a version > where documentation of SupportStatusInfo mentioned device types, > but I decided to leave that out until we actually implement a > device deprecation info API. > >> >> - Is deprecation as a status enough, or do we want to carry more >> information here? E.g. is the machine maintained or orphan? Is it >> stable or rather experimental? And didn't Gerd have also some >> patches for this last year? ... yes, I think it was this series here: >> http://lists.gnu.org/archive/html/qemu-ppc/2018-11/msg00039.html >> ... actually, I like that idea with QemuSupportState... maybe you >> could base your work on that series instead? > > We might want to carry more information eventually. The > possibility of extending the data later is the main reason I > called the struct SupportStatusInfo and not just DeprecationInfo. Ok. I was just a little bit afraid that we define an interface here that we have to change again completely once we want to carry more information. For example whether "deprecated" should be a "bool" here, or rather one of the "enum" entries like in Gerd's series. But after reading through https://patchwork.kernel.org/patch/10660677/ again, I think I agree that "deprecated" is orthogonal to the support state, e.g. a device could still be supported (in the sense that there is a maintainer for it), while it has been marked as "deprecated" already. So no more objections from my side here. Thomas
On Wed, Apr 24, 2019 at 03:10:49PM -0300, Eduardo Habkost wrote: > On Wed, Apr 24, 2019 at 09:56:53AM +0200, Thomas Huth wrote: > > On 23/04/2019 23.22, Eduardo Habkost wrote: > > > This series adds machine type deprecation information to the > > > output of the `query-machines` QMP command. With this, libvirt > > > and management software will be able to show this information to > > > users and/or suggest changes to VM configuration to avoid > > > deprecated machine types. > > > > > > Eduardo Habkost (3): > > > qapi: SupportStatusInfo struct > > > machine: Use SupportStatusInfo for deprecation info > > > qmp: Add deprecation information to query-machines > > > > > > qapi/common.json | 24 ++++++++++++++++++++++++ > > > qapi/misc.json | 5 ++++- > > > include/hw/boards.h | 7 ++++--- > > > hw/i386/pc_piix.c | 4 +++- > > > hw/ppc/prep.c | 4 +++- > > > vl.c | 19 +++++++++++++++---- > > > tests/acceptance/query_machines.py | 27 +++++++++++++++++++++++++++ > > > 7 files changed, 80 insertions(+), 10 deletions(-) > > > create mode 100644 tests/acceptance/query_machines.py > > > > Good idea, but some questions come to my mind: > > > > - What about devices? IIRC Gerd wrote a patch series last year that does > > something similar for devices... It would be good to synchronize the > > work, so that we do not have two completely interfaces between devices > > and machines here in the end... > > My plan is to support this on devices, too. I even had a version > where documentation of SupportStatusInfo mentioned device types, > but I decided to leave that out until we actually implement a > device deprecation info API. We need it on CPU models, and on CPU features too potentially. Essentially we should consider it applicable to anything we report in QMP. Regards, Daniel
Eduardo Habkost <ehabkost@redhat.com> writes: > This series adds machine type deprecation information to the > output of the `query-machines` QMP command. With this, libvirt > and management software will be able to show this information to > users and/or suggest changes to VM configuration to avoid > deprecated machine types. This overlaps with something I want to try, namely using Kevin's proposed QAPI feature flags for deprecation markings. Let's compare the two. To mark something as deprecated with your patches, you add a @support-status member somewhere, where "somewhere" is related to "something" by "provides information on". Example: MachineInfo (returned by query-machines) provides information on possible values of -machine parameter type. If -machine was QAPIfied, it would provide information on possible values of a QAPI object type's member. The type might be anonymous. The member should be an enum (we currently use 'str' in MachineInfo). Example: say we want to deprecate block driver "vfat", i.e. BlockdevDriver member @vfat. Type BlockdevDriver is used in multiple places; let's ignore all but BlockdevOptions. We need to add @support-status to something that provides information on BlockdevDriver, or maybe on BlockdevOptions. There is no ad hoc query providing information on either of the two, because QAPI/QMP introspection has been sufficient. What now? Can we add deprecation information to (general) QAPI/QMP introspection instead of ad hoc queries? Kevin's proposed QAPI feature flags[*] extend the QAPI language so that struct types can optionally have a list of feature flags, which are strings. Struct types suffice for his immediate needs. I'd like to use feature flags to mark deprecation by tacking a "deprecated" feature onto whatever is deprecated. This obviously needs feature support for everything we want to be able to deprecate: commands, and events, as well as members of enum and object types. Example: to deprecate block driver "vfat", add feature "deprecated" to BlockdevDriver member @vfat. Unlike your patches, this does not require finding a "somewhere" that provides information on "something". You simply tack "deprecated" right onto "something". Your patches provide more information, however: human-readable messages. Food for thought :) [*] Hiding in Subject: [PATCH 0/4] file-posix: Add dynamic-auto-read-only QAPI feature Date: Mon, 8 Apr 2019 16:35:39 +0200 Message-Id: <20190408143543.3982-1-kwolf@redhat.com>
On Tue, May 07, 2019 at 07:07:04AM +0200, Markus Armbruster wrote: > Eduardo Habkost <ehabkost@redhat.com> writes: > > > This series adds machine type deprecation information to the > > output of the `query-machines` QMP command. With this, libvirt > > and management software will be able to show this information to > > users and/or suggest changes to VM configuration to avoid > > deprecated machine types. > > This overlaps with something I want to try, namely using Kevin's > proposed QAPI feature flags for deprecation markings. Let's compare the > two. > > To mark something as deprecated with your patches, you add a > @support-status member somewhere, where "somewhere" is related to > "something" by "provides information on". > > Example: MachineInfo (returned by query-machines) provides information > on possible values of -machine parameter type. If -machine was > QAPIfied, it would provide information on possible values of a QAPI > object type's member. The type might be anonymous. The member should > be an enum (we currently use 'str' in MachineInfo). QAPIfying -machine, -cpu, and -device would be wonderful. > > Example: say we want to deprecate block driver "vfat", > i.e. BlockdevDriver member @vfat. Type BlockdevDriver is used in > multiple places; let's ignore all but BlockdevOptions. We need to add > @support-status to something that provides information on > BlockdevDriver, or maybe on BlockdevOptions. There is no ad hoc query > providing information on either of the two, because QAPI/QMP > introspection has been sufficient. What now? > > Can we add deprecation information to (general) QAPI/QMP introspection Yes, we can. I think it's a good idea. But: > instead of ad hoc queries? I'm not sure about the "instead of" part. I don't want perfect to be the enemy of done, and I don't want QAPIfication of -machine to be a requirement to start reporting machine type deprecation information. > > Kevin's proposed QAPI feature flags[*] extend the QAPI language so that > struct types can optionally have a list of feature flags, which are > strings. Struct types suffice for his immediate needs. I'd like to use > feature flags to mark deprecation by tacking a "deprecated" feature onto > whatever is deprecated. This obviously needs feature support for > everything we want to be able to deprecate: commands, and events, as > well as members of enum and object types. > > Example: to deprecate block driver "vfat", add feature "deprecated" to > BlockdevDriver member @vfat. > > Unlike your patches, this does not require finding a "somewhere" that > provides information on "something". You simply tack "deprecated" right > onto "something". > > Your patches provide more information, however: human-readable messages. It also includes a machine-friendly suggested alternative (which I think is even more important that the human-readable message). We could extend QAPI introspection to return that if necessary, right? > > Food for thought :) > > > [*] Hiding in > Subject: [PATCH 0/4] file-posix: Add dynamic-auto-read-only QAPI feature > Date: Mon, 8 Apr 2019 16:35:39 +0200 > Message-Id: <20190408143543.3982-1-kwolf@redhat.com>
Eduardo Habkost <ehabkost@redhat.com> writes: > On Tue, May 07, 2019 at 07:07:04AM +0200, Markus Armbruster wrote: >> Eduardo Habkost <ehabkost@redhat.com> writes: >> >> > This series adds machine type deprecation information to the >> > output of the `query-machines` QMP command. With this, libvirt >> > and management software will be able to show this information to >> > users and/or suggest changes to VM configuration to avoid >> > deprecated machine types. >> >> This overlaps with something I want to try, namely using Kevin's >> proposed QAPI feature flags for deprecation markings. Let's compare the >> two. >> >> To mark something as deprecated with your patches, you add a >> @support-status member somewhere, where "somewhere" is related to >> "something" by "provides information on". >> >> Example: MachineInfo (returned by query-machines) provides information >> on possible values of -machine parameter type. If -machine was >> QAPIfied, it would provide information on possible values of a QAPI >> object type's member. The type might be anonymous. The member should >> be an enum (we currently use 'str' in MachineInfo). > > QAPIfying -machine, -cpu, and -device would be wonderful. > >> >> Example: say we want to deprecate block driver "vfat", >> i.e. BlockdevDriver member @vfat. Type BlockdevDriver is used in >> multiple places; let's ignore all but BlockdevOptions. We need to add >> @support-status to something that provides information on >> BlockdevDriver, or maybe on BlockdevOptions. There is no ad hoc query >> providing information on either of the two, because QAPI/QMP >> introspection has been sufficient. What now? >> >> Can we add deprecation information to (general) QAPI/QMP introspection > > Yes, we can. I think it's a good idea. But: > >> instead of ad hoc queries? > > I'm not sure about the "instead of" part. I don't want perfect > to be the enemy of done, and I don't want QAPIfication of > -machine to be a requirement to start reporting machine type > deprecation information. Valid point. Still, I believe we should at least try to predict how the pieces we create now would fit with the pieces we plan to create later on. Note that full QAPIfication of -machine isn't necessary to make QAPI feature "deprecated" work for machine types. Turning MachineInfo member @name into an enum, so we can tack "deprecated" onto its values, would suffice. Such a QAPIfication of machine types is still hard: QOM types are defined at compile time just like the QAPI schema, but their definition is distributed, and collected into one place only at run time. I discussed this on slide 39 of my "QEMU interface introspection: From hacks to solutions" talk (KVM Form 2015). Just for device_add, but it's just a special case of QOM. Choices listed there: * Collect drivers at compile time? Hard... * Make QAPI schema dynamic? Hard... * Forgo driver-specific arguments in schema? Defeats introspection... I'd like to add to the last item: Provide QOM introspection on par with QAPI schema introspection The QOM introspection we have (qom-list-types etc. is not on par. Back to exposing machine type deprecation. I'm doubtful your proposed solution can be applied widely. It relies on adding @support-status to something that provides information on whatever is deprecated. The initial use is with a something that is an ad hoc query, namely query-machines. To use it, the management application needs to understand what query-machines' @support-status applies to. Certainly feasible. But I fear every use will be a special case. Furthermore, a suitable ad hoc query need not exist. What then? Create suitable ad hoc queries just for communicating deprecation? Instead, I'd like us to think about a more genral solution. Or perhaps two: one for properly QAPIfied stuff, and one for QOM. >> Kevin's proposed QAPI feature flags[*] extend the QAPI language so that >> struct types can optionally have a list of feature flags, which are >> strings. Struct types suffice for his immediate needs. I'd like to use >> feature flags to mark deprecation by tacking a "deprecated" feature onto >> whatever is deprecated. This obviously needs feature support for >> everything we want to be able to deprecate: commands, and events, as >> well as members of enum and object types. >> >> Example: to deprecate block driver "vfat", add feature "deprecated" to >> BlockdevDriver member @vfat. >> >> Unlike your patches, this does not require finding a "somewhere" that >> provides information on "something". You simply tack "deprecated" right >> onto "something". >> >> Your patches provide more information, however: human-readable messages. > > It also includes a machine-friendly suggested alternative (which > I think is even more important that the human-readable message). I agree we should point to a preferred replacement whenever we deprecate something. We have to do it in documentation. And we generally do, in qemu-deprecated.texi. How useful would doing it in QMP as well be? Depends on what management applications can do with the additional information. > We could extend QAPI introspection to return that if necessary, > right? I'm confident we can come up with *something*. It might kill the neat and simple "use QAPI features to communicate deprecation" idea, though. >> Food for thought :) >> >> >> [*] Hiding in >> Subject: [PATCH 0/4] file-posix: Add dynamic-auto-read-only QAPI feature >> Date: Mon, 8 Apr 2019 16:35:39 +0200 >> Message-Id: <20190408143543.3982-1-kwolf@redhat.com>
On Wed, May 08, 2019 at 11:16:50AM +0200, Markus Armbruster wrote: > Eduardo Habkost <ehabkost@redhat.com> writes: > > > On Tue, May 07, 2019 at 07:07:04AM +0200, Markus Armbruster wrote: > >> Eduardo Habkost <ehabkost@redhat.com> writes: > >> > >> > This series adds machine type deprecation information to the > >> > output of the `query-machines` QMP command. With this, libvirt > >> > and management software will be able to show this information to > >> > users and/or suggest changes to VM configuration to avoid > >> > deprecated machine types. > >> > >> This overlaps with something I want to try, namely using Kevin's > >> proposed QAPI feature flags for deprecation markings. Let's compare the > >> two. > >> > >> To mark something as deprecated with your patches, you add a > >> @support-status member somewhere, where "somewhere" is related to > >> "something" by "provides information on". > >> > >> Example: MachineInfo (returned by query-machines) provides information > >> on possible values of -machine parameter type. If -machine was > >> QAPIfied, it would provide information on possible values of a QAPI > >> object type's member. The type might be anonymous. The member should > >> be an enum (we currently use 'str' in MachineInfo). > > > > QAPIfying -machine, -cpu, and -device would be wonderful. > > > >> > >> Example: say we want to deprecate block driver "vfat", > >> i.e. BlockdevDriver member @vfat. Type BlockdevDriver is used in > >> multiple places; let's ignore all but BlockdevOptions. We need to add > >> @support-status to something that provides information on > >> BlockdevDriver, or maybe on BlockdevOptions. There is no ad hoc query > >> providing information on either of the two, because QAPI/QMP > >> introspection has been sufficient. What now? > >> > >> Can we add deprecation information to (general) QAPI/QMP introspection > > > > Yes, we can. I think it's a good idea. But: > > > >> instead of ad hoc queries? > > > > I'm not sure about the "instead of" part. I don't want perfect > > to be the enemy of done, and I don't want QAPIfication of > > -machine to be a requirement to start reporting machine type > > deprecation information. > > Valid point. Still, I believe we should at least try to predict how the > pieces we create now would fit with the pieces we plan to create later > on. Sure. > > Note that full QAPIfication of -machine isn't necessary to make QAPI > feature "deprecated" work for machine types. Turning MachineInfo member > @name into an enum, so we can tack "deprecated" onto its values, would > suffice. > > Such a QAPIfication of machine types is still hard: QOM types are > defined at compile time just like the QAPI schema, but their definition > is distributed, and collected into one place only at run time. I > discussed this on slide 39 of my "QEMU interface introspection: From > hacks to solutions" talk (KVM Form 2015). Just for device_add, but it's > just a special case of QOM. Choices listed there: > > * Collect drivers at compile time? Hard... > * Make QAPI schema dynamic? Hard... > * Forgo driver-specific arguments in schema? > Defeats introspection... > > I'd like to add to the last item: > > Provide QOM introspection on par with QAPI schema introspection > > The QOM introspection we have (qom-list-types etc. is not on par. Agreed, but do we really want to do it? We have been avoiding exposing QOM internals to the outside on purpose. I believe there are at least two reasons for that: 1) Not every QOM type/property is supposed to be visible to the outside (and nobody really knows what's the full set of supported external QOM interfaces); 2) QAPI is our preferred interface interface specification/introspection mechanism. > > Back to exposing machine type deprecation. > > I'm doubtful your proposed solution can be applied widely. It relies on > adding @support-status to something that provides information on > whatever is deprecated. The initial use is with a something that is an > ad hoc query, namely query-machines. To use it, the management > application needs to understand what query-machines' @support-status > applies to. Certainly feasible. But I fear every use will be a special > case. Furthermore, a suitable ad hoc query need not exist. What then? > Create suitable ad hoc queries just for communicating deprecation? > > Instead, I'd like us to think about a more genral solution. Or perhaps > two: one for properly QAPIfied stuff, and one for QOM. Should we really spend our time designing a second solution, if we could build this on top of QAPI abstractions? Making the QAPI schema dynamic might be hard, but reinventing QAPI and maintaining the two systems in parallel seems harder. > > >> Kevin's proposed QAPI feature flags[*] extend the QAPI language so that > >> struct types can optionally have a list of feature flags, which are > >> strings. Struct types suffice for his immediate needs. I'd like to use > >> feature flags to mark deprecation by tacking a "deprecated" feature onto > >> whatever is deprecated. This obviously needs feature support for > >> everything we want to be able to deprecate: commands, and events, as > >> well as members of enum and object types. > >> > >> Example: to deprecate block driver "vfat", add feature "deprecated" to > >> BlockdevDriver member @vfat. > >> > >> Unlike your patches, this does not require finding a "somewhere" that > >> provides information on "something". You simply tack "deprecated" right > >> onto "something". > >> > >> Your patches provide more information, however: human-readable messages. > > > > It also includes a machine-friendly suggested alternative (which > > I think is even more important that the human-readable message). > > I agree we should point to a preferred replacement whenever we deprecate > something. > > We have to do it in documentation. And we generally do, in > qemu-deprecated.texi. > > How useful would doing it in QMP as well be? Depends on what management > applications can do with the additional information. I expect it to be useful for things that have obvious replacements, like old machine type or CPU model versions. > > > We could extend QAPI introspection to return that if necessary, > > right? > > I'm confident we can come up with *something*. It might kill the neat > and simple "use QAPI features to communicate deprecation" idea, though. > If something is important enough to be communicated through stderr, it's important enough to be communicated through QMP. Is that enough reason to provide something more complex? Do we need QAPI features to be just strings? Can't they be a more complex type, like a QAPI alternate? > >> Food for thought :) > >> > >> > >> [*] Hiding in > >> Subject: [PATCH 0/4] file-posix: Add dynamic-auto-read-only QAPI feature > >> Date: Mon, 8 Apr 2019 16:35:39 +0200 > >> Message-Id: <20190408143543.3982-1-kwolf@redhat.com>
We've wandered into the QAPI vs. QOM swamp. Cc: Paolo. Eduardo Habkost <ehabkost@redhat.com> writes: > On Wed, May 08, 2019 at 11:16:50AM +0200, Markus Armbruster wrote: >> Eduardo Habkost <ehabkost@redhat.com> writes: >> >> > On Tue, May 07, 2019 at 07:07:04AM +0200, Markus Armbruster wrote: >> >> Eduardo Habkost <ehabkost@redhat.com> writes: >> >> >> >> > This series adds machine type deprecation information to the >> >> > output of the `query-machines` QMP command. With this, libvirt >> >> > and management software will be able to show this information to >> >> > users and/or suggest changes to VM configuration to avoid >> >> > deprecated machine types. >> >> >> >> This overlaps with something I want to try, namely using Kevin's >> >> proposed QAPI feature flags for deprecation markings. Let's compare the >> >> two. >> >> >> >> To mark something as deprecated with your patches, you add a >> >> @support-status member somewhere, where "somewhere" is related to >> >> "something" by "provides information on". >> >> >> >> Example: MachineInfo (returned by query-machines) provides information >> >> on possible values of -machine parameter type. If -machine was >> >> QAPIfied, it would provide information on possible values of a QAPI >> >> object type's member. The type might be anonymous. The member should >> >> be an enum (we currently use 'str' in MachineInfo). >> > >> > QAPIfying -machine, -cpu, and -device would be wonderful. >> > >> >> >> >> Example: say we want to deprecate block driver "vfat", >> >> i.e. BlockdevDriver member @vfat. Type BlockdevDriver is used in >> >> multiple places; let's ignore all but BlockdevOptions. We need to add >> >> @support-status to something that provides information on >> >> BlockdevDriver, or maybe on BlockdevOptions. There is no ad hoc query >> >> providing information on either of the two, because QAPI/QMP >> >> introspection has been sufficient. What now? >> >> >> >> Can we add deprecation information to (general) QAPI/QMP introspection >> > >> > Yes, we can. I think it's a good idea. But: >> > >> >> instead of ad hoc queries? >> > >> > I'm not sure about the "instead of" part. I don't want perfect >> > to be the enemy of done, and I don't want QAPIfication of >> > -machine to be a requirement to start reporting machine type >> > deprecation information. >> >> Valid point. Still, I believe we should at least try to predict how the >> pieces we create now would fit with the pieces we plan to create later >> on. > > Sure. > >> >> Note that full QAPIfication of -machine isn't necessary to make QAPI >> feature "deprecated" work for machine types. Turning MachineInfo member >> @name into an enum, so we can tack "deprecated" onto its values, would >> suffice. >> >> Such a QAPIfication of machine types is still hard: QOM types are >> defined at compile time just like the QAPI schema, but their definition >> is distributed, and collected into one place only at run time. I >> discussed this on slide 39 of my "QEMU interface introspection: From >> hacks to solutions" talk (KVM Form 2015). Just for device_add, but it's >> just a special case of QOM. Choices listed there: >> >> * Collect drivers at compile time? Hard... >> * Make QAPI schema dynamic? Hard... >> * Forgo driver-specific arguments in schema? >> Defeats introspection... >> >> I'd like to add to the last item: >> >> Provide QOM introspection on par with QAPI schema introspection >> >> The QOM introspection we have (qom-list-types etc. is not on par. > > Agreed, but do we really want to do it? We have been avoiding > exposing QOM internals to the outside on purpose. I believe > there are at least two reasons for that: > > 1) Not every QOM type/property is supposed to be visible to the > outside True. However, the parts of QOM exposed via device_add and object-add are definitely part of the stable external interface (unless explicitly marked unstable). > (and nobody really knows what's the full set of > supported external QOM interfaces); Also true. And terrible. > 2) QAPI is our preferred interface interface specification/introspection > mechanism. When preferences and requirements collide, preferences tend to get run over. The QAPI schema is *declarative*: the schema declares QAPI objects and properties. We generate C from the schema, which we then compile and link into QEMU. QOM is by design *imperative*: we execute compiled C at QEMU run-time to define QOM objects and properties. Maximizes flexibility. See also Turing tarpit. No matter how much we'd prefer to use QAPI to specify external interfaces to QOM, we can't without making QAPI much more dynamic or QOM much more static. Either is hard. Quite possibly infeasible. We could try to extend QAPI/QMP introspection to somehow merge in additional information at run-time[1]. Could be regarded as a limited way to make QAPI more dynamic. This is in the "vague idea, not sure it's feasible" stage. We could try to rearchitect QOM so that you can optionally specify QOM stuff in the QAPI schema, then require that for QOM stuff that's part of a stable external interface. Same "vague idea" stage, feasibility even more doubtful. If I remember correctly, Paolo suggested to abandon the idea to have query-qmp-schema cover device_add as infeasible. Instead, cover it separately. I think this separate introspection mechanism should be as similar to query-qmp-schema as practical. It should cover not just device_add, but also object-add, and whatever other external interface to QOM we deem stable. Should there be more stable external interfaces that can't be defined in the QAPI schema, we might want to cover them in this second introspection mechanism, too. >> Back to exposing machine type deprecation. >> >> I'm doubtful your proposed solution can be applied widely. It relies on >> adding @support-status to something that provides information on >> whatever is deprecated. The initial use is with a something that is an >> ad hoc query, namely query-machines. To use it, the management >> application needs to understand what query-machines' @support-status >> applies to. Certainly feasible. But I fear every use will be a special >> case. Furthermore, a suitable ad hoc query need not exist. What then? >> Create suitable ad hoc queries just for communicating deprecation? >> >> Instead, I'd like us to think about a more genral solution. Or perhaps >> two: one for properly QAPIfied stuff, and one for QOM. > > Should we really spend our time designing a second solution, if > we could build this on top of QAPI abstractions? Making the QAPI > schema dynamic might be hard, but reinventing QAPI and > maintaining the two systems in parallel seems harder. I'd love to see a practical way forward that keeps QAPI/QMP introspection the only game in town. >> >> Kevin's proposed QAPI feature flags[*] extend the QAPI language so that >> >> struct types can optionally have a list of feature flags, which are >> >> strings. Struct types suffice for his immediate needs. I'd like to use >> >> feature flags to mark deprecation by tacking a "deprecated" feature onto >> >> whatever is deprecated. This obviously needs feature support for >> >> everything we want to be able to deprecate: commands, and events, as >> >> well as members of enum and object types. >> >> >> >> Example: to deprecate block driver "vfat", add feature "deprecated" to >> >> BlockdevDriver member @vfat. >> >> >> >> Unlike your patches, this does not require finding a "somewhere" that >> >> provides information on "something". You simply tack "deprecated" right >> >> onto "something". >> >> >> >> Your patches provide more information, however: human-readable messages. >> > >> > It also includes a machine-friendly suggested alternative (which >> > I think is even more important that the human-readable message). >> >> I agree we should point to a preferred replacement whenever we deprecate >> something. >> >> We have to do it in documentation. And we generally do, in >> qemu-deprecated.texi. >> >> How useful would doing it in QMP as well be? Depends on what management >> applications can do with the additional information. > > I expect it to be useful for things that have obvious > replacements, like old machine type or CPU model versions. I doubt a management application should apply suggested replacements automatically, and I doubt libvirt would. Not even when QEMU developers deem them "obvious". >> > We could extend QAPI introspection to return that if necessary, >> > right? >> >> I'm confident we can come up with *something*. It might kill the neat >> and simple "use QAPI features to communicate deprecation" idea, though. > > If something is important enough to be communicated through > stderr, it's important enough to be communicated through QMP. Mostly. Differences are due to the different consumers. stderr is primarily for human users. We print stuff useful to human users. QMP is primarily for machines, secondarily for the humans building these machines. We send stuff useful to the machines themselves, and stuff the machines can use to be more useful for their users (which may be machines or humans). We can also send stuff to help the humans building the machines. In any case, the information we provide is limited by the cost to provide it. > Is that enough reason to provide something more complex? We need to consider cost / benefit. On benefit, I'd like to know what libvirt would do with the additional information beyond logging it. Is the additional information you propose to provide static or dynamic? By "static", I mean each occurence of a feature in the QAPI schema is tied to one fixed instance of "additional information". > Do we need QAPI features to be just strings? Can't they be a > more complex type, like a QAPI alternate? Adds complexity. We currently imagine QAPI features enum-like, i.e. a list of strings, optionally with conditions. The conditions are evaluated at QAPI generation time, and not visible in introspection. This is probably the stupidest solution that could possibly work. The structure of features is trivial. More expressive solutions include: * List of 'any'. Feels like a bad idea, because it's completely unstructured. * List of some 'QapiFeatures' object type. Lets us expose the variable structure of features in introspection. * List of some 'QapiFeatures' alternate type. Like the previous, but permits coding simple feature flags as strings. All of the more expressive solutions I listed treat the additional information as dynamic. Overly general in case the information is actually always static. >> >> Food for thought :) >> >> >> >> >> >> [*] Hiding in >> >> Subject: [PATCH 0/4] file-posix: Add dynamic-auto-read-only QAPI feature >> >> Date: Mon, 8 Apr 2019 16:35:39 +0200 >> >> Message-Id: <20190408143543.3982-1-kwolf@redhat.com> [1] Special case of "merge additional information into QAPI/QMP introspection at run-time, relegated to a footnote, because I think it's of mostly historical interest now. Qdev was designed to be declarative, but the declarations always got collected only at run-time. It has since been rebased onto imperative QOM. Since you can mix declarative qdev and imperative QOM, the combination is effectively imperative. We could collect the declarative parts of qdev at compile-time somehow, and fold them into QAPI introspection. That's the "Collect drives at compile time" item from the slide I quoted. I think this idea has become pretty much worthless, since it would cover device_add only in part, and object-add not at all.
On Thu, May 09, 2019 at 10:31:46AM +0200, Markus Armbruster wrote: > We've wandered into the QAPI vs. QOM swamp. Cc: Paolo. > > Eduardo Habkost <ehabkost@redhat.com> writes: > > > On Wed, May 08, 2019 at 11:16:50AM +0200, Markus Armbruster wrote: > >> Eduardo Habkost <ehabkost@redhat.com> writes: > >> > >> > On Tue, May 07, 2019 at 07:07:04AM +0200, Markus Armbruster wrote: > >> >> Eduardo Habkost <ehabkost@redhat.com> writes: > >> >> > >> >> > This series adds machine type deprecation information to the > >> >> > output of the `query-machines` QMP command. With this, libvirt > >> >> > and management software will be able to show this information to > >> >> > users and/or suggest changes to VM configuration to avoid > >> >> > deprecated machine types. > >> >> > >> >> This overlaps with something I want to try, namely using Kevin's > >> >> proposed QAPI feature flags for deprecation markings. Let's compare the > >> >> two. > >> >> > >> >> To mark something as deprecated with your patches, you add a > >> >> @support-status member somewhere, where "somewhere" is related to > >> >> "something" by "provides information on". > >> >> > >> >> Example: MachineInfo (returned by query-machines) provides information > >> >> on possible values of -machine parameter type. If -machine was > >> >> QAPIfied, it would provide information on possible values of a QAPI > >> >> object type's member. The type might be anonymous. The member should > >> >> be an enum (we currently use 'str' in MachineInfo). > >> > > >> > QAPIfying -machine, -cpu, and -device would be wonderful. > >> > > >> >> > >> >> Example: say we want to deprecate block driver "vfat", > >> >> i.e. BlockdevDriver member @vfat. Type BlockdevDriver is used in > >> >> multiple places; let's ignore all but BlockdevOptions. We need to add > >> >> @support-status to something that provides information on > >> >> BlockdevDriver, or maybe on BlockdevOptions. There is no ad hoc query > >> >> providing information on either of the two, because QAPI/QMP > >> >> introspection has been sufficient. What now? > >> >> > >> >> Can we add deprecation information to (general) QAPI/QMP introspection > >> > > >> > Yes, we can. I think it's a good idea. But: > >> > > >> >> instead of ad hoc queries? > >> > > >> > I'm not sure about the "instead of" part. I don't want perfect > >> > to be the enemy of done, and I don't want QAPIfication of > >> > -machine to be a requirement to start reporting machine type > >> > deprecation information. > >> > >> Valid point. Still, I believe we should at least try to predict how the > >> pieces we create now would fit with the pieces we plan to create later > >> on. > > > > Sure. > > > >> > >> Note that full QAPIfication of -machine isn't necessary to make QAPI > >> feature "deprecated" work for machine types. Turning MachineInfo member > >> @name into an enum, so we can tack "deprecated" onto its values, would > >> suffice. > >> > >> Such a QAPIfication of machine types is still hard: QOM types are > >> defined at compile time just like the QAPI schema, but their definition > >> is distributed, and collected into one place only at run time. I > >> discussed this on slide 39 of my "QEMU interface introspection: From > >> hacks to solutions" talk (KVM Form 2015). Just for device_add, but it's > >> just a special case of QOM. Choices listed there: > >> > >> * Collect drivers at compile time? Hard... > >> * Make QAPI schema dynamic? Hard... > >> * Forgo driver-specific arguments in schema? > >> Defeats introspection... > >> > >> I'd like to add to the last item: > >> > >> Provide QOM introspection on par with QAPI schema introspection > >> > >> The QOM introspection we have (qom-list-types etc. is not on par. > > > > Agreed, but do we really want to do it? We have been avoiding > > exposing QOM internals to the outside on purpose. I believe > > there are at least two reasons for that: > > > > 1) Not every QOM type/property is supposed to be visible to the > > outside > > True. > > However, the parts of QOM exposed via device_add and object-add are > definitely part of the stable external interface (unless explicitly > marked unstable). > > > (and nobody really knows what's the full set of > > supported external QOM interfaces); > > Also true. And terrible. > > > 2) QAPI is our preferred interface interface specification/introspection > > mechanism. > > When preferences and requirements collide, preferences tend to get run > over. > > The QAPI schema is *declarative*: the schema declares QAPI objects and > properties. We generate C from the schema, which we then compile and > link into QEMU. > > QOM is by design *imperative*: we execute compiled C at QEMU run-time to > define QOM objects and properties. Maximizes flexibility. See also > Turing tarpit. > > No matter how much we'd prefer to use QAPI to specify external > interfaces to QOM, we can't without making QAPI much more dynamic or QOM > much more static. Either is hard. Quite possibly infeasible. > > We could try to extend QAPI/QMP introspection to somehow merge in > additional information at run-time[1]. Could be regarded as a limited > way to make QAPI more dynamic. This is in the "vague idea, not sure > it's feasible" stage. > > We could try to rearchitect QOM so that you can optionally specify QOM > stuff in the QAPI schema, then require that for QOM stuff that's part of > a stable external interface. Same "vague idea" stage, feasibility even > more doubtful. I think it is feasible to use QAPI to declare the QOM object type, QOM parent type, list of QOM interfaces and QOM properties in a declarative manner. From that generate all the tedious boilerplate code. The dev then just has to provide the "interesting" code for the object. I'd really like to do a PoC of this but never have free time :-( Perhaps an interesting task for someone who wants to delve into some new work... > >> >> Example: to deprecate block driver "vfat", add feature "deprecated" to > >> >> BlockdevDriver member @vfat. > >> >> > >> >> Unlike your patches, this does not require finding a "somewhere" that > >> >> provides information on "something". You simply tack "deprecated" right > >> >> onto "something". > >> >> > >> >> Your patches provide more information, however: human-readable messages. > >> > > >> > It also includes a machine-friendly suggested alternative (which > >> > I think is even more important that the human-readable message). > >> > >> I agree we should point to a preferred replacement whenever we deprecate > >> something. > >> > >> We have to do it in documentation. And we generally do, in > >> qemu-deprecated.texi. > >> > >> How useful would doing it in QMP as well be? Depends on what management > >> applications can do with the additional information. > > > > I expect it to be useful for things that have obvious > > replacements, like old machine type or CPU model versions. > > I doubt a management application should apply suggested replacements > automatically, and I doubt libvirt would. Not even when QEMU developers > deem them "obvious". We certainly won't apply the suggested replacement as in many cases it is not going to be a functionally equivalent drop-in. If QEMU logs it to stderr, it will end up in the per-VM log file libvirt has under /var/log/libvirt/qemu/$GUESTNAME.log. If QEMU doesn't log it to stderr, then libvirt would just write it to that same log file itself. If libvirt gains some API or event for notifying apps of deprecation we might bubble it up to the mgmt app that way. I still feel it is useful to have the suggested replacement in the logs, rather than only leaving it in qemu-deprecated.texi. This way the info is immediately visible to both app developers and any support person dealing with bugs. If the app dev see the suggested replacement upfront they're more likely to make an immediate decision to update their code if the suggestion is trivial. If they need to go find the QEMU docs to lookup what action is required I feel they'll more likely just put the item on their long todo list where it will languish. Regards, Daniel
On Thu, May 09, 2019 at 10:14:52AM +0100, Daniel P. Berrangé wrote: > On Thu, May 09, 2019 at 10:31:46AM +0200, Markus Armbruster wrote: > > We've wandered into the QAPI vs. QOM swamp. Cc: Paolo. > > > > Eduardo Habkost <ehabkost@redhat.com> writes: > > > > > On Wed, May 08, 2019 at 11:16:50AM +0200, Markus Armbruster wrote: > > >> Eduardo Habkost <ehabkost@redhat.com> writes: > > >> > > >> > On Tue, May 07, 2019 at 07:07:04AM +0200, Markus Armbruster wrote: > > >> >> Eduardo Habkost <ehabkost@redhat.com> writes: > > >> >> > > >> >> > This series adds machine type deprecation information to the > > >> >> > output of the `query-machines` QMP command. With this, libvirt > > >> >> > and management software will be able to show this information to > > >> >> > users and/or suggest changes to VM configuration to avoid > > >> >> > deprecated machine types. > > >> >> > > >> >> This overlaps with something I want to try, namely using Kevin's > > >> >> proposed QAPI feature flags for deprecation markings. Let's compare the > > >> >> two. > > >> >> > > >> >> To mark something as deprecated with your patches, you add a > > >> >> @support-status member somewhere, where "somewhere" is related to > > >> >> "something" by "provides information on". > > >> >> > > >> >> Example: MachineInfo (returned by query-machines) provides information > > >> >> on possible values of -machine parameter type. If -machine was > > >> >> QAPIfied, it would provide information on possible values of a QAPI > > >> >> object type's member. The type might be anonymous. The member should > > >> >> be an enum (we currently use 'str' in MachineInfo). > > >> > > > >> > QAPIfying -machine, -cpu, and -device would be wonderful. > > >> > > > >> >> > > >> >> Example: say we want to deprecate block driver "vfat", > > >> >> i.e. BlockdevDriver member @vfat. Type BlockdevDriver is used in > > >> >> multiple places; let's ignore all but BlockdevOptions. We need to add > > >> >> @support-status to something that provides information on > > >> >> BlockdevDriver, or maybe on BlockdevOptions. There is no ad hoc query > > >> >> providing information on either of the two, because QAPI/QMP > > >> >> introspection has been sufficient. What now? > > >> >> > > >> >> Can we add deprecation information to (general) QAPI/QMP introspection > > >> > > > >> > Yes, we can. I think it's a good idea. But: > > >> > > > >> >> instead of ad hoc queries? > > >> > > > >> > I'm not sure about the "instead of" part. I don't want perfect > > >> > to be the enemy of done, and I don't want QAPIfication of > > >> > -machine to be a requirement to start reporting machine type > > >> > deprecation information. > > >> > > >> Valid point. Still, I believe we should at least try to predict how the > > >> pieces we create now would fit with the pieces we plan to create later > > >> on. > > > > > > Sure. > > > > > >> > > >> Note that full QAPIfication of -machine isn't necessary to make QAPI > > >> feature "deprecated" work for machine types. Turning MachineInfo member > > >> @name into an enum, so we can tack "deprecated" onto its values, would > > >> suffice. > > >> > > >> Such a QAPIfication of machine types is still hard: QOM types are > > >> defined at compile time just like the QAPI schema, but their definition > > >> is distributed, and collected into one place only at run time. I > > >> discussed this on slide 39 of my "QEMU interface introspection: From > > >> hacks to solutions" talk (KVM Form 2015). Just for device_add, but it's > > >> just a special case of QOM. Choices listed there: > > >> > > >> * Collect drivers at compile time? Hard... > > >> * Make QAPI schema dynamic? Hard... > > >> * Forgo driver-specific arguments in schema? > > >> Defeats introspection... > > >> > > >> I'd like to add to the last item: > > >> > > >> Provide QOM introspection on par with QAPI schema introspection > > >> > > >> The QOM introspection we have (qom-list-types etc. is not on par. > > > > > > Agreed, but do we really want to do it? We have been avoiding > > > exposing QOM internals to the outside on purpose. I believe > > > there are at least two reasons for that: > > > > > > 1) Not every QOM type/property is supposed to be visible to the > > > outside > > > > True. > > > > However, the parts of QOM exposed via device_add and object-add are > > definitely part of the stable external interface (unless explicitly > > marked unstable). > > > > > (and nobody really knows what's the full set of > > > supported external QOM interfaces); > > > > Also true. And terrible. > > > > > 2) QAPI is our preferred interface interface specification/introspection > > > mechanism. > > > > When preferences and requirements collide, preferences tend to get run > > over. > > > > The QAPI schema is *declarative*: the schema declares QAPI objects and > > properties. We generate C from the schema, which we then compile and > > link into QEMU. > > > > QOM is by design *imperative*: we execute compiled C at QEMU run-time to > > define QOM objects and properties. Maximizes flexibility. See also > > Turing tarpit. > > > > No matter how much we'd prefer to use QAPI to specify external > > interfaces to QOM, we can't without making QAPI much more dynamic or QOM > > much more static. Either is hard. Quite possibly infeasible. > > > > We could try to extend QAPI/QMP introspection to somehow merge in > > additional information at run-time[1]. Could be regarded as a limited > > way to make QAPI more dynamic. This is in the "vague idea, not sure > > it's feasible" stage. > > > > We could try to rearchitect QOM so that you can optionally specify QOM > > stuff in the QAPI schema, then require that for QOM stuff that's part of > > a stable external interface. Same "vague idea" stage, feasibility even > > more doubtful. > > I think it is feasible to use QAPI to declare the QOM object type, > QOM parent type, list of QOM interfaces and QOM properties in a > declarative manner. From that generate all the tedious boilerplate > code. The dev then just has to provide the "interesting" code for > the object. > > I'd really like to do a PoC of this but never have free time :-( > Perhaps an interesting task for someone who wants to delve into > some new work... > > > > >> >> Example: to deprecate block driver "vfat", add feature "deprecated" to > > >> >> BlockdevDriver member @vfat. > > >> >> > > >> >> Unlike your patches, this does not require finding a "somewhere" that > > >> >> provides information on "something". You simply tack "deprecated" right > > >> >> onto "something". > > >> >> > > >> >> Your patches provide more information, however: human-readable messages. > > >> > > > >> > It also includes a machine-friendly suggested alternative (which > > >> > I think is even more important that the human-readable message). > > >> > > >> I agree we should point to a preferred replacement whenever we deprecate > > >> something. > > >> > > >> We have to do it in documentation. And we generally do, in > > >> qemu-deprecated.texi. > > >> > > >> How useful would doing it in QMP as well be? Depends on what management > > >> applications can do with the additional information. > > > > > > I expect it to be useful for things that have obvious > > > replacements, like old machine type or CPU model versions. > > > > I doubt a management application should apply suggested replacements > > automatically, and I doubt libvirt would. Not even when QEMU developers > > deem them "obvious". > > We certainly won't apply the suggested replacement as in many cases > it is not going to be a functionally equivalent drop-in. Who's "we"? > > If QEMU logs it to stderr, it will end up in the per-VM log file > libvirt has under /var/log/libvirt/qemu/$GUESTNAME.log. If QEMU > doesn't log it to stderr, then libvirt would just write it to > that same log file itself. > > If libvirt gains some API or event for notifying apps of deprecation > we might bubble it up to the mgmt app that way. > > I still feel it is useful to have the suggested replacement in the > logs, rather than only leaving it in qemu-deprecated.texi. This > way the info is immediately visible to both app developers and any > support person dealing with bugs. > > If the app dev see the suggested replacement upfront they're more > likely to make an immediate decision to update their code if the > suggestion is trivial. If they need to go find the QEMU docs to > lookup what action is required I feel they'll more likely just > put the item on their long todo list where it will languish. Agreed. However, note that the audience for deprecation information is not just developers and support people. End users need to know when they are relying on a deprecated feature, and applications should make it as easy as possible for them to update their configurations. I'm not suggesting the alternative would be applied automatically. But having the alternative available in a machine-friendly way may be the difference between a unhelpful UI that just tells the user there's some problem but can't give a solution, and one that can really assist the user to fix the problem.
On Thu, May 09, 2019 at 12:52:47PM -0300, Eduardo Habkost wrote: > On Thu, May 09, 2019 at 10:14:52AM +0100, Daniel P. Berrangé wrote: > > On Thu, May 09, 2019 at 10:31:46AM +0200, Markus Armbruster wrote: > > > We've wandered into the QAPI vs. QOM swamp. Cc: Paolo. > > > > > > Eduardo Habkost <ehabkost@redhat.com> writes: > > > > > > > On Wed, May 08, 2019 at 11:16:50AM +0200, Markus Armbruster wrote: > > > >> Eduardo Habkost <ehabkost@redhat.com> writes: > > > >> > > > >> > On Tue, May 07, 2019 at 07:07:04AM +0200, Markus Armbruster wrote: > > > >> >> Eduardo Habkost <ehabkost@redhat.com> writes: > > > >> >> > > > >> >> > This series adds machine type deprecation information to the > > > >> >> > output of the `query-machines` QMP command. With this, libvirt > > > >> >> > and management software will be able to show this information to > > > >> >> > users and/or suggest changes to VM configuration to avoid > > > >> >> > deprecated machine types. > > > >> >> > > > >> >> This overlaps with something I want to try, namely using Kevin's > > > >> >> proposed QAPI feature flags for deprecation markings. Let's compare the > > > >> >> two. > > > >> >> > > > >> >> To mark something as deprecated with your patches, you add a > > > >> >> @support-status member somewhere, where "somewhere" is related to > > > >> >> "something" by "provides information on". > > > >> >> > > > >> >> Example: MachineInfo (returned by query-machines) provides information > > > >> >> on possible values of -machine parameter type. If -machine was > > > >> >> QAPIfied, it would provide information on possible values of a QAPI > > > >> >> object type's member. The type might be anonymous. The member should > > > >> >> be an enum (we currently use 'str' in MachineInfo). > > > >> > > > > >> > QAPIfying -machine, -cpu, and -device would be wonderful. > > > >> > > > > >> >> > > > >> >> Example: say we want to deprecate block driver "vfat", > > > >> >> i.e. BlockdevDriver member @vfat. Type BlockdevDriver is used in > > > >> >> multiple places; let's ignore all but BlockdevOptions. We need to add > > > >> >> @support-status to something that provides information on > > > >> >> BlockdevDriver, or maybe on BlockdevOptions. There is no ad hoc query > > > >> >> providing information on either of the two, because QAPI/QMP > > > >> >> introspection has been sufficient. What now? > > > >> >> > > > >> >> Can we add deprecation information to (general) QAPI/QMP introspection > > > >> > > > > >> > Yes, we can. I think it's a good idea. But: > > > >> > > > > >> >> instead of ad hoc queries? > > > >> > > > > >> > I'm not sure about the "instead of" part. I don't want perfect > > > >> > to be the enemy of done, and I don't want QAPIfication of > > > >> > -machine to be a requirement to start reporting machine type > > > >> > deprecation information. > > > >> > > > >> Valid point. Still, I believe we should at least try to predict how the > > > >> pieces we create now would fit with the pieces we plan to create later > > > >> on. > > > > > > > > Sure. > > > > > > > >> > > > >> Note that full QAPIfication of -machine isn't necessary to make QAPI > > > >> feature "deprecated" work for machine types. Turning MachineInfo member > > > >> @name into an enum, so we can tack "deprecated" onto its values, would > > > >> suffice. > > > >> > > > >> Such a QAPIfication of machine types is still hard: QOM types are > > > >> defined at compile time just like the QAPI schema, but their definition > > > >> is distributed, and collected into one place only at run time. I > > > >> discussed this on slide 39 of my "QEMU interface introspection: From > > > >> hacks to solutions" talk (KVM Form 2015). Just for device_add, but it's > > > >> just a special case of QOM. Choices listed there: > > > >> > > > >> * Collect drivers at compile time? Hard... > > > >> * Make QAPI schema dynamic? Hard... > > > >> * Forgo driver-specific arguments in schema? > > > >> Defeats introspection... > > > >> > > > >> I'd like to add to the last item: > > > >> > > > >> Provide QOM introspection on par with QAPI schema introspection > > > >> > > > >> The QOM introspection we have (qom-list-types etc. is not on par. > > > > > > > > Agreed, but do we really want to do it? We have been avoiding > > > > exposing QOM internals to the outside on purpose. I believe > > > > there are at least two reasons for that: > > > > > > > > 1) Not every QOM type/property is supposed to be visible to the > > > > outside > > > > > > True. > > > > > > However, the parts of QOM exposed via device_add and object-add are > > > definitely part of the stable external interface (unless explicitly > > > marked unstable). > > > > > > > (and nobody really knows what's the full set of > > > > supported external QOM interfaces); > > > > > > Also true. And terrible. > > > > > > > 2) QAPI is our preferred interface interface specification/introspection > > > > mechanism. > > > > > > When preferences and requirements collide, preferences tend to get run > > > over. > > > > > > The QAPI schema is *declarative*: the schema declares QAPI objects and > > > properties. We generate C from the schema, which we then compile and > > > link into QEMU. > > > > > > QOM is by design *imperative*: we execute compiled C at QEMU run-time to > > > define QOM objects and properties. Maximizes flexibility. See also > > > Turing tarpit. > > > > > > No matter how much we'd prefer to use QAPI to specify external > > > interfaces to QOM, we can't without making QAPI much more dynamic or QOM > > > much more static. Either is hard. Quite possibly infeasible. > > > > > > We could try to extend QAPI/QMP introspection to somehow merge in > > > additional information at run-time[1]. Could be regarded as a limited > > > way to make QAPI more dynamic. This is in the "vague idea, not sure > > > it's feasible" stage. > > > > > > We could try to rearchitect QOM so that you can optionally specify QOM > > > stuff in the QAPI schema, then require that for QOM stuff that's part of > > > a stable external interface. Same "vague idea" stage, feasibility even > > > more doubtful. > > > > I think it is feasible to use QAPI to declare the QOM object type, > > QOM parent type, list of QOM interfaces and QOM properties in a > > declarative manner. From that generate all the tedious boilerplate > > code. The dev then just has to provide the "interesting" code for > > the object. > > > > I'd really like to do a PoC of this but never have free time :-( > > Perhaps an interesting task for someone who wants to delve into > > some new work... > > > > > > > >> >> Example: to deprecate block driver "vfat", add feature "deprecated" to > > > >> >> BlockdevDriver member @vfat. > > > >> >> > > > >> >> Unlike your patches, this does not require finding a "somewhere" that > > > >> >> provides information on "something". You simply tack "deprecated" right > > > >> >> onto "something". > > > >> >> > > > >> >> Your patches provide more information, however: human-readable messages. > > > >> > > > > >> > It also includes a machine-friendly suggested alternative (which > > > >> > I think is even more important that the human-readable message). > > > >> > > > >> I agree we should point to a preferred replacement whenever we deprecate > > > >> something. > > > >> > > > >> We have to do it in documentation. And we generally do, in > > > >> qemu-deprecated.texi. > > > >> > > > >> How useful would doing it in QMP as well be? Depends on what management > > > >> applications can do with the additional information. > > > > > > > > I expect it to be useful for things that have obvious > > > > replacements, like old machine type or CPU model versions. > > > > > > I doubt a management application should apply suggested replacements > > > automatically, and I doubt libvirt would. Not even when QEMU developers > > > deem them "obvious". > > > > We certainly won't apply the suggested replacement as in many cases > > it is not going to be a functionally equivalent drop-in. > > Who's "we"? I was refering to libvirt here. > > If QEMU logs it to stderr, it will end up in the per-VM log file > > libvirt has under /var/log/libvirt/qemu/$GUESTNAME.log. If QEMU > > doesn't log it to stderr, then libvirt would just write it to > > that same log file itself. > > > > If libvirt gains some API or event for notifying apps of deprecation > > we might bubble it up to the mgmt app that way. > > > > I still feel it is useful to have the suggested replacement in the > > logs, rather than only leaving it in qemu-deprecated.texi. This > > way the info is immediately visible to both app developers and any > > support person dealing with bugs. > > > > If the app dev see the suggested replacement upfront they're more > > likely to make an immediate decision to update their code if the > > suggestion is trivial. If they need to go find the QEMU docs to > > lookup what action is required I feel they'll more likely just > > put the item on their long todo list where it will languish. > > Agreed. However, note that the audience for deprecation > information is not just developers and support people. End users > need to know when they are relying on a deprecated feature, and > applications should make it as easy as possible for them to > update their configurations. > > I'm not suggesting the alternative would be applied > automatically. But having the alternative available in a > machine-friendly way may be the difference between a unhelpful UI > that just tells the user there's some problem but can't give a > solution, and one that can really assist the user to fix the > problem. For some aspects of QEMU it might be possible, but considering the broader set of things which can be deprecated, I don't think it is possible to expose a machine consumable "suggestion". Consider the deprecation of the ACL feature. We deprecated monitor commands "acl_add", "acl_policy", "acl_reset", etc. The suggested replacement is to use one of the many possible QAuthZ types combined with the -object arg. Even if we invented some way to express this in the schema, I don't think any app would usefully consume it. Regards, Daniel
On Thu, May 09, 2019 at 05:08:11PM +0100, Daniel P. Berrangé wrote: > On Thu, May 09, 2019 at 12:52:47PM -0300, Eduardo Habkost wrote: > > On Thu, May 09, 2019 at 10:14:52AM +0100, Daniel P. Berrangé wrote: > > > On Thu, May 09, 2019 at 10:31:46AM +0200, Markus Armbruster wrote: > > > > We've wandered into the QAPI vs. QOM swamp. Cc: Paolo. > > > > > > > > Eduardo Habkost <ehabkost@redhat.com> writes: > > > > > > > > > On Wed, May 08, 2019 at 11:16:50AM +0200, Markus Armbruster wrote: > > > > >> Eduardo Habkost <ehabkost@redhat.com> writes: > > > > >> > > > > >> > On Tue, May 07, 2019 at 07:07:04AM +0200, Markus Armbruster wrote: > > > > >> >> Eduardo Habkost <ehabkost@redhat.com> writes: > > > > >> >> > > > > >> >> > This series adds machine type deprecation information to the > > > > >> >> > output of the `query-machines` QMP command. With this, libvirt > > > > >> >> > and management software will be able to show this information to > > > > >> >> > users and/or suggest changes to VM configuration to avoid > > > > >> >> > deprecated machine types. > > > > >> >> > > > > >> >> This overlaps with something I want to try, namely using Kevin's > > > > >> >> proposed QAPI feature flags for deprecation markings. Let's compare the > > > > >> >> two. > > > > >> >> > > > > >> >> To mark something as deprecated with your patches, you add a > > > > >> >> @support-status member somewhere, where "somewhere" is related to > > > > >> >> "something" by "provides information on". > > > > >> >> > > > > >> >> Example: MachineInfo (returned by query-machines) provides information > > > > >> >> on possible values of -machine parameter type. If -machine was > > > > >> >> QAPIfied, it would provide information on possible values of a QAPI > > > > >> >> object type's member. The type might be anonymous. The member should > > > > >> >> be an enum (we currently use 'str' in MachineInfo). > > > > >> > > > > > >> > QAPIfying -machine, -cpu, and -device would be wonderful. > > > > >> > > > > > >> >> > > > > >> >> Example: say we want to deprecate block driver "vfat", > > > > >> >> i.e. BlockdevDriver member @vfat. Type BlockdevDriver is used in > > > > >> >> multiple places; let's ignore all but BlockdevOptions. We need to add > > > > >> >> @support-status to something that provides information on > > > > >> >> BlockdevDriver, or maybe on BlockdevOptions. There is no ad hoc query > > > > >> >> providing information on either of the two, because QAPI/QMP > > > > >> >> introspection has been sufficient. What now? > > > > >> >> > > > > >> >> Can we add deprecation information to (general) QAPI/QMP introspection > > > > >> > > > > > >> > Yes, we can. I think it's a good idea. But: > > > > >> > > > > > >> >> instead of ad hoc queries? > > > > >> > > > > > >> > I'm not sure about the "instead of" part. I don't want perfect > > > > >> > to be the enemy of done, and I don't want QAPIfication of > > > > >> > -machine to be a requirement to start reporting machine type > > > > >> > deprecation information. > > > > >> > > > > >> Valid point. Still, I believe we should at least try to predict how the > > > > >> pieces we create now would fit with the pieces we plan to create later > > > > >> on. > > > > > > > > > > Sure. > > > > > > > > > >> > > > > >> Note that full QAPIfication of -machine isn't necessary to make QAPI > > > > >> feature "deprecated" work for machine types. Turning MachineInfo member > > > > >> @name into an enum, so we can tack "deprecated" onto its values, would > > > > >> suffice. > > > > >> > > > > >> Such a QAPIfication of machine types is still hard: QOM types are > > > > >> defined at compile time just like the QAPI schema, but their definition > > > > >> is distributed, and collected into one place only at run time. I > > > > >> discussed this on slide 39 of my "QEMU interface introspection: From > > > > >> hacks to solutions" talk (KVM Form 2015). Just for device_add, but it's > > > > >> just a special case of QOM. Choices listed there: > > > > >> > > > > >> * Collect drivers at compile time? Hard... > > > > >> * Make QAPI schema dynamic? Hard... > > > > >> * Forgo driver-specific arguments in schema? > > > > >> Defeats introspection... > > > > >> > > > > >> I'd like to add to the last item: > > > > >> > > > > >> Provide QOM introspection on par with QAPI schema introspection > > > > >> > > > > >> The QOM introspection we have (qom-list-types etc. is not on par. > > > > > > > > > > Agreed, but do we really want to do it? We have been avoiding > > > > > exposing QOM internals to the outside on purpose. I believe > > > > > there are at least two reasons for that: > > > > > > > > > > 1) Not every QOM type/property is supposed to be visible to the > > > > > outside > > > > > > > > True. > > > > > > > > However, the parts of QOM exposed via device_add and object-add are > > > > definitely part of the stable external interface (unless explicitly > > > > marked unstable). > > > > > > > > > (and nobody really knows what's the full set of > > > > > supported external QOM interfaces); > > > > > > > > Also true. And terrible. > > > > > > > > > 2) QAPI is our preferred interface interface specification/introspection > > > > > mechanism. > > > > > > > > When preferences and requirements collide, preferences tend to get run > > > > over. > > > > > > > > The QAPI schema is *declarative*: the schema declares QAPI objects and > > > > properties. We generate C from the schema, which we then compile and > > > > link into QEMU. > > > > > > > > QOM is by design *imperative*: we execute compiled C at QEMU run-time to > > > > define QOM objects and properties. Maximizes flexibility. See also > > > > Turing tarpit. > > > > > > > > No matter how much we'd prefer to use QAPI to specify external > > > > interfaces to QOM, we can't without making QAPI much more dynamic or QOM > > > > much more static. Either is hard. Quite possibly infeasible. > > > > > > > > We could try to extend QAPI/QMP introspection to somehow merge in > > > > additional information at run-time[1]. Could be regarded as a limited > > > > way to make QAPI more dynamic. This is in the "vague idea, not sure > > > > it's feasible" stage. > > > > > > > > We could try to rearchitect QOM so that you can optionally specify QOM > > > > stuff in the QAPI schema, then require that for QOM stuff that's part of > > > > a stable external interface. Same "vague idea" stage, feasibility even > > > > more doubtful. > > > > > > I think it is feasible to use QAPI to declare the QOM object type, > > > QOM parent type, list of QOM interfaces and QOM properties in a > > > declarative manner. From that generate all the tedious boilerplate > > > code. The dev then just has to provide the "interesting" code for > > > the object. > > > > > > I'd really like to do a PoC of this but never have free time :-( > > > Perhaps an interesting task for someone who wants to delve into > > > some new work... > > > > > > > > > > >> >> Example: to deprecate block driver "vfat", add feature "deprecated" to > > > > >> >> BlockdevDriver member @vfat. > > > > >> >> > > > > >> >> Unlike your patches, this does not require finding a "somewhere" that > > > > >> >> provides information on "something". You simply tack "deprecated" right > > > > >> >> onto "something". > > > > >> >> > > > > >> >> Your patches provide more information, however: human-readable messages. > > > > >> > > > > > >> > It also includes a machine-friendly suggested alternative (which > > > > >> > I think is even more important that the human-readable message). > > > > >> > > > > >> I agree we should point to a preferred replacement whenever we deprecate > > > > >> something. > > > > >> > > > > >> We have to do it in documentation. And we generally do, in > > > > >> qemu-deprecated.texi. > > > > >> > > > > >> How useful would doing it in QMP as well be? Depends on what management > > > > >> applications can do with the additional information. > > > > > > > > > > I expect it to be useful for things that have obvious > > > > > replacements, like old machine type or CPU model versions. > > > > > > > > I doubt a management application should apply suggested replacements > > > > automatically, and I doubt libvirt would. Not even when QEMU developers > > > > deem them "obvious". > > > > > > We certainly won't apply the suggested replacement as in many cases > > > it is not going to be a functionally equivalent drop-in. > > > > Who's "we"? > > I was refering to libvirt here. > > > > If QEMU logs it to stderr, it will end up in the per-VM log file > > > libvirt has under /var/log/libvirt/qemu/$GUESTNAME.log. If QEMU > > > doesn't log it to stderr, then libvirt would just write it to > > > that same log file itself. > > > > > > If libvirt gains some API or event for notifying apps of deprecation > > > we might bubble it up to the mgmt app that way. > > > > > > I still feel it is useful to have the suggested replacement in the > > > logs, rather than only leaving it in qemu-deprecated.texi. This > > > way the info is immediately visible to both app developers and any > > > support person dealing with bugs. > > > > > > If the app dev see the suggested replacement upfront they're more > > > likely to make an immediate decision to update their code if the > > > suggestion is trivial. If they need to go find the QEMU docs to > > > lookup what action is required I feel they'll more likely just > > > put the item on their long todo list where it will languish. > > > > Agreed. However, note that the audience for deprecation > > information is not just developers and support people. End users > > need to know when they are relying on a deprecated feature, and > > applications should make it as easy as possible for them to > > update their configurations. > > > > I'm not suggesting the alternative would be applied > > automatically. But having the alternative available in a > > machine-friendly way may be the difference between a unhelpful UI > > that just tells the user there's some problem but can't give a > > solution, and one that can really assist the user to fix the > > problem. > > For some aspects of QEMU it might be possible, but considering the > broader set of things which can be deprecated, I don't think it is > possible to expose a machine consumable "suggestion". > > Consider the deprecation of the ACL feature. We deprecated monitor > commands "acl_add", "acl_policy", "acl_reset", etc. The suggested > replacement is to use one of the many possible QAuthZ types combined > with the -object arg. Even if we invented some way to express this > in the schema, I don't think any app would usefully consume it. No problem, we don't need to suggest a machine consumable alternative for everything. I'm thinking about features that are visible to the end user and require user action to fix their configuration, like machine type versions or CPU model versions.
On Thu, May 09, 2019 at 10:31:46AM +0200, Markus Armbruster wrote: > We've wandered into the QAPI vs. QOM swamp. Cc: Paolo. > > Eduardo Habkost <ehabkost@redhat.com> writes: > > > On Wed, May 08, 2019 at 11:16:50AM +0200, Markus Armbruster wrote: > >> Eduardo Habkost <ehabkost@redhat.com> writes: > >> > >> > On Tue, May 07, 2019 at 07:07:04AM +0200, Markus Armbruster wrote: > >> >> Eduardo Habkost <ehabkost@redhat.com> writes: > >> >> > >> >> > This series adds machine type deprecation information to the > >> >> > output of the `query-machines` QMP command. With this, libvirt > >> >> > and management software will be able to show this information to > >> >> > users and/or suggest changes to VM configuration to avoid > >> >> > deprecated machine types. > >> >> > >> >> This overlaps with something I want to try, namely using Kevin's > >> >> proposed QAPI feature flags for deprecation markings. Let's compare the > >> >> two. > >> >> > >> >> To mark something as deprecated with your patches, you add a > >> >> @support-status member somewhere, where "somewhere" is related to > >> >> "something" by "provides information on". > >> >> > >> >> Example: MachineInfo (returned by query-machines) provides information > >> >> on possible values of -machine parameter type. If -machine was > >> >> QAPIfied, it would provide information on possible values of a QAPI > >> >> object type's member. The type might be anonymous. The member should > >> >> be an enum (we currently use 'str' in MachineInfo). > >> > > >> > QAPIfying -machine, -cpu, and -device would be wonderful. > >> > > >> >> > >> >> Example: say we want to deprecate block driver "vfat", > >> >> i.e. BlockdevDriver member @vfat. Type BlockdevDriver is used in > >> >> multiple places; let's ignore all but BlockdevOptions. We need to add > >> >> @support-status to something that provides information on > >> >> BlockdevDriver, or maybe on BlockdevOptions. There is no ad hoc query > >> >> providing information on either of the two, because QAPI/QMP > >> >> introspection has been sufficient. What now? > >> >> > >> >> Can we add deprecation information to (general) QAPI/QMP introspection > >> > > >> > Yes, we can. I think it's a good idea. But: > >> > > >> >> instead of ad hoc queries? > >> > > >> > I'm not sure about the "instead of" part. I don't want perfect > >> > to be the enemy of done, and I don't want QAPIfication of > >> > -machine to be a requirement to start reporting machine type > >> > deprecation information. > >> > >> Valid point. Still, I believe we should at least try to predict how the > >> pieces we create now would fit with the pieces we plan to create later > >> on. > > > > Sure. > > > >> > >> Note that full QAPIfication of -machine isn't necessary to make QAPI > >> feature "deprecated" work for machine types. Turning MachineInfo member > >> @name into an enum, so we can tack "deprecated" onto its values, would > >> suffice. > >> > >> Such a QAPIfication of machine types is still hard: QOM types are > >> defined at compile time just like the QAPI schema, but their definition > >> is distributed, and collected into one place only at run time. I > >> discussed this on slide 39 of my "QEMU interface introspection: From > >> hacks to solutions" talk (KVM Form 2015). Just for device_add, but it's > >> just a special case of QOM. Choices listed there: > >> > >> * Collect drivers at compile time? Hard... > >> * Make QAPI schema dynamic? Hard... > >> * Forgo driver-specific arguments in schema? > >> Defeats introspection... > >> > >> I'd like to add to the last item: > >> > >> Provide QOM introspection on par with QAPI schema introspection > >> > >> The QOM introspection we have (qom-list-types etc. is not on par. > > > > Agreed, but do we really want to do it? We have been avoiding > > exposing QOM internals to the outside on purpose. I believe > > there are at least two reasons for that: > > > > 1) Not every QOM type/property is supposed to be visible to the > > outside > > True. > > However, the parts of QOM exposed via device_add and object-add are > definitely part of the stable external interface (unless explicitly > marked unstable). > > > (and nobody really knows what's the full set of > > supported external QOM interfaces); > > Also true. And terrible. > > > 2) QAPI is our preferred interface interface specification/introspection > > mechanism. > > When preferences and requirements collide, preferences tend to get run > over. > > The QAPI schema is *declarative*: the schema declares QAPI objects and > properties. We generate C from the schema, which we then compile and > link into QEMU. > > QOM is by design *imperative*: we execute compiled C at QEMU run-time to > define QOM objects and properties. Maximizes flexibility. See also > Turing tarpit. > > No matter how much we'd prefer to use QAPI to specify external > interfaces to QOM, we can't without making QAPI much more dynamic or QOM > much more static. Either is hard. Quite possibly infeasible. > > We could try to extend QAPI/QMP introspection to somehow merge in > additional information at run-time[1]. Could be regarded as a limited > way to make QAPI more dynamic. This is in the "vague idea, not sure > it's feasible" stage. > > We could try to rearchitect QOM so that you can optionally specify QOM > stuff in the QAPI schema, then require that for QOM stuff that's part of > a stable external interface. Same "vague idea" stage, feasibility even > more doubtful. > > If I remember correctly, Paolo suggested to abandon the idea to have > query-qmp-schema cover device_add as infeasible. Instead, cover it > separately. > > I think this separate introspection mechanism should be as similar to > query-qmp-schema as practical. It should cover not just device_add, but > also object-add, and whatever other external interface to QOM we deem > stable. I think we are in agreement here. We don't need to make the new introspection mechanism part of query-qmp-schema, but I think it should use the same abstractions. > > Should there be more stable external interfaces that can't be defined in > the QAPI schema, we might want to cover them in this second > introspection mechanism, too. Agreed. > > >> Back to exposing machine type deprecation. > >> > >> I'm doubtful your proposed solution can be applied widely. It relies on > >> adding @support-status to something that provides information on > >> whatever is deprecated. The initial use is with a something that is an > >> ad hoc query, namely query-machines. To use it, the management > >> application needs to understand what query-machines' @support-status > >> applies to. Certainly feasible. But I fear every use will be a special > >> case. Furthermore, a suitable ad hoc query need not exist. What then? > >> Create suitable ad hoc queries just for communicating deprecation? > >> > >> Instead, I'd like us to think about a more genral solution. Or perhaps > >> two: one for properly QAPIfied stuff, and one for QOM. > > > > Should we really spend our time designing a second solution, if > > we could build this on top of QAPI abstractions? Making the QAPI > > schema dynamic might be hard, but reinventing QAPI and > > maintaining the two systems in parallel seems harder. > > I'd love to see a practical way forward that keeps QAPI/QMP > introspection the only game in town. > > >> >> Kevin's proposed QAPI feature flags[*] extend the QAPI language so that > >> >> struct types can optionally have a list of feature flags, which are > >> >> strings. Struct types suffice for his immediate needs. I'd like to use > >> >> feature flags to mark deprecation by tacking a "deprecated" feature onto > >> >> whatever is deprecated. This obviously needs feature support for > >> >> everything we want to be able to deprecate: commands, and events, as > >> >> well as members of enum and object types. > >> >> > >> >> Example: to deprecate block driver "vfat", add feature "deprecated" to > >> >> BlockdevDriver member @vfat. > >> >> > >> >> Unlike your patches, this does not require finding a "somewhere" that > >> >> provides information on "something". You simply tack "deprecated" right > >> >> onto "something". > >> >> > >> >> Your patches provide more information, however: human-readable messages. > >> > > >> > It also includes a machine-friendly suggested alternative (which > >> > I think is even more important that the human-readable message). > >> > >> I agree we should point to a preferred replacement whenever we deprecate > >> something. > >> > >> We have to do it in documentation. And we generally do, in > >> qemu-deprecated.texi. > >> > >> How useful would doing it in QMP as well be? Depends on what management > >> applications can do with the additional information. > > > > I expect it to be useful for things that have obvious > > replacements, like old machine type or CPU model versions. > > I doubt a management application should apply suggested replacements > automatically, and I doubt libvirt would. Not even when QEMU developers > deem them "obvious". > > >> > We could extend QAPI introspection to return that if necessary, > >> > right? > >> > >> I'm confident we can come up with *something*. It might kill the neat > >> and simple "use QAPI features to communicate deprecation" idea, though. > > > > If something is important enough to be communicated through > > stderr, it's important enough to be communicated through QMP. > > Mostly. Differences are due to the different consumers. > > stderr is primarily for human users. We print stuff useful to human > users. We have users that don't have access to stderr. They might have access to log files, but log files are pretty bad user interfaces. If it's important for some set of human users, apps using libvirt or QMP need access to that information so they can show it to their human users too. > > QMP is primarily for machines, secondarily for the humans building these > machines. We send stuff useful to the machines themselves, and stuff > the machines can use to be more useful for their users (which may be > machines or humans). We can also send stuff to help the humans building > the machines. > > In any case, the information we provide is limited by the cost to > provide it. Absolutely. > > > Is that enough reason to provide something more complex? > > We need to consider cost / benefit. > > On benefit, I'd like to know what libvirt would do with the additional > information beyond logging it. I'd say it should provide it to apps, otherwise this won't be more useful than the existing log files. > > Is the additional information you propose to provide static or dynamic? > > By "static", I mean each occurence of a feature in the QAPI schema is > tied to one fixed instance of "additional information". I don't think I understand this description of "static". I expect the data to be fixed at build time, but I expect it to be different in downstream distributions of QEMU. > > > Do we need QAPI features to be just strings? Can't they be a > > more complex type, like a QAPI alternate? > > Adds complexity. > > We currently imagine QAPI features enum-like, i.e. a list of strings, > optionally with conditions. The conditions are evaluated at QAPI > generation time, and not visible in introspection. > > This is probably the stupidest solution that could possibly work. The > structure of features is trivial. > > More expressive solutions include: > > * List of 'any'. Feels like a bad idea, because it's completely > unstructured. Agreed. > > * List of some 'QapiFeatures' object type. Lets us expose the variable > structure of features in introspection. > > * List of some 'QapiFeatures' alternate type. Like the previous, but > permits coding simple feature flags as strings. > > All of the more expressive solutions I listed treat the additional > information as dynamic. Overly general in case the information is > actually always static. The static vs. dynamic distinction is getting me confused. Why are the more expressive solutions more dynamic than "list of strings"? Can you give examples? Also, why do we want to place all info inside the same "features" attribute instead of just adding new fields to SchemaInfoObject? i.e. why are these options: { "members" [ ... ], "features": [ "dynamic-read-only", "deprecated" ] } { "members" [ ... ], "features": [ "dynamic-read-only", { "deprecated": true, "hint": "FOO" } ] } better than these options: { "members" [ ... ], "features": [ "dynamic-read-only" ], "deprecated": true } { "members" [ ... ], "features": [ "dynamic-read-only" ], "deprecation-info": { "deprecated": true, "hint": "FOO" } } > > >> >> Food for thought :) > >> >> > >> >> > >> >> [*] Hiding in > >> >> Subject: [PATCH 0/4] file-posix: Add dynamic-auto-read-only QAPI feature > >> >> Date: Mon, 8 Apr 2019 16:35:39 +0200 > >> >> Message-Id: <20190408143543.3982-1-kwolf@redhat.com> > > > [1] Special case of "merge additional information into QAPI/QMP > introspection at run-time, relegated to a footnote, because I think it's > of mostly historical interest now. > > Qdev was designed to be declarative, but the declarations always got > collected only at run-time. It has since been rebased onto imperative > QOM. Since you can mix declarative qdev and imperative QOM, the > combination is effectively imperative. > > We could collect the declarative parts of qdev at compile-time somehow, > and fold them into QAPI introspection. That's the "Collect drives at > compile time" item from the slide I quoted. I think this idea has > become pretty much worthless, since it would cover device_add only in > part, and object-add not at all.