| Message ID | 20180925160248.30801-3-kchamart@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | Deprecate QMP `cpu-add`; document vCPU hotplug | expand |
On Tue, 25 Sep 2018 18:02:48 +0200 Kashyap Chamarthy <kchamart@redhat.com> wrote: > Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com> > --- > docs/cpu-hotplug.rst | 140 +++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 140 insertions(+) > create mode 100644 docs/cpu-hotplug.rst > > diff --git a/docs/cpu-hotplug.rst b/docs/cpu-hotplug.rst > new file mode 100644 > index 0000000000..8f9e63a9f7 > --- /dev/null > +++ b/docs/cpu-hotplug.rst > @@ -0,0 +1,140 @@ > +=================== > +Virtual CPU hotplug > +=================== > + > +A complete example of vCPU hotplug (and hot-unplug) using QMP > +``device_add`` and ``device_del``. > + > +vCPU hotplug > +------------ > + > +(1) Launch QEMU as follows (note that the "maxcpus" is mandatory to > + allow vCPU hotplug):: > + > + $ qemu-system-x86_64 -display none -no-user-config -m 2048 \ > + -nodefaults -monitor stdio -machine pc,accel=kvm,usb=off \ > + -smp 1,maxcpus=2 -cpu IvyBridge-IBRS \ > + -blockdev node-name=node-Base,driver=qcow2,file.driver=file,file.filename=./base.qcow2 \ > + -device virtio-blk,drive=node-Base,id=virtio0 -qmp unix:/tmp/qmp-sock,server,nowait all options beside of -smp and -qmp are not relevant here and mostly masking parts that matter, I'd simplify it as much as possible: qemu-system-x86_64 -smp 1,maxcpus=2 -qmp unix:/tmp/qmp-sock,server,nowait ... Also example only covers x86 target, but there are ppc target with spapr machine (add/del) and s390 target (add only). probably should be mentioned here as well In addition one could coldplug cpus the same way as other devices using -device CLI option. > +(2) Run 'qmp-shell' (located in the source tree) to connect to the > + just-launched QEMU:: > + > + $> ./qmp/qmp-shell -p -v /tmp/qmp-sock > + [...] > + (QEMU) > + > +(3) Check which socket is free to allow hotplugging a CPU:: may be: which cpus are possible to plug (an entry with qom-path property describes an existing cpu) > + > + (QEMU) query-hotpluggable-cpus > + { > + "execute": "query-hotpluggable-cpus", > + "arguments": {} > + } > + { > + "return": [ > + { > + "type": "IvyBridge-IBRS-x86_64-cpu", > + "vcpus-count": 1, > + "props": { > + "socket-id": 1, > + "core-id": 0, > + "thread-id": 0 > + } > + }, > + { > + "qom-path": "/machine/unattached/device[0]", > + "type": "IvyBridge-IBRS-x86_64-cpu", > + "vcpus-count": 1, > + "props": { > + "socket-id": 0, > + "core-id": 0, > + "thread-id": 0 > + } > + } > + ] > + } > + (QEMU) > + > +(4) We can see that socket 1 is free, so use `device_add` to hotplug ... and 'arguments' provide a list of property/value pairs to create corresponding cpu. > + "IvyBridge-IBRS-x86_64-cpu":: > + > + (QEMU) device_add id=cpu-2 driver=IvyBridge-IBRS-x86_64-cpu socket-id=1 core-id=0 thread-id=0 > + { > + "execute": "device_add", > + "arguments": { > + "socket-id": 1, > + "driver": "IvyBridge-IBRS-x86_64-cpu", > + "id": "cpu-2", > + "core-id": 0, > + "thread-id": 0 > + } > + } > + { > + "return": {} > + } > + (QEMU) > + > +(5) Optionally, run QMP `query-cpus-fast` for some details about the > + vCPUs:: I'd replace it with another query-hotpluggable-cpus call > + > + (QEMU) query-cpus-fast > + { > + "execute": "query-cpus-fast", > + "arguments": {} > + } > + { > + "return": [ > + { > + "qom-path": "/machine/unattached/device[0]", > + "target": "x86_64", > + "thread-id": 11534, > + "cpu-index": 0, > + "props": { > + "socket-id": 0, > + "core-id": 0, > + "thread-id": 0 > + }, > + "arch": "x86" > + }, > + { > + "qom-path": "/machine/peripheral/cpu-2", > + "target": "x86_64", > + "thread-id": 12106, > + "cpu-index": 1, > + "props": { > + "socket-id": 1, > + "core-id": 0, > + "thread-id": 0 > + }, > + "arch": "x86" > + } > + ] > + } > + (QEMU) > + > + > +vCPU hot-unplug > +--------------- > + > +From the 'qmp-shell', invoke the QMP ``device_del`` command:: > + > + (QEMU) device_del id=cpu-2 > + { > + "execute": "device_del", > + "arguments": { > + "id": "cpu-2" > + } > + } > + { > + "return": {} > + } > + (QEMU) > + > +.. note:: > + vCPU hot-unplug requires guest cooperation; so the ``device_del`` > + command above does not guarantee vCPU removal -- it's a "request to > + unplug". At this point, the guest will get a System Control > + Interupt (SCI) and calls the ACPI handler for the affected vCPU > + device. Then the guest kernel will bring the vCPU offline and tells > + QEMU to unplug it. s/tells/tell/
Igor Mammedov <imammedo@redhat.com> writes: > On Tue, 25 Sep 2018 18:02:48 +0200 > Kashyap Chamarthy <kchamart@redhat.com> wrote: > >> Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com> >> --- >> docs/cpu-hotplug.rst | 140 +++++++++++++++++++++++++++++++++++++++++++ >> 1 file changed, 140 insertions(+) >> create mode 100644 docs/cpu-hotplug.rst >> >> diff --git a/docs/cpu-hotplug.rst b/docs/cpu-hotplug.rst >> new file mode 100644 >> index 0000000000..8f9e63a9f7 >> --- /dev/null >> +++ b/docs/cpu-hotplug.rst >> @@ -0,0 +1,140 @@ >> +=================== >> +Virtual CPU hotplug >> +=================== >> + >> +A complete example of vCPU hotplug (and hot-unplug) using QMP >> +``device_add`` and ``device_del``. >> + >> +vCPU hotplug >> +------------ >> + >> +(1) Launch QEMU as follows (note that the "maxcpus" is mandatory to >> + allow vCPU hotplug):: >> + >> + $ qemu-system-x86_64 -display none -no-user-config -m 2048 \ >> + -nodefaults -monitor stdio -machine pc,accel=kvm,usb=off \ >> + -smp 1,maxcpus=2 -cpu IvyBridge-IBRS \ >> + -blockdev node-name=node-Base,driver=qcow2,file.driver=file,file.filename=./base.qcow2 \ >> + -device virtio-blk,drive=node-Base,id=virtio0 -qmp unix:/tmp/qmp-sock,server,nowait > all options beside of -smp and -qmp are not relevant here and mostly masking > parts that matter, I'd simplify it as much as possible: > qemu-system-x86_64 -smp 1,maxcpus=2 -qmp unix:/tmp/qmp-sock,server,nowait ... > > Also example only covers x86 target, but there are ppc target with spapr machine (add/del) > and s390 target (add only). probably should be mentioned here as well > > In addition one could coldplug cpus the same way as other devices using -device CLI option. > > >> +(2) Run 'qmp-shell' (located in the source tree) to connect to the >> + just-launched QEMU:: >> + >> + $> ./qmp/qmp-shell -p -v /tmp/qmp-sock >> + [...] >> + (QEMU) >> + >> +(3) Check which socket is free to allow hotplugging a CPU:: > may be: which cpus are possible to plug (an entry with qom-path property describes an existing cpu) Suggest (3) Find out which CPU types could be plugged, and into which sockets: >> + >> + (QEMU) query-hotpluggable-cpus >> + { >> + "execute": "query-hotpluggable-cpus", >> + "arguments": {} >> + } >> + { >> + "return": [ >> + { >> + "type": "IvyBridge-IBRS-x86_64-cpu", >> + "vcpus-count": 1, >> + "props": { >> + "socket-id": 1, >> + "core-id": 0, >> + "thread-id": 0 >> + } >> + }, >> + { >> + "qom-path": "/machine/unattached/device[0]", >> + "type": "IvyBridge-IBRS-x86_64-cpu", >> + "vcpus-count": 1, >> + "props": { >> + "socket-id": 0, >> + "core-id": 0, >> + "thread-id": 0 >> + } >> + } >> + ] >> + } >> + (QEMU) >> + >> +(4) We can see that socket 1 is free, How? I know, but only because I just read the documentation of query-hotpluggable-cpus. Which by the way sucks. For instance, will the command always return exactly one HotpluggableCPU object per socket? Anyway, what about this: The command returns an object with a "qom-path" member for each present CPU. In this case, it shows an IvyBridge-IBRS-x86_64-cpu in socket 0. It returns an object without a "qom-path" for every possibly CPU hot-plug. In this case, it shows you can plug an IvyBridge-IBRS-x86_64-cpu into socket 1, and the additional properties you need to pass to device_add for that. >> so use `device_add` to hotplug > ... and 'arguments' provide a list of property/value pairs to create corresponding cpu. > >> + "IvyBridge-IBRS-x86_64-cpu":: Suggest (4) Hot-plug an additional CPU: >> + >> + (QEMU) device_add id=cpu-2 driver=IvyBridge-IBRS-x86_64-cpu socket-id=1 core-id=0 thread-id=0 >> + { >> + "execute": "device_add", >> + "arguments": { >> + "socket-id": 1, >> + "driver": "IvyBridge-IBRS-x86_64-cpu", >> + "id": "cpu-2", >> + "core-id": 0, >> + "thread-id": 0 >> + } >> + } >> + { >> + "return": {} >> + } >> + (QEMU) >> + [...]
On Wed, Sep 26, 2018 at 05:24:27PM +0200, Igor Mammedov wrote: > On Tue, 25 Sep 2018 18:02:48 +0200 > Kashyap Chamarthy <kchamart@redhat.com> wrote: [...] > > +(1) Launch QEMU as follows (note that the "maxcpus" is mandatory to > > + allow vCPU hotplug):: > > + > > + $ qemu-system-x86_64 -display none -no-user-config -m 2048 \ > > + -nodefaults -monitor stdio -machine pc,accel=kvm,usb=off \ > > + -smp 1,maxcpus=2 -cpu IvyBridge-IBRS \ > > + -blockdev node-name=node-Base,driver=qcow2,file.driver=file,file.filename=./base.qcow2 \ > > + -device virtio-blk,drive=node-Base,id=virtio0 -qmp unix:/tmp/qmp-sock,server,nowait > > > all options beside of -smp and -qmp are not relevant here and mostly masking > parts that matter, I'd simplify it as much as possible: > qemu-system-x86_64 -smp 1,maxcpus=2 -qmp unix:/tmp/qmp-sock,server,nowait ... Sure, I can cut it down. (That specific command-line is what I normally use when playing with the QEMU Block Layer.) > Also example only covers x86 target, but there are ppc target with > spapr machine (add/del) and s390 target (add only). probably should be > mentioned here as well I'll probably just mention about PPC and s390. But adding examples for them too is a bit redundant. > In addition one could coldplug cpus the same way as other devices > using -device CLI option. Good point, I'll mention it. > > +(2) Run 'qmp-shell' (located in the source tree) to connect to the > > + just-launched QEMU:: > > + > > + $> ./qmp/qmp-shell -p -v /tmp/qmp-sock > > + [...] > > + (QEMU) > > + > > +(3) Check which socket is free to allow hotplugging a CPU:: > > > may be: which cpus are possible to plug (an entry with qom-path > property describes an existing cpu) As a user of QMP, I don't find that phrasing clear as well, I'm afraid. Wonder if there's a way better way to tell what's happening here. [...] > > +(4) We can see that socket 1 is free, so use `device_add` to hotplug > > > ... and 'arguments' provide a list of property/value pairs to create > corresponding cpu. Noted. [...] > > +(5) Optionally, run QMP `query-cpus-fast` for some details about the > > + vCPUs:: > > > I'd replace it with another query-hotpluggable-cpus call Will do. > > + [...] > > +vCPU hot-unplug > > +--------------- > > + > > +From the 'qmp-shell', invoke the QMP ``device_del`` command:: > > + > > + (QEMU) device_del id=cpu-2 > > + { > > + "execute": "device_del", > > + "arguments": { > > + "id": "cpu-2" > > + } > > + } > > + { > > + "return": {} > > + } > > + (QEMU) > > + > > +.. note:: > > + vCPU hot-unplug requires guest cooperation; so the ``device_del`` > > + command above does not guarantee vCPU removal -- it's a "request to > > + unplug". At this point, the guest will get a System Control > > + Interupt (SCI) and calls the ACPI handler for the affected vCPU > > + device. Then the guest kernel will bring the vCPU offline and tells > > + QEMU to unplug it. > s/tells/tell/ Will fix. Thanks for the review, Igor!
On Thu, Sep 27, 2018 at 04:33:16PM +0200, Markus Armbruster wrote: > Igor Mammedov <imammedo@redhat.com> writes: > > > On Tue, 25 Sep 2018 18:02:48 +0200 > > Kashyap Chamarthy <kchamart@redhat.com> wrote: [...] > >> +(3) Check which socket is free to allow hotplugging a CPU:: > > may be: which cpus are possible to plug (an entry with qom-path > > property describes an existing cpu) > > Suggest > > (3) Find out which CPU types could be plugged, and into which sockets: Yeah, clearer. [...] > >> +(4) We can see that socket 1 is free, > > How? I know, but only because I just read the documentation of > query-hotpluggable-cpus. Which by the way sucks. For instance, will > the command always return exactly one HotpluggableCPU object per socket? About the 'how', I was not entirely sure, hence my request in the cover letter. > Anyway, what about this: > > The command returns an object with a "qom-path" member for each > present CPU. In this case, it shows an IvyBridge-IBRS-x86_64-cpu in > socket 0. > > It returns an object without a "qom-path" for every possibly CPU > hot-plug. In this case, it shows you can plug an > IvyBridge-IBRS-x86_64-cpu into socket 1, and the additional > properties you need to pass to device_add for that. Crystal clear. Many thanks for the review! > > ... and 'arguments' provide a list of property/value pairs to create > > corresponding cpu. > > > >> + "IvyBridge-IBRS-x86_64-cpu":: > > Suggest > > (4) Hot-plug an additional CPU: [...]
On Mon, 1 Oct 2018 10:18:45 +0200 Kashyap Chamarthy <kchamart@redhat.com> wrote: > On Thu, Sep 27, 2018 at 04:33:16PM +0200, Markus Armbruster wrote: > > Igor Mammedov <imammedo@redhat.com> writes: > > > > > On Tue, 25 Sep 2018 18:02:48 +0200 > > > Kashyap Chamarthy <kchamart@redhat.com> wrote: > > [...] > > > >> +(3) Check which socket is free to allow hotplugging a CPU:: > > > may be: which cpus are possible to plug (an entry with qom-path > > > property describes an existing cpu) > > > > Suggest > > > > (3) Find out which CPU types could be plugged, and into which sockets: > > Yeah, clearer. > > [...] > > > >> +(4) We can see that socket 1 is free, > > > > How? I know, but only because I just read the documentation of > > query-hotpluggable-cpus. Which by the way sucks. For instance, will > > the command always return exactly one HotpluggableCPU object per socket? > > About the 'how', I was not entirely sure, hence my request in the cover > letter. > > > Anyway, what about this: > > > > The command returns an object with a "qom-path" member for each > > present CPU. In this case, it shows an IvyBridge-IBRS-x86_64-cpu in > > socket 0. > > > > It returns an object without a "qom-path" for every possibly CPU > > hot-plug. In this case, it shows you can plug an > > IvyBridge-IBRS-x86_64-cpu into socket 1, and the additional > > properties you need to pass to device_add for that. not really sure my English (CCed Eric) but to match 'an object' with the rest of sentence: It returns an object without a "qom-path" for a possible to hot-plug CPU. + In this case, it shows you can plug an IvyBridge-IBRS-x86_64-cpu into socket 1/core = 0/thread 0, where 'props' list describes additional properties you need to pass to device_add for hot-pluging that CPU. > > Crystal clear. > > Many thanks for the review! > > > > ... and 'arguments' provide a list of property/value pairs to create > > > corresponding cpu. > > > > > >> + "IvyBridge-IBRS-x86_64-cpu":: > > > > Suggest > > > > (4) Hot-plug an additional CPU: > > [...] >
On 10/1/18 3:59 AM, Igor Mammedov wrote: >>> Anyway, what about this: >>> >>> The command returns an object with a "qom-path" member for each >>> present CPU. In this case, it shows an IvyBridge-IBRS-x86_64-cpu in >>> socket 0. >>> >>> It returns an object without a "qom-path" for every possibly CPU >>> hot-plug. In this case, it shows you can plug an >>> IvyBridge-IBRS-x86_64-cpu into socket 1, and the additional >>> properties you need to pass to device_add for that. > not really sure my English (CCed Eric) but to match 'an object' with > the rest of sentence: > > It returns an object without a "qom-path" for a possible to hot-plug CPU. > + > In this case, it shows you can plug an IvyBridge-IBRS-x86_64-cpu > into socket 1/core = 0/thread 0, where 'props' list describes > additional properties you need to pass to device_add for hot-pluging > that CPU. Maybe: The command returns an object for CPUs that are present (containing a "qom-path" member) or which may be hot-plugged (no "qom-path" member). In this example, an IvyBridge-IBRS-x86_64-cpu is present in socket 0, while hot-plugging a CPU into socket 1 requires passing the listed properties to device_add.
On Mon, 8 Oct 2018 14:10:50 -0500 Eric Blake <eblake@redhat.com> wrote: > On 10/1/18 3:59 AM, Igor Mammedov wrote: > > >>> Anyway, what about this: > >>> > >>> The command returns an object with a "qom-path" member for each > >>> present CPU. In this case, it shows an IvyBridge-IBRS-x86_64-cpu in > >>> socket 0. > >>> > >>> It returns an object without a "qom-path" for every possibly CPU > >>> hot-plug. In this case, it shows you can plug an > >>> IvyBridge-IBRS-x86_64-cpu into socket 1, and the additional > >>> properties you need to pass to device_add for that. > > not really sure my English (CCed Eric) but to match 'an object' with > > the rest of sentence: > > > > It returns an object without a "qom-path" for a possible to hot-plug CPU. > > + > > In this case, it shows you can plug an IvyBridge-IBRS-x86_64-cpu > > into socket 1/core = 0/thread 0, where 'props' list describes > > additional properties you need to pass to device_add for hot-pluging > > that CPU. > > Maybe: > > The command returns an object for CPUs that are present (containing a > "qom-path" member) or which may be hot-plugged (no "qom-path" member). > In this example, an IvyBridge-IBRS-x86_64-cpu is present in socket 0, > while hot-plugging a CPU into socket 1 requires passing the listed > properties to device_add. to be precise it's a logical cpu in socket/core/thread, but considering example has only 2 socket and 2 cpus total, suggested variant probably is good too.
diff --git a/docs/cpu-hotplug.rst b/docs/cpu-hotplug.rst new file mode 100644 index 0000000000..8f9e63a9f7 --- /dev/null +++ b/docs/cpu-hotplug.rst @@ -0,0 +1,140 @@ +=================== +Virtual CPU hotplug +=================== + +A complete example of vCPU hotplug (and hot-unplug) using QMP +``device_add`` and ``device_del``. + +vCPU hotplug +------------ + +(1) Launch QEMU as follows (note that the "maxcpus" is mandatory to + allow vCPU hotplug):: + + $ qemu-system-x86_64 -display none -no-user-config -m 2048 \ + -nodefaults -monitor stdio -machine pc,accel=kvm,usb=off \ + -smp 1,maxcpus=2 -cpu IvyBridge-IBRS \ + -blockdev node-name=node-Base,driver=qcow2,file.driver=file,file.filename=./base.qcow2 \ + -device virtio-blk,drive=node-Base,id=virtio0 -qmp unix:/tmp/qmp-sock,server,nowait + +(2) Run 'qmp-shell' (located in the source tree) to connect to the + just-launched QEMU:: + + $> ./qmp/qmp-shell -p -v /tmp/qmp-sock + [...] + (QEMU) + +(3) Check which socket is free to allow hotplugging a CPU:: + + (QEMU) query-hotpluggable-cpus + { + "execute": "query-hotpluggable-cpus", + "arguments": {} + } + { + "return": [ + { + "type": "IvyBridge-IBRS-x86_64-cpu", + "vcpus-count": 1, + "props": { + "socket-id": 1, + "core-id": 0, + "thread-id": 0 + } + }, + { + "qom-path": "/machine/unattached/device[0]", + "type": "IvyBridge-IBRS-x86_64-cpu", + "vcpus-count": 1, + "props": { + "socket-id": 0, + "core-id": 0, + "thread-id": 0 + } + } + ] + } + (QEMU) + +(4) We can see that socket 1 is free, so use `device_add` to hotplug + "IvyBridge-IBRS-x86_64-cpu":: + + (QEMU) device_add id=cpu-2 driver=IvyBridge-IBRS-x86_64-cpu socket-id=1 core-id=0 thread-id=0 + { + "execute": "device_add", + "arguments": { + "socket-id": 1, + "driver": "IvyBridge-IBRS-x86_64-cpu", + "id": "cpu-2", + "core-id": 0, + "thread-id": 0 + } + } + { + "return": {} + } + (QEMU) + +(5) Optionally, run QMP `query-cpus-fast` for some details about the + vCPUs:: + + (QEMU) query-cpus-fast + { + "execute": "query-cpus-fast", + "arguments": {} + } + { + "return": [ + { + "qom-path": "/machine/unattached/device[0]", + "target": "x86_64", + "thread-id": 11534, + "cpu-index": 0, + "props": { + "socket-id": 0, + "core-id": 0, + "thread-id": 0 + }, + "arch": "x86" + }, + { + "qom-path": "/machine/peripheral/cpu-2", + "target": "x86_64", + "thread-id": 12106, + "cpu-index": 1, + "props": { + "socket-id": 1, + "core-id": 0, + "thread-id": 0 + }, + "arch": "x86" + } + ] + } + (QEMU) + + +vCPU hot-unplug +--------------- + +From the 'qmp-shell', invoke the QMP ``device_del`` command:: + + (QEMU) device_del id=cpu-2 + { + "execute": "device_del", + "arguments": { + "id": "cpu-2" + } + } + { + "return": {} + } + (QEMU) + +.. note:: + vCPU hot-unplug requires guest cooperation; so the ``device_del`` + command above does not guarantee vCPU removal -- it's a "request to + unplug". At this point, the guest will get a System Control + Interupt (SCI) and calls the ACPI handler for the affected vCPU + device. Then the guest kernel will bring the vCPU offline and tells + QEMU to unplug it.
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com> --- docs/cpu-hotplug.rst | 140 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+) create mode 100644 docs/cpu-hotplug.rst