| Message ID | 1365172636-28628-25-git-send-email-imammedo@redhat.com |
|---|---|
| State | New |
| Headers | show |
On Fri, Apr 05, 2013 at 04:37:16PM +0200, Igor Mammedov wrote: [...] > diff --git a/qapi-schema.json b/qapi-schema.json > index db542f6..a760ed5 100644 > --- a/qapi-schema.json > +++ b/qapi-schema.json > @@ -1387,6 +1387,17 @@ > { 'command': 'cpu', 'data': {'index': 'int'} } > > ## > +# @cpu-add > +# > +# Adds CPU with specified id > +# > +# @id: cpu id of CPU to be created Can we have the semantics/constraints of "id" documented here? Is it an arbitrary ID chosen by the caller? Does it have to be the APIC ID? Does it have to be the index of the CPU in the CPU list? How the IDs of existing CPUs set using "-smp" are allocated? I am looking at the code right now to understand how this implementation works, but the documentation could contain or point to documentation on how the "id" parameter is used and interpreted. > +# > +# Returns: Nothing on success > +## > +{ 'command': 'cpu-add', 'data': {'id': 'int'} } > + > +##
On Fri, Apr 05, 2013 at 02:10:54PM -0300, Eduardo Habkost wrote: > On Fri, Apr 05, 2013 at 04:37:16PM +0200, Igor Mammedov wrote: > [...] > > diff --git a/qapi-schema.json b/qapi-schema.json > > index db542f6..a760ed5 100644 > > --- a/qapi-schema.json > > +++ b/qapi-schema.json > > @@ -1387,6 +1387,17 @@ > > { 'command': 'cpu', 'data': {'index': 'int'} } > > > > ## > > +# @cpu-add > > +# > > +# Adds CPU with specified id > > +# > > +# @id: cpu id of CPU to be created > > Can we have the semantics/constraints of "id" documented here? Is it an > arbitrary ID chosen by the caller? Does it have to be the APIC ID? Does > it have to be the index of the CPU in the CPU list? How the IDs of > existing CPUs set using "-smp" are allocated? > > I am looking at the code right now to understand how this implementation > works, but the documentation could contain or point to documentation on > how the "id" parameter is used and interpreted. So, the answer to my own question seems to be on patch 21/22. Let me check if I understand this correctly: @id: cpu id of the CPU to be created. It must be one of the the available IDs listed at the "/machine/icc-bridge/cpu[0..N]" links. Is the above correct? Now, my question is: suppose a caller starts QEMU as: $ qemu -smp 18,cores=3,threads=3,maxcpus=36 \ -numa node,cpus=0-8 -numa node,cpus=9-17 \ -numa node,cpus=18-26 -numa node,cpus=27-35 and the caller wants to hot-add all VCPUs in the last CPU socket (that means: all the VCPUs in the last NUMA node, that means: CPU indexes 27-35). What should the caller do to find out which of the /machine/icc-bridge/cpu[0..N] links/IDs really correspond to those VCPUs? > > > +# > > +# Returns: Nothing on success > > +## > > +{ 'command': 'cpu-add', 'data': {'id': 'int'} } > > + > > +## > -- > Eduardo >
On Fri, 5 Apr 2013 14:10:54 -0300 Eduardo Habkost <ehabkost@redhat.com> wrote: > On Fri, Apr 05, 2013 at 04:37:16PM +0200, Igor Mammedov wrote: > [...] > > diff --git a/qapi-schema.json b/qapi-schema.json > > index db542f6..a760ed5 100644 > > --- a/qapi-schema.json > > +++ b/qapi-schema.json > > @@ -1387,6 +1387,17 @@ > > { 'command': 'cpu', 'data': {'index': 'int'} } > > > > ## > > +# @cpu-add > > +# > > +# Adds CPU with specified id > > +# > > +# @id: cpu id of CPU to be created > > Can we have the semantics/constraints of "id" documented here? Is it an > arbitrary ID chosen by the caller? Does it have to be the APIC ID? Does it's generic function so documenting it as APIC ID is not appropriate. I for sure should document it on cpu-hotplug wiki page though, for x86 use case for starters. i.e. how to use QMP to get a list of available/free IDs. and in which order to use them. > it have to be the index of the CPU in the CPU list? How the IDs of > existing CPUs set using "-smp" are allocated? With current -smp implementation the same way as it was before, and for migration to work hot-plugged CPU has to be the next unused APIC ID in their sequence, so that target qemu could be started with "-smp n+1". But -smp along with -numa should be reworked to allow specifying guest visible CPU IDs for arbitrary CPU hotplug to work. when we done with QOMifying CPUs it might be possible to use -device for them and keeping -smp for compat/shorcut purposes. > > I am looking at the code right now to understand how this implementation > works, but the documentation could contain or point to documentation on > how the "id" parameter is used and interpreted. I'll add pointer to wiki and describe there target-i386 use-case. > > > +# > > +# Returns: Nothing on success > > +## > > +{ 'command': 'cpu-add', 'data': {'id': 'int'} } > > + > > +## > -- > Eduardo
On Tue, Apr 09, 2013 at 10:19:11PM +0200, Igor Mammedov wrote: > On Fri, 5 Apr 2013 14:10:54 -0300 > Eduardo Habkost <ehabkost@redhat.com> wrote: > > > On Fri, Apr 05, 2013 at 04:37:16PM +0200, Igor Mammedov wrote: > > [...] > > > diff --git a/qapi-schema.json b/qapi-schema.json > > > index db542f6..a760ed5 100644 > > > --- a/qapi-schema.json > > > +++ b/qapi-schema.json > > > @@ -1387,6 +1387,17 @@ > > > { 'command': 'cpu', 'data': {'index': 'int'} } > > > > > > ## > > > +# @cpu-add > > > +# > > > +# Adds CPU with specified id > > > +# > > > +# @id: cpu id of CPU to be created > > > > Can we have the semantics/constraints of "id" documented here? Is it an > > arbitrary ID chosen by the caller? Does it have to be the APIC ID? Does > it's generic function so documenting it as APIC ID is not appropriate. > > I for sure should document it on cpu-hotplug wiki page though, for x86 use > case for starters. i.e. how to use QMP to get a list of available/free IDs. > and in which order to use them. > > > it have to be the index of the CPU in the CPU list? How the IDs of > > existing CPUs set using "-smp" are allocated? > With current -smp implementation the same way as it was before, > and for migration to work hot-plugged CPU has to be the next unused APIC > ID in their sequence, so that target qemu could be started with "-smp n+1". The problem is that it's hard to find out what's the APIC ID for each CPU mentioned in the command-line. For example, if you use "-smp 18,cores=3,threads=3,maxcpus=36" thread ID will use 2 bits, core ID will use 2 bits, the APIC IDs on startup will be: online on startup: package 0, core 0: 0 1 2 package 0, core 1: 4 5 6 package 0, core 2: 8 9 10 package 1, core 0: 16 17 18 package 1, core 1: 20 21 22 package 1, core 2: 24 25 26 offline on startup: package 2, core 0: 32 33 34 package 2, core 1: 36 37 38 package 2, core 2: 40 41 42 package 3, core 0: 48 49 50 package 3, core 1: 52 53 54 package 3, core 2: 56 57 58 What should the caller do to find out the correct ID for each of the 36 VCPUs? This should be clearly documented. > > But -smp along with -numa should be reworked to allow specifying guest visible > CPU IDs for arbitrary CPU hotplug to work. I'm curious how you plan to make this work while keeping command-line compatibility. See the question I sent on my other message, about how to map the IDs used on -numa (that are "CPU indexes") to the IDs required by cpu-add. > > when we done with QOMifying CPUs it might be possible to use -device for them > and keeping -smp for compat/shorcut purposes. > > > > > I am looking at the code right now to understand how this implementation > > works, but the documentation could contain or point to documentation on > > how the "id" parameter is used and interpreted. > I'll add pointer to wiki and describe there target-i386 use-case. Thanks! Could you try to document it succintly inside qapi-schema.json as well? Maybe just a pointer to other documents would be useful.
On Tue, 9 Apr 2013 17:46:21 -0300 Eduardo Habkost <ehabkost@redhat.com> wrote: > On Tue, Apr 09, 2013 at 10:19:11PM +0200, Igor Mammedov wrote: > > On Fri, 5 Apr 2013 14:10:54 -0300 > > Eduardo Habkost <ehabkost@redhat.com> wrote: > > > > > On Fri, Apr 05, 2013 at 04:37:16PM +0200, Igor Mammedov wrote: > > > [...] > > > > diff --git a/qapi-schema.json b/qapi-schema.json > > > > index db542f6..a760ed5 100644 > > > > --- a/qapi-schema.json > > > > +++ b/qapi-schema.json > > > > @@ -1387,6 +1387,17 @@ > > > > { 'command': 'cpu', 'data': {'index': 'int'} } > > > > > > > > ## > > > > +# @cpu-add > > > > +# > > > > +# Adds CPU with specified id > > > > +# > > > > +# @id: cpu id of CPU to be created > > > > > > Can we have the semantics/constraints of "id" documented here? Is it an > > > arbitrary ID chosen by the caller? Does it have to be the APIC ID? Does > > it's generic function so documenting it as APIC ID is not appropriate. > > > > I for sure should document it on cpu-hotplug wiki page though, for x86 use > > case for starters. i.e. how to use QMP to get a list of available/free IDs. > > and in which order to use them. > > > > > it have to be the index of the CPU in the CPU list? How the IDs of > > > existing CPUs set using "-smp" are allocated? > > With current -smp implementation the same way as it was before, > > and for migration to work hot-plugged CPU has to be the next unused APIC > > ID in their sequence, so that target qemu could be started with "-smp n+1". > > The problem is that it's hard to find out what's the APIC ID for each > CPU mentioned in the command-line. > > For example, if you use "-smp 18,cores=3,threads=3,maxcpus=36" thread ID > will use 2 bits, core ID will use 2 bits, the APIC IDs on startup will > be: > > online on startup: > package 0, core 0: 0 1 2 > package 0, core 1: 4 5 6 > package 0, core 2: 8 9 10 > package 1, core 0: 16 17 18 > package 1, core 1: 20 21 22 > package 1, core 2: 24 25 26 > > offline on startup: > package 2, core 0: 32 33 34 > package 2, core 1: 36 37 38 > package 2, core 2: 40 41 42 > package 3, core 0: 48 49 50 > package 3, core 1: 52 53 54 > package 3, core 2: 56 57 58 > > > What should the caller do to find out the correct ID for each of the 36 > VCPUs? This should be clearly documented. Patch 21/22 exposes all APIC IDs (including offline) as links via QOM as /machine/icc-bridge/cpu[0..n] And since in above sequence IDs are monotonously increasing it's enough to use the next unused APIC ID to make -smp n+1 work. > > > > > > But -smp along with -numa should be reworked to allow specifying guest visible > > CPU IDs for arbitrary CPU hotplug to work. > > I'm curious how you plan to make this work while keeping command-line > compatibility. See the question I sent on my other message, about how to > map the IDs used on -numa (that are "CPU indexes") to the IDs required > by cpu-add. It doesn't mean that we should stick to bad/insufficient interface forever. We could add new one that does it right and keep old one for a time being for compatibility. > > > > > when we done with QOMifying CPUs it might be possible to use -device for them > > and keeping -smp for compat/shorcut purposes. > > > > > > > > I am looking at the code right now to understand how this implementation > > > works, but the documentation could contain or point to documentation on > > > how the "id" parameter is used and interpreted. > > I'll add pointer to wiki and describe there target-i386 use-case. > > Thanks! Could you try to document it succintly inside qapi-schema.json > as well? Maybe just a pointer to other documents would be useful. It's very target specific, so separate document probably would make more sense. > -- > Eduardo >
On Tue, Apr 09, 2013 at 11:05:26PM +0200, Igor Mammedov wrote: > On Tue, 9 Apr 2013 17:46:21 -0300 > Eduardo Habkost <ehabkost@redhat.com> wrote: > > > On Tue, Apr 09, 2013 at 10:19:11PM +0200, Igor Mammedov wrote: > > > On Fri, 5 Apr 2013 14:10:54 -0300 > > > Eduardo Habkost <ehabkost@redhat.com> wrote: > > > > > > > On Fri, Apr 05, 2013 at 04:37:16PM +0200, Igor Mammedov wrote: > > > > [...] > > > > > diff --git a/qapi-schema.json b/qapi-schema.json > > > > > index db542f6..a760ed5 100644 > > > > > --- a/qapi-schema.json > > > > > +++ b/qapi-schema.json > > > > > @@ -1387,6 +1387,17 @@ > > > > > { 'command': 'cpu', 'data': {'index': 'int'} } > > > > > > > > > > ## > > > > > +# @cpu-add > > > > > +# > > > > > +# Adds CPU with specified id > > > > > +# > > > > > +# @id: cpu id of CPU to be created > > > > > > > > Can we have the semantics/constraints of "id" documented here? Is it an > > > > arbitrary ID chosen by the caller? Does it have to be the APIC ID? Does > > > it's generic function so documenting it as APIC ID is not appropriate. > > > > > > I for sure should document it on cpu-hotplug wiki page though, for x86 use > > > case for starters. i.e. how to use QMP to get a list of available/free IDs. > > > and in which order to use them. > > > > > > > it have to be the index of the CPU in the CPU list? How the IDs of > > > > existing CPUs set using "-smp" are allocated? > > > With current -smp implementation the same way as it was before, > > > and for migration to work hot-plugged CPU has to be the next unused APIC > > > ID in their sequence, so that target qemu could be started with "-smp n+1". > > > > The problem is that it's hard to find out what's the APIC ID for each > > CPU mentioned in the command-line. > > > > For example, if you use "-smp 18,cores=3,threads=3,maxcpus=36" thread ID > > will use 2 bits, core ID will use 2 bits, the APIC IDs on startup will > > be: > > > > online on startup: > > package 0, core 0: 0 1 2 > > package 0, core 1: 4 5 6 > > package 0, core 2: 8 9 10 > > package 1, core 0: 16 17 18 > > package 1, core 1: 20 21 22 > > package 1, core 2: 24 25 26 > > > > offline on startup: > > package 2, core 0: 32 33 34 > > package 2, core 1: 36 37 38 > > package 2, core 2: 40 41 42 > > package 3, core 0: 48 49 50 > > package 3, core 1: 52 53 54 > > package 3, core 2: 56 57 58 > > > > > > What should the caller do to find out the correct ID for each of the 36 > > VCPUs? This should be clearly documented. > Patch 21/22 exposes all APIC IDs (including offline) as links via QOM > as /machine/icc-bridge/cpu[0..n] > And since in above sequence IDs are monotonously increasing it's enough to use > the next unused APIC ID to make -smp n+1 work. This can be one method to map CPU indexes to APIC IDs, yes. I find it hard to explain and hard to use, and it imposes constraints on the way the target-specific IDs are calculated (requiring them to be monotonically increasing). But it may be a reasonable solution by now. I have one additional question about the icc-bridge paths: are the link paths on icc-bridge explicitly documented/required to be APIC IDs, or the caller should assume they are arbitrary IDs with no specific meaning? > > > > > > > > > > But -smp along with -numa should be reworked to allow specifying guest visible > > > CPU IDs for arbitrary CPU hotplug to work. > > > > I'm curious how you plan to make this work while keeping command-line > > compatibility. See the question I sent on my other message, about how to > > map the IDs used on -numa (that are "CPU indexes") to the IDs required > > by cpu-add. > It doesn't mean that we should stick to bad/insufficient interface forever. > We could add new one that does it right and keep old one for a time being for > compatibility. The problem is that I only see three possible kinds of CPU identifiers that could work in the command-line: * Arbitrary user-defined IDs * CPU indexes (the current interface) * Topology-based identifiers/paths (e.g. "/machine/numa_node[0]/cpu_socket[1]/core[2]/thread[1]") I don't think we can asily use APIC IDs on the command-line because the caller simply doesn't know what will be the APIC ID for each VCPU. But this is a problem we can discuss and solve later. By now, we are stuck with the legacy CPU-index-based interfaces. > > > > > > > > > when we done with QOMifying CPUs it might be possible to use -device for them > > > and keeping -smp for compat/shorcut purposes. > > > > > > > > > > > I am looking at the code right now to understand how this implementation > > > > works, but the documentation could contain or point to documentation on > > > > how the "id" parameter is used and interpreted. > > > I'll add pointer to wiki and describe there target-i386 use-case. > > > > Thanks! Could you try to document it succintly inside qapi-schema.json > > as well? Maybe just a pointer to other documents would be useful. > It's very target specific, so separate document probably would make more sense. The IDs are target-specific, but do we really need to make the interface specification/usage to be target-specific? We could have a target-independent interface to find out what are the available/valid IDs to use on cpu-add.
On Fri, 5 Apr 2013 14:24:35 -0300 Eduardo Habkost <ehabkost@redhat.com> wrote: > On Fri, Apr 05, 2013 at 02:10:54PM -0300, Eduardo Habkost wrote: > > On Fri, Apr 05, 2013 at 04:37:16PM +0200, Igor Mammedov wrote: > > [...] > > > diff --git a/qapi-schema.json b/qapi-schema.json > > > index db542f6..a760ed5 100644 > > > --- a/qapi-schema.json > > > +++ b/qapi-schema.json > > > @@ -1387,6 +1387,17 @@ > > > { 'command': 'cpu', 'data': {'index': 'int'} } > > > > > > ## > > > +# @cpu-add > > > +# > > > +# Adds CPU with specified id > > > +# > > > +# @id: cpu id of CPU to be created > > > > Can we have the semantics/constraints of "id" documented here? Is it an > > arbitrary ID chosen by the caller? Does it have to be the APIC ID? Does > > it have to be the index of the CPU in the CPU list? How the IDs of > > existing CPUs set using "-smp" are allocated? > > > > I am looking at the code right now to understand how this implementation > > works, but the documentation could contain or point to documentation on > > how the "id" parameter is used and interpreted. > > So, the answer to my own question seems to be on patch 21/22. Let me > check if I understand this correctly: > > @id: cpu id of the CPU to be created. It must be one of the the > available IDs listed at the "/machine/icc-bridge/cpu[0..N]" links. > > Is the above correct? yes. > > Now, my question is: suppose a caller starts QEMU as: > > $ qemu -smp 18,cores=3,threads=3,maxcpus=36 \ > -numa node,cpus=0-8 -numa node,cpus=9-17 \ > -numa node,cpus=18-26 -numa node,cpus=27-35 > > and the caller wants to hot-add all VCPUs in the last CPU socket (that > means: all the VCPUs in the last NUMA node, that means: CPU indexes > 27-35). What should the caller do to find out which of the > /machine/icc-bridge/cpu[0..N] links/IDs really correspond to those > VCPUs? If one wants plug in a specific CPU, It won't work with -numa yet. to make it work we need to specify sockets on -numa cmd. line and then construct synthetic QOM containers tree that could looklike: /machine/numa_nodes/[0..N]/sockets/[0..M]/cpus/[0..X] then command line could look like: $ qemu -smp 18,cores=3,threads=3,maxcpus=36 \ -numa node,sockets=0-1 -numa node,sockets=2-3 > > > > > > > +# > > > +# Returns: Nothing on success > > > +## > > > +{ 'command': 'cpu-add', 'data': {'id': 'int'} } > > > + > > > +## > > -- > > Eduardo > > > > -- > Eduardo >
On Thu, 11 Apr 2013 12:12:37 -0300 Eduardo Habkost <ehabkost@redhat.com> wrote: > On Tue, Apr 09, 2013 at 11:05:26PM +0200, Igor Mammedov wrote: > > On Tue, 9 Apr 2013 17:46:21 -0300 > > Eduardo Habkost <ehabkost@redhat.com> wrote: > > > > > On Tue, Apr 09, 2013 at 10:19:11PM +0200, Igor Mammedov wrote: > > > > On Fri, 5 Apr 2013 14:10:54 -0300 > > > > Eduardo Habkost <ehabkost@redhat.com> wrote: > > > > > > > > > On Fri, Apr 05, 2013 at 04:37:16PM +0200, Igor Mammedov wrote: > > > > > [...] > > > > > > diff --git a/qapi-schema.json b/qapi-schema.json > > > > > > index db542f6..a760ed5 100644 > > > > > > --- a/qapi-schema.json > > > > > > +++ b/qapi-schema.json > > > > > > @@ -1387,6 +1387,17 @@ > > > > > > { 'command': 'cpu', 'data': {'index': 'int'} } > > > > > > > > > > > > ## > > > > > > +# @cpu-add > > > > > > +# > > > > > > +# Adds CPU with specified id > > > > > > +# > > > > > > +# @id: cpu id of CPU to be created > > > > > > > > > > Can we have the semantics/constraints of "id" documented here? Is it an > > > > > arbitrary ID chosen by the caller? Does it have to be the APIC ID? Does > > > > it's generic function so documenting it as APIC ID is not appropriate. > > > > > > > > I for sure should document it on cpu-hotplug wiki page though, for x86 use > > > > case for starters. i.e. how to use QMP to get a list of available/free IDs. > > > > and in which order to use them. > > > > > > > > > it have to be the index of the CPU in the CPU list? How the IDs of > > > > > existing CPUs set using "-smp" are allocated? > > > > With current -smp implementation the same way as it was before, > > > > and for migration to work hot-plugged CPU has to be the next unused APIC > > > > ID in their sequence, so that target qemu could be started with "-smp n+1". > > > > > > The problem is that it's hard to find out what's the APIC ID for each > > > CPU mentioned in the command-line. > > > > > > For example, if you use "-smp 18,cores=3,threads=3,maxcpus=36" thread ID > > > will use 2 bits, core ID will use 2 bits, the APIC IDs on startup will > > > be: > > > > > > online on startup: > > > package 0, core 0: 0 1 2 > > > package 0, core 1: 4 5 6 > > > package 0, core 2: 8 9 10 > > > package 1, core 0: 16 17 18 > > > package 1, core 1: 20 21 22 > > > package 1, core 2: 24 25 26 > > > > > > offline on startup: > > > package 2, core 0: 32 33 34 > > > package 2, core 1: 36 37 38 > > > package 2, core 2: 40 41 42 > > > package 3, core 0: 48 49 50 > > > package 3, core 1: 52 53 54 > > > package 3, core 2: 56 57 58 > > > > > > > > > What should the caller do to find out the correct ID for each of the 36 > > > VCPUs? This should be clearly documented. > > Patch 21/22 exposes all APIC IDs (including offline) as links via QOM > > as /machine/icc-bridge/cpu[0..n] > > And since in above sequence IDs are monotonously increasing it's enough to use > > the next unused APIC ID to make -smp n+1 work. > > This can be one method to map CPU indexes to APIC IDs, yes. I find it > hard to explain and hard to use, and it imposes constraints on the way > the target-specific IDs are calculated (requiring them to be > monotonically increasing). But it may be a reasonable solution by now. Arbitrary CPU hotplug + migration, require ability to specify which CPU to create on target. It's probably post CPU-unplug topic or might be fixed with it. > > I have one additional question about the icc-bridge paths: are the link > paths on icc-bridge explicitly documented/required to be APIC IDs, or > the caller should assume they are arbitrary IDs with no specific > meaning? since it's on icc-bridge, they are APIC IDs, but from user's POV I wouldn't care and treat it as opaque. If we do it in a generic way, then I'll say they are arbitrary IDs. > > > > > > > > > > > > > > > But -smp along with -numa should be reworked to allow specifying guest visible > > > > CPU IDs for arbitrary CPU hotplug to work. > > > > > > I'm curious how you plan to make this work while keeping command-line > > > compatibility. See the question I sent on my other message, about how to > > > map the IDs used on -numa (that are "CPU indexes") to the IDs required > > > by cpu-add. > > It doesn't mean that we should stick to bad/insufficient interface forever. > > We could add new one that does it right and keep old one for a time being for > > compatibility. > > The problem is that I only see three possible kinds of CPU identifiers > that could work in the command-line: > > * Arbitrary user-defined IDs > * CPU indexes (the current interface) > * Topology-based identifiers/paths (e.g. > "/machine/numa_node[0]/cpu_socket[1]/core[2]/thread[1]") +1 to topology, others will allow to specify bad configuration > > I don't think we can asily use APIC IDs on the command-line because the > caller simply doesn't know what will be the APIC ID for each VCPU. Agree, especially regarding -numa option, it should consume sockets instead. user might still potentially use opaque ID in cmd line if he will add CPUs with -device apic_id=xxx, but first he should probe qemu with the same topology and read complete topology with IDs qemu provides. > > But this is a problem we can discuss and solve later. By now, we are > stuck with the legacy CPU-index-based interfaces. > > > > > > > > > > > > > > when we done with QOMifying CPUs it might be possible to use -device for them > > > > and keeping -smp for compat/shorcut purposes. > > > > > > > > > > > > > > I am looking at the code right now to understand how this implementation > > > > > works, but the documentation could contain or point to documentation on > > > > > how the "id" parameter is used and interpreted. > > > > I'll add pointer to wiki and describe there target-i386 use-case. > > > > > > Thanks! Could you try to document it succintly inside qapi-schema.json > > > as well? Maybe just a pointer to other documents would be useful. > > It's very target specific, so separate document probably would make more sense. > > The IDs are target-specific, but do we really need to make the interface > specification/usage to be target-specific? We could have a > target-independent interface to find out what are the available/valid > IDs to use on cpu-add. Answered to this in another email about -numa and showed how ID discovery could be made in unified cross target way. > > -- > Eduardo
On Thu, Apr 11, 2013 at 05:17:29PM +0200, Igor Mammedov wrote: [...] > > > > Now, my question is: suppose a caller starts QEMU as: > > > > $ qemu -smp 18,cores=3,threads=3,maxcpus=36 \ > > -numa node,cpus=0-8 -numa node,cpus=9-17 \ > > -numa node,cpus=18-26 -numa node,cpus=27-35 > > > > and the caller wants to hot-add all VCPUs in the last CPU socket (that > > means: all the VCPUs in the last NUMA node, that means: CPU indexes > > 27-35). What should the caller do to find out which of the > > /machine/icc-bridge/cpu[0..N] links/IDs really correspond to those > > VCPUs? > If one wants plug in a specific CPU, It won't work with -numa yet. OK, documenting it as unsupported by now would be enough to me. :-) It looks like -numa is the only interface that requires CPUs to be individually identified in the command-line, right? > > to make it work we need to specify sockets on -numa cmd. line and then > construct synthetic QOM containers tree that could looklike: > > /machine/numa_nodes/[0..N]/sockets/[0..M]/cpus/[0..X] > > then command line could look like: > $ qemu -smp 18,cores=3,threads=3,maxcpus=36 \ > -numa node,sockets=0-1 -numa node,sockets=2-3 Makes sense to me. Using socket numbers is one way of making the CPU identifiers be topology-based (as mentioned on a previous message), and also make it stricter to allow only topologies that actually make sense for a "/machine/numa_nodes/.../sockets/.../threads/..." tree. > [...]
diff --git a/hw/boards.h b/hw/boards.h index 425bdc7..de8f92a 100644 --- a/hw/boards.h +++ b/hw/boards.h @@ -18,6 +18,8 @@ typedef struct QEMUMachineInitArgs { const char *cpu_model; } QEMUMachineInitArgs; +extern QEMUMachineInitArgs *machine_args; + typedef void QEMUMachineInitFunc(QEMUMachineInitArgs *args); typedef void QEMUMachineResetFunc(void); @@ -43,6 +45,7 @@ typedef struct QEMUMachine { GlobalProperty *compat_props; struct QEMUMachine *next; const char *hw_version; + void (*hot_add_cpu)(const int64_t id, Error **errp); } QEMUMachine; int qemu_register_machine(QEMUMachine *m); diff --git a/hw/i386/pc.c b/hw/i386/pc.c index 86e5365..1f94134 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -53,6 +53,7 @@ #include "qemu/bitmap.h" #include "qemu/config-file.h" #include "hw/icc_bus.h" +#include "hw/boards.h" /* debug PC/ISA interrupts */ //#define DEBUG_IRQ @@ -896,6 +897,23 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, return cpu; } +static void do_cpu_hot_add(const int64_t id, Error **errp) +{ + if (cpu_exists(id)) { + error_setg(errp, "Unable to add CPU with APIC ID: 0x%" PRIx64 + ", it already exists", id); + return; + } + + if (id >= pc_apic_id_limit(max_cpus)) { + error_setg(errp, "Unable to add CPU with APIC ID: 0x%" PRIx64 + ", max allowed: 0x%x", id, pc_apic_id_limit(max_cpus) - 1); + return; + } + + pc_new_cpu(machine_args->cpu_model, id, NULL, errp); +} + void pc_cpus_init(const char *cpu_model) { int i; @@ -910,7 +928,9 @@ void pc_cpus_init(const char *cpu_model) #else cpu_model = "qemu32"; #endif + machine_args->cpu_model = cpu_model; } + current_machine->hot_add_cpu = do_cpu_hot_add; ib = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge", TYPE_ICC_BRIDGE, NULL)); diff --git a/qapi-schema.json b/qapi-schema.json index db542f6..a760ed5 100644 --- a/qapi-schema.json +++ b/qapi-schema.json @@ -1387,6 +1387,17 @@ { 'command': 'cpu', 'data': {'index': 'int'} } ## +# @cpu-add +# +# Adds CPU with specified id +# +# @id: cpu id of CPU to be created +# +# Returns: Nothing on success +## +{ 'command': 'cpu-add', 'data': {'id': 'int'} } + +## # @memsave: # # Save a portion of guest memory to a file. diff --git a/qmp-commands.hx b/qmp-commands.hx index 1e0e11e..79b2ba7 100644 --- a/qmp-commands.hx +++ b/qmp-commands.hx @@ -407,6 +407,29 @@ Example: EQMP { + .name = "cpu-add", + .args_type = "id:i", + .mhandler.cmd_new = qmp_marshal_input_cpu_add, + }, + +SQMP +cpu-add +------- + +Adds virtual cpu + +Arguments: + +- "id": cpu id (json-int) + +Example: + +-> { "execute": "cpu-add", "arguments": { "id": 2 } } +<- { "return": {} } + +EQMP + + { .name = "memsave", .args_type = "val:l,size:i,filename:s,cpu:i?", .mhandler.cmd_new = qmp_marshal_input_memsave, diff --git a/qmp.c b/qmp.c index 55b056b..8a9fd9e 100644 --- a/qmp.c +++ b/qmp.c @@ -24,6 +24,7 @@ #include "hw/qdev.h" #include "sysemu/blockdev.h" #include "qom/qom-qobject.h" +#include "hw/boards.h" NameInfo *qmp_query_name(Error **errp) { @@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp) /* Just do nothing */ } +void qmp_cpu_add(int64_t id, Error **errp) +{ + if (current_machine->hot_add_cpu) { + current_machine->hot_add_cpu(id, errp); + } else { + error_setg(errp, "Not supported"); + } +} + #ifndef CONFIG_VNC /* If VNC support is enabled, the "true" query-vnc command is defined in the VNC subsystem */ diff --git a/stubs/do_cpu_hot_add.c b/stubs/do_cpu_hot_add.c new file mode 100644 index 0000000..1f6d7a6 --- /dev/null +++ b/stubs/do_cpu_hot_add.c @@ -0,0 +1,7 @@ +#include "qapi/error.h" +#include "sysemu/sysemu.h" + +void do_cpu_hot_add(const int64_t id, Error **errp) +{ + error_setg(errp, "Not implemented"); +} diff --git a/vl.c b/vl.c index 97f0349..b2ea58c 100644 --- a/vl.c +++ b/vl.c @@ -179,6 +179,8 @@ int main(int argc, char **argv) #define MAX_VIRTIO_CONSOLES 1 #define MAX_SCLP_CONSOLES 1 +QEMUMachineInitArgs *machine_args; + static const char *data_dir[16]; static int data_dir_idx; const char *bios_name = NULL; @@ -4310,13 +4312,15 @@ int main(int argc, char **argv, char **envp) .kernel_cmdline = kernel_cmdline, .initrd_filename = initrd_filename, .cpu_model = cpu_model }; + machine_args = &args; + current_machine = machine; + machine->init(&args); cpu_synchronize_all_post_init(); set_numa_modes(); - current_machine = machine; /* init USB devices */ if (usb_enabled(false)) {
... via current_machine->cpu_hot_add() hook called by cpu-set QMP command, for x86 target. Signed-off-by: Igor Mammedov <imammedo@redhat.com> --- v4: * merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU hot-add" patches * move notifier call to CPUCLass.realize() * add hook cpu_hot_add to QEMUMachine * make QEMUMachineInitArgs global and keep default cpu_model there v3: * it appears that 'online/offline' in cpu-set are confusing people with what command actually does and users might have to distinguish if 'offline' is not implemented by parsing error message. To simplify things replace cpu-set with cpu-add command to show more clear what command does and just add cpu-del when CPU remove is implemented. v2: * s/cpu_set/cpu-set/ * qmp doc style fix * use bool type instead of opencodding online/offline string suggested-by: Eric Blake <eblake@redhat.com> --- hw/boards.h | 3 +++ hw/i386/pc.c | 20 ++++++++++++++++++++ qapi-schema.json | 11 +++++++++++ qmp-commands.hx | 23 +++++++++++++++++++++++ qmp.c | 10 ++++++++++ stubs/do_cpu_hot_add.c | 7 +++++++ vl.c | 6 +++++- 7 files changed, 79 insertions(+), 1 deletion(-) create mode 100644 stubs/do_cpu_hot_add.c