[22/22] machine: introduce -machine-def option to define a machine via config

Since we have MachineCore and can represent a machine entirely via default
options, we can introduce a new option that let's us dynamically register a
machine based on those options.

For instance, we could add the following to target-x86_64.conf:

[machine-def]
 name = "pc-0.11"
 desc = "Standard PC"
 acpi = "on"
 pci = "on"
 cpu = "qemu64"
 max_cpus = "255"
 virtio-blk-pci.vectors = "0"
 virtio-serial-pci.max_nr_ports = "1"
 virtio-serial-pci.vectors = "0"
 ide-drive.ver = "0.11"
 scsi-disk.ver = "0.11"
 PCI.rombar = "0"

What's really exciting, is that a user can then define their own machines
that better suite their desires:

[kvmpc]
 name = "kvmpc"
 accel = "kvm|tcg"
 ram_size = "512M"
 max_cpus = "64"
 sockets = "16"
 default_drive = "virtio"

I'd eventually like to move all PC compatibility machines to the default
config but for now, I wanted to keep this simple.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

On 06/07/2010 06:52 PM, Anthony Liguori wrote:
> Since we have MachineCore and can represent a machine entirely via default
> options, we can introduce a new option that let's us dynamically register a
> machine based on those options.
>
> For instance, we could add the following to target-x86_64.conf:
>
> [machine-def]
>   name = "pc-0.11"
>   desc = "Standard PC"
>   acpi = "on"
>   pci = "on"
>   cpu = "qemu64"
>   max_cpus = "255"
>   virtio-blk-pci.vectors = "0"
>   virtio-serial-pci.max_nr_ports = "1"
>   virtio-serial-pci.vectors = "0"
>   ide-drive.ver = "0.11"
>   scsi-disk.ver = "0.11"
>   PCI.rombar = "0"
>
> What's really exciting, is that a user can then define their own machines
> that better suite their desires:
>
> [kvmpc]
>   name = "kvmpc"
>   accel = "kvm|tcg"
>   ram_size = "512M"
>   max_cpus = "64"
>   sockets = "16"
>   default_drive = "virtio"
>
> I'd eventually like to move all PC compatibility machines to the default
> config but for now, I wanted to keep this simple.
>
> Signed-off-by: Anthony Liguori<aliguori@us.ibm.com>
>    

 From the perspective of a tool like libvirt, I think there are a couple 
ways it could handle something like this and I think it's worth 
discussing the options.

Assume we move all the compat machine definitions into a config file, 
since libvirt presumably uses -nodefconfig today, it could simply 
include it's own machine definitions for each qemu version based on the 
definitions we ship.  That makes sure that the definition is always 
static for libvirt.

Another option would be for libvirt to not use -nodefconfig, and instead 
to let the user's global configs be read.  libvirt would then read the 
config file from the running qemu instance to sync it's state up.

The later option is a bit more work up front but longer term, I think it 
addresses a couple things nicely.  It provides a way for a user 
specified config to co-exist with libvirt.  It also let's tools tweak 
power config options in a way that's compatible with libvirt.

If libvirt can embed the qemu config description in its own XML, then 
there is no problem for libvirt to recreate the system on a different 
box even if the global configuration is different.

Regards,

Anthony Liguori

> diff --git a/qemu-config.c b/qemu-config.c
> index a0cb34f..9b5415d 100644
> --- a/qemu-config.c
> +++ b/qemu-config.c
> @@ -345,6 +345,15 @@ QemuOptsList qemu_machine_opts = {
>       },
>   };
>
> +QemuOptsList qemu_machine_def_opts = {
> +    .name = "machine-def",
> +    .implied_opt_name = "name",
> +    .head = QTAILQ_HEAD_INITIALIZER(qemu_machine_def_opts.head),
> +    .desc = {
> +        { /* end of list */ }
> +    },
> +};
> +
>   static QemuOptsList *vm_config_groups[] = {
>       &qemu_drive_opts,
>       &qemu_chardev_opts,
> @@ -356,6 +365,7 @@ static QemuOptsList *vm_config_groups[] = {
>       &qemu_mon_opts,
>       &qemu_cpudef_opts,
>       &qemu_machine_opts,
> +&qemu_machine_def_opts,
>       NULL,
>   };
>
> diff --git a/qemu-config.h b/qemu-config.h
> index 6f52188..1b7324c 100644
> --- a/qemu-config.h
> +++ b/qemu-config.h
> @@ -15,6 +15,7 @@ extern QemuOptsList qemu_global_opts;
>   extern QemuOptsList qemu_mon_opts;
>   extern QemuOptsList qemu_cpudef_opts;
>   extern QemuOptsList qemu_machine_opts;
> +extern QemuOptsList qemu_machine_def_opts;
>
>   QemuOptsList *qemu_find_opts(const char *group);
>   int qemu_set_option(const char *str);
> diff --git a/qemu-options.hx b/qemu-options.hx
> index 32fd32a..2bafe22 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -43,6 +43,14 @@ STEXI
>   Select the emulated @var{machine}
>   ETEXI
>
> +DEF("machine-def", HAS_ARG, QEMU_OPTION_machine_def,
> +    "-machine-def name[,opt=val...] define a new machine\n", QEMU_ARCH_ALL)
> +STEXI
> +@item -machine-def @var{name}[,@var{opt}=@var{val}...]
> +@findex -machine-def
> +Define a new machine
> +ETEXI
> +
>   DEF("cpu", HAS_ARG, QEMU_OPTION_cpu,
>       "-cpu cpu        select CPU (-cpu ? for list)\n", QEMU_ARCH_ALL)
>   STEXI
> diff --git a/vl.c b/vl.c
> index 150dd41..7797187 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -1683,6 +1683,59 @@ void machine_set_default(const char *name)
>       default_machine = name;
>   }
>
> +typedef struct MachineDefineHelper
> +{
> +    QemuOptValue *defaults;
> +    int length;
> +    int capacity;
> +} MachineDefineHelper;
> +
> +static void helper_grow(MachineDefineHelper *helper)
> +{
> +    if ((helper->capacity - helper->length)<  2) {
> +        helper->capacity += 100;
> +        helper->defaults = qemu_realloc(helper->defaults,
> +                                        helper->capacity * sizeof(QemuOptValue));
> +    }
> +}
> +
> +static int machine_define_prop(const char *name, const char *value, void *opaque)
> +{
> +    MachineDefineHelper *helper = opaque;
> +    QemuOptValue *v;
> +
> +    if (strcmp(name, "core") == 0) {
> +        return 0;
> +    }
> +
> +    helper_grow(helper);
> +    v =&helper->defaults[helper->length++];
> +    v->name = qemu_strdup(name);
> +    v->value = qemu_strdup(value);
> +
> +    return 0;
> +}
> +
> +static int machine_define(QemuOpts *opts, void *opaque)
> +{
> +    MachineDefineHelper *helper;
> +    const char *core;
> +
> +    core = qemu_opt_get(opts, "core");
> +    if (!core) {
> +        fprintf(stderr, "machine-def: No core specified\n");
> +        return -1;
> +    }
> +
> +    helper = qemu_mallocz(sizeof(*helper));
> +    qemu_opt_foreach(opts, machine_define_prop, helper, 1);
> +    helper->defaults[helper->length].name = NULL;
> +    machine_create_from_core(core, helper->defaults);
> +    qemu_free(helper);
> +
> +    return 0;
> +}
> +
>   static Machine *find_machine(const char *name)
>   {
>       Machine *m;
> @@ -2812,6 +2865,7 @@ int main(int argc, char **argv, char **envp)
>           }
>       }
>       cpudef_init();
> +    qemu_opts_foreach(&qemu_machine_def_opts, machine_define, NULL, 1);
>
>       /* second pass of option parsing */
>       optind = 1;
> @@ -2853,6 +2907,14 @@ int main(int argc, char **argv, char **envp)
>                       exit(1);
>                   }
>                   break;
> +            case QEMU_OPTION_machine_def:
> +                opts = qemu_opts_parse(&qemu_machine_def_opts, optarg, 1);
> +                if (!opts) {
> +                    exit(1);
> +                }
> +
> +                machine_define(opts, NULL);
> +                break;
>               case QEMU_OPTION_cpu:
>                   /* hw initialization will check this */
>                   if (*optarg == '?') {
>

On Mon, Jun 07, 2010 at 07:50:14PM -0500, Anthony Liguori wrote:
> On 06/07/2010 06:52 PM, Anthony Liguori wrote:
> >Since we have MachineCore and can represent a machine entirely via default
> >options, we can introduce a new option that let's us dynamically register a
> >machine based on those options.
> >
> >For instance, we could add the following to target-x86_64.conf:
> >
> >[machine-def]
> >  name = "pc-0.11"
> >  desc = "Standard PC"
> >  acpi = "on"
> >  pci = "on"
> >  cpu = "qemu64"
> >  max_cpus = "255"
> >  virtio-blk-pci.vectors = "0"
> >  virtio-serial-pci.max_nr_ports = "1"
> >  virtio-serial-pci.vectors = "0"
> >  ide-drive.ver = "0.11"
> >  scsi-disk.ver = "0.11"
> >  PCI.rombar = "0"
> >
> >What's really exciting, is that a user can then define their own machines
> >that better suite their desires:
> >
> >[kvmpc]
> >  name = "kvmpc"
> >  accel = "kvm|tcg"
> >  ram_size = "512M"
> >  max_cpus = "64"
> >  sockets = "16"
> >  default_drive = "virtio"
> >
> >I'd eventually like to move all PC compatibility machines to the default
> >config but for now, I wanted to keep this simple.
> >
> >Signed-off-by: Anthony Liguori<aliguori@us.ibm.com>
> >   
> 
> From the perspective of a tool like libvirt, I think there are a couple 
> ways it could handle something like this and I think it's worth 
> discussing the options.
> 
> Assume we move all the compat machine definitions into a config file, 
> since libvirt presumably uses -nodefconfig today, it could simply 
> include it's own machine definitions for each qemu version based on the 
> definitions we ship.  That makes sure that the definition is always 
> static for libvirt.

Due to a screwup on my part, we don't currently use -nodefconfig
but we should be. I had originally thought '-nodefaults' turned off
all defaults, but I see it only does defaults hardware, but not
default configs. 

> Another option would be for libvirt to not use -nodefconfig, and instead 
> to let the user's global configs be read.  libvirt would then read the 
> config file from the running qemu instance to sync it's state up.

The tricky thing I'm seeing here is the scope of the stuff you can 
put in the configuration files. 

On the one had there are config options that effectively provide new 
capabilities to the QEMU binary eg new machine types, new CPU definitions.
These don't cause any trouble, since that are a complete no-op unless you
launch a guest that actually requests to make use of them eg by adding a
-M mycustommachine or  a  -cpu mycustomCPUmodel flag. A '-M pc-010' guest
will never be impacted by fact that you added some new machine types in
the global config.

On the other hand there are config options that immediately change the 
virtual hardware in all guests launched, eg if I edit the 
/etc/qemu/target-i386.conf and add

  [drive]
    if = "ide"
    file = "foo.iso"

then every single guest gets a new piece of hardware, which is what we
tried to avoid with the '-nodefaults' flag already.

> The later option is a bit more work up front but longer term, I think it 
> addresses a couple things nicely.  It provides a way for a user 
> specified config to co-exist with libvirt.  It also let's tools tweak 
> power config options in a way that's compatible with libvirt.
> 
> If libvirt can embed the qemu config description in its own XML, then 
> there is no problem for libvirt to recreate the system on a different 
> box even if the global configuration is different.

If the global config is just adding new capabilities (machine types,
cpu types, etc) I see no problem with having these loaded by default
for any libvirt guest.

When the global config can add extra hardware (eg drives) this becomes
very tricky to re-concile, which is exactly why we had '-nodefaults'
to turn off extra global hardware. 

We want all hardware libvirt knows about to be visible in the XML. 
eg, if the default config contained a [drive] section, you'd expect 
that to appear as a <disk> in libvirt XML. So if we parsed the default 
global config to sync it to the libvirt XML, when we come to launch the
guest, we have even more fun figuring out which of the disks in the XML
config needs a '-drive' on the ARGV, and which don't need any arg because
they're in the global config. To make that practical we'd need to read 
the global config, turn it into libvirt XML, and then launch the guest
with -nodefconfig and just use -drive as normal for everything. But then
we loose useful things like new machine types & cpu types :-(

Is it practical to a way to separate the global config into two global
configs. One config that is used to define extra capabilities (machine
types, cpu types, etc) that on their own are guarenteed to never impact
any existing guest config. One that is used to add default hardware 
(disks nics, etc) which clearly does impact every guest.

Then, we could let the global capabilities config be in effect at all 
times, QEMU wouldn't even need a way to turn that off. The global
hardware config could be enabled/disable as per the needs of the mgmt
app, reconciled with their config as required.

Daniel

On Thu, Jun 10, 2010 at 06:48:42PM +0100, Daniel P. Berrange wrote:
> On Mon, Jun 07, 2010 at 07:50:14PM -0500, Anthony Liguori wrote:
> > On 06/07/2010 06:52 PM, Anthony Liguori wrote:
> > >Since we have MachineCore and can represent a machine entirely via default
> > >options, we can introduce a new option that let's us dynamically register a
> > >machine based on those options.
> > >
> > >For instance, we could add the following to target-x86_64.conf:
> > >
> > >[machine-def]
> > >  name = "pc-0.11"
> > >  desc = "Standard PC"
> > >  acpi = "on"
> > >  pci = "on"
> > >  cpu = "qemu64"
> > >  max_cpus = "255"
> > >  virtio-blk-pci.vectors = "0"
> > >  virtio-serial-pci.max_nr_ports = "1"
> > >  virtio-serial-pci.vectors = "0"
> > >  ide-drive.ver = "0.11"
> > >  scsi-disk.ver = "0.11"
> > >  PCI.rombar = "0"
> > >
> > >What's really exciting, is that a user can then define their own machines
> > >that better suite their desires:
> > >
> > >[kvmpc]
> > >  name = "kvmpc"
> > >  accel = "kvm|tcg"
> > >  ram_size = "512M"
> > >  max_cpus = "64"
> > >  sockets = "16"
> > >  default_drive = "virtio"
> > >
> > >I'd eventually like to move all PC compatibility machines to the default
> > >config but for now, I wanted to keep this simple.
> > >
> > >Signed-off-by: Anthony Liguori<aliguori@us.ibm.com>
> > >   
> > 
> > From the perspective of a tool like libvirt, I think there are a couple 
> > ways it could handle something like this and I think it's worth 
> > discussing the options.
> > 
> > Assume we move all the compat machine definitions into a config file, 
> > since libvirt presumably uses -nodefconfig today, it could simply 
> > include it's own machine definitions for each qemu version based on the 
> > definitions we ship.  That makes sure that the definition is always 
> > static for libvirt.
> 
> Due to a screwup on my part, we don't currently use -nodefconfig
> but we should be. I had originally thought '-nodefaults' turned off
> all defaults, but I see it only does defaults hardware, but not
> default configs. 
> 
> > Another option would be for libvirt to not use -nodefconfig, and instead 
> > to let the user's global configs be read.  libvirt would then read the 
> > config file from the running qemu instance to sync it's state up.
> 
> The tricky thing I'm seeing here is the scope of the stuff you can 
> put in the configuration files. 
> 
> On the one had there are config options that effectively provide new 
> capabilities to the QEMU binary eg new machine types, new CPU definitions.
> These don't cause any trouble, since that are a complete no-op unless you
> launch a guest that actually requests to make use of them eg by adding a
> -M mycustommachine or  a  -cpu mycustomCPUmodel flag. A '-M pc-010' guest
> will never be impacted by fact that you added some new machine types in
> the global config.
> 
> On the other hand there are config options that immediately change the 
> virtual hardware in all guests launched, eg if I edit the 
> /etc/qemu/target-i386.conf and add
> 
>   [drive]
>     if = "ide"
>     file = "foo.iso"
> 
> then every single guest gets a new piece of hardware, which is what we
> tried to avoid with the '-nodefaults' flag already.
> 
> > The later option is a bit more work up front but longer term, I think it 
> > addresses a couple things nicely.  It provides a way for a user 
> > specified config to co-exist with libvirt.  It also let's tools tweak 
> > power config options in a way that's compatible with libvirt.
> > 
> > If libvirt can embed the qemu config description in its own XML, then 
> > there is no problem for libvirt to recreate the system on a different 
> > box even if the global configuration is different.
> 
> If the global config is just adding new capabilities (machine types,
> cpu types, etc) I see no problem with having these loaded by default
> for any libvirt guest.
> 
> When the global config can add extra hardware (eg drives) this becomes
> very tricky to re-concile, which is exactly why we had '-nodefaults'
> to turn off extra global hardware. 
> 
> We want all hardware libvirt knows about to be visible in the XML. 
> eg, if the default config contained a [drive] section, you'd expect 
> that to appear as a <disk> in libvirt XML. So if we parsed the default 
> global config to sync it to the libvirt XML, when we come to launch the
> guest, we have even more fun figuring out which of the disks in the XML
> config needs a '-drive' on the ARGV, and which don't need any arg because
> they're in the global config. To make that practical we'd need to read 
> the global config, turn it into libvirt XML, and then launch the guest
> with -nodefconfig and just use -drive as normal for everything. But then
> we loose useful things like new machine types & cpu types :-(
> 
> Is it practical to a way to separate the global config into two global
> configs. One config that is used to define extra capabilities (machine
> types, cpu types, etc) that on their own are guarenteed to never impact
> any existing guest config. One that is used to add default hardware 
> (disks nics, etc) which clearly does impact every guest.
> 
> Then, we could let the global capabilities config be in effect at all 
> times, QEMU wouldn't even need a way to turn that off. The global
> hardware config could be enabled/disable as per the needs of the mgmt
> app, reconciled with their config as required.

Actually thinking about it some more, it doesn't require a separation of
global configs. Instead we're just use my capabilities patches to query
the desired CPU & machine definitions from the global, and write them out 
to a new config and then use -nodefconfig to turn off the global config,
and -readconfig re-add just the bits of the global config we wanted.


Daniel