| Message ID | 20221226053329.157905-1-het.gala@nutanix.com |
|---|---|
| Headers | show |
| Series | migration: Modified 'migrate' QAPI command for migration | expand |
On 26/12/22 11:03 am, Het Gala wrote: > Current QAPI 'migrate' command design (for initiating a migration > stream) contains information regarding different migrate transport mechanism > (tcp / unix / exec), dest-host IP address, and binding port number in form of > a string. Thus the design does seem to have some design issues. Some of the > issues, stated below are: > > 1. Use of string URIs is a data encoding scheme within a data encoding scheme. > QEMU code should directly be able to work with the results from QAPI, > without resorting to do a second level of parsing (eg. socket_parse()). > 2. For features / parameters related to migration, the migration tunables needs > to be defined and updated upfront. For example, 'migrate-set-capability' > and 'migrate-set-parameter' is required to enable multifd capability and > multifd-number of channels respectively. Instead, 'Multifd-channels' can > directly be represented as a single additional parameter to 'migrate' > QAPI. 'migrate-set-capability' and 'migrate-set-parameter' commands could > be used for runtime tunables that need setting after migration has already > started. > > The current patchset focuses on solving the first problem of multi-level > encoding of URIs. The patch defines 'migrate' command as a QAPI discriminated > union for the various transport backends (like socket, exec and rdma), and on > basis of transport backends, different migration parameters are defined. > > (uri) string --> (channel) Channel-type > Transport-type > Migration parameters based on transport type I hope all of you had nice a lovely christmas :) and wish you all a beautiful new year!! Just a gentle reminder for patch review. This patchset series focuses on the idea of modifying qemu 'migration' QAPI syntax into a well defined data structure. Hoping for suggestions and active discussions on the patchset series from all the maintainers. Regards, Het Gala
On 02/01/23 12:48 pm, Het Gala wrote: > > On 26/12/22 11:03 am, Het Gala wrote: >> Current QAPI 'migrate' command design (for initiating a migration >> stream) contains information regarding different migrate transport >> mechanism >> (tcp / unix / exec), dest-host IP address, and binding port number in >> form of >> a string. Thus the design does seem to have some design issues. Some >> of the >> issues, stated below are: >> >> 1. Use of string URIs is a data encoding scheme within a data >> encoding scheme. >> QEMU code should directly be able to work with the results from >> QAPI, >> without resorting to do a second level of parsing (eg. >> socket_parse()). >> 2. For features / parameters related to migration, the migration >> tunables needs >> to be defined and updated upfront. For example, >> 'migrate-set-capability' >> and 'migrate-set-parameter' is required to enable multifd >> capability and >> multifd-number of channels respectively. Instead, >> 'Multifd-channels' can >> directly be represented as a single additional parameter to >> 'migrate' >> QAPI. 'migrate-set-capability' and 'migrate-set-parameter' >> commands could >> be used for runtime tunables that need setting after migration >> has already >> started. >> >> The current patchset focuses on solving the first problem of multi-level >> encoding of URIs. The patch defines 'migrate' command as a QAPI >> discriminated >> union for the various transport backends (like socket, exec and >> rdma), and on >> basis of transport backends, different migration parameters are defined. >> >> (uri) string --> (channel) Channel-type >> Transport-type >> Migration parameters based on transport >> type > > I hope all of you had nice a lovely christmas :) and wish you all a > beautiful new year!! > > Just a gentle reminder for patch review. > This patchset series focuses on the idea of modifying qemu 'migration' > QAPI syntax into a well defined data structure. > Hoping for suggestions and active discussions on the patchset series > from all the maintainers. A gentle reminder once again :) This patchset could prove to the base of changing wire protocol around migration QAPIs. It could help in taking other decisions regarding restructuring around live migration code base in future. Would be glad to have some feedback on the patches from the maintainers. Waiting for a positive response from the upstream community. Regards, Het Gala
Hi, On 12/26/22 06:33, Het Gala wrote: > Current QAPI 'migrate' command design (for initiating a migration > stream) contains information regarding different migrate transport mechanism > (tcp / unix / exec), dest-host IP address, and binding port number in form of > a string. Thus the design does seem to have some design issues. Some of the > issues, stated below are: > > 1. Use of string URIs is a data encoding scheme within a data encoding scheme. > QEMU code should directly be able to work with the results from QAPI, > without resorting to do a second level of parsing (eg. socket_parse()). > 2. For features / parameters related to migration, the migration tunables needs > to be defined and updated upfront. For example, 'migrate-set-capability' > and 'migrate-set-parameter' is required to enable multifd capability and > multifd-number of channels respectively. Instead, 'Multifd-channels' can > directly be represented as a single additional parameter to 'migrate' > QAPI. 'migrate-set-capability' and 'migrate-set-parameter' commands could > be used for runtime tunables that need setting after migration has already > started. Is efficient and parallel migration to file of large VMs in scope for this design? Thanks, Claudio > > The current patchset focuses on solving the first problem of multi-level > encoding of URIs. The patch defines 'migrate' command as a QAPI discriminated > union for the various transport backends (like socket, exec and rdma), and on > basis of transport backends, different migration parameters are defined. > > (uri) string --> (channel) Channel-type > Transport-type > Migration parameters based on transport type > > ----------------------------------------------------------------------------- > > Author Het Gala (5): > migration: Updated QAPI format for 'migrate' qemu monitor command > migration: HMP side changes for modified 'migrate' QAPI design > migration: Avoid multiple parsing of uri in migration code flow > migration: Modified 'migrate-incoming' QAPI and HMP side changes on > the destination interface. > migration: Established connection for listener sockets on the dest > interface > > migration/migration.c | 133 +++++++++++++++++++++++++++++---------- > migration/socket.c | 31 +-------- > migration/socket.h | 5 +- > monitor/hmp-cmds.c | 101 ++++++++++++++++++++++++++++- > qapi/migration.json | 143 ++++++++++++++++++++++++++++++++++++++++-- > softmmu/vl.c | 2 +- > 6 files changed, 344 insertions(+), 71 deletions(-) >
On 17/01/23 4:22 pm, Claudio Fontana wrote: > Hi, > > On 12/26/22 06:33, Het Gala wrote: >> Current QAPI 'migrate' command design (for initiating a migration >> stream) contains information regarding different migrate transport mechanism >> (tcp / unix / exec), dest-host IP address, and binding port number in form of >> a string. Thus the design does seem to have some design issues. Some of the >> issues, stated below are: >> >> 1. Use of string URIs is a data encoding scheme within a data encoding scheme. >> QEMU code should directly be able to work with the results from QAPI, >> without resorting to do a second level of parsing (eg. socket_parse()). >> 2. For features / parameters related to migration, the migration tunables needs >> to be defined and updated upfront. For example, 'migrate-set-capability' >> and 'migrate-set-parameter' is required to enable multifd capability and >> multifd-number of channels respectively. Instead, 'Multifd-channels' can >> directly be represented as a single additional parameter to 'migrate' >> QAPI. 'migrate-set-capability' and 'migrate-set-parameter' commands could >> be used for runtime tunables that need setting after migration has already >> started. > Is efficient and parallel migration to file of large VMs in scope for this design? > > Thanks, > > Claudio This patch's design right now mainly focuses on revamping the design for 'migrate' command. In the upcomig patchset series in future, it will try to accomodate multifd as a feature in the same QAPI command and try to build multiple interface support on top of multifd feature. Main aim is to increase network bandwidth for migration with help of multiple interface and multifd. Regards, Het Gala.
On 1/18/23 06:52, Het Gala wrote: > > On 17/01/23 4:22 pm, Claudio Fontana wrote: >> Hi, >> >> On 12/26/22 06:33, Het Gala wrote: >>> Current QAPI 'migrate' command design (for initiating a migration >>> stream) contains information regarding different migrate transport mechanism >>> (tcp / unix / exec), dest-host IP address, and binding port number in form of >>> a string. Thus the design does seem to have some design issues. Some of the >>> issues, stated below are: >>> >>> 1. Use of string URIs is a data encoding scheme within a data encoding scheme. >>> QEMU code should directly be able to work with the results from QAPI, >>> without resorting to do a second level of parsing (eg. socket_parse()). >>> 2. For features / parameters related to migration, the migration tunables needs >>> to be defined and updated upfront. For example, 'migrate-set-capability' >>> and 'migrate-set-parameter' is required to enable multifd capability and >>> multifd-number of channels respectively. Instead, 'Multifd-channels' can >>> directly be represented as a single additional parameter to 'migrate' >>> QAPI. 'migrate-set-capability' and 'migrate-set-parameter' commands could >>> be used for runtime tunables that need setting after migration has already >>> started. >> Is efficient and parallel migration to file of large VMs in scope for this design? >> >> Thanks, >> >> Claudio > > This patch's design right now mainly focuses on revamping the design for > 'migrate' command. > > In the upcomig patchset series in future, it will try to accomodate > multifd as a feature in the same QAPI command and try to build multiple > interface support on top of multifd feature. Main aim is to increase > network bandwidth for migration with help of multiple interface and multifd. > > Regards, > Het Gala. Understand, hopefully we can make sure that we can have a design that allows also increasing disk bandwidth for direct migration to disk. Currently upstream migration to fast disks of medium to large size VMs is badly bottlenecked by qemu/libvirt interfaces. Just FYI for existing work if interested see: https://www.mail-archive.com/libvir-list@redhat.com/msg230248.html (not upstreamable, but dramatically improves VM save/restore performance) https://lists.gnu.org/archive/html/qemu-devel/2022-10/msg02870.html (attempt to address the issue in QEMU project itself via migrating to file:///). Ciao, C
Current QAPI 'migrate' command design (for initiating a migration stream) contains information regarding different migrate transport mechanism (tcp / unix / exec), dest-host IP address, and binding port number in form of a string. Thus the design does seem to have some design issues. Some of the issues, stated below are: 1. Use of string URIs is a data encoding scheme within a data encoding scheme. QEMU code should directly be able to work with the results from QAPI, without resorting to do a second level of parsing (eg. socket_parse()). 2. For features / parameters related to migration, the migration tunables needs to be defined and updated upfront. For example, 'migrate-set-capability' and 'migrate-set-parameter' is required to enable multifd capability and multifd-number of channels respectively. Instead, 'Multifd-channels' can directly be represented as a single additional parameter to 'migrate' QAPI. 'migrate-set-capability' and 'migrate-set-parameter' commands could be used for runtime tunables that need setting after migration has already started. The current patchset focuses on solving the first problem of multi-level encoding of URIs. The patch defines 'migrate' command as a QAPI discriminated union for the various transport backends (like socket, exec and rdma), and on basis of transport backends, different migration parameters are defined. (uri) string --> (channel) Channel-type Transport-type Migration parameters based on transport type ----------------------------------------------------------------------------- Author Het Gala (5): migration: Updated QAPI format for 'migrate' qemu monitor command migration: HMP side changes for modified 'migrate' QAPI design migration: Avoid multiple parsing of uri in migration code flow migration: Modified 'migrate-incoming' QAPI and HMP side changes on the destination interface. migration: Established connection for listener sockets on the dest interface migration/migration.c | 133 +++++++++++++++++++++++++++++---------- migration/socket.c | 31 +-------- migration/socket.h | 5 +- monitor/hmp-cmds.c | 101 ++++++++++++++++++++++++++++- qapi/migration.json | 143 ++++++++++++++++++++++++++++++++++++++++-- softmmu/vl.c | 2 +- 6 files changed, 344 insertions(+), 71 deletions(-)