| Message ID | 1434450415-11339-20-git-send-email-dgilbert@redhat.com |
|---|---|
| State | New |
| Headers | show |
"Dr. David Alan Gilbert (git)" <dgilbert@redhat.com> wrote: > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com> > > MIG_CMD_PACKAGED is a migration command that wraps a chunk of migration > stream inside a package whose length can be determined purely by reading > its header. The destination guarantees that the whole MIG_CMD_PACKAGED > is read off the stream prior to parsing the contents. > > This is used by postcopy to load device state (from the package) > while leaving the main stream free to receive memory pages. > > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > +/* We have a buffer of data to send; we don't want that all to be loaded > + * by the command itself, so the command contains just the length of the > + * extra buffer that we then send straight after it. > + * TODO: Must be a better way to organise that Famous words of advise O:-) I have to read the rest of series to make my mind on this one.
On (Tue) 16 Jun 2015 [11:26:32], Dr. David Alan Gilbert (git) wrote: > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com> > > MIG_CMD_PACKAGED is a migration command that wraps a chunk of migration > stream inside a package whose length can be determined purely by reading > its header. The destination guarantees that the whole MIG_CMD_PACKAGED > is read off the stream prior to parsing the contents. > > This is used by postcopy to load device state (from the package) > while leaving the main stream free to receive memory pages. Not sure why this is necessary. I suppose I'll have to go read the documentation in patch 1.. However: > --- a/migration/savevm.c > +++ b/migration/savevm.c > @@ -718,6 +718,50 @@ void qemu_savevm_send_open_return_path(QEMUFile *f) > qemu_savevm_command_send(f, MIG_CMD_OPEN_RETURN_PATH, 0, NULL); > } > > +/* We have a buffer of data to send; we don't want that all to be loaded > + * by the command itself, so the command contains just the length of the > + * extra buffer that we then send straight after it. > + * TODO: Must be a better way to organise that > + * > + * Returns: > + * 0 on success > + * -ve on error > + */ > +int qemu_savevm_send_packaged(QEMUFile *f, const QEMUSizedBuffer *qsb) > +{ > + size_t cur_iov; > + size_t len = qsb_get_length(qsb); > + uint32_t tmp; > + > + if (len > MAX_VM_CMD_PACKAGED_SIZE) { > + error_report("%s: Unreasonably large packaged state: %zu", > + __func__, len); > + return -1; > + } > + > + tmp = cpu_to_be32(len); > + > + trace_qemu_savevm_send_packaged(); > + qemu_savevm_command_send(f, MIG_CMD_PACKAGED, 4, (uint8_t *)&tmp); > + > + /* all the data follows (concatinating the iov's) */ > + for (cur_iov = 0; cur_iov < qsb->n_iov; cur_iov++) { > + /* The iov entries are partially filled */ > + size_t towrite = (qsb->iov[cur_iov].iov_len > len) ? > + len : > + qsb->iov[cur_iov].iov_len; If iov_len was > len, we only wrote part of the current buffer, and we skip to the next? Amit
* Amit Shah (amit.shah@redhat.com) wrote: > On (Tue) 16 Jun 2015 [11:26:32], Dr. David Alan Gilbert (git) wrote: > > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com> > > > > MIG_CMD_PACKAGED is a migration command that wraps a chunk of migration > > stream inside a package whose length can be determined purely by reading > > its header. The destination guarantees that the whole MIG_CMD_PACKAGED > > is read off the stream prior to parsing the contents. > > > > This is used by postcopy to load device state (from the package) > > while leaving the main stream free to receive memory pages. > > Not sure why this is necessary. I suppose I'll have to go read the > documentation in patch 1.. Yep - or one of the previous replies where I explained it. > However: > > > --- a/migration/savevm.c > > +++ b/migration/savevm.c > > @@ -718,6 +718,50 @@ void qemu_savevm_send_open_return_path(QEMUFile *f) > > qemu_savevm_command_send(f, MIG_CMD_OPEN_RETURN_PATH, 0, NULL); > > } > > > > +/* We have a buffer of data to send; we don't want that all to be loaded > > + * by the command itself, so the command contains just the length of the > > + * extra buffer that we then send straight after it. > > + * TODO: Must be a better way to organise that > > + * > > + * Returns: > > + * 0 on success > > + * -ve on error > > + */ > > +int qemu_savevm_send_packaged(QEMUFile *f, const QEMUSizedBuffer *qsb) > > +{ > > + size_t cur_iov; > > + size_t len = qsb_get_length(qsb); > > + uint32_t tmp; > > + > > + if (len > MAX_VM_CMD_PACKAGED_SIZE) { > > + error_report("%s: Unreasonably large packaged state: %zu", > > + __func__, len); > > + return -1; > > + } > > + > > + tmp = cpu_to_be32(len); > > + > > + trace_qemu_savevm_send_packaged(); > > + qemu_savevm_command_send(f, MIG_CMD_PACKAGED, 4, (uint8_t *)&tmp); > > + > > + /* all the data follows (concatinating the iov's) */ > > + for (cur_iov = 0; cur_iov < qsb->n_iov; cur_iov++) { > > + /* The iov entries are partially filled */ > > + size_t towrite = (qsb->iov[cur_iov].iov_len > len) ? > > + len : > > + qsb->iov[cur_iov].iov_len; > > If iov_len was > len, we only wrote part of the current buffer, and we > skip to the next? Yes; this is just the end case; the qsb allocates iov entries in 'chunks' but then the data that gets added often doesn't use the whole chunk. 'len' - set above from qsb_get_length - gives the used contents of the qsb, and that's all we want to write. This is normally the case on the last entry in the qsb anyway; however since 'len' gets decremented by the amount written we might go once more around the loop and the 'if (!towrite) { break; }' might break us out of the loop instead. Dave > > > Amit -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
On (Tue) 16 Jun 2015 [11:26:32], Dr. David Alan Gilbert (git) wrote: > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com> > > MIG_CMD_PACKAGED is a migration command that wraps a chunk of migration > stream inside a package whose length can be determined purely by reading > its header. The destination guarantees that the whole MIG_CMD_PACKAGED > is read off the stream prior to parsing the contents. > > This is used by postcopy to load device state (from the package) > while leaving the main stream free to receive memory pages. > > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Amit Shah <amit.shah@redhat.com> Amit
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h index c5738f5..5bf8f80 100644 --- a/include/sysemu/sysemu.h +++ b/include/sysemu/sysemu.h @@ -87,6 +87,7 @@ enum qemu_vm_cmd { MIG_CMD_INVALID = 0, /* Must be 0 */ MIG_CMD_OPEN_RETURN_PATH, /* Tell the dest to open the Return path */ MIG_CMD_PING, /* Request a PONG on the RP */ + MIG_CMD_PACKAGED, /* Send a wrapped stream within this stream */ MIG_CMD_POSTCOPY_ADVISE = 20, /* Prior to any page transfers, just warn we might want to do PC */ @@ -100,6 +101,8 @@ enum qemu_vm_cmd { }; +#define MAX_VM_CMD_PACKAGED_SIZE (1ul << 24) + bool qemu_savevm_state_blocked(Error **errp); void qemu_savevm_state_begin(QEMUFile *f, const MigrationParams *params); @@ -112,6 +115,7 @@ void qemu_savevm_command_send(QEMUFile *f, enum qemu_vm_cmd command, uint16_t len, uint8_t *data); void qemu_savevm_send_ping(QEMUFile *f, uint32_t value); void qemu_savevm_send_open_return_path(QEMUFile *f); +int qemu_savevm_send_packaged(QEMUFile *f, const QEMUSizedBuffer *qsb); void qemu_savevm_send_postcopy_advise(QEMUFile *f); void qemu_savevm_send_postcopy_listen(QEMUFile *f); void qemu_savevm_send_postcopy_run(QEMUFile *f); diff --git a/migration/savevm.c b/migration/savevm.c index 7b2f086..2c4cbe1 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -718,6 +718,50 @@ void qemu_savevm_send_open_return_path(QEMUFile *f) qemu_savevm_command_send(f, MIG_CMD_OPEN_RETURN_PATH, 0, NULL); } +/* We have a buffer of data to send; we don't want that all to be loaded + * by the command itself, so the command contains just the length of the + * extra buffer that we then send straight after it. + * TODO: Must be a better way to organise that + * + * Returns: + * 0 on success + * -ve on error + */ +int qemu_savevm_send_packaged(QEMUFile *f, const QEMUSizedBuffer *qsb) +{ + size_t cur_iov; + size_t len = qsb_get_length(qsb); + uint32_t tmp; + + if (len > MAX_VM_CMD_PACKAGED_SIZE) { + error_report("%s: Unreasonably large packaged state: %zu", + __func__, len); + return -1; + } + + tmp = cpu_to_be32(len); + + trace_qemu_savevm_send_packaged(); + qemu_savevm_command_send(f, MIG_CMD_PACKAGED, 4, (uint8_t *)&tmp); + + /* all the data follows (concatinating the iov's) */ + for (cur_iov = 0; cur_iov < qsb->n_iov; cur_iov++) { + /* The iov entries are partially filled */ + size_t towrite = (qsb->iov[cur_iov].iov_len > len) ? + len : + qsb->iov[cur_iov].iov_len; + len -= towrite; + + if (!towrite) { + break; + } + + qemu_put_buffer(f, qsb->iov[cur_iov].iov_base, towrite); + } + + return 0; +} + /* Send prior to any postcopy transfer */ void qemu_savevm_send_postcopy_advise(QEMUFile *f) { @@ -1253,6 +1297,48 @@ static int loadvm_process_command_simple_lencheck(const char *name, return 0; } +/* Immediately following this command is a blob of data containing an embedded + * chunk of migration stream; read it and load it. + */ +static int loadvm_handle_cmd_packaged(MigrationIncomingState *mis, + uint32_t length) +{ + int ret; + uint8_t *buffer; + QEMUSizedBuffer *qsb; + + trace_loadvm_handle_cmd_packaged(length); + + if (length > MAX_VM_CMD_PACKAGED_SIZE) { + error_report("Unreasonably large packaged state: %u", length); + return -1; + } + buffer = g_malloc0(length); + ret = qemu_get_buffer(mis->file, buffer, (int)length); + if (ret != length) { + g_free(buffer); + error_report("CMD_PACKAGED: Buffer receive fail ret=%d length=%d\n", + ret, length); + return (ret < 0) ? ret : -EAGAIN; + } + trace_loadvm_handle_cmd_packaged_received(ret); + + /* Setup a dummy QEMUFile that actually reads from the buffer */ + qsb = qsb_create(buffer, length); + g_free(buffer); /* Because qsb_create copies */ + if (!qsb) { + error_report("Unable to create qsb"); + } + QEMUFile *packf = qemu_bufopen("r", qsb); + + ret = qemu_loadvm_state_main(packf, mis); + trace_loadvm_handle_cmd_packaged_main(ret); + qemu_fclose(packf); + qsb_free(qsb); + + return ret; +} + /* * Process an incoming 'QEMU_VM_COMMAND' * 0 just a normal return @@ -1304,6 +1390,14 @@ static int loadvm_process_command(QEMUFile *f) migrate_send_rp_pong(mis, tmp32); break; + case MIG_CMD_PACKAGED: + if (loadvm_process_command_simple_lencheck("CMD_POSTCOPY_PACKAGED", + len, 4)) { + return -1; + } + tmp32 = qemu_get_be32(f); + return loadvm_handle_cmd_packaged(mis, tmp32); + case MIG_CMD_POSTCOPY_ADVISE: if (loadvm_process_command_simple_lencheck("CMD_POSTCOPY_ADVISE", len, 16)) { diff --git a/trace-events b/trace-events index 44ac831..299805b 100644 --- a/trace-events +++ b/trace-events @@ -1187,6 +1187,10 @@ qemu_loadvm_state_main(void) "" qemu_loadvm_state_main_quit_parent(void) "" qemu_loadvm_state_post_main(int ret) "%d" qemu_loadvm_state_section_startfull(uint32_t section_id, const char *idstr, uint32_t instance_id, uint32_t version_id) "%u(%s) %u %u" +qemu_savevm_send_packaged(void) "" +loadvm_handle_cmd_packaged(unsigned int length) "%u" +loadvm_handle_cmd_packaged_main(int ret) "%d" +loadvm_handle_cmd_packaged_received(int ret) "%d" loadvm_postcopy_handle_advise(void) "" loadvm_postcopy_handle_listen(void) "" loadvm_postcopy_handle_run(void) ""