From patchwork Thu Jan 22 14:01:39 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Graf X-Patchwork-Id: 431822 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 968511402A1 for ; Fri, 23 Jan 2015 01:02:44 +1100 (AEDT) Received: from localhost ([::1]:53428 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YEILK-0002AU-HD for incoming@patchwork.ozlabs.org; Thu, 22 Jan 2015 09:02:42 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32914) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YEIKa-0000pY-71 for qemu-devel@nongnu.org; Thu, 22 Jan 2015 09:02:01 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YEIKN-00017S-M4 for qemu-devel@nongnu.org; Thu, 22 Jan 2015 09:01:56 -0500 Received: from cantor2.suse.de ([195.135.220.15]:39671 helo=mx2.suse.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YEIKN-00016y-Ce for qemu-devel@nongnu.org; Thu, 22 Jan 2015 09:01:43 -0500 Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 21415ACD4; Thu, 22 Jan 2015 14:01:41 +0000 (UTC) From: Alexander Graf To: qemu-devel@nongnu.org Date: Thu, 22 Jan 2015 15:01:39 +0100 Message-Id: <1421935300-8579-4-git-send-email-agraf@suse.de> X-Mailer: git-send-email 1.7.12.4 In-Reply-To: <1421935300-8579-1-git-send-email-agraf@suse.de> References: <1421935300-8579-1-git-send-email-agraf@suse.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x (no timestamps) [generic] X-Received-From: 195.135.220.15 Cc: quintela@redhat.com, amit.shah@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org, afaerber@suse.de Subject: [Qemu-devel] [PATCH v4 3/4] migration: Append JSON description of migration stream X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org One of the annoyances of the current migration format is the fact that it's not self-describing. In fact, it's not properly describing at all. Some code randomly scattered throughout QEMU elaborates roughly how to read and write a stream of bytes. We discussed an idea during KVM Forum 2013 to add a JSON description of the migration protocol itself to the migration stream. This patch adds a section after the VM_END migration end marker that contains description data on what the device sections of the stream are composed of. This approach is backwards compatible with any QEMU version reading the stream, because QEMU just stops reading after the VM_END marker and ignores any data following it. With an additional external program this allows us to decipher the contents of any migration stream and hopefully make migration bugs easier to track down. Signed-off-by: Alexander Graf --- v2 -> v3: - Dont compress objects with subsections - Destroy QJSON object v3 -> v4: - Update patch description --- hw/pci/pci.c | 2 +- hw/scsi/spapr_vscsi.c | 2 +- hw/virtio/virtio.c | 2 +- include/migration/migration.h | 1 + include/migration/vmstate.h | 3 +- migration/vmstate.c | 186 ++++++++++++++++++++++++++++++++++++++++-- savevm.c | 54 ++++++++++-- tests/test-vmstate.c | 6 +- 8 files changed, 237 insertions(+), 19 deletions(-) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 371699c..38d0b7b 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -512,7 +512,7 @@ void pci_device_save(PCIDevice *s, QEMUFile *f) * This makes us compatible with old devices * which never set or clear this bit. */ s->config[PCI_STATUS] &= ~PCI_STATUS_INTERRUPT; - vmstate_save_state(f, pci_get_vmstate(s), s); + vmstate_save_state(f, pci_get_vmstate(s), s, NULL); /* Restore the interrupt status bit. */ pci_update_irq_status(s); } diff --git a/hw/scsi/spapr_vscsi.c b/hw/scsi/spapr_vscsi.c index 20b20f0..3639235 100644 --- a/hw/scsi/spapr_vscsi.c +++ b/hw/scsi/spapr_vscsi.c @@ -630,7 +630,7 @@ static void vscsi_save_request(QEMUFile *f, SCSIRequest *sreq) vscsi_req *req = sreq->hba_private; assert(req->active); - vmstate_save_state(f, &vmstate_spapr_vscsi_req, req); + vmstate_save_state(f, &vmstate_spapr_vscsi_req, req, NULL); DPRINTF("VSCSI: saving tag=%u, current desc#%d, offset=%x\n", req->qtag, req->cur_desc_num, req->cur_desc_offset); diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 013979a..d735343 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -955,7 +955,7 @@ void virtio_save(VirtIODevice *vdev, QEMUFile *f) } /* Subsections */ - vmstate_save_state(f, &vmstate_virtio, vdev); + vmstate_save_state(f, &vmstate_virtio, vdev, NULL); } int virtio_set_features(VirtIODevice *vdev, uint32_t val) diff --git a/include/migration/migration.h b/include/migration/migration.h index 3cb5ba8..f37348b 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -33,6 +33,7 @@ #define QEMU_VM_SECTION_END 0x03 #define QEMU_VM_SECTION_FULL 0x04 #define QEMU_VM_SUBSECTION 0x05 +#define QEMU_VM_VMDESCRIPTION 0x06 struct MigrationParams { bool blk; diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index d712a65..42cc27e 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -29,6 +29,7 @@ #ifndef CONFIG_USER_ONLY #include #endif +#include typedef void SaveStateHandler(QEMUFile *f, void *opaque); typedef int LoadStateHandler(QEMUFile *f, void *opaque, int version_id); @@ -779,7 +780,7 @@ extern const VMStateInfo vmstate_info_bitmap; int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd, void *opaque, int version_id); void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd, - void *opaque); + void *opaque, QJSON *vmdesc); int vmstate_register_with_alias_id(DeviceState *dev, int instance_id, const VMStateDescription *vmsd, diff --git a/migration/vmstate.c b/migration/vmstate.c index 3dde574..2a91f1f 100644 --- a/migration/vmstate.c +++ b/migration/vmstate.c @@ -4,9 +4,10 @@ #include "migration/vmstate.h" #include "qemu/bitops.h" #include "trace.h" +#include "qjson.h" static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd, - void *opaque); + void *opaque, QJSON *vmdesc); static int vmstate_subsection_load(QEMUFile *f, const VMStateDescription *vmsd, void *opaque); @@ -138,32 +139,181 @@ int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd, return 0; } +static int vmfield_name_num(VMStateField *start, VMStateField *search) +{ + VMStateField *field; + int found = 0; + + for (field = start; field->name; field++) { + if (!strcmp(field->name, search->name)) { + if (field == search) { + return found; + } + found++; + } + } + + return -1; +} + +static bool vmfield_name_is_unique(VMStateField *start, VMStateField *search) +{ + VMStateField *field; + int found = 0; + + for (field = start; field->name; field++) { + if (!strcmp(field->name, search->name)) { + found++; + /* name found more than once, so it's not unique */ + if (found > 1) { + return false; + } + } + } + + return true; +} + +static const char *vmfield_get_type_name(VMStateField *field) +{ + const char *type = "unknown"; + + if (field->flags & VMS_STRUCT) { + type = "struct"; + } else if (field->info->name) { + type = field->info->name; + } + + return type; +} + +static bool vmsd_can_compress(VMStateField *field) +{ + if (field->field_exists) { + /* Dynamically existing fields mess up compression */ + return false; + } + + if (field->flags & VMS_STRUCT) { + VMStateField *sfield = field->vmsd->fields; + while (sfield->name) { + if (!vmsd_can_compress(sfield)) { + /* Child elements can't compress, so can't we */ + return false; + } + sfield++; + } + + if (field->vmsd->subsections) { + /* Subsections may come and go, better don't compress */ + return false; + } + } + + return true; +} + +static void vmsd_desc_field_start(const VMStateDescription *vmsd, QJSON *vmdesc, + VMStateField *field, int i, int max) +{ + char *name, *old_name; + bool is_array = max > 1; + bool can_compress = vmsd_can_compress(field); + + if (!vmdesc) { + return; + } + + name = g_strdup(field->name); + + /* Field name is not unique, need to make it unique */ + if (!vmfield_name_is_unique(vmsd->fields, field)) { + int num = vmfield_name_num(vmsd->fields, field); + old_name = name; + name = g_strdup_printf("%s[%d]", name, num); + g_free(old_name); + } + + json_start_object(vmdesc, NULL); + json_prop_str(vmdesc, "name", name); + if (is_array) { + if (can_compress) { + json_prop_int(vmdesc, "array_len", max); + } else { + json_prop_int(vmdesc, "index", i); + } + } + json_prop_str(vmdesc, "type", vmfield_get_type_name(field)); + + if (field->flags & VMS_STRUCT) { + json_start_object(vmdesc, "struct"); + } + + g_free(name); +} + +static void vmsd_desc_field_end(const VMStateDescription *vmsd, QJSON *vmdesc, + VMStateField *field, size_t size, int i) +{ + if (!vmdesc) { + return; + } + + if (field->flags & VMS_STRUCT) { + /* We printed a struct in between, close its child object */ + json_end_object(vmdesc); + } + + json_prop_int(vmdesc, "size", size); + json_end_object(vmdesc); +} + void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd, - void *opaque) + void *opaque, QJSON *vmdesc) { VMStateField *field = vmsd->fields; if (vmsd->pre_save) { vmsd->pre_save(opaque); } + + if (vmdesc) { + json_prop_str(vmdesc, "vmsd_name", vmsd->name); + json_prop_int(vmdesc, "version", vmsd->version_id); + json_start_array(vmdesc, "fields"); + } + while (field->name) { if (!field->field_exists || field->field_exists(opaque, vmsd->version_id)) { void *base_addr = vmstate_base_addr(opaque, field, false); int i, n_elems = vmstate_n_elems(opaque, field); int size = vmstate_size(opaque, field); + int64_t old_offset, written_bytes; + QJSON *vmdesc_loop = vmdesc; for (i = 0; i < n_elems; i++) { void *addr = base_addr + size * i; + vmsd_desc_field_start(vmsd, vmdesc_loop, field, i, n_elems); + old_offset = qemu_ftell_fast(f); + if (field->flags & VMS_ARRAY_OF_POINTER) { addr = *(void **)addr; } if (field->flags & VMS_STRUCT) { - vmstate_save_state(f, field->vmsd, addr); + vmstate_save_state(f, field->vmsd, addr, vmdesc_loop); } else { field->info->put(f, addr, size); } + + written_bytes = qemu_ftell_fast(f) - old_offset; + vmsd_desc_field_end(vmsd, vmdesc_loop, field, written_bytes, i); + + /* Compressed arrays only care about the first element */ + if (vmdesc_loop && vmsd_can_compress(field)) { + vmdesc_loop = NULL; + } } } else { if (field->flags & VMS_MUST_EXIST) { @@ -174,7 +324,12 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd, } field++; } - vmstate_subsection_save(f, vmsd, opaque); + + if (vmdesc) { + json_end_array(vmdesc); + } + + vmstate_subsection_save(f, vmsd, opaque, vmdesc); } static const VMStateDescription * @@ -231,24 +386,43 @@ static int vmstate_subsection_load(QEMUFile *f, const VMStateDescription *vmsd, } static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd, - void *opaque) + void *opaque, QJSON *vmdesc) { const VMStateSubsection *sub = vmsd->subsections; + bool subsection_found = false; while (sub && sub->needed) { if (sub->needed(opaque)) { const VMStateDescription *vmsd = sub->vmsd; uint8_t len; + if (vmdesc) { + /* Only create subsection array when we have any */ + if (!subsection_found) { + json_start_array(vmdesc, "subsections"); + subsection_found = true; + } + + json_start_object(vmdesc, NULL); + } + qemu_put_byte(f, QEMU_VM_SUBSECTION); len = strlen(vmsd->name); qemu_put_byte(f, len); qemu_put_buffer(f, (uint8_t *)vmsd->name, len); qemu_put_be32(f, vmsd->version_id); - vmstate_save_state(f, vmsd, opaque); + vmstate_save_state(f, vmsd, opaque, vmdesc); + + if (vmdesc) { + json_end_object(vmdesc); + } } sub++; } + + if (vmdesc && subsection_found) { + json_end_array(vmdesc); + } } /* bool */ diff --git a/savevm.c b/savevm.c index 08ec678..48e3e5c 100644 --- a/savevm.c +++ b/savevm.c @@ -572,14 +572,34 @@ static int vmstate_load(QEMUFile *f, SaveStateEntry *se, int version_id) return vmstate_load_state(f, se->vmsd, se->opaque, version_id); } -static void vmstate_save(QEMUFile *f, SaveStateEntry *se) +static void vmstate_save_old_style(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc) +{ + int64_t old_offset, size; + + old_offset = qemu_ftell_fast(f); + se->ops->save_state(f, se->opaque); + size = qemu_ftell_fast(f) - old_offset; + + if (vmdesc) { + json_prop_int(vmdesc, "size", size); + json_start_array(vmdesc, "fields"); + json_start_object(vmdesc, NULL); + json_prop_str(vmdesc, "name", "data"); + json_prop_int(vmdesc, "size", size); + json_prop_str(vmdesc, "type", "buffer"); + json_end_object(vmdesc); + json_end_array(vmdesc); + } +} + +static void vmstate_save(QEMUFile *f, SaveStateEntry *se, QJSON *vmdesc) { trace_vmstate_save(se->idstr, se->vmsd ? se->vmsd->name : "(old)"); - if (!se->vmsd) { /* Old style */ - se->ops->save_state(f, se->opaque); + if (!se->vmsd) { + vmstate_save_old_style(f, se, vmdesc); return; } - vmstate_save_state(f, se->vmsd, se->opaque); + vmstate_save_state(f, se->vmsd, se->opaque, vmdesc); } bool qemu_savevm_state_blocked(Error **errp) @@ -692,6 +712,8 @@ int qemu_savevm_state_iterate(QEMUFile *f) void qemu_savevm_state_complete(QEMUFile *f) { + QJSON *vmdesc; + int vmdesc_len; SaveStateEntry *se; int ret; @@ -721,6 +743,9 @@ void qemu_savevm_state_complete(QEMUFile *f) } } + vmdesc = qjson_new(); + json_prop_int(vmdesc, "page_size", TARGET_PAGE_SIZE); + json_start_array(vmdesc, "devices"); QTAILQ_FOREACH(se, &savevm_handlers, entry) { int len; @@ -728,6 +753,11 @@ void qemu_savevm_state_complete(QEMUFile *f) continue; } trace_savevm_section_start(se->idstr, se->section_id); + + json_start_object(vmdesc, NULL); + json_prop_str(vmdesc, "name", se->idstr); + json_prop_int(vmdesc, "instance_id", se->instance_id); + /* Section type */ qemu_put_byte(f, QEMU_VM_SECTION_FULL); qemu_put_be32(f, se->section_id); @@ -740,11 +770,23 @@ void qemu_savevm_state_complete(QEMUFile *f) qemu_put_be32(f, se->instance_id); qemu_put_be32(f, se->version_id); - vmstate_save(f, se); + vmstate_save(f, se, vmdesc); + + json_end_object(vmdesc); trace_savevm_section_end(se->idstr, se->section_id); } qemu_put_byte(f, QEMU_VM_EOF); + + json_end_array(vmdesc); + qjson_finish(vmdesc); + vmdesc_len = strlen(qjson_get_str(vmdesc)); + + qemu_put_byte(f, QEMU_VM_VMDESCRIPTION); + qemu_put_be32(f, vmdesc_len); + qemu_put_buffer(f, (uint8_t *)qjson_get_str(vmdesc), vmdesc_len); + object_unref(OBJECT(vmdesc)); + qemu_fflush(f); } @@ -843,7 +885,7 @@ static int qemu_save_device_state(QEMUFile *f) qemu_put_be32(f, se->instance_id); qemu_put_be32(f, se->version_id); - vmstate_save(f, se); + vmstate_save(f, se, NULL); } qemu_put_byte(f, QEMU_VM_EOF); diff --git a/tests/test-vmstate.c b/tests/test-vmstate.c index 39b7b01..1d620e0 100644 --- a/tests/test-vmstate.c +++ b/tests/test-vmstate.c @@ -85,7 +85,7 @@ static void save_vmstate(const VMStateDescription *desc, void *obj) QEMUFile *f = open_test_file(true); /* Save file with vmstate */ - vmstate_save_state(f, desc, obj); + vmstate_save_state(f, desc, obj, NULL); qemu_put_byte(f, QEMU_VM_EOF); g_assert(!qemu_file_get_error(f)); qemu_fclose(f); @@ -394,7 +394,7 @@ static void test_save_noskip(void) QEMUFile *fsave = qemu_bufopen("w", NULL); TestStruct obj = { .a = 1, .b = 2, .c = 3, .d = 4, .e = 5, .f = 6, .skip_c_e = false }; - vmstate_save_state(fsave, &vmstate_skipping, &obj); + vmstate_save_state(fsave, &vmstate_skipping, &obj, NULL); g_assert(!qemu_file_get_error(fsave)); uint8_t expected[] = { @@ -414,7 +414,7 @@ static void test_save_skip(void) QEMUFile *fsave = qemu_bufopen("w", NULL); TestStruct obj = { .a = 1, .b = 2, .c = 3, .d = 4, .e = 5, .f = 6, .skip_c_e = true }; - vmstate_save_state(fsave, &vmstate_skipping, &obj); + vmstate_save_state(fsave, &vmstate_skipping, &obj, NULL); g_assert(!qemu_file_get_error(fsave)); uint8_t expected[] = {