From patchwork Tue Nov 24 03:33:57 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haozhong Zhang X-Patchwork-Id: 547808 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 881D01402A5 for ; Tue, 24 Nov 2015 14:34:42 +1100 (AEDT) Received: from localhost ([::1]:35882 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a14NM-00029i-Cz for incoming@patchwork.ozlabs.org; Mon, 23 Nov 2015 22:34:40 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37747) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a14Mu-0001G8-HG for qemu-devel@nongnu.org; Mon, 23 Nov 2015 22:34:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a14Mt-0002tX-7z for qemu-devel@nongnu.org; Mon, 23 Nov 2015 22:34:12 -0500 Received: from mga03.intel.com ([134.134.136.65]:8849) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a14Ms-0002tN-Sx for qemu-devel@nongnu.org; Mon, 23 Nov 2015 22:34:11 -0500 Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga103.jf.intel.com with ESMTP; 23 Nov 2015 19:34:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,338,1444719600"; d="scan'208";a="1334342" Received: from hzzhang-optiplex-9020.sh.intel.com (HELO localhost) ([10.239.12.31]) by fmsmga004.fm.intel.com with ESMTP; 23 Nov 2015 19:34:08 -0800 From: Haozhong Zhang To: , Eduardo Habkost , "Dr. David Alan Gilbert" Date: Tue, 24 Nov 2015 11:33:57 +0800 Message-Id: <1448336037-6035-4-git-send-email-haozhong.zhang@intel.com> X-Mailer: git-send-email 2.4.8 In-Reply-To: <1448336037-6035-1-git-send-email-haozhong.zhang@intel.com> References: <1448336037-6035-1-git-send-email-haozhong.zhang@intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 134.134.136.65 Cc: Haozhong Zhang , kvm@vger.kernel.org, "Michael S. Tsirkin" , Marcelo Tosatti , Paolo Bonzini , afaerber@suse.de, Richard Henderson Subject: [Qemu-devel] [PATCH v6 3/3] target-i386: add support to migrate vcpu's TSC rate X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This patch enables migrating vcpu's TSC rate. If KVM on the destination machine supports TSC scaling, guest programs will observe a consistent TSC rate across the migration. If TSC scaling is not supported on the destination machine, the migration will not be aborted and QEMU on the destination will not set vcpu's TSC rate to the migrated value. If vcpu's TSC rate specified by CPU option 'tsc-freq' on the destination machine is inconsistent with the migrated TSC rate, the migration will be aborted. For backwards compatibility, the migration of vcpu's TSC rate is disabled on pc-*-2.4 and older machine types. Signed-off-by: Haozhong Zhang Reviewed-by: Eduardo Habkost --- hw/i386/pc.c | 1 + hw/i386/pc_piix.c | 1 + hw/i386/pc_q35.c | 1 + include/hw/i386/pc.h | 1 + target-i386/cpu.c | 2 +- target-i386/cpu.h | 1 + target-i386/kvm.c | 22 ++++++++++++++++++++++ target-i386/machine.c | 30 ++++++++++++++++++++++++++++++ 8 files changed, 58 insertions(+), 1 deletion(-) diff --git a/hw/i386/pc.c b/hw/i386/pc.c index 5e20e07..72d9b9c 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -1952,6 +1952,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data) HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc); pcmc->get_hotplug_handler = mc->get_hotplug_handler; + pcmc->save_tsc_khz = true; mc->get_hotplug_handler = pc_get_hotpug_handler; mc->cpu_index_to_socket_id = pc_cpu_index_to_socket_id; mc->default_boot_order = "cad"; diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c index 07d0baa..7c5b0d2 100644 --- a/hw/i386/pc_piix.c +++ b/hw/i386/pc_piix.c @@ -489,6 +489,7 @@ static void pc_i440fx_2_4_machine_options(MachineClass *m) m->alias = NULL; m->is_default = 0; pcmc->broken_reserved_end = true; + pcmc->save_tsc_khz = false; SET_MACHINE_COMPAT(m, PC_COMPAT_2_4); } diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c index 0fdae09..fd8efe3 100644 --- a/hw/i386/pc_q35.c +++ b/hw/i386/pc_q35.c @@ -387,6 +387,7 @@ static void pc_q35_2_4_machine_options(MachineClass *m) m->hw_version = "2.4.0"; m->alias = NULL; pcmc->broken_reserved_end = true; + pcmc->save_tsc_khz = false; SET_MACHINE_COMPAT(m, PC_COMPAT_2_4); } diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h index 854c330..3b8f368 100644 --- a/include/hw/i386/pc.h +++ b/include/hw/i386/pc.h @@ -60,6 +60,7 @@ struct PCMachineClass { /*< public >*/ bool broken_reserved_end; + bool save_tsc_khz; HotplugHandler *(*get_hotplug_handler)(MachineState *machine, DeviceState *dev); }; diff --git a/target-i386/cpu.c b/target-i386/cpu.c index 11e5e39..3c0b720 100644 --- a/target-i386/cpu.c +++ b/target-i386/cpu.c @@ -1728,7 +1728,7 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque, return; } - cpu->env.tsc_khz = value / 1000; + cpu->env.tsc_khz = cpu->env.user_tsc_khz = value / 1000; } static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque, diff --git a/target-i386/cpu.h b/target-i386/cpu.h index fc4a605..ffe0bce 100644 --- a/target-i386/cpu.h +++ b/target-i386/cpu.h @@ -973,6 +973,7 @@ typedef struct CPUX86State { uint32_t sipi_vector; bool tsc_valid; int64_t tsc_khz; + int64_t user_tsc_khz; /* for sanity check only */ void *kvm_xsave_buf; uint64_t mcg_cap; diff --git a/target-i386/kvm.c b/target-i386/kvm.c index 1e811ee..2a0fd54 100644 --- a/target-i386/kvm.c +++ b/target-i386/kvm.c @@ -2381,6 +2381,28 @@ int kvm_arch_put_registers(CPUState *cpu, int level) } } + if (level == KVM_PUT_FULL_STATE) { + /* kvm_arch_set_tsc_khz() below can be called in two control flows and + * we don't need to handle its errors in both of them. + * + * One is the control flow that creates a vcpu, where + * kvm_arch_set_tsc_khz() has already been called once before by + * kvm_arch_init_vcpu(). The latter will abort the control flow if there + * are any errors of kvm_arch_set_tsc_khz(). Thus, in this control flow, + * kvm_arch_set_tsc_khz() below never fails and we can safely ignore its + * return values here. + * + * Another is the control flow of migration that sets vcpu's TSC + * frequency on the destination. The only error that can fail the + * migration is the mismatch between the migrated and the user-specified + * TSC frequencies, which has been handled by cpu_post_load(). Other + * errors, i.e. those from kvm_arch_set_tsc_khz(), never fail the + * migration, so we also safely ignore its return values in this control + * flow. + */ + kvm_arch_set_tsc_khz(cpu); + } + ret = kvm_getput_regs(x86_cpu, 1); if (ret < 0) { return ret; diff --git a/target-i386/machine.c b/target-i386/machine.c index a18e16e..e560ca3 100644 --- a/target-i386/machine.c +++ b/target-i386/machine.c @@ -6,6 +6,8 @@ #include "cpu.h" #include "sysemu/kvm.h" +#include "qemu/error-report.h" + static const VMStateDescription vmstate_segment = { .name = "segment", .version_id = 1, @@ -331,6 +333,13 @@ static int cpu_post_load(void *opaque, int version_id) CPUX86State *env = &cpu->env; int i; + if (env->tsc_khz && env->user_tsc_khz && + env->tsc_khz != env->user_tsc_khz) { + error_report("Mismatch between user-specified TSC frequency and " + "migrated TSC frequency"); + return -EINVAL; + } + /* * Real mode guest segments register DPL should be zero. * Older KVM version were setting it wrongly. @@ -775,6 +784,26 @@ static const VMStateDescription vmstate_xss = { } }; +static bool tsc_khz_needed(void *opaque) +{ + X86CPU *cpu = opaque; + CPUX86State *env = &cpu->env; + MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine()); + PCMachineClass *pcmc = PC_MACHINE_CLASS(mc); + return env->tsc_khz && pcmc->save_tsc_khz; +} + +static const VMStateDescription vmstate_tsc_khz = { + .name = "cpu/tsc_khz", + .version_id = 1, + .minimum_version_id = 1, + .needed = tsc_khz_needed, + .fields = (VMStateField[]) { + VMSTATE_INT64(env.tsc_khz, X86CPU), + VMSTATE_END_OF_LIST() + } +}; + VMStateDescription vmstate_x86_cpu = { .name = "cpu", .version_id = 12, @@ -895,6 +924,7 @@ VMStateDescription vmstate_x86_cpu = { &vmstate_msr_hyperv_runtime, &vmstate_avx512, &vmstate_xss, + &vmstate_tsc_khz, NULL } };