From patchwork Thu Aug 28 03:36:42 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Gu Zheng X-Patchwork-Id: 383679 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id BA63E140170 for ; Thu, 28 Aug 2014 14:03:47 +1000 (EST) Received: from localhost ([::1]:34394 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XMqw5-00022n-Tb for incoming@patchwork.ozlabs.org; Thu, 28 Aug 2014 00:03:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47750) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XMqvc-0001Nr-1c for qemu-devel@nongnu.org; Thu, 28 Aug 2014 00:03:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XMqvW-0005e7-2j for qemu-devel@nongnu.org; Thu, 28 Aug 2014 00:03:15 -0400 Received: from fgwmail5.fujitsu.co.jp ([192.51.44.35]:39585) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XMqvV-0005cz-Ey for qemu-devel@nongnu.org; Thu, 28 Aug 2014 00:03:10 -0400 Received: from kw-mxq.gw.nic.fujitsu.com (unknown [10.0.237.131]) by fgwmail5.fujitsu.co.jp (Postfix) with ESMTP id AD06D3EE0E7 for ; Thu, 28 Aug 2014 13:03:08 +0900 (JST) Received: from s3.gw.fujitsu.co.jp (s3.gw.fujitsu.co.jp [10.0.50.93]) by kw-mxq.gw.nic.fujitsu.com (Postfix) with ESMTP id B8057AC0885 for ; Thu, 28 Aug 2014 13:03:07 +0900 (JST) Received: from s00.gw.fujitsu.co.jp (s00.gw.nic.fujitsu.com [133.161.11.15]) by s3.gw.fujitsu.co.jp (Postfix) with ESMTP id 6778F1DB8038 for ; Thu, 28 Aug 2014 13:03:07 +0900 (JST) Received: from s00.gw.fujitsu.co.jp (kw-mxio2.gw.nic.fujitsu.com [10.0.237.142]) by s00.gw.fujitsu.co.jp (Postfix) with ESMTP id 4BB401180C3; Thu, 28 Aug 2014 13:03:07 +0900 (JST) Received: from guz (unknown [10.167.226.100]) by s00.gw.fujitsu.co.jp (Postfix) with ESMTP id 2128D8A002; Thu, 28 Aug 2014 13:03:07 +0900 (JST) Received: (from root@localhost) by guz (8.14.5/8.14.5/Submit) id s7S3m15m009768; Thu, 28 Aug 2014 11:48:01 +0800 From: Gu Zheng To: qemu-devel@nongnu.org, imammedo@redhat.com, afaerber@suse.de Date: Thu, 28 Aug 2014 11:36:42 +0800 Message-Id: <1409197002-9498-11-git-send-email-guz.fnst@cn.fujitsu.com> X-Mailer: git-send-email 1.7.7 In-Reply-To: <1409197002-9498-1-git-send-email-guz.fnst@cn.fujitsu.com> References: <1409197002-9498-1-git-send-email-guz.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-TM-AS-MML: disable X-TMASE-MatchedRID: 6mw5WAT+gZ9cNOa3zFU5XxIRh9wkXSlFJ7AUYFBH3abejCctWUsIIA6G St6DPcN2P5vI5ECwbmy+daZEFcM/u1VBWgEsQzu04pdq9sdj8LWrH40UyBdnXuOxOq7LQlGLRj9 q7BEmcwsryFZ2sFu6D0Uxj2KvNFNZourMhFrz5MC628cXbnOhT5KLNrbpy/A0ow/V8X6BDY6uRS LNCjomEaLqebnC2s87w60UCgKjnSSON+GapO42veTuT3JcmKqqf2g6KJZtxl1Y8awbSmPh07+xg 266Cxva0BpcSFb1f15ftuJwrFEhTY2j49Ftap9EOwBXM346/+wif6EYaSE3+tGxM2AWQY7d9H0f HUk/7w6mfnozoUbFXzDv5pVqGf0R X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 192.51.44.35 Cc: chen.fan.fnst@cn.fujitsu.com, anshul.makkar@profitbricks.com, isimatu.yasuaki@jp.fujitsu.com, Gu Zheng , tangchen@cn.fujitsu.com Subject: [Qemu-devel] [RFC V2 10/10] cpus: reclaim allocated vCPU objects X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org After ACPI get a signal to eject a vCPU, the vCPU must be removed from CPU list，before the vCPU really removed, then release the all related vCPU objects. But we do not close KVM vcpu fd, just record it into a list, in order to reuse it. Signed-off-by: Chen Fan Signed-off-by: Gu Zheng --- cpus.c | 37 ++++++++++++++++++++++++++++++++ include/sysemu/kvm.h | 1 + kvm-all.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 94 insertions(+), 1 deletions(-) diff --git a/cpus.c b/cpus.c index eee693b..0608b41 100644 --- a/cpus.c +++ b/cpus.c @@ -851,6 +851,24 @@ void async_run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data) qemu_cpu_kick(cpu); } +static void qemu_kvm_destroy_vcpu(CPUState *cpu) +{ + CPU_REMOVE(cpu); + + if (kvm_destroy_vcpu(cpu) < 0) { + fprintf(stderr, "kvm_destroy_vcpu failed.\n"); + exit(1); + } + + object_unparent(OBJECT(cpu)); +} + +static void qemu_tcg_destroy_vcpu(CPUState *cpu) +{ + CPU_REMOVE(cpu); + object_unparent(OBJECT(cpu)); +} + static void flush_queued_work(CPUState *cpu) { struct qemu_work_item *wi; @@ -942,6 +960,11 @@ static void *qemu_kvm_cpu_thread_fn(void *arg) } } qemu_kvm_wait_io_event(cpu); + if (cpu->exit && !cpu_can_run(cpu)) { + qemu_kvm_destroy_vcpu(cpu); + qemu_mutex_unlock(&qemu_global_mutex); + return NULL; + } } return NULL; @@ -994,6 +1017,7 @@ static void tcg_exec_all(void); static void *qemu_tcg_cpu_thread_fn(void *arg) { CPUState *cpu = arg; + CPUState *remove_cpu = NULL; qemu_tcg_init_cpu_signals(); qemu_thread_get_self(cpu->thread); @@ -1026,6 +1050,16 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) } } qemu_tcg_wait_io_event(); + CPU_FOREACH(cpu) { + if (cpu->exit && !cpu_can_run(cpu)) { + remove_cpu = cpu; + break; + } + } + if (remove_cpu) { + qemu_tcg_destroy_vcpu(remove_cpu); + remove_cpu = NULL; + } } return NULL; @@ -1383,6 +1417,9 @@ static void tcg_exec_all(void) break; } } else if (cpu->stop || cpu->stopped) { + if (cpu->exit) { + next_cpu = CPU_NEXT(cpu); + } break; } } diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index 174ea36..88e2403 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -178,6 +178,7 @@ int kvm_has_intx_set_mask(void); int kvm_init_vcpu(CPUState *cpu); int kvm_cpu_exec(CPUState *cpu); +int kvm_destroy_vcpu(CPUState *cpu); #ifdef NEED_CPU_H diff --git a/kvm-all.c b/kvm-all.c index 1402f4f..d0caeff 100644 --- a/kvm-all.c +++ b/kvm-all.c @@ -74,6 +74,12 @@ typedef struct KVMSlot typedef struct kvm_dirty_log KVMDirtyLog; +struct KVMParkedVcpu { + unsigned long vcpu_id; + int kvm_fd; + QLIST_ENTRY(KVMParkedVcpu) node; +}; + struct KVMState { KVMSlot *slots; @@ -108,6 +114,7 @@ struct KVMState QTAILQ_HEAD(msi_hashtab, KVMMSIRoute) msi_hashtab[KVM_MSI_HASHTAB_SIZE]; bool direct_msi; #endif + QLIST_HEAD(, KVMParkedVcpu) kvm_parked_vcpus; }; KVMState *kvm_state; @@ -226,6 +233,53 @@ static int kvm_set_user_memory_region(KVMState *s, KVMSlot *slot) return kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem); } +int kvm_destroy_vcpu(CPUState *cpu) +{ + KVMState *s = kvm_state; + long mmap_size; + struct KVMParkedVcpu *vcpu = NULL; + int ret = 0; + + DPRINTF("kvm_destroy_vcpu\n"); + + mmap_size = kvm_ioctl(s, KVM_GET_VCPU_MMAP_SIZE, 0); + if (mmap_size < 0) { + ret = mmap_size; + DPRINTF("KVM_GET_VCPU_MMAP_SIZE failed\n"); + goto err; + } + + ret = munmap(cpu->kvm_run, mmap_size); + if (ret < 0) { + goto err; + } + + vcpu = g_malloc0(sizeof(*vcpu)); + vcpu->vcpu_id = kvm_arch_vcpu_id(cpu); + vcpu->kvm_fd = cpu->kvm_fd; + QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node); +err: + return ret; +} + +static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id) +{ + struct KVMParkedVcpu *cpu; + + QLIST_FOREACH(cpu, &s->kvm_parked_vcpus, node) { + if (cpu->vcpu_id == vcpu_id) { + int kvm_fd; + + QLIST_REMOVE(cpu, node); + kvm_fd = cpu->kvm_fd; + g_free(cpu); + return kvm_fd; + } + } + + return kvm_vm_ioctl(s, KVM_CREATE_VCPU, (void *)vcpu_id); +} + int kvm_init_vcpu(CPUState *cpu) { KVMState *s = kvm_state; @@ -234,7 +288,7 @@ int kvm_init_vcpu(CPUState *cpu) DPRINTF("kvm_init_vcpu\n"); - ret = kvm_vm_ioctl(s, KVM_CREATE_VCPU, (void *)kvm_arch_vcpu_id(cpu)); + ret = kvm_get_vcpu(s, kvm_arch_vcpu_id(cpu)); if (ret < 0) { DPRINTF("kvm_create_vcpu failed\n"); goto err; @@ -1404,6 +1458,7 @@ int kvm_init(MachineClass *mc) #ifdef KVM_CAP_SET_GUEST_DEBUG QTAILQ_INIT(&s->kvm_sw_breakpoints); #endif + QLIST_INIT(&s->kvm_parked_vcpus); s->vmfd = -1; s->fd = qemu_open("/dev/kvm", O_RDWR); if (s->fd == -1) {