From patchwork Fri Jun 22 22:45:01 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Kiszka X-Patchwork-Id: 166680 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id EFD9DB6FA2 for ; Sat, 23 Jun 2012 08:50:23 +1000 (EST) Received: from localhost ([::1]:38366 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SiCgG-00024k-RY for incoming@patchwork.ozlabs.org; Fri, 22 Jun 2012 18:50:20 -0400 Received: from eggs.gnu.org ([208.118.235.92]:57758) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SiCg8-00024f-Eq for qemu-devel@nongnu.org; Fri, 22 Jun 2012 18:50:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SiCg6-00034L-Bg for qemu-devel@nongnu.org; Fri, 22 Jun 2012 18:50:12 -0400 Received: from mout.web.de ([212.227.17.12]:55411) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SiCg6-000343-0t for qemu-devel@nongnu.org; Fri, 22 Jun 2012 18:50:10 -0400 Received: from mchn199C.mchp.siemens.de ([95.157.56.37]) by smtp.web.de (mrweb101) with ESMTPSA (Nemesis) id 0MAvCS-1SYGQj2wom-00ANRj; Sat, 23 Jun 2012 00:45:02 +0200 Message-ID: <4FE4F56D.1020201@web.de> Date: Sat, 23 Jun 2012 00:45:01 +0200 From: Jan Kiszka User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 To: qemu-devel X-Enigmail-Version: 1.4.2 X-Provags-ID: V02:K0:7Hiukwo8QHBZ9QHvPMz5MJQ2ed0AtYSgQR3+pyIUdjv 1T5IFvND33TEE4ZqMqWIGbwxm7708Pyy5on+6LfLhPOVu+sLeD CFG2DCzrWmlIUqxrpAHcJsAAneCoQuq32ZQaln7Z2GFhzVyHkA Qdypfa+bWJHoapRDhFHJEGStIE8T9KIDc1MMIqvlwSKyZExuwM 0wRpgEpMd4RjnXUT3Ea7A== X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 212.227.17.12 Cc: Liu Ping Fan , Alexander Graf , Anthony Liguori Subject: [Qemu-devel] [PATCH] kvm: First step to push iothread lock out of inner run loop X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This sketches a possible path to get rid of the iothread lock on vmexits in KVM mode. On x86, the the in-kernel irqchips has to be used because we otherwise need to synchronize APIC and other per-cpu state accesses that could be changed concurrently. Not yet fully analyzed is the NMI injection path in the absence of an APIC. s390x should be fine without specific locking as their pre/post-run callbacks are empty. Power requires locking for the pre-run callback. This patch is untested, but a similar version was successfully used in a x86 setup with a network I/O path that needed no central iothread locking anymore (required special MMIO exit handling). --- kvm-all.c | 18 ++++++++++++++++-- target-i386/kvm.c | 7 +++++++ target-ppc/kvm.c | 4 ++++ 3 files changed, 27 insertions(+), 2 deletions(-) diff --git a/kvm-all.c b/kvm-all.c index f8e4328..9c3e26f 100644 --- a/kvm-all.c +++ b/kvm-all.c @@ -1460,6 +1460,8 @@ int kvm_cpu_exec(CPUArchState *env) return EXCP_HLT; } + qemu_mutex_unlock_iothread(); + do { if (env->kvm_vcpu_dirty) { kvm_arch_put_registers(env, KVM_PUT_RUNTIME_STATE); @@ -1476,14 +1478,16 @@ int kvm_cpu_exec(CPUArchState *env) */ qemu_cpu_kick_self(); } - qemu_mutex_unlock_iothread(); run_ret = kvm_vcpu_ioctl(env, KVM_RUN, 0); - qemu_mutex_lock_iothread(); kvm_arch_post_run(env, run); + /* TODO: push coalesced mmio flushing to the point where we access + * devices that are using it (currently VGA and E1000). */ + qemu_mutex_lock_iothread(); kvm_flush_coalesced_mmio_buffer(); + qemu_mutex_unlock_iothread(); if (run_ret < 0) { if (run_ret == -EINTR || run_ret == -EAGAIN) { @@ -1499,19 +1503,23 @@ int kvm_cpu_exec(CPUArchState *env) switch (run->exit_reason) { case KVM_EXIT_IO: DPRINTF("handle_io\n"); + qemu_mutex_lock_iothread(); kvm_handle_io(run->io.port, (uint8_t *)run + run->io.data_offset, run->io.direction, run->io.size, run->io.count); + qemu_mutex_unlock_iothread(); ret = 0; break; case KVM_EXIT_MMIO: DPRINTF("handle_mmio\n"); + qemu_mutex_lock_iothread(); cpu_physical_memory_rw(run->mmio.phys_addr, run->mmio.data, run->mmio.len, run->mmio.is_write); + qemu_mutex_unlock_iothread(); ret = 0; break; case KVM_EXIT_IRQ_WINDOW_OPEN: @@ -1520,7 +1528,9 @@ int kvm_cpu_exec(CPUArchState *env) break; case KVM_EXIT_SHUTDOWN: DPRINTF("shutdown\n"); + qemu_mutex_lock_iothread(); qemu_system_reset_request(); + qemu_mutex_unlock_iothread(); ret = EXCP_INTERRUPT; break; case KVM_EXIT_UNKNOWN: @@ -1533,11 +1543,15 @@ int kvm_cpu_exec(CPUArchState *env) break; default: DPRINTF("kvm_arch_handle_exit\n"); + qemu_mutex_lock_iothread(); ret = kvm_arch_handle_exit(env, run); + qemu_mutex_unlock_iothread(); break; } } while (ret == 0); + qemu_mutex_lock_iothread(); + if (ret < 0) { cpu_dump_state(env, stderr, fprintf, CPU_DUMP_CODE); vm_stop(RUN_STATE_INTERNAL_ERROR); diff --git a/target-i386/kvm.c b/target-i386/kvm.c index 0d0d8f6..0ad64d1 100644 --- a/target-i386/kvm.c +++ b/target-i386/kvm.c @@ -1631,7 +1631,10 @@ void kvm_arch_pre_run(CPUX86State *env, struct kvm_run *run) /* Inject NMI */ if (env->interrupt_request & CPU_INTERRUPT_NMI) { + qemu_mutex_lock_iothread(); env->interrupt_request &= ~CPU_INTERRUPT_NMI; + qemu_mutex_unlock_iothread(); + DPRINTF("injected NMI\n"); ret = kvm_vcpu_ioctl(env, KVM_NMI); if (ret < 0) { @@ -1641,6 +1644,8 @@ void kvm_arch_pre_run(CPUX86State *env, struct kvm_run *run) } if (!kvm_irqchip_in_kernel()) { + qemu_mutex_lock_iothread(); + /* Force the VCPU out of its inner loop to process any INIT requests * or pending TPR access reports. */ if (env->interrupt_request & @@ -1682,6 +1687,8 @@ void kvm_arch_pre_run(CPUX86State *env, struct kvm_run *run) DPRINTF("setting tpr\n"); run->cr8 = cpu_get_apic_tpr(env->apic_state); + + qemu_mutex_unlock_iothread(); } } diff --git a/target-ppc/kvm.c b/target-ppc/kvm.c index c09cc39..60d91a5 100644 --- a/target-ppc/kvm.c +++ b/target-ppc/kvm.c @@ -471,6 +471,8 @@ void kvm_arch_pre_run(CPUPPCState *env, struct kvm_run *run) int r; unsigned irq; + qemu_mutex_lock_iothread(); + /* PowerPC QEMU tracks the various core input pins (interrupt, critical * interrupt, reset, etc) in PPC-specific env->irq_input_state. */ if (!cap_interrupt_level && @@ -497,6 +499,8 @@ void kvm_arch_pre_run(CPUPPCState *env, struct kvm_run *run) /* We don't know if there are more interrupts pending after this. However, * the guest will return to userspace in the course of handling this one * anyways, so we will get a chance to deliver the rest. */ + + qemu_mutex_unlock_iothread(); } void kvm_arch_post_run(CPUPPCState *env, struct kvm_run *run)