From patchwork Wed Jun 24 16:25:07 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 488148 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id ACE18140319 for ; Thu, 25 Jun 2015 02:32:45 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b=Jvr6ASbD; dkim-atps=neutral Received: from localhost ([::1]:51865 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z7nbO-0001oL-JK for incoming@patchwork.ozlabs.org; Wed, 24 Jun 2015 12:32:42 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59961) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z7nb9-0001WE-5G for qemu-devel@nongnu.org; Wed, 24 Jun 2015 12:32:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z7nb5-0007Xh-KY for qemu-devel@nongnu.org; Wed, 24 Jun 2015 12:32:27 -0400 Received: from mail-wi0-x235.google.com ([2a00:1450:400c:c05::235]:35728) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z7nb5-0007Xc-B5 for qemu-devel@nongnu.org; Wed, 24 Jun 2015 12:32:23 -0400 Received: by wiga1 with SMTP id a1so140806513wig.0 for ; Wed, 24 Jun 2015 09:32:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=l0czh2YBtqfOUfnhx/MIYY+Z6hY56EDM6nCFWv2JGC0=; b=Jvr6ASbDxPGUBKiZ+5uBlQXoUXLWNUvBqB0rDsEE50WW21YBHLec9NJJfBP+NVqKuS /BwN3EsYnHh0zXbD6Q/i3CxiLyDY0NoKlvESM+UvtU5ujmthmjNutY3H2wp4nOCocg2I jrSB5Fftc4lrvM5HA3vE7SV0XxFifr0bOQha70l2pLawakVYoGhGZA23w9VI1bQskHJ9 35gOuzq5vqpNcy3/xue0aGnhVcXIpHUO6Dwwpg4kiBgqxtnSX5HBvDVxMExU3vdd3sol ylR0w++KaNgzJY/36UtGlJGWnyqDLzYhV+MxpeO/S70awelBRcNyoeYy0HmNu3yokmHI wzKQ== X-Received: by 10.180.94.35 with SMTP id cz3mr6461843wib.85.1435163122109; Wed, 24 Jun 2015 09:25:22 -0700 (PDT) Received: from 640k.localdomain (dynamic-adsl-94-39-132-37.clienti.tiscali.it. [94.39.132.37]) by mx.google.com with ESMTPSA id gt10sm3305953wib.20.2015.06.24.09.25.20 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Jun 2015 09:25:21 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Wed, 24 Jun 2015 18:25:07 +0200 Message-Id: <1435163110-2724-7-git-send-email-pbonzini@redhat.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1435163110-2724-1-git-send-email-pbonzini@redhat.com> References: <1435163110-2724-1-git-send-email-pbonzini@redhat.com> X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2a00:1450:400c:c05::235 Cc: borntraeger@de.ibm.com, famz@redhat.com, Jan Kiszka Subject: [Qemu-devel] [PATCH 6/9] kvm: First step to push iothread lock out of inner run loop X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Jan Kiszka This opens the path to get rid of the iothread lock on vmexits in KVM mode. On x86, the in-kernel irqchips has to be used because we otherwise need to synchronize APIC and other per-cpu state accesses that could be changed concurrently. Regarding pre/post-run callbacks, s390x and ARM should be fine without specific locking as the callbacks are empty. MIPS and POWER require locking for the pre-run callback. For the handle_exit callback, it is non-empty in x86, POWER and s390. Some POWER cases could do without the locking, but it is left in place for now. Signed-off-by: Jan Kiszka Signed-off-by: Paolo Bonzini Message-Id: <1434646046-27150-7-git-send-email-pbonzini@redhat.com> --- kvm-all.c | 10 ++++++++-- target-i386/kvm.c | 24 ++++++++++++++++++++++++ target-mips/kvm.c | 4 ++++ target-ppc/kvm.c | 7 +++++++ target-s390x/kvm.c | 3 +++ 5 files changed, 46 insertions(+), 2 deletions(-) diff --git a/kvm-all.c b/kvm-all.c index 53e01d4..af3d10b 100644 --- a/kvm-all.c +++ b/kvm-all.c @@ -1752,6 +1752,8 @@ int kvm_cpu_exec(CPUState *cpu) return EXCP_HLT; } + qemu_mutex_unlock_iothread(); + do { MemTxAttrs attrs; @@ -1770,11 +1772,9 @@ int kvm_cpu_exec(CPUState *cpu) */ qemu_cpu_kick_self(); } - qemu_mutex_unlock_iothread(); run_ret = kvm_vcpu_ioctl(cpu, KVM_RUN, 0); - qemu_mutex_lock_iothread(); attrs = kvm_arch_post_run(cpu, run); if (run_ret < 0) { @@ -1801,20 +1801,24 @@ int kvm_cpu_exec(CPUState *cpu) switch (run->exit_reason) { case KVM_EXIT_IO: DPRINTF("handle_io\n"); + qemu_mutex_lock_iothread(); kvm_handle_io(run->io.port, attrs, (uint8_t *)run + run->io.data_offset, run->io.direction, run->io.size, run->io.count); + qemu_mutex_unlock_iothread(); ret = 0; break; case KVM_EXIT_MMIO: DPRINTF("handle_mmio\n"); + qemu_mutex_lock_iothread(); address_space_rw(&address_space_memory, run->mmio.phys_addr, attrs, run->mmio.data, run->mmio.len, run->mmio.is_write); + qemu_mutex_unlock_iothread(); ret = 0; break; case KVM_EXIT_IRQ_WINDOW_OPEN: @@ -1857,6 +1861,8 @@ int kvm_cpu_exec(CPUState *cpu) } } while (ret == 0); + qemu_mutex_lock_iothread(); + if (ret < 0) { cpu_dump_state(cpu, stderr, fprintf, CPU_DUMP_CODE); vm_stop(RUN_STATE_INTERNAL_ERROR); diff --git a/target-i386/kvm.c b/target-i386/kvm.c index 5a236e3..45fd3b0 100644 --- a/target-i386/kvm.c +++ b/target-i386/kvm.c @@ -2192,7 +2192,10 @@ void kvm_arch_pre_run(CPUState *cpu, struct kvm_run *run) /* Inject NMI */ if (cpu->interrupt_request & CPU_INTERRUPT_NMI) { + qemu_mutex_lock_iothread(); cpu->interrupt_request &= ~CPU_INTERRUPT_NMI; + qemu_mutex_unlock_iothread(); + DPRINTF("injected NMI\n"); ret = kvm_vcpu_ioctl(cpu, KVM_NMI); if (ret < 0) { @@ -2201,6 +2204,10 @@ void kvm_arch_pre_run(CPUState *cpu, struct kvm_run *run) } } + if (!kvm_irqchip_in_kernel()) { + qemu_mutex_lock_iothread(); + } + /* Force the VCPU out of its inner loop to process any INIT requests * or (for userspace APIC, but it is cheap to combine the checks here) * pending TPR access reports. @@ -2244,6 +2251,8 @@ void kvm_arch_pre_run(CPUState *cpu, struct kvm_run *run) DPRINTF("setting tpr\n"); run->cr8 = cpu_get_apic_tpr(x86_cpu->apic_state); + + qemu_mutex_unlock_iothread(); } } @@ -2257,8 +2266,17 @@ MemTxAttrs kvm_arch_post_run(CPUState *cpu, struct kvm_run *run) } else { env->eflags &= ~IF_MASK; } + + /* We need to protect the apic state against concurrent accesses from + * different threads in case the userspace irqchip is used. */ + if (!kvm_irqchip_in_kernel()) { + qemu_mutex_lock_iothread(); + } cpu_set_apic_tpr(x86_cpu->apic_state, run->cr8); cpu_set_apic_base(x86_cpu->apic_state, run->apic_base); + if (!kvm_irqchip_in_kernel()) { + qemu_mutex_unlock_iothread(); + } return cpu_get_mem_attrs(env); } @@ -2551,13 +2569,17 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) switch (run->exit_reason) { case KVM_EXIT_HLT: DPRINTF("handle_hlt\n"); + qemu_mutex_lock_iothread(); ret = kvm_handle_halt(cpu); + qemu_mutex_unlock_iothread(); break; case KVM_EXIT_SET_TPR: ret = 0; break; case KVM_EXIT_TPR_ACCESS: + qemu_mutex_lock_iothread(); ret = kvm_handle_tpr_access(cpu); + qemu_mutex_unlock_iothread(); break; case KVM_EXIT_FAIL_ENTRY: code = run->fail_entry.hardware_entry_failure_reason; @@ -2583,7 +2605,9 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) break; case KVM_EXIT_DEBUG: DPRINTF("kvm_exit_debug\n"); + qemu_mutex_lock_iothread(); ret = kvm_handle_debug(cpu, &run->debug.arch); + qemu_mutex_unlock_iothread(); break; default: fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason); diff --git a/target-mips/kvm.c b/target-mips/kvm.c index 948619f..7d2293d 100644 --- a/target-mips/kvm.c +++ b/target-mips/kvm.c @@ -99,6 +99,8 @@ void kvm_arch_pre_run(CPUState *cs, struct kvm_run *run) int r; struct kvm_mips_interrupt intr; + qemu_mutex_lock_iothread(); + if ((cs->interrupt_request & CPU_INTERRUPT_HARD) && cpu_mips_io_interrupts_pending(cpu)) { intr.cpu = -1; @@ -109,6 +111,8 @@ void kvm_arch_pre_run(CPUState *cs, struct kvm_run *run) __func__, cs->cpu_index, intr.irq); } } + + qemu_mutex_unlock_iothread(); } MemTxAttrs kvm_arch_post_run(CPUState *cs, struct kvm_run *run) diff --git a/target-ppc/kvm.c b/target-ppc/kvm.c index afb4696..ddf469f 100644 --- a/target-ppc/kvm.c +++ b/target-ppc/kvm.c @@ -1242,6 +1242,8 @@ void kvm_arch_pre_run(CPUState *cs, struct kvm_run *run) int r; unsigned irq; + qemu_mutex_lock_iothread(); + /* PowerPC QEMU tracks the various core input pins (interrupt, critical * interrupt, reset, etc) in PPC-specific env->irq_input_state. */ if (!cap_interrupt_level && @@ -1269,6 +1271,8 @@ void kvm_arch_pre_run(CPUState *cs, struct kvm_run *run) /* We don't know if there are more interrupts pending after this. However, * the guest will return to userspace in the course of handling this one * anyways, so we will get a chance to deliver the rest. */ + + qemu_mutex_unlock_iothread(); } MemTxAttrs kvm_arch_post_run(CPUState *cs, struct kvm_run *run) @@ -1570,6 +1574,8 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) CPUPPCState *env = &cpu->env; int ret; + qemu_mutex_lock_iothread(); + switch (run->exit_reason) { case KVM_EXIT_DCR: if (run->dcr.is_write) { @@ -1620,6 +1626,7 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) break; } + qemu_mutex_unlock_iothread(); return ret; } diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c index b02ff8d..1ebcd18 100644 --- a/target-s390x/kvm.c +++ b/target-s390x/kvm.c @@ -2007,6 +2007,8 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) S390CPU *cpu = S390_CPU(cs); int ret = 0; + qemu_mutex_lock_iothread(); + switch (run->exit_reason) { case KVM_EXIT_S390_SIEIC: ret = handle_intercept(cpu); @@ -2027,6 +2029,7 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) fprintf(stderr, "Unknown KVM exit: %d\n", run->exit_reason); break; } + qemu_mutex_unlock_iothread(); if (ret == 0) { ret = EXCP_INTERRUPT;