From patchwork Thu Jul 12 02:00:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 942751 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="gR2RLtYC"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41Qzh71bHcz9s19 for ; Thu, 12 Jul 2018 12:01:43 +1000 (AEST) Received: from localhost ([::1]:56340 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdQvM-0000K8-P5 for incoming@patchwork.ozlabs.org; Wed, 11 Jul 2018 22:01:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34055) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdQup-0000Hw-0q for qemu-devel@nongnu.org; Wed, 11 Jul 2018 22:01:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fdQul-000067-Ty for qemu-devel@nongnu.org; Wed, 11 Jul 2018 22:01:07 -0400 Received: from mail-pl0-x242.google.com ([2607:f8b0:400e:c01::242]:40425) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fdQul-000057-L0 for qemu-devel@nongnu.org; Wed, 11 Jul 2018 22:01:03 -0400 Received: by mail-pl0-x242.google.com with SMTP id s17-v6so567949plp.7 for ; Wed, 11 Jul 2018 19:01:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=nHDJoSLdrBVEqInOUCDTsdkQHbhpwT3MWQJIhd4RI6s=; b=gR2RLtYCSRfi+sD5OidKQh57Ls3opuApOkR7xGeZR3wlSa1Lz17lhkXfkZKRwAvCMR VkZXMy0Y8tKZxJGtpQbQfbTq8scw9cVEO3O3t39P20PckvHQEU23unt1bG0S7Lo3kY+X oFfZKBKUDFKxYPDksanDlTrPhjOWXDhOwzMys3Zpb+TUOQ134A3FgcitR/Q1D0uHSGa/ Kf7wRiwzCrlB/lDmPLtYEge/lS05A5868IbR+ox5lUAf8NJ0ogU7gIL078twvJCBlGoo CMSBZFX8r313Q/FJVdfXzNWUFUT1IDecBH0k+km6J5yObxJjfOgb1nJpcVtbH82oidMy 9k3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=nHDJoSLdrBVEqInOUCDTsdkQHbhpwT3MWQJIhd4RI6s=; b=dmynsWirGkhHhB269MMjh6aYoss/XrYufOnSOPdr78wzkCEwTuS0RCeWqoatTg7001 hE2HK4kJ1oUik7bsKPiHbKb7UHBTBVDS7AmG2oQ9ElsqFLFMSPrIy+Xju3InKgPR5Zs6 Dpy8ResDQaje58soFbzjikiy7d0bQQ0Uxb/i0rJbO+j8xkpMJnf0m+c0Qfu5qZk/AjZ2 rSBN7xItj0V4oyxr0LWjjA7YVullka+zZpTp2SQPqG+GhXmznaffJCZPev87JP76nOYG zl+Bj3NpmiynwH3OnxfBGi4sRQyg7CRl+DSp16nWewLWdRugV7kUTUyaN2QcA3NUUPKX jrlA== X-Gm-Message-State: AOUpUlF/7SYagSBYG587c+C2ikcatDHF00kc/9zcvjKxdfq9S2GaMTwE iXk19pPCyCp5/SawkBr0keoqZQ== X-Google-Smtp-Source: AAOMgpf04lWcFW7AN6/05PYxzQnDjLtokoetl33bUdYIVK9xwEVOKwtf3nDArXupoG1u18hREymDKg== X-Received: by 2002:a17:902:7446:: with SMTP id e6-v6mr293209plt.161.1531360862351; Wed, 11 Jul 2018 19:01:02 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.123]) by smtp.googlemail.com with ESMTPSA id z9-v6sm311115pfi.150.2018.07.11.19.01.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 11 Jul 2018 19:01:01 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: qemu-devel@nongnu.org, kvm@vger.kernel.org Date: Thu, 12 Jul 2018 10:00:57 +0800 Message-Id: <1531360857-32374-1-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::242 Subject: [Qemu-devel] [PATCH v2] target-i386: coalesced PIO support for RTC X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Peng Hao , Eduardo Habkost , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Peng Hao Windows I/O, such as the real-time clock. The address register (port 0x70 in the RTC case) can use coalesced I/O, cutting the number of userspace exits by half when reading or writing the RTC. Guest access rtc like this: write register index to 0x70, then write or read data from 0x71. writing 0x70 port is just as index and do nothing else. So we can use coalesced mmio to handle this scene to reduce VM-EXIT time. In our environment, 12 windows guest running on a Skylake server: Before patch: IO Port Access Samples Samples% Time% Avg time 0x70:POUT 20675 46.04% 92.72% 67.15us ( +- 7.93% ) After patch: IO Port Access Samples Samples% Time% Avg time 0x70:POUT 17509 45.42% 42.08% 6.37us ( +- 20.37% ) Cc: Paolo Bonzini Cc: Radim Krčmář Cc: Eduardo Habkost Cc: Peng Hao Signed-off-by: Peng Hao Signed-off-by: Wanpeng Li --- v1 -> v2: * add the original author accel/kvm/kvm-all.c | 56 +++++++++++++++++++++++++++++++++++++++++++---- hw/timer/mc146818rtc.c | 8 +++++++ include/exec/memattrs.h | 1 + include/exec/memory.h | 5 +++++ include/sysemu/kvm.h | 8 +++++++ linux-headers/linux/kvm.h | 5 +++-- memory.c | 5 +++++ 7 files changed, 82 insertions(+), 6 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index eb7db92..7a12341 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -128,6 +128,7 @@ bool kvm_direct_msi_allowed; bool kvm_ioeventfd_any_length_allowed; bool kvm_msi_use_devid; static bool kvm_immediate_exit; +bool kvm_coalesced_pio_allowed; static const KVMCapabilityInfo kvm_required_capabilites[] = { KVM_CAP_INFO(USER_MEMORY), @@ -536,7 +537,7 @@ static void kvm_coalesce_mmio_region(MemoryListener *listener, zone.addr = start; zone.size = size; - zone.pad = 0; + zone.pio = 0; (void)kvm_vm_ioctl(s, KVM_REGISTER_COALESCED_MMIO, &zone); } @@ -553,7 +554,7 @@ static void kvm_uncoalesce_mmio_region(MemoryListener *listener, zone.addr = start; zone.size = size; - zone.pad = 0; + zone.pio = 0; (void)kvm_vm_ioctl(s, KVM_UNREGISTER_COALESCED_MMIO, &zone); } @@ -877,6 +878,45 @@ static void kvm_io_ioeventfd_del(MemoryListener *listener, } } +static void kvm_coalesce_io_add(MemoryListener *listener, + MemoryRegionSection *section, + hwaddr start, hwaddr size) +{ + KVMState *s = kvm_state; + + if (kvm_coalesced_pio_allowed) { + struct kvm_coalesced_mmio_zone zone; + + zone.addr = start; + zone.size = size; + zone.pio = 1; + + (void)kvm_vm_ioctl(s, KVM_REGISTER_COALESCED_MMIO, &zone); + } +} + +static void kvm_coalesce_io_del(MemoryListener *listener, + MemoryRegionSection *section, + hwaddr start, hwaddr size) +{ + KVMState *s = kvm_state; + + if (kvm_coalesced_pio_allowed) { + struct kvm_coalesced_mmio_zone zone; + + zone.addr = start; + zone.size = size; + zone.pio = 1; + + (void)kvm_vm_ioctl(s, KVM_UNREGISTER_COALESCED_MMIO, &zone); + } +} + +static MemoryListener kvm_coalesced_io_listener = { + .coalesced_mmio_add = kvm_coalesce_io_add, + .coalesced_mmio_del = kvm_coalesce_io_del, + .priority = 10, +}; void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml, AddressSpace *as, int as_id) { @@ -1615,6 +1655,8 @@ static int kvm_init(MachineState *ms) } s->coalesced_mmio = kvm_check_extension(s, KVM_CAP_COALESCED_MMIO); + kvm_coalesced_pio_allowed = s->coalesced_mmio && + kvm_check_extension(s, KVM_CAP_COALESCED_PIO); #ifdef KVM_CAP_VCPU_EVENTS s->vcpu_events = kvm_check_extension(s, KVM_CAP_VCPU_EVENTS); @@ -1694,6 +1736,8 @@ static int kvm_init(MachineState *ms) &address_space_memory, 0); memory_listener_register(&kvm_io_listener, &address_space_io); + memory_listener_register(&kvm_coalesced_io_listener, + &address_space_io); s->many_ioeventfds = kvm_check_many_ioeventfds(); @@ -1775,8 +1819,12 @@ void kvm_flush_coalesced_mmio_buffer(void) struct kvm_coalesced_mmio *ent; ent = &ring->coalesced_mmio[ring->first]; - - cpu_physical_memory_write(ent->phys_addr, ent->data, ent->len); + if (ent->pio) { + address_space_rw(&address_space_io, ent->phys_addr, + MEMTXATTRS_NONE, ent->data, ent->len, true); + } else { + cpu_physical_memory_write(ent->phys_addr, ent->data, ent->len); + } smp_wmb(); ring->first = (ring->first + 1) % KVM_COALESCED_MMIO_MAX; } diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c index 6f1f723..8bd8682 100644 --- a/hw/timer/mc146818rtc.c +++ b/hw/timer/mc146818rtc.c @@ -70,6 +70,7 @@ typedef struct RTCState { ISADevice parent_obj; MemoryRegion io; + MemoryRegion coalesced_io; uint8_t cmos_data[128]; uint8_t cmos_index; int32_t base_year; @@ -990,6 +991,13 @@ static void rtc_realizefn(DeviceState *dev, Error **errp) memory_region_init_io(&s->io, OBJECT(s), &cmos_ops, s, "rtc", 2); isa_register_ioport(isadev, &s->io, base); + if (memory_allow_coalesced_pio()) { + memory_region_set_flush_coalesced(&s->io); + memory_region_init_io(&s->coalesced_io, OBJECT(s), &cmos_ops, + s, "rtc1", 1); + isa_register_ioport(isadev, &s->coalesced_io, base); + memory_region_add_coalescing(&s->coalesced_io, 0, 1); + } qdev_set_legacy_instance_id(dev, base, 3); qemu_register_reset(rtc_reset, s); diff --git a/include/exec/memattrs.h b/include/exec/memattrs.h index d4a1642..97884d8 100644 --- a/include/exec/memattrs.h +++ b/include/exec/memattrs.h @@ -45,6 +45,7 @@ typedef struct MemTxAttrs { * from "didn't specify" if necessary). */ #define MEMTXATTRS_UNSPECIFIED ((MemTxAttrs) { .unspecified = 1 }) +#define MEMTXATTRS_NONE ((MemTxAttrs) { 0 }) /* New-style MMIO accessors can indicate that the transaction failed. * A zero (MEMTX_OK) response means success; anything else is a failure diff --git a/include/exec/memory.h b/include/exec/memory.h index 448d41a..2854907 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -1405,6 +1405,11 @@ void memory_region_set_flush_coalesced(MemoryRegion *mr); void memory_region_clear_flush_coalesced(MemoryRegion *mr); /** + * memory_allow_coalesced_pio: Check whether coalesced pio allowed. + */ +bool memory_allow_coalesced_pio(void); + +/** * memory_region_clear_global_locking: Declares that access processing does * not depend on the QEMU global lock. * diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index 0b64b8e..08366b2 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -45,6 +45,7 @@ extern bool kvm_readonly_mem_allowed; extern bool kvm_direct_msi_allowed; extern bool kvm_ioeventfd_any_length_allowed; extern bool kvm_msi_use_devid; +extern bool kvm_coalesced_pio_allowed; #define kvm_enabled() (kvm_allowed) /** @@ -167,6 +168,12 @@ extern bool kvm_msi_use_devid; */ #define kvm_msi_devid_required() (kvm_msi_use_devid) +/** + * kvm_coalesced_pio_enabled: + * Returns: true if KVM allow coalesced pio + */ +#define kvm_coalesced_pio_enabled() (kvm_coalesced_pio_allowed) + #else #define kvm_enabled() (0) @@ -184,6 +191,7 @@ extern bool kvm_msi_use_devid; #define kvm_direct_msi_enabled() (false) #define kvm_ioeventfd_any_length_enabled() (false) #define kvm_msi_devid_required() (false) +#define kvm_coalesced_pio_enabled() (false) #endif /* CONFIG_KVM_IS_POSSIBLE */ diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h index 98f389a..747b473 100644 --- a/linux-headers/linux/kvm.h +++ b/linux-headers/linux/kvm.h @@ -420,13 +420,13 @@ struct kvm_run { struct kvm_coalesced_mmio_zone { __u64 addr; __u32 size; - __u32 pad; + __u32 pio; }; struct kvm_coalesced_mmio { __u64 phys_addr; __u32 len; - __u32 pad; + __u32 pio; __u8 data[8]; }; @@ -949,6 +949,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_GET_MSR_FEATURES 153 #define KVM_CAP_HYPERV_EVENTFD 154 #define KVM_CAP_HYPERV_TLBFLUSH 155 +#define KVM_CAP_COALESCED_PIO 156 #ifdef KVM_CAP_IRQ_ROUTING diff --git a/memory.c b/memory.c index e9cd446..4a32817 100644 --- a/memory.c +++ b/memory.c @@ -2211,6 +2211,11 @@ void memory_region_clear_global_locking(MemoryRegion *mr) mr->global_locking = false; } +bool memory_allow_coalesced_pio(void) +{ + return kvm_enabled() && kvm_coalesced_pio_enabled(); +} + static bool userspace_eventfd_warning; void memory_region_add_eventfd(MemoryRegion *mr,