| Message ID | 20171006215244.27104-8-ehabkost@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | x86: Rework KVM-defaults compat code, enable kvm_pv_unhalt by default | expand |
On 06/10/2017 23:52, Eduardo Habkost wrote: > Commit f010bc643a (target-i386: add feature kvm_pv_unhalt) introduced the > kvm_pv_unhalt feature but didn't enable it by default. > > Without kvm_pv_unhalt we see a measurable degradation in scheduling > performance, so enabling it by default does make sense IMHO. This patch > just flips it to default to on by default. > > [With kvm_pv_unhalt disabled] > $ perf bench sched messaging -l 10000 > Total time: 8.573 [sec] > > [With kvm_pv_unhalt enabled] > $ perf bench sched messaging -l 10000 > Total time: 4.416 [sec] I cannot reproduce this: Host CPU model: Haswell-EP (Xeon E5-2697 v3 @ 2.60 GHz) Host physical CPUs: 56 (2 sockets 14 cores/sockets, 2 thread/core) Host Linux kernel: 4.14 (more or less :)) Host memory: 64 GB Guest Linux kernel: 4.10.13 QEMU command line: /usr/libexec/qemu-kvm -cpu host,+kvm_pv_unhalt -M q35 \ -m XXX -smp YYY \ /path/to/vm.qcow2 (XXX = MiB of guest memory, YYY = number of guest processors) "perf bench sched messaging -l 50000" has the following result for me: Guest vCPUs Guest memory without PV unhalt with PV unhalt 1*96 32 GB 24.6 s 24.2 s 2*96 24 GB 47.9 s (both VMs) 46.8 s (both VMs) 2*48 16 GB 50.4 s (both VMs) 49.3 s (both VMs) 4*24 12 GB 82.1 - 89.0 s 82.3 - 88.8 s 4*4 12 GB 87.2 - 91.3 s 90.3 - 94.9 s All but the first line are running the benchmark in multiple guests, concurrently. The improvement seems to be about 2-3% for guests larger than 1 NUMA node, and zero or negative for smaller guests (especially as the host is no longer overcommitted). The difference for large NUMA guests is small but I ran the benchmark multiple times and it is statistically significant---but not as large as what Alex reported. Paolo
On 09.10.17 16:40, Paolo Bonzini wrote: > On 06/10/2017 23:52, Eduardo Habkost wrote: >> Commit f010bc643a (target-i386: add feature kvm_pv_unhalt) introduced the >> kvm_pv_unhalt feature but didn't enable it by default. >> >> Without kvm_pv_unhalt we see a measurable degradation in scheduling >> performance, so enabling it by default does make sense IMHO. This patch >> just flips it to default to on by default. >> >> [With kvm_pv_unhalt disabled] >> $ perf bench sched messaging -l 10000 >> Total time: 8.573 [sec] >> >> [With kvm_pv_unhalt enabled] >> $ perf bench sched messaging -l 10000 >> Total time: 4.416 [sec] > > I cannot reproduce this: > > Host CPU model: Haswell-EP (Xeon E5-2697 v3 @ 2.60 GHz) > Host physical CPUs: 56 (2 sockets 14 cores/sockets, 2 thread/core) > Host Linux kernel: 4.14 (more or less :)) > Host memory: 64 GB > Guest Linux kernel: 4.10.13 > > QEMU command line: > > /usr/libexec/qemu-kvm -cpu host,+kvm_pv_unhalt -M q35 \ -cpu host already enables kvm_pv_unhalt. It's actually the only CPU type that does :). The problem arises when you use libvirt with automatic cpu type detection, as it then generates something that doesn't do -cpu host. Alex
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h index d2742dd0bc..19b42de224 100644 --- a/include/hw/i386/pc.h +++ b/include/hw/i386/pc.h @@ -375,6 +375,10 @@ bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *); .driver = TYPE_X86_CPU,\ .property = "x-hv-max-vps",\ .value = "0x40",\ + },{\ + .driver = TYPE_X86_CPU,\ + .property = "x-kvm-auto-enable-pv-unhalt",\ + .value = "off",\ }, #define PC_COMPAT_2_9 \ diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 7e5bf86921..db1eecff00 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -1276,6 +1276,9 @@ struct X86CPU { /* KVM automatically enables kvm-pv-eoi if not explicitly disabled by user */ bool kvm_auto_enable_pv_eoi; + /* KVM automatically enables kvm-pv-unhalt if not explicitly disabled */ + bool kvm_auto_enable_pv_unhalt; + /* Number of physical address bits supported */ uint32_t phys_bits; diff --git a/target/i386/cpu.c b/target/i386/cpu.c index 2160738a37..7461c2a25e 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -3515,6 +3515,11 @@ static void x86_cpu_expand_features(X86CPU *cpu, Error **errp) x86_cpu_expand_feature(cpu, FEAT_KVM, (1 << KVM_FEATURE_PV_EOI), (1 << KVM_FEATURE_PV_EOI)); } + + if (cpu->kvm_auto_enable_pv_unhalt) { + x86_cpu_expand_feature(cpu, FEAT_KVM, (1 << KVM_FEATURE_PV_UNHALT), + (1 << KVM_FEATURE_PV_UNHALT)); + } } /*TODO: Now cpu->max_features doesn't overwrite features @@ -4164,6 +4169,8 @@ static Property x86_cpu_properties[] = { X86CPU, kvm_auto_enable_x2apic, true), DEFINE_PROP_BOOL("x-kvm-auto-enable-pv-eoi", X86CPU, kvm_auto_enable_pv_eoi, true), + DEFINE_PROP_BOOL("x-kvm-auto-enable-pv-unhalt", + X86CPU, kvm_auto_enable_pv_unhalt, true), DEFINE_PROP_BOOL("vmware-cpuid-freq", X86CPU, vmware_cpuid_freq, true), DEFINE_PROP_BOOL("tcg-cpuid", X86CPU, expose_tcg, true), diff --git a/qemu-doc.texi b/qemu-doc.texi index be45b6b6f6..73c831383a 100644 --- a/qemu-doc.texi +++ b/qemu-doc.texi @@ -2355,7 +2355,7 @@ Run the emulation in single step mode. @section KVM kernel module On x86_64 hosts, the default set of CPU features enabled by the KVM accelerator -require the host to be running Linux v3.6 or newer. +require the host to be running Linux v3.12 or newer. @include qemu-tech.texi
Original description of a patch by Alexander Graf that did something similar: Commit f010bc643a (target-i386: add feature kvm_pv_unhalt) introduced the kvm_pv_unhalt feature but didn't enable it by default. Without kvm_pv_unhalt we see a measurable degradation in scheduling performance, so enabling it by default does make sense IMHO. This patch just flips it to default to on by default. [With kvm_pv_unhalt disabled] $ perf bench sched messaging -l 10000 Total time: 8.573 [sec] [With kvm_pv_unhalt enabled] $ perf bench sched messaging -l 10000 Total time: 4.416 [sec] This patch does the same as that patch, but using a kvm_auto_enable_pv_unhalt flag to keep compatibility on older machine-types. kvm_pv_unhalt support was added to Linux on v3.12 (Linux commit 6aef266c6e17b798a1740cf70cd34f90664740b3), so update the system requirements section in qemu-doc.texi accordingly. Suggested-by: Alexander Graf <agraf@suse.de> Cc: Alexander Graf <agraf@suse.de> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com> --- include/hw/i386/pc.h | 4 ++++ target/i386/cpu.h | 3 +++ target/i386/cpu.c | 7 +++++++ qemu-doc.texi | 2 +- 4 files changed, 15 insertions(+), 1 deletion(-)