| Message ID | cover.1596768588.git.haibo.xu@linaro.org |
|---|---|
| Headers | show |
| Series | target/arm: Add vSPE support to KVM guest | expand |
On Fri, Aug 07, 2020 at 08:10:30AM +0000, Haibo Xu wrote: > This series add support for SPE(Statistical Profiling Extension)[1] > in KVM guest. It's based on Andrew Murray's kernel KVM patches V2[2], > and has been tested to ensure that guest can use SPE with valid data. > E.g. > > In host: > $ ./qemu-system-aarch64 \ > -cpu host -M virt,accel=kvm,gic-version=3 -nographic -m 2048M \ > -kernel ./Image-new \ > -initrd /boot/initrd.img-5.6.0-rc2+ \ > -append "root=/dev/vda rw console=ttyAMA0" -nodefaults -serial stdio\ > -drive if=none,file=./xenial.rootfs.ext4,id=hd0,format=raw \ > -device virtio-blk-device,drive=hd0 \ > > In guest: > $ perf record -e arm_spe/ts_enable=1,pa_enable=1,pct_enable=1/ \ > dd if=/dev/zero of=/dev/null count=1000 > $ perf report --dump-raw-trace > spe_buf.txt > > The spe_buf.txt should contain similar data as below: > > . ... ARM SPE data: size 135944 bytes > . 00000000: b0 f4 d3 29 10 00 80 ff a0 PC 0xff80001029d3f4 el1 ns=1 > . 00000009: 99 0b 00 LAT 11 ISSUE > . 0000000c: 98 0d 00 LAT 13 TOT > . 0000000f: 52 16 00 EV RETIRED L1D-ACCESS TLB-ACCESS > . 00000012: 49 00 LD > . 00000014: b2 d0 40 d8 70 00 00 ff 00 VA 0xff000070d840d0 > . 0000001d: 9a 01 00 LAT 1 XLAT > . 00000020: 00 00 00 PAD > . 00000023: 71 a5 1f b3 20 14 00 00 00 TS 86447955877 > . 0000002c: b0 7c f9 29 10 00 80 ff a0 PC 0xff80001029f97c el1 ns=1 > . 00000035: 99 02 00 LAT 2 ISSUE > . 00000038: 98 03 00 LAT 3 TOT > . 0000003b: 52 02 00 EV RETIRED > . 0000003e: 48 00 INSN-OTHER > . 00000040: 00 00 00 PAD > . 00000043: 71 ef 1f b3 20 14 00 00 00 TS 86447955951 > . 0000004c: b0 f0 e9 29 10 00 80 ff a0 PC 0xff80001029e9f0 el1 ns=1 > . 00000055: 99 02 00 LAT 2 ISSUE > . 00000058: 98 03 00 LAT 3 TOT > . 0000005b: 52 02 00 EV RETIRED > > If you want to disable the vSPE support, you can use the 'spe=off' cpu > property: > > ./qemu-system-aarch64 \ > -cpu host,spe=off -M virt,accel=kvm,gic-version=3 -nographic -m 2048M \ > -kernel ./Image-new \ > -initrd /boot/initrd.img-5.6.0-rc2+ \ > -append "root=/dev/vda rw console=ttyAMA0" -nodefaults -serial stdio\ > -drive if=none,file=./xenial.rootfs.ext4,id=hd0,format=raw \ > -device virtio-blk-device,drive=hd0 \ > > Note: > (1) Since the kernel patches are still under review, some of the macros > in the header files may be changed after merging. We may need to > update them accordingly. > (2) These patches only add vSPE support in KVM mode, for TCG mode, I'm > not sure whether we need to support it. > (3) Just followed the 'pmu' property, we only allow this feature to be > removed from CPUs which enable it by default. But since the SPE is > an optional feature extension for Armv8.2, I think a better way may > be to disable it by default, and only enable it when the host cpu > do have the feature. * When not using KVM, if TCG supports SPE emulation, then it should be enabled by default with '-cpu max'. * When using KVM, if KVM supports the feature, then it should be enabled by default with '-cpu max' or '-cpu host' (which are currently the only ways to use KVM on AArch64) * When using KVM, if KVM does not support the feature (either the CPU doesn't support it, or the version of KVM in use doesn't know what to do with it), then the feature should obviously be disabled for the VM by default, since there's no way to enable it. > > [1]https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/ > posts/statistical-profiling-extension-for-armv8-a > [2]https://www.spinics.net/lists/arm-kernel/msg776228.html > > Haibo Xu (7): > update Linux headers with new vSPE macros > target/arm/kvm: spe: Add helper to detect SPE when using KVM > target/arm/cpu: spe: Add an option to turn on/off vSPE support > target/arm/kvm: spe: Unify device attr operatioin helper > target/arm/kvm: spe: Add device init and set_irq operations > hw/arm/virt: spe: Add SPE fdt binding for virt machine > target/arm/cpu: spe: Enable spe to work with host cpu > > hw/arm/virt-acpi-build.c | 3 +++ > hw/arm/virt.c | 42 ++++++++++++++++++++++++++++++ > include/hw/acpi/acpi-defs.h | 1 + > include/hw/arm/virt.h | 1 + > linux-headers/asm-arm64/kvm.h | 4 +++ > linux-headers/linux/kvm.h | 2 ++ > target/arm/cpu.c | 34 +++++++++++++++++++++++++ > target/arm/cpu.h | 5 ++++ > target/arm/kvm.c | 11 ++++++++ > target/arm/kvm64.c | 48 ++++++++++++++++++++++++++++++++--- > target/arm/kvm_arm.h | 18 +++++++++++++ This series should also be updating these files: docs/system/arm/cpu-features.rst target/arm/monitor.c tests/qtest/arm-cpu-features.c Thanks, drew > 11 files changed, 166 insertions(+), 3 deletions(-) > > -- > 2.17.1 > >
Hi Haibo, On 8/7/20 10:10 AM, Haibo Xu wrote: > This series add support for SPE(Statistical Profiling Extension)[1] > in KVM guest. It's based on Andrew Murray's kernel KVM patches V2[2], > and has been tested to ensure that guest can use SPE with valid data. > E.g. > > In host: > $ ./qemu-system-aarch64 \ > -cpu host -M virt,accel=kvm,gic-version=3 -nographic -m 2048M \ > -kernel ./Image-new \ > -initrd /boot/initrd.img-5.6.0-rc2+ \ > -append "root=/dev/vda rw console=ttyAMA0" -nodefaults -serial stdio\ > -drive if=none,file=./xenial.rootfs.ext4,id=hd0,format=raw \ > -device virtio-blk-device,drive=hd0 \ > > In guest: > $ perf record -e arm_spe/ts_enable=1,pa_enable=1,pct_enable=1/ \ > dd if=/dev/zero of=/dev/null count=1000 > $ perf report --dump-raw-trace > spe_buf.txt > > The spe_buf.txt should contain similar data as below: > > . ... ARM SPE data: size 135944 bytes > . 00000000: b0 f4 d3 29 10 00 80 ff a0 PC 0xff80001029d3f4 el1 ns=1 > . 00000009: 99 0b 00 LAT 11 ISSUE > . 0000000c: 98 0d 00 LAT 13 TOT > . 0000000f: 52 16 00 EV RETIRED L1D-ACCESS TLB-ACCESS > . 00000012: 49 00 LD > . 00000014: b2 d0 40 d8 70 00 00 ff 00 VA 0xff000070d840d0 > . 0000001d: 9a 01 00 LAT 1 XLAT > . 00000020: 00 00 00 PAD > . 00000023: 71 a5 1f b3 20 14 00 00 00 TS 86447955877 > . 0000002c: b0 7c f9 29 10 00 80 ff a0 PC 0xff80001029f97c el1 ns=1 > . 00000035: 99 02 00 LAT 2 ISSUE > . 00000038: 98 03 00 LAT 3 TOT > . 0000003b: 52 02 00 EV RETIRED > . 0000003e: 48 00 INSN-OTHER > . 00000040: 00 00 00 PAD > . 00000043: 71 ef 1f b3 20 14 00 00 00 TS 86447955951 > . 0000004c: b0 f0 e9 29 10 00 80 ff a0 PC 0xff80001029e9f0 el1 ns=1 > . 00000055: 99 02 00 LAT 2 ISSUE > . 00000058: 98 03 00 LAT 3 TOT > . 0000005b: 52 02 00 EV RETIRED > > If you want to disable the vSPE support, you can use the 'spe=off' cpu > property: > > ./qemu-system-aarch64 \ > -cpu host,spe=off -M virt,accel=kvm,gic-version=3 -nographic -m 2048M \ > -kernel ./Image-new \ > -initrd /boot/initrd.img-5.6.0-rc2+ \ > -append "root=/dev/vda rw console=ttyAMA0" -nodefaults -serial stdio\ > -drive if=none,file=./xenial.rootfs.ext4,id=hd0,format=raw \ > -device virtio-blk-device,drive=hd0 \ > > Note: > (1) Since the kernel patches are still under review, some of the macros > in the header files may be changed after merging. We may need to > update them accordingly. to be more explicit one needs to replace on the kernel 5.5-rc2 based series -#define KVM_CAP_ARM_SPE_V1 179 +#define KVM_CAP_ARM_SPE_V1 184 I got misleaded ;-) + Andrew in CC as he contributed the kernel part. For information, I have been working on a kvm unit test series for testing SPE. I will send an RFC, most probably this week. At the moment I still face some weirdness such as some unexpected Service state in the syndrome register. Anyway I will share the existing code so that we can discuss the issues. Are there any plans to respin the kernel series Thanks Eric > (2) These patches only add vSPE support in KVM mode, for TCG mode, I'm > not sure whether we need to support it. > (3) Just followed the 'pmu' property, we only allow this feature to be > removed from CPUs which enable it by default. But since the SPE is > an optional feature extension for Armv8.2, I think a better way may > be to disable it by default, and only enable it when the host cpu > do have the feature. > > [1]https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/ > posts/statistical-profiling-extension-for-armv8-a > [2]https://www.spinics.net/lists/arm-kernel/msg776228.html > > Haibo Xu (7): > update Linux headers with new vSPE macros > target/arm/kvm: spe: Add helper to detect SPE when using KVM > target/arm/cpu: spe: Add an option to turn on/off vSPE support > target/arm/kvm: spe: Unify device attr operatioin helper > target/arm/kvm: spe: Add device init and set_irq operations > hw/arm/virt: spe: Add SPE fdt binding for virt machine > target/arm/cpu: spe: Enable spe to work with host cpu > > hw/arm/virt-acpi-build.c | 3 +++ > hw/arm/virt.c | 42 ++++++++++++++++++++++++++++++ > include/hw/acpi/acpi-defs.h | 1 + > include/hw/arm/virt.h | 1 + > linux-headers/asm-arm64/kvm.h | 4 +++ > linux-headers/linux/kvm.h | 2 ++ > target/arm/cpu.c | 34 +++++++++++++++++++++++++ > target/arm/cpu.h | 5 ++++ > target/arm/kvm.c | 11 ++++++++ > target/arm/kvm64.c | 48 ++++++++++++++++++++++++++++++++--- > target/arm/kvm_arm.h | 18 +++++++++++++ > 11 files changed, 166 insertions(+), 3 deletions(-) >
On Mon, 31 Aug 2020 at 15:56, Auger Eric <eric.auger@redhat.com> wrote: > > Hi Haibo, > > On 8/7/20 10:10 AM, Haibo Xu wrote: > > This series add support for SPE(Statistical Profiling Extension)[1] > > in KVM guest. It's based on Andrew Murray's kernel KVM patches V2[2], > > and has been tested to ensure that guest can use SPE with valid data. > > E.g. > > > > In host: > > $ ./qemu-system-aarch64 \ > > -cpu host -M virt,accel=kvm,gic-version=3 -nographic -m 2048M \ > > -kernel ./Image-new \ > > -initrd /boot/initrd.img-5.6.0-rc2+ \ > > -append "root=/dev/vda rw console=ttyAMA0" -nodefaults -serial stdio\ > > -drive if=none,file=./xenial.rootfs.ext4,id=hd0,format=raw \ > > -device virtio-blk-device,drive=hd0 \ > > > > In guest: > > $ perf record -e arm_spe/ts_enable=1,pa_enable=1,pct_enable=1/ \ > > dd if=/dev/zero of=/dev/null count=1000 > > $ perf report --dump-raw-trace > spe_buf.txt > > > > The spe_buf.txt should contain similar data as below: > > > > . ... ARM SPE data: size 135944 bytes > > . 00000000: b0 f4 d3 29 10 00 80 ff a0 PC 0xff80001029d3f4 el1 ns=1 > > . 00000009: 99 0b 00 LAT 11 ISSUE > > . 0000000c: 98 0d 00 LAT 13 TOT > > . 0000000f: 52 16 00 EV RETIRED L1D-ACCESS TLB-ACCESS > > . 00000012: 49 00 LD > > . 00000014: b2 d0 40 d8 70 00 00 ff 00 VA 0xff000070d840d0 > > . 0000001d: 9a 01 00 LAT 1 XLAT > > . 00000020: 00 00 00 PAD > > . 00000023: 71 a5 1f b3 20 14 00 00 00 TS 86447955877 > > . 0000002c: b0 7c f9 29 10 00 80 ff a0 PC 0xff80001029f97c el1 ns=1 > > . 00000035: 99 02 00 LAT 2 ISSUE > > . 00000038: 98 03 00 LAT 3 TOT > > . 0000003b: 52 02 00 EV RETIRED > > . 0000003e: 48 00 INSN-OTHER > > . 00000040: 00 00 00 PAD > > . 00000043: 71 ef 1f b3 20 14 00 00 00 TS 86447955951 > > . 0000004c: b0 f0 e9 29 10 00 80 ff a0 PC 0xff80001029e9f0 el1 ns=1 > > . 00000055: 99 02 00 LAT 2 ISSUE > > . 00000058: 98 03 00 LAT 3 TOT > > . 0000005b: 52 02 00 EV RETIRED > > > > If you want to disable the vSPE support, you can use the 'spe=off' cpu > > property: > > > > ./qemu-system-aarch64 \ > > -cpu host,spe=off -M virt,accel=kvm,gic-version=3 -nographic -m 2048M \ > > -kernel ./Image-new \ > > -initrd /boot/initrd.img-5.6.0-rc2+ \ > > -append "root=/dev/vda rw console=ttyAMA0" -nodefaults -serial stdio\ > > -drive if=none,file=./xenial.rootfs.ext4,id=hd0,format=raw \ > > -device virtio-blk-device,drive=hd0 \ > > > > Note: > > (1) Since the kernel patches are still under review, some of the macros > > in the header files may be changed after merging. We may need to > > update them accordingly. > to be more explicit one needs to replace on the kernel 5.5-rc2 based series > > -#define KVM_CAP_ARM_SPE_V1 179 > +#define KVM_CAP_ARM_SPE_V1 184 > > I got misleaded ;-) > > + Andrew in CC as he contributed the kernel part. > > For information, I have been working on a kvm unit test series for > testing SPE. I will send an RFC, most probably this week. At the moment > I still face some weirdness such as some unexpected Service state in the > syndrome register. Anyway I will share the existing code so that we can > discuss the issues. > > Are there any plans to respin the kernel series > > Thanks > > Eric > Hi Eric, Thanks for elaborating on the above macro definition! The next version of the kernel series are supposed to be sent out in mid-September, and it should not change so much except for some macro definitions. Regards, Haibo > > (2) These patches only add vSPE support in KVM mode, for TCG mode, I'm > > not sure whether we need to support it. > > (3) Just followed the 'pmu' property, we only allow this feature to be > > removed from CPUs which enable it by default. But since the SPE is > > an optional feature extension for Armv8.2, I think a better way may > > be to disable it by default, and only enable it when the host cpu > > do have the feature. > > > > [1]https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/ > > posts/statistical-profiling-extension-for-armv8-a > > [2]https://www.spinics.net/lists/arm-kernel/msg776228.html > > > > Haibo Xu (7): > > update Linux headers with new vSPE macros > > target/arm/kvm: spe: Add helper to detect SPE when using KVM > > target/arm/cpu: spe: Add an option to turn on/off vSPE support > > target/arm/kvm: spe: Unify device attr operatioin helper > > target/arm/kvm: spe: Add device init and set_irq operations > > hw/arm/virt: spe: Add SPE fdt binding for virt machine > > target/arm/cpu: spe: Enable spe to work with host cpu > > > > hw/arm/virt-acpi-build.c | 3 +++ > > hw/arm/virt.c | 42 ++++++++++++++++++++++++++++++ > > include/hw/acpi/acpi-defs.h | 1 + > > include/hw/arm/virt.h | 1 + > > linux-headers/asm-arm64/kvm.h | 4 +++ > > linux-headers/linux/kvm.h | 2 ++ > > target/arm/cpu.c | 34 +++++++++++++++++++++++++ > > target/arm/cpu.h | 5 ++++ > > target/arm/kvm.c | 11 ++++++++ > > target/arm/kvm64.c | 48 ++++++++++++++++++++++++++++++++--- > > target/arm/kvm_arm.h | 18 +++++++++++++ > > 11 files changed, 166 insertions(+), 3 deletions(-) > > >
This series add support for SPE(Statistical Profiling Extension)[1] in KVM guest. It's based on Andrew Murray's kernel KVM patches V2[2], and has been tested to ensure that guest can use SPE with valid data. E.g. In host: $ ./qemu-system-aarch64 \ -cpu host -M virt,accel=kvm,gic-version=3 -nographic -m 2048M \ -kernel ./Image-new \ -initrd /boot/initrd.img-5.6.0-rc2+ \ -append "root=/dev/vda rw console=ttyAMA0" -nodefaults -serial stdio\ -drive if=none,file=./xenial.rootfs.ext4,id=hd0,format=raw \ -device virtio-blk-device,drive=hd0 \ In guest: $ perf record -e arm_spe/ts_enable=1,pa_enable=1,pct_enable=1/ \ dd if=/dev/zero of=/dev/null count=1000 $ perf report --dump-raw-trace > spe_buf.txt The spe_buf.txt should contain similar data as below: . ... ARM SPE data: size 135944 bytes . 00000000: b0 f4 d3 29 10 00 80 ff a0 PC 0xff80001029d3f4 el1 ns=1 . 00000009: 99 0b 00 LAT 11 ISSUE . 0000000c: 98 0d 00 LAT 13 TOT . 0000000f: 52 16 00 EV RETIRED L1D-ACCESS TLB-ACCESS . 00000012: 49 00 LD . 00000014: b2 d0 40 d8 70 00 00 ff 00 VA 0xff000070d840d0 . 0000001d: 9a 01 00 LAT 1 XLAT . 00000020: 00 00 00 PAD . 00000023: 71 a5 1f b3 20 14 00 00 00 TS 86447955877 . 0000002c: b0 7c f9 29 10 00 80 ff a0 PC 0xff80001029f97c el1 ns=1 . 00000035: 99 02 00 LAT 2 ISSUE . 00000038: 98 03 00 LAT 3 TOT . 0000003b: 52 02 00 EV RETIRED . 0000003e: 48 00 INSN-OTHER . 00000040: 00 00 00 PAD . 00000043: 71 ef 1f b3 20 14 00 00 00 TS 86447955951 . 0000004c: b0 f0 e9 29 10 00 80 ff a0 PC 0xff80001029e9f0 el1 ns=1 . 00000055: 99 02 00 LAT 2 ISSUE . 00000058: 98 03 00 LAT 3 TOT . 0000005b: 52 02 00 EV RETIRED If you want to disable the vSPE support, you can use the 'spe=off' cpu property: ./qemu-system-aarch64 \ -cpu host,spe=off -M virt,accel=kvm,gic-version=3 -nographic -m 2048M \ -kernel ./Image-new \ -initrd /boot/initrd.img-5.6.0-rc2+ \ -append "root=/dev/vda rw console=ttyAMA0" -nodefaults -serial stdio\ -drive if=none,file=./xenial.rootfs.ext4,id=hd0,format=raw \ -device virtio-blk-device,drive=hd0 \ Note: (1) Since the kernel patches are still under review, some of the macros in the header files may be changed after merging. We may need to update them accordingly. (2) These patches only add vSPE support in KVM mode, for TCG mode, I'm not sure whether we need to support it. (3) Just followed the 'pmu' property, we only allow this feature to be removed from CPUs which enable it by default. But since the SPE is an optional feature extension for Armv8.2, I think a better way may be to disable it by default, and only enable it when the host cpu do have the feature. [1]https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/ posts/statistical-profiling-extension-for-armv8-a [2]https://www.spinics.net/lists/arm-kernel/msg776228.html Haibo Xu (7): update Linux headers with new vSPE macros target/arm/kvm: spe: Add helper to detect SPE when using KVM target/arm/cpu: spe: Add an option to turn on/off vSPE support target/arm/kvm: spe: Unify device attr operatioin helper target/arm/kvm: spe: Add device init and set_irq operations hw/arm/virt: spe: Add SPE fdt binding for virt machine target/arm/cpu: spe: Enable spe to work with host cpu hw/arm/virt-acpi-build.c | 3 +++ hw/arm/virt.c | 42 ++++++++++++++++++++++++++++++ include/hw/acpi/acpi-defs.h | 1 + include/hw/arm/virt.h | 1 + linux-headers/asm-arm64/kvm.h | 4 +++ linux-headers/linux/kvm.h | 2 ++ target/arm/cpu.c | 34 +++++++++++++++++++++++++ target/arm/cpu.h | 5 ++++ target/arm/kvm.c | 11 ++++++++ target/arm/kvm64.c | 48 ++++++++++++++++++++++++++++++++--- target/arm/kvm_arm.h | 18 +++++++++++++ 11 files changed, 166 insertions(+), 3 deletions(-)