| Message ID | 20190724143105.307042-1-sgarzare@redhat.com |
|---|---|
| Headers | show |
| Series | pc: mmap kernel (ELF image) and initrd | expand |
On 24/07/19 16:31, Stefano Garzarella wrote: > In order to reduce the memory footprint when PVH kernel and initrd > are used, we map them into memory instead of reading them. > In this way we can share them between multiple instances of QEMU. > > v4: > - Patch 1: fix the rom_add_elf_program() comment [Paolo] > - Patch 2: > ~ fix the missing of g_mapped_file_unref() in the success case [Paolo] > ~ fix the rom_add_elf_program() comment [Paolo] > > v3: https://patchew.org/QEMU/20190724112531.232260-1-sgarzare@redhat.com/ > v2: https://patchew.org/QEMU/20190723140445.12748-1-sgarzare@redhat.com/ > > These are the results using a PVH kernel and initrd (cpio): > - memory footprint (using smem) [MB] > QEMU before now > # instances USS PSS USS PSS > 1 102.0M 105.8M 102.3M 106.2M > 2 94.6M 101.2M 72.3M 90.1M > 4 94.1M 98.0M 72.0M 81.5M > 8 94.0M 96.2M 71.8M 76.9M > 16 93.9M 95.1M 71.6M 74.3M > > Initrd size: 3.0M > Kernel > image size: 28M > sections size [size -A -d vmlinux]: 18.9M > > - boot time [ms] > before now > qemu_init_end: 63.85 55.91 > linux_start_kernel: 82.11 (+18.26) 74.51 (+18.60) > linux_start_user: 169.94 (+87.83) 159.06 (+84.56) > > QEMU command used: > ./qemu-system-x86_64 -bios /path/to/seabios/out/bios.bin -no-hpet \ > -machine q35,accel=kvm,kernel_irqchip,nvdimm,sata=off,smbus=off,vmport=off \ > -cpu host -m 1G -smp 1 -vga none -display none -no-user-config -nodefaults \ > -kernel /path/to/vmlinux -initrd /path/to/rootfs.cpio \ > -append 'root=/dev/mem0 ro console=hvc0 pci=lastbus=0 nosmap' > > Stefano Garzarella (3): > loader: Handle memory-mapped ELFs > elf-ops.h: Map into memory the ELF to load > hw/i386/pc: Map into memory the initrd > > hw/core/loader.c | 38 +++++++++++++++++++----- > hw/i386/pc.c | 17 ++++++++--- > include/hw/elf_ops.h | 71 ++++++++++++++++++++++++++------------------ > include/hw/i386/pc.h | 1 + > include/hw/loader.h | 5 ++-- > 5 files changed, 89 insertions(+), 43 deletions(-) > Queued, thanks. Paolo
* Stefano Garzarella (sgarzare@redhat.com) wrote: > In order to reduce the memory footprint when PVH kernel and initrd > are used, we map them into memory instead of reading them. > In this way we can share them between multiple instances of QEMU. > > v4: > - Patch 1: fix the rom_add_elf_program() comment [Paolo] > - Patch 2: > ~ fix the missing of g_mapped_file_unref() in the success case [Paolo] > ~ fix the rom_add_elf_program() comment [Paolo] > > v3: https://patchew.org/QEMU/20190724112531.232260-1-sgarzare@redhat.com/ > v2: https://patchew.org/QEMU/20190723140445.12748-1-sgarzare@redhat.com/ Two high level questions: a) What happens if someone tries to migrate the VM - I don't think it's too unusual for people to run with -kernel/-initrd in situations where they migrate. b) Are there situations where you can't mmap but you can validly read it? For example, running with an ELF built for 4k page alignment on a host with 64k host pages? Dave > > These are the results using a PVH kernel and initrd (cpio): > - memory footprint (using smem) [MB] > QEMU before now > # instances USS PSS USS PSS > 1 102.0M 105.8M 102.3M 106.2M > 2 94.6M 101.2M 72.3M 90.1M > 4 94.1M 98.0M 72.0M 81.5M > 8 94.0M 96.2M 71.8M 76.9M > 16 93.9M 95.1M 71.6M 74.3M > > Initrd size: 3.0M > Kernel > image size: 28M > sections size [size -A -d vmlinux]: 18.9M > > - boot time [ms] > before now > qemu_init_end: 63.85 55.91 > linux_start_kernel: 82.11 (+18.26) 74.51 (+18.60) > linux_start_user: 169.94 (+87.83) 159.06 (+84.56) > > QEMU command used: > ./qemu-system-x86_64 -bios /path/to/seabios/out/bios.bin -no-hpet \ > -machine q35,accel=kvm,kernel_irqchip,nvdimm,sata=off,smbus=off,vmport=off \ > -cpu host -m 1G -smp 1 -vga none -display none -no-user-config -nodefaults \ > -kernel /path/to/vmlinux -initrd /path/to/rootfs.cpio \ > -append 'root=/dev/mem0 ro console=hvc0 pci=lastbus=0 nosmap' > > Stefano Garzarella (3): > loader: Handle memory-mapped ELFs > elf-ops.h: Map into memory the ELF to load > hw/i386/pc: Map into memory the initrd > > hw/core/loader.c | 38 +++++++++++++++++++----- > hw/i386/pc.c | 17 ++++++++--- > include/hw/elf_ops.h | 71 ++++++++++++++++++++++++++------------------ > include/hw/i386/pc.h | 1 + > include/hw/loader.h | 5 ++-- > 5 files changed, 89 insertions(+), 43 deletions(-) > > -- > 2.20.1 > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
Hi Stefano Here the results https://pasteboard.co/Ipu3DO4.png https://pasteboard.co/Ipu3L69.png boot time with initrd is a little bit better Thanks - Julio On Wed, 2019-07-24 at 16:31 +0200, Stefano Garzarella wrote: > In order to reduce the memory footprint when PVH kernel and initrd > are used, we map them into memory instead of reading them. > In this way we can share them between multiple instances of QEMU. > > v4: > - Patch 1: fix the rom_add_elf_program() comment [Paolo] > - Patch 2: > ~ fix the missing of g_mapped_file_unref() in the success case > [Paolo] > ~ fix the rom_add_elf_program() comment [Paolo] > > v3: > https://patchew.org/QEMU/20190724112531.232260-1-sgarzare@redhat.com/ > v2: > https://patchew.org/QEMU/20190723140445.12748-1-sgarzare@redhat.com/ > > These are the results using a PVH kernel and initrd (cpio): > - memory footprint (using smem) [MB] > QEMU before now > # instances USS PSS USS PSS > 1 102.0M 105.8M 102.3M 106.2M > 2 94.6M 101.2M 72.3M 90.1M > 4 94.1M 98.0M 72.0M 81.5M > 8 94.0M 96.2M 71.8M 76.9M > 16 93.9M 95.1M 71.6M 74.3M > > Initrd size: 3.0M > Kernel > image size: 28M > sections size [size -A -d vmlinux]: 18.9M > > - boot time [ms] > before now > qemu_init_end: 63.85 55.91 > linux_start_kernel: 82.11 (+18.26) 74.51 (+18.60) > linux_start_user: 169.94 (+87.83) 159.06 (+84.56) > > QEMU command used: > ./qemu-system-x86_64 -bios /path/to/seabios/out/bios.bin -no-hpet \ > -machine > q35,accel=kvm,kernel_irqchip,nvdimm,sata=off,smbus=off,vmport=off \ > -cpu host -m 1G -smp 1 -vga none -display none -no-user-config > -nodefaults \ > -kernel /path/to/vmlinux -initrd /path/to/rootfs.cpio \ > -append 'root=/dev/mem0 ro console=hvc0 pci=lastbus=0 nosmap' > > Stefano Garzarella (3): > loader: Handle memory-mapped ELFs > elf-ops.h: Map into memory the ELF to load > hw/i386/pc: Map into memory the initrd > > hw/core/loader.c | 38 +++++++++++++++++++----- > hw/i386/pc.c | 17 ++++++++--- > include/hw/elf_ops.h | 71 ++++++++++++++++++++++++++-------------- > ---- > include/hw/i386/pc.h | 1 + > include/hw/loader.h | 5 ++-- > 5 files changed, 89 insertions(+), 43 deletions(-) >
On Wed, Jul 24, 2019 at 04:35:23PM +0000, Montes, Julio wrote: > Hi Stefano > > Here the results > > https://pasteboard.co/Ipu3DO4.png > https://pasteboard.co/Ipu3L69.png > > boot time with initrd is a little bit better > Thanks for sharing! It is interesting that 'inkernel' time with initrd is almost twice the time of nvdimm. Stefano
In order to reduce the memory footprint when PVH kernel and initrd are used, we map them into memory instead of reading them. In this way we can share them between multiple instances of QEMU. v4: - Patch 1: fix the rom_add_elf_program() comment [Paolo] - Patch 2: ~ fix the missing of g_mapped_file_unref() in the success case [Paolo] ~ fix the rom_add_elf_program() comment [Paolo] v3: https://patchew.org/QEMU/20190724112531.232260-1-sgarzare@redhat.com/ v2: https://patchew.org/QEMU/20190723140445.12748-1-sgarzare@redhat.com/ These are the results using a PVH kernel and initrd (cpio): - memory footprint (using smem) [MB] QEMU before now # instances USS PSS USS PSS 1 102.0M 105.8M 102.3M 106.2M 2 94.6M 101.2M 72.3M 90.1M 4 94.1M 98.0M 72.0M 81.5M 8 94.0M 96.2M 71.8M 76.9M 16 93.9M 95.1M 71.6M 74.3M Initrd size: 3.0M Kernel image size: 28M sections size [size -A -d vmlinux]: 18.9M - boot time [ms] before now qemu_init_end: 63.85 55.91 linux_start_kernel: 82.11 (+18.26) 74.51 (+18.60) linux_start_user: 169.94 (+87.83) 159.06 (+84.56) QEMU command used: ./qemu-system-x86_64 -bios /path/to/seabios/out/bios.bin -no-hpet \ -machine q35,accel=kvm,kernel_irqchip,nvdimm,sata=off,smbus=off,vmport=off \ -cpu host -m 1G -smp 1 -vga none -display none -no-user-config -nodefaults \ -kernel /path/to/vmlinux -initrd /path/to/rootfs.cpio \ -append 'root=/dev/mem0 ro console=hvc0 pci=lastbus=0 nosmap' Stefano Garzarella (3): loader: Handle memory-mapped ELFs elf-ops.h: Map into memory the ELF to load hw/i386/pc: Map into memory the initrd hw/core/loader.c | 38 +++++++++++++++++++----- hw/i386/pc.c | 17 ++++++++--- include/hw/elf_ops.h | 71 ++++++++++++++++++++++++++------------------ include/hw/i386/pc.h | 1 + include/hw/loader.h | 5 ++-- 5 files changed, 89 insertions(+), 43 deletions(-)