[RFC,00/21] migration: Support hugetlb doublemaps

Based-on: <20221213213850.1481858-1-peterx@redhat.com>
  [PATCH 0/5] migration: Fix disorder of channel creations

Trees for reference:
  https://github.com/xzpeter/linux/releases/tag/doublemap-v0.1
  https://github.com/xzpeter/qemu/releases/tag/doublemap-v0.1

This is an RFC series that only for early discussion purpose but not for
merging.

This patchset allows postcopy to work with huge pages better by migrating
huge pages in small page sizes.  It relies on a kernel feature called
"hugetlb HGM" which is currently proposed on the Linux kernel mailing list
by James Houghton, latest version v1:

https://lore.kernel.org/r/20230105101844.1893104-1-jthoughton@google.com

[PS: The kernel v1 patchset may need a few fixups to make QEMU work, which
 are all contained in the tree link provided tagged doublemap-v0.1 above]

The kernel series is still during review upstream, so the API is still not
stable.

I kept the old name of "doublemap" in this QEMU patchset to represent HGM.
With that, huge pages can be mapped with even smaller sizes than the huge
page itself.  It can drastically reduce page fault latencies during
postcopy if the guest has hugepage backed memories and make postcopy start
working with huge pages.  The average page request latency can drop from
~1sec to ~250us for 1G backed in the initial test results.

UFFDIO_COPY doesn't support mapping huge pages in small sizes, so one major
part of this series introduced UFFDIO_CONTINUE to resolve page faults for
hugetlb mappings.

Sampled page latency histogram for 18G guest with/without doublemap
(preempt=on, single thread busy spin workload over 18G map):

Before:

@delay_us:
[64, 128)              3 |@                                                   |
[128, 256)            84 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[256, 512)            10 |@@@@@@                                              |
[512, 1K)              1 |                                                    |
[1K, 2K)               0 |                                                    |
[2K, 4K)               0 |                                                    |
[4K, 8K)               0 |                                                    |
[8K, 16K)              0 |                                                    |
[16K, 32K)             0 |                                                    |
[32K, 64K)             0 |                                                    |
[64K, 128K)            0 |                                                    |
[128K, 256K)           0 |                                                    |
[256K, 512K)           0 |                                                    |
[512K, 1M)             0 |                                                    |
[1M, 2M)              17 |@@@@@@@@@@                                          |
[2M, 4M)              21 |@@@@@@@@@@@@@                                       |
[4M, 8M)               8 |@@@@                                                |
[8M, 16M)              4 |@@                                                  |

After:

@delay_us:
[16, 32)               6 |                                                    |
[32, 64)               6 |                                                    |
[64, 128)           3117 |@@                                                  |
[128, 256)         70815 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[256, 512)         30460 |@@@@@@@@@@@@@@@@@@@@@@                              |
[512, 1K)           1135 |                                                    |
[1K, 2K)              34 |                                                    |
[2K, 4K)              42 |                                                    |
[4K, 8K)             126 |                                                    |
[8K, 16K)             91 |                                                    |
[16K, 32K)             0 |                                                    |
[32K, 64K)             1 |                                                    |

Any early comment welcomed.  Thanks.

Peter Xu (21):
  update linux headers
  util: Include osdep.h first in util/mmap-alloc.c
  physmem: Add qemu_ram_is_hugetlb()
  madvise: Include linux/mman.h under linux-headers/
  madvise: Add QEMU_MADV_SPLIT
  madvise: Add QEMU_MADV_COLLAPSE
  ramblock: Cache file offset for file-backed ramblocks
  ramblock: Cache the length to do file mmap() on ramblocks
  ramblock: Add RAM_READONLY
  ramblock: Add ramblock_file_map()
  migration: Add hugetlb-doublemap cap
  migration: Introduce page size for-migration-only
  migration: Add migration_ram_pagesize_largest()
  migration: Map hugetlbfs ramblocks twice, and pre-allocate
  migration: Teach qemu about minor faults and doublemap
  migration: Enable doublemap with MADV_SPLIT
  migration: Rework ram discard logic for hugetlb double-map
  migration: Allow postcopy_register_shared_ufd() to fail
  migration: Add postcopy_mark_received()
  migration: Handle page faults using UFFDIO_CONTINUE
  migration: Collapse huge pages again after postcopy finished

 backends/hostmem-file.c                       |   3 +-
 hw/virtio/vhost-user.c                        |   9 +-
 include/exec/cpu-common.h                     |   3 +-
 include/exec/memory.h                         |   4 +-
 include/exec/ram_addr.h                       |   6 +-
 include/exec/ramblock.h                       |  14 +
 include/qemu/madvise.h                        |  18 ++
 include/standard-headers/drm/drm_fourcc.h     |  63 +++-
 include/standard-headers/linux/ethtool.h      |  81 ++++-
 include/standard-headers/linux/fuse.h         |  20 +-
 .../linux/input-event-codes.h                 |   4 +
 include/standard-headers/linux/pci_regs.h     |   2 +
 include/standard-headers/linux/virtio_blk.h   |  19 ++
 include/standard-headers/linux/virtio_bt.h    |   8 +
 include/standard-headers/linux/virtio_net.h   |   4 +
 linux-headers/asm-arm64/kvm.h                 |   1 +
 linux-headers/asm-generic/hugetlb_encode.h    |  26 +-
 linux-headers/asm-generic/mman-common.h       |   4 +
 linux-headers/asm-mips/mman.h                 |   4 +
 linux-headers/asm-riscv/kvm.h                 |   7 +
 linux-headers/asm-x86/kvm.h                   |  11 +-
 linux-headers/linux/kvm.h                     |  32 +-
 linux-headers/linux/psci.h                    |  14 +
 linux-headers/linux/userfaultfd.h             |   4 +
 linux-headers/linux/vfio.h                    | 278 +++++++++++++++++-
 migration/migration.c                         |  56 +++-
 migration/migration.h                         |   1 +
 migration/postcopy-ram.c                      | 228 +++++++++++---
 migration/postcopy-ram.h                      |   5 +-
 migration/ram.c                               | 165 ++++++++++-
 migration/ram.h                               |   2 +
 migration/trace-events                        |   6 +-
 qapi/migration.json                           |   7 +-
 softmmu/memory.c                              |   8 +-
 softmmu/physmem.c                             |  92 ++++--
 util/mmap-alloc.c                             |   2 +-
 36 files changed, 1051 insertions(+), 160 deletions(-)