| Series |
mm: Generalize HVO for HugeTLB and device DAX
|
expand
-
[v2,00/69] mm: Generalize HVO for HugeTLB and device DAX
-
[v2,01/69] mm/hugetlb: Fix boot panic with CONFIG_DEBUG_VM and HVO bootmem pages
-
[v2,02/69] mm/hugetlb_vmemmap: Fix __hugetlb_vmemmap_optimize_folios()
-
[v2,03/69] powerpc/mm: Fix wrong addr_pfn tracking in compound vmemmap population
-
[v2,04/69] mm/hugetlb: Initialize gigantic bootmem hugepage struct pages earlier
-
[v2,05/69] mm/mm_init: Simplify deferred_free_pages() migratetype init
-
[v2,06/69] mm/sparse: Panic on memmap and usemap allocation failure
-
[v2,07/69] mm/sparse: Move subsection_map_init() into sparse_init()
-
[v2,08/69] mm/mm_init: Defer sparse_init() until after zone initialization
-
[v2,09/69] mm/mm_init: Defer hugetlb reservation until after zone initialization
-
[v2,10/69] mm/mm_init: Remove set_pageblock_order() call from sparse_init()
-
[v2,11/69] mm/sparse: Move sparse_vmemmap_init_nid_late() into sparse_init_nid()
-
[v2,12/69] mm/hugetlb_cma: Validate hugetlb CMA range by zone at reserve time
-
[v2,13/69] mm/hugetlb: Refactor early boot gigantic hugepage allocation
-
[v2,14/69] mm/hugetlb: Free cross-zone bootmem gigantic pages after allocation
-
[v2,15/69] mm/hugetlb_vmemmap: Move bootmem HVO setup to early init
-
[v2,16/69] mm/hugetlb: Remove obsolete bootmem cross-zone checks
-
[v2,17/69] mm/sparse-vmemmap: Remove sparse_vmemmap_init_nid_late()
-
[v2,18/69] mm/hugetlb: Remove unused bootmem cma field
-
[v2,19/69] mm/mm_init: Make __init_page_from_nid() static
-
[v2,20/69] mm/sparse-vmemmap: Drop VMEMMAP_POPULATE_PAGEREF
-
[v2,21/69] mm: Rename vmemmap optimization macros around folio semantics
-
[v2,22/69] mm/sparse: Drop power-of-2 size requirement for struct mem_section
-
[v2,23/69] mm/sparse-vmemmap: track compound page order in struct mem_section
-
[v2,24/69] mm/mm_init: Skip initializing shared vmemmap tail pages
-
[v2,25/69] mm/sparse-vmemmap: Initialize shared tail vmemmap pages on allocation
-
[v2,26/69] mm/sparse-vmemmap: Support section-based vmemmap accounting
-
[v2,27/69] mm/sparse-vmemmap: Support section-based vmemmap optimization
-
[v2,28/69] mm/hugetlb: Use generic vmemmap optimization macros
-
[v2,29/69] mm/sparse: Mark memblocks present earlier
-
[v2,30/69] mm/hugetlb: Switch HugeTLB to section-based vmemmap optimization
-
[v2,31/69] mm/sparse: Remove section_map_size()
-
[v2,32/69] mm/mm_init: Factor out pfn_to_zone() as a shared helper
-
[v2,33/69] mm/sparse: Remove SPARSEMEM_VMEMMAP_PREINIT
-
[v2,34/69] mm/sparse: Inline usemap allocation into sparse_init_nid()
-
[v2,35/69] mm/hugetlb: Remove HUGE_BOOTMEM_HVO
-
[v2,36/69] mm/hugetlb: Remove HUGE_BOOTMEM_CMA
-
[v2,37/69] mm/sparse-vmemmap: Factor out shared vmemmap page allocation
-
[v2,38/69] mm/sparse-vmemmap: Introduce CONFIG_SPARSEMEM_VMEMMAP_OPTIMIZATION
-
[v2,39/69] mm/sparse-vmemmap: Switch DAX to vmemmap_shared_tail_page()
-
[v2,40/69] powerpc/mm: Switch DAX to vmemmap_shared_tail_page()
-
[v2,41/69] mm/sparse-vmemmap: Drop the extra tail page from DAX reservation
-
[v2,42/69] mm/sparse-vmemmap: Switch DAX to section-based vmemmap optimization
-
[v2,43/69] mm/sparse-vmemmap: Unify DAX and HugeTLB population paths
-
[v2,44/69] mm/sparse-vmemmap: Remove the unused ptpfn argument
-
[v2,45/69] powerpc/mm: Make vmemmap_populate_compound_pages() static
-
[v2,46/69] mm/sparse-vmemmap: Map shared vmemmap tail pages read-only
-
[v2,47/69] powerpc/mm: Map shared vmemmap tail pages read-only
-
[v2,48/69] mm/sparse-vmemmap: Inline vmemmap_populate_address() into its caller
-
[v2,49/69] mm/hugetlb_vmemmap: Remove vmemmap_wrprotect_hvo()
-
[v2,50/69] mm/sparse: Simplify section_nr_vmemmap_pages()
-
[v2,51/69] mm/sparse-vmemmap: Introduce vmemmap_nr_struct_pages()
-
[v2,52/69] powerpc/mm: Drop powerpc vmemmap_can_optimize()
-
[v2,53/69] mm/sparse-vmemmap: Drop vmemmap_can_optimize()
-
[v2,54/69] mm/sparse-vmemmap: Drop @pgmap from vmemmap population APIs
-
[v2,55/69] mm/sparse: Decouple section activation from ZONE_DEVICE
-
[v2,56/69] mm: Redefine HVO as Hugepage Vmemmap Optimization
-
[v2,57/69] mm/sparse-vmemmap: Consolidate HVO enable checks
-
[v2,58/69] mm/hugetlb: Make HVO optimizable checks depend on generic logic
-
[v2,59/69] mm/sparse-vmemmap: Localize init_compound_tail()
-
[v2,60/69] mm/mm_init: Check zone consistency on optimized vmemmap sections
-
[v2,61/69] mm/hugetlb: Drop boot-time HVO handling for gigantic folios
-
[v2,62/69] mm/hugetlb: Simplify hugetlb_folio_init_vmemmap()
-
[v2,63/69] mm/hugetlb: Initialize the full bootmem hugepage in hugetlb code
-
[v2,64/69] mm/mm_init: Factor out compound page initialization
-
[v2,65/69] mm/mm_init: Make __init_single_page() static
-
[v2,66/69] mm/cma: Move CMA pageblock initialization into cma_activate_area()
-
[v2,67/69] mm/cma: Move init_cma_pageblock() into cma.c
-
[v2,68/69] mm/mm_init: Initialize pageblock migratetype in memmap init helpers
-
[v2,69/69] Documentation/mm: Rewrite vmemmap_dedup.rst for unified HVO
|
In this series, HVO is redefined as Hugepage Vmemmap Optimization: a general vmemmap optimization model for large hugepage-backed mappings, rather than a HugeTLB-only implementation detail. The existing code grew around the original HugeTLB-specific HVO path, while device DAX developed similar but separate vmemmap optimization handling. As a result, the current implementation carries duplicated logic, boot-time special cases, and subsystem-specific interfaces around what is fundamentally the same sparse-vmemmap optimization. This series generalizes that optimization into a common framework used by both HugeTLB and device DAX. The first few patches include some minor bug fixes found during AI-aided review of the current code. These fixes are not the main goal of the series, but the later refactoring and unification work depends on them, so they are included here as preparatory changes. The series then reworks the relevant early boot and sparse initialization paths, introduces a generic section-based sparse-vmemmap optimization infrastructure, switches HugeTLB and device DAX over to the shared implementation, and removes the old special-case code. At a high level, the series does the following: - apply a small set of preparatory bug fixes - reorder early boot and sparse initialization so optimized vmemmap setup has the required zone and pageblock state - introduce generic section-based vmemmap optimization infrastructure - switch HugeTLB and device DAX to the shared implementation - consolidate HVO enablement and naming - remove obsolete HugeTLB-specific boot-time and architecture-specific optimization code - rewrite the documentation around the unified design This brings a few concrete benefits: - HugeTLB and device DAX share one vmemmap optimization framework, reducing duplicated logic and long-term maintenance overhead - when CONFIG_DEFERRED_STRUCT_PAGE_INIT is disabled, optimized struct pages can skip the usual memmap_init() initialization work, which helps reduce boot-time overhead - all architectures that support HVO benefit from the generic sparse-vmemmap optimization path without extra architecture-specific preinit handling - device DAX improves its struct page savings further by dropping the extra reserved tail page - shared vmemmap tail pages are mapped read-only, improving robustness I have only built and tested this series on x86. I do not currently have a powerpc test environment, so any testing or feedback on powerpc would be much appreciated. Changes since v1: - rebased onto current next tree - added the preparatory minor bug fixes found during AI-aided review - added further refactoring on top of the new infrastructure Muchun Song (69): mm/hugetlb: Fix boot panic with CONFIG_DEBUG_VM and HVO bootmem pages mm/hugetlb_vmemmap: Fix __hugetlb_vmemmap_optimize_folios() powerpc/mm: Fix wrong addr_pfn tracking in compound vmemmap population mm/hugetlb: Initialize gigantic bootmem hugepage struct pages earlier mm/mm_init: Simplify deferred_free_pages() migratetype init mm/sparse: Panic on memmap and usemap allocation failure mm/sparse: Move subsection_map_init() into sparse_init() mm/mm_init: Defer sparse_init() until after zone initialization mm/mm_init: Defer hugetlb reservation until after zone initialization mm/mm_init: Remove set_pageblock_order() call from sparse_init() mm/sparse: Move sparse_vmemmap_init_nid_late() into sparse_init_nid() mm/hugetlb_cma: Validate hugetlb CMA range by zone at reserve time mm/hugetlb: Refactor early boot gigantic hugepage allocation mm/hugetlb: Free cross-zone bootmem gigantic pages after allocation mm/hugetlb_vmemmap: Move bootmem HVO setup to early init mm/hugetlb: Remove obsolete bootmem cross-zone checks mm/sparse-vmemmap: Remove sparse_vmemmap_init_nid_late() mm/hugetlb: Remove unused bootmem cma field mm/mm_init: Make __init_page_from_nid() static mm/sparse-vmemmap: Drop VMEMMAP_POPULATE_PAGEREF mm: Rename vmemmap optimization macros around folio semantics mm/sparse: Drop power-of-2 size requirement for struct mem_section mm/sparse-vmemmap: track compound page order in struct mem_section mm/mm_init: Skip initializing shared vmemmap tail pages mm/sparse-vmemmap: Initialize shared tail vmemmap pages on allocation mm/sparse-vmemmap: Support section-based vmemmap accounting mm/sparse-vmemmap: Support section-based vmemmap optimization mm/hugetlb: Use generic vmemmap optimization macros mm/sparse: Mark memblocks present earlier mm/hugetlb: Switch HugeTLB to section-based vmemmap optimization mm/sparse: Remove section_map_size() mm/mm_init: Factor out pfn_to_zone() as a shared helper mm/sparse: Remove SPARSEMEM_VMEMMAP_PREINIT mm/sparse: Inline usemap allocation into sparse_init_nid() mm/hugetlb: Remove HUGE_BOOTMEM_HVO mm/hugetlb: Remove HUGE_BOOTMEM_CMA mm/sparse-vmemmap: Factor out shared vmemmap page allocation mm/sparse-vmemmap: Introduce CONFIG_SPARSEMEM_VMEMMAP_OPTIMIZATION mm/sparse-vmemmap: Switch DAX to vmemmap_shared_tail_page() powerpc/mm: Switch DAX to vmemmap_shared_tail_page() mm/sparse-vmemmap: Drop the extra tail page from DAX reservation mm/sparse-vmemmap: Switch DAX to section-based vmemmap optimization mm/sparse-vmemmap: Unify DAX and HugeTLB population paths mm/sparse-vmemmap: Remove the unused ptpfn argument powerpc/mm: Make vmemmap_populate_compound_pages() static mm/sparse-vmemmap: Map shared vmemmap tail pages read-only powerpc/mm: Map shared vmemmap tail pages read-only mm/sparse-vmemmap: Inline vmemmap_populate_address() into its caller mm/hugetlb_vmemmap: Remove vmemmap_wrprotect_hvo() mm/sparse: Simplify section_nr_vmemmap_pages() mm/sparse-vmemmap: Introduce vmemmap_nr_struct_pages() powerpc/mm: Drop powerpc vmemmap_can_optimize() mm/sparse-vmemmap: Drop vmemmap_can_optimize() mm/sparse-vmemmap: Drop @pgmap from vmemmap population APIs mm/sparse: Decouple section activation from ZONE_DEVICE mm: Redefine HVO as Hugepage Vmemmap Optimization mm/sparse-vmemmap: Consolidate HVO enable checks mm/hugetlb: Make HVO optimizable checks depend on generic logic mm/sparse-vmemmap: Localize init_compound_tail() mm/mm_init: Check zone consistency on optimized vmemmap sections mm/hugetlb: Drop boot-time HVO handling for gigantic folios mm/hugetlb: Simplify hugetlb_folio_init_vmemmap() mm/hugetlb: Initialize the full bootmem hugepage in hugetlb code mm/mm_init: Factor out compound page initialization mm/mm_init: Make __init_single_page() static mm/cma: Move CMA pageblock initialization into cma_activate_area() mm/cma: Move init_cma_pageblock() into cma.c mm/mm_init: Initialize pageblock migratetype in memmap init helpers Documentation/mm: Rewrite vmemmap_dedup.rst for unified HVO .../admin-guide/kernel-parameters.txt | 2 +- Documentation/admin-guide/mm/hugetlbpage.rst | 4 +- .../admin-guide/mm/memory-hotplug.rst | 2 +- Documentation/admin-guide/sysctl/vm.rst | 3 +- Documentation/arch/powerpc/index.rst | 1 - Documentation/arch/powerpc/vmemmap_dedup.rst | 101 ---- Documentation/mm/vmemmap_dedup.rst | 217 ++------ arch/arm64/mm/mmu.c | 5 +- arch/loongarch/mm/init.c | 5 +- arch/powerpc/include/asm/book3s/64/radix.h | 12 - arch/powerpc/mm/book3s64/radix_pgtable.c | 154 +----- arch/powerpc/mm/hugetlbpage.c | 11 +- arch/powerpc/mm/init_64.c | 1 + arch/powerpc/mm/mem.c | 5 +- arch/riscv/mm/init.c | 5 +- arch/s390/mm/init.c | 5 +- arch/x86/Kconfig | 1 - arch/x86/entry/vdso/vdso32/fake_32bit_build.h | 1 - arch/x86/mm/init_64.c | 5 +- drivers/dax/Kconfig | 1 + fs/Kconfig | 6 +- include/linux/hugetlb.h | 23 +- include/linux/memory_hotplug.h | 12 +- include/linux/mm.h | 44 +- include/linux/mm_types.h | 3 +- include/linux/mmzone.h | 151 ++++-- include/linux/page-flags-layout.h | 2 + include/linux/page-flags.h | 31 +- kernel/bounds.c | 5 + mm/Kconfig | 9 +- mm/bootmem_info.c | 5 +- mm/cma.c | 18 +- mm/hugetlb.c | 337 ++++-------- mm/hugetlb_cma.c | 41 +- mm/hugetlb_cma.h | 4 +- mm/hugetlb_vmemmap.c | 266 +-------- mm/hugetlb_vmemmap.h | 64 +-- mm/internal.h | 72 ++- mm/memory-failure.c | 6 +- mm/memory_hotplug.c | 22 +- mm/memremap.c | 4 +- mm/mm_init.c | 241 ++++----- mm/sparse-vmemmap.c | 511 ++++++------------ mm/sparse.c | 129 +---- mm/util.c | 2 +- scripts/gdb/linux/mm.py | 6 +- 46 files changed, 743 insertions(+), 1812 deletions(-) delete mode 100644 Documentation/arch/powerpc/vmemmap_dedup.rst base-commit: e98d21c170b01ddef366f023bbfcf6b31509fa83