| Message ID | 20221118002146.25979-5-rmclure@linux.ibm.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [v5,1/5] powerpc: mm: Replace p{u,m,4}d_is_leaf with p{u,m,4}_leaf | expand |
Rohan McLure <rmclure@linux.ibm.com> writes: > On creation and clearing of a page table mapping, instrument such calls > by invoking page_table_check_pte_set and page_table_check_pte_clear > respectively. These calls serve as a sanity check against illegal > mappings. > > Enable ARCH_SUPPORTS_PAGE_TABLE_CHECK for all ppc64, and 32-bit > platforms implementing Book3S. > > Change pud_pfn to be a runtime bug rather than a build bug as it is > consumed by page_table_check_pud_{clear,set} which are not called. > > See also: > > riscv support in commit 3fee229a8eb9 ("riscv/mm: enable > ARCH_SUPPORTS_PAGE_TABLE_CHECK") > arm64 in commit 42b2547137f5 ("arm64/mm: enable > ARCH_SUPPORTS_PAGE_TABLE_CHECK") > x86_64 in commit d283d422c6c4 ("x86: mm: add x86_64 support for page table > check") > > Reviewed-by: Russell Currey <ruscur@russell.cc> > Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu> > Signed-off-by: Rohan McLure <rmclure@linux.ibm.com> This blows up for me when checking is enabled. This is a qemu pseries KVM guest on a P9 host, booting Fedora 34. I haven't dug into what is wrong yet. cheers [ 0.600480][ T63] ------------[ cut here ]------------ [ 0.600546][ T63] kernel BUG at mm/page_table_check.c:115! [ 0.600596][ T63] Oops: Exception in kernel mode, sig: 5 [#1] [ 0.600645][ T63] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries [ 0.600703][ T63] Modules linked in: [ 0.600736][ T63] CPU: 0 PID: 63 Comm: systemd-bless-b Not tainted 6.1.0-rc2-00178-gf0c0e10f5162 #65 [ 0.600803][ T63] Hardware name: IBM pSeries (emulated by qemu) POWER9 (raw) 0x4e1202 0xf000005 of:SLOF,git-5b4c5a hv:linux,kvm pSeries [ 0.600885][ T63] NIP: c0000000004a7d58 LR: c0000000004a7d74 CTR: 0000000000000000 [ 0.600942][ T63] REGS: c0000000067635e0 TRAP: 0700 Not tainted (6.1.0-rc2-00178-gf0c0e10f5162) [ 0.601008][ T63] MSR: 8000000000029033 <SF,EE,ME,IR,DR,RI,LE> CR: 44424420 XER: 00000003 [ 0.601088][ T63] CFAR: c0000000004a7d4c IRQMASK: 0 [ 0.601088][ T63] GPR00: c0000000004a7d74 c000000006763880 c000000001332500 c000000003a00408 [ 0.601088][ T63] GPR04: 0000000000000001 0000000000000001 8603402f000000c0 0000000000000000 [ 0.601088][ T63] GPR08: 00000000000005e0 0000000000000001 c000000002702500 0000000000002000 [ 0.601088][ T63] GPR12: 00007fffbc810000 c000000002a20000 0000000040000000 0000000040000000 [ 0.601088][ T63] GPR16: 0000000040000000 0000000000000000 0000000000000000 0000000040000000 [ 0.601088][ T63] GPR20: ff7fffffffffefbf 0000000000000000 c000000003555a00 0000000000000001 [ 0.601088][ T63] GPR24: c0000000028dda60 c00c000000020ac0 c0000000082a1100 c00c0000000bd000 [ 0.601088][ T63] GPR28: 0000000000000001 c0000000026fcf60 0000000000000001 c000000003a00400 [ 0.601614][ T63] NIP [c0000000004a7d58] page_table_check_set.part.0+0xc8/0x170 [ 0.601675][ T63] LR [c0000000004a7d74] page_table_check_set.part.0+0xe4/0x170 [ 0.601734][ T63] Call Trace: [ 0.601764][ T63] [c000000006763880] [c0000000067638c0] 0xc0000000067638c0 (unreliable) [ 0.601825][ T63] [c0000000067638c0] [c000000000087c28] set_pte_at+0x68/0x210 [ 0.601884][ T63] [c000000006763910] [c000000000483fd8] __split_huge_pmd+0x7f8/0x11c0 [ 0.601947][ T63] [c000000006763a20] [c000000000485908] vma_adjust_trans_huge+0x158/0x2d0 [ 0.602006][ T63] [c000000006763a70] [c00000000040b5dc] __vma_adjust+0x13c/0xbe0 [ 0.602067][ T63] [c000000006763b80] [c00000000040d708] __split_vma+0x158/0x270 [ 0.602128][ T63] [c000000006763bd0] [c00000000040d938] do_mas_align_munmap.constprop.0+0x118/0x610 [ 0.602196][ T63] [c000000006763cd0] [c000000000416228] sys_mremap+0x3c8/0x850 [ 0.602255][ T63] [c000000006763e10] [c00000000002fab8] system_call_exception+0x128/0x330 [ 0.602314][ T63] [c000000006763e50] [c00000000000d05c] system_call_vectored_common+0x15c/0x2ec [ 0.602384][ T63] --- interrupt: 3000 at 0x7fffbd94f86c [ 0.602425][ T63] NIP: 00007fffbd94f86c LR: 0000000000000000 CTR: 0000000000000000 [ 0.602481][ T63] REGS: c000000006763e80 TRAP: 3000 Not tainted (6.1.0-rc2-00178-gf0c0e10f5162) [ 0.602546][ T63] MSR: 800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE> CR: 44004422 XER: 00000000 [ 0.602635][ T63] IRQMASK: 0 [ 0.602635][ T63] GPR00: 00000000000000a3 00007ffff2c4e670 00007fffbda37000 00007fffbc400000 [ 0.602635][ T63] GPR04: 0000000000410000 0000000000010000 0000000000000001 0000000000000000 [ 0.602635][ T63] GPR08: 00007fffbc410000 0000000000000000 0000000000000000 0000000000000000 [ 0.602635][ T63] GPR12: 0000000000000000 00007fffbc937d50 0000000000000000 0000000000000000 [ 0.602635][ T63] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 0.602635][ T63] GPR20: 000000000000046b 00000000003fffff 00007ffff2c4e7e8 0000000000400000 [ 0.602635][ T63] GPR24: 0000000000000000 0000000000000000 0000000000000480 0000000000410000 [ 0.602635][ T63] GPR28: 00007fffbc400000 0000000000000000 0000000000010000 0000000000100000 [ 0.603146][ T63] NIP [00007fffbd94f86c] 0x7fffbd94f86c [ 0.603186][ T63] LR [0000000000000000] 0x0 [ 0.603226][ T63] --- interrupt: 3000 [ 0.603256][ T63] Code: 0b090000 7c0004ac 7d201828 31290001 7d20192d 40c2fff4 7c0004ac 2c090001 7f89e378 41810008 39200000 5529063e <0b090000> e93d0000 37deffff 7fff4a14 [ 0.603384][ T63] ---[ end trace 0000000000000000 ]---
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 699df27b0e2f..1d943a13a204 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -150,6 +150,7 @@ config PPC select ARCH_STACKWALK select ARCH_SUPPORTS_ATOMIC_RMW select ARCH_SUPPORTS_DEBUG_PAGEALLOC if PPC_BOOK3S || PPC_8xx || 40x + select ARCH_SUPPORTS_PAGE_TABLE_CHECK select ARCH_USE_BUILTIN_BSWAP select ARCH_USE_CMPXCHG_LOCKREF if PPC64 select ARCH_USE_MEMTEST diff --git a/arch/powerpc/include/asm/book3s/32/pgtable.h b/arch/powerpc/include/asm/book3s/32/pgtable.h index b2fdd3cc81de..44703c8c590c 100644 --- a/arch/powerpc/include/asm/book3s/32/pgtable.h +++ b/arch/powerpc/include/asm/book3s/32/pgtable.h @@ -53,6 +53,8 @@ #ifndef __ASSEMBLY__ +#include <linux/page_table_check.h> + static inline bool pte_user(pte_t pte) { return pte_val(pte) & _PAGE_USER; @@ -337,7 +339,11 @@ static inline int __ptep_test_and_clear_young(struct mm_struct *mm, static inline pte_t ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { - return __pte(pte_update(mm, addr, ptep, ~_PAGE_HASHPTE, 0, 0)); + pte_t old_pte = __pte(pte_update(mm, addr, ptep, ~_PAGE_HASHPTE, 0, 0)); + + page_table_check_pte_clear(mm, addr, old_pte); + + return old_pte; } #define __HAVE_ARCH_PTEP_SET_WRPROTECT @@ -529,6 +535,7 @@ static inline bool pmd_user(pmd_t pmd) static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, int percpu) { + page_table_check_pte_set(mm, addr, ptep, pte); #if defined(CONFIG_SMP) && !defined(CONFIG_PTE_64BIT) /* First case is 32-bit Hash MMU in SMP mode with 32-bit PTEs. We use the * helper pte_update() which does an atomic update. We need to do that diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index e04e3cd378a6..436632d04304 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -162,6 +162,8 @@ #define PAGE_KERNEL_ROX __pgprot(_PAGE_BASE | _PAGE_KERNEL_ROX) #ifndef __ASSEMBLY__ +#include <linux/page_table_check.h> + /* * page table defines */ @@ -465,8 +467,11 @@ static inline void huge_ptep_set_wrprotect(struct mm_struct *mm, static inline pte_t ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { - unsigned long old = pte_update(mm, addr, ptep, ~0UL, 0, 0); - return __pte(old); + pte_t old_pte = __pte(pte_update(mm, addr, ptep, ~0UL, 0, 0)); + + page_table_check_pte_clear(mm, addr, old_pte); + + return old_pte; } #define __HAVE_ARCH_PTEP_GET_AND_CLEAR_FULL @@ -475,11 +480,16 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm, pte_t *ptep, int full) { if (full && radix_enabled()) { + pte_t old_pte; + /* * We know that this is a full mm pte clear and * hence can be sure there is no parallel set_pte. */ - return radix__ptep_get_and_clear_full(mm, addr, ptep, full); + old_pte = radix__ptep_get_and_clear_full(mm, addr, ptep, full); + page_table_check_pte_clear(mm, addr, old_pte); + + return old_pte; } return ptep_get_and_clear(mm, addr, ptep); } @@ -865,6 +875,8 @@ static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr, */ pte = __pte_raw(pte_raw(pte) | cpu_to_be64(_PAGE_PTE)); + page_table_check_pte_set(mm, addr, ptep, pte); + if (radix_enabled()) return radix__set_pte_at(mm, addr, ptep, pte, percpu); return hash__set_pte_at(mm, addr, ptep, pte, percpu); diff --git a/arch/powerpc/include/asm/nohash/32/pgtable.h b/arch/powerpc/include/asm/nohash/32/pgtable.h index 94b2a53f73d5..b3b8a843a1bd 100644 --- a/arch/powerpc/include/asm/nohash/32/pgtable.h +++ b/arch/powerpc/include/asm/nohash/32/pgtable.h @@ -166,6 +166,7 @@ void unmap_kernel_page(unsigned long va); #define _PAGE_CHG_MASK (PTE_RPN_MASK | _PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_SPECIAL) #ifndef __ASSEMBLY__ +#include <linux/page_table_check.h> #define pte_clear(mm, addr, ptep) \ do { pte_update(mm, addr, ptep, ~0, 0, 0); } while (0) @@ -310,7 +311,11 @@ static inline int __ptep_test_and_clear_young(struct mm_struct *mm, static inline pte_t ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { - return __pte(pte_update(mm, addr, ptep, ~0, 0, 0)); + pte_t old_pte = __pte(pte_update(mm, addr, ptep, ~0, 0, 0)); + + page_table_check_pte_clear(mm, addr, old_pte); + + return old_pte; } #define __HAVE_ARCH_PTEP_SET_WRPROTECT diff --git a/arch/powerpc/include/asm/nohash/64/pgtable.h b/arch/powerpc/include/asm/nohash/64/pgtable.h index 69304159aafc..2488da8f0deb 100644 --- a/arch/powerpc/include/asm/nohash/64/pgtable.h +++ b/arch/powerpc/include/asm/nohash/64/pgtable.h @@ -83,6 +83,7 @@ #define H_PAGE_4K_PFN 0 #ifndef __ASSEMBLY__ +#include <linux/page_table_check.h> /* pte_clear moved to later in this file */ static inline pte_t pte_mkwrite(pte_t pte) @@ -258,8 +259,11 @@ static inline void huge_ptep_set_wrprotect(struct mm_struct *mm, static inline pte_t ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { - unsigned long old = pte_update(mm, addr, ptep, ~0UL, 0, 0); - return __pte(old); + pte_t old_pte = __pte(pte_update(mm, addr, ptep, ~0UL, 0, 0)); + + page_table_check_pte_clear(mm, addr, old_pte); + + return old_pte; } static inline void pte_clear(struct mm_struct *mm, unsigned long addr, diff --git a/arch/powerpc/include/asm/nohash/pgtable.h b/arch/powerpc/include/asm/nohash/pgtable.h index 455ae13822ee..0454ed762d39 100644 --- a/arch/powerpc/include/asm/nohash/pgtable.h +++ b/arch/powerpc/include/asm/nohash/pgtable.h @@ -170,6 +170,7 @@ extern void set_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, int percpu) { + page_table_check_pte_set(mm, addr, ptep, pte); /* Second case is 32-bit with 64-bit PTE. In this case, we * can just store as long as we do the two halves in the right order * with a barrier in between.
On creation and clearing of a page table mapping, instrument such calls by invoking page_table_check_pte_set and page_table_check_pte_clear respectively. These calls serve as a sanity check against illegal mappings. Enable ARCH_SUPPORTS_PAGE_TABLE_CHECK for all ppc64, and 32-bit platforms implementing Book3S. Change pud_pfn to be a runtime bug rather than a build bug as it is consumed by page_table_check_pud_{clear,set} which are not called. See also: riscv support in commit 3fee229a8eb9 ("riscv/mm: enable ARCH_SUPPORTS_PAGE_TABLE_CHECK") arm64 in commit 42b2547137f5 ("arm64/mm: enable ARCH_SUPPORTS_PAGE_TABLE_CHECK") x86_64 in commit d283d422c6c4 ("x86: mm: add x86_64 support for page table check") Reviewed-by: Russell Currey <ruscur@russell.cc> Reviewed-by: Christophe Leroy <christophe.leroy@csgroup.eu> Signed-off-by: Rohan McLure <rmclure@linux.ibm.com> --- V2: Update spacing and types assigned to pte_update calls. V3: Update one last pte_update call to remove __pte invocation. --- arch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/book3s/32/pgtable.h | 9 ++++++++- arch/powerpc/include/asm/book3s/64/pgtable.h | 18 +++++++++++++++--- arch/powerpc/include/asm/nohash/32/pgtable.h | 7 ++++++- arch/powerpc/include/asm/nohash/64/pgtable.h | 8 ++++++-- arch/powerpc/include/asm/nohash/pgtable.h | 1 + 6 files changed, 37 insertions(+), 7 deletions(-)