| Message ID | 20260922-rcu-pagetable-freeing-v4-8-fe1ad1f1e303@kernel.org |
|---|---|
| State | New |
| Headers | show |
| Series | mm: make userland page table freeing RCU-safe | expand |
On Tue, Sep 22, 2026 at 04:35:39PM +0100, Lorenzo Stoakes (ARM) wrote: >Currently, non-x2 sh specifies CONFIG_MMU_GATHER_RCU_TABLE_FREE allowing >RCU page table freeing. > >sh-X2 is problematic because it utilises slab-allocated PMD page tables, >and thus tlb_remove_ptdesc() cannot be used in these cases. > >All other sh variants are fine as commit e3ecf7c7d082 ("mm: pgtable: >convert some architectures to use tlb_remove_ptdesc()") already converted >page table freeing to use tlb_remove_ptdesc(), which does so after an RCU >grace period when CONFIG_MMU_GATHER_RCU_TABLE_FREE is specified. > >Resolve this issue by firstly specifying CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE >for sh-X2, so the arch can provide its own __tlb_remove_table() >implementation (called after the RCU grace period). > >Then, convert __pmd_free_tlb() to tag the pointer to the PMD, and have >__tlb_remove_table() check this tag to determine whether to free via the >slab or to use pagetable_dtor_free(). Makes sense to me, so RCU callback won't mistake a slab PMD for a PTE page :) >This follows the pattern used by sparc64 as implemented in commit >4a0100f7546f ("sparc64: use RCU page table freeing"). > >Previously __pmd_free_tlb() freed PMD page tables immediately, before any >TLB flush IPI. This seems to be a pre-existing bug, which this change also >resolves. > >CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE is only specified for sh-X2, as setting >it disables CONFIG_PT_RECLAIM and causes __tlb_remove_table_one() to call >tlb_remove_table_sync_rcu() and synchronize_rcu() in turn, and this is not >necessary for other sh variants. > >This forms part of an overall effort to switch every architecture to this >mode. > >Acked-by: Kiryl Shutsemau (Meta) <kas@kernel.org> >Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> >--- LGTM, feel free to add: Acked-by: Lance Yang <lance.yang@linux.dev>
On Wed, Sep 23, 2026 at 07:54:17PM +0800, Lance Yang wrote: > > On Tue, Sep 22, 2026 at 04:35:39PM +0100, Lorenzo Stoakes (ARM) wrote: > >Currently, non-x2 sh specifies CONFIG_MMU_GATHER_RCU_TABLE_FREE allowing > >RCU page table freeing. > > > >sh-X2 is problematic because it utilises slab-allocated PMD page tables, > >and thus tlb_remove_ptdesc() cannot be used in these cases. > > > >All other sh variants are fine as commit e3ecf7c7d082 ("mm: pgtable: > >convert some architectures to use tlb_remove_ptdesc()") already converted > >page table freeing to use tlb_remove_ptdesc(), which does so after an RCU > >grace period when CONFIG_MMU_GATHER_RCU_TABLE_FREE is specified. > > > >Resolve this issue by firstly specifying CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE > >for sh-X2, so the arch can provide its own __tlb_remove_table() > >implementation (called after the RCU grace period). > > > >Then, convert __pmd_free_tlb() to tag the pointer to the PMD, and have > >__tlb_remove_table() check this tag to determine whether to free via the > >slab or to use pagetable_dtor_free(). > > Makes sense to me, so RCU callback won't mistake a slab PMD for a PTE > page :) Yes indeed :) > > >This follows the pattern used by sparc64 as implemented in commit > >4a0100f7546f ("sparc64: use RCU page table freeing"). > > > >Previously __pmd_free_tlb() freed PMD page tables immediately, before any > >TLB flush IPI. This seems to be a pre-existing bug, which this change also > >resolves. > > > >CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE is only specified for sh-X2, as setting > >it disables CONFIG_PT_RECLAIM and causes __tlb_remove_table_one() to call > >tlb_remove_table_sync_rcu() and synchronize_rcu() in turn, and this is not > >necessary for other sh variants. > > > >This forms part of an overall effort to switch every architecture to this > >mode. > > > >Acked-by: Kiryl Shutsemau (Meta) <kas@kernel.org> > >Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> > >--- > > LGTM, feel free to add: > > Acked-by: Lance Yang <lance.yang@linux.dev> Thanks! -- Cheers, Lorenzo
diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig index 204f64912f0e..75236bef6f16 100644 --- a/arch/sh/Kconfig +++ b/arch/sh/Kconfig @@ -33,6 +33,7 @@ config SUPERH select HAVE_ARCH_AUDITSYSCALL select HAVE_ARCH_KGDB select HAVE_ARCH_SECCOMP_FILTER + select HAVE_ARCH_TLB_REMOVE_TABLE if X2TLB select HAVE_ARCH_TRACEHOOK select HAVE_DEBUG_BUGVERBOSE select HAVE_DEBUG_KMEMLEAK @@ -61,7 +62,7 @@ config SUPERH select HAVE_SYSCALL_TRACEPOINTS select IRQ_FORCED_THREADING select LOCK_MM_AND_FIND_VMA - select MMU_GATHER_RCU_TABLE_FREE if MMU && !X2TLB + select MMU_GATHER_RCU_TABLE_FREE if MMU select MODULES_USE_ELF_RELA select NEED_SG_DMA_LENGTH select NO_DMA if !MMU && !DMA_COHERENT diff --git a/arch/sh/include/asm/pgalloc.h b/arch/sh/include/asm/pgalloc.h index 6fe7123d38fa..67ce7fa23fa1 100644 --- a/arch/sh/include/asm/pgalloc.h +++ b/arch/sh/include/asm/pgalloc.h @@ -17,7 +17,11 @@ extern void pgd_free(struct mm_struct *mm, pgd_t *pgd); extern void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmd); extern pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address); extern void pmd_free(struct mm_struct *mm, pmd_t *pmd); -#define __pmd_free_tlb(tlb, pmdp, addr) pmd_free((tlb)->mm, (pmdp)) +extern void __tlb_remove_table(void *table); + +/* PMDs are slab-allocated, tag so they are freed correctly. */ +#define __pmd_free_tlb(tlb, pmdp, addr) \ + tlb_remove_table((tlb), (void *)((unsigned long)(pmdp) | 1)) #endif static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, diff --git a/arch/sh/mm/pgtable.c b/arch/sh/mm/pgtable.c index 3a4085ea0161..f6184b86b89c 100644 --- a/arch/sh/mm/pgtable.c +++ b/arch/sh/mm/pgtable.c @@ -56,4 +56,24 @@ void pmd_free(struct mm_struct *mm, pmd_t *pmd) { kmem_cache_free(pmd_cachep, pmd); } + +static void __tlb_remove_table_slab(void *table) +{ + kmem_cache_free(pmd_cachep, table); +} + +static void __tlb_remove_table_pgtable(void *table) +{ + pagetable_dtor_free(table); +} + +void __tlb_remove_table(void *table) +{ + const unsigned long addr = (unsigned long)table; + + if (addr & 1) + __tlb_remove_table_slab((void *)(addr & ~1UL)); + else + __tlb_remove_table_pgtable(table); +} #endif /* PAGETABLE_LEVELS > 2 */
Currently, non-x2 sh specifies CONFIG_MMU_GATHER_RCU_TABLE_FREE allowing RCU page table freeing. sh-X2 is problematic because it utilises slab-allocated PMD page tables, and thus tlb_remove_ptdesc() cannot be used in these cases. All other sh variants are fine as commit e3ecf7c7d082 ("mm: pgtable: convert some architectures to use tlb_remove_ptdesc()") already converted page table freeing to use tlb_remove_ptdesc(), which does so after an RCU grace period when CONFIG_MMU_GATHER_RCU_TABLE_FREE is specified. Resolve this issue by firstly specifying CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE for sh-X2, so the arch can provide its own __tlb_remove_table() implementation (called after the RCU grace period). Then, convert __pmd_free_tlb() to tag the pointer to the PMD, and have __tlb_remove_table() check this tag to determine whether to free via the slab or to use pagetable_dtor_free(). This follows the pattern used by sparc64 as implemented in commit 4a0100f7546f ("sparc64: use RCU page table freeing"). Previously __pmd_free_tlb() freed PMD page tables immediately, before any TLB flush IPI. This seems to be a pre-existing bug, which this change also resolves. CONFIG_HAVE_ARCH_TLB_REMOVE_TABLE is only specified for sh-X2, as setting it disables CONFIG_PT_RECLAIM and causes __tlb_remove_table_one() to call tlb_remove_table_sync_rcu() and synchronize_rcu() in turn, and this is not necessary for other sh variants. This forms part of an overall effort to switch every architecture to this mode. Acked-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> --- arch/sh/Kconfig | 3 ++- arch/sh/include/asm/pgalloc.h | 6 +++++- arch/sh/mm/pgtable.c | 20 ++++++++++++++++++++ 3 files changed, 27 insertions(+), 2 deletions(-)