| Message ID | 20190920195047.7703-2-leonardo@linux.ibm.com (mailing list archive) |
|---|---|
| State | Superseded, archived |
| Headers | show |
| Series | Introduces new count-based method for monitoring lockless pagetable wakls | expand |
| Context | Check | Description |
|---|---|---|
| snowpatch_ozlabs/apply_patch | success | Successfully applied on branch next (39d90246212423715005d06e4deac033174bae44) |
| snowpatch_ozlabs/checkpatch | success | total: 0 errors, 0 warnings, 0 checks, 39 lines checked |
On 9/20/19 12:50 PM, Leonardo Bras wrote: > It's necessary to monitor lockless pagetable walks, in order to avoid doing > THP splitting/collapsing during them. > > Some methods rely on local_irq_{save,restore}, but that can be slow on > cases with a lot of cpus are used for the process. > > In order to speedup some cases, I propose a refcount-based approach, that > counts the number of lockless pagetable walks happening on the process. > > This method does not exclude the current irq-oriented method. It works as a > complement to skip unnecessary waiting. > > start_lockless_pgtbl_walk(mm) > Insert before starting any lockless pgtable walk > end_lockless_pgtbl_walk(mm) > Insert after the end of any lockless pgtable walk > (Mostly after the ptep is last used) > running_lockless_pgtbl_walk(mm) > Returns the number of lockless pgtable walks running > > Signed-off-by: Leonardo Bras <leonardo@linux.ibm.com> > --- > arch/powerpc/include/asm/book3s/64/mmu.h | 3 +++ > arch/powerpc/mm/book3s64/mmu_context.c | 1 + > arch/powerpc/mm/book3s64/pgtable.c | 17 +++++++++++++++++ > 3 files changed, 21 insertions(+) > > diff --git a/arch/powerpc/include/asm/book3s/64/mmu.h b/arch/powerpc/include/asm/book3s/64/mmu.h > index 23b83d3593e2..13b006e7dde4 100644 > --- a/arch/powerpc/include/asm/book3s/64/mmu.h > +++ b/arch/powerpc/include/asm/book3s/64/mmu.h > @@ -116,6 +116,9 @@ typedef struct { > /* Number of users of the external (Nest) MMU */ > atomic_t copros; > > + /* Number of running instances of lockless pagetable walk*/ > + atomic_t lockless_pgtbl_walk_count; > + > struct hash_mm_context *hash_context; > > unsigned long vdso_base; > diff --git a/arch/powerpc/mm/book3s64/mmu_context.c b/arch/powerpc/mm/book3s64/mmu_context.c > index 2d0cb5ba9a47..3dd01c0ca5be 100644 > --- a/arch/powerpc/mm/book3s64/mmu_context.c > +++ b/arch/powerpc/mm/book3s64/mmu_context.c > @@ -200,6 +200,7 @@ int init_new_context(struct task_struct *tsk, struct mm_struct *mm) > #endif > atomic_set(&mm->context.active_cpus, 0); > atomic_set(&mm->context.copros, 0); > + atomic_set(&mm->context.lockless_pgtbl_walk_count, 0); > > return 0; > } > diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c > index 7d0e0d0d22c4..13239b17a22c 100644 > --- a/arch/powerpc/mm/book3s64/pgtable.c > +++ b/arch/powerpc/mm/book3s64/pgtable.c > @@ -98,6 +98,23 @@ void serialize_against_pte_lookup(struct mm_struct *mm) > smp_call_function_many(mm_cpumask(mm), do_nothing, NULL, 1); > } > Somewhere, there should be a short comment that explains how the following functions are meant to be used. And it should include the interaction with irqs, so maybe if you end up adding that combined wrapper function that does both, that's where the documentation would go. If not, then here is probably where it goes. > +void start_lockless_pgtbl_walk(struct mm_struct *mm) > +{ > + atomic_inc(&mm->context.lockless_pgtbl_walk_count); > +} > +EXPORT_SYMBOL(start_lockless_pgtbl_walk); > + > +void end_lockless_pgtbl_walk(struct mm_struct *mm) > +{ > + atomic_dec(&mm->context.lockless_pgtbl_walk_count); > +} > +EXPORT_SYMBOL(end_lockless_pgtbl_walk); > + > +int running_lockless_pgtbl_walk(struct mm_struct *mm) > +{ > + return atomic_read(&mm->context.lockless_pgtbl_walk_count); > +} > + thanks,
On Mon, 2019-09-23 at 13:42 -0700, John Hubbard wrote: > Somewhere, there should be a short comment that explains how the following functions > are meant to be used. And it should include the interaction with irqs, so maybe > if you end up adding that combined wrapper function that does both, that's > where the documentation would go. If not, then here is probably where it goes. Thanks for the feedback. I will make sure to add comments here for v3.
diff --git a/arch/powerpc/include/asm/book3s/64/mmu.h b/arch/powerpc/include/asm/book3s/64/mmu.h index 23b83d3593e2..13b006e7dde4 100644 --- a/arch/powerpc/include/asm/book3s/64/mmu.h +++ b/arch/powerpc/include/asm/book3s/64/mmu.h @@ -116,6 +116,9 @@ typedef struct { /* Number of users of the external (Nest) MMU */ atomic_t copros; + /* Number of running instances of lockless pagetable walk*/ + atomic_t lockless_pgtbl_walk_count; + struct hash_mm_context *hash_context; unsigned long vdso_base; diff --git a/arch/powerpc/mm/book3s64/mmu_context.c b/arch/powerpc/mm/book3s64/mmu_context.c index 2d0cb5ba9a47..3dd01c0ca5be 100644 --- a/arch/powerpc/mm/book3s64/mmu_context.c +++ b/arch/powerpc/mm/book3s64/mmu_context.c @@ -200,6 +200,7 @@ int init_new_context(struct task_struct *tsk, struct mm_struct *mm) #endif atomic_set(&mm->context.active_cpus, 0); atomic_set(&mm->context.copros, 0); + atomic_set(&mm->context.lockless_pgtbl_walk_count, 0); return 0; } diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c index 7d0e0d0d22c4..13239b17a22c 100644 --- a/arch/powerpc/mm/book3s64/pgtable.c +++ b/arch/powerpc/mm/book3s64/pgtable.c @@ -98,6 +98,23 @@ void serialize_against_pte_lookup(struct mm_struct *mm) smp_call_function_many(mm_cpumask(mm), do_nothing, NULL, 1); } +void start_lockless_pgtbl_walk(struct mm_struct *mm) +{ + atomic_inc(&mm->context.lockless_pgtbl_walk_count); +} +EXPORT_SYMBOL(start_lockless_pgtbl_walk); + +void end_lockless_pgtbl_walk(struct mm_struct *mm) +{ + atomic_dec(&mm->context.lockless_pgtbl_walk_count); +} +EXPORT_SYMBOL(end_lockless_pgtbl_walk); + +int running_lockless_pgtbl_walk(struct mm_struct *mm) +{ + return atomic_read(&mm->context.lockless_pgtbl_walk_count); +} + /* * We use this to invalidate a pmdp entry before switching from a * hugepte to regular pmd entry.
It's necessary to monitor lockless pagetable walks, in order to avoid doing THP splitting/collapsing during them. Some methods rely on local_irq_{save,restore}, but that can be slow on cases with a lot of cpus are used for the process. In order to speedup some cases, I propose a refcount-based approach, that counts the number of lockless pagetable walks happening on the process. This method does not exclude the current irq-oriented method. It works as a complement to skip unnecessary waiting. start_lockless_pgtbl_walk(mm) Insert before starting any lockless pgtable walk end_lockless_pgtbl_walk(mm) Insert after the end of any lockless pgtable walk (Mostly after the ptep is last used) running_lockless_pgtbl_walk(mm) Returns the number of lockless pgtable walks running Signed-off-by: Leonardo Bras <leonardo@linux.ibm.com> --- arch/powerpc/include/asm/book3s/64/mmu.h | 3 +++ arch/powerpc/mm/book3s64/mmu_context.c | 1 + arch/powerpc/mm/book3s64/pgtable.c | 17 +++++++++++++++++ 3 files changed, 21 insertions(+)