[v6,02/11] mm/gup: Use functions to track lockless pgtbl walks on gup_pgd_range
diff mbox series

Message ID 20200206030900.147032-3-leonardo@linux.ibm.com
State Superseded
Headers show
Series
  • Introduces new functions for tracking lockless pagetable walks
Related show

Checks

Context Check Description
snowpatch_ozlabs/needsstable success Patch has no Fixes tags
snowpatch_ozlabs/checkpatch success total: 0 errors, 0 warnings, 0 checks, 30 lines checked
snowpatch_ozlabs/apply_patch success Successfully applied on branch powerpc/merge (530a1cfd52af0aba1af4b1c9a7bc66a202a459b1)

Commit Message

Leonardo Bras Feb. 6, 2020, 3:08 a.m. UTC
As described, gup_pgd_range is a lockless pagetable walk. So, in order to
track against THP split/collapse, it disables/enables irq around it.

To make use of the new tracking functions, it replaces irq disable/enable
by {begin,end}_lockless_pgtbl_walk().

As local_irq_{save,restore} is present inside
{begin,end}_lockless_pgtbl_walk, there should be no change in the
workings.

Signed-off-by: Leonardo Bras <leonardo@linux.ibm.com>
---
 mm/gup.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

Comments

Leonardo Bras Feb. 6, 2020, 3:25 a.m. UTC | #1
On Thu, 2020-02-06 at 00:08 -0300, Leonardo Bras wrote:
>                 gup_pgd_range(addr, end, gup_flags, pages, &nr);
> -               local_irq_enable();
> +               end_lockless_pgtbl_walk(IRQS_ENABLED);
>                 ret = nr;
>         }
>  

Just noticed IRQS_ENABLED is not available on other archs than ppc64.
I will fix this for v7.
kbuild test robot Feb. 7, 2020, 1:19 a.m. UTC | #2
Hi Leonardo,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on powerpc/next]
[also build test ERROR on paulus-powerpc/kvm-ppc-next linus/master v5.5 next-20200206]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Leonardo-Bras/Introduces-new-functions-for-tracking-lockless-pagetable-walks/20200207-071035
base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
config: i386-tinyconfig (attached as .config)
compiler: gcc-7 (Debian 7.5.0-3) 7.5.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   mm/gup.c: In function 'get_user_pages_fast':
>> mm/gup.c:2435:27: error: 'IRQS_ENABLED' undeclared (first use in this function); did you mean 'IS_ENABLED'?
      end_lockless_pgtbl_walk(IRQS_ENABLED);
                              ^~~~~~~~~~~~
                              IS_ENABLED
   mm/gup.c:2435:27: note: each undeclared identifier is reported only once for each function it appears in

vim +2435 mm/gup.c

  2395	
  2396	/**
  2397	 * get_user_pages_fast() - pin user pages in memory
  2398	 * @start:	starting user address
  2399	 * @nr_pages:	number of pages from start to pin
  2400	 * @gup_flags:	flags modifying pin behaviour
  2401	 * @pages:	array that receives pointers to the pages pinned.
  2402	 *		Should be at least nr_pages long.
  2403	 *
  2404	 * Attempt to pin user pages in memory without taking mm->mmap_sem.
  2405	 * If not successful, it will fall back to taking the lock and
  2406	 * calling get_user_pages().
  2407	 *
  2408	 * Returns number of pages pinned. This may be fewer than the number
  2409	 * requested. If nr_pages is 0 or negative, returns 0. If no pages
  2410	 * were pinned, returns -errno.
  2411	 */
  2412	int get_user_pages_fast(unsigned long start, int nr_pages,
  2413				unsigned int gup_flags, struct page **pages)
  2414	{
  2415		unsigned long addr, len, end;
  2416		int nr = 0, ret = 0;
  2417	
  2418		if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM)))
  2419			return -EINVAL;
  2420	
  2421		start = untagged_addr(start) & PAGE_MASK;
  2422		addr = start;
  2423		len = (unsigned long) nr_pages << PAGE_SHIFT;
  2424		end = start + len;
  2425	
  2426		if (end <= start)
  2427			return 0;
  2428		if (unlikely(!access_ok((void __user *)start, len)))
  2429			return -EFAULT;
  2430	
  2431		if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) &&
  2432		    gup_fast_permitted(start, end)) {
  2433			begin_lockless_pgtbl_walk();
  2434			gup_pgd_range(addr, end, gup_flags, pages, &nr);
> 2435			end_lockless_pgtbl_walk(IRQS_ENABLED);

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
kbuild test robot Feb. 7, 2020, 8:01 a.m. UTC | #3
Hi Leonardo,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on powerpc/next]
[also build test ERROR on asm-generic/master paulus-powerpc/kvm-ppc-next linus/master v5.5 next-20200207]
[cannot apply to kvm-ppc/kvm-ppc-next]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Leonardo-Bras/Introduces-new-functions-for-tracking-lockless-pagetable-walks/20200207-071035
base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
config: mips-randconfig-a001-20200207 (attached as .config)
compiler: mipsel-linux-gcc (GCC) 5.5.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        GCC_VERSION=5.5.0 make.cross ARCH=mips 

If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   mm/gup.c: In function 'get_user_pages_fast':
>> mm/gup.c:2435:27: error: 'IRQS_ENABLED' undeclared (first use in this function)
      end_lockless_pgtbl_walk(IRQS_ENABLED);
                              ^
   mm/gup.c:2435:27: note: each undeclared identifier is reported only once for each function it appears in

vim +/IRQS_ENABLED +2435 mm/gup.c

  2395	
  2396	/**
  2397	 * get_user_pages_fast() - pin user pages in memory
  2398	 * @start:	starting user address
  2399	 * @nr_pages:	number of pages from start to pin
  2400	 * @gup_flags:	flags modifying pin behaviour
  2401	 * @pages:	array that receives pointers to the pages pinned.
  2402	 *		Should be at least nr_pages long.
  2403	 *
  2404	 * Attempt to pin user pages in memory without taking mm->mmap_sem.
  2405	 * If not successful, it will fall back to taking the lock and
  2406	 * calling get_user_pages().
  2407	 *
  2408	 * Returns number of pages pinned. This may be fewer than the number
  2409	 * requested. If nr_pages is 0 or negative, returns 0. If no pages
  2410	 * were pinned, returns -errno.
  2411	 */
  2412	int get_user_pages_fast(unsigned long start, int nr_pages,
  2413				unsigned int gup_flags, struct page **pages)
  2414	{
  2415		unsigned long addr, len, end;
  2416		int nr = 0, ret = 0;
  2417	
  2418		if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM)))
  2419			return -EINVAL;
  2420	
  2421		start = untagged_addr(start) & PAGE_MASK;
  2422		addr = start;
  2423		len = (unsigned long) nr_pages << PAGE_SHIFT;
  2424		end = start + len;
  2425	
  2426		if (end <= start)
  2427			return 0;
  2428		if (unlikely(!access_ok((void __user *)start, len)))
  2429			return -EFAULT;
  2430	
  2431		if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) &&
  2432		    gup_fast_permitted(start, end)) {
  2433			begin_lockless_pgtbl_walk();
  2434			gup_pgd_range(addr, end, gup_flags, pages, &nr);
> 2435			end_lockless_pgtbl_walk(IRQS_ENABLED);

---
0-DAY kernel test infrastructure                 Open Source Technology Center
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org Intel Corporation
John Hubbard Feb. 7, 2020, 10:54 p.m. UTC | #4
On 2/5/20 7:25 PM, Leonardo Bras wrote:
> On Thu, 2020-02-06 at 00:08 -0300, Leonardo Bras wrote:
>>                 gup_pgd_range(addr, end, gup_flags, pages, &nr);
>> -               local_irq_enable();
>> +               end_lockless_pgtbl_walk(IRQS_ENABLED);
>>                 ret = nr;
>>         }
>>  
> 
> Just noticed IRQS_ENABLED is not available on other archs than ppc64.
> I will fix this for v7.
> 

What's the fix going to look like, approximately?


thanks,
Leonardo Bras Feb. 17, 2020, 8:55 p.m. UTC | #5
Hello John, comments inline;

On Fri, 2020-02-07 at 14:54 -0800, John Hubbard wrote:
> On 2/5/20 7:25 PM, Leonardo Bras wrote:
> > On Thu, 2020-02-06 at 00:08 -0300, Leonardo Bras wrote:
> > >                 gup_pgd_range(addr, end, gup_flags, pages, &nr);
> > > -               local_irq_enable();
> > > +               end_lockless_pgtbl_walk(IRQS_ENABLED);
> > >                 ret = nr;
> > >         }
> > >  
> > 
> > Just noticed IRQS_ENABLED is not available on other archs than ppc64.
> > I will fix this for v7.
> > 
> 
> What's the fix going to look like, approximately?

I am not sure what is the best approach yet. 

1. On irq_mask == 0, always enable irq on end_lockless_pgtbl_walk().
   Not sure how bat would that affect other archs.

2. Add another function like end_lockless_pgtbl_walk_irqen() that
always enables IRQ.

3. Add another parameter in end_lockless_pgtbl_walk(), so that caller
can choose ii IRQ must be enabled.

Also, not sure if internal_get_user_pages_fast() can possibly be called
with IRQ disabled, and then return with it enabled. Maybe just
saving/restoring should be fine.

Other suggestions are welcome.

> 
> 
> thanks,

Best regards,

Leonardo Bras

Patch
diff mbox series

diff --git a/mm/gup.c b/mm/gup.c
index 1b521e0ac1de..04e6f46993b6 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -2369,7 +2369,7 @@  int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
 			  struct page **pages)
 {
 	unsigned long len, end;
-	unsigned long flags;
+	unsigned long irq_mask;
 	int nr = 0;
 
 	start = untagged_addr(start) & PAGE_MASK;
@@ -2395,9 +2395,9 @@  int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
 
 	if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) &&
 	    gup_fast_permitted(start, end)) {
-		local_irq_save(flags);
+		irq_mask = begin_lockless_pgtbl_walk();
 		gup_pgd_range(start, end, write ? FOLL_WRITE : 0, pages, &nr);
-		local_irq_restore(flags);
+		end_lockless_pgtbl_walk(irq_mask);
 	}
 
 	return nr;
@@ -2450,9 +2450,9 @@  static int internal_get_user_pages_fast(unsigned long start, int nr_pages,
 
 	if (IS_ENABLED(CONFIG_HAVE_FAST_GUP) &&
 	    gup_fast_permitted(start, end)) {
-		local_irq_disable();
+		begin_lockless_pgtbl_walk();
 		gup_pgd_range(addr, end, gup_flags, pages, &nr);
-		local_irq_enable();
+		end_lockless_pgtbl_walk(IRQS_ENABLED);
 		ret = nr;
 	}