[3/5] On a Linux kernel ask explicitely for a huge page in ggc

From: Andi Kleen <ak@linux.intel.com>

Benchmarks show slightly faster build times on a kernel
build, near the measurement error unfortunately.

This will only work with a recent glibc that defines MADV_HUGEPAGE.

2011-10-08   Andi Kleen <ak@linux.intel.com>

	* ggc-page.c (alloc_page): Add madvise for hugepage
---
 gcc/ggc-page.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

On Sun, Oct 9, 2011 at 9:55 PM, Andi Kleen <andi@firstfloor.org> wrote:
> From: Andi Kleen <ak@linux.intel.com>
>
> Benchmarks show slightly faster build times on a kernel
> build, near the measurement error unfortunately.
>
> This will only work with a recent glibc that defines MADV_HUGEPAGE.

Will partial unmaps fail or split the page?

Richard.

> 2011-10-08   Andi Kleen <ak@linux.intel.com>
>
>        * ggc-page.c (alloc_page): Add madvise for hugepage
> ---
>  gcc/ggc-page.c |    5 +++++
>  1 files changed, 5 insertions(+), 0 deletions(-)
>
> diff --git a/gcc/ggc-page.c b/gcc/ggc-page.c
> index 1f52b56..6e08cda 100644
> --- a/gcc/ggc-page.c
> +++ b/gcc/ggc-page.c
> @@ -779,6 +779,11 @@ alloc_page (unsigned order)
>
>       page = alloc_anon (NULL, G.pagesize * GGC_QUIRE_SIZE);
>
> +#if defined(HAVE_MADVISE) && defined(MADV_HUGEPAGE)
> +      /* Kernel, I would like to have hugepages, please. */
> +      madvise(page, G.pagesize * GGC_QUIRE_SIZE, MADV_HUGEPAGE);
> +#endif
> +
>       /* This loop counts down so that the chain will be in ascending
>         memory order.  */
>       for (i = GGC_QUIRE_SIZE - 1; i >= 1; i--)
> --
> 1.7.5.4
>
>

On Mon, Oct 10, 2011 at 12:15:14PM +0200, Richard Guenther wrote:
> On Sun, Oct 9, 2011 at 9:55 PM, Andi Kleen <andi@firstfloor.org> wrote:
> > From: Andi Kleen <ak@linux.intel.com>
> >
> > Benchmarks show slightly faster build times on a kernel
> > build, near the measurement error unfortunately.
> >
> > This will only work with a recent glibc that defines MADV_HUGEPAGE.
> 
> Will partial unmaps fail or split the page?

I think it will not do anything, because with G.pagesize * GGC_QUIRE_SIZE
being just 2MB (and most likely not 2MB aligned either) it would do
something only if it happens to be 2MB aligned or the following VMA
is also MADV_HUGEPAGE hinted and no pages in the 2MB region have been
paged in yet.
So, either we need to ensure that the address is likely 2MB aligned,
or use on x86_64 even bigger GGC_QUIRE_SIZE (such as 16MB or 32MB) and
then huge pages would be likely used at least for the aligned inner
huge pages in the region.

Additionally, IMHO we shouldn't use this MADV_HUGEPAGE size on all hosts
that define it, that is a useless syscall for hosts which don't support
huge pages.  So should be guarded by some macro defined in host headers.
> > +#if defined(HAVE_MADVISE) && defined(MADV_HUGEPAGE)
> > +      /* Kernel, I would like to have hugepages, please. */
> > +      madvise(page, G.pagesize * GGC_QUIRE_SIZE, MADV_HUGEPAGE);

Please watch formatting, space is missing after madvise.

	Jakub

On Mon, Oct 10, 2011 at 12:15:14PM +0200, Richard Guenther wrote:
> On Sun, Oct 9, 2011 at 9:55 PM, Andi Kleen <andi@firstfloor.org> wrote:
> > From: Andi Kleen <ak@linux.intel.com>
> >
> > Benchmarks show slightly faster build times on a kernel
> > build, near the measurement error unfortunately.
> >
> > This will only work with a recent glibc that defines MADV_HUGEPAGE.
> 
> Will partial unmaps fail or split the page?

They split the page.

-Andi

On Mon, Oct 10, 2011 at 12:29:03PM +0200, Jakub Jelinek wrote:
> On Mon, Oct 10, 2011 at 12:15:14PM +0200, Richard Guenther wrote:
> > On Sun, Oct 9, 2011 at 9:55 PM, Andi Kleen <andi@firstfloor.org> wrote:
> > > From: Andi Kleen <ak@linux.intel.com>
> > >
> > > Benchmarks show slightly faster build times on a kernel
> > > build, near the measurement error unfortunately.
> > >
> > > This will only work with a recent glibc that defines MADV_HUGEPAGE.
> > 
> > Will partial unmaps fail or split the page?
> 
> I think it will not do anything, because with G.pagesize * GGC_QUIRE_SIZE

On large builds (LTO) I see the vmstat THP counter increasing, 
it doesn't do too much on small builds.

> being just 2MB (and most likely not 2MB aligned either) it would do
> something only if it happens to be 2MB aligned or the following VMA
> is also MADV_HUGEPAGE hinted and no pages in the 2MB region have been
> paged in yet.
> So, either we need to ensure that the address is likely 2MB aligned,
> or use on x86_64 even bigger GGC_QUIRE_SIZE (such as 16MB or 32MB) and
> then huge pages would be likely used at least for the aligned inner
> huge pages in the region.

Hmm, that gets too complicated for mee. I'll drop the patch for now.
It's not strictly needed to fix the fragmentation problem
and it's also possible to force THP from the kernel.

-Andi