| Message ID | 1455860327-22913-1-git-send-email-rashmicy@gmail.com (mailing list archive) |
|---|---|
| State | Accepted |
| Headers | show |
On 19/02/16 16:38, Rashmica Gupta wrote: > Currently on PPC64 changing kernel pagesize from 4K to 64K leaves > FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. > > The error occurs because of the following constraint (from > include/linux/mmzone.h) being violated: > > MAX_ORDER -1 + PAGESHIFT <= SECTION_SIZE_BITS. > > Expanding this out, we get: > > FORCE_MAX_ZONEBITS <= 25 - PAGESHIFT, > > which requires, for a 64K page, FORCE_MAX_ZONEBITS <= 9. Thus set max > value of FORCE_MAX_ZONEORDER for 64K pages to 9, and 4K pages to 13. > > Also, check the minimum value: > In include/linux/huge_mm.h, we have the constraint HPAGE_PMD_ORDER < > MAX_ORDER which expands out to: > > PTE_INDEX_SIZE < FORCE_MAX_ZONEORDER. > > PTE_INDEX_SIZE is: > 9 (4k hash or no hash 4K pgtable) or > 8 (64K hash or no hash 64K pgtable). > Thus a min value of 8 for 64K pages and 9 for 4K pages is reasonable. > > So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages > and from 13-64 to 9-13 for 4K pages. > > Signed-off-by: Rashmica Gupta <rashmicy@gmail.com> > --- > > v2: Changed the range for 4K pages and minimum for 64K pages as suggested > by Balbir Singh. > > > arch/powerpc/Kconfig | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig > index e4824fd04bb7..b933530821fb 100644 > --- a/arch/powerpc/Kconfig > +++ b/arch/powerpc/Kconfig > @@ -585,9 +585,9 @@ endchoice > > config FORCE_MAX_ZONEORDER > int "Maximum zone order" > - range 9 64 if PPC64 && PPC_64K_PAGES > + range 8 9 if PPC64 && PPC_64K_PAGES > default "9" if PPC64 && PPC_64K_PAGES > - range 13 64 if PPC64 && !PPC_64K_PAGES > + range 9 13 if PPC64 && !PPC_64K_PAGES > default "13" if PPC64 && !PPC_64K_PAGES > range 9 64 if PPC32 && PPC_16K_PAGES > default "9" if PPC32 && PPC_16K_PAGES Reviewed-by: Balbir Singh <bsingharora@gmail.com> Balbir Singh
On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: > Currently on PPC64 changing kernel pagesize from 4K to 64K leaves > FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. > ... > So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages > and from 13-64 to 9-13 for 4K pages. > > Signed-off-by: Rashmica Gupta <rashmicy@gmail.com> > Reviewed-by: Balbir Singh <bsingharora@gmail.com> Applied to powerpc next, thanks. https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea cheers
Michael Ellerman <mpe@ellerman.id.au> writes: > On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: >> Currently on PPC64 changing kernel pagesize from 4K to 64K leaves >> FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. >> > ... >> So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages >> and from 13-64 to 9-13 for 4K pages. >> >> Signed-off-by: Rashmica Gupta <rashmicy@gmail.com> >> Reviewed-by: Balbir Singh <bsingharora@gmail.com> > > Applied to powerpc next, thanks. > > https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea > HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page size. We do have the below constraint w.r.t hugetlb pages static inline bool hstate_is_gigantic(struct hstate *h) { return huge_page_order(h) >= MAX_ORDER; } That require MAX_ORDER to be greater than 12. Did we test hugetlbfs 4k config with this patch ? Will it work if we start marking hugepage as gigantic page ? -aneesh
On 20/04/16 00:59, Aneesh Kumar K.V wrote: > Michael Ellerman <mpe@ellerman.id.au> writes: > >> On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: >>> Currently on PPC64 changing kernel pagesize from 4K to 64K leaves >>> FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. >>> >> ... >>> So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages >>> and from 13-64 to 9-13 for 4K pages. >>> >>> Signed-off-by: Rashmica Gupta <rashmicy@gmail.com> >>> Reviewed-by: Balbir Singh <bsingharora@gmail.com> >> >> Applied to powerpc next, thanks. >> >> https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea >> > > HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page > size. We do have the below constraint w.r.t hugetlb pages > > static inline bool hstate_is_gigantic(struct hstate *h) > { > return huge_page_order(h) >= MAX_ORDER; > } > > That require MAX_ORDER to be greater than 12. > The build will fail for MAX_ZONEORDER beyond the specified limits. MAX_ORDER > 12 for what page size? My understanding is this 1. gigantic refers to the fact the regular allocators cannot allocate this page 2. Use alloc_contig_range() with CONFIG_CMA for gigantic pages I could be wrong > Did we test hugetlbfs 4k config with this patch ? Will it work if we > start marking hugepage as gigantic page ? Nope.. I did not Thanks for the review! Balbir Singh
> Michael Ellerman <mpe@ellerman.id.au> writes: > >> On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: >>> Currently on PPC64 changing kernel pagesize from 4K to 64K leaves >>> FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. >>> >> ... >>> So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages >>> and from 13-64 to 9-13 for 4K pages. >>> >>> Signed-off-by: Rashmica Gupta <rashmicy@gmail.com> >>> Reviewed-by: Balbir Singh <bsingharora@gmail.com> >> Applied to powerpc next, thanks. >> >> https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea >> > HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page > size. We do have the below constraint w.r.t hugetlb pages > > static inline bool hstate_is_gigantic(struct hstate *h) > { > return huge_page_order(h) >= MAX_ORDER; > } > > That require MAX_ORDER to be greater than 12. > > Did we test hugetlbfs 4k config with this patch ? Will it work if we > start marking hugepage as gigantic page ? > > -aneesh > Hello Rashmica, With upstream linux kernel 4.8.0-rc1-00006-gbae9cc6 compiled with linux 4k page size we are not able set hugepages, Aneesh had a look at the problem and he mentioned this commit is causing the issue. *Details:* We are using pkvm ubuntu 16.04 guest with upstream kernel [4.8.0-rc1-00006-gbae9cc6] compiled with 4k page size o/p from guest: HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 16384 kB Page sizes from device-tree: [dmesg] [ 0.000000] base_shift=12: shift=12, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=0 [ 0.000000] base_shift=12: shift=24, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=56 [ 0.000000] base_shift=24: shift=24, sllp=0x0100, avpnm=0x00000001, tlbiel=0, penc=0 while trying to configure the hugepages inside the guest it throws the below error: echo 100 > /proc/sys/vm/nr_hugepages -bash: echo: write error: Invalid argument *Note*: we do not see the problem when the linux page is 64k Thanks, Santhosh G
santhosh <santhog4@linux.vnet.ibm.com> writes: >> Michael Ellerman <mpe@ellerman.id.au> writes: >> >>> On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: >>>> Currently on PPC64 changing kernel pagesize from 4K to 64K leaves >>>> FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. >>>> >>> ... >>>> So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages >>>> and from 13-64 to 9-13 for 4K pages. >>> >>> https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea >>> >> HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page >> size. We do have the below constraint w.r.t hugetlb pages >> >> static inline bool hstate_is_gigantic(struct hstate *h) >> { >> return huge_page_order(h) >= MAX_ORDER; >> } >> >> That require MAX_ORDER to be greater than 12. So have you tried that fix? cheers
On 14/09/16 20:40, santhosh wrote: > >> Michael Ellerman <mpe@ellerman.id.au> writes: >> >>> On Fri, 2016-19-02 at 05:38:47 UTC, Rashmica Gupta wrote: >>>> Currently on PPC64 changing kernel pagesize from 4K to 64K leaves >>>> FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. >>>> >>> ... >>>> So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages >>>> and from 13-64 to 9-13 for 4K pages. >>>> >>>> Signed-off-by: Rashmica Gupta <rashmicy@gmail.com> >>>> Reviewed-by: Balbir Singh <bsingharora@gmail.com> >>> Applied to powerpc next, thanks. >>> >>> https://git.kernel.org/powerpc/c/a7ee539584acf4a565b7439cea >>> >> HPAGE_PMD_ORDER is not something we should check w.r.t 4k linux page >> size. We do have the below constraint w.r.t hugetlb pages >> >> static inline bool hstate_is_gigantic(struct hstate *h) >> { >> return huge_page_order(h) >= MAX_ORDER; >> } >> >> That require MAX_ORDER to be greater than 12. >> 9 to 13 was done based on calculations you can find the commit >> Did we test hugetlbfs 4k config with this patch ? Will it work if we >> start marking hugepage as gigantic page ? >> >> -aneesh >> > Hello Rashmica, > > With upstream linux kernel 4.8.0-rc1-00006-gbae9cc6 compiled with linux 4k page size we are not able set hugepages, Aneesh had a look at the problem and he mentioned this commit is causing the issue. > > *Details:* > We are using pkvm ubuntu 16.04 guest with upstream kernel [4.8.0-rc1-00006-gbae9cc6] compiled with 4k page size > > o/p from guest: > HugePages_Total: 0 > HugePages_Free: 0 > HugePages_Rsvd: 0 > HugePages_Surp: 0 > Hugepagesize: 16384 kB > > Page sizes from device-tree: [dmesg] > [ 0.000000] base_shift=12: shift=12, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=0 > [ 0.000000] base_shift=12: shift=24, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=56 > [ 0.000000] base_shift=24: shift=24, sllp=0x0100, avpnm=0x00000001, tlbiel=0, penc=0 > > while trying to configure the hugepages inside the guest it throws the below error: > > echo 100 > /proc/sys/vm/nr_hugepages > -bash: echo: write error: Invalid argument > > *Note*: we do not see the problem when the linux page is 64k Just to reiterate you are seeing this problem using 4k page size and 16M as the hugepage size. With FORCE_MAX_ZONEORDER set to 9 to 13 for 4k pages, you can do upto 32M if FORCE_MAX_ZONEORDER is 13 and same for 64K with FORCE_MAX_ZONEORDER set to 9. Basically the constraint is FORCE_MAX_ZONEBITS <= 25 - PAGESHIFT What is your value of FORCE_MAX_ZONEORDER in the .config? Balbir Singh.
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index e4824fd04bb7..b933530821fb 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -585,9 +585,9 @@ endchoice config FORCE_MAX_ZONEORDER int "Maximum zone order" - range 9 64 if PPC64 && PPC_64K_PAGES + range 8 9 if PPC64 && PPC_64K_PAGES default "9" if PPC64 && PPC_64K_PAGES - range 13 64 if PPC64 && !PPC_64K_PAGES + range 9 13 if PPC64 && !PPC_64K_PAGES default "13" if PPC64 && !PPC_64K_PAGES range 9 64 if PPC32 && PPC_16K_PAGES default "9" if PPC32 && PPC_16K_PAGES
Currently on PPC64 changing kernel pagesize from 4K to 64K leaves FORCE_MAX_ZONEORDER set to 13 - which produces a compile error. The error occurs because of the following constraint (from include/linux/mmzone.h) being violated: MAX_ORDER -1 + PAGESHIFT <= SECTION_SIZE_BITS. Expanding this out, we get: FORCE_MAX_ZONEBITS <= 25 - PAGESHIFT, which requires, for a 64K page, FORCE_MAX_ZONEBITS <= 9. Thus set max value of FORCE_MAX_ZONEORDER for 64K pages to 9, and 4K pages to 13. Also, check the minimum value: In include/linux/huge_mm.h, we have the constraint HPAGE_PMD_ORDER < MAX_ORDER which expands out to: PTE_INDEX_SIZE < FORCE_MAX_ZONEORDER. PTE_INDEX_SIZE is: 9 (4k hash or no hash 4K pgtable) or 8 (64K hash or no hash 64K pgtable). Thus a min value of 8 for 64K pages and 9 for 4K pages is reasonable. So, update the range of FORCE_MAX_ZONEORDER from 9-64 to 8-9 for 64K pages and from 13-64 to 9-13 for 4K pages. Signed-off-by: Rashmica Gupta <rashmicy@gmail.com> --- v2: Changed the range for 4K pages and minimum for 64K pages as suggested by Balbir Singh. arch/powerpc/Kconfig | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)