| Message ID | 20160130012335.GA27534@fluid.dannf |
|---|---|
| State | New |
| Headers | show |
On Fri, Jan 29, 2016 at 06:23:35PM -0700, dann frazier wrote: > On NUMA implementations of Cavium ThunderX, node1 memory addresses start with > bit 40 set to 1, and therefore requires >= 41 bits of VA. > > Since we are using 4K pages, this also means moving from 3 to 4 level page > tables. The alternative would be increasing our page size. Upstream supports > 16K and 64K pages; 16K pages are not universally supported by ARM64 SoCs. > > 64K pages are pretty universal, but has issues of it's own: > https://bugs.launchpad.net/skipper/+bug/1522204 Oops - that should be: http://bugs.launchpad.net/bugs/1520162 -dann > https://lists.ubuntu.com/archives/ubuntu-devel/2014-December/038572.html > (and not an ideal change to introduce before an LTS) > > Signed-off-by: dann frazier <dann.frazier@canonical.com> > --- > debian.master/config/arm64/config.common.arm64 | 2 +- > debian.master/config/config.common.ubuntu | 6 +++--- > 2 files changed, 4 insertions(+), 4 deletions(-) > > diff --git a/debian.master/config/arm64/config.common.arm64 b/debian.master/config/arm64/config.common.arm64 > index 70ab3c0..e98bd48 100644 > --- a/debian.master/config/arm64/config.common.arm64 > +++ b/debian.master/config/arm64/config.common.arm64 > @@ -272,7 +272,7 @@ CONFIG_PCIPCWATCHDOG=m > CONFIG_PCI_PASID=y > CONFIG_PCI_PRI=y > CONFIG_PCI_QUIRKS=y > -CONFIG_PGTABLE_LEVELS=3 > +CONFIG_PGTABLE_LEVELS=4 > CONFIG_PGTABLE_MAPPING=y > CONFIG_PHANTOM=m > CONFIG_PHONET=m > diff --git a/debian.master/config/config.common.ubuntu b/debian.master/config/config.common.ubuntu > index a7de418..163e7af 100644 > --- a/debian.master/config/config.common.ubuntu > +++ b/debian.master/config/config.common.ubuntu > @@ -499,9 +499,9 @@ CONFIG_ARM64_HW_AFDBM=y > CONFIG_ARM64_PAN=y > # CONFIG_ARM64_PTDUMP is not set > # CONFIG_ARM64_RANDOMIZE_TEXT_OFFSET is not set > -CONFIG_ARM64_VA_BITS=39 > -CONFIG_ARM64_VA_BITS_39=y > -# CONFIG_ARM64_VA_BITS_48 is not set > +CONFIG_ARM64_VA_BITS=48 > +# CONFIG_ARM64_VA_BITS_39 is not set > +CONFIG_ARM64_VA_BITS_48=y > CONFIG_ARMADA375_USBCLUSTER_PHY=y > CONFIG_ARMADA_370_CLK=y > CONFIG_ARMADA_370_XP_TIMER=y
On 02/01/2016 03:31 PM, dann frazier wrote: > On Fri, Jan 29, 2016 at 06:23:35PM -0700, dann frazier wrote: >> On NUMA implementations of Cavium ThunderX, node1 memory addresses start with >> bit 40 set to 1, and therefore requires >= 41 bits of VA. >> >> Since we are using 4K pages, this also means moving from 3 to 4 level page >> tables. The alternative would be increasing our page size. Upstream supports >> 16K and 64K pages; 16K pages are not universally supported by ARM64 SoCs. >> >> 64K pages are pretty universal, but has issues of it's own: >> https://bugs.launchpad.net/skipper/+bug/1522204 > > Oops - that should be: http://bugs.launchpad.net/bugs/1520162 > That bug makes it sound like we might be opening a can of worms with 64K pages. Any thoughts on how much carnage this might cause ? Maybe we should wait until binutils has been adjusted so that 4K applications are 64K aligned ? rtg
On 02/02/2016 05:30 AM, Tim Gardner wrote: > On 02/01/2016 03:31 PM, dann frazier wrote: >> On Fri, Jan 29, 2016 at 06:23:35PM -0700, dann frazier wrote: >>> On NUMA implementations of Cavium ThunderX, node1 memory addresses start with >>> bit 40 set to 1, and therefore requires >= 41 bits of VA. >>> >>> Since we are using 4K pages, this also means moving from 3 to 4 level page >>> tables. The alternative would be increasing our page size. Upstream supports >>> 16K and 64K pages; 16K pages are not universally supported by ARM64 SoCs. >>> >>> 64K pages are pretty universal, but has issues of it's own: >>> https://bugs.launchpad.net/skipper/+bug/1522204 >> >> Oops - that should be: http://bugs.launchpad.net/bugs/1520162 >> > > That bug makes it sound like we might be opening a can of worms with 64K > pages. Any thoughts on how much carnage this might cause ? Maybe we > should wait until binutils has been adjusted so that 4K applications are > 64K aligned ? > > rtg > Never mind. Paolo explained to me what this patch actually does. Could you add an annotation in debian.master/config/annotations so that we remember why we've made this change ? rtg
On Tue, Feb 2, 2016 at 6:04 AM, Tim Gardner <tim.gardner@canonical.com> wrote: > On 02/02/2016 05:30 AM, Tim Gardner wrote: >> On 02/01/2016 03:31 PM, dann frazier wrote: >>> On Fri, Jan 29, 2016 at 06:23:35PM -0700, dann frazier wrote: >>>> On NUMA implementations of Cavium ThunderX, node1 memory addresses start with >>>> bit 40 set to 1, and therefore requires >= 41 bits of VA. >>>> >>>> Since we are using 4K pages, this also means moving from 3 to 4 level page >>>> tables. The alternative would be increasing our page size. Upstream supports >>>> 16K and 64K pages; 16K pages are not universally supported by ARM64 SoCs. >>>> >>>> 64K pages are pretty universal, but has issues of it's own: >>>> https://bugs.launchpad.net/skipper/+bug/1522204 >>> >>> Oops - that should be: http://bugs.launchpad.net/bugs/1520162 >>> >> >> That bug makes it sound like we might be opening a can of worms with 64K >> pages. Any thoughts on how much carnage this might cause ? Maybe we >> should wait until binutils has been adjusted so that 4K applications are >> 64K aligned ? >> >> rtg >> > > Never mind. Paolo explained to me what this patch actually does. Could > you add an annotation in debian.master/config/annotations so that we > remember why we've made this change ? Right, sorry - I was just trying to explain the other options we considered before choosing the 4K/4 level page table route. I'll follow up w/ a patch that clarifies the text and adds an annotation. -dann > rtg > > -- > Tim Gardner tim.gardner@canonical.com
diff --git a/debian.master/config/arm64/config.common.arm64 b/debian.master/config/arm64/config.common.arm64 index 70ab3c0..e98bd48 100644 --- a/debian.master/config/arm64/config.common.arm64 +++ b/debian.master/config/arm64/config.common.arm64 @@ -272,7 +272,7 @@ CONFIG_PCIPCWATCHDOG=m CONFIG_PCI_PASID=y CONFIG_PCI_PRI=y CONFIG_PCI_QUIRKS=y -CONFIG_PGTABLE_LEVELS=3 +CONFIG_PGTABLE_LEVELS=4 CONFIG_PGTABLE_MAPPING=y CONFIG_PHANTOM=m CONFIG_PHONET=m diff --git a/debian.master/config/config.common.ubuntu b/debian.master/config/config.common.ubuntu index a7de418..163e7af 100644 --- a/debian.master/config/config.common.ubuntu +++ b/debian.master/config/config.common.ubuntu @@ -499,9 +499,9 @@ CONFIG_ARM64_HW_AFDBM=y CONFIG_ARM64_PAN=y # CONFIG_ARM64_PTDUMP is not set # CONFIG_ARM64_RANDOMIZE_TEXT_OFFSET is not set -CONFIG_ARM64_VA_BITS=39 -CONFIG_ARM64_VA_BITS_39=y -# CONFIG_ARM64_VA_BITS_48 is not set +CONFIG_ARM64_VA_BITS=48 +# CONFIG_ARM64_VA_BITS_39 is not set +CONFIG_ARM64_VA_BITS_48=y CONFIG_ARMADA375_USBCLUSTER_PHY=y CONFIG_ARMADA_370_CLK=y CONFIG_ARMADA_370_XP_TIMER=y
On NUMA implementations of Cavium ThunderX, node1 memory addresses start with bit 40 set to 1, and therefore requires >= 41 bits of VA. Since we are using 4K pages, this also means moving from 3 to 4 level page tables. The alternative would be increasing our page size. Upstream supports 16K and 64K pages; 16K pages are not universally supported by ARM64 SoCs. 64K pages are pretty universal, but has issues of it's own: https://bugs.launchpad.net/skipper/+bug/1522204 https://lists.ubuntu.com/archives/ubuntu-devel/2014-December/038572.html (and not an ideal change to introduce before an LTS) Signed-off-by: dann frazier <dann.frazier@canonical.com> --- debian.master/config/arm64/config.common.arm64 | 2 +- debian.master/config/config.common.ubuntu | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-)