| Message ID | 4D024CD1.3060407@canonical.com |
|---|---|
| State | Accepted |
| Headers | show |
On 12/10/2010 08:52 AM, Stefan Bader wrote: > On 12/10/2010 04:02 PM, Stefan Bader wrote: >> On 12/10/2010 03:52 PM, Tim Gardner wrote: >>> On 12/10/2010 07:10 AM, Stefan Bader wrote: >>>> SRU justification: >>>> >>>> Impact: The config option XEN_MAX_DOMAIN_MEMORY controls how much memory >>>> a Xen instance is seeing. The default for 64bit is 32GB, which is the >>>> reason that m2.4xlarge instances only report this amount of memory. >>>> >>>> Fix: Setting this limit to 70GB as there is a known restriction for >>>> t1.micro instances at about 80GB. >>>> >>>> Testcase: Booted m2.4xlarge with this set to 32GB shows 32GB of memory, >>>> when set to 70GB, it correctly reports 68GB. Booted a t1.micro as well >>>> to verify this has not caused problems there. >>>> >>>> From c72705539ee164cdd100570c1340887f81a63d98 Mon Sep 17 00:00:00 2001 >>>> From: Stefan Bader<stefan.bader@canonical.com> >>>> Date: Thu, 9 Dec 2010 17:59:36 +0100 >>>> Subject: [PATCH] UBUNTU: Set maximum of domain visible memory to 70G >>>> >>>> BugLink: http://bugs.launchpad.net/bugs/667796 >>>> >>>> Otherwise the largest instances will be stuck with 32G of memory. 70G >>>> was chosen as there are some known problems on small instances around >>>> 80GB and the largest instance is using 68GB. >>>> >>>> Signed-off-by: Stefan Bader<stefan.bader@canonical.com> >>>> --- >>>> debian.master/config/config.common.ubuntu | 2 +- >>>> 1 files changed, 1 insertions(+), 1 deletions(-) >>>> >>>> diff --git a/debian.master/config/config.common.ubuntu >>>> b/debian.master/config/config.common.ubuntu >>>> index b0c486b..ed8ad01 100644 >>>> --- a/debian.master/config/config.common.ubuntu >>>> +++ b/debian.master/config/config.common.ubuntu >>>> @@ -5014,7 +5014,7 @@ CONFIG_XEN_COMPAT_XENFS=y >>>> CONFIG_XEN_DEV_EVTCHN=m >>>> CONFIG_XEN_FBDEV_FRONTEND=m >>>> CONFIG_XEN_KBDDEV_FRONTEND=m >>>> -CONFIG_XEN_MAX_DOMAIN_MEMORY=32 >>>> +CONFIG_XEN_MAX_DOMAIN_MEMORY=70 >>>> CONFIG_XEN_SAVE_RESTORE=y >>>> CONFIG_XEN_SCRUB_PAGES=y >>>> CONFIG_XEN_SYS_HYPERVISOR=y >>> >>> Any thoughts on the impact this change will have on -generic and >>> -server? Perhaps we should restrict this change to the -ec2 flavour only. >>> >>> rtg >> >> As far as I understand it, this value is only used in the xen mmu code when >> setting up domU. The comment there says, it is in that case using a page per GB. >> But as we have the virtual flavour, it probably makes sense to use it. >> >> -Stefan >> > > Ok, so here the version 2 which only changes amd64-virtual > > -Stefan > Acked-by: Tim Gardner <tim.gardner@canonical.com>
On 12/10/2010 07:52 AM, Stefan Bader wrote: > On 12/10/2010 04:02 PM, Stefan Bader wrote: >> On 12/10/2010 03:52 PM, Tim Gardner wrote: >>> On 12/10/2010 07:10 AM, Stefan Bader wrote: >>>> SRU justification: >>>> >>>> Impact: The config option XEN_MAX_DOMAIN_MEMORY controls how much memory >>>> a Xen instance is seeing. The default for 64bit is 32GB, which is the >>>> reason that m2.4xlarge instances only report this amount of memory. >>>> >>>> Fix: Setting this limit to 70GB as there is a known restriction for >>>> t1.micro instances at about 80GB. >>>> >>>> Testcase: Booted m2.4xlarge with this set to 32GB shows 32GB of memory, >>>> when set to 70GB, it correctly reports 68GB. Booted a t1.micro as well >>>> to verify this has not caused problems there. >>>> >>>> From c72705539ee164cdd100570c1340887f81a63d98 Mon Sep 17 00:00:00 2001 >>>> From: Stefan Bader<stefan.bader@canonical.com> >>>> Date: Thu, 9 Dec 2010 17:59:36 +0100 >>>> Subject: [PATCH] UBUNTU: Set maximum of domain visible memory to 70G >>>> >>>> BugLink: http://bugs.launchpad.net/bugs/667796 >>>> >>>> Otherwise the largest instances will be stuck with 32G of memory. 70G >>>> was chosen as there are some known problems on small instances around >>>> 80GB and the largest instance is using 68GB. >>>> >>>> Signed-off-by: Stefan Bader<stefan.bader@canonical.com> >>>> --- >>>> debian.master/config/config.common.ubuntu | 2 +- >>>> 1 files changed, 1 insertions(+), 1 deletions(-) >>>> >>>> diff --git a/debian.master/config/config.common.ubuntu >>>> b/debian.master/config/config.common.ubuntu >>>> index b0c486b..ed8ad01 100644 >>>> --- a/debian.master/config/config.common.ubuntu >>>> +++ b/debian.master/config/config.common.ubuntu >>>> @@ -5014,7 +5014,7 @@ CONFIG_XEN_COMPAT_XENFS=y >>>> CONFIG_XEN_DEV_EVTCHN=m >>>> CONFIG_XEN_FBDEV_FRONTEND=m >>>> CONFIG_XEN_KBDDEV_FRONTEND=m >>>> -CONFIG_XEN_MAX_DOMAIN_MEMORY=32 >>>> +CONFIG_XEN_MAX_DOMAIN_MEMORY=70 >>>> CONFIG_XEN_SAVE_RESTORE=y >>>> CONFIG_XEN_SCRUB_PAGES=y >>>> CONFIG_XEN_SYS_HYPERVISOR=y >>> >>> Any thoughts on the impact this change will have on -generic and >>> -server? Perhaps we should restrict this change to the -ec2 flavour only. >>> >>> rtg >> >> As far as I understand it, this value is only used in the xen mmu code when >> setting up domU. The comment there says, it is in that case using a page per GB. >> But as we have the virtual flavour, it probably makes sense to use it. >> >> -Stefan >> > > Ok, so here the version 2 which only changes amd64-virtual > Acked-by: John Johansen <john.johansen@canonical.com>
On 12/10/2010 08:52 AM, Stefan Bader wrote: > On 12/10/2010 04:02 PM, Stefan Bader wrote: >> On 12/10/2010 03:52 PM, Tim Gardner wrote: >>> On 12/10/2010 07:10 AM, Stefan Bader wrote: >>>> SRU justification: >>>> >>>> Impact: The config option XEN_MAX_DOMAIN_MEMORY controls how much memory >>>> a Xen instance is seeing. The default for 64bit is 32GB, which is the >>>> reason that m2.4xlarge instances only report this amount of memory. >>>> >>>> Fix: Setting this limit to 70GB as there is a known restriction for >>>> t1.micro instances at about 80GB. >>>> >>>> Testcase: Booted m2.4xlarge with this set to 32GB shows 32GB of memory, >>>> when set to 70GB, it correctly reports 68GB. Booted a t1.micro as well >>>> to verify this has not caused problems there. >>>> >>>> From c72705539ee164cdd100570c1340887f81a63d98 Mon Sep 17 00:00:00 2001 >>>> From: Stefan Bader<stefan.bader@canonical.com> >>>> Date: Thu, 9 Dec 2010 17:59:36 +0100 >>>> Subject: [PATCH] UBUNTU: Set maximum of domain visible memory to 70G >>>> >>>> BugLink: http://bugs.launchpad.net/bugs/667796 >>>> >>>> Otherwise the largest instances will be stuck with 32G of memory. 70G >>>> was chosen as there are some known problems on small instances around >>>> 80GB and the largest instance is using 68GB. >>>> >>>> Signed-off-by: Stefan Bader<stefan.bader@canonical.com> >>>> --- >>>> debian.master/config/config.common.ubuntu | 2 +- >>>> 1 files changed, 1 insertions(+), 1 deletions(-) >>>> >>>> diff --git a/debian.master/config/config.common.ubuntu >>>> b/debian.master/config/config.common.ubuntu >>>> index b0c486b..ed8ad01 100644 >>>> --- a/debian.master/config/config.common.ubuntu >>>> +++ b/debian.master/config/config.common.ubuntu >>>> @@ -5014,7 +5014,7 @@ CONFIG_XEN_COMPAT_XENFS=y >>>> CONFIG_XEN_DEV_EVTCHN=m >>>> CONFIG_XEN_FBDEV_FRONTEND=m >>>> CONFIG_XEN_KBDDEV_FRONTEND=m >>>> -CONFIG_XEN_MAX_DOMAIN_MEMORY=32 >>>> +CONFIG_XEN_MAX_DOMAIN_MEMORY=70 >>>> CONFIG_XEN_SAVE_RESTORE=y >>>> CONFIG_XEN_SCRUB_PAGES=y >>>> CONFIG_XEN_SYS_HYPERVISOR=y >>> >>> Any thoughts on the impact this change will have on -generic and >>> -server? Perhaps we should restrict this change to the -ec2 flavour only. >>> >>> rtg >> >> As far as I understand it, this value is only used in the xen mmu code when >> setting up domU. The comment there says, it is in that case using a page per GB. >> But as we have the virtual flavour, it probably makes sense to use it. >> >> -Stefan >> > > Ok, so here the version 2 which only changes amd64-virtual > > -Stefan > Applied V2
On 12/10/2010 08:52 AM, Stefan Bader wrote: > On 12/10/2010 04:02 PM, Stefan Bader wrote: >> On 12/10/2010 03:52 PM, Tim Gardner wrote: >>> On 12/10/2010 07:10 AM, Stefan Bader wrote: >>>> SRU justification: >>>> >>>> Impact: The config option XEN_MAX_DOMAIN_MEMORY controls how much memory >>>> a Xen instance is seeing. The default for 64bit is 32GB, which is the >>>> reason that m2.4xlarge instances only report this amount of memory. >>>> >>>> Fix: Setting this limit to 70GB as there is a known restriction for >>>> t1.micro instances at about 80GB. >>>> >>>> Testcase: Booted m2.4xlarge with this set to 32GB shows 32GB of memory, >>>> when set to 70GB, it correctly reports 68GB. Booted a t1.micro as well >>>> to verify this has not caused problems there. >>>> >>>> From c72705539ee164cdd100570c1340887f81a63d98 Mon Sep 17 00:00:00 2001 >>>> From: Stefan Bader<stefan.bader@canonical.com> >>>> Date: Thu, 9 Dec 2010 17:59:36 +0100 >>>> Subject: [PATCH] UBUNTU: Set maximum of domain visible memory to 70G >>>> >>>> BugLink: http://bugs.launchpad.net/bugs/667796 >>>> >>>> Otherwise the largest instances will be stuck with 32G of memory. 70G >>>> was chosen as there are some known problems on small instances around >>>> 80GB and the largest instance is using 68GB. >>>> >>>> Signed-off-by: Stefan Bader<stefan.bader@canonical.com> >>>> --- >>>> debian.master/config/config.common.ubuntu | 2 +- >>>> 1 files changed, 1 insertions(+), 1 deletions(-) >>>> >>>> diff --git a/debian.master/config/config.common.ubuntu >>>> b/debian.master/config/config.common.ubuntu >>>> index b0c486b..ed8ad01 100644 >>>> --- a/debian.master/config/config.common.ubuntu >>>> +++ b/debian.master/config/config.common.ubuntu >>>> @@ -5014,7 +5014,7 @@ CONFIG_XEN_COMPAT_XENFS=y >>>> CONFIG_XEN_DEV_EVTCHN=m >>>> CONFIG_XEN_FBDEV_FRONTEND=m >>>> CONFIG_XEN_KBDDEV_FRONTEND=m >>>> -CONFIG_XEN_MAX_DOMAIN_MEMORY=32 >>>> +CONFIG_XEN_MAX_DOMAIN_MEMORY=70 >>>> CONFIG_XEN_SAVE_RESTORE=y >>>> CONFIG_XEN_SCRUB_PAGES=y >>>> CONFIG_XEN_SYS_HYPERVISOR=y >>> >>> Any thoughts on the impact this change will have on -generic and >>> -server? Perhaps we should restrict this change to the -ec2 flavour only. >>> >>> rtg >> >> As far as I understand it, this value is only used in the xen mmu code when >> setting up domU. The comment there says, it is in that case using a page per GB. >> But as we have the virtual flavour, it probably makes sense to use it. >> >> -Stefan >> > > Ok, so here the version 2 which only changes amd64-virtual > > -Stefan > So, whats the story for Natty where CONFIG_XEN_MAX_DOMAIN_MEMORY=128 ? Won't that preclude booting on t1.micro ?
On 12/11/2010 05:14 PM, Tim Gardner wrote: > On 12/10/2010 08:52 AM, Stefan Bader wrote: >> On 12/10/2010 04:02 PM, Stefan Bader wrote: >>> On 12/10/2010 03:52 PM, Tim Gardner wrote: >>>> On 12/10/2010 07:10 AM, Stefan Bader wrote: >>>>> SRU justification: >>>>> >>>>> Impact: The config option XEN_MAX_DOMAIN_MEMORY controls how much memory >>>>> a Xen instance is seeing. The default for 64bit is 32GB, which is the >>>>> reason that m2.4xlarge instances only report this amount of memory. >>>>> >>>>> Fix: Setting this limit to 70GB as there is a known restriction for >>>>> t1.micro instances at about 80GB. >>>>> >>>>> Testcase: Booted m2.4xlarge with this set to 32GB shows 32GB of memory, >>>>> when set to 70GB, it correctly reports 68GB. Booted a t1.micro as well >>>>> to verify this has not caused problems there. >>>>> >>>>> From c72705539ee164cdd100570c1340887f81a63d98 Mon Sep 17 00:00:00 2001 >>>>> From: Stefan Bader<stefan.bader@canonical.com> >>>>> Date: Thu, 9 Dec 2010 17:59:36 +0100 >>>>> Subject: [PATCH] UBUNTU: Set maximum of domain visible memory to 70G >>>>> >>>>> BugLink: http://bugs.launchpad.net/bugs/667796 >>>>> >>>>> Otherwise the largest instances will be stuck with 32G of memory. 70G >>>>> was chosen as there are some known problems on small instances around >>>>> 80GB and the largest instance is using 68GB. >>>>> >>>>> Signed-off-by: Stefan Bader<stefan.bader@canonical.com> >>>>> --- >>>>> debian.master/config/config.common.ubuntu | 2 +- >>>>> 1 files changed, 1 insertions(+), 1 deletions(-) >>>>> >>>>> diff --git a/debian.master/config/config.common.ubuntu >>>>> b/debian.master/config/config.common.ubuntu >>>>> index b0c486b..ed8ad01 100644 >>>>> --- a/debian.master/config/config.common.ubuntu >>>>> +++ b/debian.master/config/config.common.ubuntu >>>>> @@ -5014,7 +5014,7 @@ CONFIG_XEN_COMPAT_XENFS=y >>>>> CONFIG_XEN_DEV_EVTCHN=m >>>>> CONFIG_XEN_FBDEV_FRONTEND=m >>>>> CONFIG_XEN_KBDDEV_FRONTEND=m >>>>> -CONFIG_XEN_MAX_DOMAIN_MEMORY=32 >>>>> +CONFIG_XEN_MAX_DOMAIN_MEMORY=70 >>>>> CONFIG_XEN_SAVE_RESTORE=y >>>>> CONFIG_XEN_SCRUB_PAGES=y >>>>> CONFIG_XEN_SYS_HYPERVISOR=y >>>> >>>> Any thoughts on the impact this change will have on -generic and >>>> -server? Perhaps we should restrict this change to the -ec2 flavour only. >>>> >>>> rtg >>> >>> As far as I understand it, this value is only used in the xen mmu code when >>> setting up domU. The comment there says, it is in that case using a page per GB. >>> But as we have the virtual flavour, it probably makes sense to use it. >>> >>> -Stefan >>> >> >> Ok, so here the version 2 which only changes amd64-virtual >> >> -Stefan >> > > So, whats the story for Natty where CONFIG_XEN_MAX_DOMAIN_MEMORY=128 ? Won't > that preclude booting on t1.micro ? > That really was my first thought on seeing this. The longer story is, when upstream changed this from being 8/32 as a default, they also made it non-configurable and at the same time completely rewrote the mm code to make their pfn-to-memory structure use a tree. So in the end, yes this is probably the best hint to follow but no, forcing this to 70G does not resolve the problem (I tried). -Stefan
Natty no longer has this option as a tunable so this does not apply there. -apw
From 8cac300da1d56629ed55789dce9c654ff46a6dcc Mon Sep 17 00:00:00 2001 From: Stefan Bader <stefan.bader@canonical.com> Date: Thu, 9 Dec 2010 17:59:36 +0100 Subject: [PATCH v2] UBUNTU: Set maximum of domain visible memory to 70G BugLink: http://bugs.launchpad.net/bugs/667796 Otherwise the largest instances will be stuck with 32G of memory. Doing this only to the amd64 virtual flavour to keep cross effects to a minimum. Also, the i386 instances only come with much less memory. Signed-off-by: Stefan Bader <stefan.bader@canonical.com> --- debian.master/config/amd64/config.flavour.generic | 1 + debian.master/config/amd64/config.flavour.server | 1 + debian.master/config/amd64/config.flavour.virtual | 1 + debian.master/config/config.common.ubuntu | 1 - debian.master/config/i386/config.common.i386 | 1 + 5 files changed, 4 insertions(+), 1 deletions(-) diff --git a/debian.master/config/amd64/config.flavour.generic b/debian.master/config/amd64/config.flavour.generic index 0cda20a..4804006 100644 --- a/debian.master/config/amd64/config.flavour.generic +++ b/debian.master/config/amd64/config.flavour.generic @@ -16,4 +16,5 @@ CONFIG_VIRTIO_NET=m CONFIG_VIRTIO_PCI=m CONFIG_VIRTIO_RING=m CONFIG_XEN_BLKDEV_FRONTEND=m +CONFIG_XEN_MAX_DOMAIN_MEMORY=32 CONFIG_XEN_NETDEV_FRONTEND=m diff --git a/debian.master/config/amd64/config.flavour.server b/debian.master/config/amd64/config.flavour.server index 3fdffb1..b69832c 100644 --- a/debian.master/config/amd64/config.flavour.server +++ b/debian.master/config/amd64/config.flavour.server @@ -16,4 +16,5 @@ CONFIG_VIRTIO_NET=y CONFIG_VIRTIO_PCI=y CONFIG_VIRTIO_RING=y CONFIG_XEN_BLKDEV_FRONTEND=m +CONFIG_XEN_MAX_DOMAIN_MEMORY=32 CONFIG_XEN_NETDEV_FRONTEND=m diff --git a/debian.master/config/amd64/config.flavour.virtual b/debian.master/config/amd64/config.flavour.virtual index 97a6244..217ed9a 100644 --- a/debian.master/config/amd64/config.flavour.virtual +++ b/debian.master/config/amd64/config.flavour.virtual @@ -16,4 +16,5 @@ CONFIG_VIRTIO_NET=y CONFIG_VIRTIO_PCI=y CONFIG_VIRTIO_RING=y CONFIG_XEN_BLKDEV_FRONTEND=y +CONFIG_XEN_MAX_DOMAIN_MEMORY=70 CONFIG_XEN_NETDEV_FRONTEND=y diff --git a/debian.master/config/config.common.ubuntu b/debian.master/config/config.common.ubuntu index b0c486b..93a7fa5 100644 --- a/debian.master/config/config.common.ubuntu +++ b/debian.master/config/config.common.ubuntu @@ -5014,7 +5014,6 @@ CONFIG_XEN_COMPAT_XENFS=y CONFIG_XEN_DEV_EVTCHN=m CONFIG_XEN_FBDEV_FRONTEND=m CONFIG_XEN_KBDDEV_FRONTEND=m -CONFIG_XEN_MAX_DOMAIN_MEMORY=32 CONFIG_XEN_SAVE_RESTORE=y CONFIG_XEN_SCRUB_PAGES=y CONFIG_XEN_SYS_HYPERVISOR=y diff --git a/debian.master/config/i386/config.common.i386 b/debian.master/config/i386/config.common.i386 index 7d65433..41e88bb 100644 --- a/debian.master/config/i386/config.common.i386 +++ b/debian.master/config/i386/config.common.i386 @@ -294,5 +294,6 @@ CONFIG_X86_INTERNODE_CACHE_SHIFT=6 CONFIG_X86_MINIMUM_CPU_FAMILY=5 CONFIG_X86_SPEEDSTEP_LIB=y # CONFIG_X86_X2APIC is not set +CONFIG_XEN_MAX_DOMAIN_MEMORY=32 # CONFIG_ZONE_DMA32 is not set CONFIG_ZONE_DMA_FLAG=1 -- 1.7.0.4