| Message ID | 1479160961-25840-3-git-send-email-arbab@linux.vnet.ibm.com (mailing list archive) |
|---|---|
| State | Awaiting Upstream, archived |
| Headers | show |
On 15/11/16 09:02, Reza Arbab wrote: > In commit c5320926e370 ("mem-hotplug: introduce movable_node boot > option"), the memblock allocation direction is changed to bottom-up and > then back to top-down like this: > > 1. memblock_set_bottom_up(true), called by cmdline_parse_movable_node(). > 2. memblock_set_bottom_up(false), called by x86's numa_init(). > > Even though (1) occurs in generic mm code, it is wrapped by #ifdef > CONFIG_MOVABLE_NODE, which depends on X86_64. > > This means that when we extend CONFIG_MOVABLE_NODE to non-x86 arches, > things will be unbalanced. (1) will happen for them, but (2) will not. > > This toggle was added in the first place because x86 has a delay between > adding memblocks and marking them as hotpluggable. Since other arches do > this marking either immediately or not at all, they do not require the > bottom-up toggle. > > So, resolve things by moving (1) from cmdline_parse_movable_node() to > x86's setup_arch(), immediately after the movable_node parameter has > been parsed. > > Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com> Acked-by: Balbir Singh <bsingharora@gmail.com>
Reza Arbab <arbab@linux.vnet.ibm.com> writes: > In commit c5320926e370 ("mem-hotplug: introduce movable_node boot > option"), the memblock allocation direction is changed to bottom-up and > then back to top-down like this: > > 1. memblock_set_bottom_up(true), called by cmdline_parse_movable_node(). > 2. memblock_set_bottom_up(false), called by x86's numa_init(). > > Even though (1) occurs in generic mm code, it is wrapped by #ifdef > CONFIG_MOVABLE_NODE, which depends on X86_64. > > This means that when we extend CONFIG_MOVABLE_NODE to non-x86 arches, > things will be unbalanced. (1) will happen for them, but (2) will not. > > This toggle was added in the first place because x86 has a delay between > adding memblocks and marking them as hotpluggable. Since other arches do > this marking either immediately or not at all, they do not require the > bottom-up toggle. > > So, resolve things by moving (1) from cmdline_parse_movable_node() to > x86's setup_arch(), immediately after the movable_node parameter has > been parsed. Considering that we now can mark memblock hotpluggable, do we need to enable the bottom up allocation for ppc64 also ? > > Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com> > --- > Documentation/kernel-parameters.txt | 2 +- > arch/x86/kernel/setup.c | 24 ++++++++++++++++++++++++ -aneesh
On Tue, Nov 15, 2016 at 12:35:42PM +0530, Aneesh Kumar K.V wrote: >Considering that we now can mark memblock hotpluggable, do we need to >enable the bottom up allocation for ppc64 also ? No, we don't, because early_init_dt_scan_memory() marks the memblocks hotpluggable immediately when they are added. There is no gap between the addition and the marking, as there is on x86, during which an allocation might accidentally occur in a movable node.
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index 37babf9..adcccd5 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -2401,7 +2401,7 @@ bytes respectively. Such letter suffixes can also be entirely omitted. that the amount of memory usable for all allocations is not too small. - movable_node [KNL,X86] Boot-time switch to enable the effects + movable_node [KNL] Boot-time switch to enable the effects of CONFIG_MOVABLE_NODE=y. See mm/Kconfig for details. MTD_Partition= [MTD] diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 9c337b0..4cfba94 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -985,6 +985,30 @@ void __init setup_arch(char **cmdline_p) parse_early_param(); +#ifdef CONFIG_MEMORY_HOTPLUG + /* + * Memory used by the kernel cannot be hot-removed because Linux + * cannot migrate the kernel pages. When memory hotplug is + * enabled, we should prevent memblock from allocating memory + * for the kernel. + * + * ACPI SRAT records all hotpluggable memory ranges. But before + * SRAT is parsed, we don't know about it. + * + * The kernel image is loaded into memory at very early time. We + * cannot prevent this anyway. So on NUMA system, we set any + * node the kernel resides in as un-hotpluggable. + * + * Since on modern servers, one node could have double-digit + * gigabytes memory, we can assume the memory around the kernel + * image is also un-hotpluggable. So before SRAT is parsed, just + * allocate memory near the kernel image to try the best to keep + * the kernel away from hotpluggable memory. + */ + if (movable_node_is_enabled()) + memblock_set_bottom_up(true); +#endif + x86_report_nx(); /* after early param, so could get panic from serial */ diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index cad4b91..e43142c1 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1727,26 +1727,6 @@ static bool can_offline_normal(struct zone *zone, unsigned long nr_pages) static int __init cmdline_parse_movable_node(char *p) { #ifdef CONFIG_MOVABLE_NODE - /* - * Memory used by the kernel cannot be hot-removed because Linux - * cannot migrate the kernel pages. When memory hotplug is - * enabled, we should prevent memblock from allocating memory - * for the kernel. - * - * ACPI SRAT records all hotpluggable memory ranges. But before - * SRAT is parsed, we don't know about it. - * - * The kernel image is loaded into memory at very early time. We - * cannot prevent this anyway. So on NUMA system, we set any - * node the kernel resides in as un-hotpluggable. - * - * Since on modern servers, one node could have double-digit - * gigabytes memory, we can assume the memory around the kernel - * image is also un-hotpluggable. So before SRAT is parsed, just - * allocate memory near the kernel image to try the best to keep - * the kernel away from hotpluggable memory. - */ - memblock_set_bottom_up(true); movable_node_enabled = true; #else pr_warn("movable_node option not supported\n");
In commit c5320926e370 ("mem-hotplug: introduce movable_node boot option"), the memblock allocation direction is changed to bottom-up and then back to top-down like this: 1. memblock_set_bottom_up(true), called by cmdline_parse_movable_node(). 2. memblock_set_bottom_up(false), called by x86's numa_init(). Even though (1) occurs in generic mm code, it is wrapped by #ifdef CONFIG_MOVABLE_NODE, which depends on X86_64. This means that when we extend CONFIG_MOVABLE_NODE to non-x86 arches, things will be unbalanced. (1) will happen for them, but (2) will not. This toggle was added in the first place because x86 has a delay between adding memblocks and marking them as hotpluggable. Since other arches do this marking either immediately or not at all, they do not require the bottom-up toggle. So, resolve things by moving (1) from cmdline_parse_movable_node() to x86's setup_arch(), immediately after the movable_node parameter has been parsed. Signed-off-by: Reza Arbab <arbab@linux.vnet.ibm.com> --- Documentation/kernel-parameters.txt | 2 +- arch/x86/kernel/setup.c | 24 ++++++++++++++++++++++++ mm/memory_hotplug.c | 20 -------------------- 3 files changed, 25 insertions(+), 21 deletions(-)