| Message ID | 1470330729-6273-2-git-send-email-srikar@linux.vnet.ibm.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
On Thu, 4 Aug 2016 22:42:09 +0530 Srikar Dronamraju <srikar@linux.vnet.ibm.com> wrote: > Fadump kernel reserves large chunks of memory even before the pages are > initialized. This could mean memory that corresponds to several nodes might > fall in memblock reserved regions. > > Kernels compiled with CONFIG_DEFERRED_STRUCT_PAGE_INIT will initialize > only certain size memory per node. The certain size takes into account > the dentry and inode cache sizes. Currently the cache sizes are > calculated based on the total system memory including the reserved > memory. However such a kernel when booting the same kernel as fadump > kernel will not be able to allocate the required amount of memory to > suffice for the dentry and inode caches. This results in crashes like > the below on large systems such as 32 TB systems. > > Dentry cache hash table entries: 536870912 (order: 16, 4294967296 bytes) > vmalloc: allocation failure, allocated 4097114112 of 17179934720 bytes > swapper/0: page allocation failure: order:0, mode:0x2080020(GFP_ATOMIC) > CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6-master+ #3 > Call Trace: > [c00000000108fb10] [c0000000007fac88] dump_stack+0xb0/0xf0 (unreliable) > [c00000000108fb50] [c000000000235264] warn_alloc_failed+0x114/0x160 > [c00000000108fbf0] [c000000000281484] __vmalloc_node_range+0x304/0x340 > [c00000000108fca0] [c00000000028152c] __vmalloc+0x6c/0x90 > [c00000000108fd40] [c000000000aecfb0] > alloc_large_system_hash+0x1b8/0x2c0 > [c00000000108fe00] [c000000000af7240] inode_init+0x94/0xe4 > [c00000000108fe80] [c000000000af6fec] vfs_caches_init+0x8c/0x13c > [c00000000108ff00] [c000000000ac4014] start_kernel+0x50c/0x578 > [c00000000108ff90] [c000000000008c6c] start_here_common+0x20/0xa8 > > Register the memory reserved by fadump, so that the cache sizes are > calculated based on the free memory (i.e Total memory - reserved > memory). Looks harmless enough to me. I'll schedule the patches for 4.8. But it sounds like they should be backported into older kernels?
* Andrew Morton <akpm@linux-foundation.org> [2016-08-04 14:01:33]: > > Register the memory reserved by fadump, so that the cache sizes are > > calculated based on the free memory (i.e Total memory - reserved > > memory). > > Looks harmless enough to me. I'll schedule the patches for 4.8. But > it sounds like they should be backported into older kernels? > Based on the v2 feedback, I just posted a v3 at http://lkml.kernel.org/r/1472476010-4709-1-git-send-email-srikar@linux.vnet.ibm.com that tries to reduce the large system hash based on tha reserved memory. Hence please drop the v2 patches.
diff --git a/arch/powerpc/kernel/fadump.c b/arch/powerpc/kernel/fadump.c index 3cb3b02a..ca5ec88 100644 --- a/arch/powerpc/kernel/fadump.c +++ b/arch/powerpc/kernel/fadump.c @@ -330,6 +330,7 @@ int __init fadump_reserve_mem(void) } fw_dump.reserve_dump_area_start = base; fw_dump.reserve_dump_area_size = size; + set_memory_reserve(size/PAGE_SIZE, true); return 1; }
Fadump kernel reserves large chunks of memory even before the pages are initialized. This could mean memory that corresponds to several nodes might fall in memblock reserved regions. Kernels compiled with CONFIG_DEFERRED_STRUCT_PAGE_INIT will initialize only certain size memory per node. The certain size takes into account the dentry and inode cache sizes. Currently the cache sizes are calculated based on the total system memory including the reserved memory. However such a kernel when booting the same kernel as fadump kernel will not be able to allocate the required amount of memory to suffice for the dentry and inode caches. This results in crashes like the below on large systems such as 32 TB systems. Dentry cache hash table entries: 536870912 (order: 16, 4294967296 bytes) vmalloc: allocation failure, allocated 4097114112 of 17179934720 bytes swapper/0: page allocation failure: order:0, mode:0x2080020(GFP_ATOMIC) CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.6-master+ #3 Call Trace: [c00000000108fb10] [c0000000007fac88] dump_stack+0xb0/0xf0 (unreliable) [c00000000108fb50] [c000000000235264] warn_alloc_failed+0x114/0x160 [c00000000108fbf0] [c000000000281484] __vmalloc_node_range+0x304/0x340 [c00000000108fca0] [c00000000028152c] __vmalloc+0x6c/0x90 [c00000000108fd40] [c000000000aecfb0] alloc_large_system_hash+0x1b8/0x2c0 [c00000000108fe00] [c000000000af7240] inode_init+0x94/0xe4 [c00000000108fe80] [c000000000af6fec] vfs_caches_init+0x8c/0x13c [c00000000108ff00] [c000000000ac4014] start_kernel+0x50c/0x578 [c00000000108ff90] [c000000000008c6c] start_here_common+0x20/0xa8 Register the memory reserved by fadump, so that the cache sizes are calculated based on the free memory (i.e Total memory - reserved memory). Suggested-by: Mel Gorman <mgorman@techsingularity.net> Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com> --- arch/powerpc/kernel/fadump.c | 1 + 1 file changed, 1 insertion(+)