| Message ID | 20210127115606.28985-1-mdoucha@suse.cz |
|---|---|
| State | Rejected |
| Headers | show |
| Series | Fix tst_pollute_memory() safety margin for huge systems | expand |
Hi! Actually I've been thinking about this one and since this is a strictly best effort action I think we should be more aggressive about the safety margin. I also started to talk to kernel developers about this and generally about tests that try to allocate maximal amount of memory without triggering OOM since we have a few in the tree. I will send the results of this discussion once I have any here as well.
According my test,malloc amount greater than available memory,then invoke the oom-killer.
#man free
Estimation of how much memory is available for starting new applications, without swapping.
[root@liuxp mywork]# head /proc/meminfo
MemTotal: 198101744 kB
MemFree: 189303148 kB
MemAvailable: 188566732 kB
Set safety margin to larger value, may be not sensible.
From: Cyril Hrubis
Date: 2021-01-27 20:03
To: Martin Doucha
CC: liuxp11; ltp
Subject: Re: [LTP] [PATCH] Fix tst_pollute_memory() safety margin for huge systems
Hi!
Actually I've been thinking about this one and since this is a strictly
best effort action I think we should be more aggressive about the safety
margin.
I also started to talk to kernel developers about this and generally
about tests that try to allocate maximal amount of memory without
triggering OOM since we have a few in the tree. I will send the results
of this discussion once I have any here as well.
Hi!
> According my test,malloc amount greater than available memory,then invoke the oom-killer.
However that does not mean that allocating MemAvailable or slightly less
than MemAvailable is safe. In fact it's not, at least that I've been
told be kernel developers. I will try to figure out something safe
enough with their help.
OK,I also wanna know how to fix it. Thanks! From: Cyril Hrubis Date: 2021-01-27 20:28 To: liuxp11@chinatelecom.cn CC: mdoucha; ltp Subject: Re: Re: [LTP] [PATCH] Fix tst_pollute_memory() safety margin for huge systems Hi! > According my test,malloc amount greater than available memory,then invoke the oom-killer. However that does not mean that allocating MemAvailable or slightly less than MemAvailable is safe. In fact it's not, at least that I've been told be kernel developers. I will try to figure out something safe enough with their help.
On Wed, Jan 27, 2021 at 8:27 PM Cyril Hrubis <chrubis@suse.cz> wrote: > Hi! > > According my test,malloc amount greater than available memory,then > invoke the oom-killer. > > However that does not mean that allocating MemAvailable or slightly less > than MemAvailable is safe. In fact it's not, at least that I've been > told be kernel developers. I will try to figure out something safe > enough with their help. > If we only hope to avoid trigger OOM during allocating until the MAP_FAILED, I think the knob 'proc/sys/vm/overcommit_memory' maybe helpful. e.g. set overcommit_memory = 2, overcommit_ratio = 85 or 90; ... memory allocating as much as you can ... restore the value of overcommit_*
diff --git a/lib/tst_memutils.c b/lib/tst_memutils.c index dd09db490..7cdb3dbe0 100644 --- a/lib/tst_memutils.c +++ b/lib/tst_memutils.c @@ -20,8 +20,11 @@ void tst_pollute_memory(size_t maxsize, int fillchar) struct sysinfo info; SAFE_SYSINFO(&info); - safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); + SAFE_FILE_SCANF("/proc/sys/vm/min_free_kbytes", "%zd", &safety); + safety = MAX(2048 * safety, 128 * 1024 * 1024); + safety = MAX(safety, 4096 * SAFE_SYSCONF(_SC_PAGESIZE)); safety /= info.mem_unit; + safety = MAX(safety, info.freeram / 128); if (info.freeswap > safety) safety = 0;
tst_pollute_memory() still has OOM issues on system with huge amounts of RAM. Set safety margin to the largest value of: - 2 * min_free_kbytes - 128MB - 4096 pages - Free RAM / 128 (to account for memory allocation overhead) Signed-off-by: Martin Doucha <mdoucha@suse.cz> --- lib/tst_memutils.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)