From patchwork Sat Feb 21 09:19:07 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 442178 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 53EEE140182 for ; Sat, 21 Feb 2015 20:19:53 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751889AbbBUJTw (ORCPT ); Sat, 21 Feb 2015 04:19:52 -0500 Received: from mail.linuxfoundation.org ([140.211.169.12]:49641 "EHLO mail.linuxfoundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750749AbbBUJTs (ORCPT ); Sat, 21 Feb 2015 04:19:48 -0500 Received: from localhost (c-67-161-9-76.hsd1.ca.comcast.net [67.161.9.76]) by mail.linuxfoundation.org (Postfix) with ESMTPSA id 6BB848A1; Sat, 21 Feb 2015 09:19:47 +0000 (UTC) Date: Sat, 21 Feb 2015 01:19:07 -0800 From: Andrew Morton To: "Theodore Ts'o" Cc: Dave Chinner , Tetsuo Handa , hannes@cmpxchg.org, mhocko@suse.cz, dchinner@redhat.com, linux-mm@kvack.org, rientjes@google.com, oleg@redhat.com, mgorman@suse.de, torvalds@linux-foundation.org, xfs@oss.sgi.com, linux-ext4@vger.kernel.org Subject: Re: How to handle TIF_MEMDIE stalls? Message-Id: <20150221011907.2d26c979.akpm@linux-foundation.org> In-Reply-To: <20150221032000.GC7922@thunk.org> References: <201502172123.JIE35470.QOLMVOFJSHOFFt@I-love.SAKURA.ne.jp> <20150217125315.GA14287@phnom.home.cmpxchg.org> <20150217225430.GJ4251@dastard> <20150219102431.GA15569@phnom.home.cmpxchg.org> <20150219225217.GY12722@dastard> <201502201936.HBH34799.SOLFFFQtHOMOJV@I-love.SAKURA.ne.jp> <20150220231511.GH12722@dastard> <20150221032000.GC7922@thunk.org> X-Mailer: Sylpheed 2.7.1 (GTK+ 2.18.9; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org On Fri, 20 Feb 2015 22:20:00 -0500 "Theodore Ts'o" wrote: > +akpm I was hoping not to have to read this thread ;) afaict there are two (main) issues: a) whether to oom-kill when __GFP_FS is not set. The kernel hasn't been doing this for ages and nothing has changed recently. b) whether to keep looping when __GFP_NOFAIL is not set and __GFP_FS is not set and we can't oom-kill anything (which goes without saying, because __GFP_FS isn't set!). And 9879de7373fc ("mm: page_alloc: embed OOM killing naturally into allocation slowpath") somewhat inadvertently changed this policy - the allocation attempt will now promptly return ENOMEM if !__GFP_NOFAIL and !__GFP_FS. Correct enough? Question a) seems a bit of red herring and we can park it for now. What I'm not really understanding is why the pre-3.19 implementation actually worked. We've exhausted the free pages, we're not succeeding at reclaiming anything, we aren't able to oom-kill anyone. Yet it *does* work - we eventually find that memory and everything proceeds. How come? Where did that memory come from? Short term, we need to fix 3.19.x and 3.20 and that appears to be by applying Johannes's akpm-doesnt-know-why-it-works patch: Have people adequately confirmed that this gets us out of trouble? And yes, I agree that sites such as xfs's kmem_alloc() should be passing __GFP_NOFAIL to tell the page allocator what's going on. I don't think it matters a lot whether kmem_alloc() retains its retry loop. If __GFP_NOFAIL is working correctly then it will never loop anyway... Also, this: On Wed, 18 Feb 2015 09:54:30 +1100 Dave Chinner wrote: > Right now, the oom killer is a liability. Over the past 6 months > I've slowly had to exclude filesystem regression tests from running > on small memory machines because the OOM killer is now so unreliable > that it kills the test harness regularly rather than the process > generating memory pressure. David, I did not know this! If you've been telling us about this then perhaps it wasn't loud enough. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Acked-by: David Rientjes --- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2382,8 +2382,15 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order, if (high_zoneidx < ZONE_NORMAL) goto out; /* The OOM killer does not compensate for light reclaim */ - if (!(gfp_mask & __GFP_FS)) + if (!(gfp_mask & __GFP_FS)) { + /* + * XXX: Page reclaim didn't yield anything, + * and the OOM killer can't be invoked, but + * keep looping as per should_alloc_retry(). + */ + *did_some_progress = 1; goto out; + } /* * GFP_THISNODE contains __GFP_NORETRY and we never hit this. * Sanity check for bare calls of __GFP_THISNODE, not real OOM.