From patchwork Wed Jun 20 11:43:57 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 166044 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 0B550B6FB4 for ; Wed, 20 Jun 2012 21:50:42 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755609Ab2FTLuY (ORCPT ); Wed, 20 Jun 2012 07:50:24 -0400 Received: from cantor2.suse.de ([195.135.220.15]:40112 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753913Ab2FTLoR (ORCPT ); Wed, 20 Jun 2012 07:44:17 -0400 Received: from relay2.suse.de (unknown [195.135.220.254]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx2.suse.de (Postfix) with ESMTP id 9E93BA3CEE; Wed, 20 Jun 2012 13:44:16 +0200 (CEST) From: Mel Gorman To: Andrew Morton Cc: Linux-MM , Linux-Netdev , LKML , David Miller , Neil Brown , Peter Zijlstra , Mike Christie , Eric B Munson , Sebastian Andrzej Siewior , Mel Gorman Subject: [PATCH 02/17] mm: slub: Optimise the SLUB fast path to avoid pfmemalloc checks Date: Wed, 20 Jun 2012 12:43:57 +0100 Message-Id: <1340192652-31658-3-git-send-email-mgorman@suse.de> X-Mailer: git-send-email 1.7.9.2 In-Reply-To: <1340192652-31658-1-git-send-email-mgorman@suse.de> References: <1340192652-31658-1-git-send-email-mgorman@suse.de> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Christoph Lameter This patch removes the check for pfmemalloc from the alloc hotpath and puts the logic after the election of a new per cpu slab. For a pfmemalloc page we do not use the fast path but force the use of the slow path which is also used for the debug case. This has the side-effect of weakening pfmemalloc processing in the following way; 1. A process that is allocating for network swap calls __slab_alloc. pfmemalloc_match is true so the freelist is loaded and c->freelist is now pointing to a pfmemalloc page. 2. A process that is attempting normal allocations calls slab_alloc, finds the pfmemalloc page on the freelist and uses it because it did not check pfmemalloc_match() The patch allows non-pfmemalloc allocations to use pfmemalloc pages with the kmalloc slabs being the most vunerable caches on the grounds they are most likely to have a mix of pfmemalloc and !pfmemalloc requests. A later patch will still protect the system as processes will get throttled if the pfmemalloc reserves get depleted but performance will not degrade as smoothly. [mgorman@suse.de: Expanded changelog] Signed-off-by: Christoph Lameter Signed-off-by: Mel Gorman --- mm/slub.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 3cf24b4..dd13305 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2301,11 +2301,11 @@ new_slab: } } - if (likely(!kmem_cache_debug(s))) + if (likely(!kmem_cache_debug(s) && pfmemalloc_match(c, gfpflags))) goto load_freelist; /* Only entered in the debug case */ - if (!alloc_debug_processing(s, c->page, object, addr)) + if (kmem_cache_debug(s) && !alloc_debug_processing(s, c->page, object, addr)) goto new_slab; /* Slab failed checks. Next slab needed */ c->freelist = get_freepointer(s, object); @@ -2355,8 +2355,7 @@ redo: barrier(); object = c->freelist; - if (unlikely(!object || !node_match(c, node) || - !pfmemalloc_match(c, gfpflags))) + if (unlikely(!object || !node_match(c, node))) object = __slab_alloc(s, gfpflags, node, addr, c); else {