From patchwork Fri Apr 28 06:13:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 756234 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3wDk6N0pc0z9s3w for ; Fri, 28 Apr 2017 16:14:04 +1000 (AEST) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3wDk6M3qQJzDqKl for ; Fri, 28 Apr 2017 16:14:03 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Received: from pegase1.c-s.fr (pegase1.c-s.fr [93.17.236.30]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3wDk5F41TXzDq7Z for ; Fri, 28 Apr 2017 16:13:05 +1000 (AEST) Received: from localhost (mailhub1-int [192.168.12.234]) by localhost (Postfix) with ESMTP id 3wDk4t3CvZz9ttBf; Fri, 28 Apr 2017 08:12:46 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [192.168.12.234]) (amavisd-new, port 10024) with ESMTP id igqUeT4JpwTm; Fri, 28 Apr 2017 08:12:46 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 3wDk4t2g8Kz9ttBL; Fri, 28 Apr 2017 08:12:46 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 6CE9D8B7AD; Fri, 28 Apr 2017 08:13:01 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id Ure32J3h31w9; Fri, 28 Apr 2017 08:13:01 +0200 (CEST) Received: from PO15451.localdomain (po15451.idsi0.si.c-s.fr [172.25.231.16]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 4617C8B795; Fri, 28 Apr 2017 08:13:01 +0200 (CEST) Received: by localhost.localdomain (Postfix, from userid 0) id 27B826E713; Fri, 28 Apr 2017 08:13:01 +0200 (CEST) From: Christophe Leroy Subject: [PATCH v2] powerpc/mm: Only read faulting instruction when necessary in do_page_fault() To: Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Scott Wood Message-Id: <20170428061301.27B826E713@localhost.localdomain> Date: Fri, 28 Apr 2017 08:13:01 +0200 (CEST) X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Commit a7a9dcd882a67 ("powerpc: Avoid taking a data miss on every userspace instruction miss") has shown that limiting the read of faulting instruction to likely cases improves performance. This patch goes further into this direction by limiting the read of the faulting instruction to the only cases where it is definitly needed. On an MPC885, with the same benchmark app as in the commit referred above, we see a reduction of 4000 dTLB misses (approx 3%): Before the patch: Performance counter stats for './fault 500' (10 runs): 720495838 cpu-cycles ( +- 0.04% ) 141769 dTLB-load-misses ( +- 0.02% ) 52722 iTLB-load-misses ( +- 0.01% ) 19611 faults ( +- 0.02% ) 5.750535176 seconds time elapsed ( +- 0.16% ) With the patch: Performance counter stats for './fault 500' (10 runs): 717669123 cpu-cycles ( +- 0.02% ) 137344 dTLB-load-misses ( +- 0.03% ) 52731 iTLB-load-misses ( +- 0.01% ) 19614 faults ( +- 0.03% ) 5.728423115 seconds time elapsed ( +- 0.14% ) Signed-off-by: Christophe Leroy --- v2: Changes 'if (cond1) if (cond2)' by 'if (cond1 && cond2)' In case the instruction we read has value 0, store_update_sp() will return false, so it will bail out. This patch applies after the serie "powerpc/mm: some cleanup of do_page_fault()" arch/powerpc/mm/fault.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 400f2d0d42f8..2ec82a279d28 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -280,14 +280,6 @@ int do_page_fault(struct pt_regs *regs, unsigned long address, perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address); - /* - * We want to do this outside mmap_sem, because reading code around nip - * can result in fault, which will cause a deadlock when called with - * mmap_sem held - */ - if (is_write && is_user) - __get_user(inst, (unsigned int __user *)regs->nip); - if (is_user) flags |= FAULT_FLAG_USER; @@ -356,8 +348,18 @@ int do_page_fault(struct pt_regs *regs, unsigned long address, * between the last mapped region and the stack will * expand the stack rather than segfaulting. */ - if (address + 2048 < uregs->gpr[1] && !store_updates_sp(inst)) - goto bad_area; + if (address + 2048 < uregs->gpr[1] && !inst) { + /* + * We want to do this outside mmap_sem, because reading + * code around nip can result in fault, which will cause + * a deadlock when called with mmap_sem held + */ + up_read(&mm->mmap_sem); + __get_user(inst, (unsigned int __user *)regs->nip); + if (!store_updates_sp(inst)) + goto bad_area_nosemaphore; + goto retry; + } } if (expand_stack(vma, address)) goto bad_area;