From patchwork Thu May 24 17:58:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 920048 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40sHZ45DgMz9ryk for ; Fri, 25 May 2018 04:13:32 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="BGjxx1Ai"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40sHZ43gr3zF1Q8 for ; Fri, 25 May 2018 04:13:32 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="BGjxx1Ai"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:400e:c00::244; helo=mail-pf0-x244.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="BGjxx1Ai"; dkim-atps=neutral Received: from mail-pf0-x244.google.com (mail-pf0-x244.google.com [IPv6:2607:f8b0:400e:c00::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40sHFk4ff6zF1QS for ; Fri, 25 May 2018 03:59:22 +1000 (AEST) Received: by mail-pf0-x244.google.com with SMTP id q22-v6so1257879pff.11 for ; Thu, 24 May 2018 10:59:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=+B2cPAVQCD9aE8L1VMjvQ8RjTXkhQbFtDIcYirz6jhQ=; b=BGjxx1AiWgl55AnlILSJjsm/ZQlroSWP4WJIK01BhrnqSSUucpj98jsOtMgmEOdVX4 aFlm9iswZYtIBa/gSN1FEJJcOF6Xy72AFHm9Ft6BtSXIaYqTik9cEs7iRQU2M/YcW2bk Maflw64vYZtXqbDr27IVSzIbJad7RFfC2KQZeuOGj93hB+R9AIVQa7+ODRjyuLosEu24 YfdmlyVWDQc77WBbM/ONYuZN438aV+c6dmaQZnZzHAMnDYb87Yac3Zk1qMxNGIa2KJfP X2FzuHVW0tl0aU94oTMCqABZa0OW0SHq1VvQKxS6V+6joNS4DxdmuAXM/FK5crTY7rop 3CVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=+B2cPAVQCD9aE8L1VMjvQ8RjTXkhQbFtDIcYirz6jhQ=; b=bYFHLU11RWGQRMl+CE4491l4D0z2w5bbNok8aJ8aj+AN+bFu2pqxGnrxFTq0i3/hKv iL0+bEb3400XPIGlBzZ0Zf2HVw8+A2oAeYz5NRBPJFV4yG3qVC41H1g3wr4FTab+IkZz 3JxODcBPUBdUHdAGDcFO0Rptz5pgQ34iFBW8VIO3yhY0ARnnwzXvk28keOik3U55KJ0Z LMse7p9g2p8FIp2PYq6RcWzRHQLiuWppOJ4X1ijZ+OqGv8BoOV+BWF57Ufw2UdlkO9uz uAXeqT816qsl++HMnCQANoCenw6F6RiiHDw2C3TTWh7dOkCPNV1XInUqGOBt9wB1EeMm nvkw== X-Gm-Message-State: ALKqPwcTJ0bOraZUc7mqhW4yONjA9gUJy0Ocdv1M2Jwfjae0vCKXhYD1 +OnszYoWS6PWFS7jv3HGsM5BeCMp X-Google-Smtp-Source: AB8JxZrpg9r4OnlP23+UU0f209jCFMvVLI5ogsFEemvIYT9aITqkdPXPxCWgdfGxG/V3izigfN6QVg== X-Received: by 2002:aa7:864d:: with SMTP id a13-v6mr8353893pfo.199.1527184760482; Thu, 24 May 2018 10:59:20 -0700 (PDT) Received: from roar.au.ibm.com ([1.128.253.12]) by smtp.gmail.com with ESMTPSA id v14-v6sm34833483pfn.105.2018.05.24.10.59.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 24 May 2018 10:59:19 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v3 5/7] powerpc/64s/radix: avoid ptesync after set_pte and ptep_set_access_flags Date: Fri, 25 May 2018 03:58:51 +1000 Message-Id: <20180524175853.19695-6-npiggin@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180524175853.19695-1-npiggin@gmail.com> References: <20180524175853.19695-1-npiggin@gmail.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" The ISA suggests ptesync after setting a pte, to prevent a table walk initiated by a subsequent access from missing that store and causing a spurious fault. This is an architectual allowance that allows an implementation's page table walker to be incoherent with the store queue. However there is no correctness problem in taking a spurious fault in userspace -- the kernel copes with these at any time, so the updated pte will be found eventually. Spurious kernel faults on vmap memory must be avoided, so a ptesync is put into flush_cache_vmap. On POWER9 so far I have not found a measurable window where this can result in more minor faults, so as an optimisation, remove the costly ptesync from pte updates. If an implementation benefits from ptesync, it would be better to add it back in update_mmu_cache, so it's not done for things like fork(2). fork --fork --exec benchmark improved 5.2% (12400->13100). Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/book3s/64/radix.h | 2 -- arch/powerpc/include/asm/cacheflush.h | 13 +++++++++++++ 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/include/asm/book3s/64/radix.h b/arch/powerpc/include/asm/book3s/64/radix.h index fcd92f9b6ec0..45bf1e1b1d33 100644 --- a/arch/powerpc/include/asm/book3s/64/radix.h +++ b/arch/powerpc/include/asm/book3s/64/radix.h @@ -209,7 +209,6 @@ static inline void radix__ptep_set_access_flags(struct mm_struct *mm, __radix_pte_update(ptep, 0, new_pte); } else __radix_pte_update(ptep, 0, set); - asm volatile("ptesync" : : : "memory"); } static inline int radix__pte_same(pte_t pte_a, pte_t pte_b) @@ -226,7 +225,6 @@ static inline void radix__set_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, int percpu) { *ptep = pte; - asm volatile("ptesync" : : : "memory"); } static inline int radix__pmd_bad(pmd_t pmd) diff --git a/arch/powerpc/include/asm/cacheflush.h b/arch/powerpc/include/asm/cacheflush.h index 11843e37d9cf..e9662648e72d 100644 --- a/arch/powerpc/include/asm/cacheflush.h +++ b/arch/powerpc/include/asm/cacheflush.h @@ -26,6 +26,19 @@ #define flush_cache_vmap(start, end) do { } while (0) #define flush_cache_vunmap(start, end) do { } while (0) +#ifdef CONFIG_BOOK3S_64 +/* + * Book3s has no ptesync after setting a pte, so without this ptesync it's + * possible for a kernel virtual mapping access to return a spurious fault + * if it's accessed right after the pte is set. The page fault handler does + * not expect this type of fault. flush_cache_vmap is not exactly the right + * place to put this, but it seems to work well enough. + */ +#define flush_cache_vmap(start, end) do { asm volatile("ptesync"); } while (0) +#else +#define flush_cache_vmap(start, end) do { } while (0) +#endif + #define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 1 extern void flush_dcache_page(struct page *page); #define flush_dcache_mmap_lock(mapping) do { } while (0)