{"id":811062,"url":"http://patchwork.ozlabs.org/api/patches/811062/?format=json","web_url":"http://patchwork.ozlabs.org/project/linuxppc-dev/patch/20170907145148.24398-4-npiggin@gmail.com/","project":{"id":2,"url":"http://patchwork.ozlabs.org/api/projects/2/?format=json","name":"Linux PPC development","link_name":"linuxppc-dev","list_id":"linuxppc-dev.lists.ozlabs.org","list_email":"linuxppc-dev@lists.ozlabs.org","web_url":"https://github.com/linuxppc/wiki/wiki","scm_url":"https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git","webscm_url":"https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/","list_archive_url":"https://lore.kernel.org/linuxppc-dev/","list_archive_url_format":"https://lore.kernel.org/linuxppc-dev/{}/","commit_url_format":"https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id={}"},"msgid":"<20170907145148.24398-4-npiggin@gmail.com>","list_archive_url":"https://lore.kernel.org/linuxppc-dev/20170907145148.24398-4-npiggin@gmail.com/","date":"2017-09-07T14:51:43","name":"[RFC,3/8] powerpc/64s/radix: optimize TLB range flush barriers","commit_ref":null,"pull_url":null,"state":"superseded","archived":true,"hash":"ce34af7c91b250a25c98f3b313d3da3874d18585","submitter":{"id":69518,"url":"http://patchwork.ozlabs.org/api/people/69518/?format=json","name":"Nicholas Piggin","email":"npiggin@gmail.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/linuxppc-dev/patch/20170907145148.24398-4-npiggin@gmail.com/mbox/","series":[{"id":2010,"url":"http://patchwork.ozlabs.org/api/series/2010/?format=json","web_url":"http://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=2010","date":"2017-09-07T14:51:40","name":"Further radix TLB flush optimisations","version":1,"mbox":"http://patchwork.ozlabs.org/series/2010/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/811062/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/811062/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xp3WW2DsTz9s81\n\tfor ;\n\tFri, 8 Sep 2017 00:59:19 +1000 (AEST)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3xp3WW14c6zDrYt\n\tfor ;\n\tFri, 8 Sep 2017 00:59:19 +1000 (AEST)","from mail-pg0-x244.google.com (mail-pg0-x244.google.com\n\t[IPv6:2607:f8b0:400e:c05::244])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3xp3MX6qLlzDrWX\n\tfor ;\n\tFri, 8 Sep 2017 00:52:24 +1000 (AEST)","by mail-pg0-x244.google.com with SMTP id v82so4426355pgb.1\n\tfor ;\n\tThu, 07 Sep 2017 07:52:24 -0700 (PDT)","from roar.au.ibm.com (203-219-56-202.tpgi.com.au. [203.219.56.202])\n\tby smtp.gmail.com with ESMTPSA id\n\ta6sm4642791pfa.76.2017.09.07.07.52.19\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tThu, 07 Sep 2017 07:52:22 -0700 (PDT)"],"Authentication-Results":["ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"DUJcPvvf\"; dkim-atps=neutral","lists.ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"DUJcPvvf\"; dkim-atps=neutral","ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=gmail.com\n\t(client-ip=2607:f8b0:400e:c05::244; helo=mail-pg0-x244.google.com;\n\tenvelope-from=npiggin@gmail.com; receiver=)","lists.ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"DUJcPvvf\"; dkim-atps=neutral"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=from:to:cc:subject:date:message-id:in-reply-to:references;\n\tbh=UDIibrf762Cv/CDNL2CMxskCyimmje6rG63nwqMD0cM=;\n\tb=DUJcPvvfIi+z/bhswggtfW25oJ8iip+b12WI+OblQSZsRN00ECbW++iuqqKpPo83L8\n\ts0Zzeh6T/cHxcYJuyXxn3B8UIaccvqFcyrgYgS7fRCHTGqEszD5fNqlTpxP+V97lUrsW\n\tFv1w9VMXCBZ5WicqLTUjV0RHms2gsgxlN9RUB6dpF1mtwVDjaiHgM1YullATrOaWZR6a\n\tAK11j6DWmoOtm9bIBF0HNMJUQSTNwTz9jLYPR2hiFe8ZG1OffiY4VChaAFmX58MLWtRV\n\t56zhpIFekuDGYnH2JxYr7/E/WpbxIrcOx61n2VSTYHo2Y5qdwlyjEZ1dFWNe+GnodB7b\n\twajg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to\n\t:references;\n\tbh=UDIibrf762Cv/CDNL2CMxskCyimmje6rG63nwqMD0cM=;\n\tb=e0DPCKT4RIV8F9d2CvpIOtRIT7LMwfyqiFeU4F7bS2MM0lYycMCIxnWrntj2gSXSbR\n\tS8yME58TKAk1qcmoQY+kAOzPF3ltZigU+U11Kv7bQe2QANfPBiDzyJKtiLHXamgvQY4z\n\tZUCuVBjtDYesmDZn2myViZeN6FRthg711aEPn3qqzFRTfXJDW/d0IKYDg7uqIUaYuaha\n\t+J6yh3dG8e7V/+hBnZRVSHmtIl9oKVNHJ9rPyFMJebSj4UoFPCXAEZtmcB9aGFxb92w8\n\toDAu7mYiYB93OBmpAGdbjQPwDSi4bihmOKn+oJDMjqLtTgnt2D01h/e382m0o6k9MtWY\n\tfMVw==","X-Gm-Message-State":"AHPjjUjk7zBy9YXMYJAL/gfS6pGm1UI+kJ6QuzQ/TZwfG6bP0q+eEAEG\n\tnsFE+CzdY/7wPP3e","X-Google-Smtp-Source":"ADKCNb5+915ru8ATJqdPREt1F8oNvuGSsUWO6cjlzgrAsiKraVHXDYaCd/5Kv5HcCHPjHHMHu1b1sA==","X-Received":"by 10.84.233.66 with SMTP id k2mr3241999plt.57.1504795942944;\n\tThu, 07 Sep 2017 07:52:22 -0700 (PDT)","From":"Nicholas Piggin ","To":"linuxppc-dev@lists.ozlabs.org","Subject":"[RFC PATCH 3/8] powerpc/64s/radix: optimize TLB range flush barriers","Date":"Fri, 8 Sep 2017 00:51:43 +1000","Message-Id":"<20170907145148.24398-4-npiggin@gmail.com>","X-Mailer":"git-send-email 2.13.3","In-Reply-To":"<20170907145148.24398-1-npiggin@gmail.com>","References":"<20170907145148.24398-1-npiggin@gmail.com>","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.23","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"Aneesh Kumar K . V\" ,\n\tNicholas Piggin , Anton Blanchard ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"},"content":"Short range flushes issue a sequences of tlbie(l) instructions for\nindividual effective addresses. These do not all require individual\nbarrier sequences, only one set around all instructions.\n\nCommit f7327e0ba3 (\"powerpc/mm/radix: Remove unnecessary ptesync\")\nmade a similar optimization for tlbiel for PID flushing.\n\nFor tlbie, the ISA says:\n\n The tlbsync instruction provides an ordering function for the\n effects of all tlbie instructions executed by the thread executing\n the tlbsync instruction, with respect to the memory barrier\n created by a subsequent ptesync instruction executed by the same\n thread.\n\nSigned-off-by: Nicholas Piggin \n---\n arch/powerpc/mm/tlb-radix.c | 41 ++++++++++++++++++++++++++++++++---------\n 1 file changed, 32 insertions(+), 9 deletions(-)","diff":"diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c\nindex 1ed61baf58da..c30f3faf5356 100644\n--- a/arch/powerpc/mm/tlb-radix.c\n+++ b/arch/powerpc/mm/tlb-radix.c\n@@ -84,7 +84,7 @@ static inline void _tlbie_pid(unsigned long pid, unsigned long ric)\n \ttrace_tlbie(0, 0, rb, rs, ric, prs, r);\n }\n \n-static inline void _tlbiel_va(unsigned long va, unsigned long pid,\n+static inline void __tlbiel_va(unsigned long va, unsigned long pid,\n \t\t\t unsigned long ap, unsigned long ric)\n {\n \tunsigned long rb,rs,prs,r;\n@@ -95,14 +95,20 @@ static inline void _tlbiel_va(unsigned long va, unsigned long pid,\n \tprs = 1; /* process scoped */\n \tr = 1; /* raidx format */\n \n-\tasm volatile(\"ptesync\": : :\"memory\");\n \tasm volatile(PPC_TLBIEL(%0, %4, %3, %2, %1)\n \t\t : : \"r\"(rb), \"i\"(r), \"i\"(prs), \"i\"(ric), \"r\"(rs) : \"memory\");\n-\tasm volatile(\"ptesync\": : :\"memory\");\n \ttrace_tlbie(0, 1, rb, rs, ric, prs, r);\n }\n \n-static inline void _tlbie_va(unsigned long va, unsigned long pid,\n+static inline void _tlbiel_va(unsigned long va, unsigned long pid,\n+\t\t\t unsigned long ap, unsigned long ric)\n+{\n+\tasm volatile(\"ptesync\": : :\"memory\");\n+\t__tlbiel_va(va, pid, ap, ric);\n+\tasm volatile(\"ptesync\": : :\"memory\");\n+}\n+\n+static inline void __tlbie_va(unsigned long va, unsigned long pid,\n \t\t\t unsigned long ap, unsigned long ric)\n {\n \tunsigned long rb,rs,prs,r;\n@@ -113,13 +119,20 @@ static inline void _tlbie_va(unsigned long va, unsigned long pid,\n \tprs = 1; /* process scoped */\n \tr = 1; /* raidx format */\n \n-\tasm volatile(\"ptesync\": : :\"memory\");\n \tasm volatile(PPC_TLBIE_5(%0, %4, %3, %2, %1)\n \t\t : : \"r\"(rb), \"i\"(r), \"i\"(prs), \"i\"(ric), \"r\"(rs) : \"memory\");\n-\tasm volatile(\"eieio; tlbsync; ptesync\": : :\"memory\");\n \ttrace_tlbie(0, 0, rb, rs, ric, prs, r);\n }\n \n+static inline void _tlbie_va(unsigned long va, unsigned long pid,\n+\t\t\t unsigned long ap, unsigned long ric)\n+{\n+\tasm volatile(\"ptesync\": : :\"memory\");\n+\t__tlbie_va(va, pid, ap, ric);\n+\tasm volatile(\"eieio; tlbsync; ptesync\": : :\"memory\");\n+}\n+\n+\n /*\n * Base TLB flushing operations:\n *\n@@ -335,14 +348,20 @@ void radix__flush_tlb_range_psize(struct mm_struct *mm, unsigned long start,\n \t\t\t_tlbiel_pid(pid, RIC_FLUSH_TLB);\n \t\telse\n \t\t\t_tlbie_pid(pid, RIC_FLUSH_TLB);\n+\n \t} else {\n+\t\tasm volatile(\"ptesync\": : :\"memory\");\n \t\tfor (addr = start; addr < end; addr += page_size) {\n \n \t\t\tif (local)\n-\t\t\t\t_tlbiel_va(addr, pid, ap, RIC_FLUSH_TLB);\n+\t\t\t\t__tlbiel_va(addr, pid, ap, RIC_FLUSH_TLB);\n \t\t\telse\n-\t\t\t\t_tlbie_va(addr, pid, ap, RIC_FLUSH_TLB);\n+\t\t\t\t__tlbie_va(addr, pid, ap, RIC_FLUSH_TLB);\n \t\t}\n+\t\tif (local)\n+\t\t\tasm volatile(\"ptesync\": : :\"memory\");\n+\t\telse\n+\t\t\tasm volatile(\"eieio; tlbsync; ptesync\": : :\"memory\");\n \t}\n \tpreempt_enable();\n }\n@@ -373,6 +392,7 @@ void radix__flush_tlb_collapsed_pmd(struct mm_struct *mm, unsigned long addr)\n \t\t_tlbie_pid(pid, RIC_FLUSH_PWC);\n \n \t/* Then iterate the pages */\n+\tasm volatile(\"ptesync\": : :\"memory\");\n \tend = addr + HPAGE_PMD_SIZE;\n \tfor (; addr < end; addr += PAGE_SIZE) {\n \t\tif (local)\n@@ -380,7 +400,10 @@ void radix__flush_tlb_collapsed_pmd(struct mm_struct *mm, unsigned long addr)\n \t\telse\n \t\t\t_tlbie_va(addr, pid, ap, RIC_FLUSH_TLB);\n \t}\n-\n+\tif (local)\n+\t\tasm volatile(\"ptesync\": : :\"memory\");\n+\telse\n+\t\tasm volatile(\"eieio; tlbsync; ptesync\": : :\"memory\");\n \tpreempt_enable();\n }\n #endif /* CONFIG_TRANSPARENT_HUGEPAGE */\n","prefixes":["RFC","3/8"]}