[{"id":1794612,"web_url":"http://patchwork.ozlabs.org/comment/1794612/","msgid":"<20171027043430.GA27483@fergus.ozlabs.ibm.com>","date":"2017-10-27T04:34:30","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":67079,"url":"http://patchwork.ozlabs.org/api/people/67079/","name":"Paul Mackerras","email":"paulus@ozlabs.org"},"content":"On Fri, Oct 27, 2017 at 09:38:17AM +0530, Aneesh Kumar K.V wrote:\n> Hi,\n> \n> With hash translation mode we always tracked the hash pte slot details in linux page table.\n> This occupied space in the linux page table and also limitted our ability to support\n> linux features that require additional PTE bits. This series attempt to lift this\n> limitation by not tracking slot number in linux page table. We still track slot details\n> w.r.t Transparent Hugepage entries because an invalidate there requires us to go through\n> all the 256 hash pte slots. So tracking whether hash page table entry is valid helps us in\n> avoiding a lot of hcalls there. With THP entries we don't keep slot details in the primary\n> linux page table entry but in the second half of page table. Hence tracking slot details\n> for THP doesn't take up space in PTE.\n> \n> Even though we don't track slot, for removing/updating hash page table entry, PAPR hcalls expect\n> hash page table slot details. On pseries we find slot using H_READ hcall using H_READ_4 flags.\n> This implies an additional 2 hcalls in the updatepp and remove paths. The patch series also\n> attempt to limit the impact of this by adding new hcalls that does remove/update of hash page table\n> entry using hash value instead of hash page table slot.\n> \n> Below is the performance numbers observed when running a workload that does the below sequence\n> \n> for(5000) {\n> mmap(128M)\n> touch every page of 2048 page\n> munmap()\n> }\n> \n> The test is run with address randomization off, swap disabled in both host and guest.\n> \n> \n> |------------+----------+---------------+--------------------------+-----------------------|\n> | iterations | platform | without patch | With series and no hcall | With series and hcall |\n> |------------+----------+---------------+--------------------------+-----------------------|\n> | 1 | powernv | | 50.818343 | |\n> | 2 | powernv | | 50.744123 | |\n> | 3 | powernv | | 50.721603 | |\n> | 4 | powernv | | 50.739922 | |\n> | 5 | powernv | | 50.638555 | |\n> | 1 | powernv | 51.388249 | | |\n> | 2 | powernv | 51.789701 | | |\n> | 3 | powernv | 52.240394 | | |\n> | 4 | powernv | 51.432255 | | |\n> | 5 | powernv | 51.392947 | | |\n> |------------+----------+---------------+--------------------------+-----------------------|\n> | 1 | pseries | | | 123.154394 |\n> | 2 | pseries | | | 122.253956 |\n> | 3 | pseries | | | 117.666344 |\n> | 4 | pseries | | | 117.681479 |\n> | 5 | pseries | | | 117.735808 |\n> | 1 | pseries | | 119.424940 | |\n> | 2 | pseries | | 117.663078 | |\n> | 3 | pseries | | 118.345584 | |\n> | 4 | pseries | | 119.620934 | |\n> | 5 | pseries | | 119.463185 | |\n> | 1 | pseries | 122.810867 | | |\n> | 2 | pseries | 115.760801 | | |\n> | 3 | pseries | 115.257030 | | |\n> | 4 | pseries | 116.617884 | | |\n> | 5 | pseries | 117.247036 | | |\n> |------------+----------+---------------+--------------------------+-----------------------|\n>\n\nHow do we interpret these numbers? Are they times, or speed? Is\nlarger better or worse?\n\nCan you give us the mean and standard deviation for each set of 5\nplease?\n\nPaul.","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yNWKJ6Yw0z9s0g\n\tfor ;\n\tFri, 27 Oct 2017 15:36:04 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yNWKJ5BwLzDshH\n\tfor ;\n\tFri, 27 Oct 2017 15:36:04 +1100 (AEDT)","from ozlabs.org (bilbo.ozlabs.org [103.22.144.67])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yNWHj6JR7zDqjm\n\tfor ;\n\tFri, 27 Oct 2017 15:34:41 +1100 (AEDT)","by ozlabs.org (Postfix, from userid 1003)\n\tid 3yNWHj4tCNz9t2d; Fri, 27 Oct 2017 15:34:41 +1100 (AEDT)"],"Authentication-Results":["ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"IMJKm1NP\";\n\tdkim-atps=neutral","lists.ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"IMJKm1NP\";\n\tdkim-atps=neutral","lists.ozlabs.org; dkim=pass (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"IMJKm1NP\"; \n\tdkim-atps=neutral"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ozlabs.org; s=201707; \n\tt=1509078881; bh=9Dootl2XaIb+kAwaTOj4bawJJ7AyRTpg96iqvmjsWvE=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=IMJKm1NPBA/cwp45WuOEV4sa0M6TcIjeW0N+gbmHaY4ZSPHaZT5EDFgcZ8g/V1Yx7\n\txJufi1JS91MPlszOFmz5Nu1YtuhW6DMQg5LSDUenwj6VKC0yB4GHSI7aydeafTGC/q\n\tEDabo02H5P9DWbCYw9Ku1te/4zF4i/oKgFTkKrv6Yvn4M57XbaW8WkAae91yKTbgMC\n\tnxO9TpgrftiCyY6LZlEO4oCeoHLhq5Zpzkis9IJicSnF2ZUHXUSJhFxHOOCMDc7+zA\n\t0EI9Sxy68xGm17FMORrCv4ujOF8GHOgR7o8kGdginaMuamo0Gk72YKO5ar2t6ThKJJ\n\tEexfIOMqdPsvA==","Date":"Fri, 27 Oct 2017 15:34:30 +1100","From":"Paul Mackerras ","To":"\"Aneesh Kumar K.V\" ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","Message-ID":"<20171027043430.GA27483@fergus.ozlabs.ibm.com>","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"linuxppc-dev@lists.ozlabs.org","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1794630,"web_url":"http://patchwork.ozlabs.org/comment/1794630/","msgid":"","date":"2017-10-27T05:27:13","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":664,"url":"http://patchwork.ozlabs.org/api/people/664/","name":"Aneesh Kumar K.V","email":"aneesh.kumar@linux.vnet.ibm.com"},"content":"On 10/27/2017 10:04 AM, Paul Mackerras wrote:\n> On Fri, Oct 27, 2017 at 09:38:17AM +0530, Aneesh Kumar K.V wrote:\n>> Hi,\n>>\n>> With hash translation mode we always tracked the hash pte slot details in linux page table.\n>> This occupied space in the linux page table and also limitted our ability to support\n>> linux features that require additional PTE bits. This series attempt to lift this\n>> limitation by not tracking slot number in linux page table. We still track slot details\n>> w.r.t Transparent Hugepage entries because an invalidate there requires us to go through\n>> all the 256 hash pte slots. So tracking whether hash page table entry is valid helps us in\n>> avoiding a lot of hcalls there. With THP entries we don't keep slot details in the primary\n>> linux page table entry but in the second half of page table. Hence tracking slot details\n>> for THP doesn't take up space in PTE.\n>>\n>> Even though we don't track slot, for removing/updating hash page table entry, PAPR hcalls expect\n>> hash page table slot details. On pseries we find slot using H_READ hcall using H_READ_4 flags.\n>> This implies an additional 2 hcalls in the updatepp and remove paths. The patch series also\n>> attempt to limit the impact of this by adding new hcalls that does remove/update of hash page table\n>> entry using hash value instead of hash page table slot.\n>>\n>> Below is the performance numbers observed when running a workload that does the below sequence\n>>\n>> for(5000) {\n>> mmap(128M)\n>> touch every page of 2048 page\n>> munmap()\n>> }\n>>\n>> The test is run with address randomization off, swap disabled in both host and guest.\n>>\n>>\n>> |------------+----------+---------------+--------------------------+-----------------------|\n>> | iterations | platform | without patch | With series and no hcall | With series and hcall |\n>> |------------+----------+---------------+--------------------------+-----------------------|\n>> | 1 | powernv | | 50.818343 | |\n>> | 2 | powernv | | 50.744123 | |\n>> | 3 | powernv | | 50.721603 | |\n>> | 4 | powernv | | 50.739922 | |\n>> | 5 | powernv | | 50.638555 | |\n>> | 1 | powernv | 51.388249 | | |\n>> | 2 | powernv | 51.789701 | | |\n>> | 3 | powernv | 52.240394 | | |\n>> | 4 | powernv | 51.432255 | | |\n>> | 5 | powernv | 51.392947 | | |\n>> |------------+----------+---------------+--------------------------+-----------------------|\n>> | 1 | pseries | | | 123.154394 |\n>> | 2 | pseries | | | 122.253956 |\n>> | 3 | pseries | | | 117.666344 |\n>> | 4 | pseries | | | 117.681479 |\n>> | 5 | pseries | | | 117.735808 |\n>> | 1 | pseries | | 119.424940 | |\n>> | 2 | pseries | | 117.663078 | |\n>> | 3 | pseries | | 118.345584 | |\n>> | 4 | pseries | | 119.620934 | |\n>> | 5 | pseries | | 119.463185 | |\n>> | 1 | pseries | 122.810867 | | |\n>> | 2 | pseries | 115.760801 | | |\n>> | 3 | pseries | 115.257030 | | |\n>> | 4 | pseries | 116.617884 | | |\n>> | 5 | pseries | 117.247036 | | |\n>> |------------+----------+---------------+--------------------------+-----------------------|\n>>\n> \n> How do we interpret these numbers? Are they times, or speed? Is\n> larger better or worse?\n\nSorry for not including the details. They are time in seconds. Test case \nis a modified mmap_bench included in powerpc/selftest.\n\n> \n> Can you give us the mean and standard deviation for each set of 5\n> please?\n> \n\npowernv without patch\nmedian= 51.432255\nstdev = 0.370835\n\nwith patch\nmedian = 50.739922\nstdev = 0.06419662\n\npseries without patch\nmedian = 116.617884\nstdev = 3.04531023\n\nwith patch no hcall\nmedian = 119.42494\nstdev = 0.85874552\n\nwith patch and hcall\nmedian = 117.735808\nstdev = 2.7624151\n\n-aneesh","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yNXTq0tWmz9t2h\n\tfor ;\n\tFri, 27 Oct 2017 16:28:31 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yNXTq00HLzDrj7\n\tfor ;\n\tFri, 27 Oct 2017 16:28:31 +1100 (AEDT)","from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com\n\t[148.163.158.5])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yNXSY6781zDqjm\n\tfor ;\n\tFri, 27 Oct 2017 16:27:25 +1100 (AEDT)","from pps.filterd (m0098413.ppops.net [127.0.0.1])\n\tby mx0b-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id\n\tv9R5NgOU070645\n\tfor ; Fri, 27 Oct 2017 01:27:23 -0400","from e38.co.us.ibm.com (e38.co.us.ibm.com [32.97.110.159])\n\tby mx0b-001b2d01.pphosted.com with ESMTP id 2duuqmyq50-1\n\t(version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT)\n\tfor ; Fri, 27 Oct 2017 01:27:22 -0400","from localhost\n\tby e38.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use\n\tOnly! Violators will be prosecuted\n\tfor from\n\t; Thu, 26 Oct 2017 23:27:21 -0600","from b03cxnp08025.gho.boulder.ibm.com (9.17.130.17)\n\tby e38.co.us.ibm.com (192.168.1.138) with IBM ESMTP SMTP Gateway:\n\tAuthorized Use Only! Violators will be prosecuted; \n\tThu, 26 Oct 2017 23:27:18 -0600","from b03ledav003.gho.boulder.ibm.com\n\t(b03ledav003.gho.boulder.ibm.com [9.17.130.234])\n\tby b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with\n\tESMTP id v9R5RH46786904; Thu, 26 Oct 2017 22:27:17 -0700","from b03ledav003.gho.boulder.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 7AD3A6A03F;\n\tThu, 26 Oct 2017 23:27:18 -0600 (MDT)","from [9.85.199.61] (unknown [9.85.199.61])\n\tby b03ledav003.gho.boulder.ibm.com (Postfix) with ESMTP id 4AE046A03C;\n\tThu, 26 Oct 2017 23:27:16 -0600 (MDT)"],"Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com\n\t(client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com;\n\tenvelope-from=aneesh.kumar@linux.vnet.ibm.com; receiver=)","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","To":"Paul Mackerras ","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>","From":"\"Aneesh Kumar K.V\" ","Date":"Fri, 27 Oct 2017 10:57:13 +0530","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20171027043430.GA27483@fergus.ozlabs.ibm.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-TM-AS-GCONF":"00","x-cbid":"17102705-0028-0000-0000-0000088F7CCA","X-IBM-SpamModules-Scores":"","X-IBM-SpamModules-Versions":"BY=3.00007959; HX=3.00000241; KW=3.00000007;\n\tPH=3.00000004; SC=3.00000239; SDB=6.00937060; UDB=6.00472261;\n\tIPR=6.00717323; \n\tBA=6.00005660; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009;\n\tZB=6.00000000; \n\tZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00017736;\n\tXFM=3.00000015; UTC=2017-10-27 05:27:20","X-IBM-AV-DETECTION":"SAVI=unused REMOTE=unused XFE=unused","x-cbparentid":"17102705-0029-0000-0000-0000381810EC","Message-Id":"","X-Proofpoint-Virus-Version":"vendor=fsecure engine=2.50.10432:, ,\n\tdefinitions=2017-10-27_02:, , signatures=0","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tpriorityscore=1501\n\tmalwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0\n\tclxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0\n\tclassifier=spam adjust=0 reason=mlx scancount=1\n\tengine=8.0.1-1707230000\n\tdefinitions=main-1710270071","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"linuxppc-dev@lists.ozlabs.org","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1794633,"web_url":"http://patchwork.ozlabs.org/comment/1794633/","msgid":"<20171027054136.GC27483@fergus.ozlabs.ibm.com>","date":"2017-10-27T05:41:36","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":67079,"url":"http://patchwork.ozlabs.org/api/people/67079/","name":"Paul Mackerras","email":"paulus@ozlabs.org"},"content":"On Fri, Oct 27, 2017 at 10:57:13AM +0530, Aneesh Kumar K.V wrote:\n> \n> \n> On 10/27/2017 10:04 AM, Paul Mackerras wrote:\n> >How do we interpret these numbers? Are they times, or speed? Is\n> >larger better or worse?\n> \n> Sorry for not including the details. They are time in seconds. Test case is\n> a modified mmap_bench included in powerpc/selftest.\n> \n> >\n> >Can you give us the mean and standard deviation for each set of 5\n> >please?\n> >\n> \n> powernv without patch\n> median= 51.432255\n> stdev = 0.370835\n> \n> with patch\n> median = 50.739922\n> stdev = 0.06419662\n> \n> pseries without patch\n> median = 116.617884\n> stdev = 3.04531023\n> \n> with patch no hcall\n> median = 119.42494\n> stdev = 0.85874552\n> \n> with patch and hcall\n> median = 117.735808\n> stdev = 2.7624151\n\nSo on powernv, the patch set *improves* performance by about 1.3%\n(almost 2 standard deviations). Do we know why that is?\n\nOn pseries, performance is about 2.4% worse without new hcalls, but\nthat is less than 1 standard deviation. With new hcalls, performance\nis 0.95% worse, only a third of a standard deviation. I think we need\nto do more measurements to try to get a more accurate picture here.\n\nWere the pseries numbers done on KVM or PowerVM? Could you do a set\nof measurements on the other one too please? (I assume the numbers\nwith the new hcall were done on KVM, and can't be done on PowerVM.)\n\nPaul.","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yNXpd32c1z9t30\n\tfor ;\n\tFri, 27 Oct 2017 16:43:05 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yNXpc6s7GzDrcd\n\tfor ;\n\tFri, 27 Oct 2017 16:43:04 +1100 (AEDT)","from ozlabs.org (bilbo.ozlabs.org [103.22.144.67])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yNXmz6k8dzDrbl\n\tfor ;\n\tFri, 27 Oct 2017 16:41:39 +1100 (AEDT)","by ozlabs.org (Postfix, from userid 1003)\n\tid 3yNXmz5pgQz9t2S; Fri, 27 Oct 2017 16:41:39 +1100 (AEDT)"],"Authentication-Results":["ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"J3yynToY\";\n\tdkim-atps=neutral","lists.ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"J3yynToY\";\n\tdkim-atps=neutral","lists.ozlabs.org; dkim=pass (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"J3yynToY\"; \n\tdkim-atps=neutral"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ozlabs.org; s=201707; \n\tt=1509082899; bh=QyalV3bUTsEQnvF1pflorQ+Dn3WiQ1L8/6cFw3/QUFg=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=J3yynToYkschJdGs+cqRvzMKbPFuhCPVO6JFYMORSe2dE9ka2hDAUi73d4jFthpuv\n\tfHeaQcNGsNTwA5ucIfNGtWIkURCV+xupM3+H6g+DUa97nR1QdPMAnNKk42sbJ2T9Rd\n\tej3oQ1NY0Dymg3A/PhOgDEuLMl9aL/A+AkGNEwvkdIsrGtacT7ebGF2dIi5yKwiElm\n\tnON6PNCUJjdqg6uK5HR3uh7UOuP8p9Lc7b6RHTLhV5C6PHz6ryM4Lbkt2TSuNQvXzB\n\tSCe0MBT2g+3MjldhOj72Hx0b9u2YZre2CVhT8PcyBu2/aJq1DtoDH/KTW9SRth5SHX\n\tCznvMtegZDHHw==","Date":"Fri, 27 Oct 2017 16:41:36 +1100","From":"Paul Mackerras ","To":"\"Aneesh Kumar K.V\" ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","Message-ID":"<20171027054136.GC27483@fergus.ozlabs.ibm.com>","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"linuxppc-dev@lists.ozlabs.org","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1795350,"web_url":"http://patchwork.ozlabs.org/comment/1795350/","msgid":"<20171028223532.GA5587@ram.oc3035372033.ibm.com>","date":"2017-10-28T22:35:32","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":2667,"url":"http://patchwork.ozlabs.org/api/people/2667/","name":"Ram Pai","email":"linuxram@us.ibm.com"},"content":"On Fri, Oct 27, 2017 at 10:57:13AM +0530, Aneesh Kumar K.V wrote:\n> \n> \n> On 10/27/2017 10:04 AM, Paul Mackerras wrote:\n> >On Fri, Oct 27, 2017 at 09:38:17AM +0530, Aneesh Kumar K.V wrote:\n> >>Hi,\n> >>\n> >>With hash translation mode we always tracked the hash pte slot details in linux page table.\n> >>This occupied space in the linux page table and also limitted our ability to support\n> >>linux features that require additional PTE bits. This series attempt to lift this\n> >>limitation by not tracking slot number in linux page table. We still track slot details\n> >>w.r.t Transparent Hugepage entries because an invalidate there requires us to go through\n> >>all the 256 hash pte slots. So tracking whether hash page table entry is valid helps us in\n> >>avoiding a lot of hcalls there. With THP entries we don't keep slot details in the primary\n> >>linux page table entry but in the second half of page table. Hence tracking slot details\n> >>for THP doesn't take up space in PTE.\n> >>\n> >>Even though we don't track slot, for removing/updating hash page table entry, PAPR hcalls expect\n> >>hash page table slot details. On pseries we find slot using H_READ hcall using H_READ_4 flags.\n> >>This implies an additional 2 hcalls in the updatepp and remove paths. The patch series also\n> >>attempt to limit the impact of this by adding new hcalls that does remove/update of hash page table\n> >>entry using hash value instead of hash page table slot.\n> >>\n> >>Below is the performance numbers observed when running a workload that does the below sequence\n> >>\n> >>for(5000) {\n> >>mmap(128M)\n> >>touch every page of 2048 page\n> >>munmap()\n> >>}\n> >>\n\nI like the idea of not tracking the slots at all. It is something the\nguest should not be knowing or tracking.\n\n> >>The test is run with address randomization off, swap disabled in both host and guest.\n> >>\n> >>\n> >>|------------+----------+---------------+--------------------------+-----------------------|\n> >>| iterations | platform | without patch | With series and no hcall | With series and hcall |\n> >>|------------+----------+---------------+--------------------------+-----------------------|\n> >>| 1 | powernv | | 50.818343 | |\n> >>| 2 | powernv | | 50.744123 | |\n> >>| 3 | powernv | | 50.721603 | |\n> >>| 4 | powernv | | 50.739922 | |\n> >>| 5 | powernv | | 50.638555 | |\n> >>| 1 | powernv | 51.388249 | | |\n> >>| 2 | powernv | 51.789701 | | |\n> >>| 3 | powernv | 52.240394 | | |\n> >>| 4 | powernv | 51.432255 | | |\n> >>| 5 | powernv | 51.392947 | | |\n> >>|------------+----------+---------------+--------------------------+-----------------------|\n> >>| 1 | pseries | | | 123.154394 |\n> >>| 2 | pseries | | | 122.253956 |\n> >>| 3 | pseries | | | 117.666344 |\n> >>| 4 | pseries | | | 117.681479 |\n> >>| 5 | pseries | | | 117.735808 |\n> >>| 1 | pseries | | 119.424940 | |\n> >>| 2 | pseries | | 117.663078 | |\n> >>| 3 | pseries | | 118.345584 | |\n> >>| 4 | pseries | | 119.620934 | |\n> >>| 5 | pseries | | 119.463185 | |\n> >>| 1 | pseries | 122.810867 | | |\n> >>| 2 | pseries | 115.760801 | | |\n> >>| 3 | pseries | 115.257030 | | |\n> >>| 4 | pseries | 116.617884 | | |\n> >>| 5 | pseries | 117.247036 | | |\n> >>|------------+----------+---------------+--------------------------+-----------------------|\n> >>\n> >\n\nWhat does 'With series and no hcall' mean? does it mean -- no calls to new hcalls,\n instead use H_READ_4 followed by old HCALLs?\n\nAnd I am assuming the code is not using any of my slot-move-to-secondary-pte changes.\n\nRP","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yPbFt2kFFz9t2Z\n\tfor ;\n\tSun, 29 Oct 2017 09:36:50 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yPbFt1nvgzDrnZ\n\tfor ;\n\tSun, 29 Oct 2017 09:36:50 +1100 (AEDT)","from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com\n\t[148.163.158.5])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yPbDb0pzNzDqs4\n\tfor ;\n\tSun, 29 Oct 2017 09:35:42 +1100 (AEDT)","from pps.filterd (m0098420.ppops.net [127.0.0.1])\n\tby mx0b-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id\n\tv9SMXiXh110028\n\tfor ; Sat, 28 Oct 2017 18:35:39 -0400","from e38.co.us.ibm.com (e38.co.us.ibm.com [32.97.110.159])\n\tby mx0b-001b2d01.pphosted.com with ESMTP id 2dvp3cqjvj-1\n\t(version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT)\n\tfor ; Sat, 28 Oct 2017 18:35:39 -0400","from localhost\n\tby e38.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use\n\tOnly! Violators will be prosecuted\n\tfor from ;\n\tSat, 28 Oct 2017 16:35:38 -0600","from b03cxnp08025.gho.boulder.ibm.com (9.17.130.17)\n\tby e38.co.us.ibm.com (192.168.1.138) with IBM ESMTP SMTP Gateway:\n\tAuthorized Use Only! Violators will be prosecuted; \n\tSat, 28 Oct 2017 16:35:36 -0600","from b03ledav004.gho.boulder.ibm.com\n\t(b03ledav004.gho.boulder.ibm.com [9.17.130.235])\n\tby b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with\n\tESMTP id v9SMZZVB4063612; Sat, 28 Oct 2017 15:35:35 -0700","from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id BD59B7803F;\n\tSat, 28 Oct 2017 16:35:35 -0600 (MDT)","from ram.oc3035372033.ibm.com (unknown [9.85.153.11])\n\tby b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTPS id\n\t1373E78037; Sat, 28 Oct 2017 16:35:34 -0600 (MDT)"],"Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=us.ibm.com\n\t(client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com;\n\tenvelope-from=linuxram@us.ibm.com; receiver=)","Date":"Sat, 28 Oct 2017 15:35:32 -0700","From":"Ram Pai ","To":"\"Aneesh Kumar K.V\" ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.5.20 (2009-12-10)","X-TM-AS-GCONF":"00","x-cbid":"17102822-0028-0000-0000-00000891692B","X-IBM-SpamModules-Scores":"","X-IBM-SpamModules-Versions":"BY=3.00007970; HX=3.00000241; KW=3.00000007;\n\tPH=3.00000004; SC=3.00000239; SDB=6.00937881; UDB=6.00472699;\n\tIPR=6.00718141; \n\tBA=6.00005662; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009;\n\tZB=6.00000000; \n\tZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00017769;\n\tXFM=3.00000015; UTC=2017-10-28 22:35:37","X-IBM-AV-DETECTION":"SAVI=unused REMOTE=unused XFE=unused","x-cbparentid":"17102822-0029-0000-0000-0000381EF509","Message-Id":"<20171028223532.GA5587@ram.oc3035372033.ibm.com>","X-Proofpoint-Virus-Version":"vendor=fsecure engine=2.50.10432:, ,\n\tdefinitions=2017-10-28_06:, , signatures=0","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tpriorityscore=1501\n\tmalwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0\n\tclxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0\n\tclassifier=spam adjust=0 reason=mlx scancount=1\n\tengine=8.0.1-1707230000\n\tdefinitions=main-1710280319","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"Ram Pai ","Cc":"linuxppc-dev@lists.ozlabs.org","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1795454,"web_url":"http://patchwork.ozlabs.org/comment/1795454/","msgid":"<87zi8a123g.fsf@linux.vnet.ibm.com>","date":"2017-10-29T14:05:23","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":664,"url":"http://patchwork.ozlabs.org/api/people/664/","name":"Aneesh Kumar K.V","email":"aneesh.kumar@linux.vnet.ibm.com"},"content":"Ram Pai writes:\n\n> On Fri, Oct 27, 2017 at 10:57:13AM +0530, Aneesh Kumar K.V wrote:\n>> \n>> \n> >>\n>> >\n>\n> What does 'With series and no hcall' mean? does it mean -- no calls to new hcalls,\n> instead use H_READ_4 followed by old HCALLs?\n\nThat is correct.\n\n>\n> And I am assuming the code is not using any of my slot-move-to-secondary-pte changes.\n>\n\nThat is correct.\n\n-aneesh","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yPztd6yRLz9t2M\n\tfor ;\n\tMon, 30 Oct 2017 01:06:33 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yPztd69VdzDrj7\n\tfor ;\n\tMon, 30 Oct 2017 01:06:33 +1100 (AEDT)","from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com\n\t[148.163.158.5])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yPzsT2cjqzDqkH\n\tfor ;\n\tMon, 30 Oct 2017 01:05:33 +1100 (AEDT)","from pps.filterd (m0098414.ppops.net [127.0.0.1])\n\tby mx0b-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id\n\tv9TE4S8n022594\n\tfor ; Sun, 29 Oct 2017 10:05:30 -0400","from e06smtp13.uk.ibm.com (e06smtp13.uk.ibm.com [195.75.94.109])\n\tby mx0b-001b2d01.pphosted.com with ESMTP id 2dwepqbggh-1\n\t(version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT)\n\tfor ; Sun, 29 Oct 2017 10:05:30 -0400","from localhost\n\tby e06smtp13.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use\n\tOnly! Violators will be prosecuted\n\tfor from\n\t; Sun, 29 Oct 2017 14:05:29 -0000","from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197)\n\tby e06smtp13.uk.ibm.com (192.168.101.143) with IBM ESMTP SMTP\n\tGateway: Authorized Use Only! Violators will be prosecuted; \n\tSun, 29 Oct 2017 14:05:28 -0000","from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com\n\t[9.149.105.232])\n\tby b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with\n\tESMTP id v9TE5RUJ29294598; Sun, 29 Oct 2017 14:05:27 GMT","from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 1CC815204B;\n\tSun, 29 Oct 2017 12:59:32 +0000 (GMT)","from skywalker (unknown [9.80.232.74])\n\tby d06av21.portsmouth.uk.ibm.com (Postfix) with SMTP id CB4CE52043;\n\tSun, 29 Oct 2017 12:59:30 +0000 (GMT)","(nullmailer pid 22045 invoked by uid 1000);\n\tSun, 29 Oct 2017 14:05:23 -0000"],"Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com\n\t(client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com;\n\tenvelope-from=aneesh.kumar@linux.vnet.ibm.com; receiver=)","From":"\"Aneesh Kumar K.V\" ","To":"Ram Pai ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","In-Reply-To":"<20171028223532.GA5587@ram.oc3035372033.ibm.com>","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>\n\t\n\t<20171028223532.GA5587@ram.oc3035372033.ibm.com>","Date":"Sun, 29 Oct 2017 19:35:23 +0530","MIME-Version":"1.0","Content-Type":"text/plain","X-TM-AS-GCONF":"00","x-cbid":"17102914-0012-0000-0000-0000058723A9","X-IBM-AV-DETECTION":"SAVI=unused REMOTE=unused XFE=unused","x-cbparentid":"17102914-0013-0000-0000-00001901A6EB","Message-Id":"<87zi8a123g.fsf@linux.vnet.ibm.com>","X-Proofpoint-Virus-Version":"vendor=fsecure engine=2.50.10432:, ,\n\tdefinitions=2017-10-29_08:, , signatures=0","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tpriorityscore=1501\n\tmalwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0\n\tclxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0\n\tclassifier=spam adjust=0 reason=mlx scancount=1\n\tengine=8.0.1-1707230000\n\tdefinitions=main-1710290201","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"linuxppc-dev@lists.ozlabs.org","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1795544,"web_url":"http://patchwork.ozlabs.org/comment/1795544/","msgid":"<20171029220417.GA3317@fergus.ozlabs.ibm.com>","date":"2017-10-29T22:04:17","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":67079,"url":"http://patchwork.ozlabs.org/api/people/67079/","name":"Paul Mackerras","email":"paulus@ozlabs.org"},"content":"On Sat, Oct 28, 2017 at 03:35:32PM -0700, Ram Pai wrote:\n> \n> I like the idea of not tracking the slots at all. It is something the\n> guest should not be knowing or tracking.\n\nWhy do you say that?\n\nPaul.","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yQBWk0j6wz9sQl\n\tfor ;\n\tMon, 30 Oct 2017 09:05:54 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yQBWj6cJXzDrd2\n\tfor ;\n\tMon, 30 Oct 2017 09:05:53 +1100 (AEDT)","from ozlabs.org (ozlabs.org [IPv6:2401:3900:2:1::2])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yQBTw04sKzDqlM\n\tfor ;\n\tMon, 30 Oct 2017 09:04:19 +1100 (AEDT)","by ozlabs.org (Postfix, from userid 1003)\n\tid 3yQBTv63GDz9t3l; Mon, 30 Oct 2017 09:04:19 +1100 (AEDT)"],"Authentication-Results":["ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"OSO+bz/b\";\n\tdkim-atps=neutral","lists.ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"OSO+bz/b\";\n\tdkim-atps=neutral","lists.ozlabs.org; dkim=pass (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"OSO+bz/b\"; \n\tdkim-atps=neutral"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ozlabs.org; s=201707; \n\tt=1509314659; bh=oAP5qp1tmFnD48G4KoDgk/uE188/MMtw7cYAQK+bZGE=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=OSO+bz/btvnl77lsU0PXNlwixGqu/2WF4tETAX3qiLgVGbbZu7DWUCc1sVFZtbx4j\n\trRvNj3rqiNU8eujP5Lbwlj6IvNr9Ib6pKtc2i4gEvCdUQPyOkOnot1lDv41NLtrfGU\n\twG1wi7ZTUifca4yAJlqbL+MdAUEEw6fYr+kBWNFhz1aE7sAf6+eHVZDKpzY+fo1UfI\n\tcv+AnYlxDMvUoIyldMVsBMwuEMOV+9+XAOWpH8/3Ehc39fWHsl/WQoWH0EnsZwXARB\n\tR9nvL6hwFioZbSl3h+kXBzM07HYL6MB3UOhDGlp2d+86XgnC5uBu+4ToFnafgn6lUg\n\tGeKJV9H8c6CVg==","Date":"Mon, 30 Oct 2017 09:04:17 +1100","From":"Paul Mackerras ","To":"Ram Pai ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","Message-ID":"<20171029220417.GA3317@fergus.ozlabs.ibm.com>","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>\n\t\n\t<20171028223532.GA5587@ram.oc3035372033.ibm.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20171028223532.GA5587@ram.oc3035372033.ibm.com>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"linuxppc-dev@lists.ozlabs.org,\n\t\"Aneesh Kumar K.V\" ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1795573,"web_url":"http://patchwork.ozlabs.org/comment/1795573/","msgid":"<20171030005106.GD5587@ram.oc3035372033.ibm.com>","date":"2017-10-30T00:51:06","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":2667,"url":"http://patchwork.ozlabs.org/api/people/2667/","name":"Ram Pai","email":"linuxram@us.ibm.com"},"content":"On Mon, Oct 30, 2017 at 09:04:17AM +1100, Paul Mackerras wrote:\n> On Sat, Oct 28, 2017 at 03:35:32PM -0700, Ram Pai wrote:\n> > \n> > I like the idea of not tracking the slots at all. It is something the\n> > guest should not be knowing or tracking.\n> \n> Why do you say that?\n\n'slot' is a internal mechanism used by the hash table to accelerate\nmapping an address to a hash-page. If the hash-table implementation\nchoose to use a different mechanism to accelerate the mapping, it can't\nbecause that mechanism is baked into the logic of all the consumers.\n\nRP","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yQGCm4mcbz9t2V\n\tfor ;\n\tMon, 30 Oct 2017 11:52:20 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yQGCm3lldzDrmy\n\tfor ;\n\tMon, 30 Oct 2017 11:52:20 +1100 (AEDT)","from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com\n\t[148.163.156.1])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yQGBd6G4JzDqlM\n\tfor ;\n\tMon, 30 Oct 2017 11:51:21 +1100 (AEDT)","from pps.filterd (m0098394.ppops.net [127.0.0.1])\n\tby mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id\n\tv9U0mk4J022869\n\tfor ; Sun, 29 Oct 2017 20:51:19 -0400","from e34.co.us.ibm.com (e34.co.us.ibm.com [32.97.110.152])\n\tby mx0a-001b2d01.pphosted.com with ESMTP id 2dwevc68ea-1\n\t(version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT)\n\tfor ; Sun, 29 Oct 2017 20:51:18 -0400","from localhost\n\tby e34.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use\n\tOnly! Violators will be prosecuted\n\tfor from ;\n\tSun, 29 Oct 2017 18:51:18 -0600","from b03cxnp08027.gho.boulder.ibm.com (9.17.130.19)\n\tby e34.co.us.ibm.com (192.168.1.134) with IBM ESMTP SMTP Gateway:\n\tAuthorized Use Only! Violators will be prosecuted; \n\tSun, 29 Oct 2017 18:51:15 -0600","from b03ledav005.gho.boulder.ibm.com\n\t(b03ledav005.gho.boulder.ibm.com [9.17.130.236])\n\tby b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with\n\tESMTP id v9U0pFNE64815198; Sun, 29 Oct 2017 17:51:15 -0700","from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 2452FBE03A;\n\tSun, 29 Oct 2017 18:51:15 -0600 (MDT)","from ram.oc3035372033.ibm.com (unknown [9.85.185.28])\n\tby b03ledav005.gho.boulder.ibm.com (Postfix) with ESMTPS id\n\t0E10BBE038; Sun, 29 Oct 2017 18:51:12 -0600 (MDT)"],"Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=us.ibm.com\n\t(client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com;\n\tenvelope-from=linuxram@us.ibm.com; receiver=)","Date":"Sun, 29 Oct 2017 17:51:06 -0700","From":"Ram Pai ","To":"Paul Mackerras ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>\n\t\n\t<20171028223532.GA5587@ram.oc3035372033.ibm.com>\n\t<20171029220417.GA3317@fergus.ozlabs.ibm.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20171029220417.GA3317@fergus.ozlabs.ibm.com>","User-Agent":"Mutt/1.5.20 (2009-12-10)","X-TM-AS-GCONF":"00","x-cbid":"17103000-0016-0000-0000-000007BA3018","X-IBM-SpamModules-Scores":"","X-IBM-SpamModules-Versions":"BY=3.00007976; HX=3.00000241; KW=3.00000007;\n\tPH=3.00000004; SC=3.00000239; SDB=6.00938385; UDB=6.00473001;\n\tIPR=6.00718647; \n\tBA=6.00005662; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009;\n\tZB=6.00000000; \n\tZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00017781;\n\tXFM=3.00000015; UTC=2017-10-30 00:51:16","X-IBM-AV-DETECTION":"SAVI=unused REMOTE=unused XFE=unused","x-cbparentid":"17103000-0017-0000-0000-00003C0CADBF","Message-Id":"<20171030005106.GD5587@ram.oc3035372033.ibm.com>","X-Proofpoint-Virus-Version":"vendor=fsecure engine=2.50.10432:, ,\n\tdefinitions=2017-10-29_12:, , signatures=0","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tpriorityscore=1501\n\tmalwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0\n\tclxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0\n\tclassifier=spam adjust=0 reason=mlx scancount=1\n\tengine=8.0.1-1707230000\n\tdefinitions=main-1710300009","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"Ram Pai ","Cc":"linuxppc-dev@lists.ozlabs.org,\n\t\"Aneesh Kumar K.V\" ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1795680,"web_url":"http://patchwork.ozlabs.org/comment/1795680/","msgid":"<87wp3d130t.fsf@linux.vnet.ibm.com>","date":"2017-10-30T07:57:38","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":664,"url":"http://patchwork.ozlabs.org/api/people/664/","name":"Aneesh Kumar K.V","email":"aneesh.kumar@linux.vnet.ibm.com"},"content":"Paul Mackerras writes:\n\n> On Fri, Oct 27, 2017 at 10:57:13AM +0530, Aneesh Kumar K.V wrote:\n>> \n>> \n>> On 10/27/2017 10:04 AM, Paul Mackerras wrote:\n>> >How do we interpret these numbers? Are they times, or speed? Is\n>> >larger better or worse?\n>> \n>> Sorry for not including the details. They are time in seconds. Test case is\n>> a modified mmap_bench included in powerpc/selftest.\n>> \n>> >\n>> >Can you give us the mean and standard deviation for each set of 5\n>> >please?\n>> >\n>> \n>> powernv without patch\n>> median= 51.432255\n>> stdev = 0.370835\n>> \n>> with patch\n>> median = 50.739922\n>> stdev = 0.06419662\n>> \n>> pseries without patch\n>> median = 116.617884\n>> stdev = 3.04531023\n>> \n>> with patch no hcall\n>> median = 119.42494\n>> stdev = 0.85874552\n>> \n>> with patch and hcall\n>> median = 117.735808\n>> stdev = 2.7624151\n>\n> So on powernv, the patch set *improves* performance by about 1.3%\n> (almost 2 standard deviations). Do we know why that is?\n\nI haven't looked at that closely. I was considering it within runtime\nvariance (no impact with patch series). I will get perf record collected\nand will see if that points to any details.\n\n>\n> On pseries, performance is about 2.4% worse without new hcalls, but\n> that is less than 1 standard deviation. With new hcalls, performance\n> is 0.95% worse, only a third of a standard deviation. I think we need\n> to do more measurements to try to get a more accurate picture here.\n>\n> Were the pseries numbers done on KVM or PowerVM? Could you do a set\n> of measurements on the other one too please? (I assume the numbers\n> with the new hcall were done on KVM, and can't be done on PowerVM.)\n>\n\n\nThe above pseries numbers were collected on KVM.\n\nPowerVM numbers on a different machine:\nWithout patch\n31.194165 \n31.372913 \n31.253494\n31.416198\n31.199180\nMEDIAN = 31.253494\nSTDEV = 0.1018900\n\nWith patch series\n31.538281\n31.385996\n31.492737\n31.452514\n31.259461\nMEDIAN = 31.452514\nSTDEV = 0.108511\n\n-aneesh","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yQRhW092tz9t2r\n\tfor ;\n\tMon, 30 Oct 2017 18:59:23 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yQRhV4j9WzDrcq\n\tfor ;\n\tMon, 30 Oct 2017 18:59:22 +1100 (AEDT)","from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com\n\t[148.163.156.1])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yQRfh1xnkzDqmt\n\tfor ;\n\tMon, 30 Oct 2017 18:57:47 +1100 (AEDT)","from pps.filterd (m0098393.ppops.net [127.0.0.1])\n\tby mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id\n\tv9U7vgKZ130468\n\tfor ; Mon, 30 Oct 2017 03:57:45 -0400","from e06smtp11.uk.ibm.com (e06smtp11.uk.ibm.com [195.75.94.107])\n\tby mx0a-001b2d01.pphosted.com with ESMTP id 2dwxpxkw6u-1\n\t(version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT)\n\tfor ; Mon, 30 Oct 2017 03:57:45 -0400","from localhost\n\tby e06smtp11.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use\n\tOnly! Violators will be prosecuted\n\tfor from\n\t; Mon, 30 Oct 2017 07:57:43 -0000","from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196)\n\tby e06smtp11.uk.ibm.com (192.168.101.141) with IBM ESMTP SMTP\n\tGateway: Authorized Use Only! Violators will be prosecuted; \n\tMon, 30 Oct 2017 07:57:41 -0000","from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com\n\t[9.149.105.58])\n\tby b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with\n\tESMTP id v9U7veU219923026; Mon, 30 Oct 2017 07:57:40 GMT","from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id BA6764C04E;\n\tMon, 30 Oct 2017 07:53:12 +0000 (GMT)","from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id F000C4C052;\n\tMon, 30 Oct 2017 07:53:11 +0000 (GMT)","from skywalker (unknown [9.124.35.86])\n\tby d06av22.portsmouth.uk.ibm.com (Postfix) with SMTP;\n\tMon, 30 Oct 2017 07:53:11 +0000 (GMT)","(nullmailer pid 8449 invoked by uid 1000);\n\tMon, 30 Oct 2017 07:57:38 -0000"],"Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com\n\t(client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com;\n\tenvelope-from=aneesh.kumar@linux.vnet.ibm.com; receiver=)","From":"\"Aneesh Kumar K.V\" ","To":"Paul Mackerras ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","In-Reply-To":"<20171027054136.GC27483@fergus.ozlabs.ibm.com>","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>\n\t\n\t<20171027054136.GC27483@fergus.ozlabs.ibm.com>","Date":"Mon, 30 Oct 2017 13:27:38 +0530","MIME-Version":"1.0","Content-Type":"text/plain","X-TM-AS-GCONF":"00","x-cbid":"17103007-0040-0000-0000-0000040853EA","X-IBM-AV-DETECTION":"SAVI=unused REMOTE=unused XFE=unused","x-cbparentid":"17103007-0041-0000-0000-000020AAD1C4","Message-Id":"<87wp3d130t.fsf@linux.vnet.ibm.com>","X-Proofpoint-Virus-Version":"vendor=fsecure engine=2.50.10432:, ,\n\tdefinitions=2017-10-30_02:, , signatures=0","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tpriorityscore=1501\n\tmalwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0\n\tclxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0\n\tclassifier=spam adjust=0 reason=mlx scancount=1\n\tengine=8.0.1-1707230000\n\tdefinitions=main-1710300112","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"linuxppc-dev@lists.ozlabs.org","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1795866,"web_url":"http://patchwork.ozlabs.org/comment/1795866/","msgid":"<87tvyg26us.fsf@linux.vnet.ibm.com>","date":"2017-10-30T11:49:31","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":664,"url":"http://patchwork.ozlabs.org/api/people/664/","name":"Aneesh Kumar K.V","email":"aneesh.kumar@linux.vnet.ibm.com"},"content":"Paul Mackerras writes:\n\n> On Fri, Oct 27, 2017 at 10:57:13AM +0530, Aneesh Kumar K.V wrote:\n>> \n>> \n>> On 10/27/2017 10:04 AM, Paul Mackerras wrote:\n>> >How do we interpret these numbers? Are they times, or speed? Is\n>> >larger better or worse?\n>> \n>> Sorry for not including the details. They are time in seconds. Test case is\n>> a modified mmap_bench included in powerpc/selftest.\n>> \n>> >\n>> >Can you give us the mean and standard deviation for each set of 5\n>> >please?\n>> >\n>> \n>> powernv without patch\n>> median= 51.432255\n>> stdev = 0.370835\n>> \n>> with patch\n>> median = 50.739922\n>> stdev = 0.06419662\n>> \n>> pseries without patch\n>> median = 116.617884\n>> stdev = 3.04531023\n>> \n>> with patch no hcall\n>> median = 119.42494\n>> stdev = 0.85874552\n>> \n>> with patch and hcall\n>> median = 117.735808\n>> stdev = 2.7624151\n>\n> So on powernv, the patch set *improves* performance by about 1.3%\n> (almost 2 standard deviations). Do we know why that is?\n>\n\nI looked at the perf data and with the test, we are doing larger number\nof hash faults and then around 10k flush_hash_range. Can the small\nimprovement in number be due to the fact that we are not storing slot\nnumber when doing an insert now?. Also in the flush path we are now not\nusing real_pte_t.\n\naneesh","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yQXrj5Qjbz9s81\n\tfor ;\n\tMon, 30 Oct 2017 22:51:49 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yQXrj4Ns8zDrcv\n\tfor ;\n\tMon, 30 Oct 2017 22:51:49 +1100 (AEDT)","from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com\n\t[148.163.158.5])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yQXpL1CZqzDqmt\n\tfor ;\n\tMon, 30 Oct 2017 22:49:42 +1100 (AEDT)","from pps.filterd (m0098413.ppops.net [127.0.0.1])\n\tby mx0b-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id\n\tv9UBj8Hp097751\n\tfor ; Mon, 30 Oct 2017 07:49:39 -0400","from e06smtp13.uk.ibm.com (e06smtp13.uk.ibm.com [195.75.94.109])\n\tby mx0b-001b2d01.pphosted.com with ESMTP id 2dx0vsredd-1\n\t(version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT)\n\tfor ; Mon, 30 Oct 2017 07:49:39 -0400","from localhost\n\tby e06smtp13.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use\n\tOnly! Violators will be prosecuted\n\tfor from\n\t; Mon, 30 Oct 2017 11:49:38 -0000","from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195)\n\tby e06smtp13.uk.ibm.com (192.168.101.143) with IBM ESMTP SMTP\n\tGateway: Authorized Use Only! Violators will be prosecuted; \n\tMon, 30 Oct 2017 11:49:36 -0000","from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com\n\t[9.149.105.59])\n\tby b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with\n\tESMTP id v9UBnZ3B31064122; Mon, 30 Oct 2017 11:49:35 GMT","from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 49426A405D;\n\tMon, 30 Oct 2017 11:44:45 +0000 (GMT)","from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 6F664A4040;\n\tMon, 30 Oct 2017 11:44:43 +0000 (GMT)","from skywalker (unknown [9.85.185.102])\n\tby d06av23.portsmouth.uk.ibm.com (Postfix) with SMTP;\n\tMon, 30 Oct 2017 11:44:43 +0000 (GMT)","(nullmailer pid 15197 invoked by uid 1000);\n\tMon, 30 Oct 2017 11:49:31 -0000"],"Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com\n\t(client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com;\n\tenvelope-from=aneesh.kumar@linux.vnet.ibm.com; receiver=)","From":"\"Aneesh Kumar K.V\" ","To":"Paul Mackerras ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","In-Reply-To":"<20171027054136.GC27483@fergus.ozlabs.ibm.com>","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>\n\t\n\t<20171027054136.GC27483@fergus.ozlabs.ibm.com>","Date":"Mon, 30 Oct 2017 17:19:31 +0530","MIME-Version":"1.0","Content-Type":"text/plain","X-TM-AS-GCONF":"00","x-cbid":"17103011-0012-0000-0000-0000058765FE","X-IBM-AV-DETECTION":"SAVI=unused REMOTE=unused XFE=unused","x-cbparentid":"17103011-0013-0000-0000-00001901EBBA","Message-Id":"<87tvyg26us.fsf@linux.vnet.ibm.com>","X-Proofpoint-Virus-Version":"vendor=fsecure engine=2.50.10432:, ,\n\tdefinitions=2017-10-30_03:, , signatures=0","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tpriorityscore=1501\n\tmalwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0\n\tclxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0\n\tclassifier=spam adjust=0 reason=mlx scancount=1\n\tengine=8.0.1-1707230000\n\tdefinitions=main-1710300164","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"linuxppc-dev@lists.ozlabs.org","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1795920,"web_url":"http://patchwork.ozlabs.org/comment/1795920/","msgid":"<87r2tk22wo.fsf@linux.vnet.ibm.com>","date":"2017-10-30T13:14:47","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":664,"url":"http://patchwork.ozlabs.org/api/people/664/","name":"Aneesh Kumar K.V","email":"aneesh.kumar@linux.vnet.ibm.com"},"content":"\"Aneesh Kumar K.V\" writes:\n\n\n> I looked at the perf data and with the test, we are doing larger number\n> of hash faults and then around 10k flush_hash_range. Can the small\n> improvement in number be due to the fact that we are not storing slot\n> number when doing an insert now?. Also in the flush path we are now not\n> using real_pte_t.\n>\n\nWith THP disabled I am finding below.\n\nWithout patch\n\n 35.62% a.out [kernel.vmlinux] [k] clear_user_page\n 8.54% a.out [kernel.vmlinux] [k] __lock_acquire\n 3.86% a.out [kernel.vmlinux] [k] native_flush_hash_range\n 3.38% a.out [kernel.vmlinux] [k] save_context_stack\n 2.98% a.out a.out [.] main\n 2.59% a.out [kernel.vmlinux] [k] lock_acquire\n 2.29% a.out [kernel.vmlinux] [k] mark_lock\n 2.23% a.out [kernel.vmlinux] [k] native_hpte_insert\n 1.87% a.out [kernel.vmlinux] [k] get_mem_cgroup_from_mm\n 1.71% a.out [kernel.vmlinux] [k] rcu_lockdep_current_cpu_online\n 1.68% a.out [kernel.vmlinux] [k] lock_release\n 1.47% a.out [kernel.vmlinux] [k] __handle_mm_fault\n 1.41% a.out [kernel.vmlinux] [k] validate_sp\n\n\nWith patch\n 35.40% a.out [kernel.vmlinux] [k] clear_user_page\n 8.82% a.out [kernel.vmlinux] [k] __lock_acquire\n 3.66% a.out a.out [.] main\n 3.49% a.out [kernel.vmlinux] [k] save_context_stack\n 2.77% a.out [kernel.vmlinux] [k] lock_acquire\n 2.45% a.out [kernel.vmlinux] [k] mark_lock\n 1.80% a.out [kernel.vmlinux] [k] get_mem_cgroup_from_mm\n 1.80% a.out [kernel.vmlinux] [k] native_hpte_insert\n 1.79% a.out [kernel.vmlinux] [k] rcu_lockdep_current_cpu_online\n 1.78% a.out [kernel.vmlinux] [k] lock_release\n 1.73% a.out [kernel.vmlinux] [k] native_flush_hash_range\n 1.53% a.out [kernel.vmlinux] [k] __handle_mm_fault\n\nThat is we are now spending less time in native_flush_hash_range.\n\n-aneesh","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yQZkg1kMVz9s3T\n\tfor ;\n\tTue, 31 Oct 2017 00:16:43 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yQZkg0v4zzDrqK\n\tfor ;\n\tTue, 31 Oct 2017 00:16:43 +1100 (AEDT)","from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com\n\t[148.163.156.1])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yQZjT2VRGzDqmt\n\tfor ;\n\tTue, 31 Oct 2017 00:15:41 +1100 (AEDT)","from pps.filterd (m0098394.ppops.net [127.0.0.1])\n\tby mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id\n\tv9UDFA5m095446\n\tfor ; Mon, 30 Oct 2017 09:15:39 -0400","from e06smtp12.uk.ibm.com (e06smtp12.uk.ibm.com [195.75.94.108])\n\tby mx0a-001b2d01.pphosted.com with ESMTP id 2dx285gh6c-1\n\t(version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT)\n\tfor ; Mon, 30 Oct 2017 09:15:37 -0400","from localhost\n\tby e06smtp12.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use\n\tOnly! Violators will be prosecuted\n\tfor from\n\t; Mon, 30 Oct 2017 13:14:53 -0000","from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195)\n\tby e06smtp12.uk.ibm.com (192.168.101.142) with IBM ESMTP SMTP\n\tGateway: Authorized Use Only! Violators will be prosecuted; \n\tMon, 30 Oct 2017 13:14:51 -0000","from d06av24.portsmouth.uk.ibm.com (mk.ibm.com [9.149.105.60])\n\tby b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with\n\tESMTP id v9UDEpf018481264; Mon, 30 Oct 2017 13:14:51 GMT","from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 6E2C14204C;\n\tMon, 30 Oct 2017 13:10:07 +0000 (GMT)","from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 940684203F;\n\tMon, 30 Oct 2017 13:10:05 +0000 (GMT)","from skywalker (unknown [9.85.185.102])\n\tby d06av24.portsmouth.uk.ibm.com (Postfix) with SMTP;\n\tMon, 30 Oct 2017 13:10:05 +0000 (GMT)","(nullmailer pid 17545 invoked by uid 1000);\n\tMon, 30 Oct 2017 13:14:47 -0000"],"Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com\n\t(client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com;\n\tenvelope-from=aneesh.kumar@linux.vnet.ibm.com; receiver=)","From":"\"Aneesh Kumar K.V\" ","To":"Paul Mackerras ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","In-Reply-To":"<87tvyg26us.fsf@linux.vnet.ibm.com>","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>\n\t\n\t<20171027054136.GC27483@fergus.ozlabs.ibm.com>\n\t<87tvyg26us.fsf@linux.vnet.ibm.com>","Date":"Mon, 30 Oct 2017 18:44:47 +0530","MIME-Version":"1.0","Content-Type":"text/plain","X-TM-AS-GCONF":"00","x-cbid":"17103013-0008-0000-0000-000004A574BB","X-IBM-AV-DETECTION":"SAVI=unused REMOTE=unused XFE=unused","x-cbparentid":"17103013-0009-0000-0000-00001E37F0BB","Message-Id":"<87r2tk22wo.fsf@linux.vnet.ibm.com>","X-Proofpoint-Virus-Version":"vendor=fsecure engine=2.50.10432:, ,\n\tdefinitions=2017-10-30_04:, , signatures=0","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tpriorityscore=1501\n\tmalwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0\n\tclxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0\n\tclassifier=spam adjust=0 reason=mlx scancount=1\n\tengine=8.0.1-1707230000\n\tdefinitions=main-1710300181","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"linuxppc-dev@lists.ozlabs.org","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1795951,"web_url":"http://patchwork.ozlabs.org/comment/1795951/","msgid":"<87o9oo21ay.fsf@linux.vnet.ibm.com>","date":"2017-10-30T13:49:25","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":664,"url":"http://patchwork.ozlabs.org/api/people/664/","name":"Aneesh Kumar K.V","email":"aneesh.kumar@linux.vnet.ibm.com"},"content":"\"Aneesh Kumar K.V\" writes:\n\n> \"Aneesh Kumar K.V\" writes:\n>\n>\n>> I looked at the perf data and with the test, we are doing larger number\n>> of hash faults and then around 10k flush_hash_range. Can the small\n>> improvement in number be due to the fact that we are not storing slot\n>> number when doing an insert now?. Also in the flush path we are now not\n>> using real_pte_t.\n>>\n>\n> With THP disabled I am finding below.\n>\n> Without patch\n>\n> 35.62% a.out [kernel.vmlinux] [k] clear_user_page\n> 8.54% a.out [kernel.vmlinux] [k] __lock_acquire\n> 3.86% a.out [kernel.vmlinux] [k] native_flush_hash_range\n> 3.38% a.out [kernel.vmlinux] [k] save_context_stack\n> 2.98% a.out a.out [.] main\n> 2.59% a.out [kernel.vmlinux] [k] lock_acquire\n> 2.29% a.out [kernel.vmlinux] [k] mark_lock\n> 2.23% a.out [kernel.vmlinux] [k] native_hpte_insert\n> 1.87% a.out [kernel.vmlinux] [k] get_mem_cgroup_from_mm\n> 1.71% a.out [kernel.vmlinux] [k] rcu_lockdep_current_cpu_online\n> 1.68% a.out [kernel.vmlinux] [k] lock_release\n> 1.47% a.out [kernel.vmlinux] [k] __handle_mm_fault\n> 1.41% a.out [kernel.vmlinux] [k] validate_sp\n>\n>\n> With patch\n> 35.40% a.out [kernel.vmlinux] [k] clear_user_page\n> 8.82% a.out [kernel.vmlinux] [k] __lock_acquire\n> 3.66% a.out a.out [.] main\n> 3.49% a.out [kernel.vmlinux] [k] save_context_stack\n> 2.77% a.out [kernel.vmlinux] [k] lock_acquire\n> 2.45% a.out [kernel.vmlinux] [k] mark_lock\n> 1.80% a.out [kernel.vmlinux] [k] get_mem_cgroup_from_mm\n> 1.80% a.out [kernel.vmlinux] [k] native_hpte_insert\n> 1.79% a.out [kernel.vmlinux] [k] rcu_lockdep_current_cpu_online\n> 1.78% a.out [kernel.vmlinux] [k] lock_release\n> 1.73% a.out [kernel.vmlinux] [k] native_flush_hash_range\n> 1.53% a.out [kernel.vmlinux] [k] __handle_mm_fault\n>\n> That is we are now spending less time in native_flush_hash_range.\n>\n> -aneesh\n\nOne possible explanation is, with slot tracking we do\n\n\tslot += hidx & _PTEIDX_GROUP_IX;\n\thptep = htab_address + slot;\n\twant_v = hpte_encode_avpn(vpn, psize, ssize);\n\tnative_lock_hpte(hptep);\n\thpte_v = be64_to_cpu(hptep->v);\n\tif (cpu_has_feature(CPU_FTR_ARCH_300))\n\t\thpte_v = hpte_new_to_old_v(hpte_v,\n\t\t\t\tbe64_to_cpu(hptep->r));\n\tif (!HPTE_V_COMPARE(hpte_v, want_v) ||\n\t !(hpte_v & HPTE_V_VALID))\n\t\tnative_unlock_hpte(hptep);\n\n\nand without slot tracking we do\n\n\tfor (i = 0; i < HPTES_PER_GROUP; i++, hptep++) {\n\t\t/* check locklessly first */\n\t\thpte_v = be64_to_cpu(hptep->v);\n\t\tif (cpu_has_feature(CPU_FTR_ARCH_300))\n\t\t\thpte_v = hpte_new_to_old_v(hpte_v, be64_to_cpu(hptep->r));\n\t\tif (!HPTE_V_COMPARE(hpte_v, want_v) || !(hpte_v & HPTE_V_VALID))\n\t\t\tcontinue;\n\n\t\tnative_lock_hpte(hptep);\n\nThat is without the patch series, we take the hpte lock always even if the\nhpte didn't match. Hence in perf annotate we find the lock to be\nhighly contended without patch series.\n\nI will change that to compare pte without taking lock and see if that\nhas any impact.\n\n-aneesh","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yQbTx6YzGz9sRV\n\tfor ;\n\tTue, 31 Oct 2017 00:50:45 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yQbTx5JwYzDrhJ\n\tfor ;\n\tTue, 31 Oct 2017 00:50:45 +1100 (AEDT)","from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com\n\t[148.163.156.1])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yQbSd0HG5zDqmt\n\tfor ;\n\tTue, 31 Oct 2017 00:49:36 +1100 (AEDT)","from pps.filterd (m0098393.ppops.net [127.0.0.1])\n\tby mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id\n\tv9UDlvYn076690\n\tfor ; Mon, 30 Oct 2017 09:49:34 -0400","from e06smtp14.uk.ibm.com (e06smtp14.uk.ibm.com [195.75.94.110])\n\tby mx0a-001b2d01.pphosted.com with ESMTP id 2dx2xm8x36-1\n\t(version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT)\n\tfor ; Mon, 30 Oct 2017 09:49:34 -0400","from localhost\n\tby e06smtp14.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use\n\tOnly! Violators will be prosecuted\n\tfor from\n\t; Mon, 30 Oct 2017 13:49:31 -0000","from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195)\n\tby e06smtp14.uk.ibm.com (192.168.101.144) with IBM ESMTP SMTP\n\tGateway: Authorized Use Only! Violators will be prosecuted; \n\tMon, 30 Oct 2017 13:49:30 -0000","from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com\n\t[9.149.105.62])\n\tby b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with\n\tESMTP id v9UDnTml25690312; Mon, 30 Oct 2017 13:49:29 GMT","from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id E3B32AE04D;\n\tMon, 30 Oct 2017 13:43:18 +0000 (GMT)","from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id C0E7DAE051;\n\tMon, 30 Oct 2017 13:43:16 +0000 (GMT)","from skywalker (unknown [9.85.185.102])\n\tby d06av26.portsmouth.uk.ibm.com (Postfix) with SMTP;\n\tMon, 30 Oct 2017 13:43:16 +0000 (GMT)","(nullmailer pid 20230 invoked by uid 1000);\n\tMon, 30 Oct 2017 13:49:25 -0000"],"Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com\n\t(client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com;\n\tenvelope-from=aneesh.kumar@linux.vnet.ibm.com; receiver=)","From":"\"Aneesh Kumar K.V\" ","To":"Paul Mackerras ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","In-Reply-To":"<87r2tk22wo.fsf@linux.vnet.ibm.com>","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>\n\t\n\t<20171027054136.GC27483@fergus.ozlabs.ibm.com>\n\t<87tvyg26us.fsf@linux.vnet.ibm.com>\n\t<87r2tk22wo.fsf@linux.vnet.ibm.com>","Date":"Mon, 30 Oct 2017 19:19:25 +0530","MIME-Version":"1.0","Content-Type":"text/plain","X-TM-AS-GCONF":"00","x-cbid":"17103013-0016-0000-0000-000004FB7496","X-IBM-AV-DETECTION":"SAVI=unused REMOTE=unused XFE=unused","x-cbparentid":"17103013-0017-0000-0000-00002836F84F","Message-Id":"<87o9oo21ay.fsf@linux.vnet.ibm.com>","X-Proofpoint-Virus-Version":"vendor=fsecure engine=2.50.10432:, ,\n\tdefinitions=2017-10-30_04:, , signatures=0","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tpriorityscore=1501\n\tmalwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0\n\tclxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0\n\tclassifier=spam adjust=0 reason=mlx scancount=1\n\tengine=8.0.1-1707230000\n\tdefinitions=main-1710300188","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"linuxppc-dev@lists.ozlabs.org","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1797095,"web_url":"http://patchwork.ozlabs.org/comment/1797095/","msgid":"<878tfq8v2o.fsf@concordia.ellerman.id.au>","date":"2017-11-01T04:46:39","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":46580,"url":"http://patchwork.ozlabs.org/api/people/46580/","name":"Michael Ellerman","email":"mpe@ellerman.id.au"},"content":"Ram Pai writes:\n\n> On Mon, Oct 30, 2017 at 09:04:17AM +1100, Paul Mackerras wrote:\n>> On Sat, Oct 28, 2017 at 03:35:32PM -0700, Ram Pai wrote:\n>> > \n>> > I like the idea of not tracking the slots at all. It is something the\n>> > guest should not be knowing or tracking.\n>> \n>> Why do you say that?\n>\n> 'slot' is a internal mechanism used by the hash table to accelerate\n> mapping an address to a hash-page. If the hash-table implementation\n> choose to use a different mechanism to accelerate the mapping, it can't\n> because that mechanism is baked into the logic of all the consumers.\n\nThe slots are defined in the architecture, so it's entirely fine for\nguests to track them.\n\ncheers","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yRbLQ2zqCz9t3x\n\tfor ;\n\tWed, 1 Nov 2017 15:47:42 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yRbLQ25GwzDrJy\n\tfor ;\n\tWed, 1 Nov 2017 15:47:42 +1100 (AEDT)","from ozlabs.org (bilbo.ozlabs.org [103.22.144.67])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yRbKH3JkLzDr1w\n\tfor ;\n\tWed, 1 Nov 2017 15:46:43 +1100 (AEDT)","from authenticated.ozlabs.org (localhost [127.0.0.1])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPSA id 3yRbKH1nDtz9sNc;\n\tWed, 1 Nov 2017 15:46:43 +1100 (AEDT)"],"From":"Michael Ellerman ","To":"Ram Pai , Paul Mackerras ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","In-Reply-To":"<20171030005106.GD5587@ram.oc3035372033.ibm.com>","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>\n\t\n\t<20171028223532.GA5587@ram.oc3035372033.ibm.com>\n\t<20171029220417.GA3317@fergus.ozlabs.ibm.com>\n\t<20171030005106.GD5587@ram.oc3035372033.ibm.com>","Date":"Wed, 01 Nov 2017 15:46:39 +1100","Message-ID":"<878tfq8v2o.fsf@concordia.ellerman.id.au>","MIME-Version":"1.0","Content-Type":"text/plain","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"linuxppc-dev@lists.ozlabs.org,\n\t\"Aneesh Kumar K.V\" ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1797193,"web_url":"http://patchwork.ozlabs.org/comment/1797193/","msgid":"<20171101110241.GA17203@fergus.ozlabs.ibm.com>","date":"2017-11-01T11:02:41","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":67079,"url":"http://patchwork.ozlabs.org/api/people/67079/","name":"Paul Mackerras","email":"paulus@ozlabs.org"},"content":"On Sun, Oct 29, 2017 at 05:51:06PM -0700, Ram Pai wrote:\n> On Mon, Oct 30, 2017 at 09:04:17AM +1100, Paul Mackerras wrote:\n> > On Sat, Oct 28, 2017 at 03:35:32PM -0700, Ram Pai wrote:\n> > > \n> > > I like the idea of not tracking the slots at all. It is something the\n> > > guest should not be knowing or tracking.\n> > \n> > Why do you say that?\n> \n> 'slot' is a internal mechanism used by the hash table to accelerate\n> mapping an address to a hash-page. If the hash-table implementation\n> choose to use a different mechanism to accelerate the mapping, it can't\n> because that mechanism is baked into the logic of all the consumers.\n\nNot all operating systems use the HPT as a cache of translations that\nare also stored somewhere, as Linux does. Those OSes are perfectly\nentitled to control the slot allocation for their own purposes in\nwhatever way they want. So having the interface use the slot number\nis fine; just as having alternative interfaces that don't need to\nspecify the slot number for the kind of usage that Linux does is also\nfine.\n\nPaul.","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yRlhk3c3hz9t2f\n\tfor ;\n\tWed, 1 Nov 2017 22:04:06 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yRlhk0WZDzDr4m\n\tfor ;\n\tWed, 1 Nov 2017 22:04:06 +1100 (AEDT)","from ozlabs.org (ozlabs.org [IPv6:2401:3900:2:1::2])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yRlg74sR7zDr4H\n\tfor ;\n\tWed, 1 Nov 2017 22:02:43 +1100 (AEDT)","by ozlabs.org (Postfix, from userid 1003)\n\tid 3yRlg73vXpz9t2m; Wed, 1 Nov 2017 22:02:43 +1100 (AEDT)"],"Authentication-Results":["ozlabs.org; dkim=pass (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"KdGXz/uC\";\n\tdkim-atps=neutral","lists.ozlabs.org; dkim=pass (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"KdGXz/uC\";\n\tdkim-atps=neutral","lists.ozlabs.org; dkim=pass (2048-bit key;\n\tsecure) header.d=ozlabs.org header.i=@ozlabs.org header.b=\"KdGXz/uC\"; \n\tdkim-atps=neutral"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ozlabs.org; s=201707; \n\tt=1509534163; bh=lLJRGArdW15VLPvzgVq11UvpQDbSkBdmyLQa741SUdY=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=KdGXz/uCmbB02JgIoMd0OU/+wbY9mjeGUf5vcrq7EgTG46XjETv/3XDQzMPMTEvVv\n\tzahT7vbDVmyHx4SHp3IpzC8+GyvarhdSuTPlL175fetka7IwMB7GZUQN/rwayavCLQ\n\tQZ+4LqHeu6iL+l4dbSh9Ro/K23AUBvuJ4ccSEUGhIZ0ilfx9xwLih7O7Ny0OywhUs4\n\t+9ebH+ovZ7p2OvgB2TnURz1QMbD0v6TsFBgBwO+XfsPc/Kma9OWk8QnAa6iB5OKYin\n\ttPvHQOLRNbZxWSDFDAHee6XgQIYTkKaifuQDOhtq8f69AHjV6UhrwC0E69Kcfnq6PR\n\t1Ete7/EYQbnNg==","Date":"Wed, 1 Nov 2017 22:02:41 +1100","From":"Paul Mackerras ","To":"Ram Pai ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","Message-ID":"<20171101110241.GA17203@fergus.ozlabs.ibm.com>","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>\n\t\n\t<20171028223532.GA5587@ram.oc3035372033.ibm.com>\n\t<20171029220417.GA3317@fergus.ozlabs.ibm.com>\n\t<20171030005106.GD5587@ram.oc3035372033.ibm.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20171030005106.GD5587@ram.oc3035372033.ibm.com>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"linuxppc-dev@lists.ozlabs.org,\n\t\"Aneesh Kumar K.V\" ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1807907,"web_url":"http://patchwork.ozlabs.org/comment/1807907/","msgid":"<871sks10s7.fsf@linux.vnet.ibm.com>","date":"2017-11-21T08:41:44","subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","submitter":{"id":664,"url":"http://patchwork.ozlabs.org/api/people/664/","name":"Aneesh Kumar K.V","email":"aneesh.kumar@linux.vnet.ibm.com"},"content":"Paul Mackerras writes:\n> On pseries, performance is about 2.4% worse without new hcalls, but\n> that is less than 1 standard deviation. With new hcalls, performance\n> is 0.95% worse, only a third of a standard deviation. I think we need\n> to do more measurements to try to get a more accurate picture here.\n>\n> Were the pseries numbers done on KVM or PowerVM? Could you do a set\n> of measurements on the other one too please? (I assume the numbers\n> with the new hcall were done on KVM, and can't be done on PowerVM.)\n>\n\nI got ebizzy and kernel compile run on powernv and powervm config. You\ncan find the numbers below. I did 10 iterations and only added stdev and\nmedian below. I do find powernv do better with patch series.\n\nebizzy run\n-----------\nPowerNV (ebizzy -m -n 1000 -P -s 512000 -S 100 -t 100):\nWith patches, 10 iterations results records/sec.\nstdev =\t37.60\nmedian = 7411.5\n\nWithout patch:\nstdev = 23.071\nmedian = 7350\n\nPowerVM numbers(./ebizzy -m -n 1000 -P -s 512000 -S 100 -t 30):\nWith patch (no new hcalls):\nstdev = 20.721\nmedian = 6955.5\n\t\nWithout patch\t\nstdev =\t35.049\nmedian = 7081\n\nkernel compile:(time -p)\n---------------------------\nPowerNV:\nWith patches:\nReal\t\n----\t\nstdev = 1.624\nmedian = 61.56\n\t\nUser:\t\nstdev =\t61.204\nmedian = 4816.73\n\t\nSys:\t\nstdev =\t4.367\nmedian = 387.575\n\nWithout patches:\nReal:\t\nstdev =\t1.318\nmedian = 63.635\n\t\nUser:\t\nstdev =\t50.531\nmedian = 4820.51\n\t\n\t\nSys:\t\nstdev =\t6.409\nmedian = 389.765\n\nPowerVM numbers:\n-------------------\nWith patches (no new hcalls):\nReal:\t\nstdev =\t3.016\nmedian = 442.745\n\t\n\t\nUser:\t\nstdev =\t9.738\nmedian = 5507.87\n\t\nSys:\t\nstdev =\t0.223\nmedian = 176.455\n\nWitout patches:\nReal:\nstdev = 0.720\nmedian = 442.445\n\nUser:\nstdev = 8.621\nmedian = 5501.615\n\nSys:\nstdev = 0.189\nmedian = 173.3","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3ygzcs1KnSz9s7F\n\tfor ;\n\tTue, 21 Nov 2017 19:43:09 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3ygzcs0TpszDrnn\n\tfor ;\n\tTue, 21 Nov 2017 19:43:09 +1100 (AEDT)","from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com\n\t[148.163.158.5])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3ygzbS6tnFzDrTs\n\tfor ;\n\tTue, 21 Nov 2017 19:41:56 +1100 (AEDT)","from pps.filterd (m0098414.ppops.net [127.0.0.1])\n\tby mx0b-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id\n\tvAL8ddwN127033\n\tfor ; Tue, 21 Nov 2017 03:41:53 -0500","from e06smtp10.uk.ibm.com (e06smtp10.uk.ibm.com [195.75.94.106])\n\tby mx0b-001b2d01.pphosted.com with ESMTP id 2ece900ffb-1\n\t(version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT)\n\tfor ; Tue, 21 Nov 2017 03:41:53 -0500","from localhost\n\tby e06smtp10.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use\n\tOnly! Violators will be prosecuted\n\tfor from\n\t; Tue, 21 Nov 2017 08:41:51 -0000","from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195)\n\tby e06smtp10.uk.ibm.com (192.168.101.140) with IBM ESMTP SMTP\n\tGateway: Authorized Use Only! Violators will be prosecuted; \n\tTue, 21 Nov 2017 08:41:48 -0000","from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com\n\t[9.149.105.62])\n\tby b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with\n\tESMTP id vAL8fmsT3342624; Tue, 21 Nov 2017 08:41:48 GMT","from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 1C484AE045;\n\tTue, 21 Nov 2017 08:35:05 +0000 (GMT)","from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1])\n\tby IMSVA (Postfix) with ESMTP id 8ADE2AE056;\n\tTue, 21 Nov 2017 08:35:03 +0000 (GMT)","from skywalker (unknown [9.85.195.218])\n\tby d06av26.portsmouth.uk.ibm.com (Postfix) with SMTP;\n\tTue, 21 Nov 2017 08:35:03 +0000 (GMT)","(nullmailer pid 16588 invoked by uid 1000);\n\tTue, 21 Nov 2017 08:41:44 -0000"],"Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com\n\t(client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com;\n\tenvelope-from=aneesh.kumar@linux.vnet.ibm.com; receiver=)","From":"\"Aneesh Kumar K.V\" ","To":"Paul Mackerras ","Subject":"Re: [PATCH 00/16] Remove hash page table slot tracking from linux\n\tPTE","In-Reply-To":"<20171027054136.GC27483@fergus.ozlabs.ibm.com>","References":"<20171027040833.3644-1-aneesh.kumar@linux.vnet.ibm.com>\n\t<20171027043430.GA27483@fergus.ozlabs.ibm.com>\n\t\n\t<20171027054136.GC27483@fergus.ozlabs.ibm.com>","Date":"Tue, 21 Nov 2017 14:11:44 +0530","MIME-Version":"1.0","Content-Type":"text/plain","X-TM-AS-GCONF":"00","x-cbid":"17112108-0040-0000-0000-000003F0CA0A","X-IBM-AV-DETECTION":"SAVI=unused REMOTE=unused XFE=unused","x-cbparentid":"17112108-0041-0000-0000-000025F39017","Message-Id":"<871sks10s7.fsf@linux.vnet.ibm.com>","X-Proofpoint-Virus-Version":"vendor=fsecure engine=2.50.10432:, ,\n\tdefinitions=2017-11-21_03:, , signatures=0","X-Proofpoint-Spam-Details":"rule=outbound_notspam policy=outbound score=0\n\tpriorityscore=1501\n\tmalwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0\n\tclxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0\n\tclassifier=spam adjust=0 reason=mlx scancount=1\n\tengine=8.0.1-1709140000\n\tdefinitions=main-1711210119","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"linuxppc-dev@lists.ozlabs.org","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}}]