[{"id":1772690,"web_url":"http://patchwork.ozlabs.org/comment/1772690/","msgid":"<37e3ce0e-717d-156d-fef3-27559aff980e@arm.com>","list_archive_url":null,"date":"2017-09-21T11:11:23","subject":"Re: [PATCH 2/4] numa, iommu/io-pgtable-arm: Use NUMA aware memory\n\tallocation for smmu translation tables","submitter":{"id":65641,"url":"http://patchwork.ozlabs.org/api/people/65641/","name":"Robin Murphy","email":"robin.murphy@arm.com"},"content":"On 21/09/17 09:59, Ganapatrao Kulkarni wrote:\n> function __arm_lpae_alloc_pages is used to allcoated memory for smmu\n> translation tables. updating function to allocate memory/pages\n> from the proximity domain of SMMU device.\n\nAFAICS, data->pgd_size always works out to a power-of-two number of\npages, so I'm not sure why we've ever needed alloc_pages_exact() here. I\nthink we could simply use alloc_pages_node() and drop patch #1.\n\nRobin.\n\n> Signed-off-by: Ganapatrao Kulkarni \n> ---\n> drivers/iommu/io-pgtable-arm.c | 4 +++-\n> 1 file changed, 3 insertions(+), 1 deletion(-)\n> \n> diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c\n> index e8018a3..f6d01f6 100644\n> --- a/drivers/iommu/io-pgtable-arm.c\n> +++ b/drivers/iommu/io-pgtable-arm.c\n> @@ -215,8 +215,10 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp,\n> {\n> \tstruct device *dev = cfg->iommu_dev;\n> \tdma_addr_t dma;\n> -\tvoid *pages = alloc_pages_exact(size, gfp | __GFP_ZERO);\n> +\tvoid *pages;\n> \n> +\tpages = alloc_pages_exact_nid(dev_to_node(dev), size,\n> +\t\t\tgfp | __GFP_ZERO);\n> \tif (!pages)\n> \t\treturn NULL;\n> \n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"D5LnbIPU\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyYpk4TrVz9t3w\n\tfor ;\n\tThu, 21 Sep 2017 21:11:58 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1duzOc-0000Os-QB; Thu, 21 Sep 2017 11:11:54 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1duzOY-0000JL-WF for linux-arm-kernel@lists.infradead.org;\n\tThu, 21 Sep 2017 11:11:52 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7D66780D;\n\tThu, 21 Sep 2017 04:11:27 -0700 (PDT)","from [10.1.210.88] (e110467-lin.cambridge.arm.com [10.1.210.88])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tEDB423F58C; Thu, 21 Sep 2017 04:11:24 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=uoIwaIRap9DzplwrYg56nXfHq6riTIQTwqaZVYmWELA=;\n\tb=D5LnbIPU6STNVG\n\tBkTDPGocR9LW+W+4DFf/dWGOtTbULnX36SEbXCDx2zFkDXweByhVXXGz2WdHZ+KPohAFa/KsFRRxy\n\ts31I/LMCgjs+1gm1vu8WFkFEzmkNJjgSNx3u72GK1t1tDzc4A3xKaaD45H5fOg3v7obV0nCwudyXp\n\t8eLZ57Z48lMAPZGNAjf0v0+JysFqIE/ayPp0h695/EAoJRGXE9obOhAeR5/78T4RVkbiJWZ1msnvY\n\t8M4KMW0nYMTCE384BqJFIgkxEs8vMvjOEUV1MHKNsV2Tw9zd1JDA/Gmbq/XLV9zdy88Q5k5imDqyM\n\tYKfx2FHjdJxGmBRoaS/w==;","Subject":"Re: [PATCH 2/4] numa, iommu/io-pgtable-arm: Use NUMA aware memory\n\tallocation for smmu translation tables","To":"Ganapatrao Kulkarni ,\n\tlinux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org,\n\tiommu@lists.linux-foundation.org, linux-mm@kvack.org","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<20170921085922.11659-3-ganapatrao.kulkarni@cavium.com>","From":"Robin Murphy ","Message-ID":"<37e3ce0e-717d-156d-fef3-27559aff980e@arm.com>","Date":"Thu, 21 Sep 2017 12:11:23 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170921085922.11659-3-ganapatrao.kulkarni@cavium.com>","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170921_041151_053117_AE0BEB41 ","X-CRM114-Status":"GOOD ( 11.75 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"lorenzo.pieralisi@arm.com, joro@8bytes.org, Will.Deacon@arm.com,\n\tgklkml16@gmail.com, Tomasz.Nowicki@cavium.com,\n\tRobert.Richter@cavium.com, \n\tmhocko@suse.com, hanjun.guo@linaro.org, akpm@linux-foundation.org,\n\tvbabka@suse.cz, jnair@caviumnetworks.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1772728,"web_url":"http://patchwork.ozlabs.org/comment/1772728/","msgid":"<9d65676f-e4e8-e0a6-602c-361d83ce83c1@arm.com>","list_archive_url":null,"date":"2017-09-21T11:41:11","subject":"Re: [PATCH 4/4] iommu/dma,\n\tnuma: Use NUMA aware memory allocations in __iommu_dma_alloc_pages","submitter":{"id":65641,"url":"http://patchwork.ozlabs.org/api/people/65641/","name":"Robin Murphy","email":"robin.murphy@arm.com"},"content":"On 21/09/17 09:59, Ganapatrao Kulkarni wrote:\n> Change function __iommu_dma_alloc_pages to allocate memory/pages\n> for dma from respective device numa node.\n> \n> Signed-off-by: Ganapatrao Kulkarni \n> ---\n> drivers/iommu/dma-iommu.c | 17 ++++++++++-------\n> 1 file changed, 10 insertions(+), 7 deletions(-)\n> \n> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c\n> index 9d1cebe..0626b58 100644\n> --- a/drivers/iommu/dma-iommu.c\n> +++ b/drivers/iommu/dma-iommu.c\n> @@ -428,20 +428,21 @@ static void __iommu_dma_free_pages(struct page **pages, int count)\n> \tkvfree(pages);\n> }\n> \n> -static struct page **__iommu_dma_alloc_pages(unsigned int count,\n> -\t\tunsigned long order_mask, gfp_t gfp)\n> +static struct page **__iommu_dma_alloc_pages(struct device *dev,\n> +\t\tunsigned int count, unsigned long order_mask, gfp_t gfp)\n> {\n> \tstruct page **pages;\n> \tunsigned int i = 0, array_size = count * sizeof(*pages);\n> +\tint numa_node = dev_to_node(dev);\n> \n> \torder_mask &= (2U << MAX_ORDER) - 1;\n> \tif (!order_mask)\n> \t\treturn NULL;\n> \n> \tif (array_size <= PAGE_SIZE)\n> -\t\tpages = kzalloc(array_size, GFP_KERNEL);\n> +\t\tpages = kzalloc_node(array_size, GFP_KERNEL, numa_node);\n> \telse\n> -\t\tpages = vzalloc(array_size);\n> +\t\tpages = vzalloc_node(array_size, numa_node);\n\nkvzalloc{,_node}() didn't exist when this code was first written, but it\ndoes now - since you're touching it you may as well get rid of the whole\nif-else and array_size local.\n\nFurther nit: some of the indentation below is a bit messed up.\n\nRobin.\n\n> \tif (!pages)\n> \t\treturn NULL;\n> \n> @@ -462,8 +463,9 @@ static struct page **__iommu_dma_alloc_pages(unsigned int count,\n> \t\t\tunsigned int order = __fls(order_mask);\n> \n> \t\t\torder_size = 1U << order;\n> -\t\t\tpage = alloc_pages((order_mask - order_size) ?\n> -\t\t\t\t\t gfp | __GFP_NORETRY : gfp, order);\n> +\t\t\tpage = alloc_pages_node(numa_node,\n> +\t\t\t\t\t(order_mask - order_size) ?\n> +\t\t\t\t gfp | __GFP_NORETRY : gfp, order);\n> \t\t\tif (!page)\n> \t\t\t\tcontinue;\n> \t\t\tif (!order)\n> @@ -548,7 +550,8 @@ struct page **iommu_dma_alloc(struct device *dev, size_t size, gfp_t gfp,\n> \t\talloc_sizes = min_size;\n> \n> \tcount = PAGE_ALIGN(size) >> PAGE_SHIFT;\n> -\tpages = __iommu_dma_alloc_pages(count, alloc_sizes >> PAGE_SHIFT, gfp);\n> +\tpages = __iommu_dma_alloc_pages(dev, count, alloc_sizes >> PAGE_SHIFT,\n> +\t\t\tgfp);\n> \tif (!pages)\n> \t\treturn NULL;\n> \n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"UvVovsaQ\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyZT16CQTz9s03\n\tfor ;\n\tThu, 21 Sep 2017 21:41:41 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1duzrO-00007C-S1; Thu, 21 Sep 2017 11:41:38 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1duzrL-0008VQ-AT for linux-arm-kernel@lists.infradead.org;\n\tThu, 21 Sep 2017 11:41:36 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0A1A780D;\n\tThu, 21 Sep 2017 04:41:15 -0700 (PDT)","from [10.1.210.88] (e110467-lin.cambridge.arm.com [10.1.210.88])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t7A2B23F58C; Thu, 21 Sep 2017 04:41:12 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=dxL1KGSw+tVhFXTDLJBy9iAfiDTffQhFKvxIGyCUY7A=;\n\tb=UvVovsaQdXJuqt\n\tk3srn7N9Q4JLT1Ikm5chHrEJDuAvxPrODw2z7aL7aifIElZkmS3cpPHQCwcKqdq9yKp5TyIIwfF6i\n\tn1ZXrIHrObzStxuY1pMKiBRSnRQBa+Z35Qi07DJAlAGmSCOVfmb1sMqACh/cGv7CamIUgQ0egf08w\n\tFCuGQF511ISPAvqACTDVb2QXZgukp07qYYZHy/TsI/tbFXfiJIQHsZvbAQfWu59kXu7Uf4V2HLD3q\n\tJhD3Ca9Utz+fe1TOcU8UCGNbH8STzGo9i/a+Cp7PEvF0Z24bhaoQZDLQpW7cDd7e5PcDi6X6NsX4X\n\tPg1xRsbbsOVheN19JzCQ==;","Subject":"Re: [PATCH 4/4] iommu/dma,\n\tnuma: Use NUMA aware memory allocations in __iommu_dma_alloc_pages","To":"Ganapatrao Kulkarni ,\n\tlinux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org,\n\tiommu@lists.linux-foundation.org, linux-mm@kvack.org","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<20170921085922.11659-5-ganapatrao.kulkarni@cavium.com>","From":"Robin Murphy ","Message-ID":"<9d65676f-e4e8-e0a6-602c-361d83ce83c1@arm.com>","Date":"Thu, 21 Sep 2017 12:41:11 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170921085922.11659-5-ganapatrao.kulkarni@cavium.com>","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170921_044135_409700_F8C51EFB ","X-CRM114-Status":"GOOD ( 13.40 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"lorenzo.pieralisi@arm.com, joro@8bytes.org, Will.Deacon@arm.com,\n\tgklkml16@gmail.com, Tomasz.Nowicki@cavium.com,\n\tRobert.Richter@cavium.com, \n\tmhocko@suse.com, hanjun.guo@linaro.org, akpm@linux-foundation.org,\n\tvbabka@suse.cz, jnair@caviumnetworks.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1772742,"web_url":"http://patchwork.ozlabs.org/comment/1772742/","msgid":"","list_archive_url":null,"date":"2017-09-21T11:58:04","subject":"Re: [PATCH 3/4] iommu/arm-smmu-v3: Use NUMA memory allocations for\n\tstream tables and comamnd queues","submitter":{"id":65641,"url":"http://patchwork.ozlabs.org/api/people/65641/","name":"Robin Murphy","email":"robin.murphy@arm.com"},"content":"[+Christoph and Marek]\n\nOn 21/09/17 09:59, Ganapatrao Kulkarni wrote:\n> Introduce smmu_alloc_coherent and smmu_free_coherent functions to\n> allocate/free dma coherent memory from NUMA node associated with SMMU.\n> Replace all calls of dmam_alloc_coherent with smmu_alloc_coherent\n> for SMMU stream tables and command queues.\n\nThis doesn't work - not only do you lose the 'managed' aspect and risk\nleaking various tables on probe failure or device removal, but more\nimportantly, unless you add DMA syncs around all the CPU accesses to the\ntables, you lose the critical 'coherent' aspect, and that's a horribly\ninvasive change that I really don't want to make.\n\nChristoph, Marek; how reasonable do you think it is to expect\ndma_alloc_coherent() to be inherently NUMA-aware on NUMA-capable\nsystems? SWIOTLB looks fairly straightforward to fix up (for the simple\nallocation case; I'm not sure it's even worth it for bounce-buffering),\nbut the likes of CMA might be a little trickier...\n\nRobin.\n\n> Signed-off-by: Ganapatrao Kulkarni \n> ---\n> drivers/iommu/arm-smmu-v3.c | 57 ++++++++++++++++++++++++++++++++++++++++-----\n> 1 file changed, 51 insertions(+), 6 deletions(-)\n> \n> diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c\n> index e67ba6c..bc4ba1f 100644\n> --- a/drivers/iommu/arm-smmu-v3.c\n> +++ b/drivers/iommu/arm-smmu-v3.c\n> @@ -1158,6 +1158,50 @@ static void arm_smmu_init_bypass_stes(u64 *strtab, unsigned int nent)\n> \t}\n> }\n> \n> +static void *smmu_alloc_coherent(struct arm_smmu_device *smmu, size_t size,\n> +\t\tdma_addr_t *dma_handle,\tgfp_t gfp)\n> +{\n> +\tstruct device *dev = smmu->dev;\n> +\tvoid *pages;\n> +\tdma_addr_t dma;\n> +\tint numa_node = dev_to_node(dev);\n> +\n> +\tpages = alloc_pages_exact_nid(numa_node, size, gfp | __GFP_ZERO);\n> +\tif (!pages)\n> +\t\treturn NULL;\n> +\n> +\tif (!(smmu->features & ARM_SMMU_FEAT_COHERENCY)) {\n> +\t\tdma = dma_map_single(dev, pages, size, DMA_TO_DEVICE);\n> +\t\tif (dma_mapping_error(dev, dma))\n> +\t\t\tgoto out_free;\n> +\t\t/*\n> +\t\t * We depend on the SMMU being able to work with any physical\n> +\t\t * address directly, so if the DMA layer suggests otherwise by\n> +\t\t * translating or truncating them, that bodes very badly...\n> +\t\t */\n> +\t\tif (dma != virt_to_phys(pages))\n> +\t\t\tgoto out_unmap;\n> +\t}\n> +\n> +\t*dma_handle = (dma_addr_t)virt_to_phys(pages);\n> +\treturn pages;\n> +\n> +out_unmap:\n> +\tdev_err(dev, \"Cannot accommodate DMA translation for IOMMU page tables\\n\");\n> +\tdma_unmap_single(dev, dma, size, DMA_TO_DEVICE);\n> +out_free:\n> +\tfree_pages_exact(pages, size);\n> +\treturn NULL;\n> +}\n> +\n> +static void smmu_free_coherent(struct arm_smmu_device *smmu, size_t size,\n> +\t\tvoid *pages, dma_addr_t dma_handle)\n> +{\n> +\tif (!(smmu->features & ARM_SMMU_FEAT_COHERENCY))\n> +\t\tdma_unmap_single(smmu->dev, dma_handle, size, DMA_TO_DEVICE);\n> +\tfree_pages_exact(pages, size);\n> +}\n> +\n> static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)\n> {\n> \tsize_t size;\n> @@ -1172,7 +1216,7 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)\n> \tstrtab = &cfg->strtab[(sid >> STRTAB_SPLIT) * STRTAB_L1_DESC_DWORDS];\n> \n> \tdesc->span = STRTAB_SPLIT + 1;\n> -\tdesc->l2ptr = dmam_alloc_coherent(smmu->dev, size, &desc->l2ptr_dma,\n> +\tdesc->l2ptr = smmu_alloc_coherent(smmu, size, &desc->l2ptr_dma,\n> \t\t\t\t\t GFP_KERNEL | __GFP_ZERO);\n> \tif (!desc->l2ptr) {\n> \t\tdev_err(smmu->dev,\n> @@ -1487,7 +1531,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain)\n> \t\tstruct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg;\n> \n> \t\tif (cfg->cdptr) {\n> -\t\t\tdmam_free_coherent(smmu_domain->smmu->dev,\n> +\t\t\tsmmu_free_coherent(smmu,\n> \t\t\t\t\t CTXDESC_CD_DWORDS << 3,\n> \t\t\t\t\t cfg->cdptr,\n> \t\t\t\t\t cfg->cdptr_dma);\n> @@ -1515,7 +1559,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain,\n> \tif (asid < 0)\n> \t\treturn asid;\n> \n> -\tcfg->cdptr = dmam_alloc_coherent(smmu->dev, CTXDESC_CD_DWORDS << 3,\n> +\tcfg->cdptr = smmu_alloc_coherent(smmu, CTXDESC_CD_DWORDS << 3,\n> \t\t\t\t\t &cfg->cdptr_dma,\n> \t\t\t\t\t GFP_KERNEL | __GFP_ZERO);\n> \tif (!cfg->cdptr) {\n> @@ -1984,7 +2028,7 @@ static int arm_smmu_init_one_queue(struct arm_smmu_device *smmu,\n> {\n> \tsize_t qsz = ((1 << q->max_n_shift) * dwords) << 3;\n> \n> -\tq->base = dmam_alloc_coherent(smmu->dev, qsz, &q->base_dma, GFP_KERNEL);\n> +\tq->base = smmu_alloc_coherent(smmu, qsz, &q->base_dma, GFP_KERNEL);\n> \tif (!q->base) {\n> \t\tdev_err(smmu->dev, \"failed to allocate queue (0x%zx bytes)\\n\",\n> \t\t\tqsz);\n> @@ -2069,7 +2113,7 @@ static int arm_smmu_init_strtab_2lvl(struct arm_smmu_device *smmu)\n> \t\t\t size, smmu->sid_bits);\n> \n> \tl1size = cfg->num_l1_ents * (STRTAB_L1_DESC_DWORDS << 3);\n> -\tstrtab = dmam_alloc_coherent(smmu->dev, l1size, &cfg->strtab_dma,\n> +\tstrtab = smmu_alloc_coherent(smmu, l1size, &cfg->strtab_dma,\n> \t\t\t\t GFP_KERNEL | __GFP_ZERO);\n> \tif (!strtab) {\n> \t\tdev_err(smmu->dev,\n> @@ -2097,8 +2141,9 @@ static int arm_smmu_init_strtab_linear(struct arm_smmu_device *smmu)\n> \tu32 size;\n> \tstruct arm_smmu_strtab_cfg *cfg = &smmu->strtab_cfg;\n> \n> +\n> \tsize = (1 << smmu->sid_bits) * (STRTAB_STE_DWORDS << 3);\n> -\tstrtab = dmam_alloc_coherent(smmu->dev, size, &cfg->strtab_dma,\n> +\tstrtab = smmu_alloc_coherent(smmu, size, &cfg->strtab_dma,\n> \t\t\t\t GFP_KERNEL | __GFP_ZERO);\n> \tif (!strtab) {\n> \t\tdev_err(smmu->dev,\n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"n2R/F+H+\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyZrf52Zrz9s5L\n\tfor ;\n\tThu, 21 Sep 2017 21:58:42 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dv07q-00075G-C7; Thu, 21 Sep 2017 11:58:38 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dv07g-0006v4-RY for linux-arm-kernel@lists.infradead.org;\n\tThu, 21 Sep 2017 11:58:35 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8BEB480D;\n\tThu, 21 Sep 2017 04:58:08 -0700 (PDT)","from [10.1.210.88] (e110467-lin.cambridge.arm.com [10.1.210.88])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tA28A43F58C; Thu, 21 Sep 2017 04:58:05 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=rQS3BAgKmWDFvnHOnzaxeUeV7rSVRYjz3Mx1fgEpV+4=;\n\tb=n2R/F+H+n4PA8J\n\tmOiO0ubxboE3LYgX2gDxjtt1MMATDFSHFXGA7SHyqcj9ixcDWgbA0BvUB8FAup6MjmaI0NBz9G9nV\n\tSWYg5Esk1egNydYY0HbP54J9BUc9yPF1AkZ+V/7Fcl7xLFbcBe7uvFu3aMrEdq/ICPHq2mBZwm+Dy\n\tMw4daw3r08nywgKmvnlq39cC8wcmT+G1KBGU43hWKeF6ezRRBAwuN4bGXtPQpKmYUjJQhJsgcvB9V\n\t3pp2cceSLn7Ta0NDHtUmI53gH3b1CdmHX4dgBqs5g238+RKHv7uB/dVFuF29DGSmSPnyIU7pr5tAQ\n\tAaLoZs8jU7CnzAByxDTw==;","Subject":"Re: [PATCH 3/4] iommu/arm-smmu-v3: Use NUMA memory allocations for\n\tstream tables and comamnd queues","To":"Ganapatrao Kulkarni ,\n\tlinux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org,\n\tiommu@lists.linux-foundation.org, linux-mm@kvack.org,\n\tChristoph Hellwig ,\n\tMarek Szyprowski ","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<20170921085922.11659-4-ganapatrao.kulkarni@cavium.com>","From":"Robin Murphy ","Message-ID":"","Date":"Thu, 21 Sep 2017 12:58:04 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170921085922.11659-4-ganapatrao.kulkarni@cavium.com>","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170921_045829_012189_0AAB6C27 ","X-CRM114-Status":"GOOD ( 19.33 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"lorenzo.pieralisi@arm.com, joro@8bytes.org, Will.Deacon@arm.com,\n\tgklkml16@gmail.com, Tomasz.Nowicki@cavium.com,\n\tRobert.Richter@cavium.com, \n\tmhocko@suse.com, hanjun.guo@linaro.org, akpm@linux-foundation.org,\n\tvbabka@suse.cz, jnair@caviumnetworks.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1772841,"web_url":"http://patchwork.ozlabs.org/comment/1772841/","msgid":"<20170921142639.GA18211@lst.de>","list_archive_url":null,"date":"2017-09-21T14:26:39","subject":"Re: [PATCH 3/4] iommu/arm-smmu-v3: Use NUMA memory allocations for\n\tstream tables and comamnd queues","submitter":{"id":82,"url":"http://patchwork.ozlabs.org/api/people/82/","name":"Christoph Hellwig","email":"hch@lst.de"},"content":"On Thu, Sep 21, 2017 at 12:58:04PM +0100, Robin Murphy wrote:\n> Christoph, Marek; how reasonable do you think it is to expect\n> dma_alloc_coherent() to be inherently NUMA-aware on NUMA-capable\n> systems? SWIOTLB looks fairly straightforward to fix up (for the simple\n> allocation case; I'm not sure it's even worth it for bounce-buffering),\n> but the likes of CMA might be a little trickier...\n\nI think allocating data node local to dev is a good default. I'm not\nsure if we'd still need a version that takes an explicit node, though.\n\nOn the one hand devices like NVMe or RDMA nics have queues that are\nassigned to specific cpus and thus have an inherent affinity to given\nnodes. On the other hand we'd still need to access the PCIe device,\nso for it to make sense we'd need to access the dma memory a lot more\nfrom the host than from the device, and I'm not sure if we ever have\ndevices where that is the case (which would not be optimal to start\nwith).","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"MmoCFh6B\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyf7w5xJkz9s7g\n\tfor ;\n\tFri, 22 Sep 2017 00:27:08 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dv2RV-0003t5-QL; Thu, 21 Sep 2017 14:27:05 +0000","from verein.lst.de ([213.95.11.211] helo=newverein.lst.de)\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dv2RR-0003pe-Tm for linux-arm-kernel@lists.infradead.org;\n\tThu, 21 Sep 2017 14:27:03 +0000","by newverein.lst.de (Postfix, from userid 2407)\n\tid A82FD68CFF; Thu, 21 Sep 2017 16:26:39 +0200 (CEST)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:\n\tMessage-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=Y3SZGwER/o0TaE1ZH+K+ASEC/03+pICsCnmiBO66Fu8=;\n\tb=MmoCFh6BOqb8ko\n\tOhF0hnOHqfFXHft10En/Ks3BVsa/uh3UKWky33ycaPMcnq88uB/ur0vFMw9Kwb7dh2S86y9xYqhEX\n\tALznDY0bfg50u/SVzy0wJKJwM/OyxbqkQenJgVCoUaRWuf6o1RIHxeJB14dQiVTYTzj40N4Ranlto\n\tDXaiF9IAxt8lr9FiSZwYmiAsHJ5BGoWzj9D55Z81H5i83GcPGAM+5xcpY5aYn7b6nK35LH2F6Rj6w\n\thfdIfmyZo3UMNYDUGIU0sOHNM/LO63e10/Bi0BbSjTfvGipr0bIS9MUO4aUTaCcFZty4nDVwrGe0g\n\t21SZluCDBzgerXgJCv4w==;","Date":"Thu, 21 Sep 2017 16:26:39 +0200","From":"Christoph Hellwig ","To":"Robin Murphy ","Subject":"Re: [PATCH 3/4] iommu/arm-smmu-v3: Use NUMA memory allocations for\n\tstream tables and comamnd queues","Message-ID":"<20170921142639.GA18211@lst.de>","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<20170921085922.11659-4-ganapatrao.kulkarni@cavium.com>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.5.17 (2007-11-01)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170921_072702_211050_50F37141 ","X-CRM114-Status":"GOOD ( 13.12 )","X-Spam-Score":"-1.9 (-)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-1.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,\n\tno trust [213.95.11.211 listed in list.dnswl.org]\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"lorenzo.pieralisi@arm.com, Tomasz.Nowicki@cavium.com, vbabka@suse.cz,\n\tjnair@caviumnetworks.com, joro@8bytes.org, Will.Deacon@arm.com,\n\tlinux-kernel@vger.kernel.org, gklkml16@gmail.com,\n\tRobert.Richter@cavium.com, \n\tlinux-mm@kvack.org, iommu@lists.linux-foundation.org, mhocko@suse.com,\n\thanjun.guo@linaro.org,\n\tGanapatrao Kulkarni , \n\takpm@linux-foundation.org, Christoph Hellwig ,\n\tlinux-arm-kernel@lists.infradead.org,\n\tMarek Szyprowski ","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1773696,"web_url":"http://patchwork.ozlabs.org/comment/1773696/","msgid":"","list_archive_url":null,"date":"2017-09-22T15:33:24","subject":"Re: [PATCH 2/4] numa, iommu/io-pgtable-arm: Use NUMA aware memory\n\tallocation for smmu translation tables","submitter":{"id":72420,"url":"http://patchwork.ozlabs.org/api/people/72420/","name":"Ganapatrao Kulkarni","email":"gklkml16@gmail.com"},"content":"On Thu, Sep 21, 2017 at 4:41 PM, Robin Murphy wrote:\n> On 21/09/17 09:59, Ganapatrao Kulkarni wrote:\n>> function __arm_lpae_alloc_pages is used to allcoated memory for smmu\n>> translation tables. updating function to allocate memory/pages\n>> from the proximity domain of SMMU device.\n>\n> AFAICS, data->pgd_size always works out to a power-of-two number of\n> pages, so I'm not sure why we've ever needed alloc_pages_exact() here. I\n> think we could simply use alloc_pages_node() and drop patch #1.\n\nthanks Robin, i think we can replace with alloc_pages_node.\ni will change as suggested in next version.\n\n>\n> Robin.\n>\n>> Signed-off-by: Ganapatrao Kulkarni \n>> ---\n>> drivers/iommu/io-pgtable-arm.c | 4 +++-\n>> 1 file changed, 3 insertions(+), 1 deletion(-)\n>>\n>> diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c\n>> index e8018a3..f6d01f6 100644\n>> --- a/drivers/iommu/io-pgtable-arm.c\n>> +++ b/drivers/iommu/io-pgtable-arm.c\n>> @@ -215,8 +215,10 @@ static void *__arm_lpae_alloc_pages(size_t size, gfp_t gfp,\n>> {\n>> struct device *dev = cfg->iommu_dev;\n>> dma_addr_t dma;\n>> - void *pages = alloc_pages_exact(size, gfp | __GFP_ZERO);\n>> + void *pages;\n>>\n>> + pages = alloc_pages_exact_nid(dev_to_node(dev), size,\n>> + gfp | __GFP_ZERO);\n>> if (!pages)\n>> return NULL;\n>>\n>>\n>\n\nthanks\nGanapat","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"n0gCwc8v\"; \n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"rg46uGlX\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xzHZd1Gg1z9s7h\n\tfor ;\n\tSat, 23 Sep 2017 01:33:59 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dvPxg-0008J8-9c; Fri, 22 Sep 2017 15:33:52 +0000","from mail-lf0-x242.google.com ([2a00:1450:4010:c07::242])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dvPxc-0008Ev-GT for linux-arm-kernel@lists.infradead.org;\n\tFri, 22 Sep 2017 15:33:50 +0000","by mail-lf0-x242.google.com with SMTP id h80so795128lfe.1\n\tfor ;\n\tFri, 22 Sep 2017 08:33:27 -0700 (PDT)","by 10.25.87.84 with HTTP; Fri, 22 Sep 2017 08:33:24 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:\n\tReferences:In-Reply-To:MIME-Version:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=ls1G8RY/H5mASXSG8E/OJM7Nd7bDBvpzOM0UApO+v3Q=;\n\tb=n0gCwc8vdgt1g+\n\tkxNpVDU8Ow5/vaw15FhNG/taEKwyAxRlvy1pLw94XJyLJR7aZ6mYxLfDx27XY11M+rapY6lrj3xOZ\n\tgZBolvrKESM9HJGFWSsqgKAcxGyCEgRpSEa4VnjsWZ1tCquezExJzqZGIPd9xFiiQRlYYV9yN7aNh\n\tWSQSn08w3oQTcC95nI5e97FuQAbkaNLCCuNiivGzIMN3OlH/m78nFsz9khaVs+52kOOpYIEG3+Eom\n\tmmkwHQgRYFfFPm2v6WmhTUf+DhzYQSW4Q1by06lwl8+T9DxNaMgmGSQNjkdqLeAMn++rcXciNO/IH\n\tuHeIcf/QSpYgCng6ndjw==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=LEiX1RG/PYoSDK1/Mz2/hCiTvZ65J6/fpHyxDMa/ZdI=;\n\tb=rg46uGlXFu1vp0bqx7O6LXm9EvoG5sUCgQLGv2da4m8dYqAbq1E0wAzel94VhboTl/\n\tvwyZyxk6uSv5z49QcpGX2m7de6/oPxAvEenzm6sxZAJA1i5Q3sV6X/XZaSuAixfdU67r\n\tPu66rSq0pzcmBDCMQgqS4U5EajV2OFOT4MICToaN7dPPWGoEV/cip1pSsB+kO3R81zsO\n\tljClcOL8wWhRX+D1SoPI3DSoBtE8F7pYQw3Cm2TNBZahXVmKkgmf24P0exCbX42Ocmtq\n\t0r9nVmd0506lKxvfwNPunS3GrF78ui/6Exo5NsaBIbDe69HvFETijEcxpRgkarzqq4cA\n\tWCrA=="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=LEiX1RG/PYoSDK1/Mz2/hCiTvZ65J6/fpHyxDMa/ZdI=;\n\tb=VjGq5phx9QjHMBsaQwGtcUzp9SjMNwie+gkMdRatLCN4dZ2vw0cFCyUzBk1f0n+ndG\n\tGmvN1ZWoV2ZWE5BNSO1XmAM2ORmyp5T9rsb87M/UPae2UzRl57wiKXfG3D3Mr8FZgsC7\n\tFFZWp6iuG0eyHYsUlRiQPGgnW+cHinUhsE6ebSjJZzr895jV7BChKEnP9SWs9R8Qm184\n\ts6r+5KtVF0xaQBszkJnjqx0YQwMfynE1kS/F9x83AVkInsFO0dAyDDzkgA+aPJPMsSmt\n\tOtYGQKUj+ZKA13/NEObKEDS+ACvRSf9SiwsSamycPnIn8f73ReTFWOp68peD707ldbX3\n\tAWvg==","X-Gm-Message-State":"AHPjjUgk84MdMh/4Qi7lVHKztc1shiV4m+G8mI2k8nztQm3WZGDa3BTu\n\tYKtw8QP9kjESX84jiBrjppQhdysRfAnkotGYsj0=","X-Google-Smtp-Source":"AOwi7QCGDCnvGR5c5s3vOVTEMUoHhH0RETmX5h0ABwROhH5K/WRb4kbnv9An51IVqz+d0XTlTt9G8/bFQpC6WRro5bA=","X-Received":"by 10.46.25.81 with SMTP id p78mr2783304lje.65.1506094405273;\n\tFri, 22 Sep 2017 08:33:25 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<37e3ce0e-717d-156d-fef3-27559aff980e@arm.com>","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<20170921085922.11659-3-ganapatrao.kulkarni@cavium.com>\n\t<37e3ce0e-717d-156d-fef3-27559aff980e@arm.com>","From":"Ganapatrao Kulkarni ","Date":"Fri, 22 Sep 2017 21:03:24 +0530","Message-ID":"","Subject":"Re: [PATCH 2/4] numa, iommu/io-pgtable-arm: Use NUMA aware memory\n\tallocation for smmu translation tables","To":"Robin Murphy ","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170922_083348_772260_0A5D096C ","X-CRM114-Status":"GOOD ( 13.22 )","X-Spam-Score":"-1.7 (-)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-1.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t0.0 HK_RANDOM_FROM From username looks random\n\t0.0 HK_RANDOM_ENVFROM Envelope sender username looks random\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends\n\tin digit (gklkml16[at]gmail.com)\n\t0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail\n\tprovider (gklkml16[at]gmail.com)\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Lorenzo Pieralisi , Tomasz.Nowicki@cavium.com,\n\tvbabka@suse.cz, Joerg Roedel ,\n\tWill Deacon , linux-kernel@vger.kernel.org,\n\tRobert.Richter@cavium.com, linux-mm@kvack.org,\n\tiommu@lists.linux-foundation.org, mhocko@suse.com,\n\tHanjun Guo ,\n\tGanapatrao Kulkarni ,\n\takpm@linux-foundation.org, linux-arm-kernel@lists.infradead.org,\n\tjnair@caviumnetworks.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1773706,"web_url":"http://patchwork.ozlabs.org/comment/1773706/","msgid":"","list_archive_url":null,"date":"2017-09-22T15:44:51","subject":"Re: [PATCH 4/4] iommu/dma,\n\tnuma: Use NUMA aware memory allocations in __iommu_dma_alloc_pages","submitter":{"id":72420,"url":"http://patchwork.ozlabs.org/api/people/72420/","name":"Ganapatrao Kulkarni","email":"gklkml16@gmail.com"},"content":"Hi Robin,\n\n\nOn Thu, Sep 21, 2017 at 5:11 PM, Robin Murphy wrote:\n> On 21/09/17 09:59, Ganapatrao Kulkarni wrote:\n>> Change function __iommu_dma_alloc_pages to allocate memory/pages\n>> for dma from respective device numa node.\n>>\n>> Signed-off-by: Ganapatrao Kulkarni \n>> ---\n>> drivers/iommu/dma-iommu.c | 17 ++++++++++-------\n>> 1 file changed, 10 insertions(+), 7 deletions(-)\n>>\n>> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c\n>> index 9d1cebe..0626b58 100644\n>> --- a/drivers/iommu/dma-iommu.c\n>> +++ b/drivers/iommu/dma-iommu.c\n>> @@ -428,20 +428,21 @@ static void __iommu_dma_free_pages(struct page **pages, int count)\n>> kvfree(pages);\n>> }\n>>\n>> -static struct page **__iommu_dma_alloc_pages(unsigned int count,\n>> - unsigned long order_mask, gfp_t gfp)\n>> +static struct page **__iommu_dma_alloc_pages(struct device *dev,\n>> + unsigned int count, unsigned long order_mask, gfp_t gfp)\n>> {\n>> struct page **pages;\n>> unsigned int i = 0, array_size = count * sizeof(*pages);\n>> + int numa_node = dev_to_node(dev);\n>>\n>> order_mask &= (2U << MAX_ORDER) - 1;\n>> if (!order_mask)\n>> return NULL;\n>>\n>> if (array_size <= PAGE_SIZE)\n>> - pages = kzalloc(array_size, GFP_KERNEL);\n>> + pages = kzalloc_node(array_size, GFP_KERNEL, numa_node);\n>> else\n>> - pages = vzalloc(array_size);\n>> + pages = vzalloc_node(array_size, numa_node);\n>\n> kvzalloc{,_node}() didn't exist when this code was first written, but it\n> does now - since you're touching it you may as well get rid of the whole\n> if-else and array_size local.\n\nthanks, i will update in next version.\n>\n> Further nit: some of the indentation below is a bit messed up.\n\nok, will fix it.\n>\n> Robin.\n>\n>> if (!pages)\n>> return NULL;\n>>\n>> @@ -462,8 +463,9 @@ static struct page **__iommu_dma_alloc_pages(unsigned int count,\n>> unsigned int order = __fls(order_mask);\n>>\n>> order_size = 1U << order;\n>> - page = alloc_pages((order_mask - order_size) ?\n>> - gfp | __GFP_NORETRY : gfp, order);\n>> + page = alloc_pages_node(numa_node,\n>> + (order_mask - order_size) ?\n>> + gfp | __GFP_NORETRY : gfp, order);\n>> if (!page)\n>> continue;\n>> if (!order)\n>> @@ -548,7 +550,8 @@ struct page **iommu_dma_alloc(struct device *dev, size_t size, gfp_t gfp,\n>> alloc_sizes = min_size;\n>>\n>> count = PAGE_ALIGN(size) >> PAGE_SHIFT;\n>> - pages = __iommu_dma_alloc_pages(count, alloc_sizes >> PAGE_SHIFT, gfp);\n>> + pages = __iommu_dma_alloc_pages(dev, count, alloc_sizes >> PAGE_SHIFT,\n>> + gfp);\n>> if (!pages)\n>> return NULL;\n>>\n>>\n>\n\nthanks\nGanapat","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"KAfC4/+m\"; \n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"m6up4bs+\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xzHqr5VyBz9t16\n\tfor ;\n\tSat, 23 Sep 2017 01:45:28 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dvQ8r-0005dS-UD; Fri, 22 Sep 2017 15:45:25 +0000","from mail-lf0-x244.google.com ([2a00:1450:4010:c07::244])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dvQ8j-0004MT-PA for linux-arm-kernel@lists.infradead.org;\n\tFri, 22 Sep 2017 15:45:24 +0000","by mail-lf0-x244.google.com with SMTP id c8so807126lfe.2\n\tfor ;\n\tFri, 22 Sep 2017 08:44:53 -0700 (PDT)","by 10.25.87.84 with HTTP; Fri, 22 Sep 2017 08:44:51 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:\n\tReferences:In-Reply-To:MIME-Version:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=qRw/9wjhCkBmpgiHSGr1o0vcK5OXOnNIBYfWMTbOIzo=;\n\tb=KAfC4/+mgBGLK5\n\t+UY8LE4mBpwi1KS1EHJVNXzfMtbFx11mzgQmE4hfN7pnq+dOPkoQchIpMAWo7+Zj5UXD+MwMz6Tny\n\tLB5MLL/cbwC+acbwkR1SNBmJTrucDPgPNKHdyNE/fVbUMDhbv3Ndfk7pBPUasNs3QTE91219J8Iev\n\tJdbTr9ZJkhWJajcGGGGP4hCiIeEdQYmPPSmQd/2+cEIQKCuHi0MfaSFDs7J/SrhISeYM90ogi9Zym\n\tvfhIeG5U7Ihfl2WhYwb/kW9lnK7DKiwBCFD1nCouOhjz9Kvyupfqsq0eJnRrW2EEB0c6VyO1FUwF+\n\tc/svq98bURQFarypDK3g==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=lrsCKVpg99UeO4/dK6UO7UNLerUIs0RXlYWLmytcXj4=;\n\tb=m6up4bs+0nE9fZJbM2npAzj+Jd/YiwYpIBf4h4VT5tatuCj2pJwvfP3gR5hO5Y9zsf\n\te6tE1u9M7/g62IOlkMidtAiYCPOEGEMxmbcrm6NWGlm4F1jUKfe3SsMRabpjDA/GWXQt\n\tLGa3c9y6bcNbb8KsvI5u9T8BeBJe/39mG2elULSw37g5D+p4N+g+tpSTcBTH9XtbrDAH\n\tq1Xzzj9EXsWLQWDLSavH6bm40P5LUR2OTZyNayV66wmAiXmxHzLIIU3ec7dQEzjLjyzY\n\tKUEx85QZPM8tLGSUYqx+qFXavoUTTmah8KxPWo1IvQaY3WJvAt5oVRFcxJXg0ggQgk9o\n\tAfKw=="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=lrsCKVpg99UeO4/dK6UO7UNLerUIs0RXlYWLmytcXj4=;\n\tb=F1FBfAvphpLIxdXi1AHbpIF07xzoTxGU+8sJW9pizqfglmuVsV2nu3WX7mqM+idnaI\n\tP6iG8RQE300UlWIgfWIojSUZCmNB+kVqxTonn2HEOLUBc5KbJEE/cQ0qWkbE2qtVFF6K\n\tA2ElYpw71I8Z1qgkQnlY0yM+wZwQWeqo9E0kyUOTYU3jlQS8t+uLMo+Y6wjh0S9BXk3w\n\t0vVtow31XsDDaseocIleGuiLvQ72xA59sxvBVhkzdtnM2n2glgcXql+mLM3ep0LgUWWQ\n\tG0UmSTuuVkbfYO6At7mAHwut6o+Q0m3jyDbFvX/DXz59qQmvpHTPa/8z9Mf5ur4VWOAP\n\t379Q==","X-Gm-Message-State":"AHPjjUiqD4a6sE4hVWOOwgz7/tZrgZ0YH9CM4UcDrVRqLTpIS34GFDOh\n\t95wLu9LWsozGDHfjcZy7bjLeACp+Z4RHGNMat7I=","X-Google-Smtp-Source":"AOwi7QAXuGcE7luM8me3xYBJ8BiJZQulZmouTlq2tbK8s/R6lBTMd1+jZU8g5UkoGREHCoQR/z8aUaqZ4ys5tUPV7BM=","X-Received":"by 10.25.17.42 with SMTP id g42mr2301582lfi.183.1506095092051;\n\tFri, 22 Sep 2017 08:44:52 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<9d65676f-e4e8-e0a6-602c-361d83ce83c1@arm.com>","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<20170921085922.11659-5-ganapatrao.kulkarni@cavium.com>\n\t<9d65676f-e4e8-e0a6-602c-361d83ce83c1@arm.com>","From":"Ganapatrao Kulkarni ","Date":"Fri, 22 Sep 2017 21:14:51 +0530","Message-ID":"","Subject":"Re: [PATCH 4/4] iommu/dma,\n\tnuma: Use NUMA aware memory allocations in __iommu_dma_alloc_pages","To":"Robin Murphy ","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170922_084522_215635_9D2D2B0D ","X-CRM114-Status":"GOOD ( 14.13 )","X-Spam-Score":"-1.7 (-)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-1.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,\n\tno\n\ttrust [2a00:1450:4010:c07:0:0:0:244 listed in] [list.dnswl.org]\n\t0.0 HK_RANDOM_FROM From username looks random\n\t0.0 HK_RANDOM_ENVFROM Envelope sender username looks random\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends\n\tin digit (gklkml16[at]gmail.com)\n\t0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail\n\tprovider (gklkml16[at]gmail.com)\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Lorenzo Pieralisi , Tomasz.Nowicki@cavium.com,\n\tvbabka@suse.cz, Joerg Roedel ,\n\tWill Deacon , linux-kernel@vger.kernel.org,\n\tRobert.Richter@cavium.com, linux-mm@kvack.org,\n\tiommu@lists.linux-foundation.org, mhocko@suse.com,\n\tHanjun Guo ,\n\tGanapatrao Kulkarni ,\n\takpm@linux-foundation.org, linux-arm-kernel@lists.infradead.org,\n\tjnair@caviumnetworks.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1775506,"web_url":"http://patchwork.ozlabs.org/comment/1775506/","msgid":"<20170926133525.2h55vfa25gjiu5ts@dhcp22.suse.cz>","list_archive_url":null,"date":"2017-09-26T13:35:25","subject":"Re: [PATCH 1/4] mm: move function alloc_pages_exact_nid out of\n\t__meminit","submitter":{"id":66979,"url":"http://patchwork.ozlabs.org/api/people/66979/","name":"Michal Hocko","email":"mhocko@kernel.org"},"content":"On Thu 21-09-17 14:29:19, Ganapatrao Kulkarni wrote:\n> This function can be used on NUMA systems in place of alloc_pages_exact\n> Adding code to export and to remove __meminit section tagging.\n\nIt is usually better to fold such a change into a patch which adds a new\nuser. Other than that I do not have any objections.\n\n> Signed-off-by: Ganapatrao Kulkarni \n\nAcked-by: Michal Hocko \n\n> ---\n> include/linux/gfp.h | 2 +-\n> mm/page_alloc.c | 3 ++-\n> 2 files changed, 3 insertions(+), 2 deletions(-)\n> \n> diff --git a/include/linux/gfp.h b/include/linux/gfp.h\n> index f780718..a4bd234 100644\n> --- a/include/linux/gfp.h\n> +++ b/include/linux/gfp.h\n> @@ -528,7 +528,7 @@ extern unsigned long get_zeroed_page(gfp_t gfp_mask);\n> \n> void *alloc_pages_exact(size_t size, gfp_t gfp_mask);\n> void free_pages_exact(void *virt, size_t size);\n> -void * __meminit alloc_pages_exact_nid(int nid, size_t size, gfp_t gfp_mask);\n> +void *alloc_pages_exact_nid(int nid, size_t size, gfp_t gfp_mask);\n> \n> #define __get_free_page(gfp_mask) \\\n> \t\t__get_free_pages((gfp_mask), 0)\n> diff --git a/mm/page_alloc.c b/mm/page_alloc.c\n> index c841af8..7975870 100644\n> --- a/mm/page_alloc.c\n> +++ b/mm/page_alloc.c\n> @@ -4442,7 +4442,7 @@ EXPORT_SYMBOL(alloc_pages_exact);\n> * Like alloc_pages_exact(), but try to allocate on node nid first before falling\n> * back.\n> */\n> -void * __meminit alloc_pages_exact_nid(int nid, size_t size, gfp_t gfp_mask)\n> +void *alloc_pages_exact_nid(int nid, size_t size, gfp_t gfp_mask)\n> {\n> \tunsigned int order = get_order(size);\n> \tstruct page *p = alloc_pages_node(nid, gfp_mask, order);\n> @@ -4450,6 +4450,7 @@ void * __meminit alloc_pages_exact_nid(int nid, size_t size, gfp_t gfp_mask)\n> \t\treturn NULL;\n> \treturn make_alloc_exact((unsigned long)page_address(p), order, size);\n> }\n> +EXPORT_SYMBOL(alloc_pages_exact_nid);\n> \n> /**\n> * free_pages_exact - release memory allocated via alloc_pages_exact()\n> -- \n> 2.9.4\n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"A7EByp9X\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y1hmg01mSz9tXq\n\tfor ;\n\tTue, 26 Sep 2017 23:36:01 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dwq1i-0005z5-5a; Tue, 26 Sep 2017 13:35:54 +0000","from mx2.suse.de ([195.135.220.15] helo=mx1.suse.de)\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dwq1d-0005r3-0y for linux-arm-kernel@lists.infradead.org;\n\tTue, 26 Sep 2017 13:35:51 +0000","from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254])\n\tby mx1.suse.de (Postfix) with ESMTP id 59FE8AB5D;\n\tTue, 26 Sep 2017 13:35:26 +0000 (UTC)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:\n\tMessage-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=DYbqNRApEiQ9Q/9mKZ0Z8Xg8VzQWNFYfW6Xf45CYFjg=;\n\tb=A7EByp9XwRqeG2\n\t04MgCvZ3eKeGoq+TScH3lWXk5NsRoC97M/sxnkGwSP4QuJjIrG+lmqYJV7yPwiNF5XhTvBECw0oYM\n\tm+UTKRkv/yg6qDDgqUO1V26sbQKbXtHdVz0O8Hvj+r6L7jsIzhxLIFG+xgTi/ZH46f08V8n2fkTjp\n\t9MZ6JqYb/Q2MUDfBQyhYqHg5bT4oxW5ISqjcfmvbU8HAxN2M24WlNLBb4OmxLSYmn+pA9Nq0ydcev\n\tKQ67/0F7mMeHNFz4hi/6EJW9xYiBscMVhl4t2ddij2Ob4/v/3JS2LENEq6NEUlGP+6xT+XHSwxuu3\n\tDQcHWtxK96qNQbNCjLIQ==;","X-Virus-Scanned":"by amavisd-new at test-mx.suse.de","Date":"Tue, 26 Sep 2017 15:35:25 +0200","From":"Michal Hocko ","To":"Ganapatrao Kulkarni ","Subject":"Re: [PATCH 1/4] mm: move function alloc_pages_exact_nid out of\n\t__meminit","Message-ID":"<20170926133525.2h55vfa25gjiu5ts@dhcp22.suse.cz>","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<20170921085922.11659-2-ganapatrao.kulkarni@cavium.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20170921085922.11659-2-ganapatrao.kulkarni@cavium.com>","User-Agent":"NeoMutt/20170609 (1.8.3)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170926_063549_268767_D39D5FC9 ","X-CRM114-Status":"GOOD ( 12.06 )","X-Spam-Score":"-4.2 (----)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-4.2 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/,\n\tmedium trust [195.135.220.15 listed in list.dnswl.org]\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"lorenzo.pieralisi@arm.com, Tomasz.Nowicki@cavium.com, vbabka@suse.cz,\n\tgklkml16@gmail.com, joro@8bytes.org, Will.Deacon@arm.com,\n\tlinux-kernel@vger.kernel.org, Robert.Richter@cavium.com,\n\tlinux-mm@kvack.org, \n\tiommu@lists.linux-foundation.org, hanjun.guo@linaro.org,\n\takpm@linux-foundation.org, robin.murphy@arm.com,\n\tlinux-arm-kernel@lists.infradead.org, jnair@caviumnetworks.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1777514,"web_url":"http://patchwork.ozlabs.org/comment/1777514/","msgid":"<7e270ffc-963c-c1c7-410c-ff3c2e767984@samsung.com>","list_archive_url":null,"date":"2017-09-29T12:13:50","subject":"Re: [PATCH 3/4] iommu/arm-smmu-v3: Use NUMA memory allocations for\n\tstream tables and comamnd queues","submitter":{"id":4023,"url":"http://patchwork.ozlabs.org/api/people/4023/","name":"Marek Szyprowski","email":"m.szyprowski@samsung.com"},"content":"Hi Robin,\n\nOn 2017-09-21 13:58, Robin Murphy wrote:\n> [+Christoph and Marek]\n>\n> On 21/09/17 09:59, Ganapatrao Kulkarni wrote:\n>> Introduce smmu_alloc_coherent and smmu_free_coherent functions to\n>> allocate/free dma coherent memory from NUMA node associated with SMMU.\n>> Replace all calls of dmam_alloc_coherent with smmu_alloc_coherent\n>> for SMMU stream tables and command queues.\n> This doesn't work - not only do you lose the 'managed' aspect and risk\n> leaking various tables on probe failure or device removal, but more\n> importantly, unless you add DMA syncs around all the CPU accesses to the\n> tables, you lose the critical 'coherent' aspect, and that's a horribly\n> invasive change that I really don't want to make.\n>\n> Christoph, Marek; how reasonable do you think it is to expect\n> dma_alloc_coherent() to be inherently NUMA-aware on NUMA-capable\n> systems? SWIOTLB looks fairly straightforward to fix up (for the simple\n> allocation case; I'm not sure it's even worth it for bounce-buffering),\n> but the likes of CMA might be a little trickier...\n\nI'm not sure if there is any dma-coherent implementation that is NUMA aware.\n\nMaybe author should provide some benchmarks, which show that those \nstructures\nshould be allocated in NUMA-aware way?\n\nOn the other hand it is not that hard to add required dma_sync_* calls \naround\nall the code which updated those tables.\n\n > ...\n\nBest regards","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"cdOW23Fq\"; \n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=infradead.org header.i=@infradead.org\n\theader.b=\"jNzH9wyy\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y3W7k3929z9t3H\n\tfor ;\n\tFri, 29 Sep 2017 22:28:50 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dxuPO-0005Z4-5F; Fri, 29 Sep 2017 12:28:46 +0000","from casper.infradead.org ([2001:8b0:10b:1236::1])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dxuLV-00016l-CU for linux-arm-kernel@bombadil.infradead.org;\n\tFri, 29 Sep 2017 12:24:45 +0000","from mailout2.w1.samsung.com ([210.118.77.12])\n\tby casper.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dxuBO-0007ac-Rq for linux-arm-kernel@lists.infradead.org;\n\tFri, 29 Sep 2017 12:14:21 +0000","from eucas1p1.samsung.com (unknown [182.198.249.206])\n\tby mailout2.w1.samsung.com (KnoxPortal) with ESMTP id\n\t20170929121354euoutp02cfc5f88543fc9f5de3916e5aa1005b41~o1A6RvyBW0647906479euoutp02k;\n\tFri, 29 Sep 2017 12:13:54 +0000 (GMT)","from eusmges4.samsung.com (unknown [203.254.199.244]) by\n\teucas1p2.samsung.com (KnoxPortal) with ESMTP id\n\t20170929121353eucas1p2c98cd5d67a36cf9eb64a5691122d049f~o1A5rMAyP1642616426eucas1p2b;\n\tFri, 29 Sep 2017 12:13:53 +0000 (GMT)","from eucas1p1.samsung.com ( [182.198.249.206]) by\n\teusmges4.samsung.com (EUCPMTA) with SMTP id 77.63.12944.1093EC95;\n\tFri, 29 Sep 2017 13:13:53 +0100 (BST)","from eusmgms1.samsung.com (unknown [182.198.249.179]) by\n\teucas1p1.samsung.com (KnoxPortal) with ESMTP id\n\t20170929121352eucas1p1d2a433655951c075a61af1c83325c09a~o1A4zaGIB1137711377eucas1p1j;\n\tFri, 29 Sep 2017 12:13:52 +0000 (GMT)","from eusync4.samsung.com ( [203.254.199.214]) by\n\teusmgms1.samsung.com (EUCPMTA) with SMTP id AF.C8.18832.0093EC95;\n\tFri, 29 Sep 2017 13:13:52 +0100 (BST)","from [106.116.147.30] by eusync4.samsung.com (Oracle\n\tCommunications Messaging Server 7.0.5.31.0 64bit (built May 5 2014))\n\twith ESMTPA id <0OX10028GJB3FV80@eusync4.samsung.com>;\n\tFri, 29 Sep 2017 13:13:52 +0100 (BST)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:References:In-reply-to:MIME-version:Date:Message-id:\n\tFrom:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=sUNO32RpQWKcGkLgjIZVLUtwTyC+q9yCLyzkMp27tj4=;\n\tb=cdOW23FqoBt2/bsarGX7eL+6W\n\tLJaYa/rErz8l1n7QNHAM4XNNw6+8rf3N8epZJkjgfWyW3edorKmEoRsrssobcFnGw1FrWDoP0qVtA\n\tqlc8REDTjCWNSEZFVThLur5O/DVVK5uOOglaoyefhOO1PXRbTGt1hnBnexLhC3hUsG1ZugWl2QfjA\n\tAD9B8OJVHNpfpx7BybdOKF8MYhUAl/QWrww07P8uziOPZv0bLwuh6MJOofUdX/6G6xx6VS6rLaN3Q\n\tuJE9yF+XvB+aEkI/ZNiu0qg5FtMPkteXAkNzn091BS1SC8IoHAEzJIKkc4GEyH8ZZ1U1DFFz5+WF7\n\tOs9OylKDg==;","v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=infradead.org; s=casper.20170209;\n\th=References:Content-transfer-encoding:\n\tContent-type:In-reply-to:MIME-version:Date:Message-id:From:Cc:To:Subject:\n\tSender:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:\n\tResent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:\n\tList-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;\n\tbh=NYwkVw7Gub9dSpD1SckOb6mtvqpIQVUW2OaXvI0QxsY=;\n\tb=jNzH9wyyQkwVIKxssY4XZ6Dq3L\n\t8uSb7sH3s2qpzSosdmpk0dHKSvnUeaq6eWgt0MChtolNaIK8FdGiuwyf5a8v6OlDRnMVFrw5vQ7oU\n\tYhyTw9TnblsAkL7sbwlYE9NQq1W0FghM89+8BS9rHX1x7m2ZTCmiIxMA1PxINd2vzIFvKdbjTuSkz\n\tBAGkHqf+PlhoYiIX0cyRThUuwdChoIf2OAoqGF971Kj2zcoD8JOWjKFMJjO0gVMRgFJhh7ETXwXM9\n\tqGkcbJ33LmlxJlDCugL3MgxYw1yGpkQVdssHIZ/gUAUIMu+5DDdu2wFjzoiISKlnVF7WrSUW/nVaz\n\trAqn83Xw==;"],"X-AuditID":"cbfec7f4-f79ab6d000003290-a3-59ce3901c99a","Subject":"Re: [PATCH 3/4] iommu/arm-smmu-v3: Use NUMA memory allocations for\n\tstream tables and comamnd queues","To":"Robin Murphy , Ganapatrao Kulkarni\n\t, linux-kernel@vger.kernel.org,\n\tlinux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org,\n\tlinux-mm@kvack.org, Christoph Hellwig ","From":"Marek Szyprowski ","Message-id":"<7e270ffc-963c-c1c7-410c-ff3c2e767984@samsung.com>","Date":"Fri, 29 Sep 2017 14:13:50 +0200","User-Agent":"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-version":"1.0","In-reply-to":"","Content-language":"en-US","X-Brightmail-Tracker":["H4sIAAAAAAAAA02Se0hTcRTH+e3e3Xu3mtzmyl/2EAYa9PBBQr9eZiB1/6jor8RIauhlWtuK\n\tLaVV1DI1t8rVZjhmhakVudnGlOGjDKeplI3sIY1yRVpqZZSP0qTVtqvgf59zzvf8zvkefhQm\n\t/sePpnJVx1m1SqaQEkLc3TntXQc2ejMSvz5dga477ASafHufRJP39DzkbLzIR/dsj3mo8tFm\n\tZPxdTyB9hZNEroE+PnrZfJ1Afvs/Pvo2M0yi96Wp6K9lhERtPwaDKXcnhirOlQI08rMbTxUz\n\tg203eYz9ph0wlpkqnHF2G/lMk7WfZFxjJpJ51/eAYLotMzhTX3OWMb+5C5gWn45gHA2vcaan\n\tsoNkxl0r90bsF27JZhW5+aw6IeWQMMf2Yow85ltwYvhDANeBZoEBCChIJ8Ni8x8ex0vgc7+D\n\tMAAhJaZvA1j2yYlzwTiAP9w12FyH4VI5GWIxfQfAqe/bONEQgAX3h8KFSFoFrT5LuFtC/wXQ\n\tUWjDQgFG63hwqthKhFQEnQQNo4Ywi+gUaBpuDzNOx8IvIwP8EC+mD8ALHh/GaRbBKbMfD7GA\n\t3gzbJy6F8xi9CX4OFPE5joH19tHZfBQ8X+QLbwHpjyQ0Xp0MiqhgkAb7TXLOTiT80tVAcrwc\n\t6kvaZo9hDNopWsOxBUDvqIjj4Nyu3tlZEdDkLse4J0WwpFjMSZjg2Buz19oOO4ZHeNyFCniw\n\t4JWfuAJirPPsWOdZsM6zYJ1noRLgtUDC5mmUclaTHK+RKTV5Knl81lGlCwS/5dNA10QjqO7c\n\t5AE0BaQLRW9WPssQ82X5Gq3SAyCFSSWi1vXeDLEoW6Y9yaqPHlTnKViNByyjcGmUaOv+4gwx\n\tLZcdZ4+w7DFWPVflUYJoHdgS0Z7Z6vpUenmgqiyqrnffhsF0oWAoam2asayu9kS61la+eGnL\n\tE21sYpN/1caKpO3TXXE7Tlt2/tLtsST4el4GUgoTD58iFZMTPSLpbpR1S2U+GEM1pZa/rop7\n\tJ6n0w5xe/ZC/T9uXVv1Nnvmw4tmuM65C07UEvWeptjRgTm6V4pocWdJqTK2R/QcJFyghkgMA\n\tAA==","H4sIAAAAAAAAA+NgFrrKIsWRmVeSWpSXmKPExsVy+t/xa7oMluciDRoP6lvMWb+GzeLr7XXs\n\tFl9XdjJZbNjRzWqxcvVRJosF+60t+r9vZrPonL2B3WLT42usFpd3zWGzuLfmP6vFm98v2C3u\n\t9zlY/J3xkt3i4IcnQKFtx5gtZjf2MVq8/HiCxUHI48nBeUwea+atYfSY8XsRi8eGE/2sHjtn\n\t3WX32PRpErvHnWt72DxOzPjN4rF5Sb3H5BvLGT1232xg81i/5SqLx5kFR9g9Pm+SC+CL4rJJ\n\tSc3JLEst0rdL4MpYfekTe8FN7ooXD/6xNDDu4uxi5OSQEDCR6OqZzg5hi0lcuLeerYuRi0NI\n\tYAmjxLSF95kgnOeMEhN3fGUDqRIWyJOYdXMGC4gtIvCfUeLBrwCQImaBBiaJcz+egI0SEmhi\n\tknjzIx3EZhMwlOh62wXWzCtgJzHpxWEwm0VAVeLVy8esILaoQIzEz0uPWCBqBCV+TL4HZnMK\n\tWEsc/tLDDGIzC5hJfHl5mBXClpfYvOYtVFxcorn1JssERsFZSNpnIWmZhaRlFpKWBYwsqxhF\n\tUkuLc9Nziw31ihNzi0vz0vWS83M3MQIje9uxn5t3MF7aGHyIUYCDUYmH94bc2Ugh1sSy4src\n\tQ4wSHMxKIrz7jM9FCvGmJFZWpRblxxeV5qQWH2KU5mBREuft3bM6UkggPbEkNTs1tSC1CCbL\n\txMEp1cBotrbku2cV2zGGsnm8LJdbmeSj1c0rPNs2/vz6cu/DQH5PgWUftaKddqU3nEj/uYtd\n\t58s5pwtcV17VhTbtvmPzX2lqbFla1/kjVj6ZnJd4zx5dovrNXWhmwd6+5C0T2i8vL70cVXeM\n\txbak6dtLfb9iHv75zh3M5v95AmsMG7jCllx/7c86u1CJpTgj0VCLuag4EQCOEhya6AIAAA=="],"X-CMS-MailID":"20170929121352eucas1p1d2a433655951c075a61af1c83325c09a","X-Msg-Generator":"CA","X-Sender-IP":"182.198.249.179","X-Local-Sender":"=?utf-8?q?Marek_Szyprowski=1BSRPOL-Kernel_=28TP=29=1B?=\n\t=?utf-8?b?7IK87ISx7KCE7J6QG1NlbmlvciBTb2Z0d2FyZSBFbmdpbmVlcg==?=","X-Global-Sender":"=?utf-8?q?Marek_Szyprowski=1BSRPOL-Kernel_=28TP=29=1BSam?=\n\t=?utf-8?q?sung_Electronics=1BSenior_Software_Engineer?=","X-Sender-Code":"=?utf-8?q?C10=1BEHQ=1BC10CD02CD027392?=","CMS-TYPE":"201P","X-CMS-RootMailID":"20170921115811epcas5p49012d8793fc765bb86830bfc377b6d36","X-RootMTR":"20170921115811epcas5p49012d8793fc765bb86830bfc377b6d36","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<20170921085922.11659-4-ganapatrao.kulkarni@cavium.com>\n\t\n\t","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170929_131418_932910_AC0C833D ","X-CRM114-Status":"GOOD ( 13.76 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on casper.infradead.org summary:\n\tContent analysis details: (-6.9 points, 5.0 required)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [210.118.77.12 listed in list.dnswl.org]\n\t-0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3)\n\t[210.118.77.12 listed in wl.mailspike.net]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-0.0 SPF_HELO_PASS SPF: HELO matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.0 RCVD_IN_MSPIKE_WL Mailspike good senders","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"lorenzo.pieralisi@arm.com, joro@8bytes.org, Will.Deacon@arm.com,\n\tgklkml16@gmail.com, Tomasz.Nowicki@cavium.com,\n\tRobert.Richter@cavium.com, \n\tmhocko@suse.com, hanjun.guo@linaro.org, akpm@linux-foundation.org,\n\tvbabka@suse.cz, jnair@caviumnetworks.com","Content-Transfer-Encoding":"7bit","Content-Type":"text/plain; charset=\"us-ascii\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1779832,"web_url":"http://patchwork.ozlabs.org/comment/1779832/","msgid":"","list_archive_url":null,"date":"2017-10-04T13:53:03","subject":"Re: [PATCH 3/4] iommu/arm-smmu-v3: Use NUMA memory allocations for\n\tstream tables and comamnd queues","submitter":{"id":72420,"url":"http://patchwork.ozlabs.org/api/people/72420/","name":"Ganapatrao Kulkarni","email":"gklkml16@gmail.com"},"content":"Hi Robin,\n\n\nOn Thu, Sep 21, 2017 at 5:28 PM, Robin Murphy wrote:\n> [+Christoph and Marek]\n>\n> On 21/09/17 09:59, Ganapatrao Kulkarni wrote:\n>> Introduce smmu_alloc_coherent and smmu_free_coherent functions to\n>> allocate/free dma coherent memory from NUMA node associated with SMMU.\n>> Replace all calls of dmam_alloc_coherent with smmu_alloc_coherent\n>> for SMMU stream tables and command queues.\n>\n> This doesn't work - not only do you lose the 'managed' aspect and risk\n> leaking various tables on probe failure or device removal, but more\n> importantly, unless you add DMA syncs around all the CPU accesses to the\n> tables, you lose the critical 'coherent' aspect, and that's a horribly\n> invasive change that I really don't want to make.\n\nthis implementation is similar to function used to allocate memory for\ntranslation tables.\nwhy do you see it affects to stream tables and not to page tables.\nat runtime, both tables are accessed by SMMU only.\n\nAs said in cover letter, having stream table from respective NUMA node\nis yielding\naround 30% performance!\nplease suggest, if there is any better way to address this issue?\n\n>\n> Christoph, Marek; how reasonable do you think it is to expect\n> dma_alloc_coherent() to be inherently NUMA-aware on NUMA-capable\n> systems? SWIOTLB looks fairly straightforward to fix up (for the simple\n> allocation case; I'm not sure it's even worth it for bounce-buffering),\n> but the likes of CMA might be a little trickier...\n>\n> Robin.\n>\n>> Signed-off-by: Ganapatrao Kulkarni \n>> ---\n>> drivers/iommu/arm-smmu-v3.c | 57 ++++++++++++++++++++++++++++++++++++++++-----\n>> 1 file changed, 51 insertions(+), 6 deletions(-)\n>>\n>> diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c\n>> index e67ba6c..bc4ba1f 100644\n>> --- a/drivers/iommu/arm-smmu-v3.c\n>> +++ b/drivers/iommu/arm-smmu-v3.c\n>> @@ -1158,6 +1158,50 @@ static void arm_smmu_init_bypass_stes(u64 *strtab, unsigned int nent)\n>> }\n>> }\n>>\n>> +static void *smmu_alloc_coherent(struct arm_smmu_device *smmu, size_t size,\n>> + dma_addr_t *dma_handle, gfp_t gfp)\n>> +{\n>> + struct device *dev = smmu->dev;\n>> + void *pages;\n>> + dma_addr_t dma;\n>> + int numa_node = dev_to_node(dev);\n>> +\n>> + pages = alloc_pages_exact_nid(numa_node, size, gfp | __GFP_ZERO);\n>> + if (!pages)\n>> + return NULL;\n>> +\n>> + if (!(smmu->features & ARM_SMMU_FEAT_COHERENCY)) {\n>> + dma = dma_map_single(dev, pages, size, DMA_TO_DEVICE);\n>> + if (dma_mapping_error(dev, dma))\n>> + goto out_free;\n>> + /*\n>> + * We depend on the SMMU being able to work with any physical\n>> + * address directly, so if the DMA layer suggests otherwise by\n>> + * translating or truncating them, that bodes very badly...\n>> + */\n>> + if (dma != virt_to_phys(pages))\n>> + goto out_unmap;\n>> + }\n>> +\n>> + *dma_handle = (dma_addr_t)virt_to_phys(pages);\n>> + return pages;\n>> +\n>> +out_unmap:\n>> + dev_err(dev, \"Cannot accommodate DMA translation for IOMMU page tables\\n\");\n>> + dma_unmap_single(dev, dma, size, DMA_TO_DEVICE);\n>> +out_free:\n>> + free_pages_exact(pages, size);\n>> + return NULL;\n>> +}\n>> +\n>> +static void smmu_free_coherent(struct arm_smmu_device *smmu, size_t size,\n>> + void *pages, dma_addr_t dma_handle)\n>> +{\n>> + if (!(smmu->features & ARM_SMMU_FEAT_COHERENCY))\n>> + dma_unmap_single(smmu->dev, dma_handle, size, DMA_TO_DEVICE);\n>> + free_pages_exact(pages, size);\n>> +}\n>> +\n>> static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)\n>> {\n>> size_t size;\n>> @@ -1172,7 +1216,7 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)\n>> strtab = &cfg->strtab[(sid >> STRTAB_SPLIT) * STRTAB_L1_DESC_DWORDS];\n>>\n>> desc->span = STRTAB_SPLIT + 1;\n>> - desc->l2ptr = dmam_alloc_coherent(smmu->dev, size, &desc->l2ptr_dma,\n>> + desc->l2ptr = smmu_alloc_coherent(smmu, size, &desc->l2ptr_dma,\n>> GFP_KERNEL | __GFP_ZERO);\n>> if (!desc->l2ptr) {\n>> dev_err(smmu->dev,\n>> @@ -1487,7 +1531,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain)\n>> struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg;\n>>\n>> if (cfg->cdptr) {\n>> - dmam_free_coherent(smmu_domain->smmu->dev,\n>> + smmu_free_coherent(smmu,\n>> CTXDESC_CD_DWORDS << 3,\n>> cfg->cdptr,\n>> cfg->cdptr_dma);\n>> @@ -1515,7 +1559,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain,\n>> if (asid < 0)\n>> return asid;\n>>\n>> - cfg->cdptr = dmam_alloc_coherent(smmu->dev, CTXDESC_CD_DWORDS << 3,\n>> + cfg->cdptr = smmu_alloc_coherent(smmu, CTXDESC_CD_DWORDS << 3,\n>> &cfg->cdptr_dma,\n>> GFP_KERNEL | __GFP_ZERO);\n>> if (!cfg->cdptr) {\n>> @@ -1984,7 +2028,7 @@ static int arm_smmu_init_one_queue(struct arm_smmu_device *smmu,\n>> {\n>> size_t qsz = ((1 << q->max_n_shift) * dwords) << 3;\n>>\n>> - q->base = dmam_alloc_coherent(smmu->dev, qsz, &q->base_dma, GFP_KERNEL);\n>> + q->base = smmu_alloc_coherent(smmu, qsz, &q->base_dma, GFP_KERNEL);\n>> if (!q->base) {\n>> dev_err(smmu->dev, \"failed to allocate queue (0x%zx bytes)\\n\",\n>> qsz);\n>> @@ -2069,7 +2113,7 @@ static int arm_smmu_init_strtab_2lvl(struct arm_smmu_device *smmu)\n>> size, smmu->sid_bits);\n>>\n>> l1size = cfg->num_l1_ents * (STRTAB_L1_DESC_DWORDS << 3);\n>> - strtab = dmam_alloc_coherent(smmu->dev, l1size, &cfg->strtab_dma,\n>> + strtab = smmu_alloc_coherent(smmu, l1size, &cfg->strtab_dma,\n>> GFP_KERNEL | __GFP_ZERO);\n>> if (!strtab) {\n>> dev_err(smmu->dev,\n>> @@ -2097,8 +2141,9 @@ static int arm_smmu_init_strtab_linear(struct arm_smmu_device *smmu)\n>> u32 size;\n>> struct arm_smmu_strtab_cfg *cfg = &smmu->strtab_cfg;\n>>\n>> +\n>> size = (1 << smmu->sid_bits) * (STRTAB_STE_DWORDS << 3);\n>> - strtab = dmam_alloc_coherent(smmu->dev, size, &cfg->strtab_dma,\n>> + strtab = smmu_alloc_coherent(smmu, size, &cfg->strtab_dma,\n>> GFP_KERNEL | __GFP_ZERO);\n>> if (!strtab) {\n>> dev_err(smmu->dev,\n>>\n>\n\nthanks\nGanapat","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"uXjjtVfV\"; \n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"hB5Lqjnu\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y6cnZ4TnRz9s7M\n\tfor ;\n\tThu, 5 Oct 2017 00:53:54 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzk7S-0002uh-Q7; Wed, 04 Oct 2017 13:53:50 +0000","from mail-lf0-x243.google.com ([2a00:1450:4010:c07::243])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dzk75-0002RU-1c for linux-arm-kernel@lists.infradead.org;\n\tWed, 04 Oct 2017 13:53:37 +0000","by mail-lf0-x243.google.com with SMTP id 90so5305283lfs.13\n\tfor ;\n\tWed, 04 Oct 2017 06:53:05 -0700 (PDT)","by 10.25.147.207 with HTTP; Wed, 4 Oct 2017 06:53:03 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:\n\tReferences:In-Reply-To:MIME-Version:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=LpbyUbjmmNUaJVA0vv3nbDQCQZaPSZOKwlf+UnT2ppE=;\n\tb=uXjjtVfVGleg/u\n\tSB5kJHqD4D0RwOOxWyzr2YqL4g4WqZ+Vnk3avxPUy9dGnplsujYeA1OMBKjTww2G4JZfV6ti2pMq0\n\t6kkkQ/2imsuf8QGar8rRENoepwrQPFmBUiGkahtmLbMxbIqA05Puk0YWWaJ3l7YKLNyg1864vE0oz\n\taIQ6aQaa9a0oRxeqIppdxJdoieqtw/YhUfhqyWZrIFd5jYjeRmrk4LB5JGdRvgU+3owaWtQeFCo46\n\tYP1OcTTWo3naPFFmh+hxZh4/IOVdVeNDgiEZWF5eYsPvr3U6/tPoAXQPjV53TmRaKpqrMWQua5Wvh\n\tEhyNBOzNPP/WDIaj+MFA==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=+Do70G+1DSt/jq/aAAwKpvEQkQfvb8HRrsRy5ZLUZpA=;\n\tb=hB5LqjnuBKUdACVlxzt7ON5iukvFCUKZuI+qrO2GYNZ1XUO1UTUQVq0VAB0spRxiCa\n\t5V7wNgigotW3hOmRHkKWaM5Wjx1/M4xkPapYc8suZWeSfdqIyRI2e4y173npb0wGy9va\n\tba0lVtrevSrn9HXfjPGPNG82BZebsDBOi1qJHfJtkJQhmG/o9CJXvQ9HBgeBUATxO8gn\n\t8tWP48VMBsnrcC6stcA+QOW6+vR9X6TyUWBbcr2HBFt3P1vYws8VZMVRJupA2kGeU32E\n\tPyCGaFDvJbg4bXOgISG5vXViUiZIGyQdPATgv409v5WqfLvqnj/xkScMnE63nWkgCUmR\n\tS8pg=="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=+Do70G+1DSt/jq/aAAwKpvEQkQfvb8HRrsRy5ZLUZpA=;\n\tb=JC40ca4LwQ1mASRJLH1IbyNOlCKgHh7JiTpt5kc6viXJCTyeyOs51kca9J7VW4piDu\n\tA+9I3UWsB84ZsMx1oHBedxQ3CPV3u1kYv/4ZmLqQK2PUOMfiUy98QnHSwD7ao86KAGYO\n\tSAdnhs3Xv2JksnqbWSXJ1bPmdSpOy0SHrlon8CX0sYJYLd877IJofpU7JsDd2AKEMGJm\n\tr8bU2f2ex3uAGybVNd1QgR1ud+zRrNvgdZC3TuJO/BI2KUPOoNHnTkyifitAOoNYG2/w\n\tvhJYo/th6zcyZFzPHA0wPHnEMVpJH6nIiOSruv4o0w/l1CDk77y+a2NtRIBx1IktqYdk\n\tiZ3A==","X-Gm-Message-State":"AMCzsaUnPeVS9ZItLnTb/aHoAXKov0NNEwZzaL/pIxrk4vWoORe72NWP\n\te84t9+yeVYBoKopLnNkLo1fhxzfqfyHjGwYQfr8=","X-Google-Smtp-Source":"AOwi7QAEbm+PCsxwTSGtGGtHNkF0It0c6PQ6Y+JuQyVt22YYqEZ8/cr2AtU3r2HFUG6nV2/tAwpMUcTg0dkpfL8I460=","X-Received":"by 10.25.99.65 with SMTP id x62mr4183957lfb.129.1507125184273;\n\tWed, 04 Oct 2017 06:53:04 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<20170921085922.11659-4-ganapatrao.kulkarni@cavium.com>\n\t","From":"Ganapatrao Kulkarni ","Date":"Wed, 4 Oct 2017 19:23:03 +0530","Message-ID":"","Subject":"Re: [PATCH 3/4] iommu/arm-smmu-v3: Use NUMA memory allocations for\n\tstream tables and comamnd queues","To":"Robin Murphy ","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171004_065327_937626_D73C2BA1 ","X-CRM114-Status":"GOOD ( 21.93 )","X-Spam-Score":"-1.7 (-)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-1.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t0.0 HK_RANDOM_FROM From username looks random\n\t0.0 HK_RANDOM_ENVFROM Envelope sender username looks random\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends\n\tin digit (gklkml16[at]gmail.com)\n\t0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail\n\tprovider (gklkml16[at]gmail.com)\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Lorenzo Pieralisi , Tomasz.Nowicki@cavium.com,\n\tvbabka@suse.cz, jnair@caviumnetworks.com,\n\tJoerg Roedel , \n\tWill Deacon , linux-kernel@vger.kernel.org,\n\tRobert.Richter@cavium.com, linux-mm@kvack.org,\n\tiommu@lists.linux-foundation.org, mhocko@suse.com,\n\tHanjun Guo ,\n\tGanapatrao Kulkarni ,\n\takpm@linux-foundation.org, Christoph Hellwig ,\n\tlinux-arm-kernel@lists.infradead.org,\n\tMarek Szyprowski ","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1789569,"web_url":"http://patchwork.ozlabs.org/comment/1789569/","msgid":"<20171018132759.GA21820@arm.com>","list_archive_url":null,"date":"2017-10-18T13:28:00","subject":"Re: [PATCH 0/4] numa, iommu/smmu: IOMMU/SMMU driver optimization for\n\tNUMA systems","submitter":{"id":7916,"url":"http://patchwork.ozlabs.org/api/people/7916/","name":"Will Deacon","email":"will.deacon@arm.com"},"content":"Hi Ganapat,\n\nOn Thu, Sep 21, 2017 at 02:29:18PM +0530, Ganapatrao Kulkarni wrote:\n> Adding numa aware memory allocations used for iommu dma allocation and\n> memory allocated for SMMU stream tables, page walk tables and command queues.\n> \n> With this patch, iperf testing on ThunderX2, with 40G NIC card on\n> NODE 1 PCI shown same performance(around 30% improvement) as NODE 0.\n\nAre you planning to repost this series? The idea looks good, but it needs\nsome rework before it can be merged.\n\nThanks,\n\nWill","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"KRTw7zNW\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yHCYm6p87z9t4c\n\tfor ;\n\tThu, 19 Oct 2017 00:28:28 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4oOQ-0005zn-5K; Wed, 18 Oct 2017 13:28:18 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4oOM-0005yG-SQ for linux-arm-kernel@lists.infradead.org;\n\tWed, 18 Oct 2017 13:28:16 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9AB4EF;\n\tWed, 18 Oct 2017 06:27:54 -0700 (PDT)","from edgewater-inn.cambridge.arm.com\n\t(usa-sjc-imap-foss1.foss.arm.com [10.72.51.249])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id\n\t6B2E73F483; Wed, 18 Oct 2017 06:27:54 -0700 (PDT)","by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000)\n\tid 18C5B1AE3BC9; Wed, 18 Oct 2017 14:28:00 +0100 (BST)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:\n\tMessage-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=tQkNcvJPIJGut6e8d3y3Eiw/4eZJMc1c/0xhr69B2YI=;\n\tb=KRTw7zNWv2XqYo\n\tqrOuD3Re1CyK6tuR4R6qDkk0wRaWsKwk1o9SkQxnkPQrGRBmqT3/z7BLxxiBIyf8hzbk02752eimr\n\tiXRj2BDzmNE+APi4WCU4SbXrtISv4uOh9BfpcWKaxb4DsE6FzWYmb8BBP9xXHwihonDFSt1SkVo1T\n\titnPu74Wl3lBsXpu5989j1a/JYFWJNWNYlcPJEGoF/PZ99W1zrT8tYnRbGQXDKuyIjgZsAn9oow+r\n\tLvXMaDfdoQIOcASLRHHFS1Qrzj7IbAgSpB42iUCMa/ODo4qxOr8ipkjP51NEQYoqJwTo0aMWL6El5\n\tdHu7YBlhvnET3Aqo//dA==;","Date":"Wed, 18 Oct 2017 14:28:00 +0100","From":"Will Deacon ","To":"Ganapatrao Kulkarni ","Subject":"Re: [PATCH 0/4] numa, iommu/smmu: IOMMU/SMMU driver optimization for\n\tNUMA systems","Message-ID":"<20171018132759.GA21820@arm.com>","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>","User-Agent":"Mutt/1.5.23 (2014-03-12)","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171018_062814_925012_299A29DC ","X-CRM114-Status":"UNSURE ( 7.36 )","X-CRM114-Notice":"Please train this message.","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"lorenzo.pieralisi@arm.com, Tomasz.Nowicki@cavium.com, vbabka@suse.cz,\n\tgklkml16@gmail.com, joro@8bytes.org, linux-kernel@vger.kernel.org,\n\tRobert.Richter@cavium.com, linux-mm@kvack.org,\n\tiommu@lists.linux-foundation.org, mhocko@suse.com,\n\thanjun.guo@linaro.org, akpm@linux-foundation.org, robin.murphy@arm.com,\n\tlinux-arm-kernel@lists.infradead.org, jnair@caviumnetworks.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1789586,"web_url":"http://patchwork.ozlabs.org/comment/1789586/","msgid":"","list_archive_url":null,"date":"2017-10-18T13:36:03","subject":"Re: [PATCH 3/4] iommu/arm-smmu-v3: Use NUMA memory allocations for\n\tstream tables and comamnd queues","submitter":{"id":65641,"url":"http://patchwork.ozlabs.org/api/people/65641/","name":"Robin Murphy","email":"robin.murphy@arm.com"},"content":"On 04/10/17 14:53, Ganapatrao Kulkarni wrote:\n> Hi Robin,\n> \n> \n> On Thu, Sep 21, 2017 at 5:28 PM, Robin Murphy wrote:\n>> [+Christoph and Marek]\n>>\n>> On 21/09/17 09:59, Ganapatrao Kulkarni wrote:\n>>> Introduce smmu_alloc_coherent and smmu_free_coherent functions to\n>>> allocate/free dma coherent memory from NUMA node associated with SMMU.\n>>> Replace all calls of dmam_alloc_coherent with smmu_alloc_coherent\n>>> for SMMU stream tables and command queues.\n>>\n>> This doesn't work - not only do you lose the 'managed' aspect and risk\n>> leaking various tables on probe failure or device removal, but more\n>> importantly, unless you add DMA syncs around all the CPU accesses to the\n>> tables, you lose the critical 'coherent' aspect, and that's a horribly\n>> invasive change that I really don't want to make.\n> \n> this implementation is similar to function used to allocate memory for\n> translation tables.\n\nThe concept is similar, yes, and would work if implemented *correctly*\nwith the aforementioned comprehensive and hugely invasive changes. The\nimplementation as presented in this patch, however, is incomplete and\nbadly broken.\n\nBy way of comparison, the io-pgtable implementations contain all the\nnecessary dma_sync_* calls, never relied on devres, and only have one\nDMA direction to worry about (hint: the queues don't all work\nidentically). There are also a couple of practical reasons for using\nstreaming mappings with the DMA == phys restriction there - tracking\nboth the CPU and DMA addresses for each table would significantly\nincrease the memory overhead, and using the cacheable linear map address\nin all cases sidesteps any potential problems with the atomic PTE\nupdates. Neither of those concerns apply to the SMMUv3 data structures,\nwhich are textbook coherent DMA allocations (being tied to the lifetime\nof the device, rather than transient).\n\n> why do you see it affects to stream tables and not to page tables.\n> at runtime, both tables are accessed by SMMU only.\n> \n> As said in cover letter, having stream table from respective NUMA node\n> is yielding\n> around 30% performance!\n> please suggest, if there is any better way to address this issue?\n\nI fully agree that NUMA-aware allocations are a worthwhile thing that we\nwant. I just don't like the idea of going around individual drivers\nreplacing coherent API usage with bodged-up streaming mappings - I\nreally think it's worth making the effort to to tackle it once, in the\nproper place, in a way that benefits all users together.\n\nRobin.\n\n>>\n>> Christoph, Marek; how reasonable do you think it is to expect\n>> dma_alloc_coherent() to be inherently NUMA-aware on NUMA-capable\n>> systems? SWIOTLB looks fairly straightforward to fix up (for the simple\n>> allocation case; I'm not sure it's even worth it for bounce-buffering),\n>> but the likes of CMA might be a little trickier...\n>>\n>> Robin.\n>>\n>>> Signed-off-by: Ganapatrao Kulkarni \n>>> ---\n>>> drivers/iommu/arm-smmu-v3.c | 57 ++++++++++++++++++++++++++++++++++++++++-----\n>>> 1 file changed, 51 insertions(+), 6 deletions(-)\n>>>\n>>> diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c\n>>> index e67ba6c..bc4ba1f 100644\n>>> --- a/drivers/iommu/arm-smmu-v3.c\n>>> +++ b/drivers/iommu/arm-smmu-v3.c\n>>> @@ -1158,6 +1158,50 @@ static void arm_smmu_init_bypass_stes(u64 *strtab, unsigned int nent)\n>>> }\n>>> }\n>>>\n>>> +static void *smmu_alloc_coherent(struct arm_smmu_device *smmu, size_t size,\n>>> + dma_addr_t *dma_handle, gfp_t gfp)\n>>> +{\n>>> + struct device *dev = smmu->dev;\n>>> + void *pages;\n>>> + dma_addr_t dma;\n>>> + int numa_node = dev_to_node(dev);\n>>> +\n>>> + pages = alloc_pages_exact_nid(numa_node, size, gfp | __GFP_ZERO);\n>>> + if (!pages)\n>>> + return NULL;\n>>> +\n>>> + if (!(smmu->features & ARM_SMMU_FEAT_COHERENCY)) {\n>>> + dma = dma_map_single(dev, pages, size, DMA_TO_DEVICE);\n>>> + if (dma_mapping_error(dev, dma))\n>>> + goto out_free;\n>>> + /*\n>>> + * We depend on the SMMU being able to work with any physical\n>>> + * address directly, so if the DMA layer suggests otherwise by\n>>> + * translating or truncating them, that bodes very badly...\n>>> + */\n>>> + if (dma != virt_to_phys(pages))\n>>> + goto out_unmap;\n>>> + }\n>>> +\n>>> + *dma_handle = (dma_addr_t)virt_to_phys(pages);\n>>> + return pages;\n>>> +\n>>> +out_unmap:\n>>> + dev_err(dev, \"Cannot accommodate DMA translation for IOMMU page tables\\n\");\n>>> + dma_unmap_single(dev, dma, size, DMA_TO_DEVICE);\n>>> +out_free:\n>>> + free_pages_exact(pages, size);\n>>> + return NULL;\n>>> +}\n>>> +\n>>> +static void smmu_free_coherent(struct arm_smmu_device *smmu, size_t size,\n>>> + void *pages, dma_addr_t dma_handle)\n>>> +{\n>>> + if (!(smmu->features & ARM_SMMU_FEAT_COHERENCY))\n>>> + dma_unmap_single(smmu->dev, dma_handle, size, DMA_TO_DEVICE);\n>>> + free_pages_exact(pages, size);\n>>> +}\n>>> +\n>>> static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)\n>>> {\n>>> size_t size;\n>>> @@ -1172,7 +1216,7 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)\n>>> strtab = &cfg->strtab[(sid >> STRTAB_SPLIT) * STRTAB_L1_DESC_DWORDS];\n>>>\n>>> desc->span = STRTAB_SPLIT + 1;\n>>> - desc->l2ptr = dmam_alloc_coherent(smmu->dev, size, &desc->l2ptr_dma,\n>>> + desc->l2ptr = smmu_alloc_coherent(smmu, size, &desc->l2ptr_dma,\n>>> GFP_KERNEL | __GFP_ZERO);\n>>> if (!desc->l2ptr) {\n>>> dev_err(smmu->dev,\n>>> @@ -1487,7 +1531,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain)\n>>> struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg;\n>>>\n>>> if (cfg->cdptr) {\n>>> - dmam_free_coherent(smmu_domain->smmu->dev,\n>>> + smmu_free_coherent(smmu,\n>>> CTXDESC_CD_DWORDS << 3,\n>>> cfg->cdptr,\n>>> cfg->cdptr_dma);\n>>> @@ -1515,7 +1559,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain,\n>>> if (asid < 0)\n>>> return asid;\n>>>\n>>> - cfg->cdptr = dmam_alloc_coherent(smmu->dev, CTXDESC_CD_DWORDS << 3,\n>>> + cfg->cdptr = smmu_alloc_coherent(smmu, CTXDESC_CD_DWORDS << 3,\n>>> &cfg->cdptr_dma,\n>>> GFP_KERNEL | __GFP_ZERO);\n>>> if (!cfg->cdptr) {\n>>> @@ -1984,7 +2028,7 @@ static int arm_smmu_init_one_queue(struct arm_smmu_device *smmu,\n>>> {\n>>> size_t qsz = ((1 << q->max_n_shift) * dwords) << 3;\n>>>\n>>> - q->base = dmam_alloc_coherent(smmu->dev, qsz, &q->base_dma, GFP_KERNEL);\n>>> + q->base = smmu_alloc_coherent(smmu, qsz, &q->base_dma, GFP_KERNEL);\n>>> if (!q->base) {\n>>> dev_err(smmu->dev, \"failed to allocate queue (0x%zx bytes)\\n\",\n>>> qsz);\n>>> @@ -2069,7 +2113,7 @@ static int arm_smmu_init_strtab_2lvl(struct arm_smmu_device *smmu)\n>>> size, smmu->sid_bits);\n>>>\n>>> l1size = cfg->num_l1_ents * (STRTAB_L1_DESC_DWORDS << 3);\n>>> - strtab = dmam_alloc_coherent(smmu->dev, l1size, &cfg->strtab_dma,\n>>> + strtab = smmu_alloc_coherent(smmu, l1size, &cfg->strtab_dma,\n>>> GFP_KERNEL | __GFP_ZERO);\n>>> if (!strtab) {\n>>> dev_err(smmu->dev,\n>>> @@ -2097,8 +2141,9 @@ static int arm_smmu_init_strtab_linear(struct arm_smmu_device *smmu)\n>>> u32 size;\n>>> struct arm_smmu_strtab_cfg *cfg = &smmu->strtab_cfg;\n>>>\n>>> +\n>>> size = (1 << smmu->sid_bits) * (STRTAB_STE_DWORDS << 3);\n>>> - strtab = dmam_alloc_coherent(smmu->dev, size, &cfg->strtab_dma,\n>>> + strtab = smmu_alloc_coherent(smmu, size, &cfg->strtab_dma,\n>>> GFP_KERNEL | __GFP_ZERO);\n>>> if (!strtab) {\n>>> dev_err(smmu->dev,\n>>>\n>>\n> \n> thanks\n> Ganapat\n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"PngnjThX\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yHClv2Gf5z9s4s\n\tfor ;\n\tThu, 19 Oct 2017 00:37:15 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4oWy-00057f-CT; Wed, 18 Oct 2017 13:37:08 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1e4oWL-0004Ty-J0 for linux-arm-kernel@lists.infradead.org;\n\tWed, 18 Oct 2017 13:36:39 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A709CF;\n\tWed, 18 Oct 2017 06:36:08 -0700 (PDT)","from [10.1.210.88] (e110467-lin.cambridge.arm.com [10.1.210.88])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t2A4CA3F483; Wed, 18 Oct 2017 06:36:05 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:References:To:Subject:From:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=GfZmlnbPtw36x0AzSXsWNrAF1o6EpF0hYdkYwHJ83zg=;\n\tb=PngnjThXBs69qo\n\t4fqlwBH7POp8WKf+tjhlvpxW7DY7ciDKywnc+P0lXgA0ezrSfans5rXEEFtiIl0ufSzxby402w7WP\n\tGUNIDZzY/gYS/Orrhys4XeWF4qv1HkkzKQ1VHpUSANTePxRWEMMXH2X6lIkw9R3TPEzrhz/h1mr16\n\tkguBnt8vzirNLK9QLaNNbUWCMuYcSMFHQu5HNLh6hOdquTyzArxDMgPmLe8t+NBVELhWhOyghlS2H\n\tpv6+j4jzOD4U0fsLbWVhH8N4x4kyXsWsA6yJFU5NTOS5MxPqIt5NW7vWIPYD+M5OTOBhFo0etmDdF\n\tt5Aft3wDxnUQbTkTn5SA==;","From":"Robin Murphy ","Subject":"Re: [PATCH 3/4] iommu/arm-smmu-v3: Use NUMA memory allocations for\n\tstream tables and comamnd queues","To":"Ganapatrao Kulkarni ","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<20170921085922.11659-4-ganapatrao.kulkarni@cavium.com>\n\t\n\t","Message-ID":"","Date":"Wed, 18 Oct 2017 14:36:03 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.4.0","MIME-Version":"1.0","In-Reply-To":"","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171018_063630_421899_B8800CB0 ","X-CRM114-Status":"GOOD ( 25.25 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Lorenzo Pieralisi , Tomasz.Nowicki@cavium.com,\n\tvbabka@suse.cz, jnair@caviumnetworks.com,\n\tJoerg Roedel , \n\tWill Deacon , linux-kernel@vger.kernel.org,\n\tRobert.Richter@cavium.com, linux-mm@kvack.org,\n\tiommu@lists.linux-foundation.org, mhocko@suse.com,\n\tHanjun Guo ,\n\tGanapatrao Kulkarni ,\n\takpm@linux-foundation.org, Christoph Hellwig ,\n\tlinux-arm-kernel@lists.infradead.org,\n\tMarek Szyprowski ","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1799517,"web_url":"http://patchwork.ozlabs.org/comment/1799517/","msgid":"","list_archive_url":null,"date":"2017-11-06T09:04:52","subject":"Re: [PATCH 3/4] iommu/arm-smmu-v3: Use NUMA memory allocations for\n\tstream tables and comamnd queues","submitter":{"id":72420,"url":"http://patchwork.ozlabs.org/api/people/72420/","name":"Ganapatrao Kulkarni","email":"gklkml16@gmail.com"},"content":"On Wed, Oct 18, 2017 at 7:06 PM, Robin Murphy wrote:\n> On 04/10/17 14:53, Ganapatrao Kulkarni wrote:\n>> Hi Robin,\n>>\n>>\n>> On Thu, Sep 21, 2017 at 5:28 PM, Robin Murphy wrote:\n>>> [+Christoph and Marek]\n>>>\n>>> On 21/09/17 09:59, Ganapatrao Kulkarni wrote:\n>>>> Introduce smmu_alloc_coherent and smmu_free_coherent functions to\n>>>> allocate/free dma coherent memory from NUMA node associated with SMMU.\n>>>> Replace all calls of dmam_alloc_coherent with smmu_alloc_coherent\n>>>> for SMMU stream tables and command queues.\n>>>\n>>> This doesn't work - not only do you lose the 'managed' aspect and risk\n>>> leaking various tables on probe failure or device removal, but more\n>>> importantly, unless you add DMA syncs around all the CPU accesses to the\n>>> tables, you lose the critical 'coherent' aspect, and that's a horribly\n>>> invasive change that I really don't want to make.\n>>\n>> this implementation is similar to function used to allocate memory for\n>> translation tables.\n>\n> The concept is similar, yes, and would work if implemented *correctly*\n> with the aforementioned comprehensive and hugely invasive changes. The\n> implementation as presented in this patch, however, is incomplete and\n> badly broken.\n>\n> By way of comparison, the io-pgtable implementations contain all the\n> necessary dma_sync_* calls, never relied on devres, and only have one\n> DMA direction to worry about (hint: the queues don't all work\n> identically). There are also a couple of practical reasons for using\n> streaming mappings with the DMA == phys restriction there - tracking\n> both the CPU and DMA addresses for each table would significantly\n> increase the memory overhead, and using the cacheable linear map address\n> in all cases sidesteps any potential problems with the atomic PTE\n> updates. Neither of those concerns apply to the SMMUv3 data structures,\n> which are textbook coherent DMA allocations (being tied to the lifetime\n> of the device, rather than transient).\n>\n>> why do you see it affects to stream tables and not to page tables.\n>> at runtime, both tables are accessed by SMMU only.\n>>\n>> As said in cover letter, having stream table from respective NUMA node\n>> is yielding\n>> around 30% performance!\n>> please suggest, if there is any better way to address this issue?\n>\n> I fully agree that NUMA-aware allocations are a worthwhile thing that we\n> want. I just don't like the idea of going around individual drivers\n> replacing coherent API usage with bodged-up streaming mappings - I\n> really think it's worth making the effort to to tackle it once, in the\n> proper place, in a way that benefits all users together.\n>\n> Robin.\n>\n>>>\n>>> Christoph, Marek; how reasonable do you think it is to expect\n>>> dma_alloc_coherent() to be inherently NUMA-aware on NUMA-capable\n>>> systems? SWIOTLB looks fairly straightforward to fix up (for the simple\n>>> allocation case; I'm not sure it's even worth it for bounce-buffering),\n>>> but the likes of CMA might be a little trickier...\n\nIIUC, having DMA allocation per node may become issue for 32 bit PCI\ndevices connected on NODE 1 on IOMMU less platforms.\nmost of the platforms may have NODE 1 RAM located beyond 4GB and\nhaving DMA allocation beyond 32bit for NODE1(and above) devices may\nmake 32 bit pci devices not usable.\n\nDMA/IOMMU experts, please advise?\n\n>>>\n>>> Robin.\n>>>\n>>>> Signed-off-by: Ganapatrao Kulkarni \n>>>> ---\n>>>> drivers/iommu/arm-smmu-v3.c | 57 ++++++++++++++++++++++++++++++++++++++++-----\n>>>> 1 file changed, 51 insertions(+), 6 deletions(-)\n>>>>\n>>>> diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c\n>>>> index e67ba6c..bc4ba1f 100644\n>>>> --- a/drivers/iommu/arm-smmu-v3.c\n>>>> +++ b/drivers/iommu/arm-smmu-v3.c\n>>>> @@ -1158,6 +1158,50 @@ static void arm_smmu_init_bypass_stes(u64 *strtab, unsigned int nent)\n>>>> }\n>>>> }\n>>>>\n>>>> +static void *smmu_alloc_coherent(struct arm_smmu_device *smmu, size_t size,\n>>>> + dma_addr_t *dma_handle, gfp_t gfp)\n>>>> +{\n>>>> + struct device *dev = smmu->dev;\n>>>> + void *pages;\n>>>> + dma_addr_t dma;\n>>>> + int numa_node = dev_to_node(dev);\n>>>> +\n>>>> + pages = alloc_pages_exact_nid(numa_node, size, gfp | __GFP_ZERO);\n>>>> + if (!pages)\n>>>> + return NULL;\n>>>> +\n>>>> + if (!(smmu->features & ARM_SMMU_FEAT_COHERENCY)) {\n>>>> + dma = dma_map_single(dev, pages, size, DMA_TO_DEVICE);\n>>>> + if (dma_mapping_error(dev, dma))\n>>>> + goto out_free;\n>>>> + /*\n>>>> + * We depend on the SMMU being able to work with any physical\n>>>> + * address directly, so if the DMA layer suggests otherwise by\n>>>> + * translating or truncating them, that bodes very badly...\n>>>> + */\n>>>> + if (dma != virt_to_phys(pages))\n>>>> + goto out_unmap;\n>>>> + }\n>>>> +\n>>>> + *dma_handle = (dma_addr_t)virt_to_phys(pages);\n>>>> + return pages;\n>>>> +\n>>>> +out_unmap:\n>>>> + dev_err(dev, \"Cannot accommodate DMA translation for IOMMU page tables\\n\");\n>>>> + dma_unmap_single(dev, dma, size, DMA_TO_DEVICE);\n>>>> +out_free:\n>>>> + free_pages_exact(pages, size);\n>>>> + return NULL;\n>>>> +}\n>>>> +\n>>>> +static void smmu_free_coherent(struct arm_smmu_device *smmu, size_t size,\n>>>> + void *pages, dma_addr_t dma_handle)\n>>>> +{\n>>>> + if (!(smmu->features & ARM_SMMU_FEAT_COHERENCY))\n>>>> + dma_unmap_single(smmu->dev, dma_handle, size, DMA_TO_DEVICE);\n>>>> + free_pages_exact(pages, size);\n>>>> +}\n>>>> +\n>>>> static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)\n>>>> {\n>>>> size_t size;\n>>>> @@ -1172,7 +1216,7 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid)\n>>>> strtab = &cfg->strtab[(sid >> STRTAB_SPLIT) * STRTAB_L1_DESC_DWORDS];\n>>>>\n>>>> desc->span = STRTAB_SPLIT + 1;\n>>>> - desc->l2ptr = dmam_alloc_coherent(smmu->dev, size, &desc->l2ptr_dma,\n>>>> + desc->l2ptr = smmu_alloc_coherent(smmu, size, &desc->l2ptr_dma,\n>>>> GFP_KERNEL | __GFP_ZERO);\n>>>> if (!desc->l2ptr) {\n>>>> dev_err(smmu->dev,\n>>>> @@ -1487,7 +1531,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain)\n>>>> struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg;\n>>>>\n>>>> if (cfg->cdptr) {\n>>>> - dmam_free_coherent(smmu_domain->smmu->dev,\n>>>> + smmu_free_coherent(smmu,\n>>>> CTXDESC_CD_DWORDS << 3,\n>>>> cfg->cdptr,\n>>>> cfg->cdptr_dma);\n>>>> @@ -1515,7 +1559,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain,\n>>>> if (asid < 0)\n>>>> return asid;\n>>>>\n>>>> - cfg->cdptr = dmam_alloc_coherent(smmu->dev, CTXDESC_CD_DWORDS << 3,\n>>>> + cfg->cdptr = smmu_alloc_coherent(smmu, CTXDESC_CD_DWORDS << 3,\n>>>> &cfg->cdptr_dma,\n>>>> GFP_KERNEL | __GFP_ZERO);\n>>>> if (!cfg->cdptr) {\n>>>> @@ -1984,7 +2028,7 @@ static int arm_smmu_init_one_queue(struct arm_smmu_device *smmu,\n>>>> {\n>>>> size_t qsz = ((1 << q->max_n_shift) * dwords) << 3;\n>>>>\n>>>> - q->base = dmam_alloc_coherent(smmu->dev, qsz, &q->base_dma, GFP_KERNEL);\n>>>> + q->base = smmu_alloc_coherent(smmu, qsz, &q->base_dma, GFP_KERNEL);\n>>>> if (!q->base) {\n>>>> dev_err(smmu->dev, \"failed to allocate queue (0x%zx bytes)\\n\",\n>>>> qsz);\n>>>> @@ -2069,7 +2113,7 @@ static int arm_smmu_init_strtab_2lvl(struct arm_smmu_device *smmu)\n>>>> size, smmu->sid_bits);\n>>>>\n>>>> l1size = cfg->num_l1_ents * (STRTAB_L1_DESC_DWORDS << 3);\n>>>> - strtab = dmam_alloc_coherent(smmu->dev, l1size, &cfg->strtab_dma,\n>>>> + strtab = smmu_alloc_coherent(smmu, l1size, &cfg->strtab_dma,\n>>>> GFP_KERNEL | __GFP_ZERO);\n>>>> if (!strtab) {\n>>>> dev_err(smmu->dev,\n>>>> @@ -2097,8 +2141,9 @@ static int arm_smmu_init_strtab_linear(struct arm_smmu_device *smmu)\n>>>> u32 size;\n>>>> struct arm_smmu_strtab_cfg *cfg = &smmu->strtab_cfg;\n>>>>\n>>>> +\n>>>> size = (1 << smmu->sid_bits) * (STRTAB_STE_DWORDS << 3);\n>>>> - strtab = dmam_alloc_coherent(smmu->dev, size, &cfg->strtab_dma,\n>>>> + strtab = smmu_alloc_coherent(smmu, size, &cfg->strtab_dma,\n>>>> GFP_KERNEL | __GFP_ZERO);\n>>>> if (!strtab) {\n>>>> dev_err(smmu->dev,\n>>>>\n>>>\n>>\n>> thanks\n>> Ganapat\n>>\n>\n\nthanks\nGanapat","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"uvGDaApp\"; \n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"MJI5/KUL\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yVmqW696cz9s4q\n\tfor ;\n\tMon, 6 Nov 2017 20:05:27 +1100 (AEDT)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1eBdLO-0002QR-6A; Mon, 06 Nov 2017 09:05:22 +0000","from mail-lf0-x244.google.com ([2a00:1450:4010:c07::244])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1eBdLI-00014Z-IX for linux-arm-kernel@lists.infradead.org;\n\tMon, 06 Nov 2017 09:05:19 +0000","by mail-lf0-x244.google.com with SMTP id a2so9632420lfh.11\n\tfor ;\n\tMon, 06 Nov 2017 01:04:55 -0800 (PST)","by 10.25.196.198 with HTTP; Mon, 6 Nov 2017 01:04:52 -0800 (PST)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:\n\tReferences:In-Reply-To:MIME-Version:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=T6DFIQFykwePALGDkbqMyB3nhSKWJNierV0OFi1pu8o=;\n\tb=uvGDaApp/fanA3\n\tc5YInMpG3tl5S/hbL6o1zPmV4mlj0nAiA9mv4opzi8gdIMxiBCYlylbGfdQpoHdJhStzYwG91d1lV\n\tNCdw+eN4z0b6rDFAOUMUiHJ2GlDnOxU4Hv4wWMNEPxYDKtIeJNZmQQgn/pd61KjMYF4IjxMru8Xfo\n\t+ole5xAvERcQ/zC6jnxAw4rqMotu9ASsJQB8ZdLsTtHgRr2H5vxmOHR2ozJqBAOjoqIzwgsyBzcgV\n\tfP1vx6I3tMjM1G98Z+PM5CkZgoVxnUE2AhQbksbcKAWcyef9f6rVERdoCJ+ran0YEGCOC70tLGMNj\n\tVwHQxtjf8FoHV4nvnxzA==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=zprEdZ+a1+YrcgpT+V0n1RD9uXei3a3m50osSkmMNdc=;\n\tb=MJI5/KULfrqUu62wqze8KgySsEsQVsis363l0Y4Ol1IOor34g/95t2zThHHKXfZru+\n\tfmaAjSwaTEgrUGsYQg0z3nkR187ViQiMJkFy2nK8UX3hJ2ZF6oh1PMw/lIBZbajgX+8P\n\teNlFCo8sLlp9S5h1ioG5KkA2bwTScxyjJ+BMOdyVg3kt+JZWeTYTJU+6ZhwwlWlysHQn\n\tSG1HAqUV92+qI1QM8TlOLwIGWt6jE7ZPERzmU8xmvshDmNCAMFbDHcz74x+nmUT8rWXe\n\tjpHUMF70tLj5umCKmEaCZVl7Y9OpRwEbfDSj4EAp/P8qPEIL3guqHp8DFLhnC6ATOoS2\n\tCqeA=="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=zprEdZ+a1+YrcgpT+V0n1RD9uXei3a3m50osSkmMNdc=;\n\tb=Iebq9PxqHqT0tHvVyQSAHQSBy/5ciF2+4yrn8AabwPN5MT6d8GM+kkeNMsWX1cwS5y\n\tOF+es86h/hYwdFuRHaiUjVHkGoCreTo1fg06trxCYEirQjd0riX5TTR//bWZrrDpwKzj\n\tATaSRzAa0HzWkNcK11MfiAX2bHvRLIe22jnXtNwZ5vj5F4whDQ2PLeDBe2oDqfq8dEG8\n\t3bZPyRAlIqOUHa/vfBdujk47JPhwID8/gQIm0jvj1kPvvIwpOvKjcMMPWLYWYgvGmBop\n\tmrlqHKP2mI/z+WD10P9yiEt3QbHVKSv+mkcK4Hiqj6fNIcKDEKvVCtzmXT4lHSH21FlG\n\tXjng==","X-Gm-Message-State":"AJaThX6pfdUQ9pF1nxe0Rd47Qkm7jgvsd50GmpElUxVvKBxKoFGWzGhh\n\tyvEg383sfbj8d1vd1aaOshejgfxl5TCk5O/GR7s=","X-Google-Smtp-Source":"ABhQp+TiiolhQyUZVZ588Jjms6MHo/uRv9Y3Z37JeJ51KnBJuVcgrAw70UTE3E6+mmiUI5N8oCD7pl+UO1hSSxzUFU8=","X-Received":"by 10.25.100.18 with SMTP id y18mr6120846lfb.260.1509959093910; \n\tMon, 06 Nov 2017 01:04:53 -0800 (PST)","MIME-Version":"1.0","In-Reply-To":"","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<20170921085922.11659-4-ganapatrao.kulkarni@cavium.com>\n\t\n\t\n\t","From":"Ganapatrao Kulkarni ","Date":"Mon, 6 Nov 2017 14:34:52 +0530","Message-ID":"","Subject":"Re: [PATCH 3/4] iommu/arm-smmu-v3: Use NUMA memory allocations for\n\tstream tables and comamnd queues","To":"Robin Murphy , Will Deacon ","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20171106_010517_011300_327D6D50 ","X-CRM114-Status":"GOOD ( 28.78 )","X-Spam-Score":"-1.7 (-)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-1.7 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,\n\tno\n\ttrust [2a00:1450:4010:c07:0:0:0:244 listed in] [list.dnswl.org]\n\t0.0 HK_RANDOM_FROM From username looks random\n\t0.0 HK_RANDOM_ENVFROM Envelope sender username looks random\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends\n\tin digit (gklkml16[at]gmail.com)\n\t0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail\n\tprovider (gklkml16[at]gmail.com)\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Lorenzo Pieralisi , Tomasz.Nowicki@cavium.com,\n\tvbabka@suse.cz, jnair@caviumnetworks.com,\n\tJoerg Roedel , linux-kernel@vger.kernel.org,\n\tRobert Richter , \n\tlinux-mm@kvack.org, iommu@lists.linux-foundation.org, mhocko@suse.com,\n\tHanjun Guo ,\n\tGanapatrao Kulkarni ,\n\takpm@linux-foundation.org, Christoph Hellwig ,\n\tlinux-arm-kernel@lists.infradead.org,\n\tMarek Szyprowski ","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1977948,"web_url":"http://patchwork.ozlabs.org/comment/1977948/","msgid":"<452f1665-eb3a-5e8c-f671-099ef4a15d84@huawei.com>","list_archive_url":null,"date":"2018-08-22T13:44:33","subject":"Re: [PATCH 0/4] numa, iommu/smmu: IOMMU/SMMU driver optimization for\n\tNUMA systems","submitter":{"id":67406,"url":"http://patchwork.ozlabs.org/api/people/67406/","name":"John Garry","email":"john.garry@huawei.com"},"content":"On 21/09/2017 09:59, Ganapatrao Kulkarni wrote:\n> Adding numa aware memory allocations used for iommu dma allocation and\n> memory allocated for SMMU stream tables, page walk tables and command queues.\n>\n> With this patch, iperf testing on ThunderX2, with 40G NIC card on\n> NODE 1 PCI shown same performance(around 30% improvement) as NODE 0.\n>\n> Ganapatrao Kulkarni (4):\n> mm: move function alloc_pages_exact_nid out of __meminit\n> numa, iommu/io-pgtable-arm: Use NUMA aware memory allocation for smmu\n> translation tables\n> iommu/arm-smmu-v3: Use NUMA memory allocations for stream tables and\n> comamnd queues\n> iommu/dma, numa: Use NUMA aware memory allocations in\n> __iommu_dma_alloc_pages\n>\n> drivers/iommu/arm-smmu-v3.c | 57 +++++++++++++++++++++++++++++++++++++-----\n> drivers/iommu/dma-iommu.c | 17 +++++++------\n> drivers/iommu/io-pgtable-arm.c | 4 ++-\n> include/linux/gfp.h | 2 +-\n> mm/page_alloc.c | 3 ++-\n> 5 files changed, 67 insertions(+), 16 deletions(-)\n>\n\nHi Ganapatrao,\n\nHave you any plans for further work on this patchset? I have not seen \nanything since this v1 was posted+discussed.\n\nThanks,\nJohn\n\n> --\n> 2.9.4\n>\n> _______________________________________________\n> iommu mailing list\n> iommu@lists.linux-foundation.org\n> https://lists.linuxfoundation.org/mailman/listinfo/iommu\n>\n> .\n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org;\n\tdmarc=none (p=none dis=none) header.from=huawei.com","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"naBG7z6O\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[IPv6:2607:7c80:54:e::133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 41wTM520jDz9s5c\n\tfor ;\n\tWed, 22 Aug 2018 23:45:21 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux))\n\tid 1fsTRl-0006RZ-3h; Wed, 22 Aug 2018 13:45:17 +0000","from szxga04-in.huawei.com ([45.249.212.190] helo=huawei.com)\n\tby bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat\n\tLinux)) id 1fsTRh-0005Aj-Sl\n\tfor linux-arm-kernel@lists.infradead.org;\n\tWed, 22 Aug 2018 13:45:15 +0000","from DGGEMS407-HUB.china.huawei.com (unknown [172.30.72.59])\n\tby Forcepoint Email with ESMTP id 7C1293B39CA00;\n\tWed, 22 Aug 2018 21:44:52 +0800 (CST)","from [127.0.0.1] (10.202.226.41) by DGGEMS407-HUB.china.huawei.com\n\t(10.3.19.207) with Microsoft SMTP Server id 14.3.399.0;\n\tWed, 22 Aug 2018 21:44:43 +0800"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=5tt/e8+rSjFYSfhnh9z6zp3FxRJigPmdiHMtmdsspgU=;\n\tb=naBG7z6OjP3uQ10785pt2iksu\n\tbdxr7zSFSOmqT8ziWS9r+VIP16ahBZ6b0EIGttjlCKNX4X5p94YZde47MUM1Ac3cXctnuZbDD0WPr\n\tCaJQsk3PXMpX8wAijIsCvL6mBjbQWElAeEGtl8+/VajvZm2lKchYsnIj+VBVJZ9alPg6E1uK5y5yV\n\tcyLsi3mfz1DdSgXdXIbbrRe3burXECtMQBtjtqqY57AfxTvcJvlk/AFmApoqVzDu/UC/wm3sw8vjX\n\txGYVfFYsIzNCbeMFVD200PII4fH6af4ktEZfl4PoQjopXaNE/fR3Ov8s9fMAJXbwLJIRqhf3SDHN1\n\tBbb68+NlQ==;","Subject":"Re: [PATCH 0/4] numa, iommu/smmu: IOMMU/SMMU driver optimization for\n\tNUMA systems","To":"Ganapatrao Kulkarni ","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>","From":"John Garry ","Message-ID":"<452f1665-eb3a-5e8c-f671-099ef4a15d84@huawei.com>","Date":"Wed, 22 Aug 2018 14:44:33 +0100","User-Agent":"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101\n\tThunderbird/45.3.0","MIME-Version":"1.0","In-Reply-To":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>","X-Originating-IP":"[10.202.226.41]","X-CFilter-Loop":"Reflected","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20180822_064514_145636_E12644D3 ","X-CRM114-Status":"GOOD ( 12.23 )","X-Spam-Score":"-0.0 (/)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-0.0 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,\n\tno trust [45.249.212.190 listed in list.dnswl.org]\n\t-0.0 SPF_HELO_PASS SPF: HELO matches SPF record\n\t-0.0 SPF_PASS SPF: sender matches SPF record","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"mhocko@suse.com\" ,\n\t\"Tomasz.Nowicki@cavium.com\" ,\n\t\"Robert.Richter@cavium.com\" ,\n\t\"Will.Deacon@arm.com\" ,\n\t\"iommu@lists.linux-foundation.org\" ,\n\tLinuxarm ,\n\t\"linux-kernel@vger.kernel.org\" ,\n\t\"linux-mm@kvack.org\" ,\n\t\"gklkml16@gmail.com\" ,\n\t\"vbabka@suse.cz\" , \n\t\"thunder.leizhen@huawei.com\" ,\n\t\"akpm@linux-foundation.org\" ,\n\tMarek Szyprowski ,\n\tRobin Murphy , Christoph Hellwig , \n\t\"linux-arm-kernel@lists.infradead.org\"\n\t, \n\t\"jnair@caviumnetworks.com\" ","Content-Transfer-Encoding":"7bit","Content-Type":"text/plain; charset=\"us-ascii\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1977985,"web_url":"http://patchwork.ozlabs.org/comment/1977985/","msgid":"","list_archive_url":null,"date":"2018-08-22T14:56:43","subject":"Re: [PATCH 0/4] numa, iommu/smmu: IOMMU/SMMU driver optimization for\n\tNUMA systems","submitter":{"id":65641,"url":"http://patchwork.ozlabs.org/api/people/65641/","name":"Robin Murphy","email":"robin.murphy@arm.com"},"content":"Hi John,\n\nOn 22/08/18 14:44, John Garry wrote:\n> On 21/09/2017 09:59, Ganapatrao Kulkarni wrote:\n>> Adding numa aware memory allocations used for iommu dma allocation and\n>> memory allocated for SMMU stream tables, page walk tables and command \n>> queues.\n>>\n>> With this patch, iperf testing on ThunderX2, with 40G NIC card on\n>> NODE 1 PCI shown same performance(around 30% improvement) as NODE 0.\n>>\n>> Ganapatrao Kulkarni (4):\n>>   mm: move function alloc_pages_exact_nid out of __meminit\n>>   numa, iommu/io-pgtable-arm: Use NUMA aware memory allocation for smmu\n>>     translation tables\n>>   iommu/arm-smmu-v3: Use NUMA memory allocations for stream tables and\n>>     comamnd queues\n>>   iommu/dma, numa: Use NUMA aware memory allocations in\n>>     __iommu_dma_alloc_pages\n>>\n>>  drivers/iommu/arm-smmu-v3.c    | 57 \n>> +++++++++++++++++++++++++++++++++++++-----\n>>  drivers/iommu/dma-iommu.c      | 17 +++++++------\n>>  drivers/iommu/io-pgtable-arm.c |  4 ++-\n>>  include/linux/gfp.h            |  2 +-\n>>  mm/page_alloc.c                |  3 ++-\n>>  5 files changed, 67 insertions(+), 16 deletions(-)\n>>\n> \n> Hi Ganapatrao,\n> \n> Have you any plans for further work on this patchset? I have not seen \n> anything since this v1 was posted+discussed.\n\nLooks like I ended up doing the version of the io-pgtable change that I \nsuggested here, which was merged recently (4b123757eeaa). Patch #3 \nshould also be effectively obsolete now since the SWIOTLB/dma-direct \nrework (21f237e4d085). Apparently I also started reworking patch #4 in \nmy tree at some point but sidelined it - I think that was at least \npartly due to another thread[1] which made it seem less clear-cut \nwhether this is always the right thing to do.\n\nRobin.\n\n[1] \nhttps://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1693026.html","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org;\n\tdmarc=none (p=none dis=none) header.from=arm.com","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"LLgNJRhq\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[IPv6:2607:7c80:54:e::133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 41wVy15QLQz9s4Z\n\tfor ;\n\tThu, 23 Aug 2018 00:57:13 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux))\n\tid 1fsUZH-0002Rf-Oj; Wed, 22 Aug 2018 14:57:07 +0000","from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]\n\thelo=foss.arm.com)\n\tby bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux))\n\tid 1fsUZE-0002FU-DN for linux-arm-kernel@lists.infradead.org;\n\tWed, 22 Aug 2018 14:57:05 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 185FDED1;\n\tWed, 22 Aug 2018 07:56:51 -0700 (PDT)","from [10.4.12.131] (e110467-lin.Emea.Arm.com [10.4.12.131])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t3A5963F2EA; Wed, 22 Aug 2018 07:56:48 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:References:\n\tTo:Subject:From:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=A7s+hdkGGeE0JlZ0PBBljTg3mAFLNeRodEyIZrwlxI8=;\n\tb=LLgNJRhq60bUtL83JWTUfP9MX\n\tiygss3mzw6+Pao7+IUIUyje27BrhkJGnroCEj8KgqbiRuX/nutmOMlpoupSqs7gsZGyw2T3TJm4Xt\n\tTns1PExHhk6aLU2m2Dyer338vFxSZ5OdBwVJ2xo56Rhp4rRetnG8p9BI/BF2BXzAZmVZQm1tKWgjX\n\tELS3pDT5xFSX25dvZjyP4MR8jC2ejzTn94DJHiSvr9g/v/UBT+C8IomWlLSp4nwRCMRbvHh/Hc4XF\n\tcs53nN7SnssygJ8IwMHvRtcsRoeOpoeqQA8LWjVrr9YDbApzR8zbiSbAg11Q1bPp8W6JLlDjtTWs7\n\tROcbhvSkA==;","From":"Robin Murphy ","Subject":"Re: [PATCH 0/4] numa, iommu/smmu: IOMMU/SMMU driver optimization for\n\tNUMA systems","To":"John Garry ,\n\tGanapatrao Kulkarni ","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<452f1665-eb3a-5e8c-f671-099ef4a15d84@huawei.com>","Message-ID":"","Date":"Wed, 22 Aug 2018 15:56:43 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.9.1","MIME-Version":"1.0","In-Reply-To":"<452f1665-eb3a-5e8c-f671-099ef4a15d84@huawei.com>","Content-Language":"en-GB","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20180822_075704_452420_304D6F91 ","X-CRM114-Status":"GOOD ( 20.66 )","X-Spam-Score":"-5.0 (-----)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-5.0 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"mhocko@suse.com\" ,\n\t\"Tomasz.Nowicki@cavium.com\" ,\n\t\"Robert.Richter@cavium.com\" ,\n\t\"Will.Deacon@arm.com\" ,\n\t\"iommu@lists.linux-foundation.org\" ,\n\tLinuxarm ,\n\t\"linux-kernel@vger.kernel.org\" ,\n\t\"linux-mm@kvack.org\" ,\n\t\"gklkml16@gmail.com\" ,\n\t\"vbabka@suse.cz\" , \n\t\"thunder.leizhen@huawei.com\" ,\n\t\"akpm@linux-foundation.org\" ,\n\tMarek Szyprowski ,\n\tChristoph Hellwig , \"linux-arm-kernel@lists.infradead.org\"\n\t, \n\t\"jnair@caviumnetworks.com\" ","Content-Transfer-Encoding":"quoted-printable","Content-Type":"text/plain; charset=\"windows-1252\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1978093,"web_url":"http://patchwork.ozlabs.org/comment/1978093/","msgid":"","list_archive_url":null,"date":"2018-08-22T16:07:55","subject":"Re: [PATCH 0/4] numa, iommu/smmu: IOMMU/SMMU driver optimization for\n\tNUMA systems","submitter":{"id":67406,"url":"http://patchwork.ozlabs.org/api/people/67406/","name":"John Garry","email":"john.garry@huawei.com"},"content":"On 22/08/2018 15:56, Robin Murphy wrote:\n> Hi John,\n>\n> On 22/08/18 14:44, John Garry wrote:\n>> On 21/09/2017 09:59, Ganapatrao Kulkarni wrote:\n>>> Adding numa aware memory allocations used for iommu dma allocation and\n>>> memory allocated for SMMU stream tables, page walk tables and command\n>>> queues.\n>>>\n>>> With this patch, iperf testing on ThunderX2, with 40G NIC card on\n>>> NODE 1 PCI shown same performance(around 30% improvement) as NODE 0.\n>>>\n>>> Ganapatrao Kulkarni (4):\n>>> mm: move function alloc_pages_exact_nid out of __meminit\n>>> numa, iommu/io-pgtable-arm: Use NUMA aware memory allocation for smmu\n>>> translation tables\n>>> iommu/arm-smmu-v3: Use NUMA memory allocations for stream tables and\n>>> comamnd queues\n>>> iommu/dma, numa: Use NUMA aware memory allocations in\n>>> __iommu_dma_alloc_pages\n>>>\n>>> drivers/iommu/arm-smmu-v3.c | 57\n>>> +++++++++++++++++++++++++++++++++++++-----\n>>> drivers/iommu/dma-iommu.c | 17 +++++++------\n>>> drivers/iommu/io-pgtable-arm.c | 4 ++-\n>>> include/linux/gfp.h | 2 +-\n>>> mm/page_alloc.c | 3 ++-\n>>> 5 files changed, 67 insertions(+), 16 deletions(-)\n>>>\n>>\n>> Hi Ganapatrao,\n>>\n>> Have you any plans for further work on this patchset? I have not seen\n>> anything since this v1 was posted+discussed.\n>\n\nHi Robin,\n\nThanks for the info. I thought I remembered 4b12 but couldn't put my \nfinger on it.\n\n> Looks like I ended up doing the version of the io-pgtable change that I\n> suggested here, which was merged recently (4b123757eeaa). Patch #3\n> should also be effectively obsolete now since the SWIOTLB/dma-direct\n> rework (21f237e4d085). Apparently I also started reworking patch #4 in\n> my tree at some point but sidelined it - I think that was at least\n> partly due to another thread[1] which made it seem less clear-cut\n> whether this is always the right thing to do.\n\nRight, so #4 seems less straightforward and not directly related to \nIOMMU driver anyway.\n\nCheers,\nJohn\n\n>\n> Robin.\n>\n> [1]\n> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1693026.html\n>\n> .\n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org;\n\tdmarc=none (p=none dis=none) header.from=huawei.com","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"DW7G9Xl8\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[IPv6:2607:7c80:54:e::133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 41wgRZ0lXSz9s0n\n\tfor ;\n\tThu, 23 Aug 2018 07:19:54 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux))\n\tid 1fsVgj-000774-57; Wed, 22 Aug 2018 16:08:53 +0000","from szxga06-in.huawei.com ([45.249.212.32] helo=huawei.com)\n\tby bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat\n\tLinux)) id 1fsVgg-00073r-2d\n\tfor linux-arm-kernel@lists.infradead.org;\n\tWed, 22 Aug 2018 16:08:51 +0000","from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.60])\n\tby Forcepoint Email with ESMTP id 7195968B9F11E;\n\tThu, 23 Aug 2018 00:08:32 +0800 (CST)","from [127.0.0.1] (10.202.226.41) by DGGEMS403-HUB.china.huawei.com\n\t(10.3.19.203) with Microsoft SMTP Server id 14.3.399.0;\n\tThu, 23 Aug 2018 00:08:10 +0800"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=TCB5vzJkYfeXrWxGiAS0fJCRdrBrvJNXz5639FMpKy4=;\n\tb=DW7G9Xl8D7kmQ0ft2DGwJBy9j\n\tvOin/qh88vhLzPBOQEyBi+Y46dFHzlqc/aDxNsS+v9nkSKgMMb21fJjZR1BrM5EQGrWYGqUG+vGCL\n\t715kXiw/LicGtkOyJ1H/3kCFYWp95plgO/FgvLZ4h3YqHkB65cQoGr1/9GSojkFwmS8qvD9DDmg2N\n\tvlKCVEasl0mmW9LysFf+qYNKe+gZ5buUbqjxRVaOrbsjqMepPsDVKvx4JLPQUxhCRMIWVaxpjy5qG\n\t4nLka0wGiyWqPIFO2lL6D8BXRcHPjiWz43/Li8lfHb7UnQt+mRz1cmINuCBadHSW5lwrliX3Qy1gd\n\td5o+EbU+A==;","Subject":"Re: [PATCH 0/4] numa, iommu/smmu: IOMMU/SMMU driver optimization for\n\tNUMA systems","To":"Robin Murphy , Ganapatrao Kulkarni\n\t","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<452f1665-eb3a-5e8c-f671-099ef4a15d84@huawei.com>\n\t","From":"John Garry ","Message-ID":"","Date":"Wed, 22 Aug 2018 17:07:55 +0100","User-Agent":"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101\n\tThunderbird/45.3.0","MIME-Version":"1.0","In-Reply-To":"","X-Originating-IP":"[10.202.226.41]","X-CFilter-Loop":"Reflected","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20180822_090850_329215_88C4D496 ","X-CRM114-Status":"GOOD ( 15.74 )","X-Spam-Score":"-0.0 (/)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-0.0 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 SPF_HELO_PASS SPF: HELO matches SPF record\n\t-0.0 SPF_PASS SPF: sender matches SPF record","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"mhocko@suse.com\" ,\n\t\"Tomasz.Nowicki@cavium.com\" ,\n\t\"Robert.Richter@cavium.com\" ,\n\t\"Will.Deacon@arm.com\" ,\n\t\"iommu@lists.linux-foundation.org\" ,\n\tLinuxarm ,\n\t\"linux-kernel@vger.kernel.org\" ,\n\t\"linux-mm@kvack.org\" ,\n\t\"gklkml16@gmail.com\" ,\n\t\"vbabka@suse.cz\" , \n\t\"thunder.leizhen@huawei.com\" ,\n\t\"akpm@linux-foundation.org\" ,\n\tMarek Szyprowski , Christoph\n\tHellwig , \"linux-arm-kernel@lists.infradead.org\"\n\t, \n\t\"jnair@caviumnetworks.com\" ","Content-Transfer-Encoding":"7bit","Content-Type":"text/plain; charset=\"us-ascii\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1978101,"web_url":"http://patchwork.ozlabs.org/comment/1978101/","msgid":"","list_archive_url":null,"date":"2018-08-22T17:57:07","subject":"Re: [PATCH 0/4] numa, iommu/smmu: IOMMU/SMMU driver optimization for\n\tNUMA systems","submitter":{"id":72420,"url":"http://patchwork.ozlabs.org/api/people/72420/","name":"Ganapatrao Kulkarni","email":"gklkml16@gmail.com"},"content":"On Wed, Aug 22, 2018 at 9:08 AM John Garry wrote:\n>\n> On 22/08/2018 15:56, Robin Murphy wrote:\n> > Hi John,\n> >\n> > On 22/08/18 14:44, John Garry wrote:\n> >> On 21/09/2017 09:59, Ganapatrao Kulkarni wrote:\n> >>> Adding numa aware memory allocations used for iommu dma allocation and\n> >>> memory allocated for SMMU stream tables, page walk tables and command\n> >>> queues.\n> >>>\n> >>> With this patch, iperf testing on ThunderX2, with 40G NIC card on\n> >>> NODE 1 PCI shown same performance(around 30% improvement) as NODE 0.\n> >>>\n> >>> Ganapatrao Kulkarni (4):\n> >>> mm: move function alloc_pages_exact_nid out of __meminit\n> >>> numa, iommu/io-pgtable-arm: Use NUMA aware memory allocation for smmu\n> >>> translation tables\n> >>> iommu/arm-smmu-v3: Use NUMA memory allocations for stream tables and\n> >>> comamnd queues\n> >>> iommu/dma, numa: Use NUMA aware memory allocations in\n> >>> __iommu_dma_alloc_pages\n> >>>\n> >>> drivers/iommu/arm-smmu-v3.c | 57\n> >>> +++++++++++++++++++++++++++++++++++++-----\n> >>> drivers/iommu/dma-iommu.c | 17 +++++++------\n> >>> drivers/iommu/io-pgtable-arm.c | 4 ++-\n> >>> include/linux/gfp.h | 2 +-\n> >>> mm/page_alloc.c | 3 ++-\n> >>> 5 files changed, 67 insertions(+), 16 deletions(-)\n> >>>\n> >>\n> >> Hi Ganapatrao,\n> >>\n> >> Have you any plans for further work on this patchset? I have not seen\n> >> anything since this v1 was posted+discussed.\n> >\n>\n> Hi Robin,\n>\n> Thanks for the info. I thought I remembered 4b12 but couldn't put my\n> finger on it.\n>\n> > Looks like I ended up doing the version of the io-pgtable change that I\n> > suggested here, which was merged recently (4b123757eeaa). Patch #3\n> > should also be effectively obsolete now since the SWIOTLB/dma-direct\n> > rework (21f237e4d085). Apparently I also started reworking patch #4 in\n> > my tree at some point but sidelined it - I think that was at least\n> > partly due to another thread[1] which made it seem less clear-cut\n> > whether this is always the right thing to do.\n>\n> Right, so #4 seems less straightforward and not directly related to\n> IOMMU driver anyway.\n>\n\nthanks Robin for pulling up the patch. I couldn't followup with this\ndue to other tasks.\n\n> Cheers,\n> John\n>\n> >\n> > Robin.\n> >\n> > [1]\n> > https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1693026.html\n> >\n> > .\n> >\n>\n>\nthanks,\nGanapat","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org;\n\tdmarc=fail (p=none dis=none) header.from=gmail.com","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"pL9r1PIT\"; \n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"ozVOMdGL\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[IPv6:2607:7c80:54:e::133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 41wgSJ1mWNz9s3C\n\tfor ;\n\tThu, 23 Aug 2018 07:20:32 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux))\n\tid 1fsXNu-0006Ti-M5; Wed, 22 Aug 2018 17:57:34 +0000","from mail-oi0-x242.google.com ([2607:f8b0:4003:c06::242])\n\tby bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat\n\tLinux)) id 1fsXNr-0006SM-4o\n\tfor linux-arm-kernel@lists.infradead.org;\n\tWed, 22 Aug 2018 17:57:32 +0000","by mail-oi0-x242.google.com with SMTP id 8-v6so4703300oip.0\n\tfor ;\n\tWed, 22 Aug 2018 10:57:20 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:\n\tIn-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description:\n\tResent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=hZchR1K0lZNDTCsAjRT1H1VonAn5HeSSjB4/ZI5oRjI=;\n\tb=pL9r1PITGy5HrB\n\tQvxJKMEqabOYJ1zJwQuQkTc07hjQnlNMv4B9BUUSgiPd5ejQoThp4ahwPh1UtVLDoHx+/flv8bfpu\n\tZlYoupNKcCJJaZiLSpmgtGtvNjHYZO3Hhb1fmSausbzUczJ1ysuKEDEY9LgJ84MySHzwQcmF2kCZr\n\tUXl5YpsRe0wdDtIot9ZdPQNV+65grAeBzcuXsGTkoCOPgqJCNdSHj/xfOLsz1mAcE5cj4Gkytazqo\n\tHGIvQID7966e7UD6l28ApQzKkHOCnasy1LDIlHIAW5K0K7QqZnNWUMoki1KuKLzJjKgQSL7dx48+u\n\tNTgA/MGd4/cqElg0i4uQ==;","v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=j+Duo5Zd3dc2U2bmtFSOK65PK3BsP/GWImhAiwC5ulM=;\n\tb=ozVOMdGLX1CMFwqXsupwbyOx7vnJ2JfyjwrfbGhxcF+XUWBtJ/jJwBRCwiOdNAr/26\n\t90vWlmXbBCNTSUslmkkb43ucBGB3f4lK9Pap7Foclypt68sI2epyQ+TD2nfy4zBfbZde\n\t1AnwLR4sdlVSopCfQ7WCNxtrmAHBCw6evEmEKSq7BGozy0kvWJPf5O7I3COyOo+HUYE4\n\tGUo4qmmoxCwcbX2Kop4EIyWeIbmSKjm4eOMXr7euMjJEtmYninAKL5eIfoW2L9nEDZJv\n\tTA/gPYC4x6okdhOPDH6geIzuEXD6Wll4qjrrTXXjjbQnFgapcI4LDW+wXFJxKAtZdlDR\n\tVo4g=="],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=j+Duo5Zd3dc2U2bmtFSOK65PK3BsP/GWImhAiwC5ulM=;\n\tb=tviILsPA6XzKS9Xxzy7Ynjtq3+mxvnZgmTegeFOi9J5DrMsYpgKqY2lAo8p+YNgM6B\n\tMCLpriiG5rqAc3et4zFVqw2ltB+eDCmT/ZKUyBXRhSoRuwF0+RUkBelqs+WPj8o7+hUN\n\tbmh9kwvnIP7qzk7QyPKdheEPaA5XKHGX5sSru+fFiAjoe49DtJtrhC7BA16Bt8CAI56k\n\t37+x1sxvqEamDOwxdlmhAMcykvC7WsufW2oNWvI6yLo/4VajPKLB4+c8YXYbKPE4dknA\n\t0xSkOOH3XFZr2fDpDQrQyICChLy0qjDVfenHJ6XdVR9b6kfIjeniiCPYSoCHdKNHliXY\n\tVcTQ==","X-Gm-Message-State":"APzg51Dre8iS6TLF+V52ci0AUdtsMIuROxDdcFze42IlVIylLBCWF+gC\n\theHCG1/LN+JiECck/Tc93c4U8eUi1SLFyigjpLA=","X-Google-Smtp-Source":"ANB0VdbL5ZBQahBU/hJM1c/Z1hc6gEtBlK8496R8CJA6ULSHMUBbTNgRk6PkodHyZdc8F7drM6O9B0usWmQ251ufB0M=","X-Received":"by 2002:aca:a12:: with SMTP id\n\t18-v6mr5263336oik.292.1534960639722; \n\tWed, 22 Aug 2018 10:57:19 -0700 (PDT)","MIME-Version":"1.0","References":"<20170921085922.11659-1-ganapatrao.kulkarni@cavium.com>\n\t<452f1665-eb3a-5e8c-f671-099ef4a15d84@huawei.com>\n\t\n\t","In-Reply-To":"","From":"Ganapatrao Kulkarni ","Date":"Wed, 22 Aug 2018 10:57:07 -0700","Message-ID":"","Subject":"Re: [PATCH 0/4] numa, iommu/smmu: IOMMU/SMMU driver optimization for\n\tNUMA systems","To":"John Garry ","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20180822_105731_190342_A11CDBFC ","X-CRM114-Status":"GOOD ( 20.93 )","X-Spam-Score":"0.8 (/)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (0.8 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,\n\tno\n\ttrust [2607:f8b0:4003:c06:0:0:0:242 listed in] [list.dnswl.org]\n\t0.6 HK_RANDOM_ENVFROM Envelope sender username looks random\n\t0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends\n\tin digit (gklkml16[at]gmail.com)\n\t0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail\n\tprovider (gklkml16[at]gmail.com)\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"mhocko@suse.com, Tomasz.Nowicki@cavium.com,\n\t\"Leizhen \\(ThunderTown\\)\" ,\n\tWill Deacon , LKML ,\n\tLinuxarm ,\n\tRobert Richter , \n\tlinux-mm@kvack.org, iommu@lists.linux-foundation.org, vbabka@suse.cz, \n\tGanapatrao Kulkarni ,\n\takpm@linux-foundation.org, Marek Szyprowski ,\n\tRobin Murphy , Christoph Hellwig , \n\tlinux-arm-kernel@lists.infradead.org, jnair@caviumnetworks.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}}]