[{"id":1770280,"web_url":"http://patchwork.ozlabs.org/comment/1770280/","msgid":"<250f41c9-fb37-49b7-c336-68e163f77e16@arm.com>","list_archive_url":null,"date":"2017-09-18T16:02:22","subject":"Re: [PATCH 1/1] iommu/iova: Make rcache flush optional on IOVA\n\tallocation failure","submitter":{"id":65641,"url":"http://patchwork.ozlabs.org/api/people/65641/","name":"Robin Murphy","email":"robin.murphy@arm.com"},"content":"Hi Tomasz,\n\nOn 18/09/17 11:56, Tomasz Nowicki wrote:\n> Since IOVA allocation failure is not unusual case we need to flush\n> CPUs' rcache in hope we will succeed in next round.\n> \n> However, it is useful to decide whether we need rcache flush step because\n> of two reasons:\n> - Not scalability. On large system with ~100 CPUs iterating and flushing\n> rcache for each CPU becomes serious bottleneck so we may want to deffer it.\n> - free_cpu_cached_iovas() does not care about max PFN we are interested in.\n> Thus we may flush our rcaches and still get no new IOVA like in the\n> commonly used scenario:\n> \n> if (dma_limit > DMA_BIT_MASK(32) && dev_is_pci(dev))\n> iova = alloc_iova_fast(iovad, iova_len, DMA_BIT_MASK(32) >> shift);\n> \n> if (!iova)\n> iova = alloc_iova_fast(iovad, iova_len, dma_limit >> shift);\n> \n> 1. First alloc_iova_fast() call is limited to DMA_BIT_MASK(32) to get\n> PCI devices a SAC address\n> 2. alloc_iova() fails due to full 32-bit space\n> 3. rcaches contain PFNs out of 32-bit space so free_cpu_cached_iovas()\n> throws entries away for nothing and alloc_iova() fails again\n> 4. Next alloc_iova_fast() call cannot take advantage of rcache since we\n> have just defeated caches. In this case we pick the slowest option\n> to proceed.\n> \n> This patch reworks flushed_rcache local flag to be additional function\n> argument instead and control rcache flush step. Also, it updates all users\n> to do the flush as the last chance.\n\nLooks like you've run into the same thing Nate found[1] - I came up with\nalmost the exact same patch, only with separate alloc_iova_fast() and\nalloc_iova_fast_noretry() wrapper functions, but on reflection, just\nexposing the bool to callers is probably simpler. One nit, can you\ndocument it in the kerneldoc comment too? With that:\n\nReviewed-by: Robin Murphy \n\nThanks,\nRobin.\n\n[1]:https://www.mail-archive.com/iommu@lists.linux-foundation.org/msg19758.html\n\n> \n> Signed-off-by: Tomasz Nowicki \n> ---\n> drivers/iommu/amd_iommu.c | 5 +++--\n> drivers/iommu/dma-iommu.c | 6 ++++--\n> drivers/iommu/intel-iommu.c | 5 +++--\n> drivers/iommu/iova.c | 7 +++----\n> include/linux/iova.h | 5 +++--\n> 5 files changed, 16 insertions(+), 12 deletions(-)\n> \n> diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c\n> index 8d2ec60..ce68986 100644\n> --- a/drivers/iommu/amd_iommu.c\n> +++ b/drivers/iommu/amd_iommu.c\n> @@ -1604,10 +1604,11 @@ static unsigned long dma_ops_alloc_iova(struct device *dev,\n> \n> \tif (dma_mask > DMA_BIT_MASK(32))\n> \t\tpfn = alloc_iova_fast(&dma_dom->iovad, pages,\n> -\t\t\t\t IOVA_PFN(DMA_BIT_MASK(32)));\n> +\t\t\t\t IOVA_PFN(DMA_BIT_MASK(32)), false);\n> \n> \tif (!pfn)\n> -\t\tpfn = alloc_iova_fast(&dma_dom->iovad, pages, IOVA_PFN(dma_mask));\n> +\t\tpfn = alloc_iova_fast(&dma_dom->iovad, pages,\n> +\t\t\t\t IOVA_PFN(dma_mask), true);\n> \n> \treturn (pfn << PAGE_SHIFT);\n> }\n> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c\n> index 191be9c..25914d3 100644\n> --- a/drivers/iommu/dma-iommu.c\n> +++ b/drivers/iommu/dma-iommu.c\n> @@ -370,10 +370,12 @@ static dma_addr_t iommu_dma_alloc_iova(struct iommu_domain *domain,\n> \n> \t/* Try to get PCI devices a SAC address */\n> \tif (dma_limit > DMA_BIT_MASK(32) && dev_is_pci(dev))\n> -\t\tiova = alloc_iova_fast(iovad, iova_len, DMA_BIT_MASK(32) >> shift);\n> +\t\tiova = alloc_iova_fast(iovad, iova_len,\n> +\t\t\t\t DMA_BIT_MASK(32) >> shift, false);\n> \n> \tif (!iova)\n> -\t\tiova = alloc_iova_fast(iovad, iova_len, dma_limit >> shift);\n> +\t\tiova = alloc_iova_fast(iovad, iova_len, dma_limit >> shift,\n> +\t\t\t\t true);\n> \n> \treturn (dma_addr_t)iova << shift;\n> }\n> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c\n> index 05c0c3a..75c8320 100644\n> --- a/drivers/iommu/intel-iommu.c\n> +++ b/drivers/iommu/intel-iommu.c\n> @@ -3460,11 +3460,12 @@ static unsigned long intel_alloc_iova(struct device *dev,\n> \t\t * from higher range\n> \t\t */\n> \t\tiova_pfn = alloc_iova_fast(&domain->iovad, nrpages,\n> -\t\t\t\t\t IOVA_PFN(DMA_BIT_MASK(32)));\n> +\t\t\t\t\t IOVA_PFN(DMA_BIT_MASK(32)), false);\n> \t\tif (iova_pfn)\n> \t\t\treturn iova_pfn;\n> \t}\n> -\tiova_pfn = alloc_iova_fast(&domain->iovad, nrpages, IOVA_PFN(dma_mask));\n> +\tiova_pfn = alloc_iova_fast(&domain->iovad, nrpages,\n> +\t\t\t\t IOVA_PFN(dma_mask), true);\n> \tif (unlikely(!iova_pfn)) {\n> \t\tpr_err(\"Allocating %ld-page iova for %s failed\",\n> \t\t nrpages, dev_name(dev));\n> diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c\n> index f88acad..1a18b14 100644\n> --- a/drivers/iommu/iova.c\n> +++ b/drivers/iommu/iova.c\n> @@ -358,9 +358,8 @@ EXPORT_SYMBOL_GPL(free_iova);\n> */\n> unsigned long\n> alloc_iova_fast(struct iova_domain *iovad, unsigned long size,\n> -\t\tunsigned long limit_pfn)\n> +\t\tunsigned long limit_pfn, bool flush_rcache)\n> {\n> -\tbool flushed_rcache = false;\n> \tunsigned long iova_pfn;\n> \tstruct iova *new_iova;\n> \n> @@ -373,11 +372,11 @@ alloc_iova_fast(struct iova_domain *iovad, unsigned long size,\n> \tif (!new_iova) {\n> \t\tunsigned int cpu;\n> \n> -\t\tif (flushed_rcache)\n> +\t\tif (!flush_rcache)\n> \t\t\treturn 0;\n> \n> \t\t/* Try replenishing IOVAs by flushing rcache. */\n> -\t\tflushed_rcache = true;\n> +\t\tflush_rcache = false;\n> \t\tfor_each_online_cpu(cpu)\n> \t\t\tfree_cpu_cached_iovas(cpu, iovad);\n> \t\tgoto retry;\n> diff --git a/include/linux/iova.h b/include/linux/iova.h\n> index 58c2a36..8fdcb66 100644\n> --- a/include/linux/iova.h\n> +++ b/include/linux/iova.h\n> @@ -97,7 +97,7 @@ struct iova *alloc_iova(struct iova_domain *iovad, unsigned long size,\n> void free_iova_fast(struct iova_domain *iovad, unsigned long pfn,\n> \t\t unsigned long size);\n> unsigned long alloc_iova_fast(struct iova_domain *iovad, unsigned long size,\n> -\t\t\t unsigned long limit_pfn);\n> +\t\t\t unsigned long limit_pfn, bool flush_rcache);\n> struct iova *reserve_iova(struct iova_domain *iovad, unsigned long pfn_lo,\n> \tunsigned long pfn_hi);\n> void copy_reserved_iova(struct iova_domain *from, struct iova_domain *to);\n> @@ -151,7 +151,8 @@ static inline void free_iova_fast(struct iova_domain *iovad,\n> \n> static inline unsigned long alloc_iova_fast(struct iova_domain *iovad,\n> \t\t\t\t\t unsigned long size,\n> -\t\t\t\t\t unsigned long limit_pfn)\n> +\t\t\t\t\t unsigned long limit_pfn,\n> +\t\t\t\t\t bool flush_rcache)\n> {\n> \treturn 0;\n> }\n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org\n\theader.b=\"sFd99a+N\"; dkim-atps=neutral"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwrPt3ZyTz9s78\n\tfor ;\n\tTue, 19 Sep 2017 02:02:58 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dtyVa-0005G1-Fl; Mon, 18 Sep 2017 16:02:54 +0000","from foss.arm.com ([217.140.101.70])\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1dtyVS-00059v-DL for linux-arm-kernel@lists.infradead.org;\n\tMon, 18 Sep 2017 16:02:52 +0000","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A81881435;\n\tMon, 18 Sep 2017 09:02:25 -0700 (PDT)","from [10.1.210.88] (e110467-lin.cambridge.arm.com [10.1.210.88])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\tD83023F53D; Mon, 18 Sep 2017 09:02:23 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:\n\tContent-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post:\n\tList-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:\n\tMessage-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description\n\t:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:\n\tList-Owner; bh=Ck5hz4WE1Z/TqiVrCZNSrVVyvPXhTTHaTNmPqKEGnEM=;\n\tb=sFd99a+NaOVEQF\n\tl7xs0YugVg2ewZZaDS8R7Q6oo+BEndNsjSI/J9/SLFr8M8FCfkwv73OBKSLNng6avMac3AgC8zoNa\n\tIv01ff3GYxebgNoUQY1os7W0p5oetx3hGKM5q70bzwnp0NZJZh0k6bXn8jJ9kXUMklvP49ns91JBt\n\tv5TTcR38pdIM5H+kx1kwxN/lKQbLuXd1OvSwgr3GYgqx+52D5YfYPcVxFdJuqKWpLV62/O42gU1dR\n\t65zR58H6X5uQXSMIw5ta2Qu23ZUbWP83lHi7+pTz/zS7Bo6N9gzi4RyQahYfq9p6U9+dIZYc+v2h2\n\tlX1LrYFmhr72fvzuLASg==;","Subject":"Re: [PATCH 1/1] iommu/iova: Make rcache flush optional on IOVA\n\tallocation failure","To":"Tomasz Nowicki , joro@8bytes.org,\n\twill.deacon@arm.com","References":"<1505732214-9052-1-git-send-email-tomasz.nowicki@caviumnetworks.com>\n\t<1505732214-9052-2-git-send-email-tomasz.nowicki@caviumnetworks.com>","From":"Robin Murphy ","Message-ID":"<250f41c9-fb37-49b7-c336-68e163f77e16@arm.com>","Date":"Mon, 18 Sep 2017 17:02:22 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<1505732214-9052-2-git-send-email-tomasz.nowicki@caviumnetworks.com>","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170918_090246_595000_9FC23604 ","X-CRM114-Status":"GOOD ( 21.59 )","X-Spam-Score":"-6.9 (------)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-6.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/,\n\thigh trust [217.140.101.70 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Jayachandran.Nair@cavium.com, lorenzo.pieralisi@arm.com,\n\tard.biesheuvel@linaro.org, linux-kernel@vger.kernel.org,\n\tiommu@lists.linux-foundation.org,\n\tNate Watterson , \n\tGanapatrao.Kulkarni@cavium.com, linux-arm-kernel@lists.infradead.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1770624,"web_url":"http://patchwork.ozlabs.org/comment/1770624/","msgid":"","list_archive_url":null,"date":"2017-09-19T02:57:32","subject":"Re: [PATCH 1/1] iommu/iova: Make rcache flush optional on IOVA\n\tallocation failure","submitter":{"id":69609,"url":"http://patchwork.ozlabs.org/api/people/69609/","name":"Nate Watterson","email":"nwatters@codeaurora.org"},"content":"Hi Tomasz,\n\nOn 9/18/2017 12:02 PM, Robin Murphy wrote:\n> Hi Tomasz,\n> \n> On 18/09/17 11:56, Tomasz Nowicki wrote:\n>> Since IOVA allocation failure is not unusual case we need to flush\n>> CPUs' rcache in hope we will succeed in next round.\n>>\n>> However, it is useful to decide whether we need rcache flush step because\n>> of two reasons:\n>> - Not scalability. On large system with ~100 CPUs iterating and flushing\n>> rcache for each CPU becomes serious bottleneck so we may want to deffer it.\ns/deffer/defer\n\n>> - free_cpu_cached_iovas() does not care about max PFN we are interested in.\n>> Thus we may flush our rcaches and still get no new IOVA like in the\n>> commonly used scenario:\n>>\n>> if (dma_limit > DMA_BIT_MASK(32) && dev_is_pci(dev))\n>> iova = alloc_iova_fast(iovad, iova_len, DMA_BIT_MASK(32) >> shift);\n>>\n>> if (!iova)\n>> iova = alloc_iova_fast(iovad, iova_len, dma_limit >> shift);\n>>\n>> 1. First alloc_iova_fast() call is limited to DMA_BIT_MASK(32) to get\n>> PCI devices a SAC address\n>> 2. alloc_iova() fails due to full 32-bit space\n>> 3. rcaches contain PFNs out of 32-bit space so free_cpu_cached_iovas()\n>> throws entries away for nothing and alloc_iova() fails again\n>> 4. Next alloc_iova_fast() call cannot take advantage of rcache since we\n>> have just defeated caches. In this case we pick the slowest option\n>> to proceed.\n>>\n>> This patch reworks flushed_rcache local flag to be additional function\n>> argument instead and control rcache flush step. Also, it updates all users\n>> to do the flush as the last chance.\n> \n> Looks like you've run into the same thing Nate found[1] - I came up with\n> almost the exact same patch, only with separate alloc_iova_fast() and\n> alloc_iova_fast_noretry() wrapper functions, but on reflection, just\n> exposing the bool to callers is probably simpler. One nit, can you\n> document it in the kerneldoc comment too? With that:\n> \n> Reviewed-by: Robin Murphy \n> \n> Thanks,\n> Robin.\n> \n> [1]:https://www.mail-archive.com/iommu@lists.linux-foundation.org/msg19758.html\nThis patch completely resolves the issue I reported in [1]!!\nTested-by: Nate Watterson \n\nThanks,\nNate\n> \n>>\n>> Signed-off-by: Tomasz Nowicki \n>> ---\n>> drivers/iommu/amd_iommu.c | 5 +++--\n>> drivers/iommu/dma-iommu.c | 6 ++++--\n>> drivers/iommu/intel-iommu.c | 5 +++--\n>> drivers/iommu/iova.c | 7 +++----\n>> include/linux/iova.h | 5 +++--\n>> 5 files changed, 16 insertions(+), 12 deletions(-)\n>>\n>> diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c\n>> index 8d2ec60..ce68986 100644\n>> --- a/drivers/iommu/amd_iommu.c\n>> +++ b/drivers/iommu/amd_iommu.c\n>> @@ -1604,10 +1604,11 @@ static unsigned long dma_ops_alloc_iova(struct device *dev,\n>> \n>> \tif (dma_mask > DMA_BIT_MASK(32))\n>> \t\tpfn = alloc_iova_fast(&dma_dom->iovad, pages,\n>> -\t\t\t\t IOVA_PFN(DMA_BIT_MASK(32)));\n>> +\t\t\t\t IOVA_PFN(DMA_BIT_MASK(32)), false);\n>> \n>> \tif (!pfn)\n>> -\t\tpfn = alloc_iova_fast(&dma_dom->iovad, pages, IOVA_PFN(dma_mask));\n>> +\t\tpfn = alloc_iova_fast(&dma_dom->iovad, pages,\n>> +\t\t\t\t IOVA_PFN(dma_mask), true);\n>> \n>> \treturn (pfn << PAGE_SHIFT);\n>> }\n>> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c\n>> index 191be9c..25914d3 100644\n>> --- a/drivers/iommu/dma-iommu.c\n>> +++ b/drivers/iommu/dma-iommu.c\n>> @@ -370,10 +370,12 @@ static dma_addr_t iommu_dma_alloc_iova(struct iommu_domain *domain,\n>> \n>> \t/* Try to get PCI devices a SAC address */\n>> \tif (dma_limit > DMA_BIT_MASK(32) && dev_is_pci(dev))\n>> -\t\tiova = alloc_iova_fast(iovad, iova_len, DMA_BIT_MASK(32) >> shift);\n>> +\t\tiova = alloc_iova_fast(iovad, iova_len,\n>> +\t\t\t\t DMA_BIT_MASK(32) >> shift, false);\n>> \n>> \tif (!iova)\n>> -\t\tiova = alloc_iova_fast(iovad, iova_len, dma_limit >> shift);\n>> +\t\tiova = alloc_iova_fast(iovad, iova_len, dma_limit >> shift,\n>> +\t\t\t\t true);\n>> \n>> \treturn (dma_addr_t)iova << shift;\n>> }\n>> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c\n>> index 05c0c3a..75c8320 100644\n>> --- a/drivers/iommu/intel-iommu.c\n>> +++ b/drivers/iommu/intel-iommu.c\n>> @@ -3460,11 +3460,12 @@ static unsigned long intel_alloc_iova(struct device *dev,\n>> \t\t * from higher range\n>> \t\t */\n>> \t\tiova_pfn = alloc_iova_fast(&domain->iovad, nrpages,\n>> -\t\t\t\t\t IOVA_PFN(DMA_BIT_MASK(32)));\n>> +\t\t\t\t\t IOVA_PFN(DMA_BIT_MASK(32)), false);\n>> \t\tif (iova_pfn)\n>> \t\t\treturn iova_pfn;\n>> \t}\n>> -\tiova_pfn = alloc_iova_fast(&domain->iovad, nrpages, IOVA_PFN(dma_mask));\n>> +\tiova_pfn = alloc_iova_fast(&domain->iovad, nrpages,\n>> +\t\t\t\t IOVA_PFN(dma_mask), true);\n>> \tif (unlikely(!iova_pfn)) {\n>> \t\tpr_err(\"Allocating %ld-page iova for %s failed\",\n>> \t\t nrpages, dev_name(dev));\n>> diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c\n>> index f88acad..1a18b14 100644\n>> --- a/drivers/iommu/iova.c\n>> +++ b/drivers/iommu/iova.c\n>> @@ -358,9 +358,8 @@ EXPORT_SYMBOL_GPL(free_iova);\n>> */\n>> unsigned long\n>> alloc_iova_fast(struct iova_domain *iovad, unsigned long size,\n>> -\t\tunsigned long limit_pfn)\n>> +\t\tunsigned long limit_pfn, bool flush_rcache)\n>> {\n>> -\tbool flushed_rcache = false;\n>> \tunsigned long iova_pfn;\n>> \tstruct iova *new_iova;\n>> \n>> @@ -373,11 +372,11 @@ alloc_iova_fast(struct iova_domain *iovad, unsigned long size,\n>> \tif (!new_iova) {\n>> \t\tunsigned int cpu;\n>> \n>> -\t\tif (flushed_rcache)\n>> +\t\tif (!flush_rcache)\n>> \t\t\treturn 0;\n>> \n>> \t\t/* Try replenishing IOVAs by flushing rcache. */\n>> -\t\tflushed_rcache = true;\n>> +\t\tflush_rcache = false;\n>> \t\tfor_each_online_cpu(cpu)\n>> \t\t\tfree_cpu_cached_iovas(cpu, iovad);\n>> \t\tgoto retry;\n>> diff --git a/include/linux/iova.h b/include/linux/iova.h\n>> index 58c2a36..8fdcb66 100644\n>> --- a/include/linux/iova.h\n>> +++ b/include/linux/iova.h\n>> @@ -97,7 +97,7 @@ struct iova *alloc_iova(struct iova_domain *iovad, unsigned long size,\n>> void free_iova_fast(struct iova_domain *iovad, unsigned long pfn,\n>> \t\t unsigned long size);\n>> unsigned long alloc_iova_fast(struct iova_domain *iovad, unsigned long size,\n>> -\t\t\t unsigned long limit_pfn);\n>> +\t\t\t unsigned long limit_pfn, bool flush_rcache);\n>> struct iova *reserve_iova(struct iova_domain *iovad, unsigned long pfn_lo,\n>> \tunsigned long pfn_hi);\n>> void copy_reserved_iova(struct iova_domain *from, struct iova_domain *to);\n>> @@ -151,7 +151,8 @@ static inline void free_iova_fast(struct iova_domain *iovad,\n>> \n>> static inline unsigned long alloc_iova_fast(struct iova_domain *iovad,\n>> \t\t\t\t\t unsigned long size,\n>> -\t\t\t\t\t unsigned long limit_pfn)\n>> +\t\t\t\t\t unsigned long limit_pfn,\n>> +\t\t\t\t\t bool flush_rcache)\n>> {\n>> \treturn 0;\n>> }\n>>\n>","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"tLQ9QrKv\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=codeaurora.org header.i=@codeaurora.org\n\theader.b=\"QlDGCcH7\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key)\n\theader.d=codeaurora.org header.i=@codeaurora.org header.b=\"SmE6+T7G\"; \n\tdkim-atps=neutral","pdx-caf-mail.web.codeaurora.org;\n\tdmarc=none (p=none dis=none) header.from=codeaurora.org","pdx-caf-mail.web.codeaurora.org;\n\tspf=none smtp.mailfrom=nwatters@codeaurora.org"],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xx6yF1Yy4z9rxm\n\tfor ;\n\tTue, 19 Sep 2017 12:58:29 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1du8jx-0002sT-GE; Tue, 19 Sep 2017 02:58:26 +0000","from smtp.codeaurora.org ([198.145.29.96])\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1du8jW-0002lg-Go for linux-arm-kernel@lists.infradead.org;\n\tTue, 19 Sep 2017 02:58:01 +0000","by smtp.codeaurora.org (Postfix, from userid 1000)\n\tid 7047760710; Tue, 19 Sep 2017 02:57:36 +0000 (UTC)","from [192.168.86.148] (cpe-107-15-192-128.nc.res.rr.com\n\t[107.15.192.128])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128\n\tbits)) (No client certificate requested)\n\t(Authenticated sender: nwatters@smtp.codeaurora.org)\n\tby smtp.codeaurora.org (Postfix) with ESMTPSA id E9F2D601C4;\n\tTue, 19 Sep 2017 02:57:33 +0000 (UTC)"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=kDq7awZAnpXJsKyRnb7eUafik/kRvBj35dwIlchmx3A=;\n\tb=tLQ9QrKvuYTWeilmvc6B+J8kG\n\ttAZio/LhGD4ZfMnh/2chJtb91Dp8/jTELlDAmqSVY44X3b/8NNPSzic9xVCMMF/7OSTSf4CeKUx8l\n\tgdZGoYSg8j3iZC5BKA7xkNXeUIugTItq3Bb1wSd3V3W/FKxbn0wdMXlVsE2IQtl18l9TAA/vJSSyD\n\tX4Epmqz+kMluVw05q8VVvy0xbvjEq7wwRc6jUYqQOKNj0tefu7ZzIFsTUb8RcLsMn3/1Eibt6oK56\n\thiockCAIGk8mlxOLaDQmPc6IWZ4tIUdh0kjJ0EJU4UlYwmClGhwGarSaWyN0AipFuPIALNSm6+cnA\n\tY+oQQ0Lxw==;","v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org;\n\ts=default; t=1505789856;\n\tbh=dnSKnJDuKu7v2cfM4cZ62d8roxIOHvi9qFBjnI3V3lM=;\n\th=Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=QlDGCcH7e83RbKlZBgIPlyo1kLLoka0Jfc9hp2Ft9xwTyeqJlx5CwAH019hlE8Hs+\n\tJ7HFQw4d6S5id5KPYsrTmxLdB7avu3qZGgUcDuQqaw063n9rWKWy2rFR50Dox7uG0E\n\tce3UkiWH38WgaO1n4s+3y8EMrcj7eOdwJ+MYv5Ac=","v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org;\n\ts=default; t=1505789855;\n\tbh=dnSKnJDuKu7v2cfM4cZ62d8roxIOHvi9qFBjnI3V3lM=;\n\th=Subject:To:Cc:References:From:Date:In-Reply-To:From;\n\tb=SmE6+T7GdU4Eic+SQOrjExG6f/Os3tZgn6SAFyg32cAoQ84aNA8iSi9UjbKhqGAqw\n\tFAUcwNr7CGr4SCXmigwPlfi1nLoFSbU8TICg/k69+aILTfRX9T8sWpNCfN8VjLhloS\n\tkTQChohgIFe653OLM3ZJMVTLfkjPBs3dIDZSAKEk="],"X-Spam-Checker-Version":"SpamAssassin 3.4.0 (2014-02-07) on\n\tpdx-caf-mail.web.codeaurora.org","X-Spam-Level":"","X-Spam-Status":"No, score=-2.8 required=2.0 tests=ALL_TRUSTED,BAYES_00,\n\tDKIM_SIGNED,\n\tT_DKIM_INVALID autolearn=no autolearn_force=no version=3.4.0","DMARC-Filter":"OpenDMARC Filter v1.3.2 smtp.codeaurora.org E9F2D601C4","Subject":"Re: [PATCH 1/1] iommu/iova: Make rcache flush optional on IOVA\n\tallocation failure","To":"Robin Murphy ,\n\tTomasz Nowicki , joro@8bytes.org, \n\twill.deacon@arm.com","References":"<1505732214-9052-1-git-send-email-tomasz.nowicki@caviumnetworks.com>\n\t<1505732214-9052-2-git-send-email-tomasz.nowicki@caviumnetworks.com>\n\t<250f41c9-fb37-49b7-c336-68e163f77e16@arm.com>","From":"Nate Watterson ","Message-ID":"","Date":"Mon, 18 Sep 2017 22:57:32 -0400","User-Agent":"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<250f41c9-fb37-49b7-c336-68e163f77e16@arm.com>","Content-Language":"en-US","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170918_195758_644427_0342E7CF ","X-CRM114-Status":"GOOD ( 18.44 )","X-Spam-Score":"-4.3 (----)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-4.3 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/,\n\tmedium trust [198.145.29.96 listed in list.dnswl.org]\n\t-0.0 SPF_PASS SPF: sender matches SPF record\n\t-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay\n\tdomain\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from\n\tauthor's domain","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Jayachandran.Nair@cavium.com, lorenzo.pieralisi@arm.com,\n\tard.biesheuvel@linaro.org, linux-kernel@vger.kernel.org,\n\tiommu@lists.linux-foundation.org, Ganapatrao.Kulkarni@cavium.com,\n\tlinux-arm-kernel@lists.infradead.org","Content-Transfer-Encoding":"7bit","Content-Type":"text/plain; charset=\"us-ascii\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1770726,"web_url":"http://patchwork.ozlabs.org/comment/1770726/","msgid":"<5632227b-3d9b-0f03-db3d-2b2d7ccc8079@caviumnetworks.com>","list_archive_url":null,"date":"2017-09-19T08:03:47","subject":"Re: [PATCH 1/1] iommu/iova: Make rcache flush optional on IOVA\n\tallocation failure","submitter":{"id":72364,"url":"http://patchwork.ozlabs.org/api/people/72364/","name":"Tomasz Nowicki","email":"tnowicki@caviumnetworks.com"},"content":"Hi Robin,\n\nOn 18.09.2017 18:02, Robin Murphy wrote:\n> Hi Tomasz,\n> \n> On 18/09/17 11:56, Tomasz Nowicki wrote:\n>> Since IOVA allocation failure is not unusual case we need to flush\n>> CPUs' rcache in hope we will succeed in next round.\n>>\n>> However, it is useful to decide whether we need rcache flush step because\n>> of two reasons:\n>> - Not scalability. On large system with ~100 CPUs iterating and flushing\n>> rcache for each CPU becomes serious bottleneck so we may want to deffer it.\n>> - free_cpu_cached_iovas() does not care about max PFN we are interested in.\n>> Thus we may flush our rcaches and still get no new IOVA like in the\n>> commonly used scenario:\n>>\n>> if (dma_limit > DMA_BIT_MASK(32) && dev_is_pci(dev))\n>> iova = alloc_iova_fast(iovad, iova_len, DMA_BIT_MASK(32) >> shift);\n>>\n>> if (!iova)\n>> iova = alloc_iova_fast(iovad, iova_len, dma_limit >> shift);\n>>\n>> 1. First alloc_iova_fast() call is limited to DMA_BIT_MASK(32) to get\n>> PCI devices a SAC address\n>> 2. alloc_iova() fails due to full 32-bit space\n>> 3. rcaches contain PFNs out of 32-bit space so free_cpu_cached_iovas()\n>> throws entries away for nothing and alloc_iova() fails again\n>> 4. Next alloc_iova_fast() call cannot take advantage of rcache since we\n>> have just defeated caches. In this case we pick the slowest option\n>> to proceed.\n>>\n>> This patch reworks flushed_rcache local flag to be additional function\n>> argument instead and control rcache flush step. Also, it updates all users\n>> to do the flush as the last chance.\n> \n> Looks like you've run into the same thing Nate found[1] - I came up with\n> almost the exact same patch, only with separate alloc_iova_fast() and\n> alloc_iova_fast_noretry() wrapper functions, but on reflection, just\n> exposing the bool to callers is probably simpler. One nit, can you\n> document it in the kerneldoc comment too? With that:\n> \n> Reviewed-by: Robin Murphy \n\nThanks! I will add missing comment.\n\nTomasz","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"gexOVx3F\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=CAVIUMNETWORKS.onmicrosoft.com\n\theader.i=@CAVIUMNETWORKS.onmicrosoft.com header.b=\"R7MhID8x\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=Tomasz.Nowicki@cavium.com; "],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxFlX1ZWcz9ryT\n\tfor ;\n\tTue, 19 Sep 2017 18:04:40 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1duDWE-00072B-If; Tue, 19 Sep 2017 08:04:34 +0000","from mail-sn1nam01on0072.outbound.protection.outlook.com\n\t([104.47.32.72] helo=NAM01-SN1-obe.outbound.protection.outlook.com)\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1duDW8-0006uq-Bd for linux-arm-kernel@lists.infradead.org;\n\tTue, 19 Sep 2017 08:04:33 +0000","from [10.0.0.85] (31.172.191.173) by\n\tMWHPR0701MB3658.namprd07.prod.outlook.com (2603:10b6:301:7d::39) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7;\n\tTue, 19 Sep 2017 08:03:57 +0000"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=hiFepTRJNsHL1vpLsODxDxNl3b9VdOwTOUtlZKG6uiY=;\n\tb=gexOVx3FUf3f89NzGpqGeK5Ad\n\tgTVZf3xaQjJv1np1SQdeYbldxmMZTH7bTUQfC0xupc2leEuwKeGtMB436iD2OlTM7UDSA4PsoW2Bn\n\tkZ/nRTa0B4Gl4gW6U3T4jwyFqFbHscaAoauLltLTlGip4L+3jqZtAFqbzDKSIy/VsRFdmywDSr4NT\n\t7C7YQbuNdDJoL59YZoaeIxEXrQzO8KWbysu4uC2BHp6cLhzvgwn5E2VYxh4V1XeRAZ2MvKFscBM0h\n\tPAedvkcf7DqsWuS5hPnH0IIvq1lCGgPn57zgMzqrUzw9Ad1duTTzZoW6qs+jbOZiSRIVunqLp0odp\n\to8Y4vaeQw==;","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=JO/VRMUDBPQ9qwmQVZ1e5p1bUVRbWQwcLJ/0r12kYsU=;\n\tb=R7MhID8xPbhEairWuPIw53TGFpZa+dwmKh+Oj+0SyS2/In/vo/XgF0l3aAdL0A6UtuAnUFCpxlGqzKTlZo/JIelw3pyqBmdkSBrd/v8lgvi0xAiH+3acek9WHVV8Daob5QG3YpcsTtKg0qBfl5P3+aNYudo594K7gyYp87CQ9Ao="],"Subject":"Re: [PATCH 1/1] iommu/iova: Make rcache flush optional on IOVA\n\tallocation failure","To":"Robin Murphy ,\n\tTomasz Nowicki , joro@8bytes.org, \n\twill.deacon@arm.com","References":"<1505732214-9052-1-git-send-email-tomasz.nowicki@caviumnetworks.com>\n\t<1505732214-9052-2-git-send-email-tomasz.nowicki@caviumnetworks.com>\n\t<250f41c9-fb37-49b7-c336-68e163f77e16@arm.com>","From":"Tomasz Nowicki ","Message-ID":"<5632227b-3d9b-0f03-db3d-2b2d7ccc8079@caviumnetworks.com>","Date":"Tue, 19 Sep 2017 10:03:47 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 Thunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<250f41c9-fb37-49b7-c336-68e163f77e16@arm.com>","Content-Language":"en-GB","X-Originating-IP":"[31.172.191.173]","X-ClientProxiedBy":"VI1PR08CA0109.eurprd08.prod.outlook.com\n\t(2603:10a6:800:d4::11) To MWHPR0701MB3658.namprd07.prod.outlook.com\n\t(2603:10b6:301:7d::39)","X-MS-PublicTrafficType":"Email","X-MS-Office365-Filtering-Correlation-Id":"4d05768f-4a54-4a05-ae48-08d4ff34fc03","X-Microsoft-Antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:MWHPR0701MB3658; ","X-Microsoft-Exchange-Diagnostics":["1; MWHPR0701MB3658;\n\t3:tMwMModlgXsA0ZWx2Vn1h178wc7WG25aFdn8xcJ1ORuVY0lJtw/61xVAwuBZM1aTXTXui/T51j1li414CL1waLOh4ev6eTKIWKyFVVZBd6bhOzreKvRTeh1YWxXs9n8OLCfyiVjmji/VFP1QUQB6+aCE6U7Pa+ZJJmORKPkk74eAOXpF5oMfgLvEQOPLyyjNlwhIaXnFy+acYN35nLLixINUpZiaPIghnvVxHOxnJDVnlqFcHvukjVOZK4ycIGjR;\n\t25:FSYqJs254hoCLXVinricEu5hqPGEEaEPopMXCaOLP3LH5Iej3tVLOJQodkyRSGK/9oYGVYkRm4Oa5CQp90deRt81Z7GHi82J9iQbojCoC/kk0fdwjgvp+ejNL345tAPO270/QiGf/sgMFGfmeP0XiRGbzNXbAoJ3P+6+IUyQ8LLzG/0n8niDLfuGpfaIV7dPt7OELOo3LpklYt+NDl7LqDqPdXc4r0ZaEH1rTh3GfswWknWc2DKtcVQlm5zifYXYRdujprNTTlUNUaat98pPtaLq/qO5+iMTFr8dbK6grxRnbi5LJvfYxLGEjDx+4K5JKqbH1IgP+kcxUQtj9Wp+6w==;\n\t31:T1CH1zkk15lGsacCs8la1MWXYBmqjgQ5uEdRsx128DsAaDL59ZVjnIDCEJycwMi0PHEa1WKaOAC/Z+bgkXP88juc5tWbPtbaIvX5FuuiFuqJoMQHTic5UisjlpFpTRsEkE5052CHZ6Ziz+caR6MH2jSS774g9okWeBXfemuzm95b+8LY+4sAEJK8cw8XUyniSbLQSup71hNA2GbWNSX8j0+af+ioLiIt4nb1vJsCfbw=","1; MWHPR0701MB3658;\n\t20:4aHtSjBqAcLciglyRW2SbdEBNHSr4cR3HFEU41KEGa0CQpYqPZBJWQxsCPVPz0o69P8LJaZdEABz7FI2U1+UhIYq4UuzsWQSjrepmL0d5dX5G7sdjjLn4AFNPfESJZEQZvOUhc1MQtOls3hIlCwwYXD0/uBhulH+VkWbXl8lneWzthJnum6PI+sd+8gKdX6puKnmOM5Kt2Sd3d2uhC/2vNDvXJj+evd9tbRO1yU44gBlRnCesiFprKcxviDPP84NV5PmxBBg92+RHwEcksrFYhUBW5Z93txccadt5/+aKicbQMrnyDhygmlGmVQNRMzGy/m+In3L5fIri40rdJPmqyCi7EG9SR0usWp6W3GHpc/jsj80PzcgO1rlZWHgOvjv45Rb45C3XXzv9FaiiP+4Z8KH3OjmfI1FS6acbtiBEQFC8RUcAz6oV9k/fTB/l3/6EjQEkGoRcRQCCF6OofjYSbm4cikqMn/sdQvJUl8MLEmSkP/b9IPyMYMvzOc2PzrnpUBed3u87wLzZeS55hXxyhyyCzt6sgmBjVwhPhU+Kvez3tHvmCeINQFsoC5c4+A03b1zCeTlgl80a6m/IHXt+T5+lpsYRtcf3K2mq+56MdA=;\n\t4:65diqiUt57aP65y1h2CHjvl6KFr7opu8UA2Q6DLZHGrj69zuuzBZ3XkjTNhoPjXijFM/uAodefuUA3tH+27YeXaYT1+cZz/p7bOgaE0uApZdMt9gxe+fEOrWoT419s5nTJRleaK2BTrdxTUtatSaOkPWmX+aFTry8Y1gCLzONc65UyGDD4+OHLF8+GEB3IXpYgq+T/GE9tagx2LXBKogSz+rDOpKHvE1l0rW1Ymdf13IHsYN/AZQu0m9IxywSJjd4N18D9X7wlfzw/X5VbaGdfxJW4i8IDhbhqzXTEMkdOw=","=?utf-8?q?1=3BMWHPR0701MB3658=3B23=3AdW?=\n\t=?utf-8?q?KR1JBiNaLPi0lkDz5vB8JFAtVZ3ep/aXGH/X+N/8uDhlMiVzLxWBy2Ad?=\n\t=?utf-8?q?NZD6jjCLMQtmynxVZ08i9Zp0hr8hgdzGWFPsLkfoR7eymY6wcRhnjY7I?=\n\t=?utf-8?q?BYN44euzRXpitgXlzQOaF3DIuxVlv3Xf/JZCnnunVwAF7yYv6D7gWCgO?=\n\t=?utf-8?q?DBAE3bCLosMsiIiObLK0Dzg3U1ZEgKYZ/g+H6cGRoDlhbyjt0NI7x7HC?=\n\t=?utf-8?q?QrxK0vw9vtVrPWt+Jbc+QFbqReYR1fuX819x8+xk4wz5CS9RLgmLl2e/?=\n\t=?utf-8?q?jwglL4+EMqac8A7zFkC3kUCq1hib4xZPw6BhK/1NtvBp0C5UFUbcOaHw?=\n\t=?utf-8?q?e/VSisNFWCqQ3vj4zYXLSvOqZeYk256F8j2CtouJQH1NLdUks22rJSAg?=\n\t=?utf-8?q?vpTMSNR9DnMrqVDDR7TEgAg4PZsJK+3Jd9Vpkuk8UMP5yKiMPbrrUHYV?=\n\t=?utf-8?q?n1SLVPDjvFDuruusTop8a+nY3M/455ZM4Taf5NjJxG/VB1uSg5rfpPrh?=\n\t=?utf-8?q?SyIPFKV7UMFTAmzsL6AAUDCbTivQ1Recd4gXNZYA4YEBtk2F5NCQvGOA?=\n\t=?utf-8?q?U8wqJd1XvaNBKPZgXH7hpTGJnicOq0TxjZL0lkxJMzmCsoX5XzMi4XQL?=\n\t=?utf-8?q?BVAnqG3VoHh97AblWJaetiUiSaAh5k56hxdU6SKpZVsaAMCJpgEoiG2l?=\n\t=?utf-8?q?Zp1bSRB/woJgAjPf9WtH7W+N230auxmwnadDaStthnw/30/VHRCaT/0D?=\n\t=?utf-8?q?UxYNNAc2K5fUy3KTfCJwougRcdvTTw7aKMWcL49iXFIiSy3ReR9R2Top?=\n\t=?utf-8?q?8JBPYiSTSqJ5eW5tnui4wA3HNYMpyFKmataRmu0SiKWgdZfbBXhKGlqA?=\n\t=?utf-8?q?PyzcFScJv8dSskhd+yfGnr69DU2Ro/+g909kQm0zuXsXubMcNYjCzxay?=\n\t=?utf-8?q?lT/kNSOYIwy/iurJzm9ZRzK0zr1qfgaTfBwExbkyxyceelw5x4ylfAVR?=\n\t=?utf-8?q?aAxZfHBixxe0Lgf5dGk8lYmeUWEb6WAKOEkYnRTJeTNU/hx1exV/MrD8?=\n\t=?utf-8?q?cz9j1PzEs+DLWXQGdHhmkfBoJcVJMqfGgyKo383NK0754fjWv/3BNzB8?=\n\t=?utf-8?q?iCTvvQHRk0HbFlUoAqe4iJVyDjrmn+r0X426upH69ipFE5Bk+ZLdjggO?=\n\t=?utf-8?q?UB1sPW5uIaKKC23aUenbRcaK0kJiiMyaYL+o6jsynfR8/0K0sBwk3PFl?=\n\t=?utf-8?q?qaFTSLz0EeAqe9otA2RLbfBzJvsr8rqcGLZHV9G751/R14jqGytC79le?=\n\t=?utf-8?q?4s5XN9gIS7dmEIJ5h11neZbywnCQQfBIBUBdAeUokrjxs05KHf+FR2Ma?=\n\t=?utf-8?q?f326bGVDSFVA=3D=3D?=","1; MWHPR0701MB3658;\n\t6:84a1HLVtRi+Mri0jp2gOdE8kqXd+bzrCQl9UsQEfqJmBBGlqJGg0YvZ1zYXomO5t/CSGcXv74wDJyhbT5SeiU1F67LVchOPVuTTKK9jZVXpmnXkhFiavVt8zep0J0boHyNaeiRtboxVyf+4nVkwVWPLfKbQf4p98QjZ8BUPudnohMS/I4vhOSaW4ZREwI3ekPy3r+/pvkGRIcFxIMWjetw/BKUsHNdRjFhO4zYm4v+3+bU3NPynxYpd0uFYbxvIf02tcHzfFVV0bFMVr8NB74RAJnpwNplwwpH42pc5L6xQ0eaiQ5XdT80l4f7T1Ps4cDznUvAzB9d2ad0WB+pbFhg==;\n\t5:C1T13VCzvctnlYX2mTFHyDJ39RmVw9jjSb8zqd987FgVKfzigG8GxdSPm4h+JTD+M7pebu+TLgM1WQRNT4H0izbiUXWh5afNIwE5OeMBHCwndQRPDTS4eKnrN6wj06c4NjrF11xiJisLKpRxF8VInQ==;\n\t24:LP70P0vVbbs68PkY4B4obSCC32N9k3GjUwt/VqEnVBfz1vB17EF7AEdG6TK9KIVRv79sV6XYY80hoMAfgqwutbd6QHbnpZsslas+NCRRbYI=;\n\t7:/tyuVmmSxCw0j6tRGWoWv1RJ1Kih8oMOvyx07NraHBQG8MSD95QxHfCUqZ7flF8CPTPh1qpSeINRiN+rLGsmMAd1kOz8jUWl6vEjNuVy4xDBI2uyNT32rL3Dhl6ZPy5zI3TN0yf//qxqEiqmCWRhxsqc/k1PE1teGFbHVgEDObEQEhqEQ0c+oOo3yLfsQSKNLRy5AxMOgNx2INlxwpurvmN+x+4d4VcNNjbpVdyWJeY="],"X-MS-TrafficTypeDiagnostic":"MWHPR0701MB3658:","X-Exchange-Antispam-Report-Test":"UriScan:(180628864354917);","X-Microsoft-Antispam-PRVS":"","X-Exchange-Antispam-Report-CFA-Test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(93006095)(100000703101)(100105400095)(3002001)(10201501046)(6041248)(20161123562025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123558100)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:MWHPR0701MB3658; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:MWHPR0701MB3658; ","X-Forefront-PRVS":"04359FAD81","X-Forefront-Antispam-Report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(6049001)(376002)(346002)(189002)(24454002)(199003)(42882006)(16526017)(31696002)(230700001)(81156014)(25786009)(2950100002)(64126003)(6666003)(81166006)(53936002)(6116002)(3846002)(8936002)(53546010)(31686004)(72206003)(33646002)(478600001)(6246003)(105586002)(16576012)(8676002)(189998001)(68736007)(229853002)(54356999)(106356001)(110136005)(65956001)(66066001)(47776003)(76176999)(50986999)(316002)(36756003)(23676002)(4326008)(77096006)(6486002)(101416001)(97736004)(305945005)(58126008)(50466002)(7736002)(2906002)(83506001)(5660300001);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR0701MB3658; H:[10.0.0.85]; FPR:;\n\tSPF:None; \n\tPTR:InfoNoRecords; MX:3; A:1; LANG:en; ","Received-SPF":"None (protection.outlook.com: cavium.com does not designate\n\tpermitted sender hosts)","SpamDiagnosticOutput":"1:99","SpamDiagnosticMetadata":"NSPM","X-OriginatorOrg":"caviumnetworks.com","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"19 Sep 2017 08:03:57.1782\n\t(UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"711e4ccf-2e9b-4bcf-a551-4094005b6194","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"MWHPR0701MB3658","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170919_010428_653049_21A2261C ","X-CRM114-Status":"GOOD ( 12.65 )","X-Spam-Score":"-1.9 (-)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-1.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/,\n\tno trust [104.47.32.72 listed in list.dnswl.org]\n\t-0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3)\n\t[104.47.32.72 listed in wl.mailspike.net]\n\t0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level\n\tmail domains are different\n\t-0.0 SPF_HELO_PASS SPF: HELO matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid\n\t-0.0 RCVD_IN_MSPIKE_WL Mailspike good senders","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Jayachandran.Nair@cavium.com, lorenzo.pieralisi@arm.com,\n\tard.biesheuvel@linaro.org, linux-kernel@vger.kernel.org,\n\tiommu@lists.linux-foundation.org,\n\tNate Watterson , \n\tGanapatrao.Kulkarni@cavium.com, linux-arm-kernel@lists.infradead.org","Content-Transfer-Encoding":"7bit","Content-Type":"text/plain; charset=\"us-ascii\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}},{"id":1770734,"web_url":"http://patchwork.ozlabs.org/comment/1770734/","msgid":"<03297a05-8490-a86d-12ab-a99cf73c09ba@caviumnetworks.com>","list_archive_url":null,"date":"2017-09-19T08:10:26","subject":"Re: [PATCH 1/1] iommu/iova: Make rcache flush optional on IOVA\n\tallocation failure","submitter":{"id":72364,"url":"http://patchwork.ozlabs.org/api/people/72364/","name":"Tomasz Nowicki","email":"tnowicki@caviumnetworks.com"},"content":"Hi Nate,\n\nOn 19.09.2017 04:57, Nate Watterson wrote:\n> Hi Tomasz,\n> \n> On 9/18/2017 12:02 PM, Robin Murphy wrote:\n>> Hi Tomasz,\n>>\n>> On 18/09/17 11:56, Tomasz Nowicki wrote:\n>>> Since IOVA allocation failure is not unusual case we need to flush\n>>> CPUs' rcache in hope we will succeed in next round.\n>>>\n>>> However, it is useful to decide whether we need rcache flush step \n>>> because\n>>> of two reasons:\n>>> - Not scalability. On large system with ~100 CPUs iterating and flushing\n>>> rcache for each CPU becomes serious bottleneck so we may want to \n>>> deffer it.\n> s/deffer/defer\n> \n>>> - free_cpu_cached_iovas() does not care about max PFN we are \n>>> interested in.\n>>> Thus we may flush our rcaches and still get no new IOVA like in the\n>>> commonly used scenario:\n>>>\n>>> if (dma_limit > DMA_BIT_MASK(32) && dev_is_pci(dev))\n>>> iova = alloc_iova_fast(iovad, iova_len, DMA_BIT_MASK(32) >> \n>>> shift);\n>>>\n>>> if (!iova)\n>>> iova = alloc_iova_fast(iovad, iova_len, dma_limit >> shift);\n>>>\n>>> 1. First alloc_iova_fast() call is limited to DMA_BIT_MASK(32) to \n>>> get\n>>> PCI devices a SAC address\n>>> 2. alloc_iova() fails due to full 32-bit space\n>>> 3. rcaches contain PFNs out of 32-bit space so \n>>> free_cpu_cached_iovas()\n>>> throws entries away for nothing and alloc_iova() fails again\n>>> 4. Next alloc_iova_fast() call cannot take advantage of rcache \n>>> since we\n>>> have just defeated caches. In this case we pick the slowest \n>>> option\n>>> to proceed.\n>>>\n>>> This patch reworks flushed_rcache local flag to be additional function\n>>> argument instead and control rcache flush step. Also, it updates all \n>>> users\n>>> to do the flush as the last chance.\n>>\n>> Looks like you've run into the same thing Nate found[1] - I came up with\n>> almost the exact same patch, only with separate alloc_iova_fast() and\n>> alloc_iova_fast_noretry() wrapper functions, but on reflection, just\n>> exposing the bool to callers is probably simpler. One nit, can you\n>> document it in the kerneldoc comment too? With that:\n>>\n>> Reviewed-by: Robin Murphy \n>>\n>> Thanks,\n>> Robin.\n>>\n>> [1]:https://www.mail-archive.com/iommu@lists.linux-foundation.org/msg19758.html \n>>\n> This patch completely resolves the issue I reported in [1]!!\n\nI somehow missed your observations in [1] :/\nAnyway, it's great it fixes performance for you too.\n\n> Tested-by: Nate Watterson \n\nThanks!\nTomasz","headers":{"Return-Path":"","X-Original-To":"incoming-imx@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming-imx@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=lists.infradead.org\n\t(client-ip=65.50.211.133; helo=bombadil.infradead.org;\n\tenvelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org;\n\treceiver=)","ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=lists.infradead.org\n\theader.i=@lists.infradead.org header.b=\"qkrADzqC\"; \n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=CAVIUMNETWORKS.onmicrosoft.com\n\theader.i=@CAVIUMNETWORKS.onmicrosoft.com header.b=\"GsBj1Jof\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=Tomasz.Nowicki@cavium.com; "],"Received":["from bombadil.infradead.org (bombadil.infradead.org\n\t[65.50.211.133])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxFv30lnPz9rvt\n\tfor ;\n\tTue, 19 Sep 2017 18:11:11 +1000 (AEST)","from localhost ([127.0.0.1] helo=bombadil.infradead.org)\n\tby bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux))\n\tid 1duDcZ-0002bu-T8; Tue, 19 Sep 2017 08:11:07 +0000","from mail-co1nam03on062c.outbound.protection.outlook.com\n\t([2a01:111:f400:fe48::62c]\n\thelo=NAM03-CO1-obe.outbound.protection.outlook.com)\n\tby bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux))\n\tid 1duDcV-0002Vk-6b for linux-arm-kernel@lists.infradead.org;\n\tTue, 19 Sep 2017 08:11:05 +0000","from [10.0.0.85] (31.172.191.173) by\n\tMWHPR0701MB3659.namprd07.prod.outlook.com (2603:10b6:301:7e::10) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.56.11;\n\tTue, 19 Sep 2017 08:10:36 +0000"],"DKIM-Signature":["v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;\n\td=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type:\n\tContent-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive:\n\tList-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From:\n\tReferences:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date:\n\tResent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner;\n\tbh=onUVt43tW3NDNeKFxDS9NL/nad03sDTHgfJol3bXDoc=;\n\tb=qkrADzqCtpfwjJx+8q5S1zYUj\n\tCcns5Qi3fHKzLEGcVR8c6PcRIAuWCeHWosKR57WrtgVFNQk9AJCr0drJilPFQ3LGHoj3hrFbdzgcH\n\tjH0XKmpSUo1MPF18/NFnuEesrVt8kWrP5nM78/waknjFSkZuEcjT3IduBwvAlm4Bbl3KhB043iDzk\n\t73rzo7q0QPdylgxXUOktZJUNyR1a76hVLuxFoRL22ioNTogp9/ujCY0kjt2AuLRQtCeLX8g5DI4Um\n\tZa5v1cCZ+dl51C4P2U+JgQGCET9D4HWuQTZh6/INzkZVVDke1GKQewaelx74/eFP9ErqQuLiqrtsq\n\tEWCSbMB7w==;","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=RLsNl9HohR9aPL99vZH1CwT5aDbTWkGSMpel1jrYbI0=;\n\tb=GsBj1JofGO0/dzIMXbMjf1r7kL0RYsIdrU2zAdH3lxgRjyDakhDExSlhu1KRMNkxzjVdna72A7rBKpTTC9VQzILsfHd/LUhh7nDY9rkiqLWjRJNBA15Gt9JUt/vgYqDPW0/zEqv/UvYSJrZJAaA2NKO+9Dp18/yISrGz0mGV2Xw="],"Subject":"Re: [PATCH 1/1] iommu/iova: Make rcache flush optional on IOVA\n\tallocation failure","To":"Nate Watterson ,\n\tRobin Murphy ,\n\tTomasz Nowicki , joro@8bytes.org, \n\twill.deacon@arm.com","References":"<1505732214-9052-1-git-send-email-tomasz.nowicki@caviumnetworks.com>\n\t<1505732214-9052-2-git-send-email-tomasz.nowicki@caviumnetworks.com>\n\t<250f41c9-fb37-49b7-c336-68e163f77e16@arm.com>\n\t","From":"Tomasz Nowicki ","Message-ID":"<03297a05-8490-a86d-12ab-a99cf73c09ba@caviumnetworks.com>","Date":"Tue, 19 Sep 2017 10:10:26 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 Thunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Language":"en-GB","X-Originating-IP":"[31.172.191.173]","X-ClientProxiedBy":"VI1PR08CA0132.eurprd08.prod.outlook.com\n\t(2603:10a6:800:d4::34) To MWHPR0701MB3659.namprd07.prod.outlook.com\n\t(2603:10b6:301:7e::10)","X-MS-PublicTrafficType":"Email","X-MS-Office365-Filtering-Correlation-Id":"72118d64-8b69-4ed7-e40c-08d4ff35ea3b","X-Microsoft-Antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(300000502095)(300135100095)(22001)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:MWHPR0701MB3659; ","X-Microsoft-Exchange-Diagnostics":["1; MWHPR0701MB3659;\n\t3:lHnIk/CgixOjEP9QoJx5oHYquV3/4uaOebtLt6cCOg7FNDkZ2qA6KURFCSQmifSZSlPtD9UfIQGCFDL3koM34T7K3kb1lyq/2bLJAu4BX58NVrMZsTxwQGuqGZSE8Ue+hwfV797aFACpj4Itrr7reqGyYsIheEVt0xFG2zR38kmB7qS6zSHBZa2NhdaAkgkppLtGgOneSfpKI8DHIptEB1TWefG05UKw2Ewvb6JWXR2Z5uZ2RdB1YObQ3AKTKDTF;\n\t25:ufIir3LgpmnCnDTrWtRwafmeDj9SUeyXl/ZXRrIATr1BrBIMmDNVQsXqGX5p2AyB/9BLE24BH6mo1QvvtBSfuN0ZZ5St+JyV3Eb2B6JYImyvoZrt4+18YIaNUUkDWvlqxvAo6Pt6XeBCzqU8f9PHvGdd+//+bjEF5v2BHz5ZhiX8wCUCHtLk2Lnho+UqY4+zih9oYnejTfg+sjN9DnJ1GrxZQO0zq1Qt+hmH7/jzTCQ2AH72JvQL8Lj21/iysLQrxhSgjxcZ76MXKsP5T1krDSpb7ykXr+FSZjPdNmEqYna75Dt0T1wi3TGbqdCxZFkBUho57vFOO70gCuB4JjaXiA==;\n\t31:qIpbUZX1tpHNE8uwGtmVUO/h7Jga1JpB0asAcZPrtzkAD6AJLCNasvUYy1kSFrlnYnvpkg7blnE58BPlLETH3xg/8KOdIj4G6ZYWPa4QfgmIZLpzv9jkKMvfVrqTdOKbBFddjhJ/cv+DJV7xH9VXTEpkPAftzFywlv3X0ROSU6kcnhM1ZgN5e9ryUzTQrcHwlloN2xQI9ymQXqFo0K2TZVqs9m8WIXmgA7erbMUeXgM=","1; MWHPR0701MB3659;\n\t20:jzYmIqyApBK8dK1hU91VKRJtCY+ZzHf+hTjAM1cCJ1iPpLZFDwBpcteBHDrziQTgYG2CWXatxlAOmgxscewDPWr57Vg3fPW00bYXzrTXjXBztwfmcljT3EGWVtX6KJjc1UiAqv54be38khzEYwY9uTyqRywTX9pXplzYWcqL98u4ddTyC3g5G7G/Kd9ddQB+ny7yYACNZfbadHtcIvdMuhWaddv/wiKZKJQ6ZlB9dKa2dzaVwMPEzeGKmRhvlSdR67h4g1o4kLMI4yBbjBdlw9kuPqXH7ULLA43np117W8eTFmTFLUsQODHOei5HgtaTqRsbM5qxFtHvM+xGiv5o/HEPcg7v0W5HNQwuwBEsF7vuM4k8mzPWXNuEdwAN17d71LGcwZVlooIZuBxqp5h2+cjuPmc6cYKbQp9dR/LDzAvYWJzrbrND4hrN6/duQ7kBikzVCcbAer43cJp6jmiunPQWUIdzDWTPAziwPApxoXM8uQ1h2MY1a4GWvU+U8l2jzQpqc72A/I5w6XWw1X1N1rPeWJ/jVgKYCF55/9BqbngHFEKc857iq04DvyE5OtkYBKcq3K4p95//WtHYsbC4cfCH8Eq1HtNj1LWHrEEtGiM=","1; MWHPR0701MB3659;\n\t4:asC+NUa4jUopema8vRFAuWrB9WlmFZ7pNvxs9DpQSnhVs/QeBHgDyo6kHchRXAvhlOGJsCostxW89Er0jA99iq8Iq0IBc75euaeiTmKzasqpPGAthPFj6jznszmgaszXjNyryogP5Nw4guK3+RtBD0bQGbGzqAoaXjdImsS42O1xcjtSkiLTEceN5oxEJ1Hy+ayjySRJWXdB9qYSfljj1/jbD/h+tvSAvWHKxyhHq3LfVik1leb+x9n+dAu6vuvPE7uRUN0NeLvgT4NHceA8NXQu33TtkvoVUUoSAqxFVL+nS+kpUtUlQom/XrVgmp3oJIZ3jh6W5mos3MWjkOYTFQ==","=?utf-8?q?1=3BMWHPR0701MB3659=3B23=3AxW?=\n\t=?utf-8?q?9PL3J8guUiMWs/nYTjouzW3RTw6xLsu3NkVt0l/OaFM0p4R6fCR1qP8o?=\n\t=?utf-8?q?AuSnJO+lz81G1MrYaXiBbFF/cA371cwWN0NBSTuRp5oS8k+DI0mNHz1D?=\n\t=?utf-8?q?ykdfyIPSwEPoSycgvP3/squBWmP86RxI9LVwr2pdgT/Lx7I9M8IX1Vad?=\n\t=?utf-8?q?ApgE/gEqKvUVOEybOLf2DxUbwhOVxoi6nTEve1WZpEJuVCcRufkNi5Pt?=\n\t=?utf-8?q?l+ILexfZI5/dFeKVePIFeA1v6TpbenU/qQqKYI8nqPaNlpyV/U5bE8fm?=\n\t=?utf-8?q?2jGSadCweJeb/vh1rjJL7jsxxSIPJMZXvduqGPG3M8ulVIBxlbTmQt4J?=\n\t=?utf-8?q?22HnqtQNiw96OkC+9gzOKPZapSZRt5Q81fQQFOasiOLMq3pyFQGK4Oq8?=\n\t=?utf-8?q?5ADHyNyK11aaQAFwfevIvzaEfn+gphZl2rt3FuyLeFYToeObgM8vlUdh?=\n\t=?utf-8?q?g/5RD6epblKmv9h7tVGMmQNExCBkHvrf+5EXXspOSvc0HNTnvQn3zrnI?=\n\t=?utf-8?q?Ekw/TwwUc765Xz8K+0vxK/vSjclu3q2BOeCILFctrtl6J0va+nolpb05?=\n\t=?utf-8?q?OwktBoe/TEZ+Q8tUzxer+nRXLjVdo7zhm2tM/KDJiB2Gd1s7SMaPDmdf?=\n\t=?utf-8?q?/0tqJLqvbJH4ZpAx1Cb66itEAd2UVkw4QO2B3CRWA6bKE58VL2h7uTor?=\n\t=?utf-8?q?xwEh9Yi7+vs+Lq5NpOH46Mi2Bosb3k22Mt+pPyJejJTFUlygfkr4yqev?=\n\t=?utf-8?q?dW8UuEQej/QBMbmjP1HGpxDQYrYQV+7Rn6yoy71+VfukfO2zGg8fxW9X?=\n\t=?utf-8?q?QoYsFXTAAgqmA+ODP7uqPAQpYGE/OGd3AwQ2WsUehjyKqBBUMnXH0zBq?=\n\t=?utf-8?q?/FQmoEMsFGv3vNbFyFd8jL5gFR3FcFyoxs0u80EvOHSyTvTDVzTT9UEF?=\n\t=?utf-8?q?yBEv2Zdbz6CAT0ZWmxPs0Q5ZCT7b5iH+Pf8y9cTh8my3VHWW92yLOH4E?=\n\t=?utf-8?q?NZJtlffZ7uSGJUmtJUfcB12A6T6/FjCMhHlMgYlW/eO6rivvoKkdoIHU?=\n\t=?utf-8?q?25NyJETsM+9dK6iEPSgnjocG2qPIGQoAJL6i/HA5I/0h5pfuZyPDFAJC?=\n\t=?utf-8?q?UYe/CD/6b0Sw8lr/yMtfrk2bOJlAAff5KNczb7D3JKYFhL1LMrERwSPI?=\n\t=?utf-8?q?G/WP9qlYcu+x/X/XE5DEfJTzDE7AKZ1aKZnVwnEJERf6aVb96//shTjh?=\n\t=?utf-8?q?ppU2UcSWOGxzeEc80983oMbl7YlVyoKagcz6JMbsutBZexsx5C2gEIq2?=\n\t=?utf-8?q?hAyA2eJ8LgvmWYc2/4xUwZ1JbZsfcV/NHVscGUfPrfUTR6lwj5Ci4SAU?=\n\t=?utf-8?q?GbMWkJ0q1rT3KJnNlcAhst7kj5nmmjzOkhHpE6dIbmzcSiXFJEdDxTIe?=\n\t=?utf-8?q?ks+t+ghxHEV0Dty+pfD7hHq0K2S8BWjhZvAmbZHJk=3D?=","1; MWHPR0701MB3659;\n\t6:mFXWcoWsAXDmnwkwla7B3zcjItNmjwqsRTVv/HU02fHIRK6lFcdfinc1NMKsqSYOoZJ7587+N7uprBuOxRNjr2LFSAdVe97/jeZkRx7bXnvV0tRoX8utXxlIuI/BASyDAfJIBnETq93D5KCyxlZteHXgvpWcdC/m7xIlqP8pa59d8mMy54msGw9jsN7uQxdKqOaBECbPdveAryKxSqLlswtOO4FynAkrDbQ+bpxgzTSPoqjWA/7kD8wurpMq33f/QoNrAmm3f0o0leFqF9yLzICvl3bRhdvqzrhE7ySSndw9H7SYaz+uVfx9M0e23AcHIc4WczYnpA+HHtYZA36A5Q==;\n\t5:n2yV2ZhdqEpQ++xYmx4ZIB2PfK5g5/VLcrHLDfHt2B8dBreGqu9teKrnJ+/CYVe+V1WjMob88TyfMUZSBQfY/lXUpYbQeRdWGbrjZLFBE4SUWL50+AnK0FZeO9oOnis0YDmBEiCoOqA4BSA+514ixA==;\n\t24:49NmGAxUDX4U+BHPK14cqdQl2NpKIAlQ9KQ//lA05lpuCSxd6bK4K/li5v3KgzBrixzlH5/sLTXTMBcPOcqskX3VDiRWi0iRMHnNRYDu9vI=;\n\t7:zEQqTMfp1MZQQ6yHePWOnKXggbJVAN2W6tC7W42dihkxBXJhyISmZI41CAvsLAgbt0Xym8Z7GCCj1ik5DswP1RLzQPKJejxYpzQvBqoc5UEShVn0sG/hNR1s1P738pIdF2zF3xMI2bupjt7tnkcJgJZg5BguPI4iZIE0jK8TOdc8W2DfCtde/TZjHw0lD9zsoaSfbPo5Fz7dL5FBbNq7oLGQpIBM4d4jm4+S82vD35M="],"X-MS-TrafficTypeDiagnostic":"MWHPR0701MB3659:","X-Exchange-Antispam-Report-Test":"UriScan:(180628864354917)(278021516957215); ","X-Microsoft-Antispam-PRVS":"","X-Exchange-Antispam-Report-CFA-Test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(3002001)(100000703101)(100105400095)(93006095)(6041248)(20161123555025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123564025)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:MWHPR0701MB3659; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:MWHPR0701MB3659; ","X-Forefront-PRVS":"04359FAD81","X-Forefront-Antispam-Report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(6049001)(346002)(376002)(24454002)(199003)(52314003)(377454003)(189002)(16576012)(16526017)(72206003)(53546010)(8676002)(97736004)(23676002)(81166006)(81156014)(189998001)(36756003)(8936002)(110136005)(58126008)(478600001)(77096006)(6486002)(64126003)(305945005)(53936002)(50466002)(93886005)(83506001)(76176999)(50986999)(31686004)(2906002)(33646002)(6306002)(7736002)(5660300001)(68736007)(31696002)(66066001)(65956001)(6116002)(3846002)(2950100002)(4326008)(316002)(229853002)(42882006)(6246003)(101416001)(6666003)(106356001)(230700001)(25786009)(105586002)(47776003)(54356999);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR0701MB3659; H:[10.0.0.85]; FPR:;\n\tSPF:None; \n\tPTR:InfoNoRecords; A:1; MX:1; LANG:en; ","Received-SPF":"None (protection.outlook.com: cavium.com does not designate\n\tpermitted sender hosts)","SpamDiagnosticOutput":"1:99","SpamDiagnosticMetadata":"NSPM","X-OriginatorOrg":"caviumnetworks.com","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"19 Sep 2017 08:10:36.8444\n\t(UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"711e4ccf-2e9b-4bcf-a551-4094005b6194","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"MWHPR0701MB3659","X-CRM114-Version":"20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 ","X-CRM114-CacheID":"sfid-20170919_011103_317917_EC65080C ","X-CRM114-Status":"GOOD ( 11.84 )","X-Spam-Score":"-1.9 (-)","X-Spam-Report":"SpamAssassin version 3.4.1 on bombadil.infradead.org summary:\n\tContent analysis details: (-1.9 points)\n\tpts rule name description\n\t---- ----------------------\n\t--------------------------------------------------\n\t0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level\n\tmail domains are different\n\t-0.0 SPF_HELO_PASS SPF: HELO matches SPF record\n\t-1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%\n\t[score: 0.0000]\n\t-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature\n\t0.1 DKIM_SIGNED Message has a DKIM or DK signature,\n\tnot necessarily valid","X-BeenThere":"linux-arm-kernel@lists.infradead.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Jayachandran.Nair@cavium.com, lorenzo.pieralisi@arm.com,\n\tard.biesheuvel@linaro.org, linux-kernel@vger.kernel.org,\n\tiommu@lists.linux-foundation.org, Ganapatrao.Kulkarni@cavium.com,\n\tlinux-arm-kernel@lists.infradead.org","Content-Transfer-Encoding":"7bit","Content-Type":"text/plain; charset=\"us-ascii\"; Format=\"flowed\"","Sender":"\"linux-arm-kernel\" ","Errors-To":"linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org","List-Id":"linux-imx-kernel.lists.patchwork.ozlabs.org"}}]