[{"id":1804011,"web_url":"http://patchwork.ozlabs.org/comment/1804011/","msgid":"<8c58fcaa-dedc-e6be-44bf-2cb349ef9e38@ozlabs.ru>","date":"2017-11-13T23:47:12","subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 27/10/17 14:00, Alexey Kardashevskiy wrote:\n> This adds trace_map/trace_unmap tracepoints to spapr driver. Type1 already\n> uses these via the IOMMU API (iommu_map/__iommu_unmap).\n> \n> Signed-off-by: Alexey Kardashevskiy \n\nPing?\n\n\n\n> ---\n> \n> Example:\n> qemu-system-ppc-8655 [096] 724.662740: unmap: IOMMU: iova=0x000000003ffff000 size=4096 unmapped_size=4096\n> qemu-system-ppc-8656 [104] 724.970912: map: IOMMU: iova=0x0800000000000000 paddr=0x00007ffef7ff0000 size=65536\n> ---\n> drivers/vfio/vfio_iommu_spapr_tce.c | 12 ++++++++++--\n> 1 file changed, 10 insertions(+), 2 deletions(-)\n> \n> diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c\n> index 63112c36ab2d..4531486c77c6 100644\n> --- a/drivers/vfio/vfio_iommu_spapr_tce.c\n> +++ b/drivers/vfio/vfio_iommu_spapr_tce.c\n> @@ -22,6 +22,7 @@\n> #include \n> #include \n> #include \n> +#include \n> \n> #include \n> #include \n> @@ -502,17 +503,19 @@ static int tce_iommu_clear(struct tce_container *container,\n> \t\tstruct iommu_table *tbl,\n> \t\tunsigned long entry, unsigned long pages)\n> {\n> -\tunsigned long oldhpa;\n> +\tunsigned long oldhpa, unmapped, firstentry = entry, totalpages = pages;\n> \tlong ret;\n> \tenum dma_data_direction direction;\n> \n> -\tfor ( ; pages; --pages, ++entry) {\n> +\tfor (unmapped = 0; pages; --pages, ++entry) {\n> \t\tdirection = DMA_NONE;\n> \t\toldhpa = 0;\n> \t\tret = iommu_tce_xchg(tbl, entry, &oldhpa, &direction);\n> \t\tif (ret)\n> \t\t\tcontinue;\n> \n> +\t\t++unmapped;\n> +\n> \t\tif (direction == DMA_NONE)\n> \t\t\tcontinue;\n> \n> @@ -523,6 +526,9 @@ static int tce_iommu_clear(struct tce_container *container,\n> \n> \t\ttce_iommu_unuse_page(container, oldhpa);\n> \t}\n> +\ttrace_unmap(firstentry << tbl->it_page_shift,\n> +\t\t\ttotalpages << tbl->it_page_shift,\n> +\t\t\tunmapped << tbl->it_page_shift);\n> \n> \treturn 0;\n> }\n> @@ -965,6 +971,8 @@ static long tce_iommu_ioctl(void *iommu_data,\n> \t\t\t\t\tdirection);\n> \n> \t\tiommu_flush_tce(tbl);\n> +\t\tif (!ret)\n> +\t\t\ttrace_map(param.iova, param.vaddr, param.size);\n> \n> \t\treturn ret;\n> \t}\n>","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3ybS5Z2FHSz9s1h\n\tfor ;\n\tTue, 14 Nov 2017 10:48:50 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3ybS5Z11w0zDqm7\n\tfor ;\n\tTue, 14 Nov 2017 10:48:50 +1100 (AEDT)","from mail-pf0-x244.google.com (mail-pf0-x244.google.com\n\t[IPv6:2607:f8b0:400e:c00::244])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3ybS3s5lBfzDqF9\n\tfor ;\n\tTue, 14 Nov 2017 10:47:19 +1100 (AEDT)","by mail-pf0-x244.google.com with SMTP id l24so572892pfj.6\n\tfor ;\n\tMon, 13 Nov 2017 15:47:19 -0800 (PST)","from [192.168.10.22] (124-171-205-99.dyn.iinet.net.au.\n\t[124.171.205.99]) by smtp.googlemail.com with ESMTPSA id\n\tq13sm38245873pfi.166.2017.11.13.15.47.15\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tMon, 13 Nov 2017 15:47:17 -0800 (PST)"],"Authentication-Results":["ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"JjKJ0MEW\"; dkim-atps=neutral","lists.ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"JjKJ0MEW\"; dkim-atps=neutral","ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=ozlabs.ru\n\t(client-ip=2607:f8b0:400e:c00::244; helo=mail-pf0-x244.google.com;\n\tenvelope-from=aik@ozlabs.ru; receiver=)","lists.ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com header.b=\"JjKJ0MEW\"; \n\tdkim-atps=neutral"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=Hl76ccwwQRoIPBHUbPKWXUJNW75HQK9Yimn14UEprlU=;\n\tb=JjKJ0MEWjtOiPSkaulrd1o6kWU4c/vCealrSDU5aM5Fg0tc+M5n2PPbv9cgo1/H+Ey\n\taGfFZ3hjY/hC1LRhPayNjlXkfHL1Qe81Snw8/kro5WSEBhkhxJ58RU/xw9iYkC9/g2hm\n\trdJX5bK0b4WDsV27BV75nJR0EKJvC3Ty27A15vJGaVww4D7E0aJEtFgJVi+KZGekzLKX\n\t8Yh1KpcN12jCwE2qNSlhhAXxxmN7IaASzGJeEfDpQ53KYVPmvLBhqXmHU5++vNzI5zkx\n\tqHpogk5GtmHpzEcAsrcIR08cRghS5Hb9Rs6ar5YR6mO7ci8nMP/D0hJ/s9la2+qTsex1\n\tv+QQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=Hl76ccwwQRoIPBHUbPKWXUJNW75HQK9Yimn14UEprlU=;\n\tb=StKq+aYIRWfwI3a+YsarzJ9ADwi5qNgVpRg2cQc32n3hNlJhscs9N15JXSq2sj9+Zt\n\tP6zTNj+2Z999x++KVIPfpVF8x98HNOpqdD88YT56vcRUnhNDqZ1Q3ER/t3AW9FmfXZFP\n\tOJnowuH1U5M87Bx5sNd8HHkD9/YvYSqgCKi1/NBKdgXBmAXexJpOBqcgpMwd81Bl+7Xl\n\ti7238llaewudl4BhuCZ2hAOanPO4DUfNemPZ4eHIkX9gw602svf0exb7ji/b3b4euw0N\n\tllwtTaXElWZJhjqTU1q8IWmeLYe6k3an+r/t6J0kEYgJNLWCDuHBRfjKrCesh0mNEBQi\n\tEH3w==","X-Gm-Message-State":"AJaThX7ugl5fwguyJwVAC+jGkTD0gpr/b3RvMCLW8vgiTxWLIRc302UX\n\tK9n+kOCMhCH38uoFP8QB/VUjdA==","X-Google-Smtp-Source":"AGs4zMY4GXPHgcnmy51pGJr4cJMfPEq7vZzypfCVnxVrDMvyI/WhyCfSSGOq2+QnA3l+/j2Clav2Jw==","X-Received":"by 10.99.159.25 with SMTP id g25mr6005776pge.176.1510616837935; \n\tMon, 13 Nov 2017 15:47:17 -0800 (PST)","Subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","To":"linuxppc-dev@lists.ozlabs.org","References":"<20171027030008.17200-1-aik@ozlabs.ru>","From":"Alexey Kardashevskiy ","Message-ID":"<8c58fcaa-dedc-e6be-44bf-2cb349ef9e38@ozlabs.ru>","Date":"Tue, 14 Nov 2017 10:47:12 +1100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.4.0","MIME-Version":"1.0","In-Reply-To":"<20171027030008.17200-1-aik@ozlabs.ru>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alex Williamson , kvm@vger.kernel.org,\n\tkvm-ppc@vger.kernel.org, David Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1806040,"web_url":"http://patchwork.ozlabs.org/comment/1806040/","msgid":"<20171116171301.6552869b@t450s.home>","date":"2017-11-17T00:13:01","subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","submitter":{"id":4123,"url":"http://patchwork.ozlabs.org/api/people/4123/","name":"Alex Williamson","email":"alex.williamson@redhat.com"},"content":"On Tue, 14 Nov 2017 10:47:12 +1100\nAlexey Kardashevskiy wrote:\n\n> On 27/10/17 14:00, Alexey Kardashevskiy wrote:\n> > This adds trace_map/trace_unmap tracepoints to spapr driver. Type1 already\n> > uses these via the IOMMU API (iommu_map/__iommu_unmap).\n> > \n> > Signed-off-by: Alexey Kardashevskiy \n\nIs this really legitimate to include tracepoints from a different\nsubsystem? The vfio type1 backend gets these trace points by virtue of\nit actually using the IOMMU API, it doesn't call them itself. I'm kind\nof surprised these are actually available to be called from a module.\nI suspect the way to do this is probably to define our own tracepoints\nin the vfio/spapr backend or insert tracepoints into the IOMMU layers\nthat that code calls into rather than masquerading as tracepoints from\na different subsystem. Right? Thanks,\n\nAlex\n\n> > ---\n> > \n> > Example:\n> > qemu-system-ppc-8655 [096] 724.662740: unmap: IOMMU: iova=0x000000003ffff000 size=4096 unmapped_size=4096\n> > qemu-system-ppc-8656 [104] 724.970912: map: IOMMU: iova=0x0800000000000000 paddr=0x00007ffef7ff0000 size=65536\n> > ---\n> > drivers/vfio/vfio_iommu_spapr_tce.c | 12 ++++++++++--\n> > 1 file changed, 10 insertions(+), 2 deletions(-)\n> > \n> > diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c\n> > index 63112c36ab2d..4531486c77c6 100644\n> > --- a/drivers/vfio/vfio_iommu_spapr_tce.c\n> > +++ b/drivers/vfio/vfio_iommu_spapr_tce.c\n> > @@ -22,6 +22,7 @@\n> > #include \n> > #include \n> > #include \n> > +#include \n> > \n> > #include \n> > #include \n> > @@ -502,17 +503,19 @@ static int tce_iommu_clear(struct tce_container *container,\n> > \t\tstruct iommu_table *tbl,\n> > \t\tunsigned long entry, unsigned long pages)\n> > {\n> > -\tunsigned long oldhpa;\n> > +\tunsigned long oldhpa, unmapped, firstentry = entry, totalpages = pages;\n> > \tlong ret;\n> > \tenum dma_data_direction direction;\n> > \n> > -\tfor ( ; pages; --pages, ++entry) {\n> > +\tfor (unmapped = 0; pages; --pages, ++entry) {\n> > \t\tdirection = DMA_NONE;\n> > \t\toldhpa = 0;\n> > \t\tret = iommu_tce_xchg(tbl, entry, &oldhpa, &direction);\n> > \t\tif (ret)\n> > \t\t\tcontinue;\n> > \n> > +\t\t++unmapped;\n> > +\n> > \t\tif (direction == DMA_NONE)\n> > \t\t\tcontinue;\n> > \n> > @@ -523,6 +526,9 @@ static int tce_iommu_clear(struct tce_container *container,\n> > \n> > \t\ttce_iommu_unuse_page(container, oldhpa);\n> > \t}\n> > +\ttrace_unmap(firstentry << tbl->it_page_shift,\n> > +\t\t\ttotalpages << tbl->it_page_shift,\n> > +\t\t\tunmapped << tbl->it_page_shift);\n> > \n> > \treturn 0;\n> > }\n> > @@ -965,6 +971,8 @@ static long tce_iommu_ioctl(void *iommu_data,\n> > \t\t\t\t\tdirection);\n> > \n> > \t\tiommu_flush_tce(tbl);\n> > +\t\tif (!ret)\n> > +\t\t\ttrace_map(param.iova, param.vaddr, param.size);\n> > \n> > \t\treturn ret;\n> > \t}\n> > \n> \n>","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3ydJWg6fHtz9ryr\n\tfor ;\n\tFri, 17 Nov 2017 11:14:23 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3ydJWg4qJ4zDrMZ\n\tfor ;\n\tFri, 17 Nov 2017 11:14:23 +1100 (AEDT)","from mx1.redhat.com (mx1.redhat.com [209.132.183.28])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3ydJVD0pxNzDrDh\n\tfor ;\n\tFri, 17 Nov 2017 11:13:07 +1100 (AEDT)","from smtp.corp.redhat.com\n\t(int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id DDB3A80F7A;\n\tFri, 17 Nov 2017 00:13:04 +0000 (UTC)","from t450s.home (ovpn-116-16.phx2.redhat.com [10.3.116.16])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id B2B115C3FD;\n\tFri, 17 Nov 2017 00:13:02 +0000 (UTC)"],"Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=redhat.com\n\t(client-ip=209.132.183.28; helo=mx1.redhat.com;\n\tenvelope-from=alex.williamson@redhat.com; receiver=)","Date":"Thu, 16 Nov 2017 17:13:01 -0700","From":"Alex Williamson ","To":"Alexey Kardashevskiy ","Subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","Message-ID":"<20171116171301.6552869b@t450s.home>","In-Reply-To":"<8c58fcaa-dedc-e6be-44bf-2cb349ef9e38@ozlabs.ru>","References":"<20171027030008.17200-1-aik@ozlabs.ru>\n\t<8c58fcaa-dedc-e6be-44bf-2cb349ef9e38@ozlabs.ru>","MIME-Version":"1.0","Content-Type":"text/plain; charset=US-ASCII","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.16","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.27]); Fri, 17 Nov 2017 00:13:05 +0000 (UTC)","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kvm@vger.kernel.org, Shuah Khan ,\n\tkvm-ppc@vger.kernel.org, iommu@lists.linux-foundation.org,\n\tlinuxppc-dev@lists.ozlabs.org, David Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1806139,"web_url":"http://patchwork.ozlabs.org/comment/1806139/","msgid":"<52a19cfb-431f-522b-11de-c1c47f09a4c8@ozlabs.ru>","date":"2017-11-17T06:58:32","subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 17/11/17 11:13, Alex Williamson wrote:\n> On Tue, 14 Nov 2017 10:47:12 +1100\n> Alexey Kardashevskiy wrote:\n> \n>> On 27/10/17 14:00, Alexey Kardashevskiy wrote:\n>>> This adds trace_map/trace_unmap tracepoints to spapr driver. Type1 already\n>>> uses these via the IOMMU API (iommu_map/__iommu_unmap).\n>>>\n>>> Signed-off-by: Alexey Kardashevskiy \n> \n> Is this really legitimate to include tracepoints from a different\n> subsystem?> The vfio type1 backend gets these trace points by virtue of\n> it actually using the IOMMU API, it doesn't call them itself. I'm kind\n> of surprised these are actually available to be called from a module.\n\nThey are explicitly exported:\n\nEXPORT_TRACEPOINT_SYMBOL_GPL(map);\nEXPORT_TRACEPOINT_SYMBOL_GPL(unmap);\n\nI would think this is for things like drivers/vfio/vfio_iommu_spapr_tce.c ,\nwhy else?...\n\n\n> I suspect the way to do this is probably to define our own tracepoints\n> in the vfio/spapr backend or insert tracepoints into the IOMMU layers\n> that that code calls into rather than masquerading as tracepoints from\n> a different subsystem. Right?\n\nThis makes sense too. But it is going to be just cut-n-paste and some\nconfusion -\n/sys/kernel/debug/tracing/events/iommu/map will still be present but\nwon't work and\n/sys/kernel/debug/tracing/events/vfio/vfio_iommu_spapr_tce/map will.\n\nJudges? :)\n\n\n\n\n> Thanks,\n> \n> Alex\n> \n>>> ---\n>>>\n>>> Example:\n>>> qemu-system-ppc-8655 [096] 724.662740: unmap: IOMMU: iova=0x000000003ffff000 size=4096 unmapped_size=4096\n>>> qemu-system-ppc-8656 [104] 724.970912: map: IOMMU: iova=0x0800000000000000 paddr=0x00007ffef7ff0000 size=65536\n>>> ---\n>>> drivers/vfio/vfio_iommu_spapr_tce.c | 12 ++++++++++--\n>>> 1 file changed, 10 insertions(+), 2 deletions(-)\n>>>\n>>> diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c\n>>> index 63112c36ab2d..4531486c77c6 100644\n>>> --- a/drivers/vfio/vfio_iommu_spapr_tce.c\n>>> +++ b/drivers/vfio/vfio_iommu_spapr_tce.c\n>>> @@ -22,6 +22,7 @@\n>>> #include \n>>> #include \n>>> #include \n>>> +#include \n>>> \n>>> #include \n>>> #include \n>>> @@ -502,17 +503,19 @@ static int tce_iommu_clear(struct tce_container *container,\n>>> \t\tstruct iommu_table *tbl,\n>>> \t\tunsigned long entry, unsigned long pages)\n>>> {\n>>> -\tunsigned long oldhpa;\n>>> +\tunsigned long oldhpa, unmapped, firstentry = entry, totalpages = pages;\n>>> \tlong ret;\n>>> \tenum dma_data_direction direction;\n>>> \n>>> -\tfor ( ; pages; --pages, ++entry) {\n>>> +\tfor (unmapped = 0; pages; --pages, ++entry) {\n>>> \t\tdirection = DMA_NONE;\n>>> \t\toldhpa = 0;\n>>> \t\tret = iommu_tce_xchg(tbl, entry, &oldhpa, &direction);\n>>> \t\tif (ret)\n>>> \t\t\tcontinue;\n>>> \n>>> +\t\t++unmapped;\n>>> +\n>>> \t\tif (direction == DMA_NONE)\n>>> \t\t\tcontinue;\n>>> \n>>> @@ -523,6 +526,9 @@ static int tce_iommu_clear(struct tce_container *container,\n>>> \n>>> \t\ttce_iommu_unuse_page(container, oldhpa);\n>>> \t}\n>>> +\ttrace_unmap(firstentry << tbl->it_page_shift,\n>>> +\t\t\ttotalpages << tbl->it_page_shift,\n>>> +\t\t\tunmapped << tbl->it_page_shift);\n>>> \n>>> \treturn 0;\n>>> }\n>>> @@ -965,6 +971,8 @@ static long tce_iommu_ioctl(void *iommu_data,\n>>> \t\t\t\t\tdirection);\n>>> \n>>> \t\tiommu_flush_tce(tbl);\n>>> +\t\tif (!ret)\n>>> +\t\t\ttrace_map(param.iova, param.vaddr, param.size);\n>>> \n>>> \t\treturn ret;\n>>> \t}\n>>> \n>>\n>>\n>","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3ydTWt3fyVz9rxj\n\tfor ;\n\tFri, 17 Nov 2017 18:00:10 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3ydTWt1zxdzDrKc\n\tfor ;\n\tFri, 17 Nov 2017 18:00:10 +1100 (AEDT)","from mail-it0-x242.google.com (mail-it0-x242.google.com\n\t[IPv6:2607:f8b0:4001:c0b::242])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3ydTVC6c6zzDrK3\n\tfor ;\n\tFri, 17 Nov 2017 17:58:41 +1100 (AEDT)","by mail-it0-x242.google.com with SMTP id b5so2926526itc.3\n\tfor ;\n\tThu, 16 Nov 2017 22:58:41 -0800 (PST)","from [192.168.10.22] (124-171-65-237.dyn.iinet.net.au.\n\t[124.171.65.237]) by smtp.googlemail.com with ESMTPSA id\n\tm72sm1143541ioe.40.2017.11.16.22.58.35\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tThu, 16 Nov 2017 22:58:38 -0800 (PST)"],"Authentication-Results":["ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"EfdcdYsp\"; dkim-atps=neutral","lists.ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"EfdcdYsp\"; dkim-atps=neutral","ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=ozlabs.ru\n\t(client-ip=2607:f8b0:4001:c0b::242; helo=mail-it0-x242.google.com;\n\tenvelope-from=aik@ozlabs.ru; receiver=)","lists.ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com header.b=\"EfdcdYsp\"; \n\tdkim-atps=neutral"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=O6MHq0ClggkK3s1Z4YgnCxnuSB4q6ouxDDTLGmaRfAE=;\n\tb=EfdcdYsp+J4G4LhttJ99SH/Jgh9Zedd10QPH0T+/LwvQ3kBAbsdMmhntXd95GTWx8r\n\tSt+2HS1tNSOpXYBOWRnh/7UuP/dOep1vfHy/VUAyPJraKauqFROEExb2IJKLMImy3FiW\n\ttqDTokCieNP9mQdGJFhjluPidOFROx9Nf3VRS5evIfpzAYjdnCD7gIu9OTpI935x7865\n\t/G6/tgZynhKEnQaBEEI5OqVB56fCffZcxpqM0xjpRjl5qAbxULhKFtZDdeuO5JzL2b4i\n\toW74JgQABmQUGxPuIKb0nxToBx3YWL3BamzDxJv6IlVUAO1EsRSn7nLqeFK1NEXGyWRn\n\tFpgw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=O6MHq0ClggkK3s1Z4YgnCxnuSB4q6ouxDDTLGmaRfAE=;\n\tb=TvO0qMtc/RqnC8PD4gq9ke54bPRbOF80tME7z1y4En9/c61r6FI5HKG5De5EJYbGIf\n\tdwda/jXyVc4hm1MvqYfGUd/MR3u0O1SAkI3CKO9ma3GSFzFFvQl62T1cnbdKojG5uOYI\n\trZK6J3S69eORd1lRk+HcpEC+6+Vs81wnJ/l6W36lvkLgecViAzppKQLi4YwKbaAaWN8W\n\twFnN5NDHo38sqizRFobhy92aSXF1aXjRWVert6749LrOht1PHR8zhTlV9ypG3/g+ezRY\n\tsLiPW414ZLlOPg0JLmvPbn4ngNVGlP2Fw/fhUXDzv6l0UMvil6M5ryVCIHLjnm3Tk4vn\n\t6gfg==","X-Gm-Message-State":"AJaThX6YavNQnEGxcvm8RCJpGi0ll9vogu+4sozQAz5gKyv8rlkraGnB\n\t/dX2juqCKYXZs7QxAbWUTRvkQQ==","X-Google-Smtp-Source":"AGs4zMYWzGVLX8CUcQ4dSYEsvAn2GjlJbJq7KJt4huyzR5qcJYYymYnqTTuyGWoi59aZDxDuAp4Pcg==","X-Received":"by 10.36.86.10 with SMTP id o10mr458773itb.86.1510901919543;\n\tThu, 16 Nov 2017 22:58:39 -0800 (PST)","Subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","To":"Alex Williamson ","References":"<20171027030008.17200-1-aik@ozlabs.ru>\n\t<8c58fcaa-dedc-e6be-44bf-2cb349ef9e38@ozlabs.ru>\n\t<20171116171301.6552869b@t450s.home>","From":"Alexey Kardashevskiy ","Message-ID":"<52a19cfb-431f-522b-11de-c1c47f09a4c8@ozlabs.ru>","Date":"Fri, 17 Nov 2017 17:58:32 +1100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.4.0","MIME-Version":"1.0","In-Reply-To":"<20171116171301.6552869b@t450s.home>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kvm@vger.kernel.org, Joerg Roedel ,\n\tShuah Khan , kvm-ppc@vger.kernel.org,\n\tiommu@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org,\n\tDavid Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1809037,"web_url":"http://patchwork.ozlabs.org/comment/1809037/","msgid":"","date":"2017-11-23T04:13:37","subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 17/11/17 17:58, Alexey Kardashevskiy wrote:\n> On 17/11/17 11:13, Alex Williamson wrote:\n>> On Tue, 14 Nov 2017 10:47:12 +1100\n>> Alexey Kardashevskiy wrote:\n>>\n>>> On 27/10/17 14:00, Alexey Kardashevskiy wrote:\n>>>> This adds trace_map/trace_unmap tracepoints to spapr driver. Type1 already\n>>>> uses these via the IOMMU API (iommu_map/__iommu_unmap).\n>>>>\n>>>> Signed-off-by: Alexey Kardashevskiy \n>>\n>> Is this really legitimate to include tracepoints from a different\n>> subsystem?> The vfio type1 backend gets these trace points by virtue of\n>> it actually using the IOMMU API, it doesn't call them itself. I'm kind\n>> of surprised these are actually available to be called from a module.\n> \n> They are explicitly exported:\n> \n> EXPORT_TRACEPOINT_SYMBOL_GPL(map);\n> EXPORT_TRACEPOINT_SYMBOL_GPL(unmap);\n> \n> I would think this is for things like drivers/vfio/vfio_iommu_spapr_tce.c ,\n> why else?...\n> \n> \n>> I suspect the way to do this is probably to define our own tracepoints\n>> in the vfio/spapr backend or insert tracepoints into the IOMMU layers\n>> that that code calls into rather than masquerading as tracepoints from\n>> a different subsystem. Right?\n> \n> This makes sense too. But it is going to be just cut-n-paste and some\n> confusion -\n> /sys/kernel/debug/tracing/events/iommu/map will still be present but\n> won't work and\n> /sys/kernel/debug/tracing/events/vfio/vfio_iommu_spapr_tce/map will.\n> \n> Judges? :)\n\n\nStill nak? I discussed this locally, the conclusion was it is a matter of\ntaste and this proposal is not that disgusting. Thanks.\n\n\n> \n> \n> \n>> Thanks,\n>>\n>> Alex\n>>\n>>>> ---\n>>>>\n>>>> Example:\n>>>> qemu-system-ppc-8655 [096] 724.662740: unmap: IOMMU: iova=0x000000003ffff000 size=4096 unmapped_size=4096\n>>>> qemu-system-ppc-8656 [104] 724.970912: map: IOMMU: iova=0x0800000000000000 paddr=0x00007ffef7ff0000 size=65536\n>>>> ---\n>>>> drivers/vfio/vfio_iommu_spapr_tce.c | 12 ++++++++++--\n>>>> 1 file changed, 10 insertions(+), 2 deletions(-)\n>>>>\n>>>> diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c\n>>>> index 63112c36ab2d..4531486c77c6 100644\n>>>> --- a/drivers/vfio/vfio_iommu_spapr_tce.c\n>>>> +++ b/drivers/vfio/vfio_iommu_spapr_tce.c\n>>>> @@ -22,6 +22,7 @@\n>>>> #include \n>>>> #include \n>>>> #include \n>>>> +#include \n>>>> \n>>>> #include \n>>>> #include \n>>>> @@ -502,17 +503,19 @@ static int tce_iommu_clear(struct tce_container *container,\n>>>> \t\tstruct iommu_table *tbl,\n>>>> \t\tunsigned long entry, unsigned long pages)\n>>>> {\n>>>> -\tunsigned long oldhpa;\n>>>> +\tunsigned long oldhpa, unmapped, firstentry = entry, totalpages = pages;\n>>>> \tlong ret;\n>>>> \tenum dma_data_direction direction;\n>>>> \n>>>> -\tfor ( ; pages; --pages, ++entry) {\n>>>> +\tfor (unmapped = 0; pages; --pages, ++entry) {\n>>>> \t\tdirection = DMA_NONE;\n>>>> \t\toldhpa = 0;\n>>>> \t\tret = iommu_tce_xchg(tbl, entry, &oldhpa, &direction);\n>>>> \t\tif (ret)\n>>>> \t\t\tcontinue;\n>>>> \n>>>> +\t\t++unmapped;\n>>>> +\n>>>> \t\tif (direction == DMA_NONE)\n>>>> \t\t\tcontinue;\n>>>> \n>>>> @@ -523,6 +526,9 @@ static int tce_iommu_clear(struct tce_container *container,\n>>>> \n>>>> \t\ttce_iommu_unuse_page(container, oldhpa);\n>>>> \t}\n>>>> +\ttrace_unmap(firstentry << tbl->it_page_shift,\n>>>> +\t\t\ttotalpages << tbl->it_page_shift,\n>>>> +\t\t\tunmapped << tbl->it_page_shift);\n>>>> \n>>>> \treturn 0;\n>>>> }\n>>>> @@ -965,6 +971,8 @@ static long tce_iommu_ioctl(void *iommu_data,\n>>>> \t\t\t\t\tdirection);\n>>>> \n>>>> \t\tiommu_flush_tce(tbl);\n>>>> +\t\tif (!ret)\n>>>> +\t\t\ttrace_map(param.iova, param.vaddr, param.size);\n>>>> \n>>>> \t\treturn ret;\n>>>> \t}\n>>>> \n>>>\n>>>\n>>\n> \n>","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yj5bT2djQz9t16\n\tfor ;\n\tThu, 23 Nov 2017 15:15:49 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yj5bT1TrpzDrp7\n\tfor ;\n\tThu, 23 Nov 2017 15:15:49 +1100 (AEDT)","from mail-pl0-x243.google.com (mail-pl0-x243.google.com\n\t[IPv6:2607:f8b0:400e:c01::243])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128\n\tbits)) (No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yj5Y53XxbzDrRp\n\tfor ;\n\tThu, 23 Nov 2017 15:13:45 +1100 (AEDT)","by mail-pl0-x243.google.com with SMTP id h3so1985349pln.10\n\tfor ;\n\tWed, 22 Nov 2017 20:13:45 -0800 (PST)","from [10.61.2.175] ([122.99.82.10])\n\tby smtp.googlemail.com with ESMTPSA id\n\tk8sm25126076pgt.22.2017.11.22.20.13.39\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tWed, 22 Nov 2017 20:13:42 -0800 (PST)"],"Authentication-Results":["ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"F4Xfml5a\"; dkim-atps=neutral","lists.ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"F4Xfml5a\"; dkim-atps=neutral","ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=ozlabs.ru\n\t(client-ip=2607:f8b0:400e:c01::243; helo=mail-pl0-x243.google.com;\n\tenvelope-from=aik@ozlabs.ru; receiver=)","lists.ozlabs.org; dkim=pass (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com header.b=\"F4Xfml5a\"; \n\tdkim-atps=neutral"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:from:to:cc:references:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=ifr2FLjRMQrlhaobRrSsMmHkz8QQ+YgfzYXzTsjI1EE=;\n\tb=F4Xfml5a64PU+pk5hvrcQBsdj6L1IgPRjBedMwMZReRN6BhB8UyevHI0WggU20tcV5\n\tQlACHCLajlAkWquRXtl4PFutiRk2glbbtwKgSRw6flFPWTHAh9DITzu+FVRFuiLQHJxR\n\tGeVuKCzmHJMUsXEi25cDczmBkhkN3kP6hXX5on/HPyUTeVUlnkVg5mjtHgogCwJnLbUs\n\tM0zNkR4QvZMnumIMLT0iSyMupbdYdvZwfNVPwbsUQNvuhVU4UgrYs2vMuMNNPKz/dB8Z\n\tvuoTF6m/EQkCbII6B2EpT9QxEF7mgDv+8rYuxGOw7AN9KWZ1yEBBSDP//E63MclLN8xr\n\tOyFw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:from:to:cc:references:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=ifr2FLjRMQrlhaobRrSsMmHkz8QQ+YgfzYXzTsjI1EE=;\n\tb=Mz3yqbMrXtcE7thABlN9+hVH0DDCBM47bP7Pgw38L4kwiy6kMrQFyK2xk0OTlcfQvI\n\tC3Hs9FsWZNfkwKXfkeKfyGLmUIBjcja2S/AMDph8zo6lD9G0DwuerXmLUPWh6oT+ygnh\n\tes76/aiDa9kCYqxRjF4EUJTkDG3L+BYUDRnS9/jcasimCtsyId9Wjs4OmB86Pf1SJ88y\n\tW2BgNhNgAJXBKKTt6zC6NDvo5FJr9WZuzivP/69y4Fh5l64qKgn6ALSBrNsSTGG6k+tD\n\tpDHo44BDwNHBDd84r6FUI/9cN/TruD/nebPAFqg1/CGtiA74hJhrNpTaTckLLB0LkGuq\n\tI//w==","X-Gm-Message-State":"AJaThX57dPODisNWB16XphDGPTe4G3B7QsAVvntZ31cBuSsi8ozsJo99\n\txl96ApFM0vPGXQK8s092UYYllgAF","X-Google-Smtp-Source":"AGs4zMaeEd/yaUp/JW1s/Jscw4aIf/HZ8Z8eQT11mwhJoDaAVBFdRssX/zSuwmvcIfvok8y+FBiFVA==","X-Received":"by 10.84.160.233 with SMTP id v38mr453863plg.179.1511410423301; \n\tWed, 22 Nov 2017 20:13:43 -0800 (PST)","Subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","From":"Alexey Kardashevskiy ","To":"Alex Williamson ","References":"<20171027030008.17200-1-aik@ozlabs.ru>\n\t<8c58fcaa-dedc-e6be-44bf-2cb349ef9e38@ozlabs.ru>\n\t<20171116171301.6552869b@t450s.home>\n\t<52a19cfb-431f-522b-11de-c1c47f09a4c8@ozlabs.ru>","Message-ID":"","Date":"Thu, 23 Nov 2017 15:13:37 +1100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.4.0","MIME-Version":"1.0","In-Reply-To":"<52a19cfb-431f-522b-11de-c1c47f09a4c8@ozlabs.ru>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kvm@vger.kernel.org, Joerg Roedel ,\n\tShuah Khan , kvm-ppc@vger.kernel.org,\n\tiommu@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org,\n\tDavid Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1812436,"web_url":"http://patchwork.ozlabs.org/comment/1812436/","msgid":"<20171129093244.0849fd7c@t450s.home>","date":"2017-11-29T16:32:44","subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","submitter":{"id":4123,"url":"http://patchwork.ozlabs.org/api/people/4123/","name":"Alex Williamson","email":"alex.williamson@redhat.com"},"content":"On Thu, 23 Nov 2017 15:13:37 +1100\nAlexey Kardashevskiy wrote:\n\n> On 17/11/17 17:58, Alexey Kardashevskiy wrote:\n> > On 17/11/17 11:13, Alex Williamson wrote: \n> >> On Tue, 14 Nov 2017 10:47:12 +1100\n> >> Alexey Kardashevskiy wrote:\n> >> \n> >>> On 27/10/17 14:00, Alexey Kardashevskiy wrote: \n> >>>> This adds trace_map/trace_unmap tracepoints to spapr driver. Type1 already\n> >>>> uses these via the IOMMU API (iommu_map/__iommu_unmap).\n> >>>>\n> >>>> Signed-off-by: Alexey Kardashevskiy \n> >>\n> >> Is this really legitimate to include tracepoints from a different \n> >> subsystem?> The vfio type1 backend gets these trace points by virtue of \n> >> it actually using the IOMMU API, it doesn't call them itself. I'm kind\n> >> of surprised these are actually available to be called from a module. \n> > \n> > They are explicitly exported:\n> > \n> > EXPORT_TRACEPOINT_SYMBOL_GPL(map);\n> > EXPORT_TRACEPOINT_SYMBOL_GPL(unmap);\n> > \n> > I would think this is for things like drivers/vfio/vfio_iommu_spapr_tce.c ,\n> > why else?...\n> > \n> > \n> >> I suspect the way to do this is probably to define our own tracepoints\n> >> in the vfio/spapr backend or insert tracepoints into the IOMMU layers\n> >> that that code calls into rather than masquerading as tracepoints from\n> >> a different subsystem. Right? \n> > \n> > This makes sense too. But it is going to be just cut-n-paste and some\n> > confusion -\n> > /sys/kernel/debug/tracing/events/iommu/map will still be present but\n> > won't work and\n> > /sys/kernel/debug/tracing/events/vfio/vfio_iommu_spapr_tce/map will.\n\nBut iommu/map does work, it's just not called by the vfio spapr tce\nbackend, which is absolutely correct. In fact, that's part of my\nreservation about this approach is that it could be interpreted as\nimplying a call path that doesn't exist on this arch.\n\n> > Judges? :) \n> \n> \n> Still nak? I discussed this locally, the conclusion was it is a matter of\n> taste and this proposal is not that disgusting. Thanks.\n\nIs that our goal now, proposals that aren't too disgusting? I think\nit's an opportunity to introduce our own tracing infrastructure and it\nfeels a lot more correct to do that than to generalize existing trace\npoints from other subsystems. Thanks,\n\nAlex\n\n\n> >>>> ---\n> >>>>\n> >>>> Example:\n> >>>> qemu-system-ppc-8655 [096] 724.662740: unmap: IOMMU: iova=0x000000003ffff000 size=4096 unmapped_size=4096\n> >>>> qemu-system-ppc-8656 [104] 724.970912: map: IOMMU: iova=0x0800000000000000 paddr=0x00007ffef7ff0000 size=65536\n> >>>> ---\n> >>>> drivers/vfio/vfio_iommu_spapr_tce.c | 12 ++++++++++--\n> >>>> 1 file changed, 10 insertions(+), 2 deletions(-)\n> >>>>\n> >>>> diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c\n> >>>> index 63112c36ab2d..4531486c77c6 100644\n> >>>> --- a/drivers/vfio/vfio_iommu_spapr_tce.c\n> >>>> +++ b/drivers/vfio/vfio_iommu_spapr_tce.c\n> >>>> @@ -22,6 +22,7 @@\n> >>>> #include \n> >>>> #include \n> >>>> #include \n> >>>> +#include \n> >>>> \n> >>>> #include \n> >>>> #include \n> >>>> @@ -502,17 +503,19 @@ static int tce_iommu_clear(struct tce_container *container,\n> >>>> \t\tstruct iommu_table *tbl,\n> >>>> \t\tunsigned long entry, unsigned long pages)\n> >>>> {\n> >>>> -\tunsigned long oldhpa;\n> >>>> +\tunsigned long oldhpa, unmapped, firstentry = entry, totalpages = pages;\n> >>>> \tlong ret;\n> >>>> \tenum dma_data_direction direction;\n> >>>> \n> >>>> -\tfor ( ; pages; --pages, ++entry) {\n> >>>> +\tfor (unmapped = 0; pages; --pages, ++entry) {\n> >>>> \t\tdirection = DMA_NONE;\n> >>>> \t\toldhpa = 0;\n> >>>> \t\tret = iommu_tce_xchg(tbl, entry, &oldhpa, &direction);\n> >>>> \t\tif (ret)\n> >>>> \t\t\tcontinue;\n> >>>> \n> >>>> +\t\t++unmapped;\n> >>>> +\n> >>>> \t\tif (direction == DMA_NONE)\n> >>>> \t\t\tcontinue;\n> >>>> \n> >>>> @@ -523,6 +526,9 @@ static int tce_iommu_clear(struct tce_container *container,\n> >>>> \n> >>>> \t\ttce_iommu_unuse_page(container, oldhpa);\n> >>>> \t}\n> >>>> +\ttrace_unmap(firstentry << tbl->it_page_shift,\n> >>>> +\t\t\ttotalpages << tbl->it_page_shift,\n> >>>> +\t\t\tunmapped << tbl->it_page_shift);\n> >>>> \n> >>>> \treturn 0;\n> >>>> }\n> >>>> @@ -965,6 +971,8 @@ static long tce_iommu_ioctl(void *iommu_data,\n> >>>> \t\t\t\t\tdirection);\n> >>>> \n> >>>> \t\tiommu_flush_tce(tbl);\n> >>>> +\t\tif (!ret)\n> >>>> +\t\t\ttrace_map(param.iova, param.vaddr, param.size);\n> >>>> \n> >>>> \t\treturn ret;\n> >>>> \t}\n> >>>> \n> >>>\n> >>> \n> >> \n> > \n> > \n> \n>","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yn5hx0q4Dz9sBZ\n\tfor ;\n\tThu, 30 Nov 2017 03:34:25 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yn5hw6vQWzDrbj\n\tfor ;\n\tThu, 30 Nov 2017 03:34:24 +1100 (AEDT)","from mx1.redhat.com (mx1.redhat.com [209.132.183.28])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby lists.ozlabs.org (Postfix) with ESMTPS id 3yn5g42XyVzDrb7\n\tfor ;\n\tThu, 30 Nov 2017 03:32:48 +1100 (AEDT)","from smtp.corp.redhat.com\n\t(int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 2C32813A53;\n\tWed, 29 Nov 2017 16:32:46 +0000 (UTC)","from t450s.home (ovpn-116-16.phx2.redhat.com [10.3.116.16])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id F3FC85D753;\n\tWed, 29 Nov 2017 16:32:44 +0000 (UTC)"],"Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=redhat.com\n\t(client-ip=209.132.183.28; helo=mx1.redhat.com;\n\tenvelope-from=alex.williamson@redhat.com; receiver=)","Date":"Wed, 29 Nov 2017 09:32:44 -0700","From":"Alex Williamson ","To":"Alexey Kardashevskiy ","Subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","Message-ID":"<20171129093244.0849fd7c@t450s.home>","In-Reply-To":"","References":"<20171027030008.17200-1-aik@ozlabs.ru>\n\t<8c58fcaa-dedc-e6be-44bf-2cb349ef9e38@ozlabs.ru>\n\t<20171116171301.6552869b@t450s.home>\n\t<52a19cfb-431f-522b-11de-c1c47f09a4c8@ozlabs.ru>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=US-ASCII","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.15","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.29]); Wed, 29 Nov 2017 16:32:46 +0000 (UTC)","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kvm@vger.kernel.org, Joerg Roedel ,\n\tShuah Khan , kvm-ppc@vger.kernel.org,\n\tiommu@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org,\n\tDavid Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1812452,"web_url":"http://patchwork.ozlabs.org/comment/1812452/","msgid":"<24f7ea0f-8a18-752f-6841-7001f88d1e00@arm.com>","date":"2017-11-29T16:43:46","subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","submitter":{"id":65641,"url":"http://patchwork.ozlabs.org/api/people/65641/","name":"Robin Murphy","email":"robin.murphy@arm.com"},"content":"On 29/11/17 16:32, Alex Williamson wrote:\n> On Thu, 23 Nov 2017 15:13:37 +1100\n> Alexey Kardashevskiy wrote:\n> \n>> On 17/11/17 17:58, Alexey Kardashevskiy wrote:\n>>> On 17/11/17 11:13, Alex Williamson wrote:\n>>>> On Tue, 14 Nov 2017 10:47:12 +1100\n>>>> Alexey Kardashevskiy wrote:\n>>>> \n>>>>> On 27/10/17 14:00, Alexey Kardashevskiy wrote:\n>>>>>> This adds trace_map/trace_unmap tracepoints to spapr driver. Type1 already\n>>>>>> uses these via the IOMMU API (iommu_map/__iommu_unmap).\n>>>>>>\n>>>>>> Signed-off-by: Alexey Kardashevskiy \n>>>>\n>>>> Is this really legitimate to include tracepoints from a different\n>>>> subsystem?> The vfio type1 backend gets these trace points by virtue of\n>>>> it actually using the IOMMU API, it doesn't call them itself. I'm kind\n>>>> of surprised these are actually available to be called from a module.\n>>>\n>>> They are explicitly exported:\n>>>\n>>> EXPORT_TRACEPOINT_SYMBOL_GPL(map);\n>>> EXPORT_TRACEPOINT_SYMBOL_GPL(unmap);\n>>>\n>>> I would think this is for things like drivers/vfio/vfio_iommu_spapr_tce.c ,\n>>> why else?...\n>>>\n>>> \n>>>> I suspect the way to do this is probably to define our own tracepoints\n>>>> in the vfio/spapr backend or insert tracepoints into the IOMMU layers\n>>>> that that code calls into rather than masquerading as tracepoints from\n>>>> a different subsystem. Right?\n>>>\n>>> This makes sense too. But it is going to be just cut-n-paste and some\n>>> confusion -\n>>> /sys/kernel/debug/tracing/events/iommu/map will still be present but\n>>> won't work and\n>>> /sys/kernel/debug/tracing/events/vfio/vfio_iommu_spapr_tce/map will.\n> \n> But iommu/map does work, it's just not called by the vfio spapr tce\n> backend, which is absolutely correct. In fact, that's part of my\n> reservation about this approach is that it could be interpreted as\n> implying a call path that doesn't exist on this arch.\n\nCome to think of it, the inverse also applies - if users are led to \nexpect x86-like behaviour where the iommu/map tracepoint just happens to \nreflect VFIO calls, they'll likely also get a nasty surprise on \nplatforms like arm64 where regular DMA API calls by drivers may be \nhammering iommu_map() in the background.\n\nRobin.\n\n>>> Judges? :)\n>>\n>>\n>> Still nak? I discussed this locally, the conclusion was it is a matter of\n>> taste and this proposal is not that disgusting. Thanks.\n> \n> Is that our goal now, proposals that aren't too disgusting? I think\n> it's an opportunity to introduce our own tracing infrastructure and it\n> feels a lot more correct to do that than to generalize existing trace\n> points from other subsystems. Thanks,\n> \n> Alex\n> \n> \n>>>>>> ---\n>>>>>>\n>>>>>> Example:\n>>>>>> qemu-system-ppc-8655 [096] 724.662740: unmap: IOMMU: iova=0x000000003ffff000 size=4096 unmapped_size=4096\n>>>>>> qemu-system-ppc-8656 [104] 724.970912: map: IOMMU: iova=0x0800000000000000 paddr=0x00007ffef7ff0000 size=65536\n>>>>>> ---\n>>>>>> drivers/vfio/vfio_iommu_spapr_tce.c | 12 ++++++++++--\n>>>>>> 1 file changed, 10 insertions(+), 2 deletions(-)\n>>>>>>\n>>>>>> diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c\n>>>>>> index 63112c36ab2d..4531486c77c6 100644\n>>>>>> --- a/drivers/vfio/vfio_iommu_spapr_tce.c\n>>>>>> +++ b/drivers/vfio/vfio_iommu_spapr_tce.c\n>>>>>> @@ -22,6 +22,7 @@\n>>>>>> #include \n>>>>>> #include \n>>>>>> #include \n>>>>>> +#include \n>>>>>> \n>>>>>> #include \n>>>>>> #include \n>>>>>> @@ -502,17 +503,19 @@ static int tce_iommu_clear(struct tce_container *container,\n>>>>>> \t\tstruct iommu_table *tbl,\n>>>>>> \t\tunsigned long entry, unsigned long pages)\n>>>>>> {\n>>>>>> -\tunsigned long oldhpa;\n>>>>>> +\tunsigned long oldhpa, unmapped, firstentry = entry, totalpages = pages;\n>>>>>> \tlong ret;\n>>>>>> \tenum dma_data_direction direction;\n>>>>>> \n>>>>>> -\tfor ( ; pages; --pages, ++entry) {\n>>>>>> +\tfor (unmapped = 0; pages; --pages, ++entry) {\n>>>>>> \t\tdirection = DMA_NONE;\n>>>>>> \t\toldhpa = 0;\n>>>>>> \t\tret = iommu_tce_xchg(tbl, entry, &oldhpa, &direction);\n>>>>>> \t\tif (ret)\n>>>>>> \t\t\tcontinue;\n>>>>>> \n>>>>>> +\t\t++unmapped;\n>>>>>> +\n>>>>>> \t\tif (direction == DMA_NONE)\n>>>>>> \t\t\tcontinue;\n>>>>>> \n>>>>>> @@ -523,6 +526,9 @@ static int tce_iommu_clear(struct tce_container *container,\n>>>>>> \n>>>>>> \t\ttce_iommu_unuse_page(container, oldhpa);\n>>>>>> \t}\n>>>>>> +\ttrace_unmap(firstentry << tbl->it_page_shift,\n>>>>>> +\t\t\ttotalpages << tbl->it_page_shift,\n>>>>>> +\t\t\tunmapped << tbl->it_page_shift);\n>>>>>> \n>>>>>> \treturn 0;\n>>>>>> }\n>>>>>> @@ -965,6 +971,8 @@ static long tce_iommu_ioctl(void *iommu_data,\n>>>>>> \t\t\t\t\tdirection);\n>>>>>> \n>>>>>> \t\tiommu_flush_tce(tbl);\n>>>>>> +\t\tif (!ret)\n>>>>>> +\t\t\ttrace_map(param.iova, param.vaddr, param.size);\n>>>>>> \n>>>>>> \t\treturn ret;\n>>>>>> \t}\n>>>>>> \n>>>>>\n>>>>> \n>>>> \n>>>\n>>> \n>>\n>>\n> \n> _______________________________________________\n> iommu mailing list\n> iommu@lists.linux-foundation.org\n> https://lists.linuxfoundation.org/mailman/listinfo/iommu\n>","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yn5yb4GHkz9s5L\n\tfor ;\n\tThu, 30 Nov 2017 03:46:15 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yn5yb1jpMzDrny\n\tfor ;\n\tThu, 30 Nov 2017 03:46:15 +1100 (AEDT)","from foss.arm.com (foss.arm.com [217.140.101.70])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yn5vt1tFqzDrLW\n\tfor ;\n\tThu, 30 Nov 2017 03:43:53 +1100 (AEDT)","from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249])\n\tby usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A344E1435;\n\tWed, 29 Nov 2017 08:43:49 -0800 (PST)","from [10.1.210.88] (e110467-lin.cambridge.arm.com [10.1.210.88])\n\tby usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id\n\t462F83F236; Wed, 29 Nov 2017 08:43:48 -0800 (PST)"],"Authentication-Results":"ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=arm.com\n\t(client-ip=217.140.101.70; helo=foss.arm.com;\n\tenvelope-from=robin.murphy@arm.com; receiver=)","Subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","To":"Alex Williamson ,\n\tAlexey Kardashevskiy ","References":"<20171027030008.17200-1-aik@ozlabs.ru>\n\t<8c58fcaa-dedc-e6be-44bf-2cb349ef9e38@ozlabs.ru>\n\t<20171116171301.6552869b@t450s.home>\n\t<52a19cfb-431f-522b-11de-c1c47f09a4c8@ozlabs.ru>\n\t\n\t<20171129093244.0849fd7c@t450s.home>","From":"Robin Murphy ","Message-ID":"<24f7ea0f-8a18-752f-6841-7001f88d1e00@arm.com>","Date":"Wed, 29 Nov 2017 16:43:46 +0000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.4.0","MIME-Version":"1.0","In-Reply-To":"<20171129093244.0849fd7c@t450s.home>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kvm@vger.kernel.org, Shuah Khan ,\n\tkvm-ppc@vger.kernel.org, iommu@lists.linux-foundation.org,\n\tlinuxppc-dev@lists.ozlabs.org, David Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}},{"id":1812455,"web_url":"http://patchwork.ozlabs.org/comment/1812455/","msgid":"","date":"2017-11-29T16:44:13","subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","submitter":{"id":65093,"url":"http://patchwork.ozlabs.org/api/people/65093/","name":"Shuah Khan","email":"shuahkh@osg.samsung.com"},"content":"On 11/29/2017 09:32 AM, Alex Williamson wrote:\n> On Thu, 23 Nov 2017 15:13:37 +1100\n> Alexey Kardashevskiy wrote:\n> \n>> On 17/11/17 17:58, Alexey Kardashevskiy wrote:\n>>> On 17/11/17 11:13, Alex Williamson wrote: \n>>>> On Tue, 14 Nov 2017 10:47:12 +1100\n>>>> Alexey Kardashevskiy wrote:\n>>>> \n>>>>> On 27/10/17 14:00, Alexey Kardashevskiy wrote: \n>>>>>> This adds trace_map/trace_unmap tracepoints to spapr driver. Type1 already\n>>>>>> uses these via the IOMMU API (iommu_map/__iommu_unmap).\n>>>>>>\n>>>>>> Signed-off-by: Alexey Kardashevskiy \n>>>>\n>>>> Is this really legitimate to include tracepoints from a different \n>>>> subsystem?> The vfio type1 backend gets these trace points by virtue of \n>>>> it actually using the IOMMU API, it doesn't call them itself. I'm kind\n>>>> of surprised these are actually available to be called from a module. \n>>>\n>>> They are explicitly exported:\n>>>\n>>> EXPORT_TRACEPOINT_SYMBOL_GPL(map);\n>>> EXPORT_TRACEPOINT_SYMBOL_GPL(unmap);\n\nTracepoints need to be exported as per the tracepoint sub-system\nrequirements.\n\n>>>\n>>> I would think this is for things like drivers/vfio/vfio_iommu_spapr_tce.c ,\n>>> why else?...\n>>>\n>>> \n>>>> I suspect the way to do this is probably to define our own tracepoints\n>>>> in the vfio/spapr backend or insert tracepoints into the IOMMU layers\n>>>> that that code calls into rather than masquerading as tracepoints from\n>>>> a different subsystem. Right? \n>>>\n>>> This makes sense too. But it is going to be just cut-n-paste and some\n>>> confusion -\n>>> /sys/kernel/debug/tracing/events/iommu/map will still be present but\n>>> won't work and\n>>> /sys/kernel/debug/tracing/events/vfio/vfio_iommu_spapr_tce/map will.\n> \n> But iommu/map does work, it's just not called by the vfio spapr tce\n> backend, which is absolutely correct. In fact, that's part of my\n> reservation about this approach is that it could be interpreted as\n> implying a call path that doesn't exist on this arch.\n\nThese tracepoints are intended to be called after a iommo/map/unmap\ncalls from IOMMU API. It doesn't make any sense to call them from\nrandom places. It is totally useless to do so and makes it difficult\nto debug problems.\n\nAs an author of these tracepoints - that is my opinion.\n\nthanks,\n-- Shuah","headers":{"Return-Path":"","X-Original-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Delivered-To":["patchwork-incoming@ozlabs.org","linuxppc-dev@lists.ozlabs.org"],"Received":["from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68])\n\t(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yn61G42BZz9ryr\n\tfor ;\n\tThu, 30 Nov 2017 03:48:34 +1100 (AEDT)","from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yn61G2X2qzDrZv\n\tfor ;\n\tThu, 30 Nov 2017 03:48:34 +1100 (AEDT)","from osg.samsung.com (osg.samsung.com [64.30.133.232])\n\tby lists.ozlabs.org (Postfix) with ESMTP id 3yn5wh2QkXzDrns\n\tfor ;\n\tThu, 30 Nov 2017 03:44:35 +1100 (AEDT)","from localhost (localhost [127.0.0.1])\n\tby osg.samsung.com (Postfix) with ESMTP id F0CCB2E462;\n\tWed, 29 Nov 2017 08:44:32 -0800 (PST)","from osg.samsung.com ([127.0.0.1])\n\tby localhost (localhost [127.0.0.1]) (amavisd-new, port 10024)\n\twith ESMTP id bLfjYA27fIaw; Wed, 29 Nov 2017 08:44:21 -0800 (PST)","from [192.168.1.87] (c-24-9-64-241.hsd1.co.comcast.net\n\t[24.9.64.241])\n\tby osg.samsung.com (Postfix) with ESMTPSA id 774002E457;\n\tWed, 29 Nov 2017 08:44:21 -0800 (PST)"],"Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=osg.samsung.com\n\t(client-ip=64.30.133.232; helo=osg.samsung.com;\n\tenvelope-from=shuahkh@osg.samsung.com; receiver=)","X-Virus-Scanned":"Debian amavisd-new at dev.s-opensource.com","Subject":"Re: [PATCH kernel] vfio/spapr: Add trace points for map/unmap","To":"Alex Williamson ,\n\tAlexey Kardashevskiy ","References":"<20171027030008.17200-1-aik@ozlabs.ru>\n\t<8c58fcaa-dedc-e6be-44bf-2cb349ef9e38@ozlabs.ru>\n\t<20171116171301.6552869b@t450s.home>\n\t<52a19cfb-431f-522b-11de-c1c47f09a4c8@ozlabs.ru>\n\t\n\t<20171129093244.0849fd7c@t450s.home>","From":"Shuah Khan ","Message-ID":"","Date":"Wed, 29 Nov 2017 09:44:13 -0700","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.4.0","MIME-Version":"1.0","In-Reply-To":"<20171129093244.0849fd7c@t450s.home>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-BeenThere":"linuxppc-dev@lists.ozlabs.org","X-Mailman-Version":"2.1.24","Precedence":"list","List-Id":"Linux on PowerPC Developers Mail List\n\t","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kvm@vger.kernel.org, Joerg Roedel ,\n\tShuah Khan , kvm-ppc@vger.kernel.org,\n\tiommu@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org,\n\tDavid Gibson ","Errors-To":"linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org","Sender":"\"Linuxppc-dev\"\n\t"}}]