[{"id":1906280,"web_url":"http://patchwork.ozlabs.org/comment/1906280/","msgid":"<152540403226.2487.4650651377020838066@71c20359a636>","list_archive_url":null,"date":"2018-05-04T03:20:33","subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","submitter":{"id":69632,"url":"http://patchwork.ozlabs.org/api/people/69632/","name":null,"email":"no-reply@patchew.org"},"content":"Hi,\n\nThis series seems to have some coding style problems. See output below for\nmore information:\n\nType: series\nMessage-id: 20180504030811.28111-1-peterx@redhat.com\nSubject: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU, cleanups, bug fixes\n\n=== TEST SCRIPT BEGIN ===\n#!/bin/bash\n\nBASE=base\nn=1\ntotal=$(git log --oneline $BASE.. | wc -l)\nfailed=0\n\ngit config --local diff.renamelimit 0\ngit config --local diff.renames True\ngit config --local diff.algorithm histogram\n\ncommits=\"$(git log --format=%H --reverse $BASE..)\"\nfor c in $commits; do\n echo \"Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)...\"\n if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then\n failed=1\n echo\n fi\n n=$((n+1))\ndone\n\nexit $failed\n=== TEST SCRIPT END ===\n\nUpdating 3c8cf5a9c21ff8782164d1def7f44bd888713384\nFrom https://github.com/patchew-project/qemu\n t [tag update] patchew/20180503213458.2749.28566.stgit@gimli.home -> patchew/20180503213458.2749.28566.stgit@gimli.home\n * [new tag] patchew/20180504030811.28111-1-peterx@redhat.com -> patchew/20180504030811.28111-1-peterx@redhat.com\nSwitched to a new branch 'test'\n55b6c6b1f5 intel-iommu: remove notify_unmap for page walk\n72ffe9314a intel-iommu: don't unmap all for shadow page table\n7bb663682b intel-iommu: maintain per-device iova ranges\ne77921e63f util: implement simple interval tree logic\n356489cd59 intel-iommu: pass in address space when page walk\n27fcd3062f intel-iommu: introduce vtd_page_walk_info\n953a6db666 intel-iommu: only do page walk for MAP notifiers\n0ccfedc1e7 intel-iommu: add iommu lock\n47a29fd165 intel-iommu: remove IntelIOMMUNotifierNode\naacd552d3a intel-iommu: send PSI always even if across PDEs\n\n=== OUTPUT BEGIN ===\nChecking PATCH 1/10: intel-iommu: send PSI always even if across PDEs...\nChecking PATCH 2/10: intel-iommu: remove IntelIOMMUNotifierNode...\nChecking PATCH 3/10: intel-iommu: add iommu lock...\nChecking PATCH 4/10: intel-iommu: only do page walk for MAP notifiers...\nChecking PATCH 5/10: intel-iommu: introduce vtd_page_walk_info...\nChecking PATCH 6/10: intel-iommu: pass in address space when page walk...\nChecking PATCH 7/10: util: implement simple interval tree logic...\nERROR: space prohibited between function name and open parenthesis '('\n#56: FILE: include/qemu/interval-tree.h:33:\n+typedef gboolean (*it_tree_iterator)(ITValue start, ITValue end);\n\ntotal: 1 errors, 0 warnings, 343 lines checked\n\nYour patch has style problems, please review. If any of these errors\nare false positives report them to the maintainer, see\nCHECKPATCH in MAINTAINERS.\n\nChecking PATCH 8/10: intel-iommu: maintain per-device iova ranges...\nChecking PATCH 9/10: intel-iommu: don't unmap all for shadow page table...\nChecking PATCH 10/10: intel-iommu: remove notify_unmap for page walk...\n=== OUTPUT END ===\n\nTest command exited with code: 1\n\n\n---\nEmail generated automatically by Patchew [http://patchew.org/].\nPlease send your feedback to patchew-devel@redhat.com","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdmarc=none (p=none dis=none) header.from=patchew.org"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 40cckF43BMz9s35\n\tfor ;\n\tFri, 4 May 2018 13:21:41 +1000 (AEST)","from localhost ([::1]:60222 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1fERHv-0001We-6w\n\tfor incoming@patchwork.ozlabs.org; Thu, 03 May 2018 23:21:39 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:36539)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1fERHP-0001WY-09\n\tfor qemu-devel@nongnu.org; Thu, 03 May 2018 23:21:08 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1fERHL-0005jn-PP\n\tfor qemu-devel@nongnu.org; Thu, 03 May 2018 23:21:06 -0400","from sender-of-o51.zoho.com ([135.84.80.216]:21071)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1fERHL-0005jF-Fr\n\tfor qemu-devel@nongnu.org; Thu, 03 May 2018 23:21:03 -0400","from [172.17.0.2] (23.253.156.214 [23.253.156.214]) by\n\tmx.zohomail.com with SMTPS id 1525404033396641.8945887322311;\n\tThu, 3 May 2018 20:20:33 -0700 (PDT)"],"Resent-Date":"Thu, 03 May 2018 23:21:06 -0400","Resent-Message-Id":"","In-Reply-To":"<20180504030811.28111-1-peterx@redhat.com>","Message-ID":"<152540403226.2487.4650651377020838066@71c20359a636>","MIME-Version":"1.0","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Resent-From":"","From":"no-reply@patchew.org","To":"peterx@redhat.com","Date":"Thu, 3 May 2018 20:20:33 -0700 (PDT)","X-ZohoMailClient":"External","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"135.84.80.216","Subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"qemu-devel@nongnu.org","Cc":"kevin.tian@intel.com, famz@redhat.com, mst@redhat.com,\n\tjasowang@redhat.com, qemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, jintack@cs.columbia.edu","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1906286,"web_url":"http://patchwork.ozlabs.org/comment/1906286/","msgid":"<20180504034042.GA28447@xz-mi>","list_archive_url":null,"date":"2018-05-04T03:40:42","subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/people/67717/","name":"Peter Xu","email":"peterx@redhat.com"},"content":"On Thu, May 03, 2018 at 08:20:33PM -0700, no-reply@patchew.org wrote:\n\n[...]\n\n> === OUTPUT BEGIN ===\n> Checking PATCH 1/10: intel-iommu: send PSI always even if across PDEs...\n> Checking PATCH 2/10: intel-iommu: remove IntelIOMMUNotifierNode...\n> Checking PATCH 3/10: intel-iommu: add iommu lock...\n> Checking PATCH 4/10: intel-iommu: only do page walk for MAP notifiers...\n> Checking PATCH 5/10: intel-iommu: introduce vtd_page_walk_info...\n> Checking PATCH 6/10: intel-iommu: pass in address space when page walk...\n> Checking PATCH 7/10: util: implement simple interval tree logic...\n> ERROR: space prohibited between function name and open parenthesis '('\n> #56: FILE: include/qemu/interval-tree.h:33:\n> +typedef gboolean (*it_tree_iterator)(ITValue start, ITValue end);\n> \n> total: 1 errors, 0 warnings, 343 lines checked\n> \n> Your patch has style problems, please review. If any of these errors\n> are false positives report them to the maintainer, see\n> CHECKPATCH in MAINTAINERS.\n\nThis is false positive, and should be dissolved after below patch\nmerged (now in Paolo's queue):\n\n [PATCH v2] checkpatch.pl: add common glib defines to typelist","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdmarc=fail (p=none dis=none) header.from=redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 40cd980kjhz9s3D\n\tfor ;\n\tFri, 4 May 2018 13:41:32 +1000 (AEST)","from localhost ([::1]:60262 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1fERb7-00067s-RW\n\tfor incoming@patchwork.ozlabs.org; Thu, 03 May 2018 23:41:29 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:40384)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1fERad-00066m-1c\n\tfor qemu-devel@nongnu.org; Thu, 03 May 2018 23:40:59 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1fERaZ-0002T3-Tl\n\tfor qemu-devel@nongnu.org; Thu, 03 May 2018 23:40:59 -0400","from mx3-rdu2.redhat.com ([66.187.233.73]:49232\n\thelo=mx1.redhat.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1fERaZ-0002SH-O0\n\tfor qemu-devel@nongnu.org; Thu, 03 May 2018 23:40:55 -0400","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 5CDE4406C743;\n\tFri, 4 May 2018 03:40:51 +0000 (UTC)","from xz-mi (ovpn-12-46.pek2.redhat.com [10.72.12.46])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id CDACC2026DFD;\n\tFri, 4 May 2018 03:40:44 +0000 (UTC)"],"Date":"Fri, 4 May 2018 11:40:42 +0800","From":"Peter Xu ","To":"qemu-devel@nongnu.org","Message-ID":"<20180504034042.GA28447@xz-mi>","References":"<20180504030811.28111-1-peterx@redhat.com>\n\t<152540403226.2487.4650651377020838066@71c20359a636>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<152540403226.2487.4650651377020838066@71c20359a636>","User-Agent":"Mutt/1.9.3 (2018-01-21)","X-Scanned-By":"MIMEDefang 2.78 on 10.11.54.4","X-Greylist":["Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.11.55.5]);\n\tFri, 04 May 2018 03:40:51 +0000 (UTC)","inspected by milter-greylist-4.5.16 (mx1.redhat.com\n\t[10.11.55.5]); \n\tFri, 04 May 2018 03:40:51 +0000 (UTC) for IP:'10.11.54.4'\n\tDOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com'\n\tHELO:'smtp.corp.redhat.com' FROM:'peterx@redhat.com' RCPT:''"],"X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"66.187.233.73","Subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kevin.tian@intel.com, famz@redhat.com, mst@redhat.com,\n\tjasowang@redhat.com, alex.williamson@redhat.com, jintack@cs.columbia.edu","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1913544,"web_url":"http://patchwork.ozlabs.org/comment/1913544/","msgid":"<20180516063009.GG9089@xz-mi>","list_archive_url":null,"date":"2018-05-16T06:30:09","subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/people/67717/","name":"Peter Xu","email":"peterx@redhat.com"},"content":"On Fri, May 04, 2018 at 11:08:01AM +0800, Peter Xu wrote:\n> v2:\n> - fix patchew code style warnings\n> - interval tree: postpone malloc when inserting; simplify node remove\n> a bit where proper [Jason]\n> - fix up comment and commit message for iommu lock patch [Kevin]\n> - protect context cache too using the iommu lock [Kevin, Jason]\n> - add vast comment in patch 8 to explain the modify-PTE problem\n> [Jason, Kevin]\n\nWe can hold a bit on reviewing this series. Jintack reported a scp\nDMAR issue that might happen even with L1 guest with this series, and\nthe scp can stall after copied tens or hundreds of MBs randomly. I'm\nstill investigating the problem. This problem should be related to\ndeferred flushing of VT-d kernel driver, since the problem will go\naway if we use \"intel_iommu=on,strict\". However I'm still trying to\nfigure out what's the thing behind the scene even with that deferred\nflushing feature.\n\nMeanwhile, during the investigation I found another \"possibly valid\"\nuse case about the modify-PTE problem that Jason has mentioned when\nwith deferred flushing:\n\n vcpu1 vcpu2\n map page A\n explicitly send PSI for A\n queue unmap page A [1]\n map the same page A [2]\n explcitly send PSI for A [3]\n flush unmap page A [4]\n\nDue to deferred flushing, the UNMAP PSI might be postponed (or it can\nbe finally a DSI) from step [1] to step [4]. If we allocate the same\npage somewhere else, we might trigger this modify-PTE at [2] since we\nhaven't yet received the deferred PSI to unmap A from vcpu1.\n\nNote that this will not happen with latest upstream Linux, since the\nIOVA allocation algorithm in current Linux kernel made sure that the\nIOVA range won't be freed until [4], so we can't really allocate the\nsame page address at [2]. However this let me tend to agree with\nJason and Kevin's worry on future potential issues if that can be\ntriggered easily by common guest kernel bugs. So now I'm considering\nto drop my mergable interval tree but just use a simpler tree to cache\neverything including translated addresses. The metadata will possibly\ntake 2% of managed memory if with that.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdmarc=fail (p=none dis=none) header.from=redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 40m4MH2V8Qz9s19\n\tfor ;\n\tWed, 16 May 2018 16:31:07 +1000 (AEST)","from localhost ([::1]:57613 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1fIpxp-0006KH-0e\n\tfor incoming@patchwork.ozlabs.org; Wed, 16 May 2018 02:31:05 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:35214)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1fIpxB-0006IN-4C\n\tfor qemu-devel@nongnu.org; Wed, 16 May 2018 02:30:31 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1fIpx6-0004Ut-3C\n\tfor qemu-devel@nongnu.org; Wed, 16 May 2018 02:30:25 -0400","from mx3-rdu2.redhat.com ([66.187.233.73]:38068\n\thelo=mx1.redhat.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1fIpx5-0004RR-UW\n\tfor qemu-devel@nongnu.org; Wed, 16 May 2018 02:30:20 -0400","from smtp.corp.redhat.com\n\t(int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 0AB9C4000B6E;\n\tWed, 16 May 2018 06:30:19 +0000 (UTC)","from xz-mi (dhcp-14-151.nay.redhat.com [10.66.14.151])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id AB70BD74A5;\n\tWed, 16 May 2018 06:30:11 +0000 (UTC)"],"Date":"Wed, 16 May 2018 14:30:09 +0800","From":"Peter Xu ","To":"qemu-devel@nongnu.org","Message-ID":"<20180516063009.GG9089@xz-mi>","References":"<20180504030811.28111-1-peterx@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20180504030811.28111-1-peterx@redhat.com>","User-Agent":"Mutt/1.9.3 (2018-01-21)","X-Scanned-By":"MIMEDefang 2.79 on 10.11.54.5","X-Greylist":["Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.11.55.7]);\n\tWed, 16 May 2018 06:30:19 +0000 (UTC)","inspected by milter-greylist-4.5.16 (mx1.redhat.com\n\t[10.11.55.7]); \n\tWed, 16 May 2018 06:30:19 +0000 (UTC) for IP:'10.11.54.5'\n\tDOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com'\n\tHELO:'smtp.corp.redhat.com' FROM:'peterx@redhat.com' RCPT:''"],"X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"66.187.233.73","Subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Jintack Lim , Tian Kevin ,\n\tJason Wang ,\n\tAlex Williamson ,\n\t\"Michael S . Tsirkin\" ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1914012,"web_url":"http://patchwork.ozlabs.org/comment/1914012/","msgid":"","list_archive_url":null,"date":"2018-05-16T13:57:40","subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","submitter":{"id":5225,"url":"http://patchwork.ozlabs.org/api/people/5225/","name":"Jason Wang","email":"jasowang@redhat.com"},"content":"On 2018年05月16日 14:30, Peter Xu wrote:\n> On Fri, May 04, 2018 at 11:08:01AM +0800, Peter Xu wrote:\n>> v2:\n>> - fix patchew code style warnings\n>> - interval tree: postpone malloc when inserting; simplify node remove\n>> a bit where proper [Jason]\n>> - fix up comment and commit message for iommu lock patch [Kevin]\n>> - protect context cache too using the iommu lock [Kevin, Jason]\n>> - add vast comment in patch 8 to explain the modify-PTE problem\n>> [Jason, Kevin]\n> We can hold a bit on reviewing this series. Jintack reported a scp\n> DMAR issue that might happen even with L1 guest with this series, and\n> the scp can stall after copied tens or hundreds of MBs randomly. I'm\n> still investigating the problem. This problem should be related to\n> deferred flushing of VT-d kernel driver, since the problem will go\n> away if we use \"intel_iommu=on,strict\". However I'm still trying to\n> figure out what's the thing behind the scene even with that deferred\n> flushing feature.\n\nI vaguely remember recent upstream vfio support delayed flush, maybe \nit's related.\n\n>\n> Meanwhile, during the investigation I found another \"possibly valid\"\n> use case about the modify-PTE problem that Jason has mentioned when\n> with deferred flushing:\n>\n> vcpu1 vcpu2\n> map page A\n> explicitly send PSI for A\n> queue unmap page A [1]\n> map the same page A [2]\n> explcitly send PSI for A [3]\n> flush unmap page A [4]\n>\n> Due to deferred flushing, the UNMAP PSI might be postponed (or it can\n> be finally a DSI) from step [1] to step [4]. If we allocate the same\n> page somewhere else, we might trigger this modify-PTE at [2] since we\n> haven't yet received the deferred PSI to unmap A from vcpu1.\n>\n> Note that this will not happen with latest upstream Linux, since the\n> IOVA allocation algorithm in current Linux kernel made sure that the\n> IOVA range won't be freed until [4], so we can't really allocate the\n> same page address at [2].\n\nYes, so the vfio + vIOMMU work will probably uncover more bugs in the \nIOMMU driver (especially CM mode). I suspect CM mode does not have \nsufficient test (since it probably wasn't used in any production \nenvironment before the vfio + vIOMMU work).\n\n> However this let me tend to agree with\n> Jason and Kevin's worry on future potential issues if that can be\n> triggered easily by common guest kernel bugs. So now I'm considering\n> to drop my mergable interval tree but just use a simpler tree to cache\n> everything including translated addresses. The metadata will possibly\n> take 2% of managed memory if with that.\n>\n\nGood to know this, we can start from the way we know correct for sure \nthen optimizations on top.\n\nThanks","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdmarc=fail (p=none dis=none) header.from=redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 40mGHF0ZZKz9s19\n\tfor ;\n\tWed, 16 May 2018 23:58:17 +1000 (AEST)","from localhost ([::1]:47760 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1fIwwY-0006Wz-RU\n\tfor incoming@patchwork.ozlabs.org; Wed, 16 May 2018 09:58:14 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:34945)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1fIwwE-0006WN-4d\n\tfor qemu-devel@nongnu.org; Wed, 16 May 2018 09:57:55 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1fIww9-0005jM-9M\n\tfor qemu-devel@nongnu.org; Wed, 16 May 2018 09:57:54 -0400","from mx3-rdu2.redhat.com ([66.187.233.73]:51150\n\thelo=mx1.redhat.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1fIww9-0005j2-48\n\tfor qemu-devel@nongnu.org; Wed, 16 May 2018 09:57:49 -0400","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 99709402382B;\n\tWed, 16 May 2018 13:57:48 +0000 (UTC)","from [10.72.12.64] (ovpn-12-64.pek2.redhat.com [10.72.12.64])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id C76F22024CBC;\n\tWed, 16 May 2018 13:57:44 +0000 (UTC)"],"To":"Peter Xu , qemu-devel@nongnu.org","References":"<20180504030811.28111-1-peterx@redhat.com>\n\t<20180516063009.GG9089@xz-mi>","From":"Jason Wang ","Message-ID":"","Date":"Wed, 16 May 2018 21:57:40 +0800","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.7.0","MIME-Version":"1.0","In-Reply-To":"<20180516063009.GG9089@xz-mi>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-US","X-Scanned-By":"MIMEDefang 2.78 on 10.11.54.4","X-Greylist":["Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.11.55.6]);\n\tWed, 16 May 2018 13:57:48 +0000 (UTC)","inspected by milter-greylist-4.5.16 (mx1.redhat.com\n\t[10.11.55.6]); \n\tWed, 16 May 2018 13:57:48 +0000 (UTC) for IP:'10.11.54.4'\n\tDOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com'\n\tHELO:'smtp.corp.redhat.com' FROM:'jasowang@redhat.com' RCPT:''"],"Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"66.187.233.73","Subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Jintack Lim , Tian Kevin ,\n\tAlex Williamson ,\n\t\"Michael S . Tsirkin\" ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1914564,"web_url":"http://patchwork.ozlabs.org/comment/1914564/","msgid":"<20180517024544.GK9089@xz-mi>","list_archive_url":null,"date":"2018-05-17T02:45:44","subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/people/67717/","name":"Peter Xu","email":"peterx@redhat.com"},"content":"On Wed, May 16, 2018 at 09:57:40PM +0800, Jason Wang wrote:\n> \n> \n> On 2018年05月16日 14:30, Peter Xu wrote:\n> > On Fri, May 04, 2018 at 11:08:01AM +0800, Peter Xu wrote:\n> > > v2:\n> > > - fix patchew code style warnings\n> > > - interval tree: postpone malloc when inserting; simplify node remove\n> > > a bit where proper [Jason]\n> > > - fix up comment and commit message for iommu lock patch [Kevin]\n> > > - protect context cache too using the iommu lock [Kevin, Jason]\n> > > - add vast comment in patch 8 to explain the modify-PTE problem\n> > > [Jason, Kevin]\n> > We can hold a bit on reviewing this series. Jintack reported a scp\n> > DMAR issue that might happen even with L1 guest with this series, and\n> > the scp can stall after copied tens or hundreds of MBs randomly. I'm\n> > still investigating the problem. This problem should be related to\n> > deferred flushing of VT-d kernel driver, since the problem will go\n> > away if we use \"intel_iommu=on,strict\". However I'm still trying to\n> > figure out what's the thing behind the scene even with that deferred\n> > flushing feature.\n> \n> I vaguely remember recent upstream vfio support delayed flush, maybe it's\n> related.\n\nI'm a bit confused on why vfio is related to the deferred flushing.\nCould you provide a pointer for this?\n\n> \n> > \n> > Meanwhile, during the investigation I found another \"possibly valid\"\n> > use case about the modify-PTE problem that Jason has mentioned when\n> > with deferred flushing:\n> > \n> > vcpu1 vcpu2\n> > map page A\n> > explicitly send PSI for A\n> > queue unmap page A [1]\n> > map the same page A [2]\n> > explcitly send PSI for A [3]\n> > flush unmap page A [4]\n> > \n> > Due to deferred flushing, the UNMAP PSI might be postponed (or it can\n> > be finally a DSI) from step [1] to step [4]. If we allocate the same\n> > page somewhere else, we might trigger this modify-PTE at [2] since we\n> > haven't yet received the deferred PSI to unmap A from vcpu1.\n> > \n> > Note that this will not happen with latest upstream Linux, since the\n> > IOVA allocation algorithm in current Linux kernel made sure that the\n> > IOVA range won't be freed until [4], so we can't really allocate the\n> > same page address at [2].\n> \n> Yes, so the vfio + vIOMMU work will probably uncover more bugs in the IOMMU\n> driver (especially CM mode). I suspect CM mode does not have sufficient test\n> (since it probably wasn't used in any production environment before the vfio\n> + vIOMMU work).\n\nYes maybe. I might possibly continue investigating some of this after\nthis \"functional\" series. So my plan is that firstly we make it work\nfunctionally (even we can allow some trivial bugs, but scp error is\nnot in count), then we consider other things.\n\n> \n> > However this let me tend to agree with\n> > Jason and Kevin's worry on future potential issues if that can be\n> > triggered easily by common guest kernel bugs. So now I'm considering\n> > to drop my mergable interval tree but just use a simpler tree to cache\n> > everything including translated addresses. The metadata will possibly\n> > take 2% of managed memory if with that.\n> > \n> \n> Good to know this, we can start from the way we know correct for sure then\n> optimizations on top.\n\nYes.\n\nBtw, still it's not really \"correct\" here AFAIU - the correct thing\nshould be that we \"modify\" the PTE without any invalid DMA window.\nNow even if I switch to the other method we still need to unmap and\nremap, so we will still have an invalid window for that DMA range.\nFor example, if Linux kernel frees IOVA ranges after queueing the\nunmap in current Linux VT-d drivers (again, this does not exist, but\nI'm just assuming), then my current series will be affected while the\nother way won't be affected. I'll still add some comment there to\nclarify this as TODO.\n\nThanks,","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdmarc=fail (p=none dis=none) header.from=redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 40mbKT00rXz9s0x\n\tfor ;\n\tThu, 17 May 2018 12:46:19 +1000 (AEST)","from localhost ([::1]:56561 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1fJ8vn-0006LM-Fy\n\tfor incoming@patchwork.ozlabs.org; Wed, 16 May 2018 22:46:15 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:54148)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1fJ8vR-0006Kp-TA\n\tfor qemu-devel@nongnu.org; Wed, 16 May 2018 22:45:55 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1fJ8vO-000658-PO\n\tfor qemu-devel@nongnu.org; Wed, 16 May 2018 22:45:53 -0400","from mx3-rdu2.redhat.com ([66.187.233.73]:58214\n\thelo=mx1.redhat.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1fJ8vO-00064W-JL\n\tfor qemu-devel@nongnu.org; Wed, 16 May 2018 22:45:50 -0400","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 2AB1D5BCB6;\n\tThu, 17 May 2018 02:45:49 +0000 (UTC)","from xz-mi (ovpn-12-145.pek2.redhat.com [10.72.12.145])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 955FB2026990;\n\tThu, 17 May 2018 02:45:46 +0000 (UTC)"],"Date":"Thu, 17 May 2018 10:45:44 +0800","From":"Peter Xu ","To":"Jason Wang ","Message-ID":"<20180517024544.GK9089@xz-mi>","References":"<20180504030811.28111-1-peterx@redhat.com>\n\t<20180516063009.GG9089@xz-mi>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.9.3 (2018-01-21)","X-Scanned-By":"MIMEDefang 2.78 on 10.11.54.4","X-Greylist":["Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.11.55.1]);\n\tThu, 17 May 2018 02:45:49 +0000 (UTC)","inspected by milter-greylist-4.5.16 (mx1.redhat.com\n\t[10.11.55.1]); \n\tThu, 17 May 2018 02:45:49 +0000 (UTC) for IP:'10.11.54.4'\n\tDOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com'\n\tHELO:'smtp.corp.redhat.com' FROM:'peterx@redhat.com' RCPT:''"],"Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"66.187.233.73","Subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Jintack Lim , Tian Kevin ,\n\tAlex Williamson ,\n\tqemu-devel@nongnu.org, \"Michael S . Tsirkin\" ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1914577,"web_url":"http://patchwork.ozlabs.org/comment/1914577/","msgid":"<20180516213941.7c3edfdb@w520.home>","list_archive_url":null,"date":"2018-05-17T03:39:41","subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","submitter":{"id":4123,"url":"http://patchwork.ozlabs.org/api/people/4123/","name":"Alex Williamson","email":"alex.williamson@redhat.com"},"content":"On Thu, 17 May 2018 10:45:44 +0800\nPeter Xu wrote:\n\n> On Wed, May 16, 2018 at 09:57:40PM +0800, Jason Wang wrote:\n> > \n> > \n> > On 2018年05月16日 14:30, Peter Xu wrote: \n> > > On Fri, May 04, 2018 at 11:08:01AM +0800, Peter Xu wrote: \n> > > > v2:\n> > > > - fix patchew code style warnings\n> > > > - interval tree: postpone malloc when inserting; simplify node remove\n> > > > a bit where proper [Jason]\n> > > > - fix up comment and commit message for iommu lock patch [Kevin]\n> > > > - protect context cache too using the iommu lock [Kevin, Jason]\n> > > > - add vast comment in patch 8 to explain the modify-PTE problem\n> > > > [Jason, Kevin] \n> > > We can hold a bit on reviewing this series. Jintack reported a scp\n> > > DMAR issue that might happen even with L1 guest with this series, and\n> > > the scp can stall after copied tens or hundreds of MBs randomly. I'm\n> > > still investigating the problem. This problem should be related to\n> > > deferred flushing of VT-d kernel driver, since the problem will go\n> > > away if we use \"intel_iommu=on,strict\". However I'm still trying to\n> > > figure out what's the thing behind the scene even with that deferred\n> > > flushing feature. \n> > \n> > I vaguely remember recent upstream vfio support delayed flush, maybe it's\n> > related. \n> \n> I'm a bit confused on why vfio is related to the deferred flushing.\n> Could you provide a pointer for this?\n\nPerhaps referring to this:\n\nhttps://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6bd06f5a486c06023a618a86e8153b91d26f75f4\n\nRather than calling iommu_unmap() for each chunk of a mapping we'll\nmake multiple calls to iommu_unmap_fast() and flush with\niommu_tlb_sync() to defer and batch the hardware flushing. Thanks,\n\nAlex","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdmarc=fail (p=none dis=none) header.from=redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 40mcWv62DNz9s33\n\tfor ;\n\tThu, 17 May 2018 13:40:26 +1000 (AEST)","from localhost ([::1]:58787 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1fJ9mB-0007uf-00\n\tfor incoming@patchwork.ozlabs.org; Wed, 16 May 2018 23:40:23 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:37095)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1fJ9lj-0007uR-3H\n\tfor qemu-devel@nongnu.org; Wed, 16 May 2018 23:39:56 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1fJ9lg-0000PA-0B\n\tfor qemu-devel@nongnu.org; Wed, 16 May 2018 23:39:55 -0400","from mx1.redhat.com ([209.132.183.28]:45950)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1fJ9lf-0000OX-Qq\n\tfor qemu-devel@nongnu.org; Wed, 16 May 2018 23:39:51 -0400","from smtp.corp.redhat.com\n\t(int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.27])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 1C44330BE830;\n\tThu, 17 May 2018 03:39:50 +0000 (UTC)","from w520.home (ovpn-116-135.phx2.redhat.com [10.3.116.135])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id 3228E9B3F0;\n\tThu, 17 May 2018 03:39:41 +0000 (UTC)"],"Date":"Wed, 16 May 2018 21:39:41 -0600","From":"Alex Williamson ","To":"Peter Xu ","Message-ID":"<20180516213941.7c3edfdb@w520.home>","In-Reply-To":"<20180517024544.GK9089@xz-mi>","References":"<20180504030811.28111-1-peterx@redhat.com>\n\t<20180516063009.GG9089@xz-mi>\n\t\n\t<20180517024544.GK9089@xz-mi>","MIME-Version":"1.0","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"quoted-printable","X-Scanned-By":"MIMEDefang 2.84 on 10.5.11.27","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.40]);\n\tThu, 17 May 2018 03:39:50 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Tian Kevin , Jintack Lim ,\n\tJason Wang , qemu-devel@nongnu.org,\n\t\"Michael S . Tsirkin\" ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1914584,"web_url":"http://patchwork.ozlabs.org/comment/1914584/","msgid":"<20180517041608.GO9089@xz-mi>","list_archive_url":null,"date":"2018-05-17T04:16:08","subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/people/67717/","name":"Peter Xu","email":"peterx@redhat.com"},"content":"On Wed, May 16, 2018 at 09:39:41PM -0600, Alex Williamson wrote:\n> On Thu, 17 May 2018 10:45:44 +0800\n> Peter Xu wrote:\n> \n> > On Wed, May 16, 2018 at 09:57:40PM +0800, Jason Wang wrote:\n> > > \n> > > \n> > > On 2018年05月16日 14:30, Peter Xu wrote: \n> > > > On Fri, May 04, 2018 at 11:08:01AM +0800, Peter Xu wrote: \n> > > > > v2:\n> > > > > - fix patchew code style warnings\n> > > > > - interval tree: postpone malloc when inserting; simplify node remove\n> > > > > a bit where proper [Jason]\n> > > > > - fix up comment and commit message for iommu lock patch [Kevin]\n> > > > > - protect context cache too using the iommu lock [Kevin, Jason]\n> > > > > - add vast comment in patch 8 to explain the modify-PTE problem\n> > > > > [Jason, Kevin] \n> > > > We can hold a bit on reviewing this series. Jintack reported a scp\n> > > > DMAR issue that might happen even with L1 guest with this series, and\n> > > > the scp can stall after copied tens or hundreds of MBs randomly. I'm\n> > > > still investigating the problem. This problem should be related to\n> > > > deferred flushing of VT-d kernel driver, since the problem will go\n> > > > away if we use \"intel_iommu=on,strict\". However I'm still trying to\n> > > > figure out what's the thing behind the scene even with that deferred\n> > > > flushing feature. \n> > > \n> > > I vaguely remember recent upstream vfio support delayed flush, maybe it's\n> > > related. \n> > \n> > I'm a bit confused on why vfio is related to the deferred flushing.\n> > Could you provide a pointer for this?\n> \n> Perhaps referring to this:\n> \n> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6bd06f5a486c06023a618a86e8153b91d26f75f4\n> \n> Rather than calling iommu_unmap() for each chunk of a mapping we'll\n> make multiple calls to iommu_unmap_fast() and flush with\n> iommu_tlb_sync() to defer and batch the hardware flushing. Thanks,\n\nThanks for the link!\n\nIt seems to be a good performance enhancement for vfio, but it might\nnot related to current problem.\n\nMy latest clue shows that the issue is possibly caused by replaying\neach IOMMU region twice on a single DSI. For example, when we get one\nDSI we'll actually call vtd_page_walk() twice (the IOMMU region is\nsplitted by the MSI region 0xfeeXXXXX, so we have two notifiers for\neach vfio-pci device now... and memory_region_iommu_replay_all will\ncall vtd_page_walk twice). So that confused the IOVA tree a bit. I'll\nverify that and see how I can fix it up.\n\n(PS: it seems that in above patch unmapped_region_cnt is not needed in\n vfio_unmap_unpin considering that we already have\n unmapped_region_list there? If that's correct, then we can remove\n all the references too, e.g. we don't need to pass in unmapped_cnt\n into unmap_unpin_fast as well.)\n\nThanks,","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdmarc=fail (p=none dis=none) header.from=redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 40mdKv2d3mz9s0y\n\tfor ;\n\tThu, 17 May 2018 14:16:49 +1000 (AEST)","from localhost ([::1]:60250 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1fJALM-0007Zs-M9\n\tfor incoming@patchwork.ozlabs.org; Thu, 17 May 2018 00:16:44 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:41010)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1fJAKx-0007ZU-Cg\n\tfor qemu-devel@nongnu.org; Thu, 17 May 2018 00:16:20 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1fJAKs-0005xK-DT\n\tfor qemu-devel@nongnu.org; Thu, 17 May 2018 00:16:19 -0400","from mx3-rdu2.redhat.com ([66.187.233.73]:47434\n\thelo=mx1.redhat.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1fJAKs-0005wn-7v\n\tfor qemu-devel@nongnu.org; Thu, 17 May 2018 00:16:14 -0400","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 8909A79D36;\n\tThu, 17 May 2018 04:16:13 +0000 (UTC)","from xz-mi (ovpn-12-145.pek2.redhat.com [10.72.12.145])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 008042026E0E;\n\tThu, 17 May 2018 04:16:10 +0000 (UTC)"],"Date":"Thu, 17 May 2018 12:16:08 +0800","From":"Peter Xu ","To":"Alex Williamson ","Message-ID":"<20180517041608.GO9089@xz-mi>","References":"<20180504030811.28111-1-peterx@redhat.com>\n\t<20180516063009.GG9089@xz-mi>\n\t\n\t<20180517024544.GK9089@xz-mi> <20180516213941.7c3edfdb@w520.home>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20180516213941.7c3edfdb@w520.home>","User-Agent":"Mutt/1.9.3 (2018-01-21)","X-Scanned-By":"MIMEDefang 2.78 on 10.11.54.4","X-Greylist":["Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.11.55.2]);\n\tThu, 17 May 2018 04:16:13 +0000 (UTC)","inspected by milter-greylist-4.5.16 (mx1.redhat.com\n\t[10.11.55.2]); \n\tThu, 17 May 2018 04:16:13 +0000 (UTC) for IP:'10.11.54.4'\n\tDOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com'\n\tHELO:'smtp.corp.redhat.com' FROM:'peterx@redhat.com' RCPT:''"],"Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"66.187.233.73","Subject":"Re: [Qemu-devel] [PATCH v2 00/10] intel-iommu: nested vIOMMU,\n\tcleanups, bug fixes","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Tian Kevin , Jintack Lim ,\n\tJason Wang , qemu-devel@nongnu.org,\n\t\"Michael S . Tsirkin\" ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]