[{"id":1764701,"web_url":"http://patchwork.ozlabs.org/comment/1764701/","msgid":"","list_archive_url":null,"date":"2017-09-07T12:39:58","subject":"Re: [Qemu-devel] [PATCH v7 00/20] ARM SMMUv3 Emulation Support","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 1 September 2017 at 18:21, Eric Auger wrote:\n> This series implements the emulation code for ARM SMMUv3.\n>\n> Changes since v6:\n> - DPDK testpmd now running on guest with 2 assigned VFs\n> - Changed the instantiation method: add the following option to\n> the QEMU command line\n> -device smmuv3 # for virtio/vhost use cases\n> -device smmuv3,caching-mode # for vfio use cases (based on [1])\n> - splitted the series into smaller patches to allow the review\n> - the VFIO integration based on \"tlbi-on-map\" smmuv3 driver\n> is isolated from the rest: last 2 patches, not for upstream.\n> This is shipped for testing/bench until a better solution is found.\n> - Reworked permission flag checks and event generation\n\nHi Eric -- I see you've upgraded this from an RFC to a PATCH set.\nDo you want the patches reviewed and (eventually) taken into git\nnow?\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"gIO84bsz\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xp0Rr6bsWz9s9Y\n\tfor ;\n\tThu, 7 Sep 2017 22:40:55 +1000 (AEST)","from localhost ([::1]:40314 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dpw72-0007Xx-3q\n\tfor incoming@patchwork.ozlabs.org; Thu, 07 Sep 2017 08:40:52 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:41148)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dpw6a-0007XI-QT\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 08:40:29 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dpw6W-0006qJ-9y\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 08:40:24 -0400","from mail-wr0-x235.google.com ([2a00:1450:400c:c0c::235]:34821)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1dpw6W-0006pu-2w\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 08:40:20 -0400","by mail-wr0-x235.google.com with SMTP id m18so2641615wrm.2\n\tfor ; Thu, 07 Sep 2017 05:40:19 -0700 (PDT)","by 10.223.159.68 with HTTP; Thu, 7 Sep 2017 05:39:58 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=YGXEyickdfupltUiBew3k0VwVAmRv2rT9p5Q2tiYek4=;\n\tb=gIO84bszplUnUaDplhH/h/kL2oj/8ZZLLxQuX4FNc9XJx6rCek+JY/9vvrISZPuQDg\n\tIKyhtGH478JqCfCRAbsMeDp3y575vNGGspWiiwFAuJ2P2RL81TavDNDDKop19b2DIjCa\n\tgV0M4bxPRAsFOtCINRCIvFGkLDmodjCYsaBkE=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=YGXEyickdfupltUiBew3k0VwVAmRv2rT9p5Q2tiYek4=;\n\tb=qex3Z9bRn/kFPkz+mQ6VJqz8u8USR7znKvBzmr4UNFQIf0bxqC9OX4ViVsEplJY1mE\n\t/YGj/v2n5oyNaw1OcvA3xNovfln09UC+MdryI7RL9CQ7nIFtuahSbIjCQD/o/Jg/FznE\n\tGvHkqqulaBJE3v/MMwTfAHNLhxL8dqAX77ktZ7fVPCTgJkoQPkuuA0tYhmf5/uGrwyIC\n\tFtowildD6Vjif6sdJpYnMiylLs6WcuEAp9tZRA9nwYp7zz0FUKCqWEHJ2TYbktwoH1EF\n\tsg3ks3UmZe9pi97enwt2UCcWWAIawLVWps+yIQJ809B4cIWi4pogRUg0Kz0MoZNvRMhZ\n\tnTjQ==","X-Gm-Message-State":"AHPjjUhzKGy0ptqBSGgLfq3Xx8P+TYBPGj7a7lFXeALusrudp61XN0o/\n\tiG0bRLGOBHF2tTMvnt8Trdn2XiuysA+Y","X-Google-Smtp-Source":"ADKCNb6hL1M3StmY2pPRP3qRMs16W3JtghEypsS/YN79vHer0C8DPDn7pjMeW74zBcteVqIO8kmXVGaBs4LHNNOwH88=","X-Received":"by 10.223.163.71 with SMTP id d7mr1898793wrb.161.1504788018719; \n\tThu, 07 Sep 2017 05:40:18 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>","From":"Peter Maydell ","Date":"Thu, 7 Sep 2017 13:39:58 +0100","Message-ID":"","To":"Eric Auger ","Content-Type":"text/plain; charset=\"UTF-8\"","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c0c::235","Subject":"Re: [Qemu-devel] [PATCH v7 00/20] ARM SMMUv3 Emulation Support","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Radha Mohan , Andrew Jones ,\n\tTrey Cain ,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\t\"Michael S. Tsirkin\" ,\n\tjean-philippe.brucker@arm.com, Tomasz Nowicki ,\n\tWill Deacon ,\n\tQEMU Developers , Peter Xu ,\n\tAlex Williamson , \n\tqemu-arm ,\n\tChristoffer Dall ,\n\t\"Edgar E. Iglesias\" ,\n\trobin.murphy@arm.com, wtownsen@redhat.com,\n\tBharat Bhushan ,\n\tPrem Mallappa , eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1765090,"web_url":"http://patchwork.ozlabs.org/comment/1765090/","msgid":"<20170908084657-mutt-send-email-mst@kernel.org>","list_archive_url":null,"date":"2017-09-08T05:47:52","subject":"Re: [Qemu-devel] [PATCH v7 00/20] ARM SMMUv3 Emulation Support","submitter":{"id":2235,"url":"http://patchwork.ozlabs.org/api/people/2235/","name":"Michael S. Tsirkin","email":"mst@redhat.com"},"content":"On Fri, Sep 01, 2017 at 07:21:03PM +0200, Eric Auger wrote:\n> This series implements the emulation code for ARM SMMUv3.\n\nCan you add some code to block using vfio with this\nuntil patches 19+20 are ready?\nThen 1-18 could be applied.\n\n> Changes since v6:\n> - DPDK testpmd now running on guest with 2 assigned VFs\n> - Changed the instantiation method: add the following option to\n> the QEMU command line\n> -device smmuv3 # for virtio/vhost use cases\n> -device smmuv3,caching-mode # for vfio use cases (based on [1])\n> - splitted the series into smaller patches to allow the review\n> - the VFIO integration based on \"tlbi-on-map\" smmuv3 driver\n> is isolated from the rest: last 2 patches, not for upstream.\n> This is shipped for testing/bench until a better solution is found.\n> - Reworked permission flag checks and event generation\n> \n> testing:\n> - in dt and ACPI modes\n> - virtio-net-pci and vhost-net devices using dma ops with various\n> guest page sizes [2]\n> - assigned VFs using dma ops [3]:\n> - AMD Overdrive and igbvf passthrough (using gsi direct mapping)\n> - Cavium ThunderX and ixgbevf passthrough (using KVM MSI routing)\n> - DPDK testpmd on guest running with VFIO user space drivers (2 igbvf) [3]\n> with guest and host page size equal (4kB)\n> \n> Known limitations:\n> - no VMSAv8-32 suport\n> - no nested stage support (S1 + S2)\n> - no support for HYP mappings\n> - register fine emulation, commands, interrupts and errors were\n> not accurately tested. Handling is sufficient to run use cases\n> described above though.\n> - interrupts and event generation not observed yet.\n> \n> Best Regards\n> \n> Eric\n> \n> This series can be found at:\n> v7: https://github.com/eauger/qemu/tree/v2.10.0-SMMU-v7\n> Previous version at:\n> v6: https://github.com/eauger/qemu/tree/v2.10.0-rc2-SMMU-v6\n> \n> References:\n> [1] [RFC v2 0/4] arm-smmu-v3 tlbi-on-map option\n> https://lkml.org/lkml/2017/8/11/426\n> \n> [2] qemu cmd line excerpt:\n> -device smmuv3 \\\n> -netdev tap,id=tap0,script=no,downscript=no,ifname=tap0,vhost=off \\\n> -device virtio-net-pci,netdev=tap0,mac=6a:f5:10:b1:3d:d2,iommu_platform,disable-modern=off,disable-legacy=on \\\n> [3] use -device smmuv3,caching-mode\n> \n> \n> History:\n> v6 -> v7:\n> - see above\n> \n> v5 -> v6:\n> - Rebase on 2.10 and IOMMUMemoryRegion\n> - add ACPI TLBI_ON_MAP support (VFIO integration also works in\n> ACPI mode)\n> - fix block replay\n> - handle implementation defined SMMU_CMD_TLBI_NH_VA_AM cmd\n> (goes along with TLBI_ON_MAP FW quirk)\n> - replay systematically unmap the whole range first\n> - smmuv3_map_hook does not unmap anymore and the unmap is done\n> before the replay\n> - add and use smmuv3_context_device_invalidate instead of\n> blindly replaying everything\n> \n> v4 -> v5:\n> - initial_level now part of SMMUTransCfg\n> - smmu_page_walk_64 takes into account the max input size\n> - implement sys->iommu_ops.replay and sys->iommu_ops.notify_flag_changed\n> - smmuv3_translate: bug fix: don't walk on bypass\n> - smmu_update_qreg: fix PROD index update\n> - I did not yet address Peter's comments as the code is not mature enough\n> to be split into sub patches.\n> \n> v3 -> v4 [Eric]:\n> - page table walk rewritten to allow scan of the page table within a\n> range of IOVA. This prepares for VFIO integration and replay.\n> - configuration parsing partially reworked.\n> - do not advertise unsupported/untested features: S2, S1 + S2, HYP,\n> PRI, ATS, ..\n> - added ACPI table generation\n> - migrated to dynamic traces\n> - mingw compilation fix\n> \n> v2 -> v3 [Eric]:\n> - rebased on 2.9\n> - mostly code and patch reorganization to ease the review process\n> - optional patches removed. They may be handled separately. I am currently\n> working on ACPI enablement.\n> - optional instantiation of the smmu in mach-virt\n> - removed [2/9] (fdt functions) since not mandated\n> - start splitting main patch into base and derived object\n> - no new function feature added\n> \n> v1 -> v2 [Prem]:\n> - Adopted review comments from Eric Auger\n> - Make SMMU_DPRINTF to internally call qemu_log\n> (since translation requests are too many, we need control\n> on the type of log we want)\n> - SMMUTransCfg modified to suite simplicity\n> - Change RegInfo to uint64 register array\n> - Code cleanup\n> - Test cleanups\n> - Reshuffled patches\n> \n> v0 -> v1 [Prem]:\n> - As per SMMUv3 spec 16.0 (only is_ste_consistant() is noticeable)\n> - Reworked register access/update logic\n> - Factored out translation code for\n> - single point bug fix\n> - sharing/removal in future\n> - (optional) Unit tests added, with PCI test device\n> - S1 with 4k/64k, S1+S2 with 4k/64k\n> - (S1 or S2) only can be verified by Linux 4.7 driver\n> - (optional) Priliminary ACPI support\n> \n> v0 [Prem]:\n> - Implements SMMUv3 spec 11.0\n> - Supported for PCIe devices,\n> - Command Queue and Event Queue supported\n> - LPAE only, S1 is supported and Tested, S2 not tested\n> - BE mode Translation not supported\n> - IRQ support (legacy, no MSI)\n> \n> Eric Auger (18):\n> hw/arm/smmu-common: smmu base device and datatypes\n> hw/arm/smmu-common: IOMMU memory region and address space setup\n> hw/arm/smmu-common: smmu_read/write_sysmem\n> hw/arm/smmu-common: VMSAv8-64 page table walk\n> hw/arm/smmuv3: Wired IRQ and GERROR helpers\n> hw/arm/smmuv3: Queue helpers\n> hw/arm/smmuv3: Implement MMIO write operations\n> hw/arm/smmuv3: Event queue recording helper\n> hw/arm/smmuv3: Implement translate callback\n> target/arm/kvm: Translate the MSI doorbell in kvm_arch_fixup_msi_route\n> hw/arm/smmuv3: Implement data structure and TLB invalidation\n> notifications\n> hw/arm/smmuv3: Implement IOMMU memory region replay callback\n> hw/arm/virt: Store the PCI host controller dt phandle\n> hw/arm/sysbus-fdt: Pass the VirtMachineState to the node creation\n> functions\n> hw/arm/sysbus-fdt: Pass the platform bus base address in\n> PlatformBusFDTData\n> hw/arm/sysbus-fdt: Allow smmuv3 dynamic instantiation\n> hw/arm/smmuv3: [not for upstream] add SMMU_CMD_TLBI_NH_VA_AM handling\n> hw/arm/smmuv3: [not for upstream] Add caching-mode option\n> \n> Prem Mallappa (2):\n> hw/arm/smmuv3: Skeleton\n> hw/arm/virt-acpi-build: Add smmuv3 node in IORT table\n> \n> default-configs/aarch64-softmmu.mak | 1 +\n> hw/arm/Makefile.objs | 1 +\n> hw/arm/smmu-common.c | 527 ++++++++++++++++\n> hw/arm/smmu-internal.h | 105 ++++\n> hw/arm/smmuv3-internal.h | 584 +++++++++++++++++\n> hw/arm/smmuv3.c | 1181 +++++++++++++++++++++++++++++++++++\n> hw/arm/sysbus-fdt.c | 129 +++-\n> hw/arm/trace-events | 48 ++\n> hw/arm/virt-acpi-build.c | 63 +-\n> hw/arm/virt.c | 6 +-\n> include/hw/acpi/acpi-defs.h | 15 +\n> include/hw/arm/smmu-common.h | 123 ++++\n> include/hw/arm/smmuv3.h | 80 +++\n> include/hw/arm/sysbus-fdt.h | 2 +\n> include/hw/arm/virt.h | 15 +\n> target/arm/kvm.c | 27 +\n> target/arm/trace-events | 3 +\n> 17 files changed, 2886 insertions(+), 24 deletions(-)\n> create mode 100644 hw/arm/smmu-common.c\n> create mode 100644 hw/arm/smmu-internal.h\n> create mode 100644 hw/arm/smmuv3-internal.h\n> create mode 100644 hw/arm/smmuv3.c\n> create mode 100644 include/hw/arm/smmu-common.h\n> create mode 100644 include/hw/arm/smmuv3.h\n> \n> -- \n> 2.5.5","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx04.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx04.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=mst@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xpRFw1Ldgz9sPk\n\tfor ;\n\tFri, 8 Sep 2017 15:48:50 +1000 (AEST)","from localhost ([::1]:43408 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dqC9o-0000CK-4p\n\tfor incoming@patchwork.ozlabs.org; Fri, 08 Sep 2017 01:48:48 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:54949)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dqC9M-0000BB-66\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 01:48:25 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dqC9G-0007yR-Oy\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 01:48:20 -0400","from mx1.redhat.com ([209.132.183.28]:44666)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dqC95-0007o1-Ra; Fri, 08 Sep 2017 01:48:04 -0400","from smtp.corp.redhat.com\n\t(int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id A4D8D80461;\n\tFri, 8 Sep 2017 05:48:01 +0000 (UTC)","from redhat.com (ovpn-120-144.rdu2.redhat.com [10.10.120.144])\n\tby smtp.corp.redhat.com (Postfix) with SMTP id E1AC05C882;\n\tFri, 8 Sep 2017 05:47:53 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com A4D8D80461","Date":"Fri, 8 Sep 2017 08:47:52 +0300","From":"\"Michael S. Tsirkin\" ","To":"Eric Auger ","Message-ID":"<20170908084657-mutt-send-email-mst@kernel.org>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.16","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.28]);\n\tFri, 08 Sep 2017 05:48:02 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH v7 00/20] ARM SMMUv3 Emulation Support","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, will.deacon@arm.com, qemu-devel@nongnu.org,\n\tpeterx@redhat.com, alex.williamson@redhat.com,\n\tqemu-arm@nongnu.org, christoffer.dall@linaro.org,\n\tedgar.iglesias@gmail.com, robin.murphy@arm.com,\n\twtownsen@redhat.com, bharat.bhushan@nxp.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1765152,"web_url":"http://patchwork.ozlabs.org/comment/1765152/","msgid":"<05efe60e-e4d0-24e1-548b-b587bf5153ff@redhat.com>","list_archive_url":null,"date":"2017-09-08T08:35:28","subject":"Re: [Qemu-devel] [PATCH v7 00/20] ARM SMMUv3 Emulation Support","submitter":{"id":69187,"url":"http://patchwork.ozlabs.org/api/people/69187/","name":"Eric Auger","email":"eric.auger@redhat.com"},"content":"Hi Peter,\n\nOn 07/09/2017 14:39, Peter Maydell wrote:\n> On 1 September 2017 at 18:21, Eric Auger wrote:\n>> This series implements the emulation code for ARM SMMUv3.\n>>\n>> Changes since v6:\n>> - DPDK testpmd now running on guest with 2 assigned VFs\n>> - Changed the instantiation method: add the following option to\n>> the QEMU command line\n>> -device smmuv3 # for virtio/vhost use cases\n>> -device smmuv3,caching-mode # for vfio use cases (based on [1])\n>> - splitted the series into smaller patches to allow the review\n>> - the VFIO integration based on \"tlbi-on-map\" smmuv3 driver\n>> is isolated from the rest: last 2 patches, not for upstream.\n>> This is shipped for testing/bench until a better solution is found.\n>> - Reworked permission flag checks and event generation\n> \n> Hi Eric -- I see you've upgraded this from an RFC to a PATCH set.\n> Do you want the patches reviewed and (eventually) taken into git\n> now?\n\nYes I split the series to make it more reviewable and from a functional\npoint of view I have run all major use cases. So now I would encourage\npeople to start reviewing the series (focusing on patches 1-18).\n\nThanks\n\nEric\n> \n> thanks\n> -- PMM\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx01.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx01.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eric.auger@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xpVzc3Y9Yz9s8J\n\tfor ;\n\tFri, 8 Sep 2017 18:36:43 +1000 (AEST)","from localhost ([::1]:43888 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dqEmD-0007lE-Nn\n\tfor incoming@patchwork.ozlabs.org; Fri, 08 Sep 2017 04:36:37 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:33775)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dqElg-0007hp-97\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 04:36:13 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dqElb-0004Wp-DZ\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 04:36:04 -0400","from mx1.redhat.com ([209.132.183.28]:46098)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dqElL-0004DS-FL; Fri, 08 Sep 2017 04:35:43 -0400","from smtp.corp.redhat.com\n\t(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 53DC981DE4;\n\tFri, 8 Sep 2017 08:35:41 +0000 (UTC)","from localhost.localdomain (ovpn-117-1.ams2.redhat.com\n\t[10.36.117.1])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id F1663600C0;\n\tFri, 8 Sep 2017 08:35:29 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 53DC981DE4","To":"Peter Maydell ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t","From":"Auger Eric ","Message-ID":"<05efe60e-e4d0-24e1-548b-b587bf5153ff@redhat.com>","Date":"Fri, 8 Sep 2017 10:35:28 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101\n\tThunderbird/45.4.0","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.11","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.25]);\n\tFri, 08 Sep 2017 08:35:41 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH v7 00/20] ARM SMMUv3 Emulation Support","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Radha Mohan , Andrew Jones ,\n\tTrey Cain ,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\t\"Michael S. Tsirkin\" ,\n\tjean-philippe.brucker@arm.com, Tomasz Nowicki ,\n\tWill Deacon ,\n\tQEMU Developers , Peter Xu ,\n\tAlex Williamson , \n\tqemu-arm ,\n\tChristoffer Dall ,\n\t\"Edgar E. Iglesias\" ,\n\trobin.murphy@arm.com, wtownsen@redhat.com,\n\tBharat Bhushan ,\n\tPrem Mallappa , eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1765153,"web_url":"http://patchwork.ozlabs.org/comment/1765153/","msgid":"<47f736e8-b80e-6926-b050-cf18796f342e@redhat.com>","list_archive_url":null,"date":"2017-09-08T08:36:33","subject":"Re: [Qemu-devel] [PATCH v7 00/20] ARM SMMUv3 Emulation Support","submitter":{"id":69187,"url":"http://patchwork.ozlabs.org/api/people/69187/","name":"Eric Auger","email":"eric.auger@redhat.com"},"content":"Hi Michael,\n\nOn 08/09/2017 07:47, Michael S. Tsirkin wrote:\n> On Fri, Sep 01, 2017 at 07:21:03PM +0200, Eric Auger wrote:\n>> This series implements the emulation code for ARM SMMUv3.\n> \n> Can you add some code to block using vfio with this\n> until patches 19+20 are ready?\nSure.\n\nThanks\n\nEric\n> Then 1-18 could be applied.\n> \n>> Changes since v6:\n>> - DPDK testpmd now running on guest with 2 assigned VFs\n>> - Changed the instantiation method: add the following option to\n>> the QEMU command line\n>> -device smmuv3 # for virtio/vhost use cases\n>> -device smmuv3,caching-mode # for vfio use cases (based on [1])\n>> - splitted the series into smaller patches to allow the review\n>> - the VFIO integration based on \"tlbi-on-map\" smmuv3 driver\n>> is isolated from the rest: last 2 patches, not for upstream.\n>> This is shipped for testing/bench until a better solution is found.\n>> - Reworked permission flag checks and event generation\n>>\n>> testing:\n>> - in dt and ACPI modes\n>> - virtio-net-pci and vhost-net devices using dma ops with various\n>> guest page sizes [2]\n>> - assigned VFs using dma ops [3]:\n>> - AMD Overdrive and igbvf passthrough (using gsi direct mapping)\n>> - Cavium ThunderX and ixgbevf passthrough (using KVM MSI routing)\n>> - DPDK testpmd on guest running with VFIO user space drivers (2 igbvf) [3]\n>> with guest and host page size equal (4kB)\n>>\n>> Known limitations:\n>> - no VMSAv8-32 suport\n>> - no nested stage support (S1 + S2)\n>> - no support for HYP mappings\n>> - register fine emulation, commands, interrupts and errors were\n>> not accurately tested. Handling is sufficient to run use cases\n>> described above though.\n>> - interrupts and event generation not observed yet.\n>>\n>> Best Regards\n>>\n>> Eric\n>>\n>> This series can be found at:\n>> v7: https://github.com/eauger/qemu/tree/v2.10.0-SMMU-v7\n>> Previous version at:\n>> v6: https://github.com/eauger/qemu/tree/v2.10.0-rc2-SMMU-v6\n>>\n>> References:\n>> [1] [RFC v2 0/4] arm-smmu-v3 tlbi-on-map option\n>> https://lkml.org/lkml/2017/8/11/426\n>>\n>> [2] qemu cmd line excerpt:\n>> -device smmuv3 \\\n>> -netdev tap,id=tap0,script=no,downscript=no,ifname=tap0,vhost=off \\\n>> -device virtio-net-pci,netdev=tap0,mac=6a:f5:10:b1:3d:d2,iommu_platform,disable-modern=off,disable-legacy=on \\\n>> [3] use -device smmuv3,caching-mode\n>>\n>>\n>> History:\n>> v6 -> v7:\n>> - see above\n>>\n>> v5 -> v6:\n>> - Rebase on 2.10 and IOMMUMemoryRegion\n>> - add ACPI TLBI_ON_MAP support (VFIO integration also works in\n>> ACPI mode)\n>> - fix block replay\n>> - handle implementation defined SMMU_CMD_TLBI_NH_VA_AM cmd\n>> (goes along with TLBI_ON_MAP FW quirk)\n>> - replay systematically unmap the whole range first\n>> - smmuv3_map_hook does not unmap anymore and the unmap is done\n>> before the replay\n>> - add and use smmuv3_context_device_invalidate instead of\n>> blindly replaying everything\n>>\n>> v4 -> v5:\n>> - initial_level now part of SMMUTransCfg\n>> - smmu_page_walk_64 takes into account the max input size\n>> - implement sys->iommu_ops.replay and sys->iommu_ops.notify_flag_changed\n>> - smmuv3_translate: bug fix: don't walk on bypass\n>> - smmu_update_qreg: fix PROD index update\n>> - I did not yet address Peter's comments as the code is not mature enough\n>> to be split into sub patches.\n>>\n>> v3 -> v4 [Eric]:\n>> - page table walk rewritten to allow scan of the page table within a\n>> range of IOVA. This prepares for VFIO integration and replay.\n>> - configuration parsing partially reworked.\n>> - do not advertise unsupported/untested features: S2, S1 + S2, HYP,\n>> PRI, ATS, ..\n>> - added ACPI table generation\n>> - migrated to dynamic traces\n>> - mingw compilation fix\n>>\n>> v2 -> v3 [Eric]:\n>> - rebased on 2.9\n>> - mostly code and patch reorganization to ease the review process\n>> - optional patches removed. They may be handled separately. I am currently\n>> working on ACPI enablement.\n>> - optional instantiation of the smmu in mach-virt\n>> - removed [2/9] (fdt functions) since not mandated\n>> - start splitting main patch into base and derived object\n>> - no new function feature added\n>>\n>> v1 -> v2 [Prem]:\n>> - Adopted review comments from Eric Auger\n>> - Make SMMU_DPRINTF to internally call qemu_log\n>> (since translation requests are too many, we need control\n>> on the type of log we want)\n>> - SMMUTransCfg modified to suite simplicity\n>> - Change RegInfo to uint64 register array\n>> - Code cleanup\n>> - Test cleanups\n>> - Reshuffled patches\n>>\n>> v0 -> v1 [Prem]:\n>> - As per SMMUv3 spec 16.0 (only is_ste_consistant() is noticeable)\n>> - Reworked register access/update logic\n>> - Factored out translation code for\n>> - single point bug fix\n>> - sharing/removal in future\n>> - (optional) Unit tests added, with PCI test device\n>> - S1 with 4k/64k, S1+S2 with 4k/64k\n>> - (S1 or S2) only can be verified by Linux 4.7 driver\n>> - (optional) Priliminary ACPI support\n>>\n>> v0 [Prem]:\n>> - Implements SMMUv3 spec 11.0\n>> - Supported for PCIe devices,\n>> - Command Queue and Event Queue supported\n>> - LPAE only, S1 is supported and Tested, S2 not tested\n>> - BE mode Translation not supported\n>> - IRQ support (legacy, no MSI)\n>>\n>> Eric Auger (18):\n>> hw/arm/smmu-common: smmu base device and datatypes\n>> hw/arm/smmu-common: IOMMU memory region and address space setup\n>> hw/arm/smmu-common: smmu_read/write_sysmem\n>> hw/arm/smmu-common: VMSAv8-64 page table walk\n>> hw/arm/smmuv3: Wired IRQ and GERROR helpers\n>> hw/arm/smmuv3: Queue helpers\n>> hw/arm/smmuv3: Implement MMIO write operations\n>> hw/arm/smmuv3: Event queue recording helper\n>> hw/arm/smmuv3: Implement translate callback\n>> target/arm/kvm: Translate the MSI doorbell in kvm_arch_fixup_msi_route\n>> hw/arm/smmuv3: Implement data structure and TLB invalidation\n>> notifications\n>> hw/arm/smmuv3: Implement IOMMU memory region replay callback\n>> hw/arm/virt: Store the PCI host controller dt phandle\n>> hw/arm/sysbus-fdt: Pass the VirtMachineState to the node creation\n>> functions\n>> hw/arm/sysbus-fdt: Pass the platform bus base address in\n>> PlatformBusFDTData\n>> hw/arm/sysbus-fdt: Allow smmuv3 dynamic instantiation\n>> hw/arm/smmuv3: [not for upstream] add SMMU_CMD_TLBI_NH_VA_AM handling\n>> hw/arm/smmuv3: [not for upstream] Add caching-mode option\n>>\n>> Prem Mallappa (2):\n>> hw/arm/smmuv3: Skeleton\n>> hw/arm/virt-acpi-build: Add smmuv3 node in IORT table\n>>\n>> default-configs/aarch64-softmmu.mak | 1 +\n>> hw/arm/Makefile.objs | 1 +\n>> hw/arm/smmu-common.c | 527 ++++++++++++++++\n>> hw/arm/smmu-internal.h | 105 ++++\n>> hw/arm/smmuv3-internal.h | 584 +++++++++++++++++\n>> hw/arm/smmuv3.c | 1181 +++++++++++++++++++++++++++++++++++\n>> hw/arm/sysbus-fdt.c | 129 +++-\n>> hw/arm/trace-events | 48 ++\n>> hw/arm/virt-acpi-build.c | 63 +-\n>> hw/arm/virt.c | 6 +-\n>> include/hw/acpi/acpi-defs.h | 15 +\n>> include/hw/arm/smmu-common.h | 123 ++++\n>> include/hw/arm/smmuv3.h | 80 +++\n>> include/hw/arm/sysbus-fdt.h | 2 +\n>> include/hw/arm/virt.h | 15 +\n>> target/arm/kvm.c | 27 +\n>> target/arm/trace-events | 3 +\n>> 17 files changed, 2886 insertions(+), 24 deletions(-)\n>> create mode 100644 hw/arm/smmu-common.c\n>> create mode 100644 hw/arm/smmu-internal.h\n>> create mode 100644 hw/arm/smmuv3-internal.h\n>> create mode 100644 hw/arm/smmuv3.c\n>> create mode 100644 include/hw/arm/smmu-common.h\n>> create mode 100644 include/hw/arm/smmuv3.h\n>>\n>> -- \n>> 2.5.5\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx10.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx10.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eric.auger@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xpW0l2hpPz9s8J\n\tfor ;\n\tFri, 8 Sep 2017 18:37:43 +1000 (AEST)","from localhost ([::1]:43892 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dqEnF-0008Ey-Fx\n\tfor incoming@patchwork.ozlabs.org; Fri, 08 Sep 2017 04:37:41 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:34259)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dqEmd-00088O-Rj\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 04:37:09 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dqEmX-0005e8-Ta\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 04:37:03 -0400","from mx1.redhat.com ([209.132.183.28]:48926)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dqEmM-0005S7-MY; Fri, 08 Sep 2017 04:36:46 -0400","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 78A505D68D;\n\tFri, 8 Sep 2017 08:36:45 +0000 (UTC)","from localhost.localdomain (ovpn-117-1.ams2.redhat.com\n\t[10.36.117.1])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 487085D97A;\n\tFri, 8 Sep 2017 08:36:35 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 78A505D68D","To":"\"Michael S. Tsirkin\" ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<20170908084657-mutt-send-email-mst@kernel.org>","From":"Auger Eric ","Message-ID":"<47f736e8-b80e-6926-b050-cf18796f342e@redhat.com>","Date":"Fri, 8 Sep 2017 10:36:33 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101\n\tThunderbird/45.4.0","MIME-Version":"1.0","In-Reply-To":"<20170908084657-mutt-send-email-mst@kernel.org>","Content-Type":"text/plain; charset=windows-1252","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.14","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.39]);\n\tFri, 08 Sep 2017 08:36:45 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH v7 00/20] ARM SMMUv3 Emulation Support","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, bharat.bhushan@nxp.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tprem.mallappa@gmail.com, edgar.iglesias@gmail.com,\n\teric.auger.pro@gmail.com, robin.murphy@arm.com,\n\tchristoffer.dall@linaro.org, wtownsen@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1765247,"web_url":"http://patchwork.ozlabs.org/comment/1765247/","msgid":"<20170908105214.GA16412@virtx40>","list_archive_url":null,"date":"2017-09-08T10:52:14","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 05/20] hw/arm/smmuv3: Skeleton","submitter":{"id":72195,"url":"http://patchwork.ozlabs.org/api/people/72195/","name":"Linu Cherian","email":"linuc.decode@gmail.com"},"content":"Hi Eric,\n\nOn Fri Sep 01, 2017 at 07:21:08PM +0200, Eric Auger wrote:\n> From: Prem Mallappa \n> \n> This patch implements a skeleton for the smmuv3 device.\n> Datatypes and register definitions are introduced. The MMIO\n> region, the interrupts and the queue are initialized (PRI is\n> not supported).\n> \n> Only the MMIO read operation is implemented here.\n> \n> Signed-off-by: Prem Mallappa \n> Signed-off-by: Eric Auger \n> \n> ---\n> v6 -> v7:\n> - split into several patches\n> \n> v5 -> v6:\n> - Use IOMMUMemoryregion\n> - regs become uint32_t and fix 64b MMIO access (.impl)\n> - trace_smmuv3_write/read_mmio take the size param\n> \n> v4 -> v5:\n> - change smmuv3_translate proto (IOMMUAccessFlags flag)\n> - has_stagex replaced by is_ste_stagex\n> - smmu_cfg_populate removed\n> - added smmuv3_decode_config and reworked error management\n> - remwork the naming of IOMMU mrs\n> - fix SMMU_CMDQ_CONS offset\n> \n> v3 -> v4\n> - smmu_irq_update\n> - fix hash key allocation\n> - set smmu_iommu_ops\n> - set SMMU_REG_CR0,\n> - smmuv3_translate: ret.perm not set in bypass mode\n> - use trace events\n> - renamed STM2U64 into L1STD_L2PTR and STMSPAN into L1STD_SPAN\n> - rework smmu_find_ste\n> - fix tg2granule in TT0/0b10 corresponds to 16kB\n> \n> v2 -> v3:\n> - move creation of include/hw/arm/smmuv3.h to this patch to fix compil issue\n> - compilation allowed\n> - fix sbus allocation in smmu_init_pci_iommu\n> - restructure code into headers\n> - misc cleanups\n> ---\n> hw/arm/Makefile.objs | 2 +-\n> hw/arm/smmuv3-internal.h | 201 +++++++++++++++++++++++++++++++++++++++\n> hw/arm/smmuv3.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++\n> hw/arm/trace-events | 3 +\n> include/hw/arm/smmuv3.h | 79 ++++++++++++++++\n> 5 files changed, 523 insertions(+), 1 deletion(-)\n> create mode 100644 hw/arm/smmuv3-internal.h\n> create mode 100644 hw/arm/smmuv3.c\n> create mode 100644 include/hw/arm/smmuv3.h\n> \n> diff --git a/hw/arm/Makefile.objs b/hw/arm/Makefile.objs\n> index 5b2d38d..a7c808b 100644\n> --- a/hw/arm/Makefile.objs\n> +++ b/hw/arm/Makefile.objs\n> @@ -19,4 +19,4 @@ obj-$(CONFIG_FSL_IMX31) += fsl-imx31.o kzm.o\n> obj-$(CONFIG_FSL_IMX6) += fsl-imx6.o sabrelite.o\n> obj-$(CONFIG_ASPEED_SOC) += aspeed_soc.o aspeed.o\n> obj-$(CONFIG_MPS2) += mps2.o\n> -obj-$(CONFIG_ARM_SMMUV3) += smmu-common.o\n> +obj-$(CONFIG_ARM_SMMUV3) += smmu-common.o smmuv3.o\n> diff --git a/hw/arm/smmuv3-internal.h b/hw/arm/smmuv3-internal.h\n> new file mode 100644\n> index 0000000..488acc8\n> --- /dev/null\n> +++ b/hw/arm/smmuv3-internal.h\n> @@ -0,0 +1,201 @@\n> +/*\n> + * ARM SMMUv3 support - Internal API\n> + *\n> + * Copyright (C) 2014-2016 Broadcom Corporation\n> + * Copyright (c) 2017 Red Hat, Inc.\n> + * Written by Prem Mallappa, Eric Auger\n> + *\n> + * This program is free software; you can redistribute it and/or modify\n> + * it under the terms of the GNU General Public License as published by\n> + * the Free Software Foundation, either version 2 of the License, or\n> + * (at your option) any later version.\n> + *\n> + * This program is distributed in the hope that it will be useful,\n> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n> + * GNU General Public License for more details.\n> + *\n> + * You should have received a copy of the GNU General Public License along\n> + * with this program; if not, see .\n> + */\n> +\n> +#ifndef HW_ARM_SMMU_V3_INTERNAL_H\n> +#define HW_ARM_SMMU_V3_INTERNAL_H\n> +\n> +#include \"trace.h\"\n> +#include \"qemu/error-report.h\"\n> +#include \"hw/arm/smmu-common.h\"\n> +\n> +/*****************************\n> + * MMIO Register\n> + *****************************/\n> +enum {\n> + SMMU_REG_IDR0 = 0x0,\n> +\n> +/* IDR0 Field Values and supported features */\n> +\n> +#define SMMU_IDR0_S2P 1 /* stage 2 */\n> +#define SMMU_IDR0_S1P 1 /* stage 1 */\n> +#define SMMU_IDR0_TTF 2 /* Aarch64 only - not Aarch32 (LPAE) */\n> +#define SMMU_IDR0_COHACC 1 /* IO coherent access */\n> +#define SMMU_IDR0_HTTU 2 /* Access and Dirty flag update */\n> +#define SMMU_IDR0_HYP 0 /* Hypervisor Stage 1 contexts */\n> +#define SMMU_IDR0_ATS 0 /* PCIe RC ATS */\n> +#define SMMU_IDR0_ASID16 1 /* 16-bit ASID */\n> +#define SMMU_IDR0_PRI 0 /* Page Request Interface */\n> +#define SMMU_IDR0_VMID16 0 /* 16-bit VMID */\n> +#define SMMU_IDR0_CD2L 0 /* 2-level Context Descriptor table */\n> +#define SMMU_IDR0_STALL 1 /* Stalling fault model */\n> +#define SMMU_IDR0_TERM 1 /* Termination model behaviour */\n> +#define SMMU_IDR0_STLEVEL 1 /* Multi-level Stream Table */\n> +\n> +#define SMMU_IDR0_S2P_SHIFT 0\n> +#define SMMU_IDR0_S1P_SHIFT 1\n> +#define SMMU_IDR0_TTF_SHIFT 2\n> +#define SMMU_IDR0_COHACC_SHIFT 4\n> +#define SMMU_IDR0_HTTU_SHIFT 6\n> +#define SMMU_IDR0_HYP_SHIFT 9\n> +#define SMMU_IDR0_ATS_SHIFT 10\n> +#define SMMU_IDR0_ASID16_SHIFT 12\n> +#define SMMU_IDR0_PRI_SHIFT 16\n> +#define SMMU_IDR0_VMID16_SHIFT 18\n> +#define SMMU_IDR0_CD2L_SHIFT 19\n> +#define SMMU_IDR0_STALL_SHIFT 24\n> +#define SMMU_IDR0_TERM_SHIFT 26\n> +#define SMMU_IDR0_STLEVEL_SHIFT 27\n> +\n> + SMMU_REG_IDR1 = 0x4,\n> +#define SMMU_IDR1_SIDSIZE 16\n> + SMMU_REG_IDR2 = 0x8,\n> + SMMU_REG_IDR3 = 0xc,\n> + SMMU_REG_IDR4 = 0x10,\n> + SMMU_REG_IDR5 = 0x14,\n> +#define SMMU_IDR5_GRAN_SHIFT 4\n> +#define SMMU_IDR5_GRAN 0b101 /* GRAN4K, GRAN64K */\n> +#define SMMU_IDR5_OAS 4 /* 44 bits */\n> + SMMU_REG_IIDR = 0x1c,\n> + SMMU_REG_CR0 = 0x20,\n> +\n> +#define SMMU_CR0_SMMU_ENABLE (1 << 0)\n> +#define SMMU_CR0_PRIQ_ENABLE (1 << 1)\n> +#define SMMU_CR0_EVTQ_ENABLE (1 << 2)\n> +#define SMMU_CR0_CMDQ_ENABLE (1 << 3)\n> +#define SMMU_CR0_ATS_CHECK (1 << 4)\n> +\n> + SMMU_REG_CR0_ACK = 0x24,\n> + SMMU_REG_CR1 = 0x28,\n> + SMMU_REG_CR2 = 0x2c,\n> +\n> + SMMU_REG_STATUSR = 0x40,\n> +\n> + SMMU_REG_IRQ_CTRL = 0x50,\n> + SMMU_REG_IRQ_CTRL_ACK = 0x54,\n> +\n> +#define SMMU_IRQ_CTRL_GERROR_EN (1 << 0)\n> +#define SMMU_IRQ_CTRL_EVENT_EN (1 << 1)\n> +#define SMMU_IRQ_CTRL_PRI_EN (1 << 2)\n> +\n> + SMMU_REG_GERROR = 0x60,\n> +\n> +#define SMMU_GERROR_CMDQ (1 << 0)\n> +#define SMMU_GERROR_EVENTQ_ABT (1 << 2)\n> +#define SMMU_GERROR_PRIQ_ABT (1 << 3)\n> +#define SMMU_GERROR_MSI_CMDQ_ABT (1 << 4)\n> +#define SMMU_GERROR_MSI_EVENTQ_ABT (1 << 5)\n> +#define SMMU_GERROR_MSI_PRIQ_ABT (1 << 6)\n> +#define SMMU_GERROR_MSI_GERROR_ABT (1 << 7)\n> +#define SMMU_GERROR_SFM_ERR (1 << 8)\n> +\n> + SMMU_REG_GERRORN = 0x64,\n> + SMMU_REG_GERROR_IRQ_CFG0 = 0x68,\n> + SMMU_REG_GERROR_IRQ_CFG1 = 0x70,\n> + SMMU_REG_GERROR_IRQ_CFG2 = 0x74,\n> +\n> + /* SMMU_BASE_RA Applies to STRTAB_BASE, CMDQ_BASE and EVTQ_BASE */\n> +#define SMMU_BASE_RA (1ULL << 62)\n> + SMMU_REG_STRTAB_BASE = 0x80,\n> + SMMU_REG_STRTAB_BASE_CFG = 0x88,\n> +\n> + SMMU_REG_CMDQ_BASE = 0x90,\n> + SMMU_REG_CMDQ_PROD = 0x98,\n> + SMMU_REG_CMDQ_CONS = 0x9c,\n> + /* CMD Consumer (CONS) */\n> +#define SMMU_CMD_CONS_ERR_SHIFT 24\n> +#define SMMU_CMD_CONS_ERR_BITS 7\n> +\n> + SMMU_REG_EVTQ_BASE = 0xa0,\n> + SMMU_REG_EVTQ_PROD = 0xa8,\n> + SMMU_REG_EVTQ_CONS = 0xac,\n> + SMMU_REG_EVTQ_IRQ_CFG0 = 0xb0,\n> + SMMU_REG_EVTQ_IRQ_CFG1 = 0xb8,\n> + SMMU_REG_EVTQ_IRQ_CFG2 = 0xbc,\n> +\n> + SMMU_REG_PRIQ_BASE = 0xc0,\n> + SMMU_REG_PRIQ_PROD = 0xc8,\n> + SMMU_REG_PRIQ_CONS = 0xcc,\n> + SMMU_REG_PRIQ_IRQ_CFG0 = 0xd0,\n> + SMMU_REG_PRIQ_IRQ_CFG1 = 0xd8,\n> + SMMU_REG_PRIQ_IRQ_CFG2 = 0xdc,\n> +\n> + SMMU_ID_REGS_OFFSET = 0xfd0,\n> +\n> + /* Secure registers are not used for now */\n> + SMMU_SECURE_OFFSET = 0x8000,\n> +};\n> +\n> +/**********************\n> + * Data Structures\n> + **********************/\n> +\n> +struct __smmu_data2 {\n> + uint32_t word[2];\n> +};\n> +\n> +struct __smmu_data8 {\n> + uint32_t word[8];\n> +};\n> +\n> +struct __smmu_data16 {\n> + uint32_t word[16];\n> +};\n> +\n> +struct __smmu_data4 {\n> + uint32_t word[4];\n> +};\n> +\n> +typedef struct __smmu_data4 Cmd; /* Command Entry */\n> +typedef struct __smmu_data8 Evt; /* Event Entry */\n> +\n> +/*****************************\n> + * Register Access Primitives\n> + *****************************/\n> +\n> +static inline void smmu_write32_reg(SMMUV3State *s, uint32_t addr, uint32_t val)\n> +{\n> + s->regs[addr >> 2] = val;\n> +}\n> +\n> +static inline void smmu_write64_reg(SMMUV3State *s, uint32_t addr, uint64_t val)\n> +{\n> + addr >>= 2;\n> + s->regs[addr] = extract64(val, 0, 32);\n> + s->regs[addr + 1] = extract64(val, 32, 32);\n> +}\n> +\n> +static inline uint32_t smmu_read32_reg(SMMUV3State *s, uint32_t addr)\n> +{\n> + return s->regs[addr >> 2];\n> +}\n> +\n> +static inline uint64_t smmu_read64_reg(SMMUV3State *s, uint32_t addr)\n> +{\n> + addr >>= 2;\n> + return s->regs[addr] | ((uint64_t)(s->regs[addr + 1]) << 32);\n> +}\n> +\n> +static inline int smmu_enabled(SMMUV3State *s)\n> +{\n> + return smmu_read32_reg(s, SMMU_REG_CR0) & SMMU_CR0_SMMU_ENABLE;\n> +}\n> +\n> +#endif\n> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n> new file mode 100644\n> index 0000000..0a7cd1c\n> --- /dev/null\n> +++ b/hw/arm/smmuv3.c\n> @@ -0,0 +1,239 @@\n> +/*\n> + * Copyright (C) 2014-2016 Broadcom Corporation\n> + * Copyright (c) 2017 Red Hat, Inc.\n> + * Written by Prem Mallappa, Eric Auger\n> + *\n> + * This program is free software; you can redistribute it and/or modify\n> + * it under the terms of the GNU General Public License as published by\n> + * the Free Software Foundation, either version 2 of the License, or\n> + * (at your option) any later version.\n> + *\n> + * This program is distributed in the hope that it will be useful,\n> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n> + * GNU General Public License for more details.\n> + *\n> + * You should have received a copy of the GNU General Public License along\n> + * with this program; if not, see .\n> + */\n> +\n> +#include \"qemu/osdep.h\"\n> +#include \"hw/boards.h\"\n> +#include \"sysemu/sysemu.h\"\n> +#include \"hw/sysbus.h\"\n> +#include \"hw/pci/pci.h\"\n> +#include \"exec/address-spaces.h\"\n> +#include \"trace.h\"\n> +#include \"qemu/error-report.h\"\n> +\n> +#include \"hw/arm/smmuv3.h\"\n> +#include \"smmuv3-internal.h\"\n> +\n> +static void smmuv3_init_regs(SMMUV3State *s)\n> +{\n> + uint32_t data =\n> + SMMU_IDR0_STLEVEL << SMMU_IDR0_STLEVEL_SHIFT |\n> + SMMU_IDR0_TERM << SMMU_IDR0_TERM_SHIFT |\n> + SMMU_IDR0_STALL << SMMU_IDR0_STALL_SHIFT |\n> + SMMU_IDR0_VMID16 << SMMU_IDR0_VMID16_SHIFT |\n> + SMMU_IDR0_PRI << SMMU_IDR0_PRI_SHIFT |\n> + SMMU_IDR0_ASID16 << SMMU_IDR0_ASID16_SHIFT |\n> + SMMU_IDR0_ATS << SMMU_IDR0_ATS_SHIFT |\n> + SMMU_IDR0_HYP << SMMU_IDR0_HYP_SHIFT |\n> + SMMU_IDR0_HTTU << SMMU_IDR0_HTTU_SHIFT |\n> + SMMU_IDR0_COHACC << SMMU_IDR0_COHACC_SHIFT |\n> + SMMU_IDR0_TTF << SMMU_IDR0_TTF_SHIFT |\n> + SMMU_IDR0_S1P << SMMU_IDR0_S1P_SHIFT |\n> + SMMU_IDR0_S2P << SMMU_IDR0_S2P_SHIFT;\n> +\n> + smmu_write32_reg(s, SMMU_REG_IDR0, data);\n> +\n> +#define SMMU_QUEUE_SIZE_LOG2 19\n> + data =\n> + 1 << 27 | /* Attr Types override */\n> + SMMU_QUEUE_SIZE_LOG2 << 21 | /* Cmd Q size */\n> + SMMU_QUEUE_SIZE_LOG2 << 16 | /* Event Q size */\n> + SMMU_QUEUE_SIZE_LOG2 << 11 | /* PRI Q size */\n> + 0 << 6 | /* SSID not supported */\n> + SMMU_IDR1_SIDSIZE;\n> +\n> + smmu_write32_reg(s, SMMU_REG_IDR1, data);\n> +\n> + s->sid_size = SMMU_IDR1_SIDSIZE;\n> +\n> + data = SMMU_IDR5_GRAN << SMMU_IDR5_GRAN_SHIFT | SMMU_IDR5_OAS;\n\nFor VFIO case, should we not set the granule size based on underlying \npagesize bitmap derived from VFIO_IOMMU_GET_INFO. Else if guest kernel\nis build with 4k page size and the host kernel is 64k we would start\ngetting map errors. \n\n\n\n> +\n> + smmu_write32_reg(s, SMMU_REG_IDR5, data);\n> +}\n> +\n> +static void smmuv3_init_queues(SMMUV3State *s)\n> +{\n> + s->cmdq.prod = 0;\n> + s->cmdq.cons = 0;\n> + s->cmdq.wrap.prod = 0;\n> + s->cmdq.wrap.cons = 0;\n> +\n> + s->evtq.prod = 0;\n> + s->evtq.cons = 0;\n> + s->evtq.wrap.prod = 0;\n> + s->evtq.wrap.cons = 0;\n> +\n> + s->cmdq.entries = SMMU_QUEUE_SIZE_LOG2;\n> + s->cmdq.ent_size = sizeof(Cmd);\n> + s->evtq.entries = SMMU_QUEUE_SIZE_LOG2;\n> + s->evtq.ent_size = sizeof(Evt);\n> +}\n> +\n> +static void smmuv3_init(SMMUV3State *s)\n> +{\n> + smmuv3_init_regs(s);\n> + smmuv3_init_queues(s);\n> +}\n> +\n> +static inline void smmu_update_base_reg(SMMUV3State *s, uint64_t *base,\n> + uint64_t val)\n> +{\n> + *base = val & ~(SMMU_BASE_RA | 0x3fULL);\n> +}\n> +\n> +static void smmu_write_mmio_fixup(SMMUV3State *s, hwaddr *addr)\n> +{\n> + switch (*addr) {\n> + case 0x100a8: case 0x100ac: /* Aliasing => page0 registers */\n> + case 0x100c8: case 0x100cc:\n> + *addr ^= (hwaddr)0x10000;\n> + }\n> +}\n> +\n> +static void smmu_write_mmio(void *opaque, hwaddr addr,\n> + uint64_t val, unsigned size)\n> +{\n> +}\n> +\n> +static uint64_t smmu_read_mmio(void *opaque, hwaddr addr, unsigned size)\n> +{\n> + SMMUState *sys = opaque;\n> + SMMUV3State *s = SMMU_V3_DEV(sys);\n> + uint64_t val;\n> +\n> + smmu_write_mmio_fixup(s, &addr);\n> +\n> + /* Primecell/Corelink ID registers */\n> + switch (addr) {\n> + case 0xFF0 ... 0xFFC:\n> + case 0xFDC ... 0xFE4:\n> + val = 0;\n> + error_report(\"addr:0x%\"PRIx64\" val:0x%\"PRIx64, addr, val);\n> + break;\n> + case SMMU_REG_STRTAB_BASE ... SMMU_REG_CMDQ_BASE:\n> + case SMMU_REG_EVTQ_BASE:\n> + case SMMU_REG_PRIQ_BASE ... SMMU_REG_PRIQ_IRQ_CFG1:\n> + val = smmu_read64_reg(s, addr);\n> + break;\n> + default:\n> + val = (uint64_t)smmu_read32_reg(s, addr);\n> + break;\n> + }\n> +\n> + trace_smmuv3_read_mmio(addr, val, size);\n> + return val;\n> +}\n> +\n> +static const MemoryRegionOps smmu_mem_ops = {\n> + .read = smmu_read_mmio,\n> + .write = smmu_write_mmio,\n> + .endianness = DEVICE_LITTLE_ENDIAN,\n> + .valid = {\n> + .min_access_size = 4,\n> + .max_access_size = 8,\n> + },\n> + .impl = {\n> + .min_access_size = 4,\n> + .max_access_size = 8,\n> + },\n> +};\n> +\n> +static void smmu_init_irq(SMMUV3State *s, SysBusDevice *dev)\n> +{\n> + int i;\n> +\n> + for (i = 0; i < ARRAY_SIZE(s->irq); i++) {\n> + sysbus_init_irq(dev, &s->irq[i]);\n> + }\n> +}\n> +\n> +static void smmu_reset(DeviceState *dev)\n> +{\n> + SMMUV3State *s = SMMU_V3_DEV(dev);\n> + smmuv3_init(s);\n> +}\n> +\n> +static void smmu_realize(DeviceState *d, Error **errp)\n> +{\n> + SMMUState *sys = SMMU_SYS_DEV(d);\n> + SMMUV3State *s = SMMU_V3_DEV(sys);\n> + SysBusDevice *dev = SYS_BUS_DEVICE(d);\n> +\n> + memory_region_init_io(&sys->iomem, OBJECT(s),\n> + &smmu_mem_ops, sys, TYPE_SMMU_V3_DEV, 0x20000);\n> +\n> + sys->mrtypename = g_strdup(TYPE_SMMUV3_IOMMU_MEMORY_REGION);\n> +\n> + sysbus_init_mmio(dev, &sys->iomem);\n> +\n> + smmu_init_irq(s, dev);\n> +}\n> +\n> +static const VMStateDescription vmstate_smmuv3 = {\n> + .name = \"smmuv3\",\n> + .version_id = 1,\n> + .minimum_version_id = 1,\n> + .fields = (VMStateField[]) {\n> + VMSTATE_UINT32_ARRAY(regs, SMMUV3State, SMMU_NREGS),\n> + VMSTATE_END_OF_LIST(),\n> + },\n> +};\n> +\n> +static void smmuv3_instance_init(Object *obj)\n> +{\n> + /* Nothing much to do here as of now */\n> +}\n> +\n> +static void smmuv3_class_init(ObjectClass *klass, void *data)\n> +{\n> + DeviceClass *dc = DEVICE_CLASS(klass);\n> +\n> + dc->reset = smmu_reset;\n> + dc->vmsd = &vmstate_smmuv3;\n> + dc->realize = smmu_realize;\n> +}\n> +\n> +static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass,\n> + void *data)\n> +{\n> +}\n> +\n> +static const TypeInfo smmuv3_type_info = {\n> + .name = TYPE_SMMU_V3_DEV,\n> + .parent = TYPE_SMMU_DEV_BASE,\n> + .instance_size = sizeof(SMMUV3State),\n> + .instance_init = smmuv3_instance_init,\n> + .class_data = NULL,\n> + .class_size = sizeof(SMMUV3Class),\n> + .class_init = smmuv3_class_init,\n> +};\n> +\n> +static const TypeInfo smmuv3_iommu_memory_region_info = {\n> + .parent = TYPE_IOMMU_MEMORY_REGION,\n> + .name = TYPE_SMMUV3_IOMMU_MEMORY_REGION,\n> + .class_init = smmuv3_iommu_memory_region_class_init,\n> +};\n> +\n> +static void smmuv3_register_types(void)\n> +{\n> + type_register(&smmuv3_type_info);\n> + type_register(&smmuv3_iommu_memory_region_info);\n> +}\n> +\n> +type_init(smmuv3_register_types)\n> +\n> diff --git a/hw/arm/trace-events b/hw/arm/trace-events\n> index c67cd39..8affbf7 100644\n> --- a/hw/arm/trace-events\n> +++ b/hw/arm/trace-events\n> @@ -14,3 +14,6 @@ smmu_page_walk_level_block_pte(int stage, int level, uint64_t baseaddr, uint64_t\n> smmu_page_walk_level_table_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t address) \"stage=%d, level=%d base@=0x%\"PRIx64\" pte@=0x%\"PRIx64\" pte=0x%\"PRIx64\" next table address = 0x%\"PRIx64\n> smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte) \"baseaddr=0x%\"PRIx64\" index=0x%x, pteaddr=0x%\"PRIx64\", pte=0x%\"PRIx64\n> smmu_set_translated_address(hwaddr iova, hwaddr pa) \"iova = 0x%\"PRIx64\" -> pa = 0x%\"PRIx64\n> +\n> +#hw/arm/smmuv3.c\n> +smmuv3_read_mmio(hwaddr addr, uint64_t val, unsigned size) \"addr: 0x%\"PRIx64\" val:0x%\"PRIx64\" size: 0x%x\"\n> diff --git a/include/hw/arm/smmuv3.h b/include/hw/arm/smmuv3.h\n> new file mode 100644\n> index 0000000..0c8973d\n> --- /dev/null\n> +++ b/include/hw/arm/smmuv3.h\n> @@ -0,0 +1,79 @@\n> +/*\n> + * Copyright (C) 2014-2016 Broadcom Corporation\n> + * Copyright (c) 2017 Red Hat, Inc.\n> + * Written by Prem Mallappa, Eric Auger\n> + *\n> + * This program is free software; you can redistribute it and/or modify\n> + * it under the terms of the GNU General Public License as published by\n> + * the Free Software Foundation, either version 2 of the License, or\n> + * (at your option) any later version.\n> + *\n> + * This program is distributed in the hope that it will be useful,\n> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n> + * GNU General Public License for more details.\n> + *\n> + * You should have received a copy of the GNU General Public License along\n> + * with this program; if not, see .\n> + */\n> +\n> +#ifndef HW_ARM_SMMUV3_H\n> +#define HW_ARM_SMMUV3_H\n> +\n> +#include \"hw/arm/smmu-common.h\"\n> +\n> +#define TYPE_SMMUV3_IOMMU_MEMORY_REGION \"smmuv3-iommu-memory-region\"\n> +\n> +#define SMMU_NREGS 0x200\n> +\n> +typedef struct SMMUQueue {\n> + hwaddr base;\n> + uint32_t prod;\n> + uint32_t cons;\n> + union {\n> + struct {\n> + uint8_t prod:1;\n> + uint8_t cons:1;\n> + };\n> + uint8_t unused;\n> + } wrap;\n> +\n> + uint16_t entries; /* Number of entries */\n> + uint8_t ent_size; /* Size of entry in bytes */\n> + uint8_t shift; /* Size in log2 */\n> +} SMMUQueue;\n> +\n> +typedef struct SMMUV3State {\n> + SMMUState smmu_state;\n> +\n> + /* Local cache of most-frequently used registers */\n> +#define SMMU_FEATURE_2LVL_STE (1 << 0)\n> + uint32_t features;\n> + uint16_t sid_size;\n> + uint16_t sid_split;\n> + uint64_t strtab_base;\n> +\n> + uint32_t regs[SMMU_NREGS];\n> +\n> + qemu_irq irq[4];\n> + SMMUQueue cmdq, evtq;\n> +\n> +} SMMUV3State;\n> +\n> +typedef enum {\n> + SMMU_IRQ_EVTQ,\n> + SMMU_IRQ_PRIQ,\n> + SMMU_IRQ_CMD_SYNC,\n> + SMMU_IRQ_GERROR,\n> +} SMMUIrq;\n> +\n> +typedef struct {\n> + SMMUBaseClass smmu_base_class;\n> +} SMMUV3Class;\n> +\n> +#define TYPE_SMMU_V3_DEV \"smmuv3\"\n> +#define SMMU_V3_DEV(obj) OBJECT_CHECK(SMMUV3State, (obj), TYPE_SMMU_V3_DEV)\n> +#define SMMU_V3_DEVICE_GET_CLASS(obj) \\\n> + OBJECT_GET_CLASS(SMMUBaseClass, (obj), TYPE_SMMU_V3_DEV)\n> +\n> +#endif\n> -- \n> 2.5.5\n> \n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"W0W8oz8P\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xpZTR6jBfz9s83\n\tfor ;\n\tFri, 8 Sep 2017 21:14:19 +1000 (AEST)","from localhost ([::1]:44675 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dqHEo-0001zU-0G\n\tfor incoming@patchwork.ozlabs.org; Fri, 08 Sep 2017 07:14:18 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:37837)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dqGu4-00013b-Q1\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 06:52:59 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dqGty-0001Kn-K4\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 06:52:52 -0400","from mail-pg0-x243.google.com ([2607:f8b0:400e:c05::243]:38838)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1dqGtl-0001CV-7e; Fri, 08 Sep 2017 06:52:33 -0400","by mail-pg0-x243.google.com with SMTP id t3so1153763pgt.5;\n\tFri, 08 Sep 2017 03:52:31 -0700 (PDT)","from virtx40 ([111.93.218.67]) by smtp.gmail.com with ESMTPSA id\n\tf29sm3358036pff.136.2017.09.08.03.52.23\n\t(version=TLS1_2 cipher=AES128-SHA bits=128/128);\n\tFri, 08 Sep 2017 03:52:29 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=JJbyP33CB49A0OOSpBan0QDkCR+0bX4gYwcCtJtWtvY=;\n\tb=W0W8oz8PPZ8PoMd3Rr/sXwHFGdhjDXmrGOFHWaDDBza8GhN83LxVWpCYUkI0G+WKhP\n\tQJkVNxo4BSbS8uA5vqMHTYAiDOyoCysxJqmg6SF++UyRIb3FJUphezKkVQfqAE7VNQ+Q\n\tmQz+Khzsxx9BJ5+xMsMPS7pd9shlpCkGZnDFaGLYBZOaJEDT9rxlYngd4GP337Be3hwN\n\t8jAtDkbR3JtTTW5XQ8R0suT8B6zC3Qg/btqojqfbB654MgBOyZ1vet1/ccNVqd4yfFVs\n\t+tAA6w2nCYxrtbQJJ1y/zlufCtyTVt8Wd9qhSoKFtb6vLVpvDJAy2j0ihQ9atdXb3Vok\n\t+pAQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=JJbyP33CB49A0OOSpBan0QDkCR+0bX4gYwcCtJtWtvY=;\n\tb=VPNUSzMMumRokvMlNnUxq2Kp/091iMEPz+hpr0oyVeCnbees3aN53VT9ISghuV7mPc\n\ttW532xMuYb5Ia01npCKOOcMYCeWAhZ+KTHgFAqzudaDs9Lc+laRMzfALF62OW/m7qNWG\n\t2/KZG5O3VyIU560HQNq4KjUigbDYYYy9XdZOm25fcjAeRHKJnA5VRQkzBi9tog1+S3zt\n\tJj6pwLTqf2v3qBTFmiOML3MvIQa5u5BH73bPm29vgNtvthxsdCo6jWq8VBhq1m1cu2Lp\n\t0GH6mz4h5WLiHkRkus0ayGYh9y2y/4mb/FBdpSD4F9IVbcpbqVEKAZ3BrUnjoXuckRSR\n\tGH+Q==","X-Gm-Message-State":"AHPjjUgdAk8RBFXmz7/ST40OfORjVqitrByWxNh7kAHEqdMKZtsKta8v\n\trjVpfTq3lxjIVA==","X-Google-Smtp-Source":"ADKCNb69VrKq4LC1w1gtYWNVk5Jh3kR/mpo2htmkiINOWFgdXREsJJW5DHguqo5R5PY47KCzzDJu6A==","X-Received":"by 10.84.241.67 with SMTP id u3mr2971586plm.135.1504867950497;\n\tFri, 08 Sep 2017 03:52:30 -0700 (PDT)","Date":"Fri, 8 Sep 2017 16:22:14 +0530","From":"Linu Cherian ","To":"Eric Auger ","Message-ID":"<20170908105214.GA16412@virtx40>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-6-git-send-email-eric.auger@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<1504286483-23327-6-git-send-email-eric.auger@redhat.com>","User-Agent":"Mutt/1.5.21 (2010-09-15)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:400e:c05::243","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 05/20] hw/arm/smmuv3: Skeleton","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, bharat.bhushan@nxp.com, mst@redhat.com,\n\twill.deacon@arm.com, qemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, linu.cherian@cavium.com,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1765455,"web_url":"http://patchwork.ozlabs.org/comment/1765455/","msgid":"","list_archive_url":null,"date":"2017-09-08T15:18:19","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 05/20] hw/arm/smmuv3: Skeleton","submitter":{"id":69187,"url":"http://patchwork.ozlabs.org/api/people/69187/","name":"Eric Auger","email":"eric.auger@redhat.com"},"content":"Hi Linu,\n\nOn 08/09/2017 12:52, Linu Cherian wrote:\n> Hi Eric,\n> \n> On Fri Sep 01, 2017 at 07:21:08PM +0200, Eric Auger wrote:\n>> From: Prem Mallappa \n>>\n>> This patch implements a skeleton for the smmuv3 device.\n>> Datatypes and register definitions are introduced. The MMIO\n>> region, the interrupts and the queue are initialized (PRI is\n>> not supported).\n>>\n>> Only the MMIO read operation is implemented here.\n>>\n>> Signed-off-by: Prem Mallappa \n>> Signed-off-by: Eric Auger \n>>\n>> ---\n>> v6 -> v7:\n>> - split into several patches\n>>\n>> v5 -> v6:\n>> - Use IOMMUMemoryregion\n>> - regs become uint32_t and fix 64b MMIO access (.impl)\n>> - trace_smmuv3_write/read_mmio take the size param\n>>\n>> v4 -> v5:\n>> - change smmuv3_translate proto (IOMMUAccessFlags flag)\n>> - has_stagex replaced by is_ste_stagex\n>> - smmu_cfg_populate removed\n>> - added smmuv3_decode_config and reworked error management\n>> - remwork the naming of IOMMU mrs\n>> - fix SMMU_CMDQ_CONS offset\n>>\n>> v3 -> v4\n>> - smmu_irq_update\n>> - fix hash key allocation\n>> - set smmu_iommu_ops\n>> - set SMMU_REG_CR0,\n>> - smmuv3_translate: ret.perm not set in bypass mode\n>> - use trace events\n>> - renamed STM2U64 into L1STD_L2PTR and STMSPAN into L1STD_SPAN\n>> - rework smmu_find_ste\n>> - fix tg2granule in TT0/0b10 corresponds to 16kB\n>>\n>> v2 -> v3:\n>> - move creation of include/hw/arm/smmuv3.h to this patch to fix compil issue\n>> - compilation allowed\n>> - fix sbus allocation in smmu_init_pci_iommu\n>> - restructure code into headers\n>> - misc cleanups\n>> ---\n>> hw/arm/Makefile.objs | 2 +-\n>> hw/arm/smmuv3-internal.h | 201 +++++++++++++++++++++++++++++++++++++++\n>> hw/arm/smmuv3.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++\n>> hw/arm/trace-events | 3 +\n>> include/hw/arm/smmuv3.h | 79 ++++++++++++++++\n>> 5 files changed, 523 insertions(+), 1 deletion(-)\n>> create mode 100644 hw/arm/smmuv3-internal.h\n>> create mode 100644 hw/arm/smmuv3.c\n>> create mode 100644 include/hw/arm/smmuv3.h\n>>\n>> diff --git a/hw/arm/Makefile.objs b/hw/arm/Makefile.objs\n>> index 5b2d38d..a7c808b 100644\n>> --- a/hw/arm/Makefile.objs\n>> +++ b/hw/arm/Makefile.objs\n>> @@ -19,4 +19,4 @@ obj-$(CONFIG_FSL_IMX31) += fsl-imx31.o kzm.o\n>> obj-$(CONFIG_FSL_IMX6) += fsl-imx6.o sabrelite.o\n>> obj-$(CONFIG_ASPEED_SOC) += aspeed_soc.o aspeed.o\n>> obj-$(CONFIG_MPS2) += mps2.o\n>> -obj-$(CONFIG_ARM_SMMUV3) += smmu-common.o\n>> +obj-$(CONFIG_ARM_SMMUV3) += smmu-common.o smmuv3.o\n>> diff --git a/hw/arm/smmuv3-internal.h b/hw/arm/smmuv3-internal.h\n>> new file mode 100644\n>> index 0000000..488acc8\n>> --- /dev/null\n>> +++ b/hw/arm/smmuv3-internal.h\n>> @@ -0,0 +1,201 @@\n>> +/*\n>> + * ARM SMMUv3 support - Internal API\n>> + *\n>> + * Copyright (C) 2014-2016 Broadcom Corporation\n>> + * Copyright (c) 2017 Red Hat, Inc.\n>> + * Written by Prem Mallappa, Eric Auger\n>> + *\n>> + * This program is free software; you can redistribute it and/or modify\n>> + * it under the terms of the GNU General Public License as published by\n>> + * the Free Software Foundation, either version 2 of the License, or\n>> + * (at your option) any later version.\n>> + *\n>> + * This program is distributed in the hope that it will be useful,\n>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n>> + * GNU General Public License for more details.\n>> + *\n>> + * You should have received a copy of the GNU General Public License along\n>> + * with this program; if not, see .\n>> + */\n>> +\n>> +#ifndef HW_ARM_SMMU_V3_INTERNAL_H\n>> +#define HW_ARM_SMMU_V3_INTERNAL_H\n>> +\n>> +#include \"trace.h\"\n>> +#include \"qemu/error-report.h\"\n>> +#include \"hw/arm/smmu-common.h\"\n>> +\n>> +/*****************************\n>> + * MMIO Register\n>> + *****************************/\n>> +enum {\n>> + SMMU_REG_IDR0 = 0x0,\n>> +\n>> +/* IDR0 Field Values and supported features */\n>> +\n>> +#define SMMU_IDR0_S2P 1 /* stage 2 */\n>> +#define SMMU_IDR0_S1P 1 /* stage 1 */\n>> +#define SMMU_IDR0_TTF 2 /* Aarch64 only - not Aarch32 (LPAE) */\n>> +#define SMMU_IDR0_COHACC 1 /* IO coherent access */\n>> +#define SMMU_IDR0_HTTU 2 /* Access and Dirty flag update */\n>> +#define SMMU_IDR0_HYP 0 /* Hypervisor Stage 1 contexts */\n>> +#define SMMU_IDR0_ATS 0 /* PCIe RC ATS */\n>> +#define SMMU_IDR0_ASID16 1 /* 16-bit ASID */\n>> +#define SMMU_IDR0_PRI 0 /* Page Request Interface */\n>> +#define SMMU_IDR0_VMID16 0 /* 16-bit VMID */\n>> +#define SMMU_IDR0_CD2L 0 /* 2-level Context Descriptor table */\n>> +#define SMMU_IDR0_STALL 1 /* Stalling fault model */\n>> +#define SMMU_IDR0_TERM 1 /* Termination model behaviour */\n>> +#define SMMU_IDR0_STLEVEL 1 /* Multi-level Stream Table */\n>> +\n>> +#define SMMU_IDR0_S2P_SHIFT 0\n>> +#define SMMU_IDR0_S1P_SHIFT 1\n>> +#define SMMU_IDR0_TTF_SHIFT 2\n>> +#define SMMU_IDR0_COHACC_SHIFT 4\n>> +#define SMMU_IDR0_HTTU_SHIFT 6\n>> +#define SMMU_IDR0_HYP_SHIFT 9\n>> +#define SMMU_IDR0_ATS_SHIFT 10\n>> +#define SMMU_IDR0_ASID16_SHIFT 12\n>> +#define SMMU_IDR0_PRI_SHIFT 16\n>> +#define SMMU_IDR0_VMID16_SHIFT 18\n>> +#define SMMU_IDR0_CD2L_SHIFT 19\n>> +#define SMMU_IDR0_STALL_SHIFT 24\n>> +#define SMMU_IDR0_TERM_SHIFT 26\n>> +#define SMMU_IDR0_STLEVEL_SHIFT 27\n>> +\n>> + SMMU_REG_IDR1 = 0x4,\n>> +#define SMMU_IDR1_SIDSIZE 16\n>> + SMMU_REG_IDR2 = 0x8,\n>> + SMMU_REG_IDR3 = 0xc,\n>> + SMMU_REG_IDR4 = 0x10,\n>> + SMMU_REG_IDR5 = 0x14,\n>> +#define SMMU_IDR5_GRAN_SHIFT 4\n>> +#define SMMU_IDR5_GRAN 0b101 /* GRAN4K, GRAN64K */\n>> +#define SMMU_IDR5_OAS 4 /* 44 bits */\n>> + SMMU_REG_IIDR = 0x1c,\n>> + SMMU_REG_CR0 = 0x20,\n>> +\n>> +#define SMMU_CR0_SMMU_ENABLE (1 << 0)\n>> +#define SMMU_CR0_PRIQ_ENABLE (1 << 1)\n>> +#define SMMU_CR0_EVTQ_ENABLE (1 << 2)\n>> +#define SMMU_CR0_CMDQ_ENABLE (1 << 3)\n>> +#define SMMU_CR0_ATS_CHECK (1 << 4)\n>> +\n>> + SMMU_REG_CR0_ACK = 0x24,\n>> + SMMU_REG_CR1 = 0x28,\n>> + SMMU_REG_CR2 = 0x2c,\n>> +\n>> + SMMU_REG_STATUSR = 0x40,\n>> +\n>> + SMMU_REG_IRQ_CTRL = 0x50,\n>> + SMMU_REG_IRQ_CTRL_ACK = 0x54,\n>> +\n>> +#define SMMU_IRQ_CTRL_GERROR_EN (1 << 0)\n>> +#define SMMU_IRQ_CTRL_EVENT_EN (1 << 1)\n>> +#define SMMU_IRQ_CTRL_PRI_EN (1 << 2)\n>> +\n>> + SMMU_REG_GERROR = 0x60,\n>> +\n>> +#define SMMU_GERROR_CMDQ (1 << 0)\n>> +#define SMMU_GERROR_EVENTQ_ABT (1 << 2)\n>> +#define SMMU_GERROR_PRIQ_ABT (1 << 3)\n>> +#define SMMU_GERROR_MSI_CMDQ_ABT (1 << 4)\n>> +#define SMMU_GERROR_MSI_EVENTQ_ABT (1 << 5)\n>> +#define SMMU_GERROR_MSI_PRIQ_ABT (1 << 6)\n>> +#define SMMU_GERROR_MSI_GERROR_ABT (1 << 7)\n>> +#define SMMU_GERROR_SFM_ERR (1 << 8)\n>> +\n>> + SMMU_REG_GERRORN = 0x64,\n>> + SMMU_REG_GERROR_IRQ_CFG0 = 0x68,\n>> + SMMU_REG_GERROR_IRQ_CFG1 = 0x70,\n>> + SMMU_REG_GERROR_IRQ_CFG2 = 0x74,\n>> +\n>> + /* SMMU_BASE_RA Applies to STRTAB_BASE, CMDQ_BASE and EVTQ_BASE */\n>> +#define SMMU_BASE_RA (1ULL << 62)\n>> + SMMU_REG_STRTAB_BASE = 0x80,\n>> + SMMU_REG_STRTAB_BASE_CFG = 0x88,\n>> +\n>> + SMMU_REG_CMDQ_BASE = 0x90,\n>> + SMMU_REG_CMDQ_PROD = 0x98,\n>> + SMMU_REG_CMDQ_CONS = 0x9c,\n>> + /* CMD Consumer (CONS) */\n>> +#define SMMU_CMD_CONS_ERR_SHIFT 24\n>> +#define SMMU_CMD_CONS_ERR_BITS 7\n>> +\n>> + SMMU_REG_EVTQ_BASE = 0xa0,\n>> + SMMU_REG_EVTQ_PROD = 0xa8,\n>> + SMMU_REG_EVTQ_CONS = 0xac,\n>> + SMMU_REG_EVTQ_IRQ_CFG0 = 0xb0,\n>> + SMMU_REG_EVTQ_IRQ_CFG1 = 0xb8,\n>> + SMMU_REG_EVTQ_IRQ_CFG2 = 0xbc,\n>> +\n>> + SMMU_REG_PRIQ_BASE = 0xc0,\n>> + SMMU_REG_PRIQ_PROD = 0xc8,\n>> + SMMU_REG_PRIQ_CONS = 0xcc,\n>> + SMMU_REG_PRIQ_IRQ_CFG0 = 0xd0,\n>> + SMMU_REG_PRIQ_IRQ_CFG1 = 0xd8,\n>> + SMMU_REG_PRIQ_IRQ_CFG2 = 0xdc,\n>> +\n>> + SMMU_ID_REGS_OFFSET = 0xfd0,\n>> +\n>> + /* Secure registers are not used for now */\n>> + SMMU_SECURE_OFFSET = 0x8000,\n>> +};\n>> +\n>> +/**********************\n>> + * Data Structures\n>> + **********************/\n>> +\n>> +struct __smmu_data2 {\n>> + uint32_t word[2];\n>> +};\n>> +\n>> +struct __smmu_data8 {\n>> + uint32_t word[8];\n>> +};\n>> +\n>> +struct __smmu_data16 {\n>> + uint32_t word[16];\n>> +};\n>> +\n>> +struct __smmu_data4 {\n>> + uint32_t word[4];\n>> +};\n>> +\n>> +typedef struct __smmu_data4 Cmd; /* Command Entry */\n>> +typedef struct __smmu_data8 Evt; /* Event Entry */\n>> +\n>> +/*****************************\n>> + * Register Access Primitives\n>> + *****************************/\n>> +\n>> +static inline void smmu_write32_reg(SMMUV3State *s, uint32_t addr, uint32_t val)\n>> +{\n>> + s->regs[addr >> 2] = val;\n>> +}\n>> +\n>> +static inline void smmu_write64_reg(SMMUV3State *s, uint32_t addr, uint64_t val)\n>> +{\n>> + addr >>= 2;\n>> + s->regs[addr] = extract64(val, 0, 32);\n>> + s->regs[addr + 1] = extract64(val, 32, 32);\n>> +}\n>> +\n>> +static inline uint32_t smmu_read32_reg(SMMUV3State *s, uint32_t addr)\n>> +{\n>> + return s->regs[addr >> 2];\n>> +}\n>> +\n>> +static inline uint64_t smmu_read64_reg(SMMUV3State *s, uint32_t addr)\n>> +{\n>> + addr >>= 2;\n>> + return s->regs[addr] | ((uint64_t)(s->regs[addr + 1]) << 32);\n>> +}\n>> +\n>> +static inline int smmu_enabled(SMMUV3State *s)\n>> +{\n>> + return smmu_read32_reg(s, SMMU_REG_CR0) & SMMU_CR0_SMMU_ENABLE;\n>> +}\n>> +\n>> +#endif\n>> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n>> new file mode 100644\n>> index 0000000..0a7cd1c\n>> --- /dev/null\n>> +++ b/hw/arm/smmuv3.c\n>> @@ -0,0 +1,239 @@\n>> +/*\n>> + * Copyright (C) 2014-2016 Broadcom Corporation\n>> + * Copyright (c) 2017 Red Hat, Inc.\n>> + * Written by Prem Mallappa, Eric Auger\n>> + *\n>> + * This program is free software; you can redistribute it and/or modify\n>> + * it under the terms of the GNU General Public License as published by\n>> + * the Free Software Foundation, either version 2 of the License, or\n>> + * (at your option) any later version.\n>> + *\n>> + * This program is distributed in the hope that it will be useful,\n>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n>> + * GNU General Public License for more details.\n>> + *\n>> + * You should have received a copy of the GNU General Public License along\n>> + * with this program; if not, see .\n>> + */\n>> +\n>> +#include \"qemu/osdep.h\"\n>> +#include \"hw/boards.h\"\n>> +#include \"sysemu/sysemu.h\"\n>> +#include \"hw/sysbus.h\"\n>> +#include \"hw/pci/pci.h\"\n>> +#include \"exec/address-spaces.h\"\n>> +#include \"trace.h\"\n>> +#include \"qemu/error-report.h\"\n>> +\n>> +#include \"hw/arm/smmuv3.h\"\n>> +#include \"smmuv3-internal.h\"\n>> +\n>> +static void smmuv3_init_regs(SMMUV3State *s)\n>> +{\n>> + uint32_t data =\n>> + SMMU_IDR0_STLEVEL << SMMU_IDR0_STLEVEL_SHIFT |\n>> + SMMU_IDR0_TERM << SMMU_IDR0_TERM_SHIFT |\n>> + SMMU_IDR0_STALL << SMMU_IDR0_STALL_SHIFT |\n>> + SMMU_IDR0_VMID16 << SMMU_IDR0_VMID16_SHIFT |\n>> + SMMU_IDR0_PRI << SMMU_IDR0_PRI_SHIFT |\n>> + SMMU_IDR0_ASID16 << SMMU_IDR0_ASID16_SHIFT |\n>> + SMMU_IDR0_ATS << SMMU_IDR0_ATS_SHIFT |\n>> + SMMU_IDR0_HYP << SMMU_IDR0_HYP_SHIFT |\n>> + SMMU_IDR0_HTTU << SMMU_IDR0_HTTU_SHIFT |\n>> + SMMU_IDR0_COHACC << SMMU_IDR0_COHACC_SHIFT |\n>> + SMMU_IDR0_TTF << SMMU_IDR0_TTF_SHIFT |\n>> + SMMU_IDR0_S1P << SMMU_IDR0_S1P_SHIFT |\n>> + SMMU_IDR0_S2P << SMMU_IDR0_S2P_SHIFT;\n>> +\n>> + smmu_write32_reg(s, SMMU_REG_IDR0, data);\n>> +\n>> +#define SMMU_QUEUE_SIZE_LOG2 19\n>> + data =\n>> + 1 << 27 | /* Attr Types override */\n>> + SMMU_QUEUE_SIZE_LOG2 << 21 | /* Cmd Q size */\n>> + SMMU_QUEUE_SIZE_LOG2 << 16 | /* Event Q size */\n>> + SMMU_QUEUE_SIZE_LOG2 << 11 | /* PRI Q size */\n>> + 0 << 6 | /* SSID not supported */\n>> + SMMU_IDR1_SIDSIZE;\n>> +\n>> + smmu_write32_reg(s, SMMU_REG_IDR1, data);\n>> +\n>> + s->sid_size = SMMU_IDR1_SIDSIZE;\n>> +\n>> + data = SMMU_IDR5_GRAN << SMMU_IDR5_GRAN_SHIFT | SMMU_IDR5_OAS;\n> \n> For VFIO case, should we not set the granule size based on underlying \n> pagesize bitmap derived from VFIO_IOMMU_GET_INFO. Else if guest kernel\n> is build with 4k page size and the host kernel is 64k we would start\n> getting map errors. \n\nyes at the moment this is not implemented (1st target of the series is\nvirtio/vhost).\n\nOn Intel if I understand correctly the minimum requested is 4K, 2MB.\n1GB is optional. I understand the emulated model does not expose 1GB\n(FL1GP = 0).\n\nOn ARM nothing is mandatory although 4K and 64K minimal granules are\n\"strongly recommended\", leading to the following additional sizes.\n\n if (reg & IDR5_GRAN64K)\n smmu->pgsize_bitmap |= SZ_64K | SZ_512M;\n if (reg & IDR5_GRAN16K)\n smmu->pgsize_bitmap |= SZ_16K | SZ_32M;\n if (reg & IDR5_GRAN4K)\n smmu->pgsize_bitmap |= SZ_4K | SZ_2M | SZ_1G;\n\nMaybe we can override the ID5 values using the vfio_memory_listener. I\nwill try to prototype this idea.\n\nThanks\n\nEric\n\n\n> \n> \n> \n>> +\n>> + smmu_write32_reg(s, SMMU_REG_IDR5, data);\n>> +}\n>> +\n>> +static void smmuv3_init_queues(SMMUV3State *s)\n>> +{\n>> + s->cmdq.prod = 0;\n>> + s->cmdq.cons = 0;\n>> + s->cmdq.wrap.prod = 0;\n>> + s->cmdq.wrap.cons = 0;\n>> +\n>> + s->evtq.prod = 0;\n>> + s->evtq.cons = 0;\n>> + s->evtq.wrap.prod = 0;\n>> + s->evtq.wrap.cons = 0;\n>> +\n>> + s->cmdq.entries = SMMU_QUEUE_SIZE_LOG2;\n>> + s->cmdq.ent_size = sizeof(Cmd);\n>> + s->evtq.entries = SMMU_QUEUE_SIZE_LOG2;\n>> + s->evtq.ent_size = sizeof(Evt);\n>> +}\n>> +\n>> +static void smmuv3_init(SMMUV3State *s)\n>> +{\n>> + smmuv3_init_regs(s);\n>> + smmuv3_init_queues(s);\n>> +}\n>> +\n>> +static inline void smmu_update_base_reg(SMMUV3State *s, uint64_t *base,\n>> + uint64_t val)\n>> +{\n>> + *base = val & ~(SMMU_BASE_RA | 0x3fULL);\n>> +}\n>> +\n>> +static void smmu_write_mmio_fixup(SMMUV3State *s, hwaddr *addr)\n>> +{\n>> + switch (*addr) {\n>> + case 0x100a8: case 0x100ac: /* Aliasing => page0 registers */\n>> + case 0x100c8: case 0x100cc:\n>> + *addr ^= (hwaddr)0x10000;\n>> + }\n>> +}\n>> +\n>> +static void smmu_write_mmio(void *opaque, hwaddr addr,\n>> + uint64_t val, unsigned size)\n>> +{\n>> +}\n>> +\n>> +static uint64_t smmu_read_mmio(void *opaque, hwaddr addr, unsigned size)\n>> +{\n>> + SMMUState *sys = opaque;\n>> + SMMUV3State *s = SMMU_V3_DEV(sys);\n>> + uint64_t val;\n>> +\n>> + smmu_write_mmio_fixup(s, &addr);\n>> +\n>> + /* Primecell/Corelink ID registers */\n>> + switch (addr) {\n>> + case 0xFF0 ... 0xFFC:\n>> + case 0xFDC ... 0xFE4:\n>> + val = 0;\n>> + error_report(\"addr:0x%\"PRIx64\" val:0x%\"PRIx64, addr, val);\n>> + break;\n>> + case SMMU_REG_STRTAB_BASE ... SMMU_REG_CMDQ_BASE:\n>> + case SMMU_REG_EVTQ_BASE:\n>> + case SMMU_REG_PRIQ_BASE ... SMMU_REG_PRIQ_IRQ_CFG1:\n>> + val = smmu_read64_reg(s, addr);\n>> + break;\n>> + default:\n>> + val = (uint64_t)smmu_read32_reg(s, addr);\n>> + break;\n>> + }\n>> +\n>> + trace_smmuv3_read_mmio(addr, val, size);\n>> + return val;\n>> +}\n>> +\n>> +static const MemoryRegionOps smmu_mem_ops = {\n>> + .read = smmu_read_mmio,\n>> + .write = smmu_write_mmio,\n>> + .endianness = DEVICE_LITTLE_ENDIAN,\n>> + .valid = {\n>> + .min_access_size = 4,\n>> + .max_access_size = 8,\n>> + },\n>> + .impl = {\n>> + .min_access_size = 4,\n>> + .max_access_size = 8,\n>> + },\n>> +};\n>> +\n>> +static void smmu_init_irq(SMMUV3State *s, SysBusDevice *dev)\n>> +{\n>> + int i;\n>> +\n>> + for (i = 0; i < ARRAY_SIZE(s->irq); i++) {\n>> + sysbus_init_irq(dev, &s->irq[i]);\n>> + }\n>> +}\n>> +\n>> +static void smmu_reset(DeviceState *dev)\n>> +{\n>> + SMMUV3State *s = SMMU_V3_DEV(dev);\n>> + smmuv3_init(s);\n>> +}\n>> +\n>> +static void smmu_realize(DeviceState *d, Error **errp)\n>> +{\n>> + SMMUState *sys = SMMU_SYS_DEV(d);\n>> + SMMUV3State *s = SMMU_V3_DEV(sys);\n>> + SysBusDevice *dev = SYS_BUS_DEVICE(d);\n>> +\n>> + memory_region_init_io(&sys->iomem, OBJECT(s),\n>> + &smmu_mem_ops, sys, TYPE_SMMU_V3_DEV, 0x20000);\n>> +\n>> + sys->mrtypename = g_strdup(TYPE_SMMUV3_IOMMU_MEMORY_REGION);\n>> +\n>> + sysbus_init_mmio(dev, &sys->iomem);\n>> +\n>> + smmu_init_irq(s, dev);\n>> +}\n>> +\n>> +static const VMStateDescription vmstate_smmuv3 = {\n>> + .name = \"smmuv3\",\n>> + .version_id = 1,\n>> + .minimum_version_id = 1,\n>> + .fields = (VMStateField[]) {\n>> + VMSTATE_UINT32_ARRAY(regs, SMMUV3State, SMMU_NREGS),\n>> + VMSTATE_END_OF_LIST(),\n>> + },\n>> +};\n>> +\n>> +static void smmuv3_instance_init(Object *obj)\n>> +{\n>> + /* Nothing much to do here as of now */\n>> +}\n>> +\n>> +static void smmuv3_class_init(ObjectClass *klass, void *data)\n>> +{\n>> + DeviceClass *dc = DEVICE_CLASS(klass);\n>> +\n>> + dc->reset = smmu_reset;\n>> + dc->vmsd = &vmstate_smmuv3;\n>> + dc->realize = smmu_realize;\n>> +}\n>> +\n>> +static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass,\n>> + void *data)\n>> +{\n>> +}\n>> +\n>> +static const TypeInfo smmuv3_type_info = {\n>> + .name = TYPE_SMMU_V3_DEV,\n>> + .parent = TYPE_SMMU_DEV_BASE,\n>> + .instance_size = sizeof(SMMUV3State),\n>> + .instance_init = smmuv3_instance_init,\n>> + .class_data = NULL,\n>> + .class_size = sizeof(SMMUV3Class),\n>> + .class_init = smmuv3_class_init,\n>> +};\n>> +\n>> +static const TypeInfo smmuv3_iommu_memory_region_info = {\n>> + .parent = TYPE_IOMMU_MEMORY_REGION,\n>> + .name = TYPE_SMMUV3_IOMMU_MEMORY_REGION,\n>> + .class_init = smmuv3_iommu_memory_region_class_init,\n>> +};\n>> +\n>> +static void smmuv3_register_types(void)\n>> +{\n>> + type_register(&smmuv3_type_info);\n>> + type_register(&smmuv3_iommu_memory_region_info);\n>> +}\n>> +\n>> +type_init(smmuv3_register_types)\n>> +\n>> diff --git a/hw/arm/trace-events b/hw/arm/trace-events\n>> index c67cd39..8affbf7 100644\n>> --- a/hw/arm/trace-events\n>> +++ b/hw/arm/trace-events\n>> @@ -14,3 +14,6 @@ smmu_page_walk_level_block_pte(int stage, int level, uint64_t baseaddr, uint64_t\n>> smmu_page_walk_level_table_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t address) \"stage=%d, level=%d base@=0x%\"PRIx64\" pte@=0x%\"PRIx64\" pte=0x%\"PRIx64\" next table address = 0x%\"PRIx64\n>> smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte) \"baseaddr=0x%\"PRIx64\" index=0x%x, pteaddr=0x%\"PRIx64\", pte=0x%\"PRIx64\n>> smmu_set_translated_address(hwaddr iova, hwaddr pa) \"iova = 0x%\"PRIx64\" -> pa = 0x%\"PRIx64\n>> +\n>> +#hw/arm/smmuv3.c\n>> +smmuv3_read_mmio(hwaddr addr, uint64_t val, unsigned size) \"addr: 0x%\"PRIx64\" val:0x%\"PRIx64\" size: 0x%x\"\n>> diff --git a/include/hw/arm/smmuv3.h b/include/hw/arm/smmuv3.h\n>> new file mode 100644\n>> index 0000000..0c8973d\n>> --- /dev/null\n>> +++ b/include/hw/arm/smmuv3.h\n>> @@ -0,0 +1,79 @@\n>> +/*\n>> + * Copyright (C) 2014-2016 Broadcom Corporation\n>> + * Copyright (c) 2017 Red Hat, Inc.\n>> + * Written by Prem Mallappa, Eric Auger\n>> + *\n>> + * This program is free software; you can redistribute it and/or modify\n>> + * it under the terms of the GNU General Public License as published by\n>> + * the Free Software Foundation, either version 2 of the License, or\n>> + * (at your option) any later version.\n>> + *\n>> + * This program is distributed in the hope that it will be useful,\n>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n>> + * GNU General Public License for more details.\n>> + *\n>> + * You should have received a copy of the GNU General Public License along\n>> + * with this program; if not, see .\n>> + */\n>> +\n>> +#ifndef HW_ARM_SMMUV3_H\n>> +#define HW_ARM_SMMUV3_H\n>> +\n>> +#include \"hw/arm/smmu-common.h\"\n>> +\n>> +#define TYPE_SMMUV3_IOMMU_MEMORY_REGION \"smmuv3-iommu-memory-region\"\n>> +\n>> +#define SMMU_NREGS 0x200\n>> +\n>> +typedef struct SMMUQueue {\n>> + hwaddr base;\n>> + uint32_t prod;\n>> + uint32_t cons;\n>> + union {\n>> + struct {\n>> + uint8_t prod:1;\n>> + uint8_t cons:1;\n>> + };\n>> + uint8_t unused;\n>> + } wrap;\n>> +\n>> + uint16_t entries; /* Number of entries */\n>> + uint8_t ent_size; /* Size of entry in bytes */\n>> + uint8_t shift; /* Size in log2 */\n>> +} SMMUQueue;\n>> +\n>> +typedef struct SMMUV3State {\n>> + SMMUState smmu_state;\n>> +\n>> + /* Local cache of most-frequently used registers */\n>> +#define SMMU_FEATURE_2LVL_STE (1 << 0)\n>> + uint32_t features;\n>> + uint16_t sid_size;\n>> + uint16_t sid_split;\n>> + uint64_t strtab_base;\n>> +\n>> + uint32_t regs[SMMU_NREGS];\n>> +\n>> + qemu_irq irq[4];\n>> + SMMUQueue cmdq, evtq;\n>> +\n>> +} SMMUV3State;\n>> +\n>> +typedef enum {\n>> + SMMU_IRQ_EVTQ,\n>> + SMMU_IRQ_PRIQ,\n>> + SMMU_IRQ_CMD_SYNC,\n>> + SMMU_IRQ_GERROR,\n>> +} SMMUIrq;\n>> +\n>> +typedef struct {\n>> + SMMUBaseClass smmu_base_class;\n>> +} SMMUV3Class;\n>> +\n>> +#define TYPE_SMMU_V3_DEV \"smmuv3\"\n>> +#define SMMU_V3_DEV(obj) OBJECT_CHECK(SMMUV3State, (obj), TYPE_SMMU_V3_DEV)\n>> +#define SMMU_V3_DEVICE_GET_CLASS(obj) \\\n>> + OBJECT_GET_CLASS(SMMUBaseClass, (obj), TYPE_SMMU_V3_DEV)\n>> +\n>> +#endif\n>> -- \n>> 2.5.5\n>>\n>>\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eric.auger@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xpgvz5Jvkz9s2G\n\tfor ;\n\tSat, 9 Sep 2017 01:19:10 +1000 (AEST)","from localhost ([::1]:45924 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dqL3k-0006rF-Ac\n\tfor incoming@patchwork.ozlabs.org; Fri, 08 Sep 2017 11:19:08 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:42093)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dqL3L-0006ot-Qe\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 11:18:47 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dqL3I-0000xX-Sx\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 11:18:43 -0400","from mx1.redhat.com ([209.132.183.28]:47081)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dqL3C-0000sO-Cx; Fri, 08 Sep 2017 11:18:34 -0400","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 1134CC047B75;\n\tFri, 8 Sep 2017 15:18:33 +0000 (UTC)","from localhost.localdomain (ovpn-117-1.ams2.redhat.com\n\t[10.36.117.1])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id DF1595D967;\n\tFri, 8 Sep 2017 15:18:20 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 1134CC047B75","To":"Linu Cherian ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-6-git-send-email-eric.auger@redhat.com>\n\t<20170908105214.GA16412@virtx40>","From":"Auger Eric ","Message-ID":"","Date":"Fri, 8 Sep 2017 17:18:19 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101\n\tThunderbird/45.4.0","MIME-Version":"1.0","In-Reply-To":"<20170908105214.GA16412@virtx40>","Content-Type":"text/plain; charset=windows-1252","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.14","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.31]);\n\tFri, 08 Sep 2017 15:18:33 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 05/20] hw/arm/smmuv3: Skeleton","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tprem.mallappa@gmail.com, linu.cherian@cavium.com,\n\trobin.murphy@arm.com, eric.auger.pro@gmail.com,\n\tbharat.bhushan@nxp.com, christoffer.dall@linaro.org, wtownsen@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1766699,"web_url":"http://patchwork.ozlabs.org/comment/1766699/","msgid":"<20170912061421.GA23102@virtx40>","list_archive_url":null,"date":"2017-09-12T06:14:21","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 05/20] hw/arm/smmuv3: Skeleton","submitter":{"id":72349,"url":"http://patchwork.ozlabs.org/api/people/72349/","name":"Cherian, Linu","email":"linu.cherian@cavium.com"},"content":"On Fri Sep 08, 2017 at 05:18:19PM +0200, Auger Eric wrote:\n> Hi Linu,\n> \n> On 08/09/2017 12:52, Linu Cherian wrote:\n> > Hi Eric,\n> > \n> > On Fri Sep 01, 2017 at 07:21:08PM +0200, Eric Auger wrote:\n> >> From: Prem Mallappa \n> >>\n> >> This patch implements a skeleton for the smmuv3 device.\n> >> Datatypes and register definitions are introduced. The MMIO\n> >> region, the interrupts and the queue are initialized (PRI is\n> >> not supported).\n> >>\n> >> Only the MMIO read operation is implemented here.\n> >>\n> >> Signed-off-by: Prem Mallappa \n> >> Signed-off-by: Eric Auger \n> >>\n> >> ---\n> >> v6 -> v7:\n> >> - split into several patches\n> >>\n> >> v5 -> v6:\n> >> - Use IOMMUMemoryregion\n> >> - regs become uint32_t and fix 64b MMIO access (.impl)\n> >> - trace_smmuv3_write/read_mmio take the size param\n> >>\n> >> v4 -> v5:\n> >> - change smmuv3_translate proto (IOMMUAccessFlags flag)\n> >> - has_stagex replaced by is_ste_stagex\n> >> - smmu_cfg_populate removed\n> >> - added smmuv3_decode_config and reworked error management\n> >> - remwork the naming of IOMMU mrs\n> >> - fix SMMU_CMDQ_CONS offset\n> >>\n> >> v3 -> v4\n> >> - smmu_irq_update\n> >> - fix hash key allocation\n> >> - set smmu_iommu_ops\n> >> - set SMMU_REG_CR0,\n> >> - smmuv3_translate: ret.perm not set in bypass mode\n> >> - use trace events\n> >> - renamed STM2U64 into L1STD_L2PTR and STMSPAN into L1STD_SPAN\n> >> - rework smmu_find_ste\n> >> - fix tg2granule in TT0/0b10 corresponds to 16kB\n> >>\n> >> v2 -> v3:\n> >> - move creation of include/hw/arm/smmuv3.h to this patch to fix compil issue\n> >> - compilation allowed\n> >> - fix sbus allocation in smmu_init_pci_iommu\n> >> - restructure code into headers\n> >> - misc cleanups\n> >> ---\n> >> hw/arm/Makefile.objs | 2 +-\n> >> hw/arm/smmuv3-internal.h | 201 +++++++++++++++++++++++++++++++++++++++\n> >> hw/arm/smmuv3.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++\n> >> hw/arm/trace-events | 3 +\n> >> include/hw/arm/smmuv3.h | 79 ++++++++++++++++\n> >> 5 files changed, 523 insertions(+), 1 deletion(-)\n> >> create mode 100644 hw/arm/smmuv3-internal.h\n> >> create mode 100644 hw/arm/smmuv3.c\n> >> create mode 100644 include/hw/arm/smmuv3.h\n> >>\n> >> diff --git a/hw/arm/Makefile.objs b/hw/arm/Makefile.objs\n> >> index 5b2d38d..a7c808b 100644\n> >> --- a/hw/arm/Makefile.objs\n> >> +++ b/hw/arm/Makefile.objs\n> >> @@ -19,4 +19,4 @@ obj-$(CONFIG_FSL_IMX31) += fsl-imx31.o kzm.o\n> >> obj-$(CONFIG_FSL_IMX6) += fsl-imx6.o sabrelite.o\n> >> obj-$(CONFIG_ASPEED_SOC) += aspeed_soc.o aspeed.o\n> >> obj-$(CONFIG_MPS2) += mps2.o\n> >> -obj-$(CONFIG_ARM_SMMUV3) += smmu-common.o\n> >> +obj-$(CONFIG_ARM_SMMUV3) += smmu-common.o smmuv3.o\n> >> diff --git a/hw/arm/smmuv3-internal.h b/hw/arm/smmuv3-internal.h\n> >> new file mode 100644\n> >> index 0000000..488acc8\n> >> --- /dev/null\n> >> +++ b/hw/arm/smmuv3-internal.h\n> >> @@ -0,0 +1,201 @@\n> >> +/*\n> >> + * ARM SMMUv3 support - Internal API\n> >> + *\n> >> + * Copyright (C) 2014-2016 Broadcom Corporation\n> >> + * Copyright (c) 2017 Red Hat, Inc.\n> >> + * Written by Prem Mallappa, Eric Auger\n> >> + *\n> >> + * This program is free software; you can redistribute it and/or modify\n> >> + * it under the terms of the GNU General Public License as published by\n> >> + * the Free Software Foundation, either version 2 of the License, or\n> >> + * (at your option) any later version.\n> >> + *\n> >> + * This program is distributed in the hope that it will be useful,\n> >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n> >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n> >> + * GNU General Public License for more details.\n> >> + *\n> >> + * You should have received a copy of the GNU General Public License along\n> >> + * with this program; if not, see .\n> >> + */\n> >> +\n> >> +#ifndef HW_ARM_SMMU_V3_INTERNAL_H\n> >> +#define HW_ARM_SMMU_V3_INTERNAL_H\n> >> +\n> >> +#include \"trace.h\"\n> >> +#include \"qemu/error-report.h\"\n> >> +#include \"hw/arm/smmu-common.h\"\n> >> +\n> >> +/*****************************\n> >> + * MMIO Register\n> >> + *****************************/\n> >> +enum {\n> >> + SMMU_REG_IDR0 = 0x0,\n> >> +\n> >> +/* IDR0 Field Values and supported features */\n> >> +\n> >> +#define SMMU_IDR0_S2P 1 /* stage 2 */\n> >> +#define SMMU_IDR0_S1P 1 /* stage 1 */\n> >> +#define SMMU_IDR0_TTF 2 /* Aarch64 only - not Aarch32 (LPAE) */\n> >> +#define SMMU_IDR0_COHACC 1 /* IO coherent access */\n> >> +#define SMMU_IDR0_HTTU 2 /* Access and Dirty flag update */\n> >> +#define SMMU_IDR0_HYP 0 /* Hypervisor Stage 1 contexts */\n> >> +#define SMMU_IDR0_ATS 0 /* PCIe RC ATS */\n> >> +#define SMMU_IDR0_ASID16 1 /* 16-bit ASID */\n> >> +#define SMMU_IDR0_PRI 0 /* Page Request Interface */\n> >> +#define SMMU_IDR0_VMID16 0 /* 16-bit VMID */\n> >> +#define SMMU_IDR0_CD2L 0 /* 2-level Context Descriptor table */\n> >> +#define SMMU_IDR0_STALL 1 /* Stalling fault model */\n> >> +#define SMMU_IDR0_TERM 1 /* Termination model behaviour */\n> >> +#define SMMU_IDR0_STLEVEL 1 /* Multi-level Stream Table */\n> >> +\n> >> +#define SMMU_IDR0_S2P_SHIFT 0\n> >> +#define SMMU_IDR0_S1P_SHIFT 1\n> >> +#define SMMU_IDR0_TTF_SHIFT 2\n> >> +#define SMMU_IDR0_COHACC_SHIFT 4\n> >> +#define SMMU_IDR0_HTTU_SHIFT 6\n> >> +#define SMMU_IDR0_HYP_SHIFT 9\n> >> +#define SMMU_IDR0_ATS_SHIFT 10\n> >> +#define SMMU_IDR0_ASID16_SHIFT 12\n> >> +#define SMMU_IDR0_PRI_SHIFT 16\n> >> +#define SMMU_IDR0_VMID16_SHIFT 18\n> >> +#define SMMU_IDR0_CD2L_SHIFT 19\n> >> +#define SMMU_IDR0_STALL_SHIFT 24\n> >> +#define SMMU_IDR0_TERM_SHIFT 26\n> >> +#define SMMU_IDR0_STLEVEL_SHIFT 27\n> >> +\n> >> + SMMU_REG_IDR1 = 0x4,\n> >> +#define SMMU_IDR1_SIDSIZE 16\n> >> + SMMU_REG_IDR2 = 0x8,\n> >> + SMMU_REG_IDR3 = 0xc,\n> >> + SMMU_REG_IDR4 = 0x10,\n> >> + SMMU_REG_IDR5 = 0x14,\n> >> +#define SMMU_IDR5_GRAN_SHIFT 4\n> >> +#define SMMU_IDR5_GRAN 0b101 /* GRAN4K, GRAN64K */\n> >> +#define SMMU_IDR5_OAS 4 /* 44 bits */\n> >> + SMMU_REG_IIDR = 0x1c,\n> >> + SMMU_REG_CR0 = 0x20,\n> >> +\n> >> +#define SMMU_CR0_SMMU_ENABLE (1 << 0)\n> >> +#define SMMU_CR0_PRIQ_ENABLE (1 << 1)\n> >> +#define SMMU_CR0_EVTQ_ENABLE (1 << 2)\n> >> +#define SMMU_CR0_CMDQ_ENABLE (1 << 3)\n> >> +#define SMMU_CR0_ATS_CHECK (1 << 4)\n> >> +\n> >> + SMMU_REG_CR0_ACK = 0x24,\n> >> + SMMU_REG_CR1 = 0x28,\n> >> + SMMU_REG_CR2 = 0x2c,\n> >> +\n> >> + SMMU_REG_STATUSR = 0x40,\n> >> +\n> >> + SMMU_REG_IRQ_CTRL = 0x50,\n> >> + SMMU_REG_IRQ_CTRL_ACK = 0x54,\n> >> +\n> >> +#define SMMU_IRQ_CTRL_GERROR_EN (1 << 0)\n> >> +#define SMMU_IRQ_CTRL_EVENT_EN (1 << 1)\n> >> +#define SMMU_IRQ_CTRL_PRI_EN (1 << 2)\n> >> +\n> >> + SMMU_REG_GERROR = 0x60,\n> >> +\n> >> +#define SMMU_GERROR_CMDQ (1 << 0)\n> >> +#define SMMU_GERROR_EVENTQ_ABT (1 << 2)\n> >> +#define SMMU_GERROR_PRIQ_ABT (1 << 3)\n> >> +#define SMMU_GERROR_MSI_CMDQ_ABT (1 << 4)\n> >> +#define SMMU_GERROR_MSI_EVENTQ_ABT (1 << 5)\n> >> +#define SMMU_GERROR_MSI_PRIQ_ABT (1 << 6)\n> >> +#define SMMU_GERROR_MSI_GERROR_ABT (1 << 7)\n> >> +#define SMMU_GERROR_SFM_ERR (1 << 8)\n> >> +\n> >> + SMMU_REG_GERRORN = 0x64,\n> >> + SMMU_REG_GERROR_IRQ_CFG0 = 0x68,\n> >> + SMMU_REG_GERROR_IRQ_CFG1 = 0x70,\n> >> + SMMU_REG_GERROR_IRQ_CFG2 = 0x74,\n> >> +\n> >> + /* SMMU_BASE_RA Applies to STRTAB_BASE, CMDQ_BASE and EVTQ_BASE */\n> >> +#define SMMU_BASE_RA (1ULL << 62)\n> >> + SMMU_REG_STRTAB_BASE = 0x80,\n> >> + SMMU_REG_STRTAB_BASE_CFG = 0x88,\n> >> +\n> >> + SMMU_REG_CMDQ_BASE = 0x90,\n> >> + SMMU_REG_CMDQ_PROD = 0x98,\n> >> + SMMU_REG_CMDQ_CONS = 0x9c,\n> >> + /* CMD Consumer (CONS) */\n> >> +#define SMMU_CMD_CONS_ERR_SHIFT 24\n> >> +#define SMMU_CMD_CONS_ERR_BITS 7\n> >> +\n> >> + SMMU_REG_EVTQ_BASE = 0xa0,\n> >> + SMMU_REG_EVTQ_PROD = 0xa8,\n> >> + SMMU_REG_EVTQ_CONS = 0xac,\n> >> + SMMU_REG_EVTQ_IRQ_CFG0 = 0xb0,\n> >> + SMMU_REG_EVTQ_IRQ_CFG1 = 0xb8,\n> >> + SMMU_REG_EVTQ_IRQ_CFG2 = 0xbc,\n> >> +\n> >> + SMMU_REG_PRIQ_BASE = 0xc0,\n> >> + SMMU_REG_PRIQ_PROD = 0xc8,\n> >> + SMMU_REG_PRIQ_CONS = 0xcc,\n> >> + SMMU_REG_PRIQ_IRQ_CFG0 = 0xd0,\n> >> + SMMU_REG_PRIQ_IRQ_CFG1 = 0xd8,\n> >> + SMMU_REG_PRIQ_IRQ_CFG2 = 0xdc,\n> >> +\n> >> + SMMU_ID_REGS_OFFSET = 0xfd0,\n> >> +\n> >> + /* Secure registers are not used for now */\n> >> + SMMU_SECURE_OFFSET = 0x8000,\n> >> +};\n> >> +\n> >> +/**********************\n> >> + * Data Structures\n> >> + **********************/\n> >> +\n> >> +struct __smmu_data2 {\n> >> + uint32_t word[2];\n> >> +};\n> >> +\n> >> +struct __smmu_data8 {\n> >> + uint32_t word[8];\n> >> +};\n> >> +\n> >> +struct __smmu_data16 {\n> >> + uint32_t word[16];\n> >> +};\n> >> +\n> >> +struct __smmu_data4 {\n> >> + uint32_t word[4];\n> >> +};\n> >> +\n> >> +typedef struct __smmu_data4 Cmd; /* Command Entry */\n> >> +typedef struct __smmu_data8 Evt; /* Event Entry */\n> >> +\n> >> +/*****************************\n> >> + * Register Access Primitives\n> >> + *****************************/\n> >> +\n> >> +static inline void smmu_write32_reg(SMMUV3State *s, uint32_t addr, uint32_t val)\n> >> +{\n> >> + s->regs[addr >> 2] = val;\n> >> +}\n> >> +\n> >> +static inline void smmu_write64_reg(SMMUV3State *s, uint32_t addr, uint64_t val)\n> >> +{\n> >> + addr >>= 2;\n> >> + s->regs[addr] = extract64(val, 0, 32);\n> >> + s->regs[addr + 1] = extract64(val, 32, 32);\n> >> +}\n> >> +\n> >> +static inline uint32_t smmu_read32_reg(SMMUV3State *s, uint32_t addr)\n> >> +{\n> >> + return s->regs[addr >> 2];\n> >> +}\n> >> +\n> >> +static inline uint64_t smmu_read64_reg(SMMUV3State *s, uint32_t addr)\n> >> +{\n> >> + addr >>= 2;\n> >> + return s->regs[addr] | ((uint64_t)(s->regs[addr + 1]) << 32);\n> >> +}\n> >> +\n> >> +static inline int smmu_enabled(SMMUV3State *s)\n> >> +{\n> >> + return smmu_read32_reg(s, SMMU_REG_CR0) & SMMU_CR0_SMMU_ENABLE;\n> >> +}\n> >> +\n> >> +#endif\n> >> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n> >> new file mode 100644\n> >> index 0000000..0a7cd1c\n> >> --- /dev/null\n> >> +++ b/hw/arm/smmuv3.c\n> >> @@ -0,0 +1,239 @@\n> >> +/*\n> >> + * Copyright (C) 2014-2016 Broadcom Corporation\n> >> + * Copyright (c) 2017 Red Hat, Inc.\n> >> + * Written by Prem Mallappa, Eric Auger\n> >> + *\n> >> + * This program is free software; you can redistribute it and/or modify\n> >> + * it under the terms of the GNU General Public License as published by\n> >> + * the Free Software Foundation, either version 2 of the License, or\n> >> + * (at your option) any later version.\n> >> + *\n> >> + * This program is distributed in the hope that it will be useful,\n> >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n> >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n> >> + * GNU General Public License for more details.\n> >> + *\n> >> + * You should have received a copy of the GNU General Public License along\n> >> + * with this program; if not, see .\n> >> + */\n> >> +\n> >> +#include \"qemu/osdep.h\"\n> >> +#include \"hw/boards.h\"\n> >> +#include \"sysemu/sysemu.h\"\n> >> +#include \"hw/sysbus.h\"\n> >> +#include \"hw/pci/pci.h\"\n> >> +#include \"exec/address-spaces.h\"\n> >> +#include \"trace.h\"\n> >> +#include \"qemu/error-report.h\"\n> >> +\n> >> +#include \"hw/arm/smmuv3.h\"\n> >> +#include \"smmuv3-internal.h\"\n> >> +\n> >> +static void smmuv3_init_regs(SMMUV3State *s)\n> >> +{\n> >> + uint32_t data =\n> >> + SMMU_IDR0_STLEVEL << SMMU_IDR0_STLEVEL_SHIFT |\n> >> + SMMU_IDR0_TERM << SMMU_IDR0_TERM_SHIFT |\n> >> + SMMU_IDR0_STALL << SMMU_IDR0_STALL_SHIFT |\n> >> + SMMU_IDR0_VMID16 << SMMU_IDR0_VMID16_SHIFT |\n> >> + SMMU_IDR0_PRI << SMMU_IDR0_PRI_SHIFT |\n> >> + SMMU_IDR0_ASID16 << SMMU_IDR0_ASID16_SHIFT |\n> >> + SMMU_IDR0_ATS << SMMU_IDR0_ATS_SHIFT |\n> >> + SMMU_IDR0_HYP << SMMU_IDR0_HYP_SHIFT |\n> >> + SMMU_IDR0_HTTU << SMMU_IDR0_HTTU_SHIFT |\n> >> + SMMU_IDR0_COHACC << SMMU_IDR0_COHACC_SHIFT |\n> >> + SMMU_IDR0_TTF << SMMU_IDR0_TTF_SHIFT |\n> >> + SMMU_IDR0_S1P << SMMU_IDR0_S1P_SHIFT |\n> >> + SMMU_IDR0_S2P << SMMU_IDR0_S2P_SHIFT;\n> >> +\n> >> + smmu_write32_reg(s, SMMU_REG_IDR0, data);\n> >> +\n> >> +#define SMMU_QUEUE_SIZE_LOG2 19\n> >> + data =\n> >> + 1 << 27 | /* Attr Types override */\n> >> + SMMU_QUEUE_SIZE_LOG2 << 21 | /* Cmd Q size */\n> >> + SMMU_QUEUE_SIZE_LOG2 << 16 | /* Event Q size */\n> >> + SMMU_QUEUE_SIZE_LOG2 << 11 | /* PRI Q size */\n> >> + 0 << 6 | /* SSID not supported */\n> >> + SMMU_IDR1_SIDSIZE;\n> >> +\n> >> + smmu_write32_reg(s, SMMU_REG_IDR1, data);\n> >> +\n> >> + s->sid_size = SMMU_IDR1_SIDSIZE;\n> >> +\n> >> + data = SMMU_IDR5_GRAN << SMMU_IDR5_GRAN_SHIFT | SMMU_IDR5_OAS;\n> > \n> > For VFIO case, should we not set the granule size based on underlying \n> > pagesize bitmap derived from VFIO_IOMMU_GET_INFO. Else if guest kernel\n> > is build with 4k page size and the host kernel is 64k we would start\n> > getting map errors. \n> \n> yes at the moment this is not implemented (1st target of the series is\n> virtio/vhost).\n> \n> On Intel if I understand correctly the minimum requested is 4K, 2MB.\n> 1GB is optional. I understand the emulated model does not expose 1GB\n> (FL1GP = 0).\n> \n> On ARM nothing is mandatory although 4K and 64K minimal granules are\n> \"strongly recommended\", leading to the following additional sizes.\n> \n> if (reg & IDR5_GRAN64K)\n> smmu->pgsize_bitmap |= SZ_64K | SZ_512M;\n> if (reg & IDR5_GRAN16K)\n> smmu->pgsize_bitmap |= SZ_16K | SZ_32M;\n> if (reg & IDR5_GRAN4K)\n> smmu->pgsize_bitmap |= SZ_4K | SZ_2M | SZ_1G;\n> \n> Maybe we can override the ID5 values using the vfio_memory_listener. I\n> will try to prototype this idea.\n\nOk. Thanks.\n\n\n> \n> Thanks\n> \n> Eric\n> \n> \n> > \n> > \n> > \n> >> +\n> >> + smmu_write32_reg(s, SMMU_REG_IDR5, data);\n> >> +}\n> >> +\n> >> +static void smmuv3_init_queues(SMMUV3State *s)\n> >> +{\n> >> + s->cmdq.prod = 0;\n> >> + s->cmdq.cons = 0;\n> >> + s->cmdq.wrap.prod = 0;\n> >> + s->cmdq.wrap.cons = 0;\n> >> +\n> >> + s->evtq.prod = 0;\n> >> + s->evtq.cons = 0;\n> >> + s->evtq.wrap.prod = 0;\n> >> + s->evtq.wrap.cons = 0;\n> >> +\n> >> + s->cmdq.entries = SMMU_QUEUE_SIZE_LOG2;\n> >> + s->cmdq.ent_size = sizeof(Cmd);\n> >> + s->evtq.entries = SMMU_QUEUE_SIZE_LOG2;\n> >> + s->evtq.ent_size = sizeof(Evt);\n> >> +}\n> >> +\n> >> +static void smmuv3_init(SMMUV3State *s)\n> >> +{\n> >> + smmuv3_init_regs(s);\n> >> + smmuv3_init_queues(s);\n> >> +}\n> >> +\n> >> +static inline void smmu_update_base_reg(SMMUV3State *s, uint64_t *base,\n> >> + uint64_t val)\n> >> +{\n> >> + *base = val & ~(SMMU_BASE_RA | 0x3fULL);\n> >> +}\n> >> +\n> >> +static void smmu_write_mmio_fixup(SMMUV3State *s, hwaddr *addr)\n> >> +{\n> >> + switch (*addr) {\n> >> + case 0x100a8: case 0x100ac: /* Aliasing => page0 registers */\n> >> + case 0x100c8: case 0x100cc:\n> >> + *addr ^= (hwaddr)0x10000;\n> >> + }\n> >> +}\n> >> +\n> >> +static void smmu_write_mmio(void *opaque, hwaddr addr,\n> >> + uint64_t val, unsigned size)\n> >> +{\n> >> +}\n> >> +\n> >> +static uint64_t smmu_read_mmio(void *opaque, hwaddr addr, unsigned size)\n> >> +{\n> >> + SMMUState *sys = opaque;\n> >> + SMMUV3State *s = SMMU_V3_DEV(sys);\n> >> + uint64_t val;\n> >> +\n> >> + smmu_write_mmio_fixup(s, &addr);\n> >> +\n> >> + /* Primecell/Corelink ID registers */\n> >> + switch (addr) {\n> >> + case 0xFF0 ... 0xFFC:\n> >> + case 0xFDC ... 0xFE4:\n> >> + val = 0;\n> >> + error_report(\"addr:0x%\"PRIx64\" val:0x%\"PRIx64, addr, val);\n> >> + break;\n> >> + case SMMU_REG_STRTAB_BASE ... SMMU_REG_CMDQ_BASE:\n> >> + case SMMU_REG_EVTQ_BASE:\n> >> + case SMMU_REG_PRIQ_BASE ... SMMU_REG_PRIQ_IRQ_CFG1:\n> >> + val = smmu_read64_reg(s, addr);\n> >> + break;\n> >> + default:\n> >> + val = (uint64_t)smmu_read32_reg(s, addr);\n> >> + break;\n> >> + }\n> >> +\n> >> + trace_smmuv3_read_mmio(addr, val, size);\n> >> + return val;\n> >> +}\n> >> +\n> >> +static const MemoryRegionOps smmu_mem_ops = {\n> >> + .read = smmu_read_mmio,\n> >> + .write = smmu_write_mmio,\n> >> + .endianness = DEVICE_LITTLE_ENDIAN,\n> >> + .valid = {\n> >> + .min_access_size = 4,\n> >> + .max_access_size = 8,\n> >> + },\n> >> + .impl = {\n> >> + .min_access_size = 4,\n> >> + .max_access_size = 8,\n> >> + },\n> >> +};\n> >> +\n> >> +static void smmu_init_irq(SMMUV3State *s, SysBusDevice *dev)\n> >> +{\n> >> + int i;\n> >> +\n> >> + for (i = 0; i < ARRAY_SIZE(s->irq); i++) {\n> >> + sysbus_init_irq(dev, &s->irq[i]);\n> >> + }\n> >> +}\n> >> +\n> >> +static void smmu_reset(DeviceState *dev)\n> >> +{\n> >> + SMMUV3State *s = SMMU_V3_DEV(dev);\n> >> + smmuv3_init(s);\n> >> +}\n> >> +\n> >> +static void smmu_realize(DeviceState *d, Error **errp)\n> >> +{\n> >> + SMMUState *sys = SMMU_SYS_DEV(d);\n> >> + SMMUV3State *s = SMMU_V3_DEV(sys);\n> >> + SysBusDevice *dev = SYS_BUS_DEVICE(d);\n> >> +\n> >> + memory_region_init_io(&sys->iomem, OBJECT(s),\n> >> + &smmu_mem_ops, sys, TYPE_SMMU_V3_DEV, 0x20000);\n> >> +\n> >> + sys->mrtypename = g_strdup(TYPE_SMMUV3_IOMMU_MEMORY_REGION);\n> >> +\n> >> + sysbus_init_mmio(dev, &sys->iomem);\n> >> +\n> >> + smmu_init_irq(s, dev);\n> >> +}\n> >> +\n> >> +static const VMStateDescription vmstate_smmuv3 = {\n> >> + .name = \"smmuv3\",\n> >> + .version_id = 1,\n> >> + .minimum_version_id = 1,\n> >> + .fields = (VMStateField[]) {\n> >> + VMSTATE_UINT32_ARRAY(regs, SMMUV3State, SMMU_NREGS),\n> >> + VMSTATE_END_OF_LIST(),\n> >> + },\n> >> +};\n> >> +\n> >> +static void smmuv3_instance_init(Object *obj)\n> >> +{\n> >> + /* Nothing much to do here as of now */\n> >> +}\n> >> +\n> >> +static void smmuv3_class_init(ObjectClass *klass, void *data)\n> >> +{\n> >> + DeviceClass *dc = DEVICE_CLASS(klass);\n> >> +\n> >> + dc->reset = smmu_reset;\n> >> + dc->vmsd = &vmstate_smmuv3;\n> >> + dc->realize = smmu_realize;\n> >> +}\n> >> +\n> >> +static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass,\n> >> + void *data)\n> >> +{\n> >> +}\n> >> +\n> >> +static const TypeInfo smmuv3_type_info = {\n> >> + .name = TYPE_SMMU_V3_DEV,\n> >> + .parent = TYPE_SMMU_DEV_BASE,\n> >> + .instance_size = sizeof(SMMUV3State),\n> >> + .instance_init = smmuv3_instance_init,\n> >> + .class_data = NULL,\n> >> + .class_size = sizeof(SMMUV3Class),\n> >> + .class_init = smmuv3_class_init,\n> >> +};\n> >> +\n> >> +static const TypeInfo smmuv3_iommu_memory_region_info = {\n> >> + .parent = TYPE_IOMMU_MEMORY_REGION,\n> >> + .name = TYPE_SMMUV3_IOMMU_MEMORY_REGION,\n> >> + .class_init = smmuv3_iommu_memory_region_class_init,\n> >> +};\n> >> +\n> >> +static void smmuv3_register_types(void)\n> >> +{\n> >> + type_register(&smmuv3_type_info);\n> >> + type_register(&smmuv3_iommu_memory_region_info);\n> >> +}\n> >> +\n> >> +type_init(smmuv3_register_types)\n> >> +\n> >> diff --git a/hw/arm/trace-events b/hw/arm/trace-events\n> >> index c67cd39..8affbf7 100644\n> >> --- a/hw/arm/trace-events\n> >> +++ b/hw/arm/trace-events\n> >> @@ -14,3 +14,6 @@ smmu_page_walk_level_block_pte(int stage, int level, uint64_t baseaddr, uint64_t\n> >> smmu_page_walk_level_table_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t address) \"stage=%d, level=%d base@=0x%\"PRIx64\" pte@=0x%\"PRIx64\" pte=0x%\"PRIx64\" next table address = 0x%\"PRIx64\n> >> smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte) \"baseaddr=0x%\"PRIx64\" index=0x%x, pteaddr=0x%\"PRIx64\", pte=0x%\"PRIx64\n> >> smmu_set_translated_address(hwaddr iova, hwaddr pa) \"iova = 0x%\"PRIx64\" -> pa = 0x%\"PRIx64\n> >> +\n> >> +#hw/arm/smmuv3.c\n> >> +smmuv3_read_mmio(hwaddr addr, uint64_t val, unsigned size) \"addr: 0x%\"PRIx64\" val:0x%\"PRIx64\" size: 0x%x\"\n> >> diff --git a/include/hw/arm/smmuv3.h b/include/hw/arm/smmuv3.h\n> >> new file mode 100644\n> >> index 0000000..0c8973d\n> >> --- /dev/null\n> >> +++ b/include/hw/arm/smmuv3.h\n> >> @@ -0,0 +1,79 @@\n> >> +/*\n> >> + * Copyright (C) 2014-2016 Broadcom Corporation\n> >> + * Copyright (c) 2017 Red Hat, Inc.\n> >> + * Written by Prem Mallappa, Eric Auger\n> >> + *\n> >> + * This program is free software; you can redistribute it and/or modify\n> >> + * it under the terms of the GNU General Public License as published by\n> >> + * the Free Software Foundation, either version 2 of the License, or\n> >> + * (at your option) any later version.\n> >> + *\n> >> + * This program is distributed in the hope that it will be useful,\n> >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n> >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n> >> + * GNU General Public License for more details.\n> >> + *\n> >> + * You should have received a copy of the GNU General Public License along\n> >> + * with this program; if not, see .\n> >> + */\n> >> +\n> >> +#ifndef HW_ARM_SMMUV3_H\n> >> +#define HW_ARM_SMMUV3_H\n> >> +\n> >> +#include \"hw/arm/smmu-common.h\"\n> >> +\n> >> +#define TYPE_SMMUV3_IOMMU_MEMORY_REGION \"smmuv3-iommu-memory-region\"\n> >> +\n> >> +#define SMMU_NREGS 0x200\n> >> +\n> >> +typedef struct SMMUQueue {\n> >> + hwaddr base;\n> >> + uint32_t prod;\n> >> + uint32_t cons;\n> >> + union {\n> >> + struct {\n> >> + uint8_t prod:1;\n> >> + uint8_t cons:1;\n> >> + };\n> >> + uint8_t unused;\n> >> + } wrap;\n> >> +\n> >> + uint16_t entries; /* Number of entries */\n> >> + uint8_t ent_size; /* Size of entry in bytes */\n> >> + uint8_t shift; /* Size in log2 */\n> >> +} SMMUQueue;\n> >> +\n> >> +typedef struct SMMUV3State {\n> >> + SMMUState smmu_state;\n> >> +\n> >> + /* Local cache of most-frequently used registers */\n> >> +#define SMMU_FEATURE_2LVL_STE (1 << 0)\n> >> + uint32_t features;\n> >> + uint16_t sid_size;\n> >> + uint16_t sid_split;\n> >> + uint64_t strtab_base;\n> >> +\n> >> + uint32_t regs[SMMU_NREGS];\n> >> +\n> >> + qemu_irq irq[4];\n> >> + SMMUQueue cmdq, evtq;\n> >> +\n> >> +} SMMUV3State;\n> >> +\n> >> +typedef enum {\n> >> + SMMU_IRQ_EVTQ,\n> >> + SMMU_IRQ_PRIQ,\n> >> + SMMU_IRQ_CMD_SYNC,\n> >> + SMMU_IRQ_GERROR,\n> >> +} SMMUIrq;\n> >> +\n> >> +typedef struct {\n> >> + SMMUBaseClass smmu_base_class;\n> >> +} SMMUV3Class;\n> >> +\n> >> +#define TYPE_SMMU_V3_DEV \"smmuv3\"\n> >> +#define SMMU_V3_DEV(obj) OBJECT_CHECK(SMMUV3State, (obj), TYPE_SMMU_V3_DEV)\n> >> +#define SMMU_V3_DEVICE_GET_CLASS(obj) \\\n> >> + OBJECT_GET_CLASS(SMMUBaseClass, (obj), TYPE_SMMU_V3_DEV)\n> >> +\n> >> +#endif\n> >> -- \n> >> 2.5.5\n> >>\n> >>\n> >","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=CAVIUMNETWORKS.onmicrosoft.com\n\theader.i=@CAVIUMNETWORKS.onmicrosoft.com header.b=\"Y15Kyvr6\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=Linu.Cherian@cavium.com; "],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xrvg22tslz9s76\n\tfor ;\n\tTue, 12 Sep 2017 16:15:41 +1000 (AEST)","from localhost ([::1]:33829 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dreTy-0005pw-RO\n\tfor incoming@patchwork.ozlabs.org; Tue, 12 Sep 2017 02:15:38 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:54567)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dreTY-0005mX-Ot\n\tfor qemu-devel@nongnu.org; Tue, 12 Sep 2017 02:15:18 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dreTV-0000Kv-Nu\n\tfor qemu-devel@nongnu.org; Tue, 12 Sep 2017 02:15:12 -0400","from mail-by2nam01on0081.outbound.protection.outlook.com\n\t([104.47.34.81]:31008\n\thelo=NAM01-BY2-obe.outbound.protection.outlook.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dreTL-0000GR-ME; Tue, 12 Sep 2017 02:15:00 -0400","from localhost (111.93.218.67) by\n\tBN6PR07MB3602.namprd07.prod.outlook.com (10.161.153.157) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id\n\t15.20.35.12; Tue, 12 Sep 2017 06:14:47 +0000","by localhost (sSMTP sendmail emulation);\n\tTue, 12 Sep 2017 11:44:21 +0530"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=0eDOgTxsuYRnGMDIPPzve83M07w7fM3gs/wnm2aH3PI=;\n\tb=Y15Kyvr6a1wpwn4IGD9GP+XKNJOvF+F5kWnv8iz0YR1vUwEpLEa80XF4HPkq9deQU2GQ/msos47jzey+5vHb8eMLm6GUNsBx3iM9LQykFd9OLWyBd2pCpaLlPeWXWfiu3LRKNWnS1cUTipBX77mpH+JtW7Kqu1arLOOsfl0HCm0=","Date":"Tue, 12 Sep 2017 11:44:21 +0530","From":"Linu Cherian ","To":"Auger Eric ","Message-ID":"<20170912061421.GA23102@virtx40>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-6-git-send-email-eric.auger@redhat.com>\n\t<20170908105214.GA16412@virtx40>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.5.21 (2010-09-15)","X-Originating-IP":"[111.93.218.67]","X-ClientProxiedBy":"PN1PR01CA0102.INDPRD01.PROD.OUTLOOK.COM (10.174.144.18) To\n\tBN6PR07MB3602.namprd07.prod.outlook.com (10.161.153.157)","X-MS-PublicTrafficType":"Email","X-MS-Office365-Filtering-Correlation-Id":"25f057a3-fe40-430d-7cbe-08d4f9a5951a","X-Microsoft-Antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:BN6PR07MB3602; ","X-Microsoft-Exchange-Diagnostics":["1; BN6PR07MB3602;\n\t3:xofgyi9u/tmP2wxpC5WPWPg4mpntrK1WAHxD8zw3PnrI+x+P0yT2dxwVgwfCOJsvfbpcOdCvih3plnF0+PFWRXkxSzKRc5DWbCgiix1tMYm8kiYUAgP/BAFceItPaTmQePGT2QhsbDg4+h1Pf+xrkJaEY8mINnmjnPSZo+rMnd0dZ9XWce3g/6piIT3KL3uKiK9az7DRJBPWUk4Wq5fKn0tFmuTQfAzz6/ko5JiHb5lx5I16M7d+CkZXsRZ34jQP;\n\t25:8gkxaFRfUWS1nyfOMRNHr0f913LmNaG0TC9daQkEm5Ca5mTU6cqcp4BpgnjysHFJQ5NL6GIjTHxW3h0mHNIhkQ70DvIpbl9F0dv1ufSkt9BNetNK7TW8DB5aMg8A+vrZdQhdP4U7fPmUo2CXlpvbAHSy32SvoK7NpczeGnU6JZoNIoLaL7cJKyo5lfRRnkIt4m/MDDrTNJfIPbFlimyOCwF2IrWJanC1IkedyUJXBBCIgrbII7F40cM8pSP8vl4EcfqA8Oeo5EpXpfsHkka2dZn0SGO3VDlyTOGHrTpNoork/naBvxx+fkL2C1bCDaQV98GHp9CpgqaDImOGXJDcew==;\n\t31:2wEJ72SD+kln0JaFWqrEyo44LncvkrWs+1bnoJjI2hM/VCGhzLzGNqJoRP1zEY6c8His0fylBwHsVOC2Xr8rg9RbzNHMqCvTYy6TEsDeFLYQcyd5QUYbGBqpxccvuMvdBEUj1Ceot533B5zomRACNCqEt0DZ+5NTHMpqCxF8uxj/CLlzvGDaatJRTKwYSBmnCg6pvGHmXbPnUl1b1T54hya/mNq3Cv98qTrartB17BY=","1; BN6PR07MB3602;\n\t20:rpvYPL3N6TW2wwVTU2YCDyFWX+KicBcWD6DpCgbaqkKgXexg1eLNbNpFsqAKsKOgXBTyTt3E/ZiWIFmy2PdVKZGNUcj9Zv9Wub5n1SWUaA9sMxbAVSx4rMU1crwzM+8K5Piw84IOL6I1qMizmeKm9+8BPmQ9DOTZzv1M5yAvXq0KSbryGPhM4ACGx2p6hd58IREbpzCfjfVn05APvMinfZLj1oUu+6L4lHplB2+XGylezkEkO+/SN+hPiRuPhzDLzCtthNqApRX9UolUEoB0wE7wAd7s5ayz7A8imqpRxQIpeC55ImC5LU0Mm+T2BGuy+rD5ziHnhvvZL6rK2YuTGKODDqoK21d21J0GGwfA5/igN1lV0NtpnDn5kShr1ekW2rDqWnX3XS4W94zxUpUzQKw5sqYlYsNnfcKOQCfVQd2b6LxPYXxgp7XTNTNxZ1IZ1r99k86slwNGST8HNSuPGAEFgWFOmXD3QlheTVah5RpGjeCdkHCdcRctVBW+FdkI;\n\t4:X81k3KYowZNmPrDs61WzLh5I1+VJlYES/B45zMiK+V+rUzfEJ5bjr8/z35LgYGRGcxfNj7l36mXAJ1dUTx8f3GSahgXdJUdSSAULLp9e9vy8M8AHQTHn15k1qeUvnpWrXIt+8H9S0Bp8nVUkTWD9ZCTUBdtFyfMMXAWd0emWZR8kwXZ+THMfO3uSJSQT7TegNPQxDl5H7vDKJwfjgIAk4XVD8aaej8EEt7k+0NbGY4KVU5fbK9MmO8a4oixz0LSstr7wHmZ1vYbmUDYrCZsW4IKIbnGLQ6FpdRQeRJTTUvXljE8FnCmKY9Ox0PsgW5njMyR96YHaKcen1wsUlLcpTg==","=?us-ascii?Q?1; BN6PR07MB3602;\n\t23:a54EltgSNDGmwpAM71mRZ7B/KZ6Nxytn9ejFm1AOo?=\n\tNzGixYob5Rn7m/1/5A9GbUlsLC89G8ZeJYhWGnx6sFazajKaL9eBB1hF8/VPq7srATq6hjy0+i/XwltRjWZsT2fPYFbCuZJaoTJUguIeG72kImWIvYX1N2p3QD0lXjlBN9YuF1jAZG8wzpo8THvi1igqNWlBtLg+Y9+BkbI0C851Noh7SNTG6Jw7UamAofW5QvOzcjSl1bEI2K+uJObgPIYgEalFB6xacf+N9jt0LT9UrkLBMxcqz6/wAwH4mNxJtXC3kxuHn00ViAKmfe0irkgyjnVSnEnNdb479ynXJc79W6NYnxiXYHovDV7bELrqjU55//ZqEbl745nPZ2nZUtCymBIHs3nqBkKqnb9UuIXAZ8UsSIKfOvp8a/SYM+qAGv4H0VqAWecQp2inlht+UrnYCMWJWPFimFW3sLuNlNV/gxtMUQc1MZRccSlixDJBm+UYqydUwZbrhEFI/uhlPas0O0FGUVvVoRoALH92NoSs4c+nOTQUkEOwGYbY+3F4CprHFPU9ozSBQvYd+rsoopKWAaBOeJbPOMQ8h9hJhU6DyF4KHcgrE/LDZCJ4zSHGggRwgqn4pMiFVAa6Lehc9bjNKJEKVwS/jswfUI4ZF2LbNpGcMf0xKi9LmOzGdnU+IfnKXfM9zyShVOi6XvkPtvibkL95jRqXf1+C6iBZbsEIlXGGCpq5mEZ8Ub7Uxhr0Yq/02Kn7Z4Pfe0rItn/tnO4QLyFgPynr3LlfbnyQvc18E6r54RKqRXxV1zPBfFHreDcWsN4JtVYr7tvDPOKDklEUg18SOt2r6skeeZLGKgwla//tywY+flCJShETL5APNlQ7oVjmA5NQAZRQS5gnFEg/S2NM8mp1z1qIf1VSycCpTl2AA0Zdv0HTB7cgSq7uVknTJiQYbD8FSzJ+i3mSjsoyWD7JMiOnFerGRD7T3f5N0/61MhTj1fji4N4mchhfAZDa5lRS35lyQEj4tgWehJ2PtHAqb1B+ygSvBd+BlINop4UwCkYMHv9kyAQnf62vH97rX1/TpX5dEidieItQKJNjWxHDwnH5DO28ukBm9vzjR/617aQ7/MjlKZNrzvLthH7qOizT/vANUUFobxSE6UBpjCs/KcQ8stkhE3mKC2woqXUjDu31j31OhMIy2qM8TbOD1T5Wb1qZHEaza176SujI4EqiR+U9+1kCQrn9xF9JelG8KTqExb5mLX5WuddTdOicbspg7q1SC6iHdscn06AfJp99aFDtWfsVgMHfc+QtA==","1; BN6PR07MB3602;\n\t6:nijFNGKSCsQlRcoeJPMCrGShRxKRtmavrLpF//QHXkOjQfRAgXdwDx53RWptikaPl4A9FX2BSCsDnFTa3VKhlHD68qGmJX6QEHS94keGZGAxjOQiXBv0Ui1Y6NQ198N+e7XgCdXFMl3hBjpOg631UTKgWBVo897Iw84qDceoxJWB7mVr7yzZpw9mDSVyTM+9OlE4p7KMVcF3b33w0uLFgSEqA8Qd9Zho3Nym/BHa1vjjO+Q/nHBnF3yc4MEB4ovx2tWWfw/QfIe8v27vuA52+DzL34MARJImkwh+SY3dYP3ZwFOh3E61fXutXqKJ/eVY12pSAyupO6LXnZwkz/5gYg==;\n\t5:36RyteLl8NSAcbp69YzwbIoTsdFOcV1CYE5n5ppsx08QGMFkR6Z1L+6BgW2SmOYe3ZyuIyMifxfFIM4nWnvZKB3NMJlXVAfo9ad5kAFcrJljyVOIHcK511hRXEYq6tWByKLDXdkK/RI/SceoB7jvEw==;\n\t24:W37Ng0O60E5ldb4GCr2HolGeyofjiMkUIQWX6UMZzL7rNZXVUmpnrT6U2T6GTHU+ID8HZu9I138+mabsCLDDmuS+O9s03J9nkwHkhfpBZBA=;\n\t7:VpsWC6TE2choWbBG3ohk5xS1Q3zGXAEsh9xifl7HwM0cFx++xGtMjNhGftctk4hO+mOy7n/Q5ghuOaMdHGS8VI/3eNdA794d46sVcDNPOalmPlXdO2SHAKXbL+FIWX82xb68S7OgevUFmv+AR/Qfh3rQmeyvUD5zBc3br/Ie8uL5Vz8XYbcJqYKm86NRY5coj4lsDi3egmyHlFTn67oyrmFNJnfNARk6qJp0wbXNBGk="],"X-MS-TrafficTypeDiagnostic":"BN6PR07MB3602:","X-Exchange-Antispam-Report-Test":"UriScan:(250305191791016)(22074186197030);","X-Microsoft-Antispam-PRVS":"","X-Exchange-Antispam-Report-CFA-Test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(10201501046)(100000703101)(100105400095)(93006095)(93001095)(3002001)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123562025)(20161123558100)(20161123555025)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:BN6PR07MB3602; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:BN6PR07MB3602; ","X-Forefront-PRVS":"042857DBB5","X-Forefront-Antispam-Report":"SFV:NSPM;\n\tSFS:(10009020)(6069001)(6009001)(199003)(189002)(24454002)(6306002)(23726003)(93886005)(6116002)(3846002)(66066001)(4326008)(305945005)(68736007)(33716001)(53936002)(478600001)(1076002)(9686003)(8656003)(7416002)(7736002)(105586002)(316002)(53546010)(42186005)(72206003)(122856001)(83506001)(8936002)(86362001)(97736004)(106356001)(101416001)(6916009)(2950100002)(8676002)(5660300001)(39060400002)(110136004)(6246003)(47776003)(50466002)(4001350100001)(189998001)(229853002)(2906002)(33656002)(76176999)(81166006)(81156014)(50986999)(54356999)(18370500001)(2004002);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR07MB3602; H:localhost; FPR:;\n\tSPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; ","Received-SPF":"None (protection.outlook.com: cavium.com does not designate\n\tpermitted sender hosts)","SpamDiagnosticOutput":"1:99","SpamDiagnosticMetadata":"NSPM","X-OriginatorOrg":"cavium.com","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"12 Sep 2017 06:14:47.5355\n\t(UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"711e4ccf-2e9b-4bcf-a551-4094005b6194","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"BN6PR07MB3602","X-detected-operating-system":"by eggs.gnu.org: Windows 7 or 8 [fuzzy]","X-Received-From":"104.47.34.81","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 05/20] hw/arm/smmuv3: Skeleton","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tprem.mallappa@gmail.com, robin.murphy@arm.com,\n\tLinu Cherian , eric.auger.pro@gmail.com,\n\tbharat.bhushan@nxp.com, christoffer.dall@linaro.org, wtownsen@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1766700,"web_url":"http://patchwork.ozlabs.org/comment/1766700/","msgid":"<20170912061815.GA23124@virtx40>","list_archive_url":null,"date":"2017-09-12T06:18:15","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 00/20] ARM SMMUv3 Emulation\n\tSupport","submitter":{"id":72195,"url":"http://patchwork.ozlabs.org/api/people/72195/","name":"Linu Cherian","email":"linuc.decode@gmail.com"},"content":"Hi Eric,\n\nOn Fri Sep 01, 2017 at 07:21:03PM +0200, Eric Auger wrote:\n> This series implements the emulation code for ARM SMMUv3.\n> \n> Changes since v6:\n> - DPDK testpmd now running on guest with 2 assigned VFs\n> - Changed the instantiation method: add the following option to\n> the QEMU command line\n> -device smmuv3 # for virtio/vhost use cases\n> -device smmuv3,caching-mode # for vfio use cases (based on [1])\n> - splitted the series into smaller patches to allow the review\n> - the VFIO integration based on \"tlbi-on-map\" smmuv3 driver\n> is isolated from the rest: last 2 patches, not for upstream.\n> This is shipped for testing/bench until a better solution is found.\n> - Reworked permission flag checks and event generation\n> \n> testing:\n> - in dt and ACPI modes\n> - virtio-net-pci and vhost-net devices using dma ops with various\n> guest page sizes [2]\n> - assigned VFs using dma ops [3]:\n> - AMD Overdrive and igbvf passthrough (using gsi direct mapping)\n> - Cavium ThunderX and ixgbevf passthrough (using KVM MSI routing)\n> - DPDK testpmd on guest running with VFIO user space drivers (2 igbvf) [3]\n> with guest and host page size equal (4kB)\n> \n> Known limitations:\n> - no VMSAv8-32 suport\n> - no nested stage support (S1 + S2)\n> - no support for HYP mappings\n> - register fine emulation, commands, interrupts and errors were\n> not accurately tested. Handling is sufficient to run use cases\n> described above though.\n> - interrupts and event generation not observed yet.\n> \n\nBy design, shouldnt this work on hardware with smmuv2 implementations as well. \nie. Guest with smmuv3 emulation + Host with smmuv2 hardware.\n\nOr Is there any known limitations for this ?\n\n> Best Regards\n> \n> Eric\n> \n> This series can be found at:\n> v7: https://github.com/eauger/qemu/tree/v2.10.0-SMMU-v7\n> Previous version at:\n> v6: https://github.com/eauger/qemu/tree/v2.10.0-rc2-SMMU-v6\n> \n> References:\n> [1] [RFC v2 0/4] arm-smmu-v3 tlbi-on-map option\n> https://lkml.org/lkml/2017/8/11/426\n> \n> [2] qemu cmd line excerpt:\n> -device smmuv3 \\\n> -netdev tap,id=tap0,script=no,downscript=no,ifname=tap0,vhost=off \\\n> -device virtio-net-pci,netdev=tap0,mac=6a:f5:10:b1:3d:d2,iommu_platform,disable-modern=off,disable-legacy=on \\\n> [3] use -device smmuv3,caching-mode\n> \n> \n> History:\n> v6 -> v7:\n> - see above\n> \n> v5 -> v6:\n> - Rebase on 2.10 and IOMMUMemoryRegion\n> - add ACPI TLBI_ON_MAP support (VFIO integration also works in\n> ACPI mode)\n> - fix block replay\n> - handle implementation defined SMMU_CMD_TLBI_NH_VA_AM cmd\n> (goes along with TLBI_ON_MAP FW quirk)\n> - replay systematically unmap the whole range first\n> - smmuv3_map_hook does not unmap anymore and the unmap is done\n> before the replay\n> - add and use smmuv3_context_device_invalidate instead of\n> blindly replaying everything\n> \n> v4 -> v5:\n> - initial_level now part of SMMUTransCfg\n> - smmu_page_walk_64 takes into account the max input size\n> - implement sys->iommu_ops.replay and sys->iommu_ops.notify_flag_changed\n> - smmuv3_translate: bug fix: don't walk on bypass\n> - smmu_update_qreg: fix PROD index update\n> - I did not yet address Peter's comments as the code is not mature enough\n> to be split into sub patches.\n> \n> v3 -> v4 [Eric]:\n> - page table walk rewritten to allow scan of the page table within a\n> range of IOVA. This prepares for VFIO integration and replay.\n> - configuration parsing partially reworked.\n> - do not advertise unsupported/untested features: S2, S1 + S2, HYP,\n> PRI, ATS, ..\n> - added ACPI table generation\n> - migrated to dynamic traces\n> - mingw compilation fix\n> \n> v2 -> v3 [Eric]:\n> - rebased on 2.9\n> - mostly code and patch reorganization to ease the review process\n> - optional patches removed. They may be handled separately. I am currently\n> working on ACPI enablement.\n> - optional instantiation of the smmu in mach-virt\n> - removed [2/9] (fdt functions) since not mandated\n> - start splitting main patch into base and derived object\n> - no new function feature added\n> \n> v1 -> v2 [Prem]:\n> - Adopted review comments from Eric Auger\n> - Make SMMU_DPRINTF to internally call qemu_log\n> (since translation requests are too many, we need control\n> on the type of log we want)\n> - SMMUTransCfg modified to suite simplicity\n> - Change RegInfo to uint64 register array\n> - Code cleanup\n> - Test cleanups\n> - Reshuffled patches\n> \n> v0 -> v1 [Prem]:\n> - As per SMMUv3 spec 16.0 (only is_ste_consistant() is noticeable)\n> - Reworked register access/update logic\n> - Factored out translation code for\n> - single point bug fix\n> - sharing/removal in future\n> - (optional) Unit tests added, with PCI test device\n> - S1 with 4k/64k, S1+S2 with 4k/64k\n> - (S1 or S2) only can be verified by Linux 4.7 driver\n> - (optional) Priliminary ACPI support\n> \n> v0 [Prem]:\n> - Implements SMMUv3 spec 11.0\n> - Supported for PCIe devices,\n> - Command Queue and Event Queue supported\n> - LPAE only, S1 is supported and Tested, S2 not tested\n> - BE mode Translation not supported\n> - IRQ support (legacy, no MSI)\n> \n> Eric Auger (18):\n> hw/arm/smmu-common: smmu base device and datatypes\n> hw/arm/smmu-common: IOMMU memory region and address space setup\n> hw/arm/smmu-common: smmu_read/write_sysmem\n> hw/arm/smmu-common: VMSAv8-64 page table walk\n> hw/arm/smmuv3: Wired IRQ and GERROR helpers\n> hw/arm/smmuv3: Queue helpers\n> hw/arm/smmuv3: Implement MMIO write operations\n> hw/arm/smmuv3: Event queue recording helper\n> hw/arm/smmuv3: Implement translate callback\n> target/arm/kvm: Translate the MSI doorbell in kvm_arch_fixup_msi_route\n> hw/arm/smmuv3: Implement data structure and TLB invalidation\n> notifications\n> hw/arm/smmuv3: Implement IOMMU memory region replay callback\n> hw/arm/virt: Store the PCI host controller dt phandle\n> hw/arm/sysbus-fdt: Pass the VirtMachineState to the node creation\n> functions\n> hw/arm/sysbus-fdt: Pass the platform bus base address in\n> PlatformBusFDTData\n> hw/arm/sysbus-fdt: Allow smmuv3 dynamic instantiation\n> hw/arm/smmuv3: [not for upstream] add SMMU_CMD_TLBI_NH_VA_AM handling\n> hw/arm/smmuv3: [not for upstream] Add caching-mode option\n> \n> Prem Mallappa (2):\n> hw/arm/smmuv3: Skeleton\n> hw/arm/virt-acpi-build: Add smmuv3 node in IORT table\n> \n> default-configs/aarch64-softmmu.mak | 1 +\n> hw/arm/Makefile.objs | 1 +\n> hw/arm/smmu-common.c | 527 ++++++++++++++++\n> hw/arm/smmu-internal.h | 105 ++++\n> hw/arm/smmuv3-internal.h | 584 +++++++++++++++++\n> hw/arm/smmuv3.c | 1181 +++++++++++++++++++++++++++++++++++\n> hw/arm/sysbus-fdt.c | 129 +++-\n> hw/arm/trace-events | 48 ++\n> hw/arm/virt-acpi-build.c | 63 +-\n> hw/arm/virt.c | 6 +-\n> include/hw/acpi/acpi-defs.h | 15 +\n> include/hw/arm/smmu-common.h | 123 ++++\n> include/hw/arm/smmuv3.h | 80 +++\n> include/hw/arm/sysbus-fdt.h | 2 +\n> include/hw/arm/virt.h | 15 +\n> target/arm/kvm.c | 27 +\n> target/arm/trace-events | 3 +\n> 17 files changed, 2886 insertions(+), 24 deletions(-)\n> create mode 100644 hw/arm/smmu-common.c\n> create mode 100644 hw/arm/smmu-internal.h\n> create mode 100644 hw/arm/smmuv3-internal.h\n> create mode 100644 hw/arm/smmuv3.c\n> create mode 100644 include/hw/arm/smmu-common.h\n> create mode 100644 include/hw/arm/smmuv3.h\n> \n> -- \n> 2.5.5\n> \n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"D7Z+l9ia\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xrvl22YDsz9s7g\n\tfor ;\n\tTue, 12 Sep 2017 16:19:10 +1000 (AEST)","from localhost ([::1]:33836 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dreXM-0007IZ-Dk\n\tfor incoming@patchwork.ozlabs.org; Tue, 12 Sep 2017 02:19:08 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:55291)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dreWy-0007Gv-Pb\n\tfor qemu-devel@nongnu.org; Tue, 12 Sep 2017 02:18:46 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dreWw-0001VY-Ub\n\tfor qemu-devel@nongnu.org; Tue, 12 Sep 2017 02:18:44 -0400","from mail-pg0-x241.google.com ([2607:f8b0:400e:c05::241]:34618)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1dreWq-0001Rx-8t; Tue, 12 Sep 2017 02:18:36 -0400","by mail-pg0-x241.google.com with SMTP id v82so5734603pgb.1;\n\tMon, 11 Sep 2017 23:18:34 -0700 (PDT)","from virtx40 ([111.93.218.67]) by smtp.gmail.com with ESMTPSA id\n\ts76sm9867102pfj.119.2017.09.11.23.18.27\n\t(version=TLS1_2 cipher=AES128-SHA bits=128/128);\n\tMon, 11 Sep 2017 23:18:32 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=XXWhnC32K/VuDNj2Jh2rsIVaXhmqg8pgcwCEsaIfHnA=;\n\tb=D7Z+l9iancM8uKR/IHANdWfSbder4NMvbXMAfMkuBCFQMSQYU/M+GWeyfeYb7gpl4M\n\t7rjQ2vhKHrsFUSSXGB3+NBUP9le0tHOIlsBfY9K40cJyIyslR/iM7the2eYtCZzI2wyX\n\twk6q7So1CI6NXh50iL0aiT4rrGJ86EqAr9aQhXv+hTGtz+ulAs/KnUrrRg2r/F0jjCOd\n\t1Mq+jNMgE1NJqdKuxsoNI8WmqJ+pcMLWbD3aldxIQVs/SuRl7hVhR6ZWe8Bgwn3danwO\n\tk0Nv7Cjc6DCd4E9JpoW+Ge7QtCCI/USSuC1xM2KyCinuXBr7XuMPfNMLpB6GpNn17b2J\n\t9N4w==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=XXWhnC32K/VuDNj2Jh2rsIVaXhmqg8pgcwCEsaIfHnA=;\n\tb=p2CmhE4FdAXszy7o1uHahpZc961y/AEbmQ5yJuN0llT+GTVmh7RRu1NVoabYmmR7Uz\n\tKBBiSnENdvxVQFLYdYYHxCb59fORa17o03iGvV/3CGG61u18S4EpCgseAnBqlobmZO9q\n\tILbx/ywL/s5pUK3NJEdjA1+V+3rwBiGY0xdRLyoL2gVD+RVqXeSTfYokKvQnVaD7/n/W\n\t4UuX1VXb0ZNneDSAJzzLRzwG8FpnvXk4T2i6dSsXqAlAIm9v7zuIQxLd6Q3H+wBGNW0L\n\tL/VWrsJwlzWTMZVVUp9K/NkCgznNn9zKkxOoZ/jYLMQ9tS8th3r3RLR0BhCLu3oEUypo\n\tU76A==","X-Gm-Message-State":"AHPjjUj7dH9W9TiqmHkeWJWgm7S77o/7K1ZJNdboqX8LQeztGRD4O6Oa\n\tF8SP8oMpGVVGEw==","X-Google-Smtp-Source":"ADKCNb4VdQ7jXdGalaL2ipnmHP1Zl6T80vKZqZKDOn6pf8eHyDLliRz1kra+flOxg+KUng3ZNavnvg==","X-Received":"by 10.84.210.73 with SMTP id z67mr16317096plh.306.1505197113791; \n\tMon, 11 Sep 2017 23:18:33 -0700 (PDT)","Date":"Tue, 12 Sep 2017 11:48:15 +0530","From":"Linu Cherian ","To":"Eric Auger ","Message-ID":"<20170912061815.GA23124@virtx40>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>","User-Agent":"Mutt/1.5.21 (2010-09-15)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:400e:c05::241","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 00/20] ARM SMMUv3 Emulation\n\tSupport","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, bharat.bhushan@nxp.com, mst@redhat.com,\n\twill.deacon@arm.com, qemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, linu.cherian@cavium.com,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1766706,"web_url":"http://patchwork.ozlabs.org/comment/1766706/","msgid":"","list_archive_url":null,"date":"2017-09-12T06:38:47","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 00/20] ARM SMMUv3 Emulation\n\tSupport","submitter":{"id":69187,"url":"http://patchwork.ozlabs.org/api/people/69187/","name":"Eric Auger","email":"eric.auger@redhat.com"},"content":"Hi Linu,\n\nOn 12/09/2017 08:18, Linu Cherian wrote:\n> Hi Eric,\n> \n> On Fri Sep 01, 2017 at 07:21:03PM +0200, Eric Auger wrote:\n>> This series implements the emulation code for ARM SMMUv3.\n>>\n>> Changes since v6:\n>> - DPDK testpmd now running on guest with 2 assigned VFs\n>> - Changed the instantiation method: add the following option to\n>> the QEMU command line\n>> -device smmuv3 # for virtio/vhost use cases\n>> -device smmuv3,caching-mode # for vfio use cases (based on [1])\n>> - splitted the series into smaller patches to allow the review\n>> - the VFIO integration based on \"tlbi-on-map\" smmuv3 driver\n>> is isolated from the rest: last 2 patches, not for upstream.\n>> This is shipped for testing/bench until a better solution is found.\n>> - Reworked permission flag checks and event generation\n>>\n>> testing:\n>> - in dt and ACPI modes\n>> - virtio-net-pci and vhost-net devices using dma ops with various\n>> guest page sizes [2]\n>> - assigned VFs using dma ops [3]:\n>> - AMD Overdrive and igbvf passthrough (using gsi direct mapping)\n>> - Cavium ThunderX and ixgbevf passthrough (using KVM MSI routing)\n>> - DPDK testpmd on guest running with VFIO user space drivers (2 igbvf) [3]\n>> with guest and host page size equal (4kB)\n>>\n>> Known limitations:\n>> - no VMSAv8-32 suport\n>> - no nested stage support (S1 + S2)\n>> - no support for HYP mappings\n>> - register fine emulation, commands, interrupts and errors were\n>> not accurately tested. Handling is sufficient to run use cases\n>> described above though.\n>> - interrupts and event generation not observed yet.\n>>\n> \n> By design, shouldnt this work on hardware with smmuv2 implementations as well. \n> ie. Guest with smmuv3 emulation + Host with smmuv2 hardware.\n\nYes indeed. I am mostly testing with a host featuring smmuv2 at the moment.\n\nThanks\n\nEric\n> \n> Or Is there any known limitations for this ?\n> \n>> Best Regards\n>>\n>> Eric\n>>\n>> This series can be found at:\n>> v7: https://github.com/eauger/qemu/tree/v2.10.0-SMMU-v7\n>> Previous version at:\n>> v6: https://github.com/eauger/qemu/tree/v2.10.0-rc2-SMMU-v6\n>>\n>> References:\n>> [1] [RFC v2 0/4] arm-smmu-v3 tlbi-on-map option\n>> https://lkml.org/lkml/2017/8/11/426\n>>\n>> [2] qemu cmd line excerpt:\n>> -device smmuv3 \\\n>> -netdev tap,id=tap0,script=no,downscript=no,ifname=tap0,vhost=off \\\n>> -device virtio-net-pci,netdev=tap0,mac=6a:f5:10:b1:3d:d2,iommu_platform,disable-modern=off,disable-legacy=on \\\n>> [3] use -device smmuv3,caching-mode\n>>\n>>\n>> History:\n>> v6 -> v7:\n>> - see above\n>>\n>> v5 -> v6:\n>> - Rebase on 2.10 and IOMMUMemoryRegion\n>> - add ACPI TLBI_ON_MAP support (VFIO integration also works in\n>> ACPI mode)\n>> - fix block replay\n>> - handle implementation defined SMMU_CMD_TLBI_NH_VA_AM cmd\n>> (goes along with TLBI_ON_MAP FW quirk)\n>> - replay systematically unmap the whole range first\n>> - smmuv3_map_hook does not unmap anymore and the unmap is done\n>> before the replay\n>> - add and use smmuv3_context_device_invalidate instead of\n>> blindly replaying everything\n>>\n>> v4 -> v5:\n>> - initial_level now part of SMMUTransCfg\n>> - smmu_page_walk_64 takes into account the max input size\n>> - implement sys->iommu_ops.replay and sys->iommu_ops.notify_flag_changed\n>> - smmuv3_translate: bug fix: don't walk on bypass\n>> - smmu_update_qreg: fix PROD index update\n>> - I did not yet address Peter's comments as the code is not mature enough\n>> to be split into sub patches.\n>>\n>> v3 -> v4 [Eric]:\n>> - page table walk rewritten to allow scan of the page table within a\n>> range of IOVA. This prepares for VFIO integration and replay.\n>> - configuration parsing partially reworked.\n>> - do not advertise unsupported/untested features: S2, S1 + S2, HYP,\n>> PRI, ATS, ..\n>> - added ACPI table generation\n>> - migrated to dynamic traces\n>> - mingw compilation fix\n>>\n>> v2 -> v3 [Eric]:\n>> - rebased on 2.9\n>> - mostly code and patch reorganization to ease the review process\n>> - optional patches removed. They may be handled separately. I am currently\n>> working on ACPI enablement.\n>> - optional instantiation of the smmu in mach-virt\n>> - removed [2/9] (fdt functions) since not mandated\n>> - start splitting main patch into base and derived object\n>> - no new function feature added\n>>\n>> v1 -> v2 [Prem]:\n>> - Adopted review comments from Eric Auger\n>> - Make SMMU_DPRINTF to internally call qemu_log\n>> (since translation requests are too many, we need control\n>> on the type of log we want)\n>> - SMMUTransCfg modified to suite simplicity\n>> - Change RegInfo to uint64 register array\n>> - Code cleanup\n>> - Test cleanups\n>> - Reshuffled patches\n>>\n>> v0 -> v1 [Prem]:\n>> - As per SMMUv3 spec 16.0 (only is_ste_consistant() is noticeable)\n>> - Reworked register access/update logic\n>> - Factored out translation code for\n>> - single point bug fix\n>> - sharing/removal in future\n>> - (optional) Unit tests added, with PCI test device\n>> - S1 with 4k/64k, S1+S2 with 4k/64k\n>> - (S1 or S2) only can be verified by Linux 4.7 driver\n>> - (optional) Priliminary ACPI support\n>>\n>> v0 [Prem]:\n>> - Implements SMMUv3 spec 11.0\n>> - Supported for PCIe devices,\n>> - Command Queue and Event Queue supported\n>> - LPAE only, S1 is supported and Tested, S2 not tested\n>> - BE mode Translation not supported\n>> - IRQ support (legacy, no MSI)\n>>\n>> Eric Auger (18):\n>> hw/arm/smmu-common: smmu base device and datatypes\n>> hw/arm/smmu-common: IOMMU memory region and address space setup\n>> hw/arm/smmu-common: smmu_read/write_sysmem\n>> hw/arm/smmu-common: VMSAv8-64 page table walk\n>> hw/arm/smmuv3: Wired IRQ and GERROR helpers\n>> hw/arm/smmuv3: Queue helpers\n>> hw/arm/smmuv3: Implement MMIO write operations\n>> hw/arm/smmuv3: Event queue recording helper\n>> hw/arm/smmuv3: Implement translate callback\n>> target/arm/kvm: Translate the MSI doorbell in kvm_arch_fixup_msi_route\n>> hw/arm/smmuv3: Implement data structure and TLB invalidation\n>> notifications\n>> hw/arm/smmuv3: Implement IOMMU memory region replay callback\n>> hw/arm/virt: Store the PCI host controller dt phandle\n>> hw/arm/sysbus-fdt: Pass the VirtMachineState to the node creation\n>> functions\n>> hw/arm/sysbus-fdt: Pass the platform bus base address in\n>> PlatformBusFDTData\n>> hw/arm/sysbus-fdt: Allow smmuv3 dynamic instantiation\n>> hw/arm/smmuv3: [not for upstream] add SMMU_CMD_TLBI_NH_VA_AM handling\n>> hw/arm/smmuv3: [not for upstream] Add caching-mode option\n>>\n>> Prem Mallappa (2):\n>> hw/arm/smmuv3: Skeleton\n>> hw/arm/virt-acpi-build: Add smmuv3 node in IORT table\n>>\n>> default-configs/aarch64-softmmu.mak | 1 +\n>> hw/arm/Makefile.objs | 1 +\n>> hw/arm/smmu-common.c | 527 ++++++++++++++++\n>> hw/arm/smmu-internal.h | 105 ++++\n>> hw/arm/smmuv3-internal.h | 584 +++++++++++++++++\n>> hw/arm/smmuv3.c | 1181 +++++++++++++++++++++++++++++++++++\n>> hw/arm/sysbus-fdt.c | 129 +++-\n>> hw/arm/trace-events | 48 ++\n>> hw/arm/virt-acpi-build.c | 63 +-\n>> hw/arm/virt.c | 6 +-\n>> include/hw/acpi/acpi-defs.h | 15 +\n>> include/hw/arm/smmu-common.h | 123 ++++\n>> include/hw/arm/smmuv3.h | 80 +++\n>> include/hw/arm/sysbus-fdt.h | 2 +\n>> include/hw/arm/virt.h | 15 +\n>> target/arm/kvm.c | 27 +\n>> target/arm/trace-events | 3 +\n>> 17 files changed, 2886 insertions(+), 24 deletions(-)\n>> create mode 100644 hw/arm/smmu-common.c\n>> create mode 100644 hw/arm/smmu-internal.h\n>> create mode 100644 hw/arm/smmuv3-internal.h\n>> create mode 100644 hw/arm/smmuv3.c\n>> create mode 100644 include/hw/arm/smmu-common.h\n>> create mode 100644 include/hw/arm/smmuv3.h\n>>\n>> -- \n>> 2.5.5\n>>\n>>\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eric.auger@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xrwBm6TF8z9s7g\n\tfor ;\n\tTue, 12 Sep 2017 16:39:43 +1000 (AEST)","from localhost ([::1]:33897 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1drerD-000385-IP\n\tfor incoming@patchwork.ozlabs.org; Tue, 12 Sep 2017 02:39:39 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:60174)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dreqn-00035P-RH\n\tfor qemu-devel@nongnu.org; Tue, 12 Sep 2017 02:39:15 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dreql-0000qk-Vj\n\tfor qemu-devel@nongnu.org; Tue, 12 Sep 2017 02:39:13 -0400","from mx1.redhat.com ([209.132.183.28]:33152)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dreqg-0000od-Bc; Tue, 12 Sep 2017 02:39:06 -0400","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 600F6C04B92A;\n\tTue, 12 Sep 2017 06:39:04 +0000 (UTC)","from localhost.localdomain (ovpn-117-228.ams2.redhat.com\n\t[10.36.117.228])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id E8E026C518;\n\tTue, 12 Sep 2017 06:38:49 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 600F6C04B92A","To":"Linu Cherian ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<20170912061815.GA23124@virtx40>","From":"Auger Eric ","Message-ID":"","Date":"Tue, 12 Sep 2017 08:38:47 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101\n\tThunderbird/45.4.0","MIME-Version":"1.0","In-Reply-To":"<20170912061815.GA23124@virtx40>","Content-Type":"text/plain; charset=windows-1252","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.14","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.31]);\n\tTue, 12 Sep 2017 06:39:04 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 00/20] ARM SMMUv3 Emulation\n\tSupport","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, bharat.bhushan@nxp.com, mst@redhat.com,\n\twill.deacon@arm.com, qemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, linu.cherian@cavium.com,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1768450,"web_url":"http://patchwork.ozlabs.org/comment/1768450/","msgid":"<20170914092751.GA3336@virtx40>","list_archive_url":null,"date":"2017-09-14T09:27:52","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","submitter":{"id":72195,"url":"http://patchwork.ozlabs.org/api/people/72195/","name":"Linu Cherian","email":"linuc.decode@gmail.com"},"content":"Hi Eric,\n\nOn Fri Sep 01, 2017 at 07:21:16PM +0200, Eric Auger wrote:\n> memory_region_iommu_replay() is used for VFIO integration.\n> \n> However its default implementation is not adapted to SMMUv3\n> IOMMU memory region. Indeed the input address range is too\n> huge and its execution is too slow as it calls the translate()\n> callback on each granule.\n> \n> Let's implement the replay callback which hierarchically walk\n> over the page table structure and notify only the segments\n> that are populated with valid entries.\n> \n> Signed-off-by: Eric Auger \n> ---\n> hw/arm/smmuv3.c | 36 ++++++++++++++++++++++++++++++++++++\n> hw/arm/trace-events | 1 +\n> 2 files changed, 37 insertions(+)\n> \n> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n> index 8e7d10d..c43bd93 100644\n> --- a/hw/arm/smmuv3.c\n> +++ b/hw/arm/smmuv3.c\n> @@ -657,6 +657,41 @@ static int smmuv3_notify_entry(IOMMUTLBEntry *entry, void *private)\n> return 0;\n> }\n> \n> +/* Unmap the whole notifier's range */\n> +static void smmuv3_unmap_notifier_range(IOMMUNotifier *n)\n> +{\n> + IOMMUTLBEntry entry;\n> + hwaddr size = n->end - n->start + 1;\n> +\n> + entry.target_as = &address_space_memory;\n> + entry.iova = n->start & ~(size - 1);\n> + entry.perm = IOMMU_NONE;\n> + entry.addr_mask = size - 1;\n> +\n> + memory_region_notify_one(n, &entry);\n> +}\n> +\n> +static void smmuv3_replay(IOMMUMemoryRegion *mr, IOMMUNotifier *n)\n> +{\n> + SMMUTransCfg cfg = {};\n> + int ret;\n> +\n> + trace_smmuv3_replay(mr->parent_obj.name, n, n->start, n->end);\n> + smmuv3_unmap_notifier_range(n);\n> +\n> + ret = smmuv3_decode_config(mr, &cfg);\n> + if (ret) {\n> + error_report(\"%s error decoding the configuration for iommu mr=%s\",\n> + __func__, mr->parent_obj.name);\n> + }\n> \n\nOn an invalid config being found, shouldnt we return rather than proceeding with\npage table walk. For example on an invalid Stream table entry.\n\n+\n> + if (cfg.disabled || cfg.bypassed) {\n> + return;\n> + }\n> + /* walk the page tables and replay valid entries */\n> + smmu_page_walk(&cfg, 0, (1ULL << (64 - cfg.tsz)) - 1, false,\n> + smmuv3_notify_entry, n);\n> +}\n> static void smmuv3_notify_iova_range(IOMMUMemoryRegion *mr, IOMMUNotifier *n,\n> uint64_t iova, size_t size)\n> {\n> @@ -1095,6 +1130,7 @@ static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass,\n> \n> imrc->translate = smmuv3_translate;\n> imrc->notify_flag_changed = smmuv3_notify_flag_changed;\n> + imrc->replay = smmuv3_replay;\n> }\n> \n> static const TypeInfo smmuv3_type_info = {\n> diff --git a/hw/arm/trace-events b/hw/arm/trace-events\n> index 4ac264d..15f84d6 100644\n> --- a/hw/arm/trace-events\n> +++ b/hw/arm/trace-events\n> @@ -46,5 +46,6 @@ smmuv3_cfg_stage(int s, uint32_t oas, uint32_t tsz, uint64_t ttbr, bool aa64, ui\n> smmuv3_notify_flag_add(const char *iommu) \"ADD SMMUNotifier node for iommu mr=%s\"\n> smmuv3_notify_flag_del(const char *iommu) \"DEL SMMUNotifier node for iommu mr=%s\"\n> smmuv3_replay_mr(const char *name) \"iommu mr=%s\"\n> +smmuv3_replay(const char *name, void *n, hwaddr start, hwaddr end) \"iommu mr=%s notifier=%p [0x%\"PRIx64\",0x%\"PRIx64\"]\"\n> smmuv3_notify_entry(hwaddr iova, hwaddr pa, hwaddr mask, int perm) \"iova=0x%\"PRIx64\" pa=0x%\" PRIx64\" mask=0x%\"PRIx64\" perm=%d\"\n> smmuv3_notify_iova_range(const char *name, uint64_t iova, size_t size, void *n) \"iommu mr=%s iova=0x%\"PRIx64\" size=0x%lx n=%p\"\n> -- \n> 2.5.5\n> \n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"TlcJ7c8Z\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtCs00FVjz9sPt\n\tfor ;\n\tThu, 14 Sep 2017 19:28:50 +1000 (AEST)","from localhost ([::1]:46568 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsQS0-0005EH-DJ\n\tfor incoming@patchwork.ozlabs.org; Thu, 14 Sep 2017 05:28:48 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:42422)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsQRZ-0005CY-M3\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 05:28:22 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsQRY-0000La-I7\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 05:28:21 -0400","from mail-pf0-x241.google.com ([2607:f8b0:400e:c00::241]:32961)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1dsQRS-0000D8-Rn; Thu, 14 Sep 2017 05:28:15 -0400","by mail-pf0-x241.google.com with SMTP id h4so1272667pfk.0;\n\tThu, 14 Sep 2017 02:28:13 -0700 (PDT)","from virtx40 ([111.93.218.67]) by smtp.gmail.com with ESMTPSA id\n\t184sm26062178pfg.18.2017.09.14.02.28.06\n\t(version=TLS1_2 cipher=AES128-SHA bits=128/128);\n\tThu, 14 Sep 2017 02:28:11 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=o6jPFwc9u+F2UTXyKF+rRaUk/ZcLeVsy2bgeyK1jLuE=;\n\tb=TlcJ7c8Z/ZE6JN4Ofn6F3hxGb7mqEDNOjU5eFu0Ggc/eXQhBLZJttA6zpF8bc3b9TI\n\tmvvN6sX9hkClQD1s7N0iUuir4GzLEUCcaP454+u5LO7PIbycEZ2rSakyIEStN6Fpe8tc\n\tQEUhgb06X92Sv6+kILTXZg7XQ0JVB7vVjdSgjR4cHrYo4A7qNkuC26E8nE1qIqQJt900\n\tdTf2SlCAZuJfTQ6Lt45cFwZYVEO2TiM0ZJFhtbBnW4Vb13cahrXbWWhYE+kf8NNuKwhl\n\t3pPtt5iuEovi67vDbFnPVRY4a6i4xqrqg5TY7WgosO4OrnDkcRCc9tlRTi1H2Yn4NRVK\n\tVlwQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=o6jPFwc9u+F2UTXyKF+rRaUk/ZcLeVsy2bgeyK1jLuE=;\n\tb=I/IeWcXM0tSOurS6zBddgBhnKS0V6ANXChy2aqmzle4jF3mlhd+WIBo9QMKcb3JwcZ\n\ts1GVj8LOmy3nw4QuAZ05BtdE40c4nlMNuMtbqhj+TY1OTZeocWxxRdntuVN4YTBDc3Q5\n\t+4dK85qOn3HaBquqUUQjW+5B+UfYhcGuwklgwmmBlvUy0nloTW3E9AtHQUGDklVhbsEj\n\t4Phx0k81fLvTOMG7Bua3ACRY/OhRpLWq/j53g6cFDQtRGYim2gsZ0N2U59MA01mytpBE\n\tr9vaBeXlVUt2OvtkRG/+gfJOltuUmHpWeJgsgs19eKxEDdMjoWGzJXgtmJ6ojMVl02FB\n\tqHWw==","X-Gm-Message-State":"AHPjjUg7VbU9icEcrQJPcYRV5FqVMIFH5lSzx9pqa0zZiyxKzlwyIqZv\n\tztNFwX+89gC78A==","X-Google-Smtp-Source":"ADKCNb79JOUmwLg17GJLHPaDl69eSjXXYiUoXNMIm4AYEDqZHWscBp64Zb4oPIql9YJuFTJ9lw+egg==","X-Received":"by 10.98.102.149 with SMTP id s21mr20556275pfj.294.1505381292461;\n\tThu, 14 Sep 2017 02:28:12 -0700 (PDT)","Date":"Thu, 14 Sep 2017 14:57:52 +0530","From":"Linu Cherian ","To":"Eric Auger ","Message-ID":"<20170914092751.GA3336@virtx40>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-14-git-send-email-eric.auger@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<1504286483-23327-14-git-send-email-eric.auger@redhat.com>","User-Agent":"Mutt/1.5.21 (2010-09-15)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:400e:c00::241","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, bharat.bhushan@nxp.com, mst@redhat.com,\n\twill.deacon@arm.com, qemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, linu.cherian@cavium.com,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1768632,"web_url":"http://patchwork.ozlabs.org/comment/1768632/","msgid":"<4dbbccae-64e1-f4ac-4dcc-3abd9766f324@caviumnetworks.com>","list_archive_url":null,"date":"2017-09-14T14:31:58","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","submitter":{"id":72364,"url":"http://patchwork.ozlabs.org/api/people/72364/","name":"Tomasz Nowicki","email":"tnowicki@caviumnetworks.com"},"content":"Hi Eric,\n\nOn 14.09.2017 11:27, Linu Cherian wrote:\n> Hi Eric,\n> \n> On Fri Sep 01, 2017 at 07:21:16PM +0200, Eric Auger wrote:\n>> memory_region_iommu_replay() is used for VFIO integration.\n>>\n>> However its default implementation is not adapted to SMMUv3\n>> IOMMU memory region. Indeed the input address range is too\n>> huge and its execution is too slow as it calls the translate()\n>> callback on each granule.\n>>\n>> Let's implement the replay callback which hierarchically walk\n>> over the page table structure and notify only the segments\n>> that are populated with valid entries.\n>>\n>> Signed-off-by: Eric Auger \n>> ---\n>> hw/arm/smmuv3.c | 36 ++++++++++++++++++++++++++++++++++++\n>> hw/arm/trace-events | 1 +\n>> 2 files changed, 37 insertions(+)\n>>\n>> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n>> index 8e7d10d..c43bd93 100644\n>> --- a/hw/arm/smmuv3.c\n>> +++ b/hw/arm/smmuv3.c\n>> @@ -657,6 +657,41 @@ static int smmuv3_notify_entry(IOMMUTLBEntry *entry, void *private)\n>> return 0;\n>> }\n>> \n>> +/* Unmap the whole notifier's range */\n>> +static void smmuv3_unmap_notifier_range(IOMMUNotifier *n)\n>> +{\n>> + IOMMUTLBEntry entry;\n>> + hwaddr size = n->end - n->start + 1;\n>> +\n>> + entry.target_as = &address_space_memory;\n>> + entry.iova = n->start & ~(size - 1);\n>> + entry.perm = IOMMU_NONE;\n>> + entry.addr_mask = size - 1;\n>> +\n>> + memory_region_notify_one(n, &entry);\n>> +}\n>> +\n>> +static void smmuv3_replay(IOMMUMemoryRegion *mr, IOMMUNotifier *n)\n>> +{\n>> + SMMUTransCfg cfg = {};\n>> + int ret;\n>> +\n>> + trace_smmuv3_replay(mr->parent_obj.name, n, n->start, n->end);\n>> + smmuv3_unmap_notifier_range(n);\n>> +\n>> + ret = smmuv3_decode_config(mr, &cfg);\n>> + if (ret) {\n>> + error_report(\"%s error decoding the configuration for iommu mr=%s\",\n>> + __func__, mr->parent_obj.name);\n>> + }\n>>\n> \n> On an invalid config being found, shouldnt we return rather than proceeding with\n> page table walk. For example on an invalid Stream table entry.\n\nIndeed, without return here vhost case is not working for me.\n\nThanks,\nTomasz\n\n> \n> +\n>> + if (cfg.disabled || cfg.bypassed) {\n>> + return;\n>> + }\n>> + /* walk the page tables and replay valid entries */\n>> + smmu_page_walk(&cfg, 0, (1ULL << (64 - cfg.tsz)) - 1, false,\n>> + smmuv3_notify_entry, n);\n>> +}\n>> static void smmuv3_notify_iova_range(IOMMUMemoryRegion *mr, IOMMUNotifier *n,\n>> uint64_t iova, size_t size)\n>> {\n>> @@ -1095,6 +1130,7 @@ static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass,\n>> \n>> imrc->translate = smmuv3_translate;\n>> imrc->notify_flag_changed = smmuv3_notify_flag_changed;\n>> + imrc->replay = smmuv3_replay;\n>> }\n>> \n>> static const TypeInfo smmuv3_type_info = {\n>> diff --git a/hw/arm/trace-events b/hw/arm/trace-events\n>> index 4ac264d..15f84d6 100644\n>> --- a/hw/arm/trace-events\n>> +++ b/hw/arm/trace-events\n>> @@ -46,5 +46,6 @@ smmuv3_cfg_stage(int s, uint32_t oas, uint32_t tsz, uint64_t ttbr, bool aa64, ui\n>> smmuv3_notify_flag_add(const char *iommu) \"ADD SMMUNotifier node for iommu mr=%s\"\n>> smmuv3_notify_flag_del(const char *iommu) \"DEL SMMUNotifier node for iommu mr=%s\"\n>> smmuv3_replay_mr(const char *name) \"iommu mr=%s\"\n>> +smmuv3_replay(const char *name, void *n, hwaddr start, hwaddr end) \"iommu mr=%s notifier=%p [0x%\"PRIx64\",0x%\"PRIx64\"]\"\n>> smmuv3_notify_entry(hwaddr iova, hwaddr pa, hwaddr mask, int perm) \"iova=0x%\"PRIx64\" pa=0x%\" PRIx64\" mask=0x%\"PRIx64\" perm=%d\"\n>> smmuv3_notify_iova_range(const char *name, uint64_t iova, size_t size, void *n) \"iommu mr=%s iova=0x%\"PRIx64\" size=0x%lx n=%p\"\n>> -- \n>> 2.5.5\n>>\n>>\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=CAVIUMNETWORKS.onmicrosoft.com\n\theader.i=@CAVIUMNETWORKS.onmicrosoft.com header.b=\"U6kkyLXW\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=Tomasz.Nowicki@cavium.com; "],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtLc40P7Pz9s4s\n\tfor ;\n\tFri, 15 Sep 2017 00:33:08 +1000 (AEST)","from localhost ([::1]:48133 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsVCU-0001t6-00\n\tfor incoming@patchwork.ozlabs.org; Thu, 14 Sep 2017 10:33:06 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:41789)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsVBz-0001pX-Ac\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 10:32:39 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsVBt-000796-Dc\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 10:32:35 -0400","from mail-sn1nam01on0066.outbound.protection.outlook.com\n\t([104.47.32.66]:12761\n\thelo=NAM01-SN1-obe.outbound.protection.outlook.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dsVBh-0006ae-5h; Thu, 14 Sep 2017 10:32:17 -0400","from [10.0.0.85] (31.172.191.173) by\n\tSN4PR0701MB3664.namprd07.prod.outlook.com (2603:10b6:803:4d::18) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.35.12;\n\tThu, 14 Sep 2017 14:32:09 +0000"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=0o7tT9h/31wbsg0qO9t4qnhsVmTHw6RPdVJ9YtBt1b0=;\n\tb=U6kkyLXW8IlS7m5/zqQfiYJBQSLtquoeqRq3dxtwdH/7PSiEVNb/cX66AmUHjyyIXOg9Pk8tZqI4koZTedosPIy/es9/nz/PXpMW/rRtPf4AIkwkOz/Ykp8zHlzWeeeRMUFDOAZbtkTCA3t3OCakMrWJ/PVNRth8Yxzcgrjo6/g=","To":"Linu Cherian , Eric Auger ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-14-git-send-email-eric.auger@redhat.com>\n\t<20170914092751.GA3336@virtx40>","From":"Tomasz Nowicki ","Message-ID":"<4dbbccae-64e1-f4ac-4dcc-3abd9766f324@caviumnetworks.com>","Date":"Thu, 14 Sep 2017 16:31:58 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 Thunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<20170914092751.GA3336@virtx40>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-GB","Content-Transfer-Encoding":"7bit","X-Originating-IP":"[31.172.191.173]","X-ClientProxiedBy":"AM5PR0701CA0056.eurprd07.prod.outlook.com\n\t(2603:10a6:203:2::18) To SN4PR0701MB3664.namprd07.prod.outlook.com\n\t(2603:10b6:803:4d::18)","X-MS-PublicTrafficType":"Email","X-MS-Office365-Filtering-Correlation-Id":"4d0dd8a8-9f81-40b2-8b62-08d4fb7d641f","X-Microsoft-Antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(300000502095)(300135100095)(22001)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:SN4PR0701MB3664; ","X-Microsoft-Exchange-Diagnostics":["1; SN4PR0701MB3664;\n\t3:i4xWWKZg84bhqFFwlNSLXwGzuT6DNOROSUX45fy3mz3RfpWTaaQu06jPCNTfp073r4InrLROEyolkA5aku3Qi1dzT4+xhZhIgU3geu+4Em2os7PGgGwuqBfqEYyVmO9h4EdID19KnMtxDQUZvytlOFZisVU2p43w2ct7WyDaXWYzDWptXKRVAfDwYhpe5goKtSxRHB+yBkBEkI03mTHtsyqOCsgvvYkfmqsPAd/pAxu655+g2acRtC6fIOb+Q4kX;\n\t25:tKY+VRP0RaPTJ6ymFK3IqoDVkFZR424HGelxoyS79jiKk3UkcBqFB+ChuyRTq19ejelqjf0ODTPRr9MYq3PG4bMLgAS1uGKZmH535V26X3IBjeaY0jdr+caqiQo5MPANok58EMgbH/RFK5iyTJtZfbArdCo/Jcazy5dn3jWk5haDFXJGj+BicxA5VE6n9F88+MWG7aDY8MEGgduEYbgaLQ3m4k/cQILvQYnTfPXFzJ2npk5JyNr4dkAh/x1nXLfmvB8CzKyH+4XjA730oz7lHEnrWPbdzkvlLFXgseU2ydryAgvjytMLYlyH/ny8viQTCEIq5WbLe84HhvYf9rv/Aw==;\n\t31:uTRiQwpvVtHtvVE9h8Xpar96LK2OQ2j4KXmtw/zO7d9zqNmN4fphjfT9zuRynwUs+6QQAPoBujW1W0rC4V0I6X6bZ5HzkPTkyea715Igr8g6rNZhhBfI6cYCTo/N5WMgqisks5pOZ5OiK0/Kik3UGuAm9W/Ye74H2t9S/7UAjH0IP3WhUd21vmeLjpXS1Jski2cldVB38gR2CGfJpduycIskUyTs0ROHAv7B94fIRMY=","1; SN4PR0701MB3664;\n\t20:GOseN1wWjnQxyyFR1Lfktsg9J9xgPwiLRD0cn0N2N8dQBI60kVZa4yv0ArfAuN2xPjtx7EP36ucIUJogjlH0gzo9sI9A/A93kscptpgRkWZREGJ7Si3Zc/v8P/pUq7/U+y26D2rXo5FpqcbYVmv9NS6qiY8wCFGnLqjJj9/kNqqSgcmGtRcns9bJZAaFXRbwEp9IbvW8caKYnQBGdPu4Vqk9wf9pGgfE8QIns6OVDzqh9aQuSVET95879j7HtQvuPb4jmC/TsSOaPnK7sGCSDQ0XuNn7i40WCq714y9GDpLQi8tvEW4P0Sq8BU6pA+eS6sOeh1JEaEiA+jgBmZ33VshqHnfM9HhFj+Qp729BFKCmMsAHxiVtIWaBttvdMlg018D9QV9ItoA0n0SHODFWJkMzKHwcfKv5q33x9rE61qAJGffu3A7xUk+M8dU32xi8tXIKrwTsoWyx9ZNe8jUTcCfSS/5qMBlwrB8hQKPXdd0t4YPlWDuaPyuU5cRKv6E+R2XzSiOt79himIlYf17fRDqgOYGJFNQwOb+VlXgntxCYh+iptRoFSfA0iiXY1oTkLFVVk+rorLV7ic13IzPYSmDsJlrbu5nJhu+opteL5p4=;\n\t4:j3hdqxURhxkCiHplA2vuL/Bb3oBvr6l/TGlnPimi41ma/z0zrltKjvv7Env2syT4rOwe5pyj3LfqFnQT/gwsG7TkmS9yFYek+gua/Q1WvS+o0BW6fP45UVdNpnacWXJVPWoflTEXqjQez17omIoiB2RWOaLyP3GJ9XKvY9t4DN62N/I+R4MBbAI9sX1Ca44iEI0uyIaNHxUzz2ovNqCLtkXCg8+Vkdm8+ltfi2Pb6RD1q0HUlb265hcSA/DX8/q8","=?utf-8?q?1=3BSN4PR0701MB3664=3B23=3Aqt?=\n\t=?utf-8?q?qTbyIDTew5Llr7tLBNdmHrWgyTdwGy4WP/i1sRDxk+Xp6z6TFYJIAeQL?=\n\t=?utf-8?q?+VYXIm7zpX5Yq/kBFbMXVyaY+LkW2cew+2C9AC5LihjhVf+1E1z2uVCr?=\n\t=?utf-8?q?QTLsdLt0ACGEXj2NVGVgHvatkfqc+xO+niLIsD0wHOyjTP0ObmTKHMfG?=\n\t=?utf-8?q?LNarKvcNZALFu/vfFkjj2jTAiM9R9XecRcrDQ1xW+gsL/mHoG8pVhqYn?=\n\t=?utf-8?q?dpuICtU3Kk9yy5ZsqflpD86BPs6jnSsE4jL0q20Eg9WLUs4ExRkROUsm?=\n\t=?utf-8?q?GUwKGHM3B5dRFXQmvYt/yycrFkhfKc9/fjAerllElG4lP91v5Ti72P52?=\n\t=?utf-8?q?ixEg2/2w/guViqYLTbOal4MU7qhj0juaNVb7pr28YSfKpNoYgTGvC0im?=\n\t=?utf-8?q?UqzAUEyYHjDRdlxW6dGvDUOrURw3j8PJSvS/eiG3hUbHKnW47TY8g6vM?=\n\t=?utf-8?q?XSW/EfrQfSArsA7ERFaYV0IBOm9z+GXlN0tZzwXfmaiZM325gizXhKSy?=\n\t=?utf-8?q?llyN8aRrgtHOdQajbBt/JmluomhvgSLTik3bDDHiucKjaEnmUwatgrBD?=\n\t=?utf-8?q?gLWblkTfH17NpYmU/dOX4USrCo3vAtLMYpU9gv2/BihHo5eFN6hNWYHh?=\n\t=?utf-8?q?f0eIvtE0jQ3/6SIRMHHsya8kJL9KgqpPqvJpf9BDXB4vbmZ1/ROXP1fi?=\n\t=?utf-8?q?asN5PWx8MLz3rkPu+w9kPd640WzE98NrIE7QWDr1htLq7pxwNAFM8t6q?=\n\t=?utf-8?q?QSxI4cFyoQNWKryjeH71H/l4/Qu5Ox0PBjmbgwdrR2U1CAomwkrtStsq?=\n\t=?utf-8?q?nlYBAKrePeXeN8MEv2xRHLt2y6BfVwXQk2jwyMXHcSepsN2LQwzlpmk0?=\n\t=?utf-8?q?ywoKuOGBz1es8R8Ka5Z3ZuAS5sL+kx7sm8Lm33UBFuwMsS/kPGcJaMyt?=\n\t=?utf-8?q?ht54YnmpzAHzeAgP1yQwE98UHEJoMErAe6uDONUAJy42/0/VSuxMUHsZ?=\n\t=?utf-8?q?imxPplj5+1VKNdOAXAKeXSahq1BLYt5Xv8k7v+hfmpktsBJ7sO4ap5Aw?=\n\t=?utf-8?q?/uhrc13KBDSPoLtMWgecAeQ59XecAmGYvUqVLmAo3LXVajddHTxfLAzT?=\n\t=?utf-8?q?5nfIHJGHWAaOfUEHdO/+oR05TinfVr1spJ+/853U5QDZ/leccJdtsRmr?=\n\t=?utf-8?q?NmejR/7V+syGDRgqJcZ2rMrbrZXKYYSUfLMNs5CGZi1+BYTQ7/SrThwc?=\n\t=?utf-8?q?3/HR0YLwT96hmc50oHk101xM7XOJc8C7T/b1oq0yZxc8QwZscrPFe6YZ?=\n\t=?utf-8?q?6LbNzc1TzDQLgmdTOVJbDYOBDx44Zxg5SeiVW8vLHaqtEbMdxXAne/TE?=\n\t=?utf-8?q?SoxJDKZwcwGayhpuFSYQ/DH/YysinrY6D2xOXmiuEcV0FgK8akrWknAb?=\n\t=?utf-8?q?ctv0Ec7vBsf2m6LhlA35sBA3xjaPvew9WJ3PMfabw=3D?=","1; SN4PR0701MB3664;\n\t6:B4JUusgmsN830dLNb2IqP0UiGujfqykPUaNghD/CBLg1Oy0J3eFKplyKuh/5mhtF5Pk3gaJEElfvXzB8rPfJLxQ1AH9EPTUts4sLMbuSOrucDev5Ap4sKcIegLZ1896rZGcAxkbTCj8uNmXBmTrdIO62BkwPvU+jzlcpHQue2FGA3Ie2vwlmPv4KyUS93tsPamZuB79jjHWPgsl8CLx1ZgRzDeHiWzxqNSbe9MZtqCx4QwIF1JsaBqVAvTN8PIALG1pSdHUqSSwH5iuU//R5DEf5osWoOad8LagKEbOitF17uO0ZDUG9eEkEgD60AyhGgfjt0CD0n153dlELvG9Zag==;\n\t5:rd5N+q6qstouau6kNDXhdsmWpYRpEPGBMjLpLFEclu8pMaNJ3PK03IzmAQqW0OWURd3j1PVXgZC0F6U8bKGroz+0rqE7amD6TBb/bnYjCbjtMvVTBQ4QrMsC5VjfpyzyVMCcUZUsmvc3OWxxvaoFkg==;\n\t24:rPDexi4K0V/OsTFHNUiHyAPAiasycjXWV2GycHYkXWd3Nb8DfnZLgHDe+DwzcET8/S7nsBlzZ1vXGbhGxZI03CFtVtaSUW6aOGhm5d6qxPM=;\n\t7:LkgaVkn3a4U6+SjFojUrdJXjNVNkJdQE2VI8jkJ9UoiwrRiZ698qyKr6IyjwHpiwkGCR78EPBuuH8AIAoqrCLBn+zDRAAaib3J8CoohVVL1ygw8DU1Uh19LV2+/04Lzcy58khTQpkSFiHB1K2yo1eF3rFeMEmkbuHau9WfhcVQc+ouNDFSdc262ndGWoWlUNMT+6zEvy8p6DSdo6LGBNTIUfqzEhGzmTWpiQIr1jy8g="],"X-MS-TrafficTypeDiagnostic":"SN4PR0701MB3664:","X-Exchange-Antispam-Report-Test":"UriScan:;","X-Microsoft-Antispam-PRVS":"","X-Exchange-Antispam-Report-CFA-Test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(10201501046)(100000703101)(100105400095)(93006095)(3002001)(6041248)(20161123562025)(20161123555025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:SN4PR0701MB3664; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:SN4PR0701MB3664; ","X-Forefront-PRVS":"0430FA5CB7","X-Forefront-Antispam-Report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(6049001)(376002)(346002)(24454002)(189002)(199003)(106356001)(53546010)(33646002)(3846002)(25786009)(64126003)(31696002)(101416001)(76176999)(50466002)(478600001)(105586002)(6116002)(230700001)(54356999)(97736004)(31686004)(72206003)(4001350100001)(68736007)(50986999)(5660300001)(8936002)(316002)(107886003)(77096006)(83506001)(81156014)(7416002)(8676002)(36756003)(47776003)(2906002)(53936002)(6486002)(4326008)(81166006)(6666003)(229853002)(8656003)(2950100002)(42882006)(66066001)(16576012)(16526017)(6246003)(189998001)(23676002)(65956001)(39060400002)(305945005)(7736002);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:SN4PR0701MB3664; H:[10.0.0.85];\n\tFPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; ","Received-SPF":"None (protection.outlook.com: cavium.com does not designate\n\tpermitted sender hosts)","SpamDiagnosticOutput":"1:99","SpamDiagnosticMetadata":"NSPM","X-OriginatorOrg":"caviumnetworks.com","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"14 Sep 2017 14:32:09.6448\n\t(UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"711e4ccf-2e9b-4bcf-a551-4094005b6194","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"SN4PR0701MB3664","X-detected-operating-system":"by eggs.gnu.org: Windows 7 or 8 [fuzzy]","X-Received-From":"104.47.32.66","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\tbharat.bhushan@nxp.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, linu.cherian@cavium.com,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1768656,"web_url":"http://patchwork.ozlabs.org/comment/1768656/","msgid":"<795d99ba-e492-04ce-dda4-709682baf6cd@caviumnetworks.com>","list_archive_url":null,"date":"2017-09-14T14:43:20","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","submitter":{"id":72364,"url":"http://patchwork.ozlabs.org/api/people/72364/","name":"Tomasz Nowicki","email":"tnowicki@caviumnetworks.com"},"content":"On 14.09.2017 16:31, Tomasz Nowicki wrote:\n> Hi Eric,\n> \n> On 14.09.2017 11:27, Linu Cherian wrote:\n>> Hi Eric,\n>>\n>> On Fri Sep 01, 2017 at 07:21:16PM +0200, Eric Auger wrote:\n>>> memory_region_iommu_replay() is used for VFIO integration.\n>>>\n>>> However its default implementation is not adapted to SMMUv3\n>>> IOMMU memory region. Indeed the input address range is too\n>>> huge and its execution is too slow as it calls the translate()\n>>> callback on each granule.\n>>>\n>>> Let's implement the replay callback which hierarchically walk\n>>> over the page table structure and notify only the segments\n>>> that are populated with valid entries.\n>>>\n>>> Signed-off-by: Eric Auger \n>>> ---\n>>> hw/arm/smmuv3.c | 36 ++++++++++++++++++++++++++++++++++++\n>>> hw/arm/trace-events | 1 +\n>>> 2 files changed, 37 insertions(+)\n>>>\n>>> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n>>> index 8e7d10d..c43bd93 100644\n>>> --- a/hw/arm/smmuv3.c\n>>> +++ b/hw/arm/smmuv3.c\n>>> @@ -657,6 +657,41 @@ static int smmuv3_notify_entry(IOMMUTLBEntry \n>>> *entry, void *private)\n>>> return 0;\n>>> }\n>>> +/* Unmap the whole notifier's range */\n>>> +static void smmuv3_unmap_notifier_range(IOMMUNotifier *n)\n>>> +{\n>>> + IOMMUTLBEntry entry;\n>>> + hwaddr size = n->end - n->start + 1;\n>>> +\n>>> + entry.target_as = &address_space_memory;\n>>> + entry.iova = n->start & ~(size - 1);\n>>> + entry.perm = IOMMU_NONE;\n>>> + entry.addr_mask = size - 1;\n>>> +\n>>> + memory_region_notify_one(n, &entry);\n>>> +}\n>>> +\n>>> +static void smmuv3_replay(IOMMUMemoryRegion *mr, IOMMUNotifier *n)\n>>> +{\n>>> + SMMUTransCfg cfg = {};\n>>> + int ret;\n>>> +\n>>> + trace_smmuv3_replay(mr->parent_obj.name, n, n->start, n->end);\n>>> + smmuv3_unmap_notifier_range(n);\n>>> +\n>>> + ret = smmuv3_decode_config(mr, &cfg);\n>>> + if (ret) {\n>>> + error_report(\"%s error decoding the configuration for iommu \n>>> mr=%s\",\n>>> + __func__, mr->parent_obj.name);\n>>> + }\n>>>\n>>\n>> On an invalid config being found, shouldnt we return rather than \n>> proceeding with\n>> page table walk. For example on an invalid Stream table entry.\n> \n> Indeed, without return here vhost case is not working for me.\n\nI was just lucky one time. return here has no influence. Vhost still not \nworking. Sorry for noise.\n\nTomasz\n\n> \n> Thanks,\n> Tomasz\n> \n>>\n>> +\n>>> + if (cfg.disabled || cfg.bypassed) {\n>>> + return;\n>>> + }\n>>> + /* walk the page tables and replay valid entries */\n>>> + smmu_page_walk(&cfg, 0, (1ULL << (64 - cfg.tsz)) - 1, false,\n>>> + smmuv3_notify_entry, n);\n>>> +}\n>>> static void smmuv3_notify_iova_range(IOMMUMemoryRegion *mr, \n>>> IOMMUNotifier *n,\n>>> uint64_t iova, size_t size)\n>>> {\n>>> @@ -1095,6 +1130,7 @@ static void \n>>> smmuv3_iommu_memory_region_class_init(ObjectClass *klass,\n>>> imrc->translate = smmuv3_translate;\n>>> imrc->notify_flag_changed = smmuv3_notify_flag_changed;\n>>> + imrc->replay = smmuv3_replay;\n>>> }\n>>> static const TypeInfo smmuv3_type_info = {\n>>> diff --git a/hw/arm/trace-events b/hw/arm/trace-events\n>>> index 4ac264d..15f84d6 100644\n>>> --- a/hw/arm/trace-events\n>>> +++ b/hw/arm/trace-events\n>>> @@ -46,5 +46,6 @@ smmuv3_cfg_stage(int s, uint32_t oas, uint32_t tsz, \n>>> uint64_t ttbr, bool aa64, ui\n>>> smmuv3_notify_flag_add(const char *iommu) \"ADD SMMUNotifier node \n>>> for iommu mr=%s\"\n>>> smmuv3_notify_flag_del(const char *iommu) \"DEL SMMUNotifier node \n>>> for iommu mr=%s\"\n>>> smmuv3_replay_mr(const char *name) \"iommu mr=%s\"\n>>> +smmuv3_replay(const char *name, void *n, hwaddr start, hwaddr end) \n>>> \"iommu mr=%s notifier=%p [0x%\"PRIx64\",0x%\"PRIx64\"]\"\n>>> smmuv3_notify_entry(hwaddr iova, hwaddr pa, hwaddr mask, int perm) \n>>> \"iova=0x%\"PRIx64\" pa=0x%\" PRIx64\" mask=0x%\"PRIx64\" perm=%d\"\n>>> smmuv3_notify_iova_range(const char *name, uint64_t iova, size_t \n>>> size, void *n) \"iommu mr=%s iova=0x%\"PRIx64\" size=0x%lx n=%p\"\n>>> -- \n>>> 2.5.5\n>>>\n>>>\n>>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=CAVIUMNETWORKS.onmicrosoft.com\n\theader.i=@CAVIUMNETWORKS.onmicrosoft.com header.b=\"PFG91o29\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=Tomasz.Nowicki@cavium.com; "],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtMMW6Hk8z9sPk\n\tfor ;\n\tFri, 15 Sep 2017 01:07:19 +1000 (AEST)","from localhost ([::1]:48321 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsVjZ-0007ti-Pt\n\tfor incoming@patchwork.ozlabs.org; Thu, 14 Sep 2017 11:07:17 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:48249)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsVMu-0008VL-S1\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 10:43:58 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsVMq-0004Ds-Kg\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 10:43:52 -0400","from mail-dm3nam03on0055.outbound.protection.outlook.com\n\t([104.47.41.55]:59552\n\thelo=NAM03-DM3-obe.outbound.protection.outlook.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dsVMg-00048U-NM; Thu, 14 Sep 2017 10:43:39 -0400","from [10.0.0.85] (31.172.191.173) by\n\tMWHPR0701MB3659.namprd07.prod.outlook.com (2603:10b6:301:7e::10) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.56.11;\n\tThu, 14 Sep 2017 14:43:30 +0000"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=/GWoYjR/cshcH2qUBLkO/PQjFmy6u/xzQfBOoCQ2sGo=;\n\tb=PFG91o29ERYn0y053pjY8MxWTxGmcC+ttbSePyfbtBmO2rSDkXD89sbDJfLYk4L1zc/GB1b2BI2Nv/MyB3Wod8I43EpgnynTs3TndqZEwpNM7OKOSKVR6vJeDT2vDmmdCCulPII3MRpnfTfuqPELAPL1eMdlmcMV0l0GRamBgn0=","From":"Tomasz Nowicki ","To":"Linu Cherian , Eric Auger ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-14-git-send-email-eric.auger@redhat.com>\n\t<20170914092751.GA3336@virtx40>\n\t<4dbbccae-64e1-f4ac-4dcc-3abd9766f324@caviumnetworks.com>","Message-ID":"<795d99ba-e492-04ce-dda4-709682baf6cd@caviumnetworks.com>","Date":"Thu, 14 Sep 2017 16:43:20 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 Thunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<4dbbccae-64e1-f4ac-4dcc-3abd9766f324@caviumnetworks.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-GB","Content-Transfer-Encoding":"7bit","X-Originating-IP":"[31.172.191.173]","X-ClientProxiedBy":"AM5PR0502CA0023.eurprd05.prod.outlook.com\n\t(2603:10a6:203:91::33) To MWHPR0701MB3659.namprd07.prod.outlook.com\n\t(2603:10b6:301:7e::10)","X-MS-PublicTrafficType":"Email","X-MS-Office365-Filtering-Correlation-Id":"02be227c-8233-4ede-a6d5-08d4fb7efa45","X-Microsoft-Antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(300000502095)(300135100095)(22001)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:MWHPR0701MB3659; ","X-Microsoft-Exchange-Diagnostics":["1; MWHPR0701MB3659;\n\t3:fcsl1W3U22z2XN0egP8WT3+C9026DIfTbsorc1TngmwzDqHX7jzM8f2yz0F3IIyZt1KiBkpQ0t9GR1gfAUPj/XlfN5hDjjA/2STFAN2Yh+EeKFcvb8/UGQVbSQYiUZEDi5JihyNC8yASuNLMoB5S1014pTyWQLXFV+aVPB4LgEadv21c88Wol+tySqPy029z+mGLJVivhL8ASXZZ1OaQe5QzNRt5qrx6VLZWfASODQbx/dyaH9ORMYMDKdw11QMQ;\n\t25:K6KWCNd7RiaLqTkPv71dRt0FUcXe6nk2g+k6EbL0I7FT5VCFRPPJSvy5FDmPRX1uiC6HEGhhEXlr5eYbsIT8OCN/P/oBl6Zw2OxdmJDZoYHpdqwbQV3o59Cm9qlF3xIAEPD+dYoCMdBUsPwwHGrmfkx7tKC+9wFK6PJB7q3ijlM1Os5NFOs5g811u4ol9lq+FEQxMoDiSpXnA2Cw8slw5Mw2NFQz+plZI/A8qOSGPAJckrHLocAHjqSXsNOYD8DB3TlAicR2akHYhm6lPOyPp744naNtwe4/nV5oBVCkjemJbiICrD/Q83ceQKYYAqOMAIykU3JYpUjYFTv5MHkdlQ==;\n\t31:NU9ZUacmqc0rwtFe6qEvK1+5dhMOGVX+GrROriVPW3LM4mIK+etAt5+sCt+S0AdCSYEDLNE58T9MTEsXk8ubHlTjy5mnJJFqZyu/qjivWnxUWi3kUYp9MOnIkB9TyK3SuEXTw6hDth5Mc6udIbtF7FSWozJj1WIygu4Lu1NBZ3kJCTfYWyvnRCVL+SmB/z9Id8ko7Sqacl9k5UwsIcJaAfxBAgVDlDwP6eKj/aewPLw=","1; MWHPR0701MB3659;\n\t20:ZDWj0F/Xxjtd+ynWnhFwwVbC9tmVPv5YBqUxutGrs8ghfx2QDbog+x5aLl5UK6mjEX0drpYSQfmGqp1dkwxvLEEaPLfnR2Gq+alVZl6ko2fSRL1QUYCpUBF9IdjSg1/fDzjxH4YIDIrMllh3lTpIep9g6Sfha4XCdOyzbTnLm/ffN2N5YpFK+RbIYkgXamTBHTRQsl3l+Qeta+YBkhYpTPVPGDWqGRKsZDufZzqhUpaFMWSeNk47xFkM46fCvkCUxVWhMt1+uYnxMM8mZsVBMwJlW/wQqIWMUS5j4LHMSZfShsrZ8sDpcsn+FXll8s/Kh21+YaZtzsrAZLY9NB427W5VP0MZNUjsIQefOnaqAGGUjQQ92ZyZfD+/8tiXx10bFQpWCDqL58RVnDOLeCHT07/TyVjRL+UMGs+5/w3oiomxFVAddLrC+H1Ir3rx5XNbSjIOGfC9oPT5hozmlgrsih5nQ51dpEhAoboxqBowZigLD9og3voOtB34agnAG7oYihp3ioLqpxdy4HjPG7OID2O49n/pyCPGql9wpcE01Gq+crt6HuOZPkwBCW4kcCKx9FXVTvgGPzWCwXDUUT8iQSe8i7ifULxfnKwWTWBCOZE=;\n\t4:qWYa8oHhqjpG2vwiIfizGb6gi9rOTgx4MkmQm/gsyRSl36BbRRWNdAGOFvPs1AHSPs/EKL/o8bya7TZ65J2HOOn7RwP6arrcORpZf6X77QOGPfcBInoq8Tqu0lHHUXgpsIXPCBis1jaP1BWsG5ESkxBROKfezY89SzD4y3fVmeHPAuueEmtrt5Jd2SR8HmQ5FGEZ083FvNzxvCVvv0QbXo7dS+yzBU02tu2iOrTMo9XpN6kEKrR0vPhSsfhmm3/W","=?utf-8?q?1=3BMWHPR0701MB3659=3B23=3AIf?=\n\t=?utf-8?q?st9hQU5PZU80HD65ZBuY9kXFdiluONoosuNAJ0ph7+XfxiBz3mOWN0I8?=\n\t=?utf-8?q?FWBzC4U76u0eK9U3IKRCBO0mxkJXXZiDlRZS7ebib/gli8/roNQiDFav?=\n\t=?utf-8?q?QED4DOaisYx1Nwdnz+Jc7CSfTRTr1OxQgHBtIrj5Zc6Rhj8IXxMoxeCi?=\n\t=?utf-8?q?Ya0X5ApHogwQ9jq+ikAWQkN91eUXN29D4o9r9tx5x3sAMmbXNtZTj1GB?=\n\t=?utf-8?q?SBF8wWiAUrHSt2gpnkPrOOVrYns1Vfdf8iRrq7fFWzlzoqDCrDj2SQDb?=\n\t=?utf-8?q?K7jbAsodQmf2DOYXi+2OqGy1LGMl5wDwX9CDpF0nVSs8GoARD7Q5c+12?=\n\t=?utf-8?q?bQAXVZY7faBBotB4apuyM2vFVoVyenn2zfYkTnOplTJc0WK3KP6NUbPy?=\n\t=?utf-8?q?YfEz/l2744bzj1FT/Yjgfl9a6F5lhX0MCSzUy/YlXwQxO+jwoma00U32?=\n\t=?utf-8?q?NZdHTT8eODd6fa3uKWYBVkC+mKpWczaSYGRmLEwKstsHCQOzGtpRF1/K?=\n\t=?utf-8?q?xzj8fkaY/eHn+Pz5iu61k8jfKN6DMAUPLCvb3MkG88LQ7Pkb0VFXTtYk?=\n\t=?utf-8?q?55oILaSn8PSy+QIoScWr/WaxluSTdjgzYqxE7iFydmwUNJ3oFdaT1oOI?=\n\t=?utf-8?q?PJKp/3ral0kw4iHqQqAYOgiojwP2dyVfaOndSTKMuUXqzHBnPXvz8m4u?=\n\t=?utf-8?q?FONcdCE2xLetkDitzds6HVEwBOVI/nvovqN/6teiaFrnjNN5iNPVaQKI?=\n\t=?utf-8?q?hqPi74/uh0P/lRPfFd7W8xbv5pSgfNgDTAdj8Z2jCQBAMOCiMF4nObWB?=\n\t=?utf-8?q?GtGYMbl0ek3vIgi2W/otLye259vu6fwIaT2Ce7LSuPPG87i0ea8cGsqW?=\n\t=?utf-8?q?+szgIiiZIDAZLbr4Kz1Lf1Bbhe58zCMdfQOtHctfjtJl6crbKZkhVIYf?=\n\t=?utf-8?q?bn1y4PsqwF4s1mL+89s0gdgiWKUPGbX7JAG9vSANdBPMqYhWYrYrYSO+?=\n\t=?utf-8?q?NaLAdOfisXyjIYoBE/O+cdU9or0IRxVX5ug+zv4OLgB5KaLuZ96lyi+2?=\n\t=?utf-8?q?j7I0os97ySEad17rFBz2kzxVpOXeka0Gpzc/m70y/SbpIhTQpx3XXH8l?=\n\t=?utf-8?q?PSvbIKnYRYUu+DWVLCzibrETGDk7BYKJ04u3Qkm83Dh8vM5MDL8Tebot?=\n\t=?utf-8?q?XGmWYYvcp7JWIOm511pS+kMsPqOMJmlE3hIlOeG1hx0wPnh3dYF3YZKd?=\n\t=?utf-8?q?hc4Zkqgo055llrTw4RjAl+oLPNMNMaZPXVGtEbX7LhgqZ5Gm1lz69BD2?=\n\t=?utf-8?q?r0JRTiIRcBORFf3LxHeES89q63CTsDDkT7j2Ng6s6ydY2aGoYPqNv/Ar?=\n\t=?utf-8?q?mGX4BaYHQM4VS07f9gsT0AeSe2GGLtdv5wt9CfGA0qodJc+O5wmlWQXw?=\n\t=?utf-8?q?EF2+f3ghB1lilNmzAAYkahKiFf8I2ev0RAcI6TszIQADhKtm2qKPDkdg?=\n\t=?utf-8?q?4TEZ1H?=","1; MWHPR0701MB3659;\n\t6:Jg4eDvWmforKN6KDi65pCBmd0RfumacyaAVpI1WdTUdLXtWGdsn4a4D0X7Z/kvHhKx5+TD+tWP1eBbwwVMYGqgaKcLntH75HCPDd5MGSbRpXWM9Mus+0BHopPfZ+8ormg35EgJV6D3y7ippYgbPkdMcm68mBm3A+fc/khduZjLcvJKs4jCpw5IC7xMDRWlmul6gOmG7En0uJNIpROcuhNZrYzRFJ1889e6vT6YOmFIJcEw0YvBEPRVIJp8pDurK2EKK5n9FCWTjq07Mi51DbIOoseiMoeCbmnXM+87EOASIk9q3NHprPn9HHDekUR0jWtGZCjUeH0wVagMhPOely6A==;\n\t5:ysvHepHyQ1073pbxzwB1bxQ/R4OGEdiV8qtsYmbuuTs/jmBmSZF2zcXdeIR0116UvashNh9mA5QqGmZpDzfVE9b1k1ghfyDt6QJPdM8xEfSl4xyjeEfpHX1R2gmZ2Ilu+7fs8lLA44R7pGxdWJQaAA==;\n\t24:V7aiNjMGbxF1PlRooyGBhHJSn0nJyV8Ae9YdzcqHJTXiNO4KqRFndZdLq18q0dW4nLzBEZ/QhdkjNvpSgbgsKeHlCKOgV8ZkXbUONbDvqUY=;\n\t7:6HEzCxppZy9jLTqcHzG+rRg30Xu3lyezYZz7dxRu29OTrM6PT3L2gNSIxFJW4XJ2xmwDZPHJqYFECgBpJjgaeqUWI9bG36cAlcOUyxV2J/KK+ehL4UJp7svSSX2swMhV+gpNqxRNkw2bQe8si0IvVeQbgYnth0ioRHPRxVvaD6UrImW2KlueZ1GRe3OywkvkLXLBLK2/HCVms8ChtSjpMO75qyS0jhWtB3k/vg2snT0="],"X-MS-TrafficTypeDiagnostic":"MWHPR0701MB3659:","X-Exchange-Antispam-Report-Test":"UriScan:;","X-Microsoft-Antispam-PRVS":"","X-Exchange-Antispam-Report-CFA-Test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(93006095)(3002001)(10201501046)(6041248)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123555025)(20161123558100)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:MWHPR0701MB3659; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:MWHPR0701MB3659; ","X-Forefront-PRVS":"0430FA5CB7","X-Forefront-Antispam-Report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(6049001)(346002)(376002)(24454002)(199003)(189002)(65956001)(8936002)(7736002)(189998001)(8676002)(305945005)(4326008)(31696002)(23676002)(101416001)(33646002)(81156014)(93886005)(81166006)(8656003)(39060400002)(42882006)(2950100002)(229853002)(6246003)(6666003)(53936002)(5660300001)(7416002)(107886003)(77096006)(2906002)(3846002)(6116002)(36756003)(31686004)(4001350100001)(68736007)(97736004)(316002)(16526017)(16576012)(47776003)(6486002)(230700001)(53546010)(66066001)(106356001)(64126003)(76176999)(54356999)(50986999)(25786009)(83506001)(50466002)(105586002)(72206003)(478600001);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR0701MB3659; H:[10.0.0.85];\n\tFPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; ","Received-SPF":"None (protection.outlook.com: cavium.com does not designate\n\tpermitted sender hosts)","SpamDiagnosticOutput":"1:99","SpamDiagnosticMetadata":"NSPM","X-OriginatorOrg":"caviumnetworks.com","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"14 Sep 2017 14:43:30.4245\n\t(UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"711e4ccf-2e9b-4bcf-a551-4094005b6194","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"MWHPR0701MB3659","X-detected-operating-system":"by eggs.gnu.org: Windows 7 or 8 [fuzzy]","X-Received-From":"104.47.41.55","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\tbharat.bhushan@nxp.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, linu.cherian@cavium.com,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1768985,"web_url":"http://patchwork.ozlabs.org/comment/1768985/","msgid":"<403a9798-fed2-671f-fc6e-7cc0fd53c90c@redhat.com>","list_archive_url":null,"date":"2017-09-15T07:23:57","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","submitter":{"id":69187,"url":"http://patchwork.ozlabs.org/api/people/69187/","name":"Eric Auger","email":"eric.auger@redhat.com"},"content":"Hi Linu,\n\nOn 14/09/2017 11:27, Linu Cherian wrote:\n> Hi Eric,\n> \n> On Fri Sep 01, 2017 at 07:21:16PM +0200, Eric Auger wrote:\n>> memory_region_iommu_replay() is used for VFIO integration.\n>>\n>> However its default implementation is not adapted to SMMUv3\n>> IOMMU memory region. Indeed the input address range is too\n>> huge and its execution is too slow as it calls the translate()\n>> callback on each granule.\n>>\n>> Let's implement the replay callback which hierarchically walk\n>> over the page table structure and notify only the segments\n>> that are populated with valid entries.\n>>\n>> Signed-off-by: Eric Auger \n>> ---\n>> hw/arm/smmuv3.c | 36 ++++++++++++++++++++++++++++++++++++\n>> hw/arm/trace-events | 1 +\n>> 2 files changed, 37 insertions(+)\n>>\n>> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n>> index 8e7d10d..c43bd93 100644\n>> --- a/hw/arm/smmuv3.c\n>> +++ b/hw/arm/smmuv3.c\n>> @@ -657,6 +657,41 @@ static int smmuv3_notify_entry(IOMMUTLBEntry *entry, void *private)\n>> return 0;\n>> }\n>> \n>> +/* Unmap the whole notifier's range */\n>> +static void smmuv3_unmap_notifier_range(IOMMUNotifier *n)\n>> +{\n>> + IOMMUTLBEntry entry;\n>> + hwaddr size = n->end - n->start + 1;\n>> +\n>> + entry.target_as = &address_space_memory;\n>> + entry.iova = n->start & ~(size - 1);\n>> + entry.perm = IOMMU_NONE;\n>> + entry.addr_mask = size - 1;\n>> +\n>> + memory_region_notify_one(n, &entry);\n>> +}\n>> +\n>> +static void smmuv3_replay(IOMMUMemoryRegion *mr, IOMMUNotifier *n)\n>> +{\n>> + SMMUTransCfg cfg = {};\n>> + int ret;\n>> +\n>> + trace_smmuv3_replay(mr->parent_obj.name, n, n->start, n->end);\n>> + smmuv3_unmap_notifier_range(n);\n>> +\n>> + ret = smmuv3_decode_config(mr, &cfg);\n>> + if (ret) {\n>> + error_report(\"%s error decoding the configuration for iommu mr=%s\",\n>> + __func__, mr->parent_obj.name);\n>> + }\n>>\n> \n> On an invalid config being found, shouldnt we return rather than proceeding with\n> page table walk. For example on an invalid Stream table entry.\nYes that's correct. I am going to fix that.\n\nThanks!\n\nEric\n> \n> +\n>> + if (cfg.disabled || cfg.bypassed) {\n>> + return;\n>> + }\n>> + /* walk the page tables and replay valid entries */\n>> + smmu_page_walk(&cfg, 0, (1ULL << (64 - cfg.tsz)) - 1, false,\n>> + smmuv3_notify_entry, n);\n>> +}\n>> static void smmuv3_notify_iova_range(IOMMUMemoryRegion *mr, IOMMUNotifier *n,\n>> uint64_t iova, size_t size)\n>> {\n>> @@ -1095,6 +1130,7 @@ static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass,\n>> \n>> imrc->translate = smmuv3_translate;\n>> imrc->notify_flag_changed = smmuv3_notify_flag_changed;\n>> + imrc->replay = smmuv3_replay;\n>> }\n>> \n>> static const TypeInfo smmuv3_type_info = {\n>> diff --git a/hw/arm/trace-events b/hw/arm/trace-events\n>> index 4ac264d..15f84d6 100644\n>> --- a/hw/arm/trace-events\n>> +++ b/hw/arm/trace-events\n>> @@ -46,5 +46,6 @@ smmuv3_cfg_stage(int s, uint32_t oas, uint32_t tsz, uint64_t ttbr, bool aa64, ui\n>> smmuv3_notify_flag_add(const char *iommu) \"ADD SMMUNotifier node for iommu mr=%s\"\n>> smmuv3_notify_flag_del(const char *iommu) \"DEL SMMUNotifier node for iommu mr=%s\"\n>> smmuv3_replay_mr(const char *name) \"iommu mr=%s\"\n>> +smmuv3_replay(const char *name, void *n, hwaddr start, hwaddr end) \"iommu mr=%s notifier=%p [0x%\"PRIx64\",0x%\"PRIx64\"]\"\n>> smmuv3_notify_entry(hwaddr iova, hwaddr pa, hwaddr mask, int perm) \"iova=0x%\"PRIx64\" pa=0x%\" PRIx64\" mask=0x%\"PRIx64\" perm=%d\"\n>> smmuv3_notify_iova_range(const char *name, uint64_t iova, size_t size, void *n) \"iommu mr=%s iova=0x%\"PRIx64\" size=0x%lx n=%p\"\n>> -- \n>> 2.5.5\n>>\n>>\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx08.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx08.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eric.auger@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtn3Q1Bf3z9t2Z\n\tfor ;\n\tFri, 15 Sep 2017 17:24:48 +1000 (AEST)","from localhost ([::1]:51721 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dskzU-0007kY-Og\n\tfor incoming@patchwork.ozlabs.org; Fri, 15 Sep 2017 03:24:44 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:45346)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dskz4-0007hV-4e\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 03:24:19 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dskz3-0006Sl-0I\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 03:24:18 -0400","from mx1.redhat.com ([209.132.183.28]:25780)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dskyy-0006Pu-7x; Fri, 15 Sep 2017 03:24:12 -0400","from smtp.corp.redhat.com\n\t(int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 537F9C0587C7;\n\tFri, 15 Sep 2017 07:24:10 +0000 (UTC)","from localhost.localdomain (ovpn-116-217.ams2.redhat.com\n\t[10.36.116.217])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 5F9BE7AA71;\n\tFri, 15 Sep 2017 07:23:59 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 537F9C0587C7","To":"Linu Cherian ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-14-git-send-email-eric.auger@redhat.com>\n\t<20170914092751.GA3336@virtx40>","From":"Auger Eric ","Message-ID":"<403a9798-fed2-671f-fc6e-7cc0fd53c90c@redhat.com>","Date":"Fri, 15 Sep 2017 09:23:57 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101\n\tThunderbird/45.4.0","MIME-Version":"1.0","In-Reply-To":"<20170914092751.GA3336@virtx40>","Content-Type":"text/plain; charset=windows-1252","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.13","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.32]);\n\tFri, 15 Sep 2017 07:24:10 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tprem.mallappa@gmail.com, linu.cherian@cavium.com,\n\trobin.murphy@arm.com, eric.auger.pro@gmail.com,\n\tbharat.bhushan@nxp.com, christoffer.dall@linaro.org, wtownsen@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1768990,"web_url":"http://patchwork.ozlabs.org/comment/1768990/","msgid":"<4f9fc697-9e6c-5dd7-65c7-92326135aa56@redhat.com>","list_archive_url":null,"date":"2017-09-15T07:30:26","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","submitter":{"id":69187,"url":"http://patchwork.ozlabs.org/api/people/69187/","name":"Eric Auger","email":"eric.auger@redhat.com"},"content":"Hi Tomasz,\n\nOn 14/09/2017 16:43, Tomasz Nowicki wrote:\n> On 14.09.2017 16:31, Tomasz Nowicki wrote:\n>> Hi Eric,\n>>\n>> On 14.09.2017 11:27, Linu Cherian wrote:\n>>> Hi Eric,\n>>>\n>>> On Fri Sep 01, 2017 at 07:21:16PM +0200, Eric Auger wrote:\n>>>> memory_region_iommu_replay() is used for VFIO integration.\n>>>>\n>>>> However its default implementation is not adapted to SMMUv3\n>>>> IOMMU memory region. Indeed the input address range is too\n>>>> huge and its execution is too slow as it calls the translate()\n>>>> callback on each granule.\n>>>>\n>>>> Let's implement the replay callback which hierarchically walk\n>>>> over the page table structure and notify only the segments\n>>>> that are populated with valid entries.\n>>>>\n>>>> Signed-off-by: Eric Auger \n>>>> ---\n>>>> hw/arm/smmuv3.c | 36 ++++++++++++++++++++++++++++++++++++\n>>>> hw/arm/trace-events | 1 +\n>>>> 2 files changed, 37 insertions(+)\n>>>>\n>>>> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n>>>> index 8e7d10d..c43bd93 100644\n>>>> --- a/hw/arm/smmuv3.c\n>>>> +++ b/hw/arm/smmuv3.c\n>>>> @@ -657,6 +657,41 @@ static int smmuv3_notify_entry(IOMMUTLBEntry\n>>>> *entry, void *private)\n>>>> return 0;\n>>>> }\n>>>> +/* Unmap the whole notifier's range */\n>>>> +static void smmuv3_unmap_notifier_range(IOMMUNotifier *n)\n>>>> +{\n>>>> + IOMMUTLBEntry entry;\n>>>> + hwaddr size = n->end - n->start + 1;\n>>>> +\n>>>> + entry.target_as = &address_space_memory;\n>>>> + entry.iova = n->start & ~(size - 1);\n>>>> + entry.perm = IOMMU_NONE;\n>>>> + entry.addr_mask = size - 1;\n>>>> +\n>>>> + memory_region_notify_one(n, &entry);\n>>>> +}\n>>>> +\n>>>> +static void smmuv3_replay(IOMMUMemoryRegion *mr, IOMMUNotifier *n)\n>>>> +{\n>>>> + SMMUTransCfg cfg = {};\n>>>> + int ret;\n>>>> +\n>>>> + trace_smmuv3_replay(mr->parent_obj.name, n, n->start, n->end);\n>>>> + smmuv3_unmap_notifier_range(n);\n>>>> +\n>>>> + ret = smmuv3_decode_config(mr, &cfg);\n>>>> + if (ret) {\n>>>> + error_report(\"%s error decoding the configuration for iommu\n>>>> mr=%s\",\n>>>> + __func__, mr->parent_obj.name);\n>>>> + }\n>>>>\n>>>\n>>> On an invalid config being found, shouldnt we return rather than\n>>> proceeding with\n>>> page table walk. For example on an invalid Stream table entry.\n>>\n>> Indeed, without return here vhost case is not working for me.\n> \n> I was just lucky one time. return here has no influence. Vhost still not\n> working. Sorry for noise.\n\nAs far as I understand the replay() callback only is called in VFIO use\ncase. So this shouldn't impact vhost.\n\nI can't reproduce your vhost issue on my side. I will review the\ninvalidate code again and compare against the last version.\n\nWhat is the page size used by your guest?\n\nThanks\n\nEric\n> \n> Tomasz\n> \n>>\n>> Thanks,\n>> Tomasz\n>>\n>>>\n>>> +\n>>>> + if (cfg.disabled || cfg.bypassed) {\n>>>> + return;\n>>>> + }\n>>>> + /* walk the page tables and replay valid entries */\n>>>> + smmu_page_walk(&cfg, 0, (1ULL << (64 - cfg.tsz)) - 1, false,\n>>>> + smmuv3_notify_entry, n);\n>>>> +}\n>>>> static void smmuv3_notify_iova_range(IOMMUMemoryRegion *mr,\n>>>> IOMMUNotifier *n,\n>>>> uint64_t iova, size_t size)\n>>>> {\n>>>> @@ -1095,6 +1130,7 @@ static void\n>>>> smmuv3_iommu_memory_region_class_init(ObjectClass *klass,\n>>>> imrc->translate = smmuv3_translate;\n>>>> imrc->notify_flag_changed = smmuv3_notify_flag_changed;\n>>>> + imrc->replay = smmuv3_replay;\n>>>> }\n>>>> static const TypeInfo smmuv3_type_info = {\n>>>> diff --git a/hw/arm/trace-events b/hw/arm/trace-events\n>>>> index 4ac264d..15f84d6 100644\n>>>> --- a/hw/arm/trace-events\n>>>> +++ b/hw/arm/trace-events\n>>>> @@ -46,5 +46,6 @@ smmuv3_cfg_stage(int s, uint32_t oas, uint32_t\n>>>> tsz, uint64_t ttbr, bool aa64, ui\n>>>> smmuv3_notify_flag_add(const char *iommu) \"ADD SMMUNotifier node\n>>>> for iommu mr=%s\"\n>>>> smmuv3_notify_flag_del(const char *iommu) \"DEL SMMUNotifier node\n>>>> for iommu mr=%s\"\n>>>> smmuv3_replay_mr(const char *name) \"iommu mr=%s\"\n>>>> +smmuv3_replay(const char *name, void *n, hwaddr start, hwaddr end)\n>>>> \"iommu mr=%s notifier=%p [0x%\"PRIx64\",0x%\"PRIx64\"]\"\n>>>> smmuv3_notify_entry(hwaddr iova, hwaddr pa, hwaddr mask, int perm)\n>>>> \"iova=0x%\"PRIx64\" pa=0x%\" PRIx64\" mask=0x%\"PRIx64\" perm=%d\"\n>>>> smmuv3_notify_iova_range(const char *name, uint64_t iova, size_t\n>>>> size, void *n) \"iommu mr=%s iova=0x%\"PRIx64\" size=0x%lx n=%p\"\n>>>> -- \n>>>> 2.5.5\n>>>>\n>>>>\n>>>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx03.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx03.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eric.auger@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtnBp1MLYz9sPs\n\tfor ;\n\tFri, 15 Sep 2017 17:31:12 +1000 (AEST)","from localhost ([::1]:51742 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsl5h-0001sm-RL\n\tfor incoming@patchwork.ozlabs.org; Fri, 15 Sep 2017 03:31:09 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:46742)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsl5L-0001pB-34\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 03:30:48 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsl5J-0000HY-Pr\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 03:30:47 -0400","from mx1.redhat.com ([209.132.183.28]:35080)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dsl5E-0000Fy-Iv; Fri, 15 Sep 2017 03:30:40 -0400","from smtp.corp.redhat.com\n\t(int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 6DB5E83F40;\n\tFri, 15 Sep 2017 07:30:39 +0000 (UTC)","from localhost.localdomain (ovpn-116-217.ams2.redhat.com\n\t[10.36.116.217])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 186EB6942B;\n\tFri, 15 Sep 2017 07:30:27 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 6DB5E83F40","To":"Tomasz Nowicki ,\n\tLinu Cherian ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-14-git-send-email-eric.auger@redhat.com>\n\t<20170914092751.GA3336@virtx40>\n\t<4dbbccae-64e1-f4ac-4dcc-3abd9766f324@caviumnetworks.com>\n\t<795d99ba-e492-04ce-dda4-709682baf6cd@caviumnetworks.com>","From":"Auger Eric ","Message-ID":"<4f9fc697-9e6c-5dd7-65c7-92326135aa56@redhat.com>","Date":"Fri, 15 Sep 2017 09:30:26 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101\n\tThunderbird/45.4.0","MIME-Version":"1.0","In-Reply-To":"<795d99ba-e492-04ce-dda4-709682baf6cd@caviumnetworks.com>","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.15","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.27]);\n\tFri, 15 Sep 2017 07:30:39 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\tbharat.bhushan@nxp.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, linu.cherian@cavium.com,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1769000,"web_url":"http://patchwork.ozlabs.org/comment/1769000/","msgid":"","list_archive_url":null,"date":"2017-09-15T07:41:12","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","submitter":{"id":69187,"url":"http://patchwork.ozlabs.org/api/people/69187/","name":"Eric Auger","email":"eric.auger@redhat.com"},"content":"On 15/09/2017 09:30, Auger Eric wrote:\n> Hi Tomasz,\n> \n> On 14/09/2017 16:43, Tomasz Nowicki wrote:\n>> On 14.09.2017 16:31, Tomasz Nowicki wrote:\n>>> Hi Eric,\n>>>\n>>> On 14.09.2017 11:27, Linu Cherian wrote:\n>>>> Hi Eric,\n>>>>\n>>>> On Fri Sep 01, 2017 at 07:21:16PM +0200, Eric Auger wrote:\n>>>>> memory_region_iommu_replay() is used for VFIO integration.\n>>>>>\n>>>>> However its default implementation is not adapted to SMMUv3\n>>>>> IOMMU memory region. Indeed the input address range is too\n>>>>> huge and its execution is too slow as it calls the translate()\n>>>>> callback on each granule.\n>>>>>\n>>>>> Let's implement the replay callback which hierarchically walk\n>>>>> over the page table structure and notify only the segments\n>>>>> that are populated with valid entries.\n>>>>>\n>>>>> Signed-off-by: Eric Auger \n>>>>> ---\n>>>>> hw/arm/smmuv3.c | 36 ++++++++++++++++++++++++++++++++++++\n>>>>> hw/arm/trace-events | 1 +\n>>>>> 2 files changed, 37 insertions(+)\n>>>>>\n>>>>> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n>>>>> index 8e7d10d..c43bd93 100644\n>>>>> --- a/hw/arm/smmuv3.c\n>>>>> +++ b/hw/arm/smmuv3.c\n>>>>> @@ -657,6 +657,41 @@ static int smmuv3_notify_entry(IOMMUTLBEntry\n>>>>> *entry, void *private)\n>>>>> return 0;\n>>>>> }\n>>>>> +/* Unmap the whole notifier's range */\n>>>>> +static void smmuv3_unmap_notifier_range(IOMMUNotifier *n)\n>>>>> +{\n>>>>> + IOMMUTLBEntry entry;\n>>>>> + hwaddr size = n->end - n->start + 1;\n>>>>> +\n>>>>> + entry.target_as = &address_space_memory;\n>>>>> + entry.iova = n->start & ~(size - 1);\n>>>>> + entry.perm = IOMMU_NONE;\n>>>>> + entry.addr_mask = size - 1;\n>>>>> +\n>>>>> + memory_region_notify_one(n, &entry);\n>>>>> +}\n>>>>> +\n>>>>> +static void smmuv3_replay(IOMMUMemoryRegion *mr, IOMMUNotifier *n)\n>>>>> +{\n>>>>> + SMMUTransCfg cfg = {};\n>>>>> + int ret;\n>>>>> +\n>>>>> + trace_smmuv3_replay(mr->parent_obj.name, n, n->start, n->end);\n>>>>> + smmuv3_unmap_notifier_range(n);\n>>>>> +\n>>>>> + ret = smmuv3_decode_config(mr, &cfg);\n>>>>> + if (ret) {\n>>>>> + error_report(\"%s error decoding the configuration for iommu\n>>>>> mr=%s\",\n>>>>> + __func__, mr->parent_obj.name);\n>>>>> + }\n>>>>>\n>>>>\n>>>> On an invalid config being found, shouldnt we return rather than\n>>>> proceeding with\n>>>> page table walk. For example on an invalid Stream table entry.\n>>>\n>>> Indeed, without return here vhost case is not working for me.\n>>\n>> I was just lucky one time. return here has no influence. Vhost still not\n>> working. Sorry for noise.\n> \n> As far as I understand the replay() callback only is called in VFIO use\n> case. So this shouldn't impact vhost.\nForget that, that's potentially called from some invalidation commands\nalso in vhost case.\n\nThanks\n\nEric\n> \n> I can't reproduce your vhost issue on my side. I will review the\n> invalidate code again and compare against the last version.\n> \n> What is the page size used by your guest?\n> \n> Thanks\n> \n> Eric\n>>\n>> Tomasz\n>>\n>>>\n>>> Thanks,\n>>> Tomasz\n>>>\n>>>>\n>>>> +\n>>>>> + if (cfg.disabled || cfg.bypassed) {\n>>>>> + return;\n>>>>> + }\n>>>>> + /* walk the page tables and replay valid entries */\n>>>>> + smmu_page_walk(&cfg, 0, (1ULL << (64 - cfg.tsz)) - 1, false,\n>>>>> + smmuv3_notify_entry, n);\n>>>>> +}\n>>>>> static void smmuv3_notify_iova_range(IOMMUMemoryRegion *mr,\n>>>>> IOMMUNotifier *n,\n>>>>> uint64_t iova, size_t size)\n>>>>> {\n>>>>> @@ -1095,6 +1130,7 @@ static void\n>>>>> smmuv3_iommu_memory_region_class_init(ObjectClass *klass,\n>>>>> imrc->translate = smmuv3_translate;\n>>>>> imrc->notify_flag_changed = smmuv3_notify_flag_changed;\n>>>>> + imrc->replay = smmuv3_replay;\n>>>>> }\n>>>>> static const TypeInfo smmuv3_type_info = {\n>>>>> diff --git a/hw/arm/trace-events b/hw/arm/trace-events\n>>>>> index 4ac264d..15f84d6 100644\n>>>>> --- a/hw/arm/trace-events\n>>>>> +++ b/hw/arm/trace-events\n>>>>> @@ -46,5 +46,6 @@ smmuv3_cfg_stage(int s, uint32_t oas, uint32_t\n>>>>> tsz, uint64_t ttbr, bool aa64, ui\n>>>>> smmuv3_notify_flag_add(const char *iommu) \"ADD SMMUNotifier node\n>>>>> for iommu mr=%s\"\n>>>>> smmuv3_notify_flag_del(const char *iommu) \"DEL SMMUNotifier node\n>>>>> for iommu mr=%s\"\n>>>>> smmuv3_replay_mr(const char *name) \"iommu mr=%s\"\n>>>>> +smmuv3_replay(const char *name, void *n, hwaddr start, hwaddr end)\n>>>>> \"iommu mr=%s notifier=%p [0x%\"PRIx64\",0x%\"PRIx64\"]\"\n>>>>> smmuv3_notify_entry(hwaddr iova, hwaddr pa, hwaddr mask, int perm)\n>>>>> \"iova=0x%\"PRIx64\" pa=0x%\" PRIx64\" mask=0x%\"PRIx64\" perm=%d\"\n>>>>> smmuv3_notify_iova_range(const char *name, uint64_t iova, size_t\n>>>>> size, void *n) \"iommu mr=%s iova=0x%\"PRIx64\" size=0x%lx n=%p\"\n>>>>> -- \n>>>>> 2.5.5\n>>>>>\n>>>>>\n>>>>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx08.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx08.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eric.auger@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtnRG5b0nz9sPs\n\tfor ;\n\tFri, 15 Sep 2017 17:42:02 +1000 (AEST)","from localhost ([::1]:51772 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dslGC-0007Y2-VL\n\tfor incoming@patchwork.ozlabs.org; Fri, 15 Sep 2017 03:42:00 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:49618)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dslFn-0007SS-CY\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 03:41:36 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dslFm-000689-0d\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 03:41:35 -0400","from mx1.redhat.com ([209.132.183.28]:59461)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dslFf-0005sH-Dd; Fri, 15 Sep 2017 03:41:27 -0400","from smtp.corp.redhat.com\n\t(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 49E8CC057FA6;\n\tFri, 15 Sep 2017 07:41:26 +0000 (UTC)","from localhost.localdomain (ovpn-116-217.ams2.redhat.com\n\t[10.36.116.217])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id B33D878DC6;\n\tFri, 15 Sep 2017 07:41:14 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 49E8CC057FA6","To":"Tomasz Nowicki ,\n\tLinu Cherian ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-14-git-send-email-eric.auger@redhat.com>\n\t<20170914092751.GA3336@virtx40>\n\t<4dbbccae-64e1-f4ac-4dcc-3abd9766f324@caviumnetworks.com>\n\t<795d99ba-e492-04ce-dda4-709682baf6cd@caviumnetworks.com>\n\t<4f9fc697-9e6c-5dd7-65c7-92326135aa56@redhat.com>","From":"Auger Eric ","Message-ID":"","Date":"Fri, 15 Sep 2017 09:41:12 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101\n\tThunderbird/45.4.0","MIME-Version":"1.0","In-Reply-To":"<4f9fc697-9e6c-5dd7-65c7-92326135aa56@redhat.com>","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.12","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.32]);\n\tFri, 15 Sep 2017 07:41:26 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\tbharat.bhushan@nxp.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, linu.cherian@cavium.com,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1769183,"web_url":"http://patchwork.ozlabs.org/comment/1769183/","msgid":"","list_archive_url":null,"date":"2017-09-15T13:19:26","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","submitter":{"id":69187,"url":"http://patchwork.ozlabs.org/api/people/69187/","name":"Eric Auger","email":"eric.auger@redhat.com"},"content":"Hi Tomasz,\nOn 15/09/2017 12:42, tn wrote:\n> Hi Eric,\n> \n> On 15.09.2017 09:30, Auger Eric wrote:\n>> Hi Tomasz,\n>>\n>> On 14/09/2017 16:43, Tomasz Nowicki wrote:\n>>> On 14.09.2017 16:31, Tomasz Nowicki wrote:\n>>>> Hi Eric,\n>>>>\n>>>> On 14.09.2017 11:27, Linu Cherian wrote:\n>>>>> Hi Eric,\n>>>>>\n>>>>> On Fri Sep 01, 2017 at 07:21:16PM +0200, Eric Auger wrote:\n>>>>>> memory_region_iommu_replay() is used for VFIO integration.\n>>>>>>\n>>>>>> However its default implementation is not adapted to SMMUv3\n>>>>>> IOMMU memory region. Indeed the input address range is too\n>>>>>> huge and its execution is too slow as it calls the translate()\n>>>>>> callback on each granule.\n>>>>>>\n>>>>>> Let's implement the replay callback which hierarchically walk\n>>>>>> over the page table structure and notify only the segments\n>>>>>> that are populated with valid entries.\n>>>>>>\n>>>>>> Signed-off-by: Eric Auger \n>>>>>> ---\n>>>>>> hw/arm/smmuv3.c | 36 ++++++++++++++++++++++++++++++++++++\n>>>>>> hw/arm/trace-events | 1 +\n>>>>>> 2 files changed, 37 insertions(+)\n>>>>>>\n>>>>>> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n>>>>>> index 8e7d10d..c43bd93 100644\n>>>>>> --- a/hw/arm/smmuv3.c\n>>>>>> +++ b/hw/arm/smmuv3.c\n>>>>>> @@ -657,6 +657,41 @@ static int smmuv3_notify_entry(IOMMUTLBEntry\n>>>>>> *entry, void *private)\n>>>>>> return 0;\n>>>>>> }\n>>>>>> +/* Unmap the whole notifier's range */\n>>>>>> +static void smmuv3_unmap_notifier_range(IOMMUNotifier *n)\n>>>>>> +{\n>>>>>> + IOMMUTLBEntry entry;\n>>>>>> + hwaddr size = n->end - n->start + 1;\n>>>>>> +\n>>>>>> + entry.target_as = &address_space_memory;\n>>>>>> + entry.iova = n->start & ~(size - 1);\n>>>>>> + entry.perm = IOMMU_NONE;\n>>>>>> + entry.addr_mask = size - 1;\n>>>>>> +\n>>>>>> + memory_region_notify_one(n, &entry);\n>>>>>> +}\n>>>>>> +\n>>>>>> +static void smmuv3_replay(IOMMUMemoryRegion *mr, IOMMUNotifier *n)\n>>>>>> +{\n>>>>>> + SMMUTransCfg cfg = {};\n>>>>>> + int ret;\n>>>>>> +\n>>>>>> + trace_smmuv3_replay(mr->parent_obj.name, n, n->start, n->end);\n>>>>>> + smmuv3_unmap_notifier_range(n);\n>>>>>> +\n>>>>>> + ret = smmuv3_decode_config(mr, &cfg);\n>>>>>> + if (ret) {\n>>>>>> + error_report(\"%s error decoding the configuration for iommu\n>>>>>> mr=%s\",\n>>>>>> + __func__, mr->parent_obj.name);\n>>>>>> + }\n>>>>>>\n>>>>>\n>>>>> On an invalid config being found, shouldnt we return rather than\n>>>>> proceeding with\n>>>>> page table walk. For example on an invalid Stream table entry.\n>>>>\n>>>> Indeed, without return here vhost case is not working for me.\n>>>\n>>> I was just lucky one time. return here has no influence. Vhost still not\n>>> working. Sorry for noise.\n>>\n>> As far as I understand the replay() callback only is called in VFIO use\n>> case. So this shouldn't impact vhost.\n>>\n>> I can't reproduce your vhost issue on my side. I will review the\n>> invalidate code again and compare against the last version.\n>>\n>> What is the page size used by your guest?\n> \n> 64K page size for guest as well as for host.\n> \n> However, I've just checked 4K page size for guest and then vhost is\n> working fine.\nI can reproduce the issue with vhost on 64KB page guest. Currently\ninvestigating...\n\nThanks!\n\nEric\n> \n> Thanks,\n> Tomasz","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx01.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx01.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eric.auger@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtx2x75Yfz9sRm\n\tfor ;\n\tFri, 15 Sep 2017 23:24:57 +1000 (AEST)","from localhost ([::1]:53397 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsqc4-0001xY-3A\n\tfor incoming@patchwork.ozlabs.org; Fri, 15 Sep 2017 09:24:56 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:43430)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsqX3-0006DX-3b\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 09:19:46 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsqX1-00062j-Vr\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 09:19:45 -0400","from mx1.redhat.com ([209.132.183.28]:41272)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dsqWx-00061Y-6c; Fri, 15 Sep 2017 09:19:39 -0400","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 2948981DFC;\n\tFri, 15 Sep 2017 13:19:38 +0000 (UTC)","from localhost.localdomain (ovpn-116-217.ams2.redhat.com\n\t[10.36.116.217])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id AC130757BC;\n\tFri, 15 Sep 2017 13:19:27 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 2948981DFC","To":"tn ,\n\tTomasz Nowicki ,\n\tLinu Cherian ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-14-git-send-email-eric.auger@redhat.com>\n\t<20170914092751.GA3336@virtx40>\n\t<4dbbccae-64e1-f4ac-4dcc-3abd9766f324@caviumnetworks.com>\n\t<795d99ba-e492-04ce-dda4-709682baf6cd@caviumnetworks.com>\n\t<4f9fc697-9e6c-5dd7-65c7-92326135aa56@redhat.com>\n\t","From":"Auger Eric ","Message-ID":"","Date":"Fri, 15 Sep 2017 15:19:26 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101\n\tThunderbird/45.4.0","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.14","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.25]);\n\tFri, 15 Sep 2017 13:19:38 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\tbharat.bhushan@nxp.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, linu.cherian@cavium.com,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1769225,"web_url":"http://patchwork.ozlabs.org/comment/1769225/","msgid":"","list_archive_url":null,"date":"2017-09-15T10:42:42","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","submitter":{"id":72373,"url":"http://patchwork.ozlabs.org/api/people/72373/","name":"Tomasz Nowicki","email":"Tomasz.Nowicki@caviumnetworks.com"},"content":"Hi Eric,\n\nOn 15.09.2017 09:30, Auger Eric wrote:\n> Hi Tomasz,\n> \n> On 14/09/2017 16:43, Tomasz Nowicki wrote:\n>> On 14.09.2017 16:31, Tomasz Nowicki wrote:\n>>> Hi Eric,\n>>>\n>>> On 14.09.2017 11:27, Linu Cherian wrote:\n>>>> Hi Eric,\n>>>>\n>>>> On Fri Sep 01, 2017 at 07:21:16PM +0200, Eric Auger wrote:\n>>>>> memory_region_iommu_replay() is used for VFIO integration.\n>>>>>\n>>>>> However its default implementation is not adapted to SMMUv3\n>>>>> IOMMU memory region. Indeed the input address range is too\n>>>>> huge and its execution is too slow as it calls the translate()\n>>>>> callback on each granule.\n>>>>>\n>>>>> Let's implement the replay callback which hierarchically walk\n>>>>> over the page table structure and notify only the segments\n>>>>> that are populated with valid entries.\n>>>>>\n>>>>> Signed-off-by: Eric Auger \n>>>>> ---\n>>>>> hw/arm/smmuv3.c | 36 ++++++++++++++++++++++++++++++++++++\n>>>>> hw/arm/trace-events | 1 +\n>>>>> 2 files changed, 37 insertions(+)\n>>>>>\n>>>>> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n>>>>> index 8e7d10d..c43bd93 100644\n>>>>> --- a/hw/arm/smmuv3.c\n>>>>> +++ b/hw/arm/smmuv3.c\n>>>>> @@ -657,6 +657,41 @@ static int smmuv3_notify_entry(IOMMUTLBEntry\n>>>>> *entry, void *private)\n>>>>> return 0;\n>>>>> }\n>>>>> +/* Unmap the whole notifier's range */\n>>>>> +static void smmuv3_unmap_notifier_range(IOMMUNotifier *n)\n>>>>> +{\n>>>>> + IOMMUTLBEntry entry;\n>>>>> + hwaddr size = n->end - n->start + 1;\n>>>>> +\n>>>>> + entry.target_as = &address_space_memory;\n>>>>> + entry.iova = n->start & ~(size - 1);\n>>>>> + entry.perm = IOMMU_NONE;\n>>>>> + entry.addr_mask = size - 1;\n>>>>> +\n>>>>> + memory_region_notify_one(n, &entry);\n>>>>> +}\n>>>>> +\n>>>>> +static void smmuv3_replay(IOMMUMemoryRegion *mr, IOMMUNotifier *n)\n>>>>> +{\n>>>>> + SMMUTransCfg cfg = {};\n>>>>> + int ret;\n>>>>> +\n>>>>> + trace_smmuv3_replay(mr->parent_obj.name, n, n->start, n->end);\n>>>>> + smmuv3_unmap_notifier_range(n);\n>>>>> +\n>>>>> + ret = smmuv3_decode_config(mr, &cfg);\n>>>>> + if (ret) {\n>>>>> + error_report(\"%s error decoding the configuration for iommu\n>>>>> mr=%s\",\n>>>>> + __func__, mr->parent_obj.name);\n>>>>> + }\n>>>>>\n>>>>\n>>>> On an invalid config being found, shouldnt we return rather than\n>>>> proceeding with\n>>>> page table walk. For example on an invalid Stream table entry.\n>>>\n>>> Indeed, without return here vhost case is not working for me.\n>>\n>> I was just lucky one time. return here has no influence. Vhost still not\n>> working. Sorry for noise.\n> \n> As far as I understand the replay() callback only is called in VFIO use\n> case. So this shouldn't impact vhost.\n> \n> I can't reproduce your vhost issue on my side. I will review the\n> invalidate code again and compare against the last version.\n> \n> What is the page size used by your guest?\n\n64K page size for guest as well as for host.\n\nHowever, I've just checked 4K page size for guest and then vhost is \nworking fine.\n\nThanks,\nTomasz","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=CAVIUMNETWORKS.onmicrosoft.com\n\theader.i=@CAVIUMNETWORKS.onmicrosoft.com header.b=\"Xa4JFbr7\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=Tomasz.Nowicki@cavium.com; "],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtymY2WGdz9t2S\n\tfor ;\n\tSat, 16 Sep 2017 00:42:37 +1000 (AEST)","from localhost ([::1]:53634 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsrpD-00056N-5L\n\tfor incoming@patchwork.ozlabs.org; Fri, 15 Sep 2017 10:42:35 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:45742)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dso5W-0005Hz-1T\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 06:43:11 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dso5V-0005Ux-1J\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 06:43:10 -0400","from mail-sn1nam02on0073.outbound.protection.outlook.com\n\t([104.47.36.73]:34368\n\thelo=NAM02-SN1-obe.outbound.protection.outlook.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dso5P-0005FM-0J; Fri, 15 Sep 2017 06:43:03 -0400","from [192.168.2.100] (87.116.213.6) by\n\tMWHPR0701MB3660.namprd07.prod.outlook.com (2603:10b6:301:7e::11) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.56.11;\n\tFri, 15 Sep 2017 10:42:53 +0000"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=spz7t0l1GH2Fl60ZDSOezExc+2ukxISzKjkdv3QW0Yw=;\n\tb=Xa4JFbr7DdyW4l0qRHWI+MVgOOnK71W3vc2YnnrJfWxKlyrCjUuxFouA5iQX4ZOkK7cHoMmJ5q5f1T1RC261y6opf/yqaYKXz8Zd+b8DO8YQQBGj3pko5sLRq4owxMS4ip4JmsA5gPMzhydjsmZ1XYV5UpD6QaglZGCtrxxTtBE=","To":"Auger Eric ,\n\tTomasz Nowicki ,\n\tLinu Cherian ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-14-git-send-email-eric.auger@redhat.com>\n\t<20170914092751.GA3336@virtx40>\n\t<4dbbccae-64e1-f4ac-4dcc-3abd9766f324@caviumnetworks.com>\n\t<795d99ba-e492-04ce-dda4-709682baf6cd@caviumnetworks.com>\n\t<4f9fc697-9e6c-5dd7-65c7-92326135aa56@redhat.com>","From":"tn ","Date":"Fri, 15 Sep 2017 12:42:42 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<4f9fc697-9e6c-5dd7-65c7-92326135aa56@redhat.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-GB","Content-Transfer-Encoding":"7bit","X-Originating-IP":"[87.116.213.6]","X-ClientProxiedBy":"VI1P18901CA0013.EURP189.PROD.OUTLOOK.COM\n\t(2603:10a6:801::23) To MWHPR0701MB3660.namprd07.prod.outlook.com\n\t(2603:10b6:301:7e::11)","Message-ID":"","X-MS-PublicTrafficType":"Email","X-MS-Office365-Filtering-Correlation-Id":"b39c88d6-2581-4ad8-7665-08d4fc268850","X-Microsoft-Antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:MWHPR0701MB3660; ","X-Microsoft-Exchange-Diagnostics":["1; MWHPR0701MB3660;\n\t3:1W4De4YQQMP+9vdwyN64zSbmfdYcPLApURGrPxtyryV3nDni8IBtzb1ufrNR8AtJqEE/9NAPsqLE9CNkThQNOkbqf2qwkibl5Eq5JXtmZFnzYR/wj5+evwAPW8fJI3P0c73Y6ZpRX+1j5Epfm+azAsIq2hseQuV0hrJNxLnfRQjd//jAxebyVJeIpbo/FvLvm9X3jDdZWrePC3iBwZn09cwUfLKuhkHYy6GmMXFwvC6LwEoNeRdwupBrR8mGcVbV;\n\t25:jKf3kToqBvTTL0MHIbOM5xO3HOHY7/16iObtCI9ewlGf8+LpXQbhThXofdy6JAxL4DFBcBSk6JSR+pB+Oyq7t9w9WJLta+QSI1leD24UIP11S7nGCgOsD1GQhpJ9FwqNJEsUaAWJPrqf8skzhuXQpkI3Y4mcS+tMeJCe8dgvjXKzW5YqgbqmQjRS8ESt9JKBsMlsuoGnEG95OUSdrSyVpv7gIdShAxOZnQ7JdCTY+twc5NThUiEwlWPy70cNIdQdR5w3Qs+qL0doGwK1POdzmPifJJ6RE3M14ackfM068W8RqguLdg07a9fBJmBKrHYP6kjzLpEZQsEAg3ugt1v04A==;\n\t31:odaPbyxJiHeDN9/RncRt4mDXaYVobxMjaFSF3Ymc62tGa+CQgcTwpXdS5i9seAx/0Jsnj5SizTj13HYWPMqK3/eaQudgw9sMygQLeyldb2xkxtz1GTvHiOwkzSG2mnkYhsd57B2tP1IA8x68S7d/dswzqQwQ4QieVp4J3HB4x3efZaQdPtE+Zvi7wNTE3Zmqn2CU+SKaC+bCWRKyC+oXVjuMVywSMDwABIKy8EXNH28=","1; MWHPR0701MB3660;\n\t20:/LkcHQegdr1oCafVQxOAQZmu9cNoR9dgJK9cupBwm966+dT9loQD1hMBCpiG0Qa0F/SqqQCuFByMgoKFm1wXZa5gGyyYzHK8Pl2Y7IvQYOJzTfIvSIssIsgCTcDVxcFCTgD8ukq/c22074c7LBM5DIIHjIf03zkF1D//XpztvsidBmz4zsRP4Vs/hA7wjVQM1b50vY8LTmL1iJN4YbWA4A7YyCAR9cYFpOiALA/ld//fy+Cv2i5+ZiapXPi0h9zJmXpaRFF2fMfv28/Ea1MEeUykGfuOQ2Evm4ca04N1czWIAjOwybqx1bqLb5GtntsOcoiqDmPKfoz7LLIBDmlfIvepwzlj9edNT4NCRu3BeyXrFlda9NnFwQMrxqVD7WjwqjDxH7k/fxvjs91C7UEuBV7Ve022OP3w419onm1jy/AjtfizvsvNGReHJHM6Ve58gNR0LdAudFSA7CEZORGUXDIei1Om8MzEu59YUn0vMQIMXhOeHo5enqVUKLRj9v6WejqvQKnvg9InGry+4tg+d1bflg59yxIMB7KNRSiZRY7w3hYE0aBipuHFiNWmIUeNdBqOL24b7DBQoBDkgcRzBccL9vYd2OKt8lOHz6g/TLc=;\n\t4:qh7JPJbl5UU2ebJdy+N7CjZrou0m3LglyPLOEyx8+DXYYp3pp14ca8Xbbk95M2IovIm8ZUSpr5rCaly/TUqZ8UYOc1d8YXgZlN2CuEcWwrbMtVSoljpu0s2/D2Wv14g991cB+XGSiPj8Be9VEEavO5z5or66U8RQm/IG9pPWEtaBEErGtUim6vWuow3a+ti3eqiV1wyWjy9/RJKzk5ncHFOW1BFODfJLnjUFKQ5Y17zvAwdiMBQ9CKGPDvIaZK8D","=?utf-8?q?1=3BMWHPR0701MB3660=3B23=3AEw?=\n\t=?utf-8?q?rj/EodkK5gvRzvYzONirLWk/b4TP72UWtUWW5/9xAjlOgGxuFp/HDUj3?=\n\t=?utf-8?q?gJxNRIDiB/wS7w9aRaRbSmQOgtU4gw21wm6qyN4NNb3l845BPGfYvPPI?=\n\t=?utf-8?q?Y5UXZp65O00Qag4jE0eZZziKRubPMfuW+X7oxNVAgvXEMwqQ220Qz8zr?=\n\t=?utf-8?q?Zl14ZN7+S3qAaKYXr2HJLV/Bg3n+a8m6F32OumugF4lU65hp/t5HmDIy?=\n\t=?utf-8?q?2j1g3z0TCgafX8vuvdV+LwQjYIi+NveIc2e5G1360UUf5+jnHZ5Yv1za?=\n\t=?utf-8?q?3YGFWWv60Lw43S9B85qnF2RB7deZ+e7JdYZDa2J5lE82cvaYrEQ8G02R?=\n\t=?utf-8?q?MtVlQOVlen5FOn8v1EPfksyLDbfuUw+XlTkVKsd9VGhzi1gBsvWFhldi?=\n\t=?utf-8?q?7rbz1yHFDOU0sA6mJGefpUDj9YaItUHepizKTmfKaKvFGxu9OVEKVO+m?=\n\t=?utf-8?q?BT3UOBeHwxqQE4zAeOssq6s/EhZVIQYpO979ZpjCvDogk/xFPhWW2pbr?=\n\t=?utf-8?q?RIoeVfXxVVjdx0PRqKfulDyhqLUZ9ySpxJPP2QPGBtLnVU9uCXhClH5E?=\n\t=?utf-8?q?ZDaafw7kIoSTFxjJISJjtVTSXZ64zPmlzXAqWceFeg6DUfVV3tO+OjHb?=\n\t=?utf-8?q?ASRtHIqpYPwTgPN1AwRPCkWiAhhHr2pgyu07hMtypq4GVXct+DDuPL3Z?=\n\t=?utf-8?q?mBIZDNh/J9RIfg2EzyCkIcnMw7WeB9xP4zydyt+A/RTGryK9uyeQeeco?=\n\t=?utf-8?q?Z26cpgHyPphQLmhibu7Gp5zQmBRnBwxwZEpJFI3/5Dfl6FnJC+xyYyEE?=\n\t=?utf-8?q?ubQRezEzpKBsdYKL3Yw4M3w7IU3vH3YZPOVavAvCWaH9fsbhAhBC3fII?=\n\t=?utf-8?q?h0hGHQB07/k+Bd66XFXtm4nCIAM0ODZsqLvTg5wiH/vhZdaHHwMLqnph?=\n\t=?utf-8?q?pufOncwoAczruFyXZ/N7898bG7gd+nQBf3Wf9P8tqRUFQzDq6N9lJMz8?=\n\t=?utf-8?q?w3FdyiB6HVqdWLNaNxved94+owEEfiHlYnTMGkXvQTt8TjjCJ0wKLsCy?=\n\t=?utf-8?q?50ReZjcM8tmCelHlA+1SI4sYbmputRE7gvEug7RfPaEVMY4CshygiuEP?=\n\t=?utf-8?q?lBwjdxxnTd8rQ0dU/S8vit/CKUilOpCA5kPvKY4j+v8kGcyAamHr3T7U?=\n\t=?utf-8?q?gZQUWTmFVdb8kodep1ndgA1C+E37Xg0BUjweCj7VWVr52iZe9PsQ2i/9?=\n\t=?utf-8?q?LOiRW2tXFY5c4cG/mfJKgW6jGP5Ewnm+5lIV5fg9YnM/2GiI5U7Lw6D1?=\n\t=?utf-8?q?tF6JwU5kdEM76dAWSojHbxircQaDU3xnqOLMrqz3b5qRTlQYG7ySl3jy?=\n\t=?utf-8?q?bA9vsFRdRMZloWynNa7++x+8iLsSHsYzH0Ro9O//W7ROW64/KYwCbbaR?=\n\t=?utf-8?q?hJ3vnEq3fQO4PHwuEj3AcKUYQxj+CdPwh29exTJkpNj4lMiWxPm7siqO?=\n\t=?utf-8?q?2QiONDo/ONHaQd3mlwC3RLdGhTdXIm6ub027mdPnhahBxpu0t/kVXytN?=\n\t=?utf-8?q?q5VYaYA2iXGf+3?=","1; MWHPR0701MB3660;\n\t6:BrrBB2qYlf3NhSCRzBfypWtc3Of7fZDOTlplbqw/UgHgbg8391rY4diUeizTuAKlDKwsR+pkf4tzb5nwvAGpdM+FD5Eu4ArlJgqFblNAYnjzxOFq++uixadIA8iubDYoUiJe9CRDwgPV+ykRjLLAI3NTn2Y/q7EOCfAAAv5dUytatpfySP0BYessCIYlUAHSRk8zXhLn8DeuoNNxELArPzFeQ4aG4Vn+cNeyUXbG6NnyIbUuSswGZ4m8drLS+DlS0ge7Tix70gJy6S9Ajq8wG39X21YzpP+JlYG2qZxed/t9mbeSW4vQHS6/hk+y9gZuHVC4ekZ+1ZWpPw+3zgLPzQ==;\n\t5:O/Wuq4r1B+fO+tSShZ3sltuOyVJBYoQlliLuD6GUkS6XDj5IVDbK4UU/fyJ43XKonkQ3wGzdM8XMg75JLdDLdngRoM/XFsZzUyq6h9+ugkBK2G/eOzblww1XJaGD2XH3T7gh4o9297NkoRSCLdr+jA==;\n\t24:HXrVSbzpjb+GbPCF0utqSgiSUVYIRSHluVZhQIr13mm1xzdwcFY900yX5HsjS6X+3GAfMK1RPMiQJHkHt/x4qDJ1+7R3eSp1z/aRUW9MHeQ=;\n\t7:SEtGX/GJ+uFX9aruFMEjLxLfRRBDM0FbYWJCfhWz2Z9Rs4czCTL05ti133upj3W8CJseCV9yEc837fgyX//81M3dTGx83olPBkJ5CbCX7Xr/04boS/x0jL3aGjAWA46VCL66pn56Pg/j8QgWE2O9ARwfHC2TgovyqeAsc53d+ULQt6t/VikupcD27t2i0oF8Ipn1wwq7bYibsCz602Np7B84QjQOwAKUPDfzcP+G1e0="],"X-MS-TrafficTypeDiagnostic":"MWHPR0701MB3660:","X-Exchange-Antispam-Report-Test":"UriScan:;","X-Microsoft-Antispam-PRVS":"","X-Exchange-Antispam-Report-CFA-Test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(3002001)(100000703101)(100105400095)(93006095)(6041248)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123562025)(20161123564025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:MWHPR0701MB3660; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:MWHPR0701MB3660; ","X-Forefront-PRVS":"0431F981D8","X-Forefront-Antispam-Report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(6049001)(346002)(376002)(189002)(199003)(24454002)(305945005)(33656002)(7416002)(2906002)(2950100002)(6666003)(42882006)(6486002)(74316002)(229853002)(77096006)(65826007)(8676002)(81166006)(81156014)(5660300001)(8936002)(50466002)(64126003)(23676002)(66066001)(47776003)(65806001)(65956001)(3846002)(6116002)(7736002)(6246003)(101416001)(106356001)(76176999)(54356999)(50986999)(105586002)(230700001)(53546010)(86152003)(53936002)(8656003)(25786009)(4326008)(39060400002)(68736007)(107886003)(58126008)(189998001)(9686003)(83506001)(97736004)(93886005)(72206003)(16576012)(316002)(478600001)(117156002)(65966003)(43062003);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR0701MB3660;\n\tH:[192.168.2.100]; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1;\n\tLANG:en; ","Received-SPF":"None (protection.outlook.com: cavium.com does not designate\n\tpermitted sender hosts)","SpamDiagnosticOutput":"1:99","SpamDiagnosticMetadata":"NSPM","X-OriginatorOrg":"caviumnetworks.com","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"15 Sep 2017 10:42:53.7169\n\t(UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"711e4ccf-2e9b-4bcf-a551-4094005b6194","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"MWHPR0701MB3660","X-detected-operating-system":"by eggs.gnu.org: Windows 7 or 8 [fuzzy]","X-Received-From":"104.47.36.73","X-Mailman-Approved-At":"Fri, 15 Sep 2017 10:39:38 -0400","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\tbharat.bhushan@nxp.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, linu.cherian@cavium.com,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1769229,"web_url":"http://patchwork.ozlabs.org/comment/1769229/","msgid":"<64e946b3-8fb6-b918-e389-1a01c084fb47@redhat.com>","list_archive_url":null,"date":"2017-09-15T14:50:47","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","submitter":{"id":69187,"url":"http://patchwork.ozlabs.org/api/people/69187/","name":"Eric Auger","email":"eric.auger@redhat.com"},"content":"Hi,\n\nOn 15/09/2017 12:42, tn wrote:\n> Hi Eric,\n> \n> On 15.09.2017 09:30, Auger Eric wrote:\n>> Hi Tomasz,\n>>\n>> On 14/09/2017 16:43, Tomasz Nowicki wrote:\n>>> On 14.09.2017 16:31, Tomasz Nowicki wrote:\n>>>> Hi Eric,\n>>>>\n>>>> On 14.09.2017 11:27, Linu Cherian wrote:\n>>>>> Hi Eric,\n>>>>>\n>>>>> On Fri Sep 01, 2017 at 07:21:16PM +0200, Eric Auger wrote:\n>>>>>> memory_region_iommu_replay() is used for VFIO integration.\n>>>>>>\n>>>>>> However its default implementation is not adapted to SMMUv3\n>>>>>> IOMMU memory region. Indeed the input address range is too\n>>>>>> huge and its execution is too slow as it calls the translate()\n>>>>>> callback on each granule.\n>>>>>>\n>>>>>> Let's implement the replay callback which hierarchically walk\n>>>>>> over the page table structure and notify only the segments\n>>>>>> that are populated with valid entries.\n>>>>>>\n>>>>>> Signed-off-by: Eric Auger \n>>>>>> ---\n>>>>>> hw/arm/smmuv3.c | 36 ++++++++++++++++++++++++++++++++++++\n>>>>>> hw/arm/trace-events | 1 +\n>>>>>> 2 files changed, 37 insertions(+)\n>>>>>>\n>>>>>> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n>>>>>> index 8e7d10d..c43bd93 100644\n>>>>>> --- a/hw/arm/smmuv3.c\n>>>>>> +++ b/hw/arm/smmuv3.c\n>>>>>> @@ -657,6 +657,41 @@ static int smmuv3_notify_entry(IOMMUTLBEntry\n>>>>>> *entry, void *private)\n>>>>>> return 0;\n>>>>>> }\n>>>>>> +/* Unmap the whole notifier's range */\n>>>>>> +static void smmuv3_unmap_notifier_range(IOMMUNotifier *n)\n>>>>>> +{\n>>>>>> + IOMMUTLBEntry entry;\n>>>>>> + hwaddr size = n->end - n->start + 1;\n>>>>>> +\n>>>>>> + entry.target_as = &address_space_memory;\n>>>>>> + entry.iova = n->start & ~(size - 1);\n>>>>>> + entry.perm = IOMMU_NONE;\n>>>>>> + entry.addr_mask = size - 1;\n>>>>>> +\n>>>>>> + memory_region_notify_one(n, &entry);\n>>>>>> +}\n>>>>>> +\n>>>>>> +static void smmuv3_replay(IOMMUMemoryRegion *mr, IOMMUNotifier *n)\n>>>>>> +{\n>>>>>> + SMMUTransCfg cfg = {};\n>>>>>> + int ret;\n>>>>>> +\n>>>>>> + trace_smmuv3_replay(mr->parent_obj.name, n, n->start, n->end);\n>>>>>> + smmuv3_unmap_notifier_range(n);\n>>>>>> +\n>>>>>> + ret = smmuv3_decode_config(mr, &cfg);\n>>>>>> + if (ret) {\n>>>>>> + error_report(\"%s error decoding the configuration for iommu\n>>>>>> mr=%s\",\n>>>>>> + __func__, mr->parent_obj.name);\n>>>>>> + }\n>>>>>>\n>>>>>\n>>>>> On an invalid config being found, shouldnt we return rather than\n>>>>> proceeding with\n>>>>> page table walk. For example on an invalid Stream table entry.\n>>>>\n>>>> Indeed, without return here vhost case is not working for me.\n>>>\n>>> I was just lucky one time. return here has no influence. Vhost still not\n>>> working. Sorry for noise.\n>>\n>> As far as I understand the replay() callback only is called in VFIO use\n>> case. So this shouldn't impact vhost.\n>>\n>> I can't reproduce your vhost issue on my side. I will review the\n>> invalidate code again and compare against the last version.\n>>\n>> What is the page size used by your guest?\n> \n> 64K page size for guest as well as for host.\n> \n> However, I've just checked 4K page size for guest and then vhost is\n> working fine.\nSo the bug stems from the incorrect target page size used on\nSMMU_CMD_TLBI_NH_VA invalidation. This is now corrected by using the\nactual config granule size and this fixes the issue with vhost use case\nand 64KB page guest. I will release this fix early next week. Sorry for\nthe inconvenience.\n\nThanks\n\nEric\n> \n> Thanks,\n> Tomasz","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx10.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx10.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eric.auger@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtyz150PPz9sBZ\n\tfor ;\n\tSat, 16 Sep 2017 00:51:41 +1000 (AEST)","from localhost ([::1]:53696 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsrxz-0003TO-T8\n\tfor incoming@patchwork.ozlabs.org; Fri, 15 Sep 2017 10:51:39 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:39950)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsrxb-0003Rb-N2\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 10:51:18 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsrxV-0002mt-Mo\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 10:51:15 -0400","from mx1.redhat.com ([209.132.183.28]:52730)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dsrxQ-0002k2-Np; Fri, 15 Sep 2017 10:51:04 -0400","from smtp.corp.redhat.com\n\t(int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 38D5961472;\n\tFri, 15 Sep 2017 14:51:03 +0000 (UTC)","from localhost.localdomain (ovpn-116-197.ams2.redhat.com\n\t[10.36.116.197])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 6F0D571C95;\n\tFri, 15 Sep 2017 14:50:49 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 38D5961472","To":"tn ,\n\tTomasz Nowicki ,\n\tLinu Cherian ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-14-git-send-email-eric.auger@redhat.com>\n\t<20170914092751.GA3336@virtx40>\n\t<4dbbccae-64e1-f4ac-4dcc-3abd9766f324@caviumnetworks.com>\n\t<795d99ba-e492-04ce-dda4-709682baf6cd@caviumnetworks.com>\n\t<4f9fc697-9e6c-5dd7-65c7-92326135aa56@redhat.com>\n\t","From":"Auger Eric ","Message-ID":"<64e946b3-8fb6-b918-e389-1a01c084fb47@redhat.com>","Date":"Fri, 15 Sep 2017 16:50:47 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101\n\tThunderbird/45.4.0","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.16","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.39]);\n\tFri, 15 Sep 2017 14:51:03 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\tbharat.bhushan@nxp.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, linu.cherian@cavium.com,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770032,"web_url":"http://patchwork.ozlabs.org/comment/1770032/","msgid":"","list_archive_url":null,"date":"2017-09-18T09:50:39","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","submitter":{"id":72364,"url":"http://patchwork.ozlabs.org/api/people/72364/","name":"Tomasz Nowicki","email":"tnowicki@caviumnetworks.com"},"content":"Hi Eric,\n\nOn 15.09.2017 16:50, Auger Eric wrote:\n> Hi,\n> \n> On 15/09/2017 12:42, tn wrote:\n>> Hi Eric,\n>>\n>> On 15.09.2017 09:30, Auger Eric wrote:\n>>> Hi Tomasz,\n>>>\n>>> On 14/09/2017 16:43, Tomasz Nowicki wrote:\n>>>> On 14.09.2017 16:31, Tomasz Nowicki wrote:\n>>>>> Hi Eric,\n>>>>>\n>>>>> On 14.09.2017 11:27, Linu Cherian wrote:\n>>>>>> Hi Eric,\n>>>>>>\n>>>>>> On Fri Sep 01, 2017 at 07:21:16PM +0200, Eric Auger wrote:\n>>>>>>> memory_region_iommu_replay() is used for VFIO integration.\n>>>>>>>\n>>>>>>> However its default implementation is not adapted to SMMUv3\n>>>>>>> IOMMU memory region. Indeed the input address range is too\n>>>>>>> huge and its execution is too slow as it calls the translate()\n>>>>>>> callback on each granule.\n>>>>>>>\n>>>>>>> Let's implement the replay callback which hierarchically walk\n>>>>>>> over the page table structure and notify only the segments\n>>>>>>> that are populated with valid entries.\n>>>>>>>\n>>>>>>> Signed-off-by: Eric Auger \n>>>>>>> ---\n>>>>>>> hw/arm/smmuv3.c | 36 ++++++++++++++++++++++++++++++++++++\n>>>>>>> hw/arm/trace-events | 1 +\n>>>>>>> 2 files changed, 37 insertions(+)\n>>>>>>>\n>>>>>>> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n>>>>>>> index 8e7d10d..c43bd93 100644\n>>>>>>> --- a/hw/arm/smmuv3.c\n>>>>>>> +++ b/hw/arm/smmuv3.c\n>>>>>>> @@ -657,6 +657,41 @@ static int smmuv3_notify_entry(IOMMUTLBEntry\n>>>>>>> *entry, void *private)\n>>>>>>> return 0;\n>>>>>>> }\n>>>>>>> +/* Unmap the whole notifier's range */\n>>>>>>> +static void smmuv3_unmap_notifier_range(IOMMUNotifier *n)\n>>>>>>> +{\n>>>>>>> + IOMMUTLBEntry entry;\n>>>>>>> + hwaddr size = n->end - n->start + 1;\n>>>>>>> +\n>>>>>>> + entry.target_as = &address_space_memory;\n>>>>>>> + entry.iova = n->start & ~(size - 1);\n>>>>>>> + entry.perm = IOMMU_NONE;\n>>>>>>> + entry.addr_mask = size - 1;\n>>>>>>> +\n>>>>>>> + memory_region_notify_one(n, &entry);\n>>>>>>> +}\n>>>>>>> +\n>>>>>>> +static void smmuv3_replay(IOMMUMemoryRegion *mr, IOMMUNotifier *n)\n>>>>>>> +{\n>>>>>>> + SMMUTransCfg cfg = {};\n>>>>>>> + int ret;\n>>>>>>> +\n>>>>>>> + trace_smmuv3_replay(mr->parent_obj.name, n, n->start, n->end);\n>>>>>>> + smmuv3_unmap_notifier_range(n);\n>>>>>>> +\n>>>>>>> + ret = smmuv3_decode_config(mr, &cfg);\n>>>>>>> + if (ret) {\n>>>>>>> + error_report(\"%s error decoding the configuration for iommu\n>>>>>>> mr=%s\",\n>>>>>>> + __func__, mr->parent_obj.name);\n>>>>>>> + }\n>>>>>>>\n>>>>>>\n>>>>>> On an invalid config being found, shouldnt we return rather than\n>>>>>> proceeding with\n>>>>>> page table walk. For example on an invalid Stream table entry.\n>>>>>\n>>>>> Indeed, without return here vhost case is not working for me.\n>>>>\n>>>> I was just lucky one time. return here has no influence. Vhost still not\n>>>> working. Sorry for noise.\n>>>\n>>> As far as I understand the replay() callback only is called in VFIO use\n>>> case. So this shouldn't impact vhost.\n>>>\n>>> I can't reproduce your vhost issue on my side. I will review the\n>>> invalidate code again and compare against the last version.\n>>>\n>>> What is the page size used by your guest?\n>>\n>> 64K page size for guest as well as for host.\n>>\n>> However, I've just checked 4K page size for guest and then vhost is\n>> working fine.\n> So the bug stems from the incorrect target page size used on\n> SMMU_CMD_TLBI_NH_VA invalidation. This is now corrected by using the\n> actual config granule size and this fixes the issue with vhost use case\n> and 64KB page guest. I will release this fix early next week. Sorry for\n> the inconvenience.\n> \n\nYes, that was it. I will provide my t-b for the next series version.\n\nThanks,\nTomasz","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=CAVIUMNETWORKS.onmicrosoft.com\n\theader.i=@CAVIUMNETWORKS.onmicrosoft.com header.b=\"n+zMNGe2\"; \n\tdkim-atps=neutral","spf=none (sender IP is )\n\tsmtp.mailfrom=Tomasz.Nowicki@cavium.com; "],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwh9K19Nnz9ryv\n\tfor ;\n\tMon, 18 Sep 2017 19:51:33 +1000 (AEST)","from localhost ([::1]:35499 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dtsi8-0003pE-Au\n\tfor incoming@patchwork.ozlabs.org; Mon, 18 Sep 2017 05:51:28 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:34168)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtshk-0003nk-Cm\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 05:51:05 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dtshj-0000HV-8v\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 05:51:04 -0400","from mail-co1nam03on0082.outbound.protection.outlook.com\n\t([104.47.40.82]:42016\n\thelo=NAM03-CO1-obe.outbound.protection.outlook.com)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dtshc-0000Fe-I2; Mon, 18 Sep 2017 05:50:56 -0400","from [192.168.46.114] (12.108.191.226) by\n\tSN4PR0701MB3662.namprd07.prod.outlook.com (2603:10b6:803:4d::16) with\n\tMicrosoft SMTP Server (version=TLS1_2,\n\tcipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.56.11;\n\tMon, 18 Sep 2017 09:50:44 +0000"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com;\n\th=From:Date:Subject:Message-ID:Content-Type:MIME-Version;\n\tbh=6CiN6Y9Ne9MO3ssU3ZJiX28TxEe+t+B+KxnQU3w2axM=;\n\tb=n+zMNGe2qqFJvpA1F/NBnXnYi9dCC6+zepX48p4x0CrswI/wT+wwoUVEUbf8SUA5KuFHJ26SbvrRyC7zutDJXpKU4pR4FFIyP9ScVQhkrsnpgV0l+yGzmyUug96VmvhM35Pii5zLJU6VW7fZYXO1KzheDAGomg3S6LDehsRQBKk=","To":"Auger Eric ,\n\ttn , \n\tLinu Cherian ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-14-git-send-email-eric.auger@redhat.com>\n\t<20170914092751.GA3336@virtx40>\n\t<4dbbccae-64e1-f4ac-4dcc-3abd9766f324@caviumnetworks.com>\n\t<795d99ba-e492-04ce-dda4-709682baf6cd@caviumnetworks.com>\n\t<4f9fc697-9e6c-5dd7-65c7-92326135aa56@redhat.com>\n\t\n\t<64e946b3-8fb6-b918-e389-1a01c084fb47@redhat.com>","From":"Tomasz Nowicki ","Message-ID":"","Date":"Mon, 18 Sep 2017 11:50:39 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 Thunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<64e946b3-8fb6-b918-e389-1a01c084fb47@redhat.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Language":"en-GB","Content-Transfer-Encoding":"7bit","X-Originating-IP":"[12.108.191.226]","X-ClientProxiedBy":"MWHPR15CA0050.namprd15.prod.outlook.com\n\t(2603:10b6:301:4c::12) To SN4PR0701MB3662.namprd07.prod.outlook.com\n\t(2603:10b6:803:4d::16)","X-MS-PublicTrafficType":"Email","X-MS-Office365-Filtering-Correlation-Id":"7884db3c-ca36-40d4-1e16-08d4fe7abf0a","X-Microsoft-Antispam":"UriScan:; BCL:0; PCL:0;\n\tRULEID:(300000500095)(300135000095)(300000501095)(300135300095)(300000502095)(300135100095)(22001)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);\n\tSRVR:SN4PR0701MB3662; ","X-Microsoft-Exchange-Diagnostics":["1; SN4PR0701MB3662;\n\t3:3DnxDSWzKnH8LdF5zLsjInaf+ipqy/RIKb5yiRk6iSPl+X2LaBqTLZIxsaSEq7u/kEsTzzGueKcDotWzMU4pvpPEk1ITHtVNZk4S2mFnQTuFgGwXKLSoqouXnw5F46MPZWpU1daQsJrKkQicHY3d2ZaGt/Bl+4eHQupXi/ULoPeeAMpm+POdr/x2bNZ4XGaejAgvHx+GNUaAFqJ9xG5QQmzYI3JGX6WeLWsVwUpINyLIqmHvecdXiWGkdrgqy48W;\n\t25:NFV7X7KhwiDcrV332r6T991mnx8VsdICcvd83DICvIhKhQ81O5SENsnbsp2BH4pPA+S9ljoV8pm1fQSeQWZ6Pa1fAgE3qz1osclL6v99Ww3te6YjzYtsSG25GpaPYeOf1rsqmiC1PRj5zAS0VAik+CeTt8LGfgjjZJh6l57bFYkj9ujxmOphC2PbDyC9GY0Ug4yDA7eKtHq7Mm9O35lUU9RM39rIcZ42qGoTRNqrop4HOJ2OLoswB4VHQWRWd/99tg3XnMwSYyHk6dH9Gj4gh5S6WueeL2k9k8REu/sJaLxNuuBa+K/gbJsssNdxdswUq6MH1hu6Ca2bBwhY32JZag==;\n\t31:KftmGaqSP3/xKI2mabrU4LpU6oRQL2UpQZwnAv6DibT/EzFfjxW+/bnJr72YUi+qwWDRnYkYztVsYPNITYucVhLGpplRx3uTlqOEr9CPBIcwpVsNoPx50JYsF4kjJpOa6/lqTsBdVHAa0pEx3uCxFfz2TL/bykVO5VX6mNvtwmxaB6VzdlJYVOjghb9nvL6r7v1bRxqoBz383FDkXxTcoEj9Mw/j25Vw1bg623FIJfc=","1; SN4PR0701MB3662;\n\t20:VpFbfLQjKWxgGLvFMjHFjuKR8ndkgQ5A/rFrw6PBUojlKhV2hRp5Ls+wamMPgtQRSGlhgLmJLhUIAKLAagEuRDiX4bp5b/CHBj777k7N7DJGVhJAAouSuU5aYt54Ov9yn1u0+1eCTYLaQwTqMtO6gqwjyYDoqs8AFbsVJvy1utzZJTw9uAcxx2n7SlhxIiZ+JV/i16PrtNuW3AcGTg94hdS5S1b5LNQXRXBw55PW1GY11PBI+NwLM9AbzJQ13AI+7twCV6b5HHjhiLQl9Xd+ANq7Kzn2uAh9mI7RYIXk8LisauF9FNu3xQMU454cSNh2yTNFdn8z+iWWGMnD3NGaDP+NO2p3FI6WcK3lnSLU8zNi0BdRmL8oDuSsKIbyaE2qY0molBsaSy2y1Pc/ZIr99XCSOACf4qNJewJULKtlxFOhYqHzzG+SCSN6Rzt5nJdtbKJAPy5x3Otvkq70tejyTlRTm12pWkSrSdF1PVGwOWgkLpshsF++hCrOVH9sqTVjfJK0isEvoJdHJuUDJTxm8/+7fDM0AriYoMJjZx3VeYqA1PsRkT9es5a/9NVAiiGhJvjL2fmRU8FlyVj7AGMYzlIUfl4knpPkReYsU6Kwb44=;\n\t4:ojYDQMH218u/POwRvdazrVZ6rN+FqKvOrqo+5ZlzA3hU+hSx4+8k95/BwMO//DEiquSOeQx8fF5JhCAWa/K5MASlI59IBEtW6BSxxBw+XlFhK88zf1GMmicwU3KDAKYSLpZ5E8KiHHNr2JTeYDYzFTUlRAjqRzpC9av7d1xxR0xad0uyEvkMSLaF9z5O6vDvc6YfIohwq4EXQ3+7tDdfI/Cjp3Ywtng0YcQHnFf/v4qamlHjaYIDFiX2nN3LAX/p","=?utf-8?q?1=3BSN4PR0701MB3662=3B23=3Ac0?=\n\t=?utf-8?q?mWSOepJv7gaGG34xvqXZ7zjo1NHS6QScGIXhv7fOYs+eL1DczxU4CB6P?=\n\t=?utf-8?q?ffGYn8LRO1ud9kAoppNSg84tsU0boEyWLutJj+03NUX0FWc6t6TuPGUY?=\n\t=?utf-8?q?5kjlXMXqn6QvvcEak+/iNaXHEzU1EmdMJMcCVj5JFMqZQGgQYH9TFPdD?=\n\t=?utf-8?q?B7ZWDmqHuSoe4wYa16KhYhCYM2Cf/n7f+6JAMz4ztu2wctwfzE7oeDtS?=\n\t=?utf-8?q?cmtAH51CioudaNqH3i5QAX6Xj+1JiMJvxoQGGW80txwv6qXSUKj6GCxG?=\n\t=?utf-8?q?cs1s38HABrf/KFscjzI76ZH2hUA51sh+qBWrppDEE1Rxw3/Sl32qMf97?=\n\t=?utf-8?q?nWXxNVoFC9d7BCJ0KiJ4gBfhhMHOi0q2ugMnL3V3Yl5BJsfaINjcEn7d?=\n\t=?utf-8?q?CeoMZQLD7SUn/JQGEU3hNDbAk49WBSBKWMcFlC7o2YYsCfawUpqNXrpm?=\n\t=?utf-8?q?Swrv6R+Y4uBalMao5Oyk79vfg9FxR20RRk9MuQ9Zg+kyAPLS77neBRUZ?=\n\t=?utf-8?q?xRhCQ1cZWf9RX5IGVQMUbKVB6tziYmxkiC8Jt84yzgEXNUhsF/hC0ZX/?=\n\t=?utf-8?q?GXHz6Kr/e46FOA07i/NploPl4D6WVtrHiKQ+UsmtVmPQ2jsyqYUC3LY+?=\n\t=?utf-8?q?IvRw/nhjQX+2GFtKPweZ7QBrpC/DCobCTQgmQHPNBbBtdbtM4+KNgyse?=\n\t=?utf-8?q?4lRiJVRxWrV9ncQNC0g5ot1wdLegbXuipn1KtL3OqI9qYxDThtU8JcIo?=\n\t=?utf-8?q?aBavbZYJKwGn1qRhhKnA9XpqTvKELMKCZJU1X9V2hC1SAVqTyq5RsB0m?=\n\t=?utf-8?q?hi/Il7Afh9i6ST0NzJFGlNiFZVXrp8WxcoVcQXa+ImgjC2/BJoW0HVnT?=\n\t=?utf-8?q?RjyLhq/jXicmH6lXTAYsdRAwTM/S5ynU1gPL7Si6JxmEI+WO5fSnWjUQ?=\n\t=?utf-8?q?9u2tCLy5pt4n/LJcAg+E1Jnl+Fyx7y+tp79uwAGqtbvA4c27AzTuDd9r?=\n\t=?utf-8?q?06u+9OVb8dZbeFmSt0u78kl5hXalS9QDmxjpO2zO11PorbTiuR22v/qC?=\n\t=?utf-8?q?U78m69I6+9QBLK5tU/+Nm9DlFoiAZ1kIjIUQCfzDq2iba/c6arrPctfl?=\n\t=?utf-8?q?WG1hUtZW+a22safgLsYp2Diapap8M2/zMkwYUJvUpFltT1x+IhUyH08H?=\n\t=?utf-8?q?5ddcMKG+hfCYEoWtTXZ2ZtJRzqs4cSJrNQJVGZrMDPju7aotgLgs8wHJ?=\n\t=?utf-8?q?b1XSyInbowHPOQNdYg6g5L/C3wOOlJc8B1lTOgRIE1o1/3IFmF29Ulxz?=\n\t=?utf-8?q?GxNl58z313+LPxHeEHnxNdWhBRv7UFw/Txl5Rg9NRQMeKyHtTIVQApYV?=\n\t=?utf-8?q?sWC9MmIV/qRAJPi9TMTsyHLXFKJxpAupnOBFWCp1JWL4b54sENfTSmlu?=\n\t=?utf-8?q?2fWZyIqwKKpUNLM5Ga0xixVumejK1CnbMjCN9sdCkv6sIwqLwMrygxLg?=\n\t=?utf-8?q?6NUF1s?=","1; SN4PR0701MB3662;\n\t6:L5+9SJ+mTjRBKDwiLz3enU0n84LPTyiz9nuLHm2eR8ennncbP+v2BmP9QLRT/0AdHOYutlxXWjlzLCS6b6sY0NXtujQeWLNwA4DLyiwwFLzwqhmyDToqmVzkX6UlZ8JzntfH0WU4cLSjxGeVfo2fLBIqD4siS15Lu9Q3de4tRmxJHTTcdWJM1H1/4Ts7dk6if+MJkxHoBZRWTJlisvLhHuEzVD586AK+jcSx7UksnzQb38PFy1w3B8ikaTyAvWWGExy5uiecnR/RMYPAAQ1Rb7EoFEd6420WOZIu0KGY/O+/Tp47/qf6IKEzA9FGNCOQS3G44xRM2ESFqVTz9VBCVQ==;\n\t5:q/xjHJwcRoUtVct3WoxT4zrX8cuH13+tfyd2ytDnVI/raHg2YiL8qxUKXTm/Nme786cxfHhU3bvQvtGa9D22bRnp6fW7nV5AP5FYv1QqEXfCd9swkVhTFZtD9T+MxuEDMgcDnnS6YV2mvNei+4AAAw==;\n\t24:ehd91IDpQipdKqL30dosAJXR+XmDK1Nz5tw7inPdueYmuJkmYi8HaHldbmcjxXV692fyd7wQsMq8AgZ6mHmmd9vEdrPAbwT4yzqXMdhAz7c=;\n\t7:MLBQ6PJvQpaS5nIEth5/xJeuxKYORrHzG7fqguC21XFG2SfvUpwtAlSwpUe+yZ2VmNzZcBU9Yd0uTY6HiUZaZoC1I6j0wClGci5kwtT30xmV48DdqmPDz2EOU6XGukf99Z/vwBQ2BSW+Hsq4q20vcoAQdPSLpzvMmtCpxM+23h8Fl08Pxa8y+L0FOmjYH4Xkto6Uku7MZmR+0OUaTyIIzwShwdTHWggDjKjOyWrYs1o="],"X-MS-TrafficTypeDiagnostic":"SN4PR0701MB3662:","X-Exchange-Antispam-Report-Test":"UriScan:;","X-Microsoft-Antispam-PRVS":"","X-Exchange-Antispam-Report-CFA-Test":"BCL:0; PCL:0;\n\tRULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(93006095)(10201501046)(100000703101)(100105400095)(3002001)(6041248)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123555025)(20161123560025)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);\n\tSRVR:SN4PR0701MB3662; BCL:0; PCL:0;\n\tRULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);\n\tSRVR:SN4PR0701MB3662; ","X-Forefront-PRVS":"04347F8039","X-Forefront-Antispam-Report":"SFV:NSPM;\n\tSFS:(10009020)(6009001)(6049001)(376002)(346002)(199003)(189002)(24454002)(5660300001)(8936002)(8676002)(36756003)(6666003)(81156014)(81166006)(31696002)(189998001)(50466002)(7416002)(64126003)(6486002)(77096006)(229853002)(4326008)(33646002)(106356001)(39060400002)(2950100002)(42882006)(105586002)(68736007)(107886003)(53936002)(6246003)(65956001)(66066001)(230700001)(8656003)(2906002)(47776003)(54356999)(50986999)(76176999)(23676002)(31686004)(117156002)(25786009)(53546010)(83506001)(72206003)(478600001)(97736004)(316002)(58126008)(16576012)(16526017)(101416001)(3846002)(305945005)(6116002)(93886005)(7736002);\n\tDIR:OUT; SFP:1101; SCL:1; SRVR:SN4PR0701MB3662;\n\tH:[192.168.46.114]; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1;\n\tLANG:en; ","Received-SPF":"None (protection.outlook.com: cavium.com does not designate\n\tpermitted sender hosts)","SpamDiagnosticOutput":"1:99","SpamDiagnosticMetadata":"NSPM","X-OriginatorOrg":"caviumnetworks.com","X-MS-Exchange-CrossTenant-OriginalArrivalTime":"18 Sep 2017 09:50:44.8783\n\t(UTC)","X-MS-Exchange-CrossTenant-FromEntityHeader":"Hosted","X-MS-Exchange-CrossTenant-Id":"711e4ccf-2e9b-4bcf-a551-4094005b6194","X-MS-Exchange-Transport-CrossTenantHeadersStamped":"SN4PR0701MB3662","X-detected-operating-system":"by eggs.gnu.org: Windows 7 or 8 [fuzzy]","X-Received-From":"104.47.40.82","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 13/20] hw/arm/smmuv3:\n\tImplement IOMMU memory region replay callback","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\tbharat.bhushan@nxp.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, linu.cherian@cavium.com,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1776795,"web_url":"http://patchwork.ozlabs.org/comment/1776795/","msgid":"<20170928064312.GA1544@virtx40>","list_archive_url":null,"date":"2017-09-28T06:43:12","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 00/20] ARM SMMUv3 Emulation\n\tSupport","submitter":{"id":72195,"url":"http://patchwork.ozlabs.org/api/people/72195/","name":"Linu Cherian","email":"linuc.decode@gmail.com"},"content":"Hi Eric,\n\n\nOn Fri Sep 01, 2017 at 07:21:03PM +0200, Eric Auger wrote:\n> This series implements the emulation code for ARM SMMUv3.\n> \n> Changes since v6:\n> - DPDK testpmd now running on guest with 2 assigned VFs\n> - Changed the instantiation method: add the following option to\n> the QEMU command line\n> -device smmuv3 # for virtio/vhost use cases\n> -device smmuv3,caching-mode # for vfio use cases (based on [1])\n> - splitted the series into smaller patches to allow the review\n> - the VFIO integration based on \"tlbi-on-map\" smmuv3 driver\n> is isolated from the rest: last 2 patches, not for upstream.\n> This is shipped for testing/bench until a better solution is found.\n> - Reworked permission flag checks and event generation\n> e testing:\n> - in dt and ACPI modes\n> - virtio-net-pci and vhost-net devices using dma ops with various\n> guest page sizes [2]\n> - assigned VFs using dma ops [3]:\n> - AMD Overdrive and igbvf passthrough (using gsi direct mapping)\n> - Cavium ThunderX and ixgbevf passthrough (using KVM MSI routing)\n> - DPDK testpmd on guest running with VFIO user space drivers (2 igbvf) [3]\n> with guest and host page size equal (4kB)\n> \n> Known limitations:\n> - no VMSAv8-32 suport\n> - no nested stage support (S1 + S2)\n> - no support for HYP mappings\n> - register fine emulation, commands, interrupts and errors were\n> not accurately tested. Handling is sufficient to run use cases\n> described above though.\n> - interrupts and event generation not observed yet.\n\nWhile testing with vfio-pci, observed that the below two Qemu command,\nresults in two different behaviour. Is this expected by design ?\n\nCase 1:\n# -device vfio-pci,host=0002:01:00.3 -device smmuv3,caching-mode\n Here iommu is not attached to the pci bus in Qemu backend, since\n pci_setup_iommu is not called before vfio_realize.\n\nCase 2:\n# -device smmuv3,caching-mode -device vfio-pci,host=0002:01:00.3\nThis works as expected, iommu is attached to the pci bus.\n\n> \n> Best Regards\n> \n> Eric\n> \n> This series can be found at:\n> v7: https://github.com/eauger/qemu/tree/v2.10.0-SMMU-v7\n> Previous version at:\n> v6: https://github.com/eauger/qemu/tree/v2.10.0-rc2-SMMU-v6\n> \n> References:\n> [1] [RFC v2 0/4] arm-smmu-v3 tlbi-on-map option\n> https://lkml.org/lkml/2017/8/11/426\n> \n> [2] qemu cmd line excerpt:\n> -device smmuv3 \\\n> -netdev tap,id=tap0,script=no,downscript=no,ifname=tap0,vhost=off \\\n> -device virtio-net-pci,netdev=tap0,mac=6a:f5:10:b1:3d:d2,iommu_platform,disable-modern=off,disable-legacy=on \\\n> [3] use -device smmuv3,caching-mode\n> \n> \n> History:\n> v6 -> v7:\n> - see above\n> \n> v5 -> v6:\n> - Rebase on 2.10 and IOMMUMemoryRegion\n> - add ACPI TLBI_ON_MAP support (VFIO integration also works in\n> ACPI mode)\n> - fix block replay\n> - handle implementation defined SMMU_CMD_TLBI_NH_VA_AM cmd\n> (goes along with TLBI_ON_MAP FW quirk)\n> - replay systematically unmap the whole range first\n> - smmuv3_map_hook does not unmap anymore and the unmap is done\n> before the replay\n> - add and use smmuv3_context_device_invalidate instead of\n> blindly replaying everything\n> \n> v4 -> v5:\n> - initial_level now part of SMMUTransCfg\n> - smmu_page_walk_64 takes into account the max input size\n> - implement sys->iommu_ops.replay and sys->iommu_ops.notify_flag_changed\n> - smmuv3_translate: bug fix: don't walk on bypass\n> - smmu_update_qreg: fix PROD index update\n> - I did not yet address Peter's comments as the code is not mature enough\n> to be split into sub patches.\n> \n> v3 -> v4 [Eric]:\n> - page table walk rewritten to allow scan of the page table within a\n> range of IOVA. This prepares for VFIO integration and replay.\n> - configuration parsing partially reworked.\n> - do not advertise unsupported/untested features: S2, S1 + S2, HYP,\n> PRI, ATS, ..\n> - added ACPI table generation\n> - migrated to dynamic traces\n> - mingw compilation fix\n> \n> v2 -> v3 [Eric]:\n> - rebased on 2.9\n> - mostly code and patch reorganization to ease the review process\n> - optional patches removed. They may be handled separately. I am currently\n> working on ACPI enablement.\n> - optional instantiation of the smmu in mach-virt\n> - removed [2/9] (fdt functions) since not mandated\n> - start splitting main patch into base and derived object\n> - no new function feature added\n> \n> v1 -> v2 [Prem]:\n> - Adopted review comments from Eric Auger\n> - Make SMMU_DPRINTF to internally call qemu_log\n> (since translation requests are too many, we need control\n> on the type of log we want)\n> - SMMUTransCfg modified to suite simplicity\n> - Change RegInfo to uint64 register array\n> - Code cleanup\n> - Test cleanups\n> - Reshuffled patches\n> \n> v0 -> v1 [Prem]:\n> - As per SMMUv3 spec 16.0 (only is_ste_consistant() is noticeable)\n> - Reworked register access/update logic\n> - Factored out translation code for\n> - single point bug fix\n> - sharing/removal in future\n> - (optional) Unit tests added, with PCI test device\n> - S1 with 4k/64k, S1+S2 with 4k/64k\n> - (S1 or S2) only can be verified by Linux 4.7 driver\n> - (optional) Priliminary ACPI support\n> \n> v0 [Prem]:\n> - Implements SMMUv3 spec 11.0\n> - Supported for PCIe devices,\n> - Command Queue and Event Queue supported\n> - LPAE only, S1 is supported and Tested, S2 not tested\n> - BE mode Translation not supported\n> - IRQ support (legacy, no MSI)\n> \n> Eric Auger (18):\n> hw/arm/smmu-common: smmu base device and datatypes\n> hw/arm/smmu-common: IOMMU memory region and address space setup\n> hw/arm/smmu-common: smmu_read/write_sysmem\n> hw/arm/smmu-common: VMSAv8-64 page table walk\n> hw/arm/smmuv3: Wired IRQ and GERROR helpers\n> hw/arm/smmuv3: Queue helpers\n> hw/arm/smmuv3: Implement MMIO write operations\n> hw/arm/smmuv3: Event queue recording helper\n> hw/arm/smmuv3: Implement translate callback\n> target/arm/kvm: Translate the MSI doorbell in kvm_arch_fixup_msi_route\n> hw/arm/smmuv3: Implement data structure and TLB invalidation\n> notifications\n> hw/arm/smmuv3: Implement IOMMU memory region replay callback\n> hw/arm/virt: Store the PCI host controller dt phandle\n> hw/arm/sysbus-fdt: Pass the VirtMachineState to the node creation\n> functions\n> hw/arm/sysbus-fdt: Pass the platform bus base address in\n> PlatformBusFDTData\n> hw/arm/sysbus-fdt: Allow smmuv3 dynamic instantiation\n> hw/arm/smmuv3: [not for upstream] add SMMU_CMD_TLBI_NH_VA_AM handling\n> hw/arm/smmuv3: [not for upstream] Add caching-mode option\n> \n> Prem Mallappa (2):\n> hw/arm/smmuv3: Skeleton\n> hw/arm/virt-acpi-build: Add smmuv3 node in IORT table\n> \n> default-configs/aarch64-softmmu.mak | 1 +\n> hw/arm/Makefile.objs | 1 +\n> hw/arm/smmu-common.c | 527 ++++++++++++++++\n> hw/arm/smmu-internal.h | 105 ++++\n> hw/arm/smmuv3-internal.h | 584 +++++++++++++++++\n> hw/arm/smmuv3.c | 1181 +++++++++++++++++++++++++++++++++++\n> hw/arm/sysbus-fdt.c | 129 +++-\n> hw/arm/trace-events | 48 ++\n> hw/arm/virt-acpi-build.c | 63 +-\n> hw/arm/virt.c | 6 +-\n> include/hw/acpi/acpi-defs.h | 15 +\n> include/hw/arm/smmu-common.h | 123 ++++\n> include/hw/arm/smmuv3.h | 80 +++\n> include/hw/arm/sysbus-fdt.h | 2 +\n> include/hw/arm/virt.h | 15 +\n> target/arm/kvm.c | 27 +\n> target/arm/trace-events | 3 +\n> 17 files changed, 2886 insertions(+), 24 deletions(-)\n> create mode 100644 hw/arm/smmu-common.c\n> create mode 100644 hw/arm/smmu-internal.h\n> create mode 100644 hw/arm/smmuv3-internal.h\n> create mode 100644 hw/arm/smmuv3.c\n> create mode 100644 include/hw/arm/smmu-common.h\n> create mode 100644 include/hw/arm/smmuv3.h\n> \n> -- \n> 2.5.5\n> \n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"oDH18Bdh\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y2lXw3SShz9s9Y\n\tfor ;\n\tThu, 28 Sep 2017 16:44:32 +1000 (AEST)","from localhost ([::1]:57635 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dxSYg-0008EK-LC\n\tfor incoming@patchwork.ozlabs.org; Thu, 28 Sep 2017 02:44:30 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:53961)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dxSYD-0008BM-7Y\n\tfor qemu-devel@nongnu.org; Thu, 28 Sep 2017 02:44:04 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dxSY8-0007vj-I3\n\tfor qemu-devel@nongnu.org; Thu, 28 Sep 2017 02:44:01 -0400","from mail-pf0-x244.google.com ([2607:f8b0:400e:c00::244]:37575)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1dxSY3-0007tX-6p; Thu, 28 Sep 2017 02:43:51 -0400","by mail-pf0-x244.google.com with SMTP id e69so648494pfg.4;\n\tWed, 27 Sep 2017 23:43:51 -0700 (PDT)","from virtx40 ([111.93.218.67]) by smtp.gmail.com with ESMTPSA id\n\tj68sm1427449pfa.93.2017.09.27.23.43.42\n\t(version=TLS1_2 cipher=AES128-SHA bits=128/128);\n\tWed, 27 Sep 2017 23:43:48 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=bjLXDkWUc/jPKkd1DYRmYv0XxBJNzxPbm+J0WcKkOUc=;\n\tb=oDH18BdhZaXVKx5mdSzCMDf+Vig5pStEkw2fFwQujZ6vAWUmjnJofYFiB+1k0ItR9K\n\trsVT4nHXGaK+qY8RZ028ExsvLJMD+xcHetTKhQ0vjA+01YT4gHzjqpJfXS9KkZGzhL05\n\tJYm206Th/37yvkgZrvz9jKjAk+P/hDpQLXEDJGCrNrI3NYUVMpOPV+sKhdgwi5zDFUvG\n\tV+q2t5koCaXHxkPN6lVb+oXgH1bdbd4YudZCC6ihj7T+cev8sit9iwyw6kPxTd8eIckY\n\t64l2/fgYwtUYMULfecqqmlVzRRUrcXfsJ1eAaAj1PXDoaG2MRyJa4xJ9DOvj2EMey7+7\n\tikuQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=bjLXDkWUc/jPKkd1DYRmYv0XxBJNzxPbm+J0WcKkOUc=;\n\tb=fY/cDMVGYsDHNHVsXf4kkPC6+I8KAGnIfkGAqHOL1sGygOttwR8Zoc2jYU0aMJRl4c\n\tctWyzWWPiGALGghFADfynCPwHpspNJ2sr44Ryi5LAeuL0VqhrjV0ifazLV+HVzyUwZ+L\n\t+2F09mweKc1J7Tr0v1vzMQ+JWvrM0p4f+7VBKI3Mj16pdRgPK9oZJs6rwHdK3Ge9eOTH\n\t2vAzCNd+jrNLnQ20c7nYuX4/tFTAn7eLstk2+xtULe5L2U4oBkShUED2v3+5mL93jjCp\n\tOSUvMBIzGrfBwAkuFzJVg+uPJxxLS+OuKMzbeTtgLVsWEV9b3XbfoqhSkNlDmy0HReLl\n\tS66g==","X-Gm-Message-State":"AHPjjUhLHt1/GkOmf7SyVgyQ0+EBUl3vEsqYcL1Kza4l4RVjtMxOUK3h\n\tM1sWNRpsJ+D7t/5Ih7BgCEg=","X-Google-Smtp-Source":"AOwi7QAt6YXhe7IJAvbaxgo+rQO/5N/a3RlnqqiXhikYId5jw937WHmyqf2BLZ4scFldyabBKC929A==","X-Received":"by 10.101.76.207 with SMTP id n15mr3563467pgt.120.1506581029726; \n\tWed, 27 Sep 2017 23:43:49 -0700 (PDT)","Date":"Thu, 28 Sep 2017 12:13:12 +0530","From":"Linu Cherian ","To":"Eric Auger ","Message-ID":"<20170928064312.GA1544@virtx40>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>","User-Agent":"Mutt/1.5.21 (2010-09-15)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:400e:c00::244","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 00/20] ARM SMMUv3 Emulation\n\tSupport","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, bharat.bhushan@nxp.com, mst@redhat.com,\n\twill.deacon@arm.com, qemu-devel@nongnu.org, peterx@redhat.com,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tchristoffer.dall@linaro.org, wtownsen@redhat.com,\n\trobin.murphy@arm.com, prem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1776814,"web_url":"http://patchwork.ozlabs.org/comment/1776814/","msgid":"<20170928071359.GG17044@pxdev.xzpeter.org>","list_archive_url":null,"date":"2017-09-28T07:13:59","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 00/20] ARM SMMUv3 Emulation\n\tSupport","submitter":{"id":67717,"url":"http://patchwork.ozlabs.org/api/people/67717/","name":"Peter Xu","email":"peterx@redhat.com"},"content":"On Thu, Sep 28, 2017 at 12:13:12PM +0530, Linu Cherian wrote:\n> Hi Eric,\n> \n> \n> On Fri Sep 01, 2017 at 07:21:03PM +0200, Eric Auger wrote:\n> > This series implements the emulation code for ARM SMMUv3.\n> > \n> > Changes since v6:\n> > - DPDK testpmd now running on guest with 2 assigned VFs\n> > - Changed the instantiation method: add the following option to\n> > the QEMU command line\n> > -device smmuv3 # for virtio/vhost use cases\n> > -device smmuv3,caching-mode # for vfio use cases (based on [1])\n> > - splitted the series into smaller patches to allow the review\n> > - the VFIO integration based on \"tlbi-on-map\" smmuv3 driver\n> > is isolated from the rest: last 2 patches, not for upstream.\n> > This is shipped for testing/bench until a better solution is found.\n> > - Reworked permission flag checks and event generation\n> > e testing:\n> > - in dt and ACPI modes\n> > - virtio-net-pci and vhost-net devices using dma ops with various\n> > guest page sizes [2]\n> > - assigned VFs using dma ops [3]:\n> > - AMD Overdrive and igbvf passthrough (using gsi direct mapping)\n> > - Cavium ThunderX and ixgbevf passthrough (using KVM MSI routing)\n> > - DPDK testpmd on guest running with VFIO user space drivers (2 igbvf) [3]\n> > with guest and host page size equal (4kB)\n> > \n> > Known limitations:\n> > - no VMSAv8-32 suport\n> > - no nested stage support (S1 + S2)\n> > - no support for HYP mappings\n> > - register fine emulation, commands, interrupts and errors were\n> > not accurately tested. Handling is sufficient to run use cases\n> > described above though.\n> > - interrupts and event generation not observed yet.\n> \n> While testing with vfio-pci, observed that the below two Qemu command,\n> results in two different behaviour. Is this expected by design ?\n> \n> Case 1:\n> # -device vfio-pci,host=0002:01:00.3 -device smmuv3,caching-mode\n> Here iommu is not attached to the pci bus in Qemu backend, since\n> pci_setup_iommu is not called before vfio_realize.\n> \n> Case 2:\n> # -device smmuv3,caching-mode -device vfio-pci,host=0002:01:00.3\n> This works as expected, iommu is attached to the pci bus.\n\nNot sure about SMMU, but VT-d should have similar issue - the vIOMMU\ndevice needs to be created before the rest of the devices.\n\nNow for VT-d the ordering of devices should be assured by Libvirt:\n\nhttps://bugzilla.redhat.com/show_bug.cgi?id=1427005\n\nFor your reference only. Thanks,","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx04.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx04.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=peterx@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y2mCv59xhz9t5x\n\tfor ;\n\tThu, 28 Sep 2017 17:14:51 +1000 (AEST)","from localhost ([::1]:57724 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dxT21-0007ib-S3\n\tfor incoming@patchwork.ozlabs.org; Thu, 28 Sep 2017 03:14:49 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:59792)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dxT1d-0007es-0A\n\tfor qemu-devel@nongnu.org; Thu, 28 Sep 2017 03:14:25 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dxT1b-0002xR-Tt\n\tfor qemu-devel@nongnu.org; Thu, 28 Sep 2017 03:14:24 -0400","from mx1.redhat.com ([209.132.183.28]:56510)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dxT1U-0002vX-OK; Thu, 28 Sep 2017 03:14:16 -0400","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 27E6151D2A;\n\tThu, 28 Sep 2017 07:14:15 +0000 (UTC)","from pxdev.xzpeter.org (dhcp-14-247.nay.redhat.com [10.66.14.247])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id D4C975D9CA;\n\tThu, 28 Sep 2017 07:14:01 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 27E6151D2A","Date":"Thu, 28 Sep 2017 15:13:59 +0800","From":"Peter Xu ","To":"Linu Cherian ","Message-ID":"<20170928071359.GG17044@pxdev.xzpeter.org>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<20170928064312.GA1544@virtx40>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20170928064312.GA1544@virtx40>","User-Agent":"Mutt/1.5.24 (2015-08-30)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.14","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.28]);\n\tThu, 28 Sep 2017 07:14:15 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 00/20] ARM SMMUv3 Emulation\n\tSupport","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, bharat.bhushan@nxp.com, mst@redhat.com,\n\twill.deacon@arm.com, qemu-devel@nongnu.org,\n\tEric Auger , alex.williamson@redhat.com,\n\tqemu-arm@nongnu.org, christoffer.dall@linaro.org,\n\twtownsen@redhat.com, robin.murphy@arm.com,\n\tprem.mallappa@gmail.com, eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1776839,"web_url":"http://patchwork.ozlabs.org/comment/1776839/","msgid":"<42a551e1-ccc4-f162-51d8-c61ca5f86790@redhat.com>","list_archive_url":null,"date":"2017-09-28T07:54:20","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 00/20] ARM SMMUv3 Emulation\n\tSupport","submitter":{"id":69187,"url":"http://patchwork.ozlabs.org/api/people/69187/","name":"Eric Auger","email":"eric.auger@redhat.com"},"content":"Hi Linu, Peter,\n\nOn 28/09/2017 09:13, Peter Xu wrote:\n> On Thu, Sep 28, 2017 at 12:13:12PM +0530, Linu Cherian wrote:\n>> Hi Eric,\n>>\n>>\n>> On Fri Sep 01, 2017 at 07:21:03PM +0200, Eric Auger wrote:\n>>> This series implements the emulation code for ARM SMMUv3.\n>>>\n>>> Changes since v6:\n>>> - DPDK testpmd now running on guest with 2 assigned VFs\n>>> - Changed the instantiation method: add the following option to\n>>> the QEMU command line\n>>> -device smmuv3 # for virtio/vhost use cases\n>>> -device smmuv3,caching-mode # for vfio use cases (based on [1])\n>>> - splitted the series into smaller patches to allow the review\n>>> - the VFIO integration based on \"tlbi-on-map\" smmuv3 driver\n>>> is isolated from the rest: last 2 patches, not for upstream.\n>>> This is shipped for testing/bench until a better solution is found.\n>>> - Reworked permission flag checks and event generation\n>>> e testing:\n>>> - in dt and ACPI modes\n>>> - virtio-net-pci and vhost-net devices using dma ops with various\n>>> guest page sizes [2]\n>>> - assigned VFs using dma ops [3]:\n>>> - AMD Overdrive and igbvf passthrough (using gsi direct mapping)\n>>> - Cavium ThunderX and ixgbevf passthrough (using KVM MSI routing)\n>>> - DPDK testpmd on guest running with VFIO user space drivers (2 igbvf) [3]\n>>> with guest and host page size equal (4kB)\n>>>\n>>> Known limitations:\n>>> - no VMSAv8-32 suport\n>>> - no nested stage support (S1 + S2)\n>>> - no support for HYP mappings\n>>> - register fine emulation, commands, interrupts and errors were\n>>> not accurately tested. Handling is sufficient to run use cases\n>>> described above though.\n>>> - interrupts and event generation not observed yet.\n>>\n>> While testing with vfio-pci, observed that the below two Qemu command,\n>> results in two different behaviour. Is this expected by design ?\n>>\n>> Case 1:\n>> # -device vfio-pci,host=0002:01:00.3 -device smmuv3,caching-mode\n>> Here iommu is not attached to the pci bus in Qemu backend, since\n>> pci_setup_iommu is not called before vfio_realize.\n>>\n>> Case 2:\n>> # -device smmuv3,caching-mode -device vfio-pci,host=0002:01:00.3\n>> This works as expected, iommu is attached to the pci bus.\n> \n> Not sure about SMMU, but VT-d should have similar issue - the vIOMMU\n> device needs to be created before the rest of the devices.\n\nYes this is an expected limitation right now. I should have documented\nit though. As you noticed, the pci_set_iommu() is called on virtio-iommu\nrealize and it relies on the fact the PCIe devices already are realized.\n\nMaybe we could relax this constraint by calling the pci_set_iommu in a\nmachine init done notifier.\n\nThanks\n\nEric\n\n\n> \n> Now for VT-d the ordering of devices should be assured by Libvirt:\n> \n> https://bugzilla.redhat.com/show_bug.cgi?id=1427005\n> \n> For your reference only. Thanks,\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx08.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx08.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=eric.auger@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y2n6T14cyz9s9Y\n\tfor ;\n\tThu, 28 Sep 2017 17:55:11 +1000 (AEST)","from localhost ([::1]:57827 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dxTf1-0006vL-OT\n\tfor incoming@patchwork.ozlabs.org; Thu, 28 Sep 2017 03:55:07 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:39124)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dxTeY-0006s6-6r\n\tfor qemu-devel@nongnu.org; Thu, 28 Sep 2017 03:54:43 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dxTeX-0003wB-4e\n\tfor qemu-devel@nongnu.org; Thu, 28 Sep 2017 03:54:38 -0400","from mx1.redhat.com ([209.132.183.28]:39334)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dxTeU-0003vA-Nw; Thu, 28 Sep 2017 03:54:34 -0400","from smtp.corp.redhat.com\n\t(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 6DD2CC059B6C;\n\tThu, 28 Sep 2017 07:54:33 +0000 (UTC)","from localhost.localdomain (ovpn-116-163.ams2.redhat.com\n\t[10.36.116.163])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 7E41586E60;\n\tThu, 28 Sep 2017 07:54:22 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 6DD2CC059B6C","To":"Peter Xu , Linu Cherian ","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<20170928064312.GA1544@virtx40>\n\t<20170928071359.GG17044@pxdev.xzpeter.org>","From":"Auger Eric ","Message-ID":"<42a551e1-ccc4-f162-51d8-c61ca5f86790@redhat.com>","Date":"Thu, 28 Sep 2017 09:54:20 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101\n\tThunderbird/45.4.0","MIME-Version":"1.0","In-Reply-To":"<20170928071359.GG17044@pxdev.xzpeter.org>","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.12","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.32]);\n\tThu, 28 Sep 2017 07:54:33 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 00/20] ARM SMMUv3 Emulation\n\tSupport","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, alex.williamson@redhat.com,\n\tqemu-arm@nongnu.org, prem.mallappa@gmail.com,\n\trobin.murphy@arm.com, eric.auger.pro@gmail.com,\n\tbharat.bhushan@nxp.com, christoffer.dall@linaro.org, wtownsen@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1776910,"web_url":"http://patchwork.ozlabs.org/comment/1776910/","msgid":"<20170928092107.GA1813@virtx40>","list_archive_url":null,"date":"2017-09-28T09:21:08","subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 00/20] ARM SMMUv3 Emulation\n\tSupport","submitter":{"id":72195,"url":"http://patchwork.ozlabs.org/api/people/72195/","name":"Linu Cherian","email":"linuc.decode@gmail.com"},"content":"On Thu Sep 28, 2017 at 09:54:20AM +0200, Auger Eric wrote:\n> Hi Linu, Peter,\n> \n> On 28/09/2017 09:13, Peter Xu wrote:\n> > On Thu, Sep 28, 2017 at 12:13:12PM +0530, Linu Cherian wrote:\n> >> Hi Eric,\n> >>\n> >>\n> >> On Fri Sep 01, 2017 at 07:21:03PM +0200, Eric Auger wrote:\n> >>> This series implements the emulation code for ARM SMMUv3.\n> >>>\n> >>> Changes since v6:\n> >>> - DPDK testpmd now running on guest with 2 assigned VFs\n> >>> - Changed the instantiation method: add the following option to\n> >>> the QEMU command line\n> >>> -device smmuv3 # for virtio/vhost use cases\n> >>> -device smmuv3,caching-mode # for vfio use cases (based on [1])\n> >>> - splitted the series into smaller patches to allow the review\n> >>> - the VFIO integration based on \"tlbi-on-map\" smmuv3 driver\n> >>> is isolated from the rest: last 2 patches, not for upstream.\n> >>> This is shipped for testing/bench until a better solution is found.\n> >>> - Reworked permission flag checks and event generation\n> >>> e testing:\n> >>> - in dt and ACPI modes\n> >>> - virtio-net-pci and vhost-net devices using dma ops with various\n> >>> guest page sizes [2]\n> >>> - assigned VFs using dma ops [3]:\n> >>> - AMD Overdrive and igbvf passthrough (using gsi direct mapping)\n> >>> - Cavium ThunderX and ixgbevf passthrough (using KVM MSI routing)\n> >>> - DPDK testpmd on guest running with VFIO user space drivers (2 igbvf) [3]\n> >>> with guest and host page size equal (4kB)\n> >>>\n> >>> Known limitations:\n> >>> - no VMSAv8-32 suport\n> >>> - no nested stage support (S1 + S2)\n> >>> - no support for HYP mappings\n> >>> - register fine emulation, commands, interrupts and errors were\n> >>> not accurately tested. Handling is sufficient to run use cases\n> >>> described above though.\n> >>> - interrupts and event generation not observed yet.\n> >>\n> >> While testing with vfio-pci, observed that the below two Qemu command,\n> >> results in two different behaviour. Is this expected by design ?\n> >>\n> >> Case 1:\n> >> # -device vfio-pci,host=0002:01:00.3 -device smmuv3,caching-mode\n> >> Here iommu is not attached to the pci bus in Qemu backend, since\n> >> pci_setup_iommu is not called before vfio_realize.\n> >>\n> >> Case 2:\n> >> # -device smmuv3,caching-mode -device vfio-pci,host=0002:01:00.3\n> >> This works as expected, iommu is attached to the pci bus.\n> > \n> > Not sure about SMMU, but VT-d should have similar issue - the vIOMMU\n> > device needs to be created before the rest of the devices.\n> \n> Yes this is an expected limitation right now. I should have documented\n> it though. As you noticed, the pci_set_iommu() is called on virtio-iommu\n> realize and it relies on the fact the PCIe devices already are realized.\n> \n> Maybe we could relax this constraint by calling the pci_set_iommu in a\n> machine init done notifier.\n> \n> Thanks\n> \n> Eric\n\nThanks for confirming. \n\n> \n> \n> > \n> > Now for VT-d the ordering of devices should be assured by Libvirt:\n> > \n> > https://bugzilla.redhat.com/show_bug.cgi?id=1427005\n> > \n> > For your reference only. Thanks,\n> >","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"EIP2eOEN\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y2q9W2JFtz9t38\n\tfor ;\n\tThu, 28 Sep 2017 19:27:59 +1000 (AEST)","from localhost ([::1]:58132 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dxV6r-0007aa-B8\n\tfor incoming@patchwork.ozlabs.org; Thu, 28 Sep 2017 05:27:57 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:59987)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dxV0s-0002vu-Vv\n\tfor qemu-devel@nongnu.org; Thu, 28 Sep 2017 05:21:51 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dxV0n-00035A-Sj\n\tfor qemu-devel@nongnu.org; Thu, 28 Sep 2017 05:21:46 -0400","from mail-pf0-x242.google.com ([2607:f8b0:400e:c00::242]:38202)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1dxV0n-00034B-LQ; Thu, 28 Sep 2017 05:21:41 -0400","by mail-pf0-x242.google.com with SMTP id a7so983717pfj.5;\n\tThu, 28 Sep 2017 02:21:40 -0700 (PDT)","from virtx40 ([111.93.218.67]) by smtp.gmail.com with ESMTPSA id\n\tx81sm2208938pfk.40.2017.09.28.02.21.32\n\t(version=TLS1_2 cipher=AES128-SHA bits=128/128);\n\tThu, 28 Sep 2017 02:21:38 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=S+DvS8CnI8Eo08ptq7a9avxMwcBDuQpcm4dg7HYiN8g=;\n\tb=EIP2eOEN5sCouGG3KXjSd/tOQFQYxGOfWlaUs3loTBISPMN8MB3mi6k1+SNPMz0Ayj\n\tY2H4etq0yM/qWkt92nfNhgo45p0bv8/mnvaJLOQIzQOo+6z4QZU9SZBIzkSMn44comf6\n\ts71J4oT9wklX3PdvOq3D2Pmy68nC2H+KRRAL1LjibYhWj8ofCjQ/+0u4VjSN05T3aWiO\n\tFb4oCNXNIUrMdURyBGJC2fmq4W9tKErcPbBXLvj63Ti6tpsaE7wG0ynEH1gI+uB40q19\n\tFlz2OqHrrCa/EIm3aWCBM7rBk+R3UbLdWZz9F66MVRdD5J5XW6+eLx5JuaC35ECehZu7\n\tOhfA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=S+DvS8CnI8Eo08ptq7a9avxMwcBDuQpcm4dg7HYiN8g=;\n\tb=o8LkeJvQ+KUP11A9nijqL2yM4NMgqChuZdyPRKg+jTi/0ukvnwACpbW2nKijdRBPgM\n\touUXE/glQ9b3GWo4trSnizTr+SHg3w3WxUzxBHMnaWRxnztdnl1uNY2ldFH2Qe9B8XOb\n\tXdUdYTiy37JQLTuAJHzv09Jpm83ADOKH4FuvjSEuAK3/eT0V+xL/emR00lj8anXZsYUf\n\tQtzdehijAYbNqjo4MuuXKfj+Vqq6jZtoT72iTNRvm7pX1KmNgUj5aUlaZuZ93jVbpHiD\n\ttIiRDganPe5kaPeoP5p+tZcRfRP03uVJoQTRiB4krDTymVpl/ol5Qs8qsQLMHBGJK6Ce\n\tfLNQ==","X-Gm-Message-State":"AHPjjUjNAfaa8PNYMsIyM77Ux2Mtrk829ohfY74tZhGpGevYiG0GyV5V\n\t22MTgLSrejG+LCM6Yv7PI1Y=","X-Google-Smtp-Source":"AOwi7QBiKTfWrmqoCfSONJwesIkZp9kGnwUy610KsAOW5/s4H/9cDuMjzVrBNuroB75QKEbf5ZW7Cg==","X-Received":"by 10.84.132.4 with SMTP id 4mr3585023ple.449.1506590499402;\n\tThu, 28 Sep 2017 02:21:39 -0700 (PDT)","Date":"Thu, 28 Sep 2017 14:51:08 +0530","From":"Linu Cherian ","To":"Auger Eric ","Message-ID":"<20170928092107.GA1813@virtx40>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<20170928064312.GA1544@virtx40>\n\t<20170928071359.GG17044@pxdev.xzpeter.org>\n\t<42a551e1-ccc4-f162-51d8-c61ca5f86790@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<42a551e1-ccc4-f162-51d8-c61ca5f86790@redhat.com>","User-Agent":"Mutt/1.5.21 (2010-09-15)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:400e:c00::242","Subject":"Re: [Qemu-devel] [Qemu-arm] [PATCH v7 00/20] ARM SMMUv3 Emulation\n\tSupport","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"peter.maydell@linaro.org, drjones@redhat.com, tcain@qti.qualcomm.com,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\tmohun106@gmail.com, jean-philippe.brucker@arm.com,\n\ttn@semihalf.com, mst@redhat.com, will.deacon@arm.com,\n\tqemu-devel@nongnu.org, Peter Xu ,\n\talex.williamson@redhat.com, qemu-arm@nongnu.org,\n\tprem.mallappa@gmail.com, linu.cherian@cavium.com,\n\trobin.murphy@arm.com, eric.auger.pro@gmail.com,\n\tbharat.bhushan@nxp.com, christoffer.dall@linaro.org, wtownsen@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1782903,"web_url":"http://patchwork.ozlabs.org/comment/1782903/","msgid":"","list_archive_url":null,"date":"2017-10-09T14:39:40","subject":"Re: [Qemu-devel] [PATCH v7 02/20] hw/arm/smmu-common: IOMMU memory\n\tregion and address space setup","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 1 September 2017 at 18:21, Eric Auger wrote:\n> We enumerate all the PCI devices attached to the SMMU and\n> initialize an associated IOMMU memory region and address space.\n> This happens on SMMU base instance init.\n>\n> Those info are stored in SMMUDevice objects. The devices are\n> grouped according to the PCIBus they belong to. A hash table\n> indexed by the PCIBus poinet is used. Also an array indexed by\n> the bus number allows to find the list of SMMUDevices.\n>\n> Signed-off-by: Eric Auger \n> ---\n> hw/arm/smmu-common.c | 89 ++++++++++++++++++++++++++++++++++++++++++++\n> include/hw/arm/smmu-common.h | 6 +++\n> 2 files changed, 95 insertions(+)\n>\n> diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c\n> index 56608f1..3e67992 100644\n> --- a/hw/arm/smmu-common.c\n> +++ b/hw/arm/smmu-common.c\n> @@ -30,8 +30,97 @@\n> #include \"qemu/error-report.h\"\n> #include \"hw/arm/smmu-common.h\"\n>\n> +/******************/\n> +/* Infrastructure */\n> +/******************/\n\nMinor thing, but we don't really need this kind of fancy comment\nformatting.\n\n> +static inline gboolean smmu_uint64_equal(gconstpointer v1, gconstpointer v2)\n> +{\n> + return *((const uint64_t *)v1) == *((const uint64_t *)v2);\n> +}\n> +\n> +static inline guint smmu_uint64_hash(gconstpointer v)\n> +{\n> + return (guint)*(const uint64_t *)v;\n> +}\n> +\n> +SMMUPciBus *smmu_find_as_from_bus_num(SMMUState *s, uint8_t bus_num)\n> +{\n> + SMMUPciBus *smmu_pci_bus = s->smmu_as_by_bus_num[bus_num];\n> +\n> + if (!smmu_pci_bus) {\n> + GHashTableIter iter;\n> +\n> + g_hash_table_iter_init(&iter, s->smmu_as_by_busptr);\n> + while (g_hash_table_iter_next(&iter, NULL, (void **)&smmu_pci_bus)) {\n> + if (pci_bus_num(smmu_pci_bus->bus) == bus_num) {\n> + s->smmu_as_by_bus_num[bus_num] = smmu_pci_bus;\n> + return smmu_pci_bus;\n> + }\n> + }\n> + }\n> + return smmu_pci_bus;\n> +}\n> +\n> +static AddressSpace *smmu_find_add_as(PCIBus *bus, void *opaque, int devfn)\n> +{\n> + SMMUState *s = opaque;\n> + uintptr_t key = (uintptr_t)bus;\n> + SMMUPciBus *sbus = g_hash_table_lookup(s->smmu_as_by_busptr, &key);\n> + SMMUDevice *sdev;\n> +\n> + if (!sbus) {\n> + uintptr_t *new_key = g_malloc(sizeof(*new_key));\n> +\n> + *new_key = (uintptr_t)bus;\n> + sbus = g_malloc0(sizeof(SMMUPciBus) +\n> + sizeof(SMMUDevice *) * SMMU_PCI_DEVFN_MAX);\n> + sbus->bus = bus;\n> + g_hash_table_insert(s->smmu_as_by_busptr, new_key, sbus);\n\nWhy do we allocate memory containing a uintptr_t which we set to\nbe the (integer value of the) pointer to the bus, and then use the\npointer to that uintptr_t as the key, when we could just use the\npointer to the bus as the key ? That would save you having a specialist\nequal function, hash function and having to free the keys.\n\n> + }\n> +\n> + sdev = sbus->pbdev[devfn];\n> + if (!sdev) {\n> + char *name = g_strdup_printf(\"%s-%d-%d\",\n> + s->mrtypename,\n> + pci_bus_num(bus), devfn);\n> + sdev = sbus->pbdev[devfn] = g_malloc0(sizeof(SMMUDevice));\n\ng_new0() is slightly stylistically preferable for this kind of thing.\n\n> +\n> + sdev->smmu = s;\n> + sdev->bus = bus;\n> + sdev->devfn = devfn;\n> +\n> + memory_region_init_iommu(&sdev->iommu, sizeof(sdev->iommu),\n> + s->mrtypename,\n> + OBJECT(s), name, 1ULL << 48);\n\nWhat is this 1ULL << 48 ? Is it intended to be the input address\nsize, intermediate address size or output address size? It's not\nclear to me that hardcoded 1 << 48 is right in any of those cases...\n\n> + address_space_init(&sdev->as,\n> + MEMORY_REGION(&sdev->iommu), name);\n> + }\n> +\n> + return &sdev->as;\n> +}\n> +\n> +static void smmu_init_iommu_as(SMMUState *s)\n> +{\n> + PCIBus *pcibus = pci_find_primary_bus();\n\nThis looks odd. I would expect the board model to be\ninstantiating and wiring up the SMMU somehow so that\nit is in the path of whatever PCI bus it is sitting in\nfront of. It shouldn't need to look for the PCI bus like\nthis, which prevents modelling a system where there are\ntwo PCI buses each of which has its own SMMU.\n\n> +\n> + if (pcibus) {\n> + pci_setup_iommu(pcibus, smmu_find_add_as, s);\n> + } else {\n> + error_report(\"No PCI bus, SMMU is not registered\");\n> + }\n> +}\n> +\n> static void smmu_base_instance_init(Object *obj)\n> {\n> + SMMUState *s = SMMU_SYS_DEV(obj);\n> +\n> + memset(s->smmu_as_by_bus_num, 0, sizeof(s->smmu_as_by_bus_num));\n\nInstance init doesn't need to clear the data structure.\n\n> +\n> + s->smmu_as_by_busptr = g_hash_table_new_full(smmu_uint64_hash,\n> + smmu_uint64_equal,\n> + g_free, g_free);\n> + smmu_init_iommu_as(s);\n> }\n>\n> static void smmu_base_class_init(ObjectClass *klass, void *data)\n> diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h\n> index 38cd18f..20f3fe6 100644\n> --- a/include/hw/arm/smmu-common.h\n> +++ b/include/hw/arm/smmu-common.h\n> @@ -105,4 +105,10 @@ typedef struct {\n> #define SMMU_DEVICE_CLASS(klass) \\\n> OBJECT_CLASS_CHECK(SMMUBaseClass, (klass), TYPE_SMMU_DEV_BASE)\n>\n> +SMMUPciBus *smmu_find_as_from_bus_num(SMMUState *s, uint8_t bus_num);\n> +\n> +static inline uint16_t smmu_get_sid(SMMUDevice *sdev)\n> +{\n> + return ((pci_bus_num(sdev->bus) & 0xff) << 8) | sdev->devfn;\n> +}\n> #endif /* HW_ARM_SMMU_COMMON */\n> --\n> 2.5.5\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"M7DHChoW\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9jbH1Hjxz9tXv\n\tfor ;\n\tTue, 10 Oct 2017 01:40:41 +1100 (AEDT)","from localhost ([::1]:58223 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e1ZEU-000595-5E\n\tfor incoming@patchwork.ozlabs.org; Mon, 09 Oct 2017 10:40:38 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:59607)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e1ZE0-00053l-MT\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 10:40:13 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e1ZDu-0006df-He\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 10:40:08 -0400","from mail-wm0-x234.google.com ([2a00:1450:400c:c09::234]:55307)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1e1ZDu-0006d6-7U\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 10:40:02 -0400","by mail-wm0-x234.google.com with SMTP id u138so24208753wmu.4\n\tfor ; Mon, 09 Oct 2017 07:40:02 -0700 (PDT)","by 10.223.128.207 with HTTP; Mon, 9 Oct 2017 07:39:40 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=zFvSie2lfV1/3sTNXOT4fLhannVBVtqUmoWRfQJg4rM=;\n\tb=M7DHChoWRnLfCnBgl/HbeO5nbQaezvzXj3nI90FBDioZBQMzREl+IieOSw1FiKbcCa\n\tC3dYqZ0fhP6haxqLPj+Kl8iZnKGhOEAAzFTT8Sy8ZQ1ty0EFXDTdaWqf9U9wTm28kms/\n\t0qpI03riBhK1Wea7kjYFkZE5yB82aQ2hUglw4=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=zFvSie2lfV1/3sTNXOT4fLhannVBVtqUmoWRfQJg4rM=;\n\tb=MoX6jDpLifh+BXgTRejx5NUktsBDBVNmfWcburB8+ph+kPPEIxn4bG5LGJqE37wO+q\n\tunOUXoju9Bf+Q/7/TYxMmNolvcMzQ8RbvmaTGyk3yyENZAqGkoevTzC6LTOQ1EgEDY6N\n\tar/+YSgMD+TPjZWAYbh8zbH1/AnQv4/a1KIiFiOOY2Ih1O8S28AfdtPMw/GevuvpPsGF\n\tPUkxejI0Q2Kcb1rnLNmRlqSTFsV5FPsZT3PYoZRnR/kIyBh47OrAfRyeZLupMF4469qz\n\t21p6Ca1uYQ0t1yW0cuPoxtMa1UvNRvrXG3prRFtRmLE29yzK1fyDwNuciNpIcyThy3Hg\n\tKgUw==","X-Gm-Message-State":"AMCzsaWHDd9gOD50dsM08sTuKrZ1/BTXrPxEoeHPRYS3j2MYkG6tkaH5\n\tgRiK9izAbT4PEqB92Gn6aDCPz5TygFnwLx16zoesMw==","X-Google-Smtp-Source":"AOwi7QASMZPhaVKKOBjdN7xS8dg5wYzWe0/xus76+WHnUR6ixPeOtXisorlQKpUW8mZwASAnK7LaZhJjkABx8eH0sTs=","X-Received":"by 10.223.175.56 with SMTP id z53mr11037009wrc.177.1507560000967;\n\tMon, 09 Oct 2017 07:40:00 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<1504286483-23327-3-git-send-email-eric.auger@redhat.com>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-3-git-send-email-eric.auger@redhat.com>","From":"Peter Maydell ","Date":"Mon, 9 Oct 2017 15:39:40 +0100","Message-ID":"","To":"Eric Auger ","Content-Type":"text/plain; charset=\"UTF-8\"","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c09::234","Subject":"Re: [Qemu-devel] [PATCH v7 02/20] hw/arm/smmu-common: IOMMU memory\n\tregion and address space setup","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Radha Mohan , Andrew Jones ,\n\tTrey Cain ,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\t\"Michael S. Tsirkin\" ,\n\tjean-philippe.brucker@arm.com, Tomasz Nowicki ,\n\tWill Deacon ,\n\tQEMU Developers , Peter Xu ,\n\tAlex Williamson , \n\tqemu-arm ,\n\tChristoffer Dall ,\n\t\"Edgar E. Iglesias\" ,\n\trobin.murphy@arm.com, wtownsen@redhat.com,\n\tBharat Bhushan ,\n\tPrem Mallappa , eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1782914,"web_url":"http://patchwork.ozlabs.org/comment/1782914/","msgid":"","list_archive_url":null,"date":"2017-10-09T14:46:08","subject":"Re: [Qemu-devel] [PATCH v7 03/20] hw/arm/smmu-common:\n\tsmmu_read/write_sysmem","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 1 September 2017 at 18:21, Eric Auger wrote:\n> Those two functions will be used to access configuration\n> data (STE, CD) and page table entries in guest RAM.\n>\n> Signed-off-by: Eric Auger \n> ---\n> hw/arm/smmu-common.c | 37 +++++++++++++++++++++++++++++++++++++\n> include/hw/arm/smmu-common.h | 5 +++++\n> 2 files changed, 42 insertions(+)\n>\n> diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c\n> index 3e67992..2a94547 100644\n> --- a/hw/arm/smmu-common.c\n> +++ b/hw/arm/smmu-common.c\n> @@ -30,6 +30,43 @@\n> #include \"qemu/error-report.h\"\n> #include \"hw/arm/smmu-common.h\"\n>\n> +inline MemTxResult smmu_read_sysmem(dma_addr_t addr, void *buf, dma_addr_t len,\n> + bool secure)\n> +{\n> + MemTxAttrs attrs = {.unspecified = 1, .secure = secure};\n\nThis isn't right. .unspecified = 1 means \"transaction master has\nnot explicitly specified any attributes\", but you are specifying\none (the secure one).\n\n> + switch (len) {\n> + case 4:\n> + *(uint32_t *)buf = ldl_le_phys(&address_space_memory, addr);\n> + break;\n> + case 8:\n> + *(uint64_t *)buf = ldq_le_phys(&address_space_memory, addr);\n> + break;\n> + default:\n> + return address_space_rw(&address_space_memory, addr,\n> + attrs, buf, len, false);\n\nWhy do we have the special cases for 4 and 8? In particular, those\ncode paths will not correctly detect memory transaction failures.\n\n> + }\n> + return MEMTX_OK;\n> +}\n> +\n> +inline void\n> +smmu_write_sysmem(dma_addr_t addr, void *buf, dma_addr_t len, bool secure)\n> +{\n> + MemTxAttrs attrs = {.unspecified = 1, .secure = secure};\n> +\n> + switch (len) {\n> + case 4:\n> + stl_le_phys(&address_space_memory, addr, *(uint32_t *)buf);\n> + break;\n> + case 8:\n> + stq_le_phys(&address_space_memory, addr, *(uint64_t *)buf);\n> + break;\n> + default:\n> + address_space_rw(&address_space_memory, addr,\n> + attrs, buf, len, true);\n> + }\n> +}\n> +\n> /******************/\n> /* Infrastructure */\n> /******************/\n> diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h\n> index 20f3fe6..a5999b0 100644\n> --- a/include/hw/arm/smmu-common.h\n> +++ b/include/hw/arm/smmu-common.h\n> @@ -111,4 +111,9 @@ static inline uint16_t smmu_get_sid(SMMUDevice *sdev)\n> {\n> return ((pci_bus_num(sdev->bus) & 0xff) << 8) | sdev->devfn;\n> }\n> +\n> +MemTxResult smmu_read_sysmem(dma_addr_t addr, void *buf,\n> + dma_addr_t len, bool secure);\n> +void smmu_write_sysmem(dma_addr_t addr, void *buf, dma_addr_t len, bool secure);\n> +\n\nThere are so few callers of this that I'm inclined to think you\nshould just open code the right kind of memory accessor function\nin the callsites rather than having a weird switch statement.\n\n> #endif /* HW_ARM_SMMU_COMMON */\n> --\n> 2.5.5\n>\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"P2qS6DLx\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9jkh63JYz9tXv\n\tfor ;\n\tTue, 10 Oct 2017 01:47:08 +1100 (AEDT)","from localhost ([::1]:58249 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e1ZKk-0007zB-WD\n\tfor incoming@patchwork.ozlabs.org; Mon, 09 Oct 2017 10:47:07 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:32905)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e1ZKB-0007xz-MM\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 10:46:32 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e1ZKA-0004UV-IY\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 10:46:31 -0400","from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]:47564)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1e1ZKA-0004TU-AE\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 10:46:30 -0400","by mail-wm0-x231.google.com with SMTP id t69so23234967wmt.2\n\tfor ; Mon, 09 Oct 2017 07:46:30 -0700 (PDT)","by 10.223.128.207 with HTTP; Mon, 9 Oct 2017 07:46:08 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=Q1+fUyyHFk+EeB9lVbLgT/U9GjOd5MZ3yNY0sGKU32I=;\n\tb=P2qS6DLxx6NE8Rr6Jq17AhZBSo7eXmW02pxHvhll4HLwkrmhO63WzkVjdTtx9/JXo5\n\toes2OBJdZxUBJVv7jEp+aadrNGHi+JwquYbrI5y7s7LM44XtdtrNShdIvDQG4bEA5J4d\n\tmggxOm27d3/S++UngCEUqwYfeuEQzDr4JP+AM=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=Q1+fUyyHFk+EeB9lVbLgT/U9GjOd5MZ3yNY0sGKU32I=;\n\tb=cb/cuFkxTVtZ/DdNslnjlsu8EshCa++iOEP8v58oyWdgIHCOq5S2kCsASCsXBExPgQ\n\tJsqK6D3pCGQr06YFQ1hWOVkyMcUpL2eRxmwev+Uw88+eKOuZXIaNv+JkFnZz7NsATUBH\n\tiSOjFeNbz25eu2KuDL1xRHjC9AcNGfi+D0wogUHy0D12WfQJ5RiWfqRN4IlcbU90xRU0\n\tajQT5l5dzTcNmcVEvaVPTU8n8j5zKeHVk6mwOitHzbXtLtWSam4qV2uknZ6kiI9sDpbH\n\tgjrzm0KEfWjMEqPI6o+N8ITjdnUSyUrFVME2jlMKhFNOx6LGHKqjs34Wd0i4Xgt5BC9L\n\tLxyw==","X-Gm-Message-State":"AMCzsaX+ZG8DPkmWz8qgoG48g/nhd7qzj3iI7ddwN+r6bDAPvx25FtVU\n\t8eS5D7nEBngekuHlZlCE2IrjcbGCOF9bxwqko4+UTw==","X-Google-Smtp-Source":"AOwi7QA683aTOFXO1Msyrf151X/8aMl/buW1ilSz2EtxtiP/zKlR+sUj/o847tWXb8l+ntTCA4UCHgZPJu1xaS5CIws=","X-Received":"by 10.223.186.201 with SMTP id w9mr9488908wrg.230.1507560389163; \n\tMon, 09 Oct 2017 07:46:29 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<1504286483-23327-4-git-send-email-eric.auger@redhat.com>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-4-git-send-email-eric.auger@redhat.com>","From":"Peter Maydell ","Date":"Mon, 9 Oct 2017 15:46:08 +0100","Message-ID":"","To":"Eric Auger ","Content-Type":"text/plain; charset=\"UTF-8\"","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c09::231","Subject":"Re: [Qemu-devel] [PATCH v7 03/20] hw/arm/smmu-common:\n\tsmmu_read/write_sysmem","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Radha Mohan , Andrew Jones ,\n\tTrey Cain ,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\t\"Michael S. Tsirkin\" ,\n\tjean-philippe.brucker@arm.com, Tomasz Nowicki ,\n\tWill Deacon ,\n\tQEMU Developers , Peter Xu ,\n\tAlex Williamson , \n\tqemu-arm ,\n\tChristoffer Dall ,\n\t\"Edgar E. Iglesias\" ,\n\trobin.murphy@arm.com, wtownsen@redhat.com,\n\tBharat Bhushan ,\n\tPrem Mallappa , eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1782953,"web_url":"http://patchwork.ozlabs.org/comment/1782953/","msgid":"","list_archive_url":null,"date":"2017-10-09T15:36:23","subject":"Re: [Qemu-devel] [PATCH v7 04/20] hw/arm/smmu-common: VMSAv8-64\n\tpage table walk","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 1 September 2017 at 18:21, Eric Auger wrote:\n> This patch implements the page table walk for VMSAv8-64.\n>\n> The page table walk function is devised to walk the tables\n> for a range of IOVAs and to call a callback for each valid\n> leaf entry (frame or block).\n>\n> smmu_page_walk_level_64() handles the walk from a specific level.\n> The advantage of using recursivity is one easily skips invalid\n> entries at any stage. Only if the entry of level n is valid then\n> we walk the level n+1, otherwise we jump to the next index of\n> level n.\n>\n> Walk for an IOVA range will be used for SMMU memory region custom\n> replay. Translation function uses the same function for a granule.\n>\n> Signed-off-by: Eric Auger \n>\n> ---\n> v6 -> v7:\n> - fix wrong error handling in walk_page_table\n> - check perm in smmu_translate\n>\n> v5 -> v6:\n> - use IOMMUMemoryRegion\n> - remove initial_lookup_level()\n> - fix block replay\n>\n> v4 -> v5:\n> - add initial level in translation config\n> - implement block pte\n> - rename must_translate into nofail\n> - introduce call_entry_hook\n> - small changes to dynamic traces\n> - smmu_page_walk code moved from smmuv3.c to this file\n> - remove smmu_translate*\n>\n> v3 -> v4:\n> - reworked page table walk to prepare for VFIO integration\n> (capability to scan a range of IOVA). Same function is used\n> for translate for a single iova. This is largely inspired\n> from intel_iommu.c\n> - as the translate function was not straightforward to me,\n> I tried to stick more closely to the VMSA spec.\n> - remove support of nested stage (kernel driver does not\n> support it anyway)\n> - use error_report and trace events\n> - add aa64[] field in SMMUTransCfg\n> ---\n> hw/arm/smmu-common.c | 343 +++++++++++++++++++++++++++++++++++++++++++\n> hw/arm/smmu-internal.h | 105 +++++++++++++\n> hw/arm/trace-events | 12 ++\n> include/hw/arm/smmu-common.h | 4 +\n> 4 files changed, 464 insertions(+)\n> create mode 100644 hw/arm/smmu-internal.h\n>\n> diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c\n> index 2a94547..f476120 100644\n> --- a/hw/arm/smmu-common.c\n> +++ b/hw/arm/smmu-common.c\n> @@ -29,6 +29,349 @@\n>\n> #include \"qemu/error-report.h\"\n> #include \"hw/arm/smmu-common.h\"\n> +#include \"smmu-internal.h\"\n> +\n> +/*************************/\n> +/* VMSAv8-64 Translation */\n> +/*************************/\n> +\n> +/**\n> + * get_pte - Get the content of a page table entry located in\n> + * @base_addr[@index]\n> + */\n> +static uint64_t get_pte(dma_addr_t baseaddr, uint32_t index)\n> +{\n> + uint64_t pte;\n> +\n> + if (smmu_read_sysmem(baseaddr + index * sizeof(pte),\n> + &pte, sizeof(pte), false)) {\n> + error_report(\"can't read pte at address=0x%\"PRIx64,\n> + baseaddr + index * sizeof(pte));\n\nThis is just a \"guest has misprogrammed something\" error presumably;\nthose are LOG_GUEST_ERROR. Or if it can happen in normal use then\ndon't log it at all.\n\n> + pte = (uint64_t)-1;\n\nThis doesn't look right. \"Successfully read -1 from memory\" and\n\"Failed to read memory\" are different things, so you don't want to\nmash them together into the same return code.\n\n> + return pte;\n> + }\n> + trace_smmu_get_pte(baseaddr, index, baseaddr + index * sizeof(pte), pte);\n> + /* TODO: handle endianness */\n> + return pte;\n> +}\n> +\n> +/* VMSAv8-64 Translation Table Format Descriptor Decoding */\n> +\n> +#define PTE_ADDRESS(pte, shift) (extract64(pte, shift, 47 - shift) << shift)\n> +\n> +/**\n> + * get_page_pte_address - returns the L3 descriptor output address,\n> + * ie. the page frame\n> + * ARM ARM spec: Figure D4-17 VMSAv8-64 level 3 descriptor format\n> + */\n> +static inline hwaddr get_page_pte_address(uint64_t pte, int granule_sz)\n> +{\n> + return PTE_ADDRESS(pte, granule_sz);\n> +}\n> +\n> +/**\n> + * get_table_pte_address - return table descriptor output address,\n> + * ie. address of next level table\n> + * ARM ARM Figure D4-16 VMSAv8-64 level0, level1, and level 2 descriptor formats\n> + */\n> +static inline hwaddr get_table_pte_address(uint64_t pte, int granule_sz)\n> +{\n> + return PTE_ADDRESS(pte, granule_sz);\n> +}\n> +\n> +/**\n> + * get_block_pte_address - return block descriptor output address and block size\n> + * ARM ARM Figure D4-16 VMSAv8-64 level0, level1, and level 2 descriptor formats\n> + */\n> +static hwaddr get_block_pte_address(uint64_t pte, int level, int granule_sz,\n> + uint64_t *bsz)\n> +{\n> + int n;\n> +\n> + switch (granule_sz) {\n> + case 12:\n> + if (level == 1) {\n> + n = 30;\n> + } else if (level == 2) {\n> + n = 21;\n> + } else {\n> + goto error_out;\n> + }\n> + break;\n> + case 14:\n> + if (level == 2) {\n> + n = 25;\n> + } else {\n> + goto error_out;\n> + }\n> + break;\n> + case 16:\n> + if (level == 2) {\n> + n = 29;\n> + } else {\n> + goto error_out;\n> + }\n> + break;\n> + default:\n> + goto error_out;\n> + }\n\nThis is essentially a check that the initial SMMUTransCfg didn't\nspecify an incompatible initial_level and granule, right? We should\ncheck that earlier, rather than here, and if it's strictly a QEMU\ncode bug to get that wrong we should just assert. If a guest misconfig\ncan cause it then we shouldn't use error_report().\n\n> + *bsz = 1 << n;\n> + return PTE_ADDRESS(pte, n);\n> +\n> +error_out:\n> +\n> + error_report(\"unexpected granule_sz=%d/level=%d for block pte\",\n> + granule_sz, level);\n> + *bsz = 0;\n> + return (hwaddr)-1;\n> +}\n> +\n> +static int call_entry_hook(uint64_t iova, uint64_t mask, uint64_t gpa,\n> + int perm, smmu_page_walk_hook hook_fn, void *private)\n> +{\n> + IOMMUTLBEntry entry;\n> + int ret;\n> +\n> + entry.target_as = &address_space_memory;\n> + entry.iova = iova & mask;\n> + entry.translated_addr = gpa;\n> + entry.addr_mask = ~mask;\n> + entry.perm = perm;\n> +\n> + ret = hook_fn(&entry, private);\n> + if (ret) {\n> + error_report(\"%s hook returned %d\", __func__, ret);\n> + }\n> + return ret;\n> +}\n> +\n> +/**\n> + * smmu_page_walk_level_64 - Walk an IOVA range from a specific level\n> + * @baseaddr: table base address corresponding to @level\n> + * @level: level\n> + * @cfg: translation config\n> + * @start: end of the IOVA range\n> + * @end: end of the IOVA range\n> + * @hook_fn: the hook that to be called for each detected area\n> + * @private: private data for the hook function\n> + * @flags: access flags of the parent\n> + * @nofail: indicates whether each iova of the range\n> + * must be translated or whether failure is allowed\n> + *\n> + * Return 0 on success, < 0 on errors not related to translation\n> + * process, > 1 on errors related to translation process (only\n> + * if nofail is set)\n> + */\n> +static int\n> +smmu_page_walk_level_64(dma_addr_t baseaddr, int level,\n> + SMMUTransCfg *cfg, uint64_t start, uint64_t end,\n> + smmu_page_walk_hook hook_fn, void *private,\n> + IOMMUAccessFlags flags, bool nofail)\n> +{\n> + uint64_t subpage_size, subpage_mask, pte, iova = start;\n> + int ret, granule_sz, stage, perm;\n> +\n> + granule_sz = cfg->granule_sz;\n> + stage = cfg->stage;\n> + subpage_size = 1ULL << level_shift(level, granule_sz);\n> + subpage_mask = level_page_mask(level, granule_sz);\n> +\n> + trace_smmu_page_walk_level_in(level, baseaddr, granule_sz,\n> + start, end, flags, subpage_size);\n> +\n> + while (iova < end) {\n> + dma_addr_t next_table_baseaddr;\n> + uint64_t iova_next, pte_addr;\n> + uint32_t offset;\n> +\n> + iova_next = (iova & subpage_mask) + subpage_size;\n> + offset = iova_level_offset(iova, level, granule_sz);\n> + pte_addr = baseaddr + offset * sizeof(pte);\n> + pte = get_pte(baseaddr, offset);\n> +\n> + trace_smmu_page_walk_level(level, iova, subpage_size,\n> + baseaddr, offset, pte);\n> +\n> + if (pte == (uint64_t)-1) {\n> + if (nofail) {\n> + return SMMU_TRANS_ERR_WALK_EXT_ABRT;\n> + }\n> + goto next;\n> + }\n> + if (is_invalid_pte(pte) || is_reserved_pte(pte, level)) {\n> + trace_smmu_page_walk_level_res_invalid_pte(stage, level, baseaddr,\n> + pte_addr, offset, pte);\n> + if (nofail) {\n> + return SMMU_TRANS_ERR_TRANS;\n> + }\n> + goto next;\n> + }\n> +\n> + if (is_page_pte(pte, level)) {\n> + uint64_t gpa = get_page_pte_address(pte, granule_sz);\n> +\n> + perm = flags & pte_ap_to_perm(pte, true);\n> +\n> + trace_smmu_page_walk_level_page_pte(stage, level, iova,\n> + baseaddr, pte_addr, pte, gpa);\n> + ret = call_entry_hook(iova, subpage_mask, gpa, perm,\n> + hook_fn, private);\n> + if (ret) {\n> + return ret;\n> + }\n> + goto next;\n> + }\n> + if (is_block_pte(pte, level)) {\n\nA block descriptor and a page descriptor are basically the same format,\nand you can see in the CPU TLB walk code that we thus treat them basically\nthe same way. It's a bit odd that the code here handles them totally\nseparately and in apparently significantly different ways.\n\n> + size_t target_page_size = qemu_target_page_size();;\n\nStray extra semicolon. Also, this isn't really the current CPU target\npage size (which in any case the SMMU has no way of knowning) so I'm\nsuspicious that it's not what you really want. (What do you want?)\n\n> + uint64_t block_size, top_iova;\n> + hwaddr gpa, block_gpa;\n> +\n> + block_gpa = get_block_pte_address(pte, level, granule_sz,\n> + &block_size);\n> + perm = flags & pte_ap_to_perm(pte, true);\n> +\n> + if (block_gpa == -1) {\n> + if (nofail) {\n> + return SMMU_TRANS_ERR_WALK_EXT_ABRT;\n> + } else {\n> + goto next;\n> + }\n> + }\n> + trace_smmu_page_walk_level_block_pte(stage, level, baseaddr,\n> + pte_addr, pte, iova, block_gpa,\n> + (int)(block_size >> 20));\n> +\n> + gpa = block_gpa + (iova & (block_size - 1));\n> + if ((block_gpa == gpa) && (end >= iova_next - 1)) {\n> + ret = call_entry_hook(iova, ~(block_size - 1), block_gpa,\n> + perm, hook_fn, private);\n> + if (ret) {\n> + return ret;\n> + }\n> + goto next;\n> + } else {\n> + top_iova = MIN(end, iova_next);\n> + while (iova < top_iova) {\n> + gpa = block_gpa + (iova & (block_size - 1));\n> + ret = call_entry_hook(iova, ~(target_page_size - 1),\n> + gpa, perm, hook_fn, private);\n> + if (ret) {\n> + return ret;\n> + }\n> + iova += target_page_size;\n> + }\n\nNo \"goto next\" ? All the other parts of this loop seem to do that\n(though it also suggests that you want if ... else if ... else if ... else).\n\n> + }\n> + }\n> + if (level == 3) {\n> + goto next;\n\nYuck!\n\n> + }\n> + /* table pte */\n> + next_table_baseaddr = get_table_pte_address(pte, granule_sz);\n> + trace_smmu_page_walk_level_table_pte(stage, level, baseaddr, pte_addr,\n> + pte, next_table_baseaddr);\n> + perm = flags & pte_ap_to_perm(pte, false);\n\nThis is converting the architectural TLB entry attribute flags into\nthe QEMU architecture independent IOMMUAccessFlags and then passing\nthe latter onto the next stage of the TLB walk. I think it would be\nbetter to stick to using the architectural attribute flags, as the\nCPU TLB walk code does.\n\n> + ret = smmu_page_walk_level_64(next_table_baseaddr, level + 1, cfg,\n> + iova, MIN(iova_next, end),\n> + hook_fn, private, perm, nofail);\n> + if (ret) {\n> + return ret;\n> + }\n> +\n> +next:\n> + iova = iova_next;\n> + }\n\nThe usual way to write \"while (cond) { ... goto next; ... next: something; }\"\nis \"for (; cond; something) { ... ;continue; ...}\".\n\n> +\n> + return SMMU_TRANS_ERR_NONE;\n> +}\n> +\n> +/**\n> + * smmu_page_walk - walk a specific IOVA range from the initial\n> + * lookup level, and call the hook for each valid entry\n> + *\n> + * @cfg: translation config\n> + * @start: start of the IOVA range\n> + * @end: end of the IOVA range\n> + * @nofail: if true, each IOVA within the range must have a translation\n> + * @hook_fn: the hook that to be called for each detected area\n> + * @private: private data for the hook function\n> + */\n> +int smmu_page_walk(SMMUTransCfg *cfg, uint64_t start, uint64_t end,\n> + bool nofail, smmu_page_walk_hook hook_fn, void *private)\n> +{\n> + uint64_t roof = MIN(end, (1ULL << (64 - cfg->tsz)) - 1);\n> + IOMMUAccessFlags perm = IOMMU_ACCESS_FLAG(true, true);\n> + int stage = cfg->stage;\n> + dma_addr_t ttbr;\n> +\n> + if (!hook_fn) {\n> + return 0;\n> + }\n> +\n> + if (!cfg->aa64) {\n> + error_report(\"VMSAv8-32 page walk is not yet implemented\");\n> + abort();\n> + }\n> +\n> + ttbr = extract64(cfg->ttbr, 0, 48);\n> + trace_smmu_page_walk(stage, cfg->ttbr, cfg->initial_level, start, roof);\n> +\n> + return smmu_page_walk_level_64(ttbr, cfg->initial_level, cfg, start, roof,\n> + hook_fn, private, perm, nofail);\n> +}\n> +\n> +/**\n> + * set_translated_address: page table walk callback for smmu_translate\n> + *\n> + * once a leaf entry is found, applies the offset to the translated address\n> + * and check the permission\n> + *\n> + * @entry: entry filled by the page table walk function, ie. contains the\n> + * leaf entry iova/translated addr and permission flags\n> + * @private: pointer to the original entry that must be translated\n> + */\n> +static int set_translated_address(IOMMUTLBEntry *entry, void *private)\n> +{\n> + IOMMUTLBEntry *tlbe_in = (IOMMUTLBEntry *)private;\n> + size_t offset = tlbe_in->iova - entry->iova;\n> +\n> + if (((tlbe_in->perm & IOMMU_RO) && !(entry->perm & IOMMU_RO)) ||\n> + ((tlbe_in->perm & IOMMU_WO) && !(entry->perm & IOMMU_WO))) {\n> + return SMMU_TRANS_ERR_PERM;\n> + }\n> + tlbe_in->translated_addr = entry->translated_addr + offset;\n> + trace_smmu_set_translated_address(tlbe_in->iova, tlbe_in->translated_addr);\n> + return 0;\n\nreturn SMMU_TRANS_ERR_NONE; ?\n\n> +}\n> +\n> +/**\n> + * smmu_translate - Attempt to translate a given entry according to @cfg\n> + *\n> + * @cfg: translation configuration\n> + * @tlbe: entry pre-filled with the input iova, mask\n> + *\n> + * return: !=0 if no mapping is found for the tlbe->iova or access permission\n> + * does not match\n> + */\n> +int smmu_translate(SMMUTransCfg *cfg, IOMMUTLBEntry *tlbe)\n> +{\n> + int ret = 0;\n> +\n> + if (cfg->bypassed || cfg->disabled) {\n> + return 0;\n> + }\n> +\n> + ret = smmu_page_walk(cfg, tlbe->iova, tlbe->iova + 1, true /* nofail */,\n> + set_translated_address, tlbe);\n> +\n> + if (ret) {\n> + error_report(\"translation failed for iova=0x%\"PRIx64\" perm=%d (%d)\",\n> + tlbe->iova, tlbe->perm, ret);\n> + goto exit;\n> + }\n> +\n> +exit:\n\nNot much point in goto to next statement.\n\n> + return ret;\n> +}\n>\n> inline MemTxResult smmu_read_sysmem(dma_addr_t addr, void *buf, dma_addr_t len,\n> bool secure)\n> diff --git a/hw/arm/smmu-internal.h b/hw/arm/smmu-internal.h\n> new file mode 100644\n> index 0000000..aeeadd4\n> --- /dev/null\n> +++ b/hw/arm/smmu-internal.h\n> @@ -0,0 +1,105 @@\n> +/*\n> + * ARM SMMU support - Internal API\n> + *\n> + * Copyright (c) 2017 Red Hat, Inc.\n> + * Written by Eric Auger\n> + *\n> + * This program is free software; you can redistribute it and/or modify\n> + * it under the terms of the GNU General Public License as published by\n> + * the Free Software Foundation, either version 2 of the License, or\n> + * (at your option) any later version.\n> + *\n> + * This program is distributed in the hope that it will be useful,\n> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n> + * General Public License for more details.\n> + *\n> + * You should have received a copy of the GNU General Public License along\n> + * with this program; if not, see .\n> + */\n> +\n> +#ifndef HW_ARM_SMMU_INTERNAL_H\n> +#define HW_ARM_SMMU_INTERNAL_H\n> +\n> +#define ARM_LPAE_MAX_ADDR_BITS 48\n\nAnother 48... which address size limit is this intended to represent?\n\n> +#define ARM_LPAE_MAX_LEVELS 4\n> +\n> +/* PTE Manipulation */\n> +\n> +#define ARM_LPAE_PTE_TYPE_SHIFT 0\n> +#define ARM_LPAE_PTE_TYPE_MASK 0x3\n> +\n> +#define ARM_LPAE_PTE_TYPE_BLOCK 1\n> +#define ARM_LPAE_PTE_TYPE_RESERVED 1\n> +#define ARM_LPAE_PTE_TYPE_TABLE 3\n> +#define ARM_LPAE_PTE_TYPE_PAGE 3\n\nThis looks weird, because several of these are the same as each other.\nThat's because they're really distinct sets of values, some for\nL0/1/2 descriptors, and some for L3 descriptors. If you want\nto define constant names for this can you make the prefixes different\nfor the different cases, please?\n\n> +\n> +#define ARM_LPAE_PTE_VALID (1 << 0)\n> +\n> +static inline bool is_invalid_pte(uint64_t pte)\n> +{\n> + return !(pte & ARM_LPAE_PTE_VALID);\n> +}\n> +\n> +static inline bool is_reserved_pte(uint64_t pte, int level)\n> +{\n> + return ((level == 3) &&\n> + ((pte & ARM_LPAE_PTE_TYPE_MASK) == ARM_LPAE_PTE_TYPE_RESERVED));\n\nreturn isn't a function so you don't need the outer brackets here.\n\n> +}\n> +\n> +static inline bool is_block_pte(uint64_t pte, int level)\n> +{\n> + return ((level < 3) &&\n> + ((pte & ARM_LPAE_PTE_TYPE_MASK) == ARM_LPAE_PTE_TYPE_BLOCK));\n> +}\n> +\n> +static inline bool is_table_pte(uint64_t pte, int level)\n> +{\n> + return ((level < 3) &&\n> + ((pte & ARM_LPAE_PTE_TYPE_MASK) == ARM_LPAE_PTE_TYPE_TABLE));\n> +}\n> +\n> +static inline bool is_page_pte(uint64_t pte, int level)\n> +{\n> + return ((level == 3) &&\n> + ((pte & ARM_LPAE_PTE_TYPE_MASK) == ARM_LPAE_PTE_TYPE_PAGE));\n> +}\n> +\n> +static IOMMUAccessFlags pte_ap_to_perm(uint64_t pte, bool is_leaf)\n> +{\n> + int ap;\n> + IOMMUAccessFlags flags;\n> +\n> + if (is_leaf) {\n> + ap = extract64(pte, 6, 2);\n> + } else {\n> + ap = extract64(pte, 61, 2);\n> + }\n> + flags = IOMMU_ACCESS_FLAG(true, !(ap & 0x2));\n> + return flags;\n> +}\n> +\n> +/* Level Indexing */\n> +\n> +static inline int level_shift(int level, int granule_sz)\n> +{\n> + return granule_sz + (3 - level) * (granule_sz - 3);\n> +}\n> +\n> +static inline uint64_t level_page_mask(int level, int granule_sz)\n> +{\n> + return ~((1ULL << level_shift(level, granule_sz)) - 1);\n> +}\n> +\n> +/**\n> + * TODO: handle the case where the level resolves less than\n> + * granule_sz -3 IA bits.\n> + */\n> +static inline\n> +uint64_t iova_level_offset(uint64_t iova, int level, int granule_sz)\n> +{\n> + return (iova >> level_shift(level, granule_sz)) &\n> + ((1ULL << (granule_sz - 3)) - 1);\n> +}\n> +\n> +#endif\n> diff --git a/hw/arm/trace-events b/hw/arm/trace-events\n> index 193063e..c67cd39 100644\n> --- a/hw/arm/trace-events\n> +++ b/hw/arm/trace-events\n> @@ -2,3 +2,15 @@\n>\n> # hw/arm/virt-acpi-build.c\n> virt_acpi_setup(void) \"No fw cfg or ACPI disabled. Bailing out.\"\n> +\n> +# hw/arm/smmu-common.c\n> +\n> +smmu_page_walk(int stage, uint64_t baseaddr, int first_level, uint64_t start, uint64_t end) \"stage=%d, baseaddr=0x%\"PRIx64\", first level=%d, start=0x%\"PRIx64\", end=0x%\"PRIx64\n> +smmu_page_walk_level_in(int level, uint64_t baseaddr, int granule_sz, uint64_t start, uint64_t end, int flags, uint64_t subpage_size) \"level=%d baseaddr=0x%\"PRIx64\" granule=%d, start=0x%\"PRIx64\" end=0x%\"PRIx64\" flags=%d subpage_size=0x%lx\"\n> +smmu_page_walk_level(int level, uint64_t iova, size_t subpage_size, uint64_t baseaddr, uint32_t offset, uint64_t pte) \"level=%d iova=0x%lx subpage_sz=0x%lx baseaddr=0x%\"PRIx64\" offset=%d => pte=0x%lx\"\n> +smmu_page_walk_level_res_invalid_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint32_t offset, uint64_t pte) \"stage=%d level=%d base@=0x%\"PRIx64\" pte@=0x%\"PRIx64\" offset=%d pte=0x%lx\"\n> +smmu_page_walk_level_page_pte(int stage, int level, uint64_t iova, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t address) \"stage=%d level=%d iova=0x%\"PRIx64\" base@=0x%\"PRIx64\" pte@=0x%\"PRIx64\" pte=0x%\"PRIx64\" page address = 0x%\"PRIx64\n> +smmu_page_walk_level_block_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t iova, uint64_t gpa, int bsize_mb) \"stage=%d level=%d base@=0x%\"PRIx64\" pte@=0x%\"PRIx64\" pte=0x%\"PRIx64\" iova=0x%\"PRIx64\" block address = 0x%\"PRIx64\" block size = %d MiB\"\n> +smmu_page_walk_level_table_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t address) \"stage=%d, level=%d base@=0x%\"PRIx64\" pte@=0x%\"PRIx64\" pte=0x%\"PRIx64\" next table address = 0x%\"PRIx64\n> +smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte) \"baseaddr=0x%\"PRIx64\" index=0x%x, pteaddr=0x%\"PRIx64\", pte=0x%\"PRIx64\n> +smmu_set_translated_address(hwaddr iova, hwaddr pa) \"iova = 0x%\"PRIx64\" -> pa = 0x%\"PRIx64\n> diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h\n> index a5999b0..112a11c 100644\n> --- a/include/hw/arm/smmu-common.h\n> +++ b/include/hw/arm/smmu-common.h\n> @@ -116,4 +116,8 @@ MemTxResult smmu_read_sysmem(dma_addr_t addr, void *buf,\n> dma_addr_t len, bool secure);\n> void smmu_write_sysmem(dma_addr_t addr, void *buf, dma_addr_t len, bool secure);\n>\n> +int smmu_translate(SMMUTransCfg *cfg, IOMMUTLBEntry *tlbe);\n> +int smmu_page_walk(SMMUTransCfg *cfg, uint64_t start, uint64_t end,\n> + bool nofail, smmu_page_walk_hook hook_fn, void *private);\n> +\n> #endif /* HW_ARM_SMMU_COMMON */\n> --\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"eKMQ0TtC\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9krn29tkz9tXF\n\tfor ;\n\tTue, 10 Oct 2017 02:37:29 +1100 (AEDT)","from localhost ([::1]:58413 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e1a7T-0006s1-FU\n\tfor incoming@patchwork.ozlabs.org; Mon, 09 Oct 2017 11:37:27 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:45624)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e1a6t-0006oB-Q7\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 11:36:54 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e1a6n-0005WZ-Qr\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 11:36:51 -0400","from mail-wm0-x22e.google.com ([2a00:1450:400c:c09::22e]:55379)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1e1a6n-0005WD-EH\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 11:36:45 -0400","by mail-wm0-x22e.google.com with SMTP id u138so24712992wmu.4\n\tfor ; Mon, 09 Oct 2017 08:36:45 -0700 (PDT)","by 10.223.128.207 with HTTP; Mon, 9 Oct 2017 08:36:23 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc:content-transfer-encoding;\n\tbh=e+yZTwrtIkcdPjRZh+oaojoCRcEliLZ/PtOM1lqI7zY=;\n\tb=eKMQ0TtCvr8139LwMcnW61R2o1HRscD3yIvcIpI/3pCp7kOFQHYiavlCPnQyadZCLz\n\tlgENR2ZfyFNwHUT/XwPh0932k7k4Ls/dfR/1+ZDGFhZbKjzSJ9Nwvuwrye/43SWvO2yL\n\tU70rSApp5zCXFv1ydllXK1Qkkoh7zB82RCz6E=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc:content-transfer-encoding;\n\tbh=e+yZTwrtIkcdPjRZh+oaojoCRcEliLZ/PtOM1lqI7zY=;\n\tb=MqD8INqMKAQ6yvMEiKC8P3pXRb3WInZrdov9SIKVrPCszx7Kv2BrnUS4qAystYlZwv\n\tKWZ55BmuJrXULprJ2sLuI7pNF7nHb8GXBI79IZm5JeyblnCW1AqsIq3YCscREyhHOZir\n\t96b6h67jcWsLbrwXXZWJtkj05OAigVM6ChBz49KGHVpDtfFpfxMEf8hkxdS7QBVTrHs6\n\txNxZBN2EtIAIgEAwji5JqngpbBcaZt/AL+KAX7ATOY1+oacpDSWGI6bQ4eMn+91j3AZ5\n\tvZ5R7yJwHvKp9lQf3RwWp5ioY+Zm8tptaQAWHJp1hKBhd3+kTGRglXFWKNN46QZAeiEz\n\tqlCA==","X-Gm-Message-State":"AMCzsaVNwYz2Ir7l/PV0yOc085I9ucTqHZF/Qd+9OgNM4aq4IqxuiGb/\n\t7YlEf3YmYKRr58ISOt+8sYoDZ0esCXIYX4fffbr7Pw==","X-Google-Smtp-Source":"AOwi7QDfCRZCsIuyMJlNkhSOGSrriRJ6QuVAcgz0XpPCq8xsQ50uedYH/9RjdWoM+L6xnXNZK2+u/MrBV/iJ1so9Yf8=","X-Received":"by 10.28.23.3 with SMTP id 3mr8239376wmx.62.1507563403935;\n\tMon, 09 Oct 2017 08:36:43 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<1504286483-23327-5-git-send-email-eric.auger@redhat.com>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-5-git-send-email-eric.auger@redhat.com>","From":"Peter Maydell ","Date":"Mon, 9 Oct 2017 16:36:23 +0100","Message-ID":"","To":"Eric Auger ","Content-Type":"text/plain; charset=\"UTF-8\"","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c09::22e","Subject":"Re: [Qemu-devel] [PATCH v7 04/20] hw/arm/smmu-common: VMSAv8-64\n\tpage table walk","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Radha Mohan , Andrew Jones ,\n\tTrey Cain ,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\t\"Michael S. Tsirkin\" ,\n\tjean-philippe.brucker@arm.com, Tomasz Nowicki ,\n\tWill Deacon ,\n\tQEMU Developers , Peter Xu ,\n\tAlex Williamson , \n\tqemu-arm ,\n\tChristoffer Dall ,\n\t\"Edgar E. Iglesias\" ,\n\trobin.murphy@arm.com, wtownsen@redhat.com,\n\tBharat Bhushan ,\n\tPrem Mallappa , eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1782973,"web_url":"http://patchwork.ozlabs.org/comment/1782973/","msgid":"","list_archive_url":null,"date":"2017-10-09T16:17:31","subject":"Re: [Qemu-devel] [PATCH v7 05/20] hw/arm/smmuv3: Skeleton","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 1 September 2017 at 18:21, Eric Auger wrote:\n> From: Prem Mallappa \n>\n> This patch implements a skeleton for the smmuv3 device.\n> Datatypes and register definitions are introduced. The MMIO\n> region, the interrupts and the queue are initialized (PRI is\n> not supported).\n>\n> Only the MMIO read operation is implemented here.\n>\n> Signed-off-by: Prem Mallappa \n> Signed-off-by: Eric Auger \n>\n> ---\n> v6 -> v7:\n> - split into several patches\n>\n> v5 -> v6:\n> - Use IOMMUMemoryregion\n> - regs become uint32_t and fix 64b MMIO access (.impl)\n> - trace_smmuv3_write/read_mmio take the size param\n>\n> v4 -> v5:\n> - change smmuv3_translate proto (IOMMUAccessFlags flag)\n> - has_stagex replaced by is_ste_stagex\n> - smmu_cfg_populate removed\n> - added smmuv3_decode_config and reworked error management\n> - remwork the naming of IOMMU mrs\n> - fix SMMU_CMDQ_CONS offset\n>\n> v3 -> v4\n> - smmu_irq_update\n> - fix hash key allocation\n> - set smmu_iommu_ops\n> - set SMMU_REG_CR0,\n> - smmuv3_translate: ret.perm not set in bypass mode\n> - use trace events\n> - renamed STM2U64 into L1STD_L2PTR and STMSPAN into L1STD_SPAN\n> - rework smmu_find_ste\n> - fix tg2granule in TT0/0b10 corresponds to 16kB\n>\n> v2 -> v3:\n> - move creation of include/hw/arm/smmuv3.h to this patch to fix compil issue\n> - compilation allowed\n> - fix sbus allocation in smmu_init_pci_iommu\n> - restructure code into headers\n> - misc cleanups\n> ---\n> hw/arm/Makefile.objs | 2 +-\n> hw/arm/smmuv3-internal.h | 201 +++++++++++++++++++++++++++++++++++++++\n> hw/arm/smmuv3.c | 239 +++++++++++++++++++++++++++++++++++++++++++++++\n> hw/arm/trace-events | 3 +\n> include/hw/arm/smmuv3.h | 79 ++++++++++++++++\n> 5 files changed, 523 insertions(+), 1 deletion(-)\n> create mode 100644 hw/arm/smmuv3-internal.h\n> create mode 100644 hw/arm/smmuv3.c\n> create mode 100644 include/hw/arm/smmuv3.h\n>\n> diff --git a/hw/arm/Makefile.objs b/hw/arm/Makefile.objs\n> index 5b2d38d..a7c808b 100644\n> --- a/hw/arm/Makefile.objs\n> +++ b/hw/arm/Makefile.objs\n> @@ -19,4 +19,4 @@ obj-$(CONFIG_FSL_IMX31) += fsl-imx31.o kzm.o\n> obj-$(CONFIG_FSL_IMX6) += fsl-imx6.o sabrelite.o\n> obj-$(CONFIG_ASPEED_SOC) += aspeed_soc.o aspeed.o\n> obj-$(CONFIG_MPS2) += mps2.o\n> -obj-$(CONFIG_ARM_SMMUV3) += smmu-common.o\n> +obj-$(CONFIG_ARM_SMMUV3) += smmu-common.o smmuv3.o\n> diff --git a/hw/arm/smmuv3-internal.h b/hw/arm/smmuv3-internal.h\n> new file mode 100644\n> index 0000000..488acc8\n> --- /dev/null\n> +++ b/hw/arm/smmuv3-internal.h\n> @@ -0,0 +1,201 @@\n> +/*\n> + * ARM SMMUv3 support - Internal API\n> + *\n> + * Copyright (C) 2014-2016 Broadcom Corporation\n> + * Copyright (c) 2017 Red Hat, Inc.\n> + * Written by Prem Mallappa, Eric Auger\n> + *\n> + * This program is free software; you can redistribute it and/or modify\n> + * it under the terms of the GNU General Public License as published by\n> + * the Free Software Foundation, either version 2 of the License, or\n> + * (at your option) any later version.\n> + *\n> + * This program is distributed in the hope that it will be useful,\n> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n> + * GNU General Public License for more details.\n> + *\n> + * You should have received a copy of the GNU General Public License along\n> + * with this program; if not, see .\n> + */\n> +\n> +#ifndef HW_ARM_SMMU_V3_INTERNAL_H\n> +#define HW_ARM_SMMU_V3_INTERNAL_H\n> +\n> +#include \"trace.h\"\n> +#include \"qemu/error-report.h\"\n> +#include \"hw/arm/smmu-common.h\"\n> +\n> +/*****************************\n> + * MMIO Register\n> + *****************************/\n> +enum {\n> + SMMU_REG_IDR0 = 0x0,\n> +\n> +/* IDR0 Field Values and supported features */\n> +\n> +#define SMMU_IDR0_S2P 1 /* stage 2 */\n> +#define SMMU_IDR0_S1P 1 /* stage 1 */\n> +#define SMMU_IDR0_TTF 2 /* Aarch64 only - not Aarch32 (LPAE) */\n\nTwo capital As in AArch32 and AArch64.\n\n> +#define SMMU_IDR0_COHACC 1 /* IO coherent access */\n> +#define SMMU_IDR0_HTTU 2 /* Access and Dirty flag update */\n> +#define SMMU_IDR0_HYP 0 /* Hypervisor Stage 1 contexts */\n> +#define SMMU_IDR0_ATS 0 /* PCIe RC ATS */\n> +#define SMMU_IDR0_ASID16 1 /* 16-bit ASID */\n> +#define SMMU_IDR0_PRI 0 /* Page Request Interface */\n> +#define SMMU_IDR0_VMID16 0 /* 16-bit VMID */\n> +#define SMMU_IDR0_CD2L 0 /* 2-level Context Descriptor table */\n> +#define SMMU_IDR0_STALL 1 /* Stalling fault model */\n> +#define SMMU_IDR0_TERM 1 /* Termination model behaviour */\n> +#define SMMU_IDR0_STLEVEL 1 /* Multi-level Stream Table */\n> +\n> +#define SMMU_IDR0_S2P_SHIFT 0\n> +#define SMMU_IDR0_S1P_SHIFT 1\n> +#define SMMU_IDR0_TTF_SHIFT 2\n> +#define SMMU_IDR0_COHACC_SHIFT 4\n> +#define SMMU_IDR0_HTTU_SHIFT 6\n> +#define SMMU_IDR0_HYP_SHIFT 9\n> +#define SMMU_IDR0_ATS_SHIFT 10\n> +#define SMMU_IDR0_ASID16_SHIFT 12\n> +#define SMMU_IDR0_PRI_SHIFT 16\n> +#define SMMU_IDR0_VMID16_SHIFT 18\n> +#define SMMU_IDR0_CD2L_SHIFT 19\n> +#define SMMU_IDR0_STALL_SHIFT 24\n> +#define SMMU_IDR0_TERM_SHIFT 26\n> +#define SMMU_IDR0_STLEVEL_SHIFT 27\n\nOptional, but you might look at whether you like the FIELD()\nmacro in include/hw/registerfields.h for defining shift and\nmask constants.\n\n> +\n> + SMMU_REG_IDR1 = 0x4,\n> +#define SMMU_IDR1_SIDSIZE 16\n> + SMMU_REG_IDR2 = 0x8,\n> + SMMU_REG_IDR3 = 0xc,\n> + SMMU_REG_IDR4 = 0x10,\n> + SMMU_REG_IDR5 = 0x14,\n> +#define SMMU_IDR5_GRAN_SHIFT 4\n> +#define SMMU_IDR5_GRAN 0b101 /* GRAN4K, GRAN64K */\n> +#define SMMU_IDR5_OAS 4 /* 44 bits */\n> + SMMU_REG_IIDR = 0x1c,\n> + SMMU_REG_CR0 = 0x20,\n> +\n> +#define SMMU_CR0_SMMU_ENABLE (1 << 0)\n> +#define SMMU_CR0_PRIQ_ENABLE (1 << 1)\n> +#define SMMU_CR0_EVTQ_ENABLE (1 << 2)\n> +#define SMMU_CR0_CMDQ_ENABLE (1 << 3)\n> +#define SMMU_CR0_ATS_CHECK (1 << 4)\n> +\n> + SMMU_REG_CR0_ACK = 0x24,\n> + SMMU_REG_CR1 = 0x28,\n> + SMMU_REG_CR2 = 0x2c,\n> +\n> + SMMU_REG_STATUSR = 0x40,\n> +\n> + SMMU_REG_IRQ_CTRL = 0x50,\n> + SMMU_REG_IRQ_CTRL_ACK = 0x54,\n> +\n> +#define SMMU_IRQ_CTRL_GERROR_EN (1 << 0)\n> +#define SMMU_IRQ_CTRL_EVENT_EN (1 << 1)\n> +#define SMMU_IRQ_CTRL_PRI_EN (1 << 2)\n> +\n> + SMMU_REG_GERROR = 0x60,\n> +\n> +#define SMMU_GERROR_CMDQ (1 << 0)\n> +#define SMMU_GERROR_EVENTQ_ABT (1 << 2)\n> +#define SMMU_GERROR_PRIQ_ABT (1 << 3)\n> +#define SMMU_GERROR_MSI_CMDQ_ABT (1 << 4)\n> +#define SMMU_GERROR_MSI_EVENTQ_ABT (1 << 5)\n> +#define SMMU_GERROR_MSI_PRIQ_ABT (1 << 6)\n> +#define SMMU_GERROR_MSI_GERROR_ABT (1 << 7)\n> +#define SMMU_GERROR_SFM_ERR (1 << 8)\n> +\n> + SMMU_REG_GERRORN = 0x64,\n> + SMMU_REG_GERROR_IRQ_CFG0 = 0x68,\n> + SMMU_REG_GERROR_IRQ_CFG1 = 0x70,\n> + SMMU_REG_GERROR_IRQ_CFG2 = 0x74,\n> +\n> + /* SMMU_BASE_RA Applies to STRTAB_BASE, CMDQ_BASE and EVTQ_BASE */\n> +#define SMMU_BASE_RA (1ULL << 62)\n> + SMMU_REG_STRTAB_BASE = 0x80,\n> + SMMU_REG_STRTAB_BASE_CFG = 0x88,\n> +\n> + SMMU_REG_CMDQ_BASE = 0x90,\n> + SMMU_REG_CMDQ_PROD = 0x98,\n> + SMMU_REG_CMDQ_CONS = 0x9c,\n> + /* CMD Consumer (CONS) */\n> +#define SMMU_CMD_CONS_ERR_SHIFT 24\n> +#define SMMU_CMD_CONS_ERR_BITS 7\n> +\n> + SMMU_REG_EVTQ_BASE = 0xa0,\n> + SMMU_REG_EVTQ_PROD = 0xa8,\n> + SMMU_REG_EVTQ_CONS = 0xac,\n> + SMMU_REG_EVTQ_IRQ_CFG0 = 0xb0,\n> + SMMU_REG_EVTQ_IRQ_CFG1 = 0xb8,\n> + SMMU_REG_EVTQ_IRQ_CFG2 = 0xbc,\n> +\n> + SMMU_REG_PRIQ_BASE = 0xc0,\n> + SMMU_REG_PRIQ_PROD = 0xc8,\n> + SMMU_REG_PRIQ_CONS = 0xcc,\n> + SMMU_REG_PRIQ_IRQ_CFG0 = 0xd0,\n> + SMMU_REG_PRIQ_IRQ_CFG1 = 0xd8,\n> + SMMU_REG_PRIQ_IRQ_CFG2 = 0xdc,\n> +\n> + SMMU_ID_REGS_OFFSET = 0xfd0,\n> +\n> + /* Secure registers are not used for now */\n> + SMMU_SECURE_OFFSET = 0x8000,\n> +};\n> +\n> +/**********************\n> + * Data Structures\n> + **********************/\n> +\n> +struct __smmu_data2 {\n> + uint32_t word[2];\n> +};\n\nDon't use __ prefixes -- those are reserved for the system.\nBut these structures look a bit like they're not very useful\nanyway.\n\n> +\n> +struct __smmu_data8 {\n> + uint32_t word[8];\n> +};\n> +\n> +struct __smmu_data16 {\n> + uint32_t word[16];\n> +};\n> +\n> +struct __smmu_data4 {\n> + uint32_t word[4];\n> +};\n> +\n> +typedef struct __smmu_data4 Cmd; /* Command Entry */\n> +typedef struct __smmu_data8 Evt; /* Event Entry */\n> +\n> +/*****************************\n> + * Register Access Primitives\n> + *****************************/\n> +\n> +static inline void smmu_write32_reg(SMMUV3State *s, uint32_t addr, uint32_t val)\n> +{\n> + s->regs[addr >> 2] = val;\n> +}\n> +\n> +static inline void smmu_write64_reg(SMMUV3State *s, uint32_t addr, uint64_t val)\n> +{\n> + addr >>= 2;\n> + s->regs[addr] = extract64(val, 0, 32);\n> + s->regs[addr + 1] = extract64(val, 32, 32);\n> +}\n> +\n> +static inline uint32_t smmu_read32_reg(SMMUV3State *s, uint32_t addr)\n> +{\n> + return s->regs[addr >> 2];\n> +}\n> +\n> +static inline uint64_t smmu_read64_reg(SMMUV3State *s, uint32_t addr)\n> +{\n> + addr >>= 2;\n> + return s->regs[addr] | ((uint64_t)(s->regs[addr + 1]) << 32);\n> +}\n\nThis kind of thing is why I'm not a fan of implementing device\nregister state as an array.\n\n> +\n> +static inline int smmu_enabled(SMMUV3State *s)\n> +{\n> + return smmu_read32_reg(s, SMMU_REG_CR0) & SMMU_CR0_SMMU_ENABLE;\n> +}\n> +\n> +#endif\n> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n> new file mode 100644\n> index 0000000..0a7cd1c\n> --- /dev/null\n> +++ b/hw/arm/smmuv3.c\n> @@ -0,0 +1,239 @@\n> +/*\n> + * Copyright (C) 2014-2016 Broadcom Corporation\n> + * Copyright (c) 2017 Red Hat, Inc.\n> + * Written by Prem Mallappa, Eric Auger\n> + *\n> + * This program is free software; you can redistribute it and/or modify\n> + * it under the terms of the GNU General Public License as published by\n> + * the Free Software Foundation, either version 2 of the License, or\n> + * (at your option) any later version.\n> + *\n> + * This program is distributed in the hope that it will be useful,\n> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n> + * GNU General Public License for more details.\n> + *\n> + * You should have received a copy of the GNU General Public License along\n> + * with this program; if not, see .\n> + */\n> +\n> +#include \"qemu/osdep.h\"\n> +#include \"hw/boards.h\"\n> +#include \"sysemu/sysemu.h\"\n> +#include \"hw/sysbus.h\"\n> +#include \"hw/pci/pci.h\"\n> +#include \"exec/address-spaces.h\"\n> +#include \"trace.h\"\n> +#include \"qemu/error-report.h\"\n> +\n> +#include \"hw/arm/smmuv3.h\"\n> +#include \"smmuv3-internal.h\"\n> +\n> +static void smmuv3_init_regs(SMMUV3State *s)\n> +{\n> + uint32_t data =\n> + SMMU_IDR0_STLEVEL << SMMU_IDR0_STLEVEL_SHIFT |\n> + SMMU_IDR0_TERM << SMMU_IDR0_TERM_SHIFT |\n> + SMMU_IDR0_STALL << SMMU_IDR0_STALL_SHIFT |\n> + SMMU_IDR0_VMID16 << SMMU_IDR0_VMID16_SHIFT |\n> + SMMU_IDR0_PRI << SMMU_IDR0_PRI_SHIFT |\n> + SMMU_IDR0_ASID16 << SMMU_IDR0_ASID16_SHIFT |\n> + SMMU_IDR0_ATS << SMMU_IDR0_ATS_SHIFT |\n> + SMMU_IDR0_HYP << SMMU_IDR0_HYP_SHIFT |\n> + SMMU_IDR0_HTTU << SMMU_IDR0_HTTU_SHIFT |\n> + SMMU_IDR0_COHACC << SMMU_IDR0_COHACC_SHIFT |\n> + SMMU_IDR0_TTF << SMMU_IDR0_TTF_SHIFT |\n> + SMMU_IDR0_S1P << SMMU_IDR0_S1P_SHIFT |\n> + SMMU_IDR0_S2P << SMMU_IDR0_S2P_SHIFT;\n> +\n> + smmu_write32_reg(s, SMMU_REG_IDR0, data);\n> +\n> +#define SMMU_QUEUE_SIZE_LOG2 19\n> + data =\n> + 1 << 27 | /* Attr Types override */\n> + SMMU_QUEUE_SIZE_LOG2 << 21 | /* Cmd Q size */\n> + SMMU_QUEUE_SIZE_LOG2 << 16 | /* Event Q size */\n> + SMMU_QUEUE_SIZE_LOG2 << 11 | /* PRI Q size */\n> + 0 << 6 | /* SSID not supported */\n> + SMMU_IDR1_SIDSIZE;\n> +\n> + smmu_write32_reg(s, SMMU_REG_IDR1, data);\n> +\n> + s->sid_size = SMMU_IDR1_SIDSIZE;\n> +\n> + data = SMMU_IDR5_GRAN << SMMU_IDR5_GRAN_SHIFT | SMMU_IDR5_OAS;\n> +\n> + smmu_write32_reg(s, SMMU_REG_IDR5, data);\n> +}\n> +\n> +static void smmuv3_init_queues(SMMUV3State *s)\n> +{\n> + s->cmdq.prod = 0;\n> + s->cmdq.cons = 0;\n> + s->cmdq.wrap.prod = 0;\n> + s->cmdq.wrap.cons = 0;\n> +\n> + s->evtq.prod = 0;\n> + s->evtq.cons = 0;\n> + s->evtq.wrap.prod = 0;\n> + s->evtq.wrap.cons = 0;\n> +\n> + s->cmdq.entries = SMMU_QUEUE_SIZE_LOG2;\n> + s->cmdq.ent_size = sizeof(Cmd);\n> + s->evtq.entries = SMMU_QUEUE_SIZE_LOG2;\n> + s->evtq.ent_size = sizeof(Evt);\n> +}\n> +\n> +static void smmuv3_init(SMMUV3State *s)\n> +{\n> + smmuv3_init_regs(s);\n> + smmuv3_init_queues(s);\n> +}\n> +\n> +static inline void smmu_update_base_reg(SMMUV3State *s, uint64_t *base,\n> + uint64_t val)\n> +{\n> + *base = val & ~(SMMU_BASE_RA | 0x3fULL);\n> +}\n> +\n> +static void smmu_write_mmio_fixup(SMMUV3State *s, hwaddr *addr)\n> +{\n> + switch (*addr) {\n> + case 0x100a8: case 0x100ac: /* Aliasing => page0 registers */\n> + case 0x100c8: case 0x100cc:\n> + *addr ^= (hwaddr)0x10000;\n> + }\n\nMaybe we should just take advantage of the CONSTRAINED UNPREDICTABLE\nchoice to have page0 and page1 be exact aliases, and have\n addr &= ~0x10000;\nunconditionally?\n\n> +}\n> +\n> +static void smmu_write_mmio(void *opaque, hwaddr addr,\n> + uint64_t val, unsigned size)\n> +{\n> +}\n> +\n> +static uint64_t smmu_read_mmio(void *opaque, hwaddr addr, unsigned size)\n> +{\n> + SMMUState *sys = opaque;\n> + SMMUV3State *s = SMMU_V3_DEV(sys);\n> + uint64_t val;\n> +\n> + smmu_write_mmio_fixup(s, &addr);\n> +\n> + /* Primecell/Corelink ID registers */\n> + switch (addr) {\n> + case 0xFF0 ... 0xFFC:\n> + case 0xFDC ... 0xFE4:\n> + val = 0;\n\nSection \"6.3.72 ID_REGS\" in the spec defines what these registers\nshould read as, and it's not all-zeroes. We can claim to be an ARM\nimplementation, as we do with the GIC.\n\n> + error_report(\"addr:0x%\"PRIx64\" val:0x%\"PRIx64, addr, val);\n\nerror_report for the guest reading the ID regs ??\n\n\n> + break;\n> + case SMMU_REG_STRTAB_BASE ... SMMU_REG_CMDQ_BASE:\n> + case SMMU_REG_EVTQ_BASE:\n> + case SMMU_REG_PRIQ_BASE ... SMMU_REG_PRIQ_IRQ_CFG1:\n> + val = smmu_read64_reg(s, addr);\n> + break;\n> + default:\n> + val = (uint64_t)smmu_read32_reg(s, addr);\n> + break;\n> + }\n> +\n> + trace_smmuv3_read_mmio(addr, val, size);\n> + return val;\n> +}\n> +\n> +static const MemoryRegionOps smmu_mem_ops = {\n> + .read = smmu_read_mmio,\n> + .write = smmu_write_mmio,\n> + .endianness = DEVICE_LITTLE_ENDIAN,\n> + .valid = {\n> + .min_access_size = 4,\n> + .max_access_size = 8,\n> + },\n> + .impl = {\n> + .min_access_size = 4,\n> + .max_access_size = 8,\n> + },\n> +};\n> +\n> +static void smmu_init_irq(SMMUV3State *s, SysBusDevice *dev)\n> +{\n> + int i;\n> +\n> + for (i = 0; i < ARRAY_SIZE(s->irq); i++) {\n> + sysbus_init_irq(dev, &s->irq[i]);\n> + }\n> +}\n> +\n> +static void smmu_reset(DeviceState *dev)\n> +{\n> + SMMUV3State *s = SMMU_V3_DEV(dev);\n> + smmuv3_init(s);\n> +}\n> +\n> +static void smmu_realize(DeviceState *d, Error **errp)\n> +{\n> + SMMUState *sys = SMMU_SYS_DEV(d);\n> + SMMUV3State *s = SMMU_V3_DEV(sys);\n> + SysBusDevice *dev = SYS_BUS_DEVICE(d);\n> +\n> + memory_region_init_io(&sys->iomem, OBJECT(s),\n> + &smmu_mem_ops, sys, TYPE_SMMU_V3_DEV, 0x20000);\n> +\n> + sys->mrtypename = g_strdup(TYPE_SMMUV3_IOMMU_MEMORY_REGION);\n> +\n> + sysbus_init_mmio(dev, &sys->iomem);\n> +\n> + smmu_init_irq(s, dev);\n> +}\n> +\n> +static const VMStateDescription vmstate_smmuv3 = {\n> + .name = \"smmuv3\",\n> + .version_id = 1,\n> + .minimum_version_id = 1,\n> + .fields = (VMStateField[]) {\n> + VMSTATE_UINT32_ARRAY(regs, SMMUV3State, SMMU_NREGS),\n> + VMSTATE_END_OF_LIST(),\n> + },\n> +};\n> +\n> +static void smmuv3_instance_init(Object *obj)\n> +{\n> + /* Nothing much to do here as of now */\n> +}\n> +\n> +static void smmuv3_class_init(ObjectClass *klass, void *data)\n> +{\n> + DeviceClass *dc = DEVICE_CLASS(klass);\n> +\n> + dc->reset = smmu_reset;\n> + dc->vmsd = &vmstate_smmuv3;\n> + dc->realize = smmu_realize;\n> +}\n> +\n> +static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass,\n> + void *data)\n> +{\n> +}\n> +\n> +static const TypeInfo smmuv3_type_info = {\n> + .name = TYPE_SMMU_V3_DEV,\n> + .parent = TYPE_SMMU_DEV_BASE,\n> + .instance_size = sizeof(SMMUV3State),\n> + .instance_init = smmuv3_instance_init,\n> + .class_data = NULL,\n\nWhat?\n\n> + .class_size = sizeof(SMMUV3Class),\n> + .class_init = smmuv3_class_init,\n> +};\n> +\n> +static const TypeInfo smmuv3_iommu_memory_region_info = {\n> + .parent = TYPE_IOMMU_MEMORY_REGION,\n> + .name = TYPE_SMMUV3_IOMMU_MEMORY_REGION,\n> + .class_init = smmuv3_iommu_memory_region_class_init,\n> +};\n> +\n> +static void smmuv3_register_types(void)\n> +{\n> + type_register(&smmuv3_type_info);\n> + type_register(&smmuv3_iommu_memory_region_info);\n> +}\n> +\n> +type_init(smmuv3_register_types)\n> +\n> diff --git a/hw/arm/trace-events b/hw/arm/trace-events\n> index c67cd39..8affbf7 100644\n> --- a/hw/arm/trace-events\n> +++ b/hw/arm/trace-events\n> @@ -14,3 +14,6 @@ smmu_page_walk_level_block_pte(int stage, int level, uint64_t baseaddr, uint64_t\n> smmu_page_walk_level_table_pte(int stage, int level, uint64_t baseaddr, uint64_t pteaddr, uint64_t pte, uint64_t address) \"stage=%d, level=%d base@=0x%\"PRIx64\" pte@=0x%\"PRIx64\" pte=0x%\"PRIx64\" next table address = 0x%\"PRIx64\n> smmu_get_pte(uint64_t baseaddr, int index, uint64_t pteaddr, uint64_t pte) \"baseaddr=0x%\"PRIx64\" index=0x%x, pteaddr=0x%\"PRIx64\", pte=0x%\"PRIx64\n> smmu_set_translated_address(hwaddr iova, hwaddr pa) \"iova = 0x%\"PRIx64\" -> pa = 0x%\"PRIx64\n> +\n> +#hw/arm/smmuv3.c\n> +smmuv3_read_mmio(hwaddr addr, uint64_t val, unsigned size) \"addr: 0x%\"PRIx64\" val:0x%\"PRIx64\" size: 0x%x\"\n> diff --git a/include/hw/arm/smmuv3.h b/include/hw/arm/smmuv3.h\n> new file mode 100644\n> index 0000000..0c8973d\n> --- /dev/null\n> +++ b/include/hw/arm/smmuv3.h\n> @@ -0,0 +1,79 @@\n> +/*\n> + * Copyright (C) 2014-2016 Broadcom Corporation\n> + * Copyright (c) 2017 Red Hat, Inc.\n> + * Written by Prem Mallappa, Eric Auger\n> + *\n> + * This program is free software; you can redistribute it and/or modify\n> + * it under the terms of the GNU General Public License as published by\n> + * the Free Software Foundation, either version 2 of the License, or\n> + * (at your option) any later version.\n> + *\n> + * This program is distributed in the hope that it will be useful,\n> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\n> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\n> + * GNU General Public License for more details.\n> + *\n> + * You should have received a copy of the GNU General Public License along\n> + * with this program; if not, see .\n> + */\n> +\n> +#ifndef HW_ARM_SMMUV3_H\n> +#define HW_ARM_SMMUV3_H\n> +\n> +#include \"hw/arm/smmu-common.h\"\n> +\n> +#define TYPE_SMMUV3_IOMMU_MEMORY_REGION \"smmuv3-iommu-memory-region\"\n> +\n> +#define SMMU_NREGS 0x200\n> +\n> +typedef struct SMMUQueue {\n> + hwaddr base;\n> + uint32_t prod;\n> + uint32_t cons;\n> + union {\n> + struct {\n> + uint8_t prod:1;\n> + uint8_t cons:1;\n> + };\n> + uint8_t unused;\n> + } wrap;\n\nThis is an inefficient way to represent the prod/cons registers.\nThose are carefully arranged so that the wrap bit is just above\nthe queue index, so that you can implement the wrap bit toggling\nby considering the register as an N+1 width integer for\nincrement purposes, but an N width register for indexing\ninto the queue. For QEMU we should just have a uint32_t prod;\nand then we can always increment (including wrap bit handling)\nwith 'prod++', we can get the index into the queue with\n'prod & some_mask', and the register read/write would be\nhandled using a mask that's 1 bit wider than some_mask (plus\nany other high bits in the same register).\n\nThis also means you don't need weird special casing to\nhandle the architected behaviour for what happens to the\nvalue in a queue register when the guest changes the queue\nsize (see eg the spec description of SMMU_EVENTQ_CONS in\n6.3.29).\n\n> +\n> + uint16_t entries; /* Number of entries */\n> + uint8_t ent_size; /* Size of entry in bytes */\n> + uint8_t shift; /* Size in log2 */\n> +} SMMUQueue;\n> +\n> +typedef struct SMMUV3State {\n> + SMMUState smmu_state;\n> +\n> + /* Local cache of most-frequently used registers */\n> +#define SMMU_FEATURE_2LVL_STE (1 << 0)\n> + uint32_t features;\n> + uint16_t sid_size;\n> + uint16_t sid_split;\n> + uint64_t strtab_base;\n> +\n> + uint32_t regs[SMMU_NREGS];\n> +\n> + qemu_irq irq[4];\n> + SMMUQueue cmdq, evtq;\n\nThis data structure setup means you have some register state\nyou're keeping in potentially two places: regs[X] and also\nin fields in SMMUQueue. This is awkward for vmstate save/restore\nand it doesn't look like you quite get it right. You can either:\n * only save/restore the regs[] in vmstate, treating those as\n the canonical source of data, and have a post-load hook to\n reload the info into the SMMUQueue fields\n * don't have info in regs[] for queue registers, instead keeping\n the data canonically in the SMMUQueue fields, and have\n vmstate fields for migrating the SMMUQueue fields\n\n> +\n> +} SMMUV3State;\n> +\n> +typedef enum {\n> + SMMU_IRQ_EVTQ,\n> + SMMU_IRQ_PRIQ,\n> + SMMU_IRQ_CMD_SYNC,\n> + SMMU_IRQ_GERROR,\n> +} SMMUIrq;\n> +\n> +typedef struct {\n> + SMMUBaseClass smmu_base_class;\n> +} SMMUV3Class;\n> +\n> +#define TYPE_SMMU_V3_DEV \"smmuv3\"\n> +#define SMMU_V3_DEV(obj) OBJECT_CHECK(SMMUV3State, (obj), TYPE_SMMU_V3_DEV)\n> +#define SMMU_V3_DEVICE_GET_CLASS(obj) \\\n> + OBJECT_GET_CLASS(SMMUBaseClass, (obj), TYPE_SMMU_V3_DEV)\n> +\n> +#endif\n> --\n> 2.5.5\n>\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"YUSnrhgJ\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9lnm3pCtz9tXx\n\tfor ;\n\tTue, 10 Oct 2017 03:19:56 +1100 (AEDT)","from localhost ([::1]:58756 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e1amY-0008N5-KR\n\tfor incoming@patchwork.ozlabs.org; Mon, 09 Oct 2017 12:19:54 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:58950)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e1ake-0007X4-AR\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 12:17:59 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e1akb-0005xb-Jk\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 12:17:56 -0400","from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]:53310)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1e1akb-0005wO-83\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 12:17:53 -0400","by mail-wm0-x231.google.com with SMTP id q132so24952603wmd.2\n\tfor ; Mon, 09 Oct 2017 09:17:53 -0700 (PDT)","by 10.223.128.207 with HTTP; Mon, 9 Oct 2017 09:17:31 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=k2s0wlz+jFhf3VTT9uTg0+ToVzK8nEQ//1a4dRLvBLA=;\n\tb=YUSnrhgJzEyelGp/2RM71jtqH/XcbuJWMaCN6kCYIerubecPQMK805ZvKyRPRAiLuo\n\ta9H9e6R1f3yQk8RDx7KelqBxhTId0WaB2Y1YNiZ45aSgInLUn0gyNEXN8YAY9OUCs6Zk\n\tA66JAOsosEq/d0eVD4y8XwGYfRU8ZhXmjZtgk=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=k2s0wlz+jFhf3VTT9uTg0+ToVzK8nEQ//1a4dRLvBLA=;\n\tb=noEdS4TdEYHvxvkCCwkrV9pc+OXfcdqeDNev7/8Iu73XYeEu5fQpOxhNQM46I/RzDl\n\tUkWZU/adVwDhXlM8KrZ+WcvIPCxV3axnqqgkTG4U2SYxZ5DadhyPbeKBpAO3bHD7UOwY\n\tmYZkw/8pzMPJLxx7B7dufZVQ1Z0vRpoX8T9yNYudzkN83z7n+Vn+/yKhjqlZzpyLtgmj\n\tUjwPJwlYT9Mg8aJ5Km6AEi+5vLcfnibQF+WwXVI428arXxBNp9FR1jRBSiufBUcZlmoD\n\tMh8/TNP68WYfYYHmsgxppk/ntkZkd1gVmPyFFblXEHZ8hecehyBZQLul+DXOo7vBp4+E\n\tRlCQ==","X-Gm-Message-State":"AMCzsaVdjY8EKcmhklXikrEaJtPOEqlaW8U/ORNnpQ9U8Y0z+3pQC4yt\n\t0AlsVJghE9H5iAnijcPnmHOVdKg7gJt0QlvtT1RN2A==","X-Google-Smtp-Source":"AOwi7QAKARuDFq/pPHigtfQ3sLunq1/xpHoF/0hq+7Y6MFtvNh3XxtZsPZrZb7hNkbQK8uubAC6xTxz4UNvaaZQzLnc=","X-Received":"by 10.28.23.3 with SMTP id 3mr8337287wmx.62.1507565871821;\n\tMon, 09 Oct 2017 09:17:51 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<1504286483-23327-6-git-send-email-eric.auger@redhat.com>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-6-git-send-email-eric.auger@redhat.com>","From":"Peter Maydell ","Date":"Mon, 9 Oct 2017 17:17:31 +0100","Message-ID":"","To":"Eric Auger ","Content-Type":"text/plain; charset=\"UTF-8\"","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c09::231","Subject":"Re: [Qemu-devel] [PATCH v7 05/20] hw/arm/smmuv3: Skeleton","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Radha Mohan , Andrew Jones ,\n\tTrey Cain ,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\t\"Michael S. Tsirkin\" ,\n\tjean-philippe.brucker@arm.com, Tomasz Nowicki ,\n\tWill Deacon ,\n\tQEMU Developers , Peter Xu ,\n\tAlex Williamson , \n\tqemu-arm ,\n\tChristoffer Dall ,\n\t\"Edgar E. Iglesias\" ,\n\trobin.murphy@arm.com, wtownsen@redhat.com,\n\tBharat Bhushan ,\n\tPrem Mallappa , eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1783017,"web_url":"http://patchwork.ozlabs.org/comment/1783017/","msgid":"","list_archive_url":null,"date":"2017-10-09T17:01:49","subject":"Re: [Qemu-devel] [PATCH v7 06/20] hw/arm/smmuv3: Wired IRQ and\n\tGERROR helpers","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 1 September 2017 at 18:21, Eric Auger wrote:\n> We introduce some helpers to handle wired IRQs and especially\n> GERROR interrupt. SMMU writes GERROR register on GERROR event\n> and SW acks GERROR interrupts by setting GERRORn.\n>\n> The Wired interrupts are edge sensitive.\n>\n> Signed-off-by: Eric Auger \n>\n> ---\n>\n> Is CMD_SYNC interrupt enabled somewhere?\n> ---\n> hw/arm/smmuv3-internal.h | 20 ++++++++++++++++++\n> hw/arm/smmuv3.c | 55 ++++++++++++++++++++++++++++++++++++++++++++++++\n> hw/arm/trace-events | 2 ++\n> 3 files changed, 77 insertions(+)\n>\n> diff --git a/hw/arm/smmuv3-internal.h b/hw/arm/smmuv3-internal.h\n> index 488acc8..2b44ee2 100644\n> --- a/hw/arm/smmuv3-internal.h\n> +++ b/hw/arm/smmuv3-internal.h\n> @@ -198,4 +198,24 @@ static inline int smmu_enabled(SMMUV3State *s)\n> return smmu_read32_reg(s, SMMU_REG_CR0) & SMMU_CR0_SMMU_ENABLE;\n> }\n>\n> +/*****************************\n> + * Interrupts\n> + *****************************/\n> +\n> +#define smmu_evt_irq_enabled(s) \\\n> + (smmu_read64_reg(s, SMMU_REG_IRQ_CTRL) & SMMU_IRQ_CTRL_EVENT_EN)\n> +#define smmu_gerror_irq_enabled(s) \\\n> + (smmu_read64_reg(s, SMMU_REG_IRQ_CTRL) & SMMU_IRQ_CTRL_GERROR_EN)\n> +#define smmu_pri_irq_enabled(s) \\\n> + (smmu_read64_reg(s, SMMU_REG_IRQ_CTRL) & SMMU_IRQ_CTRL_PRI_EN)\n> +\n> +#define SMMU_PENDING_GERRORS(s) \\\n> + (smmu_read32_reg(s, SMMU_REG_GERROR) ^ \\\n> + smmu_read32_reg(s, SMMU_REG_GERRORN))\n\nHiding this XOR inside a macro is very confusing.\n\n> +\n> +#define SMMU_CMDQ_ERR(s) (SMMU_PENDING_GERRORS(s) & SMMU_GERROR_CMDQ)\n> +\n> +void smmuv3_irq_trigger(SMMUV3State *s, SMMUIrq irq, uint32_t gerror_val);\n> +void smmuv3_write_gerrorn(SMMUV3State *s, uint32_t gerrorn);\n> +\n> #endif\n> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n> index 0a7cd1c..468134f 100644\n> --- a/hw/arm/smmuv3.c\n> +++ b/hw/arm/smmuv3.c\n> @@ -29,6 +29,61 @@\n> #include \"hw/arm/smmuv3.h\"\n> #include \"smmuv3-internal.h\"\n>\n> +/**\n> + * smmuv3_irq_trigger - pulse @irq if enabled and update\n> + * GERROR register in case of GERROR interrupt\n> + *\n> + * @irq: irq type\n> + * @gerror: gerror new value, only relevant if @irq is GERROR\n> + */\n> +void smmuv3_irq_trigger(SMMUV3State *s, SMMUIrq irq, uint32_t gerror_val)\n> +{\n> + uint32_t pending_gerrors = SMMU_PENDING_GERRORS(s);\n> + bool pulse = false;\n> +\n> + switch (irq) {\n> + case SMMU_IRQ_EVTQ:\n> + pulse = smmu_evt_irq_enabled(s);\n> + break;\n> + case SMMU_IRQ_PRIQ:\n> + pulse = smmu_pri_irq_enabled(s);\n> + break;\n> + case SMMU_IRQ_CMD_SYNC:\n> + pulse = true;\n> + break;\n> + case SMMU_IRQ_GERROR:\n> + {\n> + /* don't toggle an already pending error */\n> + bool new_gerrors = ~pending_gerrors & gerror_val;\n> + uint32_t gerror = smmu_read32_reg(s, SMMU_REG_GERROR);\n> +\n> + smmu_write32_reg(s, SMMU_REG_GERROR, gerror | new_gerrors);\n\nThe SMMU toggles GERROR bits to indicate that they have been\nset, it doesn't just set them to 1.\n\n> +\n> + /* pulse the GERROR irq only if all fields were acked */\n> + pulse = smmu_gerror_irq_enabled(s) && !pending_gerrors;\n> + break;\n> + }\n> + }\n> + if (pulse) {\n> + trace_smmuv3_irq_trigger(irq,\n> + smmu_read32_reg(s, SMMU_REG_GERROR),\n> + SMMU_PENDING_GERRORS(s));\n> + qemu_irq_pulse(s->irq[irq]);\n\nqemu_irq_pulse() is very rarely the right thing for an interrupt\nline, but it looks like it's right here (per spec 3.18.2).\n\n> + }\n> +}\n> +\n> +void smmuv3_write_gerrorn(SMMUV3State *s, uint32_t gerrorn)\n> +{\n> + uint32_t pending_gerrors = SMMU_PENDING_GERRORS(s);\n> + uint32_t sanitized;\n> +\n> + /* Make sure SW does not toggle irqs that are not active */\n\nIf it does, this is CONSTRAINED UNPREDICTABLE. That is worth\nremarking in either a comment or a LOG_GUEST_ERROR warning.\n\n> + sanitized = gerrorn & pending_gerrors;\n\nThis isn't right -- if you want to sanitize the result\nbeing written then you need to make it only change bits\nthat are in pending_errors, but this expression allows\nthe guest to write a 0 to a field that is for a non-pending\nerror. (Or you could just not sanitize at all, that's allowed\nby the CONSTRAINED UNPREDICTABLE.)\n\n> +\n> + smmu_write32_reg(s, SMMU_REG_GERRORN, sanitized);\n> + trace_smmuv3_write_gerrorn(gerrorn, sanitized, SMMU_PENDING_GERRORS(s));\n> +}\n> +\n> static void smmuv3_init_regs(SMMUV3State *s)\n> {\n> uint32_t data =\n> diff --git a/hw/arm/trace-events b/hw/arm/trace-events\n> index 8affbf7..c1ce8eb 100644\n> --- a/hw/arm/trace-events\n> +++ b/hw/arm/trace-events\n> @@ -17,3 +17,5 @@ smmu_set_translated_address(hwaddr iova, hwaddr pa) \"iova = 0x%\"PRIx64\" -> pa =\n>\n> #hw/arm/smmuv3.c\n> smmuv3_read_mmio(hwaddr addr, uint64_t val, unsigned size) \"addr: 0x%\"PRIx64\" val:0x%\"PRIx64\" size: 0x%x\"\n> +smmuv3_irq_trigger(int irq, uint32_t gerror, uint32_t pending) \"irq=%d gerror=0x%x pending gerrors=0x%x\"\n> +smmuv3_write_gerrorn(uint32_t gerrorn, uint32_t sanitized, uint32_t pending) \"gerrorn=0x%x sanitized=0x%x pending=0x%x\"\n> --\n> 2.5.5\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"a1CM+1Ov\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9mm91Dvhz9s7f\n\tfor ;\n\tTue, 10 Oct 2017 04:03:37 +1100 (AEDT)","from localhost ([::1]:58917 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e1bSo-0006uH-VQ\n\tfor incoming@patchwork.ozlabs.org; Mon, 09 Oct 2017 13:03:34 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:44875)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e1bRd-0006PG-LE\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 13:02:25 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e1bRU-0008Oe-2N\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 13:02:21 -0400","from mail-wm0-x22a.google.com ([2a00:1450:400c:c09::22a]:51662)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1e1bRT-0008O5-Nv\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 13:02:12 -0400","by mail-wm0-x22a.google.com with SMTP id f4so25315880wme.0\n\tfor ; Mon, 09 Oct 2017 10:02:11 -0700 (PDT)","by 10.223.128.207 with HTTP; Mon, 9 Oct 2017 10:01:49 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=hpmdTa6otWt7JRf7tvLq8C++4/tzZs7iUJrnqs9Zt7c=;\n\tb=a1CM+1Ov5zKM3Hui7P3NuaQybN5e7Vf3v318RtduaRGpvLc+/E+KbtMT9TJJ6u9u1E\n\tOcNPGIJOYkS880I9zYt7Aa3REk/05QpWn4HOQFYGTn2BdGqGBYz28iiVbSC68mMxNDca\n\tsPXMLO8H8DVIjqUK2g6oHQNS8UyFic7I0FENw=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=hpmdTa6otWt7JRf7tvLq8C++4/tzZs7iUJrnqs9Zt7c=;\n\tb=r1btJn7PL/e6j1zlDOebrF+WIs6GoXs2l+sO7jJEH6kz45AGCWNycTBhFrtds0wvxR\n\t9CnCqYto4UFWBUBhdFAH/FaiEp8l3I1qwOHEWBhGFt9BqDEB+7qo3Ji+4jAegMn2XSxv\n\txCKLElr+3cYEgRCPsEcFPLN7Y4/bTYj+8NM/xnysXS1SbV+mp2kvAqLscZExV05p82lY\n\tkWM5hZiQMQISRD/e6F7NlpVRuZEvFNqu8M+Dz5a47giADSyWPnx8HskXeuNZtfAR7Stb\n\tFD8YsAURbkLZ4xZ0mKHpUj8/3Smwm8hivJ8WmAFFUmKAwjMf1vStYV4ch/CZIA1t2jKX\n\t5FnA==","X-Gm-Message-State":"AMCzsaWHU8j/Cjf7XYujcFoUCn/bi5tHYurzVXFzivHDOphTYC1T8aqM\n\tRO1Zr1xwJzdjDNAzJEzMgCLoWij5VORF+TH9YfWlfA==","X-Google-Smtp-Source":"AOwi7QABOGXBef0t7xFjE3jCo1YSPYlI3wGImjRlaKQC5HKtz8z58Ri23mlqBwyiVfie6fFpduavap0DatA+KiDUjJE=","X-Received":"by 10.28.22.66 with SMTP id 63mr8536951wmw.11.1507568530411; Mon,\n\t09 Oct 2017 10:02:10 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<1504286483-23327-7-git-send-email-eric.auger@redhat.com>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-7-git-send-email-eric.auger@redhat.com>","From":"Peter Maydell ","Date":"Mon, 9 Oct 2017 18:01:49 +0100","Message-ID":"","To":"Eric Auger ","Content-Type":"text/plain; charset=\"UTF-8\"","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c09::22a","Subject":"Re: [Qemu-devel] [PATCH v7 06/20] hw/arm/smmuv3: Wired IRQ and\n\tGERROR helpers","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Radha Mohan , Andrew Jones ,\n\tTrey Cain ,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\t\"Michael S. Tsirkin\" ,\n\tjean-philippe.brucker@arm.com, Tomasz Nowicki ,\n\tWill Deacon ,\n\tQEMU Developers , Peter Xu ,\n\tAlex Williamson , \n\tqemu-arm ,\n\tChristoffer Dall ,\n\t\"Edgar E. Iglesias\" ,\n\trobin.murphy@arm.com, wtownsen@redhat.com,\n\tBharat Bhushan ,\n\tPrem Mallappa , eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1783040,"web_url":"http://patchwork.ozlabs.org/comment/1783040/","msgid":"","list_archive_url":null,"date":"2017-10-09T17:12:36","subject":"Re: [Qemu-devel] [PATCH v7 07/20] hw/arm/smmuv3: Queue helpers","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 1 September 2017 at 18:21, Eric Auger wrote:\n> We introduce helpers to read/write into the circular queues.\n> smmuv3_read_cmdq and smmuv3_write_evtq will become static\n> later on.\n>\n> Signed-off-by: Eric Auger \n\nSee comments on a previous patch where I suggest a better\nway to implement the queue increment/wrapping handling.\n\n> +typedef enum {\n> + CMD_Q_EMPTY,\n> + CMD_Q_FULL,\n> + CMD_Q_PARTIALLY_FILLED,\n> +} SMMUQStatus;\n> +\n> +#define Q_ENTRY(q, idx) (q->base + q->ent_size * idx)\n> +#define Q_WRAP(q, pc) ((pc) >> (q)->shift)\n> +#define Q_IDX(q, pc) ((pc) & ((1 << (q)->shift) - 1))\n> +\n> +static inline SMMUQStatus __smmu_queue_status(SMMUV3State *s, SMMUQueue *q)\n\nNo __ prefixes, please.\n\n> +{\n> + uint32_t prod = Q_IDX(q, q->prod);\n> + uint32_t cons = Q_IDX(q, q->cons);\n> +\n> + if ((prod == cons) && (q->wrap.prod != q->wrap.cons)) {\n> + return CMD_Q_FULL;\n> + } else if ((prod == cons) && (q->wrap.prod == q->wrap.cons)) {\n> + return CMD_Q_EMPTY;\n> + }\n> + return CMD_Q_PARTIALLY_FILLED;\n> +}\n> +#define smmu_is_q_full(s, q) (__smmu_queue_status(s, q) == CMD_Q_FULL)\n> +#define smmu_is_q_empty(s, q) (__smmu_queue_status(s, q) == CMD_Q_EMPTY)\n> +\n> +static inline int __smmu_q_enabled(SMMUV3State *s, uint32_t q)\n> +{\n> + return smmu_read32_reg(s, SMMU_REG_CR0) & q;\n> +}\n> +#define smmu_cmd_q_enabled(s) __smmu_q_enabled(s, SMMU_CR0_CMDQ_ENABLE)\n> +#define smmu_evt_q_enabled(s) __smmu_q_enabled(s, SMMU_CR0_EVTQ_ENABLE)\n\nThis code seems to be rather macro-happy.\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"Jw8WPK2q\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9n040KKfz9t3B\n\tfor ;\n\tTue, 10 Oct 2017 04:13:56 +1100 (AEDT)","from localhost ([::1]:58963 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e1bco-0005Zo-6i\n\tfor incoming@patchwork.ozlabs.org; Mon, 09 Oct 2017 13:13:54 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:48014)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e1bbv-0005Dg-Fz\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 13:13:00 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e1bbu-0005ed-JN\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 13:12:59 -0400","from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]:50189)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1e1bbu-0005e3-DX\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 13:12:58 -0400","by mail-wm0-x231.google.com with SMTP id u138so24381387wmu.5\n\tfor ; Mon, 09 Oct 2017 10:12:58 -0700 (PDT)","by 10.223.128.207 with HTTP; Mon, 9 Oct 2017 10:12:36 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=9NP+eARie8R3T2O6MVgkSVevMEHn9HBQmr8moBz1ioA=;\n\tb=Jw8WPK2q3Y3Mc3xWTNMhso8gSivSwFo+QQkrLr7kSrkVBQRJ5pDKd92T/HdWcw3ylR\n\tkJoNywVwIeNFtSjD+6qIgqOOMQA7cRTgf3+6/GklVS48W1Q5E6aNS42XBbhwvs1lQj6A\n\tdaRwWC8Zkdg0F2WBwBeffsRQyse2+Kb4oSI2w=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=9NP+eARie8R3T2O6MVgkSVevMEHn9HBQmr8moBz1ioA=;\n\tb=MH58lZ0fcR3anzeI9gOF47mu8J3P1VhtHdh3VpSfLUvpmcU1FVSp9m39oQUntABr7q\n\tqNkhT/nqPj/yOJvo48T1nH3djYhqjaaunu8KdM4jcUKwqz9bHE3W1Ukvqr9XB9EPJGzJ\n\tHRrnabRMCPWoeaQrJo16/bgaJ5JLRxnaFeL5mWqp5wLcFLt9jhpcqHegtaKvMNj70G7/\n\tNmMkjJQalgxYIBC8ofGrSWm9KCPbQEWCkEpTKTaMlo8pybSt1mfuPgRr5MSJJNy/D0aM\n\t2Bhf5c/8/C5N9eUX0TvkWAunsfWjsQlrAhdZds2Nfv8tDCCgVzf+mtcXBXZKXVJR6UPP\n\tvvOg==","X-Gm-Message-State":"AMCzsaWz7smxvxQb+iOEXLzDLFP6TA9HBwISiOOTS3hoF08ywj88PIsY\n\tZLXm1breQiEAyDWisq6AskFDHZ8odbZjW5ZipNjBRA==","X-Google-Smtp-Source":"AOwi7QDQdIkM8PyjvrOoluBfDm5ML3W04rglmhs0P1HdkAKAlFEw59vdutalJIJsDI+omxm3gzf8IMUXkuaAMyHQa7s=","X-Received":"by 10.28.113.196 with SMTP id d65mr8025267wmi.151.1507569177402; \n\tMon, 09 Oct 2017 10:12:57 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<1504286483-23327-8-git-send-email-eric.auger@redhat.com>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-8-git-send-email-eric.auger@redhat.com>","From":"Peter Maydell ","Date":"Mon, 9 Oct 2017 18:12:36 +0100","Message-ID":"","To":"Eric Auger ","Content-Type":"text/plain; charset=\"UTF-8\"","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c09::231","Subject":"Re: [Qemu-devel] [PATCH v7 07/20] hw/arm/smmuv3: Queue helpers","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Radha Mohan , Andrew Jones ,\n\tTrey Cain ,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\t\"Michael S. Tsirkin\" ,\n\tjean-philippe.brucker@arm.com, Tomasz Nowicki ,\n\tWill Deacon ,\n\tQEMU Developers , Peter Xu ,\n\tAlex Williamson , \n\tqemu-arm ,\n\tChristoffer Dall ,\n\t\"Edgar E. Iglesias\" ,\n\trobin.murphy@arm.com, wtownsen@redhat.com,\n\tBharat Bhushan ,\n\tPrem Mallappa , eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1783056,"web_url":"http://patchwork.ozlabs.org/comment/1783056/","msgid":"","list_archive_url":null,"date":"2017-10-09T17:17:48","subject":"Re: [Qemu-devel] [PATCH v7 08/20] hw/arm/smmuv3: Implement MMIO\n\twrite operations","submitter":{"id":5111,"url":"http://patchwork.ozlabs.org/api/people/5111/","name":"Peter Maydell","email":"peter.maydell@linaro.org"},"content":"On 1 September 2017 at 18:21, Eric Auger wrote:\n> Now we have relevant helpers for queue and irq\n> management, let's implement MMIO write operations\n>\n> Signed-off-by: Eric Auger \n> ---\n> hw/arm/smmuv3-internal.h | 103 +++++++++++++++++++++++-\n> hw/arm/smmuv3.c | 204 ++++++++++++++++++++++++++++++++++++++++++++++-\n> hw/arm/trace-events | 15 ++++\n> 3 files changed, 317 insertions(+), 5 deletions(-)\n>\n> diff --git a/hw/arm/smmuv3-internal.h b/hw/arm/smmuv3-internal.h\n> index d88f141..a5d60b4 100644\n> --- a/hw/arm/smmuv3-internal.h\n> +++ b/hw/arm/smmuv3-internal.h\n> @@ -215,8 +215,6 @@ static inline int smmu_enabled(SMMUV3State *s)\n>\n> #define SMMU_CMDQ_ERR(s) (SMMU_PENDING_GERRORS(s) & SMMU_GERROR_CMDQ)\n>\n> -void smmuv3_write_gerrorn(SMMUV3State *s, uint32_t gerrorn);\n> -\n> /***************************\n> * Queue Handling\n> ***************************/\n> @@ -261,7 +259,106 @@ static inline void smmu_write_cmdq_err(SMMUV3State *s, uint32_t err_type)\n> regval | err_type << SMMU_CMD_CONS_ERR_SHIFT);\n> }\n>\n> -MemTxResult smmuv3_read_cmdq(SMMUV3State *s, Cmd *cmd);\n> void smmuv3_write_evtq(SMMUV3State *s, Evt *evt);\n>\n> +/*****************************\n> + * Commands\n> + *****************************/\n> +\n> +enum {\n> + SMMU_CMD_PREFETCH_CONFIG = 0x01,\n> + SMMU_CMD_PREFETCH_ADDR,\n> + SMMU_CMD_CFGI_STE,\n> + SMMU_CMD_CFGI_STE_RANGE,\n> + SMMU_CMD_CFGI_CD,\n> + SMMU_CMD_CFGI_CD_ALL,\n> + SMMU_CMD_CFGI_ALL,\n> + SMMU_CMD_TLBI_NH_ALL = 0x10,\n> + SMMU_CMD_TLBI_NH_ASID,\n> + SMMU_CMD_TLBI_NH_VA,\n> + SMMU_CMD_TLBI_NH_VAA,\n> + SMMU_CMD_TLBI_EL3_ALL = 0x18,\n> + SMMU_CMD_TLBI_EL3_VA = 0x1a,\n> + SMMU_CMD_TLBI_EL2_ALL = 0x20,\n> + SMMU_CMD_TLBI_EL2_ASID,\n> + SMMU_CMD_TLBI_EL2_VA,\n> + SMMU_CMD_TLBI_EL2_VAA, /* 0x23 */\n> + SMMU_CMD_TLBI_S12_VMALL = 0x28,\n> + SMMU_CMD_TLBI_S2_IPA = 0x2a,\n> + SMMU_CMD_TLBI_NSNH_ALL = 0x30,\n> + SMMU_CMD_ATC_INV = 0x40,\n> + SMMU_CMD_PRI_RESP,\n> + SMMU_CMD_RESUME = 0x44,\n> + SMMU_CMD_STALL_TERM,\n> + SMMU_CMD_SYNC, /* 0x46 */\n> +};\n> +\n> +static const char *cmd_stringify[] = {\n> + [SMMU_CMD_PREFETCH_CONFIG] = \"SMMU_CMD_PREFETCH_CONFIG\",\n> + [SMMU_CMD_PREFETCH_ADDR] = \"SMMU_CMD_PREFETCH_ADDR\",\n> + [SMMU_CMD_CFGI_STE] = \"SMMU_CMD_CFGI_STE\",\n> + [SMMU_CMD_CFGI_STE_RANGE] = \"SMMU_CMD_CFGI_STE_RANGE\",\n> + [SMMU_CMD_CFGI_CD] = \"SMMU_CMD_CFGI_CD\",\n> + [SMMU_CMD_CFGI_CD_ALL] = \"SMMU_CMD_CFGI_CD_ALL\",\n> + [SMMU_CMD_CFGI_ALL] = \"SMMU_CMD_CFGI_ALL\",\n> + [SMMU_CMD_TLBI_NH_ALL] = \"SMMU_CMD_TLBI_NH_ALL\",\n> + [SMMU_CMD_TLBI_NH_ASID] = \"SMMU_CMD_TLBI_NH_ASID\",\n> + [SMMU_CMD_TLBI_NH_VA] = \"SMMU_CMD_TLBI_NH_VA\",\n> + [SMMU_CMD_TLBI_NH_VAA] = \"SMMU_CMD_TLBI_NH_VAA\",\n> + [SMMU_CMD_TLBI_EL3_ALL] = \"SMMU_CMD_TLBI_EL3_ALL\",\n> + [SMMU_CMD_TLBI_EL3_VA] = \"SMMU_CMD_TLBI_EL3_VA\",\n> + [SMMU_CMD_TLBI_EL2_ALL] = \"SMMU_CMD_TLBI_EL2_ALL\",\n> + [SMMU_CMD_TLBI_EL2_ASID] = \"SMMU_CMD_TLBI_EL2_ASID\",\n> + [SMMU_CMD_TLBI_EL2_VA] = \"SMMU_CMD_TLBI_EL2_VA\",\n> + [SMMU_CMD_TLBI_EL2_VAA] = \"SMMU_CMD_TLBI_EL2_VAA\",\n> + [SMMU_CMD_TLBI_S12_VMALL] = \"SMMU_CMD_TLBI_S12_VMALL\",\n> + [SMMU_CMD_TLBI_S2_IPA] = \"SMMU_CMD_TLBI_S2_IPA\",\n> + [SMMU_CMD_TLBI_NSNH_ALL] = \"SMMU_CMD_TLBI_NSNH_ALL\",\n> + [SMMU_CMD_ATC_INV] = \"SMMU_CMD_ATC_INV\",\n> + [SMMU_CMD_PRI_RESP] = \"SMMU_CMD_PRI_RESP\",\n> + [SMMU_CMD_RESUME] = \"SMMU_CMD_RESUME\",\n> + [SMMU_CMD_STALL_TERM] = \"SMMU_CMD_STALL_TERM\",\n> + [SMMU_CMD_SYNC] = \"SMMU_CMD_SYNC\",\n> +};\n> +\n> +/*****************************\n> + * CMDQ fields\n> + *****************************/\n> +\n> +typedef enum {\n> + SMMU_CERROR_NONE = 0,\n> + SMMU_CERROR_ILL,\n> + SMMU_CERROR_ABT,\n> + SMMU_CERROR_ATC_INV_SYNC,\n> +} SMMUCmdError;\n> +\n> +enum { /* Command completion notification */\n> + CMD_SYNC_SIG_NONE,\n> + CMD_SYNC_SIG_IRQ,\n> + CMD_SYNC_SIG_SEV,\n> +};\n> +\n> +#define CMD_TYPE(x) extract32((x)->word[0], 0, 8)\n> +#define CMD_SEC(x) extract32((x)->word[0], 9, 1)\n> +#define CMD_SEV(x) extract32((x)->word[0], 10, 1)\n> +#define CMD_AC(x) extract32((x)->word[0], 12, 1)\n> +#define CMD_AB(x) extract32((x)->word[0], 13, 1)\n> +#define CMD_CS(x) extract32((x)->word[0], 12, 2)\n> +#define CMD_SSID(x) extract32((x)->word[0], 16, 16)\n> +#define CMD_SID(x) ((x)->word[1])\n> +#define CMD_VMID(x) extract32((x)->word[1], 0, 16)\n> +#define CMD_ASID(x) extract32((x)->word[1], 16, 16)\n> +#define CMD_STAG(x) extract32((x)->word[2], 0, 16)\n> +#define CMD_RESP(x) extract32((x)->word[2], 11, 2)\n> +#define CMD_GRPID(x) extract32((x)->word[3], 0, 8)\n> +#define CMD_SIZE(x) extract32((x)->word[3], 0, 16)\n> +#define CMD_LEAF(x) extract32((x)->word[3], 0, 1)\n> +#define CMD_SPAN(x) extract32((x)->word[3], 0, 5)\n> +#define CMD_ADDR(x) ({ \\\n> + uint64_t addr = (uint64_t)(x)->word[3]; \\\n> + addr <<= 32; \\\n> + addr |= extract32((x)->word[3], 12, 20); \\\n> + addr; \\\n> + })\n\nThis definitely seems to be reimplementing some of the registerfields.h\nfunctionality.\n\n> +\n> #endif\n> diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c\n> index 2f96463..f35fadc 100644\n> --- a/hw/arm/smmuv3.c\n> +++ b/hw/arm/smmuv3.c\n> @@ -72,7 +72,7 @@ static void smmuv3_irq_trigger(SMMUV3State *s, SMMUIrq irq, uint32_t gerror_val)\n> }\n> }\n>\n> -void smmuv3_write_gerrorn(SMMUV3State *s, uint32_t gerrorn)\n> +static void smmuv3_write_gerrorn(SMMUV3State *s, uint32_t gerrorn)\n> {\n> uint32_t pending_gerrors = SMMU_PENDING_GERRORS(s);\n> uint32_t sanitized;\n> @@ -116,7 +116,7 @@ static void smmu_q_write(SMMUQueue *q, void *data)\n> }\n> }\n>\n> -MemTxResult smmuv3_read_cmdq(SMMUV3State *s, Cmd *cmd)\n> +static MemTxResult smmuv3_read_cmdq(SMMUV3State *s, Cmd *cmd)\n> {\n> SMMUQueue *q = &s->cmdq;\n> MemTxResult ret = smmu_q_read(q, cmd);\n> @@ -224,6 +224,147 @@ static inline void smmu_update_base_reg(SMMUV3State *s, uint64_t *base,\n> *base = val & ~(SMMU_BASE_RA | 0x3fULL);\n> }\n>\n> +static int smmuv3_cmdq_consume(SMMUV3State *s)\n> +{\n> + SMMUCmdError cmd_error = SMMU_CERROR_NONE;\n> +\n> + trace_smmuv3_cmdq_consume(SMMU_CMDQ_ERR(s), smmu_cmd_q_enabled(s),\n> + s->cmdq.prod, s->cmdq.cons,\n> + s->cmdq.wrap.prod, s->cmdq.wrap.cons);\n> +\n> + if (!smmu_cmd_q_enabled(s)) {\n> + return 0;\n> + }\n> +\n> + while (!SMMU_CMDQ_ERR(s) && !smmu_is_q_empty(s, &s->cmdq)) {\n> + uint32_t type;\n> + Cmd cmd;\n> +\n> + if (smmuv3_read_cmdq(s, &cmd) != MEMTX_OK) {\n> + cmd_error = SMMU_CERROR_ABT;\n> + break;\n> + }\n> +\n> + type = CMD_TYPE(&cmd);\n> +\n> + trace_smmuv3_cmdq_opcode(cmd_stringify[type]);\n> +\n> + switch (CMD_TYPE(&cmd)) {\n> + case SMMU_CMD_SYNC:\n> + if (CMD_CS(&cmd) & CMD_SYNC_SIG_IRQ) {\n> + smmuv3_irq_trigger(s, SMMU_IRQ_CMD_SYNC, 0);\n> + }\n> + break;\n> + case SMMU_CMD_PREFETCH_CONFIG:\n> + case SMMU_CMD_PREFETCH_ADDR:\n> + break;\n> + case SMMU_CMD_CFGI_STE:\n> + {\n> + uint32_t streamid = cmd.word[1];\n> +\n> + trace_smmuv3_cmdq_cfgi_ste(streamid);\n> + break;\n> + }\n> + case SMMU_CMD_CFGI_STE_RANGE: /* same as SMMU_CMD_CFGI_ALL */\n> + {\n> + uint32_t start = cmd.word[1], range, end;\n> +\n> + range = extract32(cmd.word[2], 0, 5);\n> + end = start + (1 << (range + 1)) - 1;\n> + trace_smmuv3_cmdq_cfgi_ste_range(start, end);\n> + break;\n> + }\n> + case SMMU_CMD_CFGI_CD:\n> + case SMMU_CMD_CFGI_CD_ALL:\n> + trace_smmuv3_unhandled_cmd(type);\n\nDo we cache anything that actually needs to be invalidated?\n\n> + break;\n> + case SMMU_CMD_TLBI_NH_ALL:\n> + case SMMU_CMD_TLBI_NH_ASID:\n> + trace_smmuv3_unhandled_cmd(type);\n\nDitto.\n\n> + break;\n> + case SMMU_CMD_TLBI_NH_VA:\n> + {\n> + int asid = extract32(cmd.word[1], 16, 16);\n> + int vmid = extract32(cmd.word[1], 0, 16);\n> + uint64_t low = extract32(cmd.word[2], 12, 20);\n> + uint64_t high = cmd.word[3];\n> + uint64_t addr = high << 32 | (low << 12);\n> +\n> + trace_smmuv3_cmdq_tlbi_nh_va(asid, vmid, addr);\n> + break;\n> + }\n> + case SMMU_CMD_TLBI_NH_VAA:\n> + case SMMU_CMD_TLBI_EL3_ALL:\n> + case SMMU_CMD_TLBI_EL3_VA:\n> + case SMMU_CMD_TLBI_EL2_ALL:\n> + case SMMU_CMD_TLBI_EL2_ASID:\n> + case SMMU_CMD_TLBI_EL2_VA:\n> + case SMMU_CMD_TLBI_EL2_VAA:\n> + case SMMU_CMD_TLBI_S12_VMALL:\n> + case SMMU_CMD_TLBI_S2_IPA:\n> + case SMMU_CMD_TLBI_NSNH_ALL:\n> + trace_smmuv3_unhandled_cmd(type);\n> + break;\n> + case SMMU_CMD_ATC_INV:\n> + case SMMU_CMD_PRI_RESP:\n> + case SMMU_CMD_RESUME:\n> + case SMMU_CMD_STALL_TERM:\n> + trace_smmuv3_unhandled_cmd(type);\n> + break;\n\nYou could merge these two sets of cases (or maybe the TLBI are\ntrivial nops?)\n\n> + default:\n> + cmd_error = SMMU_CERROR_ILL;\n> + error_report(\"Illegal command type: %d\", CMD_TYPE(&cmd));\n> + break;\n> + }\n> + }\n> +\n> + if (cmd_error) {\n> + error_report(\"GERROR_CMDQ: CONS.ERR=%d\", cmd_error);\n> + smmu_write_cmdq_err(s, cmd_error);\n> + smmuv3_irq_trigger(s, SMMU_IRQ_GERROR, SMMU_GERROR_CMDQ);\n> + }\n> +\n> + trace_smmuv3_cmdq_consume_out(s->cmdq.wrap.prod, s->cmdq.prod,\n> + s->cmdq.wrap.cons, s->cmdq.cons);\n> +\n> + return 0;\n\n\n> {\n> + SMMUState *sys = opaque;\n> + SMMUV3State *s = SMMU_V3_DEV(sys);\n> +\n> + smmu_write_mmio_fixup(s, &addr);\n> +\n> + trace_smmuv3_write_mmio(addr, val, size);\n> +\n> + switch (addr) {\n> + case 0xFDC ... 0xFFC:\n> + case SMMU_REG_IDR0 ... SMMU_REG_IDR5:\n> + trace_smmuv3_write_mmio_idr(addr, val);\n> + return;\n> + case SMMU_REG_GERRORN:\n> + smmuv3_write_gerrorn(s, val);\n> + /*\n> + * By acknowledging the CMDQ_ERR, SW may notify cmds can\n> + * be processed again\n> + */\n> + smmuv3_cmdq_consume(s);\n> + return;\n> + case SMMU_REG_CR0:\n> + smmu_write32_reg(s, SMMU_REG_CR0, val);\n> + /* immediatly reflect the changes in CR0_ACK */\n> + smmu_write32_reg(s, SMMU_REG_CR0_ACK, val);\n\n\"immediately\"\n\nthanks\n-- PMM","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"ZiOZG2Et\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y9n5W2ly1z9tXx\n\tfor ;\n\tTue, 10 Oct 2017 04:18:39 +1100 (AEDT)","from localhost ([::1]:58976 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e1bhN-0006wd-F3\n\tfor incoming@patchwork.ozlabs.org; Mon, 09 Oct 2017 13:18:37 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:49122)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e1bgy-0006vF-Jf\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 13:18:14 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e1bgx-00006v-5M\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 13:18:12 -0400","from mail-wm0-x235.google.com ([2a00:1450:400c:c09::235]:55992)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1e1bgw-00006M-Rp\n\tfor qemu-devel@nongnu.org; Mon, 09 Oct 2017 13:18:11 -0400","by mail-wm0-x235.google.com with SMTP id u138so25513707wmu.4\n\tfor ; Mon, 09 Oct 2017 10:18:10 -0700 (PDT)","by 10.223.128.207 with HTTP; Mon, 9 Oct 2017 10:17:48 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=XywMrvMUhs7xz+GG/htLQm8axpyqvWmNgoqAHVFa7dw=;\n\tb=ZiOZG2EtQR4FjT3eZqEAmMFmXezo8RqMIs1MM1JjP5Wi65WCOXcvzay+DEMlezot4L\n\tgqqXpcEpQK6uZH1Y6KFoa2y9RngmX8AeyOSIgUHUWe1FRIEDE3nDlQFOwchCCGTyIRau\n\tOnaLptwtU2EO/O54140+iU8FHAWVDPC5IjAFQ=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=XywMrvMUhs7xz+GG/htLQm8axpyqvWmNgoqAHVFa7dw=;\n\tb=P9dWOWs39np3zCNDdqPzSMM9iI96LMDj4dw6ptgXNOEdqr+qVTUdOfWdfySwLSLrow\n\tt9eO6xQhGnU9eA55I6EV/L7VMFJ60VCq681Xza04uSARoKt9r3g6LxxWaXeIVKXzI/uF\n\tZV4Eb4UtVDt7u3sL3gQ8BterEdNnD62y32kNYOe8qKNrjmyscnVOxfyW6U3VvXtSlR/j\n\tAjcxH9PNY8ElW61dNbMWVEeCGZ3OMz+Z5rTKffaG2UkybOo4+N5U5lpjIalRGRR51B4Y\n\tbD42EkGAUak4LnKyJWYHfGSPKIfaOmVUEIFwfucqYA8sEHx/5Tnmw4f/5DFY/yk4BYVc\n\tYE4g==","X-Gm-Message-State":"AMCzsaWcnslkzEJXfNkqAaIeAMPIU+1aoptsIbNdOd3ksJYHHm9/Ln9+\n\tMgOXrKGrHIRKEv5Xy5vgRmGvtO6wqsgko8OKPzYevQ==","X-Google-Smtp-Source":"AOwi7QAMiz3wwNQqdN3Yh1P0vAuKnpqkyskG1xhT3Rfko6MOi5p6hAGUlQNiFwHyqF4ZZ+BMHi+6fF3702qwiHHIQZ4=","X-Received":"by 10.28.113.196 with SMTP id d65mr8035556wmi.151.1507569489512; \n\tMon, 09 Oct 2017 10:18:09 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<1504286483-23327-9-git-send-email-eric.auger@redhat.com>","References":"<1504286483-23327-1-git-send-email-eric.auger@redhat.com>\n\t<1504286483-23327-9-git-send-email-eric.auger@redhat.com>","From":"Peter Maydell ","Date":"Mon, 9 Oct 2017 18:17:48 +0100","Message-ID":"","To":"Eric Auger ","Content-Type":"text/plain; charset=\"UTF-8\"","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c09::235","Subject":"Re: [Qemu-devel] [PATCH v7 08/20] hw/arm/smmuv3: Implement MMIO\n\twrite operations","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Radha Mohan , Andrew Jones ,\n\tTrey Cain ,\n\tRadha.Chintakuntla@cavium.com, Sunil.Goutham@cavium.com,\n\t\"Michael S. Tsirkin\" ,\n\tjean-philippe.brucker@arm.com, Tomasz Nowicki ,\n\tWill Deacon ,\n\tQEMU Developers , Peter Xu ,\n\tAlex Williamson , \n\tqemu-arm ,\n\tChristoffer Dall ,\n\t\"Edgar E. Iglesias\" ,\n\trobin.murphy@arm.com, wtownsen@redhat.com,\n\tBharat Bhushan ,\n\tPrem Mallappa , eric.auger.pro@gmail.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]