mbox series

[for-5.0,v11,00/20] VIRTIO-IOMMU device

Message ID 20191122182943.4656-1-eric.auger@redhat.com
Headers show
Series VIRTIO-IOMMU device | expand

Message

Eric Auger Nov. 22, 2019, 6:29 p.m. UTC 
This series implements the QEMU virtio-iommu device.

This matches the v0.12 spec and the corresponding virtio-iommu
driver upstreamed in 5.3.

The pci proxy for the virtio-iommu device is instantiated using
"-device virtio-iommu-pci". This series still relies on ACPI IORT/DT
integration. Note the ACPI IORT integration is not yet upstreamed
and testing needs to be based on Jean-Philippe's additional
kernel patches [1].

Work is ongoing to remove IORT adherence and allow the
bindings between the IOMMU and the root complex to be defined
and written into the PCI device configuration space. The outcome
of this work is uncertain at this stage though. See [2].

So only patches 1-11 fully rely on upstreamed kernel code. Others
should be considered as RFC.

This respin allows people to test on ARM and x86. It also
brings migration support (tested on ARM) and various cleanups.
Reserved regions are now passed through an array of properties.
A libqos test also is introduced to test the virtio-iommu API.

Note integration with vhost devices and vfio devices is not part
of this series. Please follow Bharat's respins [3].

The 1st Patch ("migration: Support QLIST migration") was sent
separately [4].

Best Regards

Eric

This series can be found at:
https://github.com/eauger/qemu/tree/v4.2-rc2-virtio-iommu-v11

[1] kernel branch to be used for guest
    https://github.com/eauger/linux/tree/v5.4-rc8-virtio-iommu-iort
[2] [RFC 00/13] virtio-iommu on non-devicetree platforms
[3] VFIO/VHOST integration is not part of this series. Please follow
    [PATCH RFC v5 0/5] virtio-iommu: VFIO integration respins
[4] [PATCH v6] migration: Support QLIST migration

Testing:
- tested with guest using virtio-net-pci
  (,vhost=off,iommu_platform,disable-modern=off,disable-legacy=on)
  and virtio-blk-pci
- migration on ARM
- on x86 PC machine I get some AHCI non translated transactions,
  very early. This does not prevent the guest from boot and behaving
  properly. Warnings look like:
qemu-system-x86_64: virtio_iommu_translate sid=250 is not known!!
qemu-system-x86_64: no buffer available in event queue to report event
qemu-system-x86_64: AHCI: Failed to start FIS receive engine: bad FIS
receive buffer address

History:

v10 -> v11:
- introduce virtio_iommu_handle_req macro
- migration support
- introduce DEFINE_PROP_INTERVAL and pass reserved regions
  through an array of those
- domain gtree simplification

v9 -> v10:
- rebase on 4.1.0-rc2, compliance with 0.12 spec
- removed ACPI part
- cleanup (see individual change logs)
- moved to a PATCH series

v8 -> v9:
- virtio-iommu-pci device needs to be instantiated from the command
  line (RID is not imposed anymore).
- tail structure properly initialized

v7 -> v8:
- virtio-iommu-pci added
- virt instantiation modified
- DT and ACPI modified to exclude the iommu RID from the mapping
- VIRTIO_IOMMU_F_BYPASS, VIRTIO_F_VERSION_1 features exposed

v6 -> v7:
- rebase on qemu 3.0.0-rc3
- minor update against v0.7
- fix issue with EP not on pci.0 and ACPI probing
- change the instantiation method

v5 -> v6:
- minor update against v0.6 spec
- fix g_hash_table_lookup in virtio_iommu_find_add_as
- replace some error_reports by qemu_log_mask(LOG_GUEST_ERROR, ...)

v4 -> v5:
- event queue and fault reporting
- we now return the IOAPIC MSI region if the virtio-iommu is instantiated
  in a PC machine.
- we bypass transactions on MSI HW region and fault on reserved ones.
- We support ACPI boot with mach-virt (based on IORT proposal)
- We moved to the new driver naming conventions
- simplified mach-virt instantiation
- worked around the disappearing of pci_find_primary_bus
- in virtio_iommu_translate, check the dev->as is not NULL
- initialize as->device_list in virtio_iommu_get_as
- initialize bufstate.error to false in virtio_iommu_probe

v3 -> v4:
- probe request support although no reserved region is returned at
  the moment
- unmap semantics less strict, as specified in v0.4
- device registration, attach/detach revisited
- split into smaller patches to ease review
- propose a way to inform the IOMMU mr about the page_size_mask
  of underlying HW IOMMU, if any
- remove warning associated with the translation of the MSI doorbell

v2 -> v3:
- rebase on top of 2.10-rc0 and especially
  [PATCH qemu v9 0/2] memory/iommu: QOM'fy IOMMU MemoryRegion
- add mutex init
- fix as->mappings deletion using g_tree_ref/unref
- when a dev is attached whereas it is already attached to
  another address space, first detach it
- fix some error values
- page_sizes = TARGET_PAGE_MASK;
- I haven't changed the unmap() semantics yet, waiting for the
  next virtio-iommu spec revision.

v1 -> v2:
- fix redefinition of viommu_as typedef



Eric Auger (20):
  migration: Support QLIST migration
  virtio-iommu: Add skeleton
  virtio-iommu: Decode the command payload
  virtio-iommu: Add the iommu regions
  virtio-iommu: Endpoint and domains structs and helpers
  virtio-iommu: Implement attach/detach command
  virtio-iommu: Implement map/unmap
  virtio-iommu: Implement translate
  virtio-iommu: Implement fault reporting
  virtio-iommu-pci: Add virtio iommu pci support
  hw/arm/virt: Add the virtio-iommu device tree mappings
  qapi: Introduce DEFINE_PROP_INTERVAL
  virtio-iommu: Implement probe request
  virtio-iommu: Handle reserved regions in the translation process
  virtio-iommu-pci: Add array of Interval properties
  hw/arm/virt-acpi-build: Introduce fill_iort_idmap helper
  hw/arm/virt-acpi-build: Add virtio-iommu node in IORT table
  virtio-iommu: Support migration
  pc: Add support for virtio-iommu-pci
  tests: Add virtio-iommu test

 hw/arm/virt-acpi-build.c         |  91 ++-
 hw/arm/virt.c                    |  53 +-
 hw/core/qdev-properties.c        |  90 +++
 hw/i386/acpi-build.c             |  72 +++
 hw/i386/pc.c                     |  15 +-
 hw/virtio/Kconfig                |   5 +
 hw/virtio/Makefile.objs          |   2 +
 hw/virtio/trace-events           |  22 +
 hw/virtio/virtio-iommu-pci.c     |  91 +++
 hw/virtio/virtio-iommu.c         | 952 +++++++++++++++++++++++++++++++
 include/exec/memory.h            |   6 +
 include/hw/acpi/acpi-defs.h      |  21 +-
 include/hw/arm/virt.h            |   2 +
 include/hw/i386/pc.h             |   2 +
 include/hw/pci/pci.h             |   1 +
 include/hw/qdev-properties.h     |   3 +
 include/hw/virtio/virtio-iommu.h |  67 +++
 include/migration/vmstate.h      |  21 +
 include/qemu/queue.h             |  39 ++
 include/qemu/typedefs.h          |   1 +
 migration/trace-events           |   5 +
 migration/vmstate-types.c        |  70 +++
 qdev-monitor.c                   |   1 +
 tests/Makefile.include           |   2 +
 tests/libqos/virtio-iommu.c      | 177 ++++++
 tests/libqos/virtio-iommu.h      |  45 ++
 tests/test-vmstate.c             | 170 ++++++
 tests/virtio-iommu-test.c        | 261 +++++++++
 28 files changed, 2253 insertions(+), 34 deletions(-)
 create mode 100644 hw/virtio/virtio-iommu-pci.c
 create mode 100644 hw/virtio/virtio-iommu.c
 create mode 100644 include/hw/virtio/virtio-iommu.h
 create mode 100644 tests/libqos/virtio-iommu.c
 create mode 100644 tests/libqos/virtio-iommu.h
 create mode 100644 tests/virtio-iommu-test.c

Comments

no-reply@patchew.org Nov. 22, 2019, 9:56 p.m. UTC | #1 
Patchew URL: https://patchew.org/QEMU/20191122182943.4656-1-eric.auger@redhat.com/



Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

  CC      x86_64-softmmu/hw/virtio/virtio-scsi-pci.o
  CC      x86_64-softmmu/hw/virtio/virtio-blk-pci.o
/tmp/qemu-test/src/hw/virtio/virtio-iommu.c: In function 'int_cmp':
/tmp/qemu-test/src/hw/virtio/virtio-iommu.c:697:5: error: unknown type name 'uint'; did you mean 'guint'?
     uint ua = GPOINTER_TO_UINT(a);
     ^~~~
     guint
/tmp/qemu-test/src/hw/virtio/virtio-iommu.c:698:5: error: unknown type name 'uint'; did you mean 'guint'?
     uint ub = GPOINTER_TO_UINT(b);
     ^~~~
     guint
make[1]: *** [/tmp/qemu-test/src/rules.mak:69: hw/virtio/virtio-iommu.o] Error 1
make[1]: *** Waiting for unfinished jobs....
  CC      aarch64-softmmu/accel/tcg/tcg-runtime.o
  CC      aarch64-softmmu/accel/tcg/tcg-runtime-gvec.o
---
  CC      aarch64-softmmu/hw/block/dataplane/virtio-blk.o
  CC      aarch64-softmmu/hw/char/exynos4210_uart.o
  CC      aarch64-softmmu/hw/char/omap_uart.o
make: *** [Makefile:491: x86_64-softmmu/all] Error 2
make: *** Waiting for unfinished jobs....
  CC      aarch64-softmmu/hw/char/digic-uart.o
  CC      aarch64-softmmu/hw/char/stm32f2xx_usart.o
---
  CC      aarch64-softmmu/hw/arm/boot.o
  CC      aarch64-softmmu/hw/arm/sysbus-fdt.o
/tmp/qemu-test/src/hw/virtio/virtio-iommu.c: In function 'int_cmp':
/tmp/qemu-test/src/hw/virtio/virtio-iommu.c:697:5: error: unknown type name 'uint'; did you mean 'guint'?
     uint ua = GPOINTER_TO_UINT(a);
     ^~~~
     guint
/tmp/qemu-test/src/hw/virtio/virtio-iommu.c:698:5: error: unknown type name 'uint'; did you mean 'guint'?
     uint ub = GPOINTER_TO_UINT(b);
     ^~~~
     guint
make[1]: *** [/tmp/qemu-test/src/rules.mak:69: hw/virtio/virtio-iommu.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:491: aarch64-softmmu/all] Error 2
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 662, in <module>
    sys.exit(main())
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=028df3325d7c4927a1c334040016728f', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-cuthbude/src/docker-src.2019-11-22-16.49.52.5210:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=028df3325d7c4927a1c334040016728f
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-cuthbude/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real    6m21.012s
user    0m8.319s


The full log is available at
http://patchew.org/logs/20191122182943.4656-1-eric.auger@redhat.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Michael S. Tsirkin Dec. 11, 2019, 4:40 p.m. UTC | #2 
On Fri, Nov 22, 2019 at 07:29:23PM +0100, Eric Auger wrote:
> This series implements the QEMU virtio-iommu device.
> 
> This matches the v0.12 spec and the corresponding virtio-iommu
> driver upstreamed in 5.3.
> 
> The pci proxy for the virtio-iommu device is instantiated using
> "-device virtio-iommu-pci". This series still relies on ACPI IORT/DT
> integration. Note the ACPI IORT integration is not yet upstreamed
> and testing needs to be based on Jean-Philippe's additional
> kernel patches [1].

Or the config space approach? I really liked that one.

> 
> Work is ongoing to remove IORT adherence and allow the
> bindings between the IOMMU and the root complex to be defined
> and written into the PCI device configuration space. The outcome
> of this work is uncertain at this stage though. See [2].
> 
> So only patches 1-11 fully rely on upstreamed kernel code. Others
> should be considered as RFC.
> 
> This respin allows people to test on ARM and x86. It also
> brings migration support (tested on ARM) and various cleanups.
> Reserved regions are now passed through an array of properties.
> A libqos test also is introduced to test the virtio-iommu API.
> 
> Note integration with vhost devices and vfio devices is not part
> of this series. Please follow Bharat's respins [3].
> 
> The 1st Patch ("migration: Support QLIST migration") was sent
> separately [4].
> 
> Best Regards
> 
> Eric
> 
> This series can be found at:
> https://github.com/eauger/qemu/tree/v4.2-rc2-virtio-iommu-v11
> 
> [1] kernel branch to be used for guest
>     https://github.com/eauger/linux/tree/v5.4-rc8-virtio-iommu-iort
> [2] [RFC 00/13] virtio-iommu on non-devicetree platforms
> [3] VFIO/VHOST integration is not part of this series. Please follow
>     [PATCH RFC v5 0/5] virtio-iommu: VFIO integration respins
> [4] [PATCH v6] migration: Support QLIST migration
> 
> Testing:
> - tested with guest using virtio-net-pci
>   (,vhost=off,iommu_platform,disable-modern=off,disable-legacy=on)
>   and virtio-blk-pci
> - migration on ARM
> - on x86 PC machine I get some AHCI non translated transactions,
>   very early. This does not prevent the guest from boot and behaving
>   properly. Warnings look like:
> qemu-system-x86_64: virtio_iommu_translate sid=250 is not known!!
> qemu-system-x86_64: no buffer available in event queue to report event
> qemu-system-x86_64: AHCI: Failed to start FIS receive engine: bad FIS
> receive buffer address
> 
> History:
> 
> v10 -> v11:
> - introduce virtio_iommu_handle_req macro
> - migration support
> - introduce DEFINE_PROP_INTERVAL and pass reserved regions
>   through an array of those
> - domain gtree simplification
> 
> v9 -> v10:
> - rebase on 4.1.0-rc2, compliance with 0.12 spec
> - removed ACPI part
> - cleanup (see individual change logs)
> - moved to a PATCH series
> 
> v8 -> v9:
> - virtio-iommu-pci device needs to be instantiated from the command
>   line (RID is not imposed anymore).
> - tail structure properly initialized
> 
> v7 -> v8:
> - virtio-iommu-pci added
> - virt instantiation modified
> - DT and ACPI modified to exclude the iommu RID from the mapping
> - VIRTIO_IOMMU_F_BYPASS, VIRTIO_F_VERSION_1 features exposed
> 
> v6 -> v7:
> - rebase on qemu 3.0.0-rc3
> - minor update against v0.7
> - fix issue with EP not on pci.0 and ACPI probing
> - change the instantiation method
> 
> v5 -> v6:
> - minor update against v0.6 spec
> - fix g_hash_table_lookup in virtio_iommu_find_add_as
> - replace some error_reports by qemu_log_mask(LOG_GUEST_ERROR, ...)
> 
> v4 -> v5:
> - event queue and fault reporting
> - we now return the IOAPIC MSI region if the virtio-iommu is instantiated
>   in a PC machine.
> - we bypass transactions on MSI HW region and fault on reserved ones.
> - We support ACPI boot with mach-virt (based on IORT proposal)
> - We moved to the new driver naming conventions
> - simplified mach-virt instantiation
> - worked around the disappearing of pci_find_primary_bus
> - in virtio_iommu_translate, check the dev->as is not NULL
> - initialize as->device_list in virtio_iommu_get_as
> - initialize bufstate.error to false in virtio_iommu_probe
> 
> v3 -> v4:
> - probe request support although no reserved region is returned at
>   the moment
> - unmap semantics less strict, as specified in v0.4
> - device registration, attach/detach revisited
> - split into smaller patches to ease review
> - propose a way to inform the IOMMU mr about the page_size_mask
>   of underlying HW IOMMU, if any
> - remove warning associated with the translation of the MSI doorbell
> 
> v2 -> v3:
> - rebase on top of 2.10-rc0 and especially
>   [PATCH qemu v9 0/2] memory/iommu: QOM'fy IOMMU MemoryRegion
> - add mutex init
> - fix as->mappings deletion using g_tree_ref/unref
> - when a dev is attached whereas it is already attached to
>   another address space, first detach it
> - fix some error values
> - page_sizes = TARGET_PAGE_MASK;
> - I haven't changed the unmap() semantics yet, waiting for the
>   next virtio-iommu spec revision.
> 
> v1 -> v2:
> - fix redefinition of viommu_as typedef
> 
> 
> 
> Eric Auger (20):
>   migration: Support QLIST migration
>   virtio-iommu: Add skeleton
>   virtio-iommu: Decode the command payload
>   virtio-iommu: Add the iommu regions
>   virtio-iommu: Endpoint and domains structs and helpers
>   virtio-iommu: Implement attach/detach command
>   virtio-iommu: Implement map/unmap
>   virtio-iommu: Implement translate
>   virtio-iommu: Implement fault reporting
>   virtio-iommu-pci: Add virtio iommu pci support
>   hw/arm/virt: Add the virtio-iommu device tree mappings
>   qapi: Introduce DEFINE_PROP_INTERVAL
>   virtio-iommu: Implement probe request
>   virtio-iommu: Handle reserved regions in the translation process
>   virtio-iommu-pci: Add array of Interval properties
>   hw/arm/virt-acpi-build: Introduce fill_iort_idmap helper
>   hw/arm/virt-acpi-build: Add virtio-iommu node in IORT table
>   virtio-iommu: Support migration
>   pc: Add support for virtio-iommu-pci
>   tests: Add virtio-iommu test
> 
>  hw/arm/virt-acpi-build.c         |  91 ++-
>  hw/arm/virt.c                    |  53 +-
>  hw/core/qdev-properties.c        |  90 +++
>  hw/i386/acpi-build.c             |  72 +++
>  hw/i386/pc.c                     |  15 +-
>  hw/virtio/Kconfig                |   5 +
>  hw/virtio/Makefile.objs          |   2 +
>  hw/virtio/trace-events           |  22 +
>  hw/virtio/virtio-iommu-pci.c     |  91 +++
>  hw/virtio/virtio-iommu.c         | 952 +++++++++++++++++++++++++++++++
>  include/exec/memory.h            |   6 +
>  include/hw/acpi/acpi-defs.h      |  21 +-
>  include/hw/arm/virt.h            |   2 +
>  include/hw/i386/pc.h             |   2 +
>  include/hw/pci/pci.h             |   1 +
>  include/hw/qdev-properties.h     |   3 +
>  include/hw/virtio/virtio-iommu.h |  67 +++
>  include/migration/vmstate.h      |  21 +
>  include/qemu/queue.h             |  39 ++
>  include/qemu/typedefs.h          |   1 +
>  migration/trace-events           |   5 +
>  migration/vmstate-types.c        |  70 +++
>  qdev-monitor.c                   |   1 +
>  tests/Makefile.include           |   2 +
>  tests/libqos/virtio-iommu.c      | 177 ++++++
>  tests/libqos/virtio-iommu.h      |  45 ++
>  tests/test-vmstate.c             | 170 ++++++
>  tests/virtio-iommu-test.c        | 261 +++++++++
>  28 files changed, 2253 insertions(+), 34 deletions(-)
>  create mode 100644 hw/virtio/virtio-iommu-pci.c
>  create mode 100644 hw/virtio/virtio-iommu.c
>  create mode 100644 include/hw/virtio/virtio-iommu.h
>  create mode 100644 tests/libqos/virtio-iommu.c
>  create mode 100644 tests/libqos/virtio-iommu.h
>  create mode 100644 tests/virtio-iommu-test.c
> 
> -- 
> 2.20.1
Eric Auger Dec. 11, 2019, 4:48 p.m. UTC | #3 
Hi Michael,

On 12/11/19 5:40 PM, Michael S. Tsirkin wrote:
> On Fri, Nov 22, 2019 at 07:29:23PM +0100, Eric Auger wrote:
>> This series implements the QEMU virtio-iommu device.
>>
>> This matches the v0.12 spec and the corresponding virtio-iommu
>> driver upstreamed in 5.3.
>>
>> The pci proxy for the virtio-iommu device is instantiated using
>> "-device virtio-iommu-pci". This series still relies on ACPI IORT/DT
>> integration. Note the ACPI IORT integration is not yet upstreamed
>> and testing needs to be based on Jean-Philippe's additional
>> kernel patches [1].
> 
> Or the config space approach? I really liked that one.
Yes this corresponds to the paragraph below.
> 
>>
>> Work is ongoing to remove IORT adherence and allow the
>> bindings between the IOMMU and the root complex to be defined
>> and written into the PCI device configuration space. The outcome
>> of this work is uncertain at this stage though. See [2].

Thanks

Eric

>>
>> So only patches 1-11 fully rely on upstreamed kernel code. Others
>> should be considered as RFC.
>>
>> This respin allows people to test on ARM and x86. It also
>> brings migration support (tested on ARM) and various cleanups.
>> Reserved regions are now passed through an array of properties.
>> A libqos test also is introduced to test the virtio-iommu API.
>>
>> Note integration with vhost devices and vfio devices is not part
>> of this series. Please follow Bharat's respins [3].
>>
>> The 1st Patch ("migration: Support QLIST migration") was sent
>> separately [4].
>>
>> Best Regards
>>
>> Eric
>>
>> This series can be found at:
>> https://github.com/eauger/qemu/tree/v4.2-rc2-virtio-iommu-v11
>>
>> [1] kernel branch to be used for guest
>>     https://github.com/eauger/linux/tree/v5.4-rc8-virtio-iommu-iort
>> [2] [RFC 00/13] virtio-iommu on non-devicetree platforms
>> [3] VFIO/VHOST integration is not part of this series. Please follow
>>     [PATCH RFC v5 0/5] virtio-iommu: VFIO integration respins
>> [4] [PATCH v6] migration: Support QLIST migration
>>
>> Testing:
>> - tested with guest using virtio-net-pci
>>   (,vhost=off,iommu_platform,disable-modern=off,disable-legacy=on)
>>   and virtio-blk-pci
>> - migration on ARM
>> - on x86 PC machine I get some AHCI non translated transactions,
>>   very early. This does not prevent the guest from boot and behaving
>>   properly. Warnings look like:
>> qemu-system-x86_64: virtio_iommu_translate sid=250 is not known!!
>> qemu-system-x86_64: no buffer available in event queue to report event
>> qemu-system-x86_64: AHCI: Failed to start FIS receive engine: bad FIS
>> receive buffer address
>>
>> History:
>>
>> v10 -> v11:
>> - introduce virtio_iommu_handle_req macro
>> - migration support
>> - introduce DEFINE_PROP_INTERVAL and pass reserved regions
>>   through an array of those
>> - domain gtree simplification
>>
>> v9 -> v10:
>> - rebase on 4.1.0-rc2, compliance with 0.12 spec
>> - removed ACPI part
>> - cleanup (see individual change logs)
>> - moved to a PATCH series
>>
>> v8 -> v9:
>> - virtio-iommu-pci device needs to be instantiated from the command
>>   line (RID is not imposed anymore).
>> - tail structure properly initialized
>>
>> v7 -> v8:
>> - virtio-iommu-pci added
>> - virt instantiation modified
>> - DT and ACPI modified to exclude the iommu RID from the mapping
>> - VIRTIO_IOMMU_F_BYPASS, VIRTIO_F_VERSION_1 features exposed
>>
>> v6 -> v7:
>> - rebase on qemu 3.0.0-rc3
>> - minor update against v0.7
>> - fix issue with EP not on pci.0 and ACPI probing
>> - change the instantiation method
>>
>> v5 -> v6:
>> - minor update against v0.6 spec
>> - fix g_hash_table_lookup in virtio_iommu_find_add_as
>> - replace some error_reports by qemu_log_mask(LOG_GUEST_ERROR, ...)
>>
>> v4 -> v5:
>> - event queue and fault reporting
>> - we now return the IOAPIC MSI region if the virtio-iommu is instantiated
>>   in a PC machine.
>> - we bypass transactions on MSI HW region and fault on reserved ones.
>> - We support ACPI boot with mach-virt (based on IORT proposal)
>> - We moved to the new driver naming conventions
>> - simplified mach-virt instantiation
>> - worked around the disappearing of pci_find_primary_bus
>> - in virtio_iommu_translate, check the dev->as is not NULL
>> - initialize as->device_list in virtio_iommu_get_as
>> - initialize bufstate.error to false in virtio_iommu_probe
>>
>> v3 -> v4:
>> - probe request support although no reserved region is returned at
>>   the moment
>> - unmap semantics less strict, as specified in v0.4
>> - device registration, attach/detach revisited
>> - split into smaller patches to ease review
>> - propose a way to inform the IOMMU mr about the page_size_mask
>>   of underlying HW IOMMU, if any
>> - remove warning associated with the translation of the MSI doorbell
>>
>> v2 -> v3:
>> - rebase on top of 2.10-rc0 and especially
>>   [PATCH qemu v9 0/2] memory/iommu: QOM'fy IOMMU MemoryRegion
>> - add mutex init
>> - fix as->mappings deletion using g_tree_ref/unref
>> - when a dev is attached whereas it is already attached to
>>   another address space, first detach it
>> - fix some error values
>> - page_sizes = TARGET_PAGE_MASK;
>> - I haven't changed the unmap() semantics yet, waiting for the
>>   next virtio-iommu spec revision.
>>
>> v1 -> v2:
>> - fix redefinition of viommu_as typedef
>>
>>
>>
>> Eric Auger (20):
>>   migration: Support QLIST migration
>>   virtio-iommu: Add skeleton
>>   virtio-iommu: Decode the command payload
>>   virtio-iommu: Add the iommu regions
>>   virtio-iommu: Endpoint and domains structs and helpers
>>   virtio-iommu: Implement attach/detach command
>>   virtio-iommu: Implement map/unmap
>>   virtio-iommu: Implement translate
>>   virtio-iommu: Implement fault reporting
>>   virtio-iommu-pci: Add virtio iommu pci support
>>   hw/arm/virt: Add the virtio-iommu device tree mappings
>>   qapi: Introduce DEFINE_PROP_INTERVAL
>>   virtio-iommu: Implement probe request
>>   virtio-iommu: Handle reserved regions in the translation process
>>   virtio-iommu-pci: Add array of Interval properties
>>   hw/arm/virt-acpi-build: Introduce fill_iort_idmap helper
>>   hw/arm/virt-acpi-build: Add virtio-iommu node in IORT table
>>   virtio-iommu: Support migration
>>   pc: Add support for virtio-iommu-pci
>>   tests: Add virtio-iommu test
>>
>>  hw/arm/virt-acpi-build.c         |  91 ++-
>>  hw/arm/virt.c                    |  53 +-
>>  hw/core/qdev-properties.c        |  90 +++
>>  hw/i386/acpi-build.c             |  72 +++
>>  hw/i386/pc.c                     |  15 +-
>>  hw/virtio/Kconfig                |   5 +
>>  hw/virtio/Makefile.objs          |   2 +
>>  hw/virtio/trace-events           |  22 +
>>  hw/virtio/virtio-iommu-pci.c     |  91 +++
>>  hw/virtio/virtio-iommu.c         | 952 +++++++++++++++++++++++++++++++
>>  include/exec/memory.h            |   6 +
>>  include/hw/acpi/acpi-defs.h      |  21 +-
>>  include/hw/arm/virt.h            |   2 +
>>  include/hw/i386/pc.h             |   2 +
>>  include/hw/pci/pci.h             |   1 +
>>  include/hw/qdev-properties.h     |   3 +
>>  include/hw/virtio/virtio-iommu.h |  67 +++
>>  include/migration/vmstate.h      |  21 +
>>  include/qemu/queue.h             |  39 ++
>>  include/qemu/typedefs.h          |   1 +
>>  migration/trace-events           |   5 +
>>  migration/vmstate-types.c        |  70 +++
>>  qdev-monitor.c                   |   1 +
>>  tests/Makefile.include           |   2 +
>>  tests/libqos/virtio-iommu.c      | 177 ++++++
>>  tests/libqos/virtio-iommu.h      |  45 ++
>>  tests/test-vmstate.c             | 170 ++++++
>>  tests/virtio-iommu-test.c        | 261 +++++++++
>>  28 files changed, 2253 insertions(+), 34 deletions(-)
>>  create mode 100644 hw/virtio/virtio-iommu-pci.c
>>  create mode 100644 hw/virtio/virtio-iommu.c
>>  create mode 100644 include/hw/virtio/virtio-iommu.h
>>  create mode 100644 tests/libqos/virtio-iommu.c
>>  create mode 100644 tests/libqos/virtio-iommu.h
>>  create mode 100644 tests/virtio-iommu-test.c
>>
>> -- 
>> 2.20.1
> 
>
Michael S. Tsirkin Dec. 11, 2019, 8:40 p.m. UTC | #4 
On Wed, Dec 11, 2019 at 05:48:05PM +0100, Auger Eric wrote:
> Hi Michael,
> 
> On 12/11/19 5:40 PM, Michael S. Tsirkin wrote:
> > On Fri, Nov 22, 2019 at 07:29:23PM +0100, Eric Auger wrote:
> >> This series implements the QEMU virtio-iommu device.
> >>
> >> This matches the v0.12 spec and the corresponding virtio-iommu
> >> driver upstreamed in 5.3.
> >>
> >> The pci proxy for the virtio-iommu device is instantiated using
> >> "-device virtio-iommu-pci". This series still relies on ACPI IORT/DT
> >> integration. Note the ACPI IORT integration is not yet upstreamed
> >> and testing needs to be based on Jean-Philippe's additional
> >> kernel patches [1].
> > 
> > Or the config space approach? I really liked that one.
> Yes this corresponds to the paragraph below.
> > 
> >>
> >> Work is ongoing to remove IORT adherence and allow the
> >> bindings between the IOMMU and the root complex to be defined
> >> and written into the PCI device configuration space. The outcome
> >> of this work is uncertain at this stage though. See [2].

Oh right. Why is it uncertain? Anything can be done to help?

> Thanks
> 
> Eric
> 
> >>
> >> So only patches 1-11 fully rely on upstreamed kernel code. Others
> >> should be considered as RFC.
> >>
> >> This respin allows people to test on ARM and x86. It also
> >> brings migration support (tested on ARM) and various cleanups.
> >> Reserved regions are now passed through an array of properties.
> >> A libqos test also is introduced to test the virtio-iommu API.
> >>
> >> Note integration with vhost devices and vfio devices is not part
> >> of this series. Please follow Bharat's respins [3].
> >>
> >> The 1st Patch ("migration: Support QLIST migration") was sent
> >> separately [4].
> >>
> >> Best Regards
> >>
> >> Eric
> >>
> >> This series can be found at:
> >> https://github.com/eauger/qemu/tree/v4.2-rc2-virtio-iommu-v11
> >>
> >> [1] kernel branch to be used for guest
> >>     https://github.com/eauger/linux/tree/v5.4-rc8-virtio-iommu-iort
> >> [2] [RFC 00/13] virtio-iommu on non-devicetree platforms
> >> [3] VFIO/VHOST integration is not part of this series. Please follow
> >>     [PATCH RFC v5 0/5] virtio-iommu: VFIO integration respins
> >> [4] [PATCH v6] migration: Support QLIST migration
> >>
> >> Testing:
> >> - tested with guest using virtio-net-pci
> >>   (,vhost=off,iommu_platform,disable-modern=off,disable-legacy=on)
> >>   and virtio-blk-pci
> >> - migration on ARM
> >> - on x86 PC machine I get some AHCI non translated transactions,
> >>   very early. This does not prevent the guest from boot and behaving
> >>   properly. Warnings look like:
> >> qemu-system-x86_64: virtio_iommu_translate sid=250 is not known!!
> >> qemu-system-x86_64: no buffer available in event queue to report event
> >> qemu-system-x86_64: AHCI: Failed to start FIS receive engine: bad FIS
> >> receive buffer address
> >>
> >> History:
> >>
> >> v10 -> v11:
> >> - introduce virtio_iommu_handle_req macro
> >> - migration support
> >> - introduce DEFINE_PROP_INTERVAL and pass reserved regions
> >>   through an array of those
> >> - domain gtree simplification
> >>
> >> v9 -> v10:
> >> - rebase on 4.1.0-rc2, compliance with 0.12 spec
> >> - removed ACPI part
> >> - cleanup (see individual change logs)
> >> - moved to a PATCH series
> >>
> >> v8 -> v9:
> >> - virtio-iommu-pci device needs to be instantiated from the command
> >>   line (RID is not imposed anymore).
> >> - tail structure properly initialized
> >>
> >> v7 -> v8:
> >> - virtio-iommu-pci added
> >> - virt instantiation modified
> >> - DT and ACPI modified to exclude the iommu RID from the mapping
> >> - VIRTIO_IOMMU_F_BYPASS, VIRTIO_F_VERSION_1 features exposed
> >>
> >> v6 -> v7:
> >> - rebase on qemu 3.0.0-rc3
> >> - minor update against v0.7
> >> - fix issue with EP not on pci.0 and ACPI probing
> >> - change the instantiation method
> >>
> >> v5 -> v6:
> >> - minor update against v0.6 spec
> >> - fix g_hash_table_lookup in virtio_iommu_find_add_as
> >> - replace some error_reports by qemu_log_mask(LOG_GUEST_ERROR, ...)
> >>
> >> v4 -> v5:
> >> - event queue and fault reporting
> >> - we now return the IOAPIC MSI region if the virtio-iommu is instantiated
> >>   in a PC machine.
> >> - we bypass transactions on MSI HW region and fault on reserved ones.
> >> - We support ACPI boot with mach-virt (based on IORT proposal)
> >> - We moved to the new driver naming conventions
> >> - simplified mach-virt instantiation
> >> - worked around the disappearing of pci_find_primary_bus
> >> - in virtio_iommu_translate, check the dev->as is not NULL
> >> - initialize as->device_list in virtio_iommu_get_as
> >> - initialize bufstate.error to false in virtio_iommu_probe
> >>
> >> v3 -> v4:
> >> - probe request support although no reserved region is returned at
> >>   the moment
> >> - unmap semantics less strict, as specified in v0.4
> >> - device registration, attach/detach revisited
> >> - split into smaller patches to ease review
> >> - propose a way to inform the IOMMU mr about the page_size_mask
> >>   of underlying HW IOMMU, if any
> >> - remove warning associated with the translation of the MSI doorbell
> >>
> >> v2 -> v3:
> >> - rebase on top of 2.10-rc0 and especially
> >>   [PATCH qemu v9 0/2] memory/iommu: QOM'fy IOMMU MemoryRegion
> >> - add mutex init
> >> - fix as->mappings deletion using g_tree_ref/unref
> >> - when a dev is attached whereas it is already attached to
> >>   another address space, first detach it
> >> - fix some error values
> >> - page_sizes = TARGET_PAGE_MASK;
> >> - I haven't changed the unmap() semantics yet, waiting for the
> >>   next virtio-iommu spec revision.
> >>
> >> v1 -> v2:
> >> - fix redefinition of viommu_as typedef
> >>
> >>
> >>
> >> Eric Auger (20):
> >>   migration: Support QLIST migration
> >>   virtio-iommu: Add skeleton
> >>   virtio-iommu: Decode the command payload
> >>   virtio-iommu: Add the iommu regions
> >>   virtio-iommu: Endpoint and domains structs and helpers
> >>   virtio-iommu: Implement attach/detach command
> >>   virtio-iommu: Implement map/unmap
> >>   virtio-iommu: Implement translate
> >>   virtio-iommu: Implement fault reporting
> >>   virtio-iommu-pci: Add virtio iommu pci support
> >>   hw/arm/virt: Add the virtio-iommu device tree mappings
> >>   qapi: Introduce DEFINE_PROP_INTERVAL
> >>   virtio-iommu: Implement probe request
> >>   virtio-iommu: Handle reserved regions in the translation process
> >>   virtio-iommu-pci: Add array of Interval properties
> >>   hw/arm/virt-acpi-build: Introduce fill_iort_idmap helper
> >>   hw/arm/virt-acpi-build: Add virtio-iommu node in IORT table
> >>   virtio-iommu: Support migration
> >>   pc: Add support for virtio-iommu-pci
> >>   tests: Add virtio-iommu test
> >>
> >>  hw/arm/virt-acpi-build.c         |  91 ++-
> >>  hw/arm/virt.c                    |  53 +-
> >>  hw/core/qdev-properties.c        |  90 +++
> >>  hw/i386/acpi-build.c             |  72 +++
> >>  hw/i386/pc.c                     |  15 +-
> >>  hw/virtio/Kconfig                |   5 +
> >>  hw/virtio/Makefile.objs          |   2 +
> >>  hw/virtio/trace-events           |  22 +
> >>  hw/virtio/virtio-iommu-pci.c     |  91 +++
> >>  hw/virtio/virtio-iommu.c         | 952 +++++++++++++++++++++++++++++++
> >>  include/exec/memory.h            |   6 +
> >>  include/hw/acpi/acpi-defs.h      |  21 +-
> >>  include/hw/arm/virt.h            |   2 +
> >>  include/hw/i386/pc.h             |   2 +
> >>  include/hw/pci/pci.h             |   1 +
> >>  include/hw/qdev-properties.h     |   3 +
> >>  include/hw/virtio/virtio-iommu.h |  67 +++
> >>  include/migration/vmstate.h      |  21 +
> >>  include/qemu/queue.h             |  39 ++
> >>  include/qemu/typedefs.h          |   1 +
> >>  migration/trace-events           |   5 +
> >>  migration/vmstate-types.c        |  70 +++
> >>  qdev-monitor.c                   |   1 +
> >>  tests/Makefile.include           |   2 +
> >>  tests/libqos/virtio-iommu.c      | 177 ++++++
> >>  tests/libqos/virtio-iommu.h      |  45 ++
> >>  tests/test-vmstate.c             | 170 ++++++
> >>  tests/virtio-iommu-test.c        | 261 +++++++++
> >>  28 files changed, 2253 insertions(+), 34 deletions(-)
> >>  create mode 100644 hw/virtio/virtio-iommu-pci.c
> >>  create mode 100644 hw/virtio/virtio-iommu.c
> >>  create mode 100644 include/hw/virtio/virtio-iommu.h
> >>  create mode 100644 tests/libqos/virtio-iommu.c
> >>  create mode 100644 tests/libqos/virtio-iommu.h
> >>  create mode 100644 tests/virtio-iommu-test.c
> >>
> >> -- 
> >> 2.20.1
> > 
> >
Eric Auger Dec. 12, 2019, 3:05 p.m. UTC | #5 
Hi Michael,

On 12/11/19 9:40 PM, Michael S. Tsirkin wrote:
> On Wed, Dec 11, 2019 at 05:48:05PM +0100, Auger Eric wrote:
>> Hi Michael,
>>
>> On 12/11/19 5:40 PM, Michael S. Tsirkin wrote:
>>> On Fri, Nov 22, 2019 at 07:29:23PM +0100, Eric Auger wrote:
>>>> This series implements the QEMU virtio-iommu device.
>>>>
>>>> This matches the v0.12 spec and the corresponding virtio-iommu
>>>> driver upstreamed in 5.3.
>>>>
>>>> The pci proxy for the virtio-iommu device is instantiated using
>>>> "-device virtio-iommu-pci". This series still relies on ACPI IORT/DT
>>>> integration. Note the ACPI IORT integration is not yet upstreamed
>>>> and testing needs to be based on Jean-Philippe's additional
>>>> kernel patches [1].
>>>
>>> Or the config space approach? I really liked that one.
>> Yes this corresponds to the paragraph below.
>>>
>>>>
>>>> Work is ongoing to remove IORT adherence and allow the
>>>> bindings between the IOMMU and the root complex to be defined
>>>> and written into the PCI device configuration space. The outcome
>>>> of this work is uncertain at this stage though. See [2].
> 
> Oh right. Why is it uncertain? Anything can be done to help?

Jean's series was sent on the same day as this QEMU respin. My
understanding is we still need a way to handle platform devices. Also
the binding info layout needs to be revised and integrated into the spec
+ voted. Those are the uncertainties I wanted to point out.

Thanks

Eric
> 
>> Thanks
>>
>> Eric
>>
>>>>
>>>> So only patches 1-11 fully rely on upstreamed kernel code. Others
>>>> should be considered as RFC.
>>>>
>>>> This respin allows people to test on ARM and x86. It also
>>>> brings migration support (tested on ARM) and various cleanups.
>>>> Reserved regions are now passed through an array of properties.
>>>> A libqos test also is introduced to test the virtio-iommu API.
>>>>
>>>> Note integration with vhost devices and vfio devices is not part
>>>> of this series. Please follow Bharat's respins [3].
>>>>
>>>> The 1st Patch ("migration: Support QLIST migration") was sent
>>>> separately [4].
>>>>
>>>> Best Regards
>>>>
>>>> Eric
>>>>
>>>> This series can be found at:
>>>> https://github.com/eauger/qemu/tree/v4.2-rc2-virtio-iommu-v11
>>>>
>>>> [1] kernel branch to be used for guest
>>>>     https://github.com/eauger/linux/tree/v5.4-rc8-virtio-iommu-iort
>>>> [2] [RFC 00/13] virtio-iommu on non-devicetree platforms
>>>> [3] VFIO/VHOST integration is not part of this series. Please follow
>>>>     [PATCH RFC v5 0/5] virtio-iommu: VFIO integration respins
>>>> [4] [PATCH v6] migration: Support QLIST migration
>>>>
>>>> Testing:
>>>> - tested with guest using virtio-net-pci
>>>>   (,vhost=off,iommu_platform,disable-modern=off,disable-legacy=on)
>>>>   and virtio-blk-pci
>>>> - migration on ARM
>>>> - on x86 PC machine I get some AHCI non translated transactions,
>>>>   very early. This does not prevent the guest from boot and behaving
>>>>   properly. Warnings look like:
>>>> qemu-system-x86_64: virtio_iommu_translate sid=250 is not known!!
>>>> qemu-system-x86_64: no buffer available in event queue to report event
>>>> qemu-system-x86_64: AHCI: Failed to start FIS receive engine: bad FIS
>>>> receive buffer address
>>>>
>>>> History:
>>>>
>>>> v10 -> v11:
>>>> - introduce virtio_iommu_handle_req macro
>>>> - migration support
>>>> - introduce DEFINE_PROP_INTERVAL and pass reserved regions
>>>>   through an array of those
>>>> - domain gtree simplification
>>>>
>>>> v9 -> v10:
>>>> - rebase on 4.1.0-rc2, compliance with 0.12 spec
>>>> - removed ACPI part
>>>> - cleanup (see individual change logs)
>>>> - moved to a PATCH series
>>>>
>>>> v8 -> v9:
>>>> - virtio-iommu-pci device needs to be instantiated from the command
>>>>   line (RID is not imposed anymore).
>>>> - tail structure properly initialized
>>>>
>>>> v7 -> v8:
>>>> - virtio-iommu-pci added
>>>> - virt instantiation modified
>>>> - DT and ACPI modified to exclude the iommu RID from the mapping
>>>> - VIRTIO_IOMMU_F_BYPASS, VIRTIO_F_VERSION_1 features exposed
>>>>
>>>> v6 -> v7:
>>>> - rebase on qemu 3.0.0-rc3
>>>> - minor update against v0.7
>>>> - fix issue with EP not on pci.0 and ACPI probing
>>>> - change the instantiation method
>>>>
>>>> v5 -> v6:
>>>> - minor update against v0.6 spec
>>>> - fix g_hash_table_lookup in virtio_iommu_find_add_as
>>>> - replace some error_reports by qemu_log_mask(LOG_GUEST_ERROR, ...)
>>>>
>>>> v4 -> v5:
>>>> - event queue and fault reporting
>>>> - we now return the IOAPIC MSI region if the virtio-iommu is instantiated
>>>>   in a PC machine.
>>>> - we bypass transactions on MSI HW region and fault on reserved ones.
>>>> - We support ACPI boot with mach-virt (based on IORT proposal)
>>>> - We moved to the new driver naming conventions
>>>> - simplified mach-virt instantiation
>>>> - worked around the disappearing of pci_find_primary_bus
>>>> - in virtio_iommu_translate, check the dev->as is not NULL
>>>> - initialize as->device_list in virtio_iommu_get_as
>>>> - initialize bufstate.error to false in virtio_iommu_probe
>>>>
>>>> v3 -> v4:
>>>> - probe request support although no reserved region is returned at
>>>>   the moment
>>>> - unmap semantics less strict, as specified in v0.4
>>>> - device registration, attach/detach revisited
>>>> - split into smaller patches to ease review
>>>> - propose a way to inform the IOMMU mr about the page_size_mask
>>>>   of underlying HW IOMMU, if any
>>>> - remove warning associated with the translation of the MSI doorbell
>>>>
>>>> v2 -> v3:
>>>> - rebase on top of 2.10-rc0 and especially
>>>>   [PATCH qemu v9 0/2] memory/iommu: QOM'fy IOMMU MemoryRegion
>>>> - add mutex init
>>>> - fix as->mappings deletion using g_tree_ref/unref
>>>> - when a dev is attached whereas it is already attached to
>>>>   another address space, first detach it
>>>> - fix some error values
>>>> - page_sizes = TARGET_PAGE_MASK;
>>>> - I haven't changed the unmap() semantics yet, waiting for the
>>>>   next virtio-iommu spec revision.
>>>>
>>>> v1 -> v2:
>>>> - fix redefinition of viommu_as typedef
>>>>
>>>>
>>>>
>>>> Eric Auger (20):
>>>>   migration: Support QLIST migration
>>>>   virtio-iommu: Add skeleton
>>>>   virtio-iommu: Decode the command payload
>>>>   virtio-iommu: Add the iommu regions
>>>>   virtio-iommu: Endpoint and domains structs and helpers
>>>>   virtio-iommu: Implement attach/detach command
>>>>   virtio-iommu: Implement map/unmap
>>>>   virtio-iommu: Implement translate
>>>>   virtio-iommu: Implement fault reporting
>>>>   virtio-iommu-pci: Add virtio iommu pci support
>>>>   hw/arm/virt: Add the virtio-iommu device tree mappings
>>>>   qapi: Introduce DEFINE_PROP_INTERVAL
>>>>   virtio-iommu: Implement probe request
>>>>   virtio-iommu: Handle reserved regions in the translation process
>>>>   virtio-iommu-pci: Add array of Interval properties
>>>>   hw/arm/virt-acpi-build: Introduce fill_iort_idmap helper
>>>>   hw/arm/virt-acpi-build: Add virtio-iommu node in IORT table
>>>>   virtio-iommu: Support migration
>>>>   pc: Add support for virtio-iommu-pci
>>>>   tests: Add virtio-iommu test
>>>>
>>>>  hw/arm/virt-acpi-build.c         |  91 ++-
>>>>  hw/arm/virt.c                    |  53 +-
>>>>  hw/core/qdev-properties.c        |  90 +++
>>>>  hw/i386/acpi-build.c             |  72 +++
>>>>  hw/i386/pc.c                     |  15 +-
>>>>  hw/virtio/Kconfig                |   5 +
>>>>  hw/virtio/Makefile.objs          |   2 +
>>>>  hw/virtio/trace-events           |  22 +
>>>>  hw/virtio/virtio-iommu-pci.c     |  91 +++
>>>>  hw/virtio/virtio-iommu.c         | 952 +++++++++++++++++++++++++++++++
>>>>  include/exec/memory.h            |   6 +
>>>>  include/hw/acpi/acpi-defs.h      |  21 +-
>>>>  include/hw/arm/virt.h            |   2 +
>>>>  include/hw/i386/pc.h             |   2 +
>>>>  include/hw/pci/pci.h             |   1 +
>>>>  include/hw/qdev-properties.h     |   3 +
>>>>  include/hw/virtio/virtio-iommu.h |  67 +++
>>>>  include/migration/vmstate.h      |  21 +
>>>>  include/qemu/queue.h             |  39 ++
>>>>  include/qemu/typedefs.h          |   1 +
>>>>  migration/trace-events           |   5 +
>>>>  migration/vmstate-types.c        |  70 +++
>>>>  qdev-monitor.c                   |   1 +
>>>>  tests/Makefile.include           |   2 +
>>>>  tests/libqos/virtio-iommu.c      | 177 ++++++
>>>>  tests/libqos/virtio-iommu.h      |  45 ++
>>>>  tests/test-vmstate.c             | 170 ++++++
>>>>  tests/virtio-iommu-test.c        | 261 +++++++++
>>>>  28 files changed, 2253 insertions(+), 34 deletions(-)
>>>>  create mode 100644 hw/virtio/virtio-iommu-pci.c
>>>>  create mode 100644 hw/virtio/virtio-iommu.c
>>>>  create mode 100644 include/hw/virtio/virtio-iommu.h
>>>>  create mode 100644 tests/libqos/virtio-iommu.c
>>>>  create mode 100644 tests/libqos/virtio-iommu.h
>>>>  create mode 100644 tests/virtio-iommu-test.c
>>>>
>>>> -- 
>>>> 2.20.1
>>>
>>>
> 
>