| Message ID | cover.1607922214.git.jag.raman@oracle.com |
|---|---|
| Headers | show |
| Series | Initial support for multi-process Qemu | expand |
On Mon, Dec 14, 2020 at 12:14:40AM -0500, Jagannathan Raman wrote: > Hello, > > This is the v13 of the patchset. Thank you very much for reviewing v12 and > sharing your feedback. As mentioned on IRC, please add a ./configure option so package maintainers can easily disable multi-process QEMU without patching the source code. Thanks, Stefan
Hi,
In this series, we have incorporated the changes Marc-Andre proposed to
improve the code in v12. Following is a changelog that outlines the
differences between v12 and v13.
CHANGELOG:
--------------------
[PATCH v13 02/19] multi-process: add configure and usage information
- Dropped “tests/multiprocess/multiprocess-lsi53c895a.py” based on
feedback. We are working on acceptance tests instead.
[PATCH v13 03/19] memory: alloc RAM from file at offset
- Changed the name of argument “start” to “map_offset”
[PATCH v13 05/19] multi-process: setup PCI host bridge for remote device
- Renamed MULTIPROCESS_HOST as REMOTE_PCIHOST
- Also renamed TYPE_REMOTE_HOST_DEVICE and
REMOTE_HOST_DEVICE as TYPE_REMOTE_PCIHOST and
REMOTE_PCIHOST respectively
[PATCH v13 06/19] multi-process: setup a machine object for remote device
process
- Renamed “remote-device” as “remote-pcihost”
- Set a description for the machine class
[PATCH v13 07/19] multi-process: add qio channel function to transmit data
and fds
- Refactored qio_channel_writev_all() and qio_channel_writev_full_all()
to avoid logic duplication
[PATCH v13 08/19] multi-process: define MPQemuMsg format and transmission
functions
- Introduced qemu_in_iothread() helper function
- Using ERRP_GUARD to handle input errp parameter
- Addressed redundant typecasts to “void”
- mpqemu_msg_send() and mpqemu_msg_send() returns true/false to
indicate success/failure
- Added a comment to explain that the messaging protocol used would eventually
be replaced by vfio-user
- Prefixed the command enums with “MPQEMU_CMD_”.
- Skip calling mpqemu_read() if msg->size is zero.
[PATCH v13 09/19] multi-process: Initialize message handler in remote device
- Check for NULL IOChannel converted to an assert, and moved outside for loop
- Moved error handling outside for loop
- Shutdown with SHUTDOWN_CAUSE_HOST_ERROR with error in case of failure
- Moved exit condition into the for loop statement
[PATCH v13 10/19] multi-process: Associate fd of a PCIDevice with its object
- Removed a separate header file and merged it into the “.c” file
- Replaced Object properties with Object Class properties
- Accept the file descriptor as an integer property, and check if it is a
valid socket using fd_is_socket()
- Replaced runtime/dynamic property with a DeviceListener
- Added a comment to explain the temporary limitation for one device per
remote process
[PATCH v13 11/19] multi-process: setup memory manager for remote device
- Dropped the calls to memory_region_transaction_begin() and
memory_region_transaction_commit()
- Using ERRP_GUARD to handle input errp parameter
- Using g_autofree to automatically allocate the “name” variable
- Moved RET_MSG command definition to the patch that starts to use it
[PATCH v13 12/19] multi-process: introduce proxy object
- device class property is defined just above the class_init function
- Using ERRP_GUARD to handle input errp parameter
- Check if IOChannel is NULL before closing in the exit function, to avoid
crash
- Early return if fd is not set
- dev->migration_blocker is free’d by the exit function alone
[PATCH v13 13/19] multi-process: add proxy communication functions
- Asset that mpqemu_msg_send_and_await_reply() thread is not in co-routine
context
- Using ERRP_GUARD to handle input errp parameter
[PATCH v13 14/19] multi-process: Forward PCI config space acceses to the
remote process
- process_config_write() & process_config_read() take an Error** argument to
return error
- Use FMT_pid to format messages with PID
- Use error_prepend to avoid leak of Error*
- Check if return value from remote indicates failure in config_op_send()
- Dropped typecasting return values of config_op_send to void as it’s not
necessary
[PATCH v13 16/19] multi-process: Synchronize remote memory
- Renamed RemoteMemSync as ProxyMemoryListener
- Prefixed all related functions with “proxy_memory_listener_” for consistency
- Simplify logic in proxy_mrs_can_merge() to return efficiently and avoid
unnecessary definition of local variable.
[PATCH v13 17/19] multi-process: create IOHUB object to handle irq
- Replace calls to qemu_mutex_lock() and qemu_mutex_unlock() with
QEMU_LOCK_GUARD
Thank you very much!
> On Dec 14, 2020, at 12:14 AM, Jagannathan Raman <jag.raman@oracle.com> wrote:
>
> Hello,
>
> This is the v13 of the patchset. Thank you very much for reviewing v12 and
> sharing your feedback.
>
> We have addressed all the comments from the v12 review except for the
> following two items, which we will be addressing in subsequent patches:
> - Prefix log messages with PID in the logging subsystem.
> - Refactor vfio-user and multi-process code to avoid logic duplication.
>
> To touch upon the history of this project, we posted the Proof Of Concept
> patches before the BoF session in 2018. Subsequently, we have posted 12
> versions on the qemu-devel mailing list. You can find them by following the
> links below ([1] - [12]). Following people contributed to the design and
> implementation of this project:
> Jagannathan Raman <jag.raman@oracle.com>
> Elena Ufimtseva <elena.ufimtseva@oracle.com>
> John G Johnson <john.g.johnson@oracle.com>
> Stefan Hajnoczi <stefanha@redhat.com>
> Konrad Wilk <konrad.wilk@oracle.com>
> Kanth Ghatraju <kanth.ghatraju@oracle.com>
>
> We would like to thank the QEMU community for your feedback in the
> design and implementation of this project. Qemu wiki page:
> https://wiki.qemu.org/Features/MultiProcessQEMU
>
> For the full concept writeup about QEMU multi-process, please
> refer to docs/devel/qemu-multiprocess.rst. Also, see
> docs/qemu-multiprocess.txt for usage information.
>
> We have retained Stefan's "Reviewed-by" in the commit messages. Thank you
> very much for your time in reviewing these patches!
>
> [POC]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html
> [1]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html
> [2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html
> [3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html
> [4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html
> [5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg682429.html
> [6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg697484.html
> [7]: https://patchew.org/QEMU/cover.1593273671.git.elena.ufimtseva@oracle.com/
> [8]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg727007.html
> [9]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg734275.html
> [10]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg747638.html
> [11]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg750972.html
> [12]: https://patchew.org/QEMU/cover.1606853298.git.jag.raman@oracle.com/
>
> Elena Ufimtseva (7):
> multi-process: add configure and usage information
> multi-process: add qio channel function to transmit data and fds
> multi-process: define MPQemuMsg format and transmission functions
> multi-process: introduce proxy object
> multi-process: add proxy communication functions
> multi-process: Forward PCI config space acceses to the remote process
> multi-process: perform device reset in the remote process
>
> Jagannathan Raman (11):
> memory: alloc RAM from file at offset
> multi-process: Add config option for multi-process QEMU
> multi-process: setup PCI host bridge for remote device
> multi-process: setup a machine object for remote device process
> multi-process: Initialize message handler in remote device
> multi-process: Associate fd of a PCIDevice with its object
> multi-process: setup memory manager for remote device
> multi-process: PCI BAR read/write handling for proxy & remote
> endpoints
> multi-process: Synchronize remote memory
> multi-process: create IOHUB object to handle irq
> multi-process: Retrieve PCI info from remote process
>
> John G Johnson (1):
> multi-process: add the concept description to
> docs/devel/qemu-multiprocess
>
> docs/devel/index.rst | 1 +
> docs/devel/multi-process.rst | 966 ++++++++++++++++++++++++++++++
> docs/multi-process.rst | 64 ++
> meson.build | 1 +
> hw/remote/trace.h | 1 +
> include/exec/memory.h | 2 +
> include/exec/ram_addr.h | 2 +-
> include/hw/pci-host/remote.h | 31 +
> include/hw/pci/pci_ids.h | 3 +
> include/hw/remote/iohub.h | 42 ++
> include/hw/remote/machine.h | 40 ++
> include/hw/remote/memory.h | 19 +
> include/hw/remote/mpqemu-link.h | 99 +++
> include/hw/remote/proxy-memory-listener.h | 28 +
> include/hw/remote/proxy.h | 52 ++
> include/io/channel.h | 50 ++
> include/qemu/mmap-alloc.h | 4 +-
> include/sysemu/iothread.h | 6 +
> backends/hostmem-memfd.c | 2 +-
> hw/misc/ivshmem.c | 3 +-
> hw/pci-host/remote.c | 75 +++
> hw/remote/iohub.c | 119 ++++
> hw/remote/machine.c | 80 +++
> hw/remote/memory.c | 65 ++
> hw/remote/message.c | 230 +++++++
> hw/remote/mpqemu-link.c | 267 +++++++++
> hw/remote/proxy-memory-listener.c | 227 +++++++
> hw/remote/proxy.c | 371 ++++++++++++
> hw/remote/remote-obj.c | 206 +++++++
> io/channel.c | 63 +-
> iothread.c | 6 +
> softmmu/memory.c | 3 +-
> softmmu/physmem.c | 11 +-
> util/mmap-alloc.c | 7 +-
> util/oslib-posix.c | 2 +-
> MAINTAINERS | 24 +
> accel/Kconfig | 1 +
> hw/Kconfig | 1 +
> hw/meson.build | 1 +
> hw/pci-host/Kconfig | 3 +
> hw/pci-host/meson.build | 1 +
> hw/remote/Kconfig | 4 +
> hw/remote/meson.build | 13 +
> hw/remote/trace-events | 4 +
> 44 files changed, 3186 insertions(+), 14 deletions(-)
> create mode 100644 docs/devel/multi-process.rst
> create mode 100644 docs/multi-process.rst
> create mode 100644 hw/remote/trace.h
> create mode 100644 include/hw/pci-host/remote.h
> create mode 100644 include/hw/remote/iohub.h
> create mode 100644 include/hw/remote/machine.h
> create mode 100644 include/hw/remote/memory.h
> create mode 100644 include/hw/remote/mpqemu-link.h
> create mode 100644 include/hw/remote/proxy-memory-listener.h
> create mode 100644 include/hw/remote/proxy.h
> create mode 100644 hw/pci-host/remote.c
> create mode 100644 hw/remote/iohub.c
> create mode 100644 hw/remote/machine.c
> create mode 100644 hw/remote/memory.c
> create mode 100644 hw/remote/message.c
> create mode 100644 hw/remote/mpqemu-link.c
> create mode 100644 hw/remote/proxy-memory-listener.c
> create mode 100644 hw/remote/proxy.c
> create mode 100644 hw/remote/remote-obj.c
> create mode 100644 hw/remote/Kconfig
> create mode 100644 hw/remote/meson.build
> create mode 100644 hw/remote/trace-events
>
> --
> 1.8.3.1
>
On Mon, Dec 14, 2020 at 12:14:40AM -0500, Jagannathan Raman wrote: > This is the v13 of the patchset. Thank you very much for reviewing v12 and > sharing your feedback. > > We have addressed all the comments from the v12 review except for the > following two items, which we will be addressing in subsequent patches: > - Prefix log messages with PID in the logging subsystem. > - Refactor vfio-user and multi-process code to avoid logic duplication. Aside from the missing ./configure option needed to easily compile out this feature, I'm happy with this series. I think that should be solved before merging this patch series. The series passes Gitlab CI. I have not run multi-process QEMU since there is no test case or launcher. Please send a test case as a separate patch series. Stefan
Hello, This is the v13 of the patchset. Thank you very much for reviewing v12 and sharing your feedback. We have addressed all the comments from the v12 review except for the following two items, which we will be addressing in subsequent patches: - Prefix log messages with PID in the logging subsystem. - Refactor vfio-user and multi-process code to avoid logic duplication. To touch upon the history of this project, we posted the Proof Of Concept patches before the BoF session in 2018. Subsequently, we have posted 12 versions on the qemu-devel mailing list. You can find them by following the links below ([1] - [12]). Following people contributed to the design and implementation of this project: Jagannathan Raman <jag.raman@oracle.com> Elena Ufimtseva <elena.ufimtseva@oracle.com> John G Johnson <john.g.johnson@oracle.com> Stefan Hajnoczi <stefanha@redhat.com> Konrad Wilk <konrad.wilk@oracle.com> Kanth Ghatraju <kanth.ghatraju@oracle.com> We would like to thank the QEMU community for your feedback in the design and implementation of this project. Qemu wiki page: https://wiki.qemu.org/Features/MultiProcessQEMU For the full concept writeup about QEMU multi-process, please refer to docs/devel/qemu-multiprocess.rst. Also, see docs/qemu-multiprocess.txt for usage information. We have retained Stefan's "Reviewed-by" in the commit messages. Thank you very much for your time in reviewing these patches! [POC]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg566538.html [1]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg602285.html [2]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg624877.html [3]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg642000.html [4]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg655118.html [5]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg682429.html [6]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg697484.html [7]: https://patchew.org/QEMU/cover.1593273671.git.elena.ufimtseva@oracle.com/ [8]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg727007.html [9]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg734275.html [10]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg747638.html [11]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg750972.html [12]: https://patchew.org/QEMU/cover.1606853298.git.jag.raman@oracle.com/ Elena Ufimtseva (7): multi-process: add configure and usage information multi-process: add qio channel function to transmit data and fds multi-process: define MPQemuMsg format and transmission functions multi-process: introduce proxy object multi-process: add proxy communication functions multi-process: Forward PCI config space acceses to the remote process multi-process: perform device reset in the remote process Jagannathan Raman (11): memory: alloc RAM from file at offset multi-process: Add config option for multi-process QEMU multi-process: setup PCI host bridge for remote device multi-process: setup a machine object for remote device process multi-process: Initialize message handler in remote device multi-process: Associate fd of a PCIDevice with its object multi-process: setup memory manager for remote device multi-process: PCI BAR read/write handling for proxy & remote endpoints multi-process: Synchronize remote memory multi-process: create IOHUB object to handle irq multi-process: Retrieve PCI info from remote process John G Johnson (1): multi-process: add the concept description to docs/devel/qemu-multiprocess docs/devel/index.rst | 1 + docs/devel/multi-process.rst | 966 ++++++++++++++++++++++++++++++ docs/multi-process.rst | 64 ++ meson.build | 1 + hw/remote/trace.h | 1 + include/exec/memory.h | 2 + include/exec/ram_addr.h | 2 +- include/hw/pci-host/remote.h | 31 + include/hw/pci/pci_ids.h | 3 + include/hw/remote/iohub.h | 42 ++ include/hw/remote/machine.h | 40 ++ include/hw/remote/memory.h | 19 + include/hw/remote/mpqemu-link.h | 99 +++ include/hw/remote/proxy-memory-listener.h | 28 + include/hw/remote/proxy.h | 52 ++ include/io/channel.h | 50 ++ include/qemu/mmap-alloc.h | 4 +- include/sysemu/iothread.h | 6 + backends/hostmem-memfd.c | 2 +- hw/misc/ivshmem.c | 3 +- hw/pci-host/remote.c | 75 +++ hw/remote/iohub.c | 119 ++++ hw/remote/machine.c | 80 +++ hw/remote/memory.c | 65 ++ hw/remote/message.c | 230 +++++++ hw/remote/mpqemu-link.c | 267 +++++++++ hw/remote/proxy-memory-listener.c | 227 +++++++ hw/remote/proxy.c | 371 ++++++++++++ hw/remote/remote-obj.c | 206 +++++++ io/channel.c | 63 +- iothread.c | 6 + softmmu/memory.c | 3 +- softmmu/physmem.c | 11 +- util/mmap-alloc.c | 7 +- util/oslib-posix.c | 2 +- MAINTAINERS | 24 + accel/Kconfig | 1 + hw/Kconfig | 1 + hw/meson.build | 1 + hw/pci-host/Kconfig | 3 + hw/pci-host/meson.build | 1 + hw/remote/Kconfig | 4 + hw/remote/meson.build | 13 + hw/remote/trace-events | 4 + 44 files changed, 3186 insertions(+), 14 deletions(-) create mode 100644 docs/devel/multi-process.rst create mode 100644 docs/multi-process.rst create mode 100644 hw/remote/trace.h create mode 100644 include/hw/pci-host/remote.h create mode 100644 include/hw/remote/iohub.h create mode 100644 include/hw/remote/machine.h create mode 100644 include/hw/remote/memory.h create mode 100644 include/hw/remote/mpqemu-link.h create mode 100644 include/hw/remote/proxy-memory-listener.h create mode 100644 include/hw/remote/proxy.h create mode 100644 hw/pci-host/remote.c create mode 100644 hw/remote/iohub.c create mode 100644 hw/remote/machine.c create mode 100644 hw/remote/memory.c create mode 100644 hw/remote/message.c create mode 100644 hw/remote/mpqemu-link.c create mode 100644 hw/remote/proxy-memory-listener.c create mode 100644 hw/remote/proxy.c create mode 100644 hw/remote/remote-obj.c create mode 100644 hw/remote/Kconfig create mode 100644 hw/remote/meson.build create mode 100644 hw/remote/trace-events