| Message ID | 20191024132805.30701-1-clombard@linux.vnet.ibm.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | vfio pci: Add support for OpenCAPI devices | expand |
Hi Christophe, Sorry, I didn't have time to look at your other series yet and likely the same for this one with the upcoming KVM Forum... :-\ Anyway, for any VFIO related patch, don't forget to Cc the maintainer, Alex Williamson <alex.williamson@redhat.com> . Cheers, -- Greg On Thu, 24 Oct 2019 15:28:03 +0200 christophe lombard <clombard@linux.vnet.ibm.com> wrote: > This series adds support for the OpenCAPI devices for vfio pci. > > It builds on top of the existing ocxl driver + > http://patchwork.ozlabs.org/patch/1177999/ > > VFIO is a Linux kernel driver framework used by QEMU to make devices > directly assignable to virtual machines. > > All OpenCAPI devices on the same PCI slot will all be grouped and > assigned to the same guest. > > - Assume these are the devices you want to assign > 0007:00:00.0 Processing accelerators: IBM Device 062b > 0007:00:00.1 Processing accelerators: IBM Device 062b > > - Two Devices in the group > $ ls /sys/bus/pci/devices/0007\:00\:00.0/iommu_group/devices/ > 0007:00:00.0 0007:00:00.1 > > - Find vendor & device ID > $ lspci -n -s 0007:00:00 > 0007:00:00.0 1200: 1014:062b > 0007:00:00.1 1200: 1014:062b > > - Unbind from the current ocxl device driver if already loaded > $ rmmod ocxl > > - Load vfio-pci if it's not already done. > $ modprobe vfio-pci > > - Bind to vfio-pci > $ echo 1014 062b > /sys/bus/pci/drivers/vfio-pci/new_id > > This will result in a new device node "/dev/vfio/7", which will be > use by QEMU to setup the devices for passthrough. > > - Pass to qemu using -device vfio-pci > -device vfio-pci,multifunction=on,host=0007:00:00.0,addr=2.0 -device > vfio-pci,multifunction=on,host=0007:00:00.1,addr=2.1 > > It has been tested in a bare-metal and QEMU environment using the memcpy > and the AFP AFUs. > > christophe lombard (2): > powerpc/powernv: Register IOMMU group for OpenCAPI devices > vfio/pci: Introduce OpenCAPI devices support. > > arch/powerpc/platforms/powernv/ocxl.c | 164 ++++++++++--- > arch/powerpc/platforms/powernv/pci-ioda.c | 19 +- > arch/powerpc/platforms/powernv/pci.h | 13 + > drivers/vfio/pci/Kconfig | 7 + > drivers/vfio/pci/Makefile | 1 + > drivers/vfio/pci/vfio_pci.c | 19 ++ > drivers/vfio/pci/vfio_pci_ocxl.c | 287 ++++++++++++++++++++++ > drivers/vfio/vfio.c | 25 ++ > include/linux/vfio.h | 13 + > include/uapi/linux/vfio.h | 22 ++ > 10 files changed, 530 insertions(+), 40 deletions(-) > create mode 100644 drivers/vfio/pci/vfio_pci_ocxl.c >
On 25/10/2019 00:28, christophe lombard wrote: > This series adds support for the OpenCAPI devices for vfio pci. You can pass any PCI device via vfio already, what is missing today to make it fully working? For example, for nvlink gpus it was coherent memory and ATSD which we needed to expose to the userspace, with opencapi I thought it is going to be some sort of SRIOV-alike or a mediated device but all you do here is to pass through the entire device with no new resources shared by the vfio-pci driver. > > It builds on top of the existing ocxl driver + > http://patchwork.ozlabs.org/patch/1177999/ Next time please provide a link to the series such as: http://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=136572 > > VFIO is a Linux kernel driver framework used by QEMU to make devices > directly assignable to virtual machines. > > All OpenCAPI devices on the same PCI slot will all be grouped and > assigned to the same guest. > > - Assume these are the devices you want to assign > 0007:00:00.0 Processing accelerators: IBM Device 062b > 0007:00:00.1 Processing accelerators: IBM Device 062b > > - Two Devices in the group > $ ls /sys/bus/pci/devices/0007\:00\:00.0/iommu_group/devices/ > 0007:00:00.0 0007:00:00.1 > > - Find vendor & device ID > $ lspci -n -s 0007:00:00 > 0007:00:00.0 1200: 1014:062b > 0007:00:00.1 1200: 1014:062b > > - Unbind from the current ocxl device driver if already loaded > $ rmmod ocxl > > - Load vfio-pci if it's not already done. > $ modprobe vfio-pci > > - Bind to vfio-pci > $ echo 1014 062b > /sys/bus/pci/drivers/vfio-pci/new_id > > This will result in a new device node "/dev/vfio/7", which will be > use by QEMU to setup the devices for passthrough. > > - Pass to qemu using -device vfio-pci > -device vfio-pci,multifunction=on,host=0007:00:00.0,addr=2.0 -device > vfio-pci,multifunction=on,host=0007:00:00.1,addr=2.1 > > It has been tested in a bare-metal and QEMU environment using the memcpy > and the AFP AFUs. Is there the corresponding qemu tree somewhere? > > christophe lombard (2): > powerpc/powernv: Register IOMMU group for OpenCAPI devices > vfio/pci: Introduce OpenCAPI devices support. > > arch/powerpc/platforms/powernv/ocxl.c | 164 ++++++++++--- > arch/powerpc/platforms/powernv/pci-ioda.c | 19 +- > arch/powerpc/platforms/powernv/pci.h | 13 + > drivers/vfio/pci/Kconfig | 7 + > drivers/vfio/pci/Makefile | 1 + > drivers/vfio/pci/vfio_pci.c | 19 ++ > drivers/vfio/pci/vfio_pci_ocxl.c | 287 ++++++++++++++++++++++ > drivers/vfio/vfio.c | 25 ++ > include/linux/vfio.h | 13 + > include/uapi/linux/vfio.h | 22 ++ > 10 files changed, 530 insertions(+), 40 deletions(-) > create mode 100644 drivers/vfio/pci/vfio_pci_ocxl.c >