From patchwork Tue Apr 10 06:02:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Yulei" X-Patchwork-Id: 896492 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40KxHy3QC5z9s0n for ; Tue, 10 Apr 2018 15:56:30 +1000 (AEST) Received: from localhost ([::1]:42813 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f5mGa-0001C9-HG for incoming@patchwork.ozlabs.org; Tue, 10 Apr 2018 01:56:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54568) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f5mG2-0001AR-DB for qemu-devel@nongnu.org; Tue, 10 Apr 2018 01:55:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f5mFy-0004s5-H2 for qemu-devel@nongnu.org; Tue, 10 Apr 2018 01:55:54 -0400 Received: from mga05.intel.com ([192.55.52.43]:56980) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f5mFy-0004pt-7K for qemu-devel@nongnu.org; Tue, 10 Apr 2018 01:55:50 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Apr 2018 22:55:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,430,1517904000"; d="scan'208";a="219138815" Received: from yzhang13-kvm.sh.intel.com ([10.239.36.9]) by fmsmga006.fm.intel.com with ESMTP; 09 Apr 2018 22:55:47 -0700 From: Yulei Zhang To: qemu-devel@nongnu.org Date: Tue, 10 Apr 2018 14:02:41 +0800 Message-Id: <1523340161-8928-1-git-send-email-yulei.zhang@intel.com> X-Mailer: git-send-email 2.7.4 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.43 Subject: [Qemu-devel] [RFC PATCH V4 1/4] vfio: introduce a new VFIO subregion for mdev device migration support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kevin.tian@intel.com, quintela@redhat.com, joonas.lahtinen@linux.intel.com, kwankhede@nvidia.com, zhenyuw@linux.intel.com, dgilbert@redhat.com, Yulei Zhang , alex.williamson@redhat.com, zhi.a.wang@intel.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" New VFIO sub region VFIO_REGION_SUBTYPE_DEVICE_STATE is added to fetch and restore the status of mdev device vGPU during the live migration. Signed-off-by: Yulei Zhang --- hw/vfio/pci.c | 25 ++++++++++++++++++++++++- hw/vfio/pci.h | 2 ++ linux-headers/linux/vfio.h | 9 ++++++--- 3 files changed, 32 insertions(+), 4 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index c977ee3..f98a9dd 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -32,6 +32,7 @@ #include "pci.h" #include "trace.h" #include "qapi/error.h" +#include "migration/blocker.h" #define MSIX_CAP_LENGTH 12 @@ -2821,6 +2822,25 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) vfio_vga_quirk_setup(vdev); } + struct vfio_region_info *device_state; + /* device state region setup */ + if (!vfio_get_dev_region_info(&vdev->vbasedev, + VFIO_REGION_TYPE_PCI_VENDOR_TYPE, + VFIO_REGION_SUBTYPE_DEVICE_STATE, &device_state)) { + memcpy(&vdev->device_state, device_state, + sizeof(struct vfio_region_info)); + g_free(device_state); + } else { + error_setg(&vdev->migration_blocker, + "Migration disabled: cannot support device state region"); + migrate_add_blocker(vdev->migration_blocker, &err); + if (err) { + error_propagate(errp, err); + error_free(vdev->migration_blocker); + goto error; + } + } + for (i = 0; i < PCI_ROM_SLOT; i++) { vfio_bar_quirk_setup(vdev, i); } @@ -2884,6 +2904,10 @@ out_teardown: vfio_teardown_msi(vdev); vfio_bars_exit(vdev); error: + if (vdev->migration_blocker) { + migrate_del_blocker(vdev->migration_blocker); + error_free(vdev->migration_blocker); + } error_prepend(errp, ERR_PREFIX, vdev->vbasedev.name); } @@ -3009,7 +3033,6 @@ static Property vfio_pci_dev_properties[] = { static const VMStateDescription vfio_pci_vmstate = { .name = "vfio-pci", - .unmigratable = 1, }; static void vfio_pci_dev_class_init(ObjectClass *klass, void *data) diff --git a/hw/vfio/pci.h b/hw/vfio/pci.h index 502a575..0ee1724 100644 --- a/hw/vfio/pci.h +++ b/hw/vfio/pci.h @@ -116,6 +116,8 @@ typedef struct VFIOPCIDevice { VFIOBAR bars[PCI_NUM_REGIONS - 1]; /* No ROM */ VFIOVGA *vga; /* 0xa0000, 0x3b0, 0x3c0 */ void *igd_opregion; + struct vfio_region_info device_state; + Error *migration_blocker; PCIHostDeviceAddress host; EventNotifier err_notifier; EventNotifier req_notifier; diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index 4312e96..e3380ad 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -297,9 +297,12 @@ struct vfio_region_info_cap_type { #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0xffff) /* 8086 Vendor sub-types */ -#define VFIO_REGION_SUBTYPE_INTEL_IGD_OPREGION (1) -#define VFIO_REGION_SUBTYPE_INTEL_IGD_HOST_CFG (2) -#define VFIO_REGION_SUBTYPE_INTEL_IGD_LPC_CFG (3) +#define VFIO_REGION_SUBTYPE_INTEL_IGD_OPREGION (1) +#define VFIO_REGION_SUBTYPE_INTEL_IGD_HOST_CFG (2) +#define VFIO_REGION_SUBTYPE_INTEL_IGD_LPC_CFG (3) + +/* Mdev sub-type for device state save and restore */ +#define VFIO_REGION_SUBTYPE_DEVICE_STATE (4) /** * VFIO_DEVICE_GET_IRQ_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 9,