From patchwork Wed Aug 7 08:21:27 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Kardashevskiy X-Patchwork-Id: 265394 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id E0BA02C0224 for ; Wed, 7 Aug 2013 18:24:25 +1000 (EST) Received: from localhost ([::1]:34197 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V6z2e-00053T-27 for incoming@patchwork.ozlabs.org; Wed, 07 Aug 2013 04:24:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59135) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V6z0Z-0002Kd-2P for qemu-devel@nongnu.org; Wed, 07 Aug 2013 04:22:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1V6z0S-0007SF-Tx for qemu-devel@nongnu.org; Wed, 07 Aug 2013 04:22:14 -0400 Received: from mail-pb0-f44.google.com ([209.85.160.44]:54011) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V6z0S-0007R8-DN for qemu-devel@nongnu.org; Wed, 07 Aug 2013 04:22:08 -0400 Received: by mail-pb0-f44.google.com with SMTP id xa7so1601314pbc.31 for ; Wed, 07 Aug 2013 01:22:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=HfYusyqiCyGiJvoiPdH6WWQtWEkxJyx7PLxBNwRpe1g=; b=J80OPMsOHUjfsA2xtNVaRqh0hYeJyZoSKuQc4TEwO54e9J8w17+YgcPWxyNR4oXH0y 6RluGxTULaBuOLuB+28l1SstJenbdqNXIC4+UB3GeCGXlXbztcgM6NBviLnNWO6w0hXr pxgrMgrG4c8v0j+BdQ3xYxqwSZmTGbtRNNYxaEhn1/YoZRPneHhCIYtnX9sQ1ZwXrSA9 AeEczH6NwPNNW7d3agT+wen4noEabZXbAQxT8C8GwyItLR4mcSqaOZpJ3FsnC9/vQupn iDTdwgG5DYtoNqZbxr02DqigR8UpoMxJAnXZaDV6GZmlcV3wgTRrO8fzFQ3PF38dPfzE PyCQ== X-Gm-Message-State: ALoCoQkqq9YM6s1ZdyMvvE0gbzz8JJ4ptxnfWrMuvDYvjIqASafl1sLMIyGcXavPBaV8ILpkI8Df X-Received: by 10.68.130.2 with SMTP id oa2mr1477729pbb.134.1375863727649; Wed, 07 Aug 2013 01:22:07 -0700 (PDT) Received: from ka1.ozlabs.ibm.com (ibmaus65.lnk.telstra.net. [165.228.126.9]) by mx.google.com with ESMTPSA id 7sm8957160paf.22.2013.08.07.01.21.59 for (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 07 Aug 2013 01:22:02 -0700 (PDT) From: Alexey Kardashevskiy To: qemu-devel@nongnu.org Date: Wed, 7 Aug 2013 18:21:27 +1000 Message-Id: <1375863692-12207-4-git-send-email-aik@ozlabs.ru> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1375863692-12207-1-git-send-email-aik@ozlabs.ru> References: <1375863692-12207-1-git-send-email-aik@ozlabs.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.85.160.44 Cc: Anthony Liguori , "Michael S . Tsirkin" , aik@ozlabs.ru, Alexander Graf , Alex Williamson , qemu-ppc@nongnu.org, Paolo Bonzini , Paul Mackerras , David Gibson Subject: [Qemu-devel] [PATCH 3/8] vfio: Introduce VFIO address spaces X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: David Gibson The only model so far supported for VFIO passthrough devices is the model usually used on x86, where all of the guest's RAM is mapped into the (host) IOMMU and there is no IOMMU visible in the guest. This patch begins to relax this model, introducing the notion of a VFIOAddressSpace. This represents a logical DMA address space which will be visible to one or more VFIO devices by appropriate mapping in the (host) IOMMU. Thus the currently global list of containers becomes local to a VFIOAddressSpace, and we verify that we don't attempt to add a VFIO group to multiple address spaces. For now, only one VFIOAddressSpace is created and used, corresponding to main system memory, that will change in future patches. Signed-off-by: David Gibson --- hw/misc/vfio.c | 57 +++++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 45 insertions(+), 12 deletions(-) diff --git a/hw/misc/vfio.c b/hw/misc/vfio.c index adcd23d..bcd67a0 100644 --- a/hw/misc/vfio.c +++ b/hw/misc/vfio.c @@ -130,9 +130,17 @@ enum { VFIO_INT_MSIX = 3, }; +typedef struct VFIOAddressSpace { + AddressSpace *as; + QLIST_HEAD(, VFIOContainer) containers; +} VFIOAddressSpace; + +static VFIOAddressSpace vfio_address_space_memory; + struct VFIOGroup; typedef struct VFIOContainer { + VFIOAddressSpace *space; int fd; /* /dev/vfio/vfio, empowered by the attached groups */ struct { /* enable abstraction to support various iommu backends */ @@ -197,9 +205,6 @@ typedef struct VFIOGroup { #define MSIX_CAP_LENGTH 12 -static QLIST_HEAD(, VFIOContainer) - container_list = QLIST_HEAD_INITIALIZER(container_list); - static QLIST_HEAD(, VFIOGroup) group_list = QLIST_HEAD_INITIALIZER(group_list); @@ -2604,16 +2609,28 @@ static int vfio_load_rom(VFIODevice *vdev) return 0; } -static int vfio_connect_container(VFIOGroup *group) +static void vfio_address_space_init(VFIOAddressSpace *space, AddressSpace *as) +{ + space->as = as; + QLIST_INIT(&space->containers); +} + +static int vfio_connect_container(VFIOGroup *group, VFIOAddressSpace *space) { VFIOContainer *container; int ret, fd; if (group->container) { - return 0; + if (group->container->space == space) { + return 0; + } else { + error_report("vfio: group %d used in multiple address spaces", + group->groupid); + return -EBUSY; + } } - QLIST_FOREACH(container, &container_list, next) { + QLIST_FOREACH(container, &space->containers, next) { if (!ioctl(group->fd, VFIO_GROUP_SET_CONTAINER, &container->fd)) { group->container = container; QLIST_INSERT_HEAD(&container->group_list, group, container_next); @@ -2636,6 +2653,7 @@ static int vfio_connect_container(VFIOGroup *group) } container = g_malloc0(sizeof(*container)); + container->space = space; container->fd = fd; if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU)) { @@ -2658,7 +2676,8 @@ static int vfio_connect_container(VFIOGroup *group) container->iommu_data.listener = vfio_memory_listener; container->iommu_data.release = vfio_listener_release; - memory_listener_register(&container->iommu_data.listener, &address_space_memory); + memory_listener_register(&container->iommu_data.listener, + container->space->as); } else { error_report("vfio: No available IOMMU models"); g_free(container); @@ -2667,7 +2686,7 @@ static int vfio_connect_container(VFIOGroup *group) } QLIST_INIT(&container->group_list); - QLIST_INSERT_HEAD(&container_list, container, next); + QLIST_INSERT_HEAD(&space->containers, container, next); group->container = container; QLIST_INSERT_HEAD(&container->group_list, group, container_next); @@ -2698,7 +2717,7 @@ static void vfio_disconnect_container(VFIOGroup *group) } } -static VFIOGroup *vfio_get_group(int groupid) +static VFIOGroup *vfio_get_group(int groupid, VFIOAddressSpace *space) { VFIOGroup *group; char path[32]; @@ -2706,7 +2725,15 @@ static VFIOGroup *vfio_get_group(int groupid) QLIST_FOREACH(group, &group_list, next) { if (group->groupid == groupid) { - return group; + /* Found it. Now is it already in the right context? */ + assert(group->container); + if (group->container->space == space) { + return group; + } else { + error_report("vfio: group %d used in multiple address spaces", + group->groupid); + return NULL; + } } } @@ -2739,7 +2766,7 @@ static VFIOGroup *vfio_get_group(int groupid) group->groupid = groupid; QLIST_INIT(&group->device_list); - if (vfio_connect_container(group)) { + if (vfio_connect_container(group, space)) { error_report("vfio: failed to setup container for group %d", groupid); close(group->fd); g_free(group); @@ -3093,7 +3120,12 @@ static int vfio_initfn(PCIDevice *pdev) DPRINTF("%s(%04x:%02x:%02x.%x) group %d\n", __func__, vdev->host.domain, vdev->host.bus, vdev->host.slot, vdev->host.function, groupid); - group = vfio_get_group(groupid); + if (pci_iommu_as(pdev) != &address_space_memory) { + error_report("vfio: DMA address space must be system memory"); + return -ENXIO; + } + + group = vfio_get_group(groupid, &vfio_address_space_memory); if (!group) { error_report("vfio: failed to get group %d", groupid); return -ENOENT; @@ -3316,6 +3348,7 @@ static const TypeInfo vfio_pci_dev_info = { static void register_vfio_pci_dev_type(void) { + vfio_address_space_init(&vfio_address_space_memory, &address_space_memory); type_register_static(&vfio_pci_dev_info); }