From patchwork Tue Feb 10 18:37:33 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 438510 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48378140145 for ; Wed, 11 Feb 2015 05:39:40 +1100 (AEDT) Received: from localhost ([::1]:41315 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLFik-0004I2-I8 for incoming@patchwork.ozlabs.org; Tue, 10 Feb 2015 13:39:38 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39801) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLFgm-0000yj-T1 for qemu-devel@nongnu.org; Tue, 10 Feb 2015 13:37:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YLFgl-0006dT-Mm for qemu-devel@nongnu.org; Tue, 10 Feb 2015 13:37:36 -0500 Received: from mx1.redhat.com ([209.132.183.28]:41917) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLFgl-0006dL-FC for qemu-devel@nongnu.org; Tue, 10 Feb 2015 13:37:35 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t1AIbYkh010128 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Tue, 10 Feb 2015 13:37:35 -0500 Received: from gimli.home (ovpn-113-68.phx2.redhat.com [10.3.113.68]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t1AIbYrx014333; Tue, 10 Feb 2015 13:37:34 -0500 From: Alex Williamson To: qemu-devel@nongnu.org Date: Tue, 10 Feb 2015 11:37:33 -0700 Message-ID: <20150210183733.26101.15510.stgit@gimli.home> In-Reply-To: <20150210183106.26101.78658.stgit@gimli.home> References: <20150210183106.26101.78658.stgit@gimli.home> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: alex.williamson@redhat.com Subject: [Qemu-devel] [PULL 4/6] vfio: unmap and free BAR data in instance_finalize X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Paolo Bonzini In the case of VFIO, the unrealize callback is too early to munmap the BARs. The munmap must be delayed until memory accesses are complete. To do this, split vfio_unmap_bars in two. The removal step, now called vfio_unregister_bars, remains in vfio_exitfn. The reclamation step is vfio_unmap_bars and is moved to the instance_finalize callback. Similarly, quirk MemoryRegions have to be removed during vfio_unregister_bars, but freeing the data structure must be delayed to vfio_unmap_bars. Signed-off-by: Paolo Bonzini Signed-off-by: Alex Williamson --- hw/vfio/pci.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 57 insertions(+), 8 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 0271c80..fa6a5e9 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -1996,12 +1996,23 @@ static void vfio_vga_quirk_setup(VFIOPCIDevice *vdev) static void vfio_vga_quirk_teardown(VFIOPCIDevice *vdev) { + VFIOQuirk *quirk; + int i; + + for (i = 0; i < ARRAY_SIZE(vdev->vga.region); i++) { + QLIST_FOREACH(quirk, &vdev->vga.region[i].quirks, next) { + memory_region_del_subregion(&vdev->vga.region[i].mem, &quirk->mem); + } + } +} + +static void vfio_vga_quirk_free(VFIOPCIDevice *vdev) +{ int i; for (i = 0; i < ARRAY_SIZE(vdev->vga.region); i++) { while (!QLIST_EMPTY(&vdev->vga.region[i].quirks)) { VFIOQuirk *quirk = QLIST_FIRST(&vdev->vga.region[i].quirks); - memory_region_del_subregion(&vdev->vga.region[i].mem, &quirk->mem); object_unparent(OBJECT(&quirk->mem)); QLIST_REMOVE(quirk, next); g_free(quirk); @@ -2022,10 +2033,19 @@ static void vfio_bar_quirk_setup(VFIOPCIDevice *vdev, int nr) static void vfio_bar_quirk_teardown(VFIOPCIDevice *vdev, int nr) { VFIOBAR *bar = &vdev->bars[nr]; + VFIOQuirk *quirk; + + QLIST_FOREACH(quirk, &bar->quirks, next) { + memory_region_del_subregion(&bar->region.mem, &quirk->mem); + } +} + +static void vfio_bar_quirk_free(VFIOPCIDevice *vdev, int nr) +{ + VFIOBAR *bar = &vdev->bars[nr]; while (!QLIST_EMPTY(&bar->quirks)) { VFIOQuirk *quirk = QLIST_FIRST(&bar->quirks); - memory_region_del_subregion(&bar->region.mem, &quirk->mem); object_unparent(OBJECT(&quirk->mem)); QLIST_REMOVE(quirk, next); g_free(quirk); @@ -2281,7 +2301,7 @@ static void vfio_mmap_set_enabled(VFIOPCIDevice *vdev, bool enabled) } } -static void vfio_unmap_bar(VFIOPCIDevice *vdev, int nr) +static void vfio_unregister_bar(VFIOPCIDevice *vdev, int nr) { VFIOBAR *bar = &vdev->bars[nr]; @@ -2292,10 +2312,25 @@ static void vfio_unmap_bar(VFIOPCIDevice *vdev, int nr) vfio_bar_quirk_teardown(vdev, nr); memory_region_del_subregion(&bar->region.mem, &bar->region.mmap_mem); - munmap(bar->region.mmap, memory_region_size(&bar->region.mmap_mem)); if (vdev->msix && vdev->msix->table_bar == nr) { memory_region_del_subregion(&bar->region.mem, &vdev->msix->mmap_mem); + } +} + +static void vfio_unmap_bar(VFIOPCIDevice *vdev, int nr) +{ + VFIOBAR *bar = &vdev->bars[nr]; + + if (!bar->region.size) { + return; + } + + vfio_bar_quirk_free(vdev, nr); + + munmap(bar->region.mmap, memory_region_size(&bar->region.mmap_mem)); + + if (vdev->msix && vdev->msix->table_bar == nr) { munmap(vdev->msix->mmap, memory_region_size(&vdev->msix->mmap_mem)); } } @@ -2403,12 +2438,12 @@ static void vfio_map_bars(VFIOPCIDevice *vdev) } } -static void vfio_unmap_bars(VFIOPCIDevice *vdev) +static void vfio_unregister_bars(VFIOPCIDevice *vdev) { int i; for (i = 0; i < PCI_ROM_SLOT; i++) { - vfio_unmap_bar(vdev, i); + vfio_unregister_bar(vdev, i); } if (vdev->has_vga) { @@ -2417,6 +2452,19 @@ static void vfio_unmap_bars(VFIOPCIDevice *vdev) } } +static void vfio_unmap_bars(VFIOPCIDevice *vdev) +{ + int i; + + for (i = 0; i < PCI_ROM_SLOT; i++) { + vfio_unmap_bar(vdev, i); + } + + if (vdev->has_vga) { + vfio_vga_quirk_free(vdev); + } +} + /* * General setup */ @@ -3328,7 +3376,7 @@ static int vfio_initfn(PCIDevice *pdev) out_teardown: pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); vfio_teardown_msi(vdev); - vfio_unmap_bars(vdev); + vfio_unregister_bars(vdev); return ret; } @@ -3338,6 +3386,7 @@ static void vfio_instance_finalize(Object *obj) VFIOPCIDevice *vdev = DO_UPCAST(VFIOPCIDevice, pdev, pci_dev); VFIOGroup *group = vdev->vbasedev.group; + vfio_unmap_bars(vdev); g_free(vdev->emulated_config_bits); g_free(vdev->rom); vfio_put_device(vdev); @@ -3355,7 +3404,7 @@ static void vfio_exitfn(PCIDevice *pdev) timer_free(vdev->intx.mmap_timer); } vfio_teardown_msi(vdev); - vfio_unmap_bars(vdev); + vfio_unregister_bars(vdev); } static void vfio_pci_reset(DeviceState *dev)