From patchwork Wed Sep 26 14:45:55 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Wang X-Patchwork-Id: 187090 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 52FCA2C0094 for ; Thu, 27 Sep 2012 00:44:12 +1000 (EST) Received: from localhost ([::1]:38678 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TGsqQ-0004i8-4a for incoming@patchwork.ozlabs.org; Wed, 26 Sep 2012 10:44:10 -0400 Received: from eggs.gnu.org ([208.118.235.92]:53377) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TGsqG-0004hl-Gg for qemu-devel@nongnu.org; Wed, 26 Sep 2012 10:44:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TGsqA-000640-Bz for qemu-devel@nongnu.org; Wed, 26 Sep 2012 10:44:00 -0400 Received: from va3ehsobe001.messaging.microsoft.com ([216.32.180.11]:10557 helo=va3outboundpool.messaging.microsoft.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TGsqA-00063s-7M for qemu-devel@nongnu.org; Wed, 26 Sep 2012 10:43:54 -0400 Received: from mail80-va3-R.bigfish.com (10.7.14.242) by VA3EHSOBE002.bigfish.com (10.7.40.22) with Microsoft SMTP Server id 14.1.225.23; Wed, 26 Sep 2012 14:43:52 +0000 Received: from mail80-va3 (localhost [127.0.0.1]) by mail80-va3-R.bigfish.com (Postfix) with ESMTP id 713C9E012C; Wed, 26 Sep 2012 14:43:52 +0000 (UTC) X-Forefront-Antispam-Report: CIP:163.181.249.109; KIP:(null); UIP:(null); IPV:NLI; H:ausb3twp02.amd.com; RD:none; EFVD:NLI X-SpamScore: -1 X-BigFish: VPS-1(zzc857h4015Izz1202h1d1ah1d2ahzzz2dh668h839he5bhf0ah107ah1288h12a5h12bdh137ah34h1155h) Received: from mail80-va3 (localhost.localdomain [127.0.0.1]) by mail80-va3 (MessageSwitch) id 1348670631350563_15262; Wed, 26 Sep 2012 14:43:51 +0000 (UTC) Received: from VA3EHSMHS017.bigfish.com (unknown [10.7.14.243]) by mail80-va3.bigfish.com (Postfix) with ESMTP id 5124F60249; Wed, 26 Sep 2012 14:43:51 +0000 (UTC) Received: from ausb3twp02.amd.com (163.181.249.109) by VA3EHSMHS017.bigfish.com (10.7.99.27) with Microsoft SMTP Server id 14.1.225.23; Wed, 26 Sep 2012 14:43:50 +0000 X-WSS-ID: 0MAYOWZ-02-2VA-02 X-M-MSG: Received: from sausexedgep01.amd.com (sausexedgep01-ext.amd.com [163.181.249.72]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ausb3twp02.amd.com (Axway MailGate 3.8.1) with ESMTP id 2C50AC8147; Wed, 26 Sep 2012 09:43:46 -0500 (CDT) Received: from SAUSEXDAG06.amd.com (163.181.55.7) by sausexedgep01.amd.com (163.181.36.54) with Microsoft SMTP Server (TLS) id 8.3.192.1; Wed, 26 Sep 2012 09:44:00 -0500 Received: from storexhtp01.amd.com (172.24.4.3) by sausexdag06.amd.com (163.181.55.7) with Microsoft SMTP Server (TLS) id 14.1.323.3; Wed, 26 Sep 2012 09:43:48 -0500 Received: from gwo.osrc.amd.com (165.204.16.204) by storexhtp01.amd.com (172.24.4.3) with Microsoft SMTP Server id 8.3.213.0; Wed, 26 Sep 2012 10:43:44 -0400 Received: from [165.204.15.57] (gran.osrc.amd.com [165.204.15.57]) by gwo.osrc.amd.com (Postfix) with ESMTP id 7FFF749C14F; Wed, 26 Sep 2012 15:43:43 +0100 (BST) Message-ID: <50631523.3060602@amd.com> Date: Wed, 26 Sep 2012 16:45:55 +0200 From: Wei Wang User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:12.0) Gecko/20120421 Thunderbird/12.0 MIME-Version: 1.0 To: "qemu-devel@nongnu.org" X-OriginatorOrg: amd.com X-detected-operating-system: by eggs.gnu.org: Windows XP/2000 (RFC1323+, w+, tstamp-) X-Received-From: 216.32.180.11 Cc: "xen-devel@lists.xensource.com" Subject: [Qemu-devel] [PATCH] Add amd iommu emulation for Xen. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org HI, Attached patch adds amd iommu emulation for Xen. Please review it. Thanks, Wei From 122517435641384e4f5e36eaad8302ff273648e8 Mon Sep 17 00:00:00 2001 From: Wei Wang Date: Wed, 26 Sep 2012 16:43:40 +0200 Subject: [PATCH] Add amd iommu emulation for Xen. To passthrough amd southern islands series gpu to guest, a virtual iommu device must be registered on qemu pci bus. It uses a new hypercall xc_domain_update_iommu_msi to notify xen the msi vector of iommu. Signed-off-by: Wei Wang --- hw/i386/Makefile.objs | 2 +- hw/pc_piix.c | 6 ++ hw/xen_iommu.c | 191 +++++++++++++++++++++++++++++++++++++++++++++++++ hw/xen_pt.h | 1 + 4 files changed, 199 insertions(+), 1 deletions(-) create mode 100644 hw/xen_iommu.c diff --git a/hw/i386/Makefile.objs b/hw/i386/Makefile.objs index 8c764bb..8b231ab 100644 --- a/hw/i386/Makefile.objs +++ b/hw/i386/Makefile.objs @@ -8,7 +8,7 @@ obj-y += pc_piix.o obj-y += pc_sysfw.o obj-$(CONFIG_XEN) += xen_platform.o xen_apic.o obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen-host-pci-device.o -obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen_pt.o xen_pt_config_init.o xen_pt_msi.o +obj-$(CONFIG_XEN_PCI_PASSTHROUGH) += xen_pt.o xen_pt_config_init.o xen_pt_msi.o xen_iommu.o obj-y += kvm/ obj-$(CONFIG_SPICE) += qxl.o qxl-logger.o qxl-render.o diff --git a/hw/pc_piix.c b/hw/pc_piix.c index fd5898f..0b5d034 100644 --- a/hw/pc_piix.c +++ b/hw/pc_piix.c @@ -46,6 +46,9 @@ #ifdef CONFIG_XEN # include #endif +#ifdef CONFIG_XEN_PCI_PASSTHROUGH +# include "xen_pt.h" +#endif #define MAX_IDE_BUS 2 @@ -228,6 +231,9 @@ static void pc_init1(MemoryRegion *system_memory, pc_vga_init(isa_bus, pci_enabled ? pci_bus : NULL); if (xen_enabled()) { pci_create_simple(pci_bus, -1, "xen-platform"); +#ifdef CONFIG_XEN_PCI_PASSTHROUGH + xen_pt_iommu_create(pci_bus); +#endif } /* init basic PC hardware */ diff --git a/hw/xen_iommu.c b/hw/xen_iommu.c new file mode 100644 index 0000000..9a9ede4 --- /dev/null +++ b/hw/xen_iommu.c @@ -0,0 +1,191 @@ +/* + * amd iommu support + * + * Copyright (C) 2012 Advanced Micro Devices, Inc. + * Author: Wei Wang + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + */ + +#include "xen_pt.h" +#include "xen_backend.h" + +#pragma pack(1) + +typedef struct iommu_capability_block { + uint8_t id; + uint8_t next_ptr; + uint8_t cap_info; + uint8_t flags; + uint32_t base_low; + uint32_t base_high; + uint32_t range; + uint32_t misc; +} iommu_capability_t; + +typedef struct msi_capability_block { + uint8_t id; + uint8_t next_ptr; + uint16_t msg_ctrl; + uint32_t addr_low; + uint32_t addr_high; + uint32_t msi_data; +} msi_capability_t; + +struct amd_iommu_config { + uint16_t vendor_id; + uint16_t device_id; + uint16_t command; + uint16_t status; + uint8_t revision; + uint8_t api; + uint8_t subclass; + uint8_t class; + uint8_t cache_line_size; + uint8_t latency_timer; + uint8_t header_type; + uint8_t bist; + uint32_t base_address_regs[6]; + uint32_t reserved1; + uint16_t subsystem_vendor_id; + uint16_t subsystem_id; + uint32_t rom_addr; + uint8_t cap_ptr; + uint8_t reserved3[3]; + uint32_t reserved4; + uint8_t interrupt_line; + uint8_t interrupt_pin; + uint8_t min_gnt; + uint8_t max_lat; + iommu_capability_t cap; + msi_capability_t msi; +}; +#pragma pack() + +#ifndef PCI_CAP_ID_SEC +#define PCI_CAP_ID_SEC 0x0F +#endif +#define PCI_CLASS_SYSTEM_AMD_IOMMU 0x0806 +#define PCI_DEVICE_AMD_IOMMU_V2 0xFFFF +#define IOMMU_CAP_FLAGS_IOTLB 0 +#define IOMMU_CAP_FLAGS_EFRSUP 3 +#define IOMMU_CAP_TYPE 0x3 +#define IOMMU_CAP_REV 0x1 + +#define MSI_DATA_VECTOR_SHIFT 0 +#define MSI_DATA_DELIVERY_SHIFT 8 +#define MSI_DATA_LEVEL_SHIFT 14 +#define MSI_DATA_TRIGGER_SHIFT 15 +#define MSI_ADDR_DESTID_MASK 0xfff0000f +#define MSI_ADDR_DESTMODE_SHIFT 2 +#define MSI_ADDR_REDIRECTION_SHIFT 3 +#define MSI_TARGET_CPU_SHIFT 12 +#define PCI_STATUS_CAPABILITIES 0x010 + +static void amd_iommu_pci_write_config(PCIDevice *d, uint32_t address, + uint32_t val, int len) +{ + struct amd_iommu_config *iommu_config; + uint64_t msi_addr; + uint32_t msi_data; + uint8_t offset_msi_data, vector, en; + uint8_t dest_mode, dest, delivery_mode, trig_mode; + + pci_default_write_config(d, address, val, len); + + iommu_config = (struct amd_iommu_config *)d->config; + + offset_msi_data = iommu_config->cap.next_ptr + sizeof(uint32_t) + + sizeof(uint64_t); + + if ( address == offset_msi_data ) + { + msi_addr = (uint64_t)iommu_config->msi.addr_high << 32 | + iommu_config->msi.addr_low; + msi_data = val; + vector = msi_data & 0xFF; + en = iommu_config->msi.msg_ctrl & 0x1; + + if ( !en ) + return; + + dest_mode = (msi_addr >> MSI_ADDR_DESTMODE_SHIFT) & 0x1; + dest = (msi_addr >> MSI_TARGET_CPU_SHIFT) & 0xff; + delivery_mode = (msi_data >> MSI_DATA_DELIVERY_SHIFT) & 0x7; + trig_mode = (msi_data >> MSI_DATA_TRIGGER_SHIFT) & 0x1; + + xc_domain_update_iommu_msi(xen_xc, xen_domid, vector, dest, + dest_mode, delivery_mode, trig_mode); + } +} + +static int amd_iommu_initfn(PCIDevice *d) +{ + struct amd_iommu_config *cfg; + + cfg = (struct amd_iommu_config *)d->config; + + cfg->status = PCI_STATUS_CAPABILITIES; + cfg->cap_ptr = sizeof(struct amd_iommu_config) - + sizeof(iommu_capability_t)- sizeof(msi_capability_t); + + cfg->cap.id = PCI_CAP_ID_SEC; + cfg->cap.cap_info = IOMMU_CAP_REV << 3 | IOMMU_CAP_TYPE; + + cfg->cap.flags |= 1 << IOMMU_CAP_FLAGS_IOTLB; + cfg->cap.flags |= 1 << IOMMU_CAP_FLAGS_EFRSUP; + + cfg->cap.next_ptr = cfg->cap_ptr + sizeof(iommu_capability_t); + cfg->msi.id = PCI_CAP_ID_MSI; + cfg->msi.msg_ctrl = PCI_MSI_FLAGS_64BIT; + + return 0; +} + +static void xen_iommu_class_init(ObjectClass *klass, void *data) +{ + PCIDeviceClass *k = PCI_DEVICE_CLASS(klass); + + k->init = amd_iommu_initfn; + k->vendor_id = PCI_VENDOR_ID_AMD; + k->device_id = PCI_DEVICE_AMD_IOMMU_V2; + k->class_id = PCI_CLASS_SYSTEM_AMD_IOMMU; + k->subsystem_vendor_id = PCI_VENDOR_ID_AMD; + k->subsystem_id = PCI_DEVICE_AMD_IOMMU_V2; + k->config_write = amd_iommu_pci_write_config; +}; + +typedef struct XenIommuState { + PCIDevice dev; +} XenIommuState; + +static TypeInfo xen_iommu_info = { + .name = "xen-iommu", + .parent = TYPE_PCI_DEVICE, + .instance_size = sizeof(XenIommuState), + .class_init = xen_iommu_class_init, +}; + +static void xen_iommu_register_types(void) +{ + type_register_static(&xen_iommu_info); +} + +type_init(xen_iommu_register_types) + +void xen_pt_iommu_create(PCIBus *pci_bus) +{ + char *path; + char *iommu; + struct xs_handle *xs = xs_open(0); + + path = xs_get_domain_path(xs, xen_domid); + iommu = xenstore_read_str(path, "guest_iommu"); + + if ( !strcmp(iommu, "1") ) + pci_create_simple(pci_bus, -1, "xen-iommu"); + + free(path); + xs_close(xs); +} diff --git a/hw/xen_pt.h b/hw/xen_pt.h index 112477a..b54a0fb 100644 --- a/hw/xen_pt.h +++ b/hw/xen_pt.h @@ -291,6 +291,7 @@ void xen_pt_msix_delete(XenPCIPassthroughState *s); int xen_pt_msix_update(XenPCIPassthroughState *s); int xen_pt_msix_update_remap(XenPCIPassthroughState *s, int bar_index); void xen_pt_msix_disable(XenPCIPassthroughState *s); +void xen_pt_iommu_create(PCIBus *pci_bus); static inline bool xen_pt_has_msix_mapping(XenPCIPassthroughState *s, int bar) {