From patchwork Tue Jul 10 13:47:05 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Kardashevskiy X-Patchwork-Id: 170188 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id E3D992C0086 for ; Tue, 10 Jul 2012 23:48:34 +1000 (EST) Received: from localhost ([::1]:48154 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Soano-0004pu-SE for incoming@patchwork.ozlabs.org; Tue, 10 Jul 2012 09:48:32 -0400 Received: from eggs.gnu.org ([208.118.235.92]:53216) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Soamv-0002h6-5s for qemu-devel@nongnu.org; Tue, 10 Jul 2012 09:47:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Soamk-0001F9-Gs for qemu-devel@nongnu.org; Tue, 10 Jul 2012 09:47:36 -0400 Received: from mail-pb0-f45.google.com ([209.85.160.45]:52125) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Soamk-0001Cd-7H for qemu-devel@nongnu.org; Tue, 10 Jul 2012 09:47:26 -0400 Received: by mail-pb0-f45.google.com with SMTP id ro12so336197pbb.4 for ; Tue, 10 Jul 2012 06:47:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:to:cc:subject:date:message-id:x-mailer:in-reply-to:references :x-gm-message-state; bh=D28ZNY3pcOuMc0aQ0gKxnCtge1wwLazSkhGCBf9Ua6g=; b=ivKseuI1uINNauTNgOLVwLKmCJuiGYZHG9wLLotc8g+f9uO46QvFXdplDJMahkCnwM 3d4ZdtBPWesvnzdakiEjoNFaJbvWBDLwEthAG/3LhD+3JkcMFR+9yix/D+paN1GxwRtU NOEvEyZvLRptMl8wbScRgqX85EyUryYHxcxIa8kxEvCOVz2L16Kqo9q3aUfwTnqXbW0r dyebNxt2nl27qVTQ1ax8/7yCF+5Z3lAa1OLq5pDI3LtGJ/pyZPHBjHQlMioqNWi+61tB oaxks26hBP7VK86WYy2IJeOpPitc4cDXcrKCeqzCdBHVw2izvfub9udSN809sX4V1XLr gkvg== Received: by 10.68.200.98 with SMTP id jr2mr69051104pbc.81.1341928045252; Tue, 10 Jul 2012 06:47:25 -0700 (PDT) Received: from ka1.ozlabs.ibm.com (ibmaus65.lnk.telstra.net. [165.228.126.9]) by mx.google.com with ESMTPS id nv6sm29910038pbc.42.2012.07.10.06.47.21 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 10 Jul 2012 06:47:24 -0700 (PDT) From: Alexey Kardashevskiy To: Date: Tue, 10 Jul 2012 23:47:05 +1000 Message-Id: <1341928025-5669-4-git-send-email-aik@ozlabs.ru> X-Mailer: git-send-email 1.7.10 In-Reply-To: <1341928025-5669-1-git-send-email-aik@ozlabs.ru> References: <1341928025-5669-1-git-send-email-aik@ozlabs.ru> X-Gm-Message-State: ALoCoQkimL6mjVrvrUfpVjSg5YGqGC6+zUnazXBKKWzMXXohWV4ppWAz7sBS/qr1hdmIjqkUENNJ X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 209.85.160.45 Cc: Alexey Kardashevskiy , qemu-devel@nongnu.org, Alexander Graf , qemu-ppc@nongnu.org, David Gibson Subject: [Qemu-devel] [PATCH 3/3] pseries pci: added actual MSI/MSIX support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The first type of MSI/MSIX capable devices to support is virtio-pci. This patch includes: 1. RTAS call "ibm,change-msi" which sets up number of MSI vectors per a device. Note that this call may configure and return lesser number of vectors than requested. 2. RTAS call "ibm,query-interrupt-source-number" which translates MSI vector to interrupt controller (XICS) IRQ number. This patch depends on the "msi/msix: added functions to API to set up message address and data" patch. Signed-off-by: Alexey Kardashevskiy --- hw/spapr.c | 3 + hw/spapr_pci.c | 202 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 205 insertions(+) diff --git a/hw/spapr.c b/hw/spapr.c index 37f6026..b83f83b 100644 --- a/hw/spapr.c +++ b/hw/spapr.c @@ -41,6 +41,7 @@ #include "hw/spapr_vio.h" #include "hw/spapr_pci.h" #include "hw/xics.h" +#include "hw/msi.h" #include "kvm.h" #include "kvm_ppc.h" @@ -661,6 +662,8 @@ static void ppc_spapr_init(ram_addr_t ram_size, long load_limit, rtas_limit, fw_size; char *filename; + msi_supported = true; + spapr = g_malloc0(sizeof(*spapr)); QLIST_INIT(&spapr->phbs); diff --git a/hw/spapr_pci.c b/hw/spapr_pci.c index 650d847..014297b 100644 --- a/hw/spapr_pci.c +++ b/hw/spapr_pci.c @@ -25,6 +25,7 @@ #include "hw.h" #include "pci.h" #include "msi.h" +#include "msix.h" #include "pci_host.h" #include "hw/spapr.h" #include "hw/spapr_pci.h" @@ -34,6 +35,17 @@ #include "hw/pci_internals.h" +/* Copied from the kernel arch/powerpc/platforms/pseries/msi.c */ +#define RTAS_QUERY_FN 0 +#define RTAS_CHANGE_FN 1 +#define RTAS_RESET_FN 2 +#define RTAS_CHANGE_MSI_FN 3 +#define RTAS_CHANGE_MSIX_FN 4 + +/* Interrupt types to return on RTAS_CHANGE_* */ +#define RTAS_TYPE_MSI 1 +#define RTAS_TYPE_MSIX 2 + static sPAPRPHBState *find_phb(sPAPREnvironment *spapr, uint64_t buid) { sPAPRPHBState *phb; @@ -212,6 +224,191 @@ static void rtas_write_pci_config(sPAPREnvironment *spapr, finish_write_pci_config(spapr, 0, addr, size, val, rets); } +/* + * Find an entry with config_addr or returns the empty one if not found AND + * alloc_new is set. + * At the moment the msi_table entries are never released so there is + * no point to look till the end of the list if we need to find the free entry. + */ +static int spapr_msicfg_find(sPAPRPHBState *phb, uint32_t config_addr, + bool alloc_new) +{ + int i; + + for (i = 0; i < SPAPR_MSIX_MAX_DEVS; ++i) { + if (!phb->msi_table[i].nvec) { + break; + } + if (phb->msi_table[i].config_addr == config_addr) { + return i; + } + } + if ((i < SPAPR_MSIX_MAX_DEVS) && alloc_new) { + trace_spapr_pci_msi("Allocating new MSI config", i, config_addr); + return i; + } + + return -1; +} + +/* + * Set MSI/MSIX message data. + * This is required for msi_notify()/msix_notify() which + * will write at the addresses via spapr_msi_write(). + */ +static void spapr_msi_setmsg(PCIDevice *pdev, target_phys_addr_t addr, + bool msix, unsigned req_num) +{ + unsigned i; + MSIMessage msg = { .address = addr, .data = 0 }; + + if (!msix) { + msi_set_message(pdev, msg); + trace_spapr_pci_msi_setup(pdev->name, 0, msg.address); + return; + } + + for (i = 0; i < req_num; ++i) { + msg.address = addr | (i << 2); + msix_set_message(pdev, i, msg); + trace_spapr_pci_msi_setup(pdev->name, i, msg.address); + } +} + +static void rtas_ibm_change_msi(sPAPREnvironment *spapr, + uint32_t token, uint32_t nargs, + target_ulong args, uint32_t nret, + target_ulong rets) +{ + uint32_t config_addr = rtas_ld(args, 0); + uint64_t buid = ((uint64_t)rtas_ld(args, 1) << 32) | rtas_ld(args, 2); + unsigned int func = rtas_ld(args, 3); + unsigned int req_num = rtas_ld(args, 4); /* 0 == remove all */ + unsigned int seq_num = rtas_ld(args, 5); + unsigned int ret_intr_type; + int ndev, irq; + sPAPRPHBState *phb = NULL; + PCIDevice *pdev = NULL; + + switch (func) { + case RTAS_CHANGE_MSI_FN: + case RTAS_CHANGE_FN: + ret_intr_type = RTAS_TYPE_MSI; + break; + case RTAS_CHANGE_MSIX_FN: + ret_intr_type = RTAS_TYPE_MSIX; + break; + default: + fprintf(stderr, "rtas_ibm_change_msi(%u) is not implemented\n", func); + rtas_st(rets, 0, -3); /* Parameter error */ + return; + } + + /* Fins sPAPRPHBState */ + phb = find_phb(spapr, buid); + if (phb) { + pdev = find_dev(spapr, buid, config_addr); + } + if (!phb || !pdev) { + rtas_st(rets, 0, -3); /* Parameter error */ + return; + } + + /* Releasing MSIs */ + if (!req_num) { + ndev = spapr_msicfg_find(phb, config_addr, false); + if (ndev < 0) { + trace_spapr_pci_msi("MSI has not been enabled", -1, config_addr); + rtas_st(rets, 0, -1); /* Hardware error */ + return; + } + trace_spapr_pci_msi("Released MSIs", ndev, config_addr); + rtas_st(rets, 0, 0); + rtas_st(rets, 1, 0); + return; + } + + /* Enabling MSI */ + + /* Find a device number in the map to add or reuse the existing one */ + ndev = spapr_msicfg_find(phb, config_addr, true); + if (ndev >= SPAPR_MSIX_MAX_DEVS) { + fprintf(stderr, "No free entry for a new MSI device\n"); + rtas_st(rets, 0, -1); /* Hardware error */ + return; + } + trace_spapr_pci_msi("Configuring MSI", ndev, config_addr); + + /* Check if there is an old config and MSI number has not changed */ + if (phb->msi_table[ndev].nvec && (req_num != phb->msi_table[ndev].nvec)) { + /* Unexpected behaviour */ + fprintf(stderr, "Cannot reuse MSI config for device#%d", ndev); + rtas_st(rets, 0, -1); /* Hardware error */ + return; + } + + /* There is no cached config, allocate MSIs */ + if (!phb->msi_table[ndev].nvec) { + irq = spapr_allocate_irq_block(req_num, XICS_MSI); + if (irq < 0) { + fprintf(stderr, "Cannot allocate MSIs for device#%d", ndev); + rtas_st(rets, 0, -1); /* Hardware error */ + return; + } + phb->msi_table[ndev].dt_irq = irq; + phb->msi_table[ndev].nvec = req_num; + phb->msi_table[ndev].config_addr = config_addr; + } + + /* Setup MSI/MSIX vectors in the device (via cfgspace or MSIX BAR) */ + spapr_msi_setmsg(pdev, phb->msi_win_addr | (ndev << 16), + ret_intr_type == RTAS_TYPE_MSIX, req_num); + + rtas_st(rets, 0, 0); + rtas_st(rets, 1, req_num); + rtas_st(rets, 2, ++seq_num); + rtas_st(rets, 3, ret_intr_type); + + trace_spapr_pci_rtas_ibm_change_msi(func, req_num); +} + +static void rtas_ibm_query_interrupt_source_number(sPAPREnvironment *spapr, + uint32_t token, + uint32_t nargs, + target_ulong args, + uint32_t nret, + target_ulong rets) +{ + uint32_t config_addr = rtas_ld(args, 0); + uint64_t buid = ((uint64_t)rtas_ld(args, 1) << 32) | rtas_ld(args, 2); + unsigned int intr_src_num = -1, ioa_intr_num = rtas_ld(args, 3); + int ndev; + sPAPRPHBState *phb = NULL; + + /* Fins sPAPRPHBState */ + phb = find_phb(spapr, buid); + if (!phb) { + rtas_st(rets, 0, -3); /* Parameter error */ + return; + } + + /* Find device descriptor and start IRQ */ + ndev = spapr_msicfg_find(phb, config_addr, false); + if (ndev < 0) { + trace_spapr_pci_msi("MSI has not been enabled", -1, config_addr); + rtas_st(rets, 0, -1); /* Hardware error */ + return; + } + + intr_src_num = phb->msi_table[ndev].dt_irq + ioa_intr_num; + trace_spapr_pci_rtas_ibm_query_interrupt_source_number(ioa_intr_num, + intr_src_num); + + rtas_st(rets, 0, 0); + rtas_st(rets, 1, intr_src_num); + rtas_st(rets, 2, 1);/* 0 == level; 1 == edge */ +} + static int pci_spapr_swizzle(int slot, int pin) { return (slot + pin) % PCI_NUM_PINS; @@ -545,6 +742,11 @@ void pci_common_init(void) spapr_rtas_register("write-pci-config", rtas_write_pci_config); spapr_rtas_register("ibm,read-pci-config", rtas_ibm_read_pci_config); spapr_rtas_register("ibm,write-pci-config", rtas_ibm_write_pci_config); + if (msi_supported) { + spapr_rtas_register("ibm,query-interrupt-source-number", + rtas_ibm_query_interrupt_source_number); + spapr_rtas_register("ibm,change-msi", rtas_ibm_change_msi); + } } static void register_types(void)