Message ID | 20180730171748.18562-2-clg@kaod.org |
---|---|
State | New |
Headers | show |
Series | ppc/pnv: Add model for Power8 PHB3 PCIe Host bridge | expand |
On Mon, Jul 30, 2018 at 07:17:48PM +0200, Cédric Le Goater wrote: > From: Benjamin Herrenschmidt <benh@kernel.crashing.org> > > This is a model of the PCIe Host Bridge (PHB3) found on a Power8 > processor. It includes the PowerBus logic interface (PBCQ), IOMMU > support, a single PCIe Gen.3 Root Complex, and support for MSI and LSI > interrupt sources as found on a Power8 system using the XICS interrupt > controller. > > The Power8 processor comes in different flavors: Venice, Murano, > Naple, each having a different number of PHBs. To make things simpler, > the PHB3 model provides 3 per chip. Some platforms, like the > Firestone, can also couple PHBs on the first chip to provide more > bandwidth but this is too specific to model in QEMU. > > No default device layout is provided and PCI devices can be added on > any of the available PCIe Root Port (pcie.0 .. 2 of a Power8 chip) > with address 0x0 as the firwware (skiboot) only accepts a single > device per root port. To run a simple system with a network and a > storage adapters, use a command line options such as : > > -device e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0 > -netdev bridge,id=net0,helper=/usr/libexec/qemu-bridge-helper,br=virbr0,id=hostnet0 > > -device megasas,id=scsi0,bus=pcie.1,addr=0x0 > -drive file=$disk,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=none > -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=2 > > If more are needed, include a bridge. > > Multi chip is supported, each chip adding its set of PHB3 controllers > and its PCI busses. The model doesn't emulate the EEH error handling > (and may never do). Sorry I've let this sit for so long. I discussed the overall model here with BenH, including how we'd adapt it for PHB4 support on POWER9. We did come to a consensus on what I think is the best way to balance the need for hardware accuracy against qemu's desire for uniform configuration with -device. So, first, the overall comment on that: I think the way to go is to have all a chip's PBCQs *always* instantiated (even with -nodefaults) according to the chip's specs. The actual PHBs would be instantiated with -device, and plug into an appropriate existing PBCQ. Now, I think we might be able to do that entirely within the QOM/qdev model by creating a suitable pseudo-bus or buses, but I'm going to need a bit more research on that. Because the powernv model isn't migration stable across versions I'm ok with applying something ad-hoc initially then revising it. > Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> > [clg: rewrote the QOM models > misc fixes > lots of love and care] > Signed-off-by: Cédric Le Goater <clg@kaod.org> > --- > default-configs/ppc64-softmmu.mak | 1 + > include/hw/pci-host/pnv_phb3.h | 171 ++++ > include/hw/pci-host/pnv_phb3_regs.h | 432 ++++++++++ > include/hw/ppc/pnv.h | 22 + > include/hw/ppc/pnv_xscom.h | 9 + > include/hw/ppc/xics.h | 1 + > hw/intc/xics.c | 2 +- > hw/pci-host/pnv_phb3.c | 1146 +++++++++++++++++++++++++++ > hw/pci-host/pnv_phb3_msi.c | 318 ++++++++ > hw/pci-host/pnv_phb3_pbcq.c | 347 ++++++++ > hw/ppc/pnv.c | 75 +- > hw/ppc/pnv_xscom.c | 6 +- > hw/pci-host/Makefile.objs | 1 + > 13 files changed, 2526 insertions(+), 5 deletions(-) > create mode 100644 include/hw/pci-host/pnv_phb3.h > create mode 100644 include/hw/pci-host/pnv_phb3_regs.h > create mode 100644 hw/pci-host/pnv_phb3.c > create mode 100644 hw/pci-host/pnv_phb3_msi.c > create mode 100644 hw/pci-host/pnv_phb3_pbcq.c > > diff --git a/default-configs/ppc64-softmmu.mak b/default-configs/ppc64-softmmu.mak > index b94af6c7c62a..deebba2b044a 100644 > --- a/default-configs/ppc64-softmmu.mak > +++ b/default-configs/ppc64-softmmu.mak > @@ -9,6 +9,7 @@ CONFIG_IPMI=y > CONFIG_IPMI_LOCAL=y > CONFIG_IPMI_EXTERN=y > CONFIG_ISA_IPMI_BT=y > +CONFIG_PCIE_PORT=y AIUI the existing PCIE_PORT isn't exactly like what the PHB3 has, but is close enough for now. > # For pSeries > CONFIG_PSERIES=y > diff --git a/include/hw/pci-host/pnv_phb3.h b/include/hw/pci-host/pnv_phb3.h > new file mode 100644 > index 000000000000..eb9efaf4e13e > --- /dev/null > +++ b/include/hw/pci-host/pnv_phb3.h > @@ -0,0 +1,171 @@ > +/* > + * QEMU PowerPC PowerNV PHB3 model > + * > + * Copyright (c) 2014-2018, IBM Corporation. > + * > + * This code is licensed under the GPL version 2 or later. See the > + * COPYING file in the top-level directory. > + */ > + > +#ifndef PCI_HOST_PNV_PHB3_H > +#define PCI_HOST_PNV_PHB3_H > + > +#include "hw/pci/pcie_host.h" > +#include "hw/pci/pcie_port.h" > +#include "hw/ppc/xics.h" > + > +typedef struct PnvPHB3 PnvPHB3; > +typedef struct PnvChip PnvChip; > + > +/* > + * PHB3 XICS Source for MSIs > + */ > +#define TYPE_PHB3_MSI "phb3-msi" > +#define PHB3_MSI(obj) OBJECT_CHECK(Phb3MsiState, (obj), TYPE_PHB3_MSI) > + > +#define PHB3_MAX_MSI 2048 > + > +typedef struct Phb3MsiState { > + ICSState ics; > + > + PnvPHB3 *phb; > + uint64_t rba[PHB3_MAX_MSI / 64]; > + uint32_t rba_sum; > +} Phb3MsiState; What's the rationale for making the MSI unit a separate QOM device, rather than just folding it into the PHB device? > +void pnv_phb3_msi_update_config(Phb3MsiState *msis, uint32_t base, > + uint32_t count); > +void pnv_phb3_msi_send(Phb3MsiState *msis, uint64_t addr, uint16_t data, > + int32_t dev_pe); > +void pnv_phb3_msi_ffi(Phb3MsiState *msis, uint64_t val); > + > + > +/* We have one such address space wrapper per possible device > + * under the PHB since they need to be assigned statically at > + * qemu device creation time. The relationship to a PE is done > + * later dynamically. This means we can potentially create a lot > + * of these guys. Q35 stores them as some kind of radix tree but > + * we never really need to do fast lookups so instead we simply > + * keep a QLIST of them for now, we can add the radix if needed > + * later on. Could you start this comment of by describing exactly what one of these instances represents, it's not terribly obvious to me. > + * > + * We do cache the PE number to speed things up a bit though. > + */ > +typedef struct PnvPhb3DMASpace { > + PCIBus *bus; > + uint8_t devfn; > + int pe_num; /* Cached PE number */ > +#define PHB_INVALID_PE (-1) > + PnvPHB3 *phb; > + AddressSpace dma_as; > + IOMMUMemoryRegion dma_mr; > + MemoryRegion msi32_mr; > + MemoryRegion msi64_mr; > + bool msi32_mapped; > + bool msi64_mapped; Do you need these, or could you use memory_region_is_mapped() instead? > + QLIST_ENTRY(PnvPhb3DMASpace) list; > +} PnvPhb3DMASpace; > + > +/* > + * PHB3 Power Bus Common Queue > + */ > +#define TYPE_PNV_PBCQ "pnv-pbcq" > +#define PNV_PBCQ(obj) OBJECT_CHECK(PnvPBCQState, (obj), TYPE_PNV_PBCQ) > + > +typedef struct PnvPBCQState { > + DeviceState parent; These have MMIOs, so it probably makes sense to be a SysBusDevice. > + uint32_t nest_xbase; > + uint32_t spci_xbase; > + uint32_t pci_xbase; > +#define PBCQ_NEST_REGS_COUNT 0x46 > +#define PBCQ_PCI_REGS_COUNT 0x15 > +#define PBCQ_SPCI_REGS_COUNT 0x5 > + > + uint64_t nest_regs[PBCQ_NEST_REGS_COUNT]; > + uint64_t spci_regs[PBCQ_SPCI_REGS_COUNT]; > + uint64_t pci_regs[PBCQ_PCI_REGS_COUNT]; > + MemoryRegion mmbar0; > + MemoryRegion mmbar1; > + MemoryRegion phbbar; > + bool mmio0_mapped; > + bool mmio1_mapped; > + bool phb_mapped; > + uint64_t mmio0_base; > + uint64_t mmio0_size; > + uint64_t mmio1_base; > + uint64_t mmio1_size; > + PnvPHB3 *phb; > + > + MemoryRegion xscom_nest_regs; > + MemoryRegion xscom_pci_regs; > + MemoryRegion xscom_spci_regs; > +} PnvPBCQState; > + > +/* > + * PHB3 PCIe Root port > + */ > +#define TYPE_PNV_PHB3_ROOT_BUS "pnv-phb3-root-bus" > + > +#define TYPE_PNV_PHB3_ROOT_PORT "pnv-phb3-root-port" > + > +typedef struct PnvPHB3RootPort { > + PCIESlot parent_obj; > +} PnvPHB3RootPort; > + > +/* > + * PHB3 PCIe Host Bridge for PowerNV machines (POWER8) > + */ > +#define TYPE_PNV_PHB3 "pnv-phb3" > +#define PNV_PHB3(obj) OBJECT_CHECK(PnvPHB3, (obj), TYPE_PNV_PHB3) > + > +#define PNV_PHB3_NUM_M64 16 > +#define PNV_PHB3_NUM_REGS (0x1000 >> 3) > +#define PNV_PHB3_NUM_LSI 8 > +#define PNV_PHB3_NUM_PE 256 > + > +#define PCI_MMIO_TOTAL_SIZE (0x1ull << 60) > + > +struct PnvPHB3 { > + PCIExpressHost parent_obj; > + > + PnvPHB3RootPort root; > + > + uint32_t chip_id; > + uint32_t phb_id; > + char bus_path[8]; > + > + uint64_t regs[PNV_PHB3_NUM_REGS]; > + MemoryRegion mr_regs; > + > + MemoryRegion mr_m32; > + MemoryRegion mr_m64[PNV_PHB3_NUM_M64]; > + bool regs_mapped; > + bool m32_mapped; > + bool m64_mapped[PNV_PHB3_NUM_M64]; > + MemoryRegion pci_mmio; > + MemoryRegion pci_io; > + > + uint64_t ioda_LIST[8]; > + uint64_t ioda_LXIVT[8]; > + uint64_t ioda_TVT[512]; > + uint64_t ioda_M64BT[16]; > + uint64_t ioda_MDT[256]; > + uint64_t ioda_PEEV[4]; > + > + uint32_t total_irq; > + ICSState lsis; > + Phb3MsiState msis; > + > + PnvPBCQState pbcq; I think embedding the pbcq is the wrong way around here. > + > + QLIST_HEAD(, PnvPhb3DMASpace) dma_spaces; > +}; > + > +uint64_t pnv_phb3_reg_read(void *opaque, hwaddr off, unsigned size); > +void pnv_phb3_reg_write(void *opaque, hwaddr off, uint64_t val, unsigned size); > +void pnv_phb3_update_regions(PnvPHB3 *phb); > +void pnv_phb3_remap_irqs(PnvPHB3 *phb); > + > + > +#endif /* PCI_HOST_PNV_PHB3_H */ > diff --git a/include/hw/pci-host/pnv_phb3_regs.h b/include/hw/pci-host/pnv_phb3_regs.h > new file mode 100644 > index 000000000000..46e4e0877d51 > --- /dev/null > +++ b/include/hw/pci-host/pnv_phb3_regs.h > @@ -0,0 +1,432 @@ > +/* Copyright (c) 2013-2018, IBM Corporation. > + * > + * Licensed under the Apache License, Version 2.0 (the "License"); > + * you may not use this file except in compliance with the License. > + * You may obtain a copy of the License at > + * > + * http://www.apache.org/licenses/LICENSE-2.0 > + * > + * Unless required by applicable law or agreed to in writing, software > + * distributed under the License is distributed on an "AS IS" BASIS, > + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or > + * implied. > + * > + * See the License for the specific language governing permissions and > + * limitations under the License. > + */ > + > +#ifndef PCI_HOST_PNV_PHB3_REGS_H > +#define PCI_HOST_PNV_PHB3_REGS_H > + > +/* > + * PBCQ XSCOM registers > + */ > + > +#define PBCQ_NEST_IRSN_COMPARE 0x1a > +#define PBCQ_NEST_IRSN_COMP PPC_BITMASK(0, 18) > +#define PBCQ_NEST_IRSN_MASK 0x1b > +#define PBCQ_NEST_LSI_SRC_ID 0x1f > +#define PBCQ_NEST_LSI_SRC PPC_BITMASK(0, 7) > +#define PBCQ_NEST_REGS_COUNT 0x46 > +#define PBCQ_NEST_MMIO_BAR0 0x40 > +#define PBCQ_NEST_MMIO_BAR1 0x41 > +#define PBCQ_NEST_PHB_BAR 0x42 > +#define PBCQ_NEST_MMIO_MASK0 0x43 > +#define PBCQ_NEST_MMIO_MASK1 0x44 > +#define PBCQ_NEST_BAR_EN 0x45 > +#define PBCQ_NEST_BAR_EN_MMIO0 PPC_BIT(0) > +#define PBCQ_NEST_BAR_EN_MMIO1 PPC_BIT(1) > +#define PBCQ_NEST_BAR_EN_PHB PPC_BIT(2) > +#define PBCQ_NEST_BAR_EN_IRSN_RX PPC_BIT(3) > +#define PBCQ_NEST_BAR_EN_IRSN_TX PPC_BIT(4) > + > +#define PBCQ_PCI_REGS_COUNT 0x15 > +#define PBCQ_PCI_BAR2 0x0b > + > +#define PBCQ_SPCI_REGS_COUNT 0x5 > +#define PBCQ_SPCI_ASB_ADDR 0x0 > +#define PBCQ_SPCI_ASB_STATUS 0x1 > +#define PBCQ_SPCI_ASB_DATA 0x2 > +#define PBCQ_SPCI_AIB_CAPP_EN 0x3 > +#define PBCQ_SPCI_CAPP_SEC_TMR 0x4 > + > +/* > + * PHB MMIO registers > + */ > + > +/* PHB Fundamental register set A */ > +#define PHB_LSI_SOURCE_ID 0x100 > +#define PHB_LSI_SRC_ID PPC_BITMASK(5, 12) > +#define PHB_DMA_CHAN_STATUS 0x110 > +#define PHB_DMA_CHAN_ANY_ERR PPC_BIT(27) > +#define PHB_DMA_CHAN_ANY_ERR1 PPC_BIT(28) > +#define PHB_DMA_CHAN_ANY_FREEZE PPC_BIT(29) > +#define PHB_CPU_LOADSTORE_STATUS 0x120 > +#define PHB_CPU_LS_ANY_ERR PPC_BIT(27) > +#define PHB_CPU_LS_ANY_ERR1 PPC_BIT(28) > +#define PHB_CPU_LS_ANY_FREEZE PPC_BIT(29) > +#define PHB_DMA_MSI_NODE_ID 0x128 > +#define PHB_DMAMSI_NID_FIXED PPC_BIT(0) > +#define PHB_DMAMSI_NID PPC_BITMASK(24, 31) > +#define PHB_CONFIG_DATA 0x130 > +#define PHB_LOCK0 0x138 > +#define PHB_CONFIG_ADDRESS 0x140 > +#define PHB_CA_ENABLE PPC_BIT(0) > +#define PHB_CA_BUS PPC_BITMASK(4, 11) > +#define PHB_CA_DEV PPC_BITMASK(12, 16) > +#define PHB_CA_FUNC PPC_BITMASK(17, 19) > +#define PHB_CA_REG PPC_BITMASK(20, 31) > +#define PHB_CA_PE PPC_BITMASK(40, 47) > +#define PHB_LOCK1 0x148 > +#define PHB_IVT_BAR 0x150 > +#define PHB_IVT_BAR_ENABLE PPC_BIT(0) > +#define PHB_IVT_BASE_ADDRESS_MASK PPC_BITMASK(14, 48) > +#define PHB_IVT_LENGTH_MASK PPC_BITMASK(52, 63) > +#define PHB_RBA_BAR 0x158 > +#define PHB_RBA_BAR_ENABLE PPC_BIT(0) > +#define PHB_RBA_BASE_ADDRESS PPC_BITMASK(14, 55) > +#define PHB_PHB3_CONFIG 0x160 > +#define PHB_PHB3C_64B_TCE_EN PPC_BIT(2) > +#define PHB_PHB3C_32BIT_MSI_EN PPC_BIT(8) > +#define PHB_PHB3C_64BIT_MSI_EN PPC_BIT(14) > +#define PHB_PHB3C_M32_EN PPC_BIT(16) > +#define PHB_RTT_BAR 0x168 > +#define PHB_RTT_BAR_ENABLE PPC_BIT(0) > +#define PHB_RTT_BASE_ADDRESS_MASK PPC_BITMASK(14, 46) > +#define PHB_PELTV_BAR 0x188 > +#define PHB_PELTV_BAR_ENABLE PPC_BIT(0) > +#define PHB_PELTV_BASE_ADDRESS PPC_BITMASK(14, 50) > +#define PHB_M32_BASE_ADDR 0x190 > +#define PHB_M32_BASE_MASK 0x198 > +#define PHB_M32_START_ADDR 0x1a0 > +#define PHB_PEST_BAR 0x1a8 > +#define PHB_PEST_BAR_ENABLE PPC_BIT(0) > +#define PHB_PEST_BASE_ADDRESS PPC_BITMASK(14, 51) > +#define PHB_M64_UPPER_BITS 0x1f0 > +#define PHB_INTREP_TIMER 0x1f8 > +#define PHB_DMARD_SYNC 0x200 > +#define PHB_RTC_INVALIDATE 0x208 > +#define PHB_RTC_INVALIDATE_ALL PPC_BIT(0) > +#define PHB_RTC_INVALIDATE_RID PPC_BITMASK(16, 31) > +#define PHB_TCE_KILL 0x210 > +#define PHB_TCE_KILL_ALL PPC_BIT(0) > +#define PHB_TCE_SPEC_CTL 0x218 > +#define PHB_IODA_ADDR 0x220 > +#define PHB_IODA_AD_AUTOINC PPC_BIT(0) > +#define PHB_IODA_AD_TSEL PPC_BITMASK(11, 15) > +#define PHB_IODA_AD_TADR PPC_BITMASK(55, 63) > +#define PHB_IODA_DATA0 0x228 > +#define PHB_FFI_REQUEST 0x238 > +#define PHB_FFI_LOCK_CLEAR PPC_BIT(3) > +#define PHB_FFI_REQUEST_ISN PPC_BITMASK(49, 59) > +#define PHB_FFI_LOCK 0x240 > +#define PHB_FFI_LOCK_STATE PPC_BIT(0) > +#define PHB_XIVE_UPDATE 0x248 /* Broken in DD1 */ > +#define PHB_PHB3_GEN_CAP 0x250 > +#define PHB_PHB3_TCE_CAP 0x258 > +#define PHB_PHB3_IRQ_CAP 0x260 > +#define PHB_PHB3_EEH_CAP 0x268 > +#define PHB_IVC_INVALIDATE 0x2a0 > +#define PHB_IVC_INVALIDATE_ALL PPC_BIT(0) > +#define PHB_IVC_INVALIDATE_SID PPC_BITMASK(16, 31) > +#define PHB_IVC_UPDATE 0x2a8 > +#define PHB_IVC_UPDATE_ENABLE_P PPC_BIT(0) > +#define PHB_IVC_UPDATE_ENABLE_Q PPC_BIT(1) > +#define PHB_IVC_UPDATE_ENABLE_SERVER PPC_BIT(2) > +#define PHB_IVC_UPDATE_ENABLE_PRI PPC_BIT(3) > +#define PHB_IVC_UPDATE_ENABLE_GEN PPC_BIT(4) > +#define PHB_IVC_UPDATE_ENABLE_CON PPC_BIT(5) > +#define PHB_IVC_UPDATE_GEN_MATCH PPC_BITMASK(6, 7) > +#define PHB_IVC_UPDATE_SERVER PPC_BITMASK(8, 23) > +#define PHB_IVC_UPDATE_PRI PPC_BITMASK(24, 31) > +#define PHB_IVC_UPDATE_GEN PPC_BITMASK(32, 33) > +#define PHB_IVC_UPDATE_P PPC_BITMASK(34, 34) > +#define PHB_IVC_UPDATE_Q PPC_BITMASK(35, 35) > +#define PHB_IVC_UPDATE_SID PPC_BITMASK(48, 63) > +#define PHB_PAPR_ERR_INJ_CTL 0x2b0 > +#define PHB_PAPR_ERR_INJ_CTL_INB PPC_BIT(0) > +#define PHB_PAPR_ERR_INJ_CTL_OUTB PPC_BIT(1) > +#define PHB_PAPR_ERR_INJ_CTL_STICKY PPC_BIT(2) > +#define PHB_PAPR_ERR_INJ_CTL_CFG PPC_BIT(3) > +#define PHB_PAPR_ERR_INJ_CTL_RD PPC_BIT(4) > +#define PHB_PAPR_ERR_INJ_CTL_WR PPC_BIT(5) > +#define PHB_PAPR_ERR_INJ_CTL_FREEZE PPC_BIT(6) > +#define PHB_PAPR_ERR_INJ_ADDR 0x2b8 > +#define PHB_PAPR_ERR_INJ_ADDR_MMIO PPC_BITMASK(16, 63) > +#define PHB_PAPR_ERR_INJ_MASK 0x2c0 > +#define PHB_PAPR_ERR_INJ_MASK_CFG PPC_BITMASK(4, 11) > +#define PHB_PAPR_ERR_INJ_MASK_MMIO PPC_BITMASK(16, 63) > +#define PHB_ETU_ERR_SUMMARY 0x2c8 > + > +/* UTL registers */ > +#define UTL_SYS_BUS_CONTROL 0x400 > +#define UTL_STATUS 0x408 > +#define UTL_SYS_BUS_AGENT_STATUS 0x410 > +#define UTL_SYS_BUS_AGENT_ERR_SEVERITY 0x418 > +#define UTL_SYS_BUS_AGENT_IRQ_EN 0x420 > +#define UTL_SYS_BUS_BURST_SZ_CONF 0x440 > +#define UTL_REVISION_ID 0x448 > +#define UTL_BCLK_DOMAIN_DBG1 0x460 > +#define UTL_BCLK_DOMAIN_DBG2 0x468 > +#define UTL_BCLK_DOMAIN_DBG3 0x470 > +#define UTL_BCLK_DOMAIN_DBG4 0x478 > +#define UTL_BCLK_DOMAIN_DBG5 0x480 > +#define UTL_BCLK_DOMAIN_DBG6 0x488 > +#define UTL_OUT_POST_HDR_BUF_ALLOC 0x4c0 > +#define UTL_OUT_POST_DAT_BUF_ALLOC 0x4d0 > +#define UTL_IN_POST_HDR_BUF_ALLOC 0x4e0 > +#define UTL_IN_POST_DAT_BUF_ALLOC 0x4f0 > +#define UTL_OUT_NP_BUF_ALLOC 0x500 > +#define UTL_IN_NP_BUF_ALLOC 0x510 > +#define UTL_PCIE_TAGS_ALLOC 0x520 > +#define UTL_GBIF_READ_TAGS_ALLOC 0x530 > +#define UTL_PCIE_PORT_CONTROL 0x540 > +#define UTL_PCIE_PORT_STATUS 0x548 > +#define UTL_PCIE_PORT_ERROR_SEV 0x550 > +#define UTL_PCIE_PORT_IRQ_EN 0x558 > +#define UTL_RC_STATUS 0x560 > +#define UTL_RC_ERR_SEVERITY 0x568 > +#define UTL_RC_IRQ_EN 0x570 > +#define UTL_EP_STATUS 0x578 > +#define UTL_EP_ERR_SEVERITY 0x580 > +#define UTL_EP_ERR_IRQ_EN 0x588 > +#define UTL_PCI_PM_CTRL1 0x590 > +#define UTL_PCI_PM_CTRL2 0x598 > +#define UTL_GP_CTL1 0x5a0 > +#define UTL_GP_CTL2 0x5a8 > +#define UTL_PCLK_DOMAIN_DBG1 0x5b0 > +#define UTL_PCLK_DOMAIN_DBG2 0x5b8 > +#define UTL_PCLK_DOMAIN_DBG3 0x5c0 > +#define UTL_PCLK_DOMAIN_DBG4 0x5c8 > + > +/* PCI-E Stack registers */ > +#define PHB_PCIE_SYSTEM_CONFIG 0x600 > +#define PHB_PCIE_BUS_NUMBER 0x608 > +#define PHB_PCIE_SYSTEM_TEST 0x618 > +#define PHB_PCIE_LINK_MANAGEMENT 0x630 > +#define PHB_PCIE_LM_LINK_ACTIVE PPC_BIT(8) > +#define PHB_PCIE_DLP_TRAIN_CTL 0x640 > +#define PHB_PCIE_DLP_TCTX_DISABLE PPC_BIT(1) > +#define PHB_PCIE_DLP_TCRX_DISABLED PPC_BIT(16) > +#define PHB_PCIE_DLP_INBAND_PRESENCE PPC_BIT(19) > +#define PHB_PCIE_DLP_TC_DL_LINKUP PPC_BIT(21) > +#define PHB_PCIE_DLP_TC_DL_PGRESET PPC_BIT(22) > +#define PHB_PCIE_DLP_TC_DL_LINKACT PPC_BIT(23) > +#define PHB_PCIE_SLOP_LOOPBACK_STATUS 0x648 > +#define PHB_PCIE_SYS_LINK_INIT 0x668 > +#define PHB_PCIE_UTL_CONFIG 0x670 > +#define PHB_PCIE_DLP_CONTROL 0x678 > +#define PHB_PCIE_UTL_ERRLOG1 0x680 > +#define PHB_PCIE_UTL_ERRLOG2 0x688 > +#define PHB_PCIE_UTL_ERRLOG3 0x690 > +#define PHB_PCIE_UTL_ERRLOG4 0x698 > +#define PHB_PCIE_DLP_ERRLOG1 0x6a0 > +#define PHB_PCIE_DLP_ERRLOG2 0x6a8 > +#define PHB_PCIE_DLP_ERR_STATUS 0x6b0 > +#define PHB_PCIE_DLP_ERR_COUNTERS 0x6b8 > +#define PHB_PCIE_UTL_ERR_INJECT 0x6c0 > +#define PHB_PCIE_TLDLP_ERR_INJECT 0x6c8 > +#define PHB_PCIE_LANE_EQ_CNTL0 0x6d0 > +#define PHB_PCIE_LANE_EQ_CNTL1 0x6d8 > +#define PHB_PCIE_LANE_EQ_CNTL2 0x6e0 > +#define PHB_PCIE_LANE_EQ_CNTL3 0x6e8 > +#define PHB_PCIE_STRAPPING 0x700 > + > +/* Fundamental register set B */ > +#define PHB_VERSION 0x800 > +#define PHB_RESET 0x808 > +#define PHB_CONTROL 0x810 > +#define PHB_CTRL_IVE_128_BYTES PPC_BIT(24) > +#define PHB_AIB_RX_CRED_INIT_TIMER 0x818 > +#define PHB_AIB_RX_CMD_CRED 0x820 > +#define PHB_AIB_RX_DATA_CRED 0x828 > +#define PHB_AIB_TX_CMD_CRED 0x830 > +#define PHB_AIB_TX_DATA_CRED 0x838 > +#define PHB_AIB_TX_CHAN_MAPPING 0x840 > +#define PHB_AIB_TAG_ENABLE 0x858 > +#define PHB_AIB_FENCE_CTRL 0x860 > +#define PHB_TCE_TAG_ENABLE 0x868 > +#define PHB_TCE_WATERMARK 0x870 > +#define PHB_TIMEOUT_CTRL1 0x878 > +#define PHB_TIMEOUT_CTRL2 0x880 > +#define PHB_QUIESCE_DMA_G 0x888 > +#define PHB_AIB_TAG_STATUS 0x900 > +#define PHB_TCE_TAG_STATUS 0x908 > + > +/* FIR & Error registers */ > +#define PHB_LEM_FIR_ACCUM 0xc00 > +#define PHB_LEM_FIR_AND_MASK 0xc08 > +#define PHB_LEM_FIR_OR_MASK 0xc10 > +#define PHB_LEM_ERROR_MASK 0xc18 > +#define PHB_LEM_ERROR_AND_MASK 0xc20 > +#define PHB_LEM_ERROR_OR_MASK 0xc28 > +#define PHB_LEM_ACTION0 0xc30 > +#define PHB_LEM_ACTION1 0xc38 > +#define PHB_LEM_WOF 0xc40 > +#define PHB_ERR_STATUS 0xc80 > +#define PHB_ERR1_STATUS 0xc88 > +#define PHB_ERR_INJECT 0xc90 > +#define PHB_ERR_LEM_ENABLE 0xc98 > +#define PHB_ERR_IRQ_ENABLE 0xca0 > +#define PHB_ERR_FREEZE_ENABLE 0xca8 > +#define PHB_ERR_AIB_FENCE_ENABLE 0xcb0 > +#define PHB_ERR_LOG_0 0xcc0 > +#define PHB_ERR_LOG_1 0xcc8 > +#define PHB_ERR_STATUS_MASK 0xcd0 > +#define PHB_ERR1_STATUS_MASK 0xcd8 > + > +#define PHB_OUT_ERR_STATUS 0xd00 > +#define PHB_OUT_ERR1_STATUS 0xd08 > +#define PHB_OUT_ERR_INJECT 0xd10 > +#define PHB_OUT_ERR_LEM_ENABLE 0xd18 > +#define PHB_OUT_ERR_IRQ_ENABLE 0xd20 > +#define PHB_OUT_ERR_FREEZE_ENABLE 0xd28 > +#define PHB_OUT_ERR_AIB_FENCE_ENABLE 0xd30 > +#define PHB_OUT_ERR_LOG_0 0xd40 > +#define PHB_OUT_ERR_LOG_1 0xd48 > +#define PHB_OUT_ERR_STATUS_MASK 0xd50 > +#define PHB_OUT_ERR1_STATUS_MASK 0xd58 > + > +#define PHB_INA_ERR_STATUS 0xd80 > +#define PHB_INA_ERR1_STATUS 0xd88 > +#define PHB_INA_ERR_INJECT 0xd90 > +#define PHB_INA_ERR_LEM_ENABLE 0xd98 > +#define PHB_INA_ERR_IRQ_ENABLE 0xda0 > +#define PHB_INA_ERR_FREEZE_ENABLE 0xda8 > +#define PHB_INA_ERR_AIB_FENCE_ENABLE 0xdb0 > +#define PHB_INA_ERR_LOG_0 0xdc0 > +#define PHB_INA_ERR_LOG_1 0xdc8 > +#define PHB_INA_ERR_STATUS_MASK 0xdd0 > +#define PHB_INA_ERR1_STATUS_MASK 0xdd8 > + > +#define PHB_INB_ERR_STATUS 0xe00 > +#define PHB_INB_ERR1_STATUS 0xe08 > +#define PHB_INB_ERR_INJECT 0xe10 > +#define PHB_INB_ERR_LEM_ENABLE 0xe18 > +#define PHB_INB_ERR_IRQ_ENABLE 0xe20 > +#define PHB_INB_ERR_FREEZE_ENABLE 0xe28 > +#define PHB_INB_ERR_AIB_FENCE_ENABLE 0xe30 > +#define PHB_INB_ERR_LOG_0 0xe40 > +#define PHB_INB_ERR_LOG_1 0xe48 > +#define PHB_INB_ERR_STATUS_MASK 0xe50 > +#define PHB_INB_ERR1_STATUS_MASK 0xe58 > + > +/* Performance monitor & Debug registers */ > +#define PHB_TRACE_CONTROL 0xf80 > +#define PHB_PERFMON_CONFIG 0xf88 > +#define PHB_PERFMON_CTR0 0xf90 > +#define PHB_PERFMON_CTR1 0xf98 > +#define PHB_PERFMON_CTR2 0xfa0 > +#define PHB_PERFMON_CTR3 0xfa8 > +#define PHB_HOTPLUG_OVERRIDE 0xfb0 > +#define PHB_HPOVR_FORCE_RESAMPLE PPC_BIT(9) > +#define PHB_HPOVR_PRESENCE_A PPC_BIT(10) > +#define PHB_HPOVR_PRESENCE_B PPC_BIT(11) > +#define PHB_HPOVR_LINK_ACTIVE PPC_BIT(12) > +#define PHB_HPOVR_LINK_BIFURCATED PPC_BIT(13) > +#define PHB_HPOVR_LINK_LANE_SWAPPED PPC_BIT(14) > + > +/* > + * IODA2 on-chip tables > + */ > + > +#define IODA2_TBL_LIST 1 > +#define IODA2_TBL_LXIVT 2 > +#define IODA2_TBL_IVC_CAM 3 > +#define IODA2_TBL_RBA 4 > +#define IODA2_TBL_RCAM 5 > +#define IODA2_TBL_MRT 6 > +#define IODA2_TBL_PESTA 7 > +#define IODA2_TBL_PESTB 8 > +#define IODA2_TBL_TVT 9 > +#define IODA2_TBL_TCAM 10 > +#define IODA2_TBL_TDR 11 > +#define IODA2_TBL_M64BT 16 > +#define IODA2_TBL_M32DT 17 > +#define IODA2_TBL_PEEV 20 > + > +/* LXIVT */ > +#define IODA2_LXIVT_SERVER PPC_BITMASK(8, 23) > +#define IODA2_LXIVT_PRIORITY PPC_BITMASK(24, 31) > +#define IODA2_LXIVT_NODE_ID PPC_BITMASK(56, 63) > + > +/* IVT */ > +#define IODA2_IVT_SERVER PPC_BITMASK(0, 23) > +#define IODA2_IVT_PRIORITY PPC_BITMASK(24, 31) > +#define IODA2_IVT_GEN PPC_BITMASK(37, 38) > +#define IODA2_IVT_P PPC_BITMASK(39, 39) > +#define IODA2_IVT_Q PPC_BITMASK(47, 47) > +#define IODA2_IVT_PE PPC_BITMASK(48, 63) > + > +/* TVT */ > +#define IODA2_TVT_TABLE_ADDR PPC_BITMASK(0, 47) > +#define IODA2_TVT_NUM_LEVELS PPC_BITMASK(48, 50) > +#define IODA2_TVE_1_LEVEL 0 > +#define IODA2_TVE_2_LEVELS 1 > +#define IODA2_TVE_3_LEVELS 2 > +#define IODA2_TVE_4_LEVELS 3 > +#define IODA2_TVE_5_LEVELS 4 > +#define IODA2_TVT_TCE_TABLE_SIZE PPC_BITMASK(51, 55) > +#define IODA2_TVT_IO_PSIZE PPC_BITMASK(59, 63) > + > +/* PESTA */ > +#define IODA2_PESTA_MMIO_FROZEN PPC_BIT(0) > + > +/* PESTB */ > +#define IODA2_PESTB_DMA_STOPPED PPC_BIT(0) > + > +/* M32DT */ > +#define IODA2_M32DT_PE PPC_BITMASK(8, 15) > + > +/* M64BT */ > +#define IODA2_M64BT_ENABLE PPC_BIT(0) > +#define IODA2_M64BT_SINGLE_PE PPC_BIT(1) > +#define IODA2_M64BT_BASE PPC_BITMASK(2, 31) > +#define IODA2_M64BT_MASK PPC_BITMASK(34, 63) > +#define IODA2_M64BT_SINGLE_BASE PPC_BITMASK(2, 26) > +#define IODA2_M64BT_PE_HI PPC_BITMASK(27, 31) > +#define IODA2_M64BT_SINGLE_MASK PPC_BITMASK(34, 58) > +#define IODA2_M64BT_PE_LOW PPC_BITMASK(59, 63) > + > +/* > + * IODA2 in-memory tables > + */ > + > +/* PEST > + * > + * 2x8 bytes entries, PEST0 and PEST1 > + */ > + > +#define IODA2_PEST0_MMIO_CAUSE PPC_BIT(2) > +#define IODA2_PEST0_CFG_READ PPC_BIT(3) > +#define IODA2_PEST0_CFG_WRITE PPC_BIT(4) > +#define IODA2_PEST0_TTYPE PPC_BITMASK(5, 7) > +#define PEST_TTYPE_DMA_WRITE 0 > +#define PEST_TTYPE_MSI 1 > +#define PEST_TTYPE_DMA_READ 2 > +#define PEST_TTYPE_DMA_READ_RESP 3 > +#define PEST_TTYPE_MMIO_LOAD 4 > +#define PEST_TTYPE_MMIO_STORE 5 > +#define PEST_TTYPE_OTHER 7 > +#define IODA2_PEST0_CA_RETURN PPC_BIT(8) > +#define IODA2_PEST0_UTL_RTOS_TIMEOUT PPC_BIT(8) /* Same bit as CA return */ > +#define IODA2_PEST0_UR_RETURN PPC_BIT(9) > +#define IODA2_PEST0_UTL_NONFATAL PPC_BIT(10) > +#define IODA2_PEST0_UTL_FATAL PPC_BIT(11) > +#define IODA2_PEST0_PARITY_UE PPC_BIT(13) > +#define IODA2_PEST0_UTL_CORRECTABLE PPC_BIT(14) > +#define IODA2_PEST0_UTL_INTERRUPT PPC_BIT(15) > +#define IODA2_PEST0_MMIO_XLATE PPC_BIT(16) > +#define IODA2_PEST0_IODA2_ERROR PPC_BIT(16) /* Same bit as MMIO xlate */ > +#define IODA2_PEST0_TCE_PAGE_FAULT PPC_BIT(18) > +#define IODA2_PEST0_TCE_ACCESS_FAULT PPC_BIT(19) > +#define IODA2_PEST0_DMA_RESP_TIMEOUT PPC_BIT(20) > +#define IODA2_PEST0_AIB_SIZE_INVALID PPC_BIT(21) > +#define IODA2_PEST0_LEM_BIT PPC_BITMASK(26, 31) > +#define IODA2_PEST0_RID PPC_BITMASK(32, 47) > +#define IODA2_PEST0_MSI_DATA PPC_BITMASK(48, 63) > + > +#define IODA2_PEST1_FAIL_ADDR PPC_BITMASK(3, 63) > + > + > +#endif /* PCI_HOST_PNV_PHB3_REGS_H */ > diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h > index 86d5f54e5459..61af65fc0b76 100644 > --- a/include/hw/ppc/pnv.h > +++ b/include/hw/ppc/pnv.h > @@ -25,6 +25,7 @@ > #include "hw/ppc/pnv_lpc.h" > #include "hw/ppc/pnv_psi.h" > #include "hw/ppc/pnv_occ.h" > +#include "hw/pci-host/pnv_phb3.h" > > #define TYPE_PNV_CHIP "pnv-chip" > #define PNV_CHIP(obj) OBJECT_CHECK(PnvChip, (obj), TYPE_PNV_CHIP) > @@ -72,6 +73,9 @@ typedef struct Pnv8Chip { > PnvLpcController lpc; > PnvPsi psi; > PnvOCC occ; > + > +#define PNV8_CHIP_PHB3_MAX 4 > + PnvPHB3 phbs[PNV8_CHIP_PHB3_MAX]; > } Pnv8Chip; > > #define TYPE_PNV9_CHIP "pnv9-chip" > @@ -92,6 +96,7 @@ typedef struct PnvChipClass { > PnvChipType chip_type; > uint64_t chip_cfam_id; > uint64_t cores_mask; > + uint32_t num_phbs; You can probably drop this if you instead have max possible PBCQs always present, which may or may not have PHBs instantiated. > hwaddr xscom_base; > > @@ -171,6 +176,23 @@ static inline bool pnv_is_power9(PnvMachineState *pnv) > return pnv_chip_is_power9(pnv->chips[0]); > } > > +/* > + * This is used by devices created on the command line to find a chip > + * on which to attach to. > + */ > +static inline PnvChip *pnv_get_chip(PnvMachineState *pnv, uint chip_id) > +{ > + int i; > + > + for (i = 0; i < pnv->num_chips; i++) { > + PnvChip *chip = pnv->chips[i]; > + if (chip->chip_id == chip_id) { > + return chip; > + } > + } > + return NULL; > +} > + > #define PNV_FDT_ADDR 0x01000000 > #define PNV_TIMEBASE_FREQ 512000000ULL > > diff --git a/include/hw/ppc/pnv_xscom.h b/include/hw/ppc/pnv_xscom.h > index 255b26a5aaf6..e7ebffa5cf8a 100644 > --- a/include/hw/ppc/pnv_xscom.h > +++ b/include/hw/ppc/pnv_xscom.h > @@ -73,6 +73,15 @@ typedef struct PnvXScomInterfaceClass { > #define PNV_XSCOM_OCC_BASE 0x0066000 > #define PNV_XSCOM_OCC_SIZE 0x6000 > > +#define PNV_XSCOM_PBCQ_NEST_BASE 0x2012000 > +#define PNV_XSCOM_PBCQ_NEST_SIZE 0x46 > + > +#define PNV_XSCOM_PBCQ_PCI_BASE 0x9012000 > +#define PNV_XSCOM_PBCQ_PCI_SIZE 0x15 > + > +#define PNV_XSCOM_PBCQ_SPCI_BASE 0x9013c00 > +#define PNV_XSCOM_PBCQ_SPCI_SIZE 0x5 > + > extern void pnv_xscom_realize(PnvChip *chip, Error **errp); > extern int pnv_dt_xscom(PnvChip *chip, void *fdt, int offset); > > diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h > index 6ac8a9392da6..966a996c2eac 100644 > --- a/include/hw/ppc/xics.h > +++ b/include/hw/ppc/xics.h > @@ -194,6 +194,7 @@ void icp_set_mfrr(ICPState *icp, uint8_t mfrr); > uint32_t icp_accept(ICPState *ss); > uint32_t icp_ipoll(ICPState *ss, uint32_t *mfrr); > void icp_eoi(ICPState *icp, uint32_t xirr); > +void icp_irq(ICSState *ics, int server, int nr, uint8_t priority); > > void ics_simple_write_xive(ICSState *ics, int nr, int server, > uint8_t priority, uint8_t saved_priority); > diff --git a/hw/intc/xics.c b/hw/intc/xics.c > index c90c893228dc..30fd9eb47e90 100644 > --- a/hw/intc/xics.c > +++ b/hw/intc/xics.c > @@ -230,7 +230,7 @@ void icp_eoi(ICPState *icp, uint32_t xirr) > } > } > > -static void icp_irq(ICSState *ics, int server, int nr, uint8_t priority) > +void icp_irq(ICSState *ics, int server, int nr, uint8_t priority) > { > ICPState *icp = xics_icp_get(ics->xics, server); > > diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c > new file mode 100644 > index 000000000000..67f6dd94a705 > --- /dev/null > +++ b/hw/pci-host/pnv_phb3.c > @@ -0,0 +1,1146 @@ > +/* > + * QEMU PowerPC PowerNV PHB3 model > + * > + * Copyright (c) 2014-2018, IBM Corporation. > + * > + * This code is licensed under the GPL version 2 or later. See the > + * COPYING file in the top-level directory. > + */ > +#include "qemu/osdep.h" > +#include "qemu/log.h" > +#include "qapi/visitor.h" > +#include "qapi/error.h" > +#include "qemu-common.h" > +#include "target/ppc/cpu.h" > +#include "hw/pci-host/pnv_phb3_regs.h" > +#include "hw/pci-host/pnv_phb3.h" > +#include "hw/pci/pcie_host.h" > +#include "hw/pci/pcie_port.h" > +#include "hw/ppc/pnv.h" > + > +#define phb3_error(fmt, ...) \ > + qemu_log_mask(LOG_GUEST_ERROR, "phb3: " fmt "\n", ## __VA_ARGS__) > + > + > +static PCIDevice *pnv_phb3_find_cfg_dev(PnvPHB3 *phb) > +{ > + PCIHostState *pci = PCI_HOST_BRIDGE(phb); > + uint64_t addr = phb->regs[PHB_CONFIG_ADDRESS >> 3]; > + uint8_t bus, devfn; > + > + if (!(addr >> 63)) { > + return NULL; > + } > + bus = (addr >> 52) & 0xff; > + devfn = (addr >> 44) & 0xff; > + > + return pci_find_device(pci->bus, bus, devfn); > +} > + > +/* > + * The CONFIG_DATA register expects little endian accesses, but as the > + * region is big endian, we have to swap the value. > + */ > +static void pnv_phb3_config_write(PnvPHB3 *phb, unsigned off, > + unsigned size, uint64_t val) > +{ > + uint32_t cfg_addr, limit; > + PCIDevice *pdev; > + > + pdev = pnv_phb3_find_cfg_dev(phb); > + if (!pdev) { > + return; > + } > + cfg_addr = (phb->regs[PHB_CONFIG_ADDRESS >> 3] >> 32) & 0xffc; > + cfg_addr |= off; > + limit = pci_config_size(pdev); > + if (limit <= cfg_addr) { > + /* conventional pci device can be behind pcie-to-pci bridge. > + 256 <= addr < 4K has no effects. */ > + return; > + } > + switch (size) { > + case 1: > + break; > + case 2: > + val = bswap16(val); > + break; > + case 4: > + val = bswap32(val); > + break; > + default: > + g_assert_not_reached(); > + } > + pci_host_config_write_common(pdev, cfg_addr, limit, val, size); > +} > + > +static uint64_t pnv_phb3_config_read(PnvPHB3 *phb, unsigned off, > + unsigned size) > +{ > + uint32_t cfg_addr, limit; > + PCIDevice *pdev; > + uint64_t val; > + > + pdev = pnv_phb3_find_cfg_dev(phb); > + if (!pdev) { > + return ~0ull; > + } > + cfg_addr = (phb->regs[PHB_CONFIG_ADDRESS >> 3] >> 32) & 0xffc; > + cfg_addr |= off; > + limit = pci_config_size(pdev); > + if (limit <= cfg_addr) { > + /* conventional pci device can be behind pcie-to-pci bridge. > + 256 <= addr < 4K has no effects. */ > + return ~0ull; > + } > + val = pci_host_config_read_common(pdev, cfg_addr, limit, size); > + switch (size) { > + case 1: > + return val; > + case 2: > + return bswap16(val); > + case 4: > + return bswap32(val); > + default: > + g_assert_not_reached(); > + } > +} > + > +static void pnv_phb3_check_m32(PnvPHB3 *phb) > +{ > + uint64_t base, start, size; > + MemoryRegion *parent; > + PnvPBCQState *pbcq = &phb->pbcq; > + > + if (phb->m32_mapped) { > + memory_region_del_subregion(phb->mr_m32.container, &phb->mr_m32); > + phb->m32_mapped = false; > + } > + > + if (!(phb->regs[PHB_PHB3_CONFIG >> 3] & PHB_PHB3C_M32_EN)) { > + return; > + } > + > + /* Grab geometry from registers */ > + base = phb->regs[PHB_M32_BASE_ADDR >> 3]; > + start = phb->regs[PHB_M32_START_ADDR >> 3]; > + size = ~(phb->regs[PHB_M32_BASE_MASK >> 3] | 0xfffc000000000000ull) + 1; > + > + /* Check if it matches an enabled MMIO region in the PBCQ */ > + if (pbcq->mmio0_mapped && base >= pbcq->mmio0_base && > + (base + size) <= (pbcq->mmio0_base + pbcq->mmio0_size)) { > + parent = &pbcq->mmbar0; > + base -= pbcq->mmio0_base; > + } else if (pbcq->mmio1_mapped && base >= pbcq->mmio1_base && > + (base + size) <= (pbcq->mmio1_base + pbcq->mmio1_size)) { > + parent = &pbcq->mmbar1; > + base -= pbcq->mmio1_base; > + } else { > + return; > + } > + > + /* Create alias */ > + memory_region_init_alias(&phb->mr_m32, OBJECT(phb), "phb3-m32", > + &phb->pci_mmio, start, size); > + memory_region_add_subregion(parent, base, &phb->mr_m32); > + phb->m32_mapped = true; > +} > + > +static void pnv_phb3_check_m64(PnvPHB3 *phb, uint32_t index) > +{ > + uint64_t base, start, size, m64; > + MemoryRegion *parent; > + PnvPBCQState *pbcq = &phb->pbcq; > + > + if (phb->m64_mapped[index]) { > + /* Should we destroy it in RCU friendly way... ? */ I wouldn't think so. This should be safely under the BQL, and I don't imagine the PHB mappings are changed often, so I doubt there's a reason to optimize it. > + memory_region_del_subregion(phb->mr_m64[index].container, > + &phb->mr_m64[index]); > + phb->m64_mapped[index] = false; > + } > + > + /* Get table entry */ > + m64 = phb->ioda_M64BT[index]; > + > + if (!(m64 & IODA2_M64BT_ENABLE)) { > + return; > + } > + > + /* Grab geometry from registers */ > + base = GETFIELD(IODA2_M64BT_BASE, m64) << 20; > + if (m64 & IODA2_M64BT_SINGLE_PE) { > + base &= ~0x1ffffffull; > + } > + size = GETFIELD(IODA2_M64BT_MASK, m64) << 20; > + size |= 0xfffc000000000000ull; > + size = ~size + 1; > + start = base | (phb->regs[PHB_M64_UPPER_BITS >> 3]); > + > + /* Check if it matches an enabled MMIO region in the PBCQ */ > + if (pbcq->mmio0_mapped && base >= pbcq->mmio0_base && > + (base + size) <= (pbcq->mmio0_base + pbcq->mmio0_size)) { > + parent = &pbcq->mmbar0; > + base -= pbcq->mmio0_base; > + } else if (pbcq->mmio1_mapped && base >= pbcq->mmio1_base && > + (base + size) <= (pbcq->mmio1_base + pbcq->mmio1_size)) { > + parent = &pbcq->mmbar1; > + base -= pbcq->mmio1_base; > + } else { > + return; > + } > + > + /* Create alias */ > + memory_region_init_alias(&phb->mr_m64[index], OBJECT(phb), "phb3-m64", > + &phb->pci_mmio, start, size); > + memory_region_add_subregion(parent, base, &phb->mr_m64[index]); > + phb->m64_mapped[index] = true; > +} > + > +static void pnv_phb3_check_all_m64s(PnvPHB3 *phb) > +{ > + uint64_t i; > + > + for (i = 0; i < PNV_PHB3_NUM_M64; i++) { > + pnv_phb3_check_m64(phb, i); > + } > +} > + > +static void pnv_phb3_lxivt_write(PnvPHB3 *phb, unsigned idx, uint64_t val) > +{ > + ICSState *ics = &phb->lsis; > + uint8_t server, prio; > + > + phb->ioda_LXIVT[idx] = val & (IODA2_LXIVT_SERVER | > + IODA2_LXIVT_PRIORITY | > + IODA2_LXIVT_NODE_ID); > + server = GETFIELD(IODA2_LXIVT_SERVER, val); > + prio = GETFIELD(IODA2_LXIVT_PRIORITY, val); > + > + /* > + * The low order 2 bits are the link pointer (Type II interrupts). > + * Shift back to get a valid IRQ server. > + */ > + server >>= 2; > + > + ics_simple_write_xive(ics, idx, server, prio, prio); > +} > + > +static uint64_t *pnv_phb3_ioda_access(PnvPHB3 *phb, > + unsigned *out_table, unsigned *out_idx) > +{ > + uint64_t adreg = phb->regs[PHB_IODA_ADDR >> 3]; > + unsigned int index = GETFIELD(PHB_IODA_AD_TADR, adreg); > + unsigned int table = GETFIELD(PHB_IODA_AD_TSEL, adreg); > + unsigned int mask; > + uint64_t *tptr = NULL; > + > + switch (table) { > + case IODA2_TBL_LIST: > + tptr = phb->ioda_LIST; > + mask = 7; Looks like these masks would be clearer in hex. > + break; > + case IODA2_TBL_LXIVT: > + tptr = phb->ioda_LXIVT; > + mask = 7; > + break; > + case IODA2_TBL_IVC_CAM: > + case IODA2_TBL_RBA: > + mask = 31; > + break; > + case IODA2_TBL_RCAM: > + mask = 63; > + break; > + case IODA2_TBL_MRT: > + mask = 7; > + break; > + case IODA2_TBL_PESTA: > + case IODA2_TBL_PESTB: > + mask = 255; > + break; > + case IODA2_TBL_TVT: > + tptr = phb->ioda_TVT; > + mask = 511; > + break; > + case IODA2_TBL_TCAM: > + case IODA2_TBL_TDR: > + mask = 63; > + break; > + case IODA2_TBL_M64BT: > + tptr = phb->ioda_M64BT; > + mask = 15; > + break; > + case IODA2_TBL_M32DT: > + tptr = phb->ioda_MDT; > + mask = 255; > + break; > + case IODA2_TBL_PEEV: > + tptr = phb->ioda_PEEV; > + mask = 3; > + break; > + default: > + phb3_error("invalid IODA table %d", table); > + return NULL; > + } > + index &= mask; > + if (out_idx) { > + *out_idx = index; > + } > + if (out_table) { > + *out_table = table; > + } > + if (adreg & PHB_IODA_AD_AUTOINC) { > + index = (index + 1) & mask; > + adreg = SETFIELD(PHB_IODA_AD_TADR, adreg, index); > + } > + if (tptr) { > + tptr += index; > + } > + phb->regs[PHB_IODA_ADDR >> 3] = adreg; > + return tptr; > +} > + > +static uint64_t pnv_phb3_ioda_read(PnvPHB3 *phb) > +{ > + unsigned table; > + uint64_t *tptr; > + > + tptr = pnv_phb3_ioda_access(phb, &table, NULL); > + if (!tptr) { > + /* Return 0 on unsupported tables, not ff's */ > + return 0; > + } > + return *tptr; > +} > + > +static void pnv_phb3_ioda_write(PnvPHB3 *phb, uint64_t val) > +{ > + unsigned table, idx; > + uint64_t *tptr; > + > + tptr = pnv_phb3_ioda_access(phb, &table, &idx); > + if (!tptr) { > + return; > + } > + > + /* Handle side effects */ > + switch (table) { > + case IODA2_TBL_LXIVT: > + pnv_phb3_lxivt_write(phb, idx, val); > + break; > + case IODA2_TBL_M64BT: > + *tptr = val; > + pnv_phb3_check_m64(phb, idx); > + break; > + default: > + *tptr = val; > + } > +} > + > +/* This is called whenever the PHB LSI, MSI source ID register or > + * the PBCQ irq filters are written. > + */ > +void pnv_phb3_remap_irqs(PnvPHB3 *phb) > +{ > + ICSState *ics = &phb->lsis; > + uint32_t local, global, count, mask, comp; > + uint64_t baren; > + PnvPBCQState *pbcq = &phb->pbcq; > + > + /* First check if we are enabled. Unlike real HW we don't separate TX and RX > + * so we enable if both are set > + */ > + baren = pbcq->nest_regs[PBCQ_NEST_BAR_EN]; > + if (!(baren & PBCQ_NEST_BAR_EN_IRSN_RX) || > + !(baren & PBCQ_NEST_BAR_EN_IRSN_TX)) { > + ics->offset = 0; > + return; > + } > + > + /* Grab local LSI source ID */ > + local = GETFIELD(PHB_LSI_SRC_ID, phb->regs[PHB_LSI_SOURCE_ID >> 3]) << 3; > + > + /* Grab global one and compare */ > + global = GETFIELD(PBCQ_NEST_LSI_SRC, > + pbcq->nest_regs[PBCQ_NEST_LSI_SRC_ID]) << 3; > + if (global != local) { > + /* This happens during initialization, let's come back when we > + * are properly configured > + */ > + ics->offset = 0; > + return; > + } > + > + /* Get the base on the powerbus */ > + comp = GETFIELD(PBCQ_NEST_IRSN_COMP, > + pbcq->nest_regs[PBCQ_NEST_IRSN_COMPARE]); > + mask = GETFIELD(PBCQ_NEST_IRSN_COMP, > + pbcq->nest_regs[PBCQ_NEST_IRSN_MASK]); > + count = ((~mask) + 1) & 0x7ffff; > + phb->total_irq = count; > + > + /* Sanity checks */ > + if ((global + PNV_PHB3_NUM_LSI) > count) { > + phb3_error("LSIs out of reach: LSI base=%d total irq=%d", global, > + count); > + } > + > + if (count > 2048) { > + phb3_error("More interrupts than supported: %d", count); > + } > + > + if ((comp & mask) != comp) { > + phb3_error("IRQ compare bits not in mask: comp=0x%x mask=0x%x", > + comp, mask); > + comp &= mask; > + } > + /* Setup LSI offset */ > + ics->offset = comp + global; > + > + /* Setup MSI offset */ > + pnv_phb3_msi_update_config(&phb->msis, comp, count - PNV_PHB3_NUM_LSI); > +} > + > +static void pnv_phb3_lsi_src_id_write(PnvPHB3 *phb, uint64_t val) > +{ > + /* Sanitize content */ > + val &= PHB_LSI_SRC_ID; > + phb->regs[PHB_LSI_SOURCE_ID >> 3] = val; > + pnv_phb3_remap_irqs(phb); > +} > + > +static void pnv_phb3_rtc_invalidate(PnvPHB3 *phb, uint64_t val) > +{ > + PnvPhb3DMASpace *ds; > + > + /* Always invalidate all for now ... */ > + QLIST_FOREACH(ds, &phb->dma_spaces, list) { > + ds->pe_num = PHB_INVALID_PE; > + } > +} > + > + > +static void pnv_phb3_update_msi_regions(PnvPhb3DMASpace *ds) > +{ > + uint64_t cfg = ds->phb->regs[PHB_PHB3_CONFIG >> 3]; > + > + if (cfg & PHB_PHB3C_32BIT_MSI_EN) { > + if (!ds->msi32_mapped) { > + memory_region_add_subregion(MEMORY_REGION(&ds->dma_mr), > + 0xffff0000, &ds->msi32_mr); > + ds->msi32_mapped = true; > + } > + } else { > + if (ds->msi32_mapped) { > + memory_region_del_subregion(MEMORY_REGION(&ds->dma_mr), > + &ds->msi32_mr); > + ds->msi32_mapped = false; > + } > + } > + > + if (cfg & PHB_PHB3C_64BIT_MSI_EN) { > + if (!ds->msi64_mapped) { > + memory_region_add_subregion(MEMORY_REGION(&ds->dma_mr), > + (1ull << 60), &ds->msi64_mr); > + ds->msi64_mapped = true; > + } > + } else { > + if (ds->msi64_mapped) { > + memory_region_del_subregion(MEMORY_REGION(&ds->dma_mr), > + &ds->msi64_mr); > + ds->msi64_mapped = false; > + } > + } > +} > + > +static void pnv_phb3_update_all_msi_regions(PnvPHB3 *phb) > +{ > + PnvPhb3DMASpace *ds; > + > + QLIST_FOREACH(ds, &phb->dma_spaces, list) { > + pnv_phb3_update_msi_regions(ds); > + } > +} > + > +void pnv_phb3_reg_write(void *opaque, hwaddr off, uint64_t val, unsigned size) > +{ > + PnvPHB3 *phb = opaque; > + bool changed; > + > + /* Special case configuration data */ > + if ((off & 0xfffc) == PHB_CONFIG_DATA) { > + pnv_phb3_config_write(phb, off & 0x3, size, val); > + return; > + } > + > + /* Other registers are 64-bit only */ > + if (size != 8 || off & 0x7) { > + phb3_error("Invalid register access, offset: 0x%"PRIx64" size: %d", > + off, size); > + return; > + } > + > + /* Handle masking */ > + switch (off) { > + case PHB_M64_UPPER_BITS: > + val &= 0xfffc000000000000ull; > + break; > + } > + > + /* Record whether it changed */ > + changed = phb->regs[off >> 3] != val; > + > + /* Store in register cache first */ > + phb->regs[off >> 3] = val; > + > + /* Handle side effects */ > + switch (off) { > + case PHB_PHB3_CONFIG: > + if (changed) { > + pnv_phb3_update_all_msi_regions(phb); > + } > + /* fall through */ > + case PHB_M32_BASE_ADDR: > + case PHB_M32_BASE_MASK: > + case PHB_M32_START_ADDR: > + if (changed) { > + pnv_phb3_check_m32(phb); > + } > + break; > + case PHB_M64_UPPER_BITS: > + if (changed) { > + pnv_phb3_check_all_m64s(phb); > + } > + break; > + case PHB_LSI_SOURCE_ID: > + if (changed) { > + pnv_phb3_lsi_src_id_write(phb, val); > + } > + break; > + > + /* IODA table accesses */ > + case PHB_IODA_DATA0: > + pnv_phb3_ioda_write(phb, val); > + break; > + > + /* RTC invalidation */ > + case PHB_RTC_INVALIDATE: > + pnv_phb3_rtc_invalidate(phb, val); > + break; > + > + /* FFI request */ > + case PHB_FFI_REQUEST: > + pnv_phb3_msi_ffi(&phb->msis, val); > + break; > + > + /* Silent simple writes */ > + case PHB_CONFIG_ADDRESS: > + case PHB_IODA_ADDR: > + case PHB_TCE_KILL: > + case PHB_TCE_SPEC_CTL: > + case PHB_PEST_BAR: > + case PHB_PELTV_BAR: > + case PHB_RTT_BAR: > + case PHB_RBA_BAR: > + case PHB_IVT_BAR: > + case PHB_FFI_LOCK: > + break; > + > + /* Noise on anything else */ > + default: > + qemu_log_mask(LOG_UNIMP, "phb3: reg_write 0x%"PRIx64"=%"PRIx64"\n", > + off, val); > + } > +} > + > +uint64_t pnv_phb3_reg_read(void *opaque, hwaddr off, unsigned size) > +{ > + PnvPHB3 *phb = opaque; > + uint64_t val; > + > + if ((off & 0xfffc) == PHB_CONFIG_DATA) { > + return pnv_phb3_config_read(phb, off & 0x3, size); > + } > + > + /* Other registers are 64-bit only */ > + if (size != 8 || off & 0x7) { > + phb3_error("Invalid register access, offset: 0x%"PRIx64" size: %d", > + off, size); > + return ~0ull; > + } > + > + /* Default read from cache */ > + val = phb->regs[off >> 3]; > + > + switch (off) { > + /* Simulate venice DD2.0 */ > + case PHB_VERSION: > + return 0x000000a300000005ull; > + case PHB_PCIE_SYSTEM_CONFIG: > + return 0x441100fc30000000; > + > + /* IODA table accesses */ > + case PHB_IODA_DATA0: > + return pnv_phb3_ioda_read(phb); > + > + /* Link training always appears trained */ > + case PHB_PCIE_DLP_TRAIN_CTL: > + return PHB_PCIE_DLP_INBAND_PRESENCE | PHB_PCIE_DLP_TC_DL_LINKACT; > + > + /* FFI Lock */ > + case PHB_FFI_LOCK: > + /* Set lock and return previous value */ > + phb->regs[off >> 3] |= PHB_FFI_LOCK_STATE; > + return val; > + > + /* Silent simple reads */ > + case PHB_PHB3_CONFIG: > + case PHB_M32_BASE_ADDR: > + case PHB_M32_BASE_MASK: > + case PHB_M32_START_ADDR: > + case PHB_CONFIG_ADDRESS: > + case PHB_IODA_ADDR: > + case PHB_RTC_INVALIDATE: > + case PHB_TCE_KILL: > + case PHB_TCE_SPEC_CTL: > + case PHB_PEST_BAR: > + case PHB_PELTV_BAR: > + case PHB_RTT_BAR: > + case PHB_RBA_BAR: > + case PHB_IVT_BAR: > + case PHB_M64_UPPER_BITS: > + break; > + > + /* Noise on anything else */ > + default: > + qemu_log_mask(LOG_UNIMP, "phb3: reg_read 0x%"PRIx64"=%"PRIx64"\n", > + off, val); > + } > + return val; > +} > + > +static const MemoryRegionOps pnv_phb3_reg_ops = { > + .read = pnv_phb3_reg_read, > + .write = pnv_phb3_reg_write, > + .valid.min_access_size = 1, > + .valid.max_access_size = 8, > + .impl.min_access_size = 1, > + .impl.max_access_size = 8, > + .endianness = DEVICE_BIG_ENDIAN, > +}; > + > +static int pnv_phb3_map_irq(PCIDevice *pci_dev, int irq_num) > +{ > + /* Check that out properly ... */ > + return irq_num & 3; > +} > + > +static void pnv_phb3_set_irq(void *opaque, int irq_num, int level) > +{ > + PnvPHB3 *phb = opaque; > + > + /* LSI only ... */ > + if (irq_num > 3) { > + phb3_error("Unknown IRQ to set %d", irq_num); > + } > + qemu_set_irq(phb->lsis.qirqs[irq_num], level); > +} > + > +static bool pnv_phb3_resolve_pe(PnvPhb3DMASpace *ds) > +{ > + uint64_t rtt, addr; > + uint16_t rte; > + int bus_num; > + > + /* Already resolved ? */ > + if (ds->pe_num != PHB_INVALID_PE) { > + return true; > + } > + > + /* We need to lookup the RTT */ > + rtt = ds->phb->regs[PHB_RTT_BAR >> 3]; > + if (!(rtt & PHB_RBA_BAR_ENABLE)) { > + phb3_error("DMA with RTT BAR disabled !"); > + /* Set error bits ? fence ? ... */ > + return false; > + } > + > + /* Read RTE */ > + bus_num = pci_bus_num(ds->bus); > + addr = rtt & PHB_RTT_BASE_ADDRESS_MASK; > + addr += 2 * ((bus_num << 8) | ds->devfn); > + if (dma_memory_read(&address_space_memory, addr, &rte, sizeof(rte))) { > + phb3_error("Failed to read RTT entry at 0x%"PRIx64, addr); > + /* Set error bits ? fence ? ... */ > + return false; > + } > + rte = be16_to_cpu(rte); > + > + /* Fail upon reading of invalid PE# */ > + if (rte >= PNV_PHB3_NUM_PE) { > + phb3_error("RTE for RID 0x%x invalid (%04x", ds->devfn, rte); > + /* Set error bits ? fence ? ... */ > + return false; > + } > + ds->pe_num = rte; > + return true; > +} > + > +static void pnv_phb3_translate_tve(PnvPhb3DMASpace *ds, hwaddr addr, > + bool is_write, uint64_t tve, > + IOMMUTLBEntry *tlb) > +{ > + uint64_t tta = GETFIELD(IODA2_TVT_TABLE_ADDR, tve); > + int32_t lev = GETFIELD(IODA2_TVT_NUM_LEVELS, tve); > + uint32_t tts = GETFIELD(IODA2_TVT_TCE_TABLE_SIZE, tve); > + uint32_t tps = GETFIELD(IODA2_TVT_IO_PSIZE, tve); > + > + /* Invalid levels */ > + if (lev > 4) { > + phb3_error("Invalid #levels in TVE %d", lev); > + return; > + } > + > + /* IO Page Size of 0 means untranslated, else use TCEs */ > + if (tps == 0) { > + /* We only support non-translate in top window > + * XXX FIX THAT, Venice/Murano support it on bottom window > + * above 4G and Naples suports it on everything > + */ > + if (!(tve & PPC_BIT(51))) { > + phb3_error("xlate for invalid non-translate TVE"); > + return; > + } > + /* XXX Handle boundaries */ > + > + /* XXX Use 4k pages like q35 ... for now */ > + tlb->iova = addr & 0xfffffffffffff000ull; > + tlb->translated_addr = addr & 0x0003fffffffff000ull; > + tlb->addr_mask = 0xfffull; > + tlb->perm = IOMMU_RW; > + } else { > + uint32_t tce_shift, tbl_shift, sh; > + uint64_t base, taddr, tce, tce_mask; > + > + /* TVE disabled ? */ > + if (tts == 0) { > + phb3_error("xlate for invalid translated TVE"); > + return; > + } > + > + /* Address bits per bottom level TCE entry */ > + tce_shift = tps + 11; > + > + /* Address bits per table level */ > + tbl_shift = tts + 8; > + > + /* Top level table base address */ > + base = tta << 12; > + > + /* Total shift to first level */ > + sh = tbl_shift * lev + tce_shift; > + > + /* XXX Multi-level untested */ > + while ((lev--) >= 0) { > + /* Grab the TCE address */ > + taddr = base | (((addr >> sh) & ((1ul << tbl_shift) - 1)) << 3); > + if (dma_memory_read(&address_space_memory, taddr, &tce, > + sizeof(tce))) { > + phb3_error("Failed to read TCE at 0x%"PRIx64, taddr); > + return; > + } > + tce = be64_to_cpu(tce); > + > + /* Check permission for indirect TCE */ > + if ((lev >= 0) && !(tce & 3)) { > + phb3_error("Invalid indirect TCE at 0x%"PRIx64, taddr); > + phb3_error(" xlate %"PRIx64":%c TVE=%"PRIx64, addr, > + is_write ? 'W' : 'R', tve); > + phb3_error(" tta=%"PRIx64" lev=%d tts=%d tps=%d", > + tta, lev, tts, tps); > + return; > + } > + sh -= tbl_shift; > + base = tce & ~0xfffull; > + } > + > + /* We exit the loop with TCE being the final TCE */ > + tce_mask = ~((1ull << tce_shift) - 1); > + tlb->iova = addr & tce_mask; > + tlb->translated_addr = tce & tce_mask; > + tlb->addr_mask = ~tce_mask; > + tlb->perm = tce & 3; > + if ((is_write & !(tce & 2)) || ((!is_write) && !(tce & 1))) { > + phb3_error("TCE access fault at 0x%"PRIx64, taddr); > + phb3_error(" xlate %"PRIx64":%c TVE=%"PRIx64, addr, > + is_write ? 'W' : 'R', tve); > + phb3_error(" tta=%"PRIx64" lev=%d tts=%d tps=%d", > + tta, lev, tts, tps); > + } > + } > +} > + > +static IOMMUTLBEntry pnv_phb3_translate_iommu(IOMMUMemoryRegion *iommu, > + hwaddr addr, > + IOMMUAccessFlags flag, > + int iommu_idx) > +{ > + PnvPhb3DMASpace *ds = container_of(iommu, PnvPhb3DMASpace, dma_mr); > + int tve_sel; > + uint64_t tve, cfg; > + IOMMUTLBEntry ret = { > + .target_as = &address_space_memory, > + .iova = addr, > + .translated_addr = 0, > + .addr_mask = ~(hwaddr)0, > + .perm = IOMMU_NONE, > + }; > + > + /* Resolve PE# */ > + if (!pnv_phb3_resolve_pe(ds)) { > + phb3_error("Failed to resolve PE# for bus @%p (%d) devfn 0x%x", > + ds->bus, pci_bus_num(ds->bus), ds->devfn); > + return ret; > + } > + > + /* Check top bits */ > + switch (addr >> 60) { > + case 00: > + /* DMA or 32-bit MSI ? */ > + cfg = ds->phb->regs[PHB_PHB3_CONFIG >> 3]; > + if ((cfg & PHB_PHB3C_32BIT_MSI_EN) && > + ((addr & 0xffffffffffff0000ull) == 0xffff0000ull)) { > + phb3_error("xlate on 32-bit MSI region"); > + return ret; > + } > + /* Choose TVE XXX Use PHB3 Control Register */ > + tve_sel = (addr >> 59) & 1; > + tve = ds->phb->ioda_TVT[ds->pe_num * 2 + tve_sel]; > + pnv_phb3_translate_tve(ds, addr, flag & IOMMU_WO, tve, &ret); > + break; > + case 01: > + phb3_error("xlate on 64-bit MSI region"); > + break; > + default: > + phb3_error("xlate on unsupported address 0x%"PRIx64, addr); > + } > + return ret; > +} > + > +#define TYPE_PNV_PHB3_IOMMU_MEMORY_REGION "pnv-phb3-iommu-memory-region" > +#define PNV_PHB3_IOMMU_MEMORY_REGION(obj) \ > + OBJECT_CHECK(IOMMUMemoryRegion, (obj), TYPE_PNV_PHB3_IOMMU_MEMORY_REGION) > + > +static void pnv_phb3_iommu_memory_region_class_init(ObjectClass *klass, > + void *data) > +{ > + IOMMUMemoryRegionClass *imrc = IOMMU_MEMORY_REGION_CLASS(klass); > + > + imrc->translate = pnv_phb3_translate_iommu; > +} > + > +static const TypeInfo pnv_phb3_iommu_memory_region_info = { > + .parent = TYPE_IOMMU_MEMORY_REGION, > + .name = TYPE_PNV_PHB3_IOMMU_MEMORY_REGION, > + .class_init = pnv_phb3_iommu_memory_region_class_init, > +}; > + > +/* > + * MSI/MSIX memory region implementation. > + * The handler handles both MSI and MSIX. > + */ > +static void pnv_phb3_msi_write(void *opaque, hwaddr addr, > + uint64_t data, unsigned size) > +{ > + PnvPhb3DMASpace *ds = opaque; > + > + /* Resolve PE# */ > + if (!pnv_phb3_resolve_pe(ds)) { > + phb3_error("Failed to resolve PE# for bus @%p (%d) devfn 0x%x", > + ds->bus, pci_bus_num(ds->bus), ds->devfn); > + return; > + } > + > + pnv_phb3_msi_send(&ds->phb->msis, addr, data, ds->pe_num); > +} > + > +/* There is no .read as the read result is undefined by PCI spec */ > +static uint64_t pnv_phb3_msi_read(void *opaque, hwaddr addr, unsigned size) > +{ > + phb3_error("invalid read @ 0x%" HWADDR_PRIx, addr); > + return -1; > +} > + > +static const MemoryRegionOps pnv_phb3_msi_ops = { > + .read = pnv_phb3_msi_read, > + .write = pnv_phb3_msi_write, > + .endianness = DEVICE_LITTLE_ENDIAN > +}; > + > +static AddressSpace *pnv_phb3_dma_iommu(PCIBus *bus, void *opaque, int devfn) > +{ > + PnvPHB3 *phb = opaque; > + PnvPhb3DMASpace *ds; > + > + QLIST_FOREACH(ds, &phb->dma_spaces, list) { > + if (ds->bus == bus && ds->devfn == devfn) { > + break; > + } > + } > + > + if (ds == NULL) { > + ds = g_malloc0(sizeof(PnvPhb3DMASpace)); > + ds->bus = bus; > + ds->devfn = devfn; > + ds->pe_num = PHB_INVALID_PE; > + ds->phb = phb; > + memory_region_init_iommu(&ds->dma_mr, sizeof(ds->dma_mr), > + TYPE_PNV_PHB3_IOMMU_MEMORY_REGION, > + OBJECT(phb), "phb3_iommu", UINT64_MAX); > + address_space_init(&ds->dma_as, MEMORY_REGION(&ds->dma_mr), > + "phb3_iommu"); > + memory_region_init_io(&ds->msi32_mr, OBJECT(phb), &pnv_phb3_msi_ops, > + ds, "msi32", 0x10000); > + memory_region_init_io(&ds->msi64_mr, OBJECT(phb), &pnv_phb3_msi_ops, > + ds, "msi64", 0x100000); > + pnv_phb3_update_msi_regions(ds); > + > + QLIST_INSERT_HEAD(&phb->dma_spaces, ds, list); > + } > + return &ds->dma_as; > +} > + > +static void pnv_phb3_instance_init(Object *obj) > +{ > + PnvPHB3 *phb = PNV_PHB3(obj); > + > + QLIST_INIT(&phb->dma_spaces); > + > + /* LSI sources */ > + object_initialize(&phb->lsis, sizeof(phb->lsis), TYPE_ICS_SIMPLE); > + object_property_add_child(obj, "lsi", OBJECT(&phb->lsis), NULL); > + > + /* Default init ... will be fixed by HW inits */ > + phb->lsis.offset = 0; > + > + /* MSI sources */ > + object_initialize(&phb->msis, sizeof(phb->msis), TYPE_PHB3_MSI); > + object_property_add_const_link(OBJECT(&phb->msis), "phb", obj, > + &error_abort); > + object_property_add_child(obj, "msi", OBJECT(&phb->msis), NULL); > + > + /* Power Bus Common Queue */ > + object_initialize(&phb->pbcq, sizeof(phb->pbcq), TYPE_PNV_PBCQ); > + object_property_add_const_link(OBJECT(&phb->pbcq), "phb", obj, > + &error_abort); > + object_property_add_child(obj, "pbcq", OBJECT(&phb->pbcq), NULL); > + > + /* Root Port */ > + object_initialize(&phb->root, sizeof(phb->root), TYPE_PNV_PHB3_ROOT_PORT); > + object_property_add_child(obj, "root", OBJECT(&phb->root), NULL); > + qdev_prop_set_int32(DEVICE(&phb->root), "addr", PCI_DEVFN(0, 0)); > + qdev_prop_set_bit(DEVICE(&phb->root), "multifunction", false); > +} > + > +static void pnv_phb3_realize(DeviceState *dev, Error **errp) > +{ > + PnvPHB3 *phb = PNV_PHB3(dev); > + PCIHostState *pci = PCI_HOST_BRIDGE(dev); > + PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine()); > + Error *local_err = NULL; > + int i; > + > + if (!pnv_get_chip(pnv, phb->chip_id)) { > + error_setg(errp, "invalid chip id: %d", phb->chip_id); > + return; > + } If we go with a PBCQ-first model, I think we can do this matchup in a more QOM-ishly correct way by searching for an appropriate PBCQ instance, rather than directly looking up chip number. At least until/unless I figure out how to do it with a pseudo-bus. > + > + if (phb->phb_id >= PNV8_CHIP_PHB3_MAX) { > + error_setg(errp, "invalid PHB index: %d", phb->phb_id); > + return; > + } > + > + /* LSI sources */ > + object_property_add_const_link(OBJECT(&phb->lsis), "xics", OBJECT(pnv), > + &error_abort); > + object_property_set_int(OBJECT(&phb->lsis), PNV_PHB3_NUM_LSI, "nr-irqs", > + &error_abort); > + object_property_set_bool(OBJECT(&phb->lsis), true, "realized", &local_err); > + if (local_err) { > + error_propagate(errp, local_err); > + return; > + } > + > + for (i = 0; i < phb->lsis.nr_irqs; i++) { > + ics_set_irq_type(&phb->lsis, i, true); > + } > + > + /* MSI sources */ > + object_property_add_const_link(OBJECT(&phb->msis), "xics", OBJECT(pnv), > + &error_abort); > + object_property_set_int(OBJECT(&phb->msis), PHB3_MAX_MSI, "nr-irqs", > + &error_abort); > + object_property_set_bool(OBJECT(&phb->msis), true, "realized", &local_err); > + if (local_err) { > + error_propagate(errp, local_err); > + return; > + } > + > + /* Power Bus Common Queue */ > + object_property_set_bool(OBJECT(&phb->pbcq), true, "realized", &local_err); > + if (local_err) { > + error_propagate(errp, local_err); > + return; > + } > + > + /* Controller Registers */ > + memory_region_init_io(&phb->mr_regs, OBJECT(phb), &pnv_phb3_reg_ops, phb, > + "phb3-regs", 0x1000); > + > + /* PHB3 doesn't support IO space. However, qemu gets very upset if > + * we don't have an IO region to anchor IO BARs onto so we just > + * initialize one which we never hook up to anything > + */ > + memory_region_init(&phb->pci_io, OBJECT(phb), "pci-io", 0x10000); > + memory_region_init(&phb->pci_mmio, OBJECT(phb), "pci-mmio", > + PCI_MMIO_TOTAL_SIZE); > + > + pci->bus = pci_register_root_bus(dev, "root-bus", > + pnv_phb3_set_irq, pnv_phb3_map_irq, phb, > + &phb->pci_mmio, &phb->pci_io, > + 0, 4, TYPE_PNV_PHB3_ROOT_BUS); > + > + pci_setup_iommu(pci->bus, pnv_phb3_dma_iommu, phb); > + > + /* Add a single Root port */ > + qdev_prop_set_uint8(DEVICE(&phb->root), "chassis", phb->chip_id); > + qdev_prop_set_uint16(DEVICE(&phb->root), "slot", phb->phb_id); > + qdev_set_parent_bus(DEVICE(&phb->root), BUS(pci->bus)); > + qdev_init_nofail(DEVICE(&phb->root)); > +} > + > +void pnv_phb3_update_regions(PnvPHB3 *phb) > +{ > + PnvPBCQState *pbcq = &phb->pbcq; > + > + /* Unmap first always */ > + if (phb->regs_mapped) { > + memory_region_del_subregion(&pbcq->phbbar, &phb->mr_regs); > + phb->regs_mapped = false; > + } > + > + /* Map registers if enabled */ > + if (pbcq->phb_mapped) { > + /* XXX We should use the PHB BAR 2 register but we don't ... */ > + memory_region_add_subregion(&pbcq->phbbar, 0, &phb->mr_regs); > + phb->regs_mapped = true; > + } > + > + /* Check/update m32 */ > + if (phb->m32_mapped) { > + pnv_phb3_check_m32(phb); > + } > +} > + > +static const char *pnv_phb3_root_bus_path(PCIHostState *host_bridge, > + PCIBus *rootbus) > +{ > + PnvPHB3 *phb = PNV_PHB3(host_bridge); > + > + snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x", > + phb->chip_id, phb->phb_id); > + return phb->bus_path; > +} > + > +static Property pnv_phb3_properties[] = { > + DEFINE_PROP_UINT32("index", PnvPHB3, phb_id, 0), > + DEFINE_PROP_UINT32("chip-id", PnvPHB3, chip_id, 0), > + DEFINE_PROP_END_OF_LIST(), > +}; > + > +static void pnv_phb3_class_init(ObjectClass *klass, void *data) > +{ > + PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass); > + DeviceClass *dc = DEVICE_CLASS(klass); > + > + hc->root_bus_path = pnv_phb3_root_bus_path; > + dc->realize = pnv_phb3_realize; > + dc->props = pnv_phb3_properties; > + set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories); > +} > + > +static const TypeInfo pnv_phb3_type_info = { > + .name = TYPE_PNV_PHB3, > + .parent = TYPE_PCIE_HOST_BRIDGE, > + .instance_size = sizeof(PnvPHB3), > + .class_init = pnv_phb3_class_init, > + .instance_init = pnv_phb3_instance_init, > +}; > + > +static void pnv_phb3_root_bus_class_init(ObjectClass *klass, void *data) > +{ > + BusClass *k = BUS_CLASS(klass); > + > + /* > + * PHB3 has only a single root complex. Enforce the limit on the > + * parent bus > + */ > + k->max_dev = 1; > +} > + > +static const TypeInfo pnv_phb3_root_bus_info = { > + .name = TYPE_PNV_PHB3_ROOT_BUS, > + .parent = TYPE_PCI_BUS, > + .class_init = pnv_phb3_root_bus_class_init, > + .interfaces = (InterfaceInfo[]) { > + { INTERFACE_PCIE_DEVICE }, > + { } > + }, > +}; > + > +static void pnv_phb3_root_port_realize(DeviceState *dev, Error **errp) > +{ > + PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev); > + Error *local_err = NULL; > + > + rpc->parent_realize(dev, &local_err); > + if (local_err) { > + error_propagate(errp, local_err); > + return; > + } > +} > + > +static void pnv_phb3_root_port_class_init(ObjectClass *klass, void *data) > +{ > + DeviceClass *dc = DEVICE_CLASS(klass); > + PCIDeviceClass *k = PCI_DEVICE_CLASS(klass); > + PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass); > + > + dc->desc = "IBM PHB3 PCIE Root Port"; > + > + device_class_set_parent_realize(dc, pnv_phb3_root_port_realize, > + &rpc->parent_realize); > + > + k->vendor_id = PCI_VENDOR_ID_IBM; > + k->device_id = 0x03dc; > + k->revision = 0; > + > + rpc->exp_offset = 0x48; > + rpc->aer_offset = 0x100; > +} > + > +static const TypeInfo pnv_phb3_root_port_info = { > + .name = TYPE_PNV_PHB3_ROOT_PORT, > + .parent = TYPE_PCIE_ROOT_PORT, > + .instance_size = sizeof(PnvPHB3RootPort), > + .class_init = pnv_phb3_root_port_class_init, > +}; > + > +static void pnv_phb3_register_types(void) > +{ > + type_register_static(&pnv_phb3_root_bus_info); > + type_register_static(&pnv_phb3_root_port_info); > + type_register_static(&pnv_phb3_type_info); > + type_register_static(&pnv_phb3_iommu_memory_region_info); > +} > + > +type_init(pnv_phb3_register_types) > diff --git a/hw/pci-host/pnv_phb3_msi.c b/hw/pci-host/pnv_phb3_msi.c > new file mode 100644 > index 000000000000..588865a86de1 > --- /dev/null > +++ b/hw/pci-host/pnv_phb3_msi.c > @@ -0,0 +1,318 @@ > +/* > + * QEMU PowerPC PowerNV PHB3 model > + * > + * Copyright (c) 2014-2018, IBM Corporation. > + * > + * This code is licensed under the GPL version 2 or later. See the > + * COPYING file in the top-level directory. > + */ > +#include "qemu/osdep.h" > +#include "qemu/log.h" > +#include "qapi/error.h" > +#include "qemu-common.h" > +#include "target/ppc/cpu.h" > +#include "hw/pci-host/pnv_phb3_regs.h" > +#include "hw/pci-host/pnv_phb3.h" > +#include "hw/ppc/pnv.h" > +#include "hw/pci/msi.h" > +#include "monitor/monitor.h" > + > +static uint64_t phb3_msi_ive_addr(PnvPHB3 *phb, int srcno) > +{ > + uint64_t ivtbar = phb->regs[PHB_IVT_BAR >> 3]; > + uint64_t phbctl = phb->regs[PHB_CONTROL >> 3]; > + > + if (!(ivtbar & PHB_IVT_BAR_ENABLE)) { > + qemu_log_mask(LOG_GUEST_ERROR, "Failed access to disable IVT BAR !"); > + return 0; > + } > + > + if (srcno >= (ivtbar & PHB_IVT_LENGTH_MASK)) { > + qemu_log_mask(LOG_GUEST_ERROR, "MSI out of bounds (%d vs 0x%"PRIx64")", > + srcno, ivtbar & PHB_IVT_LENGTH_MASK); > + return 0; > + } > + > + ivtbar &= PHB_IVT_BASE_ADDRESS_MASK; > + > + if (phbctl & PHB_CTRL_IVE_128_BYTES) { > + return ivtbar + 128 * srcno; > + } else { > + return ivtbar + 16 * srcno; > + } > +} > + > +static bool phb3_msi_read_ive(PnvPHB3 *phb, int srcno, uint64_t *out_ive) > +{ > + uint64_t ive_addr, ive; > + > + ive_addr = phb3_msi_ive_addr(phb, srcno); > + if (!ive_addr) { > + return false; > + } > + > + if (dma_memory_read(&address_space_memory, ive_addr, &ive, sizeof(ive))) { > + qemu_log_mask(LOG_GUEST_ERROR, "Failed to read IVE at 0x%" PRIx64, > + ive_addr); > + return false; > + } > + *out_ive = be64_to_cpu(ive); > + > + return true; > +} > + > +static void phb3_msi_set_p(Phb3MsiState *msi, int srcno, uint8_t gen) > +{ > + uint64_t ive_addr; > + uint8_t p = 0x01 | (gen << 1); > + > + ive_addr = phb3_msi_ive_addr(msi->phb, srcno); > + if (!ive_addr) { > + return; > + } > + > + if (dma_memory_write(&address_space_memory, ive_addr + 4, &p, 1)) { > + qemu_log_mask(LOG_GUEST_ERROR, > + "Failed to write IVE (set P) at 0x%" PRIx64, ive_addr); > + } > +} > + > +static void phb3_msi_set_q(Phb3MsiState *msi, int srcno) > +{ > + uint64_t ive_addr; > + uint8_t q = 0x01; > + > + ive_addr = phb3_msi_ive_addr(msi->phb, srcno); > + if (!ive_addr) { > + return; > + } > + > + if (dma_memory_write(&address_space_memory, ive_addr + 5, &q, 1)) { > + qemu_log_mask(LOG_GUEST_ERROR, > + "Failed to write IVE (set Q) at 0x%" PRIx64, ive_addr); > + } > +} > + > +static void phb3_msi_try_send(Phb3MsiState *msi, int srcno, bool force) > +{ > + ICSState *ics = ICS_BASE(msi); > + uint64_t ive; > + uint64_t server, prio, pq, gen; > + > + if (!phb3_msi_read_ive(msi->phb, srcno, &ive)) { > + return; > + } > + > + server = GETFIELD(IODA2_IVT_SERVER, ive); > + prio = GETFIELD(IODA2_IVT_PRIORITY, ive); > + if (!force) { > + pq = GETFIELD(IODA2_IVT_Q, ive) | (GETFIELD(IODA2_IVT_P, ive) << 1); > + } else { > + pq = 0; > + } > + gen = GETFIELD(IODA2_IVT_GEN, ive); > + > + /* > + * The low order 2 bits are the link pointer (Type II interrupts). > + * Shift back to get a valid IRQ server. > + */ > + server >>= 2; > + > + switch (pq) { > + case 0: /* 00 */ > + if (prio == 0xff) { > + /* Masked, set Q */ > + phb3_msi_set_q(msi, srcno); > + } else { > + /* Enabled, set P and send */ > + phb3_msi_set_p(msi, srcno, gen); > + icp_irq(ics, server, srcno + ics->offset, prio); > + } > + break; > + case 2: /* 10 */ > + /* Already pending, set Q */ > + phb3_msi_set_q(msi, srcno); > + break; > + case 1: /* 01 */ > + case 3: /* 11 */ > + default: > + /* Just drop stuff if Q already set */ > + break; > + } > +} > + > +static void phb3_msi_set_irq(void *opaque, int srcno, int val) > +{ > + Phb3MsiState *msi = PHB3_MSI(opaque); > + > + if (val) { > + phb3_msi_try_send(msi, srcno, false); > + } > +} > + > + > +void pnv_phb3_msi_send(Phb3MsiState *msi, uint64_t addr, uint16_t data, > + int32_t dev_pe) > +{ > + ICSState *ics = ICS_BASE(msi); > + uint64_t ive; > + uint16_t pe; > + uint32_t src = ((addr >> 4) & 0xffff) | (data & 0x1f); > + > + if (src >= ics->nr_irqs) { > + qemu_log_mask(LOG_GUEST_ERROR, "MSI %d out of bounds", src); > + return; > + } > + if (dev_pe >= 0) { > + if (!phb3_msi_read_ive(msi->phb, src, &ive)) { > + return; > + } > + pe = GETFIELD(IODA2_IVT_PE, ive); > + if (pe != dev_pe) { > + qemu_log_mask(LOG_GUEST_ERROR, > + "MSI %d send by PE#%d but assigned to PE#%d", > + src, dev_pe, pe); > + return; > + } > + } > + qemu_irq_pulse(ics->qirqs[src]); > +} > + > +void pnv_phb3_msi_ffi(Phb3MsiState *msi, uint64_t val) > +{ > + /* Emit interrupt */ > + pnv_phb3_msi_send(msi, val, 0, -1); > + > + /* Clear FFI lock */ > + msi->phb->regs[PHB_FFI_LOCK >> 3] = 0; > +} > + > +static void phb3_msi_reject(ICSState *ics, uint32_t nr) > +{ > + Phb3MsiState *msi = PHB3_MSI(ics); > + unsigned int srcno = nr - ics->offset; > + unsigned int idx = srcno >> 6; > + unsigned int bit = 1ull << (srcno & 0x3f); > + > + assert(srcno < PHB3_MAX_MSI); > + > + msi->rba[idx] |= bit; > + msi->rba_sum |= (1u << idx); > +} > + > +static void phb3_msi_resend(ICSState *ics) > +{ > + Phb3MsiState *msi = PHB3_MSI(ics); > + unsigned int i, j; > + > + if (msi->rba_sum == 0) { > + return; > + } > + > + for (i = 0; i < 32; i++) { > + if ((msi->rba_sum & (1u << i)) == 0) { > + continue; > + } > + msi->rba_sum &= ~(1u << i); > + for (j = 0; j < 64; j++) { > + if ((msi->rba[i] & (1ull << j)) == 0) { > + continue; > + } > + msi->rba[i] &= ~(1u << j); > + phb3_msi_try_send(msi, i * 64 + j, true); > + } > + } > +} > + > +static void phb3_msi_reset(DeviceState *dev) > +{ > + Phb3MsiState *msi = PHB3_MSI(dev); > + ICSStateClass *icsc = ICS_BASE_GET_CLASS(dev); > + > + icsc->parent_reset(dev); > + > + memset(msi->rba, 0, sizeof(msi->rba)); > + msi->rba_sum = 0; > +} > + > +static void phb3_msi_reset_handler(void *dev) > +{ > + phb3_msi_reset(dev); > +} > + > +void pnv_phb3_msi_update_config(Phb3MsiState *msi, uint32_t base, > + uint32_t count) > +{ > + ICSState *ics = ICS_BASE(msi); > + > + if (count > PHB3_MAX_MSI) { > + count = PHB3_MAX_MSI; > + } > + ics->nr_irqs = count; > + ics->offset = base; > +} > + > +static void phb3_msi_realize(DeviceState *dev, Error **errp) > +{ > + Phb3MsiState *msi = PHB3_MSI(dev); > + ICSState *ics = ICS_BASE(msi); > + ICSStateClass *icsc = ICS_BASE_GET_CLASS(ics); > + Object *obj; > + Error *local_err = NULL; > + > + icsc->parent_realize(dev, &local_err); > + if (local_err) { > + error_propagate(errp, local_err); > + return; > + } > + > + obj = object_property_get_link(OBJECT(dev), "phb", &local_err); > + if (!obj) { > + error_propagate(errp, local_err); > + error_prepend(errp, "required link 'phb' not found: "); > + return; > + } > + msi->phb = PNV_PHB3(obj); > + > + ics->qirqs = qemu_allocate_irqs(phb3_msi_set_irq, msi, ics->nr_irqs); > + > + qemu_register_reset(phb3_msi_reset_handler, dev); > +} > + > +static void phb3_msi_instance_init(Object *obj) > +{ > + ICSState *ics = ICS_BASE(obj); > + > + /* Will be overriden later */ > + ics->offset = 0; > +} > + > +static void phb3_msi_class_init(ObjectClass *klass, void *data) > +{ > + DeviceClass *dc = DEVICE_CLASS(klass); > + ICSStateClass *isc = ICS_BASE_CLASS(klass); > + > + device_class_set_parent_realize(dc, phb3_msi_realize, > + &isc->parent_realize); > + device_class_set_parent_reset(dc, phb3_msi_reset, > + &isc->parent_reset); > + > + isc->reject = phb3_msi_reject; > + isc->resend = phb3_msi_resend; > +} > + > +static const TypeInfo phb3_msi_info = { > + .name = TYPE_PHB3_MSI, > + .parent = TYPE_ICS_BASE, > + .instance_size = sizeof(Phb3MsiState), > + .class_init = phb3_msi_class_init, > + .class_size = sizeof(ICSStateClass), > + .instance_init = phb3_msi_instance_init, > +}; > + > +static void pnv_phb3_msi_register_types(void) > +{ > + type_register_static(&phb3_msi_info); > +} > + > +type_init(pnv_phb3_msi_register_types) > diff --git a/hw/pci-host/pnv_phb3_pbcq.c b/hw/pci-host/pnv_phb3_pbcq.c > new file mode 100644 > index 000000000000..34df3ac684e4 > --- /dev/null > +++ b/hw/pci-host/pnv_phb3_pbcq.c We should maybe make a subdir rather than have these multi-barrelled filenames. > @@ -0,0 +1,347 @@ > +/* > + * QEMU PowerPC PowerNV PHB3 model > + * > + * Copyright (c) 2014-2018, IBM Corporation. > + * > + * This code is licensed under the GPL version 2 or later. See the > + * COPYING file in the top-level directory. > + */ > +#include "qemu/osdep.h" > +#include "qapi/error.h" > +#include "qemu-common.h" > +#include "qemu/log.h" > +#include "target/ppc/cpu.h" > +#include "hw/ppc/fdt.h" > +#include "hw/pci-host/pnv_phb3_regs.h" > +#include "hw/pci-host/pnv_phb3.h" > +#include "hw/ppc/pnv_xscom.h" > +#include "hw/pci/pci_bridge.h" > +#include "hw/pci/pci_bus.h" > + > +#include <libfdt.h> > + > +static uint64_t pnv_pbcq_nest_xscom_read(void *opaque, hwaddr addr, > + unsigned size) > +{ > + PnvPBCQState *pbcq = PNV_PBCQ(opaque); > + uint32_t offset = addr >> 3; > + > + return pbcq->nest_regs[offset]; > +} > + > +static uint64_t pnv_pbcq_pci_xscom_read(void *opaque, hwaddr addr, > + unsigned size) > +{ > + PnvPBCQState *pbcq = PNV_PBCQ(opaque); > + uint32_t offset = addr >> 3; > + > + return pbcq->pci_regs[offset]; > +} > + > +static uint64_t pnv_pbcq_spci_xscom_read(void *opaque, hwaddr addr, > + unsigned size) > +{ > + PnvPBCQState *pbcq = PNV_PBCQ(opaque); > + uint32_t offset = addr >> 3; > + > + if (offset == PBCQ_SPCI_ASB_DATA) { > + return pnv_phb3_reg_read(pbcq->phb, > + pbcq->spci_regs[PBCQ_SPCI_ASB_ADDR], 8); > + } > + return pbcq->spci_regs[offset]; > +} > + > +static void pnv_pbcq_update_map(PnvPBCQState *pbcq) > +{ > + uint64_t bar_en = pbcq->nest_regs[PBCQ_NEST_BAR_EN]; > + uint64_t bar, mask, size; > + > + /* > + * NOTE: This will really not work well if those are remapped > + * after the PHB has created its sub regions. We could do better > + * if we had a way to resize regions Such as memory_region_set_size(), for example? > but we don't really care > + * that much in practice as the stuff below really only happens > + * once early during boot > + */ > + > + /* Handle unmaps */ > + if (pbcq->mmio0_mapped && !(bar_en & PBCQ_NEST_BAR_EN_MMIO0)) { > + memory_region_del_subregion(get_system_memory(), &pbcq->mmbar0); > + pbcq->mmio0_mapped = false; > + } > + if (pbcq->mmio1_mapped && !(bar_en & PBCQ_NEST_BAR_EN_MMIO1)) { > + memory_region_del_subregion(get_system_memory(), &pbcq->mmbar1); > + pbcq->mmio1_mapped = false; > + } > + if (pbcq->phb_mapped && !(bar_en & PBCQ_NEST_BAR_EN_PHB)) { > + memory_region_del_subregion(get_system_memory(), &pbcq->phbbar); > + pbcq->phb_mapped = false; > + } > + > + /* Update PHB */ > + pnv_phb3_update_regions(pbcq->phb); > + > + /* Handle maps */ > + if (!pbcq->mmio0_mapped && (bar_en & PBCQ_NEST_BAR_EN_MMIO0)) { > + bar = pbcq->nest_regs[PBCQ_NEST_MMIO_BAR0] >> 14; > + mask = pbcq->nest_regs[PBCQ_NEST_MMIO_MASK0]; > + size = ((~mask) >> 14) + 1; > + memory_region_init(&pbcq->mmbar0, OBJECT(pbcq), "pbcq-mmio0", size); > + memory_region_add_subregion(get_system_memory(), bar, &pbcq->mmbar0); > + pbcq->mmio0_mapped = true; > + pbcq->mmio0_base = bar; > + pbcq->mmio0_size = size; > + } > + if (!pbcq->mmio1_mapped && (bar_en & PBCQ_NEST_BAR_EN_MMIO1)) { > + bar = pbcq->nest_regs[PBCQ_NEST_MMIO_BAR1] >> 14; > + mask = pbcq->nest_regs[PBCQ_NEST_MMIO_MASK1]; > + size = ((~mask) >> 14) + 1; > + memory_region_init(&pbcq->mmbar1, OBJECT(pbcq), "pbcq-mmio1", size); > + memory_region_add_subregion(get_system_memory(), bar, &pbcq->mmbar1); > + pbcq->mmio1_mapped = true; > + pbcq->mmio1_base = bar; > + pbcq->mmio1_size = size; > + } > + if (!pbcq->phb_mapped && (bar_en & PBCQ_NEST_BAR_EN_PHB)) { > + bar = pbcq->nest_regs[PBCQ_NEST_PHB_BAR] >> 14; > + size = 0x1000; > + memory_region_init(&pbcq->phbbar, OBJECT(pbcq), "pbcq-phb", size); > + memory_region_add_subregion(get_system_memory(), bar, &pbcq->phbbar); > + pbcq->phb_mapped = true; > + } > + > + /* Update PHB */ > + pnv_phb3_update_regions(pbcq->phb); > +} > + > +static void pnv_pbcq_nest_xscom_write(void *opaque, hwaddr addr, > + uint64_t val, unsigned size) > +{ > + PnvPBCQState *pbcq = PNV_PBCQ(opaque); > + uint32_t reg = addr >> 3; > + > + switch (reg) { > + case PBCQ_NEST_MMIO_BAR0: > + case PBCQ_NEST_MMIO_BAR1: > + case PBCQ_NEST_MMIO_MASK0: > + case PBCQ_NEST_MMIO_MASK1: > + if (pbcq->nest_regs[PBCQ_NEST_BAR_EN] & > + (PBCQ_NEST_BAR_EN_MMIO0 | > + PBCQ_NEST_BAR_EN_MMIO1)) { > + qemu_log_mask(LOG_GUEST_ERROR, > + "PHB3: Changing enabled BAR unsupported\n"); > + } > + pbcq->nest_regs[reg] = val & 0xffffffffc0000000ull; > + break; > + case PBCQ_NEST_PHB_BAR: > + if (pbcq->nest_regs[PBCQ_NEST_BAR_EN] & PBCQ_NEST_BAR_EN_PHB) { > + qemu_log_mask(LOG_GUEST_ERROR, > + "PHB3: Changing enabled BAR unsupported\n"); > + } > + pbcq->nest_regs[reg] = val & 0xfffffffffc000000ull; > + break; > + case PBCQ_NEST_BAR_EN: > + pbcq->nest_regs[reg] = val & 0xf800000000000000ull; > + pnv_pbcq_update_map(pbcq); > + pnv_phb3_remap_irqs(pbcq->phb); > + break; > + case PBCQ_NEST_IRSN_COMPARE: > + case PBCQ_NEST_IRSN_MASK: > + pbcq->nest_regs[reg] = val & PBCQ_NEST_IRSN_COMP; > + pnv_phb3_remap_irqs(pbcq->phb); > + break; > + case PBCQ_NEST_LSI_SRC_ID: > + pbcq->nest_regs[reg] = val & PBCQ_NEST_LSI_SRC; > + pnv_phb3_remap_irqs(pbcq->phb); > + break; > + } > + > + /* XXX Don't error out on other regs for now ... */ > +} > + > +static void pnv_pbcq_pci_xscom_write(void *opaque, hwaddr addr, > + uint64_t val, unsigned size) > +{ > + PnvPBCQState *pbcq = PNV_PBCQ(opaque); > + uint32_t reg = addr >> 3; > + > + switch (reg) { > + case PBCQ_PCI_BAR2: > + pbcq->pci_regs[reg] = val & 0xfffffffffc000000ull; > + pnv_pbcq_update_map(pbcq); > + break; > + } > + > + /* XXX Don't error out on other regs for now ... */ > +} > + > +static void pnv_pbcq_spci_xscom_write(void *opaque, hwaddr addr, > + uint64_t val, unsigned size) > +{ > + PnvPBCQState *pbcq = PNV_PBCQ(opaque); > + uint32_t reg = addr >> 3; > + > + switch (reg) { > + case PBCQ_SPCI_ASB_ADDR: > + pbcq->spci_regs[reg] = val & 0xfff; > + break; > + case PBCQ_SPCI_ASB_STATUS: > + pbcq->spci_regs[reg] &= ~val; > + break; > + case PBCQ_SPCI_ASB_DATA: > + pnv_phb3_reg_write(pbcq->phb, pbcq->spci_regs[PBCQ_SPCI_ASB_ADDR], > + val, 8); > + break; > + case PBCQ_SPCI_AIB_CAPP_EN: > + case PBCQ_SPCI_CAPP_SEC_TMR: > + break; > + } > + > + /* XXX Don't error out on other regs for now ... */ > +} > + > +static const MemoryRegionOps pnv_pbcq_nest_xscom_ops = { > + .read = pnv_pbcq_nest_xscom_read, > + .write = pnv_pbcq_nest_xscom_write, > + .valid.min_access_size = 8, > + .valid.max_access_size = 8, > + .impl.min_access_size = 8, > + .impl.max_access_size = 8, > + .endianness = DEVICE_BIG_ENDIAN, > +}; > + > +static const MemoryRegionOps pnv_pbcq_pci_xscom_ops = { > + .read = pnv_pbcq_pci_xscom_read, > + .write = pnv_pbcq_pci_xscom_write, > + .valid.min_access_size = 8, > + .valid.max_access_size = 8, > + .impl.min_access_size = 8, > + .impl.max_access_size = 8, > + .endianness = DEVICE_BIG_ENDIAN, > +}; > + > +static const MemoryRegionOps pnv_pbcq_spci_xscom_ops = { > + .read = pnv_pbcq_spci_xscom_read, > + .write = pnv_pbcq_spci_xscom_write, > + .valid.min_access_size = 8, > + .valid.max_access_size = 8, > + .impl.min_access_size = 8, > + .impl.max_access_size = 8, > + .endianness = DEVICE_BIG_ENDIAN, > +}; > + > +static void pnv_pbcq_default_bars(PnvPBCQState *pbcq) > +{ > + uint64_t mm0, mm1, reg; > + PnvPHB3 *phb = pbcq->phb; > + > + mm0 = 0x3d00000000000ull + 0x4000000000ull * phb->chip_id + > + 0x1000000000ull * phb->phb_id; > + mm1 = 0x3ff8000000000ull + 0x0200000000ull * phb->chip_id + > + 0x0080000000ull * phb->phb_id; > + reg = 0x3fffe40000000ull + 0x0000400000ull * phb->chip_id + > + 0x0000100000ull * phb->phb_id; > + > + pbcq->nest_regs[PBCQ_NEST_MMIO_BAR0] = mm0 << 14; > + pbcq->nest_regs[PBCQ_NEST_MMIO_BAR1] = mm1 << 14; > + pbcq->nest_regs[PBCQ_NEST_PHB_BAR] = reg << 14; > + pbcq->nest_regs[PBCQ_NEST_MMIO_MASK0] = 0x3fff000000000ull << 14; > + pbcq->nest_regs[PBCQ_NEST_MMIO_MASK1] = 0x3ffff80000000ull << 14; > + pbcq->pci_regs[PBCQ_PCI_BAR2] = reg << 14; > +} > + > +static void pnv_pbcq_realize(DeviceState *dev, Error **errp) > +{ > + PnvPBCQState *pbcq = PNV_PBCQ(dev); > + PnvPHB3 *phb; > + Object *obj; > + Error *local_err = NULL; > + char name[32]; > + > + obj = object_property_get_link(OBJECT(dev), "phb", &local_err); > + if (!obj) { > + error_propagate(errp, local_err); > + error_prepend(errp, "required link 'phb' not found: "); > + return; > + } > + phb = pbcq->phb = PNV_PHB3(obj); > + > + /* XXX Fix OPAL to do that: establish default BAR values */ > + pnv_pbcq_default_bars(pbcq); > + > + /* Initialize the XSCOM region for the PBCQ registers */ > + snprintf(name, sizeof(name), "xscom-pbcq-nest-%d.%d", > + phb->chip_id, phb->phb_id); > + pnv_xscom_region_init(&pbcq->xscom_nest_regs, OBJECT(dev), > + &pnv_pbcq_nest_xscom_ops, pbcq, name, > + PNV_XSCOM_PBCQ_NEST_SIZE); > + snprintf(name, sizeof(name), "xscom-pbcq-pci-%d.%d", > + phb->chip_id, phb->phb_id); > + pnv_xscom_region_init(&pbcq->xscom_pci_regs, OBJECT(dev), > + &pnv_pbcq_pci_xscom_ops, pbcq, name, > + PNV_XSCOM_PBCQ_PCI_SIZE); > + snprintf(name, sizeof(name), "xscom-pbcq-spci-%d.%d", > + phb->chip_id, phb->phb_id); > + pnv_xscom_region_init(&pbcq->xscom_spci_regs, OBJECT(dev), > + &pnv_pbcq_spci_xscom_ops, pbcq, name, > + PNV_XSCOM_PBCQ_SPCI_SIZE); > +} > + > +static int pnv_pbcq_dt_xscom(PnvXScomInterface *dev, void *fdt, > + int xscom_offset) > +{ > + const char compat[] = "ibm,power8-pbcq"; > + PnvPHB3 *phb = PNV_PBCQ(dev)->phb; > + char *name; > + int offset; > + uint32_t lpc_pcba = PNV_XSCOM_PBCQ_NEST_BASE + 0x400 * phb->phb_id; > + uint32_t reg[] = { > + cpu_to_be32(lpc_pcba), > + cpu_to_be32(PNV_XSCOM_PBCQ_NEST_SIZE), > + cpu_to_be32(PNV_XSCOM_PBCQ_PCI_BASE + 0x400 * phb->phb_id), > + cpu_to_be32(PNV_XSCOM_PBCQ_PCI_SIZE), > + cpu_to_be32(PNV_XSCOM_PBCQ_SPCI_BASE + 0x040 * phb->phb_id), > + cpu_to_be32(PNV_XSCOM_PBCQ_SPCI_SIZE) > + }; > + > + name = g_strdup_printf("pbcq@%x", lpc_pcba); > + offset = fdt_add_subnode(fdt, xscom_offset, name); > + _FDT(offset); > + g_free(name); > + > + _FDT((fdt_setprop(fdt, offset, "reg", reg, sizeof(reg)))); > + > + _FDT((fdt_setprop_cell(fdt, offset, "ibm,phb-index", phb->phb_id))); > + _FDT((fdt_setprop_cell(fdt, offset, "ibm,chip-id", phb->chip_id))); > + _FDT((fdt_setprop(fdt, offset, "compatible", compat, > + sizeof(compat)))); > + return 0; > +} > + > +static void pnv_pbcq_class_init(ObjectClass *klass, void *data) > +{ > + DeviceClass *dc = DEVICE_CLASS(klass); > + PnvXScomInterfaceClass *xdc = PNV_XSCOM_INTERFACE_CLASS(klass); > + > + xdc->dt_xscom = pnv_pbcq_dt_xscom; > + > + dc->realize = pnv_pbcq_realize; > +} > + > +static const TypeInfo pnv_pbcq_type_info = { > + .name = TYPE_PNV_PBCQ, > + .parent = TYPE_DEVICE, > + .instance_size = sizeof(PnvPBCQState), > + .class_init = pnv_pbcq_class_init, > + .interfaces = (InterfaceInfo[]) { > + { TYPE_PNV_XSCOM_INTERFACE }, > + { } > + } > +}; > + > +static void pnv_pbcq_register_types(void) > +{ > + type_register_static(&pnv_pbcq_type_info); > +} > + > +type_init(pnv_pbcq_register_types) > diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c > index 346f5e7aedb5..e8a8065fd25f 100644 > --- a/hw/ppc/pnv.c > +++ b/hw/ppc/pnv.c > @@ -44,6 +44,7 @@ > #include "hw/isa/isa.h" > #include "hw/char/serial.h" > #include "hw/timer/mc146818rtc.h" > +#include "hw/pci/msi.h" > > #include <libfdt.h> > > @@ -555,6 +556,9 @@ static void pnv_init(MachineState *machine) > int i; > char *chip_typename; > > + /* MSIs are supported on this platform */ > + msi_nonbroken = true; > + > /* allocate RAM */ > if (machine->ram_size < (1 * GiB)) { > warn_report("skiboot may not work with < 1GB of RAM"); > @@ -725,6 +729,8 @@ static Object *pnv_chip_power9_intc_create(PnvChip *chip, Object *child, > static void pnv_chip_power8_instance_init(Object *obj) > { > Pnv8Chip *chip8 = PNV8_CHIP(obj); > + PnvChipClass *pcc = PNV_CHIP_GET_CLASS(obj); > + int i; > > object_initialize(&chip8->psi, sizeof(chip8->psi), TYPE_PNV_PSI); > object_property_add_child(obj, "psi", OBJECT(&chip8->psi), NULL); > @@ -740,6 +746,12 @@ static void pnv_chip_power8_instance_init(Object *obj) > object_property_add_child(obj, "occ", OBJECT(&chip8->occ), NULL); > object_property_add_const_link(OBJECT(&chip8->occ), "psi", > OBJECT(&chip8->psi), &error_abort); > + > + for (i = 0; i < pcc->num_phbs; i++) { > + object_initialize(&chip8->phbs[i], sizeof(chip8->phbs[i]), > + TYPE_PNV_PHB3); > + object_property_add_child(obj, "phb[*]", OBJECT(&chip8->phbs[i]), NULL); > + } > } > > static void pnv_chip_icp_realize(Pnv8Chip *chip8, Error **errp) > @@ -780,6 +792,7 @@ static void pnv_chip_power8_realize(DeviceState *dev, Error **errp) > PnvChip *chip = PNV_CHIP(dev); > Pnv8Chip *chip8 = PNV8_CHIP(dev); > Error *local_err = NULL; > + int i; > > pcc->parent_realize(dev, &local_err); > if (local_err) { > @@ -817,6 +830,33 @@ static void pnv_chip_power8_realize(DeviceState *dev, Error **errp) > return; > } > pnv_xscom_add_subregion(chip, PNV_XSCOM_OCC_BASE, &chip8->occ.xscom_regs); > + > + /* Create the PHB3 controllers */ > + for (i = 0; i < pcc->num_phbs; i++) { > + PnvPHB3 *phb = &chip8->phbs[i]; > + PnvPBCQState *pbcq = &phb->pbcq; > + > + object_property_set_int(OBJECT(phb), i, "index", &error_fatal); > + object_property_set_int(OBJECT(phb), chip->chip_id, "chip-id", > + &error_fatal); > + object_property_set_bool(OBJECT(phb), true, "realized", &local_err); > + if (local_err) { > + error_propagate(errp, local_err); > + return; > + } > + qdev_set_parent_bus(DEVICE(phb), sysbus_get_default()); > + > + /* Populate the XSCOM address space. */ > + pnv_xscom_add_subregion(chip, > + PNV_XSCOM_PBCQ_NEST_BASE + 0x400 * phb->phb_id, > + &pbcq->xscom_nest_regs); > + pnv_xscom_add_subregion(chip, > + PNV_XSCOM_PBCQ_PCI_BASE + 0x400 * phb->phb_id, > + &pbcq->xscom_pci_regs); > + pnv_xscom_add_subregion(chip, > + PNV_XSCOM_PBCQ_SPCI_BASE + 0x040 * phb->phb_id, > + &pbcq->xscom_spci_regs); > + } > } > > static void pnv_chip_power8e_class_init(ObjectClass *klass, void *data) > @@ -827,6 +867,7 @@ static void pnv_chip_power8e_class_init(ObjectClass *klass, void *data) > k->chip_type = PNV_CHIP_POWER8E; > k->chip_cfam_id = 0x221ef04980000000ull; /* P8 Murano DD2.1 */ > k->cores_mask = POWER8E_CORE_MASK; > + k->num_phbs = 3; > k->core_pir = pnv_chip_core_pir_p8; > k->intc_create = pnv_chip_power8_intc_create; > k->isa_create = pnv_chip_power8_isa_create; > @@ -845,6 +886,7 @@ static void pnv_chip_power8_class_init(ObjectClass *klass, void *data) > k->chip_type = PNV_CHIP_POWER8; > k->chip_cfam_id = 0x220ea04980000000ull; /* P8 Venice DD2.0 */ > k->cores_mask = POWER8_CORE_MASK; > + k->num_phbs = 3; > k->core_pir = pnv_chip_core_pir_p8; > k->intc_create = pnv_chip_power8_intc_create; > k->isa_create = pnv_chip_power8_isa_create; > @@ -863,6 +905,7 @@ static void pnv_chip_power8nvl_class_init(ObjectClass *klass, void *data) > k->chip_type = PNV_CHIP_POWER8NVL; > k->chip_cfam_id = 0x120d304980000000ull; /* P8 Naples DD1.0 */ > k->cores_mask = POWER8_CORE_MASK; > + k->num_phbs = 3; > k->core_pir = pnv_chip_core_pir_p8; > k->intc_create = pnv_chip_power8_intc_create; > k->isa_create = pnv_chip_power8nvl_isa_create; > @@ -897,6 +940,7 @@ static void pnv_chip_power9_class_init(ObjectClass *klass, void *data) > k->chip_type = PNV_CHIP_POWER9; > k->chip_cfam_id = 0x220d104900008000ull; /* P9 Nimbus DD2.0 */ > k->cores_mask = POWER9_CORE_MASK; > + k->num_phbs = 0; > k->core_pir = pnv_chip_core_pir_p9; > k->intc_create = pnv_chip_power9_intc_create; > k->isa_create = pnv_chip_power9_isa_create; > @@ -1047,14 +1091,24 @@ static void pnv_chip_class_init(ObjectClass *klass, void *data) > static ICSState *pnv_ics_get(XICSFabric *xi, int irq) > { > PnvMachineState *pnv = PNV_MACHINE(xi); > - int i; > + int i, j; > > for (i = 0; i < pnv->num_chips; i++) { > + PnvChip *chip = pnv->chips[i]; > Pnv8Chip *chip8 = PNV8_CHIP(pnv->chips[i]); > + PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip); > > if (ics_valid_irq(&chip8->psi.ics, irq)) { > return &chip8->psi.ics; > } > + for (j = 0; j < pcc->num_phbs; j++) { > + if (ics_valid_irq(&chip8->phbs[j].lsis, irq)) { > + return &chip8->phbs[j].lsis; > + } > + if (ics_valid_irq(ICS_BASE(&chip8->phbs[j].msis), irq)) { > + return ICS_BASE(&chip8->phbs[j].msis); > + } > + } > } > return NULL; > } > @@ -1062,11 +1116,18 @@ static ICSState *pnv_ics_get(XICSFabric *xi, int irq) > static void pnv_ics_resend(XICSFabric *xi) > { > PnvMachineState *pnv = PNV_MACHINE(xi); > - int i; > + int i, j; > > for (i = 0; i < pnv->num_chips; i++) { > + PnvChip *chip = pnv->chips[i]; > Pnv8Chip *chip8 = PNV8_CHIP(pnv->chips[i]); > + PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip); > + > ics_resend(&chip8->psi.ics); > + for (j = 0; j < pcc->num_phbs; j++) { > + ics_resend(&chip8->phbs[j].lsis); > + ics_resend(ICS_BASE(&chip8->phbs[j].msis)); > + } > } > } > > @@ -1097,7 +1158,7 @@ static void pnv_pic_print_info(InterruptStatsProvider *obj, > Monitor *mon) > { > PnvMachineState *pnv = PNV_MACHINE(obj); > - int i; > + int i, j; > CPUState *cs; > > CPU_FOREACH(cs) { > @@ -1107,8 +1168,16 @@ static void pnv_pic_print_info(InterruptStatsProvider *obj, > } > > for (i = 0; i < pnv->num_chips; i++) { > + PnvChip *chip = pnv->chips[i]; > Pnv8Chip *chip8 = PNV8_CHIP(pnv->chips[i]); > + PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip); > + > ics_pic_print_info(&chip8->psi.ics, mon); > + > + for (j = 0; j < pcc->num_phbs; j++) { > + ics_pic_print_info(ICS_BASE(&chip8->phbs[j].msis), mon); > + ics_pic_print_info(&chip8->phbs[j].lsis, mon); > + } > } > } > > diff --git a/hw/ppc/pnv_xscom.c b/hw/ppc/pnv_xscom.c > index 46fae41f32b0..83f05d06053f 100644 > --- a/hw/ppc/pnv_xscom.c > +++ b/hw/ppc/pnv_xscom.c > @@ -252,7 +252,11 @@ int pnv_dt_xscom(PnvChip *chip, void *fdt, int root_offset) > args.fdt = fdt; > args.xscom_offset = xscom_offset; > > - object_child_foreach(OBJECT(chip), xscom_dt_child, &args); > + /* Some PnvXScomInterface objects lie a bit deeper (PnvPBCQState) > + * than the first layer, so we need to loop on the whole object > + * hierarchy to catch them > + */ > + object_child_foreach_recursive(OBJECT(chip), xscom_dt_child, &args); > return 0; > } > > diff --git a/hw/pci-host/Makefile.objs b/hw/pci-host/Makefile.objs > index 6d6597c06563..1b5b782bd986 100644 > --- a/hw/pci-host/Makefile.objs > +++ b/hw/pci-host/Makefile.objs > @@ -19,3 +19,4 @@ common-obj-$(CONFIG_PCI_GENERIC) += gpex.o > common-obj-$(CONFIG_PCI_XILINX) += xilinx-pcie.o > > common-obj-$(CONFIG_PCI_DESIGNWARE) += designware.o > +obj-$(CONFIG_POWERNV) += pnv_phb3.o pnv_phb3_pbcq.o pnv_phb3_msi.o
On 11/30/18 5:26 AM, David Gibson wrote: > On Mon, Jul 30, 2018 at 07:17:48PM +0200, Cédric Le Goater wrote: >> From: Benjamin Herrenschmidt <benh@kernel.crashing.org> >> >> This is a model of the PCIe Host Bridge (PHB3) found on a Power8 >> processor. It includes the PowerBus logic interface (PBCQ), IOMMU >> support, a single PCIe Gen.3 Root Complex, and support for MSI and LSI >> interrupt sources as found on a Power8 system using the XICS interrupt >> controller. >> >> The Power8 processor comes in different flavors: Venice, Murano, >> Naple, each having a different number of PHBs. To make things simpler, >> the PHB3 model provides 3 per chip. Some platforms, like the >> Firestone, can also couple PHBs on the first chip to provide more >> bandwidth but this is too specific to model in QEMU. >> >> No default device layout is provided and PCI devices can be added on >> any of the available PCIe Root Port (pcie.0 .. 2 of a Power8 chip) >> with address 0x0 as the firwware (skiboot) only accepts a single >> device per root port. To run a simple system with a network and a >> storage adapters, use a command line options such as : >> >> -device e1000e,netdev=net0,mac=C0:FF:EE:00:00:02,bus=pcie.0,addr=0x0 >> -netdev bridge,id=net0,helper=/usr/libexec/qemu-bridge-helper,br=virbr0,id=hostnet0 >> >> -device megasas,id=scsi0,bus=pcie.1,addr=0x0 >> -drive file=$disk,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=none >> -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=2 >> >> If more are needed, include a bridge. >> >> Multi chip is supported, each chip adding its set of PHB3 controllers >> and its PCI busses. The model doesn't emulate the EEH error handling >> (and may never do). > > Sorry I've let this sit for so long. > > I discussed the overall model here with BenH, including how we'd adapt > it for PHB4 support on POWER9. We did come to a consensus on what > I think is the best way to balance the need for hardware accuracy > against qemu's desire for uniform configuration with -device. > > So, first, the overall comment on that: I think the way to go is to > have all a chip's PBCQs *always* instantiated (even with -nodefaults) > according to the chip's specs. The actual PHBs would be instantiated > with -device, and plug into an appropriate existing PBCQ. OK. looks good to me. > Now, I think we might be able to do that entirely within the QOM/qdev > model by creating a suitable pseudo-bus or buses, but I'm going to > need a bit more research on that. OK. > Because the powernv model isn't migration stable across versions I'm > ok with applying something ad-hoc initially then revising it. Let me revise the PBCQ modeling, it has been bugging me for a while. If we can get it right first, it will help for the following steps. I won't look at the bus you propose but I will check the compatibility with PHB4. > >> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> >> [clg: rewrote the QOM models >> misc fixes >> lots of love and care] >> Signed-off-by: Cédric Le Goater <clg@kaod.org> >> --- >> default-configs/ppc64-softmmu.mak | 1 + >> include/hw/pci-host/pnv_phb3.h | 171 ++++ >> include/hw/pci-host/pnv_phb3_regs.h | 432 ++++++++++ >> include/hw/ppc/pnv.h | 22 + >> include/hw/ppc/pnv_xscom.h | 9 + >> include/hw/ppc/xics.h | 1 + >> hw/intc/xics.c | 2 +- >> hw/pci-host/pnv_phb3.c | 1146 +++++++++++++++++++++++++++ >> hw/pci-host/pnv_phb3_msi.c | 318 ++++++++ >> hw/pci-host/pnv_phb3_pbcq.c | 347 ++++++++ >> hw/ppc/pnv.c | 75 +- >> hw/ppc/pnv_xscom.c | 6 +- >> hw/pci-host/Makefile.objs | 1 + >> 13 files changed, 2526 insertions(+), 5 deletions(-) >> create mode 100644 include/hw/pci-host/pnv_phb3.h >> create mode 100644 include/hw/pci-host/pnv_phb3_regs.h >> create mode 100644 hw/pci-host/pnv_phb3.c >> create mode 100644 hw/pci-host/pnv_phb3_msi.c >> create mode 100644 hw/pci-host/pnv_phb3_pbcq.c >> >> diff --git a/default-configs/ppc64-softmmu.mak b/default-configs/ppc64-softmmu.mak >> index b94af6c7c62a..deebba2b044a 100644 >> --- a/default-configs/ppc64-softmmu.mak >> +++ b/default-configs/ppc64-softmmu.mak >> @@ -9,6 +9,7 @@ CONFIG_IPMI=y >> CONFIG_IPMI_LOCAL=y >> CONFIG_IPMI_EXTERN=y >> CONFIG_ISA_IPMI_BT=y >> +CONFIG_PCIE_PORT=y > > AIUI the existing PCIE_PORT isn't exactly like what the PHB3 has, but > is close enough for now. yes. Ben is reintroducing a custom one for PHB4 (I removed it from PHB3), I need to check why exactly. > >> # For pSeries >> CONFIG_PSERIES=y >> diff --git a/include/hw/pci-host/pnv_phb3.h b/include/hw/pci-host/pnv_phb3.h >> new file mode 100644 >> index 000000000000..eb9efaf4e13e >> --- /dev/null >> +++ b/include/hw/pci-host/pnv_phb3.h >> @@ -0,0 +1,171 @@ >> +/* >> + * QEMU PowerPC PowerNV PHB3 model >> + * >> + * Copyright (c) 2014-2018, IBM Corporation. >> + * >> + * This code is licensed under the GPL version 2 or later. See the >> + * COPYING file in the top-level directory. >> + */ >> + >> +#ifndef PCI_HOST_PNV_PHB3_H >> +#define PCI_HOST_PNV_PHB3_H >> + >> +#include "hw/pci/pcie_host.h" >> +#include "hw/pci/pcie_port.h" >> +#include "hw/ppc/xics.h" >> + >> +typedef struct PnvPHB3 PnvPHB3; >> +typedef struct PnvChip PnvChip; >> + >> +/* >> + * PHB3 XICS Source for MSIs >> + */ >> +#define TYPE_PHB3_MSI "phb3-msi" >> +#define PHB3_MSI(obj) OBJECT_CHECK(Phb3MsiState, (obj), TYPE_PHB3_MSI) >> + >> +#define PHB3_MAX_MSI 2048 >> + >> +typedef struct Phb3MsiState { >> + ICSState ics; >> + >> + PnvPHB3 *phb; >> + uint64_t rba[PHB3_MAX_MSI / 64]; >> + uint32_t rba_sum; >> +} Phb3MsiState; > > What's the rationale for making the MSI unit a separate QOM device, > rather than just folding it into the PHB device? yes. you are right. Let me check if can merge both. >> +void pnv_phb3_msi_update_config(Phb3MsiState *msis, uint32_t base, >> + uint32_t count); >> +void pnv_phb3_msi_send(Phb3MsiState *msis, uint64_t addr, uint16_t data, >> + int32_t dev_pe); >> +void pnv_phb3_msi_ffi(Phb3MsiState *msis, uint64_t val); >> + >> + >> +/* We have one such address space wrapper per possible device >> + * under the PHB since they need to be assigned statically at >> + * qemu device creation time. The relationship to a PE is done >> + * later dynamically. This means we can potentially create a lot >> + * of these guys. Q35 stores them as some kind of radix tree but >> + * we never really need to do fast lookups so instead we simply >> + * keep a QLIST of them for now, we can add the radix if needed >> + * later on. > > Could you start this comment of by describing exactly what one of > these instances represents, it's not terribly obvious to me. OK. > >> + * >> + * We do cache the PE number to speed things up a bit though. >> + */ >> +typedef struct PnvPhb3DMASpace { >> + PCIBus *bus; >> + uint8_t devfn; >> + int pe_num; /* Cached PE number */ >> +#define PHB_INVALID_PE (-1) >> + PnvPHB3 *phb; >> + AddressSpace dma_as; >> + IOMMUMemoryRegion dma_mr; >> + MemoryRegion msi32_mr; >> + MemoryRegion msi64_mr; >> + bool msi32_mapped; >> + bool msi64_mapped; > > Do you need these, or could you use memory_region_is_mapped() instead? probably. > >> + QLIST_ENTRY(PnvPhb3DMASpace) list; >> +} PnvPhb3DMASpace; >> + >> +/* >> + * PHB3 Power Bus Common Queue >> + */ >> +#define TYPE_PNV_PBCQ "pnv-pbcq" >> +#define PNV_PBCQ(obj) OBJECT_CHECK(PnvPBCQState, (obj), TYPE_PNV_PBCQ) >> + >> +typedef struct PnvPBCQState { >> + DeviceState parent; > > These have MMIOs, so it probably makes sense to be a SysBusDevice. OK. > >> + uint32_t nest_xbase; >> + uint32_t spci_xbase; >> + uint32_t pci_xbase; >> +#define PBCQ_NEST_REGS_COUNT 0x46 >> +#define PBCQ_PCI_REGS_COUNT 0x15 >> +#define PBCQ_SPCI_REGS_COUNT 0x5 >> + >> + uint64_t nest_regs[PBCQ_NEST_REGS_COUNT]; >> + uint64_t spci_regs[PBCQ_SPCI_REGS_COUNT]; >> + uint64_t pci_regs[PBCQ_PCI_REGS_COUNT]; >> + MemoryRegion mmbar0; >> + MemoryRegion mmbar1; >> + MemoryRegion phbbar; >> + bool mmio0_mapped; >> + bool mmio1_mapped; >> + bool phb_mapped; >> + uint64_t mmio0_base; >> + uint64_t mmio0_size; >> + uint64_t mmio1_base; >> + uint64_t mmio1_size; >> + PnvPHB3 *phb; >> + >> + MemoryRegion xscom_nest_regs; >> + MemoryRegion xscom_pci_regs; >> + MemoryRegion xscom_spci_regs; >> +} PnvPBCQState; >> + >> +/* >> + * PHB3 PCIe Root port >> + */ >> +#define TYPE_PNV_PHB3_ROOT_BUS "pnv-phb3-root-bus" >> + >> +#define TYPE_PNV_PHB3_ROOT_PORT "pnv-phb3-root-port" >> + >> +typedef struct PnvPHB3RootPort { >> + PCIESlot parent_obj; >> +} PnvPHB3RootPort; >> + >> +/* >> + * PHB3 PCIe Host Bridge for PowerNV machines (POWER8) >> + */ >> +#define TYPE_PNV_PHB3 "pnv-phb3" >> +#define PNV_PHB3(obj) OBJECT_CHECK(PnvPHB3, (obj), TYPE_PNV_PHB3) >> + >> +#define PNV_PHB3_NUM_M64 16 >> +#define PNV_PHB3_NUM_REGS (0x1000 >> 3) >> +#define PNV_PHB3_NUM_LSI 8 >> +#define PNV_PHB3_NUM_PE 256 >> + >> +#define PCI_MMIO_TOTAL_SIZE (0x1ull << 60) >> + >> +struct PnvPHB3 { >> + PCIExpressHost parent_obj; >> + >> + PnvPHB3RootPort root; >> + >> + uint32_t chip_id; >> + uint32_t phb_id; >> + char bus_path[8]; >> + >> + uint64_t regs[PNV_PHB3_NUM_REGS]; >> + MemoryRegion mr_regs; >> + >> + MemoryRegion mr_m32; >> + MemoryRegion mr_m64[PNV_PHB3_NUM_M64]; >> + bool regs_mapped; >> + bool m32_mapped; >> + bool m64_mapped[PNV_PHB3_NUM_M64]; >> + MemoryRegion pci_mmio; >> + MemoryRegion pci_io; >> + >> + uint64_t ioda_LIST[8]; >> + uint64_t ioda_LXIVT[8]; >> + uint64_t ioda_TVT[512]; >> + uint64_t ioda_M64BT[16]; >> + uint64_t ioda_MDT[256]; >> + uint64_t ioda_PEEV[4]; >> + >> + uint32_t total_irq; >> + ICSState lsis; >> + Phb3MsiState msis; >> + >> + PnvPBCQState pbcq; > > I think embedding the pbcq is the wrong way around here. yes. it has been a problem with xscom also. I will change. > >> + >> + QLIST_HEAD(, PnvPhb3DMASpace) dma_spaces; >> +}; >> + >> +uint64_t pnv_phb3_reg_read(void *opaque, hwaddr off, unsigned size); >> +void pnv_phb3_reg_write(void *opaque, hwaddr off, uint64_t val, unsigned size); >> +void pnv_phb3_update_regions(PnvPHB3 *phb); >> +void pnv_phb3_remap_irqs(PnvPHB3 *phb); >> + >> + >> +#endif /* PCI_HOST_PNV_PHB3_H */ >> diff --git a/include/hw/pci-host/pnv_phb3_regs.h b/include/hw/pci-host/pnv_phb3_regs.h >> new file mode 100644 >> index 000000000000..46e4e0877d51 >> --- /dev/null >> +++ b/include/hw/pci-host/pnv_phb3_regs.h >> @@ -0,0 +1,432 @@ >> +/* Copyright (c) 2013-2018, IBM Corporation. >> + * >> + * Licensed under the Apache License, Version 2.0 (the "License"); >> + * you may not use this file except in compliance with the License. >> + * You may obtain a copy of the License at >> + * >> + * http://www.apache.org/licenses/LICENSE-2.0 >> + * >> + * Unless required by applicable law or agreed to in writing, software >> + * distributed under the License is distributed on an "AS IS" BASIS, >> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or >> + * implied. >> + * >> + * See the License for the specific language governing permissions and >> + * limitations under the License. >> + */ >> + >> +#ifndef PCI_HOST_PNV_PHB3_REGS_H >> +#define PCI_HOST_PNV_PHB3_REGS_H >> + >> +/* >> + * PBCQ XSCOM registers >> + */ >> + >> +#define PBCQ_NEST_IRSN_COMPARE 0x1a >> +#define PBCQ_NEST_IRSN_COMP PPC_BITMASK(0, 18) >> +#define PBCQ_NEST_IRSN_MASK 0x1b >> +#define PBCQ_NEST_LSI_SRC_ID 0x1f >> +#define PBCQ_NEST_LSI_SRC PPC_BITMASK(0, 7) >> +#define PBCQ_NEST_REGS_COUNT 0x46 >> +#define PBCQ_NEST_MMIO_BAR0 0x40 >> +#define PBCQ_NEST_MMIO_BAR1 0x41 >> +#define PBCQ_NEST_PHB_BAR 0x42 >> +#define PBCQ_NEST_MMIO_MASK0 0x43 >> +#define PBCQ_NEST_MMIO_MASK1 0x44 >> +#define PBCQ_NEST_BAR_EN 0x45 >> +#define PBCQ_NEST_BAR_EN_MMIO0 PPC_BIT(0) >> +#define PBCQ_NEST_BAR_EN_MMIO1 PPC_BIT(1) >> +#define PBCQ_NEST_BAR_EN_PHB PPC_BIT(2) >> +#define PBCQ_NEST_BAR_EN_IRSN_RX PPC_BIT(3) >> +#define PBCQ_NEST_BAR_EN_IRSN_TX PPC_BIT(4) >> + >> +#define PBCQ_PCI_REGS_COUNT 0x15 >> +#define PBCQ_PCI_BAR2 0x0b >> + >> +#define PBCQ_SPCI_REGS_COUNT 0x5 >> +#define PBCQ_SPCI_ASB_ADDR 0x0 >> +#define PBCQ_SPCI_ASB_STATUS 0x1 >> +#define PBCQ_SPCI_ASB_DATA 0x2 >> +#define PBCQ_SPCI_AIB_CAPP_EN 0x3 >> +#define PBCQ_SPCI_CAPP_SEC_TMR 0x4 >> + >> +/* >> + * PHB MMIO registers >> + */ >> + >> +/* PHB Fundamental register set A */ >> +#define PHB_LSI_SOURCE_ID 0x100 >> +#define PHB_LSI_SRC_ID PPC_BITMASK(5, 12) >> +#define PHB_DMA_CHAN_STATUS 0x110 >> +#define PHB_DMA_CHAN_ANY_ERR PPC_BIT(27) >> +#define PHB_DMA_CHAN_ANY_ERR1 PPC_BIT(28) >> +#define PHB_DMA_CHAN_ANY_FREEZE PPC_BIT(29) >> +#define PHB_CPU_LOADSTORE_STATUS 0x120 >> +#define PHB_CPU_LS_ANY_ERR PPC_BIT(27) >> +#define PHB_CPU_LS_ANY_ERR1 PPC_BIT(28) >> +#define PHB_CPU_LS_ANY_FREEZE PPC_BIT(29) >> +#define PHB_DMA_MSI_NODE_ID 0x128 >> +#define PHB_DMAMSI_NID_FIXED PPC_BIT(0) >> +#define PHB_DMAMSI_NID PPC_BITMASK(24, 31) >> +#define PHB_CONFIG_DATA 0x130 >> +#define PHB_LOCK0 0x138 >> +#define PHB_CONFIG_ADDRESS 0x140 >> +#define PHB_CA_ENABLE PPC_BIT(0) >> +#define PHB_CA_BUS PPC_BITMASK(4, 11) >> +#define PHB_CA_DEV PPC_BITMASK(12, 16) >> +#define PHB_CA_FUNC PPC_BITMASK(17, 19) >> +#define PHB_CA_REG PPC_BITMASK(20, 31) >> +#define PHB_CA_PE PPC_BITMASK(40, 47) >> +#define PHB_LOCK1 0x148 >> +#define PHB_IVT_BAR 0x150 >> +#define PHB_IVT_BAR_ENABLE PPC_BIT(0) >> +#define PHB_IVT_BASE_ADDRESS_MASK PPC_BITMASK(14, 48) >> +#define PHB_IVT_LENGTH_MASK PPC_BITMASK(52, 63) >> +#define PHB_RBA_BAR 0x158 >> +#define PHB_RBA_BAR_ENABLE PPC_BIT(0) >> +#define PHB_RBA_BASE_ADDRESS PPC_BITMASK(14, 55) >> +#define PHB_PHB3_CONFIG 0x160 >> +#define PHB_PHB3C_64B_TCE_EN PPC_BIT(2) >> +#define PHB_PHB3C_32BIT_MSI_EN PPC_BIT(8) >> +#define PHB_PHB3C_64BIT_MSI_EN PPC_BIT(14) >> +#define PHB_PHB3C_M32_EN PPC_BIT(16) >> +#define PHB_RTT_BAR 0x168 >> +#define PHB_RTT_BAR_ENABLE PPC_BIT(0) >> +#define PHB_RTT_BASE_ADDRESS_MASK PPC_BITMASK(14, 46) >> +#define PHB_PELTV_BAR 0x188 >> +#define PHB_PELTV_BAR_ENABLE PPC_BIT(0) >> +#define PHB_PELTV_BASE_ADDRESS PPC_BITMASK(14, 50) >> +#define PHB_M32_BASE_ADDR 0x190 >> +#define PHB_M32_BASE_MASK 0x198 >> +#define PHB_M32_START_ADDR 0x1a0 >> +#define PHB_PEST_BAR 0x1a8 >> +#define PHB_PEST_BAR_ENABLE PPC_BIT(0) >> +#define PHB_PEST_BASE_ADDRESS PPC_BITMASK(14, 51) >> +#define PHB_M64_UPPER_BITS 0x1f0 >> +#define PHB_INTREP_TIMER 0x1f8 >> +#define PHB_DMARD_SYNC 0x200 >> +#define PHB_RTC_INVALIDATE 0x208 >> +#define PHB_RTC_INVALIDATE_ALL PPC_BIT(0) >> +#define PHB_RTC_INVALIDATE_RID PPC_BITMASK(16, 31) >> +#define PHB_TCE_KILL 0x210 >> +#define PHB_TCE_KILL_ALL PPC_BIT(0) >> +#define PHB_TCE_SPEC_CTL 0x218 >> +#define PHB_IODA_ADDR 0x220 >> +#define PHB_IODA_AD_AUTOINC PPC_BIT(0) >> +#define PHB_IODA_AD_TSEL PPC_BITMASK(11, 15) >> +#define PHB_IODA_AD_TADR PPC_BITMASK(55, 63) >> +#define PHB_IODA_DATA0 0x228 >> +#define PHB_FFI_REQUEST 0x238 >> +#define PHB_FFI_LOCK_CLEAR PPC_BIT(3) >> +#define PHB_FFI_REQUEST_ISN PPC_BITMASK(49, 59) >> +#define PHB_FFI_LOCK 0x240 >> +#define PHB_FFI_LOCK_STATE PPC_BIT(0) >> +#define PHB_XIVE_UPDATE 0x248 /* Broken in DD1 */ >> +#define PHB_PHB3_GEN_CAP 0x250 >> +#define PHB_PHB3_TCE_CAP 0x258 >> +#define PHB_PHB3_IRQ_CAP 0x260 >> +#define PHB_PHB3_EEH_CAP 0x268 >> +#define PHB_IVC_INVALIDATE 0x2a0 >> +#define PHB_IVC_INVALIDATE_ALL PPC_BIT(0) >> +#define PHB_IVC_INVALIDATE_SID PPC_BITMASK(16, 31) >> +#define PHB_IVC_UPDATE 0x2a8 >> +#define PHB_IVC_UPDATE_ENABLE_P PPC_BIT(0) >> +#define PHB_IVC_UPDATE_ENABLE_Q PPC_BIT(1) >> +#define PHB_IVC_UPDATE_ENABLE_SERVER PPC_BIT(2) >> +#define PHB_IVC_UPDATE_ENABLE_PRI PPC_BIT(3) >> +#define PHB_IVC_UPDATE_ENABLE_GEN PPC_BIT(4) >> +#define PHB_IVC_UPDATE_ENABLE_CON PPC_BIT(5) >> +#define PHB_IVC_UPDATE_GEN_MATCH PPC_BITMASK(6, 7) >> +#define PHB_IVC_UPDATE_SERVER PPC_BITMASK(8, 23) >> +#define PHB_IVC_UPDATE_PRI PPC_BITMASK(24, 31) >> +#define PHB_IVC_UPDATE_GEN PPC_BITMASK(32, 33) >> +#define PHB_IVC_UPDATE_P PPC_BITMASK(34, 34) >> +#define PHB_IVC_UPDATE_Q PPC_BITMASK(35, 35) >> +#define PHB_IVC_UPDATE_SID PPC_BITMASK(48, 63) >> +#define PHB_PAPR_ERR_INJ_CTL 0x2b0 >> +#define PHB_PAPR_ERR_INJ_CTL_INB PPC_BIT(0) >> +#define PHB_PAPR_ERR_INJ_CTL_OUTB PPC_BIT(1) >> +#define PHB_PAPR_ERR_INJ_CTL_STICKY PPC_BIT(2) >> +#define PHB_PAPR_ERR_INJ_CTL_CFG PPC_BIT(3) >> +#define PHB_PAPR_ERR_INJ_CTL_RD PPC_BIT(4) >> +#define PHB_PAPR_ERR_INJ_CTL_WR PPC_BIT(5) >> +#define PHB_PAPR_ERR_INJ_CTL_FREEZE PPC_BIT(6) >> +#define PHB_PAPR_ERR_INJ_ADDR 0x2b8 >> +#define PHB_PAPR_ERR_INJ_ADDR_MMIO PPC_BITMASK(16, 63) >> +#define PHB_PAPR_ERR_INJ_MASK 0x2c0 >> +#define PHB_PAPR_ERR_INJ_MASK_CFG PPC_BITMASK(4, 11) >> +#define PHB_PAPR_ERR_INJ_MASK_MMIO PPC_BITMASK(16, 63) >> +#define PHB_ETU_ERR_SUMMARY 0x2c8 >> + >> +/* UTL registers */ >> +#define UTL_SYS_BUS_CONTROL 0x400 >> +#define UTL_STATUS 0x408 >> +#define UTL_SYS_BUS_AGENT_STATUS 0x410 >> +#define UTL_SYS_BUS_AGENT_ERR_SEVERITY 0x418 >> +#define UTL_SYS_BUS_AGENT_IRQ_EN 0x420 >> +#define UTL_SYS_BUS_BURST_SZ_CONF 0x440 >> +#define UTL_REVISION_ID 0x448 >> +#define UTL_BCLK_DOMAIN_DBG1 0x460 >> +#define UTL_BCLK_DOMAIN_DBG2 0x468 >> +#define UTL_BCLK_DOMAIN_DBG3 0x470 >> +#define UTL_BCLK_DOMAIN_DBG4 0x478 >> +#define UTL_BCLK_DOMAIN_DBG5 0x480 >> +#define UTL_BCLK_DOMAIN_DBG6 0x488 >> +#define UTL_OUT_POST_HDR_BUF_ALLOC 0x4c0 >> +#define UTL_OUT_POST_DAT_BUF_ALLOC 0x4d0 >> +#define UTL_IN_POST_HDR_BUF_ALLOC 0x4e0 >> +#define UTL_IN_POST_DAT_BUF_ALLOC 0x4f0 >> +#define UTL_OUT_NP_BUF_ALLOC 0x500 >> +#define UTL_IN_NP_BUF_ALLOC 0x510 >> +#define UTL_PCIE_TAGS_ALLOC 0x520 >> +#define UTL_GBIF_READ_TAGS_ALLOC 0x530 >> +#define UTL_PCIE_PORT_CONTROL 0x540 >> +#define UTL_PCIE_PORT_STATUS 0x548 >> +#define UTL_PCIE_PORT_ERROR_SEV 0x550 >> +#define UTL_PCIE_PORT_IRQ_EN 0x558 >> +#define UTL_RC_STATUS 0x560 >> +#define UTL_RC_ERR_SEVERITY 0x568 >> +#define UTL_RC_IRQ_EN 0x570 >> +#define UTL_EP_STATUS 0x578 >> +#define UTL_EP_ERR_SEVERITY 0x580 >> +#define UTL_EP_ERR_IRQ_EN 0x588 >> +#define UTL_PCI_PM_CTRL1 0x590 >> +#define UTL_PCI_PM_CTRL2 0x598 >> +#define UTL_GP_CTL1 0x5a0 >> +#define UTL_GP_CTL2 0x5a8 >> +#define UTL_PCLK_DOMAIN_DBG1 0x5b0 >> +#define UTL_PCLK_DOMAIN_DBG2 0x5b8 >> +#define UTL_PCLK_DOMAIN_DBG3 0x5c0 >> +#define UTL_PCLK_DOMAIN_DBG4 0x5c8 >> + >> +/* PCI-E Stack registers */ >> +#define PHB_PCIE_SYSTEM_CONFIG 0x600 >> +#define PHB_PCIE_BUS_NUMBER 0x608 >> +#define PHB_PCIE_SYSTEM_TEST 0x618 >> +#define PHB_PCIE_LINK_MANAGEMENT 0x630 >> +#define PHB_PCIE_LM_LINK_ACTIVE PPC_BIT(8) >> +#define PHB_PCIE_DLP_TRAIN_CTL 0x640 >> +#define PHB_PCIE_DLP_TCTX_DISABLE PPC_BIT(1) >> +#define PHB_PCIE_DLP_TCRX_DISABLED PPC_BIT(16) >> +#define PHB_PCIE_DLP_INBAND_PRESENCE PPC_BIT(19) >> +#define PHB_PCIE_DLP_TC_DL_LINKUP PPC_BIT(21) >> +#define PHB_PCIE_DLP_TC_DL_PGRESET PPC_BIT(22) >> +#define PHB_PCIE_DLP_TC_DL_LINKACT PPC_BIT(23) >> +#define PHB_PCIE_SLOP_LOOPBACK_STATUS 0x648 >> +#define PHB_PCIE_SYS_LINK_INIT 0x668 >> +#define PHB_PCIE_UTL_CONFIG 0x670 >> +#define PHB_PCIE_DLP_CONTROL 0x678 >> +#define PHB_PCIE_UTL_ERRLOG1 0x680 >> +#define PHB_PCIE_UTL_ERRLOG2 0x688 >> +#define PHB_PCIE_UTL_ERRLOG3 0x690 >> +#define PHB_PCIE_UTL_ERRLOG4 0x698 >> +#define PHB_PCIE_DLP_ERRLOG1 0x6a0 >> +#define PHB_PCIE_DLP_ERRLOG2 0x6a8 >> +#define PHB_PCIE_DLP_ERR_STATUS 0x6b0 >> +#define PHB_PCIE_DLP_ERR_COUNTERS 0x6b8 >> +#define PHB_PCIE_UTL_ERR_INJECT 0x6c0 >> +#define PHB_PCIE_TLDLP_ERR_INJECT 0x6c8 >> +#define PHB_PCIE_LANE_EQ_CNTL0 0x6d0 >> +#define PHB_PCIE_LANE_EQ_CNTL1 0x6d8 >> +#define PHB_PCIE_LANE_EQ_CNTL2 0x6e0 >> +#define PHB_PCIE_LANE_EQ_CNTL3 0x6e8 >> +#define PHB_PCIE_STRAPPING 0x700 >> + >> +/* Fundamental register set B */ >> +#define PHB_VERSION 0x800 >> +#define PHB_RESET 0x808 >> +#define PHB_CONTROL 0x810 >> +#define PHB_CTRL_IVE_128_BYTES PPC_BIT(24) >> +#define PHB_AIB_RX_CRED_INIT_TIMER 0x818 >> +#define PHB_AIB_RX_CMD_CRED 0x820 >> +#define PHB_AIB_RX_DATA_CRED 0x828 >> +#define PHB_AIB_TX_CMD_CRED 0x830 >> +#define PHB_AIB_TX_DATA_CRED 0x838 >> +#define PHB_AIB_TX_CHAN_MAPPING 0x840 >> +#define PHB_AIB_TAG_ENABLE 0x858 >> +#define PHB_AIB_FENCE_CTRL 0x860 >> +#define PHB_TCE_TAG_ENABLE 0x868 >> +#define PHB_TCE_WATERMARK 0x870 >> +#define PHB_TIMEOUT_CTRL1 0x878 >> +#define PHB_TIMEOUT_CTRL2 0x880 >> +#define PHB_QUIESCE_DMA_G 0x888 >> +#define PHB_AIB_TAG_STATUS 0x900 >> +#define PHB_TCE_TAG_STATUS 0x908 >> + >> +/* FIR & Error registers */ >> +#define PHB_LEM_FIR_ACCUM 0xc00 >> +#define PHB_LEM_FIR_AND_MASK 0xc08 >> +#define PHB_LEM_FIR_OR_MASK 0xc10 >> +#define PHB_LEM_ERROR_MASK 0xc18 >> +#define PHB_LEM_ERROR_AND_MASK 0xc20 >> +#define PHB_LEM_ERROR_OR_MASK 0xc28 >> +#define PHB_LEM_ACTION0 0xc30 >> +#define PHB_LEM_ACTION1 0xc38 >> +#define PHB_LEM_WOF 0xc40 >> +#define PHB_ERR_STATUS 0xc80 >> +#define PHB_ERR1_STATUS 0xc88 >> +#define PHB_ERR_INJECT 0xc90 >> +#define PHB_ERR_LEM_ENABLE 0xc98 >> +#define PHB_ERR_IRQ_ENABLE 0xca0 >> +#define PHB_ERR_FREEZE_ENABLE 0xca8 >> +#define PHB_ERR_AIB_FENCE_ENABLE 0xcb0 >> +#define PHB_ERR_LOG_0 0xcc0 >> +#define PHB_ERR_LOG_1 0xcc8 >> +#define PHB_ERR_STATUS_MASK 0xcd0 >> +#define PHB_ERR1_STATUS_MASK 0xcd8 >> + >> +#define PHB_OUT_ERR_STATUS 0xd00 >> +#define PHB_OUT_ERR1_STATUS 0xd08 >> +#define PHB_OUT_ERR_INJECT 0xd10 >> +#define PHB_OUT_ERR_LEM_ENABLE 0xd18 >> +#define PHB_OUT_ERR_IRQ_ENABLE 0xd20 >> +#define PHB_OUT_ERR_FREEZE_ENABLE 0xd28 >> +#define PHB_OUT_ERR_AIB_FENCE_ENABLE 0xd30 >> +#define PHB_OUT_ERR_LOG_0 0xd40 >> +#define PHB_OUT_ERR_LOG_1 0xd48 >> +#define PHB_OUT_ERR_STATUS_MASK 0xd50 >> +#define PHB_OUT_ERR1_STATUS_MASK 0xd58 >> + >> +#define PHB_INA_ERR_STATUS 0xd80 >> +#define PHB_INA_ERR1_STATUS 0xd88 >> +#define PHB_INA_ERR_INJECT 0xd90 >> +#define PHB_INA_ERR_LEM_ENABLE 0xd98 >> +#define PHB_INA_ERR_IRQ_ENABLE 0xda0 >> +#define PHB_INA_ERR_FREEZE_ENABLE 0xda8 >> +#define PHB_INA_ERR_AIB_FENCE_ENABLE 0xdb0 >> +#define PHB_INA_ERR_LOG_0 0xdc0 >> +#define PHB_INA_ERR_LOG_1 0xdc8 >> +#define PHB_INA_ERR_STATUS_MASK 0xdd0 >> +#define PHB_INA_ERR1_STATUS_MASK 0xdd8 >> + >> +#define PHB_INB_ERR_STATUS 0xe00 >> +#define PHB_INB_ERR1_STATUS 0xe08 >> +#define PHB_INB_ERR_INJECT 0xe10 >> +#define PHB_INB_ERR_LEM_ENABLE 0xe18 >> +#define PHB_INB_ERR_IRQ_ENABLE 0xe20 >> +#define PHB_INB_ERR_FREEZE_ENABLE 0xe28 >> +#define PHB_INB_ERR_AIB_FENCE_ENABLE 0xe30 >> +#define PHB_INB_ERR_LOG_0 0xe40 >> +#define PHB_INB_ERR_LOG_1 0xe48 >> +#define PHB_INB_ERR_STATUS_MASK 0xe50 >> +#define PHB_INB_ERR1_STATUS_MASK 0xe58 >> + >> +/* Performance monitor & Debug registers */ >> +#define PHB_TRACE_CONTROL 0xf80 >> +#define PHB_PERFMON_CONFIG 0xf88 >> +#define PHB_PERFMON_CTR0 0xf90 >> +#define PHB_PERFMON_CTR1 0xf98 >> +#define PHB_PERFMON_CTR2 0xfa0 >> +#define PHB_PERFMON_CTR3 0xfa8 >> +#define PHB_HOTPLUG_OVERRIDE 0xfb0 >> +#define PHB_HPOVR_FORCE_RESAMPLE PPC_BIT(9) >> +#define PHB_HPOVR_PRESENCE_A PPC_BIT(10) >> +#define PHB_HPOVR_PRESENCE_B PPC_BIT(11) >> +#define PHB_HPOVR_LINK_ACTIVE PPC_BIT(12) >> +#define PHB_HPOVR_LINK_BIFURCATED PPC_BIT(13) >> +#define PHB_HPOVR_LINK_LANE_SWAPPED PPC_BIT(14) >> + >> +/* >> + * IODA2 on-chip tables >> + */ >> + >> +#define IODA2_TBL_LIST 1 >> +#define IODA2_TBL_LXIVT 2 >> +#define IODA2_TBL_IVC_CAM 3 >> +#define IODA2_TBL_RBA 4 >> +#define IODA2_TBL_RCAM 5 >> +#define IODA2_TBL_MRT 6 >> +#define IODA2_TBL_PESTA 7 >> +#define IODA2_TBL_PESTB 8 >> +#define IODA2_TBL_TVT 9 >> +#define IODA2_TBL_TCAM 10 >> +#define IODA2_TBL_TDR 11 >> +#define IODA2_TBL_M64BT 16 >> +#define IODA2_TBL_M32DT 17 >> +#define IODA2_TBL_PEEV 20 >> + >> +/* LXIVT */ >> +#define IODA2_LXIVT_SERVER PPC_BITMASK(8, 23) >> +#define IODA2_LXIVT_PRIORITY PPC_BITMASK(24, 31) >> +#define IODA2_LXIVT_NODE_ID PPC_BITMASK(56, 63) >> + >> +/* IVT */ >> +#define IODA2_IVT_SERVER PPC_BITMASK(0, 23) >> +#define IODA2_IVT_PRIORITY PPC_BITMASK(24, 31) >> +#define IODA2_IVT_GEN PPC_BITMASK(37, 38) >> +#define IODA2_IVT_P PPC_BITMASK(39, 39) >> +#define IODA2_IVT_Q PPC_BITMASK(47, 47) >> +#define IODA2_IVT_PE PPC_BITMASK(48, 63) >> + >> +/* TVT */ >> +#define IODA2_TVT_TABLE_ADDR PPC_BITMASK(0, 47) >> +#define IODA2_TVT_NUM_LEVELS PPC_BITMASK(48, 50) >> +#define IODA2_TVE_1_LEVEL 0 >> +#define IODA2_TVE_2_LEVELS 1 >> +#define IODA2_TVE_3_LEVELS 2 >> +#define IODA2_TVE_4_LEVELS 3 >> +#define IODA2_TVE_5_LEVELS 4 >> +#define IODA2_TVT_TCE_TABLE_SIZE PPC_BITMASK(51, 55) >> +#define IODA2_TVT_IO_PSIZE PPC_BITMASK(59, 63) >> + >> +/* PESTA */ >> +#define IODA2_PESTA_MMIO_FROZEN PPC_BIT(0) >> + >> +/* PESTB */ >> +#define IODA2_PESTB_DMA_STOPPED PPC_BIT(0) >> + >> +/* M32DT */ >> +#define IODA2_M32DT_PE PPC_BITMASK(8, 15) >> + >> +/* M64BT */ >> +#define IODA2_M64BT_ENABLE PPC_BIT(0) >> +#define IODA2_M64BT_SINGLE_PE PPC_BIT(1) >> +#define IODA2_M64BT_BASE PPC_BITMASK(2, 31) >> +#define IODA2_M64BT_MASK PPC_BITMASK(34, 63) >> +#define IODA2_M64BT_SINGLE_BASE PPC_BITMASK(2, 26) >> +#define IODA2_M64BT_PE_HI PPC_BITMASK(27, 31) >> +#define IODA2_M64BT_SINGLE_MASK PPC_BITMASK(34, 58) >> +#define IODA2_M64BT_PE_LOW PPC_BITMASK(59, 63) >> + >> +/* >> + * IODA2 in-memory tables >> + */ >> + >> +/* PEST >> + * >> + * 2x8 bytes entries, PEST0 and PEST1 >> + */ >> + >> +#define IODA2_PEST0_MMIO_CAUSE PPC_BIT(2) >> +#define IODA2_PEST0_CFG_READ PPC_BIT(3) >> +#define IODA2_PEST0_CFG_WRITE PPC_BIT(4) >> +#define IODA2_PEST0_TTYPE PPC_BITMASK(5, 7) >> +#define PEST_TTYPE_DMA_WRITE 0 >> +#define PEST_TTYPE_MSI 1 >> +#define PEST_TTYPE_DMA_READ 2 >> +#define PEST_TTYPE_DMA_READ_RESP 3 >> +#define PEST_TTYPE_MMIO_LOAD 4 >> +#define PEST_TTYPE_MMIO_STORE 5 >> +#define PEST_TTYPE_OTHER 7 >> +#define IODA2_PEST0_CA_RETURN PPC_BIT(8) >> +#define IODA2_PEST0_UTL_RTOS_TIMEOUT PPC_BIT(8) /* Same bit as CA return */ >> +#define IODA2_PEST0_UR_RETURN PPC_BIT(9) >> +#define IODA2_PEST0_UTL_NONFATAL PPC_BIT(10) >> +#define IODA2_PEST0_UTL_FATAL PPC_BIT(11) >> +#define IODA2_PEST0_PARITY_UE PPC_BIT(13) >> +#define IODA2_PEST0_UTL_CORRECTABLE PPC_BIT(14) >> +#define IODA2_PEST0_UTL_INTERRUPT PPC_BIT(15) >> +#define IODA2_PEST0_MMIO_XLATE PPC_BIT(16) >> +#define IODA2_PEST0_IODA2_ERROR PPC_BIT(16) /* Same bit as MMIO xlate */ >> +#define IODA2_PEST0_TCE_PAGE_FAULT PPC_BIT(18) >> +#define IODA2_PEST0_TCE_ACCESS_FAULT PPC_BIT(19) >> +#define IODA2_PEST0_DMA_RESP_TIMEOUT PPC_BIT(20) >> +#define IODA2_PEST0_AIB_SIZE_INVALID PPC_BIT(21) >> +#define IODA2_PEST0_LEM_BIT PPC_BITMASK(26, 31) >> +#define IODA2_PEST0_RID PPC_BITMASK(32, 47) >> +#define IODA2_PEST0_MSI_DATA PPC_BITMASK(48, 63) >> + >> +#define IODA2_PEST1_FAIL_ADDR PPC_BITMASK(3, 63) >> + >> + >> +#endif /* PCI_HOST_PNV_PHB3_REGS_H */ >> diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h >> index 86d5f54e5459..61af65fc0b76 100644 >> --- a/include/hw/ppc/pnv.h >> +++ b/include/hw/ppc/pnv.h >> @@ -25,6 +25,7 @@ >> #include "hw/ppc/pnv_lpc.h" >> #include "hw/ppc/pnv_psi.h" >> #include "hw/ppc/pnv_occ.h" >> +#include "hw/pci-host/pnv_phb3.h" >> >> #define TYPE_PNV_CHIP "pnv-chip" >> #define PNV_CHIP(obj) OBJECT_CHECK(PnvChip, (obj), TYPE_PNV_CHIP) >> @@ -72,6 +73,9 @@ typedef struct Pnv8Chip { >> PnvLpcController lpc; >> PnvPsi psi; >> PnvOCC occ; >> + >> +#define PNV8_CHIP_PHB3_MAX 4 >> + PnvPHB3 phbs[PNV8_CHIP_PHB3_MAX]; >> } Pnv8Chip; >> >> #define TYPE_PNV9_CHIP "pnv9-chip" >> @@ -92,6 +96,7 @@ typedef struct PnvChipClass { >> PnvChipType chip_type; >> uint64_t chip_cfam_id; >> uint64_t cores_mask; >> + uint32_t num_phbs; > > You can probably drop this if you instead have max possible PBCQs > always present, which may or may not have PHBs instantiated. yes. > >> hwaddr xscom_base; >> >> @@ -171,6 +176,23 @@ static inline bool pnv_is_power9(PnvMachineState *pnv) >> return pnv_chip_is_power9(pnv->chips[0]); >> } >> >> +/* >> + * This is used by devices created on the command line to find a chip >> + * on which to attach to. >> + */ >> +static inline PnvChip *pnv_get_chip(PnvMachineState *pnv, uint chip_id) >> +{ >> + int i; >> + >> + for (i = 0; i < pnv->num_chips; i++) { >> + PnvChip *chip = pnv->chips[i]; >> + if (chip->chip_id == chip_id) { >> + return chip; >> + } >> + } >> + return NULL; >> +} >> + >> #define PNV_FDT_ADDR 0x01000000 >> #define PNV_TIMEBASE_FREQ 512000000ULL >> >> diff --git a/include/hw/ppc/pnv_xscom.h b/include/hw/ppc/pnv_xscom.h >> index 255b26a5aaf6..e7ebffa5cf8a 100644 >> --- a/include/hw/ppc/pnv_xscom.h >> +++ b/include/hw/ppc/pnv_xscom.h >> @@ -73,6 +73,15 @@ typedef struct PnvXScomInterfaceClass { >> #define PNV_XSCOM_OCC_BASE 0x0066000 >> #define PNV_XSCOM_OCC_SIZE 0x6000 >> >> +#define PNV_XSCOM_PBCQ_NEST_BASE 0x2012000 >> +#define PNV_XSCOM_PBCQ_NEST_SIZE 0x46 >> + >> +#define PNV_XSCOM_PBCQ_PCI_BASE 0x9012000 >> +#define PNV_XSCOM_PBCQ_PCI_SIZE 0x15 >> + >> +#define PNV_XSCOM_PBCQ_SPCI_BASE 0x9013c00 >> +#define PNV_XSCOM_PBCQ_SPCI_SIZE 0x5 >> + >> extern void pnv_xscom_realize(PnvChip *chip, Error **errp); >> extern int pnv_dt_xscom(PnvChip *chip, void *fdt, int offset); >> >> diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h >> index 6ac8a9392da6..966a996c2eac 100644 >> --- a/include/hw/ppc/xics.h >> +++ b/include/hw/ppc/xics.h >> @@ -194,6 +194,7 @@ void icp_set_mfrr(ICPState *icp, uint8_t mfrr); >> uint32_t icp_accept(ICPState *ss); >> uint32_t icp_ipoll(ICPState *ss, uint32_t *mfrr); >> void icp_eoi(ICPState *icp, uint32_t xirr); >> +void icp_irq(ICSState *ics, int server, int nr, uint8_t priority); >> >> void ics_simple_write_xive(ICSState *ics, int nr, int server, >> uint8_t priority, uint8_t saved_priority); >> diff --git a/hw/intc/xics.c b/hw/intc/xics.c >> index c90c893228dc..30fd9eb47e90 100644 >> --- a/hw/intc/xics.c >> +++ b/hw/intc/xics.c >> @@ -230,7 +230,7 @@ void icp_eoi(ICPState *icp, uint32_t xirr) >> } >> } >> >> -static void icp_irq(ICSState *ics, int server, int nr, uint8_t priority) >> +void icp_irq(ICSState *ics, int server, int nr, uint8_t priority) >> { >> ICPState *icp = xics_icp_get(ics->xics, server); >> >> diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c >> new file mode 100644 >> index 000000000000..67f6dd94a705 >> --- /dev/null >> +++ b/hw/pci-host/pnv_phb3.c >> @@ -0,0 +1,1146 @@ >> +/* >> + * QEMU PowerPC PowerNV PHB3 model >> + * >> + * Copyright (c) 2014-2018, IBM Corporation. >> + * >> + * This code is licensed under the GPL version 2 or later. See the >> + * COPYING file in the top-level directory. >> + */ >> +#include "qemu/osdep.h" >> +#include "qemu/log.h" >> +#include "qapi/visitor.h" >> +#include "qapi/error.h" >> +#include "qemu-common.h" >> +#include "target/ppc/cpu.h" >> +#include "hw/pci-host/pnv_phb3_regs.h" >> +#include "hw/pci-host/pnv_phb3.h" >> +#include "hw/pci/pcie_host.h" >> +#include "hw/pci/pcie_port.h" >> +#include "hw/ppc/pnv.h" >> + >> +#define phb3_error(fmt, ...) \ >> + qemu_log_mask(LOG_GUEST_ERROR, "phb3: " fmt "\n", ## __VA_ARGS__) >> + >> + >> +static PCIDevice *pnv_phb3_find_cfg_dev(PnvPHB3 *phb) >> +{ >> + PCIHostState *pci = PCI_HOST_BRIDGE(phb); >> + uint64_t addr = phb->regs[PHB_CONFIG_ADDRESS >> 3]; >> + uint8_t bus, devfn; >> + >> + if (!(addr >> 63)) { >> + return NULL; >> + } >> + bus = (addr >> 52) & 0xff; >> + devfn = (addr >> 44) & 0xff; >> + >> + return pci_find_device(pci->bus, bus, devfn); >> +} >> + >> +/* >> + * The CONFIG_DATA register expects little endian accesses, but as the >> + * region is big endian, we have to swap the value. >> + */ >> +static void pnv_phb3_config_write(PnvPHB3 *phb, unsigned off, >> + unsigned size, uint64_t val) >> +{ >> + uint32_t cfg_addr, limit; >> + PCIDevice *pdev; >> + >> + pdev = pnv_phb3_find_cfg_dev(phb); >> + if (!pdev) { >> + return; >> + } >> + cfg_addr = (phb->regs[PHB_CONFIG_ADDRESS >> 3] >> 32) & 0xffc; >> + cfg_addr |= off; >> + limit = pci_config_size(pdev); >> + if (limit <= cfg_addr) { >> + /* conventional pci device can be behind pcie-to-pci bridge. >> + 256 <= addr < 4K has no effects. */ >> + return; >> + } >> + switch (size) { >> + case 1: >> + break; >> + case 2: >> + val = bswap16(val); >> + break; >> + case 4: >> + val = bswap32(val); >> + break; >> + default: >> + g_assert_not_reached(); >> + } >> + pci_host_config_write_common(pdev, cfg_addr, limit, val, size); >> +} >> + >> +static uint64_t pnv_phb3_config_read(PnvPHB3 *phb, unsigned off, >> + unsigned size) >> +{ >> + uint32_t cfg_addr, limit; >> + PCIDevice *pdev; >> + uint64_t val; >> + >> + pdev = pnv_phb3_find_cfg_dev(phb); >> + if (!pdev) { >> + return ~0ull; >> + } >> + cfg_addr = (phb->regs[PHB_CONFIG_ADDRESS >> 3] >> 32) & 0xffc; >> + cfg_addr |= off; >> + limit = pci_config_size(pdev); >> + if (limit <= cfg_addr) { >> + /* conventional pci device can be behind pcie-to-pci bridge. >> + 256 <= addr < 4K has no effects. */ >> + return ~0ull; >> + } >> + val = pci_host_config_read_common(pdev, cfg_addr, limit, size); >> + switch (size) { >> + case 1: >> + return val; >> + case 2: >> + return bswap16(val); >> + case 4: >> + return bswap32(val); >> + default: >> + g_assert_not_reached(); >> + } >> +} >> + >> +static void pnv_phb3_check_m32(PnvPHB3 *phb) >> +{ >> + uint64_t base, start, size; >> + MemoryRegion *parent; >> + PnvPBCQState *pbcq = &phb->pbcq; >> + >> + if (phb->m32_mapped) { >> + memory_region_del_subregion(phb->mr_m32.container, &phb->mr_m32); >> + phb->m32_mapped = false; >> + } >> + >> + if (!(phb->regs[PHB_PHB3_CONFIG >> 3] & PHB_PHB3C_M32_EN)) { >> + return; >> + } >> + >> + /* Grab geometry from registers */ >> + base = phb->regs[PHB_M32_BASE_ADDR >> 3]; >> + start = phb->regs[PHB_M32_START_ADDR >> 3]; >> + size = ~(phb->regs[PHB_M32_BASE_MASK >> 3] | 0xfffc000000000000ull) + 1; >> + >> + /* Check if it matches an enabled MMIO region in the PBCQ */ >> + if (pbcq->mmio0_mapped && base >= pbcq->mmio0_base && >> + (base + size) <= (pbcq->mmio0_base + pbcq->mmio0_size)) { >> + parent = &pbcq->mmbar0; >> + base -= pbcq->mmio0_base; >> + } else if (pbcq->mmio1_mapped && base >= pbcq->mmio1_base && >> + (base + size) <= (pbcq->mmio1_base + pbcq->mmio1_size)) { >> + parent = &pbcq->mmbar1; >> + base -= pbcq->mmio1_base; >> + } else { >> + return; >> + } >> + >> + /* Create alias */ >> + memory_region_init_alias(&phb->mr_m32, OBJECT(phb), "phb3-m32", >> + &phb->pci_mmio, start, size); >> + memory_region_add_subregion(parent, base, &phb->mr_m32); >> + phb->m32_mapped = true; >> +} >> + >> +static void pnv_phb3_check_m64(PnvPHB3 *phb, uint32_t index) >> +{ >> + uint64_t base, start, size, m64; >> + MemoryRegion *parent; >> + PnvPBCQState *pbcq = &phb->pbcq; >> + >> + if (phb->m64_mapped[index]) { >> + /* Should we destroy it in RCU friendly way... ? */ > > I wouldn't think so. This should be safely under the BQL, and I don't > imagine the PHB mappings are changed often, so I doubt there's a > reason to optimize it. OK > >> + memory_region_del_subregion(phb->mr_m64[index].container, >> + &phb->mr_m64[index]); >> + phb->m64_mapped[index] = false; >> + } >> + >> + /* Get table entry */ >> + m64 = phb->ioda_M64BT[index]; >> + >> + if (!(m64 & IODA2_M64BT_ENABLE)) { >> + return; >> + } >> + >> + /* Grab geometry from registers */ >> + base = GETFIELD(IODA2_M64BT_BASE, m64) << 20; >> + if (m64 & IODA2_M64BT_SINGLE_PE) { >> + base &= ~0x1ffffffull; >> + } >> + size = GETFIELD(IODA2_M64BT_MASK, m64) << 20; >> + size |= 0xfffc000000000000ull; >> + size = ~size + 1; >> + start = base | (phb->regs[PHB_M64_UPPER_BITS >> 3]); >> + >> + /* Check if it matches an enabled MMIO region in the PBCQ */ >> + if (pbcq->mmio0_mapped && base >= pbcq->mmio0_base && >> + (base + size) <= (pbcq->mmio0_base + pbcq->mmio0_size)) { >> + parent = &pbcq->mmbar0; >> + base -= pbcq->mmio0_base; >> + } else if (pbcq->mmio1_mapped && base >= pbcq->mmio1_base && >> + (base + size) <= (pbcq->mmio1_base + pbcq->mmio1_size)) { >> + parent = &pbcq->mmbar1; >> + base -= pbcq->mmio1_base; >> + } else { >> + return; >> + } >> + >> + /* Create alias */ >> + memory_region_init_alias(&phb->mr_m64[index], OBJECT(phb), "phb3-m64", >> + &phb->pci_mmio, start, size); >> + memory_region_add_subregion(parent, base, &phb->mr_m64[index]); >> + phb->m64_mapped[index] = true; >> +} >> + >> +static void pnv_phb3_check_all_m64s(PnvPHB3 *phb) >> +{ >> + uint64_t i; >> + >> + for (i = 0; i < PNV_PHB3_NUM_M64; i++) { >> + pnv_phb3_check_m64(phb, i); >> + } >> +} >> + >> +static void pnv_phb3_lxivt_write(PnvPHB3 *phb, unsigned idx, uint64_t val) >> +{ >> + ICSState *ics = &phb->lsis; >> + uint8_t server, prio; >> + >> + phb->ioda_LXIVT[idx] = val & (IODA2_LXIVT_SERVER | >> + IODA2_LXIVT_PRIORITY | >> + IODA2_LXIVT_NODE_ID); >> + server = GETFIELD(IODA2_LXIVT_SERVER, val); >> + prio = GETFIELD(IODA2_LXIVT_PRIORITY, val); >> + >> + /* >> + * The low order 2 bits are the link pointer (Type II interrupts). >> + * Shift back to get a valid IRQ server. >> + */ >> + server >>= 2; >> + >> + ics_simple_write_xive(ics, idx, server, prio, prio); >> +} >> + >> +static uint64_t *pnv_phb3_ioda_access(PnvPHB3 *phb, >> + unsigned *out_table, unsigned *out_idx) >> +{ >> + uint64_t adreg = phb->regs[PHB_IODA_ADDR >> 3]; >> + unsigned int index = GETFIELD(PHB_IODA_AD_TADR, adreg); >> + unsigned int table = GETFIELD(PHB_IODA_AD_TSEL, adreg); >> + unsigned int mask; >> + uint64_t *tptr = NULL; >> + >> + switch (table) { >> + case IODA2_TBL_LIST: >> + tptr = phb->ioda_LIST; >> + mask = 7; > > Looks like these masks would be clearer in hex. yes. > >> + break; >> + case IODA2_TBL_LXIVT: >> + tptr = phb->ioda_LXIVT; >> + mask = 7; >> + break; >> + case IODA2_TBL_IVC_CAM: >> + case IODA2_TBL_RBA: >> + mask = 31; >> + break; >> + case IODA2_TBL_RCAM: >> + mask = 63; >> + break; >> + case IODA2_TBL_MRT: >> + mask = 7; >> + break; >> + case IODA2_TBL_PESTA: >> + case IODA2_TBL_PESTB: >> + mask = 255; >> + break; >> + case IODA2_TBL_TVT: >> + tptr = phb->ioda_TVT; >> + mask = 511; >> + break; >> + case IODA2_TBL_TCAM: >> + case IODA2_TBL_TDR: >> + mask = 63; >> + break; >> + case IODA2_TBL_M64BT: >> + tptr = phb->ioda_M64BT; >> + mask = 15; >> + break; >> + case IODA2_TBL_M32DT: >> + tptr = phb->ioda_MDT; >> + mask = 255; >> + break; >> + case IODA2_TBL_PEEV: >> + tptr = phb->ioda_PEEV; >> + mask = 3; >> + break; >> + default: >> + phb3_error("invalid IODA table %d", table); >> + return NULL; >> + } >> + index &= mask; >> + if (out_idx) { >> + *out_idx = index; >> + } >> + if (out_table) { >> + *out_table = table; >> + } >> + if (adreg & PHB_IODA_AD_AUTOINC) { >> + index = (index + 1) & mask; >> + adreg = SETFIELD(PHB_IODA_AD_TADR, adreg, index); >> + } >> + if (tptr) { >> + tptr += index; >> + } >> + phb->regs[PHB_IODA_ADDR >> 3] = adreg; >> + return tptr; >> +} >> + >> +static uint64_t pnv_phb3_ioda_read(PnvPHB3 *phb) >> +{ >> + unsigned table; >> + uint64_t *tptr; >> + >> + tptr = pnv_phb3_ioda_access(phb, &table, NULL); >> + if (!tptr) { >> + /* Return 0 on unsupported tables, not ff's */ >> + return 0; >> + } >> + return *tptr; >> +} >> + >> +static void pnv_phb3_ioda_write(PnvPHB3 *phb, uint64_t val) >> +{ >> + unsigned table, idx; >> + uint64_t *tptr; >> + >> + tptr = pnv_phb3_ioda_access(phb, &table, &idx); >> + if (!tptr) { >> + return; >> + } >> + >> + /* Handle side effects */ >> + switch (table) { >> + case IODA2_TBL_LXIVT: >> + pnv_phb3_lxivt_write(phb, idx, val); >> + break; >> + case IODA2_TBL_M64BT: >> + *tptr = val; >> + pnv_phb3_check_m64(phb, idx); >> + break; >> + default: >> + *tptr = val; >> + } >> +} >> + >> +/* This is called whenever the PHB LSI, MSI source ID register or >> + * the PBCQ irq filters are written. >> + */ >> +void pnv_phb3_remap_irqs(PnvPHB3 *phb) >> +{ >> + ICSState *ics = &phb->lsis; >> + uint32_t local, global, count, mask, comp; >> + uint64_t baren; >> + PnvPBCQState *pbcq = &phb->pbcq; >> + >> + /* First check if we are enabled. Unlike real HW we don't separate TX and RX >> + * so we enable if both are set >> + */ >> + baren = pbcq->nest_regs[PBCQ_NEST_BAR_EN]; >> + if (!(baren & PBCQ_NEST_BAR_EN_IRSN_RX) || >> + !(baren & PBCQ_NEST_BAR_EN_IRSN_TX)) { >> + ics->offset = 0; >> + return; >> + } >> + >> + /* Grab local LSI source ID */ >> + local = GETFIELD(PHB_LSI_SRC_ID, phb->regs[PHB_LSI_SOURCE_ID >> 3]) << 3; >> + >> + /* Grab global one and compare */ >> + global = GETFIELD(PBCQ_NEST_LSI_SRC, >> + pbcq->nest_regs[PBCQ_NEST_LSI_SRC_ID]) << 3; >> + if (global != local) { >> + /* This happens during initialization, let's come back when we >> + * are properly configured >> + */ >> + ics->offset = 0; >> + return; >> + } >> + >> + /* Get the base on the powerbus */ >> + comp = GETFIELD(PBCQ_NEST_IRSN_COMP, >> + pbcq->nest_regs[PBCQ_NEST_IRSN_COMPARE]); >> + mask = GETFIELD(PBCQ_NEST_IRSN_COMP, >> + pbcq->nest_regs[PBCQ_NEST_IRSN_MASK]); >> + count = ((~mask) + 1) & 0x7ffff; >> + phb->total_irq = count; >> + >> + /* Sanity checks */ >> + if ((global + PNV_PHB3_NUM_LSI) > count) { >> + phb3_error("LSIs out of reach: LSI base=%d total irq=%d", global, >> + count); >> + } >> + >> + if (count > 2048) { >> + phb3_error("More interrupts than supported: %d", count); >> + } >> + >> + if ((comp & mask) != comp) { >> + phb3_error("IRQ compare bits not in mask: comp=0x%x mask=0x%x", >> + comp, mask); >> + comp &= mask; >> + } >> + /* Setup LSI offset */ >> + ics->offset = comp + global; >> + >> + /* Setup MSI offset */ >> + pnv_phb3_msi_update_config(&phb->msis, comp, count - PNV_PHB3_NUM_LSI); >> +} >> + >> +static void pnv_phb3_lsi_src_id_write(PnvPHB3 *phb, uint64_t val) >> +{ >> + /* Sanitize content */ >> + val &= PHB_LSI_SRC_ID; >> + phb->regs[PHB_LSI_SOURCE_ID >> 3] = val; >> + pnv_phb3_remap_irqs(phb); >> +} >> + >> +static void pnv_phb3_rtc_invalidate(PnvPHB3 *phb, uint64_t val) >> +{ >> + PnvPhb3DMASpace *ds; >> + >> + /* Always invalidate all for now ... */ >> + QLIST_FOREACH(ds, &phb->dma_spaces, list) { >> + ds->pe_num = PHB_INVALID_PE; >> + } >> +} >> + >> + >> +static void pnv_phb3_update_msi_regions(PnvPhb3DMASpace *ds) >> +{ >> + uint64_t cfg = ds->phb->regs[PHB_PHB3_CONFIG >> 3]; >> + >> + if (cfg & PHB_PHB3C_32BIT_MSI_EN) { >> + if (!ds->msi32_mapped) { >> + memory_region_add_subregion(MEMORY_REGION(&ds->dma_mr), >> + 0xffff0000, &ds->msi32_mr); >> + ds->msi32_mapped = true; >> + } >> + } else { >> + if (ds->msi32_mapped) { >> + memory_region_del_subregion(MEMORY_REGION(&ds->dma_mr), >> + &ds->msi32_mr); >> + ds->msi32_mapped = false; >> + } >> + } >> + >> + if (cfg & PHB_PHB3C_64BIT_MSI_EN) { >> + if (!ds->msi64_mapped) { >> + memory_region_add_subregion(MEMORY_REGION(&ds->dma_mr), >> + (1ull << 60), &ds->msi64_mr); >> + ds->msi64_mapped = true; >> + } >> + } else { >> + if (ds->msi64_mapped) { >> + memory_region_del_subregion(MEMORY_REGION(&ds->dma_mr), >> + &ds->msi64_mr); >> + ds->msi64_mapped = false; >> + } >> + } >> +} >> + >> +static void pnv_phb3_update_all_msi_regions(PnvPHB3 *phb) >> +{ >> + PnvPhb3DMASpace *ds; >> + >> + QLIST_FOREACH(ds, &phb->dma_spaces, list) { >> + pnv_phb3_update_msi_regions(ds); >> + } >> +} >> + >> +void pnv_phb3_reg_write(void *opaque, hwaddr off, uint64_t val, unsigned size) >> +{ >> + PnvPHB3 *phb = opaque; >> + bool changed; >> + >> + /* Special case configuration data */ >> + if ((off & 0xfffc) == PHB_CONFIG_DATA) { >> + pnv_phb3_config_write(phb, off & 0x3, size, val); >> + return; >> + } >> + >> + /* Other registers are 64-bit only */ >> + if (size != 8 || off & 0x7) { >> + phb3_error("Invalid register access, offset: 0x%"PRIx64" size: %d", >> + off, size); >> + return; >> + } >> + >> + /* Handle masking */ >> + switch (off) { >> + case PHB_M64_UPPER_BITS: >> + val &= 0xfffc000000000000ull; >> + break; >> + } >> + >> + /* Record whether it changed */ >> + changed = phb->regs[off >> 3] != val; >> + >> + /* Store in register cache first */ >> + phb->regs[off >> 3] = val; >> + >> + /* Handle side effects */ >> + switch (off) { >> + case PHB_PHB3_CONFIG: >> + if (changed) { >> + pnv_phb3_update_all_msi_regions(phb); >> + } >> + /* fall through */ >> + case PHB_M32_BASE_ADDR: >> + case PHB_M32_BASE_MASK: >> + case PHB_M32_START_ADDR: >> + if (changed) { >> + pnv_phb3_check_m32(phb); >> + } >> + break; >> + case PHB_M64_UPPER_BITS: >> + if (changed) { >> + pnv_phb3_check_all_m64s(phb); >> + } >> + break; >> + case PHB_LSI_SOURCE_ID: >> + if (changed) { >> + pnv_phb3_lsi_src_id_write(phb, val); >> + } >> + break; >> + >> + /* IODA table accesses */ >> + case PHB_IODA_DATA0: >> + pnv_phb3_ioda_write(phb, val); >> + break; >> + >> + /* RTC invalidation */ >> + case PHB_RTC_INVALIDATE: >> + pnv_phb3_rtc_invalidate(phb, val); >> + break; >> + >> + /* FFI request */ >> + case PHB_FFI_REQUEST: >> + pnv_phb3_msi_ffi(&phb->msis, val); >> + break; >> + >> + /* Silent simple writes */ >> + case PHB_CONFIG_ADDRESS: >> + case PHB_IODA_ADDR: >> + case PHB_TCE_KILL: >> + case PHB_TCE_SPEC_CTL: >> + case PHB_PEST_BAR: >> + case PHB_PELTV_BAR: >> + case PHB_RTT_BAR: >> + case PHB_RBA_BAR: >> + case PHB_IVT_BAR: >> + case PHB_FFI_LOCK: >> + break; >> + >> + /* Noise on anything else */ >> + default: >> + qemu_log_mask(LOG_UNIMP, "phb3: reg_write 0x%"PRIx64"=%"PRIx64"\n", >> + off, val); >> + } >> +} >> + >> +uint64_t pnv_phb3_reg_read(void *opaque, hwaddr off, unsigned size) >> +{ >> + PnvPHB3 *phb = opaque; >> + uint64_t val; >> + >> + if ((off & 0xfffc) == PHB_CONFIG_DATA) { >> + return pnv_phb3_config_read(phb, off & 0x3, size); >> + } >> + >> + /* Other registers are 64-bit only */ >> + if (size != 8 || off & 0x7) { >> + phb3_error("Invalid register access, offset: 0x%"PRIx64" size: %d", >> + off, size); >> + return ~0ull; >> + } >> + >> + /* Default read from cache */ >> + val = phb->regs[off >> 3]; >> + >> + switch (off) { >> + /* Simulate venice DD2.0 */ >> + case PHB_VERSION: >> + return 0x000000a300000005ull; >> + case PHB_PCIE_SYSTEM_CONFIG: >> + return 0x441100fc30000000; >> + >> + /* IODA table accesses */ >> + case PHB_IODA_DATA0: >> + return pnv_phb3_ioda_read(phb); >> + >> + /* Link training always appears trained */ >> + case PHB_PCIE_DLP_TRAIN_CTL: >> + return PHB_PCIE_DLP_INBAND_PRESENCE | PHB_PCIE_DLP_TC_DL_LINKACT; >> + >> + /* FFI Lock */ >> + case PHB_FFI_LOCK: >> + /* Set lock and return previous value */ >> + phb->regs[off >> 3] |= PHB_FFI_LOCK_STATE; >> + return val; >> + >> + /* Silent simple reads */ >> + case PHB_PHB3_CONFIG: >> + case PHB_M32_BASE_ADDR: >> + case PHB_M32_BASE_MASK: >> + case PHB_M32_START_ADDR: >> + case PHB_CONFIG_ADDRESS: >> + case PHB_IODA_ADDR: >> + case PHB_RTC_INVALIDATE: >> + case PHB_TCE_KILL: >> + case PHB_TCE_SPEC_CTL: >> + case PHB_PEST_BAR: >> + case PHB_PELTV_BAR: >> + case PHB_RTT_BAR: >> + case PHB_RBA_BAR: >> + case PHB_IVT_BAR: >> + case PHB_M64_UPPER_BITS: >> + break; >> + >> + /* Noise on anything else */ >> + default: >> + qemu_log_mask(LOG_UNIMP, "phb3: reg_read 0x%"PRIx64"=%"PRIx64"\n", >> + off, val); >> + } >> + return val; >> +} >> + >> +static const MemoryRegionOps pnv_phb3_reg_ops = { >> + .read = pnv_phb3_reg_read, >> + .write = pnv_phb3_reg_write, >> + .valid.min_access_size = 1, >> + .valid.max_access_size = 8, >> + .impl.min_access_size = 1, >> + .impl.max_access_size = 8, >> + .endianness = DEVICE_BIG_ENDIAN, >> +}; >> + >> +static int pnv_phb3_map_irq(PCIDevice *pci_dev, int irq_num) >> +{ >> + /* Check that out properly ... */ >> + return irq_num & 3; >> +} >> + >> +static void pnv_phb3_set_irq(void *opaque, int irq_num, int level) >> +{ >> + PnvPHB3 *phb = opaque; >> + >> + /* LSI only ... */ >> + if (irq_num > 3) { >> + phb3_error("Unknown IRQ to set %d", irq_num); >> + } >> + qemu_set_irq(phb->lsis.qirqs[irq_num], level); >> +} >> + >> +static bool pnv_phb3_resolve_pe(PnvPhb3DMASpace *ds) >> +{ >> + uint64_t rtt, addr; >> + uint16_t rte; >> + int bus_num; >> + >> + /* Already resolved ? */ >> + if (ds->pe_num != PHB_INVALID_PE) { >> + return true; >> + } >> + >> + /* We need to lookup the RTT */ >> + rtt = ds->phb->regs[PHB_RTT_BAR >> 3]; >> + if (!(rtt & PHB_RBA_BAR_ENABLE)) { >> + phb3_error("DMA with RTT BAR disabled !"); >> + /* Set error bits ? fence ? ... */ >> + return false; >> + } >> + >> + /* Read RTE */ >> + bus_num = pci_bus_num(ds->bus); >> + addr = rtt & PHB_RTT_BASE_ADDRESS_MASK; >> + addr += 2 * ((bus_num << 8) | ds->devfn); >> + if (dma_memory_read(&address_space_memory, addr, &rte, sizeof(rte))) { >> + phb3_error("Failed to read RTT entry at 0x%"PRIx64, addr); >> + /* Set error bits ? fence ? ... */ >> + return false; >> + } >> + rte = be16_to_cpu(rte); >> + >> + /* Fail upon reading of invalid PE# */ >> + if (rte >= PNV_PHB3_NUM_PE) { >> + phb3_error("RTE for RID 0x%x invalid (%04x", ds->devfn, rte); >> + /* Set error bits ? fence ? ... */ >> + return false; >> + } >> + ds->pe_num = rte; >> + return true; >> +} >> + >> +static void pnv_phb3_translate_tve(PnvPhb3DMASpace *ds, hwaddr addr, >> + bool is_write, uint64_t tve, >> + IOMMUTLBEntry *tlb) >> +{ >> + uint64_t tta = GETFIELD(IODA2_TVT_TABLE_ADDR, tve); >> + int32_t lev = GETFIELD(IODA2_TVT_NUM_LEVELS, tve); >> + uint32_t tts = GETFIELD(IODA2_TVT_TCE_TABLE_SIZE, tve); >> + uint32_t tps = GETFIELD(IODA2_TVT_IO_PSIZE, tve); >> + >> + /* Invalid levels */ >> + if (lev > 4) { >> + phb3_error("Invalid #levels in TVE %d", lev); >> + return; >> + } >> + >> + /* IO Page Size of 0 means untranslated, else use TCEs */ >> + if (tps == 0) { >> + /* We only support non-translate in top window >> + * XXX FIX THAT, Venice/Murano support it on bottom window >> + * above 4G and Naples suports it on everything >> + */ >> + if (!(tve & PPC_BIT(51))) { >> + phb3_error("xlate for invalid non-translate TVE"); >> + return; >> + } >> + /* XXX Handle boundaries */ >> + >> + /* XXX Use 4k pages like q35 ... for now */ >> + tlb->iova = addr & 0xfffffffffffff000ull; >> + tlb->translated_addr = addr & 0x0003fffffffff000ull; >> + tlb->addr_mask = 0xfffull; >> + tlb->perm = IOMMU_RW; >> + } else { >> + uint32_t tce_shift, tbl_shift, sh; >> + uint64_t base, taddr, tce, tce_mask; >> + >> + /* TVE disabled ? */ >> + if (tts == 0) { >> + phb3_error("xlate for invalid translated TVE"); >> + return; >> + } >> + >> + /* Address bits per bottom level TCE entry */ >> + tce_shift = tps + 11; >> + >> + /* Address bits per table level */ >> + tbl_shift = tts + 8; >> + >> + /* Top level table base address */ >> + base = tta << 12; >> + >> + /* Total shift to first level */ >> + sh = tbl_shift * lev + tce_shift; >> + >> + /* XXX Multi-level untested */ >> + while ((lev--) >= 0) { >> + /* Grab the TCE address */ >> + taddr = base | (((addr >> sh) & ((1ul << tbl_shift) - 1)) << 3); >> + if (dma_memory_read(&address_space_memory, taddr, &tce, >> + sizeof(tce))) { >> + phb3_error("Failed to read TCE at 0x%"PRIx64, taddr); >> + return; >> + } >> + tce = be64_to_cpu(tce); >> + >> + /* Check permission for indirect TCE */ >> + if ((lev >= 0) && !(tce & 3)) { >> + phb3_error("Invalid indirect TCE at 0x%"PRIx64, taddr); >> + phb3_error(" xlate %"PRIx64":%c TVE=%"PRIx64, addr, >> + is_write ? 'W' : 'R', tve); >> + phb3_error(" tta=%"PRIx64" lev=%d tts=%d tps=%d", >> + tta, lev, tts, tps); >> + return; >> + } >> + sh -= tbl_shift; >> + base = tce & ~0xfffull; >> + } >> + >> + /* We exit the loop with TCE being the final TCE */ >> + tce_mask = ~((1ull << tce_shift) - 1); >> + tlb->iova = addr & tce_mask; >> + tlb->translated_addr = tce & tce_mask; >> + tlb->addr_mask = ~tce_mask; >> + tlb->perm = tce & 3; >> + if ((is_write & !(tce & 2)) || ((!is_write) && !(tce & 1))) { >> + phb3_error("TCE access fault at 0x%"PRIx64, taddr); >> + phb3_error(" xlate %"PRIx64":%c TVE=%"PRIx64, addr, >> + is_write ? 'W' : 'R', tve); >> + phb3_error(" tta=%"PRIx64" lev=%d tts=%d tps=%d", >> + tta, lev, tts, tps); >> + } >> + } >> +} >> + >> +static IOMMUTLBEntry pnv_phb3_translate_iommu(IOMMUMemoryRegion *iommu, >> + hwaddr addr, >> + IOMMUAccessFlags flag, >> + int iommu_idx) >> +{ >> + PnvPhb3DMASpace *ds = container_of(iommu, PnvPhb3DMASpace, dma_mr); >> + int tve_sel; >> + uint64_t tve, cfg; >> + IOMMUTLBEntry ret = { >> + .target_as = &address_space_memory, >> + .iova = addr, >> + .translated_addr = 0, >> + .addr_mask = ~(hwaddr)0, >> + .perm = IOMMU_NONE, >> + }; >> + >> + /* Resolve PE# */ >> + if (!pnv_phb3_resolve_pe(ds)) { >> + phb3_error("Failed to resolve PE# for bus @%p (%d) devfn 0x%x", >> + ds->bus, pci_bus_num(ds->bus), ds->devfn); >> + return ret; >> + } >> + >> + /* Check top bits */ >> + switch (addr >> 60) { >> + case 00: >> + /* DMA or 32-bit MSI ? */ >> + cfg = ds->phb->regs[PHB_PHB3_CONFIG >> 3]; >> + if ((cfg & PHB_PHB3C_32BIT_MSI_EN) && >> + ((addr & 0xffffffffffff0000ull) == 0xffff0000ull)) { >> + phb3_error("xlate on 32-bit MSI region"); >> + return ret; >> + } >> + /* Choose TVE XXX Use PHB3 Control Register */ >> + tve_sel = (addr >> 59) & 1; >> + tve = ds->phb->ioda_TVT[ds->pe_num * 2 + tve_sel]; >> + pnv_phb3_translate_tve(ds, addr, flag & IOMMU_WO, tve, &ret); >> + break; >> + case 01: >> + phb3_error("xlate on 64-bit MSI region"); >> + break; >> + default: >> + phb3_error("xlate on unsupported address 0x%"PRIx64, addr); >> + } >> + return ret; >> +} >> + >> +#define TYPE_PNV_PHB3_IOMMU_MEMORY_REGION "pnv-phb3-iommu-memory-region" >> +#define PNV_PHB3_IOMMU_MEMORY_REGION(obj) \ >> + OBJECT_CHECK(IOMMUMemoryRegion, (obj), TYPE_PNV_PHB3_IOMMU_MEMORY_REGION) >> + >> +static void pnv_phb3_iommu_memory_region_class_init(ObjectClass *klass, >> + void *data) >> +{ >> + IOMMUMemoryRegionClass *imrc = IOMMU_MEMORY_REGION_CLASS(klass); >> + >> + imrc->translate = pnv_phb3_translate_iommu; >> +} >> + >> +static const TypeInfo pnv_phb3_iommu_memory_region_info = { >> + .parent = TYPE_IOMMU_MEMORY_REGION, >> + .name = TYPE_PNV_PHB3_IOMMU_MEMORY_REGION, >> + .class_init = pnv_phb3_iommu_memory_region_class_init, >> +}; >> + >> +/* >> + * MSI/MSIX memory region implementation. >> + * The handler handles both MSI and MSIX. >> + */ >> +static void pnv_phb3_msi_write(void *opaque, hwaddr addr, >> + uint64_t data, unsigned size) >> +{ >> + PnvPhb3DMASpace *ds = opaque; >> + >> + /* Resolve PE# */ >> + if (!pnv_phb3_resolve_pe(ds)) { >> + phb3_error("Failed to resolve PE# for bus @%p (%d) devfn 0x%x", >> + ds->bus, pci_bus_num(ds->bus), ds->devfn); >> + return; >> + } >> + >> + pnv_phb3_msi_send(&ds->phb->msis, addr, data, ds->pe_num); >> +} >> + >> +/* There is no .read as the read result is undefined by PCI spec */ >> +static uint64_t pnv_phb3_msi_read(void *opaque, hwaddr addr, unsigned size) >> +{ >> + phb3_error("invalid read @ 0x%" HWADDR_PRIx, addr); >> + return -1; >> +} >> + >> +static const MemoryRegionOps pnv_phb3_msi_ops = { >> + .read = pnv_phb3_msi_read, >> + .write = pnv_phb3_msi_write, >> + .endianness = DEVICE_LITTLE_ENDIAN >> +}; >> + >> +static AddressSpace *pnv_phb3_dma_iommu(PCIBus *bus, void *opaque, int devfn) >> +{ >> + PnvPHB3 *phb = opaque; >> + PnvPhb3DMASpace *ds; >> + >> + QLIST_FOREACH(ds, &phb->dma_spaces, list) { >> + if (ds->bus == bus && ds->devfn == devfn) { >> + break; >> + } >> + } >> + >> + if (ds == NULL) { >> + ds = g_malloc0(sizeof(PnvPhb3DMASpace)); >> + ds->bus = bus; >> + ds->devfn = devfn; >> + ds->pe_num = PHB_INVALID_PE; >> + ds->phb = phb; >> + memory_region_init_iommu(&ds->dma_mr, sizeof(ds->dma_mr), >> + TYPE_PNV_PHB3_IOMMU_MEMORY_REGION, >> + OBJECT(phb), "phb3_iommu", UINT64_MAX); >> + address_space_init(&ds->dma_as, MEMORY_REGION(&ds->dma_mr), >> + "phb3_iommu"); >> + memory_region_init_io(&ds->msi32_mr, OBJECT(phb), &pnv_phb3_msi_ops, >> + ds, "msi32", 0x10000); >> + memory_region_init_io(&ds->msi64_mr, OBJECT(phb), &pnv_phb3_msi_ops, >> + ds, "msi64", 0x100000); >> + pnv_phb3_update_msi_regions(ds); >> + >> + QLIST_INSERT_HEAD(&phb->dma_spaces, ds, list); >> + } >> + return &ds->dma_as; >> +} >> + >> +static void pnv_phb3_instance_init(Object *obj) >> +{ >> + PnvPHB3 *phb = PNV_PHB3(obj); >> + >> + QLIST_INIT(&phb->dma_spaces); >> + >> + /* LSI sources */ >> + object_initialize(&phb->lsis, sizeof(phb->lsis), TYPE_ICS_SIMPLE); >> + object_property_add_child(obj, "lsi", OBJECT(&phb->lsis), NULL); >> + >> + /* Default init ... will be fixed by HW inits */ >> + phb->lsis.offset = 0; >> + >> + /* MSI sources */ >> + object_initialize(&phb->msis, sizeof(phb->msis), TYPE_PHB3_MSI); >> + object_property_add_const_link(OBJECT(&phb->msis), "phb", obj, >> + &error_abort); >> + object_property_add_child(obj, "msi", OBJECT(&phb->msis), NULL); >> + >> + /* Power Bus Common Queue */ >> + object_initialize(&phb->pbcq, sizeof(phb->pbcq), TYPE_PNV_PBCQ); >> + object_property_add_const_link(OBJECT(&phb->pbcq), "phb", obj, >> + &error_abort); >> + object_property_add_child(obj, "pbcq", OBJECT(&phb->pbcq), NULL); >> + >> + /* Root Port */ >> + object_initialize(&phb->root, sizeof(phb->root), TYPE_PNV_PHB3_ROOT_PORT); >> + object_property_add_child(obj, "root", OBJECT(&phb->root), NULL); >> + qdev_prop_set_int32(DEVICE(&phb->root), "addr", PCI_DEVFN(0, 0)); >> + qdev_prop_set_bit(DEVICE(&phb->root), "multifunction", false); >> +} >> + >> +static void pnv_phb3_realize(DeviceState *dev, Error **errp) >> +{ >> + PnvPHB3 *phb = PNV_PHB3(dev); >> + PCIHostState *pci = PCI_HOST_BRIDGE(dev); >> + PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine()); >> + Error *local_err = NULL; >> + int i; >> + >> + if (!pnv_get_chip(pnv, phb->chip_id)) { >> + error_setg(errp, "invalid chip id: %d", phb->chip_id); >> + return; >> + } > > If we go with a PBCQ-first model, I think we can do this matchup in a > more QOM-ishly correct way by searching for an appropriate PBCQ > instance, rather than directly looking up chip number. At least > until/unless I figure out how to do it with a pseudo-bus. yes. that is an ugly check. >> + >> + if (phb->phb_id >= PNV8_CHIP_PHB3_MAX) { >> + error_setg(errp, "invalid PHB index: %d", phb->phb_id); >> + return; >> + } >> + >> + /* LSI sources */ >> + object_property_add_const_link(OBJECT(&phb->lsis), "xics", OBJECT(pnv), >> + &error_abort); >> + object_property_set_int(OBJECT(&phb->lsis), PNV_PHB3_NUM_LSI, "nr-irqs", >> + &error_abort); >> + object_property_set_bool(OBJECT(&phb->lsis), true, "realized", &local_err); >> + if (local_err) { >> + error_propagate(errp, local_err); >> + return; >> + } >> + >> + for (i = 0; i < phb->lsis.nr_irqs; i++) { >> + ics_set_irq_type(&phb->lsis, i, true); >> + } >> + >> + /* MSI sources */ >> + object_property_add_const_link(OBJECT(&phb->msis), "xics", OBJECT(pnv), >> + &error_abort); >> + object_property_set_int(OBJECT(&phb->msis), PHB3_MAX_MSI, "nr-irqs", >> + &error_abort); >> + object_property_set_bool(OBJECT(&phb->msis), true, "realized", &local_err); >> + if (local_err) { >> + error_propagate(errp, local_err); >> + return; >> + } >> + >> + /* Power Bus Common Queue */ >> + object_property_set_bool(OBJECT(&phb->pbcq), true, "realized", &local_err); >> + if (local_err) { >> + error_propagate(errp, local_err); >> + return; >> + } >> + >> + /* Controller Registers */ >> + memory_region_init_io(&phb->mr_regs, OBJECT(phb), &pnv_phb3_reg_ops, phb, >> + "phb3-regs", 0x1000); >> + >> + /* PHB3 doesn't support IO space. However, qemu gets very upset if >> + * we don't have an IO region to anchor IO BARs onto so we just >> + * initialize one which we never hook up to anything >> + */ >> + memory_region_init(&phb->pci_io, OBJECT(phb), "pci-io", 0x10000); >> + memory_region_init(&phb->pci_mmio, OBJECT(phb), "pci-mmio", >> + PCI_MMIO_TOTAL_SIZE); >> + >> + pci->bus = pci_register_root_bus(dev, "root-bus", >> + pnv_phb3_set_irq, pnv_phb3_map_irq, phb, >> + &phb->pci_mmio, &phb->pci_io, >> + 0, 4, TYPE_PNV_PHB3_ROOT_BUS); >> + >> + pci_setup_iommu(pci->bus, pnv_phb3_dma_iommu, phb); >> + >> + /* Add a single Root port */ >> + qdev_prop_set_uint8(DEVICE(&phb->root), "chassis", phb->chip_id); >> + qdev_prop_set_uint16(DEVICE(&phb->root), "slot", phb->phb_id); >> + qdev_set_parent_bus(DEVICE(&phb->root), BUS(pci->bus)); >> + qdev_init_nofail(DEVICE(&phb->root)); >> +} >> + >> +void pnv_phb3_update_regions(PnvPHB3 *phb) >> +{ >> + PnvPBCQState *pbcq = &phb->pbcq; >> + >> + /* Unmap first always */ >> + if (phb->regs_mapped) { >> + memory_region_del_subregion(&pbcq->phbbar, &phb->mr_regs); >> + phb->regs_mapped = false; >> + } >> + >> + /* Map registers if enabled */ >> + if (pbcq->phb_mapped) { >> + /* XXX We should use the PHB BAR 2 register but we don't ... */ >> + memory_region_add_subregion(&pbcq->phbbar, 0, &phb->mr_regs); >> + phb->regs_mapped = true; >> + } >> + >> + /* Check/update m32 */ >> + if (phb->m32_mapped) { >> + pnv_phb3_check_m32(phb); >> + } >> +} >> + >> +static const char *pnv_phb3_root_bus_path(PCIHostState *host_bridge, >> + PCIBus *rootbus) >> +{ >> + PnvPHB3 *phb = PNV_PHB3(host_bridge); >> + >> + snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x", >> + phb->chip_id, phb->phb_id); >> + return phb->bus_path; >> +} >> + >> +static Property pnv_phb3_properties[] = { >> + DEFINE_PROP_UINT32("index", PnvPHB3, phb_id, 0), >> + DEFINE_PROP_UINT32("chip-id", PnvPHB3, chip_id, 0), >> + DEFINE_PROP_END_OF_LIST(), >> +}; >> + >> +static void pnv_phb3_class_init(ObjectClass *klass, void *data) >> +{ >> + PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass); >> + DeviceClass *dc = DEVICE_CLASS(klass); >> + >> + hc->root_bus_path = pnv_phb3_root_bus_path; >> + dc->realize = pnv_phb3_realize; >> + dc->props = pnv_phb3_properties; >> + set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories); >> +} >> + >> +static const TypeInfo pnv_phb3_type_info = { >> + .name = TYPE_PNV_PHB3, >> + .parent = TYPE_PCIE_HOST_BRIDGE, >> + .instance_size = sizeof(PnvPHB3), >> + .class_init = pnv_phb3_class_init, >> + .instance_init = pnv_phb3_instance_init, >> +}; >> + >> +static void pnv_phb3_root_bus_class_init(ObjectClass *klass, void *data) >> +{ >> + BusClass *k = BUS_CLASS(klass); >> + >> + /* >> + * PHB3 has only a single root complex. Enforce the limit on the >> + * parent bus >> + */ >> + k->max_dev = 1; >> +} >> + >> +static const TypeInfo pnv_phb3_root_bus_info = { >> + .name = TYPE_PNV_PHB3_ROOT_BUS, >> + .parent = TYPE_PCI_BUS, >> + .class_init = pnv_phb3_root_bus_class_init, >> + .interfaces = (InterfaceInfo[]) { >> + { INTERFACE_PCIE_DEVICE }, >> + { } >> + }, >> +}; >> + >> +static void pnv_phb3_root_port_realize(DeviceState *dev, Error **errp) >> +{ >> + PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev); >> + Error *local_err = NULL; >> + >> + rpc->parent_realize(dev, &local_err); >> + if (local_err) { >> + error_propagate(errp, local_err); >> + return; >> + } >> +} >> + >> +static void pnv_phb3_root_port_class_init(ObjectClass *klass, void *data) >> +{ >> + DeviceClass *dc = DEVICE_CLASS(klass); >> + PCIDeviceClass *k = PCI_DEVICE_CLASS(klass); >> + PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass); >> + >> + dc->desc = "IBM PHB3 PCIE Root Port"; >> + >> + device_class_set_parent_realize(dc, pnv_phb3_root_port_realize, >> + &rpc->parent_realize); >> + >> + k->vendor_id = PCI_VENDOR_ID_IBM; >> + k->device_id = 0x03dc; >> + k->revision = 0; >> + >> + rpc->exp_offset = 0x48; >> + rpc->aer_offset = 0x100; >> +} >> + >> +static const TypeInfo pnv_phb3_root_port_info = { >> + .name = TYPE_PNV_PHB3_ROOT_PORT, >> + .parent = TYPE_PCIE_ROOT_PORT, >> + .instance_size = sizeof(PnvPHB3RootPort), >> + .class_init = pnv_phb3_root_port_class_init, >> +}; >> + >> +static void pnv_phb3_register_types(void) >> +{ >> + type_register_static(&pnv_phb3_root_bus_info); >> + type_register_static(&pnv_phb3_root_port_info); >> + type_register_static(&pnv_phb3_type_info); >> + type_register_static(&pnv_phb3_iommu_memory_region_info); >> +} >> + >> +type_init(pnv_phb3_register_types) >> diff --git a/hw/pci-host/pnv_phb3_msi.c b/hw/pci-host/pnv_phb3_msi.c >> new file mode 100644 >> index 000000000000..588865a86de1 >> --- /dev/null >> +++ b/hw/pci-host/pnv_phb3_msi.c >> @@ -0,0 +1,318 @@ >> +/* >> + * QEMU PowerPC PowerNV PHB3 model >> + * >> + * Copyright (c) 2014-2018, IBM Corporation. >> + * >> + * This code is licensed under the GPL version 2 or later. See the >> + * COPYING file in the top-level directory. >> + */ >> +#include "qemu/osdep.h" >> +#include "qemu/log.h" >> +#include "qapi/error.h" >> +#include "qemu-common.h" >> +#include "target/ppc/cpu.h" >> +#include "hw/pci-host/pnv_phb3_regs.h" >> +#include "hw/pci-host/pnv_phb3.h" >> +#include "hw/ppc/pnv.h" >> +#include "hw/pci/msi.h" >> +#include "monitor/monitor.h" >> + >> +static uint64_t phb3_msi_ive_addr(PnvPHB3 *phb, int srcno) >> +{ >> + uint64_t ivtbar = phb->regs[PHB_IVT_BAR >> 3]; >> + uint64_t phbctl = phb->regs[PHB_CONTROL >> 3]; >> + >> + if (!(ivtbar & PHB_IVT_BAR_ENABLE)) { >> + qemu_log_mask(LOG_GUEST_ERROR, "Failed access to disable IVT BAR !"); >> + return 0; >> + } >> + >> + if (srcno >= (ivtbar & PHB_IVT_LENGTH_MASK)) { >> + qemu_log_mask(LOG_GUEST_ERROR, "MSI out of bounds (%d vs 0x%"PRIx64")", >> + srcno, ivtbar & PHB_IVT_LENGTH_MASK); >> + return 0; >> + } >> + >> + ivtbar &= PHB_IVT_BASE_ADDRESS_MASK; >> + >> + if (phbctl & PHB_CTRL_IVE_128_BYTES) { >> + return ivtbar + 128 * srcno; >> + } else { >> + return ivtbar + 16 * srcno; >> + } >> +} >> + >> +static bool phb3_msi_read_ive(PnvPHB3 *phb, int srcno, uint64_t *out_ive) >> +{ >> + uint64_t ive_addr, ive; >> + >> + ive_addr = phb3_msi_ive_addr(phb, srcno); >> + if (!ive_addr) { >> + return false; >> + } >> + >> + if (dma_memory_read(&address_space_memory, ive_addr, &ive, sizeof(ive))) { >> + qemu_log_mask(LOG_GUEST_ERROR, "Failed to read IVE at 0x%" PRIx64, >> + ive_addr); >> + return false; >> + } >> + *out_ive = be64_to_cpu(ive); >> + >> + return true; >> +} >> + >> +static void phb3_msi_set_p(Phb3MsiState *msi, int srcno, uint8_t gen) >> +{ >> + uint64_t ive_addr; >> + uint8_t p = 0x01 | (gen << 1); >> + >> + ive_addr = phb3_msi_ive_addr(msi->phb, srcno); >> + if (!ive_addr) { >> + return; >> + } >> + >> + if (dma_memory_write(&address_space_memory, ive_addr + 4, &p, 1)) { >> + qemu_log_mask(LOG_GUEST_ERROR, >> + "Failed to write IVE (set P) at 0x%" PRIx64, ive_addr); >> + } >> +} >> + >> +static void phb3_msi_set_q(Phb3MsiState *msi, int srcno) >> +{ >> + uint64_t ive_addr; >> + uint8_t q = 0x01; >> + >> + ive_addr = phb3_msi_ive_addr(msi->phb, srcno); >> + if (!ive_addr) { >> + return; >> + } >> + >> + if (dma_memory_write(&address_space_memory, ive_addr + 5, &q, 1)) { >> + qemu_log_mask(LOG_GUEST_ERROR, >> + "Failed to write IVE (set Q) at 0x%" PRIx64, ive_addr); >> + } >> +} >> + >> +static void phb3_msi_try_send(Phb3MsiState *msi, int srcno, bool force) >> +{ >> + ICSState *ics = ICS_BASE(msi); >> + uint64_t ive; >> + uint64_t server, prio, pq, gen; >> + >> + if (!phb3_msi_read_ive(msi->phb, srcno, &ive)) { >> + return; >> + } >> + >> + server = GETFIELD(IODA2_IVT_SERVER, ive); >> + prio = GETFIELD(IODA2_IVT_PRIORITY, ive); >> + if (!force) { >> + pq = GETFIELD(IODA2_IVT_Q, ive) | (GETFIELD(IODA2_IVT_P, ive) << 1); >> + } else { >> + pq = 0; >> + } >> + gen = GETFIELD(IODA2_IVT_GEN, ive); >> + >> + /* >> + * The low order 2 bits are the link pointer (Type II interrupts). >> + * Shift back to get a valid IRQ server. >> + */ >> + server >>= 2; >> + >> + switch (pq) { >> + case 0: /* 00 */ >> + if (prio == 0xff) { >> + /* Masked, set Q */ >> + phb3_msi_set_q(msi, srcno); >> + } else { >> + /* Enabled, set P and send */ >> + phb3_msi_set_p(msi, srcno, gen); >> + icp_irq(ics, server, srcno + ics->offset, prio); >> + } >> + break; >> + case 2: /* 10 */ >> + /* Already pending, set Q */ >> + phb3_msi_set_q(msi, srcno); >> + break; >> + case 1: /* 01 */ >> + case 3: /* 11 */ >> + default: >> + /* Just drop stuff if Q already set */ >> + break; >> + } >> +} >> + >> +static void phb3_msi_set_irq(void *opaque, int srcno, int val) >> +{ >> + Phb3MsiState *msi = PHB3_MSI(opaque); >> + >> + if (val) { >> + phb3_msi_try_send(msi, srcno, false); >> + } >> +} >> + >> + >> +void pnv_phb3_msi_send(Phb3MsiState *msi, uint64_t addr, uint16_t data, >> + int32_t dev_pe) >> +{ >> + ICSState *ics = ICS_BASE(msi); >> + uint64_t ive; >> + uint16_t pe; >> + uint32_t src = ((addr >> 4) & 0xffff) | (data & 0x1f); >> + >> + if (src >= ics->nr_irqs) { >> + qemu_log_mask(LOG_GUEST_ERROR, "MSI %d out of bounds", src); >> + return; >> + } >> + if (dev_pe >= 0) { >> + if (!phb3_msi_read_ive(msi->phb, src, &ive)) { >> + return; >> + } >> + pe = GETFIELD(IODA2_IVT_PE, ive); >> + if (pe != dev_pe) { >> + qemu_log_mask(LOG_GUEST_ERROR, >> + "MSI %d send by PE#%d but assigned to PE#%d", >> + src, dev_pe, pe); >> + return; >> + } >> + } >> + qemu_irq_pulse(ics->qirqs[src]); >> +} >> + >> +void pnv_phb3_msi_ffi(Phb3MsiState *msi, uint64_t val) >> +{ >> + /* Emit interrupt */ >> + pnv_phb3_msi_send(msi, val, 0, -1); >> + >> + /* Clear FFI lock */ >> + msi->phb->regs[PHB_FFI_LOCK >> 3] = 0; >> +} >> + >> +static void phb3_msi_reject(ICSState *ics, uint32_t nr) >> +{ >> + Phb3MsiState *msi = PHB3_MSI(ics); >> + unsigned int srcno = nr - ics->offset; >> + unsigned int idx = srcno >> 6; >> + unsigned int bit = 1ull << (srcno & 0x3f); >> + >> + assert(srcno < PHB3_MAX_MSI); >> + >> + msi->rba[idx] |= bit; >> + msi->rba_sum |= (1u << idx); >> +} >> + >> +static void phb3_msi_resend(ICSState *ics) >> +{ >> + Phb3MsiState *msi = PHB3_MSI(ics); >> + unsigned int i, j; >> + >> + if (msi->rba_sum == 0) { >> + return; >> + } >> + >> + for (i = 0; i < 32; i++) { >> + if ((msi->rba_sum & (1u << i)) == 0) { >> + continue; >> + } >> + msi->rba_sum &= ~(1u << i); >> + for (j = 0; j < 64; j++) { >> + if ((msi->rba[i] & (1ull << j)) == 0) { >> + continue; >> + } >> + msi->rba[i] &= ~(1u << j); >> + phb3_msi_try_send(msi, i * 64 + j, true); >> + } >> + } >> +} >> + >> +static void phb3_msi_reset(DeviceState *dev) >> +{ >> + Phb3MsiState *msi = PHB3_MSI(dev); >> + ICSStateClass *icsc = ICS_BASE_GET_CLASS(dev); >> + >> + icsc->parent_reset(dev); >> + >> + memset(msi->rba, 0, sizeof(msi->rba)); >> + msi->rba_sum = 0; >> +} >> + >> +static void phb3_msi_reset_handler(void *dev) >> +{ >> + phb3_msi_reset(dev); >> +} >> + >> +void pnv_phb3_msi_update_config(Phb3MsiState *msi, uint32_t base, >> + uint32_t count) >> +{ >> + ICSState *ics = ICS_BASE(msi); >> + >> + if (count > PHB3_MAX_MSI) { >> + count = PHB3_MAX_MSI; >> + } >> + ics->nr_irqs = count; >> + ics->offset = base; >> +} >> + >> +static void phb3_msi_realize(DeviceState *dev, Error **errp) >> +{ >> + Phb3MsiState *msi = PHB3_MSI(dev); >> + ICSState *ics = ICS_BASE(msi); >> + ICSStateClass *icsc = ICS_BASE_GET_CLASS(ics); >> + Object *obj; >> + Error *local_err = NULL; >> + >> + icsc->parent_realize(dev, &local_err); >> + if (local_err) { >> + error_propagate(errp, local_err); >> + return; >> + } >> + >> + obj = object_property_get_link(OBJECT(dev), "phb", &local_err); >> + if (!obj) { >> + error_propagate(errp, local_err); >> + error_prepend(errp, "required link 'phb' not found: "); >> + return; >> + } >> + msi->phb = PNV_PHB3(obj); >> + >> + ics->qirqs = qemu_allocate_irqs(phb3_msi_set_irq, msi, ics->nr_irqs); >> + >> + qemu_register_reset(phb3_msi_reset_handler, dev); >> +} >> + >> +static void phb3_msi_instance_init(Object *obj) >> +{ >> + ICSState *ics = ICS_BASE(obj); >> + >> + /* Will be overriden later */ >> + ics->offset = 0; >> +} >> + >> +static void phb3_msi_class_init(ObjectClass *klass, void *data) >> +{ >> + DeviceClass *dc = DEVICE_CLASS(klass); >> + ICSStateClass *isc = ICS_BASE_CLASS(klass); >> + >> + device_class_set_parent_realize(dc, phb3_msi_realize, >> + &isc->parent_realize); >> + device_class_set_parent_reset(dc, phb3_msi_reset, >> + &isc->parent_reset); >> + >> + isc->reject = phb3_msi_reject; >> + isc->resend = phb3_msi_resend; >> +} >> + >> +static const TypeInfo phb3_msi_info = { >> + .name = TYPE_PHB3_MSI, >> + .parent = TYPE_ICS_BASE, >> + .instance_size = sizeof(Phb3MsiState), >> + .class_init = phb3_msi_class_init, >> + .class_size = sizeof(ICSStateClass), >> + .instance_init = phb3_msi_instance_init, >> +}; >> + >> +static void pnv_phb3_msi_register_types(void) >> +{ >> + type_register_static(&phb3_msi_info); >> +} >> + >> +type_init(pnv_phb3_msi_register_types) >> diff --git a/hw/pci-host/pnv_phb3_pbcq.c b/hw/pci-host/pnv_phb3_pbcq.c >> new file mode 100644 >> index 000000000000..34df3ac684e4 >> --- /dev/null >> +++ b/hw/pci-host/pnv_phb3_pbcq.c > > We should maybe make a subdir rather than have these multi-barrelled > filenames. > >> @@ -0,0 +1,347 @@ >> +/* >> + * QEMU PowerPC PowerNV PHB3 model >> + * >> + * Copyright (c) 2014-2018, IBM Corporation. >> + * >> + * This code is licensed under the GPL version 2 or later. See the >> + * COPYING file in the top-level directory. >> + */ >> +#include "qemu/osdep.h" >> +#include "qapi/error.h" >> +#include "qemu-common.h" >> +#include "qemu/log.h" >> +#include "target/ppc/cpu.h" >> +#include "hw/ppc/fdt.h" >> +#include "hw/pci-host/pnv_phb3_regs.h" >> +#include "hw/pci-host/pnv_phb3.h" >> +#include "hw/ppc/pnv_xscom.h" >> +#include "hw/pci/pci_bridge.h" >> +#include "hw/pci/pci_bus.h" >> + >> +#include <libfdt.h> >> + >> +static uint64_t pnv_pbcq_nest_xscom_read(void *opaque, hwaddr addr, >> + unsigned size) >> +{ >> + PnvPBCQState *pbcq = PNV_PBCQ(opaque); >> + uint32_t offset = addr >> 3; >> + >> + return pbcq->nest_regs[offset]; >> +} >> + >> +static uint64_t pnv_pbcq_pci_xscom_read(void *opaque, hwaddr addr, >> + unsigned size) >> +{ >> + PnvPBCQState *pbcq = PNV_PBCQ(opaque); >> + uint32_t offset = addr >> 3; >> + >> + return pbcq->pci_regs[offset]; >> +} >> + >> +static uint64_t pnv_pbcq_spci_xscom_read(void *opaque, hwaddr addr, >> + unsigned size) >> +{ >> + PnvPBCQState *pbcq = PNV_PBCQ(opaque); >> + uint32_t offset = addr >> 3; >> + >> + if (offset == PBCQ_SPCI_ASB_DATA) { >> + return pnv_phb3_reg_read(pbcq->phb, >> + pbcq->spci_regs[PBCQ_SPCI_ASB_ADDR], 8); >> + } >> + return pbcq->spci_regs[offset]; >> +} >> + >> +static void pnv_pbcq_update_map(PnvPBCQState *pbcq) >> +{ >> + uint64_t bar_en = pbcq->nest_regs[PBCQ_NEST_BAR_EN]; >> + uint64_t bar, mask, size; >> + >> + /* >> + * NOTE: This will really not work well if those are remapped >> + * after the PHB has created its sub regions. We could do better >> + * if we had a way to resize regions > > Such as memory_region_set_size(), for example? true. > >> but we don't really care >> + * that much in practice as the stuff below really only happens >> + * once early during boot >> + */ >> + >> + /* Handle unmaps */ >> + if (pbcq->mmio0_mapped && !(bar_en & PBCQ_NEST_BAR_EN_MMIO0)) { >> + memory_region_del_subregion(get_system_memory(), &pbcq->mmbar0); >> + pbcq->mmio0_mapped = false; >> + } >> + if (pbcq->mmio1_mapped && !(bar_en & PBCQ_NEST_BAR_EN_MMIO1)) { >> + memory_region_del_subregion(get_system_memory(), &pbcq->mmbar1); >> + pbcq->mmio1_mapped = false; >> + } >> + if (pbcq->phb_mapped && !(bar_en & PBCQ_NEST_BAR_EN_PHB)) { >> + memory_region_del_subregion(get_system_memory(), &pbcq->phbbar); >> + pbcq->phb_mapped = false; >> + } >> + >> + /* Update PHB */ >> + pnv_phb3_update_regions(pbcq->phb); >> + >> + /* Handle maps */ >> + if (!pbcq->mmio0_mapped && (bar_en & PBCQ_NEST_BAR_EN_MMIO0)) { >> + bar = pbcq->nest_regs[PBCQ_NEST_MMIO_BAR0] >> 14; >> + mask = pbcq->nest_regs[PBCQ_NEST_MMIO_MASK0]; >> + size = ((~mask) >> 14) + 1; >> + memory_region_init(&pbcq->mmbar0, OBJECT(pbcq), "pbcq-mmio0", size); >> + memory_region_add_subregion(get_system_memory(), bar, &pbcq->mmbar0); >> + pbcq->mmio0_mapped = true; >> + pbcq->mmio0_base = bar; >> + pbcq->mmio0_size = size; >> + } >> + if (!pbcq->mmio1_mapped && (bar_en & PBCQ_NEST_BAR_EN_MMIO1)) { >> + bar = pbcq->nest_regs[PBCQ_NEST_MMIO_BAR1] >> 14; >> + mask = pbcq->nest_regs[PBCQ_NEST_MMIO_MASK1]; >> + size = ((~mask) >> 14) + 1; >> + memory_region_init(&pbcq->mmbar1, OBJECT(pbcq), "pbcq-mmio1", size); >> + memory_region_add_subregion(get_system_memory(), bar, &pbcq->mmbar1); >> + pbcq->mmio1_mapped = true; >> + pbcq->mmio1_base = bar; >> + pbcq->mmio1_size = size; >> + } >> + if (!pbcq->phb_mapped && (bar_en & PBCQ_NEST_BAR_EN_PHB)) { >> + bar = pbcq->nest_regs[PBCQ_NEST_PHB_BAR] >> 14; >> + size = 0x1000; >> + memory_region_init(&pbcq->phbbar, OBJECT(pbcq), "pbcq-phb", size); >> + memory_region_add_subregion(get_system_memory(), bar, &pbcq->phbbar); >> + pbcq->phb_mapped = true; >> + } >> + >> + /* Update PHB */ >> + pnv_phb3_update_regions(pbcq->phb); >> +} >> + >> +static void pnv_pbcq_nest_xscom_write(void *opaque, hwaddr addr, >> + uint64_t val, unsigned size) >> +{ >> + PnvPBCQState *pbcq = PNV_PBCQ(opaque); >> + uint32_t reg = addr >> 3; >> + >> + switch (reg) { >> + case PBCQ_NEST_MMIO_BAR0: >> + case PBCQ_NEST_MMIO_BAR1: >> + case PBCQ_NEST_MMIO_MASK0: >> + case PBCQ_NEST_MMIO_MASK1: >> + if (pbcq->nest_regs[PBCQ_NEST_BAR_EN] & >> + (PBCQ_NEST_BAR_EN_MMIO0 | >> + PBCQ_NEST_BAR_EN_MMIO1)) { >> + qemu_log_mask(LOG_GUEST_ERROR, >> + "PHB3: Changing enabled BAR unsupported\n"); >> + } >> + pbcq->nest_regs[reg] = val & 0xffffffffc0000000ull; >> + break; >> + case PBCQ_NEST_PHB_BAR: >> + if (pbcq->nest_regs[PBCQ_NEST_BAR_EN] & PBCQ_NEST_BAR_EN_PHB) { >> + qemu_log_mask(LOG_GUEST_ERROR, >> + "PHB3: Changing enabled BAR unsupported\n"); >> + } >> + pbcq->nest_regs[reg] = val & 0xfffffffffc000000ull; >> + break; >> + case PBCQ_NEST_BAR_EN: >> + pbcq->nest_regs[reg] = val & 0xf800000000000000ull; >> + pnv_pbcq_update_map(pbcq); >> + pnv_phb3_remap_irqs(pbcq->phb); >> + break; >> + case PBCQ_NEST_IRSN_COMPARE: >> + case PBCQ_NEST_IRSN_MASK: >> + pbcq->nest_regs[reg] = val & PBCQ_NEST_IRSN_COMP; >> + pnv_phb3_remap_irqs(pbcq->phb); >> + break; >> + case PBCQ_NEST_LSI_SRC_ID: >> + pbcq->nest_regs[reg] = val & PBCQ_NEST_LSI_SRC; >> + pnv_phb3_remap_irqs(pbcq->phb); >> + break; >> + } >> + >> + /* XXX Don't error out on other regs for now ... */ >> +} >> + >> +static void pnv_pbcq_pci_xscom_write(void *opaque, hwaddr addr, >> + uint64_t val, unsigned size) >> +{ >> + PnvPBCQState *pbcq = PNV_PBCQ(opaque); >> + uint32_t reg = addr >> 3; >> + >> + switch (reg) { >> + case PBCQ_PCI_BAR2: >> + pbcq->pci_regs[reg] = val & 0xfffffffffc000000ull; >> + pnv_pbcq_update_map(pbcq); >> + break; >> + } >> + >> + /* XXX Don't error out on other regs for now ... */ >> +} >> + >> +static void pnv_pbcq_spci_xscom_write(void *opaque, hwaddr addr, >> + uint64_t val, unsigned size) >> +{ >> + PnvPBCQState *pbcq = PNV_PBCQ(opaque); >> + uint32_t reg = addr >> 3; >> + >> + switch (reg) { >> + case PBCQ_SPCI_ASB_ADDR: >> + pbcq->spci_regs[reg] = val & 0xfff; >> + break; >> + case PBCQ_SPCI_ASB_STATUS: >> + pbcq->spci_regs[reg] &= ~val; >> + break; >> + case PBCQ_SPCI_ASB_DATA: >> + pnv_phb3_reg_write(pbcq->phb, pbcq->spci_regs[PBCQ_SPCI_ASB_ADDR], >> + val, 8); >> + break; >> + case PBCQ_SPCI_AIB_CAPP_EN: >> + case PBCQ_SPCI_CAPP_SEC_TMR: >> + break; >> + } >> + >> + /* XXX Don't error out on other regs for now ... */ >> +} >> + >> +static const MemoryRegionOps pnv_pbcq_nest_xscom_ops = { >> + .read = pnv_pbcq_nest_xscom_read, >> + .write = pnv_pbcq_nest_xscom_write, >> + .valid.min_access_size = 8, >> + .valid.max_access_size = 8, >> + .impl.min_access_size = 8, >> + .impl.max_access_size = 8, >> + .endianness = DEVICE_BIG_ENDIAN, >> +}; >> + >> +static const MemoryRegionOps pnv_pbcq_pci_xscom_ops = { >> + .read = pnv_pbcq_pci_xscom_read, >> + .write = pnv_pbcq_pci_xscom_write, >> + .valid.min_access_size = 8, >> + .valid.max_access_size = 8, >> + .impl.min_access_size = 8, >> + .impl.max_access_size = 8, >> + .endianness = DEVICE_BIG_ENDIAN, >> +}; >> + >> +static const MemoryRegionOps pnv_pbcq_spci_xscom_ops = { >> + .read = pnv_pbcq_spci_xscom_read, >> + .write = pnv_pbcq_spci_xscom_write, >> + .valid.min_access_size = 8, >> + .valid.max_access_size = 8, >> + .impl.min_access_size = 8, >> + .impl.max_access_size = 8, >> + .endianness = DEVICE_BIG_ENDIAN, >> +}; >> + >> +static void pnv_pbcq_default_bars(PnvPBCQState *pbcq) >> +{ >> + uint64_t mm0, mm1, reg; >> + PnvPHB3 *phb = pbcq->phb; >> + >> + mm0 = 0x3d00000000000ull + 0x4000000000ull * phb->chip_id + >> + 0x1000000000ull * phb->phb_id; >> + mm1 = 0x3ff8000000000ull + 0x0200000000ull * phb->chip_id + >> + 0x0080000000ull * phb->phb_id; >> + reg = 0x3fffe40000000ull + 0x0000400000ull * phb->chip_id + >> + 0x0000100000ull * phb->phb_id; >> + >> + pbcq->nest_regs[PBCQ_NEST_MMIO_BAR0] = mm0 << 14; >> + pbcq->nest_regs[PBCQ_NEST_MMIO_BAR1] = mm1 << 14; >> + pbcq->nest_regs[PBCQ_NEST_PHB_BAR] = reg << 14; >> + pbcq->nest_regs[PBCQ_NEST_MMIO_MASK0] = 0x3fff000000000ull << 14; >> + pbcq->nest_regs[PBCQ_NEST_MMIO_MASK1] = 0x3ffff80000000ull << 14; >> + pbcq->pci_regs[PBCQ_PCI_BAR2] = reg << 14; >> +} >> + >> +static void pnv_pbcq_realize(DeviceState *dev, Error **errp) >> +{ >> + PnvPBCQState *pbcq = PNV_PBCQ(dev); >> + PnvPHB3 *phb; >> + Object *obj; >> + Error *local_err = NULL; >> + char name[32]; >> + >> + obj = object_property_get_link(OBJECT(dev), "phb", &local_err); >> + if (!obj) { >> + error_propagate(errp, local_err); >> + error_prepend(errp, "required link 'phb' not found: "); >> + return; >> + } >> + phb = pbcq->phb = PNV_PHB3(obj); >> + >> + /* XXX Fix OPAL to do that: establish default BAR values */ >> + pnv_pbcq_default_bars(pbcq); >> + >> + /* Initialize the XSCOM region for the PBCQ registers */ >> + snprintf(name, sizeof(name), "xscom-pbcq-nest-%d.%d", >> + phb->chip_id, phb->phb_id); >> + pnv_xscom_region_init(&pbcq->xscom_nest_regs, OBJECT(dev), >> + &pnv_pbcq_nest_xscom_ops, pbcq, name, >> + PNV_XSCOM_PBCQ_NEST_SIZE); >> + snprintf(name, sizeof(name), "xscom-pbcq-pci-%d.%d", >> + phb->chip_id, phb->phb_id); >> + pnv_xscom_region_init(&pbcq->xscom_pci_regs, OBJECT(dev), >> + &pnv_pbcq_pci_xscom_ops, pbcq, name, >> + PNV_XSCOM_PBCQ_PCI_SIZE); >> + snprintf(name, sizeof(name), "xscom-pbcq-spci-%d.%d", >> + phb->chip_id, phb->phb_id); >> + pnv_xscom_region_init(&pbcq->xscom_spci_regs, OBJECT(dev), >> + &pnv_pbcq_spci_xscom_ops, pbcq, name, >> + PNV_XSCOM_PBCQ_SPCI_SIZE); >> +} >> + >> +static int pnv_pbcq_dt_xscom(PnvXScomInterface *dev, void *fdt, >> + int xscom_offset) >> +{ >> + const char compat[] = "ibm,power8-pbcq"; >> + PnvPHB3 *phb = PNV_PBCQ(dev)->phb; >> + char *name; >> + int offset; >> + uint32_t lpc_pcba = PNV_XSCOM_PBCQ_NEST_BASE + 0x400 * phb->phb_id; >> + uint32_t reg[] = { >> + cpu_to_be32(lpc_pcba), >> + cpu_to_be32(PNV_XSCOM_PBCQ_NEST_SIZE), >> + cpu_to_be32(PNV_XSCOM_PBCQ_PCI_BASE + 0x400 * phb->phb_id), >> + cpu_to_be32(PNV_XSCOM_PBCQ_PCI_SIZE), >> + cpu_to_be32(PNV_XSCOM_PBCQ_SPCI_BASE + 0x040 * phb->phb_id), >> + cpu_to_be32(PNV_XSCOM_PBCQ_SPCI_SIZE) >> + }; >> + >> + name = g_strdup_printf("pbcq@%x", lpc_pcba); >> + offset = fdt_add_subnode(fdt, xscom_offset, name); >> + _FDT(offset); >> + g_free(name); >> + >> + _FDT((fdt_setprop(fdt, offset, "reg", reg, sizeof(reg)))); >> + >> + _FDT((fdt_setprop_cell(fdt, offset, "ibm,phb-index", phb->phb_id))); >> + _FDT((fdt_setprop_cell(fdt, offset, "ibm,chip-id", phb->chip_id))); >> + _FDT((fdt_setprop(fdt, offset, "compatible", compat, >> + sizeof(compat)))); >> + return 0; >> +} >> + >> +static void pnv_pbcq_class_init(ObjectClass *klass, void *data) >> +{ >> + DeviceClass *dc = DEVICE_CLASS(klass); >> + PnvXScomInterfaceClass *xdc = PNV_XSCOM_INTERFACE_CLASS(klass); >> + >> + xdc->dt_xscom = pnv_pbcq_dt_xscom; >> + >> + dc->realize = pnv_pbcq_realize; >> +} >> + >> +static const TypeInfo pnv_pbcq_type_info = { >> + .name = TYPE_PNV_PBCQ, >> + .parent = TYPE_DEVICE, >> + .instance_size = sizeof(PnvPBCQState), >> + .class_init = pnv_pbcq_class_init, >> + .interfaces = (InterfaceInfo[]) { >> + { TYPE_PNV_XSCOM_INTERFACE }, >> + { } >> + } >> +}; >> + >> +static void pnv_pbcq_register_types(void) >> +{ >> + type_register_static(&pnv_pbcq_type_info); >> +} >> + >> +type_init(pnv_pbcq_register_types) >> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c >> index 346f5e7aedb5..e8a8065fd25f 100644 >> --- a/hw/ppc/pnv.c >> +++ b/hw/ppc/pnv.c >> @@ -44,6 +44,7 @@ >> #include "hw/isa/isa.h" >> #include "hw/char/serial.h" >> #include "hw/timer/mc146818rtc.h" >> +#include "hw/pci/msi.h" >> >> #include <libfdt.h> >> >> @@ -555,6 +556,9 @@ static void pnv_init(MachineState *machine) >> int i; >> char *chip_typename; >> >> + /* MSIs are supported on this platform */ >> + msi_nonbroken = true; >> + >> /* allocate RAM */ >> if (machine->ram_size < (1 * GiB)) { >> warn_report("skiboot may not work with < 1GB of RAM"); >> @@ -725,6 +729,8 @@ static Object *pnv_chip_power9_intc_create(PnvChip *chip, Object *child, >> static void pnv_chip_power8_instance_init(Object *obj) >> { >> Pnv8Chip *chip8 = PNV8_CHIP(obj); >> + PnvChipClass *pcc = PNV_CHIP_GET_CLASS(obj); >> + int i; >> >> object_initialize(&chip8->psi, sizeof(chip8->psi), TYPE_PNV_PSI); >> object_property_add_child(obj, "psi", OBJECT(&chip8->psi), NULL); >> @@ -740,6 +746,12 @@ static void pnv_chip_power8_instance_init(Object *obj) >> object_property_add_child(obj, "occ", OBJECT(&chip8->occ), NULL); >> object_property_add_const_link(OBJECT(&chip8->occ), "psi", >> OBJECT(&chip8->psi), &error_abort); >> + >> + for (i = 0; i < pcc->num_phbs; i++) { >> + object_initialize(&chip8->phbs[i], sizeof(chip8->phbs[i]), >> + TYPE_PNV_PHB3); >> + object_property_add_child(obj, "phb[*]", OBJECT(&chip8->phbs[i]), NULL); >> + } >> } >> >> static void pnv_chip_icp_realize(Pnv8Chip *chip8, Error **errp) >> @@ -780,6 +792,7 @@ static void pnv_chip_power8_realize(DeviceState *dev, Error **errp) >> PnvChip *chip = PNV_CHIP(dev); >> Pnv8Chip *chip8 = PNV8_CHIP(dev); >> Error *local_err = NULL; >> + int i; >> >> pcc->parent_realize(dev, &local_err); >> if (local_err) { >> @@ -817,6 +830,33 @@ static void pnv_chip_power8_realize(DeviceState *dev, Error **errp) >> return; >> } >> pnv_xscom_add_subregion(chip, PNV_XSCOM_OCC_BASE, &chip8->occ.xscom_regs); >> + >> + /* Create the PHB3 controllers */ >> + for (i = 0; i < pcc->num_phbs; i++) { >> + PnvPHB3 *phb = &chip8->phbs[i]; >> + PnvPBCQState *pbcq = &phb->pbcq; >> + >> + object_property_set_int(OBJECT(phb), i, "index", &error_fatal); >> + object_property_set_int(OBJECT(phb), chip->chip_id, "chip-id", >> + &error_fatal); >> + object_property_set_bool(OBJECT(phb), true, "realized", &local_err); >> + if (local_err) { >> + error_propagate(errp, local_err); >> + return; >> + } >> + qdev_set_parent_bus(DEVICE(phb), sysbus_get_default()); >> + >> + /* Populate the XSCOM address space. */ >> + pnv_xscom_add_subregion(chip, >> + PNV_XSCOM_PBCQ_NEST_BASE + 0x400 * phb->phb_id, >> + &pbcq->xscom_nest_regs); >> + pnv_xscom_add_subregion(chip, >> + PNV_XSCOM_PBCQ_PCI_BASE + 0x400 * phb->phb_id, >> + &pbcq->xscom_pci_regs); >> + pnv_xscom_add_subregion(chip, >> + PNV_XSCOM_PBCQ_SPCI_BASE + 0x040 * phb->phb_id, >> + &pbcq->xscom_spci_regs); >> + } >> } >> >> static void pnv_chip_power8e_class_init(ObjectClass *klass, void *data) >> @@ -827,6 +867,7 @@ static void pnv_chip_power8e_class_init(ObjectClass *klass, void *data) >> k->chip_type = PNV_CHIP_POWER8E; >> k->chip_cfam_id = 0x221ef04980000000ull; /* P8 Murano DD2.1 */ >> k->cores_mask = POWER8E_CORE_MASK; >> + k->num_phbs = 3; >> k->core_pir = pnv_chip_core_pir_p8; >> k->intc_create = pnv_chip_power8_intc_create; >> k->isa_create = pnv_chip_power8_isa_create; >> @@ -845,6 +886,7 @@ static void pnv_chip_power8_class_init(ObjectClass *klass, void *data) >> k->chip_type = PNV_CHIP_POWER8; >> k->chip_cfam_id = 0x220ea04980000000ull; /* P8 Venice DD2.0 */ >> k->cores_mask = POWER8_CORE_MASK; >> + k->num_phbs = 3; >> k->core_pir = pnv_chip_core_pir_p8; >> k->intc_create = pnv_chip_power8_intc_create; >> k->isa_create = pnv_chip_power8_isa_create; >> @@ -863,6 +905,7 @@ static void pnv_chip_power8nvl_class_init(ObjectClass *klass, void *data) >> k->chip_type = PNV_CHIP_POWER8NVL; >> k->chip_cfam_id = 0x120d304980000000ull; /* P8 Naples DD1.0 */ >> k->cores_mask = POWER8_CORE_MASK; >> + k->num_phbs = 3; >> k->core_pir = pnv_chip_core_pir_p8; >> k->intc_create = pnv_chip_power8_intc_create; >> k->isa_create = pnv_chip_power8nvl_isa_create; >> @@ -897,6 +940,7 @@ static void pnv_chip_power9_class_init(ObjectClass *klass, void *data) >> k->chip_type = PNV_CHIP_POWER9; >> k->chip_cfam_id = 0x220d104900008000ull; /* P9 Nimbus DD2.0 */ >> k->cores_mask = POWER9_CORE_MASK; >> + k->num_phbs = 0; >> k->core_pir = pnv_chip_core_pir_p9; >> k->intc_create = pnv_chip_power9_intc_create; >> k->isa_create = pnv_chip_power9_isa_create; >> @@ -1047,14 +1091,24 @@ static void pnv_chip_class_init(ObjectClass *klass, void *data) >> static ICSState *pnv_ics_get(XICSFabric *xi, int irq) >> { >> PnvMachineState *pnv = PNV_MACHINE(xi); >> - int i; >> + int i, j; >> >> for (i = 0; i < pnv->num_chips; i++) { >> + PnvChip *chip = pnv->chips[i]; >> Pnv8Chip *chip8 = PNV8_CHIP(pnv->chips[i]); >> + PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip); >> >> if (ics_valid_irq(&chip8->psi.ics, irq)) { >> return &chip8->psi.ics; >> } >> + for (j = 0; j < pcc->num_phbs; j++) { >> + if (ics_valid_irq(&chip8->phbs[j].lsis, irq)) { >> + return &chip8->phbs[j].lsis; >> + } >> + if (ics_valid_irq(ICS_BASE(&chip8->phbs[j].msis), irq)) { >> + return ICS_BASE(&chip8->phbs[j].msis); >> + } >> + } >> } >> return NULL; >> } >> @@ -1062,11 +1116,18 @@ static ICSState *pnv_ics_get(XICSFabric *xi, int irq) >> static void pnv_ics_resend(XICSFabric *xi) >> { >> PnvMachineState *pnv = PNV_MACHINE(xi); >> - int i; >> + int i, j; >> >> for (i = 0; i < pnv->num_chips; i++) { >> + PnvChip *chip = pnv->chips[i]; >> Pnv8Chip *chip8 = PNV8_CHIP(pnv->chips[i]); >> + PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip); >> + >> ics_resend(&chip8->psi.ics); >> + for (j = 0; j < pcc->num_phbs; j++) { >> + ics_resend(&chip8->phbs[j].lsis); >> + ics_resend(ICS_BASE(&chip8->phbs[j].msis)); >> + } >> } >> } >> >> @@ -1097,7 +1158,7 @@ static void pnv_pic_print_info(InterruptStatsProvider *obj, >> Monitor *mon) >> { >> PnvMachineState *pnv = PNV_MACHINE(obj); >> - int i; >> + int i, j; >> CPUState *cs; >> >> CPU_FOREACH(cs) { >> @@ -1107,8 +1168,16 @@ static void pnv_pic_print_info(InterruptStatsProvider *obj, >> } >> >> for (i = 0; i < pnv->num_chips; i++) { >> + PnvChip *chip = pnv->chips[i]; >> Pnv8Chip *chip8 = PNV8_CHIP(pnv->chips[i]); >> + PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip); >> + >> ics_pic_print_info(&chip8->psi.ics, mon); >> + >> + for (j = 0; j < pcc->num_phbs; j++) { >> + ics_pic_print_info(ICS_BASE(&chip8->phbs[j].msis), mon); >> + ics_pic_print_info(&chip8->phbs[j].lsis, mon); >> + } >> } >> } >> >> diff --git a/hw/ppc/pnv_xscom.c b/hw/ppc/pnv_xscom.c >> index 46fae41f32b0..83f05d06053f 100644 >> --- a/hw/ppc/pnv_xscom.c >> +++ b/hw/ppc/pnv_xscom.c >> @@ -252,7 +252,11 @@ int pnv_dt_xscom(PnvChip *chip, void *fdt, int root_offset) >> args.fdt = fdt; >> args.xscom_offset = xscom_offset; >> >> - object_child_foreach(OBJECT(chip), xscom_dt_child, &args); >> + /* Some PnvXScomInterface objects lie a bit deeper (PnvPBCQState) >> + * than the first layer, so we need to loop on the whole object >> + * hierarchy to catch them >> + */ >> + object_child_foreach_recursive(OBJECT(chip), xscom_dt_child, &args); >> return 0; >> } >> >> diff --git a/hw/pci-host/Makefile.objs b/hw/pci-host/Makefile.objs >> index 6d6597c06563..1b5b782bd986 100644 >> --- a/hw/pci-host/Makefile.objs >> +++ b/hw/pci-host/Makefile.objs >> @@ -19,3 +19,4 @@ common-obj-$(CONFIG_PCI_GENERIC) += gpex.o >> common-obj-$(CONFIG_PCI_XILINX) += xilinx-pcie.o >> >> common-obj-$(CONFIG_PCI_DESIGNWARE) += designware.o >> +obj-$(CONFIG_POWERNV) += pnv_phb3.o pnv_phb3_pbcq.o pnv_phb3_msi.o >
diff --git a/default-configs/ppc64-softmmu.mak b/default-configs/ppc64-softmmu.mak index b94af6c7c62a..deebba2b044a 100644 --- a/default-configs/ppc64-softmmu.mak +++ b/default-configs/ppc64-softmmu.mak @@ -9,6 +9,7 @@ CONFIG_IPMI=y CONFIG_IPMI_LOCAL=y CONFIG_IPMI_EXTERN=y CONFIG_ISA_IPMI_BT=y +CONFIG_PCIE_PORT=y # For pSeries CONFIG_PSERIES=y diff --git a/include/hw/pci-host/pnv_phb3.h b/include/hw/pci-host/pnv_phb3.h new file mode 100644 index 000000000000..eb9efaf4e13e --- /dev/null +++ b/include/hw/pci-host/pnv_phb3.h @@ -0,0 +1,171 @@ +/* + * QEMU PowerPC PowerNV PHB3 model + * + * Copyright (c) 2014-2018, IBM Corporation. + * + * This code is licensed under the GPL version 2 or later. See the + * COPYING file in the top-level directory. + */ + +#ifndef PCI_HOST_PNV_PHB3_H +#define PCI_HOST_PNV_PHB3_H + +#include "hw/pci/pcie_host.h" +#include "hw/pci/pcie_port.h" +#include "hw/ppc/xics.h" + +typedef struct PnvPHB3 PnvPHB3; +typedef struct PnvChip PnvChip; + +/* + * PHB3 XICS Source for MSIs + */ +#define TYPE_PHB3_MSI "phb3-msi" +#define PHB3_MSI(obj) OBJECT_CHECK(Phb3MsiState, (obj), TYPE_PHB3_MSI) + +#define PHB3_MAX_MSI 2048 + +typedef struct Phb3MsiState { + ICSState ics; + + PnvPHB3 *phb; + uint64_t rba[PHB3_MAX_MSI / 64]; + uint32_t rba_sum; +} Phb3MsiState; + +void pnv_phb3_msi_update_config(Phb3MsiState *msis, uint32_t base, + uint32_t count); +void pnv_phb3_msi_send(Phb3MsiState *msis, uint64_t addr, uint16_t data, + int32_t dev_pe); +void pnv_phb3_msi_ffi(Phb3MsiState *msis, uint64_t val); + + +/* We have one such address space wrapper per possible device + * under the PHB since they need to be assigned statically at + * qemu device creation time. The relationship to a PE is done + * later dynamically. This means we can potentially create a lot + * of these guys. Q35 stores them as some kind of radix tree but + * we never really need to do fast lookups so instead we simply + * keep a QLIST of them for now, we can add the radix if needed + * later on. + * + * We do cache the PE number to speed things up a bit though. + */ +typedef struct PnvPhb3DMASpace { + PCIBus *bus; + uint8_t devfn; + int pe_num; /* Cached PE number */ +#define PHB_INVALID_PE (-1) + PnvPHB3 *phb; + AddressSpace dma_as; + IOMMUMemoryRegion dma_mr; + MemoryRegion msi32_mr; + MemoryRegion msi64_mr; + bool msi32_mapped; + bool msi64_mapped; + QLIST_ENTRY(PnvPhb3DMASpace) list; +} PnvPhb3DMASpace; + +/* + * PHB3 Power Bus Common Queue + */ +#define TYPE_PNV_PBCQ "pnv-pbcq" +#define PNV_PBCQ(obj) OBJECT_CHECK(PnvPBCQState, (obj), TYPE_PNV_PBCQ) + +typedef struct PnvPBCQState { + DeviceState parent; + + uint32_t nest_xbase; + uint32_t spci_xbase; + uint32_t pci_xbase; +#define PBCQ_NEST_REGS_COUNT 0x46 +#define PBCQ_PCI_REGS_COUNT 0x15 +#define PBCQ_SPCI_REGS_COUNT 0x5 + + uint64_t nest_regs[PBCQ_NEST_REGS_COUNT]; + uint64_t spci_regs[PBCQ_SPCI_REGS_COUNT]; + uint64_t pci_regs[PBCQ_PCI_REGS_COUNT]; + MemoryRegion mmbar0; + MemoryRegion mmbar1; + MemoryRegion phbbar; + bool mmio0_mapped; + bool mmio1_mapped; + bool phb_mapped; + uint64_t mmio0_base; + uint64_t mmio0_size; + uint64_t mmio1_base; + uint64_t mmio1_size; + PnvPHB3 *phb; + + MemoryRegion xscom_nest_regs; + MemoryRegion xscom_pci_regs; + MemoryRegion xscom_spci_regs; +} PnvPBCQState; + +/* + * PHB3 PCIe Root port + */ +#define TYPE_PNV_PHB3_ROOT_BUS "pnv-phb3-root-bus" + +#define TYPE_PNV_PHB3_ROOT_PORT "pnv-phb3-root-port" + +typedef struct PnvPHB3RootPort { + PCIESlot parent_obj; +} PnvPHB3RootPort; + +/* + * PHB3 PCIe Host Bridge for PowerNV machines (POWER8) + */ +#define TYPE_PNV_PHB3 "pnv-phb3" +#define PNV_PHB3(obj) OBJECT_CHECK(PnvPHB3, (obj), TYPE_PNV_PHB3) + +#define PNV_PHB3_NUM_M64 16 +#define PNV_PHB3_NUM_REGS (0x1000 >> 3) +#define PNV_PHB3_NUM_LSI 8 +#define PNV_PHB3_NUM_PE 256 + +#define PCI_MMIO_TOTAL_SIZE (0x1ull << 60) + +struct PnvPHB3 { + PCIExpressHost parent_obj; + + PnvPHB3RootPort root; + + uint32_t chip_id; + uint32_t phb_id; + char bus_path[8]; + + uint64_t regs[PNV_PHB3_NUM_REGS]; + MemoryRegion mr_regs; + + MemoryRegion mr_m32; + MemoryRegion mr_m64[PNV_PHB3_NUM_M64]; + bool regs_mapped; + bool m32_mapped; + bool m64_mapped[PNV_PHB3_NUM_M64]; + MemoryRegion pci_mmio; + MemoryRegion pci_io; + + uint64_t ioda_LIST[8]; + uint64_t ioda_LXIVT[8]; + uint64_t ioda_TVT[512]; + uint64_t ioda_M64BT[16]; + uint64_t ioda_MDT[256]; + uint64_t ioda_PEEV[4]; + + uint32_t total_irq; + ICSState lsis; + Phb3MsiState msis; + + PnvPBCQState pbcq; + + QLIST_HEAD(, PnvPhb3DMASpace) dma_spaces; +}; + +uint64_t pnv_phb3_reg_read(void *opaque, hwaddr off, unsigned size); +void pnv_phb3_reg_write(void *opaque, hwaddr off, uint64_t val, unsigned size); +void pnv_phb3_update_regions(PnvPHB3 *phb); +void pnv_phb3_remap_irqs(PnvPHB3 *phb); + + +#endif /* PCI_HOST_PNV_PHB3_H */ diff --git a/include/hw/pci-host/pnv_phb3_regs.h b/include/hw/pci-host/pnv_phb3_regs.h new file mode 100644 index 000000000000..46e4e0877d51 --- /dev/null +++ b/include/hw/pci-host/pnv_phb3_regs.h @@ -0,0 +1,432 @@ +/* Copyright (c) 2013-2018, IBM Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + * implied. + * + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef PCI_HOST_PNV_PHB3_REGS_H +#define PCI_HOST_PNV_PHB3_REGS_H + +/* + * PBCQ XSCOM registers + */ + +#define PBCQ_NEST_IRSN_COMPARE 0x1a +#define PBCQ_NEST_IRSN_COMP PPC_BITMASK(0, 18) +#define PBCQ_NEST_IRSN_MASK 0x1b +#define PBCQ_NEST_LSI_SRC_ID 0x1f +#define PBCQ_NEST_LSI_SRC PPC_BITMASK(0, 7) +#define PBCQ_NEST_REGS_COUNT 0x46 +#define PBCQ_NEST_MMIO_BAR0 0x40 +#define PBCQ_NEST_MMIO_BAR1 0x41 +#define PBCQ_NEST_PHB_BAR 0x42 +#define PBCQ_NEST_MMIO_MASK0 0x43 +#define PBCQ_NEST_MMIO_MASK1 0x44 +#define PBCQ_NEST_BAR_EN 0x45 +#define PBCQ_NEST_BAR_EN_MMIO0 PPC_BIT(0) +#define PBCQ_NEST_BAR_EN_MMIO1 PPC_BIT(1) +#define PBCQ_NEST_BAR_EN_PHB PPC_BIT(2) +#define PBCQ_NEST_BAR_EN_IRSN_RX PPC_BIT(3) +#define PBCQ_NEST_BAR_EN_IRSN_TX PPC_BIT(4) + +#define PBCQ_PCI_REGS_COUNT 0x15 +#define PBCQ_PCI_BAR2 0x0b + +#define PBCQ_SPCI_REGS_COUNT 0x5 +#define PBCQ_SPCI_ASB_ADDR 0x0 +#define PBCQ_SPCI_ASB_STATUS 0x1 +#define PBCQ_SPCI_ASB_DATA 0x2 +#define PBCQ_SPCI_AIB_CAPP_EN 0x3 +#define PBCQ_SPCI_CAPP_SEC_TMR 0x4 + +/* + * PHB MMIO registers + */ + +/* PHB Fundamental register set A */ +#define PHB_LSI_SOURCE_ID 0x100 +#define PHB_LSI_SRC_ID PPC_BITMASK(5, 12) +#define PHB_DMA_CHAN_STATUS 0x110 +#define PHB_DMA_CHAN_ANY_ERR PPC_BIT(27) +#define PHB_DMA_CHAN_ANY_ERR1 PPC_BIT(28) +#define PHB_DMA_CHAN_ANY_FREEZE PPC_BIT(29) +#define PHB_CPU_LOADSTORE_STATUS 0x120 +#define PHB_CPU_LS_ANY_ERR PPC_BIT(27) +#define PHB_CPU_LS_ANY_ERR1 PPC_BIT(28) +#define PHB_CPU_LS_ANY_FREEZE PPC_BIT(29) +#define PHB_DMA_MSI_NODE_ID 0x128 +#define PHB_DMAMSI_NID_FIXED PPC_BIT(0) +#define PHB_DMAMSI_NID PPC_BITMASK(24, 31) +#define PHB_CONFIG_DATA 0x130 +#define PHB_LOCK0 0x138 +#define PHB_CONFIG_ADDRESS 0x140 +#define PHB_CA_ENABLE PPC_BIT(0) +#define PHB_CA_BUS PPC_BITMASK(4, 11) +#define PHB_CA_DEV PPC_BITMASK(12, 16) +#define PHB_CA_FUNC PPC_BITMASK(17, 19) +#define PHB_CA_REG PPC_BITMASK(20, 31) +#define PHB_CA_PE PPC_BITMASK(40, 47) +#define PHB_LOCK1 0x148 +#define PHB_IVT_BAR 0x150 +#define PHB_IVT_BAR_ENABLE PPC_BIT(0) +#define PHB_IVT_BASE_ADDRESS_MASK PPC_BITMASK(14, 48) +#define PHB_IVT_LENGTH_MASK PPC_BITMASK(52, 63) +#define PHB_RBA_BAR 0x158 +#define PHB_RBA_BAR_ENABLE PPC_BIT(0) +#define PHB_RBA_BASE_ADDRESS PPC_BITMASK(14, 55) +#define PHB_PHB3_CONFIG 0x160 +#define PHB_PHB3C_64B_TCE_EN PPC_BIT(2) +#define PHB_PHB3C_32BIT_MSI_EN PPC_BIT(8) +#define PHB_PHB3C_64BIT_MSI_EN PPC_BIT(14) +#define PHB_PHB3C_M32_EN PPC_BIT(16) +#define PHB_RTT_BAR 0x168 +#define PHB_RTT_BAR_ENABLE PPC_BIT(0) +#define PHB_RTT_BASE_ADDRESS_MASK PPC_BITMASK(14, 46) +#define PHB_PELTV_BAR 0x188 +#define PHB_PELTV_BAR_ENABLE PPC_BIT(0) +#define PHB_PELTV_BASE_ADDRESS PPC_BITMASK(14, 50) +#define PHB_M32_BASE_ADDR 0x190 +#define PHB_M32_BASE_MASK 0x198 +#define PHB_M32_START_ADDR 0x1a0 +#define PHB_PEST_BAR 0x1a8 +#define PHB_PEST_BAR_ENABLE PPC_BIT(0) +#define PHB_PEST_BASE_ADDRESS PPC_BITMASK(14, 51) +#define PHB_M64_UPPER_BITS 0x1f0 +#define PHB_INTREP_TIMER 0x1f8 +#define PHB_DMARD_SYNC 0x200 +#define PHB_RTC_INVALIDATE 0x208 +#define PHB_RTC_INVALIDATE_ALL PPC_BIT(0) +#define PHB_RTC_INVALIDATE_RID PPC_BITMASK(16, 31) +#define PHB_TCE_KILL 0x210 +#define PHB_TCE_KILL_ALL PPC_BIT(0) +#define PHB_TCE_SPEC_CTL 0x218 +#define PHB_IODA_ADDR 0x220 +#define PHB_IODA_AD_AUTOINC PPC_BIT(0) +#define PHB_IODA_AD_TSEL PPC_BITMASK(11, 15) +#define PHB_IODA_AD_TADR PPC_BITMASK(55, 63) +#define PHB_IODA_DATA0 0x228 +#define PHB_FFI_REQUEST 0x238 +#define PHB_FFI_LOCK_CLEAR PPC_BIT(3) +#define PHB_FFI_REQUEST_ISN PPC_BITMASK(49, 59) +#define PHB_FFI_LOCK 0x240 +#define PHB_FFI_LOCK_STATE PPC_BIT(0) +#define PHB_XIVE_UPDATE 0x248 /* Broken in DD1 */ +#define PHB_PHB3_GEN_CAP 0x250 +#define PHB_PHB3_TCE_CAP 0x258 +#define PHB_PHB3_IRQ_CAP 0x260 +#define PHB_PHB3_EEH_CAP 0x268 +#define PHB_IVC_INVALIDATE 0x2a0 +#define PHB_IVC_INVALIDATE_ALL PPC_BIT(0) +#define PHB_IVC_INVALIDATE_SID PPC_BITMASK(16, 31) +#define PHB_IVC_UPDATE 0x2a8 +#define PHB_IVC_UPDATE_ENABLE_P PPC_BIT(0) +#define PHB_IVC_UPDATE_ENABLE_Q PPC_BIT(1) +#define PHB_IVC_UPDATE_ENABLE_SERVER PPC_BIT(2) +#define PHB_IVC_UPDATE_ENABLE_PRI PPC_BIT(3) +#define PHB_IVC_UPDATE_ENABLE_GEN PPC_BIT(4) +#define PHB_IVC_UPDATE_ENABLE_CON PPC_BIT(5) +#define PHB_IVC_UPDATE_GEN_MATCH PPC_BITMASK(6, 7) +#define PHB_IVC_UPDATE_SERVER PPC_BITMASK(8, 23) +#define PHB_IVC_UPDATE_PRI PPC_BITMASK(24, 31) +#define PHB_IVC_UPDATE_GEN PPC_BITMASK(32, 33) +#define PHB_IVC_UPDATE_P PPC_BITMASK(34, 34) +#define PHB_IVC_UPDATE_Q PPC_BITMASK(35, 35) +#define PHB_IVC_UPDATE_SID PPC_BITMASK(48, 63) +#define PHB_PAPR_ERR_INJ_CTL 0x2b0 +#define PHB_PAPR_ERR_INJ_CTL_INB PPC_BIT(0) +#define PHB_PAPR_ERR_INJ_CTL_OUTB PPC_BIT(1) +#define PHB_PAPR_ERR_INJ_CTL_STICKY PPC_BIT(2) +#define PHB_PAPR_ERR_INJ_CTL_CFG PPC_BIT(3) +#define PHB_PAPR_ERR_INJ_CTL_RD PPC_BIT(4) +#define PHB_PAPR_ERR_INJ_CTL_WR PPC_BIT(5) +#define PHB_PAPR_ERR_INJ_CTL_FREEZE PPC_BIT(6) +#define PHB_PAPR_ERR_INJ_ADDR 0x2b8 +#define PHB_PAPR_ERR_INJ_ADDR_MMIO PPC_BITMASK(16, 63) +#define PHB_PAPR_ERR_INJ_MASK 0x2c0 +#define PHB_PAPR_ERR_INJ_MASK_CFG PPC_BITMASK(4, 11) +#define PHB_PAPR_ERR_INJ_MASK_MMIO PPC_BITMASK(16, 63) +#define PHB_ETU_ERR_SUMMARY 0x2c8 + +/* UTL registers */ +#define UTL_SYS_BUS_CONTROL 0x400 +#define UTL_STATUS 0x408 +#define UTL_SYS_BUS_AGENT_STATUS 0x410 +#define UTL_SYS_BUS_AGENT_ERR_SEVERITY 0x418 +#define UTL_SYS_BUS_AGENT_IRQ_EN 0x420 +#define UTL_SYS_BUS_BURST_SZ_CONF 0x440 +#define UTL_REVISION_ID 0x448 +#define UTL_BCLK_DOMAIN_DBG1 0x460 +#define UTL_BCLK_DOMAIN_DBG2 0x468 +#define UTL_BCLK_DOMAIN_DBG3 0x470 +#define UTL_BCLK_DOMAIN_DBG4 0x478 +#define UTL_BCLK_DOMAIN_DBG5 0x480 +#define UTL_BCLK_DOMAIN_DBG6 0x488 +#define UTL_OUT_POST_HDR_BUF_ALLOC 0x4c0 +#define UTL_OUT_POST_DAT_BUF_ALLOC 0x4d0 +#define UTL_IN_POST_HDR_BUF_ALLOC 0x4e0 +#define UTL_IN_POST_DAT_BUF_ALLOC 0x4f0 +#define UTL_OUT_NP_BUF_ALLOC 0x500 +#define UTL_IN_NP_BUF_ALLOC 0x510 +#define UTL_PCIE_TAGS_ALLOC 0x520 +#define UTL_GBIF_READ_TAGS_ALLOC 0x530 +#define UTL_PCIE_PORT_CONTROL 0x540 +#define UTL_PCIE_PORT_STATUS 0x548 +#define UTL_PCIE_PORT_ERROR_SEV 0x550 +#define UTL_PCIE_PORT_IRQ_EN 0x558 +#define UTL_RC_STATUS 0x560 +#define UTL_RC_ERR_SEVERITY 0x568 +#define UTL_RC_IRQ_EN 0x570 +#define UTL_EP_STATUS 0x578 +#define UTL_EP_ERR_SEVERITY 0x580 +#define UTL_EP_ERR_IRQ_EN 0x588 +#define UTL_PCI_PM_CTRL1 0x590 +#define UTL_PCI_PM_CTRL2 0x598 +#define UTL_GP_CTL1 0x5a0 +#define UTL_GP_CTL2 0x5a8 +#define UTL_PCLK_DOMAIN_DBG1 0x5b0 +#define UTL_PCLK_DOMAIN_DBG2 0x5b8 +#define UTL_PCLK_DOMAIN_DBG3 0x5c0 +#define UTL_PCLK_DOMAIN_DBG4 0x5c8 + +/* PCI-E Stack registers */ +#define PHB_PCIE_SYSTEM_CONFIG 0x600 +#define PHB_PCIE_BUS_NUMBER 0x608 +#define PHB_PCIE_SYSTEM_TEST 0x618 +#define PHB_PCIE_LINK_MANAGEMENT 0x630 +#define PHB_PCIE_LM_LINK_ACTIVE PPC_BIT(8) +#define PHB_PCIE_DLP_TRAIN_CTL 0x640 +#define PHB_PCIE_DLP_TCTX_DISABLE PPC_BIT(1) +#define PHB_PCIE_DLP_TCRX_DISABLED PPC_BIT(16) +#define PHB_PCIE_DLP_INBAND_PRESENCE PPC_BIT(19) +#define PHB_PCIE_DLP_TC_DL_LINKUP PPC_BIT(21) +#define PHB_PCIE_DLP_TC_DL_PGRESET PPC_BIT(22) +#define PHB_PCIE_DLP_TC_DL_LINKACT PPC_BIT(23) +#define PHB_PCIE_SLOP_LOOPBACK_STATUS 0x648 +#define PHB_PCIE_SYS_LINK_INIT 0x668 +#define PHB_PCIE_UTL_CONFIG 0x670 +#define PHB_PCIE_DLP_CONTROL 0x678 +#define PHB_PCIE_UTL_ERRLOG1 0x680 +#define PHB_PCIE_UTL_ERRLOG2 0x688 +#define PHB_PCIE_UTL_ERRLOG3 0x690 +#define PHB_PCIE_UTL_ERRLOG4 0x698 +#define PHB_PCIE_DLP_ERRLOG1 0x6a0 +#define PHB_PCIE_DLP_ERRLOG2 0x6a8 +#define PHB_PCIE_DLP_ERR_STATUS 0x6b0 +#define PHB_PCIE_DLP_ERR_COUNTERS 0x6b8 +#define PHB_PCIE_UTL_ERR_INJECT 0x6c0 +#define PHB_PCIE_TLDLP_ERR_INJECT 0x6c8 +#define PHB_PCIE_LANE_EQ_CNTL0 0x6d0 +#define PHB_PCIE_LANE_EQ_CNTL1 0x6d8 +#define PHB_PCIE_LANE_EQ_CNTL2 0x6e0 +#define PHB_PCIE_LANE_EQ_CNTL3 0x6e8 +#define PHB_PCIE_STRAPPING 0x700 + +/* Fundamental register set B */ +#define PHB_VERSION 0x800 +#define PHB_RESET 0x808 +#define PHB_CONTROL 0x810 +#define PHB_CTRL_IVE_128_BYTES PPC_BIT(24) +#define PHB_AIB_RX_CRED_INIT_TIMER 0x818 +#define PHB_AIB_RX_CMD_CRED 0x820 +#define PHB_AIB_RX_DATA_CRED 0x828 +#define PHB_AIB_TX_CMD_CRED 0x830 +#define PHB_AIB_TX_DATA_CRED 0x838 +#define PHB_AIB_TX_CHAN_MAPPING 0x840 +#define PHB_AIB_TAG_ENABLE 0x858 +#define PHB_AIB_FENCE_CTRL 0x860 +#define PHB_TCE_TAG_ENABLE 0x868 +#define PHB_TCE_WATERMARK 0x870 +#define PHB_TIMEOUT_CTRL1 0x878 +#define PHB_TIMEOUT_CTRL2 0x880 +#define PHB_QUIESCE_DMA_G 0x888 +#define PHB_AIB_TAG_STATUS 0x900 +#define PHB_TCE_TAG_STATUS 0x908 + +/* FIR & Error registers */ +#define PHB_LEM_FIR_ACCUM 0xc00 +#define PHB_LEM_FIR_AND_MASK 0xc08 +#define PHB_LEM_FIR_OR_MASK 0xc10 +#define PHB_LEM_ERROR_MASK 0xc18 +#define PHB_LEM_ERROR_AND_MASK 0xc20 +#define PHB_LEM_ERROR_OR_MASK 0xc28 +#define PHB_LEM_ACTION0 0xc30 +#define PHB_LEM_ACTION1 0xc38 +#define PHB_LEM_WOF 0xc40 +#define PHB_ERR_STATUS 0xc80 +#define PHB_ERR1_STATUS 0xc88 +#define PHB_ERR_INJECT 0xc90 +#define PHB_ERR_LEM_ENABLE 0xc98 +#define PHB_ERR_IRQ_ENABLE 0xca0 +#define PHB_ERR_FREEZE_ENABLE 0xca8 +#define PHB_ERR_AIB_FENCE_ENABLE 0xcb0 +#define PHB_ERR_LOG_0 0xcc0 +#define PHB_ERR_LOG_1 0xcc8 +#define PHB_ERR_STATUS_MASK 0xcd0 +#define PHB_ERR1_STATUS_MASK 0xcd8 + +#define PHB_OUT_ERR_STATUS 0xd00 +#define PHB_OUT_ERR1_STATUS 0xd08 +#define PHB_OUT_ERR_INJECT 0xd10 +#define PHB_OUT_ERR_LEM_ENABLE 0xd18 +#define PHB_OUT_ERR_IRQ_ENABLE 0xd20 +#define PHB_OUT_ERR_FREEZE_ENABLE 0xd28 +#define PHB_OUT_ERR_AIB_FENCE_ENABLE 0xd30 +#define PHB_OUT_ERR_LOG_0 0xd40 +#define PHB_OUT_ERR_LOG_1 0xd48 +#define PHB_OUT_ERR_STATUS_MASK 0xd50 +#define PHB_OUT_ERR1_STATUS_MASK 0xd58 + +#define PHB_INA_ERR_STATUS 0xd80 +#define PHB_INA_ERR1_STATUS 0xd88 +#define PHB_INA_ERR_INJECT 0xd90 +#define PHB_INA_ERR_LEM_ENABLE 0xd98 +#define PHB_INA_ERR_IRQ_ENABLE 0xda0 +#define PHB_INA_ERR_FREEZE_ENABLE 0xda8 +#define PHB_INA_ERR_AIB_FENCE_ENABLE 0xdb0 +#define PHB_INA_ERR_LOG_0 0xdc0 +#define PHB_INA_ERR_LOG_1 0xdc8 +#define PHB_INA_ERR_STATUS_MASK 0xdd0 +#define PHB_INA_ERR1_STATUS_MASK 0xdd8 + +#define PHB_INB_ERR_STATUS 0xe00 +#define PHB_INB_ERR1_STATUS 0xe08 +#define PHB_INB_ERR_INJECT 0xe10 +#define PHB_INB_ERR_LEM_ENABLE 0xe18 +#define PHB_INB_ERR_IRQ_ENABLE 0xe20 +#define PHB_INB_ERR_FREEZE_ENABLE 0xe28 +#define PHB_INB_ERR_AIB_FENCE_ENABLE 0xe30 +#define PHB_INB_ERR_LOG_0 0xe40 +#define PHB_INB_ERR_LOG_1 0xe48 +#define PHB_INB_ERR_STATUS_MASK 0xe50 +#define PHB_INB_ERR1_STATUS_MASK 0xe58 + +/* Performance monitor & Debug registers */ +#define PHB_TRACE_CONTROL 0xf80 +#define PHB_PERFMON_CONFIG 0xf88 +#define PHB_PERFMON_CTR0 0xf90 +#define PHB_PERFMON_CTR1 0xf98 +#define PHB_PERFMON_CTR2 0xfa0 +#define PHB_PERFMON_CTR3 0xfa8 +#define PHB_HOTPLUG_OVERRIDE 0xfb0 +#define PHB_HPOVR_FORCE_RESAMPLE PPC_BIT(9) +#define PHB_HPOVR_PRESENCE_A PPC_BIT(10) +#define PHB_HPOVR_PRESENCE_B PPC_BIT(11) +#define PHB_HPOVR_LINK_ACTIVE PPC_BIT(12) +#define PHB_HPOVR_LINK_BIFURCATED PPC_BIT(13) +#define PHB_HPOVR_LINK_LANE_SWAPPED PPC_BIT(14) + +/* + * IODA2 on-chip tables + */ + +#define IODA2_TBL_LIST 1 +#define IODA2_TBL_LXIVT 2 +#define IODA2_TBL_IVC_CAM 3 +#define IODA2_TBL_RBA 4 +#define IODA2_TBL_RCAM 5 +#define IODA2_TBL_MRT 6 +#define IODA2_TBL_PESTA 7 +#define IODA2_TBL_PESTB 8 +#define IODA2_TBL_TVT 9 +#define IODA2_TBL_TCAM 10 +#define IODA2_TBL_TDR 11 +#define IODA2_TBL_M64BT 16 +#define IODA2_TBL_M32DT 17 +#define IODA2_TBL_PEEV 20 + +/* LXIVT */ +#define IODA2_LXIVT_SERVER PPC_BITMASK(8, 23) +#define IODA2_LXIVT_PRIORITY PPC_BITMASK(24, 31) +#define IODA2_LXIVT_NODE_ID PPC_BITMASK(56, 63) + +/* IVT */ +#define IODA2_IVT_SERVER PPC_BITMASK(0, 23) +#define IODA2_IVT_PRIORITY PPC_BITMASK(24, 31) +#define IODA2_IVT_GEN PPC_BITMASK(37, 38) +#define IODA2_IVT_P PPC_BITMASK(39, 39) +#define IODA2_IVT_Q PPC_BITMASK(47, 47) +#define IODA2_IVT_PE PPC_BITMASK(48, 63) + +/* TVT */ +#define IODA2_TVT_TABLE_ADDR PPC_BITMASK(0, 47) +#define IODA2_TVT_NUM_LEVELS PPC_BITMASK(48, 50) +#define IODA2_TVE_1_LEVEL 0 +#define IODA2_TVE_2_LEVELS 1 +#define IODA2_TVE_3_LEVELS 2 +#define IODA2_TVE_4_LEVELS 3 +#define IODA2_TVE_5_LEVELS 4 +#define IODA2_TVT_TCE_TABLE_SIZE PPC_BITMASK(51, 55) +#define IODA2_TVT_IO_PSIZE PPC_BITMASK(59, 63) + +/* PESTA */ +#define IODA2_PESTA_MMIO_FROZEN PPC_BIT(0) + +/* PESTB */ +#define IODA2_PESTB_DMA_STOPPED PPC_BIT(0) + +/* M32DT */ +#define IODA2_M32DT_PE PPC_BITMASK(8, 15) + +/* M64BT */ +#define IODA2_M64BT_ENABLE PPC_BIT(0) +#define IODA2_M64BT_SINGLE_PE PPC_BIT(1) +#define IODA2_M64BT_BASE PPC_BITMASK(2, 31) +#define IODA2_M64BT_MASK PPC_BITMASK(34, 63) +#define IODA2_M64BT_SINGLE_BASE PPC_BITMASK(2, 26) +#define IODA2_M64BT_PE_HI PPC_BITMASK(27, 31) +#define IODA2_M64BT_SINGLE_MASK PPC_BITMASK(34, 58) +#define IODA2_M64BT_PE_LOW PPC_BITMASK(59, 63) + +/* + * IODA2 in-memory tables + */ + +/* PEST + * + * 2x8 bytes entries, PEST0 and PEST1 + */ + +#define IODA2_PEST0_MMIO_CAUSE PPC_BIT(2) +#define IODA2_PEST0_CFG_READ PPC_BIT(3) +#define IODA2_PEST0_CFG_WRITE PPC_BIT(4) +#define IODA2_PEST0_TTYPE PPC_BITMASK(5, 7) +#define PEST_TTYPE_DMA_WRITE 0 +#define PEST_TTYPE_MSI 1 +#define PEST_TTYPE_DMA_READ 2 +#define PEST_TTYPE_DMA_READ_RESP 3 +#define PEST_TTYPE_MMIO_LOAD 4 +#define PEST_TTYPE_MMIO_STORE 5 +#define PEST_TTYPE_OTHER 7 +#define IODA2_PEST0_CA_RETURN PPC_BIT(8) +#define IODA2_PEST0_UTL_RTOS_TIMEOUT PPC_BIT(8) /* Same bit as CA return */ +#define IODA2_PEST0_UR_RETURN PPC_BIT(9) +#define IODA2_PEST0_UTL_NONFATAL PPC_BIT(10) +#define IODA2_PEST0_UTL_FATAL PPC_BIT(11) +#define IODA2_PEST0_PARITY_UE PPC_BIT(13) +#define IODA2_PEST0_UTL_CORRECTABLE PPC_BIT(14) +#define IODA2_PEST0_UTL_INTERRUPT PPC_BIT(15) +#define IODA2_PEST0_MMIO_XLATE PPC_BIT(16) +#define IODA2_PEST0_IODA2_ERROR PPC_BIT(16) /* Same bit as MMIO xlate */ +#define IODA2_PEST0_TCE_PAGE_FAULT PPC_BIT(18) +#define IODA2_PEST0_TCE_ACCESS_FAULT PPC_BIT(19) +#define IODA2_PEST0_DMA_RESP_TIMEOUT PPC_BIT(20) +#define IODA2_PEST0_AIB_SIZE_INVALID PPC_BIT(21) +#define IODA2_PEST0_LEM_BIT PPC_BITMASK(26, 31) +#define IODA2_PEST0_RID PPC_BITMASK(32, 47) +#define IODA2_PEST0_MSI_DATA PPC_BITMASK(48, 63) + +#define IODA2_PEST1_FAIL_ADDR PPC_BITMASK(3, 63) + + +#endif /* PCI_HOST_PNV_PHB3_REGS_H */ diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h index 86d5f54e5459..61af65fc0b76 100644 --- a/include/hw/ppc/pnv.h +++ b/include/hw/ppc/pnv.h @@ -25,6 +25,7 @@ #include "hw/ppc/pnv_lpc.h" #include "hw/ppc/pnv_psi.h" #include "hw/ppc/pnv_occ.h" +#include "hw/pci-host/pnv_phb3.h" #define TYPE_PNV_CHIP "pnv-chip" #define PNV_CHIP(obj) OBJECT_CHECK(PnvChip, (obj), TYPE_PNV_CHIP) @@ -72,6 +73,9 @@ typedef struct Pnv8Chip { PnvLpcController lpc; PnvPsi psi; PnvOCC occ; + +#define PNV8_CHIP_PHB3_MAX 4 + PnvPHB3 phbs[PNV8_CHIP_PHB3_MAX]; } Pnv8Chip; #define TYPE_PNV9_CHIP "pnv9-chip" @@ -92,6 +96,7 @@ typedef struct PnvChipClass { PnvChipType chip_type; uint64_t chip_cfam_id; uint64_t cores_mask; + uint32_t num_phbs; hwaddr xscom_base; @@ -171,6 +176,23 @@ static inline bool pnv_is_power9(PnvMachineState *pnv) return pnv_chip_is_power9(pnv->chips[0]); } +/* + * This is used by devices created on the command line to find a chip + * on which to attach to. + */ +static inline PnvChip *pnv_get_chip(PnvMachineState *pnv, uint chip_id) +{ + int i; + + for (i = 0; i < pnv->num_chips; i++) { + PnvChip *chip = pnv->chips[i]; + if (chip->chip_id == chip_id) { + return chip; + } + } + return NULL; +} + #define PNV_FDT_ADDR 0x01000000 #define PNV_TIMEBASE_FREQ 512000000ULL diff --git a/include/hw/ppc/pnv_xscom.h b/include/hw/ppc/pnv_xscom.h index 255b26a5aaf6..e7ebffa5cf8a 100644 --- a/include/hw/ppc/pnv_xscom.h +++ b/include/hw/ppc/pnv_xscom.h @@ -73,6 +73,15 @@ typedef struct PnvXScomInterfaceClass { #define PNV_XSCOM_OCC_BASE 0x0066000 #define PNV_XSCOM_OCC_SIZE 0x6000 +#define PNV_XSCOM_PBCQ_NEST_BASE 0x2012000 +#define PNV_XSCOM_PBCQ_NEST_SIZE 0x46 + +#define PNV_XSCOM_PBCQ_PCI_BASE 0x9012000 +#define PNV_XSCOM_PBCQ_PCI_SIZE 0x15 + +#define PNV_XSCOM_PBCQ_SPCI_BASE 0x9013c00 +#define PNV_XSCOM_PBCQ_SPCI_SIZE 0x5 + extern void pnv_xscom_realize(PnvChip *chip, Error **errp); extern int pnv_dt_xscom(PnvChip *chip, void *fdt, int offset); diff --git a/include/hw/ppc/xics.h b/include/hw/ppc/xics.h index 6ac8a9392da6..966a996c2eac 100644 --- a/include/hw/ppc/xics.h +++ b/include/hw/ppc/xics.h @@ -194,6 +194,7 @@ void icp_set_mfrr(ICPState *icp, uint8_t mfrr); uint32_t icp_accept(ICPState *ss); uint32_t icp_ipoll(ICPState *ss, uint32_t *mfrr); void icp_eoi(ICPState *icp, uint32_t xirr); +void icp_irq(ICSState *ics, int server, int nr, uint8_t priority); void ics_simple_write_xive(ICSState *ics, int nr, int server, uint8_t priority, uint8_t saved_priority); diff --git a/hw/intc/xics.c b/hw/intc/xics.c index c90c893228dc..30fd9eb47e90 100644 --- a/hw/intc/xics.c +++ b/hw/intc/xics.c @@ -230,7 +230,7 @@ void icp_eoi(ICPState *icp, uint32_t xirr) } } -static void icp_irq(ICSState *ics, int server, int nr, uint8_t priority) +void icp_irq(ICSState *ics, int server, int nr, uint8_t priority) { ICPState *icp = xics_icp_get(ics->xics, server); diff --git a/hw/pci-host/pnv_phb3.c b/hw/pci-host/pnv_phb3.c new file mode 100644 index 000000000000..67f6dd94a705 --- /dev/null +++ b/hw/pci-host/pnv_phb3.c @@ -0,0 +1,1146 @@ +/* + * QEMU PowerPC PowerNV PHB3 model + * + * Copyright (c) 2014-2018, IBM Corporation. + * + * This code is licensed under the GPL version 2 or later. See the + * COPYING file in the top-level directory. + */ +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "qapi/visitor.h" +#include "qapi/error.h" +#include "qemu-common.h" +#include "target/ppc/cpu.h" +#include "hw/pci-host/pnv_phb3_regs.h" +#include "hw/pci-host/pnv_phb3.h" +#include "hw/pci/pcie_host.h" +#include "hw/pci/pcie_port.h" +#include "hw/ppc/pnv.h" + +#define phb3_error(fmt, ...) \ + qemu_log_mask(LOG_GUEST_ERROR, "phb3: " fmt "\n", ## __VA_ARGS__) + + +static PCIDevice *pnv_phb3_find_cfg_dev(PnvPHB3 *phb) +{ + PCIHostState *pci = PCI_HOST_BRIDGE(phb); + uint64_t addr = phb->regs[PHB_CONFIG_ADDRESS >> 3]; + uint8_t bus, devfn; + + if (!(addr >> 63)) { + return NULL; + } + bus = (addr >> 52) & 0xff; + devfn = (addr >> 44) & 0xff; + + return pci_find_device(pci->bus, bus, devfn); +} + +/* + * The CONFIG_DATA register expects little endian accesses, but as the + * region is big endian, we have to swap the value. + */ +static void pnv_phb3_config_write(PnvPHB3 *phb, unsigned off, + unsigned size, uint64_t val) +{ + uint32_t cfg_addr, limit; + PCIDevice *pdev; + + pdev = pnv_phb3_find_cfg_dev(phb); + if (!pdev) { + return; + } + cfg_addr = (phb->regs[PHB_CONFIG_ADDRESS >> 3] >> 32) & 0xffc; + cfg_addr |= off; + limit = pci_config_size(pdev); + if (limit <= cfg_addr) { + /* conventional pci device can be behind pcie-to-pci bridge. + 256 <= addr < 4K has no effects. */ + return; + } + switch (size) { + case 1: + break; + case 2: + val = bswap16(val); + break; + case 4: + val = bswap32(val); + break; + default: + g_assert_not_reached(); + } + pci_host_config_write_common(pdev, cfg_addr, limit, val, size); +} + +static uint64_t pnv_phb3_config_read(PnvPHB3 *phb, unsigned off, + unsigned size) +{ + uint32_t cfg_addr, limit; + PCIDevice *pdev; + uint64_t val; + + pdev = pnv_phb3_find_cfg_dev(phb); + if (!pdev) { + return ~0ull; + } + cfg_addr = (phb->regs[PHB_CONFIG_ADDRESS >> 3] >> 32) & 0xffc; + cfg_addr |= off; + limit = pci_config_size(pdev); + if (limit <= cfg_addr) { + /* conventional pci device can be behind pcie-to-pci bridge. + 256 <= addr < 4K has no effects. */ + return ~0ull; + } + val = pci_host_config_read_common(pdev, cfg_addr, limit, size); + switch (size) { + case 1: + return val; + case 2: + return bswap16(val); + case 4: + return bswap32(val); + default: + g_assert_not_reached(); + } +} + +static void pnv_phb3_check_m32(PnvPHB3 *phb) +{ + uint64_t base, start, size; + MemoryRegion *parent; + PnvPBCQState *pbcq = &phb->pbcq; + + if (phb->m32_mapped) { + memory_region_del_subregion(phb->mr_m32.container, &phb->mr_m32); + phb->m32_mapped = false; + } + + if (!(phb->regs[PHB_PHB3_CONFIG >> 3] & PHB_PHB3C_M32_EN)) { + return; + } + + /* Grab geometry from registers */ + base = phb->regs[PHB_M32_BASE_ADDR >> 3]; + start = phb->regs[PHB_M32_START_ADDR >> 3]; + size = ~(phb->regs[PHB_M32_BASE_MASK >> 3] | 0xfffc000000000000ull) + 1; + + /* Check if it matches an enabled MMIO region in the PBCQ */ + if (pbcq->mmio0_mapped && base >= pbcq->mmio0_base && + (base + size) <= (pbcq->mmio0_base + pbcq->mmio0_size)) { + parent = &pbcq->mmbar0; + base -= pbcq->mmio0_base; + } else if (pbcq->mmio1_mapped && base >= pbcq->mmio1_base && + (base + size) <= (pbcq->mmio1_base + pbcq->mmio1_size)) { + parent = &pbcq->mmbar1; + base -= pbcq->mmio1_base; + } else { + return; + } + + /* Create alias */ + memory_region_init_alias(&phb->mr_m32, OBJECT(phb), "phb3-m32", + &phb->pci_mmio, start, size); + memory_region_add_subregion(parent, base, &phb->mr_m32); + phb->m32_mapped = true; +} + +static void pnv_phb3_check_m64(PnvPHB3 *phb, uint32_t index) +{ + uint64_t base, start, size, m64; + MemoryRegion *parent; + PnvPBCQState *pbcq = &phb->pbcq; + + if (phb->m64_mapped[index]) { + /* Should we destroy it in RCU friendly way... ? */ + memory_region_del_subregion(phb->mr_m64[index].container, + &phb->mr_m64[index]); + phb->m64_mapped[index] = false; + } + + /* Get table entry */ + m64 = phb->ioda_M64BT[index]; + + if (!(m64 & IODA2_M64BT_ENABLE)) { + return; + } + + /* Grab geometry from registers */ + base = GETFIELD(IODA2_M64BT_BASE, m64) << 20; + if (m64 & IODA2_M64BT_SINGLE_PE) { + base &= ~0x1ffffffull; + } + size = GETFIELD(IODA2_M64BT_MASK, m64) << 20; + size |= 0xfffc000000000000ull; + size = ~size + 1; + start = base | (phb->regs[PHB_M64_UPPER_BITS >> 3]); + + /* Check if it matches an enabled MMIO region in the PBCQ */ + if (pbcq->mmio0_mapped && base >= pbcq->mmio0_base && + (base + size) <= (pbcq->mmio0_base + pbcq->mmio0_size)) { + parent = &pbcq->mmbar0; + base -= pbcq->mmio0_base; + } else if (pbcq->mmio1_mapped && base >= pbcq->mmio1_base && + (base + size) <= (pbcq->mmio1_base + pbcq->mmio1_size)) { + parent = &pbcq->mmbar1; + base -= pbcq->mmio1_base; + } else { + return; + } + + /* Create alias */ + memory_region_init_alias(&phb->mr_m64[index], OBJECT(phb), "phb3-m64", + &phb->pci_mmio, start, size); + memory_region_add_subregion(parent, base, &phb->mr_m64[index]); + phb->m64_mapped[index] = true; +} + +static void pnv_phb3_check_all_m64s(PnvPHB3 *phb) +{ + uint64_t i; + + for (i = 0; i < PNV_PHB3_NUM_M64; i++) { + pnv_phb3_check_m64(phb, i); + } +} + +static void pnv_phb3_lxivt_write(PnvPHB3 *phb, unsigned idx, uint64_t val) +{ + ICSState *ics = &phb->lsis; + uint8_t server, prio; + + phb->ioda_LXIVT[idx] = val & (IODA2_LXIVT_SERVER | + IODA2_LXIVT_PRIORITY | + IODA2_LXIVT_NODE_ID); + server = GETFIELD(IODA2_LXIVT_SERVER, val); + prio = GETFIELD(IODA2_LXIVT_PRIORITY, val); + + /* + * The low order 2 bits are the link pointer (Type II interrupts). + * Shift back to get a valid IRQ server. + */ + server >>= 2; + + ics_simple_write_xive(ics, idx, server, prio, prio); +} + +static uint64_t *pnv_phb3_ioda_access(PnvPHB3 *phb, + unsigned *out_table, unsigned *out_idx) +{ + uint64_t adreg = phb->regs[PHB_IODA_ADDR >> 3]; + unsigned int index = GETFIELD(PHB_IODA_AD_TADR, adreg); + unsigned int table = GETFIELD(PHB_IODA_AD_TSEL, adreg); + unsigned int mask; + uint64_t *tptr = NULL; + + switch (table) { + case IODA2_TBL_LIST: + tptr = phb->ioda_LIST; + mask = 7; + break; + case IODA2_TBL_LXIVT: + tptr = phb->ioda_LXIVT; + mask = 7; + break; + case IODA2_TBL_IVC_CAM: + case IODA2_TBL_RBA: + mask = 31; + break; + case IODA2_TBL_RCAM: + mask = 63; + break; + case IODA2_TBL_MRT: + mask = 7; + break; + case IODA2_TBL_PESTA: + case IODA2_TBL_PESTB: + mask = 255; + break; + case IODA2_TBL_TVT: + tptr = phb->ioda_TVT; + mask = 511; + break; + case IODA2_TBL_TCAM: + case IODA2_TBL_TDR: + mask = 63; + break; + case IODA2_TBL_M64BT: + tptr = phb->ioda_M64BT; + mask = 15; + break; + case IODA2_TBL_M32DT: + tptr = phb->ioda_MDT; + mask = 255; + break; + case IODA2_TBL_PEEV: + tptr = phb->ioda_PEEV; + mask = 3; + break; + default: + phb3_error("invalid IODA table %d", table); + return NULL; + } + index &= mask; + if (out_idx) { + *out_idx = index; + } + if (out_table) { + *out_table = table; + } + if (adreg & PHB_IODA_AD_AUTOINC) { + index = (index + 1) & mask; + adreg = SETFIELD(PHB_IODA_AD_TADR, adreg, index); + } + if (tptr) { + tptr += index; + } + phb->regs[PHB_IODA_ADDR >> 3] = adreg; + return tptr; +} + +static uint64_t pnv_phb3_ioda_read(PnvPHB3 *phb) +{ + unsigned table; + uint64_t *tptr; + + tptr = pnv_phb3_ioda_access(phb, &table, NULL); + if (!tptr) { + /* Return 0 on unsupported tables, not ff's */ + return 0; + } + return *tptr; +} + +static void pnv_phb3_ioda_write(PnvPHB3 *phb, uint64_t val) +{ + unsigned table, idx; + uint64_t *tptr; + + tptr = pnv_phb3_ioda_access(phb, &table, &idx); + if (!tptr) { + return; + } + + /* Handle side effects */ + switch (table) { + case IODA2_TBL_LXIVT: + pnv_phb3_lxivt_write(phb, idx, val); + break; + case IODA2_TBL_M64BT: + *tptr = val; + pnv_phb3_check_m64(phb, idx); + break; + default: + *tptr = val; + } +} + +/* This is called whenever the PHB LSI, MSI source ID register or + * the PBCQ irq filters are written. + */ +void pnv_phb3_remap_irqs(PnvPHB3 *phb) +{ + ICSState *ics = &phb->lsis; + uint32_t local, global, count, mask, comp; + uint64_t baren; + PnvPBCQState *pbcq = &phb->pbcq; + + /* First check if we are enabled. Unlike real HW we don't separate TX and RX + * so we enable if both are set + */ + baren = pbcq->nest_regs[PBCQ_NEST_BAR_EN]; + if (!(baren & PBCQ_NEST_BAR_EN_IRSN_RX) || + !(baren & PBCQ_NEST_BAR_EN_IRSN_TX)) { + ics->offset = 0; + return; + } + + /* Grab local LSI source ID */ + local = GETFIELD(PHB_LSI_SRC_ID, phb->regs[PHB_LSI_SOURCE_ID >> 3]) << 3; + + /* Grab global one and compare */ + global = GETFIELD(PBCQ_NEST_LSI_SRC, + pbcq->nest_regs[PBCQ_NEST_LSI_SRC_ID]) << 3; + if (global != local) { + /* This happens during initialization, let's come back when we + * are properly configured + */ + ics->offset = 0; + return; + } + + /* Get the base on the powerbus */ + comp = GETFIELD(PBCQ_NEST_IRSN_COMP, + pbcq->nest_regs[PBCQ_NEST_IRSN_COMPARE]); + mask = GETFIELD(PBCQ_NEST_IRSN_COMP, + pbcq->nest_regs[PBCQ_NEST_IRSN_MASK]); + count = ((~mask) + 1) & 0x7ffff; + phb->total_irq = count; + + /* Sanity checks */ + if ((global + PNV_PHB3_NUM_LSI) > count) { + phb3_error("LSIs out of reach: LSI base=%d total irq=%d", global, + count); + } + + if (count > 2048) { + phb3_error("More interrupts than supported: %d", count); + } + + if ((comp & mask) != comp) { + phb3_error("IRQ compare bits not in mask: comp=0x%x mask=0x%x", + comp, mask); + comp &= mask; + } + /* Setup LSI offset */ + ics->offset = comp + global; + + /* Setup MSI offset */ + pnv_phb3_msi_update_config(&phb->msis, comp, count - PNV_PHB3_NUM_LSI); +} + +static void pnv_phb3_lsi_src_id_write(PnvPHB3 *phb, uint64_t val) +{ + /* Sanitize content */ + val &= PHB_LSI_SRC_ID; + phb->regs[PHB_LSI_SOURCE_ID >> 3] = val; + pnv_phb3_remap_irqs(phb); +} + +static void pnv_phb3_rtc_invalidate(PnvPHB3 *phb, uint64_t val) +{ + PnvPhb3DMASpace *ds; + + /* Always invalidate all for now ... */ + QLIST_FOREACH(ds, &phb->dma_spaces, list) { + ds->pe_num = PHB_INVALID_PE; + } +} + + +static void pnv_phb3_update_msi_regions(PnvPhb3DMASpace *ds) +{ + uint64_t cfg = ds->phb->regs[PHB_PHB3_CONFIG >> 3]; + + if (cfg & PHB_PHB3C_32BIT_MSI_EN) { + if (!ds->msi32_mapped) { + memory_region_add_subregion(MEMORY_REGION(&ds->dma_mr), + 0xffff0000, &ds->msi32_mr); + ds->msi32_mapped = true; + } + } else { + if (ds->msi32_mapped) { + memory_region_del_subregion(MEMORY_REGION(&ds->dma_mr), + &ds->msi32_mr); + ds->msi32_mapped = false; + } + } + + if (cfg & PHB_PHB3C_64BIT_MSI_EN) { + if (!ds->msi64_mapped) { + memory_region_add_subregion(MEMORY_REGION(&ds->dma_mr), + (1ull << 60), &ds->msi64_mr); + ds->msi64_mapped = true; + } + } else { + if (ds->msi64_mapped) { + memory_region_del_subregion(MEMORY_REGION(&ds->dma_mr), + &ds->msi64_mr); + ds->msi64_mapped = false; + } + } +} + +static void pnv_phb3_update_all_msi_regions(PnvPHB3 *phb) +{ + PnvPhb3DMASpace *ds; + + QLIST_FOREACH(ds, &phb->dma_spaces, list) { + pnv_phb3_update_msi_regions(ds); + } +} + +void pnv_phb3_reg_write(void *opaque, hwaddr off, uint64_t val, unsigned size) +{ + PnvPHB3 *phb = opaque; + bool changed; + + /* Special case configuration data */ + if ((off & 0xfffc) == PHB_CONFIG_DATA) { + pnv_phb3_config_write(phb, off & 0x3, size, val); + return; + } + + /* Other registers are 64-bit only */ + if (size != 8 || off & 0x7) { + phb3_error("Invalid register access, offset: 0x%"PRIx64" size: %d", + off, size); + return; + } + + /* Handle masking */ + switch (off) { + case PHB_M64_UPPER_BITS: + val &= 0xfffc000000000000ull; + break; + } + + /* Record whether it changed */ + changed = phb->regs[off >> 3] != val; + + /* Store in register cache first */ + phb->regs[off >> 3] = val; + + /* Handle side effects */ + switch (off) { + case PHB_PHB3_CONFIG: + if (changed) { + pnv_phb3_update_all_msi_regions(phb); + } + /* fall through */ + case PHB_M32_BASE_ADDR: + case PHB_M32_BASE_MASK: + case PHB_M32_START_ADDR: + if (changed) { + pnv_phb3_check_m32(phb); + } + break; + case PHB_M64_UPPER_BITS: + if (changed) { + pnv_phb3_check_all_m64s(phb); + } + break; + case PHB_LSI_SOURCE_ID: + if (changed) { + pnv_phb3_lsi_src_id_write(phb, val); + } + break; + + /* IODA table accesses */ + case PHB_IODA_DATA0: + pnv_phb3_ioda_write(phb, val); + break; + + /* RTC invalidation */ + case PHB_RTC_INVALIDATE: + pnv_phb3_rtc_invalidate(phb, val); + break; + + /* FFI request */ + case PHB_FFI_REQUEST: + pnv_phb3_msi_ffi(&phb->msis, val); + break; + + /* Silent simple writes */ + case PHB_CONFIG_ADDRESS: + case PHB_IODA_ADDR: + case PHB_TCE_KILL: + case PHB_TCE_SPEC_CTL: + case PHB_PEST_BAR: + case PHB_PELTV_BAR: + case PHB_RTT_BAR: + case PHB_RBA_BAR: + case PHB_IVT_BAR: + case PHB_FFI_LOCK: + break; + + /* Noise on anything else */ + default: + qemu_log_mask(LOG_UNIMP, "phb3: reg_write 0x%"PRIx64"=%"PRIx64"\n", + off, val); + } +} + +uint64_t pnv_phb3_reg_read(void *opaque, hwaddr off, unsigned size) +{ + PnvPHB3 *phb = opaque; + uint64_t val; + + if ((off & 0xfffc) == PHB_CONFIG_DATA) { + return pnv_phb3_config_read(phb, off & 0x3, size); + } + + /* Other registers are 64-bit only */ + if (size != 8 || off & 0x7) { + phb3_error("Invalid register access, offset: 0x%"PRIx64" size: %d", + off, size); + return ~0ull; + } + + /* Default read from cache */ + val = phb->regs[off >> 3]; + + switch (off) { + /* Simulate venice DD2.0 */ + case PHB_VERSION: + return 0x000000a300000005ull; + case PHB_PCIE_SYSTEM_CONFIG: + return 0x441100fc30000000; + + /* IODA table accesses */ + case PHB_IODA_DATA0: + return pnv_phb3_ioda_read(phb); + + /* Link training always appears trained */ + case PHB_PCIE_DLP_TRAIN_CTL: + return PHB_PCIE_DLP_INBAND_PRESENCE | PHB_PCIE_DLP_TC_DL_LINKACT; + + /* FFI Lock */ + case PHB_FFI_LOCK: + /* Set lock and return previous value */ + phb->regs[off >> 3] |= PHB_FFI_LOCK_STATE; + return val; + + /* Silent simple reads */ + case PHB_PHB3_CONFIG: + case PHB_M32_BASE_ADDR: + case PHB_M32_BASE_MASK: + case PHB_M32_START_ADDR: + case PHB_CONFIG_ADDRESS: + case PHB_IODA_ADDR: + case PHB_RTC_INVALIDATE: + case PHB_TCE_KILL: + case PHB_TCE_SPEC_CTL: + case PHB_PEST_BAR: + case PHB_PELTV_BAR: + case PHB_RTT_BAR: + case PHB_RBA_BAR: + case PHB_IVT_BAR: + case PHB_M64_UPPER_BITS: + break; + + /* Noise on anything else */ + default: + qemu_log_mask(LOG_UNIMP, "phb3: reg_read 0x%"PRIx64"=%"PRIx64"\n", + off, val); + } + return val; +} + +static const MemoryRegionOps pnv_phb3_reg_ops = { + .read = pnv_phb3_reg_read, + .write = pnv_phb3_reg_write, + .valid.min_access_size = 1, + .valid.max_access_size = 8, + .impl.min_access_size = 1, + .impl.max_access_size = 8, + .endianness = DEVICE_BIG_ENDIAN, +}; + +static int pnv_phb3_map_irq(PCIDevice *pci_dev, int irq_num) +{ + /* Check that out properly ... */ + return irq_num & 3; +} + +static void pnv_phb3_set_irq(void *opaque, int irq_num, int level) +{ + PnvPHB3 *phb = opaque; + + /* LSI only ... */ + if (irq_num > 3) { + phb3_error("Unknown IRQ to set %d", irq_num); + } + qemu_set_irq(phb->lsis.qirqs[irq_num], level); +} + +static bool pnv_phb3_resolve_pe(PnvPhb3DMASpace *ds) +{ + uint64_t rtt, addr; + uint16_t rte; + int bus_num; + + /* Already resolved ? */ + if (ds->pe_num != PHB_INVALID_PE) { + return true; + } + + /* We need to lookup the RTT */ + rtt = ds->phb->regs[PHB_RTT_BAR >> 3]; + if (!(rtt & PHB_RBA_BAR_ENABLE)) { + phb3_error("DMA with RTT BAR disabled !"); + /* Set error bits ? fence ? ... */ + return false; + } + + /* Read RTE */ + bus_num = pci_bus_num(ds->bus); + addr = rtt & PHB_RTT_BASE_ADDRESS_MASK; + addr += 2 * ((bus_num << 8) | ds->devfn); + if (dma_memory_read(&address_space_memory, addr, &rte, sizeof(rte))) { + phb3_error("Failed to read RTT entry at 0x%"PRIx64, addr); + /* Set error bits ? fence ? ... */ + return false; + } + rte = be16_to_cpu(rte); + + /* Fail upon reading of invalid PE# */ + if (rte >= PNV_PHB3_NUM_PE) { + phb3_error("RTE for RID 0x%x invalid (%04x", ds->devfn, rte); + /* Set error bits ? fence ? ... */ + return false; + } + ds->pe_num = rte; + return true; +} + +static void pnv_phb3_translate_tve(PnvPhb3DMASpace *ds, hwaddr addr, + bool is_write, uint64_t tve, + IOMMUTLBEntry *tlb) +{ + uint64_t tta = GETFIELD(IODA2_TVT_TABLE_ADDR, tve); + int32_t lev = GETFIELD(IODA2_TVT_NUM_LEVELS, tve); + uint32_t tts = GETFIELD(IODA2_TVT_TCE_TABLE_SIZE, tve); + uint32_t tps = GETFIELD(IODA2_TVT_IO_PSIZE, tve); + + /* Invalid levels */ + if (lev > 4) { + phb3_error("Invalid #levels in TVE %d", lev); + return; + } + + /* IO Page Size of 0 means untranslated, else use TCEs */ + if (tps == 0) { + /* We only support non-translate in top window + * XXX FIX THAT, Venice/Murano support it on bottom window + * above 4G and Naples suports it on everything + */ + if (!(tve & PPC_BIT(51))) { + phb3_error("xlate for invalid non-translate TVE"); + return; + } + /* XXX Handle boundaries */ + + /* XXX Use 4k pages like q35 ... for now */ + tlb->iova = addr & 0xfffffffffffff000ull; + tlb->translated_addr = addr & 0x0003fffffffff000ull; + tlb->addr_mask = 0xfffull; + tlb->perm = IOMMU_RW; + } else { + uint32_t tce_shift, tbl_shift, sh; + uint64_t base, taddr, tce, tce_mask; + + /* TVE disabled ? */ + if (tts == 0) { + phb3_error("xlate for invalid translated TVE"); + return; + } + + /* Address bits per bottom level TCE entry */ + tce_shift = tps + 11; + + /* Address bits per table level */ + tbl_shift = tts + 8; + + /* Top level table base address */ + base = tta << 12; + + /* Total shift to first level */ + sh = tbl_shift * lev + tce_shift; + + /* XXX Multi-level untested */ + while ((lev--) >= 0) { + /* Grab the TCE address */ + taddr = base | (((addr >> sh) & ((1ul << tbl_shift) - 1)) << 3); + if (dma_memory_read(&address_space_memory, taddr, &tce, + sizeof(tce))) { + phb3_error("Failed to read TCE at 0x%"PRIx64, taddr); + return; + } + tce = be64_to_cpu(tce); + + /* Check permission for indirect TCE */ + if ((lev >= 0) && !(tce & 3)) { + phb3_error("Invalid indirect TCE at 0x%"PRIx64, taddr); + phb3_error(" xlate %"PRIx64":%c TVE=%"PRIx64, addr, + is_write ? 'W' : 'R', tve); + phb3_error(" tta=%"PRIx64" lev=%d tts=%d tps=%d", + tta, lev, tts, tps); + return; + } + sh -= tbl_shift; + base = tce & ~0xfffull; + } + + /* We exit the loop with TCE being the final TCE */ + tce_mask = ~((1ull << tce_shift) - 1); + tlb->iova = addr & tce_mask; + tlb->translated_addr = tce & tce_mask; + tlb->addr_mask = ~tce_mask; + tlb->perm = tce & 3; + if ((is_write & !(tce & 2)) || ((!is_write) && !(tce & 1))) { + phb3_error("TCE access fault at 0x%"PRIx64, taddr); + phb3_error(" xlate %"PRIx64":%c TVE=%"PRIx64, addr, + is_write ? 'W' : 'R', tve); + phb3_error(" tta=%"PRIx64" lev=%d tts=%d tps=%d", + tta, lev, tts, tps); + } + } +} + +static IOMMUTLBEntry pnv_phb3_translate_iommu(IOMMUMemoryRegion *iommu, + hwaddr addr, + IOMMUAccessFlags flag, + int iommu_idx) +{ + PnvPhb3DMASpace *ds = container_of(iommu, PnvPhb3DMASpace, dma_mr); + int tve_sel; + uint64_t tve, cfg; + IOMMUTLBEntry ret = { + .target_as = &address_space_memory, + .iova = addr, + .translated_addr = 0, + .addr_mask = ~(hwaddr)0, + .perm = IOMMU_NONE, + }; + + /* Resolve PE# */ + if (!pnv_phb3_resolve_pe(ds)) { + phb3_error("Failed to resolve PE# for bus @%p (%d) devfn 0x%x", + ds->bus, pci_bus_num(ds->bus), ds->devfn); + return ret; + } + + /* Check top bits */ + switch (addr >> 60) { + case 00: + /* DMA or 32-bit MSI ? */ + cfg = ds->phb->regs[PHB_PHB3_CONFIG >> 3]; + if ((cfg & PHB_PHB3C_32BIT_MSI_EN) && + ((addr & 0xffffffffffff0000ull) == 0xffff0000ull)) { + phb3_error("xlate on 32-bit MSI region"); + return ret; + } + /* Choose TVE XXX Use PHB3 Control Register */ + tve_sel = (addr >> 59) & 1; + tve = ds->phb->ioda_TVT[ds->pe_num * 2 + tve_sel]; + pnv_phb3_translate_tve(ds, addr, flag & IOMMU_WO, tve, &ret); + break; + case 01: + phb3_error("xlate on 64-bit MSI region"); + break; + default: + phb3_error("xlate on unsupported address 0x%"PRIx64, addr); + } + return ret; +} + +#define TYPE_PNV_PHB3_IOMMU_MEMORY_REGION "pnv-phb3-iommu-memory-region" +#define PNV_PHB3_IOMMU_MEMORY_REGION(obj) \ + OBJECT_CHECK(IOMMUMemoryRegion, (obj), TYPE_PNV_PHB3_IOMMU_MEMORY_REGION) + +static void pnv_phb3_iommu_memory_region_class_init(ObjectClass *klass, + void *data) +{ + IOMMUMemoryRegionClass *imrc = IOMMU_MEMORY_REGION_CLASS(klass); + + imrc->translate = pnv_phb3_translate_iommu; +} + +static const TypeInfo pnv_phb3_iommu_memory_region_info = { + .parent = TYPE_IOMMU_MEMORY_REGION, + .name = TYPE_PNV_PHB3_IOMMU_MEMORY_REGION, + .class_init = pnv_phb3_iommu_memory_region_class_init, +}; + +/* + * MSI/MSIX memory region implementation. + * The handler handles both MSI and MSIX. + */ +static void pnv_phb3_msi_write(void *opaque, hwaddr addr, + uint64_t data, unsigned size) +{ + PnvPhb3DMASpace *ds = opaque; + + /* Resolve PE# */ + if (!pnv_phb3_resolve_pe(ds)) { + phb3_error("Failed to resolve PE# for bus @%p (%d) devfn 0x%x", + ds->bus, pci_bus_num(ds->bus), ds->devfn); + return; + } + + pnv_phb3_msi_send(&ds->phb->msis, addr, data, ds->pe_num); +} + +/* There is no .read as the read result is undefined by PCI spec */ +static uint64_t pnv_phb3_msi_read(void *opaque, hwaddr addr, unsigned size) +{ + phb3_error("invalid read @ 0x%" HWADDR_PRIx, addr); + return -1; +} + +static const MemoryRegionOps pnv_phb3_msi_ops = { + .read = pnv_phb3_msi_read, + .write = pnv_phb3_msi_write, + .endianness = DEVICE_LITTLE_ENDIAN +}; + +static AddressSpace *pnv_phb3_dma_iommu(PCIBus *bus, void *opaque, int devfn) +{ + PnvPHB3 *phb = opaque; + PnvPhb3DMASpace *ds; + + QLIST_FOREACH(ds, &phb->dma_spaces, list) { + if (ds->bus == bus && ds->devfn == devfn) { + break; + } + } + + if (ds == NULL) { + ds = g_malloc0(sizeof(PnvPhb3DMASpace)); + ds->bus = bus; + ds->devfn = devfn; + ds->pe_num = PHB_INVALID_PE; + ds->phb = phb; + memory_region_init_iommu(&ds->dma_mr, sizeof(ds->dma_mr), + TYPE_PNV_PHB3_IOMMU_MEMORY_REGION, + OBJECT(phb), "phb3_iommu", UINT64_MAX); + address_space_init(&ds->dma_as, MEMORY_REGION(&ds->dma_mr), + "phb3_iommu"); + memory_region_init_io(&ds->msi32_mr, OBJECT(phb), &pnv_phb3_msi_ops, + ds, "msi32", 0x10000); + memory_region_init_io(&ds->msi64_mr, OBJECT(phb), &pnv_phb3_msi_ops, + ds, "msi64", 0x100000); + pnv_phb3_update_msi_regions(ds); + + QLIST_INSERT_HEAD(&phb->dma_spaces, ds, list); + } + return &ds->dma_as; +} + +static void pnv_phb3_instance_init(Object *obj) +{ + PnvPHB3 *phb = PNV_PHB3(obj); + + QLIST_INIT(&phb->dma_spaces); + + /* LSI sources */ + object_initialize(&phb->lsis, sizeof(phb->lsis), TYPE_ICS_SIMPLE); + object_property_add_child(obj, "lsi", OBJECT(&phb->lsis), NULL); + + /* Default init ... will be fixed by HW inits */ + phb->lsis.offset = 0; + + /* MSI sources */ + object_initialize(&phb->msis, sizeof(phb->msis), TYPE_PHB3_MSI); + object_property_add_const_link(OBJECT(&phb->msis), "phb", obj, + &error_abort); + object_property_add_child(obj, "msi", OBJECT(&phb->msis), NULL); + + /* Power Bus Common Queue */ + object_initialize(&phb->pbcq, sizeof(phb->pbcq), TYPE_PNV_PBCQ); + object_property_add_const_link(OBJECT(&phb->pbcq), "phb", obj, + &error_abort); + object_property_add_child(obj, "pbcq", OBJECT(&phb->pbcq), NULL); + + /* Root Port */ + object_initialize(&phb->root, sizeof(phb->root), TYPE_PNV_PHB3_ROOT_PORT); + object_property_add_child(obj, "root", OBJECT(&phb->root), NULL); + qdev_prop_set_int32(DEVICE(&phb->root), "addr", PCI_DEVFN(0, 0)); + qdev_prop_set_bit(DEVICE(&phb->root), "multifunction", false); +} + +static void pnv_phb3_realize(DeviceState *dev, Error **errp) +{ + PnvPHB3 *phb = PNV_PHB3(dev); + PCIHostState *pci = PCI_HOST_BRIDGE(dev); + PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine()); + Error *local_err = NULL; + int i; + + if (!pnv_get_chip(pnv, phb->chip_id)) { + error_setg(errp, "invalid chip id: %d", phb->chip_id); + return; + } + + if (phb->phb_id >= PNV8_CHIP_PHB3_MAX) { + error_setg(errp, "invalid PHB index: %d", phb->phb_id); + return; + } + + /* LSI sources */ + object_property_add_const_link(OBJECT(&phb->lsis), "xics", OBJECT(pnv), + &error_abort); + object_property_set_int(OBJECT(&phb->lsis), PNV_PHB3_NUM_LSI, "nr-irqs", + &error_abort); + object_property_set_bool(OBJECT(&phb->lsis), true, "realized", &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + + for (i = 0; i < phb->lsis.nr_irqs; i++) { + ics_set_irq_type(&phb->lsis, i, true); + } + + /* MSI sources */ + object_property_add_const_link(OBJECT(&phb->msis), "xics", OBJECT(pnv), + &error_abort); + object_property_set_int(OBJECT(&phb->msis), PHB3_MAX_MSI, "nr-irqs", + &error_abort); + object_property_set_bool(OBJECT(&phb->msis), true, "realized", &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + + /* Power Bus Common Queue */ + object_property_set_bool(OBJECT(&phb->pbcq), true, "realized", &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + + /* Controller Registers */ + memory_region_init_io(&phb->mr_regs, OBJECT(phb), &pnv_phb3_reg_ops, phb, + "phb3-regs", 0x1000); + + /* PHB3 doesn't support IO space. However, qemu gets very upset if + * we don't have an IO region to anchor IO BARs onto so we just + * initialize one which we never hook up to anything + */ + memory_region_init(&phb->pci_io, OBJECT(phb), "pci-io", 0x10000); + memory_region_init(&phb->pci_mmio, OBJECT(phb), "pci-mmio", + PCI_MMIO_TOTAL_SIZE); + + pci->bus = pci_register_root_bus(dev, "root-bus", + pnv_phb3_set_irq, pnv_phb3_map_irq, phb, + &phb->pci_mmio, &phb->pci_io, + 0, 4, TYPE_PNV_PHB3_ROOT_BUS); + + pci_setup_iommu(pci->bus, pnv_phb3_dma_iommu, phb); + + /* Add a single Root port */ + qdev_prop_set_uint8(DEVICE(&phb->root), "chassis", phb->chip_id); + qdev_prop_set_uint16(DEVICE(&phb->root), "slot", phb->phb_id); + qdev_set_parent_bus(DEVICE(&phb->root), BUS(pci->bus)); + qdev_init_nofail(DEVICE(&phb->root)); +} + +void pnv_phb3_update_regions(PnvPHB3 *phb) +{ + PnvPBCQState *pbcq = &phb->pbcq; + + /* Unmap first always */ + if (phb->regs_mapped) { + memory_region_del_subregion(&pbcq->phbbar, &phb->mr_regs); + phb->regs_mapped = false; + } + + /* Map registers if enabled */ + if (pbcq->phb_mapped) { + /* XXX We should use the PHB BAR 2 register but we don't ... */ + memory_region_add_subregion(&pbcq->phbbar, 0, &phb->mr_regs); + phb->regs_mapped = true; + } + + /* Check/update m32 */ + if (phb->m32_mapped) { + pnv_phb3_check_m32(phb); + } +} + +static const char *pnv_phb3_root_bus_path(PCIHostState *host_bridge, + PCIBus *rootbus) +{ + PnvPHB3 *phb = PNV_PHB3(host_bridge); + + snprintf(phb->bus_path, sizeof(phb->bus_path), "00%02x:%02x", + phb->chip_id, phb->phb_id); + return phb->bus_path; +} + +static Property pnv_phb3_properties[] = { + DEFINE_PROP_UINT32("index", PnvPHB3, phb_id, 0), + DEFINE_PROP_UINT32("chip-id", PnvPHB3, chip_id, 0), + DEFINE_PROP_END_OF_LIST(), +}; + +static void pnv_phb3_class_init(ObjectClass *klass, void *data) +{ + PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass); + DeviceClass *dc = DEVICE_CLASS(klass); + + hc->root_bus_path = pnv_phb3_root_bus_path; + dc->realize = pnv_phb3_realize; + dc->props = pnv_phb3_properties; + set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories); +} + +static const TypeInfo pnv_phb3_type_info = { + .name = TYPE_PNV_PHB3, + .parent = TYPE_PCIE_HOST_BRIDGE, + .instance_size = sizeof(PnvPHB3), + .class_init = pnv_phb3_class_init, + .instance_init = pnv_phb3_instance_init, +}; + +static void pnv_phb3_root_bus_class_init(ObjectClass *klass, void *data) +{ + BusClass *k = BUS_CLASS(klass); + + /* + * PHB3 has only a single root complex. Enforce the limit on the + * parent bus + */ + k->max_dev = 1; +} + +static const TypeInfo pnv_phb3_root_bus_info = { + .name = TYPE_PNV_PHB3_ROOT_BUS, + .parent = TYPE_PCI_BUS, + .class_init = pnv_phb3_root_bus_class_init, + .interfaces = (InterfaceInfo[]) { + { INTERFACE_PCIE_DEVICE }, + { } + }, +}; + +static void pnv_phb3_root_port_realize(DeviceState *dev, Error **errp) +{ + PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(dev); + Error *local_err = NULL; + + rpc->parent_realize(dev, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } +} + +static void pnv_phb3_root_port_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + PCIDeviceClass *k = PCI_DEVICE_CLASS(klass); + PCIERootPortClass *rpc = PCIE_ROOT_PORT_CLASS(klass); + + dc->desc = "IBM PHB3 PCIE Root Port"; + + device_class_set_parent_realize(dc, pnv_phb3_root_port_realize, + &rpc->parent_realize); + + k->vendor_id = PCI_VENDOR_ID_IBM; + k->device_id = 0x03dc; + k->revision = 0; + + rpc->exp_offset = 0x48; + rpc->aer_offset = 0x100; +} + +static const TypeInfo pnv_phb3_root_port_info = { + .name = TYPE_PNV_PHB3_ROOT_PORT, + .parent = TYPE_PCIE_ROOT_PORT, + .instance_size = sizeof(PnvPHB3RootPort), + .class_init = pnv_phb3_root_port_class_init, +}; + +static void pnv_phb3_register_types(void) +{ + type_register_static(&pnv_phb3_root_bus_info); + type_register_static(&pnv_phb3_root_port_info); + type_register_static(&pnv_phb3_type_info); + type_register_static(&pnv_phb3_iommu_memory_region_info); +} + +type_init(pnv_phb3_register_types) diff --git a/hw/pci-host/pnv_phb3_msi.c b/hw/pci-host/pnv_phb3_msi.c new file mode 100644 index 000000000000..588865a86de1 --- /dev/null +++ b/hw/pci-host/pnv_phb3_msi.c @@ -0,0 +1,318 @@ +/* + * QEMU PowerPC PowerNV PHB3 model + * + * Copyright (c) 2014-2018, IBM Corporation. + * + * This code is licensed under the GPL version 2 or later. See the + * COPYING file in the top-level directory. + */ +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "qapi/error.h" +#include "qemu-common.h" +#include "target/ppc/cpu.h" +#include "hw/pci-host/pnv_phb3_regs.h" +#include "hw/pci-host/pnv_phb3.h" +#include "hw/ppc/pnv.h" +#include "hw/pci/msi.h" +#include "monitor/monitor.h" + +static uint64_t phb3_msi_ive_addr(PnvPHB3 *phb, int srcno) +{ + uint64_t ivtbar = phb->regs[PHB_IVT_BAR >> 3]; + uint64_t phbctl = phb->regs[PHB_CONTROL >> 3]; + + if (!(ivtbar & PHB_IVT_BAR_ENABLE)) { + qemu_log_mask(LOG_GUEST_ERROR, "Failed access to disable IVT BAR !"); + return 0; + } + + if (srcno >= (ivtbar & PHB_IVT_LENGTH_MASK)) { + qemu_log_mask(LOG_GUEST_ERROR, "MSI out of bounds (%d vs 0x%"PRIx64")", + srcno, ivtbar & PHB_IVT_LENGTH_MASK); + return 0; + } + + ivtbar &= PHB_IVT_BASE_ADDRESS_MASK; + + if (phbctl & PHB_CTRL_IVE_128_BYTES) { + return ivtbar + 128 * srcno; + } else { + return ivtbar + 16 * srcno; + } +} + +static bool phb3_msi_read_ive(PnvPHB3 *phb, int srcno, uint64_t *out_ive) +{ + uint64_t ive_addr, ive; + + ive_addr = phb3_msi_ive_addr(phb, srcno); + if (!ive_addr) { + return false; + } + + if (dma_memory_read(&address_space_memory, ive_addr, &ive, sizeof(ive))) { + qemu_log_mask(LOG_GUEST_ERROR, "Failed to read IVE at 0x%" PRIx64, + ive_addr); + return false; + } + *out_ive = be64_to_cpu(ive); + + return true; +} + +static void phb3_msi_set_p(Phb3MsiState *msi, int srcno, uint8_t gen) +{ + uint64_t ive_addr; + uint8_t p = 0x01 | (gen << 1); + + ive_addr = phb3_msi_ive_addr(msi->phb, srcno); + if (!ive_addr) { + return; + } + + if (dma_memory_write(&address_space_memory, ive_addr + 4, &p, 1)) { + qemu_log_mask(LOG_GUEST_ERROR, + "Failed to write IVE (set P) at 0x%" PRIx64, ive_addr); + } +} + +static void phb3_msi_set_q(Phb3MsiState *msi, int srcno) +{ + uint64_t ive_addr; + uint8_t q = 0x01; + + ive_addr = phb3_msi_ive_addr(msi->phb, srcno); + if (!ive_addr) { + return; + } + + if (dma_memory_write(&address_space_memory, ive_addr + 5, &q, 1)) { + qemu_log_mask(LOG_GUEST_ERROR, + "Failed to write IVE (set Q) at 0x%" PRIx64, ive_addr); + } +} + +static void phb3_msi_try_send(Phb3MsiState *msi, int srcno, bool force) +{ + ICSState *ics = ICS_BASE(msi); + uint64_t ive; + uint64_t server, prio, pq, gen; + + if (!phb3_msi_read_ive(msi->phb, srcno, &ive)) { + return; + } + + server = GETFIELD(IODA2_IVT_SERVER, ive); + prio = GETFIELD(IODA2_IVT_PRIORITY, ive); + if (!force) { + pq = GETFIELD(IODA2_IVT_Q, ive) | (GETFIELD(IODA2_IVT_P, ive) << 1); + } else { + pq = 0; + } + gen = GETFIELD(IODA2_IVT_GEN, ive); + + /* + * The low order 2 bits are the link pointer (Type II interrupts). + * Shift back to get a valid IRQ server. + */ + server >>= 2; + + switch (pq) { + case 0: /* 00 */ + if (prio == 0xff) { + /* Masked, set Q */ + phb3_msi_set_q(msi, srcno); + } else { + /* Enabled, set P and send */ + phb3_msi_set_p(msi, srcno, gen); + icp_irq(ics, server, srcno + ics->offset, prio); + } + break; + case 2: /* 10 */ + /* Already pending, set Q */ + phb3_msi_set_q(msi, srcno); + break; + case 1: /* 01 */ + case 3: /* 11 */ + default: + /* Just drop stuff if Q already set */ + break; + } +} + +static void phb3_msi_set_irq(void *opaque, int srcno, int val) +{ + Phb3MsiState *msi = PHB3_MSI(opaque); + + if (val) { + phb3_msi_try_send(msi, srcno, false); + } +} + + +void pnv_phb3_msi_send(Phb3MsiState *msi, uint64_t addr, uint16_t data, + int32_t dev_pe) +{ + ICSState *ics = ICS_BASE(msi); + uint64_t ive; + uint16_t pe; + uint32_t src = ((addr >> 4) & 0xffff) | (data & 0x1f); + + if (src >= ics->nr_irqs) { + qemu_log_mask(LOG_GUEST_ERROR, "MSI %d out of bounds", src); + return; + } + if (dev_pe >= 0) { + if (!phb3_msi_read_ive(msi->phb, src, &ive)) { + return; + } + pe = GETFIELD(IODA2_IVT_PE, ive); + if (pe != dev_pe) { + qemu_log_mask(LOG_GUEST_ERROR, + "MSI %d send by PE#%d but assigned to PE#%d", + src, dev_pe, pe); + return; + } + } + qemu_irq_pulse(ics->qirqs[src]); +} + +void pnv_phb3_msi_ffi(Phb3MsiState *msi, uint64_t val) +{ + /* Emit interrupt */ + pnv_phb3_msi_send(msi, val, 0, -1); + + /* Clear FFI lock */ + msi->phb->regs[PHB_FFI_LOCK >> 3] = 0; +} + +static void phb3_msi_reject(ICSState *ics, uint32_t nr) +{ + Phb3MsiState *msi = PHB3_MSI(ics); + unsigned int srcno = nr - ics->offset; + unsigned int idx = srcno >> 6; + unsigned int bit = 1ull << (srcno & 0x3f); + + assert(srcno < PHB3_MAX_MSI); + + msi->rba[idx] |= bit; + msi->rba_sum |= (1u << idx); +} + +static void phb3_msi_resend(ICSState *ics) +{ + Phb3MsiState *msi = PHB3_MSI(ics); + unsigned int i, j; + + if (msi->rba_sum == 0) { + return; + } + + for (i = 0; i < 32; i++) { + if ((msi->rba_sum & (1u << i)) == 0) { + continue; + } + msi->rba_sum &= ~(1u << i); + for (j = 0; j < 64; j++) { + if ((msi->rba[i] & (1ull << j)) == 0) { + continue; + } + msi->rba[i] &= ~(1u << j); + phb3_msi_try_send(msi, i * 64 + j, true); + } + } +} + +static void phb3_msi_reset(DeviceState *dev) +{ + Phb3MsiState *msi = PHB3_MSI(dev); + ICSStateClass *icsc = ICS_BASE_GET_CLASS(dev); + + icsc->parent_reset(dev); + + memset(msi->rba, 0, sizeof(msi->rba)); + msi->rba_sum = 0; +} + +static void phb3_msi_reset_handler(void *dev) +{ + phb3_msi_reset(dev); +} + +void pnv_phb3_msi_update_config(Phb3MsiState *msi, uint32_t base, + uint32_t count) +{ + ICSState *ics = ICS_BASE(msi); + + if (count > PHB3_MAX_MSI) { + count = PHB3_MAX_MSI; + } + ics->nr_irqs = count; + ics->offset = base; +} + +static void phb3_msi_realize(DeviceState *dev, Error **errp) +{ + Phb3MsiState *msi = PHB3_MSI(dev); + ICSState *ics = ICS_BASE(msi); + ICSStateClass *icsc = ICS_BASE_GET_CLASS(ics); + Object *obj; + Error *local_err = NULL; + + icsc->parent_realize(dev, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + + obj = object_property_get_link(OBJECT(dev), "phb", &local_err); + if (!obj) { + error_propagate(errp, local_err); + error_prepend(errp, "required link 'phb' not found: "); + return; + } + msi->phb = PNV_PHB3(obj); + + ics->qirqs = qemu_allocate_irqs(phb3_msi_set_irq, msi, ics->nr_irqs); + + qemu_register_reset(phb3_msi_reset_handler, dev); +} + +static void phb3_msi_instance_init(Object *obj) +{ + ICSState *ics = ICS_BASE(obj); + + /* Will be overriden later */ + ics->offset = 0; +} + +static void phb3_msi_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + ICSStateClass *isc = ICS_BASE_CLASS(klass); + + device_class_set_parent_realize(dc, phb3_msi_realize, + &isc->parent_realize); + device_class_set_parent_reset(dc, phb3_msi_reset, + &isc->parent_reset); + + isc->reject = phb3_msi_reject; + isc->resend = phb3_msi_resend; +} + +static const TypeInfo phb3_msi_info = { + .name = TYPE_PHB3_MSI, + .parent = TYPE_ICS_BASE, + .instance_size = sizeof(Phb3MsiState), + .class_init = phb3_msi_class_init, + .class_size = sizeof(ICSStateClass), + .instance_init = phb3_msi_instance_init, +}; + +static void pnv_phb3_msi_register_types(void) +{ + type_register_static(&phb3_msi_info); +} + +type_init(pnv_phb3_msi_register_types) diff --git a/hw/pci-host/pnv_phb3_pbcq.c b/hw/pci-host/pnv_phb3_pbcq.c new file mode 100644 index 000000000000..34df3ac684e4 --- /dev/null +++ b/hw/pci-host/pnv_phb3_pbcq.c @@ -0,0 +1,347 @@ +/* + * QEMU PowerPC PowerNV PHB3 model + * + * Copyright (c) 2014-2018, IBM Corporation. + * + * This code is licensed under the GPL version 2 or later. See the + * COPYING file in the top-level directory. + */ +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu-common.h" +#include "qemu/log.h" +#include "target/ppc/cpu.h" +#include "hw/ppc/fdt.h" +#include "hw/pci-host/pnv_phb3_regs.h" +#include "hw/pci-host/pnv_phb3.h" +#include "hw/ppc/pnv_xscom.h" +#include "hw/pci/pci_bridge.h" +#include "hw/pci/pci_bus.h" + +#include <libfdt.h> + +static uint64_t pnv_pbcq_nest_xscom_read(void *opaque, hwaddr addr, + unsigned size) +{ + PnvPBCQState *pbcq = PNV_PBCQ(opaque); + uint32_t offset = addr >> 3; + + return pbcq->nest_regs[offset]; +} + +static uint64_t pnv_pbcq_pci_xscom_read(void *opaque, hwaddr addr, + unsigned size) +{ + PnvPBCQState *pbcq = PNV_PBCQ(opaque); + uint32_t offset = addr >> 3; + + return pbcq->pci_regs[offset]; +} + +static uint64_t pnv_pbcq_spci_xscom_read(void *opaque, hwaddr addr, + unsigned size) +{ + PnvPBCQState *pbcq = PNV_PBCQ(opaque); + uint32_t offset = addr >> 3; + + if (offset == PBCQ_SPCI_ASB_DATA) { + return pnv_phb3_reg_read(pbcq->phb, + pbcq->spci_regs[PBCQ_SPCI_ASB_ADDR], 8); + } + return pbcq->spci_regs[offset]; +} + +static void pnv_pbcq_update_map(PnvPBCQState *pbcq) +{ + uint64_t bar_en = pbcq->nest_regs[PBCQ_NEST_BAR_EN]; + uint64_t bar, mask, size; + + /* + * NOTE: This will really not work well if those are remapped + * after the PHB has created its sub regions. We could do better + * if we had a way to resize regions but we don't really care + * that much in practice as the stuff below really only happens + * once early during boot + */ + + /* Handle unmaps */ + if (pbcq->mmio0_mapped && !(bar_en & PBCQ_NEST_BAR_EN_MMIO0)) { + memory_region_del_subregion(get_system_memory(), &pbcq->mmbar0); + pbcq->mmio0_mapped = false; + } + if (pbcq->mmio1_mapped && !(bar_en & PBCQ_NEST_BAR_EN_MMIO1)) { + memory_region_del_subregion(get_system_memory(), &pbcq->mmbar1); + pbcq->mmio1_mapped = false; + } + if (pbcq->phb_mapped && !(bar_en & PBCQ_NEST_BAR_EN_PHB)) { + memory_region_del_subregion(get_system_memory(), &pbcq->phbbar); + pbcq->phb_mapped = false; + } + + /* Update PHB */ + pnv_phb3_update_regions(pbcq->phb); + + /* Handle maps */ + if (!pbcq->mmio0_mapped && (bar_en & PBCQ_NEST_BAR_EN_MMIO0)) { + bar = pbcq->nest_regs[PBCQ_NEST_MMIO_BAR0] >> 14; + mask = pbcq->nest_regs[PBCQ_NEST_MMIO_MASK0]; + size = ((~mask) >> 14) + 1; + memory_region_init(&pbcq->mmbar0, OBJECT(pbcq), "pbcq-mmio0", size); + memory_region_add_subregion(get_system_memory(), bar, &pbcq->mmbar0); + pbcq->mmio0_mapped = true; + pbcq->mmio0_base = bar; + pbcq->mmio0_size = size; + } + if (!pbcq->mmio1_mapped && (bar_en & PBCQ_NEST_BAR_EN_MMIO1)) { + bar = pbcq->nest_regs[PBCQ_NEST_MMIO_BAR1] >> 14; + mask = pbcq->nest_regs[PBCQ_NEST_MMIO_MASK1]; + size = ((~mask) >> 14) + 1; + memory_region_init(&pbcq->mmbar1, OBJECT(pbcq), "pbcq-mmio1", size); + memory_region_add_subregion(get_system_memory(), bar, &pbcq->mmbar1); + pbcq->mmio1_mapped = true; + pbcq->mmio1_base = bar; + pbcq->mmio1_size = size; + } + if (!pbcq->phb_mapped && (bar_en & PBCQ_NEST_BAR_EN_PHB)) { + bar = pbcq->nest_regs[PBCQ_NEST_PHB_BAR] >> 14; + size = 0x1000; + memory_region_init(&pbcq->phbbar, OBJECT(pbcq), "pbcq-phb", size); + memory_region_add_subregion(get_system_memory(), bar, &pbcq->phbbar); + pbcq->phb_mapped = true; + } + + /* Update PHB */ + pnv_phb3_update_regions(pbcq->phb); +} + +static void pnv_pbcq_nest_xscom_write(void *opaque, hwaddr addr, + uint64_t val, unsigned size) +{ + PnvPBCQState *pbcq = PNV_PBCQ(opaque); + uint32_t reg = addr >> 3; + + switch (reg) { + case PBCQ_NEST_MMIO_BAR0: + case PBCQ_NEST_MMIO_BAR1: + case PBCQ_NEST_MMIO_MASK0: + case PBCQ_NEST_MMIO_MASK1: + if (pbcq->nest_regs[PBCQ_NEST_BAR_EN] & + (PBCQ_NEST_BAR_EN_MMIO0 | + PBCQ_NEST_BAR_EN_MMIO1)) { + qemu_log_mask(LOG_GUEST_ERROR, + "PHB3: Changing enabled BAR unsupported\n"); + } + pbcq->nest_regs[reg] = val & 0xffffffffc0000000ull; + break; + case PBCQ_NEST_PHB_BAR: + if (pbcq->nest_regs[PBCQ_NEST_BAR_EN] & PBCQ_NEST_BAR_EN_PHB) { + qemu_log_mask(LOG_GUEST_ERROR, + "PHB3: Changing enabled BAR unsupported\n"); + } + pbcq->nest_regs[reg] = val & 0xfffffffffc000000ull; + break; + case PBCQ_NEST_BAR_EN: + pbcq->nest_regs[reg] = val & 0xf800000000000000ull; + pnv_pbcq_update_map(pbcq); + pnv_phb3_remap_irqs(pbcq->phb); + break; + case PBCQ_NEST_IRSN_COMPARE: + case PBCQ_NEST_IRSN_MASK: + pbcq->nest_regs[reg] = val & PBCQ_NEST_IRSN_COMP; + pnv_phb3_remap_irqs(pbcq->phb); + break; + case PBCQ_NEST_LSI_SRC_ID: + pbcq->nest_regs[reg] = val & PBCQ_NEST_LSI_SRC; + pnv_phb3_remap_irqs(pbcq->phb); + break; + } + + /* XXX Don't error out on other regs for now ... */ +} + +static void pnv_pbcq_pci_xscom_write(void *opaque, hwaddr addr, + uint64_t val, unsigned size) +{ + PnvPBCQState *pbcq = PNV_PBCQ(opaque); + uint32_t reg = addr >> 3; + + switch (reg) { + case PBCQ_PCI_BAR2: + pbcq->pci_regs[reg] = val & 0xfffffffffc000000ull; + pnv_pbcq_update_map(pbcq); + break; + } + + /* XXX Don't error out on other regs for now ... */ +} + +static void pnv_pbcq_spci_xscom_write(void *opaque, hwaddr addr, + uint64_t val, unsigned size) +{ + PnvPBCQState *pbcq = PNV_PBCQ(opaque); + uint32_t reg = addr >> 3; + + switch (reg) { + case PBCQ_SPCI_ASB_ADDR: + pbcq->spci_regs[reg] = val & 0xfff; + break; + case PBCQ_SPCI_ASB_STATUS: + pbcq->spci_regs[reg] &= ~val; + break; + case PBCQ_SPCI_ASB_DATA: + pnv_phb3_reg_write(pbcq->phb, pbcq->spci_regs[PBCQ_SPCI_ASB_ADDR], + val, 8); + break; + case PBCQ_SPCI_AIB_CAPP_EN: + case PBCQ_SPCI_CAPP_SEC_TMR: + break; + } + + /* XXX Don't error out on other regs for now ... */ +} + +static const MemoryRegionOps pnv_pbcq_nest_xscom_ops = { + .read = pnv_pbcq_nest_xscom_read, + .write = pnv_pbcq_nest_xscom_write, + .valid.min_access_size = 8, + .valid.max_access_size = 8, + .impl.min_access_size = 8, + .impl.max_access_size = 8, + .endianness = DEVICE_BIG_ENDIAN, +}; + +static const MemoryRegionOps pnv_pbcq_pci_xscom_ops = { + .read = pnv_pbcq_pci_xscom_read, + .write = pnv_pbcq_pci_xscom_write, + .valid.min_access_size = 8, + .valid.max_access_size = 8, + .impl.min_access_size = 8, + .impl.max_access_size = 8, + .endianness = DEVICE_BIG_ENDIAN, +}; + +static const MemoryRegionOps pnv_pbcq_spci_xscom_ops = { + .read = pnv_pbcq_spci_xscom_read, + .write = pnv_pbcq_spci_xscom_write, + .valid.min_access_size = 8, + .valid.max_access_size = 8, + .impl.min_access_size = 8, + .impl.max_access_size = 8, + .endianness = DEVICE_BIG_ENDIAN, +}; + +static void pnv_pbcq_default_bars(PnvPBCQState *pbcq) +{ + uint64_t mm0, mm1, reg; + PnvPHB3 *phb = pbcq->phb; + + mm0 = 0x3d00000000000ull + 0x4000000000ull * phb->chip_id + + 0x1000000000ull * phb->phb_id; + mm1 = 0x3ff8000000000ull + 0x0200000000ull * phb->chip_id + + 0x0080000000ull * phb->phb_id; + reg = 0x3fffe40000000ull + 0x0000400000ull * phb->chip_id + + 0x0000100000ull * phb->phb_id; + + pbcq->nest_regs[PBCQ_NEST_MMIO_BAR0] = mm0 << 14; + pbcq->nest_regs[PBCQ_NEST_MMIO_BAR1] = mm1 << 14; + pbcq->nest_regs[PBCQ_NEST_PHB_BAR] = reg << 14; + pbcq->nest_regs[PBCQ_NEST_MMIO_MASK0] = 0x3fff000000000ull << 14; + pbcq->nest_regs[PBCQ_NEST_MMIO_MASK1] = 0x3ffff80000000ull << 14; + pbcq->pci_regs[PBCQ_PCI_BAR2] = reg << 14; +} + +static void pnv_pbcq_realize(DeviceState *dev, Error **errp) +{ + PnvPBCQState *pbcq = PNV_PBCQ(dev); + PnvPHB3 *phb; + Object *obj; + Error *local_err = NULL; + char name[32]; + + obj = object_property_get_link(OBJECT(dev), "phb", &local_err); + if (!obj) { + error_propagate(errp, local_err); + error_prepend(errp, "required link 'phb' not found: "); + return; + } + phb = pbcq->phb = PNV_PHB3(obj); + + /* XXX Fix OPAL to do that: establish default BAR values */ + pnv_pbcq_default_bars(pbcq); + + /* Initialize the XSCOM region for the PBCQ registers */ + snprintf(name, sizeof(name), "xscom-pbcq-nest-%d.%d", + phb->chip_id, phb->phb_id); + pnv_xscom_region_init(&pbcq->xscom_nest_regs, OBJECT(dev), + &pnv_pbcq_nest_xscom_ops, pbcq, name, + PNV_XSCOM_PBCQ_NEST_SIZE); + snprintf(name, sizeof(name), "xscom-pbcq-pci-%d.%d", + phb->chip_id, phb->phb_id); + pnv_xscom_region_init(&pbcq->xscom_pci_regs, OBJECT(dev), + &pnv_pbcq_pci_xscom_ops, pbcq, name, + PNV_XSCOM_PBCQ_PCI_SIZE); + snprintf(name, sizeof(name), "xscom-pbcq-spci-%d.%d", + phb->chip_id, phb->phb_id); + pnv_xscom_region_init(&pbcq->xscom_spci_regs, OBJECT(dev), + &pnv_pbcq_spci_xscom_ops, pbcq, name, + PNV_XSCOM_PBCQ_SPCI_SIZE); +} + +static int pnv_pbcq_dt_xscom(PnvXScomInterface *dev, void *fdt, + int xscom_offset) +{ + const char compat[] = "ibm,power8-pbcq"; + PnvPHB3 *phb = PNV_PBCQ(dev)->phb; + char *name; + int offset; + uint32_t lpc_pcba = PNV_XSCOM_PBCQ_NEST_BASE + 0x400 * phb->phb_id; + uint32_t reg[] = { + cpu_to_be32(lpc_pcba), + cpu_to_be32(PNV_XSCOM_PBCQ_NEST_SIZE), + cpu_to_be32(PNV_XSCOM_PBCQ_PCI_BASE + 0x400 * phb->phb_id), + cpu_to_be32(PNV_XSCOM_PBCQ_PCI_SIZE), + cpu_to_be32(PNV_XSCOM_PBCQ_SPCI_BASE + 0x040 * phb->phb_id), + cpu_to_be32(PNV_XSCOM_PBCQ_SPCI_SIZE) + }; + + name = g_strdup_printf("pbcq@%x", lpc_pcba); + offset = fdt_add_subnode(fdt, xscom_offset, name); + _FDT(offset); + g_free(name); + + _FDT((fdt_setprop(fdt, offset, "reg", reg, sizeof(reg)))); + + _FDT((fdt_setprop_cell(fdt, offset, "ibm,phb-index", phb->phb_id))); + _FDT((fdt_setprop_cell(fdt, offset, "ibm,chip-id", phb->chip_id))); + _FDT((fdt_setprop(fdt, offset, "compatible", compat, + sizeof(compat)))); + return 0; +} + +static void pnv_pbcq_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + PnvXScomInterfaceClass *xdc = PNV_XSCOM_INTERFACE_CLASS(klass); + + xdc->dt_xscom = pnv_pbcq_dt_xscom; + + dc->realize = pnv_pbcq_realize; +} + +static const TypeInfo pnv_pbcq_type_info = { + .name = TYPE_PNV_PBCQ, + .parent = TYPE_DEVICE, + .instance_size = sizeof(PnvPBCQState), + .class_init = pnv_pbcq_class_init, + .interfaces = (InterfaceInfo[]) { + { TYPE_PNV_XSCOM_INTERFACE }, + { } + } +}; + +static void pnv_pbcq_register_types(void) +{ + type_register_static(&pnv_pbcq_type_info); +} + +type_init(pnv_pbcq_register_types) diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c index 346f5e7aedb5..e8a8065fd25f 100644 --- a/hw/ppc/pnv.c +++ b/hw/ppc/pnv.c @@ -44,6 +44,7 @@ #include "hw/isa/isa.h" #include "hw/char/serial.h" #include "hw/timer/mc146818rtc.h" +#include "hw/pci/msi.h" #include <libfdt.h> @@ -555,6 +556,9 @@ static void pnv_init(MachineState *machine) int i; char *chip_typename; + /* MSIs are supported on this platform */ + msi_nonbroken = true; + /* allocate RAM */ if (machine->ram_size < (1 * GiB)) { warn_report("skiboot may not work with < 1GB of RAM"); @@ -725,6 +729,8 @@ static Object *pnv_chip_power9_intc_create(PnvChip *chip, Object *child, static void pnv_chip_power8_instance_init(Object *obj) { Pnv8Chip *chip8 = PNV8_CHIP(obj); + PnvChipClass *pcc = PNV_CHIP_GET_CLASS(obj); + int i; object_initialize(&chip8->psi, sizeof(chip8->psi), TYPE_PNV_PSI); object_property_add_child(obj, "psi", OBJECT(&chip8->psi), NULL); @@ -740,6 +746,12 @@ static void pnv_chip_power8_instance_init(Object *obj) object_property_add_child(obj, "occ", OBJECT(&chip8->occ), NULL); object_property_add_const_link(OBJECT(&chip8->occ), "psi", OBJECT(&chip8->psi), &error_abort); + + for (i = 0; i < pcc->num_phbs; i++) { + object_initialize(&chip8->phbs[i], sizeof(chip8->phbs[i]), + TYPE_PNV_PHB3); + object_property_add_child(obj, "phb[*]", OBJECT(&chip8->phbs[i]), NULL); + } } static void pnv_chip_icp_realize(Pnv8Chip *chip8, Error **errp) @@ -780,6 +792,7 @@ static void pnv_chip_power8_realize(DeviceState *dev, Error **errp) PnvChip *chip = PNV_CHIP(dev); Pnv8Chip *chip8 = PNV8_CHIP(dev); Error *local_err = NULL; + int i; pcc->parent_realize(dev, &local_err); if (local_err) { @@ -817,6 +830,33 @@ static void pnv_chip_power8_realize(DeviceState *dev, Error **errp) return; } pnv_xscom_add_subregion(chip, PNV_XSCOM_OCC_BASE, &chip8->occ.xscom_regs); + + /* Create the PHB3 controllers */ + for (i = 0; i < pcc->num_phbs; i++) { + PnvPHB3 *phb = &chip8->phbs[i]; + PnvPBCQState *pbcq = &phb->pbcq; + + object_property_set_int(OBJECT(phb), i, "index", &error_fatal); + object_property_set_int(OBJECT(phb), chip->chip_id, "chip-id", + &error_fatal); + object_property_set_bool(OBJECT(phb), true, "realized", &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + qdev_set_parent_bus(DEVICE(phb), sysbus_get_default()); + + /* Populate the XSCOM address space. */ + pnv_xscom_add_subregion(chip, + PNV_XSCOM_PBCQ_NEST_BASE + 0x400 * phb->phb_id, + &pbcq->xscom_nest_regs); + pnv_xscom_add_subregion(chip, + PNV_XSCOM_PBCQ_PCI_BASE + 0x400 * phb->phb_id, + &pbcq->xscom_pci_regs); + pnv_xscom_add_subregion(chip, + PNV_XSCOM_PBCQ_SPCI_BASE + 0x040 * phb->phb_id, + &pbcq->xscom_spci_regs); + } } static void pnv_chip_power8e_class_init(ObjectClass *klass, void *data) @@ -827,6 +867,7 @@ static void pnv_chip_power8e_class_init(ObjectClass *klass, void *data) k->chip_type = PNV_CHIP_POWER8E; k->chip_cfam_id = 0x221ef04980000000ull; /* P8 Murano DD2.1 */ k->cores_mask = POWER8E_CORE_MASK; + k->num_phbs = 3; k->core_pir = pnv_chip_core_pir_p8; k->intc_create = pnv_chip_power8_intc_create; k->isa_create = pnv_chip_power8_isa_create; @@ -845,6 +886,7 @@ static void pnv_chip_power8_class_init(ObjectClass *klass, void *data) k->chip_type = PNV_CHIP_POWER8; k->chip_cfam_id = 0x220ea04980000000ull; /* P8 Venice DD2.0 */ k->cores_mask = POWER8_CORE_MASK; + k->num_phbs = 3; k->core_pir = pnv_chip_core_pir_p8; k->intc_create = pnv_chip_power8_intc_create; k->isa_create = pnv_chip_power8_isa_create; @@ -863,6 +905,7 @@ static void pnv_chip_power8nvl_class_init(ObjectClass *klass, void *data) k->chip_type = PNV_CHIP_POWER8NVL; k->chip_cfam_id = 0x120d304980000000ull; /* P8 Naples DD1.0 */ k->cores_mask = POWER8_CORE_MASK; + k->num_phbs = 3; k->core_pir = pnv_chip_core_pir_p8; k->intc_create = pnv_chip_power8_intc_create; k->isa_create = pnv_chip_power8nvl_isa_create; @@ -897,6 +940,7 @@ static void pnv_chip_power9_class_init(ObjectClass *klass, void *data) k->chip_type = PNV_CHIP_POWER9; k->chip_cfam_id = 0x220d104900008000ull; /* P9 Nimbus DD2.0 */ k->cores_mask = POWER9_CORE_MASK; + k->num_phbs = 0; k->core_pir = pnv_chip_core_pir_p9; k->intc_create = pnv_chip_power9_intc_create; k->isa_create = pnv_chip_power9_isa_create; @@ -1047,14 +1091,24 @@ static void pnv_chip_class_init(ObjectClass *klass, void *data) static ICSState *pnv_ics_get(XICSFabric *xi, int irq) { PnvMachineState *pnv = PNV_MACHINE(xi); - int i; + int i, j; for (i = 0; i < pnv->num_chips; i++) { + PnvChip *chip = pnv->chips[i]; Pnv8Chip *chip8 = PNV8_CHIP(pnv->chips[i]); + PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip); if (ics_valid_irq(&chip8->psi.ics, irq)) { return &chip8->psi.ics; } + for (j = 0; j < pcc->num_phbs; j++) { + if (ics_valid_irq(&chip8->phbs[j].lsis, irq)) { + return &chip8->phbs[j].lsis; + } + if (ics_valid_irq(ICS_BASE(&chip8->phbs[j].msis), irq)) { + return ICS_BASE(&chip8->phbs[j].msis); + } + } } return NULL; } @@ -1062,11 +1116,18 @@ static ICSState *pnv_ics_get(XICSFabric *xi, int irq) static void pnv_ics_resend(XICSFabric *xi) { PnvMachineState *pnv = PNV_MACHINE(xi); - int i; + int i, j; for (i = 0; i < pnv->num_chips; i++) { + PnvChip *chip = pnv->chips[i]; Pnv8Chip *chip8 = PNV8_CHIP(pnv->chips[i]); + PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip); + ics_resend(&chip8->psi.ics); + for (j = 0; j < pcc->num_phbs; j++) { + ics_resend(&chip8->phbs[j].lsis); + ics_resend(ICS_BASE(&chip8->phbs[j].msis)); + } } } @@ -1097,7 +1158,7 @@ static void pnv_pic_print_info(InterruptStatsProvider *obj, Monitor *mon) { PnvMachineState *pnv = PNV_MACHINE(obj); - int i; + int i, j; CPUState *cs; CPU_FOREACH(cs) { @@ -1107,8 +1168,16 @@ static void pnv_pic_print_info(InterruptStatsProvider *obj, } for (i = 0; i < pnv->num_chips; i++) { + PnvChip *chip = pnv->chips[i]; Pnv8Chip *chip8 = PNV8_CHIP(pnv->chips[i]); + PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip); + ics_pic_print_info(&chip8->psi.ics, mon); + + for (j = 0; j < pcc->num_phbs; j++) { + ics_pic_print_info(ICS_BASE(&chip8->phbs[j].msis), mon); + ics_pic_print_info(&chip8->phbs[j].lsis, mon); + } } } diff --git a/hw/ppc/pnv_xscom.c b/hw/ppc/pnv_xscom.c index 46fae41f32b0..83f05d06053f 100644 --- a/hw/ppc/pnv_xscom.c +++ b/hw/ppc/pnv_xscom.c @@ -252,7 +252,11 @@ int pnv_dt_xscom(PnvChip *chip, void *fdt, int root_offset) args.fdt = fdt; args.xscom_offset = xscom_offset; - object_child_foreach(OBJECT(chip), xscom_dt_child, &args); + /* Some PnvXScomInterface objects lie a bit deeper (PnvPBCQState) + * than the first layer, so we need to loop on the whole object + * hierarchy to catch them + */ + object_child_foreach_recursive(OBJECT(chip), xscom_dt_child, &args); return 0; } diff --git a/hw/pci-host/Makefile.objs b/hw/pci-host/Makefile.objs index 6d6597c06563..1b5b782bd986 100644 --- a/hw/pci-host/Makefile.objs +++ b/hw/pci-host/Makefile.objs @@ -19,3 +19,4 @@ common-obj-$(CONFIG_PCI_GENERIC) += gpex.o common-obj-$(CONFIG_PCI_XILINX) += xilinx-pcie.o common-obj-$(CONFIG_PCI_DESIGNWARE) += designware.o +obj-$(CONFIG_POWERNV) += pnv_phb3.o pnv_phb3_pbcq.o pnv_phb3_msi.o