From patchwork Fri Mar 31 16:52:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 745716 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3vvndR3WvKz9ryj for ; Sat, 1 Apr 2017 03:53:47 +1100 (AEDT) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3vvndR1vC3zDqJn for ; Sat, 1 Apr 2017 03:53:47 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3vvncw6xqWzDqH7 for ; Sat, 1 Apr 2017 03:53:20 +1100 (AEDT) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v2VGiO72079156 for ; Fri, 31 Mar 2017 12:53:17 -0400 Received: from e06smtp11.uk.ibm.com (e06smtp11.uk.ibm.com [195.75.94.107]) by mx0a-001b2d01.pphosted.com with ESMTP id 29htsq8hw4-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Fri, 31 Mar 2017 12:53:17 -0400 Received: from localhost by e06smtp11.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 31 Mar 2017 17:53:15 +0100 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp11.uk.ibm.com (192.168.101.141) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 31 Mar 2017 17:53:14 +0100 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v2VGqlWd40370324; Fri, 31 Mar 2017 16:52:47 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8DFD9A404D; Fri, 31 Mar 2017 17:52:13 +0100 (BST) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CDE4AA4057; Fri, 31 Mar 2017 17:52:12 +0100 (BST) Received: from lombard-w520.ibm.com (unknown [9.167.246.39]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Fri, 31 Mar 2017 17:52:12 +0100 (BST) From: Christophe Lombard To: skiboot@lists.ozlabs.org, fbarrat@linux.vnet.ibm.com, imunsie@au1.ibm.com, andrew.donnellan@au1.ibm.com, christophe_lombard@fr.ibm.com Date: Fri, 31 Mar 2017 18:52:38 +0200 X-Mailer: git-send-email 2.7.4 In-Reply-To: <1490979160-14535-1-git-send-email-clombard@linux.vnet.ibm.com> References: <1490979160-14535-1-git-send-email-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 17033116-0040-0000-0000-00000377C5DB X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17033116-0041-0000-0000-00001F81E165 Message-Id: <1490979160-14535-4-git-send-email-clombard@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-03-31_14:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1703310153 Subject: [Skiboot] [PATCH V2 3/5] capi: Enable capi mode X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Enable the Coherently attached processor interface. The PHB is used as a CAPI interface. CAPI Adapters can be connected to whether PEC0 or PEC2. Single port CAPI adapter can be connected to either PEC0 or PEC2, but Dual-Port Adapter can be only connected to PEC2 CAPP0 attached to PHB0(PEC0 - single port) CAPP1 attached to PHB3(PEC2 - single or dual port) As we did for PHB3, a new specific file 'phb4-capp.h' is created to contain the CAPP register definitions. Signed-off-by: Christophe Lombard --- hw/phb4.c | 453 +++++++++++++++++++++++++++++++++++++++++++++++++++- include/chip.h | 1 + include/phb4-capp.h | 60 +++++++ include/phb4-regs.h | 8 +- include/phb4.h | 1 + 5 files changed, 517 insertions(+), 6 deletions(-) create mode 100644 include/phb4-capp.h diff --git a/hw/phb4.c b/hw/phb4.c index 571f703..e6f29bc 100644 --- a/hw/phb4.c +++ b/hw/phb4.c @@ -48,7 +48,7 @@ #include #include #include -#include +#include #include #include #include @@ -1963,6 +1963,119 @@ static int64_t phb4_freset(struct pci_slot *slot) return OPAL_HARDWARE; } +extern struct lock capi_lock; + +static void do_capp_recovery_scoms(struct phb4 *p) +{ + uint64_t reg; + uint32_t offset; + + PHBDBG(p, "Doing CAPP recovery scoms\n"); + + offset = PHB4_CAPP_REG_OFFSET(p); + /* disable snoops */ + xscom_write(p->chip_id, SNOOP_CAPI_CONFIG + offset, 0); + + /* clear err rpt reg*/ + xscom_write(p->chip_id, CAPP_ERR_RPT_CLR + offset, 0); + /* clear capp fir */ + xscom_write(p->chip_id, CAPP_FIR + offset, 0); + + xscom_read(p->chip_id, CAPP_ERR_STATUS_CTRL + offset, ®); + reg &= ~(PPC_BIT(0) | PPC_BIT(1)); + xscom_write(p->chip_id, CAPP_ERR_STATUS_CTRL + offset, reg); +} + +/* + * Disable CAPI mode on a PHB. + * + * Must be done while PHB is fenced and in recovery. Leaves CAPP in recovery - + * we can't come out of recovery until the PHB has been reinitialised. + * + * We don't reset generic error registers here - we rely on phb4_init_hw() to + * do that. + * + * Sets PHB4_CAPP_DISABLING flag when complete. + */ +static void disable_capi_mode(struct phb4 *p) +{ + struct proc_chip *chip = get_chip(p->chip_id); + uint64_t reg; + uint32_t offset = PHB4_CAPP_REG_OFFSET(p); + + lock(&capi_lock); + + xscom_read(p->chip_id, p->pe_xscom + 0x7, ®); + if (!(reg & PPC_BIT(0))) { + /* Not in CAPI mode, no action required */ + goto out; + } + + PHBDBG(p, "CAPP: Disabling CAPI mode\n"); + if (!(chip->capp_phb4_attached_mask & (1 << p->index))) + PHBERR(p, "CAPP: CAPP attached mask not set!\n"); + + xscom_read(p->chip_id, CAPP_ERR_STATUS_CTRL + offset, ®); + if (!(reg & PPC_BIT(0))) { + PHBERR(p, "CAPP: not in recovery, can't disable CAPI mode!\n"); + goto out; + } + + /* CAPI will not be decoded. */ + out_be64(p->regs + PHB_CAPI_CMPM, 0ull); + + /* Disable NBW decoding */ + out_be64(p->regs + PHB_PBL_NBW_CMPM, 0ull); + + /* Disable NBW decoding */ + out_be64(p->regs + PHB_PBL_NBW_CMPM, 0ull); + + /* APC Master PB Control Register - disable examining cResps */ + xscom_read(p->chip_id, APC_MASTER_PB_CTRL + offset, ®); + reg &= ~PPC_BIT(0); + xscom_write(p->chip_id, APC_MASTER_PB_CTRL + offset, reg); + + /* APC Master Config Register - de-select PHBs */ + xscom_read(p->chip_id, APC_MASTER_CAPI_CTRL + offset, ®); + reg &= ~PPC_BITMASK(2, 3); + xscom_write(p->chip_id, APC_MASTER_CAPI_CTRL + offset, reg); + + /* Snoop CAPI Configuration Register - disable snooping */ + xscom_write(p->chip_id, SNOOP_CAPI_CONFIG + offset, 0ull); + + /* Read Epsilon Tier0 _ default value */ + xscom_write(p->chip_id, SNOOP_CONTROL + offset, 0ull); + + /* CAPP Transport Control Register */ + xscom_write(p->chip_id, TRANSPORT_CONTROL + offset, 0x0001000000000000); + + /* Epoch and Recovery Timers Control Register */ + xscom_write(p->chip_id, EPOCH_RECOVERY_TIMERS_CTRL + offset, 0ull); + + /* Flush SUE State Map Register */ + xscom_write(p->chip_id, FLUSH_SUE_STATE_MAP + offset, 0ull); + + /* Flush SUE uOP1 Register */ + xscom_write(p->chip_id, FLUSH_SUE_UOP1 + offset, 0ull); + + /* APC FSM Read Machine Mask Register */ + xscom_write(p->chip_id, APC_FSM_READ_MASK + offset, 0ull); + + /* Snoop copy of FSM Read Machine Mask Register */ + xscom_write(p->chip_id, XPT_FSM_RMM + offset, 0ull); + + /* PE Secure CAPP Enable Register - we're all done! Disable CAPP mode! */ + xscom_write(p->chip_id, p->pe_xscom + 0x7, 0ull); + + /* Trigger CAPP recovery scoms after reinit */ + p->flags |= PHB4_CAPP_DISABLING; + + chip->capp_phb4_attached_mask &= ~(1 << p->index); + +out: + unlock(&capi_lock); +} + static int64_t phb4_creset(struct pci_slot *slot) { struct phb4 *p = phb_to_phb4(slot->phb); @@ -1972,12 +2085,11 @@ static int64_t phb4_creset(struct pci_slot *slot) case PHB4_SLOT_CRESET_START: PHBDBG(p, "CRESET: Starts\n"); - /* do steps 3-5 of capp recovery procedure */ -#if 0 if (p->flags & PHB4_CAPP_RECOVERY) do_capp_recovery_scoms(p); -#endif - /* XXX TODO XXX */ + + if (!(p->flags & PHB4_CAPP_RECOVERY)) + disable_capi_mode(p); pci_slot_set_state(slot, PHB4_SLOT_CRESET_WAIT_CQ); slot->retries = 500; @@ -1990,6 +2102,12 @@ static int64_t phb4_creset(struct pci_slot *slot) p->flags &= ~PHB4_AIB_FENCED; p->flags &= ~PHB4_CAPP_RECOVERY; phb4_init_hw(p, false); + + if (p->flags & PHB4_CAPP_DISABLING) { + do_capp_recovery_scoms(p); + p->flags &= ~PHB4_CAPP_DISABLING; + } + pci_slot_set_state(slot, PHB4_SLOT_CRESET_FRESET); return pci_slot_set_sm_timeout(slot, msecs_to_tb(100)); case PHB4_SLOT_CRESET_FRESET: @@ -2361,6 +2479,288 @@ static int64_t phb4_get_diag_data(struct phb *phb, return OPAL_SUCCESS; } +static void phb4_init_capp_regs(struct phb4 *p) +{ + uint64_t reg; + uint32_t offset; + + offset = PHB4_CAPP_REG_OFFSET(p); + + /* Enable cresp examination by CAPP */ + xscom_read(p->chip_id, APC_MASTER_PB_CTRL + offset, ®); + reg |= PPC_BIT(0); + if (p->rev == PHB4_REV_NIMBUS_DD10) { + /* disable vg not sys */ + reg |= PPC_BIT(3); + } + xscom_write(p->chip_id, APC_MASTER_PB_CTRL + offset, reg); + + /* Set PHB mode, HPC Dir State and P9 mode */ + xscom_write(p->chip_id, APC_MASTER_CAPI_CTRL + offset, 0x1072000000000000); + PHBINF(p, "CAPP: port attached\n"); + + /* Set snoop ttype decoding , dir size to 256k */ + xscom_write(p->chip_id, SNOOP_CAPI_CONFIG + offset, 0xA000000000000000); + + /* Use Read Epsilon Tier2 for all scopes, Address Pipeline Master + * Wait Count to highest(1023) and Number of rpt_hang.data to 3 + */ + xscom_write(p->chip_id, SNOOP_CONTROL + offset, 0x8000000010072000); + + /* TLBI Hang Divider = 1. LPC buffers=0. X16 PCIe(14 buffers) */ + xscom_write(p->chip_id, TRANSPORT_CONTROL + offset, 0x401404000400000B); + + /* Enable epoch timer */ + xscom_write(p->chip_id, EPOCH_RECOVERY_TIMERS_CTRL + offset, 0xC0000000FFF0FFE0); + + /* Deassert TLBI_FENCED and tlbi_psl_is_dead */ + xscom_write(p->chip_id, CAPP_ERR_STATUS_CTRL + offset, 0); + + xscom_write(p->chip_id, FLUSH_SUE_STATE_MAP + offset, + 0x1DCF5F6600000000); + xscom_write(p->chip_id, FLUSH_SUE_UOP1 + offset, + 0xE3105005C8000000); + xscom_write(p->chip_id, APC_FSM_READ_MASK + offset, + 0xFFFFFFFFFFFF0000); + xscom_write(p->chip_id, XPT_FSM_RMM + offset, + 0xFFFFFFFFFFFF0000); +} + +/* override some inits with CAPI defaults */ +static void phb4_init_capp_errors(struct phb4 *p) +{ + /* Init_77: TXE Error AIB Fence Enable Register */ + out_be64(p->regs + 0x0d30, 0xdff7ff0bf7ddfff0ull); + + /* Init_86: RXE_ARB Error AIB Fence Enable Register */ + out_be64(p->regs + 0x0db0, 0xfbffd7bbff7fbfefull); + + /* Init_95: RXE_MRG Error AIB Fence Enable Register */ + out_be64(p->regs + 0x0e30, 0xfffffeffff7fff57ull); + + /* Init_104: RXE_TCE Error AIB Fence Enable Register */ + out_be64(p->regs + 0x0eb0, 0xffaeffafffffffffull); + + /* Init_113: PHB Error AIB Fence Enable Register */ + out_be64(p->regs + 0x0cb0, 0x35777073ff000000ull); +} + +/* Power Bus Common Queue Registers + * All PBCQ and PBAIB registers are accessed via SCOM + * NestBase = 4010C00 for PEC0 + * 4011000 for PEC1 + * 4011400 for PEC2 + * + * Some registers are shared amongst all of the stacks and will only + * have 1 copy. Other registers are implemented one per stack. + * Registers that are duplicated will have an additional offset + * of “StackBase” so that they have a unique address. + * Stackoffset = 00000040 for Stack0 + * = 00000080 for Stack1 + * = 000000C0 for Stack2 + */ +static int64_t enable_capi_mode(struct phb4 *p, uint64_t pe_number) +{ + uint64_t reg; + uint32_t offset; + int i; + + xscom_read(p->chip_id, p->pe_xscom + 0x7, ®); + if (reg & PPC_BIT(0)) + PHBDBG(p, "Already in CAPP mode\n"); + + /* PEC Phase 3 (PBCQ) registers Init */ + /* poll cqstat + * CAPP0 attached to PHB0(PEC0) + * CAPP1 attached to PHB3(PEC2) + */ + offset = 0x40; + if (p->index > 0 && p->index < 3) + offset = 0x80; + else if (p->index > 2) + offset = 0xC0; + + for (i = 0; i < 500000; i++) { + xscom_read(p->chip_id, p->pe_xscom + offset + 0xC, ®); + if (!(reg & 0xC000000000000000)) + break; + time_wait_us(10); + } + if (reg & 0xC000000000000000) { + PHBERR(p, "CAPP: Timeout waiting for pending transaction\n"); + return OPAL_HARDWARE; + } + + /* Enable CAPP Mode , Set 14 CI Store buffers for CAPP, + * Set 48 Read machines for CAPP. + */ + reg = 0x8000DFFFFFFFFFFFUll; + xscom_write(p->chip_id, p->pe_xscom + 0x7, reg); + + /* PEC Phase 4 (PHB) registers adjustment + * Bit [0:7] XSL_DSNCTL[capiind] + * Init_25 - CAPI Compare/Mask + */ + out_be64(p->regs + PHB_CAPI_CMPM, + 0x0200FE0000000000Ull | PHB_CAPI_CMPM_ENABLE); + + if (!(p->rev == PHB4_REV_NIMBUS_DD10)) { + /* Init_123 : NBW Compare/Mask Register */ + out_be64(p->regs + PHB_PBL_NBW_CMPM, + 0x0300FF0000000000Ull | PHB_CAPI_NBW_ENABLE); + + /* Init_24 - ASN Compare/Mask */ + out_be64(p->regs + PHB_PBL_ASN_CMPM, + 0x0400FF0000000000Ull | PHB_CAPI_ASN_ENABLE); + } + + /* non-translate/50-bit mode */ + out_be64(p->regs + PHB_XLATE_PREFIX, 0x0000000000000000Ull); + + /* set tve no translate mode allow mmio window */ + memset(p->tve_cache, 0x0, sizeof(p->tve_cache)); + + /* + * In 50-bit non-translate mode, the fields of the TVE are + * used to perform an address range check. In this mode TCE + * Table Size(0) must be a '1' (TVE[51] = 1) + * PCI Addr(49:24) >= TVE[52:53]+TVE[0:23] and + * PCI Addr(49:24) < TVE[54:55]+TVE[24:47] + * + * TVE[51] = 1 + * TVE[56] = 1: 50-bit Non-Translate Mode Enable + * TVE[0:23] = 0x000000 + * TVE[24:47] = 0xFFFFFF + * + * capi dma mode: CAPP DMA mode needs access to all of memory + * capi mode: Allow address range (bit 14 = 1) + * 0x0002000000000000: 0x0002FFFFFFFFFFFF + * TVE[52:53] = '10' and TVE[54:55] = '10' + * + * --> we use capi dma mode by default + */ + p->tve_cache[pe_number * 2] = PPC_BIT(51); + p->tve_cache[pe_number * 2] |= IODA3_TVT_NON_TRANSLATE_50; + p->tve_cache[pe_number * 2] |= (0xfffffful << 16); + + phb4_ioda_sel(p, IODA3_TBL_TVT, 0, true); + for (i = 0; i < p->tvt_size; i++) + out_be64(p->regs + PHB_IODA_DATA0, p->tve_cache[i]); + + /* set mbt bar to pass capi mmio window. First applied cleared + * values to HW + */ + for (i = 0; i < p->mbt_size; i++) { + p->mbt_cache[i][0] = 0; + p->mbt_cache[i][1] = 0; + } + phb4_ioda_sel(p, IODA3_TBL_MBT, 0, true); + for (i = 0; i < p->mbt_size; i++) { + out_be64(p->regs + PHB_IODA_DATA0, p->mbt_cache[i][0]); + out_be64(p->regs + PHB_IODA_DATA0, p->mbt_cache[i][1]); + } + + p->mbt_cache[0][0] = IODA3_MBT0_ENABLE | + IODA3_MBT0_TYPE_M64 | + SETFIELD(IODA3_MBT0_MODE, 0ull, IODA3_MBT0_MODE_SINGLE_PE) | + SETFIELD(IODA3_MBT0_MDT_COLUMN, 0ull, 0) | + (p->mm0_base & IODA3_MBT0_BASE_ADDR); + p->mbt_cache[0][1] = IODA3_MBT1_ENABLE | + ((~(p->mm0_size - 1)) & IODA3_MBT1_MASK) | + SETFIELD(IODA3_MBT1_SINGLE_PE_NUM, 0ull, pe_number); + + p->mbt_cache[1][0] = IODA3_MBT0_ENABLE | + IODA3_MBT0_TYPE_M64 | + SETFIELD(IODA3_MBT0_MODE, 0ull, IODA3_MBT0_MODE_SINGLE_PE) | + SETFIELD(IODA3_MBT0_MDT_COLUMN, 0ull, 0) | + (0x0002000000000000ULL & IODA3_MBT0_BASE_ADDR); + p->mbt_cache[1][1] = IODA3_MBT1_ENABLE | + (0x00ff000000000000ULL & IODA3_MBT1_MASK) | + SETFIELD(IODA3_MBT1_SINGLE_PE_NUM, 0ull, pe_number); + + phb4_ioda_sel(p, IODA3_TBL_MBT, 0, true); + for (i = 0; i < p->mbt_size; i++) { + out_be64(p->regs + PHB_IODA_DATA0, p->mbt_cache[i][0]); + out_be64(p->regs + PHB_IODA_DATA0, p->mbt_cache[i][1]); + } + + phb4_init_capp_errors(p); + + phb4_init_capp_regs(p); + + if (!(p->rev == PHB4_REV_NIMBUS_DD10)) { + if (!chiptod_capp_timebase_sync(p->chip_id, CAPP_TFMR, + CAPP_TB, + PHB4_CAPP_REG_OFFSET(p))) { + PHBERR(p, "CAPP: Failed to sync timebase\n"); + return OPAL_HARDWARE; + } + } + return OPAL_SUCCESS; +} + +static int64_t phb4_set_capi_mode(struct phb *phb, uint64_t mode, + uint64_t pe_number) +{ + struct phb4 *p = phb_to_phb4(phb); + struct proc_chip *chip = get_chip(p->chip_id); + uint64_t reg; + uint32_t offset; + + lock(&capi_lock); + chip->capp_phb4_attached_mask |= 1 << p->index; + unlock(&capi_lock); + + offset = PHB4_CAPP_REG_OFFSET(p); + xscom_read(p->chip_id, CAPP_ERR_STATUS_CTRL + offset, ®); + if ((reg & PPC_BIT(5))) { + PHBERR(p, "CAPP: recovery failed (%016llx)\n", reg); + return OPAL_HARDWARE; + } else if ((reg & PPC_BIT(0)) && (!(reg & PPC_BIT(1)))) { + PHBDBG(p, "CAPP: recovery in progress\n"); + return OPAL_BUSY; + } + + switch (mode) { + case OPAL_PHB_CAPI_MODE_PCIE: + return OPAL_UNSUPPORTED; + + case OPAL_PHB_CAPI_MODE_CAPI: + return enable_capi_mode(p, pe_number); + + case OPAL_PHB_CAPI_MODE_SNOOP_OFF: + xscom_write(p->chip_id, SNOOP_CAPI_CONFIG + offset, + 0x0000000000000000); + return OPAL_SUCCESS; + + case OPAL_PHB_CAPI_MODE_SNOOP_ON: + xscom_write(p->chip_id, CAPP_ERR_STATUS_CTRL + offset, + 0x0000000000000000); + reg = 0xA000000000000000; + xscom_write(p->chip_id, SNOOP_CAPI_CONFIG + offset, reg); + + return OPAL_SUCCESS; + } + + return OPAL_UNSUPPORTED; +} + +static int64_t phb4_set_capp_recovery(struct phb *phb) +{ + struct phb4 *p = phb_to_phb4(phb); + + if (p->flags & PHB4_CAPP_RECOVERY) + return 0; + + /* set opal event flag to indicate eeh condition */ + opal_update_pending_evt(OPAL_EVENT_PCI_ERROR, + OPAL_EVENT_PCI_ERROR); + + p->flags |= PHB4_CAPP_RECOVERY; + + return 0; +} + static const struct phb_ops phb4_ops = { .cfg_read8 = phb4_pcicfg_read8, .cfg_read16 = phb4_pcicfg_read16, @@ -2393,6 +2793,8 @@ static const struct phb_ops phb4_ops = { .get_diag_data = NULL, .get_diag_data2 = phb4_get_diag_data, .tce_kill = phb4_tce_kill, + .set_capi_mode = phb4_set_capi_mode, + .set_capp_recovery = phb4_set_capp_recovery, }; static void phb4_init_ioda3(struct phb4 *p) @@ -3075,6 +3477,44 @@ static const struct irq_source_ops phb4_lsi_ops = { .attributes = phb4_lsi_attributes, }; +/* + * Trigger a creset to disable CAPI mode on kernel shutdown. + * + * This helper is called repeatedly by the host sync notifier mechanism, which + * relies on the kernel to regularly poll the OPAL_SYNC_HOST_REBOOT call as it + * shuts down. + * + * This is a somewhat hacky abuse of the host sync notifier mechanism, but the + * alternatives require a new API call which won't work for older kernels. + */ +static bool phb4_host_sync_reset(void *data) +{ + struct phb4 *p = (struct phb4 *)data; + struct pci_slot *slot = p->phb.slot; + struct proc_chip *chip = get_chip(p->chip_id); + int64_t rc; + + switch (slot->state) { + case PHB4_SLOT_NORMAL: + lock(&capi_lock); + rc = (chip->capp_phb4_attached_mask & (1 << p->index)) ? + OPAL_PHB_CAPI_MODE_CAPI : + OPAL_PHB_CAPI_MODE_PCIE; + unlock(&capi_lock); + + if (rc == OPAL_PHB_CAPI_MODE_PCIE) + return true; + + PHBINF(p, "PHB in CAPI mode, resetting\n"); + p->flags &= ~PHB4_CAPP_RECOVERY; + phb4_creset(slot); + return false; + default: + rc = slot->ops.poll(slot); + return rc <= OPAL_SUCCESS; + } +} + static void phb4_create(struct dt_node *np) { const struct dt_property *prop; @@ -3256,6 +3696,9 @@ static void phb4_create(struct dt_node *np) (p->rev == PHB4_REV_NIMBUS_DD10) ? &phb4_dd1_lsi_ops : &phb4_lsi_ops); + /* Trigger a creset to disable CAPI mode on kernel shutdown. */ + opal_add_host_sync_notifier(phb4_host_sync_reset, p); + /* Platform additional setup */ if (platform.pci_setup_phb) platform.pci_setup_phb(&p->phb, p->index); diff --git a/include/chip.h b/include/chip.h index 588db9f..a4a1e1e 100644 --- a/include/chip.h +++ b/include/chip.h @@ -190,6 +190,7 @@ struct proc_chip { /* Must hold capi_lock to change */ uint8_t capp_phb3_attached_mask; + uint8_t capp_phb4_attached_mask; uint8_t capp_ucode_loaded; /* Used by hw/centaur.c */ diff --git a/include/phb4-capp.h b/include/phb4-capp.h new file mode 100644 index 0000000..418838d --- /dev/null +++ b/include/phb4-capp.h @@ -0,0 +1,60 @@ +/* Copyright 2013-2017 IBM Corp. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + * implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef __PHB4_CAPP_H +#define __PHB4_CAPP_H + +#define CAPP_SNP_ARRAY_WRITE_REG 0x2010841 /* Satellite 2 */ +#define CAPP_SNP_ARRAY_ADDR_REG 0x2010828 +#define CAPP_APC_MASTER_ARRAY_ADDR_REG 0x201082A +#define CAPP_APC_MASTER_ARRAY_WRITE_REG 0x2010842 /* Satellite 2 */ + +#define CAPP_FIR 0x2010800 +#define CAPP_ERR_RPT_CLR 0x2010813 +#define APC_MASTER_PB_CTRL 0x2010818 +#define APC_MASTER_CAPI_CTRL 0x2010819 +#define LCO_MASTER_TARGET 0x2010821 +#define EPOCH_RECOVERY_TIMERS_CTRL 0x201082C +#define SNOOP_CAPI_CONFIG 0x201081A +#define SNOOP_CONTROL 0x201081B +#define TRANSPORT_CONTROL 0x201081C +#define CAPP_TB 0x2010826 +#define CAPP_TFMR 0x2010827 +#define CAPP_ERR_STATUS_CTRL 0x201080E +#define FLUSH_SUE_STATE_MAP 0x201080F +#define FLUSH_CPIG_STATE_MAP 0x2010820 +#define FLUSH_SUE_UOP1 0x2010843 /* Satellite 2 */ +#define APC_FSM_READ_MASK 0x2010823 +#define XPT_FSM_RMM 0x2010831 + +/* CAPP0 attached to PHB0(PEC0 - single port) + * CAPP1 attached to PHB3(PEC2 - single or dual port) + */ +#define CAPP0_PHB_INDEX 0 +#define CAPP1_PHB_INDEX 3 + +/* SCOM address Base (Ring = ‘0010’b) + * CAPP Unit Satellite SCOM address Base + * CAPP 0 S1 (sat = ‘0000’b) x02010800 + * CAPP 0 S2 (sat = ‘0001’b) x02010840 + * CAPP 1 S1 (sat = ‘0000’b) x04010800 + * CAPP 1 S2 (sat = ‘0001’b) x04010840 + */ +#define CAPP1_REG_OFFSET 0x2000000 + +#define PHB4_CAPP_REG_OFFSET(p) ((p)->index == 0 ? 0x0 : CAPP1_REG_OFFSET) + +#endif /* __PHB4_CAPP_H */ diff --git a/include/phb4-regs.h b/include/phb4-regs.h index 48953e4..4cb0b26 100644 --- a/include/phb4-regs.h +++ b/include/phb4-regs.h @@ -71,8 +71,12 @@ #define PHB_PEST_BAR 0x1a8 #define PHB_PEST_BAR_ENABLE PPC_BIT(0) #define PHB_PEST_BASE_ADDRESS PPC_BITMASK(8,51) +#define PHB_PBL_ASN_CMPM 0x1C0 +#define PHB_CAPI_ASN_ENABLE PPC_BIT(63) +#define PHB_CAPI_CMPM 0x1C8 +#define PHB_CAPI_CMPM_ENABLE PPC_BIT(63) #define PHB_M64_UPPER_BITS 0x1f0 -#define PHB_INTREP_TIMER 0x1f8 +#define PHB_XLATE_PREFIX 0x1f8 #define PHB_DMARD_SYNC 0x200 #define PHB_DMARD_SYNC_START PPC_BIT(0) #define PHB_DMARD_SYNC_COMPLETE PPC_BIT(1) @@ -157,6 +161,8 @@ #define PHB_RC_CONFIG_BASE 0x1000 #define PHB_PBL_TIMEOUT_CTRL 0x1810 +#define PHB_PBL_NBW_CMPM 0x1830 +#define PHB_CAPI_NBW_ENABLE PPC_BIT(63) // FIXME add more here #define PHB_PCIE_SCR 0x1A00 diff --git a/include/phb4.h b/include/phb4.h index bed6eef..651946c 100644 --- a/include/phb4.h +++ b/include/phb4.h @@ -222,6 +222,7 @@ struct phb4_err { #define PHB4_CFG_USE_ASB 0x00000002 #define PHB4_CFG_BLOCKED 0x00000004 #define PHB4_CAPP_RECOVERY 0x00000008 +#define PHB4_CAPP_DISABLING 0x00000010 struct phb4 { unsigned int index; /* 0..5 index inside p9 */