From patchwork Thu Aug 24 06:17:03 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Donnellan X-Patchwork-Id: 805243 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xdDc11nvYz9s8J for ; Thu, 24 Aug 2017 16:17:37 +1000 (AEST) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3xdDc0648QzDrKD for ; Thu, 24 Aug 2017 16:17:36 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3xdDbq6ZpxzDqXh for ; Thu, 24 Aug 2017 16:17:27 +1000 (AEST) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v7O6Exj4049054 for ; Thu, 24 Aug 2017 02:17:25 -0400 Received: from e23smtp08.au.ibm.com (e23smtp08.au.ibm.com [202.81.31.141]) by mx0a-001b2d01.pphosted.com with ESMTP id 2chrcvdt96-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Thu, 24 Aug 2017 02:17:25 -0400 Received: from localhost by e23smtp08.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 24 Aug 2017 16:17:22 +1000 Received: from d23relay08.au.ibm.com (202.81.31.227) by e23smtp08.au.ibm.com (202.81.31.205) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 24 Aug 2017 16:17:19 +1000 Received: from d23av01.au.ibm.com (d23av01.au.ibm.com [9.190.234.96]) by d23relay08.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v7O6HJFx39977114 for ; Thu, 24 Aug 2017 16:17:19 +1000 Received: from d23av01.au.ibm.com (localhost [127.0.0.1]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id v7O6HIRc020658 for ; Thu, 24 Aug 2017 16:17:19 +1000 Received: from ozlabs.au.ibm.com (ozlabs.au.ibm.com [9.192.253.14]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id v7O6HIGb020649; Thu, 24 Aug 2017 16:17:18 +1000 Received: from intelligence.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 3D8E6A01C4; Thu, 24 Aug 2017 16:17:18 +1000 (AEST) From: Andrew Donnellan To: skiboot@lists.ozlabs.org Date: Thu, 24 Aug 2017 16:17:03 +1000 X-Mailer: git-send-email 2.11.0 X-TM-AS-MML: disable x-cbid: 17082406-0048-0000-0000-00000259E6C9 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17082406-0049-0000-0000-0000480E1599 Message-Id: <20170824061703.374-1-andrew.donnellan@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-08-24_02:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1708240101 Subject: [Skiboot] [RFC PATCH] p9-adu: implement update sequence for PowerBus Hotplug Mode Control X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: alistair@popple.id.au MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" The PowerBus Hotplug Mode Control register is used to control various things about the PowerBus. Due to hardware timing constraints, updating this register requires more than just a simple SCOM write. There are two sets of pb_hp_mode registers, current (B) and next (A). Updating is done by a SCOM write to the next registers, followed by a SWITCH_AB pulse from the Alter/Display Unit, which triggers a hardware sequence to swap between the current and next registers. The sequence used in this code was derived from the hardware procedures used by Hostboot. This will be used in a subsequent patch as part of the OpenCAPI/NVLink init sequence. Signed-off-by: Andrew Donnellan --- makefile doesn't apply cleanly on master because this is off my private opencapi dev branch. Bikeshedding on the name "p9-adu" most welcome. --- hw/Makefile.inc | 1 + hw/p9-adu.c | 182 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ include/p9-adu.h | 57 +++++++++++++++++ 3 files changed, 240 insertions(+) create mode 100644 hw/p9-adu.c create mode 100644 include/p9-adu.h diff --git a/hw/Makefile.inc b/hw/Makefile.inc index 268c55ce..0be0f5ec 100644 --- a/hw/Makefile.inc +++ b/hw/Makefile.inc @@ -8,6 +8,7 @@ HW_OBJS += phb3.o sfc-ctrl.o fake-rtc.o bt.o p8-i2c.o prd.o HW_OBJS += dts.o lpc-rtc.o npu.o npu-hw-procedures.o xive.o phb4.o HW_OBJS += fake-nvram.o lpc-mbox.o npu2.o npu2-hw-procedures.o HW_OBJS += npu2-common.o npu2-opencapi.o phys-map.o sbe-p9.o capp.o occ-sensor.o +HW_OBJS += p9-adu.o HW=hw/built-in.o # FIXME hack this for now diff --git a/hw/p9-adu.c b/hw/p9-adu.c new file mode 100644 index 00000000..78708fa5 --- /dev/null +++ b/hw/p9-adu.c @@ -0,0 +1,182 @@ +/* Copyright 2013-2017 IBM Corp. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + * implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* + * Support code for POWER9 Fabric Alter/Display Unit (ADU) hotplug functionality + * + * The ADU acts as a bridge between the PowerBus and the Pervasive Interconnect + * Bus (PIB). + * + * Among other things, the ADU is used to support PowerBus Hotplug. Skiboot + * needs to set the PowerBus Hotplug Mode Control register as part of OpenCAPI + * and NVLink initialisation. + */ + +#include +#include +#include +#include +#include + +/* + * Lock or unlock ADU + */ +static int p9_adu_manage_lock(bool lock) +{ + int rc; + uint64_t val = 0; + struct proc_chip *chip = NULL; + if (lock) { + val |= PU_ALTD_CMD_REG_FBC_LOCKED; + val |= PU_ALTD_CMD_REG_FBC_ALTD_RESET_FSM; + val |= PU_ALTD_CMD_REG_FBC_ALTD_CLEAR_STATUS; + } + + while ((chip = next_chip(chip))) { + rc = xscom_write(chip->id, PU_ALTD_CMD_REG, val); + if (rc != OPAL_SUCCESS) { + /* TODO: Lock picking support */ + prlog(PR_ERR, + "ADU: Error %d writing command (chip %d)\n", + rc, chip->id); + return OPAL_HARDWARE; + } + } + return OPAL_SUCCESS; +} + +static uint32_t find_master_chip(void) +{ + uint64_t reg; + struct proc_chip *chip = NULL; + while ((chip = next_chip(chip))) { + xscom_read(chip->id, PB_CENT_HP_MODE_CURR, ®); + if (reg & PB_HP_MODE_MASTER_CHIP) + break; + } + return chip->id; +} + +/* + * Trigger a SWITCH_AB pulse to switch the current PowerBus Hotplug Mode Control + * register set. + * + * Overview of sequence: + * + * - acquire lock and reset all ADUs + * - configure all ADUs for AB switch + * - configure one ADU (on the fabric master chip) to issue a + * quiesce/switch/reinit + * - check status + * - clear switch selectors + * - reset all ADUs + * - unlock + */ +static void p9_adu_switch_ab(void) +{ + uint32_t gcid = find_master_chip(); + struct proc_chip *chip = NULL; + uint64_t reg; + uint64_t val; + int rc = OPAL_SUCCESS; + + /* + * There's a performance issue on P9DD1 that requires a workaround: + * see IBM defect HW397129. However, this code isn't expected to be + * used on DD1 machines. + */ + + rc = p9_adu_manage_lock(true); + if (rc) + goto err; + + /* Set PB_SWITCH_AB on all ADUs */ + while ((chip = next_chip(chip))) { + xscom_read(chip->id, PU_SND_MODE_REG, ®); + reg |= PU_SND_MODE_REG_ENABLE_PB_SWITCH_AB; + rc = xscom_write(chip->id, PU_SND_MODE_REG, reg); + if (rc) + goto err_switch; + } + + /* Set address 0 */ + rc = xscom_write(gcid, PU_ALTD_ADDR_REG, 0); + if (rc) + goto err_switch; + + /* Configure ADU to issue quiesce + switch + reinit */ + val = PU_ALTD_OPTION_REG_FBC_ALTD_WITH_PRE_QUIESCE; + val = SETFIELD(PU_ALTD_OPTION_REG_FBC_ALTD_AFTER_QUIESCE_WAIT_COUNT, + val, QUIESCE_SWITCH_WAIT_COUNT); + val |= PU_ALTD_OPTION_REG_FBC_ALTD_WITH_POST_INIT; + val = SETFIELD(PU_ALTD_OPTION_REG_FBC_ALTD_BEFORE_INIT_WAIT_COUNT, + val, INIT_SWITCH_WAIT_COUNT); + val |= PU_ALTD_OPTION_REG_FBC_ALTD_WITH_FAST_PATH; /* see HW397129, DD2 */ + rc = xscom_write(gcid, PU_ALTD_OPTION_REG, val); + if (rc) + goto err_switch; + + /* Set up command */ + val = PU_ALTD_CMD_REG_FBC_LOCKED; + val |= PU_ALTD_CMD_REG_FBC_ALTD_START_OP; + val = SETFIELD(PU_ALTD_CMD_REG_FBC_ALTD_SCOPE, val, + PU_ALTD_CMD_REG_FBC_ALTD_SCOPE_SYSTEM); + val |= PU_ALTD_CMD_REG_FBC_ALTD_DROP_PRIORITY; + val |= PU_ALTD_CMD_REG_FBC_ALTD_AXTYPE; + val = SETFIELD(PU_ALTD_CMD_REG_FBC_ALTD_TTYPE, val, + PU_ALTD_CMD_REG_FBC_ALTD_TTYPE_PMISC_OPER); + val |= PU_ALTD_CMD_REG_FBC_ALTD_WITH_TM_QUIESCE; + val = SETFIELD(PU_ALTD_CMD_REG_FBC_ALTD_TSIZE, val, + PU_ALTD_CMD_REG_FBC_ALTD_TSIZE_PMISC_1); + xscom_write(gcid, PU_ALTD_CMD_REG, val); + + /* + * TODO: check ADU status is consistent, see + * p9_build_smp_adu_check_status() in hostboot + */ + +err_switch: + /* Reset switch controls */ + chip = NULL; + while ((chip = next_chip(chip))) { + xscom_read(chip->id, PU_SND_MODE_REG, ®); + reg &= ~PU_SND_MODE_REG_ENABLE_PB_SWITCH_AB; + xscom_write(chip->id, PU_SND_MODE_REG, reg); + } + + /* Reset ADUs */ + p9_adu_manage_lock(true); + +err: + /* Unlock */ + p9_adu_manage_lock(false); +} + +void p9_adu_set_pb_hp_mode(uint32_t gcid, uint64_t val) +{ + /* Write next value */ + xscom_write(gcid, PB_WEST_HP_MODE_NEXT, val); + xscom_write(gcid, PB_CENT_HP_MODE_NEXT, val); + xscom_write(gcid, PB_EAST_HP_MODE_NEXT, val); + + /* Send switch pulse */ + p9_adu_switch_ab(); + + /* Now that switch is complete, overwrite old value */ + xscom_write(gcid, PB_WEST_HP_MODE_NEXT, val); + xscom_write(gcid, PB_CENT_HP_MODE_NEXT, val); + xscom_write(gcid, PB_EAST_HP_MODE_NEXT, val); +} diff --git a/include/p9-adu.h b/include/p9-adu.h new file mode 100644 index 00000000..a29332b1 --- /dev/null +++ b/include/p9-adu.h @@ -0,0 +1,57 @@ +/* Copyright 2013-2017 IBM Corp. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + * implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +void p9_adu_set_pb_hp_mode(uint32_t gcid, uint64_t val); + +#define PU_ALTD_ADDR_REG 0x0090000 + +#define PU_ALTD_CMD_REG 0x0090001 +#define PU_ALTD_CMD_REG_FBC_ALTD_START_OP PPC_BIT(2) +#define PU_ALTD_CMD_REG_FBC_ALTD_CLEAR_STATUS PPC_BIT(3) +#define PU_ALTD_CMD_REG_FBC_ALTD_RESET_FSM PPC_BIT(4) +#define PU_ALTD_CMD_REG_FBC_ALTD_AXTYPE PPC_BIT(6) +#define PU_ALTD_CMD_REG_FBC_LOCKED PPC_BIT(11) +#define PU_ALTD_CMD_REG_FBC_LOCK_ID PPC_BITMASK(12, 15) +#define PU_ALTD_CMD_REG_FBC_ALTD_SCOPE PPC_BITMASK(16, 18) +#define PU_ALTD_CMD_REG_FBC_ALTD_SCOPE_SYSTEM 0b101 +#define PU_ALTD_CMD_REG_FBC_ALTD_DROP_PRIORITY PPC_BIT(20) +#define PU_ALTD_CMD_REG_FBC_ALTD_WITH_TM_QUIESCE PPC_BIT(24) +#define PU_ALTD_CMD_REG_FBC_ALTD_TTYPE PPC_BITMASK(25, 31) +#define PU_ALTD_CMD_REG_FBC_ALTD_TTYPE_PMISC_OPER 0b0110001 +#define PU_ALTD_CMD_REG_FBC_ALTD_TSIZE PPC_BITMASK(32, 39) +#define PU_ALTD_CMD_REG_FBC_ALTD_TSIZE_PMISC_1 0b00000010 + +#define PU_ALTD_OPTION_REG 0x0090002 +#define PU_ALTD_OPTION_REG_FBC_ALTD_WITH_PRE_QUIESCE PPC_BIT(23) +#define PU_ALTD_OPTION_REG_FBC_ALTD_AFTER_QUIESCE_WAIT_COUNT PPC_BITMASK(28, 47) +#define PU_ALTD_OPTION_REG_FBC_ALTD_WITH_POST_INIT PPC_BIT(51) +#define PU_ALTD_OPTION_REG_FBC_ALTD_WITH_FAST_PATH PPC_BIT(52) +#define PU_ALTD_OPTION_REG_FBC_ALTD_BEFORE_INIT_WAIT_COUNT PPC_BITMASK(54, 63) + +#define QUIESCE_SWITCH_WAIT_COUNT 128 +#define INIT_SWITCH_WAIT_COUNT 128 + +#define PU_SND_MODE_REG 0x0090021 +#define PU_SND_MODE_REG_ENABLE_PB_SWITCH_AB PPC_BIT(30) + +// Hotplug registers +#define PB_WEST_HP_MODE_NEXT 0x501180B +#define PB_CENT_HP_MODE_NEXT 0x5011C0B +#define PB_EAST_HP_MODE_NEXT 0x501200B +#define PB_WEST_HP_MODE_CURR 0x501180C +#define PB_CENT_HP_MODE_CURR 0x5011C0C +#define PB_EAST_HP_MODE_CURR 0x501200C +#define PB_HP_MODE_MASTER_CHIP PPC_BIT(0)