From patchwork Wed May 23 17:25:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 919252 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40rgHg3Kwyz9s0y for ; Thu, 24 May 2018 03:58:55 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40rgHg1xNVzF189 for ; Thu, 24 May 2018 03:58:55 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40rfYb2LF5zF12G for ; Thu, 24 May 2018 03:25:55 +1000 (AEST) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4NHNwxR124536 for ; Wed, 23 May 2018 13:25:53 -0400 Received: from e06smtp14.uk.ibm.com (e06smtp14.uk.ibm.com [195.75.94.110]) by mx0a-001b2d01.pphosted.com with ESMTP id 2j5bnqaubb-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 23 May 2018 13:25:52 -0400 Received: from localhost by e06smtp14.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 23 May 2018 18:25:51 +0100 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp14.uk.ibm.com (192.168.101.144) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 23 May 2018 18:25:49 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w4NHPlwK15991062; Wed, 23 May 2018 17:25:48 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 513A911C054; Wed, 23 May 2018 18:16:53 +0100 (BST) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DCB5B11C050; Wed, 23 May 2018 18:16:52 +0100 (BST) Received: from localhost.localdomain (unknown [9.164.160.79]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 23 May 2018 18:16:52 +0100 (BST) From: Frederic Barrat To: andrew.donnellan@au1.ibm.com, skiboot@lists.ozlabs.org Date: Wed, 23 May 2018 19:25:41 +0200 X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180523172545.16016-1-fbarrat@linux.ibm.com> References: <20180523172545.16016-1-fbarrat@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18052317-0044-0000-0000-00000555BF85 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18052317-0045-0000-0000-00002897543A Message-Id: <20180523172545.16016-2-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-23_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1805230169 Subject: [Skiboot] [PATCH v2 1/5] npu2-opencapi: Use presence detection X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" From: Frederic Barrat Presence detection is not part of the opencapi specification. So each platform may choose to implement it the way it wants. All current platforms implement it through an i2c device where we can query a pin to know if a device is connected or not. ZZ and Zaius have a similar design and even use the same i2c information and pin numbers. However, presence detection on older ZZ planar (older than v4) doesn't work, so we don't activate it for now, until our lab systems are upgraded and it's better tested. Presence detection on witherspoon is still being worked on. It's shaping up to be quite different, so we may have to revisit the topic in a later patch. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- v2: define platform_ocapi with inline literals, to match existing style rework a few comments core/platform.c | 11 ++++- hw/npu2-opencapi.c | 121 ++++++++++++++++++++++++++++++++++++++--------- include/platform.h | 4 ++ platforms/astbmc/zaius.c | 6 ++- platforms/ibm-fsp/zz.c | 10 +++- 5 files changed, 127 insertions(+), 25 deletions(-) diff --git a/core/platform.c b/core/platform.c index f09ea3c1..92b118ba 100644 --- a/core/platform.c +++ b/core/platform.c @@ -175,7 +175,16 @@ const struct platform_ocapi generic_ocapi = { .i2c_offset = { 0x3, 0x1, 0x1 }, .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, - .odl_phy_swap = true, + .i2c_presence_addr = 0x20, + .i2c_presence_odl0 = (1 << 2), /* bottom connector */ + .i2c_presence_odl1 = (1 << 7), /* top connector */ + /* + * The ZZs we typically use for BML/generic platform tend to + * have old planars and presence detection is broken there, so + * force presence. + */ + .force_presence = true, + .odl_phy_swap = true, }; static struct bmc_platform generic_bmc = { diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 2cc776c7..b2bc0628 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -829,6 +829,44 @@ static void reset_ocapi_device(struct npu2_dev *dev) } } +static bool i2c_presence_detect(struct npu2_dev *dev) +{ + uint8_t state, data; + int rc; + + /* + * Opencapi presence detection is done through i2c + * + * Lagrange platforms (ZZ, Zaius) use the same default mechanism. + * Witherspoon will need a specific implementation, TBD. + */ + rc = i2c_request_send(dev->i2c_port_id_ocapi, + platform.ocapi->i2c_presence_addr, + SMBUS_READ, 0, 1, + &state, 1, 120); + if (rc) { + prlog(PR_ERR, "OCAPI: error detecting link presence: %d\n", + rc); + return true; /* assume link exists */ + } + + prlog(PR_DEBUG, "OCAPI: I2C presence detect: 0x%x\n", state); + + switch (dev->index) { + case 2: + data = platform.ocapi->i2c_presence_odl0; + break; + case 3: + data = platform.ocapi->i2c_presence_odl1; + break; + default: + prlog(PR_ERR, "OCAPI: presence detection on invalid link\n"); + return true; + } + /* Presence detect bits are active low */ + return !(state & data); +} + static int odl_train(uint32_t gcid, uint32_t index, struct npu2_dev *dev) { uint64_t reg, config_xscom; @@ -896,6 +934,20 @@ static int odl_train(uint32_t gcid, uint32_t index, struct npu2_dev *dev) return OPAL_HARDWARE; } +static int64_t npu2_opencapi_get_presence_state(struct pci_slot *slot, + uint8_t *val) +{ + bool present; + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + + if (platform.ocapi->force_presence) + present = true; + else + present = i2c_presence_detect(dev); + *val = present; + return OPAL_SUCCESS; +} + static int64_t npu2_opencapi_get_link_state(struct pci_slot *slot, uint8_t *val) { struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); @@ -926,9 +978,9 @@ static struct pci_slot *npu2_opencapi_slot_create(struct phb *phb) return slot; /* TODO: Figure out other slot functions */ - slot->ops.get_presence_state = NULL; - slot->ops.get_link_state = npu2_opencapi_get_link_state; - slot->ops.get_power_state = NULL; + slot->ops.get_presence_state = npu2_opencapi_get_presence_state; + slot->ops.get_link_state = npu2_opencapi_get_link_state; + slot->ops.get_power_state = NULL; slot->ops.get_attention_state = NULL; slot->ops.get_latch_state = NULL; slot->ops.set_power_state = NULL; @@ -1264,6 +1316,48 @@ static int setup_irq(struct npu2 *p) #define LINK_TRAINING_RETRIES 5 +static int train_link(int chip_id, struct npu2_dev *dev) +{ + bool train; + int rc; + int retries = LINK_TRAINING_RETRIES; + + if (platform.ocapi->force_presence) + train = true; + else + train = i2c_presence_detect(dev); + if (!train) { + /* + * FIXME: if there's no card on the link, we should consider + * powering off the unused lanes to save energy + */ + prlog(PR_INFO, "OCAPI: no card detected on link %d, chip %d\n", + dev->index, chip_id); + return -1; + } + + do { + rc = odl_train(chip_id, dev->index, dev); + } while (rc != OPAL_SUCCESS && --retries); + + if (rc != OPAL_SUCCESS && retries == 0) { + /** + * @fwts-label OCAPILinkTrainingFailed + * @fwts-advice The OpenCAPI link training procedure failed. + * This indicates a hardware or firmware bug. OpenCAPI + * functionality will not be available on this link. + */ + prlog(PR_ERR, + "OCAPI: Link %d on chip %u failed to train\n", + dev->index, chip_id); + prlog(PR_ERR, "OCAPI: Final link status: %016llx\n", + get_odl_status(chip_id, dev->index)); + return -1; + } + + return 0; +} + static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, struct npu2_dev *dev) { @@ -1272,7 +1366,6 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, struct pci_slot *slot; char port_name[17]; uint64_t mm_win[2]; - int retries = LINK_TRAINING_RETRIES; int rc; dev_index = dt_prop_get_u32(dn_link, "ibm,npu-link-index"); @@ -1367,27 +1460,11 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, break; case NPU2_TRAIN_DEFAULT: - do { - rc = odl_train(n->chip_id, dev->index, dev); - } while (rc != OPAL_SUCCESS && --retries); - - if (rc != OPAL_SUCCESS && retries == 0) { - /** - * @fwts-label OCAPILinkTrainingFailed - * @fwts-advice The OpenCAPI link training procedure failed. - * This indicates a hardware or firmware bug. OpenCAPI - * functionality will not be available on this link. - */ - prlog(PR_ERR, - "OCAPI: Link %d on chip %u failed to train\n", - dev->index, n->chip_id); - prlog(PR_ERR, "OCAPI: Final link status: %016llx\n", - get_odl_status(n->chip_id, dev->index)); + rc = train_link(n->chip_id, dev); + if (rc) goto failed; - } otl_enabletx(n->chip_id, n->xscom_base, dev->index); - slot = npu2_opencapi_slot_create(&dev->phb_ocapi); if (!slot) { /** diff --git a/include/platform.h b/include/platform.h index a7776446..6fa016c2 100644 --- a/include/platform.h +++ b/include/platform.h @@ -51,6 +51,10 @@ struct platform_ocapi { uint32_t i2c_offset[3]; /* Offsets on I2C device */ uint8_t i2c_odl0_data[3]; /* Data to reset ODL0 */ uint8_t i2c_odl1_data[3]; /* Data to reset ODL1 */ + uint8_t i2c_presence_addr; /* I2C address for presence detection */ + uint8_t i2c_presence_odl0; /* I2C mask for detection on ODL0 */ + uint8_t i2c_presence_odl1; /* I2C mask for detection on ODL1 */ + bool force_presence; /* don't use i2c detection */ bool odl_phy_swap; /* Swap ODL1 to use brick 2 rather than * brick 1 lanes */ }; diff --git a/platforms/astbmc/zaius.c b/platforms/astbmc/zaius.c index 7678ad12..52b072af 100644 --- a/platforms/astbmc/zaius.c +++ b/platforms/astbmc/zaius.c @@ -24,13 +24,17 @@ #include "astbmc.h" + const struct platform_ocapi zaius_ocapi = { .i2c_engine = 1, .i2c_port = 4, .i2c_offset = { 0x3, 0x1, 0x1 }, .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, - .odl_phy_swap = true, + .i2c_presence_addr = 0x20, + .i2c_presence_odl0 = (1 << 2), /* bottom connector */ + .i2c_presence_odl1 = (1 << 7), /* top connector */ + .odl_phy_swap = true, }; #define NPU_BASE 0x5011000 diff --git a/platforms/ibm-fsp/zz.c b/platforms/ibm-fsp/zz.c index 9a849290..6c8411ec 100644 --- a/platforms/ibm-fsp/zz.c +++ b/platforms/ibm-fsp/zz.c @@ -34,7 +34,15 @@ const struct platform_ocapi zz_ocapi = { .i2c_offset = { 0x3, 0x1, 0x1 }, .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, - .odl_phy_swap = true, + .i2c_presence_addr = 0x20, + .i2c_presence_odl0 = (1 << 2), /* bottom connector */ + .i2c_presence_odl1 = (1 << 7), /* top connector */ + /* + * i2c presence detection is broken on ZZ planar < v4 so we + * force the presence until all our systems are upgraded + */ + .force_presence = true, + .odl_phy_swap = true, }; static bool zz_probe(void)