From patchwork Thu Apr 12 12:45:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 897640 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40MLHm2MJBz9s2M for ; Thu, 12 Apr 2018 22:46:12 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40MLHm0wBJzF1bP for ; Thu, 12 Apr 2018 22:46:12 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40MLH43rGCzDqyL for ; Thu, 12 Apr 2018 22:45:36 +1000 (AEST) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w3CCj3ac126007 for ; Thu, 12 Apr 2018 08:45:33 -0400 Received: from e06smtp12.uk.ibm.com (e06smtp12.uk.ibm.com [195.75.94.108]) by mx0a-001b2d01.pphosted.com with ESMTP id 2ha6764mm7-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 12 Apr 2018 08:45:33 -0400 Received: from localhost by e06smtp12.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 12 Apr 2018 13:45:31 +0100 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp12.uk.ibm.com (192.168.101.142) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 12 Apr 2018 13:45:29 +0100 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w3CCjS7u37814496; Thu, 12 Apr 2018 12:45:28 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B3B805204B; Thu, 12 Apr 2018 12:36:24 +0100 (BST) Received: from borneo.lab.toulouse-stg.fr.ibm.com (unknown [9.101.4.34]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id 9434852047; Thu, 12 Apr 2018 12:36:24 +0100 (BST) From: Frederic Barrat To: andrew.donnellan@au1.ibm.com, skiboot@lists.ozlabs.org Date: Thu, 12 Apr 2018 14:45:22 +0200 X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180412124526.12662-1-fbarrat@linux.ibm.com> References: <20180412124526.12662-1-fbarrat@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18041212-0008-0000-0000-000004E9C0AF X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18041212-0009-0000-0000-00001E7DC9C2 Message-Id: <20180412124526.12662-2-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-04-12_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804120127 Subject: [Skiboot] [PATCH 1/5] npu2-opencapi: Use presence detection X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" From: Frederic Barrat Presence detection is not part of the opencapi specification. So each platform may choose to implement it the way it wants. All current platforms implement it through an i2c device where we can query a pin to know if a device is connected or not. ZZ and Zaius have a similar design and even use the same i2c information and pin numbers. However, presence detection on older ZZ planar (older than v4) doesn't work, so we don't activate it for now, until our lab systems are upgraded and it's better tested. Presence detection on witherspoon is still being worked on, but is very likely to follow something similar, i.e. an i2c pin to query per device. Signed-off-by: Frederic Barrat Acked-by: Andrew Donnellan --- core/platform.c | 14 +++++- hw/npu2-opencapi.c | 121 ++++++++++++++++++++++++++++++++++++++--------- include/platform.h | 4 ++ platforms/astbmc/zaius.c | 9 +++- platforms/ibm-fsp/zz.c | 14 +++++- 5 files changed, 137 insertions(+), 25 deletions(-) diff --git a/core/platform.c b/core/platform.c index f09ea3c1..174235d9 100644 --- a/core/platform.c +++ b/core/platform.c @@ -34,6 +34,10 @@ DEFINE_LOG_ENTRY(OPAL_RC_ABNORMAL_REBOOT, OPAL_PLATFORM_ERR_EVT, OPAL_CEC, OPAL_CEC_HARDWARE, OPAL_PREDICTIVE_ERR_FAULT_RECTIFY_REBOOT, OPAL_ABNORMAL_POWER_OFF); +#define OCAPI_I2C_PRESENCE_ADDR 0x20 +#define OCAPI_I2C_PRESENCE_BOTTOM (1 << 2) +#define OCAPI_I2C_PRESENCE_TOP (1 << 7) + /* * Various wrappers for platform functions */ @@ -175,7 +179,15 @@ const struct platform_ocapi generic_ocapi = { .i2c_offset = { 0x3, 0x1, 0x1 }, .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, - .odl_phy_swap = true, + .i2c_presence_addr = OCAPI_I2C_PRESENCE_ADDR, + .i2c_presence_odl0 = OCAPI_I2C_PRESENCE_BOTTOM, + .i2c_presence_odl1 = OCAPI_I2C_PRESENCE_TOP, + /* + * i2c presence detection is broken on ZZ planar < v4 so we + * force the presence until all our systems are upgraded + */ + .force_presence = true, + .odl_phy_swap = true, }; static struct bmc_platform generic_bmc = { diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 2cc776c7..b2bc0628 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -829,6 +829,44 @@ static void reset_ocapi_device(struct npu2_dev *dev) } } +static bool i2c_presence_detect(struct npu2_dev *dev) +{ + uint8_t state, data; + int rc; + + /* + * Opencapi presence detection is done through i2c + * + * Lagrange platforms (ZZ, Zaius) use the same default mechanism. + * Witherspoon will need a specific implementation, TBD. + */ + rc = i2c_request_send(dev->i2c_port_id_ocapi, + platform.ocapi->i2c_presence_addr, + SMBUS_READ, 0, 1, + &state, 1, 120); + if (rc) { + prlog(PR_ERR, "OCAPI: error detecting link presence: %d\n", + rc); + return true; /* assume link exists */ + } + + prlog(PR_DEBUG, "OCAPI: I2C presence detect: 0x%x\n", state); + + switch (dev->index) { + case 2: + data = platform.ocapi->i2c_presence_odl0; + break; + case 3: + data = platform.ocapi->i2c_presence_odl1; + break; + default: + prlog(PR_ERR, "OCAPI: presence detection on invalid link\n"); + return true; + } + /* Presence detect bits are active low */ + return !(state & data); +} + static int odl_train(uint32_t gcid, uint32_t index, struct npu2_dev *dev) { uint64_t reg, config_xscom; @@ -896,6 +934,20 @@ static int odl_train(uint32_t gcid, uint32_t index, struct npu2_dev *dev) return OPAL_HARDWARE; } +static int64_t npu2_opencapi_get_presence_state(struct pci_slot *slot, + uint8_t *val) +{ + bool present; + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + + if (platform.ocapi->force_presence) + present = true; + else + present = i2c_presence_detect(dev); + *val = present; + return OPAL_SUCCESS; +} + static int64_t npu2_opencapi_get_link_state(struct pci_slot *slot, uint8_t *val) { struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); @@ -926,9 +978,9 @@ static struct pci_slot *npu2_opencapi_slot_create(struct phb *phb) return slot; /* TODO: Figure out other slot functions */ - slot->ops.get_presence_state = NULL; - slot->ops.get_link_state = npu2_opencapi_get_link_state; - slot->ops.get_power_state = NULL; + slot->ops.get_presence_state = npu2_opencapi_get_presence_state; + slot->ops.get_link_state = npu2_opencapi_get_link_state; + slot->ops.get_power_state = NULL; slot->ops.get_attention_state = NULL; slot->ops.get_latch_state = NULL; slot->ops.set_power_state = NULL; @@ -1264,6 +1316,48 @@ static int setup_irq(struct npu2 *p) #define LINK_TRAINING_RETRIES 5 +static int train_link(int chip_id, struct npu2_dev *dev) +{ + bool train; + int rc; + int retries = LINK_TRAINING_RETRIES; + + if (platform.ocapi->force_presence) + train = true; + else + train = i2c_presence_detect(dev); + if (!train) { + /* + * FIXME: if there's no card on the link, we should consider + * powering off the unused lanes to save energy + */ + prlog(PR_INFO, "OCAPI: no card detected on link %d, chip %d\n", + dev->index, chip_id); + return -1; + } + + do { + rc = odl_train(chip_id, dev->index, dev); + } while (rc != OPAL_SUCCESS && --retries); + + if (rc != OPAL_SUCCESS && retries == 0) { + /** + * @fwts-label OCAPILinkTrainingFailed + * @fwts-advice The OpenCAPI link training procedure failed. + * This indicates a hardware or firmware bug. OpenCAPI + * functionality will not be available on this link. + */ + prlog(PR_ERR, + "OCAPI: Link %d on chip %u failed to train\n", + dev->index, chip_id); + prlog(PR_ERR, "OCAPI: Final link status: %016llx\n", + get_odl_status(chip_id, dev->index)); + return -1; + } + + return 0; +} + static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, struct npu2_dev *dev) { @@ -1272,7 +1366,6 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, struct pci_slot *slot; char port_name[17]; uint64_t mm_win[2]; - int retries = LINK_TRAINING_RETRIES; int rc; dev_index = dt_prop_get_u32(dn_link, "ibm,npu-link-index"); @@ -1367,27 +1460,11 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, break; case NPU2_TRAIN_DEFAULT: - do { - rc = odl_train(n->chip_id, dev->index, dev); - } while (rc != OPAL_SUCCESS && --retries); - - if (rc != OPAL_SUCCESS && retries == 0) { - /** - * @fwts-label OCAPILinkTrainingFailed - * @fwts-advice The OpenCAPI link training procedure failed. - * This indicates a hardware or firmware bug. OpenCAPI - * functionality will not be available on this link. - */ - prlog(PR_ERR, - "OCAPI: Link %d on chip %u failed to train\n", - dev->index, n->chip_id); - prlog(PR_ERR, "OCAPI: Final link status: %016llx\n", - get_odl_status(n->chip_id, dev->index)); + rc = train_link(n->chip_id, dev); + if (rc) goto failed; - } otl_enabletx(n->chip_id, n->xscom_base, dev->index); - slot = npu2_opencapi_slot_create(&dev->phb_ocapi); if (!slot) { /** diff --git a/include/platform.h b/include/platform.h index a7776446..9a04ab37 100644 --- a/include/platform.h +++ b/include/platform.h @@ -51,6 +51,10 @@ struct platform_ocapi { uint32_t i2c_offset[3]; /* Offsets on I2C device */ uint8_t i2c_odl0_data[3]; /* Data to reset ODL0 */ uint8_t i2c_odl1_data[3]; /* Data to reset ODL1 */ + uint8_t i2c_presence_addr; /* I2C address for presence detection */ + uint8_t i2c_presence_odl0; /* I2C pin to read to detect ODL0 */ + uint8_t i2c_presence_odl1; /* I2C pin to read to detect ODL1 */ + bool force_presence; /* don't use i2c detection */ bool odl_phy_swap; /* Swap ODL1 to use brick 2 rather than * brick 1 lanes */ }; diff --git a/platforms/astbmc/zaius.c b/platforms/astbmc/zaius.c index 7678ad12..279fbf8b 100644 --- a/platforms/astbmc/zaius.c +++ b/platforms/astbmc/zaius.c @@ -24,13 +24,20 @@ #include "astbmc.h" +#define OCAPI_I2C_PRESENCE_ADDR 0x20 +#define OCAPI_I2C_PRESENCE_BOTTOM (1 << 2) +#define OCAPI_I2C_PRESENCE_TOP (1 << 7) + const struct platform_ocapi zaius_ocapi = { .i2c_engine = 1, .i2c_port = 4, .i2c_offset = { 0x3, 0x1, 0x1 }, .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, - .odl_phy_swap = true, + .i2c_presence_addr = OCAPI_I2C_PRESENCE_ADDR, + .i2c_presence_odl0 = OCAPI_I2C_PRESENCE_BOTTOM, + .i2c_presence_odl1 = OCAPI_I2C_PRESENCE_TOP, + .odl_phy_swap = true, }; #define NPU_BASE 0x5011000 diff --git a/platforms/ibm-fsp/zz.c b/platforms/ibm-fsp/zz.c index 9a849290..66f29d44 100644 --- a/platforms/ibm-fsp/zz.c +++ b/platforms/ibm-fsp/zz.c @@ -27,6 +27,10 @@ #include "ibm-fsp.h" #include "lxvpd.h" +#define OCAPI_I2C_PRESENCE_ADDR 0x20 +#define OCAPI_I2C_PRESENCE_BOTTOM (1 << 2) +#define OCAPI_I2C_PRESENCE_TOP (1 << 7) + /* We don't yet create NPU device nodes on ZZ, but these values are correct */ const struct platform_ocapi zz_ocapi = { .i2c_engine = 1, @@ -34,7 +38,15 @@ const struct platform_ocapi zz_ocapi = { .i2c_offset = { 0x3, 0x1, 0x1 }, .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, - .odl_phy_swap = true, + .i2c_presence_addr = OCAPI_I2C_PRESENCE_ADDR, + .i2c_presence_odl0 = OCAPI_I2C_PRESENCE_BOTTOM, + .i2c_presence_odl1 = OCAPI_I2C_PRESENCE_TOP, + /* + * i2c presence detection is broken on ZZ planar < v4 so we + * force the presence until all our systems are upgraded + */ + .force_presence = true, + .odl_phy_swap = true, }; static bool zz_probe(void) From patchwork Thu Apr 12 12:45:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 897641 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40MLHw4ct5z9s2M for ; Thu, 12 Apr 2018 22:46:20 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40MLHw35xxzF1SP for ; Thu, 12 Apr 2018 22:46:20 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40MLH560rhzDr4R for ; Thu, 12 Apr 2018 22:45:37 +1000 (AEST) Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w3CCfE41005332 for ; Thu, 12 Apr 2018 08:45:35 -0400 Received: from e06smtp11.uk.ibm.com (e06smtp11.uk.ibm.com [195.75.94.107]) by mx0a-001b2d01.pphosted.com with ESMTP id 2ha5ckxjeu-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 12 Apr 2018 08:45:35 -0400 Received: from localhost by e06smtp11.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 12 Apr 2018 13:45:33 +0100 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp11.uk.ibm.com (192.168.101.141) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 12 Apr 2018 13:45:30 +0100 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w3CCjTec43647184; Thu, 12 Apr 2018 12:45:29 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E28CA5204D; Thu, 12 Apr 2018 12:36:25 +0100 (BST) Received: from borneo.lab.toulouse-stg.fr.ibm.com (unknown [9.101.4.34]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id C322752043; Thu, 12 Apr 2018 12:36:25 +0100 (BST) From: Frederic Barrat To: andrew.donnellan@au1.ibm.com, skiboot@lists.ozlabs.org Date: Thu, 12 Apr 2018 14:45:23 +0200 X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180412124526.12662-1-fbarrat@linux.ibm.com> References: <20180412124526.12662-1-fbarrat@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18041212-0040-0000-0000-0000044CBF13 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18041212-0041-0000-0000-000020F0F2E2 Message-Id: <20180412124526.12662-3-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-04-12_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804120127 Subject: [Skiboot] [PATCH 2/5] npu2-opencapi: Rework adapter reset X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" From: Frederic Barrat Rework a bit the code to reset the opencapi adapter: - make clearer which i2c pin is resetting which device - break the reset operation in smaller chunks. This is really to prepare for a future patch. No functional changes. Signed-off-by: Frederic Barrat Acked-by: Andrew Donnellan --- core/platform.c | 15 ++++++---- hw/npu2-opencapi.c | 71 ++++++++++++++++++++++++++++++++++-------------- include/platform.h | 6 ++-- platforms/astbmc/zaius.c | 15 ++++++---- platforms/ibm-fsp/zz.c | 15 ++++++---- 5 files changed, 81 insertions(+), 41 deletions(-) diff --git a/core/platform.c b/core/platform.c index 174235d9..b7b048f2 100644 --- a/core/platform.c +++ b/core/platform.c @@ -34,6 +34,9 @@ DEFINE_LOG_ENTRY(OPAL_RC_ABNORMAL_REBOOT, OPAL_PLATFORM_ERR_EVT, OPAL_CEC, OPAL_CEC_HARDWARE, OPAL_PREDICTIVE_ERR_FAULT_RECTIFY_REBOOT, OPAL_ABNORMAL_POWER_OFF); +#define OCAPI_I2C_RESET_ADDR 0x20 +#define OCAPI_I2C_RESET_BOTTOM (1 << 1) +#define OCAPI_I2C_RESET_TOP (1 << 6) #define OCAPI_I2C_PRESENCE_ADDR 0x20 #define OCAPI_I2C_PRESENCE_BOTTOM (1 << 2) #define OCAPI_I2C_PRESENCE_TOP (1 << 7) @@ -174,11 +177,11 @@ static int generic_start_preload_resource(enum resource_id id, uint32_t subid, /* These values will work for a ZZ booted using BML */ const struct platform_ocapi generic_ocapi = { - .i2c_engine = 1, - .i2c_port = 4, - .i2c_offset = { 0x3, 0x1, 0x1 }, - .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, - .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, + .i2c_engine = 1, + .i2c_port = 4, + .i2c_reset_addr = OCAPI_I2C_RESET_ADDR, + .i2c_reset_odl0 = OCAPI_I2C_RESET_BOTTOM, + .i2c_reset_odl1 = OCAPI_I2C_RESET_TOP, .i2c_presence_addr = OCAPI_I2C_PRESENCE_ADDR, .i2c_presence_odl0 = OCAPI_I2C_PRESENCE_BOTTOM, .i2c_presence_odl1 = OCAPI_I2C_PRESENCE_TOP, @@ -187,7 +190,7 @@ const struct platform_ocapi generic_ocapi = { * force the presence until all our systems are upgraded */ .force_presence = true, - .odl_phy_swap = true, + .odl_phy_swap = true, }; static struct bmc_platform generic_bmc = { diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index b2bc0628..be0707ea 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -794,39 +794,70 @@ static void otl_enabletx(uint32_t gcid, uint32_t scom_base, uint64_t index) static void reset_ocapi_device(struct npu2_dev *dev) { - uint8_t data[3]; + uint8_t pin, data; int rc; - int i; switch (dev->index) { case 2: case 4: - memcpy(data, platform.ocapi->i2c_odl0_data, sizeof(data)); + pin = platform.ocapi->i2c_reset_odl0; break; case 3: case 5: - memcpy(data, platform.ocapi->i2c_odl1_data, sizeof(data)); + pin = platform.ocapi->i2c_reset_odl1; break; default: assert(false); } - for (i = 0; i < 3; i++) { - rc = i2c_request_send(dev->i2c_port_id_ocapi, 0x20, SMBUS_WRITE, - platform.ocapi->i2c_offset[i], 1, - &data[i], sizeof(data[i]), 120); - if (rc) { - /** - * @fwts-label OCAPIDeviceResetFailed - * @fwts-advice There was an error attempting to send - * a reset signal over I2C to the OpenCAPI device. - */ - prlog(PR_ERR, "OCAPI: Error writing I2C reset signal: %d\n", rc); - break; - } - if (i != 0) - time_wait_ms(5); - } + /* + * set the i2c reset pin in output mode + * + * On the 9554 device, register 3 is the configuration + * register and a pin is in output mode if its value is 0 + */ + data = ~pin; + rc = i2c_request_send(dev->i2c_port_id_ocapi, + platform.ocapi->i2c_reset_addr, SMBUS_WRITE, + 0x3, 1, + &data, sizeof(data), 120); + if (rc) + goto err; + + /* + * assert i2c reset for 5ms + * register 1 controls the signal, reset is active low + */ + data = ~pin; + rc = i2c_request_send(dev->i2c_port_id_ocapi, + platform.ocapi->i2c_reset_addr, SMBUS_WRITE, + 0x1, 1, + &data, sizeof(data), 120); + if (rc) + goto err; + time_wait_ms(5); + + /* + * deassert i2c reset and wait 5ms + */ + data = 0xFF; + rc = i2c_request_send(dev->i2c_port_id_ocapi, + platform.ocapi->i2c_reset_addr, SMBUS_WRITE, + 0x1, 1, + &data, sizeof(data), 120); + if (rc) + goto err; + time_wait_ms(5); + + return; + +err: + /** + * @fwts-label OCAPIDeviceResetFailed + * @fwts-advice There was an error attempting to send + * a reset signal over I2C to the OpenCAPI device. + */ + prlog(PR_ERR, "OCAPI: Error writing I2C reset signal: %d\n", rc); } static bool i2c_presence_detect(struct npu2_dev *dev) diff --git a/include/platform.h b/include/platform.h index 9a04ab37..d71fae78 100644 --- a/include/platform.h +++ b/include/platform.h @@ -48,9 +48,9 @@ struct bmc_platform { struct platform_ocapi { uint8_t i2c_engine; /* I2C engine number */ uint8_t i2c_port; /* I2C port number */ - uint32_t i2c_offset[3]; /* Offsets on I2C device */ - uint8_t i2c_odl0_data[3]; /* Data to reset ODL0 */ - uint8_t i2c_odl1_data[3]; /* Data to reset ODL1 */ + uint8_t i2c_reset_addr; /* I2C address for reset */ + uint8_t i2c_reset_odl0; /* I2C pin to write to reset ODL0 */ + uint8_t i2c_reset_odl1; /* I2C pin to write to reset ODL1 */ uint8_t i2c_presence_addr; /* I2C address for presence detection */ uint8_t i2c_presence_odl0; /* I2C pin to read to detect ODL0 */ uint8_t i2c_presence_odl1; /* I2C pin to read to detect ODL1 */ diff --git a/platforms/astbmc/zaius.c b/platforms/astbmc/zaius.c index 279fbf8b..170d828f 100644 --- a/platforms/astbmc/zaius.c +++ b/platforms/astbmc/zaius.c @@ -24,20 +24,23 @@ #include "astbmc.h" +#define OCAPI_I2C_RESET_ADDR 0x20 +#define OCAPI_I2C_RESET_BOTTOM (1 << 1) +#define OCAPI_I2C_RESET_TOP (1 << 6) #define OCAPI_I2C_PRESENCE_ADDR 0x20 #define OCAPI_I2C_PRESENCE_BOTTOM (1 << 2) #define OCAPI_I2C_PRESENCE_TOP (1 << 7) const struct platform_ocapi zaius_ocapi = { - .i2c_engine = 1, - .i2c_port = 4, - .i2c_offset = { 0x3, 0x1, 0x1 }, - .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, - .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, + .i2c_engine = 1, + .i2c_port = 4, + .i2c_reset_addr = OCAPI_I2C_RESET_ADDR, + .i2c_reset_odl0 = OCAPI_I2C_RESET_BOTTOM, + .i2c_reset_odl1 = OCAPI_I2C_RESET_TOP, .i2c_presence_addr = OCAPI_I2C_PRESENCE_ADDR, .i2c_presence_odl0 = OCAPI_I2C_PRESENCE_BOTTOM, .i2c_presence_odl1 = OCAPI_I2C_PRESENCE_TOP, - .odl_phy_swap = true, + .odl_phy_swap = true, }; #define NPU_BASE 0x5011000 diff --git a/platforms/ibm-fsp/zz.c b/platforms/ibm-fsp/zz.c index 66f29d44..d6920110 100644 --- a/platforms/ibm-fsp/zz.c +++ b/platforms/ibm-fsp/zz.c @@ -27,17 +27,20 @@ #include "ibm-fsp.h" #include "lxvpd.h" +#define OCAPI_I2C_RESET_ADDR 0x20 +#define OCAPI_I2C_RESET_BOTTOM (1 << 1) +#define OCAPI_I2C_RESET_TOP (1 << 6) #define OCAPI_I2C_PRESENCE_ADDR 0x20 #define OCAPI_I2C_PRESENCE_BOTTOM (1 << 2) #define OCAPI_I2C_PRESENCE_TOP (1 << 7) /* We don't yet create NPU device nodes on ZZ, but these values are correct */ const struct platform_ocapi zz_ocapi = { - .i2c_engine = 1, - .i2c_port = 4, - .i2c_offset = { 0x3, 0x1, 0x1 }, - .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, - .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, + .i2c_engine = 1, + .i2c_port = 4, + .i2c_reset_addr = OCAPI_I2C_RESET_ADDR, + .i2c_reset_odl0 = OCAPI_I2C_RESET_BOTTOM, + .i2c_reset_odl1 = OCAPI_I2C_RESET_TOP, .i2c_presence_addr = OCAPI_I2C_PRESENCE_ADDR, .i2c_presence_odl0 = OCAPI_I2C_PRESENCE_BOTTOM, .i2c_presence_odl1 = OCAPI_I2C_PRESENCE_TOP, @@ -46,7 +49,7 @@ const struct platform_ocapi zz_ocapi = { * force the presence until all our systems are upgraded */ .force_presence = true, - .odl_phy_swap = true, + .odl_phy_swap = true, }; static bool zz_probe(void) From patchwork Thu Apr 12 12:45:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 897642 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40MLJB3Ls5z9s2M for ; Thu, 12 Apr 2018 22:46:34 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40MLJB0yVczF1Rs for ; Thu, 12 Apr 2018 22:46:34 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40MLH62ZFtzDrq2 for ; Thu, 12 Apr 2018 22:45:38 +1000 (AEST) Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w3CCi9cO016686 for ; Thu, 12 Apr 2018 08:45:36 -0400 Received: from e06smtp11.uk.ibm.com (e06smtp11.uk.ibm.com [195.75.94.107]) by mx0a-001b2d01.pphosted.com with ESMTP id 2ha5ckxjf2-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 12 Apr 2018 08:45:35 -0400 Received: from localhost by e06smtp11.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 12 Apr 2018 13:45:33 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp11.uk.ibm.com (192.168.101.141) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 12 Apr 2018 13:45:32 +0100 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w3CCjUgl655784; Thu, 12 Apr 2018 12:45:30 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 27D9B52047; Thu, 12 Apr 2018 12:36:27 +0100 (BST) Received: from borneo.lab.toulouse-stg.fr.ibm.com (unknown [9.101.4.34]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id F253652041; Thu, 12 Apr 2018 12:36:26 +0100 (BST) From: Frederic Barrat To: andrew.donnellan@au1.ibm.com, skiboot@lists.ozlabs.org Date: Thu, 12 Apr 2018 14:45:24 +0200 X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180412124526.12662-1-fbarrat@linux.ibm.com> References: <20180412124526.12662-1-fbarrat@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18041212-0040-0000-0000-0000044CBF14 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18041212-0041-0000-0000-000020F0F2E3 Message-Id: <20180412124526.12662-4-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-04-12_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804120127 Subject: [Skiboot] [PATCH 3/5] npu2-opencapi: Train links on fundamental reset X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" From: Frederic Barrat Reorder our link training steps so that they are executed on fundamental reset instead of during the initial setup. Skiboot always call a fundamental reset on all the PHBs during pci init. It is done through a state machine, similarly to what is done for 'real' PHBs. This is the first step for a longer term goal to be able to trigger an adapter reset from linux. We'll need the reset callbacks of the PHB to be defined. We have to handle the various delays differently, since a linux thread shouldn't stay stuck waiting in opal for too long. No functional changes. Signed-off-by: Frederic Barrat Acked-by: Andrew Donnellan --- hw/npu2-opencapi.c | 360 ++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 247 insertions(+), 113 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index be0707ea..a7a19190 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -58,6 +58,21 @@ #define TL_MAX_TEMPLATE 63 #define TL_RATE_BUF_SIZE 32 +#define OCAPI_SLOT_NORMAL PCI_SLOT_STATE_NORMAL +#define OCAPI_SLOT_LINK PCI_SLOT_STATE_LINK +#define OCAPI_SLOT_LINK_START (OCAPI_SLOT_LINK + 1) +#define OCAPI_SLOT_LINK_WAIT (OCAPI_SLOT_LINK + 2) +#define OCAPI_SLOT_LINK_TRAINED (OCAPI_SLOT_LINK + 3) +#define OCAPI_SLOT_FRESET PCI_SLOT_STATE_FRESET +#define OCAPI_SLOT_FRESET_START (OCAPI_SLOT_FRESET + 1) +#define OCAPI_SLOT_FRESET_INIT (OCAPI_SLOT_FRESET + 2) +#define OCAPI_SLOT_FRESET_ASSERT_DELAY (OCAPI_SLOT_FRESET + 3) +#define OCAPI_SLOT_FRESET_DEASSERT_DELAY (OCAPI_SLOT_FRESET + 4) +#define OCAPI_SLOT_FRESET_INIT_DELAY (OCAPI_SLOT_FRESET + 5) + +#define OCAPI_LINK_TRAINING_RETRIES 5 +#define OCAPI_LINK_TRAINING_TIMEOUT 3000 /* ms */ + enum npu2_link_training_state { NPU2_TRAIN_DEFAULT, /* fully train the link */ NPU2_TRAIN_PRBS31, /* used for Signal Integrity testing */ @@ -792,7 +807,7 @@ static void otl_enabletx(uint32_t gcid, uint32_t scom_base, uint64_t index) /* TODO: Abort if credits are zero */ } -static void reset_ocapi_device(struct npu2_dev *dev) +static void assert_reset(struct npu2_dev *dev) { uint8_t pin, data; int rc; @@ -824,10 +839,7 @@ static void reset_ocapi_device(struct npu2_dev *dev) if (rc) goto err; - /* - * assert i2c reset for 5ms - * register 1 controls the signal, reset is active low - */ + /* register 1 controls the signal, reset is active low */ data = ~pin; rc = i2c_request_send(dev->i2c_port_id_ocapi, platform.ocapi->i2c_reset_addr, SMBUS_WRITE, @@ -835,20 +847,6 @@ static void reset_ocapi_device(struct npu2_dev *dev) &data, sizeof(data), 120); if (rc) goto err; - time_wait_ms(5); - - /* - * deassert i2c reset and wait 5ms - */ - data = 0xFF; - rc = i2c_request_send(dev->i2c_port_id_ocapi, - platform.ocapi->i2c_reset_addr, SMBUS_WRITE, - 0x1, 1, - &data, sizeof(data), 120); - if (rc) - goto err; - time_wait_ms(5); - return; err: @@ -860,6 +858,26 @@ err: prlog(PR_ERR, "OCAPI: Error writing I2C reset signal: %d\n", rc); } +static void deassert_reset(struct npu2_dev *dev) +{ + uint8_t data; + int rc; + + data = 0xFF; + rc = i2c_request_send(dev->i2c_port_id_ocapi, + platform.ocapi->i2c_reset_addr, SMBUS_WRITE, + 0x1, 1, + &data, sizeof(data), 120); + if (rc) { + /** + * @fwts-label OCAPIDeviceResetFailed + * @fwts-advice There was an error attempting to send + * a reset signal over I2C to the OpenCAPI device. + */ + prlog(PR_ERR, "OCAPI: Error writing I2C reset signal: %d\n", rc); + } +} + static bool i2c_presence_detect(struct npu2_dev *dev) { uint8_t state, data; @@ -898,13 +916,11 @@ static bool i2c_presence_detect(struct npu2_dev *dev) return !(state & data); } -static int odl_train(uint32_t gcid, uint32_t index, struct npu2_dev *dev) +static void reset_odl(uint32_t gcid, struct npu2_dev *dev) { uint64_t reg, config_xscom; - int timeout = 3000; - prlog(PR_DEBUG, "OCAPI: %s: Training ODL\n", __func__); - switch (index) { + switch (dev->index) { case 2: config_xscom = OB0_ODL0_CONFIG; break; @@ -934,35 +950,60 @@ static int odl_train(uint32_t gcid, uint32_t index, struct npu2_dev *dev) reg &= ~OB_ODL_CONFIG_RESET; xscom_write(gcid, config_xscom, reg); +} - reset_ocapi_device(dev); +static void set_init_pattern(uint32_t gcid, struct npu2_dev *dev) +{ + uint64_t reg, config_xscom; + + switch (dev->index) { + case 2: + config_xscom = OB0_ODL0_CONFIG; + break; + case 3: + config_xscom = OB0_ODL1_CONFIG; + break; + case 4: + config_xscom = OB3_ODL1_CONFIG; + break; + case 5: + config_xscom = OB3_ODL0_CONFIG; + break; + default: + assert(false); + } /* Transmit Pattern A */ + xscom_read(gcid, config_xscom, ®); reg = SETFIELD(OB_ODL_CONFIG_TRAIN_MODE, reg, 0b0001); xscom_write(gcid, config_xscom, reg); - time_wait_ms(5); +} - /* Bump lanes - this improves training reliability */ - npu2_opencapi_bump_ui_lane(dev); +static void start_training(uint32_t gcid, struct npu2_dev *dev) +{ + uint64_t reg, config_xscom; + + switch (dev->index) { + case 2: + config_xscom = OB0_ODL0_CONFIG; + break; + case 3: + config_xscom = OB0_ODL1_CONFIG; + break; + case 4: + config_xscom = OB3_ODL1_CONFIG; + break; + case 5: + config_xscom = OB3_ODL0_CONFIG; + break; + default: + assert(false); + } /* Start training */ + xscom_read(gcid, config_xscom, ®); reg = SETFIELD(OB_ODL_CONFIG_TRAIN_MODE, reg, 0b1000); xscom_write(gcid, config_xscom, reg); - - do { - reg = get_odl_status(gcid, index); - if (GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg) == 0x7) { - prlog(PR_NOTICE, - "OCAPI: Link %d on chip %u trained in %dms\n", - index, gcid, 3000 - timeout); - return OPAL_SUCCESS; - } - time_wait_ms(1); - } while (timeout--); - prlog(PR_INFO, "OCAPI: Link %d on chip %u failed to train, retrying\n", - index, gcid); - prlog(PR_INFO, "OCAPI: Link status: %016llx\n", reg); - return OPAL_HARDWARE; } static int64_t npu2_opencapi_get_presence_state(struct pci_slot *slot, @@ -1000,6 +1041,150 @@ static int64_t npu2_opencapi_get_link_state(struct pci_slot *slot, uint8_t *val) return rc; } +static int64_t npu2_opencapi_retry_state(struct pci_slot *slot) +{ + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + uint32_t chip_id = dev->npu->chip_id; + + if (!slot->link_retries--) { + /** + * @fwts-label OCAPILinkTrainingFailed + * @fwts-advice The OpenCAPI link training procedure failed. + * This indicates a hardware or firmware bug. OpenCAPI + * functionality will not be available on this link. + */ + prlog(PR_ERR, + "OCAPI: Link %d on chip %u failed to train\n", + dev->index, chip_id); + prlog(PR_ERR, "OCAPI: Final link status: %016llx\n", + get_odl_status(chip_id, dev->index)); + return OPAL_HARDWARE; + } + + prlog(PR_ERR, + "OCAPI: Link %d on chip %u failed to train, retrying\n", + dev->index, chip_id); + pci_slot_set_state(slot, OCAPI_SLOT_FRESET_INIT); + return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); +} + +static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) +{ + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + uint32_t chip_id = dev->npu->chip_id; + uint64_t reg; + + switch (slot->state) { + case OCAPI_SLOT_NORMAL: + case OCAPI_SLOT_LINK_START: + prlog(PR_DEBUG, "OCAPI: LINK: Start polling\n"); + pci_slot_set_state(slot, OCAPI_SLOT_LINK_WAIT); + /* fall-through */ + case OCAPI_SLOT_LINK_WAIT: + reg = get_odl_status(chip_id, dev->index); + if (GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg) == 0x7) { + prlog(PR_NOTICE, + "OCAPI: Link %d on chip %u trained in %lld ms\n", + dev->index, chip_id, + OCAPI_LINK_TRAINING_TIMEOUT - slot->retries); + pci_slot_set_state(slot, OCAPI_SLOT_LINK_TRAINED); + return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); + } + if (slot->retries-- == 0) + return npu2_opencapi_retry_state(slot); + + return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); + + case OCAPI_SLOT_LINK_TRAINED: + otl_enabletx(chip_id, dev->npu->xscom_base, dev->index); + pci_slot_set_state(slot, OCAPI_SLOT_NORMAL); + return OPAL_SUCCESS; + + default: + prlog(PR_ERR, "OCAPI: LINK: unexpected slot state %08x\n", + slot->state); + + } + pci_slot_set_state(slot, OCAPI_SLOT_NORMAL); + return OPAL_HARDWARE; +} + +static int64_t npu2_opencapi_creset(struct pci_slot *slot __unused) +{ + prlog(PR_ERR, "OCAPI: creset not supported\n"); + return OPAL_UNSUPPORTED; +} + +static int64_t npu2_opencapi_freset(struct pci_slot *slot) +{ + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + uint32_t chip_id = dev->npu->chip_id; + uint8_t presence = 1; + + switch (slot->state) { + case OCAPI_SLOT_NORMAL: + case OCAPI_SLOT_FRESET_START: + prlog(PR_DEBUG, "OCAPI: FRESET: Starts\n"); + + if (slot->ops.get_presence_state) + slot->ops.get_presence_state(slot, &presence); + if (!presence) { + /* + * FIXME: if there's no card on the link, we + * should consider powering off the unused + * lanes to save energy + */ + prlog(PR_INFO, + "OCAPI: no card detected on link %d, chip %d\n", + dev->index, chip_id); + return OPAL_SUCCESS; + } + slot->link_retries = OCAPI_LINK_TRAINING_RETRIES; + npu2_opencapi_phy_setup(dev); + /* fall-through */ + case OCAPI_SLOT_FRESET_INIT: + reset_odl(chip_id, dev); + assert_reset(dev); + pci_slot_set_state(slot, + OCAPI_SLOT_FRESET_ASSERT_DELAY); + /* assert for 5ms */ + return pci_slot_set_sm_timeout(slot, msecs_to_tb(5)); + + case OCAPI_SLOT_FRESET_ASSERT_DELAY: + deassert_reset(dev); + pci_slot_set_state(slot, + OCAPI_SLOT_FRESET_DEASSERT_DELAY); + /* give another 5ms to device to be ready */ + return pci_slot_set_sm_timeout(slot, msecs_to_tb(5)); + + case OCAPI_SLOT_FRESET_DEASSERT_DELAY: + set_init_pattern(chip_id, dev); + pci_slot_set_state(slot, + OCAPI_SLOT_FRESET_INIT_DELAY); + return pci_slot_set_sm_timeout(slot, msecs_to_tb(5)); + + case OCAPI_SLOT_FRESET_INIT_DELAY: + /* Bump lanes - this improves training reliability */ + npu2_opencapi_bump_ui_lane(dev); + start_training(chip_id, dev); + slot->retries = OCAPI_LINK_TRAINING_TIMEOUT; + pci_slot_set_state(slot, OCAPI_SLOT_LINK_START); + return slot->ops.poll_link(slot); + + default: + prlog(PR_ERR, "OCAPI: FRESET: unexpected slot state %08x\n", + slot->state); + } + pci_slot_set_state(slot, OCAPI_SLOT_NORMAL); + return OPAL_HARDWARE; +} + +static int64_t npu2_opencapi_hreset(struct pci_slot *slot __unused) +{ + prlog(PR_ERR, "OCAPI: hreset not supported\n"); + return OPAL_UNSUPPORTED; +} + static struct pci_slot *npu2_opencapi_slot_create(struct phb *phb) { struct pci_slot *slot; @@ -1016,15 +1201,11 @@ static struct pci_slot *npu2_opencapi_slot_create(struct phb *phb) slot->ops.get_latch_state = NULL; slot->ops.set_power_state = NULL; slot->ops.set_attention_state = NULL; - /* - * Temporarily erase the run_sm callback until we support - * dynamic reset of the link. Otherwise, run_sm may call - * freset, creset, ... and we don't define them. The run_sm - * pointer is always tested before being called, at least at - * the time of this writing :-) It will go away when we - * implement dynamic reset of the link - */ - slot->ops.run_sm = NULL; + + slot->ops.poll_link = npu2_opencapi_poll_link; + slot->ops.creset = npu2_opencapi_creset; + slot->ops.freset = npu2_opencapi_freset; + slot->ops.hreset = npu2_opencapi_hreset; return slot; } @@ -1345,48 +1526,22 @@ static int setup_irq(struct npu2 *p) return 0; } -#define LINK_TRAINING_RETRIES 5 - -static int train_link(int chip_id, struct npu2_dev *dev) +static void setup_debug_training_state(struct npu2_dev *dev) { - bool train; - int rc; - int retries = LINK_TRAINING_RETRIES; - - if (platform.ocapi->force_presence) - train = true; - else - train = i2c_presence_detect(dev); - if (!train) { - /* - * FIXME: if there's no card on the link, we should consider - * powering off the unused lanes to save energy - */ - prlog(PR_INFO, "OCAPI: no card detected on link %d, chip %d\n", - dev->index, chip_id); - return -1; - } + npu2_opencapi_phy_setup(dev); - do { - rc = odl_train(chip_id, dev->index, dev); - } while (rc != OPAL_SUCCESS && --retries); + switch (npu2_ocapi_training_state) { + case NPU2_TRAIN_PRBS31: + prlog(PR_INFO, "OCAPI: sending PRBS31 pattern per NVRAM setting\n"); + npu2_opencapi_phy_prbs31(dev); + break; - if (rc != OPAL_SUCCESS && retries == 0) { - /** - * @fwts-label OCAPILinkTrainingFailed - * @fwts-advice The OpenCAPI link training procedure failed. - * This indicates a hardware or firmware bug. OpenCAPI - * functionality will not be available on this link. - */ - prlog(PR_ERR, - "OCAPI: Link %d on chip %u failed to train\n", - dev->index, chip_id); - prlog(PR_ERR, "OCAPI: Final link status: %016llx\n", - get_odl_status(chip_id, dev->index)); - return -1; + case NPU2_TRAIN_NONE: + prlog(PR_INFO, "OCAPI: link not trained per NVRAM setting\n"); + break; + default: + assert(false); } - - return 0; } static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, @@ -1397,7 +1552,6 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, struct pci_slot *slot; char port_name[17]; uint64_t mm_win[2]; - int rc; dev_index = dt_prop_get_u32(dn_link, "ibm,npu-link-index"); npu_index = dt_prop_get_u32(n->dt_node, "ibm,npu-index"); @@ -1480,22 +1634,9 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, set_fence_control(n->chip_id, n->xscom_base, dev->index, 0b00); - npu2_opencapi_phy_setup(dev); - - switch (npu2_ocapi_training_state) { - case NPU2_TRAIN_PRBS31: - prlog(PR_INFO, - "OCAPI: Link %d sending PRBS31 pattern per NVRAM setting\n", - dev->index); - npu2_opencapi_phy_prbs31(dev); - break; - - case NPU2_TRAIN_DEFAULT: - rc = train_link(n->chip_id, dev); - if (rc) - goto failed; - - otl_enabletx(n->chip_id, n->xscom_base, dev->index); + if (npu2_ocapi_training_state != NPU2_TRAIN_DEFAULT) { + setup_debug_training_state(dev); + } else { slot = npu2_opencapi_slot_create(&dev->phb_ocapi); if (!slot) { /** @@ -1505,14 +1646,7 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, */ prlog(PR_ERR, "OCAPI: Cannot create PHB slot\n"); } - break; - - case NPU2_TRAIN_NONE: - prlog(PR_INFO, "OCAPI: Link %d not trained per NVRAM setting\n", - dev->index); - break; } - pci_register_phb(&dev->phb_ocapi, OPAL_DYNAMIC_PHB_ID); return; failed: From patchwork Thu Apr 12 12:45:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 897643 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40MLJc17Y7z9s2M for ; Thu, 12 Apr 2018 22:46:56 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40MLJb6qKqzDqhm for ; Thu, 12 Apr 2018 22:46:55 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40MLH81RbfzF1Rc for ; Thu, 12 Apr 2018 22:45:39 +1000 (AEST) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w3CCdRXU016105 for ; Thu, 12 Apr 2018 08:45:37 -0400 Received: from e06smtp10.uk.ibm.com (e06smtp10.uk.ibm.com [195.75.94.106]) by mx0b-001b2d01.pphosted.com with ESMTP id 2ha5x8w91s-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 12 Apr 2018 08:45:36 -0400 Received: from localhost by e06smtp10.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 12 Apr 2018 13:45:35 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp10.uk.ibm.com (192.168.101.140) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 12 Apr 2018 13:45:33 +0100 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w3CCjWFr59834592; Thu, 12 Apr 2018 12:45:32 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 56BF352049; Thu, 12 Apr 2018 12:36:28 +0100 (BST) Received: from borneo.lab.toulouse-stg.fr.ibm.com (unknown [9.101.4.34]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id 2D71452045; Thu, 12 Apr 2018 12:36:28 +0100 (BST) From: Frederic Barrat To: andrew.donnellan@au1.ibm.com, skiboot@lists.ozlabs.org Date: Thu, 12 Apr 2018 14:45:25 +0200 X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180412124526.12662-1-fbarrat@linux.ibm.com> References: <20180412124526.12662-1-fbarrat@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18041212-0040-0000-0000-0000042D845F X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18041212-0041-0000-0000-000026319D3B Message-Id: <20180412124526.12662-5-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-04-12_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804120127 Subject: [Skiboot] [PATCH 4/5] npu2-opencapi: Cleanup traces printed during link training X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" From: Frederic Barrat Now that links may train in parallel, traces shown during training can be all mixed up. So add a prefix to all the traces to clearly identify the chip and link the trace refers to: OCAPI[:]: this is a very useful message The lower-level hardware procedures (npu2-hw-procedures.c) also print traces which would need work. But that code is being reworked to be better integrated with opencapi and nvidia, so leave it alone for now. Signed-off-by: Frederic Barrat Acked-by: Andrew Donnellan --- hw/npu2-opencapi.c | 76 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 39 insertions(+), 37 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index a7a19190..24adf5e1 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -52,6 +52,14 @@ #include #include +#define OCAPIDBG(dev, fmt, a...) prlog(PR_DEBUG, "OCAPI[%d:%d]: " fmt, \ + dev->npu->chip_id, dev->index, ## a) +#define OCAPIINF(dev, fmt, a...) prlog(PR_INFO, "OCAPI[%d:%d]: " fmt, \ + dev->npu->chip_id, dev->index, ## a) +#define OCAPIERR(dev, fmt, a...) prlog(PR_ERR, "OCAPI[%d:%d]: " fmt, \ + dev->npu->chip_id, dev->index, ## a) + + #define NPU_IRQ_LEVELS 35 #define NPU_IRQ_LEVELS_XSL 23 #define MAX_PE_HANDLE ((1 << 15) - 1) @@ -787,15 +795,16 @@ static void setup_afu_config_bars(uint32_t gcid, uint32_t scom_base, dev->bars[1].npu2_bar.size = size; } -static void otl_enabletx(uint32_t gcid, uint32_t scom_base, uint64_t index) +static void otl_enabletx(uint32_t gcid, uint32_t scom_base, + struct npu2_dev *dev) { - uint64_t stack = index_to_stack(index); - uint64_t block = index_to_block(index); + uint64_t stack = index_to_stack(dev->index); + uint64_t block = index_to_block(dev->index); uint64_t reg; /* OTL Config 2 Register */ /* Transmit Enable */ - prlog(PR_DEBUG, "OCAPI: %s: Enabling TX\n", __func__); + OCAPIDBG(dev, "Enabling TX\n"); reg = 0; reg |= NPU2_OTL_CONFIG2_TX_SEND_EN; npu2_scom_write(gcid, scom_base, NPU2_OTL_CONFIG2(stack, block), @@ -803,7 +812,7 @@ static void otl_enabletx(uint32_t gcid, uint32_t scom_base, uint64_t index) reg = npu2_scom_read(gcid, scom_base, NPU2_OTL_VC_CREDITS(stack, block), NPU2_MISC_DA_LEN_8B); - prlog(PR_DEBUG, "OCAPI: credit counter: %llx\n", reg); + OCAPIDBG(dev, "credit counter: %llx\n", reg); /* TODO: Abort if credits are zero */ } @@ -855,7 +864,7 @@ err: * @fwts-advice There was an error attempting to send * a reset signal over I2C to the OpenCAPI device. */ - prlog(PR_ERR, "OCAPI: Error writing I2C reset signal: %d\n", rc); + OCAPIERR(dev, "Error writing I2C reset signal: %d\n", rc); } static void deassert_reset(struct npu2_dev *dev) @@ -874,7 +883,7 @@ static void deassert_reset(struct npu2_dev *dev) * @fwts-advice There was an error attempting to send * a reset signal over I2C to the OpenCAPI device. */ - prlog(PR_ERR, "OCAPI: Error writing I2C reset signal: %d\n", rc); + OCAPIERR(dev, "Error writing I2C reset signal: %d\n", rc); } } @@ -894,12 +903,11 @@ static bool i2c_presence_detect(struct npu2_dev *dev) SMBUS_READ, 0, 1, &state, 1, 120); if (rc) { - prlog(PR_ERR, "OCAPI: error detecting link presence: %d\n", - rc); + OCAPIERR(dev, "error detecting link presence: %d\n", rc); return true; /* assume link exists */ } - prlog(PR_DEBUG, "OCAPI: I2C presence detect: 0x%x\n", state); + OCAPIDBG(dev, "I2C presence detect: 0x%x\n", state); switch (dev->index) { case 2: @@ -909,7 +917,7 @@ static bool i2c_presence_detect(struct npu2_dev *dev) data = platform.ocapi->i2c_presence_odl1; break; default: - prlog(PR_ERR, "OCAPI: presence detection on invalid link\n"); + OCAPIERR(dev, "presence detection on invalid link\n"); return true; } /* Presence detect bits are active low */ @@ -1053,17 +1061,13 @@ static int64_t npu2_opencapi_retry_state(struct pci_slot *slot) * This indicates a hardware or firmware bug. OpenCAPI * functionality will not be available on this link. */ - prlog(PR_ERR, - "OCAPI: Link %d on chip %u failed to train\n", - dev->index, chip_id); - prlog(PR_ERR, "OCAPI: Final link status: %016llx\n", + OCAPIERR(dev, + "Link failed to train, final link status: %016llx\n", get_odl_status(chip_id, dev->index)); return OPAL_HARDWARE; } - prlog(PR_ERR, - "OCAPI: Link %d on chip %u failed to train, retrying\n", - dev->index, chip_id); + OCAPIERR(dev, "Link failed to train, retrying\n"); pci_slot_set_state(slot, OCAPI_SLOT_FRESET_INIT); return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); } @@ -1077,15 +1081,13 @@ static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) switch (slot->state) { case OCAPI_SLOT_NORMAL: case OCAPI_SLOT_LINK_START: - prlog(PR_DEBUG, "OCAPI: LINK: Start polling\n"); + OCAPIDBG(dev, "Start polling\n"); pci_slot_set_state(slot, OCAPI_SLOT_LINK_WAIT); /* fall-through */ case OCAPI_SLOT_LINK_WAIT: reg = get_odl_status(chip_id, dev->index); if (GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg) == 0x7) { - prlog(PR_NOTICE, - "OCAPI: Link %d on chip %u trained in %lld ms\n", - dev->index, chip_id, + OCAPIINF(dev, "link trained in %lld ms\n", OCAPI_LINK_TRAINING_TIMEOUT - slot->retries); pci_slot_set_state(slot, OCAPI_SLOT_LINK_TRAINED); return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); @@ -1096,13 +1098,12 @@ static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); case OCAPI_SLOT_LINK_TRAINED: - otl_enabletx(chip_id, dev->npu->xscom_base, dev->index); + otl_enabletx(chip_id, dev->npu->xscom_base, dev); pci_slot_set_state(slot, OCAPI_SLOT_NORMAL); return OPAL_SUCCESS; default: - prlog(PR_ERR, "OCAPI: LINK: unexpected slot state %08x\n", - slot->state); + OCAPIERR(dev, "unexpected slot state %08x\n", slot->state); } pci_slot_set_state(slot, OCAPI_SLOT_NORMAL); @@ -1111,7 +1112,9 @@ static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) static int64_t npu2_opencapi_creset(struct pci_slot *slot __unused) { - prlog(PR_ERR, "OCAPI: creset not supported\n"); + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + + OCAPIERR(dev, "creset not supported\n"); return OPAL_UNSUPPORTED; } @@ -1124,7 +1127,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) switch (slot->state) { case OCAPI_SLOT_NORMAL: case OCAPI_SLOT_FRESET_START: - prlog(PR_DEBUG, "OCAPI: FRESET: Starts\n"); + OCAPIDBG(dev, "FRESET starts\n"); if (slot->ops.get_presence_state) slot->ops.get_presence_state(slot, &presence); @@ -1134,9 +1137,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) * should consider powering off the unused * lanes to save energy */ - prlog(PR_INFO, - "OCAPI: no card detected on link %d, chip %d\n", - dev->index, chip_id); + OCAPIINF(dev, "no card detected\n"); return OPAL_SUCCESS; } slot->link_retries = OCAPI_LINK_TRAINING_RETRIES; @@ -1172,7 +1173,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) return slot->ops.poll_link(slot); default: - prlog(PR_ERR, "OCAPI: FRESET: unexpected slot state %08x\n", + OCAPIERR(dev, "FRESET: unexpected slot state %08x\n", slot->state); } pci_slot_set_state(slot, OCAPI_SLOT_NORMAL); @@ -1181,7 +1182,9 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) static int64_t npu2_opencapi_hreset(struct pci_slot *slot __unused) { - prlog(PR_ERR, "OCAPI: hreset not supported\n"); + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + + OCAPIERR(dev, "hreset not supported\n"); return OPAL_UNSUPPORTED; } @@ -1532,12 +1535,12 @@ static void setup_debug_training_state(struct npu2_dev *dev) switch (npu2_ocapi_training_state) { case NPU2_TRAIN_PRBS31: - prlog(PR_INFO, "OCAPI: sending PRBS31 pattern per NVRAM setting\n"); + OCAPIINF(dev, "sending PRBS31 pattern per NVRAM setting\n"); npu2_opencapi_phy_prbs31(dev); break; case NPU2_TRAIN_NONE: - prlog(PR_INFO, "OCAPI: link not trained per NVRAM setting\n"); + OCAPIINF(dev, "link not trained per NVRAM setting\n"); break; default: assert(false); @@ -1942,7 +1945,7 @@ static bool is_template_supported(unsigned int templ, long capabilities) return !!(capabilities & (1ull << templ)); } -static int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t bdfn, +static int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t __unused bdfn, long capabilities, uint64_t rate_phys, int rate_sz) { struct phb *phb = pci_get_phb(phb_id); @@ -1995,8 +1998,7 @@ static int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t bdfn, npu2_scom_write(dev->npu->chip_id, dev->npu->xscom_base, NPU2_OTL_CONFIG1(stack, block), NPU2_MISC_DA_LEN_8B, reg); - prlog(PR_DEBUG, "OCAPI: Link %llx:%x, TL conf1 register set to %llx\n", - phb_id, bdfn, reg); + OCAPIDBG(dev, "OTL configuration 1 register set to %llx\n", reg); return OPAL_SUCCESS; } opal_call(OPAL_NPU_TL_SET, opal_npu_tl_set, 5); From patchwork Thu Apr 12 12:45:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 897644 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40MLJt0jG3z9s2M for ; Thu, 12 Apr 2018 22:47:10 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40MLJs6NjhzF1hW for ; Thu, 12 Apr 2018 22:47:09 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40MLH91W1VzF1Rc for ; Thu, 12 Apr 2018 22:45:41 +1000 (AEST) Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w3CCfXdG008432 for ; Thu, 12 Apr 2018 08:45:38 -0400 Received: from e06smtp13.uk.ibm.com (e06smtp13.uk.ibm.com [195.75.94.109]) by mx0a-001b2d01.pphosted.com with ESMTP id 2ha4f4h182-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Thu, 12 Apr 2018 08:45:38 -0400 Received: from localhost by e06smtp13.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 12 Apr 2018 13:45:36 +0100 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp13.uk.ibm.com (192.168.101.143) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 12 Apr 2018 13:45:34 +0100 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w3CCjXeJ7209208; Thu, 12 Apr 2018 12:45:33 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 870FB52041; Thu, 12 Apr 2018 12:36:29 +0100 (BST) Received: from borneo.lab.toulouse-stg.fr.ibm.com (unknown [9.101.4.34]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id 5CDD25204E; Thu, 12 Apr 2018 12:36:29 +0100 (BST) From: Frederic Barrat To: andrew.donnellan@au1.ibm.com, skiboot@lists.ozlabs.org Date: Thu, 12 Apr 2018 14:45:26 +0200 X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180412124526.12662-1-fbarrat@linux.ibm.com> References: <20180412124526.12662-1-fbarrat@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18041212-0012-0000-0000-000005CA0D06 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18041212-0013-0000-0000-00001946432F Message-Id: <20180412124526.12662-6-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-04-12_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804120127 Subject: [Skiboot] [PATCH 5/5] npu2-opencapi: Fix link state to report link down X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" From: Frederic Barrat The PHB callback 'get_link_state' is always reporting the link width, irrespective of the link status and even when the link is down. It is causing too much work (and failures) when the PHB is probed during pci init. The fix is to look at the link status first and report the link as down when appropriate. Signed-off-by: Frederic Barrat Acked-by: Andrew Donnellan --- hw/npu2-opencapi.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 24adf5e1..a0fdf22b 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -80,6 +80,7 @@ #define OCAPI_LINK_TRAINING_RETRIES 5 #define OCAPI_LINK_TRAINING_TIMEOUT 3000 /* ms */ +#define OCAPI_LINK_STATE_TRAINED 0x7 enum npu2_link_training_state { NPU2_TRAIN_DEFAULT, /* fully train the link */ @@ -1032,10 +1033,17 @@ static int64_t npu2_opencapi_get_link_state(struct pci_slot *slot, uint8_t *val) { struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); uint64_t reg; - int64_t link_width, rc = OPAL_SUCCESS; + int64_t link_width, training_status, rc = OPAL_SUCCESS; reg = get_odl_status(dev->npu->chip_id, dev->index); link_width = GETFIELD(OB_ODL_STATUS_TRAINED_MODE, reg); + training_status = GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg); + + if (training_status != OCAPI_LINK_STATE_TRAINED) { + *val = OPAL_SHPC_LINK_DOWN; + return OPAL_SUCCESS; + } + switch (link_width) { case 0b0001: *val = OPAL_SHPC_LINK_UP_x4; @@ -1086,7 +1094,8 @@ static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) /* fall-through */ case OCAPI_SLOT_LINK_WAIT: reg = get_odl_status(chip_id, dev->index); - if (GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg) == 0x7) { + if (GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg) == + OCAPI_LINK_STATE_TRAINED) { OCAPIINF(dev, "link trained in %lld ms\n", OCAPI_LINK_TRAINING_TIMEOUT - slot->retries); pci_slot_set_state(slot, OCAPI_SLOT_LINK_TRAINED);