From patchwork Wed May 23 17:25:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 919252 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40rgHg3Kwyz9s0y for ; Thu, 24 May 2018 03:58:55 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40rgHg1xNVzF189 for ; Thu, 24 May 2018 03:58:55 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40rfYb2LF5zF12G for ; Thu, 24 May 2018 03:25:55 +1000 (AEST) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4NHNwxR124536 for ; Wed, 23 May 2018 13:25:53 -0400 Received: from e06smtp14.uk.ibm.com (e06smtp14.uk.ibm.com [195.75.94.110]) by mx0a-001b2d01.pphosted.com with ESMTP id 2j5bnqaubb-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 23 May 2018 13:25:52 -0400 Received: from localhost by e06smtp14.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 23 May 2018 18:25:51 +0100 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp14.uk.ibm.com (192.168.101.144) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 23 May 2018 18:25:49 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w4NHPlwK15991062; Wed, 23 May 2018 17:25:48 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 513A911C054; Wed, 23 May 2018 18:16:53 +0100 (BST) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DCB5B11C050; Wed, 23 May 2018 18:16:52 +0100 (BST) Received: from localhost.localdomain (unknown [9.164.160.79]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 23 May 2018 18:16:52 +0100 (BST) From: Frederic Barrat To: andrew.donnellan@au1.ibm.com, skiboot@lists.ozlabs.org Date: Wed, 23 May 2018 19:25:41 +0200 X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180523172545.16016-1-fbarrat@linux.ibm.com> References: <20180523172545.16016-1-fbarrat@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18052317-0044-0000-0000-00000555BF85 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18052317-0045-0000-0000-00002897543A Message-Id: <20180523172545.16016-2-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-23_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1805230169 Subject: [Skiboot] [PATCH v2 1/5] npu2-opencapi: Use presence detection X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" From: Frederic Barrat Presence detection is not part of the opencapi specification. So each platform may choose to implement it the way it wants. All current platforms implement it through an i2c device where we can query a pin to know if a device is connected or not. ZZ and Zaius have a similar design and even use the same i2c information and pin numbers. However, presence detection on older ZZ planar (older than v4) doesn't work, so we don't activate it for now, until our lab systems are upgraded and it's better tested. Presence detection on witherspoon is still being worked on. It's shaping up to be quite different, so we may have to revisit the topic in a later patch. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- v2: define platform_ocapi with inline literals, to match existing style rework a few comments core/platform.c | 11 ++++- hw/npu2-opencapi.c | 121 ++++++++++++++++++++++++++++++++++++++--------- include/platform.h | 4 ++ platforms/astbmc/zaius.c | 6 ++- platforms/ibm-fsp/zz.c | 10 +++- 5 files changed, 127 insertions(+), 25 deletions(-) diff --git a/core/platform.c b/core/platform.c index f09ea3c1..92b118ba 100644 --- a/core/platform.c +++ b/core/platform.c @@ -175,7 +175,16 @@ const struct platform_ocapi generic_ocapi = { .i2c_offset = { 0x3, 0x1, 0x1 }, .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, - .odl_phy_swap = true, + .i2c_presence_addr = 0x20, + .i2c_presence_odl0 = (1 << 2), /* bottom connector */ + .i2c_presence_odl1 = (1 << 7), /* top connector */ + /* + * The ZZs we typically use for BML/generic platform tend to + * have old planars and presence detection is broken there, so + * force presence. + */ + .force_presence = true, + .odl_phy_swap = true, }; static struct bmc_platform generic_bmc = { diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 2cc776c7..b2bc0628 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -829,6 +829,44 @@ static void reset_ocapi_device(struct npu2_dev *dev) } } +static bool i2c_presence_detect(struct npu2_dev *dev) +{ + uint8_t state, data; + int rc; + + /* + * Opencapi presence detection is done through i2c + * + * Lagrange platforms (ZZ, Zaius) use the same default mechanism. + * Witherspoon will need a specific implementation, TBD. + */ + rc = i2c_request_send(dev->i2c_port_id_ocapi, + platform.ocapi->i2c_presence_addr, + SMBUS_READ, 0, 1, + &state, 1, 120); + if (rc) { + prlog(PR_ERR, "OCAPI: error detecting link presence: %d\n", + rc); + return true; /* assume link exists */ + } + + prlog(PR_DEBUG, "OCAPI: I2C presence detect: 0x%x\n", state); + + switch (dev->index) { + case 2: + data = platform.ocapi->i2c_presence_odl0; + break; + case 3: + data = platform.ocapi->i2c_presence_odl1; + break; + default: + prlog(PR_ERR, "OCAPI: presence detection on invalid link\n"); + return true; + } + /* Presence detect bits are active low */ + return !(state & data); +} + static int odl_train(uint32_t gcid, uint32_t index, struct npu2_dev *dev) { uint64_t reg, config_xscom; @@ -896,6 +934,20 @@ static int odl_train(uint32_t gcid, uint32_t index, struct npu2_dev *dev) return OPAL_HARDWARE; } +static int64_t npu2_opencapi_get_presence_state(struct pci_slot *slot, + uint8_t *val) +{ + bool present; + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + + if (platform.ocapi->force_presence) + present = true; + else + present = i2c_presence_detect(dev); + *val = present; + return OPAL_SUCCESS; +} + static int64_t npu2_opencapi_get_link_state(struct pci_slot *slot, uint8_t *val) { struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); @@ -926,9 +978,9 @@ static struct pci_slot *npu2_opencapi_slot_create(struct phb *phb) return slot; /* TODO: Figure out other slot functions */ - slot->ops.get_presence_state = NULL; - slot->ops.get_link_state = npu2_opencapi_get_link_state; - slot->ops.get_power_state = NULL; + slot->ops.get_presence_state = npu2_opencapi_get_presence_state; + slot->ops.get_link_state = npu2_opencapi_get_link_state; + slot->ops.get_power_state = NULL; slot->ops.get_attention_state = NULL; slot->ops.get_latch_state = NULL; slot->ops.set_power_state = NULL; @@ -1264,6 +1316,48 @@ static int setup_irq(struct npu2 *p) #define LINK_TRAINING_RETRIES 5 +static int train_link(int chip_id, struct npu2_dev *dev) +{ + bool train; + int rc; + int retries = LINK_TRAINING_RETRIES; + + if (platform.ocapi->force_presence) + train = true; + else + train = i2c_presence_detect(dev); + if (!train) { + /* + * FIXME: if there's no card on the link, we should consider + * powering off the unused lanes to save energy + */ + prlog(PR_INFO, "OCAPI: no card detected on link %d, chip %d\n", + dev->index, chip_id); + return -1; + } + + do { + rc = odl_train(chip_id, dev->index, dev); + } while (rc != OPAL_SUCCESS && --retries); + + if (rc != OPAL_SUCCESS && retries == 0) { + /** + * @fwts-label OCAPILinkTrainingFailed + * @fwts-advice The OpenCAPI link training procedure failed. + * This indicates a hardware or firmware bug. OpenCAPI + * functionality will not be available on this link. + */ + prlog(PR_ERR, + "OCAPI: Link %d on chip %u failed to train\n", + dev->index, chip_id); + prlog(PR_ERR, "OCAPI: Final link status: %016llx\n", + get_odl_status(chip_id, dev->index)); + return -1; + } + + return 0; +} + static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, struct npu2_dev *dev) { @@ -1272,7 +1366,6 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, struct pci_slot *slot; char port_name[17]; uint64_t mm_win[2]; - int retries = LINK_TRAINING_RETRIES; int rc; dev_index = dt_prop_get_u32(dn_link, "ibm,npu-link-index"); @@ -1367,27 +1460,11 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, break; case NPU2_TRAIN_DEFAULT: - do { - rc = odl_train(n->chip_id, dev->index, dev); - } while (rc != OPAL_SUCCESS && --retries); - - if (rc != OPAL_SUCCESS && retries == 0) { - /** - * @fwts-label OCAPILinkTrainingFailed - * @fwts-advice The OpenCAPI link training procedure failed. - * This indicates a hardware or firmware bug. OpenCAPI - * functionality will not be available on this link. - */ - prlog(PR_ERR, - "OCAPI: Link %d on chip %u failed to train\n", - dev->index, n->chip_id); - prlog(PR_ERR, "OCAPI: Final link status: %016llx\n", - get_odl_status(n->chip_id, dev->index)); + rc = train_link(n->chip_id, dev); + if (rc) goto failed; - } otl_enabletx(n->chip_id, n->xscom_base, dev->index); - slot = npu2_opencapi_slot_create(&dev->phb_ocapi); if (!slot) { /** diff --git a/include/platform.h b/include/platform.h index a7776446..6fa016c2 100644 --- a/include/platform.h +++ b/include/platform.h @@ -51,6 +51,10 @@ struct platform_ocapi { uint32_t i2c_offset[3]; /* Offsets on I2C device */ uint8_t i2c_odl0_data[3]; /* Data to reset ODL0 */ uint8_t i2c_odl1_data[3]; /* Data to reset ODL1 */ + uint8_t i2c_presence_addr; /* I2C address for presence detection */ + uint8_t i2c_presence_odl0; /* I2C mask for detection on ODL0 */ + uint8_t i2c_presence_odl1; /* I2C mask for detection on ODL1 */ + bool force_presence; /* don't use i2c detection */ bool odl_phy_swap; /* Swap ODL1 to use brick 2 rather than * brick 1 lanes */ }; diff --git a/platforms/astbmc/zaius.c b/platforms/astbmc/zaius.c index 7678ad12..52b072af 100644 --- a/platforms/astbmc/zaius.c +++ b/platforms/astbmc/zaius.c @@ -24,13 +24,17 @@ #include "astbmc.h" + const struct platform_ocapi zaius_ocapi = { .i2c_engine = 1, .i2c_port = 4, .i2c_offset = { 0x3, 0x1, 0x1 }, .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, - .odl_phy_swap = true, + .i2c_presence_addr = 0x20, + .i2c_presence_odl0 = (1 << 2), /* bottom connector */ + .i2c_presence_odl1 = (1 << 7), /* top connector */ + .odl_phy_swap = true, }; #define NPU_BASE 0x5011000 diff --git a/platforms/ibm-fsp/zz.c b/platforms/ibm-fsp/zz.c index 9a849290..6c8411ec 100644 --- a/platforms/ibm-fsp/zz.c +++ b/platforms/ibm-fsp/zz.c @@ -34,7 +34,15 @@ const struct platform_ocapi zz_ocapi = { .i2c_offset = { 0x3, 0x1, 0x1 }, .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, - .odl_phy_swap = true, + .i2c_presence_addr = 0x20, + .i2c_presence_odl0 = (1 << 2), /* bottom connector */ + .i2c_presence_odl1 = (1 << 7), /* top connector */ + /* + * i2c presence detection is broken on ZZ planar < v4 so we + * force the presence until all our systems are upgraded + */ + .force_presence = true, + .odl_phy_swap = true, }; static bool zz_probe(void) From patchwork Wed May 23 17:25:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 919254 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40rgJ24TPsz9s0W for ; Thu, 24 May 2018 03:59:14 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40rgJ239lRzF0xS for ; Thu, 24 May 2018 03:59:14 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40rfYg1C4tzF12j for ; Thu, 24 May 2018 03:25:58 +1000 (AEST) Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4NHNwG9030628 for ; Wed, 23 May 2018 13:25:56 -0400 Received: from e06smtp14.uk.ibm.com (e06smtp14.uk.ibm.com [195.75.94.110]) by mx0a-001b2d01.pphosted.com with ESMTP id 2j5b80cqb7-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 23 May 2018 13:25:56 -0400 Received: from localhost by e06smtp14.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 23 May 2018 18:25:53 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp14.uk.ibm.com (192.168.101.144) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 23 May 2018 18:25:50 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w4NHPnYW2425258; Wed, 23 May 2018 17:25:49 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E517711C054; Wed, 23 May 2018 18:16:54 +0100 (BST) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7CFFF11C04A; Wed, 23 May 2018 18:16:54 +0100 (BST) Received: from localhost.localdomain (unknown [9.164.160.79]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 23 May 2018 18:16:54 +0100 (BST) From: Frederic Barrat To: andrew.donnellan@au1.ibm.com, skiboot@lists.ozlabs.org Date: Wed, 23 May 2018 19:25:42 +0200 X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180523172545.16016-1-fbarrat@linux.ibm.com> References: <20180523172545.16016-1-fbarrat@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18052317-0044-0000-0000-00000555BF86 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18052317-0045-0000-0000-00002897543B Message-Id: <20180523172545.16016-3-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-23_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1805230169 Subject: [Skiboot] [PATCH v2 2/5] npu2-opencapi: Rework adapter reset X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" From: Frederic Barrat Rework a bit the code to reset the opencapi adapter: - make clearer which i2c pin is resetting which device - break the reset operation in smaller chunks. This is really to prepare for a future patch. No functional changes. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- v2: use literals in platform_ocapi structure core/platform.c | 12 ++++---- hw/npu2-opencapi.c | 71 ++++++++++++++++++++++++++++++++++-------------- include/platform.h | 6 ++-- platforms/astbmc/zaius.c | 13 ++++----- platforms/ibm-fsp/zz.c | 12 ++++---- 5 files changed, 72 insertions(+), 42 deletions(-) diff --git a/core/platform.c b/core/platform.c index 92b118ba..3282227c 100644 --- a/core/platform.c +++ b/core/platform.c @@ -170,11 +170,11 @@ static int generic_start_preload_resource(enum resource_id id, uint32_t subid, /* These values will work for a ZZ booted using BML */ const struct platform_ocapi generic_ocapi = { - .i2c_engine = 1, - .i2c_port = 4, - .i2c_offset = { 0x3, 0x1, 0x1 }, - .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, - .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, + .i2c_engine = 1, + .i2c_port = 4, + .i2c_reset_addr = 0x20, + .i2c_reset_odl0 = (1 << 1), + .i2c_reset_odl1 = (1 << 6), .i2c_presence_addr = 0x20, .i2c_presence_odl0 = (1 << 2), /* bottom connector */ .i2c_presence_odl1 = (1 << 7), /* top connector */ @@ -184,7 +184,7 @@ const struct platform_ocapi generic_ocapi = { * force presence. */ .force_presence = true, - .odl_phy_swap = true, + .odl_phy_swap = true, }; static struct bmc_platform generic_bmc = { diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index b2bc0628..be0707ea 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -794,39 +794,70 @@ static void otl_enabletx(uint32_t gcid, uint32_t scom_base, uint64_t index) static void reset_ocapi_device(struct npu2_dev *dev) { - uint8_t data[3]; + uint8_t pin, data; int rc; - int i; switch (dev->index) { case 2: case 4: - memcpy(data, platform.ocapi->i2c_odl0_data, sizeof(data)); + pin = platform.ocapi->i2c_reset_odl0; break; case 3: case 5: - memcpy(data, platform.ocapi->i2c_odl1_data, sizeof(data)); + pin = platform.ocapi->i2c_reset_odl1; break; default: assert(false); } - for (i = 0; i < 3; i++) { - rc = i2c_request_send(dev->i2c_port_id_ocapi, 0x20, SMBUS_WRITE, - platform.ocapi->i2c_offset[i], 1, - &data[i], sizeof(data[i]), 120); - if (rc) { - /** - * @fwts-label OCAPIDeviceResetFailed - * @fwts-advice There was an error attempting to send - * a reset signal over I2C to the OpenCAPI device. - */ - prlog(PR_ERR, "OCAPI: Error writing I2C reset signal: %d\n", rc); - break; - } - if (i != 0) - time_wait_ms(5); - } + /* + * set the i2c reset pin in output mode + * + * On the 9554 device, register 3 is the configuration + * register and a pin is in output mode if its value is 0 + */ + data = ~pin; + rc = i2c_request_send(dev->i2c_port_id_ocapi, + platform.ocapi->i2c_reset_addr, SMBUS_WRITE, + 0x3, 1, + &data, sizeof(data), 120); + if (rc) + goto err; + + /* + * assert i2c reset for 5ms + * register 1 controls the signal, reset is active low + */ + data = ~pin; + rc = i2c_request_send(dev->i2c_port_id_ocapi, + platform.ocapi->i2c_reset_addr, SMBUS_WRITE, + 0x1, 1, + &data, sizeof(data), 120); + if (rc) + goto err; + time_wait_ms(5); + + /* + * deassert i2c reset and wait 5ms + */ + data = 0xFF; + rc = i2c_request_send(dev->i2c_port_id_ocapi, + platform.ocapi->i2c_reset_addr, SMBUS_WRITE, + 0x1, 1, + &data, sizeof(data), 120); + if (rc) + goto err; + time_wait_ms(5); + + return; + +err: + /** + * @fwts-label OCAPIDeviceResetFailed + * @fwts-advice There was an error attempting to send + * a reset signal over I2C to the OpenCAPI device. + */ + prlog(PR_ERR, "OCAPI: Error writing I2C reset signal: %d\n", rc); } static bool i2c_presence_detect(struct npu2_dev *dev) diff --git a/include/platform.h b/include/platform.h index 6fa016c2..1a35a86a 100644 --- a/include/platform.h +++ b/include/platform.h @@ -48,9 +48,9 @@ struct bmc_platform { struct platform_ocapi { uint8_t i2c_engine; /* I2C engine number */ uint8_t i2c_port; /* I2C port number */ - uint32_t i2c_offset[3]; /* Offsets on I2C device */ - uint8_t i2c_odl0_data[3]; /* Data to reset ODL0 */ - uint8_t i2c_odl1_data[3]; /* Data to reset ODL1 */ + uint8_t i2c_reset_addr; /* I2C address for reset */ + uint8_t i2c_reset_odl0; /* I2C pin to write to reset ODL0 */ + uint8_t i2c_reset_odl1; /* I2C pin to write to reset ODL1 */ uint8_t i2c_presence_addr; /* I2C address for presence detection */ uint8_t i2c_presence_odl0; /* I2C mask for detection on ODL0 */ uint8_t i2c_presence_odl1; /* I2C mask for detection on ODL1 */ diff --git a/platforms/astbmc/zaius.c b/platforms/astbmc/zaius.c index 52b072af..8bc649c1 100644 --- a/platforms/astbmc/zaius.c +++ b/platforms/astbmc/zaius.c @@ -24,17 +24,16 @@ #include "astbmc.h" - const struct platform_ocapi zaius_ocapi = { - .i2c_engine = 1, - .i2c_port = 4, - .i2c_offset = { 0x3, 0x1, 0x1 }, - .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, - .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, + .i2c_engine = 1, + .i2c_port = 4, + .i2c_reset_addr = 0x20, + .i2c_reset_odl0 = (1 << 1), + .i2c_reset_odl1 = (1 << 6), .i2c_presence_addr = 0x20, .i2c_presence_odl0 = (1 << 2), /* bottom connector */ .i2c_presence_odl1 = (1 << 7), /* top connector */ - .odl_phy_swap = true, + .odl_phy_swap = true, }; #define NPU_BASE 0x5011000 diff --git a/platforms/ibm-fsp/zz.c b/platforms/ibm-fsp/zz.c index 6c8411ec..dc48768b 100644 --- a/platforms/ibm-fsp/zz.c +++ b/platforms/ibm-fsp/zz.c @@ -29,11 +29,11 @@ /* We don't yet create NPU device nodes on ZZ, but these values are correct */ const struct platform_ocapi zz_ocapi = { - .i2c_engine = 1, - .i2c_port = 4, - .i2c_offset = { 0x3, 0x1, 0x1 }, - .i2c_odl0_data = { 0xFD, 0xFD, 0xFF }, - .i2c_odl1_data = { 0xBF, 0xBF, 0xFF }, + .i2c_engine = 1, + .i2c_port = 4, + .i2c_reset_addr = 0x20, + .i2c_reset_odl0 = (1 << 1), + .i2c_reset_odl1 = (1 << 6), .i2c_presence_addr = 0x20, .i2c_presence_odl0 = (1 << 2), /* bottom connector */ .i2c_presence_odl1 = (1 << 7), /* top connector */ @@ -42,7 +42,7 @@ const struct platform_ocapi zz_ocapi = { * force the presence until all our systems are upgraded */ .force_presence = true, - .odl_phy_swap = true, + .odl_phy_swap = true, }; static bool zz_probe(void) From patchwork Wed May 23 17:25:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 919255 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40rgJM2WxRz9s0W for ; Thu, 24 May 2018 03:59:31 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40rgJM1C8DzDr4N for ; Thu, 24 May 2018 03:59:31 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40rfYg6rDWzF12G for ; Thu, 24 May 2018 03:25:59 +1000 (AEST) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4NHO1JB053458 for ; Wed, 23 May 2018 13:25:57 -0400 Received: from e06smtp15.uk.ibm.com (e06smtp15.uk.ibm.com [195.75.94.111]) by mx0b-001b2d01.pphosted.com with ESMTP id 2j5a8ag55b-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 23 May 2018 13:25:57 -0400 Received: from localhost by e06smtp15.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 23 May 2018 18:25:55 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp15.uk.ibm.com (192.168.101.145) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 23 May 2018 18:25:52 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w4NHPphM7733624; Wed, 23 May 2018 17:25:51 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 96DC811C052; Wed, 23 May 2018 18:16:56 +0100 (BST) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1D2C811C04A; Wed, 23 May 2018 18:16:56 +0100 (BST) Received: from localhost.localdomain (unknown [9.164.160.79]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 23 May 2018 18:16:55 +0100 (BST) From: Frederic Barrat To: andrew.donnellan@au1.ibm.com, skiboot@lists.ozlabs.org Date: Wed, 23 May 2018 19:25:43 +0200 X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180523172545.16016-1-fbarrat@linux.ibm.com> References: <20180523172545.16016-1-fbarrat@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18052317-0020-0000-0000-0000042083BA X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18052317-0021-0000-0000-000042B5C39A Message-Id: <20180523172545.16016-4-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-23_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1805230169 Subject: [Skiboot] [PATCH v2 3/5] npu2-opencapi: Train links on fundamental reset X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" From: Frederic Barrat Reorder our link training steps so that they are executed on fundamental reset instead of during the initial setup. Skiboot always call a fundamental reset on all the PHBs during pci init. It is done through a state machine, similarly to what is done for 'real' PHBs. This is the first step for a longer term goal to be able to trigger an adapter reset from linux. We'll need the reset callbacks of the PHB to be defined. We have to handle the various delays differently, since a linux thread shouldn't stay stuck waiting in opal for too long. No functional changes. Signed-off-by: Frederic Barrat Acked-by: Andrew Donnellan --- v2: Fence/unfence the OTL on reset, except during initial startup, where it is useless. It is to protect us for later, when the OS can trigger an adapter reset, as a FIR bit is raised if the link goes down and the OTL is not fenced. hw/npu2-opencapi.c | 379 +++++++++++++++++++++++++++++++++++++---------------- include/npu2.h | 2 + 2 files changed, 268 insertions(+), 113 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index be0707ea..0e4b6287 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -58,6 +58,21 @@ #define TL_MAX_TEMPLATE 63 #define TL_RATE_BUF_SIZE 32 +#define OCAPI_SLOT_NORMAL PCI_SLOT_STATE_NORMAL +#define OCAPI_SLOT_LINK PCI_SLOT_STATE_LINK +#define OCAPI_SLOT_LINK_START (OCAPI_SLOT_LINK + 1) +#define OCAPI_SLOT_LINK_WAIT (OCAPI_SLOT_LINK + 2) +#define OCAPI_SLOT_LINK_TRAINED (OCAPI_SLOT_LINK + 3) +#define OCAPI_SLOT_FRESET PCI_SLOT_STATE_FRESET +#define OCAPI_SLOT_FRESET_START (OCAPI_SLOT_FRESET + 1) +#define OCAPI_SLOT_FRESET_INIT (OCAPI_SLOT_FRESET + 2) +#define OCAPI_SLOT_FRESET_ASSERT_DELAY (OCAPI_SLOT_FRESET + 3) +#define OCAPI_SLOT_FRESET_DEASSERT_DELAY (OCAPI_SLOT_FRESET + 4) +#define OCAPI_SLOT_FRESET_INIT_DELAY (OCAPI_SLOT_FRESET + 5) + +#define OCAPI_LINK_TRAINING_RETRIES 5 +#define OCAPI_LINK_TRAINING_TIMEOUT 3000 /* ms */ + enum npu2_link_training_state { NPU2_TRAIN_DEFAULT, /* fully train the link */ NPU2_TRAIN_PRBS31, /* used for Signal Integrity testing */ @@ -792,7 +807,7 @@ static void otl_enabletx(uint32_t gcid, uint32_t scom_base, uint64_t index) /* TODO: Abort if credits are zero */ } -static void reset_ocapi_device(struct npu2_dev *dev) +static void assert_reset(struct npu2_dev *dev) { uint8_t pin, data; int rc; @@ -824,10 +839,7 @@ static void reset_ocapi_device(struct npu2_dev *dev) if (rc) goto err; - /* - * assert i2c reset for 5ms - * register 1 controls the signal, reset is active low - */ + /* register 1 controls the signal, reset is active low */ data = ~pin; rc = i2c_request_send(dev->i2c_port_id_ocapi, platform.ocapi->i2c_reset_addr, SMBUS_WRITE, @@ -835,20 +847,6 @@ static void reset_ocapi_device(struct npu2_dev *dev) &data, sizeof(data), 120); if (rc) goto err; - time_wait_ms(5); - - /* - * deassert i2c reset and wait 5ms - */ - data = 0xFF; - rc = i2c_request_send(dev->i2c_port_id_ocapi, - platform.ocapi->i2c_reset_addr, SMBUS_WRITE, - 0x1, 1, - &data, sizeof(data), 120); - if (rc) - goto err; - time_wait_ms(5); - return; err: @@ -860,6 +858,26 @@ err: prlog(PR_ERR, "OCAPI: Error writing I2C reset signal: %d\n", rc); } +static void deassert_reset(struct npu2_dev *dev) +{ + uint8_t data; + int rc; + + data = 0xFF; + rc = i2c_request_send(dev->i2c_port_id_ocapi, + platform.ocapi->i2c_reset_addr, SMBUS_WRITE, + 0x1, 1, + &data, sizeof(data), 120); + if (rc) { + /** + * @fwts-label OCAPIDeviceResetFailed + * @fwts-advice There was an error attempting to send + * a reset signal over I2C to the OpenCAPI device. + */ + prlog(PR_ERR, "OCAPI: Error writing I2C reset signal: %d\n", rc); + } +} + static bool i2c_presence_detect(struct npu2_dev *dev) { uint8_t state, data; @@ -898,13 +916,11 @@ static bool i2c_presence_detect(struct npu2_dev *dev) return !(state & data); } -static int odl_train(uint32_t gcid, uint32_t index, struct npu2_dev *dev) +static void reset_odl(uint32_t gcid, struct npu2_dev *dev) { uint64_t reg, config_xscom; - int timeout = 3000; - prlog(PR_DEBUG, "OCAPI: %s: Training ODL\n", __func__); - switch (index) { + switch (dev->index) { case 2: config_xscom = OB0_ODL0_CONFIG; break; @@ -934,35 +950,60 @@ static int odl_train(uint32_t gcid, uint32_t index, struct npu2_dev *dev) reg &= ~OB_ODL_CONFIG_RESET; xscom_write(gcid, config_xscom, reg); +} - reset_ocapi_device(dev); +static void set_init_pattern(uint32_t gcid, struct npu2_dev *dev) +{ + uint64_t reg, config_xscom; + + switch (dev->index) { + case 2: + config_xscom = OB0_ODL0_CONFIG; + break; + case 3: + config_xscom = OB0_ODL1_CONFIG; + break; + case 4: + config_xscom = OB3_ODL1_CONFIG; + break; + case 5: + config_xscom = OB3_ODL0_CONFIG; + break; + default: + assert(false); + } /* Transmit Pattern A */ + xscom_read(gcid, config_xscom, ®); reg = SETFIELD(OB_ODL_CONFIG_TRAIN_MODE, reg, 0b0001); xscom_write(gcid, config_xscom, reg); - time_wait_ms(5); +} + +static void start_training(uint32_t gcid, struct npu2_dev *dev) +{ + uint64_t reg, config_xscom; - /* Bump lanes - this improves training reliability */ - npu2_opencapi_bump_ui_lane(dev); + switch (dev->index) { + case 2: + config_xscom = OB0_ODL0_CONFIG; + break; + case 3: + config_xscom = OB0_ODL1_CONFIG; + break; + case 4: + config_xscom = OB3_ODL1_CONFIG; + break; + case 5: + config_xscom = OB3_ODL0_CONFIG; + break; + default: + assert(false); + } /* Start training */ + xscom_read(gcid, config_xscom, ®); reg = SETFIELD(OB_ODL_CONFIG_TRAIN_MODE, reg, 0b1000); xscom_write(gcid, config_xscom, reg); - - do { - reg = get_odl_status(gcid, index); - if (GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg) == 0x7) { - prlog(PR_NOTICE, - "OCAPI: Link %d on chip %u trained in %dms\n", - index, gcid, 3000 - timeout); - return OPAL_SUCCESS; - } - time_wait_ms(1); - } while (timeout--); - prlog(PR_INFO, "OCAPI: Link %d on chip %u failed to train, retrying\n", - index, gcid); - prlog(PR_INFO, "OCAPI: Link status: %016llx\n", reg); - return OPAL_HARDWARE; } static int64_t npu2_opencapi_get_presence_state(struct pci_slot *slot, @@ -1000,6 +1041,167 @@ static int64_t npu2_opencapi_get_link_state(struct pci_slot *slot, uint8_t *val) return rc; } +static int64_t npu2_opencapi_retry_state(struct pci_slot *slot) +{ + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + uint32_t chip_id = dev->npu->chip_id; + + if (!slot->link_retries--) { + /** + * @fwts-label OCAPILinkTrainingFailed + * @fwts-advice The OpenCAPI link training procedure failed. + * This indicates a hardware or firmware bug. OpenCAPI + * functionality will not be available on this link. + */ + prlog(PR_ERR, + "OCAPI: Link %d on chip %u failed to train\n", + dev->index, chip_id); + prlog(PR_ERR, "OCAPI: Final link status: %016llx\n", + get_odl_status(chip_id, dev->index)); + return OPAL_HARDWARE; + } + + prlog(PR_ERR, + "OCAPI: Link %d on chip %u failed to train, retrying\n", + dev->index, chip_id); + pci_slot_set_state(slot, OCAPI_SLOT_FRESET_INIT); + return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); +} + +static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) +{ + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + uint32_t chip_id = dev->npu->chip_id; + uint64_t reg; + + switch (slot->state) { + case OCAPI_SLOT_NORMAL: + case OCAPI_SLOT_LINK_START: + prlog(PR_DEBUG, "OCAPI: LINK: Start polling\n"); + pci_slot_set_state(slot, OCAPI_SLOT_LINK_WAIT); + /* fall-through */ + case OCAPI_SLOT_LINK_WAIT: + reg = get_odl_status(chip_id, dev->index); + if (GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg) == 0x7) { + prlog(PR_NOTICE, + "OCAPI: Link %d on chip %u trained in %lld ms\n", + dev->index, chip_id, + OCAPI_LINK_TRAINING_TIMEOUT - slot->retries); + pci_slot_set_state(slot, OCAPI_SLOT_LINK_TRAINED); + return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); + } + if (slot->retries-- == 0) + return npu2_opencapi_retry_state(slot); + + return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); + + case OCAPI_SLOT_LINK_TRAINED: + otl_enabletx(chip_id, dev->npu->xscom_base, dev->index); + pci_slot_set_state(slot, OCAPI_SLOT_NORMAL); + return OPAL_SUCCESS; + + default: + prlog(PR_ERR, "OCAPI: LINK: unexpected slot state %08x\n", + slot->state); + + } + pci_slot_set_state(slot, OCAPI_SLOT_NORMAL); + return OPAL_HARDWARE; +} + +static int64_t npu2_opencapi_creset(struct pci_slot *slot __unused) +{ + prlog(PR_ERR, "OCAPI: creset not supported\n"); + return OPAL_UNSUPPORTED; +} + +static int64_t npu2_opencapi_freset(struct pci_slot *slot) +{ + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + uint32_t chip_id = dev->npu->chip_id; + uint8_t presence = 1; + + switch (slot->state) { + case OCAPI_SLOT_NORMAL: + case OCAPI_SLOT_FRESET_START: + prlog(PR_DEBUG, "OCAPI: FRESET: Starts\n"); + + if (slot->ops.get_presence_state) + slot->ops.get_presence_state(slot, &presence); + if (!presence) { + /* + * FIXME: if there's no card on the link, we + * should consider powering off the unused + * lanes to save energy + */ + prlog(PR_INFO, + "OCAPI: no card detected on link %d, chip %d\n", + dev->index, chip_id); + return OPAL_SUCCESS; + } + if (dev->train_need_fence) { + prlog(PR_DEBUG, "Fencing OTL during reset\n"); + set_fence_control(chip_id, dev->npu->xscom_base, + dev->index, 0b11); + npu2_write(dev->npu, NPU2_MISC_FENCE_STATE, + PPC_BIT(dev->index + 6)); + dev->train_fenced = true; + } + dev->train_need_fence = true; + slot->link_retries = OCAPI_LINK_TRAINING_RETRIES; + npu2_opencapi_phy_setup(dev); + /* fall-through */ + case OCAPI_SLOT_FRESET_INIT: + reset_odl(chip_id, dev); + assert_reset(dev); + pci_slot_set_state(slot, + OCAPI_SLOT_FRESET_ASSERT_DELAY); + /* assert for 5ms */ + return pci_slot_set_sm_timeout(slot, msecs_to_tb(5)); + + case OCAPI_SLOT_FRESET_ASSERT_DELAY: + deassert_reset(dev); + pci_slot_set_state(slot, + OCAPI_SLOT_FRESET_DEASSERT_DELAY); + /* give another 5ms to device to be ready */ + return pci_slot_set_sm_timeout(slot, msecs_to_tb(5)); + + case OCAPI_SLOT_FRESET_DEASSERT_DELAY: + if (dev->train_fenced) { + prlog(PR_DEBUG, "Unfencing OTL after reset\n"); + npu2_write(dev->npu, NPU2_MISC_FENCE_STATE, PPC_BIT(dev->index)); + set_fence_control(chip_id, dev->npu->xscom_base, + dev->index, 0b00); + dev->train_fenced = false; + } + + set_init_pattern(chip_id, dev); + pci_slot_set_state(slot, + OCAPI_SLOT_FRESET_INIT_DELAY); + return pci_slot_set_sm_timeout(slot, msecs_to_tb(5)); + + case OCAPI_SLOT_FRESET_INIT_DELAY: + /* Bump lanes - this improves training reliability */ + npu2_opencapi_bump_ui_lane(dev); + start_training(chip_id, dev); + slot->retries = OCAPI_LINK_TRAINING_TIMEOUT; + pci_slot_set_state(slot, OCAPI_SLOT_LINK_START); + return slot->ops.poll_link(slot); + + default: + prlog(PR_ERR, "OCAPI: FRESET: unexpected slot state %08x\n", + slot->state); + } + pci_slot_set_state(slot, OCAPI_SLOT_NORMAL); + return OPAL_HARDWARE; +} + +static int64_t npu2_opencapi_hreset(struct pci_slot *slot __unused) +{ + prlog(PR_ERR, "OCAPI: hreset not supported\n"); + return OPAL_UNSUPPORTED; +} + static struct pci_slot *npu2_opencapi_slot_create(struct phb *phb) { struct pci_slot *slot; @@ -1016,15 +1218,11 @@ static struct pci_slot *npu2_opencapi_slot_create(struct phb *phb) slot->ops.get_latch_state = NULL; slot->ops.set_power_state = NULL; slot->ops.set_attention_state = NULL; - /* - * Temporarily erase the run_sm callback until we support - * dynamic reset of the link. Otherwise, run_sm may call - * freset, creset, ... and we don't define them. The run_sm - * pointer is always tested before being called, at least at - * the time of this writing :-) It will go away when we - * implement dynamic reset of the link - */ - slot->ops.run_sm = NULL; + + slot->ops.poll_link = npu2_opencapi_poll_link; + slot->ops.creset = npu2_opencapi_creset; + slot->ops.freset = npu2_opencapi_freset; + slot->ops.hreset = npu2_opencapi_hreset; return slot; } @@ -1345,48 +1543,22 @@ static int setup_irq(struct npu2 *p) return 0; } -#define LINK_TRAINING_RETRIES 5 - -static int train_link(int chip_id, struct npu2_dev *dev) +static void setup_debug_training_state(struct npu2_dev *dev) { - bool train; - int rc; - int retries = LINK_TRAINING_RETRIES; - - if (platform.ocapi->force_presence) - train = true; - else - train = i2c_presence_detect(dev); - if (!train) { - /* - * FIXME: if there's no card on the link, we should consider - * powering off the unused lanes to save energy - */ - prlog(PR_INFO, "OCAPI: no card detected on link %d, chip %d\n", - dev->index, chip_id); - return -1; - } + npu2_opencapi_phy_setup(dev); - do { - rc = odl_train(chip_id, dev->index, dev); - } while (rc != OPAL_SUCCESS && --retries); + switch (npu2_ocapi_training_state) { + case NPU2_TRAIN_PRBS31: + prlog(PR_INFO, "OCAPI: sending PRBS31 pattern per NVRAM setting\n"); + npu2_opencapi_phy_prbs31(dev); + break; - if (rc != OPAL_SUCCESS && retries == 0) { - /** - * @fwts-label OCAPILinkTrainingFailed - * @fwts-advice The OpenCAPI link training procedure failed. - * This indicates a hardware or firmware bug. OpenCAPI - * functionality will not be available on this link. - */ - prlog(PR_ERR, - "OCAPI: Link %d on chip %u failed to train\n", - dev->index, chip_id); - prlog(PR_ERR, "OCAPI: Final link status: %016llx\n", - get_odl_status(chip_id, dev->index)); - return -1; + case NPU2_TRAIN_NONE: + prlog(PR_INFO, "OCAPI: link not trained per NVRAM setting\n"); + break; + default: + assert(false); } - - return 0; } static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, @@ -1397,7 +1569,6 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, struct pci_slot *slot; char port_name[17]; uint64_t mm_win[2]; - int rc; dev_index = dt_prop_get_u32(dn_link, "ibm,npu-link-index"); npu_index = dt_prop_get_u32(n->dt_node, "ibm,npu-index"); @@ -1452,6 +1623,8 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, dev->lane_mask = dt_prop_get_u32(dn_link, "ibm,npu-lane-mask"); dev->link_speed = dt_prop_get_u64(dn_link, "ibm,link-speed"); dev->bdfn = 0; + dev->train_need_fence = false; + dev->train_fenced = false; n->total_devices++; /* Find I2C port for handling device reset */ @@ -1480,22 +1653,9 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, set_fence_control(n->chip_id, n->xscom_base, dev->index, 0b00); - npu2_opencapi_phy_setup(dev); - - switch (npu2_ocapi_training_state) { - case NPU2_TRAIN_PRBS31: - prlog(PR_INFO, - "OCAPI: Link %d sending PRBS31 pattern per NVRAM setting\n", - dev->index); - npu2_opencapi_phy_prbs31(dev); - break; - - case NPU2_TRAIN_DEFAULT: - rc = train_link(n->chip_id, dev); - if (rc) - goto failed; - - otl_enabletx(n->chip_id, n->xscom_base, dev->index); + if (npu2_ocapi_training_state != NPU2_TRAIN_DEFAULT) { + setup_debug_training_state(dev); + } else { slot = npu2_opencapi_slot_create(&dev->phb_ocapi); if (!slot) { /** @@ -1505,14 +1665,7 @@ static void npu2_opencapi_setup_device(struct dt_node *dn_link, struct npu2 *n, */ prlog(PR_ERR, "OCAPI: Cannot create PHB slot\n"); } - break; - - case NPU2_TRAIN_NONE: - prlog(PR_INFO, "OCAPI: Link %d not trained per NVRAM setting\n", - dev->index); - break; } - pci_register_phb(&dev->phb_ocapi, OPAL_DYNAMIC_PHB_ID); return; failed: diff --git a/include/npu2.h b/include/npu2.h index b95e1d3a..eb4af5e6 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -135,6 +135,8 @@ struct npu2_dev { /* OpenCAPI */ struct phb phb_ocapi; uint64_t i2c_port_id_ocapi; + bool train_need_fence; + bool train_fenced; }; struct npu2 { From patchwork Wed May 23 17:25:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 919256 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40rgJl0SMyz9s0W for ; Thu, 24 May 2018 03:59:51 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40rgJk5dXRzDqsy for ; Thu, 24 May 2018 03:59:50 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40rfYh18spzF12Y for ; Thu, 24 May 2018 03:25:59 +1000 (AEST) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4NHO10p021902 for ; Wed, 23 May 2018 13:25:57 -0400 Received: from e06smtp12.uk.ibm.com (e06smtp12.uk.ibm.com [195.75.94.108]) by mx0b-001b2d01.pphosted.com with ESMTP id 2j5bg6kcrq-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 23 May 2018 13:25:56 -0400 Received: from localhost by e06smtp12.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 23 May 2018 18:25:55 +0100 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp12.uk.ibm.com (192.168.101.142) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 23 May 2018 18:25:54 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w4NHPq8N7864696; Wed, 23 May 2018 17:25:52 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 36FAC11C050; Wed, 23 May 2018 18:16:58 +0100 (BST) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C31ED11C05B; Wed, 23 May 2018 18:16:57 +0100 (BST) Received: from localhost.localdomain (unknown [9.164.160.79]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 23 May 2018 18:16:57 +0100 (BST) From: Frederic Barrat To: andrew.donnellan@au1.ibm.com, skiboot@lists.ozlabs.org Date: Wed, 23 May 2018 19:25:44 +0200 X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180523172545.16016-1-fbarrat@linux.ibm.com> References: <20180523172545.16016-1-fbarrat@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18052317-0008-0000-0000-000004FA5FFE X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18052317-0009-0000-0000-00001E8E647D Message-Id: <20180523172545.16016-5-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-23_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1805230169 Subject: [Skiboot] [PATCH v2 4/5] npu2-opencapi: Cleanup traces printed during link training X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" From: Frederic Barrat Now that links may train in parallel, traces shown during training can be all mixed up. So add a prefix to all the traces to clearly identify the chip and link the trace refers to: OCAPI[:]: this is a very useful message The lower-level hardware procedures (npu2-hw-procedures.c) also print traces which would need work. But that code is being reworked to be better integrated with opencapi and nvidia, so leave it alone for now. Signed-off-by: Frederic Barrat Acked-by: Andrew Donnellan --- v2: add 2 traces around OTL fence/unfence hw/npu2-opencapi.c | 80 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 41 insertions(+), 39 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 0e4b6287..af6c4871 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -52,6 +52,14 @@ #include #include +#define OCAPIDBG(dev, fmt, a...) prlog(PR_DEBUG, "OCAPI[%d:%d]: " fmt, \ + dev->npu->chip_id, dev->index, ## a) +#define OCAPIINF(dev, fmt, a...) prlog(PR_INFO, "OCAPI[%d:%d]: " fmt, \ + dev->npu->chip_id, dev->index, ## a) +#define OCAPIERR(dev, fmt, a...) prlog(PR_ERR, "OCAPI[%d:%d]: " fmt, \ + dev->npu->chip_id, dev->index, ## a) + + #define NPU_IRQ_LEVELS 35 #define NPU_IRQ_LEVELS_XSL 23 #define MAX_PE_HANDLE ((1 << 15) - 1) @@ -787,15 +795,16 @@ static void setup_afu_config_bars(uint32_t gcid, uint32_t scom_base, dev->bars[1].npu2_bar.size = size; } -static void otl_enabletx(uint32_t gcid, uint32_t scom_base, uint64_t index) +static void otl_enabletx(uint32_t gcid, uint32_t scom_base, + struct npu2_dev *dev) { - uint64_t stack = index_to_stack(index); - uint64_t block = index_to_block(index); + uint64_t stack = index_to_stack(dev->index); + uint64_t block = index_to_block(dev->index); uint64_t reg; /* OTL Config 2 Register */ /* Transmit Enable */ - prlog(PR_DEBUG, "OCAPI: %s: Enabling TX\n", __func__); + OCAPIDBG(dev, "Enabling TX\n"); reg = 0; reg |= NPU2_OTL_CONFIG2_TX_SEND_EN; npu2_scom_write(gcid, scom_base, NPU2_OTL_CONFIG2(stack, block), @@ -803,7 +812,7 @@ static void otl_enabletx(uint32_t gcid, uint32_t scom_base, uint64_t index) reg = npu2_scom_read(gcid, scom_base, NPU2_OTL_VC_CREDITS(stack, block), NPU2_MISC_DA_LEN_8B); - prlog(PR_DEBUG, "OCAPI: credit counter: %llx\n", reg); + OCAPIDBG(dev, "credit counter: %llx\n", reg); /* TODO: Abort if credits are zero */ } @@ -855,7 +864,7 @@ err: * @fwts-advice There was an error attempting to send * a reset signal over I2C to the OpenCAPI device. */ - prlog(PR_ERR, "OCAPI: Error writing I2C reset signal: %d\n", rc); + OCAPIERR(dev, "Error writing I2C reset signal: %d\n", rc); } static void deassert_reset(struct npu2_dev *dev) @@ -874,7 +883,7 @@ static void deassert_reset(struct npu2_dev *dev) * @fwts-advice There was an error attempting to send * a reset signal over I2C to the OpenCAPI device. */ - prlog(PR_ERR, "OCAPI: Error writing I2C reset signal: %d\n", rc); + OCAPIERR(dev, "Error writing I2C reset signal: %d\n", rc); } } @@ -894,12 +903,11 @@ static bool i2c_presence_detect(struct npu2_dev *dev) SMBUS_READ, 0, 1, &state, 1, 120); if (rc) { - prlog(PR_ERR, "OCAPI: error detecting link presence: %d\n", - rc); + OCAPIERR(dev, "error detecting link presence: %d\n", rc); return true; /* assume link exists */ } - prlog(PR_DEBUG, "OCAPI: I2C presence detect: 0x%x\n", state); + OCAPIDBG(dev, "I2C presence detect: 0x%x\n", state); switch (dev->index) { case 2: @@ -909,7 +917,7 @@ static bool i2c_presence_detect(struct npu2_dev *dev) data = platform.ocapi->i2c_presence_odl1; break; default: - prlog(PR_ERR, "OCAPI: presence detection on invalid link\n"); + OCAPIERR(dev, "presence detection on invalid link\n"); return true; } /* Presence detect bits are active low */ @@ -1053,17 +1061,13 @@ static int64_t npu2_opencapi_retry_state(struct pci_slot *slot) * This indicates a hardware or firmware bug. OpenCAPI * functionality will not be available on this link. */ - prlog(PR_ERR, - "OCAPI: Link %d on chip %u failed to train\n", - dev->index, chip_id); - prlog(PR_ERR, "OCAPI: Final link status: %016llx\n", + OCAPIERR(dev, + "Link failed to train, final link status: %016llx\n", get_odl_status(chip_id, dev->index)); return OPAL_HARDWARE; } - prlog(PR_ERR, - "OCAPI: Link %d on chip %u failed to train, retrying\n", - dev->index, chip_id); + OCAPIERR(dev, "Link failed to train, retrying\n"); pci_slot_set_state(slot, OCAPI_SLOT_FRESET_INIT); return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); } @@ -1077,15 +1081,13 @@ static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) switch (slot->state) { case OCAPI_SLOT_NORMAL: case OCAPI_SLOT_LINK_START: - prlog(PR_DEBUG, "OCAPI: LINK: Start polling\n"); + OCAPIDBG(dev, "Start polling\n"); pci_slot_set_state(slot, OCAPI_SLOT_LINK_WAIT); /* fall-through */ case OCAPI_SLOT_LINK_WAIT: reg = get_odl_status(chip_id, dev->index); if (GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg) == 0x7) { - prlog(PR_NOTICE, - "OCAPI: Link %d on chip %u trained in %lld ms\n", - dev->index, chip_id, + OCAPIINF(dev, "link trained in %lld ms\n", OCAPI_LINK_TRAINING_TIMEOUT - slot->retries); pci_slot_set_state(slot, OCAPI_SLOT_LINK_TRAINED); return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); @@ -1096,13 +1098,12 @@ static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); case OCAPI_SLOT_LINK_TRAINED: - otl_enabletx(chip_id, dev->npu->xscom_base, dev->index); + otl_enabletx(chip_id, dev->npu->xscom_base, dev); pci_slot_set_state(slot, OCAPI_SLOT_NORMAL); return OPAL_SUCCESS; default: - prlog(PR_ERR, "OCAPI: LINK: unexpected slot state %08x\n", - slot->state); + OCAPIERR(dev, "unexpected slot state %08x\n", slot->state); } pci_slot_set_state(slot, OCAPI_SLOT_NORMAL); @@ -1111,7 +1112,9 @@ static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) static int64_t npu2_opencapi_creset(struct pci_slot *slot __unused) { - prlog(PR_ERR, "OCAPI: creset not supported\n"); + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + + OCAPIERR(dev, "creset not supported\n"); return OPAL_UNSUPPORTED; } @@ -1124,7 +1127,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) switch (slot->state) { case OCAPI_SLOT_NORMAL: case OCAPI_SLOT_FRESET_START: - prlog(PR_DEBUG, "OCAPI: FRESET: Starts\n"); + OCAPIDBG(dev, "FRESET starts\n"); if (slot->ops.get_presence_state) slot->ops.get_presence_state(slot, &presence); @@ -1134,13 +1137,11 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) * should consider powering off the unused * lanes to save energy */ - prlog(PR_INFO, - "OCAPI: no card detected on link %d, chip %d\n", - dev->index, chip_id); + OCAPIINF(dev, "no card detected\n"); return OPAL_SUCCESS; } if (dev->train_need_fence) { - prlog(PR_DEBUG, "Fencing OTL during reset\n"); + OCAPIDBG(dev, "Fencing OTL during reset\n"); set_fence_control(chip_id, dev->npu->xscom_base, dev->index, 0b11); npu2_write(dev->npu, NPU2_MISC_FENCE_STATE, @@ -1168,7 +1169,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) case OCAPI_SLOT_FRESET_DEASSERT_DELAY: if (dev->train_fenced) { - prlog(PR_DEBUG, "Unfencing OTL after reset\n"); + OCAPIDBG(dev, "Unfencing OTL after reset\n"); npu2_write(dev->npu, NPU2_MISC_FENCE_STATE, PPC_BIT(dev->index)); set_fence_control(chip_id, dev->npu->xscom_base, dev->index, 0b00); @@ -1189,7 +1190,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) return slot->ops.poll_link(slot); default: - prlog(PR_ERR, "OCAPI: FRESET: unexpected slot state %08x\n", + OCAPIERR(dev, "FRESET: unexpected slot state %08x\n", slot->state); } pci_slot_set_state(slot, OCAPI_SLOT_NORMAL); @@ -1198,7 +1199,9 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) static int64_t npu2_opencapi_hreset(struct pci_slot *slot __unused) { - prlog(PR_ERR, "OCAPI: hreset not supported\n"); + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + + OCAPIERR(dev, "hreset not supported\n"); return OPAL_UNSUPPORTED; } @@ -1549,12 +1552,12 @@ static void setup_debug_training_state(struct npu2_dev *dev) switch (npu2_ocapi_training_state) { case NPU2_TRAIN_PRBS31: - prlog(PR_INFO, "OCAPI: sending PRBS31 pattern per NVRAM setting\n"); + OCAPIINF(dev, "sending PRBS31 pattern per NVRAM setting\n"); npu2_opencapi_phy_prbs31(dev); break; case NPU2_TRAIN_NONE: - prlog(PR_INFO, "OCAPI: link not trained per NVRAM setting\n"); + OCAPIINF(dev, "link not trained per NVRAM setting\n"); break; default: assert(false); @@ -1961,7 +1964,7 @@ static bool is_template_supported(unsigned int templ, long capabilities) return !!(capabilities & (1ull << templ)); } -static int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t bdfn, +static int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t __unused bdfn, long capabilities, uint64_t rate_phys, int rate_sz) { struct phb *phb = pci_get_phb(phb_id); @@ -2014,8 +2017,7 @@ static int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t bdfn, npu2_scom_write(dev->npu->chip_id, dev->npu->xscom_base, NPU2_OTL_CONFIG1(stack, block), NPU2_MISC_DA_LEN_8B, reg); - prlog(PR_DEBUG, "OCAPI: Link %llx:%x, TL conf1 register set to %llx\n", - phb_id, bdfn, reg); + OCAPIDBG(dev, "OTL configuration 1 register set to %llx\n", reg); return OPAL_SUCCESS; } opal_call(OPAL_NPU_TL_SET, opal_npu_tl_set, 5); From patchwork Wed May 23 17:25:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 919259 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40rgK14jPLz9s0W for ; Thu, 24 May 2018 04:00:05 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40rgK11Y6dzF189 for ; Thu, 24 May 2018 04:00:05 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40rfYl3C50zF116 for ; Thu, 24 May 2018 03:26:03 +1000 (AEST) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4NHNxW2015695 for ; Wed, 23 May 2018 13:26:00 -0400 Received: from e06smtp12.uk.ibm.com (e06smtp12.uk.ibm.com [195.75.94.108]) by mx0a-001b2d01.pphosted.com with ESMTP id 2j5af3y8qj-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 23 May 2018 13:25:59 -0400 Received: from localhost by e06smtp12.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 23 May 2018 18:25:58 +0100 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp12.uk.ibm.com (192.168.101.142) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 23 May 2018 18:25:55 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w4NHPsIu25493554; Wed, 23 May 2018 17:25:54 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CA77411C050; Wed, 23 May 2018 18:16:59 +0100 (BST) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 63BE011C04A; Wed, 23 May 2018 18:16:59 +0100 (BST) Received: from localhost.localdomain (unknown [9.164.160.79]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 23 May 2018 18:16:59 +0100 (BST) From: Frederic Barrat To: andrew.donnellan@au1.ibm.com, skiboot@lists.ozlabs.org Date: Wed, 23 May 2018 19:25:45 +0200 X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180523172545.16016-1-fbarrat@linux.ibm.com> References: <20180523172545.16016-1-fbarrat@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18052317-0008-0000-0000-000004FA5FFF X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18052317-0009-0000-0000-00001E8E647E Message-Id: <20180523172545.16016-6-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-23_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1805230169 Subject: [Skiboot] [PATCH v2 5/5] npu2-opencapi: Fix link state to report link down X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" From: Frederic Barrat The PHB callback 'get_link_state' is always reporting the link width, irrespective of the link status and even when the link is down. It is causing too much work (and failures) when the PHB is probed during pci init. The fix is to look at the link status first and report the link as down when appropriate. Signed-off-by: Frederic Barrat Acked-by: Andrew Donnellan --- v2: no change hw/npu2-opencapi.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index af6c4871..50ef8766 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -80,6 +80,7 @@ #define OCAPI_LINK_TRAINING_RETRIES 5 #define OCAPI_LINK_TRAINING_TIMEOUT 3000 /* ms */ +#define OCAPI_LINK_STATE_TRAINED 0x7 enum npu2_link_training_state { NPU2_TRAIN_DEFAULT, /* fully train the link */ @@ -1032,10 +1033,17 @@ static int64_t npu2_opencapi_get_link_state(struct pci_slot *slot, uint8_t *val) { struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); uint64_t reg; - int64_t link_width, rc = OPAL_SUCCESS; + int64_t link_width, training_status, rc = OPAL_SUCCESS; reg = get_odl_status(dev->npu->chip_id, dev->index); link_width = GETFIELD(OB_ODL_STATUS_TRAINED_MODE, reg); + training_status = GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg); + + if (training_status != OCAPI_LINK_STATE_TRAINED) { + *val = OPAL_SHPC_LINK_DOWN; + return OPAL_SUCCESS; + } + switch (link_width) { case 0b0001: *val = OPAL_SHPC_LINK_UP_x4; @@ -1086,7 +1094,8 @@ static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) /* fall-through */ case OCAPI_SLOT_LINK_WAIT: reg = get_odl_status(chip_id, dev->index); - if (GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg) == 0x7) { + if (GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg) == + OCAPI_LINK_STATE_TRAINED) { OCAPIINF(dev, "link trained in %lld ms\n", OCAPI_LINK_TRAINING_TIMEOUT - slot->retries); pci_slot_set_state(slot, OCAPI_SLOT_LINK_TRAINED);