From patchwork Mon Jul 8 10:30:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1128989 X-Patchwork-Delegate: oohall@gmail.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45j1tV4l1vz9sN4 for ; Mon, 8 Jul 2019 20:30:26 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45j1tT6mdrzDqBj for ; Mon, 8 Jul 2019 20:30:25 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45j1tJ0mVbzDq6l for ; Mon, 8 Jul 2019 20:30:15 +1000 (AEST) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x68ASFmO019999 for ; Mon, 8 Jul 2019 06:30:11 -0400 Received: from e06smtp05.uk.ibm.com (e06smtp05.uk.ibm.com [195.75.94.101]) by mx0b-001b2d01.pphosted.com with ESMTP id 2tm345aesy-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 08 Jul 2019 06:30:11 -0400 Received: from localhost by e06smtp05.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 8 Jul 2019 11:30:08 +0100 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp05.uk.ibm.com (192.168.101.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 8 Jul 2019 11:30:07 +0100 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x68AU5vX44695680 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 8 Jul 2019 10:30:05 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7D55452050; Mon, 8 Jul 2019 10:30:05 +0000 (GMT) Received: from pic2.home (unknown [9.145.45.82]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id 372315204E; Mon, 8 Jul 2019 10:30:05 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Mon, 8 Jul 2019 12:30:05 +0200 X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19070810-0020-0000-0000-00000351429B X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19070810-0021-0000-0000-000021A4EC1A Message-Id: <20190708103005.32195-1-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-07-08_03:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1907080134 Subject: [Skiboot] [PATCH] npu2-opencapi: Add opencapi support on ZZ X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hegdevasant@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" This patch adds opencapi support on ZZ. It hard-codes the required device tree entries for the NPU and links. The alternative was to use HDAT, but it somehow proved to painful to do. The new device tree entries activate the npu2 init code on ZZ. On systems with no opencapi adapters, it should go unnoticed, as presence detection will skip link training. Signed-off-by: Frederic Barrat --- Stewart, Oliver, Vasant: it's hard to see clearly in the ever changing plan, but opencapi support may be desired for FW940. So hopefully this patch can be merged in whatever skiboot branch that fw will use (is that decided yet?). hdata/spira.c | 8 +++- platforms/ibm-fsp/zz.c | 96 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 102 insertions(+), 2 deletions(-) diff --git a/hdata/spira.c b/hdata/spira.c index 6891a9c7..c330a1fa 100644 --- a/hdata/spira.c +++ b/hdata/spira.c @@ -1369,7 +1369,13 @@ static void add_npu(struct dt_node *xscom, const struct HDIF_array_hdr *links, uint64_t speed = 0, nvlink_speed = 0; struct dt_node *node; - /* only add a link node if this link is targeted at at device */ + /* + * only add a link node if this link is targeted at a + * (GPU) device. + * If we ever activate it for an opencapi device, + * we'll need to revisit the link definitions + * hard-coded on ZZ + */ if (be32_to_cpu(link->usage) != SMP_LINK_USE_DEVICE) continue; diff --git a/platforms/ibm-fsp/zz.c b/platforms/ibm-fsp/zz.c index f44c618c..55078003 100644 --- a/platforms/ibm-fsp/zz.c +++ b/platforms/ibm-fsp/zz.c @@ -45,6 +45,97 @@ static const struct platform_ocapi zz_ocapi = { .odl_phy_swap = true, }; +#define NPU_BASE 0x5011000 +#define NPU_SIZE 0x2c +#define NPU_INDIRECT0 0x8000000009010c3f /* OB0 - no OB3 on ZZ */ + +static void create_link(struct dt_node *npu, int group, int index) +{ + struct dt_node *link; + uint32_t lane_mask; + char namebuf[32]; + + snprintf(namebuf, sizeof(namebuf), "link@%x", index); + link = dt_new(npu, namebuf); + + dt_add_property_string(link, "compatible", "ibm,npu-link"); + dt_add_property_cells(link, "ibm,npu-link-index", index); + + switch (index) { + case 2: + lane_mask = 0xf1e000; /* 0-3, 7-10 */ + break; + case 3: + lane_mask = 0x00078f; /* 13-16, 20-23 */ + break; + default: + assert(0); + } + + dt_add_property_u64s(link, "ibm,npu-phy", NPU_INDIRECT0); + dt_add_property_cells(link, "ibm,npu-lane-mask", lane_mask); + dt_add_property_cells(link, "ibm,npu-group-id", group); + dt_add_property_u64s(link, "ibm,link-speed", 25000000000ul); +} + +static void zz_opencapi_setup(void) +{ + struct dt_node *npu, *xscom; + int npu_index = 0; + int phb_index = 7; + struct dt_property *prop; + char namebuf[32]; + + /* + * In an ideal world, we should get all the NPU links + * information from HDAT. But after some effort, HDAT is still + * giving surprising information for opencapi. + * As a consequence: + * 1. the hdat parsing code in skiboot remains disabled (for + * opencapi) + * 2. we hard-code the NPU and links entries in the device + * tree. + * + * Getting the data from HDAT would have the advantage of + * providing the real link speed (20.0 vs. 25.78125 gbps), + * which is useful as there's one speed-dependent setting we + * need to do when initializing the NPU. Our hard coded + * definition assumes the higher speed and may need tuning in + * debug scenario using a lower link speed. + */ + snprintf(namebuf, sizeof(namebuf), "npu@%x", NPU_BASE); + dt_for_each_compatible(dt_root, xscom, "ibm,xscom") { + npu = dt_find_by_name(xscom, namebuf); + if (npu) { + /* + * our hdat parsing code may create NPU nodes + * with no links, so we just refresh some + * properties if needed + */ + prop = __dt_find_property(npu, "ibm,npu-index"); + if (prop) + dt_del_property(npu, prop); + prop = __dt_find_property(npu, "ibm,phb-index"); + if (prop) + dt_del_property(npu, prop); + prop = __dt_find_property(npu, "ibm,npu-links"); + if (prop) + dt_del_property(npu, prop); + } else { + npu = dt_new(xscom, namebuf); + dt_add_property_cells(npu, "reg", NPU_BASE, NPU_SIZE); + dt_add_property_strings(npu, "compatible", + "ibm,power9-npu"); + } + dt_add_property_cells(npu, "ibm,npu-index", npu_index++); + dt_add_property_cells(npu, "ibm,phb-index", phb_index++); + dt_add_property_cells(npu, "ibm,npu-links", 2); + + create_link(npu, 1, 2); + create_link(npu, 2, 3); + } +} + static bool zz_probe(void) { /* FIXME: make this neater when the dust settles */ @@ -54,8 +145,11 @@ static bool zz_probe(void) dt_node_is_compatible(dt_root, "ibm,zz-2s4u") || dt_node_is_compatible(dt_root, "ibm,zz-1s4u+gen4") || dt_node_is_compatible(dt_root, "ibm,zz-2s2u+gen4") || - dt_node_is_compatible(dt_root, "ibm,zz-2s4u+gen4")) + dt_node_is_compatible(dt_root, "ibm,zz-2s4u+gen4")) { + + zz_opencapi_setup(); return true; + } return false; }