From patchwork Mon Dec 18 07:07:51 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Donnellan X-Patchwork-Id: 849825 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3z0XKd0Spbz9sDB for ; Mon, 18 Dec 2017 18:12:21 +1100 (AEDT) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3z0XKc4YfSzDrXM for ; Mon, 18 Dec 2017 18:12:20 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=au1.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=andrew.donnellan@au1.ibm.com; receiver=) Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3z0XF73c9LzDrnF for ; Mon, 18 Dec 2017 18:08:27 +1100 (AEDT) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id vBI76uY4037151 for ; Mon, 18 Dec 2017 02:08:25 -0500 Received: from e06smtp14.uk.ibm.com (e06smtp14.uk.ibm.com [195.75.94.110]) by mx0a-001b2d01.pphosted.com with ESMTP id 2ex8yt02g3-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 18 Dec 2017 02:08:25 -0500 Received: from localhost by e06smtp14.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 18 Dec 2017 07:08:21 -0000 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp14.uk.ibm.com (192.168.101.144) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 18 Dec 2017 07:08:18 -0000 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id vBI78IVv40763552; Mon, 18 Dec 2017 07:08:18 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 96F3652049; Mon, 18 Dec 2017 06:01:18 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id E68215204B; Mon, 18 Dec 2017 06:01:17 +0000 (GMT) Received: from intelligence.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id B8B6FA0147; Mon, 18 Dec 2017 18:08:15 +1100 (AEDT) From: Andrew Donnellan To: skiboot@lists.ozlabs.org Date: Mon, 18 Dec 2017 18:07:51 +1100 X-Mailer: git-send-email 2.11.0 X-TM-AS-GCONF: 00 x-cbid: 17121807-0016-0000-0000-0000050E363D X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17121807-0017-0000-0000-0000284A58DA Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-12-18_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1712180093 Subject: [Skiboot] [PATCH 0/9] Initial OpenCAPI 3.0 Support for P9 X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.vnet.ibm.com, alistair@popple.id.au MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" This series implements OpenCAPI support for P9. The series is divided as follows: - Patches 1-3: general refactoring Add various structs and fields we'll need later. - Patches 4-5: setting up the NPU At present, we're doing NPU configuration separately from the existing NVLink code. At a later point, we intend to rework this to make it possible to support both NVLink and OpenCAPI links on the same NPU. - Patches 6-7: training links and creating OpenCAPI PHBs Unlike NVLink, which presents a single PHB to represent the entire NPU, we present a single PHB per device/link. This is necessary due to limitations in MMIO window allocation. Unfortunately this makes the structures we share with NVLink a little more complex, oh well. - Patch 8: OPAL API calls We define three new API calls for handling the Shared Process Area and setting OpenCAPI TL template capabilities. - Patch 9: platform support See below. Notable limitations: - We only support the zaius platform for now. We'll be adding ZZ after a bit more testing booting ZZs with hostboot rather than BML (probably in V2 of this series). Witherspoon support will come at a later point. - No mixing of OpenCAPI and NVLink devices on the same NPU. This will come in a later series. The only platform this currently impacts is Witherspoon. - No support for link ganging - there are also no OpenCAPI devices that support link ganging yet, so this will come when we get to it... - Link information is hardcoded per platform, and we don't have any form of presence detection apart from failing to train the link. Eventually, this will be detected via I2C once Hostboot adds the relevant link info to HDAT. - No LPC memory - this will come in a later patch once we've done a bit more testing internally. - No link reset functionality - this will come in a later patch. This series has been tested on a Zaius. I've also tested it on a GPU-equipped Witherspoon to ensure it doesn't break NVLink. Thanks to everyone who's helped us with this, especially to Alistair Popple for his advice on NVLink, and to the OpenCAPI hardware teams in Austin and Rochester who provided us with a lot of vital assistance. Comments welcome! Andrew Donnellan (8): npu2: Split out common helper functions into separate file npu2: Rework NPU data structures for OpenCAPI platform: Add fields for OpenCAPI platform data p9-adu: Implement update sequence for PowerBus Hotplug Mode Control npu2-opencapi: Configure NPU for OpenCAPI npu2-hw-procedures: Add support for OpenCAPI PHY link training npu2-opencapi: Train OpenCAPI links and setup devices platforms: Add OpenCAPI platform data and device tree nodes Frederic Barrat (1): npu2-opencapi: Add OpenCAPI OPAL API calls core/init.c | 2 +- core/platform.c | 13 +- doc/opal-api/opal-npu2-opencapi-159-160-161.rst | 126 +- hw/Makefile.inc | 3 +- hw/npu2-common.c | 98 +- hw/npu2-hw-procedures.c | 83 +- hw/npu2-opencapi.c | 1585 ++++++++++++++++- hw/npu2.c | 309 +-- hw/p9-adu.c | 182 ++- include/npu2-regs.h | 161 +- include/npu2.h | 90 +- include/opal-api.h | 5 +- include/p9-adu.h | 58 +- include/pci.h | 1 +- include/platform.h | 15 +- include/skiboot.h | 1 +- platforms/astbmc/zaius.c | 89 +- platforms/ibm-fsp/zz.c | 13 +- 18 files changed, 2620 insertions(+), 214 deletions(-) create mode 100644 doc/opal-api/opal-npu2-opencapi-159-160-161.rst create mode 100644 hw/npu2-common.c create mode 100644 hw/npu2-opencapi.c create mode 100644 hw/p9-adu.c create mode 100644 include/p9-adu.h base-commit: 70f14f4dd86eb0aa89806b5a30f4cf072f890143