From patchwork Thu Oct 14 15:56:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540983 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=KbM9JyN3; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYv70QzBz9sNH for ; Fri, 15 Oct 2021 02:57:22 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYv30k0Hz2yxx for ; Fri, 15 Oct 2021 02:57:19 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=KbM9JyN3; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=KbM9JyN3; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYtx4FRVz2yNl for ; Fri, 15 Oct 2021 02:57:12 +1100 (AEDT) Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EFhELa022287 for ; Thu, 14 Oct 2021 11:57:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=xn3hdrnDwZDcMZG180T6KoEnQ1cYXZyABhYYQeaIp20=; b=KbM9JyN3SHHJnuwNeRU1G0X5ANFYi21gm39hC/ldz3sgZUbYcpGSo+dML8DzsMh6CjLV ymsDnj3qOJB3AlyDiK25I+eMV2Xy+m1KvIyEeUAzG9IgKg8YvC4iky/Dykfye/qz2r7i L0h10z2ptQ6IkwSmJE3XZwSAk4z9YJyiDKEZ0MfGBhAibW3XUcDavynPXdp284nNBISA wPE/NvjrGsH6vcpRu9N2s0V036HW72TV9jmrYyETo5ca1I89xdRXoxTdm4atSI8fHFRH npCTP6MpPygtvNsROoYE9nO1P6h5HfSkZ+PoSBXhXTXu24VWKC+ItIT3TlfCwmyUahCZ 1w== Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 3bpm7be2hr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:09 -0400 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFXVDx029843 for ; Thu, 14 Oct 2021 15:57:07 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma03ams.nl.ibm.com with ESMTP id 3bk2qa6ysk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:07 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFpOxU63111524 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:51:24 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 33A8B11C058 for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1420411C04C for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:56:50 +0200 Message-Id: <20211014155704.46441-2-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: BErgTNtbGriwak7jyz95hBZ9o-fM2lia X-Proofpoint-ORIG-GUID: BErgTNtbGriwak7jyz95hBZ9o-fM2lia X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_09,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 suspectscore=0 adultscore=0 clxscore=1011 mlxscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 phishscore=0 impostorscore=0 bulkscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 01/15] npu2: move opal api X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Move the OPAL entry points for npu2 opencapi to the common opal NPU file. This prepares us to add same entries for PAU opencapi in this common file. No functional change. Signed-off-by: Christophe Lombard --- hw/npu-opal.c | 64 ++++++++++++++++++++++++++++++++++++++++++++++ hw/npu2-opencapi.c | 40 ++++------------------------- include/npu2.h | 7 +++++ 3 files changed, 76 insertions(+), 35 deletions(-) diff --git a/hw/npu-opal.c b/hw/npu-opal.c index c7f5f9f4..73158b1c 100644 --- a/hw/npu-opal.c +++ b/hw/npu-opal.c @@ -8,6 +8,8 @@ #include #include +#define TL_RATE_BUF_SIZE 32 + static int64_t opal_npu_init_context(uint64_t phb_id, int pid __unused, uint64_t msr, uint64_t bdf) { @@ -161,3 +163,65 @@ static int64_t opal_npu_get_relaxed_order(uint64_t phb_id, return phb4->ro_state; } opal_call(OPAL_NPU_GET_RELAXED_ORDER, opal_npu_get_relaxed_order, 2); + +#define MAX_PE_HANDLE ((1 << 15) - 1) + +static int64_t opal_npu_spa_setup(uint64_t phb_id, uint32_t bdfn, + uint64_t addr, uint64_t PE_mask) +{ + struct phb *phb = pci_get_phb(phb_id); + + if (!phb) + return OPAL_PARAMETER; + + /* 4k aligned */ + if (addr & 0xFFF) + return OPAL_PARAMETER; + + if (PE_mask > 15) + return OPAL_PARAMETER; + + if (phb->phb_type == phb_type_npu_v2_opencapi) + return npu2_opencapi_spa_setup(phb, bdfn, addr, PE_mask); + + return OPAL_PARAMETER; +} +opal_call(OPAL_NPU_SPA_SETUP, opal_npu_spa_setup, 4); + +static int64_t opal_npu_spa_clear_cache(uint64_t phb_id, uint32_t bdfn, + uint64_t PE_handle) +{ + struct phb *phb = pci_get_phb(phb_id); + + if (!phb) + return OPAL_PARAMETER; + + if (PE_handle > MAX_PE_HANDLE) + return OPAL_PARAMETER; + + if (phb->phb_type == phb_type_npu_v2_opencapi) + return npu2_opencapi_spa_clear_cache(phb, bdfn, PE_handle); + + return OPAL_PARAMETER; +} +opal_call(OPAL_NPU_SPA_CLEAR_CACHE, opal_npu_spa_clear_cache, 3); + +static int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t bdfn, + long capabilities, uint64_t rate_phys, int rate_sz) +{ + struct phb *phb = pci_get_phb(phb_id); + char *rate = (char *) rate_phys; + + if (!phb) + return OPAL_PARAMETER; + + if (!opal_addr_valid(rate) || rate_sz != TL_RATE_BUF_SIZE) + return OPAL_PARAMETER; + + if (phb->phb_type == phb_type_npu_v2_opencapi) + return npu2_opencapi_tl_set(phb, bdfn, capabilities, + rate); + + return OPAL_PARAMETER; +} +opal_call(OPAL_NPU_TL_SET, opal_npu_tl_set, 5); diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 035c6cdc..272f924b 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -41,7 +41,6 @@ #define NPU_IRQ_LEVELS_XSL 23 #define MAX_PE_HANDLE ((1 << 15) - 1) #define TL_MAX_TEMPLATE 63 -#define TL_RATE_BUF_SIZE 32 #define OCAPI_SLOT_NORMAL PCI_SLOT_STATE_NORMAL #define OCAPI_SLOT_LINK PCI_SLOT_STATE_LINK @@ -1957,24 +1956,13 @@ void npu2_opencapi_set_broken(struct npu2 *npu, int brick) } } -static int64_t opal_npu_spa_setup(uint64_t phb_id, uint32_t __unused bdfn, +int64_t npu2_opencapi_spa_setup(struct phb *phb, uint32_t __unused bdfn, uint64_t addr, uint64_t PE_mask) { uint64_t stack, block, offset, reg; - struct phb *phb = pci_get_phb(phb_id); struct npu2_dev *dev; int rc; - if (!phb || phb->phb_type != phb_type_npu_v2_opencapi) - return OPAL_PARAMETER; - - /* 4k aligned */ - if (addr & 0xFFF) - return OPAL_PARAMETER; - - if (PE_mask > 15) - return OPAL_PARAMETER; - dev = phb_to_npu2_dev_ocapi(phb); if (!dev) return OPAL_PARAMETER; @@ -1986,7 +1974,6 @@ static int64_t opal_npu_spa_setup(uint64_t phb_id, uint32_t __unused bdfn, else offset = NPU2_XSL_PSL_SPAP_A0; - lock(&dev->npu->lock); /* * set the SPAP used by the device @@ -2024,22 +2011,14 @@ out: unlock(&dev->npu->lock); return rc; } -opal_call(OPAL_NPU_SPA_SETUP, opal_npu_spa_setup, 4); -static int64_t opal_npu_spa_clear_cache(uint64_t phb_id, uint32_t __unused bdfn, - uint64_t PE_handle) +int64_t npu2_opencapi_spa_clear_cache(struct phb *phb, uint32_t __unused bdfn, + uint64_t PE_handle) { uint64_t cc_inv, stack, block, reg, rc; uint32_t retries = 5; - struct phb *phb = pci_get_phb(phb_id); struct npu2_dev *dev; - if (!phb || phb->phb_type != phb_type_npu_v2_opencapi) - return OPAL_PARAMETER; - - if (PE_handle > MAX_PE_HANDLE) - return OPAL_PARAMETER; - dev = phb_to_npu2_dev_ocapi(phb); if (!dev) return OPAL_PARAMETER; @@ -2077,7 +2056,6 @@ out: unlock(&dev->npu->lock); return rc; } -opal_call(OPAL_NPU_SPA_CLEAR_CACHE, opal_npu_spa_clear_cache, 3); static int get_template_rate(unsigned int templ, char *rate_buf) { @@ -2101,19 +2079,12 @@ static bool is_template_supported(unsigned int templ, long capabilities) return !!(capabilities & (1ull << templ)); } -static int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t __unused bdfn, - long capabilities, uint64_t rate_phys, int rate_sz) +int64_t npu2_opencapi_tl_set(struct phb *phb, uint32_t __unused bdfn, + long capabilities, char *rate) { - struct phb *phb = pci_get_phb(phb_id); struct npu2_dev *dev; uint64_t stack, block, reg, templ_rate; int i, rate_pos; - char *rate = (char *) rate_phys; - - if (!phb || phb->phb_type != phb_type_npu_v2_opencapi) - return OPAL_PARAMETER; - if (!opal_addr_valid(rate) || rate_sz != TL_RATE_BUF_SIZE) - return OPAL_PARAMETER; dev = phb_to_npu2_dev_ocapi(phb); if (!dev) @@ -2157,7 +2128,6 @@ static int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t __unused bdfn, OCAPIDBG(dev, "OTL configuration 1 register set to %llx\n", reg); return OPAL_SUCCESS; } -opal_call(OPAL_NPU_TL_SET, opal_npu_tl_set, 5); static void set_mem_bar(struct npu2_dev *dev, uint64_t base, uint64_t size) { diff --git a/include/npu2.h b/include/npu2.h index eb7c4558..23b06b4b 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -271,4 +271,11 @@ static inline int npu2_get_phb_index(unsigned int brick_index) return NPU2_PHB_INDEX_BASE + brick_index; } +int64_t npu2_opencapi_spa_setup(struct phb *phb, uint32_t __unused bdfn, + uint64_t addr, uint64_t PE_mask); +int64_t npu2_opencapi_spa_clear_cache(struct phb *phb, uint32_t __unused bdfn, + uint64_t PE_handle); +int64_t npu2_opencapi_tl_set(struct phb *phb, uint32_t __unused bdfn, + long capabilities, char *rate); + #endif /* __NPU2_H */ From patchwork Thu Oct 14 15:56:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540985 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=ELfQPh3S; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYvG5HjPz9sfG for ; Fri, 15 Oct 2021 02:57:30 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYvG4HYDz30Pj for ; Fri, 15 Oct 2021 02:57:30 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=ELfQPh3S; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=ELfQPh3S; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYtx54d6z2yPk for ; Fri, 15 Oct 2021 02:57:13 +1100 (AEDT) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EFScNJ001975 for ; Thu, 14 Oct 2021 11:57:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=HZRpsZEgSDBohQ2FyZ4a0ShqeE2iZPZMHfMzjTRglUg=; b=ELfQPh3S0j0ZrQF6b2K8ENgUIw1kZcBVPNSuFfj11e53Tc8GghJOMJrEktiTPMf5NGFM /Ks7I79a1PBo2qh1xEH4DXWmg4aZttl8jV6p3fFOYaxfrmnbhwF+hE2ACa2N5lpgc1Np tZtRmPBwGeYYFY/MYQuMl0Pn3MVojy5YjJlZrRWOUSbwK2ifyZLzTOjXyd9UPtwOa1XX ucHLKSly2dNesUHlCg6HQcglQf7wB3OiJGOvNpQK8oFnTo10nSDuHJXfiZ01MBN6wJqx HFu/P+rlYjjtDR1wqskhNJV/mZIwlWxAyDEET9SvmEWvN0psz/5tlQgs/YD2uI3EH9mk 5Q== Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0b-001b2d01.pphosted.com with ESMTP id 3bnprkde33-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:09 -0400 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFXgNQ009443 for ; Thu, 14 Oct 2021 15:57:08 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma02fra.de.ibm.com with ESMTP id 3bk2qac030-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:08 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFpPsp17957278 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:51:25 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 657F211C052 for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4241611C050 for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:56:51 +0200 Message-Id: <20211014155704.46441-3-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: iqzPwby5koBIEt0WAIEyPiz4LLKeUCqy X-Proofpoint-ORIG-GUID: iqzPwby5koBIEt0WAIEyPiz4LLKeUCqy X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_08,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0 adultscore=0 impostorscore=0 bulkscore=0 mlxlogscore=999 malwarescore=0 priorityscore=1501 spamscore=0 clxscore=1015 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 02/15] pau: introduce support X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" OpenCapi for P10 is included in the P10 chip. This requires OCAPI capable PHYs, Datalink Layer Logic and Transaction Layer Logic to be included. The PHYs are the physical connection to the OCAPI interconnect. The Datalink Layer provides link training. The Transaction Layer executes the cache coherent and data movement commands on the P10 chip. The PAU provides the Transaction Layer functionality for the OCAPI link(s) on the P10 chip. The P10 PAU supports two OCAPI links. Six accelerator units PAUs are instantiated on the P10 chip for a total of twelve OCAPI links. This patch adds PAU opencapi structure for supporting OpenCapi5. hw/pau.c file contains main of PAU management functions. Signed-off-by: Christophe Lombard --- core/init.c | 3 + hdata/spira.c | 31 +++++-- hdata/spira.h | 2 +- hw/Makefile.inc | 2 +- hw/pau.c | 224 +++++++++++++++++++++++++++++++++++++++++++++ include/pau-regs.h | 30 ++++++ include/pau.h | 94 +++++++++++++++++++ include/platform.h | 4 + include/skiboot.h | 1 + 9 files changed, 382 insertions(+), 9 deletions(-) create mode 100644 hw/pau.c create mode 100644 include/pau-regs.h create mode 100644 include/pau.h diff --git a/core/init.c b/core/init.c index 235f9055..056431ee 100644 --- a/core/init.c +++ b/core/init.c @@ -1371,6 +1371,9 @@ void __noreturn __nomcount main_cpu_entry(const void *fdt) probe_npu(); probe_npu2(); + /* Probe PAUs */ + probe_pau(); + /* Initialize PCI */ pci_init_slots(); diff --git a/hdata/spira.c b/hdata/spira.c index baa23751..1a351462 100644 --- a/hdata/spira.c +++ b/hdata/spira.c @@ -966,21 +966,38 @@ static void add_nx(void) static void add_nmmu(void) { struct dt_node *xscom, *nmmu; - u32 scom; + u32 scom1, scom2; + u32 chip_id; /* Nest MMU only exists on POWER9 or later */ if (proc_gen < proc_gen_p9) return; - if (proc_gen == proc_gen_p9) - scom = 0x5012c40; - else - scom = 0x2010c40; + if (proc_gen == proc_gen_p10) { + scom1 = 0x2010c40; + scom2 = 0x3010c40; + } else + scom1 = 0x5012c40; dt_for_each_compatible(dt_root, xscom, "ibm,xscom") { - nmmu = dt_new_addr(xscom, "nmmu", scom); + nmmu = dt_new_addr(xscom, "nmmu", scom1); dt_add_property_strings(nmmu, "compatible", "ibm,power9-nest-mmu"); - dt_add_property_cells(nmmu, "reg", scom, 0x20); + dt_add_property_cells(nmmu, "reg", scom1, 0x20); + + /* + * P10 has a second nMMU, a.k.a "south" nMMU. + * It exists only on P1 and P3 + */ + if (proc_gen == proc_gen_p10) { + + chip_id = __dt_get_chip_id(xscom); + if (chip_id != 2 && chip_id != 6) + continue; + + nmmu = dt_new_addr(xscom, "nmmu", scom2); + dt_add_property_strings(nmmu, "compatible", "ibm,power9-nest-mmu"); + dt_add_property_cells(nmmu, "reg", scom2, 0x20); + } } } diff --git a/hdata/spira.h b/hdata/spira.h index afdc9228..8def23bd 100644 --- a/hdata/spira.h +++ b/hdata/spira.h @@ -1152,7 +1152,7 @@ struct sppcrd_smp_link { __be16 pci_sideband_slot_idx; __be16 slca_idx; /* SLCA index of the *external* port */ - __be16 reserved; + __be16 opt_id; /* nvlink/ocapi detection devices */ __be32 i2c_link_cable; diff --git a/hw/Makefile.inc b/hw/Makefile.inc index c254fcbd..0b12f0be 100644 --- a/hw/Makefile.inc +++ b/hw/Makefile.inc @@ -8,7 +8,7 @@ HW_OBJS += dts.o lpc-rtc.o npu.o npu-hw-procedures.o xive.o phb4.o HW_OBJS += fake-nvram.o lpc-mbox.o npu2.o npu2-hw-procedures.o HW_OBJS += npu2-common.o npu2-opencapi.o phys-map.o sbe-p9.o capp.o HW_OBJS += occ-sensor.o vas.o sbe-p8.o dio-p9.o lpc-port80h.o cache-p9.o -HW_OBJS += npu-opal.o ocmb.o xive2.o +HW_OBJS += npu-opal.o ocmb.o xive2.o pau.o HW=hw/built-in.a include $(SRC)/hw/fsp/Makefile.inc diff --git a/hw/pau.c b/hw/pau.c new file mode 100644 index 00000000..63b69d3e --- /dev/null +++ b/hw/pau.c @@ -0,0 +1,224 @@ +/* SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later + * Copyright 2021 IBM Corp. + */ + +#include +#include +#include + +struct pau_dev *pau_next_dev(struct pau *pau, struct pau_dev *dev, + enum pau_dev_type type) +{ + uint32_t i = 0; + + if (dev) + i = dev->index + 1; + + for (; i < pau->links; i++) { + dev = &pau->devices[i]; + + if (dev->type == type || type == PAU_DEV_TYPE_ANY) + return dev; + } + + return NULL; +} + +static void pau_dt_create_link(struct dt_node *pau, uint32_t pau_index, + uint32_t dev_index) +{ + struct dt_node *link; + uint32_t phy_lane_mask = 0, pau_unit = 0; + uint32_t op_unit = 0, odl_index = 0; + + link = dt_new_addr(pau, "link", dev_index); + + dt_add_property_string(link, "compatible", "ibm,pau-link"); + dt_add_property_cells(link, "reg", dev_index); + dt_add_property_cells(link, "ibm,pau-link-index", dev_index); + + /* pau_index Interface Link - OPxA/B + * 0 OPT0 -- PAU0 + * OPT1 -- no PAU, SMP only + * OPT2 -- no PAU, SMP only + * 1 OPT3 -- PAU3 + * 2 OPT4 -- PAU4 by default, but can be muxed to use PAU5 + * 3 OPT5 -- PAU5 by default, but can be muxed to use PAU4 + * 4 OPT6 -- PAU6 by default, but can be muxed to use PAU7 + * 5 OPT7 -- PAU7 by default, but can be muxed to use PAU6 + */ + switch (pau_index) { + case 0: + /* OP0A - OP0B */ + pau_unit = 0; + op_unit = 0; + break; + case 1: + /* OP3A - OP3B */ + pau_unit = 3; + op_unit = 3; + break; + case 2: + /* OP4A - OP4B or OP5A - OP5B (TO DO) */ + pau_unit = 4; + op_unit = 4; + break; + case 3: + /* OP5A - OP5B or OP4A - OP4B (TO DO) */ + pau_unit = 5; + op_unit = 5; + break; + case 4: + /* OP6A - OP6B or OP7A - OP7B (TO DO) */ + pau_unit = 6; + op_unit = 6; + break; + case 5: + /* OP7A - OP7B or OP6A - OP6B (TO DO) */ + pau_unit = 7; + op_unit = 7; + break; + default: + return; + } + + /* ODL0 is hooked up to OTL0 */ + if (dev_index == 0) { + odl_index = 0; + phy_lane_mask = PPC_BITMASK32(0, 3); + phy_lane_mask |= PPC_BITMASK32(5, 8); + } else if (dev_index == 1) { + odl_index = 1; + phy_lane_mask = PPC_BITMASK32(9, 12); + phy_lane_mask |= PPC_BITMASK32(14, 17); + } + + dt_add_property_cells(link, "ibm,odl-index", odl_index); + dt_add_property_cells(link, "ibm,pau-unit", pau_unit); + dt_add_property_cells(link, "ibm,op-unit", op_unit); + dt_add_property_cells(link, "ibm,pau-lane-mask", phy_lane_mask); + dt_add_property_cells(link, "ibm,phb-index", pau_get_phb_index(pau_index, dev_index)); +} + +static void pau_dt_create_pau(struct dt_node *xscom, uint32_t pau_index) +{ + const uint32_t pau_base[] = { 0x10010800, 0x11010800, + 0x12010800, 0x12011000, + 0x13010800, 0x13011000}; + struct dt_node *pau; + uint32_t links; + + assert(pau_index < PAU_NBR); + pau = dt_new_addr(xscom, "pau", pau_base[pau_index]); + + dt_add_property_cells(pau, "#size-cells", 0); + dt_add_property_cells(pau, "#address-cells", 1); + dt_add_property_cells(pau, "reg", pau_base[pau_index], 0x2c); + dt_add_property_string(pau, "compatible", "ibm,power10-pau"); + dt_add_property_cells(pau, "ibm,pau-index", pau_index); + + links = PAU_LINKS_OPENCAPI_PER_PAU; + for (uint32_t i = 0; i < links; i++) + pau_dt_create_link(pau, pau_index, i); +} + +static bool pau_dt_create(void) +{ + struct dt_node *xscom; + + /* P10 chips only */ + if (proc_gen < proc_gen_p10) + return false; + + dt_for_each_compatible(dt_root, xscom, "ibm,xscom") + for (uint32_t i = 0; i < PAU_NBR; i++) + pau_dt_create_pau(xscom, i); + + return true; +} + +static struct pau *pau_create(struct dt_node *dn) +{ + struct pau *pau; + struct dt_node *link; + struct pau_dev *dev; + char *path; + uint32_t i; + + pau = zalloc(sizeof(*pau)); + assert(pau); + + init_lock(&pau->lock); + + pau->dt_node = dn; + pau->index = dt_prop_get_u32(dn, "ibm,pau-index"); + pau->xscom_base = dt_get_address(dn, 0, NULL); + + pau->chip_id = dt_get_chip_id(dn); + assert(get_chip(pau->chip_id)); + + pau->links = PAU_LINKS_OPENCAPI_PER_PAU; + dt_for_each_compatible(dn, link, "ibm,pau-link") { + i = dt_prop_get_u32(link, "ibm,pau-link-index"); + assert(i < PAU_LINKS_OPENCAPI_PER_PAU); + + dev = &pau->devices[i]; + dev->index = i; + dev->pau = pau; + dev->dn = link; + dev->odl_index = dt_prop_get_u32(link, "ibm,odl-index"); + dev->op_unit = dt_prop_get_u32(link, "ibm,op-unit"); + dev->phy_lane_mask = dt_prop_get_u32(link, "ibm,pau-lane-mask"); + }; + + path = dt_get_path(dn); + PAUINF(pau, "Found %s\n", path); + PAUINF(pau, "SCOM base: 0x%llx\n", pau->xscom_base); + free(path); + + return pau; +} + +static void pau_device_detect_fixup(struct pau_dev *dev) +{ + struct dt_node *dn = dev->dn; + + if (dev->type == PAU_DEV_TYPE_OPENCAPI) { + PAUDEVDBG(dev, "Link type opencapi\n"); + dt_add_property_strings(dn, "ibm,pau-link-type", "opencapi"); + return; + } + + PAUDEVDBG(dev, "Link type unknown\n"); + dt_add_property_strings(dn, "ibm,pau-link-type", "unknown"); +} + +static void pau_init(struct pau *pau) +{ + struct pau_dev *dev; + + platform.pau_device_detect(pau); + pau_for_each_dev(dev, pau) + pau_device_detect_fixup(dev); + +} + +void probe_pau(void) +{ + struct dt_node *dn; + struct pau *pau; + + /* This can be removed when/if we decide to use HDAT instead */ + if (!pau_dt_create()) + return; + + if (!platform.pau_device_detect) { + prlog(PR_INFO, "PAU: Platform does not support PAU\n"); + return; + } + + dt_for_each_compatible(dt_root, dn, "ibm,power10-pau") { + pau = pau_create(dn); + pau_init(pau); + } +} diff --git a/include/pau-regs.h b/include/pau-regs.h new file mode 100644 index 00000000..a35668f1 --- /dev/null +++ b/include/pau-regs.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later + * Copyright 2021 IBM Corp. + */ + +#ifndef __PAU_REGS_H +#define __PAU_REGS_H + +/* PAU FIR registers */ +#define PAU_FIR(n) (0x400 + (n) * 0x40) +#define PAU_FIR_MASK(n) (0x403 + (n) * 0x40) +#define PAU_FIR_ACTION0(n) (0x406 + (n) * 0x40) +#define PAU_FIR_ACTION1(n) (0x407 + (n) * 0x40) +#define PAU_FIR_MAX 3 + +/* PAU RING: Indirect address/data port */ +#define PAU_MISC_SCOM_IND_SCOM_ADDR 0x33e +#define PAU_MISC_DA_ADDR PPC_BITMASK(0, 23) +#define PAU_MISC_DA_LEN PPC_BITMASK(24, 25) +#define PAU_MISC_DA_LEN_4B 2 +#define PAU_MISC_DA_LEN_8B 3 +#define PAU_MISC_SCOM_IND_SCOM_DATA 0x33f + +/* PAU RING: Indirect register blocks */ +#define PAU_BLOCK(nib0, nib1) ((nib0) << 20 | (nib1) << 16) +#define PAU_REG_BLOCK(reg) ((reg) & 0xff0000) +#define PAU_REG_OFFSET(reg) ((reg) & 0xffff) + +#define PAU_BLOCK_CQ_SM(n) PAU_BLOCK(4, (n)) + +#endif /* __PAU_REGS_H */ diff --git a/include/pau.h b/include/pau.h new file mode 100644 index 00000000..2a26a65f --- /dev/null +++ b/include/pau.h @@ -0,0 +1,94 @@ +/* SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later + * Copyright 2021 IBM Corp. + */ + +#ifndef __PAU_H +#define __PAU_H + +#include +#include +#include +#include + +#define PAU_NBR 6 +#define PAU_LINKS_OPENCAPI_PER_PAU 2 + +enum pau_dev_type { + PAU_DEV_TYPE_UNKNOWN = 0, + PAU_DEV_TYPE_OPENCAPI, + PAU_DEV_TYPE_ANY = INT_MAX +}; + +struct pau_dev { + enum pau_dev_type type; + uint32_t index; + struct dt_node *dn; + + /* Associated PHY information */ + uint32_t pau_unit; /* 0,3,4,5,6,7 */ + uint32_t odl_index; + uint32_t op_unit; /* 0 -> 7 */ + uint32_t phy_lane_mask; + + struct pau *pau; +}; + +struct pau { + uint32_t index; + struct dt_node *dt_node; + uint32_t chip_id; + uint64_t xscom_base; + + /* Global MMIO window (all PAU regs) */ + uint64_t regs[2]; + + struct lock lock; + + uint32_t links; + struct pau_dev devices[PAU_LINKS_OPENCAPI_PER_PAU]; +}; + +#define PAUDBG(pau, fmt, a...) PAULOG(PR_DEBUG, pau, fmt, ##a) +#define PAUINF(pau, fmt, a...) PAULOG(PR_INFO, pau, fmt, ##a) +#define PAUERR(pau, fmt, a...) PAULOG(PR_ERR, pau, fmt, ##a) + +#define PAUDEVDBG(dev, fmt, a...) PAUDEVLOG(PR_DEBUG, dev, fmt, ##a) +#define PAUDEVINF(dev, fmt, a...) PAUDEVLOG(PR_INFO, dev, fmt, ##a) +#define PAUDEVERR(dev, fmt, a...) PAUDEVLOG(PR_ERR, dev, fmt, ##a) + +#define PAULOG(l, pau, fmt, a...) \ + prlog(l, "PAU[%d:%d]: " fmt, (pau)->chip_id, (pau)->index, ##a) + +#define PAUDEVLOG(l, dev, fmt, a...) \ + prlog(l, "PAU[%d:%d:%d]: " fmt, \ + (dev)->pau->chip_id, \ + (dev)->pau->index, \ + (dev)->index, ##a) + + +/* pau-scope index of the link */ +static inline uint32_t pau_dev_index(struct pau_dev *dev, int links) +{ + return dev->pau->index * links + dev->index; +} + +struct pau_dev *pau_next_dev(struct pau *pau, struct pau_dev *dev, + enum pau_dev_type type); + +#define pau_for_each_dev_type(dev, pau, type) \ + for (dev = NULL; (dev = pau_next_dev(pau, dev, type));) + +#define pau_for_each_opencapi_dev(dev, pau) \ + pau_for_each_dev_type(dev, pau, PAU_DEV_TYPE_OPENCAPI) + +#define pau_for_each_dev(dev, pau) \ + pau_for_each_dev_type(dev, pau, PAU_DEV_TYPE_ANY) + +#define PAU_PHB_INDEX_BASE 6 /* immediately after real PHBs */ +static inline int pau_get_phb_index(unsigned int pau_index, + unsigned int link_index) +{ + return PAU_PHB_INDEX_BASE + pau_index * 2 + link_index; +} + +#endif /* __PAU_H */ diff --git a/include/platform.h b/include/platform.h index 27a3afa0..6fafddbf 100644 --- a/include/platform.h +++ b/include/platform.h @@ -10,6 +10,7 @@ struct pci_device; struct pci_slot; struct errorlog; struct npu2; +struct pau; enum resource_id { RESOURCE_ID_KERNEL, @@ -126,6 +127,9 @@ struct platform { /* NPU device detection */ void (*npu2_device_detect)(struct npu2 *npu); + /* PAU device detection */ + void (*pau_device_detect)(struct pau *pau); + /* * Probe platform, return true on a match, called before * any allocation has been performed outside of the heap diff --git a/include/skiboot.h b/include/skiboot.h index 4f4a0057..d0db4cb7 100644 --- a/include/skiboot.h +++ b/include/skiboot.h @@ -214,6 +214,7 @@ extern int preload_capp_ucode(void); extern void preload_io_vpd(void); extern void probe_npu(void); extern void probe_npu2(void); +extern void probe_pau(void); extern void uart_init(void); extern void mbox_init(void); extern void early_uart_init(void); From patchwork Thu Oct 14 15:56:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540986 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=Ox477QWi; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYvN4qbXz9sNH for ; Fri, 15 Oct 2021 02:57:36 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYvN3JQ0z30Pj for ; Fri, 15 Oct 2021 02:57:36 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=Ox477QWi; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=Ox477QWi; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYtx4Ym9z2yP0 for ; Fri, 15 Oct 2021 02:57:12 +1100 (AEDT) Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EFtx3n020976 for ; Thu, 14 Oct 2021 11:57:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=qsV4W7HeT3Yea2NbTzV/szY1zWge3g1hhZWkw2ZXliU=; b=Ox477QWiGWAc17QTunOP2+vhV55xxxRcBtQuSYqFrL7/vJ7FTjqionaAR46+/4wNCOCV M8PtqDqwvW3BC4ohG364hdvibfJfwvHy14ET4dPk2rv5NJ5RWQZb/B4U1nsHo1zd2NeB ah8SWpxeu+OpZWX9yocH8QTiPvsGZNF4edYQE1Fh4fKnRXt5BiFVzwZTu2LUFqKAExff qLYFAHgUTqfz9X3h1enT5uXaKy/kYETw6/Wt1COSqRZeYdRwe0fnIOcsu6jqqBjl8JL5 CP8Rse3rw85+vmUjjigXKoO1COenwNvEGfDXW70KhpaFAUI+VwXq2KhZET7Sg8DwOmzr LQ== Received: from ppma06fra.de.ibm.com (48.49.7a9f.ip4.static.sl-reverse.com [159.122.73.72]) by mx0a-001b2d01.pphosted.com with ESMTP id 3bpp012v8f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:10 -0400 Received: from pps.filterd (ppma06fra.de.ibm.com [127.0.0.1]) by ppma06fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFXmfL000445 for ; Thu, 14 Oct 2021 15:57:08 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma06fra.de.ibm.com with ESMTP id 3bk2bk444r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:08 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFpP4264684398 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:51:25 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9A9E711C04A for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 746A411C05C for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:56:52 +0200 Message-Id: <20211014155704.46441-4-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: FXsgU57f7XqiU4mwnF4PbI0uOSqtV1Ci X-Proofpoint-GUID: FXsgU57f7XqiU4mwnF4PbI0uOSqtV1Ci X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_08,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 bulkscore=0 adultscore=0 mlxscore=0 priorityscore=1501 phishscore=0 clxscore=1015 impostorscore=0 mlxlogscore=999 spamscore=0 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 03/15] rainier: detect pau devices X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Update the platform_ocapi structure to store Rainier platform-specific values for detecting and resetting OpenCAPI devices via the module I2C (PCA9553) The unique number I2C bus ID associated to each OpenCapi device is get from the I2C port and engine. (De)Assert a reset and detect an OpenCapi device is available through the I2C bus id and address. Signed-off-by: Christophe Lombard --- include/pau.h | 3 + include/platform.h | 5 + platforms/astbmc/rainier.c | 239 +++++++++++++++++++++++++++++++++++++ 3 files changed, 247 insertions(+) diff --git a/include/pau.h b/include/pau.h index 2a26a65f..e946e0fd 100644 --- a/include/pau.h +++ b/include/pau.h @@ -24,6 +24,9 @@ struct pau_dev { uint32_t index; struct dt_node *dn; + /* Associated I2C information */ + uint8_t i2c_bus_id; + /* Associated PHY information */ uint32_t pau_unit; /* 0,3,4,5,6,7 */ uint32_t odl_index; diff --git a/include/platform.h b/include/platform.h index 6fafddbf..db0c0867 100644 --- a/include/platform.h +++ b/include/platform.h @@ -69,6 +69,11 @@ struct platform_ocapi { uint8_t i2c_presence_brick5; /* I2C pin to read for presence on brick 5 */ bool odl_phy_swap; /* Swap ODL1 to use brick 2 rather than * brick 1 lanes */ + uint8_t i2c_dev_addr; /* I2C device address */ + uint8_t i2c_intreset_pin; /* I2C pin to write to reset */ + uint8_t i2c_predetect_pin; /* I2C pin to read for presence */ + int64_t (*i2c_assert_reset)(uint8_t i2c_bus_id); + int64_t (*i2c_deassert_reset)(uint8_t i2c_bus_id); const char *(*ocapi_slot_label)(uint32_t chip_id, uint32_t brick_index); const struct ocapi_phy_setup *phy_setup; }; diff --git a/platforms/astbmc/rainier.c b/platforms/astbmc/rainier.c index 17d9fe2b..3e21e1b9 100644 --- a/platforms/astbmc/rainier.c +++ b/platforms/astbmc/rainier.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -99,6 +100,227 @@ static void rainier_init_slot_power(void) } } +static int64_t rainier_i2c_assert_reset(uint8_t i2c_bus_id) +{ + uint8_t data; + int64_t rc = OPAL_SUCCESS; + + /* + * Set the i2c reset pin in output mode (9553 device) + * To write a register: + * puti2c pu 0 0|1 C4 1, + * with data being a 2-nibble hex value and offset being the + * register offset from the datasheet + * + * puti2c (-p1) 0 0|1 C4 51 5 1 0 : i2c engine + * 0|1 : i2c_port + * C4 (C4 > 1 = 62) : Address + * 51 : data + * 5 : register (offset) + * 1 : offset byte + * + * 7.3.6 LS0 - LED selector register: default value 0x55 + * bit 1:0 01* LED0 selected (OpenCapi card) + * + * offset 0x05, register name: LS0, Fct: LED selector + * see Table 4. Control register definition (PCA9553) + */ + data = 0x51; + rc = i2c_request_send(i2c_bus_id, + platform.ocapi->i2c_dev_addr, + SMBUS_WRITE, 0x5, 1, + &data, sizeof(data), 120); + + return rc; +} + +static int64_t rainier_i2c_deassert_reset(uint8_t i2c_bus_id) +{ + uint8_t data; + int64_t rc = OPAL_SUCCESS; + + /* puti2c (-p1) 0 0|1 C4 55 1 + * + * offset 0x05, register name: LS0, Fct: LED selector + * see Table 4. Control register definition (PCA9553) + */ + data = 0x55; + rc = i2c_request_send(i2c_bus_id, + platform.ocapi->i2c_dev_addr, + SMBUS_WRITE, 0x5, 1, + &data, sizeof(data), 120); + + return rc; +} + +static int get_i2c_info(struct pau_dev *dev, int *engine, int *port) +{ + uint32_t chip_id = dev->pau->chip_id; + uint32_t pau_index = dev->pau->index; + uint32_t link = dev->index; + + switch (chip_id) { + case 0: + case 4: + /* + * OP3: links 0 and 1 on chip 0 + * link 0 only on chip 4 + */ + if (pau_index == 1) { + if (link == 1 && chip_id == 4) + return -1; + *engine = 1; + *port = link; + return 0; + } + break; + case 2: + case 6: + /* + * OP0: links 0 and 1 on chip 2 + * link 1 only on chip 6 + */ + if (pau_index == 0) { + if (link == 0 && chip_id == 6) + return -1; + *engine = 1; + *port = link; + return 0; + } + break; + } + return -1; +} + +static void rainier_i2c_presence_init(struct pau_dev *dev) +{ + char port_name[17]; + struct dt_node *np; + int engine, port; + + /* Find I2C port */ + if (dev->i2c_bus_id) + return; + + if (get_i2c_info(dev, &engine, &port)) + return; + + snprintf(port_name, sizeof(port_name), "p8_%08x_e%dp%d", + dev->pau->chip_id, engine, port); + + dt_for_each_compatible(dt_root, np, "ibm,power10-i2c-port") { + if (streq(port_name, dt_prop_get(np, "ibm,port-name"))) { + dev->i2c_bus_id = dt_prop_get_u32(np, "ibm,opal-id"); + break; + } + } +} + +static int64_t rainier_i2c_dev_detect(struct pau_dev *dev, + bool *presence) +{ + int64_t rc = OPAL_SUCCESS; + uint8_t detect; + + /* Read the presence value + * geti2c (-p1) pu 0 0|1 C4 1 1 + * + * offset 0x00, register name: INPUT, Fct: input register + * see Table 4. Control register definition (PCA9553) + */ + detect = 0x00; + *presence = false; + rc = i2c_request_send(dev->i2c_bus_id, + platform.ocapi->i2c_dev_addr, + SMBUS_READ, 0x00, 1, + &detect, 1, 120); + + /* LED0 (bit 0): a high level no card is plugged */ + if (!rc && !(detect & platform.ocapi->i2c_predetect_pin)) + *presence = true; + + return rc; +} + +static void rainier_pau_device_detect(struct pau *pau) +{ + struct pau_dev *dev; + bool presence; + int64_t rc; + + /* OpenCapi devices are possibly connected on Optical link pair: + * OP0 or OP3 + * pau_index Interface Link - OPxA/B + * 0 OPT0 -- PAU0 + * OPT1 -- no PAU, SMP only + * OPT2 -- no PAU, SMP only + * 1 OPT3 -- PAU3 + * 2 OPT4 -- PAU4 by default, but can be muxed to use PAU5 - N/A on Rainier + * 3 OPT5 -- PAU5 by default, but can be muxed to use PAU4 - N/A on Rainier + * 4 OPT6 -- PAU6 by default, but can be muxed to use PAU7 - N/A on Rainier + * 5 OPT7 -- PAU7 by default, but can be muxed to use PAU6 - N/A on Rainier + */ + pau_for_each_dev(dev, pau) { + dev->type = PAU_DEV_TYPE_UNKNOWN; + + rainier_i2c_presence_init(dev); + if (dev->i2c_bus_id) { + rc = rainier_i2c_dev_detect(dev, &presence); + if (!rc && presence) + dev->type = PAU_DEV_TYPE_OPENCAPI; + } + + dt_add_property_u64(dev->dn, "ibm,link-speed", 25000000000ull); + } +} + +static void rainier_pau_create_i2c_bus(void) +{ + struct dt_node *xscom, *i2cm, *i2c_bus; + + prlog(PR_DEBUG, "PLAT: Adding I2C bus device node for PAU reset\n"); + dt_for_each_compatible(dt_root, xscom, "ibm,xscom") { + i2cm = dt_find_by_name(xscom, "i2cm@a1000"); + if (!i2cm) { + prlog(PR_DEBUG, "PLAT: Adding master @a1000\n"); + i2cm = dt_new(xscom, "i2cm@a1000"); + dt_add_property_cells(i2cm, "reg", 0xa1000, 0x1000); + dt_add_property_strings(i2cm, "compatible", + "ibm,power8-i2cm", "ibm,power9-i2cm"); + dt_add_property_cells(i2cm, "#size-cells", 0x0); + dt_add_property_cells(i2cm, "#address-cells", 0x1); + dt_add_property_cells(i2cm, "chip-engine#", 0x1); + dt_add_property_cells(i2cm, "clock-frequency", 0x7735940); + } + + i2c_bus = dt_find_by_name(i2cm, "i2c-bus@0"); + if (!i2c_bus) { + prlog(PR_DEBUG, "PLAT: Adding bus 0 to master @a1000\n"); + i2c_bus = dt_new_addr(i2cm, "i2c-bus", 0); + dt_add_property_cells(i2c_bus, "reg", 0); + dt_add_property_cells(i2c_bus, "bus-frequency", 0x61a80); + dt_add_property_strings(i2c_bus, "compatible", + "ibm,opal-i2c", + "ibm,power8-i2c-port", + "ibm,power9-i2c-port", + "ibm,power10-i2c-port"); + } + + i2c_bus = dt_find_by_name(i2cm, "i2c-bus@1"); + if (!i2c_bus) { + prlog(PR_DEBUG, "PLAT: Adding bus 1 to master @a1000\n"); + i2c_bus = dt_new_addr(i2cm, "i2c-bus", 1); + dt_add_property_cells(i2c_bus, "reg", 1); + dt_add_property_cells(i2c_bus, "bus-frequency", 0x61a80); + dt_add_property_strings(i2c_bus, "compatible", + "ibm,opal-i2c", + "ibm,power8-i2c-port", + "ibm,power9-i2c-port", + "ibm,power10-i2c-port"); + } + } +} + static void rainier_init(void) { astbmc_init(); @@ -118,9 +340,24 @@ static bool rainier_probe(void) /* Setup UART for use by OPAL (Linux hvc) */ uart_set_console_policy(UART_CONSOLE_OPAL); + /* create i2c entries for PAU */ + rainier_pau_create_i2c_bus(); + return true; } +static struct platform_ocapi rainier_ocapi = { + .i2c_dev_addr = 0x62, /* C4 >> 1 */ + .i2c_intreset_pin = 0x02, /* PIN 2 - LED1 - INT/RESET */ + .i2c_predetect_pin = 0x01, /* PIN 1 - LED0 - PRE-DETECT */ + /* As previously for NPU/NPU2, we use indirect functions for + * this platform to reset the device. This makes the code more + * generic in PAU. + */ + .i2c_assert_reset = rainier_i2c_assert_reset, + .i2c_deassert_reset = rainier_i2c_deassert_reset, +}; + DECLARE_PLATFORM(rainier) = { .name = "Rainier", .probe = rainier_probe, @@ -131,6 +368,8 @@ DECLARE_PLATFORM(rainier) = { .cec_power_down = astbmc_ipmi_power_down, .cec_reboot = astbmc_ipmi_reboot, .elog_commit = ipmi_elog_commit, + .pau_device_detect = rainier_pau_device_detect, + .ocapi = &rainier_ocapi, .exit = astbmc_exit, .terminate = ipmi_terminate, }; From patchwork Thu Oct 14 15:56:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540984 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=V9gG3NhF; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYv960kfz9sNH for ; Fri, 15 Oct 2021 02:57:25 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYv94Zjmz307W for ; Fri, 15 Oct 2021 02:57:25 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=V9gG3NhF; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=V9gG3NhF; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYtx466Tz2yNY for ; Fri, 15 Oct 2021 02:57:12 +1100 (AEDT) Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EE5BD4033007 for ; Thu, 14 Oct 2021 11:57:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=hAdzI8iqPvS+OW0CECOBA92pE699lzOcya1WctZob7I=; b=V9gG3NhFaQSbNoK4dm2y3sHlIpuWgUdCCXRDMP66UBjwUGUm5/FiaE5TFDOPT5dwhLmA Q/ip7jmZp/YGHfOEa6ZVRrMT0XUxflQsBoW3ccCI8Of4DkRiEGZP4Cr6Q65hHf9yS1dv Da+OAGRGyL5g1KMKAb+l5DXxkUcUtRnJrazuIMLOXYIBP7DbaqhD42iZ7RWYE2d762xf t9hycS2TYgKVY4WoJivw+8OfiVY9b0p1RSpY5FLeDdSWckFaFUsQtFz7yszRikIpatp9 ISieOEW3nTCi5BEEqsDBTeIR8sgSXjWQH5ICB/ntzBvjoSLfWSQtSot5e8UrUaAjslyD 3g== Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com with ESMTP id 3bpm7be2j4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:10 -0400 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFW7Cx016525 for ; Thu, 14 Oct 2021 15:57:08 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma06ams.nl.ibm.com with ESMTP id 3bk2bjy3vg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:08 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFpPs664684400 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:51:25 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CE9B611C04A for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A988D11C05B for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:56:53 +0200 Message-Id: <20211014155704.46441-5-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: ISjAscO_XZrmtK910FtmAbK9v9o69vGw X-Proofpoint-ORIG-GUID: ISjAscO_XZrmtK910FtmAbK9v9o69vGw X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_09,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 suspectscore=0 adultscore=0 clxscore=1015 mlxscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 phishscore=0 impostorscore=0 bulkscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 04/15] pau: assign bars X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Configure early PAU Global MMIO BAR registers to allow PAU MMIO register accesses. This is done for each PAU. Enable the Powerbus interface is mandatory for MMIO accesses. For each OpenCAPI device, configure the bar registers to access to the AFU MMIO and to the AFU Config Addr/Data registers. AFU Config/Data registers = GENID_ADDR (from phy_map file) + 320K (= 0x50000) Signed-off-by: Christophe Lombard Reviewed-by: Frederic Barrat --- hw/pau.c | 75 +++++++++++++++++++++++++++++++++++++++++++ hw/phys-map.c | 59 +++++++++++++++++++++++++++++----- include/pau-regs.h | 26 +++++++++++++++ include/pau.h | 79 ++++++++++++++++++++++++++++++++++++++++++++++ include/phys-map.h | 4 +++ 5 files changed, 235 insertions(+), 8 deletions(-) diff --git a/hw/pau.c b/hw/pau.c index 63b69d3e..5d4b1574 100644 --- a/hw/pau.c +++ b/hw/pau.c @@ -193,6 +193,80 @@ static void pau_device_detect_fixup(struct pau_dev *dev) dt_add_property_strings(dn, "ibm,pau-link-type", "unknown"); } +static void pau_opencapi_assign_bars(struct pau *pau) +{ + struct pau_dev *dev; + uint64_t addr, size, val; + + /* Global MMIO bar (per pau) + * 16M aligned address -> 0x1000000 (bit 24) + */ + phys_map_get(pau->chip_id, PAU_REGS, pau->index, &addr, &size); + val = SETFIELD(PAU_MMIO_BAR_ADDR, 0ull, addr >> 24); + val |= PAU_MMIO_BAR_ENABLE; + pau_write(pau, PAU_MMIO_BAR, val); + + PAUINF(pau, "MMIO base: 0x%016llx (%lldMB)\n", addr, size >> 20); + pau->regs[0] = addr; + pau->regs[1] = size; + + /* NTL bar (per device) + * 64K aligned address -> 0x10000 (bit 16) + */ + pau_for_each_dev(dev, pau) { + if (dev->type == PAU_DEV_TYPE_UNKNOWN) + continue; + + phys_map_get(pau->chip_id, PAU_OCAPI_MMIO, + pau_dev_index(dev, PAU_LINKS_OPENCAPI_PER_PAU), + &addr, &size); + + val = SETFIELD(PAU_NTL_BAR_ADDR, 0ull, addr >> 16); + val = SETFIELD(PAU_NTL_BAR_SIZE, val, ilog2(size >> 16)); + pau_write(pau, PAU_NTL_BAR(dev->index), val); + + val = SETFIELD(PAU_CTL_MISC_MMIOPA_CONFIG_BAR_ADDR, 0ull, addr >> 16); + val = SETFIELD(PAU_CTL_MISC_MMIOPA_CONFIG_BAR_SIZE, val, ilog2(size >> 16)); + pau_write(pau, PAU_CTL_MISC_MMIOPA_CONFIG(dev->index), val); + + dev->ntl_bar.addr = addr; + dev->ntl_bar.size = size; + } + + /* GENID bar (logically divided per device) + * 512K aligned address -> 0x80000 (bit 19) + */ + phys_map_get(pau->chip_id, PAU_GENID, pau->index, &addr, &size); + val = SETFIELD(PAU_GENID_BAR_ADDR, 0ull, addr >> 19); + pau_write(pau, PAU_GENID_BAR, val); + + pau_for_each_dev(dev, pau) { + if (dev->type == PAU_DEV_TYPE_UNKNOWN) + continue; + + dev->genid_bar.size = size; + /* +320K = Bricks 0-4 Config Addr/Data registers */ + dev->genid_bar.cfg = addr + 0x50000; + } +} + +static void pau_opencapi_init_hw(struct pau *pau) +{ + pau_opencapi_assign_bars(pau); +} + +static void pau_opencapi_init(struct pau *pau) +{ + if (!pau_next_dev(pau, NULL, PAU_DEV_TYPE_OPENCAPI)) + return; + + assert(platform.ocapi); + + pau_opencapi_init_hw(pau); + + disable_fast_reboot("OpenCAPI device enabled"); +} + static void pau_init(struct pau *pau) { struct pau_dev *dev; @@ -201,6 +275,7 @@ static void pau_init(struct pau *pau) pau_for_each_dev(dev, pau) pau_device_detect_fixup(dev); + pau_opencapi_init(pau); } void probe_pau(void) diff --git a/hw/phys-map.c b/hw/phys-map.c index d6ff99fd..711d626e 100644 --- a/hw/phys-map.c +++ b/hw/phys-map.c @@ -82,8 +82,51 @@ static const struct phys_map_entry phys_map_table_p10[] = { { VAS_HYP_WIN , 0, 0x00060302fe000000ull, 0x0000000002000000ull }, { VAS_USER_WIN , 0, 0x0006030300000000ull, 0x0000000100000000ull }, - /* TODO: MC, OCMB, PAU */ - { RESV , 8, 0x0006030400000000ull, 0x000000f800000000ull }, + /* TODO: MC, OCMB */ + { RESV , 8, 0x0006030400000000ull, 0x0000008400000000ull }, + { PAU_OCAPI_MMIO , 0, 0x0006038800000000ull, 0x0000000800000000ull }, + { PAU_OCAPI_MMIO , 1, 0x0006039000000000ull, 0x0000000800000000ull }, + { PAU_OCAPI_MMIO , 2, 0x0006039800000000ull, 0x0000000800000000ull }, + { PAU_OCAPI_MMIO , 3, 0x000603a000000000ull, 0x0000000800000000ull }, + { PAU_OCAPI_MMIO , 4, 0x000603a800000000ull, 0x0000000800000000ull }, + { PAU_OCAPI_MMIO , 5, 0x000603b000000000ull, 0x0000000800000000ull }, + { PAU_OCAPI_MMIO , 6, 0x000603b800000000ull, 0x0000000800000000ull }, + { PAU_OCAPI_MMIO , 7, 0x000603c000000000ull, 0x0000000800000000ull }, + { PAU_OCAPI_MMIO , 8, 0x000603c800000000ull, 0x0000000800000000ull }, + { PAU_OCAPI_MMIO , 9, 0x000603d000000000ull, 0x0000000800000000ull }, + { PAU_OCAPI_MMIO ,10, 0x000603d800000000ull, 0x0000000800000000ull }, + { PAU_OCAPI_MMIO ,11, 0x000603e000000000ull, 0x0000000800000000ull }, + { PAU_REGS , 0, 0x000603e800000000ull, 0x0000000001000000ull }, + { PAU_REGS , 1, 0x000603e801000000ull, 0x0000000001000000ull }, + { PAU_REGS , 2, 0x000603e802000000ull, 0x0000000001000000ull }, + { PAU_REGS , 3, 0x000603e803000000ull, 0x0000000001000000ull }, + { PAU_REGS , 4, 0x000603e804000000ull, 0x0000000001000000ull }, + { PAU_REGS , 5, 0x000603e805000000ull, 0x0000000001000000ull }, + { RESV , 9, 0x000603e806000000ull, 0x0000000000040000ull }, + { PAU_NTL , 0, 0x000603e806040000ull, 0x0000000000020000ull }, + { PAU_NTL , 1, 0x000603e806060000ull, 0x0000000000020000ull }, + { PAU_GENID , 0, 0x000603e806080000ull, 0x0000000000080000ull }, + { RESV ,10, 0x000603e806100000ull, 0x0000000000040000ull }, + { PAU_NTL , 2, 0x000603e806140000ull, 0x0000000000020000ull }, + { PAU_NTL , 3, 0x000603e806160000ull, 0x0000000000020000ull }, + { PAU_GENID , 1, 0x000603e806180000ull, 0x0000000000080000ull }, + { RESV ,11, 0x000603e806200000ull, 0x0000000000040000ull }, + { PAU_NTL , 4, 0x000603e806240000ull, 0x0000000000020000ull }, + { PAU_NTL , 5, 0x000603e806260000ull, 0x0000000000020000ull }, + { PAU_GENID , 2, 0x000603e806280000ull, 0x0000000000080000ull }, + { RESV ,12, 0x000603e806300000ull, 0x0000000000040000ull }, + { PAU_NTL , 6, 0x000603e806340000ull, 0x0000000000020000ull }, + { PAU_NTL , 7, 0x000603e806360000ull, 0x0000000000020000ull }, + { PAU_GENID , 3, 0x000603e806380000ull, 0x0000000000080000ull }, + { RESV ,13, 0x000603e806400000ull, 0x0000000000040000ull }, + { PAU_NTL , 8, 0x000603e806440000ull, 0x0000000000020000ull }, + { PAU_NTL , 9, 0x000603e806460000ull, 0x0000000000020000ull }, + { PAU_GENID , 4, 0x000603e806480000ull, 0x0000000000080000ull }, + { RESV ,14, 0x000603e806500000ull, 0x0000000000040000ull }, + { PAU_NTL ,10, 0x000603e806540000ull, 0x0000000000020000ull }, + { PAU_NTL ,11, 0x000603e806560000ull, 0x0000000000020000ull }, + { PAU_GENID , 5, 0x000603e806580000ull, 0x0000000000080000ull }, + { RESV ,15, 0x000603e806600000ull, 0x00000013F9A00000ull }, { XSCOM , 0, 0x000603fc00000000ull, 0x0000000400000000ull }, /* 4 TB offset */ @@ -96,10 +139,10 @@ static const struct phys_map_entry phys_map_table_p10[] = { { XIVE_END , 0, 0x0006060000000000ull, 0x0000020000000000ull }, /* 8 - 13 TB offset */ - { RESV , 9, 0x0006080000000000ull, 0x0000060000000000ull }, + { RESV ,16, 0x0006080000000000ull, 0x0000060000000000ull }, /* 14 TB offset */ - { RESV ,10, 0x00060e0000000000ull, 0x0000008000000000ull }, + { RESV ,17, 0x00060e0000000000ull, 0x0000008000000000ull }, { NULL_MAP, 0, 0, 0 }, }; @@ -130,10 +173,10 @@ static const struct phys_map_entry phys_map_table_nimbus[] = { * * We don't currently support >4TB ranges. */ - { OCAPI_MEM, 0, 0x0002000000000000ull, 0x0000040000000000ull }, - { OCAPI_MEM, 1, 0x0002800000000000ull, 0x0000040000000000ull }, - { OCAPI_MEM, 2, 0x0003000000000000ull, 0x0000040000000000ull }, - { OCAPI_MEM, 3, 0x0003800000000000ull, 0x0000040000000000ull }, + { OCAPI_MEM, 0, 0x0002000000000000ull, 0x0000040000000000ull }, + { OCAPI_MEM, 1, 0x0002800000000000ull, 0x0000040000000000ull }, + { OCAPI_MEM, 2, 0x0003000000000000ull, 0x0000040000000000ull }, + { OCAPI_MEM, 3, 0x0003800000000000ull, 0x0000040000000000ull }, /* 0 TB offset @ MMIO 0x0006000000000000ull */ { PHB4_64BIT_MMIO, 0, 0x0006000000000000ull, 0x0000004000000000ull }, diff --git a/include/pau-regs.h b/include/pau-regs.h index a35668f1..afe6f958 100644 --- a/include/pau-regs.h +++ b/include/pau-regs.h @@ -26,5 +26,31 @@ #define PAU_REG_OFFSET(reg) ((reg) & 0xffff) #define PAU_BLOCK_CQ_SM(n) PAU_BLOCK(4, (n)) +#define PAU_BLOCK_CQ_CTL PAU_BLOCK(4, 4) + +/* + * CQ_SM block registers + * + * Definitions here use PAU_BLOCK_CQ_SM(0), but when pau_write() is given + * one of these, it will do corresponding writes to every CQ_SM block. + */ +#define PAU_MCP_MISC_CFG0 (PAU_BLOCK_CQ_SM(0) + 0x000) +#define PAU_MCP_MISC_CFG0_MA_MCRESP_OPT_WRP PPC_BIT(9) +#define PAU_MCP_MISC_CFG0_ENABLE_PBUS PPC_BIT(26) +#define PAU_SNP_MISC_CFG0 (PAU_BLOCK_CQ_SM(0) + 0x180) +#define PAU_SNP_MISC_CFG0_ENABLE_PBUS PPC_BIT(2) +#define PAU_NTL_BAR(brk) (PAU_BLOCK_CQ_SM(0) + 0x1b8 + (brk) * 8) +#define PAU_NTL_BAR_ADDR PPC_BITMASK(3, 35) +#define PAU_NTL_BAR_SIZE PPC_BITMASK(39, 43) +#define PAU_MMIO_BAR (PAU_BLOCK_CQ_SM(0) + 0x1e0) +#define PAU_MMIO_BAR_ENABLE PPC_BIT(0) +#define PAU_MMIO_BAR_ADDR PPC_BITMASK(3, 27) +#define PAU_GENID_BAR (PAU_BLOCK_CQ_SM(0) + 0x1e8) +#define PAU_GENID_BAR_ADDR PPC_BITMASK(3, 32) + +/* CQ_CTL block registers */ +#define PAU_CTL_MISC_MMIOPA_CONFIG(brk) (PAU_BLOCK_CQ_CTL + 0x098 + (brk) * 8) +#define PAU_CTL_MISC_MMIOPA_CONFIG_BAR_ADDR PPC_BITMASK(1, 35) +#define PAU_CTL_MISC_MMIOPA_CONFIG_BAR_SIZE PPC_BITMASK(39, 43) #endif /* __PAU_REGS_H */ diff --git a/include/pau.h b/include/pau.h index e946e0fd..be8ed26a 100644 --- a/include/pau.h +++ b/include/pau.h @@ -19,11 +19,21 @@ enum pau_dev_type { PAU_DEV_TYPE_ANY = INT_MAX }; +/* Used to expose a hardware BAR (or logical slice of it) outside skiboot */ +struct pau_bar { + uint64_t addr; + uint64_t size; + uint64_t cfg; +}; + struct pau_dev { enum pau_dev_type type; uint32_t index; struct dt_node *dn; + struct pau_bar ntl_bar; + struct pau_bar genid_bar; + /* Associated I2C information */ uint8_t i2c_bus_id; @@ -44,6 +54,7 @@ struct pau { /* Global MMIO window (all PAU regs) */ uint64_t regs[2]; + bool mmio_access; struct lock lock; @@ -94,4 +105,72 @@ static inline int pau_get_phb_index(unsigned int pau_index, return PAU_PHB_INDEX_BASE + pau_index * 2 + link_index; } +/* + * We use the indirect method because it uses the same addresses as + * the MMIO offsets (PAU RING) + */ +static inline void pau_scom_sel(struct pau *pau, uint64_t reg, + uint64_t size) +{ + uint64_t val; + + val = SETFIELD(PAU_MISC_DA_ADDR, 0ull, reg); + val = SETFIELD(PAU_MISC_DA_LEN, val, size); + xscom_write(pau->chip_id, + pau->xscom_base + PAU_MISC_SCOM_IND_SCOM_ADDR, + val); +} + +static inline void pau_scom_write(struct pau *pau, uint64_t reg, + uint64_t size, + uint64_t val) +{ + pau_scom_sel(pau, reg, size); + xscom_write(pau->chip_id, + pau->xscom_base + PAU_MISC_SCOM_IND_SCOM_DATA, + val); +} + +static inline uint64_t pau_scom_read(struct pau *pau, uint64_t reg, + uint64_t size) +{ + uint64_t val; + + pau_scom_sel(pau, reg, size); + xscom_read(pau->chip_id, + pau->xscom_base + PAU_MISC_SCOM_IND_SCOM_DATA, + &val); + + return val; +} + +static inline void pau_write(struct pau *pau, uint64_t reg, + uint64_t val) +{ + void *mmio = (void *)pau->regs[0]; + + if (pau->mmio_access) + out_be64(mmio + reg, val); + else + pau_scom_write(pau, reg, PAU_MISC_DA_LEN_8B, val); + + /* CQ_SM writes should be mirrored in all four blocks */ + if (PAU_REG_BLOCK(reg) != PAU_BLOCK_CQ_SM(0)) + return; + + for (uint32_t i = 1; i < 4; i++) + pau_write(pau, PAU_BLOCK_CQ_SM(i) + PAU_REG_OFFSET(reg), + val); +} + +static inline uint64_t pau_read(struct pau *pau, uint64_t reg) +{ + void *mmio = (void *)pau->regs[0]; + + if (pau->mmio_access) + return in_be64(mmio + reg); + + return pau_scom_read(pau, reg, PAU_MISC_DA_LEN_8B); +} + #endif /* __PAU_H */ diff --git a/include/phys-map.h b/include/phys-map.h index 1dd337a5..a53bcd04 100644 --- a/include/phys-map.h +++ b/include/phys-map.h @@ -51,6 +51,10 @@ enum phys_map_type { XIVE_NVPG, XIVE_ESB, XIVE_END, + PAU_OCAPI_MMIO, + PAU_REGS, + PAU_GENID, + PAU_NTL, }; extern void phys_map_get(uint64_t gcid, enum phys_map_type type, From patchwork Thu Oct 14 15:56:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540997 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=tltCCc6o; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYx11b0zz9sNH for ; Fri, 15 Oct 2021 02:59:01 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYx105J6z3cCL for ; Fri, 15 Oct 2021 02:59:01 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=tltCCc6o; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=tltCCc6o; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYv43Rn5z305Q for ; Fri, 15 Oct 2021 02:57:20 +1100 (AEDT) Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EFmIso017492 for ; Thu, 14 Oct 2021 11:57:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=pp1; bh=MUOKdJYKYnvb0nskGoGGbSR+sYzAYMBGFXR6NhwdcDY=; b=tltCCc6ol3OLl3+Q0tcBxHNUyPtzIqoY2JmbCO+Nxuz5+2ZlnwzbBaN8R+bPBUDYotU1 3bdXuZ/HLozcYVcAKwhzpcz26WFQ4I156V1CtzpXubkpVOY8cuQpX47f1OX0uhZ8rEuV F0ibH9q09raaQxE0l3yLWDdLWvKOLZls/PtP0STuBYwC2S1ohegTmUN16pv3GkC238o5 W7zu1OgS1zGyq0MGT73Qq2rXYMt6RNFnngLNf3yp1YP+xJrUtw4XkCfQrsdHm1IlVORL 6HBrYIvELPOCrZDHCtDuZpZ5YXbbgoOU8DEdNmA+tr6GOH39gCD+vMlIX7HiceC6g3CI mA== Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0a-001b2d01.pphosted.com with ESMTP id 3bnqmqa4bx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:11 -0400 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFXmv5009937 for ; Thu, 14 Oct 2021 15:57:08 GMT Received: from b06avi18626390.portsmouth.uk.ibm.com (b06avi18626390.portsmouth.uk.ibm.com [9.149.26.192]) by ppma02fra.de.ibm.com with ESMTP id 3bk2qac031-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:08 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFpS9f55837006 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:51:28 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 035F711C050 for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DDE8311C05E for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:05 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:56:54 +0200 Message-Id: <20211014155704.46441-6-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: SDkfWe2Q9jX6GQX1SAgqUepinXP5fPFr X-Proofpoint-ORIG-GUID: SDkfWe2Q9jX6GQX1SAgqUepinXP5fPFr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_08,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 priorityscore=1501 bulkscore=0 impostorscore=0 lowpriorityscore=0 phishscore=0 clxscore=1015 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 05/15] pau: create phb X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Implement the necessary operations for the OpenCAPI PHB type and inform the device-tree properties associated. The OpenCapi PCI config Addr/Data registers are reachable through the Generation-ID Registers MMIO BARS. The Config Address and Data registers are located at the following offsets from the AFU Config BAR plus 320 KB. • Config Address for Brick 0 – Offset 0 • Config Data for Brick 0 – Offsets: ◦ 128 – 4-byte config register • Config Address for Brick 1 – Offset 256 • Config Data for Brick 1 – Offsets: ◦ 384 – 4-byte config register Signed-off-by: Christophe Lombard Reviewed-by: Frederic Barrat --- core/pci-opal.c | 9 +- core/pci.c | 4 +- hw/pau.c | 234 ++++++++++++++++++++++++++++++++++++++++++++- include/pau-regs.h | 8 ++ include/pau.h | 13 +++ include/pci.h | 1 + 6 files changed, 264 insertions(+), 5 deletions(-) diff --git a/core/pci-opal.c b/core/pci-opal.c index aa375c6a..acbcd2a5 100644 --- a/core/pci-opal.c +++ b/core/pci-opal.c @@ -748,7 +748,8 @@ static void rescan_slot_devices(struct pci_slot *slot) * prepare_link_change() is called (if needed) by the state * machine during the slot reset or link polling */ - if (phb->phb_type != phb_type_npu_v2_opencapi) { + if ((phb->phb_type != phb_type_npu_v2_opencapi) && + (phb->phb_type != phb_type_pau_opencapi)) { pci_scan_bus(phb, pd->secondary_bus, pd->subordinate_bus, &pd->children, pd, true); pci_add_device_nodes(phb, &pd->children, pd->dn, @@ -766,7 +767,8 @@ static void remove_slot_devices(struct pci_slot *slot) struct phb *phb = slot->phb; struct pci_device *pd = slot->pd; - if (phb->phb_type != phb_type_npu_v2_opencapi) + if ((phb->phb_type != phb_type_npu_v2_opencapi) && + (phb->phb_type != phb_type_pau_opencapi)) pci_remove_bus(phb, &pd->children); else pci_remove_bus(phb, &phb->devices); @@ -817,7 +819,8 @@ static bool training_needed(struct pci_slot *slot) struct pci_device *pd = slot->pd; /* only for opencapi slots for now */ - if (!pd && phb->phb_type == phb_type_npu_v2_opencapi) + if (!pd && ((phb->phb_type == phb_type_npu_v2_opencapi) || + (phb->phb_type == phb_type_pau_opencapi))) return true; return false; } diff --git a/core/pci.c b/core/pci.c index e195ecbf..0a146c83 100644 --- a/core/pci.c +++ b/core/pci.c @@ -1517,7 +1517,9 @@ static void __noinline pci_add_one_device_node(struct phb *phb, * device has a 4KB config space. It's got nothing to do with the * standard Type 0/1 config spaces defined by PCI. */ - if (is_pcie || phb->phb_type == phb_type_npu_v2_opencapi) { + if (is_pcie || + (phb->phb_type == phb_type_npu_v2_opencapi) || + (phb->phb_type == phb_type_pau_opencapi)) { snprintf(compat, MAX_NAME, "pciex%x,%x", PCI_VENDOR_ID(pd->vdid), PCI_DEVICE_ID(pd->vdid)); dt_add_property_cells(np, "ibm,pci-config-space-type", 1); diff --git a/hw/pau.c b/hw/pau.c index 5d4b1574..1a370a97 100644 --- a/hw/pau.c +++ b/hw/pau.c @@ -2,12 +2,18 @@ * Copyright 2021 IBM Corp. */ +#include +#include #include #include #include +/* Number of PEs supported */ +#define PAU_MAX_PE_NUM 16 +#define PAU_RESERVED_PE_NUM 15 + struct pau_dev *pau_next_dev(struct pau *pau, struct pau_dev *dev, - enum pau_dev_type type) + enum pau_dev_type type) { uint32_t i = 0; @@ -250,9 +256,235 @@ static void pau_opencapi_assign_bars(struct pau *pau) } } +static void pau_opencapi_create_phb_slot(struct pau_dev *dev) +{ + struct pci_slot *slot; + + slot = pci_slot_alloc(&dev->phb, NULL); + if (!slot) { + /** + * @fwts-label OCAPICannotCreatePHBSlot + * @fwts-advice Firmware probably ran out of memory creating + * PAU slot. OpenCAPI functionality could be broken. + */ + PAUDEVERR(dev, "Cannot create PHB slot\n"); + } +} + +static int64_t pau_opencapi_pcicfg_check(struct pau_dev *dev, + uint32_t offset, + uint32_t size) +{ + if (!dev || offset > 0xfff || (offset & (size - 1))) + return OPAL_PARAMETER; + + return OPAL_SUCCESS; +} + +static int64_t pau_opencapi_pcicfg_read(struct phb *phb, uint32_t bdfn, + uint32_t offset, uint32_t size, + void *data) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(phb); + uint64_t cfg_addr, genid_base; + int64_t rc; + + rc = pau_opencapi_pcicfg_check(dev, offset, size); + if (rc) + return rc; + + /* Config Address for Brick 0 – Offset 0 + * Config Address for Brick 1 – Offset 256 + */ + genid_base = dev->genid_bar.cfg + (dev->index << 8); + + cfg_addr = PAU_CTL_MISC_CFG_ADDR_ENABLE; + cfg_addr = SETFIELD(PAU_CTL_MISC_CFG_ADDR_BUS_NBR | + PAU_CTL_MISC_CFG_ADDR_DEVICE_NBR | + PAU_CTL_MISC_CFG_ADDR_FUNCTION_NBR, + cfg_addr, bdfn); + cfg_addr = SETFIELD(PAU_CTL_MISC_CFG_ADDR_REGISTER_NBR, + cfg_addr, offset & ~3u); + + out_be64((uint64_t *)genid_base, cfg_addr); + sync(); + + switch (size) { + case 1: + *((uint8_t *)data) = + in_8((uint8_t *)(genid_base + 128 + (offset & 3))); + break; + case 2: + *((uint16_t *)data) = + in_le16((uint16_t *)(genid_base + 128 + (offset & 2))); + break; + case 4: + *((uint32_t *)data) = in_le32((uint32_t *)(genid_base + 128)); + break; + default: + return OPAL_PARAMETER; + } + + return OPAL_SUCCESS; +} + +#define PAU_OPENCAPI_PCI_CFG_READ(size, type) \ +static int64_t pau_opencapi_pcicfg_read##size(struct phb *phb, uint32_t bdfn, \ + uint32_t offset, type * data) \ +{ \ + /* Initialize data in case of error */ \ + *data = (type)0xffffffff; \ + return pau_opencapi_pcicfg_read(phb, bdfn, offset, sizeof(type), data); \ +} + +static int64_t pau_opencapi_pcicfg_write(struct phb *phb, uint32_t bdfn, + uint32_t offset, uint32_t size, + uint32_t data) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(phb); + uint64_t genid_base, cfg_addr; + int64_t rc; + + rc = pau_opencapi_pcicfg_check(dev, offset, size); + if (rc) + return rc; + + /* Config Address for Brick 0 – Offset 0 + * Config Address for Brick 1 – Offset 256 + */ + genid_base = dev->genid_bar.cfg + (dev->index << 8); + + cfg_addr = PAU_CTL_MISC_CFG_ADDR_ENABLE; + cfg_addr = SETFIELD(PAU_CTL_MISC_CFG_ADDR_BUS_NBR | + PAU_CTL_MISC_CFG_ADDR_DEVICE_NBR | + PAU_CTL_MISC_CFG_ADDR_FUNCTION_NBR, + cfg_addr, bdfn); + cfg_addr = SETFIELD(PAU_CTL_MISC_CFG_ADDR_REGISTER_NBR, + cfg_addr, offset & ~3u); + + out_be64((uint64_t *)genid_base, cfg_addr); + sync(); + + switch (size) { + case 1: + out_8((uint8_t *)(genid_base + 128 + (offset & 3)), data); + break; + case 2: + out_le16((uint16_t *)(genid_base + 128 + (offset & 2)), data); + break; + case 4: + out_le32((uint32_t *)(genid_base + 128), data); + break; + default: + return OPAL_PARAMETER; + } + + return OPAL_SUCCESS; +} + +#define PAU_OPENCAPI_PCI_CFG_WRITE(size, type) \ +static int64_t pau_opencapi_pcicfg_write##size(struct phb *phb, uint32_t bdfn, \ + uint32_t offset, type data) \ +{ \ + return pau_opencapi_pcicfg_write(phb, bdfn, offset, sizeof(type), data);\ +} + +PAU_OPENCAPI_PCI_CFG_READ(8, u8) +PAU_OPENCAPI_PCI_CFG_READ(16, u16) +PAU_OPENCAPI_PCI_CFG_READ(32, u32) +PAU_OPENCAPI_PCI_CFG_WRITE(8, u8) +PAU_OPENCAPI_PCI_CFG_WRITE(16, u16) +PAU_OPENCAPI_PCI_CFG_WRITE(32, u32) + +static const struct phb_ops pau_opencapi_ops = { + .cfg_read8 = pau_opencapi_pcicfg_read8, + .cfg_read16 = pau_opencapi_pcicfg_read16, + .cfg_read32 = pau_opencapi_pcicfg_read32, + .cfg_write8 = pau_opencapi_pcicfg_write8, + .cfg_write16 = pau_opencapi_pcicfg_write16, + .cfg_write32 = pau_opencapi_pcicfg_write32, +}; + +static void pau_opencapi_create_phb(struct pau_dev *dev) +{ + struct phb *phb = &dev->phb; + uint64_t mm_win[2]; + + mm_win[0] = dev->ntl_bar.addr; + mm_win[1] = dev->ntl_bar.size; + + phb->phb_type = phb_type_pau_opencapi; + phb->scan_map = 0; + + phb->ops = &pau_opencapi_ops; + phb->dt_node = dt_new_addr(dt_root, "pciex", mm_win[0]); + assert(phb->dt_node); + + pci_register_phb(phb, pau_get_opal_id(dev->pau->chip_id, + pau_get_phb_index(dev->pau->index, dev->index))); + pau_opencapi_create_phb_slot(dev); +} + +static void pau_opencapi_dt_add_mmio_window(struct pau_dev *dev) +{ + struct dt_node *dn = dev->phb.dt_node; + uint64_t mm_win[2]; + + mm_win[0] = dev->ntl_bar.addr; + mm_win[1] = dev->ntl_bar.size; + PAUDEVDBG(dev, "Setting AFU MMIO window to %016llx %016llx\n", + mm_win[0], mm_win[1]); + + dt_add_property(dn, "reg", mm_win, sizeof(mm_win)); + dt_add_property(dn, "ibm,mmio-window", mm_win, sizeof(mm_win)); + dt_add_property_cells(dn, "ranges", 0x02000000, + hi32(mm_win[0]), lo32(mm_win[0]), + hi32(mm_win[0]), lo32(mm_win[0]), + hi32(mm_win[1]), lo32(mm_win[1])); +} + +static void pau_opencapi_dt_add_props(struct pau_dev *dev) +{ + struct dt_node *dn = dev->phb.dt_node; + struct pau *pau = dev->pau; + + dt_add_property_strings(dn, + "compatible", + "ibm,power10-pau-opencapi-pciex", + "ibm,ioda3-pau-opencapi-phb", + "ibm,ioda2-npu2-opencapi-phb"); + + dt_add_property_cells(dn, "#address-cells", 3); + dt_add_property_cells(dn, "#size-cells", 2); + dt_add_property_cells(dn, "#interrupt-cells", 1); + dt_add_property_cells(dn, "bus-range", 0, 0xff); + dt_add_property_cells(dn, "clock-frequency", 0x200, 0); + dt_add_property_cells(dn, "interrupt-parent", get_ics_phandle()); + + dt_add_property_strings(dn, "device_type", "pciex"); + dt_add_property_cells(dn, "ibm,pau-index", pau->index); + dt_add_property_cells(dn, "ibm,chip-id", pau->chip_id); + dt_add_property_cells(dn, "ibm,xscom-base", pau->xscom_base); + dt_add_property_cells(dn, "ibm,npcq", pau->dt_node->phandle); + dt_add_property_cells(dn, "ibm,links", 1); + dt_add_property_cells(dn, "ibm,phb-diag-data-size", 0); + dt_add_property_cells(dn, "ibm,opal-num-pes", PAU_MAX_PE_NUM); + dt_add_property_cells(dn, "ibm,opal-reserved-pe", PAU_RESERVED_PE_NUM); + + pau_opencapi_dt_add_mmio_window(dev); +} + static void pau_opencapi_init_hw(struct pau *pau) { + struct pau_dev *dev = NULL; + pau_opencapi_assign_bars(pau); + + /* Create phb */ + pau_for_each_opencapi_dev(dev, pau) { + pau_opencapi_create_phb(dev); + pau_opencapi_dt_add_props(dev); + } } static void pau_opencapi_init(struct pau *pau) diff --git a/include/pau-regs.h b/include/pau-regs.h index afe6f958..57796920 100644 --- a/include/pau-regs.h +++ b/include/pau-regs.h @@ -52,5 +52,13 @@ #define PAU_CTL_MISC_MMIOPA_CONFIG(brk) (PAU_BLOCK_CQ_CTL + 0x098 + (brk) * 8) #define PAU_CTL_MISC_MMIOPA_CONFIG_BAR_ADDR PPC_BITMASK(1, 35) #define PAU_CTL_MISC_MMIOPA_CONFIG_BAR_SIZE PPC_BITMASK(39, 43) +#define PAU_CTL_MISC_CFG_ADDR(brk) (PAU_BLOCK_CQ_CTL + 0x250 + (brk) * 8) +#define PAU_CTL_MISC_CFG_ADDR_ENABLE PPC_BIT(0) +#define PAU_CTL_MISC_CFG_ADDR_STATUS PPC_BITMASK(1, 3) +#define PAU_CTL_MISC_CFG_ADDR_BUS_NBR PPC_BITMASK(4, 11) +#define PAU_CTL_MISC_CFG_ADDR_DEVICE_NBR PPC_BITMASK(12, 16) +#define PAU_CTL_MISC_CFG_ADDR_FUNCTION_NBR PPC_BITMASK(17, 19) +#define PAU_CTL_MISC_CFG_ADDR_REGISTER_NBR PPC_BITMASK(20, 31) +#define PAU_CTL_MISC_CFG_ADDR_TYPE PPC_BIT(32) #endif /* __PAU_REGS_H */ diff --git a/include/pau.h b/include/pau.h index be8ed26a..fdf85f85 100644 --- a/include/pau.h +++ b/include/pau.h @@ -8,6 +8,7 @@ #include #include #include +#include #include #define PAU_NBR 6 @@ -30,6 +31,7 @@ struct pau_dev { enum pau_dev_type type; uint32_t index; struct dt_node *dn; + struct phb phb; struct pau_bar ntl_bar; struct pau_bar genid_bar; @@ -86,6 +88,12 @@ static inline uint32_t pau_dev_index(struct pau_dev *dev, int links) return dev->pau->index * links + dev->index; } +static inline struct pau_dev *pau_phb_to_opencapi_dev(struct phb *phb) +{ + assert(phb->phb_type == phb_type_pau_opencapi); + return container_of(phb, struct pau_dev, phb); +} + struct pau_dev *pau_next_dev(struct pau *pau, struct pau_dev *dev, enum pau_dev_type type); @@ -105,6 +113,11 @@ static inline int pau_get_phb_index(unsigned int pau_index, return PAU_PHB_INDEX_BASE + pau_index * 2 + link_index; } +static inline int pau_get_opal_id(unsigned int chip_id, unsigned int index) +{ + return phb4_get_opal_id(chip_id, index); +} + /* * We use the indirect method because it uses the same addresses as * the MMIO offsets (PAU RING) diff --git a/include/pci.h b/include/pci.h index 8d467213..caae7443 100644 --- a/include/pci.h +++ b/include/pci.h @@ -352,6 +352,7 @@ enum phb_type { phb_type_pcie_v4, phb_type_npu_v2, phb_type_npu_v2_opencapi, + phb_type_pau_opencapi, }; /* Generic PCI NVRAM flags */ From patchwork Thu Oct 14 15:56:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540993 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=bWJzjyqZ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYwQ3HWrz9sNH for ; Fri, 15 Oct 2021 02:58:30 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYwQ04xSz30RC for ; Fri, 15 Oct 2021 02:58:30 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=bWJzjyqZ; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=bWJzjyqZ; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYtz5GDwz2ynX for ; Fri, 15 Oct 2021 02:57:15 +1100 (AEDT) Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EF4Lnb024839 for ; Thu, 14 Oct 2021 11:57:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=qjX1hCeUJ0c8LPEwna1t8W0F1AjxV8D7zIq61XsZLf8=; b=bWJzjyqZPezcNdbGYzBX36EYixRlqrIwg6rwnGb3nkg/AYzcO7nhZNpncPfTx7WM2w4z MhP03hdGYw8WKqpl8S2S+bpMl9rb7khbcOz2HNGifQiwQSLoe68biE7TD5ESnwxyjCtI our+wN5P24DFG7yrHt2AkMrcvjZ17gGEN7geXcaEP9K8rqBO3jQQByFSWoekkvRjmIOA zMhkua7lvHCFbReBgJFv2CWb7yb7MW78RS/ooqQbFoeUpVy6X+69M3qrduR6qCu4KRnr VPMa7RGCLvpyG57UoyYyRffF+vjr2eHt9qk/DnzQwHaBNLAvwkOETtJush3XdWGQ8rsc 8g== Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com with ESMTP id 3bnqmqa4bs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:10 -0400 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFW6ft016511 for ; Thu, 14 Oct 2021 15:57:08 GMT Received: from b06avi18626390.portsmouth.uk.ibm.com (b06avi18626390.portsmouth.uk.ibm.com [9.149.26.192]) by ppma06ams.nl.ibm.com with ESMTP id 3bk2bjy3vm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:08 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFpSVM61669644 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:51:28 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3FE7F11C04C for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1CAC811C04A for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:56:55 +0200 Message-Id: <20211014155704.46441-7-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: WOYHudkIq3h4pVfjzTSZV9ZXHSButC0g X-Proofpoint-ORIG-GUID: WOYHudkIq3h4pVfjzTSZV9ZXHSButC0g X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_08,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 priorityscore=1501 bulkscore=0 impostorscore=0 lowpriorityscore=0 phishscore=0 clxscore=1015 mlxscore=0 adultscore=0 spamscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 06/15] pau: enabling opencapi X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Enable OpenCAPI mode for each brick which are connected to be used in this mode. This is be done through 7 steps as described in the P10 OCAPI 5.0 Processing Unit Workbook document, section: 17.1.3.1 Enabling OpenCAPI. The following sequences must be performed: 1. Set Transport MUX controls to select OpenCAPI 2. Enable Clocks in XSL 3. Enable Clocks in MISC 4. Set NPCQ configuration 5. Enable XSL-XTS Interfaces 6. Enable State-machine allocation Enabling the NTL/GENID BARS allows to access to the MMIO registers. Signed-off-by: Christophe Lombard Reviewed-by: Frederic Barrat --- hw/pau.c | 222 +++++++++++++++++++++++++++++++++++++++++++++ include/pau-regs.h | 43 +++++++++ include/pau.h | 1 + 3 files changed, 266 insertions(+) diff --git a/hw/pau.c b/hw/pau.c index 1a370a97..5f181011 100644 --- a/hw/pau.c +++ b/hw/pau.c @@ -199,6 +199,42 @@ static void pau_device_detect_fixup(struct pau_dev *dev) dt_add_property_strings(dn, "ibm,pau-link-type", "unknown"); } +#define CQ_CTL_STATUS_TIMEOUT 10 /* milliseconds */ + +static int pau_opencapi_set_fence_control(struct pau_dev *dev, + uint8_t state_requested) +{ + uint64_t timeout = mftb() + msecs_to_tb(CQ_CTL_STATUS_TIMEOUT); + uint8_t status; + struct pau *pau = dev->pau; + uint64_t reg, val; + + reg = PAU_CTL_MISC_FENCE_CTRL(dev->index); + val = pau_read(pau, reg); + val = SETFIELD(PAU_CTL_MISC_FENCE_REQUEST, val, state_requested); + pau_write(pau, reg, val); + + /* Wait for fence status to update */ + do { + reg = PAU_CTL_MISC_STATUS(dev->index); + val = pau_read(pau, reg); + status = GETFIELD(PAU_CTL_MISC_STATUS_AM_FENCED(dev->index), val); + if (status == state_requested) + return OPAL_SUCCESS; + time_wait_ms(1); + } while (tb_compare(mftb(), timeout) == TB_ABEFOREB); + + /* + * @fwts-label OCAPIFenceStatusTimeout + * @fwts-advice The PAU fence status did not update as expected. This + * could be the result of a firmware or hardware bug. OpenCAPI + * functionality could be broken. + */ + PAUDEVERR(dev, "Bad fence status: expected 0x%x, got 0x%x\n", + state_requested, status); + return OPAL_HARDWARE; +} + static void pau_opencapi_assign_bars(struct pau *pau) { struct pau_dev *dev; @@ -256,6 +292,37 @@ static void pau_opencapi_assign_bars(struct pau *pau) } } +static void pau_opencapi_enable_bars(struct pau_dev *dev, bool enable) +{ + struct pau *pau = dev->pau; + uint64_t reg, val; + + if (dev->ntl_bar.enable == enable) /* No state change */ + return; + + dev->ntl_bar.enable = enable; + dev->genid_bar.enable = enable; + + reg = PAU_NTL_BAR(dev->index); + val = pau_read(pau, reg); + val = SETFIELD(PAU_NTL_BAR_ENABLE, val, enable); + pau_write(pau, reg, val); + + /* + * Generation IDs are a single space in the hardware but we split them + * per device. Only disable in hardware if every device has disabled. + */ + if (!enable) + pau_for_each_dev(dev, pau) + if (dev->genid_bar.enable) + return; + + reg = PAU_GENID_BAR; + val = pau_read(pau, reg); + val = SETFIELD(PAU_GENID_BAR_ENABLE, val, enable); + pau_write(pau, reg, val); +} + static void pau_opencapi_create_phb_slot(struct pau_dev *dev) { struct pci_slot *slot; @@ -474,6 +541,135 @@ static void pau_opencapi_dt_add_props(struct pau_dev *dev) pau_opencapi_dt_add_mmio_window(dev); } +static void pau_opencapi_set_transport_mux_controls(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint32_t typemap = 0; + uint64_t reg, val = 0; + + PAUDEVDBG(dev, "Setting transport mux controls\n"); + typemap = 0x2 >> dev->index; + + reg = PAU_MISC_OPTICAL_IO_CONFIG; + val = pau_read(pau, reg); + typemap |= GETFIELD(PAU_MISC_OPTICAL_IO_CONFIG_OTL, val); + val = SETFIELD(PAU_MISC_OPTICAL_IO_CONFIG_OTL, val, typemap); + pau_write(pau, reg, val); +} + +static void pau_opencapi_enable_xsl_clocks(struct pau *pau) +{ + uint64_t reg, val; + + PAUDBG(pau, "Enable clocks in XSL\n"); + + reg = PAU_XSL_WRAP_CFG; + val = pau_read(pau, reg); + val |= PAU_XSL_WRAP_CFG_CLOCK_ENABLE; + pau_write(pau, reg, val); +} + +static void pau_opencapi_enable_misc_clocks(struct pau *pau) +{ + uint64_t reg, val; + + PAUDBG(pau, "Enable clocks in MISC\n"); + + /* clear any spurious NDL stall or no_stall_c_err_rpts */ + reg = PAU_MISC_HOLD; + val = pau_read(pau, reg); + val = SETFIELD(PAU_MISC_HOLD_NDL_STALL, val, 0b0000); + pau_write(pau, reg, val); + + reg = PAU_MISC_CONFIG; + val = pau_read(pau, reg); + val |= PAU_MISC_CONFIG_OC_MODE; + pau_write(pau, reg, val); +} + +static void pau_opencapi_set_npcq_config(struct pau *pau) +{ + struct pau_dev *dev; + uint8_t oc_typemap = 0; + uint64_t reg, val; + + /* MCP_MISC_CFG0 + * SNP_MISC_CFG0 done in pau_opencapi_enable_pb + */ + pau_for_each_opencapi_dev(dev, pau) + oc_typemap |= 0x10 >> dev->index; + + PAUDBG(pau, "Set NPCQ Config\n"); + reg = PAU_CTL_MISC_CFG2; + val = pau_read(pau, reg); + val = SETFIELD(PAU_CTL_MISC_CFG2_OCAPI_MODE, val, oc_typemap); + val = SETFIELD(PAU_CTL_MISC_CFG2_OCAPI_4, val, oc_typemap); + val = SETFIELD(PAU_CTL_MISC_CFG2_OCAPI_C2, val, oc_typemap); + val = SETFIELD(PAU_CTL_MISC_CFG2_OCAPI_AMO, val, oc_typemap); + val = SETFIELD(PAU_CTL_MISC_CFG2_OCAPI_MEM_OS_BIT, val, oc_typemap); + pau_write(pau, reg, val); + + reg = PAU_DAT_MISC_CFG1; + val = pau_read(pau, reg); + val = SETFIELD(PAU_DAT_MISC_CFG1_OCAPI_MODE, val, oc_typemap); + pau_write(pau, reg, val); +} + +static void pau_opencapi_enable_xsl_xts_interfaces(struct pau *pau) +{ + uint64_t reg, val; + + PAUDBG(pau, "Enable XSL-XTS Interfaces\n"); + reg = PAU_XTS_CFG; + val = pau_read(pau, reg); + val |= PAU_XTS_CFG_OPENCAPI; + pau_write(pau, reg, val); + + reg = PAU_XTS_CFG2; + val = pau_read(pau, reg); + val |= PAU_XTS_CFG2_XSL2_ENA; + pau_write(pau, reg, val); +} + +static void pau_opencapi_enable_sm_allocation(struct pau *pau) +{ + uint64_t reg, val; + + PAUDBG(pau, "Enable State Machine Allocation\n"); + + reg = PAU_MISC_MACHINE_ALLOC; + val = pau_read(pau, reg); + val |= PAU_MISC_MACHINE_ALLOC_ENABLE; + pau_write(pau, reg, val); +} + +static void pau_opencapi_enable_powerbus(struct pau *pau) +{ + struct pau_dev *dev; + uint8_t oc_typemap = 0; + uint64_t reg, val; + + PAUDBG(pau, "Enable PowerBus\n"); + + pau_for_each_opencapi_dev(dev, pau) + oc_typemap |= 0x10 >> dev->index; + + /* PowerBus interfaces must be enabled prior to MMIO */ + reg = PAU_MCP_MISC_CFG0; + val = pau_read(pau, reg); + val |= PAU_MCP_MISC_CFG0_ENABLE_PBUS; + val |= PAU_MCP_MISC_CFG0_MA_MCRESP_OPT_WRP; + val = SETFIELD(PAU_MCP_MISC_CFG0_OCAPI_MODE, val, oc_typemap); + pau_write(pau, reg, val); + + reg = PAU_SNP_MISC_CFG0; + val = pau_read(pau, reg); + val |= PAU_SNP_MISC_CFG0_ENABLE_PBUS; + val = SETFIELD(PAU_SNP_MISC_CFG0_OCAPI_MODE, val, oc_typemap); + val = SETFIELD(PAU_SNP_MISC_CFG0_OCAPI_C2, val, oc_typemap); + pau_write(pau, reg, val); +} + static void pau_opencapi_init_hw(struct pau *pau) { struct pau_dev *dev = NULL; @@ -482,9 +678,35 @@ static void pau_opencapi_init_hw(struct pau *pau) /* Create phb */ pau_for_each_opencapi_dev(dev, pau) { + PAUDEVINF(dev, "Create phb\n"); pau_opencapi_create_phb(dev); + pau_opencapi_enable_bars(dev, true); pau_opencapi_dt_add_props(dev); } + + /* Procedure 17.1.3.1 - Enabling OpenCAPI */ + pau_for_each_opencapi_dev(dev, pau) { + PAUDEVINF(dev, "Configuring link ...\n"); + pau_opencapi_set_transport_mux_controls(dev); /* step 1 */ + } + pau_opencapi_enable_xsl_clocks(pau); /* step 2 */ + pau_opencapi_enable_misc_clocks(pau); /* step 3 */ + + /* OTL disabled */ + pau_for_each_opencapi_dev(dev, pau) + pau_opencapi_set_fence_control(dev, 0b01); + + pau_opencapi_set_npcq_config(pau); /* step 4 */ + pau_opencapi_enable_xsl_xts_interfaces(pau); /* step 5 */ + pau_opencapi_enable_sm_allocation(pau); /* step 6 */ + pau_opencapi_enable_powerbus(pau); /* step 7 */ + + /* + * access to the PAU registers through mmio requires setting + * up the PAU mmio BAR (in pau_opencapi_assign_bars() above) + * and machine state allocation + */ + pau->mmio_access = true; } static void pau_opencapi_init(struct pau *pau) diff --git a/include/pau-regs.h b/include/pau-regs.h index 57796920..6aeb7589 100644 --- a/include/pau-regs.h +++ b/include/pau-regs.h @@ -27,6 +27,10 @@ #define PAU_BLOCK_CQ_SM(n) PAU_BLOCK(4, (n)) #define PAU_BLOCK_CQ_CTL PAU_BLOCK(4, 4) +#define PAU_BLOCK_CQ_DAT PAU_BLOCK(4, 5) +#define PAU_BLOCK_XSL PAU_BLOCK(4, 0xE) +#define PAU_BLOCK_PAU_XTS PAU_BLOCK(7, 1) +#define PAU_BLOCK_PAU_MISC PAU_BLOCK(7, 2) /* * CQ_SM block registers @@ -37,21 +41,38 @@ #define PAU_MCP_MISC_CFG0 (PAU_BLOCK_CQ_SM(0) + 0x000) #define PAU_MCP_MISC_CFG0_MA_MCRESP_OPT_WRP PPC_BIT(9) #define PAU_MCP_MISC_CFG0_ENABLE_PBUS PPC_BIT(26) +#define PAU_MCP_MISC_CFG0_OCAPI_MODE PPC_BITMASK(44, 48) #define PAU_SNP_MISC_CFG0 (PAU_BLOCK_CQ_SM(0) + 0x180) #define PAU_SNP_MISC_CFG0_ENABLE_PBUS PPC_BIT(2) +#define PAU_SNP_MISC_CFG0_OCAPI_MODE PPC_BITMASK(32, 36) +#define PAU_SNP_MISC_CFG0_OCAPI_C2 PPC_BITMASK(45, 49) #define PAU_NTL_BAR(brk) (PAU_BLOCK_CQ_SM(0) + 0x1b8 + (brk) * 8) +#define PAU_NTL_BAR_ENABLE PPC_BIT(0) #define PAU_NTL_BAR_ADDR PPC_BITMASK(3, 35) #define PAU_NTL_BAR_SIZE PPC_BITMASK(39, 43) #define PAU_MMIO_BAR (PAU_BLOCK_CQ_SM(0) + 0x1e0) #define PAU_MMIO_BAR_ENABLE PPC_BIT(0) #define PAU_MMIO_BAR_ADDR PPC_BITMASK(3, 27) #define PAU_GENID_BAR (PAU_BLOCK_CQ_SM(0) + 0x1e8) +#define PAU_GENID_BAR_ENABLE PPC_BIT(0) #define PAU_GENID_BAR_ADDR PPC_BITMASK(3, 32) +#define PAU_MISC_MACHINE_ALLOC (PAU_BLOCK_CQ_SM(0) + 0x268) +#define PAU_MISC_MACHINE_ALLOC_ENABLE PPC_BIT(0) /* CQ_CTL block registers */ +#define PAU_CTL_MISC_CFG2 (PAU_BLOCK_CQ_CTL + 0x010) +#define PAU_CTL_MISC_CFG2_OCAPI_MODE PPC_BITMASK(0, 4) +#define PAU_CTL_MISC_CFG2_OCAPI_4 PPC_BITMASK(10, 14) +#define PAU_CTL_MISC_CFG2_OCAPI_C2 PPC_BITMASK(15, 19) +#define PAU_CTL_MISC_CFG2_OCAPI_AMO PPC_BITMASK(20, 24) +#define PAU_CTL_MISC_CFG2_OCAPI_MEM_OS_BIT PPC_BITMASK(25, 29) +#define PAU_CTL_MISC_STATUS(brk) (PAU_BLOCK_CQ_CTL + 0x060 + (brk) * 8) +#define PAU_CTL_MISC_STATUS_AM_FENCED(brk) (PPC_BITMASK(41, 42) << ((brk)*32)) #define PAU_CTL_MISC_MMIOPA_CONFIG(brk) (PAU_BLOCK_CQ_CTL + 0x098 + (brk) * 8) #define PAU_CTL_MISC_MMIOPA_CONFIG_BAR_ADDR PPC_BITMASK(1, 35) #define PAU_CTL_MISC_MMIOPA_CONFIG_BAR_SIZE PPC_BITMASK(39, 43) +#define PAU_CTL_MISC_FENCE_CTRL(brk) (PAU_BLOCK_CQ_CTL + 0x108 + (brk) * 8) +#define PAU_CTL_MISC_FENCE_REQUEST PPC_BITMASK(0, 1) #define PAU_CTL_MISC_CFG_ADDR(brk) (PAU_BLOCK_CQ_CTL + 0x250 + (brk) * 8) #define PAU_CTL_MISC_CFG_ADDR_ENABLE PPC_BIT(0) #define PAU_CTL_MISC_CFG_ADDR_STATUS PPC_BITMASK(1, 3) @@ -61,4 +82,26 @@ #define PAU_CTL_MISC_CFG_ADDR_REGISTER_NBR PPC_BITMASK(20, 31) #define PAU_CTL_MISC_CFG_ADDR_TYPE PPC_BIT(32) +/* CQ_DAT block registers */ +#define PAU_DAT_MISC_CFG1 (PAU_BLOCK_CQ_DAT + 0x008) +#define PAU_DAT_MISC_CFG1_OCAPI_MODE PPC_BITMASK(40, 44) + +/* XSL block registers */ +#define PAU_XSL_WRAP_CFG (PAU_BLOCK_XSL + 0x100) +#define PAU_XSL_WRAP_CFG_CLOCK_ENABLE PPC_BIT(0) + +/* XTS block registers */ +#define PAU_XTS_CFG (PAU_BLOCK_PAU_XTS + 0x020) +#define PAU_XTS_CFG_OPENCAPI PPC_BIT(15) +#define PAU_XTS_CFG2 (PAU_BLOCK_PAU_XTS + 0x028) +#define PAU_XTS_CFG2_XSL2_ENA PPC_BIT(55) + +/* MISC block registers */ +#define PAU_MISC_OPTICAL_IO_CONFIG (PAU_BLOCK_PAU_MISC + 0x018) +#define PAU_MISC_OPTICAL_IO_CONFIG_OTL PPC_BITMASK(2, 3) +#define PAU_MISC_HOLD (PAU_BLOCK_PAU_MISC + 0x020) +#define PAU_MISC_HOLD_NDL_STALL PPC_BITMASK(0, 3) +#define PAU_MISC_CONFIG (PAU_BLOCK_PAU_MISC + 0x030) +#define PAU_MISC_CONFIG_OC_MODE PPC_BIT(16) + #endif /* __PAU_REGS_H */ diff --git a/include/pau.h b/include/pau.h index fdf85f85..0246a63d 100644 --- a/include/pau.h +++ b/include/pau.h @@ -22,6 +22,7 @@ enum pau_dev_type { /* Used to expose a hardware BAR (or logical slice of it) outside skiboot */ struct pau_bar { + bool enable; uint64_t addr; uint64_t size; uint64_t cfg; From patchwork Thu Oct 14 15:56:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540998 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=KQbd53I4; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYxF00GYz9sNH for ; Fri, 15 Oct 2021 02:59:12 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYxD6CHdz2ygC for ; Fri, 15 Oct 2021 02:59:12 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=KQbd53I4; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0b-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=KQbd53I4; dkim-atps=neutral Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYwj3gXBz3c9f for ; Fri, 15 Oct 2021 02:58:45 +1100 (AEDT) Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EFlxZo019293 for ; Thu, 14 Oct 2021 11:58:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=xmwjAt6bSEICYnraaXbI0vQ7R4zisP0+ndX2ZA2CuzA=; b=KQbd53I4/HWHN61RsbRhbv4tDIZaVvv2hUVVcIiDdvW0JAjy9HXcA4AqUWypp2xslUSA J8dibYDF65veA1z901fDW8AlOfUdgZNhcQHTVGdCMvtN9G5Hw8OXlOUSn2va4lKLywB7 KOCDzzVrU3/4hJIm3CPpMldUMR3ehOCTDdmABvtNWNMzm/RwQTALL2EBxti4s4OmILTI /ZCNdIKmqrTJ+Nkl/vOk8dVaAW+i0DiAkOZzNF6ZM1zKbHwrxt0Dic/KBUHl2cy0PN8X kX0faW7Gk9Oveclx/LOGFTPQBXrkQD+wZUJMYwxUIIjyUApohMksYJrKa9hSnA6jm+Ze yQ== Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 3bpnv130qh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:58:42 -0400 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFXVqk029841 for ; Thu, 14 Oct 2021 15:57:09 GMT Received: from b06avi18626390.portsmouth.uk.ibm.com (b06avi18626390.portsmouth.uk.ibm.com [9.149.26.192]) by ppma03ams.nl.ibm.com with ESMTP id 3bk2qa6ysq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:08 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFpSop61669646 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:51:29 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 75DBA11C04C for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4EFC611C050 for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:56:56 +0200 Message-Id: <20211014155704.46441-8-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 8hWY74RDEmHuMQfc64EWcTJhkg9YLi5_ X-Proofpoint-GUID: 8hWY74RDEmHuMQfc64EWcTJhkg9YLi5_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_09,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 lowpriorityscore=0 clxscore=1015 priorityscore=1501 phishscore=0 mlxlogscore=999 adultscore=0 impostorscore=0 suspectscore=0 mlxscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 07/15] pau: translation layer configuration X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Next main part of the hypervisor PAU initialization. The P10 PAU supports two OpenCAPI links. The PAU provides various configuration selections for both of the OCAPI Link Transaction Layer functions (OTLs). These include a link enable, behavior controls, debug modes, and virtual channel credits to send to the AFU. The OTL Configuration 0, OTL Configuration 1, OTL Configuration 2, and TLX Credit Configuration registers are used to control these functions. This patch completes the PAU configuration following the sections 17.1.3.4 to 17.1.3.10.2 of the workbook document. Signed-off-by: Christophe Lombard Reviewed-by: Frederic Barrat --- hw/pau.c | 192 +++++++++++++++++++++++++++++++++++++++ include/pau-regs.h | 37 ++++++++ include/xscom-p10-regs.h | 2 + 3 files changed, 231 insertions(+) diff --git a/hw/pau.c b/hw/pau.c index 5f181011..36c5f224 100644 --- a/hw/pau.c +++ b/hw/pau.c @@ -7,6 +7,7 @@ #include #include #include +#include /* Number of PEs supported */ #define PAU_MAX_PE_NUM 16 @@ -173,6 +174,7 @@ static struct pau *pau_create(struct dt_node *dn) dev->pau = pau; dev->dn = link; dev->odl_index = dt_prop_get_u32(link, "ibm,odl-index"); + dev->pau_unit = dt_prop_get_u32(link, "ibm,pau-unit"); dev->op_unit = dt_prop_get_u32(link, "ibm,op-unit"); dev->phy_lane_mask = dt_prop_get_u32(link, "ibm,pau-lane-mask"); }; @@ -235,6 +237,22 @@ static int pau_opencapi_set_fence_control(struct pau_dev *dev, return OPAL_HARDWARE; } +static void pau_opencapi_mask_firs(struct pau *pau) +{ + uint64_t reg, val; + + reg = pau->xscom_base + PAU_FIR_MASK(1); + xscom_read(pau->chip_id, reg, &val); + val |= PAU_FIR1_NDL_BRICKS_0_5; + val |= PAU_FIR1_NDL_BRICKS_6_11; + xscom_write(pau->chip_id, reg, val); + + reg = pau->xscom_base + PAU_FIR_MASK(2); + xscom_read(pau->chip_id, reg, &val); + val |= PAU_FIR2_OTL_PERR; + xscom_write(pau->chip_id, reg, val); +} + static void pau_opencapi_assign_bars(struct pau *pau) { struct pau_dev *dev; @@ -670,10 +688,134 @@ static void pau_opencapi_enable_powerbus(struct pau *pau) pau_write(pau, reg, val); } +static void pau_opencapi_tl_config(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint64_t val; + + PAUDEVDBG(dev, "TL Configuration\n"); + + /* OTL Config 0 */ + val = 0; + val |= PAU_OTL_MISC_CFG0_EN; + val |= PAU_OTL_MISC_CFG0_BLOCK_PE_HANDLE; + val = SETFIELD(PAU_OTL_MISC_CFG0_BRICKID, val, dev->index); + val |= PAU_OTL_MISC_CFG0_ENABLE_4_0; + val |= PAU_OTL_MISC_CFG0_XLATE_RELEASE; + val |= PAU_OTL_MISC_CFG0_ENABLE_5_0; + pau_write(pau, PAU_OTL_MISC_CFG0(dev->index), val); + + /* OTL Config 1 */ + val = 0; + val = SETFIELD(PAU_OTL_MISC_CFG_TX_DRDY_WAIT, val, 0b010); + val = SETFIELD(PAU_OTL_MISC_CFG_TX_TEMP0_RATE, val, 0b0000); + val = SETFIELD(PAU_OTL_MISC_CFG_TX_TEMP1_RATE, val, 0b0011); + val = SETFIELD(PAU_OTL_MISC_CFG_TX_TEMP2_RATE, val, 0b0111); + val = SETFIELD(PAU_OTL_MISC_CFG_TX_TEMP3_RATE, val, 0b0010); + val = SETFIELD(PAU_OTL_MISC_CFG_TX_CRET_FREQ, val, 0b001); + pau_write(pau, PAU_OTL_MISC_CFG_TX(dev->index), val); + + /* OTL Config 2 - Done after link training, in otl_tx_send_enable() */ + + /* TLX Credit Configuration */ + val = 0; + val = SETFIELD(PAU_OTL_MISC_CFG_TLX_CREDITS_VC0, val, 0x40); + val = SETFIELD(PAU_OTL_MISC_CFG_TLX_CREDITS_VC1, val, 0x40); + val = SETFIELD(PAU_OTL_MISC_CFG_TLX_CREDITS_VC2, val, 0x40); + val = SETFIELD(PAU_OTL_MISC_CFG_TLX_CREDITS_VC3, val, 0x40); + val = SETFIELD(PAU_OTL_MISC_CFG_TLX_CREDITS_DCP0, val, 0x80); + val = SETFIELD(PAU_OTL_MISC_CFG_TLX_CREDITS_SPARE, val, 0x80); + val = SETFIELD(PAU_OTL_MISC_CFG_TLX_CREDITS_DCP2, val, 0x80); + val = SETFIELD(PAU_OTL_MISC_CFG_TLX_CREDITS_DCP3, val, 0x80); + pau_write(pau, PAU_OTL_MISC_CFG_TLX_CREDITS(dev->index), val); +} + +static void pau_opencapi_enable_otlcq_interface(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint8_t typemap = 0; + uint64_t reg, val; + + PAUDEVDBG(dev, "Enabling OTL-CQ Interface\n"); + + typemap |= 0x10 >> dev->index; + reg = PAU_CTL_MISC_CFG0; + val = pau_read(pau, reg); + typemap |= GETFIELD(PAU_CTL_MISC_CFG0_OTL_ENABLE, val); + val = SETFIELD(PAU_CTL_MISC_CFG0_OTL_ENABLE, val, typemap); + pau_write(pau, reg, val); +} + +static void pau_opencapi_address_translation_config(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint64_t reg, val; + + PAUDEVDBG(dev, "Address Translation Configuration\n"); + + /* OpenCAPI 4.0 Mode */ + reg = PAU_XSL_OSL_XLATE_CFG(dev->index); + val = pau_read(pau, reg); + val |= PAU_XSL_OSL_XLATE_CFG_AFU_DIAL; + val &= ~PAU_XSL_OSL_XLATE_CFG_OPENCAPI3; + pau_write(pau, reg, val); + + /* MMIO shootdowns (OpenCAPI 5.0) */ + reg = PAU_XTS_CFG3; + val = pau_read(pau, reg); + val |= PAU_XTS_CFG3_MMIOSD_OCAPI; + pau_write(pau, reg, val); + + /* XSL_GP - use defaults */ +} + +static void pau_opencapi_enable_ref_clock(struct pau_dev *dev) +{ + uint64_t reg, val; + int bit; + + switch (dev->pau_unit) { + case 0: + if (dev->index == 0) + bit = 16; + else + bit = 17; + break; + case 3: + if (dev->index == 0) + bit = 18; + else + bit = 19; + break; + case 4: + bit = 20; + break; + case 5: + bit = 21; + break; + case 6: + bit = 22; + break; + case 7: + bit = 23; + break; + default: + assert(false); + } + + reg = P10_ROOT_CONTROL_7; + xscom_read(dev->pau->chip_id, reg, &val); + val |= PPC_BIT(bit); + PAUDEVDBG(dev, "Enabling ref clock for PAU%d => %llx\n", + dev->pau_unit, val); + xscom_write(dev->pau->chip_id, reg, val); +} + static void pau_opencapi_init_hw(struct pau *pau) { struct pau_dev *dev = NULL; + pau_opencapi_mask_firs(pau); pau_opencapi_assign_bars(pau); /* Create phb */ @@ -707,6 +849,56 @@ static void pau_opencapi_init_hw(struct pau *pau) * and machine state allocation */ pau->mmio_access = true; + + pau_for_each_opencapi_dev(dev, pau) { + /* Procedure 17.1.3.4 - Transaction Layer Configuration + * OCAPI Link Transaction Layer functions + */ + pau_opencapi_tl_config(dev); + + /* Procedure 17.1.3.4.1 - Enabling OTL-CQ Interface */ + pau_opencapi_enable_otlcq_interface(dev); + + /* Procedure 17.1.3.4.2 - Place OTL into Reset State + * Reset (Fence) both OTL and the PowerBus for this + * Brick + */ + pau_opencapi_set_fence_control(dev, 0b11); + + /* Take PAU out of OTL Reset State + * Reset (Fence) only the PowerBus for this Brick, OTL + * will be operational + */ + pau_opencapi_set_fence_control(dev, 0b10); + + /* Procedure 17.1.3.5 - Address Translation Configuration */ + pau_opencapi_address_translation_config(dev); + + /* Procedure 17.1.3.6 - AFU Memory Range BARs */ + /* Will be done out of this process */ + + /* Procedure 17.1.3.8 - AFU MMIO Range BARs */ + /* done in pau_opencapi_assign_bars() */ + + /* Procedure 17.1.3.9 - AFU Config BARs */ + /* done in pau_opencapi_assign_bars() */ + + /* Precedure 17.1.3.10 - Relaxed Ordering Configuration */ + /* Procedure 17.1.3.10.1 - Generation-Id Registers MMIO Bars */ + /* done in pau_opencapi_assign_bars() */ + + /* Procedure 17.1.3.10.2 - Relaxed Ordering Source Configuration */ + /* For an OpenCAPI AFU that uses M2 Memory Mode, + * Relaxed Ordering can be used for accesses to the + * AFU's memory + */ + + /* Reset disabled. Place OTLs into Run State */ + pau_opencapi_set_fence_control(dev, 0b00); + + /* Enable reference clock */ + pau_opencapi_enable_ref_clock(dev); + } } static void pau_opencapi_init(struct pau *pau) diff --git a/include/pau-regs.h b/include/pau-regs.h index 6aeb7589..e4ff7cc0 100644 --- a/include/pau-regs.h +++ b/include/pau-regs.h @@ -12,6 +12,10 @@ #define PAU_FIR_ACTION1(n) (0x407 + (n) * 0x40) #define PAU_FIR_MAX 3 +#define PAU_FIR1_NDL_BRICKS_0_5 PPC_BITMASK(0, 11) +#define PAU_FIR1_NDL_BRICKS_6_11 PPC_BITMASK(47, 58) +#define PAU_FIR2_OTL_PERR PPC_BIT(18) + /* PAU RING: Indirect address/data port */ #define PAU_MISC_SCOM_IND_SCOM_ADDR 0x33e #define PAU_MISC_DA_ADDR PPC_BITMASK(0, 23) @@ -28,6 +32,7 @@ #define PAU_BLOCK_CQ_SM(n) PAU_BLOCK(4, (n)) #define PAU_BLOCK_CQ_CTL PAU_BLOCK(4, 4) #define PAU_BLOCK_CQ_DAT PAU_BLOCK(4, 5) +#define PAU_BLOCK_OTL(brk) PAU_BLOCK(4, 0xC + (brk)) #define PAU_BLOCK_XSL PAU_BLOCK(4, 0xE) #define PAU_BLOCK_PAU_XTS PAU_BLOCK(7, 1) #define PAU_BLOCK_PAU_MISC PAU_BLOCK(7, 2) @@ -60,6 +65,8 @@ #define PAU_MISC_MACHINE_ALLOC_ENABLE PPC_BIT(0) /* CQ_CTL block registers */ +#define PAU_CTL_MISC_CFG0 (PAU_BLOCK_CQ_CTL + 0x000) +#define PAU_CTL_MISC_CFG0_OTL_ENABLE PPC_BITMASK(52, 56) #define PAU_CTL_MISC_CFG2 (PAU_BLOCK_CQ_CTL + 0x010) #define PAU_CTL_MISC_CFG2_OCAPI_MODE PPC_BITMASK(0, 4) #define PAU_CTL_MISC_CFG2_OCAPI_4 PPC_BITMASK(10, 14) @@ -86,15 +93,45 @@ #define PAU_DAT_MISC_CFG1 (PAU_BLOCK_CQ_DAT + 0x008) #define PAU_DAT_MISC_CFG1_OCAPI_MODE PPC_BITMASK(40, 44) +/* OTL block registers */ +#define PAU_OTL_MISC_CFG0(brk) (PAU_BLOCK_OTL(brk) + 0x000) +#define PAU_OTL_MISC_CFG0_EN PPC_BIT(0) +#define PAU_OTL_MISC_CFG0_BLOCK_PE_HANDLE PPC_BIT(1) +#define PAU_OTL_MISC_CFG0_BRICKID PPC_BITMASK(2, 3) +#define PAU_OTL_MISC_CFG0_ENABLE_4_0 PPC_BIT(51) +#define PAU_OTL_MISC_CFG0_XLATE_RELEASE PPC_BIT(62) +#define PAU_OTL_MISC_CFG0_ENABLE_5_0 PPC_BIT(63) +#define PAU_OTL_MISC_CFG_TLX_CREDITS(brk) (PAU_BLOCK_OTL(brk) + 0x050) +#define PAU_OTL_MISC_CFG_TLX_CREDITS_VC0 PPC_BITMASK(0, 7) +#define PAU_OTL_MISC_CFG_TLX_CREDITS_VC1 PPC_BITMASK(8, 15) +#define PAU_OTL_MISC_CFG_TLX_CREDITS_VC2 PPC_BITMASK(16, 23) +#define PAU_OTL_MISC_CFG_TLX_CREDITS_VC3 PPC_BITMASK(24, 31) +#define PAU_OTL_MISC_CFG_TLX_CREDITS_DCP0 PPC_BITMASK(32, 39) +#define PAU_OTL_MISC_CFG_TLX_CREDITS_SPARE PPC_BITMASK(40, 47) +#define PAU_OTL_MISC_CFG_TLX_CREDITS_DCP2 PPC_BITMASK(48, 55) +#define PAU_OTL_MISC_CFG_TLX_CREDITS_DCP3 PPC_BITMASK(56, 63) +#define PAU_OTL_MISC_CFG_TX(brk) (PAU_BLOCK_OTL(brk) + 0x058) +#define PAU_OTL_MISC_CFG_TX_DRDY_WAIT PPC_BITMASK(5, 7) +#define PAU_OTL_MISC_CFG_TX_TEMP0_RATE PPC_BITMASK(8, 11) +#define PAU_OTL_MISC_CFG_TX_TEMP1_RATE PPC_BITMASK(12, 15) +#define PAU_OTL_MISC_CFG_TX_TEMP2_RATE PPC_BITMASK(16, 19) +#define PAU_OTL_MISC_CFG_TX_TEMP3_RATE PPC_BITMASK(20, 23) +#define PAU_OTL_MISC_CFG_TX_CRET_FREQ PPC_BITMASK(32, 34) + /* XSL block registers */ #define PAU_XSL_WRAP_CFG (PAU_BLOCK_XSL + 0x100) #define PAU_XSL_WRAP_CFG_CLOCK_ENABLE PPC_BIT(0) +#define PAU_XSL_OSL_XLATE_CFG(brk) (PAU_BLOCK_XSL + 0x040 + (brk) * 8) +#define PAU_XSL_OSL_XLATE_CFG_AFU_DIAL PPC_BIT(0) +#define PAU_XSL_OSL_XLATE_CFG_OPENCAPI3 PPC_BIT(32) /* XTS block registers */ #define PAU_XTS_CFG (PAU_BLOCK_PAU_XTS + 0x020) #define PAU_XTS_CFG_OPENCAPI PPC_BIT(15) #define PAU_XTS_CFG2 (PAU_BLOCK_PAU_XTS + 0x028) #define PAU_XTS_CFG2_XSL2_ENA PPC_BIT(55) +#define PAU_XTS_CFG3 (PAU_BLOCK_PAU_XTS + 0x068) +#define PAU_XTS_CFG3_MMIOSD_OCAPI PPC_BIT(5) /* MISC block registers */ #define PAU_MISC_OPTICAL_IO_CONFIG (PAU_BLOCK_PAU_MISC + 0x018) diff --git a/include/xscom-p10-regs.h b/include/xscom-p10-regs.h index 6045152d..21ac21f8 100644 --- a/include/xscom-p10-regs.h +++ b/include/xscom-p10-regs.h @@ -53,4 +53,6 @@ #define P10_NCU_DARN_BAR_EN PPC_BIT(0) #define P10_NCU_DARN_BAR_ADDRMSK 0x000ffffffffff000ull /* 4k aligned */ +#define P10_ROOT_CONTROL_7 0x50017 + #endif /* __XSCOM_P10_REGS_H__ */ From patchwork Thu Oct 14 15:56:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540989 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=NunpKsi3; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYvq1vTYz9sNH for ; Fri, 15 Oct 2021 02:57:59 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYvq0PKKz3c5S for ; Fri, 15 Oct 2021 02:57:59 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=NunpKsi3; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=NunpKsi3; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYty5XCyz2yPk for ; Fri, 15 Oct 2021 02:57:14 +1100 (AEDT) Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EFUb4o002323 for ; Thu, 14 Oct 2021 11:57:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=tYrMzpZlXJ7SL7y9WL2e9OtJ2/0ukRHHcsQoz63245A=; b=NunpKsi3xpC0gChuxJGZszgMsU6nmj5F0ltKABhow3zTIPIJhvNJV4PwQDRqOUghU+IJ vxVtVeZ8ZlVNi4i/lVypAj8UiyjmjoLcSX/P9sxeN5JALpEK6IJ9ubAJ1vmZ/p60XbRs OfXRZuzGyL+pL0Kcaop9bdeK7xuafySGqKCdmZjz1qsIlswdEm8ozePN0FNCFm2j0c9W bJR8qaXIjR3IcRlDrOsCmHXRM61s9beBrePp44jMtYWMm4LpDBEj2QHKFzSEynr/hEd0 J/EhMQrZJQMURLd2XZqqDYD0oWz2BYCaDUsIYPwH0G0x5KkBIbkbDOXVqiDxQytqEA7+ kw== Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0a-001b2d01.pphosted.com with ESMTP id 3bpq8kgjt9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:12 -0400 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFVsr1005770 for ; Thu, 14 Oct 2021 15:57:09 GMT Received: from b06avi18626390.portsmouth.uk.ibm.com (b06avi18626390.portsmouth.uk.ibm.com [9.149.26.192]) by ppma04fra.de.ibm.com with ESMTP id 3bk2qav2cv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:09 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFpTDK61735410 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:51:29 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AE3E311C04A for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 844F111C052 for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:56:57 +0200 Message-Id: <20211014155704.46441-9-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 78pyy06JJFatB-Bzm-W6iaQOsdab-lwx X-Proofpoint-ORIG-GUID: 78pyy06JJFatB-Bzm-W6iaQOsdab-lwx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_09,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 priorityscore=1501 phishscore=0 clxscore=1015 adultscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 suspectscore=0 mlxscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 08/15] pau: enable interrupt on error X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" The default action for the errors (unexpected errors on the opencapi link) reported in the PAU FIR2 registe is mostly set to system checkstop. This patch changes the default action of those errors so that the PAU will raise an interrupt instead. Interrupt information are logged so that the error can be debugged and linux can catch the event. Signed-off-by: Christophe Lombard Reviewed-by: Frederic Barrat --- hw/pau.c | 157 +++++++++++++++++++++++++++++++++++++++++++++ include/pau-regs.h | 12 ++++ include/pau.h | 2 + 3 files changed, 171 insertions(+) diff --git a/hw/pau.c b/hw/pau.c index 36c5f224..eb317151 100644 --- a/hw/pau.c +++ b/hw/pau.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -237,6 +238,15 @@ static int pau_opencapi_set_fence_control(struct pau_dev *dev, return OPAL_HARDWARE; } +#define PAU_DEV_STATUS_BROKEN 0x1 + +static void pau_opencapi_set_broken(struct pau_dev *dev) +{ + PAUDEVDBG(dev, "Update status to broken\n"); + + dev->status = PAU_DEV_STATUS_BROKEN; +} + static void pau_opencapi_mask_firs(struct pau *pau) { uint64_t reg, val; @@ -310,6 +320,113 @@ static void pau_opencapi_assign_bars(struct pau *pau) } } +static uint64_t pau_opencapi_ipi_attributes(struct irq_source *is, + uint32_t isn) +{ + struct pau *pau = is->data; + uint32_t level = isn - pau->irq_base; + + if (level >= 37 && level <= 40) { + /* level 37-40: OTL/XSL interrupt */ + return IRQ_ATTR_TARGET_OPAL | + IRQ_ATTR_TARGET_RARE | + IRQ_ATTR_TYPE_MSI; + } + + return IRQ_ATTR_TARGET_LINUX; +} + +static void pau_opencapi_ipi_interrupt(struct irq_source *is, + uint32_t isn) +{ + struct pau *pau = is->data; + uint32_t level = isn - pau->irq_base; + struct pau_dev *dev; + + switch (level) { + case 37 ... 40: + pau_for_each_opencapi_dev(dev, pau) + pau_opencapi_set_broken(dev); + + opal_update_pending_evt(OPAL_EVENT_PCI_ERROR, + OPAL_EVENT_PCI_ERROR); + break; + default: + PAUERR(pau, "Received unknown interrupt %d\n", level); + return; + } +} + +#define PAU_IRQ_LEVELS 60 + +static char *pau_opencapi_ipi_name(struct irq_source *is, uint32_t isn) +{ + struct pau *pau = is->data; + uint32_t level = isn - pau->irq_base; + + switch (level) { + case 0 ... 19: + return strdup("Reserved"); + case 20: + return strdup("An error event related to PAU CQ functions"); + case 21: + return strdup("An error event related to PAU MISC functions"); + case 22 ... 34: + return strdup("Reserved"); + case 35: + return strdup("Translation failure for OCAPI link 0"); + case 36: + return strdup("Translation failure for OCAPI link 1"); + case 37: + return strdup("An error event related to OTL for link 0"); + case 38: + return strdup("An error event related to OTL for link 1"); + case 39: + return strdup("An error event related to XSL for link 0"); + case 40: + return strdup("An error event related to XSL for link 1"); + case 41 ... 59: + return strdup("Reserved"); + } + + return strdup("Unknown"); +} + +static const struct irq_source_ops pau_opencapi_ipi_ops = { + .attributes = pau_opencapi_ipi_attributes, + .interrupt = pau_opencapi_ipi_interrupt, + .name = pau_opencapi_ipi_name, +}; + +static void pau_opencapi_setup_irqs(struct pau *pau) +{ + uint64_t reg, val; + uint32_t base; + + base = xive2_alloc_ipi_irqs(pau->chip_id, PAU_IRQ_LEVELS, 64); + if (base == XIVE_IRQ_ERROR) { + PAUERR(pau, "Failed to allocate interrupt sources\n"); + return; + } + + xive2_register_ipi_source(base, PAU_IRQ_LEVELS, pau, &pau_opencapi_ipi_ops); + + /* Set IPI configuration */ + reg = PAU_MISC_CONFIG; + val = pau_read(pau, reg); + val = SETFIELD(PAU_MISC_CONFIG_IPI_PS, val, PAU_MISC_CONFIG_IPI_PS_64K); + val = SETFIELD(PAU_MISC_CONFIG_IPI_OS, val, PAU_MISC_CONFIG_IPI_OS_AIX); + pau_write(pau, reg, val); + + /* Set IRQ base */ + reg = PAU_MISC_INT_BAR; + val = SETFIELD(PAU_MISC_INT_BAR_ADDR, 0ull, + (uint64_t)xive2_get_trigger_port(base) >> 12); + pau_write(pau, reg, val); + + pau->irq_base = base; +} + static void pau_opencapi_enable_bars(struct pau_dev *dev, bool enable) { struct pau *pau = dev->pau; @@ -769,6 +886,41 @@ static void pau_opencapi_address_translation_config(struct pau_dev *dev) /* XSL_GP - use defaults */ } +static void pau_opencapi_enable_interrupt_on_error(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint64_t reg, val; + + PAUDEVDBG(dev, "Enable Interrupt-on-error\n"); + + /* translation fault */ + reg = PAU_MISC_INT_2_CONFIG; + val = pau_read(pau, reg); + val |= PAU_MISC_INT_2_CONFIG_XFAULT_2_5(dev->index); + pau_write(pau, reg, val); + + /* freeze disable */ + reg = PAU_MISC_FREEZE_1_CONFIG; + val = pau_read(pau, reg); + val &= ~PAU_FIR1_NDL_BRICKS_0_5; + val &= ~PAU_FIR1_NDL_BRICKS_6_11; + pau_write(pau, reg, val); + + /* fence disable */ + reg = PAU_MISC_FENCE_1_CONFIG; + val = pau_read(pau, reg); + val &= ~PAU_FIR1_NDL_BRICKS_0_5; + val &= ~PAU_FIR1_NDL_BRICKS_6_11; + pau_write(pau, reg, val); + + /* irq disable */ + reg = PAU_MISC_INT_1_CONFIG; + val = pau_read(pau, reg); + val &= ~PAU_FIR1_NDL_BRICKS_0_5; + val &= ~PAU_FIR1_NDL_BRICKS_6_11; + pau_write(pau, reg, val); +} + static void pau_opencapi_enable_ref_clock(struct pau_dev *dev) { uint64_t reg, val; @@ -817,6 +969,7 @@ static void pau_opencapi_init_hw(struct pau *pau) pau_opencapi_mask_firs(pau); pau_opencapi_assign_bars(pau); + pau_opencapi_setup_irqs(pau); /* Create phb */ pau_for_each_opencapi_dev(dev, pau) { @@ -893,6 +1046,10 @@ static void pau_opencapi_init_hw(struct pau *pau) * AFU's memory */ + /* Procedure 17.1.3.11 - Interrupt Configuration */ + /* done in pau_opencapi_setup_irqs() */ + pau_opencapi_enable_interrupt_on_error(dev); + /* Reset disabled. Place OTLs into Run State */ pau_opencapi_set_fence_control(dev, 0b00); diff --git a/include/pau-regs.h b/include/pau-regs.h index e4ff7cc0..d98f435b 100644 --- a/include/pau-regs.h +++ b/include/pau-regs.h @@ -139,6 +139,18 @@ #define PAU_MISC_HOLD (PAU_BLOCK_PAU_MISC + 0x020) #define PAU_MISC_HOLD_NDL_STALL PPC_BITMASK(0, 3) #define PAU_MISC_CONFIG (PAU_BLOCK_PAU_MISC + 0x030) +#define PAU_MISC_CONFIG_IPI_PS PPC_BIT(11) +#define PAU_MISC_CONFIG_IPI_PS_64K 1 +#define PAU_MISC_CONFIG_IPI_OS PPC_BIT(12) +#define PAU_MISC_CONFIG_IPI_OS_AIX 0 #define PAU_MISC_CONFIG_OC_MODE PPC_BIT(16) +#define PAU_MISC_FREEZE_1_CONFIG (PAU_BLOCK_PAU_MISC + 0x048) +#define PAU_MISC_FENCE_1_CONFIG (PAU_BLOCK_PAU_MISC + 0x058) +#define PAU_MISC_INT_1_CONFIG (PAU_BLOCK_PAU_MISC + 0x068) +#define PAU_MISC_INT_BAR (PAU_BLOCK_PAU_MISC + 0x098) +#define PAU_MISC_INT_BAR_ADDR PPC_BITMASK(0, 39) +#define PAU_MISC_INT_2_CONFIG (PAU_BLOCK_PAU_MISC + 0x408) +#define PAU_MISC_INT_2_CONFIG_XFAULT_2_5(n) PPC_BIT(0 + (n)) +#define PAU_MISC_INT_2_CONFIG_XFAULT_0_1(n) PPC_BIT(54 + (n)) #endif /* __PAU_REGS_H */ diff --git a/include/pau.h b/include/pau.h index 0246a63d..b6fabe7f 100644 --- a/include/pau.h +++ b/include/pau.h @@ -33,6 +33,7 @@ struct pau_dev { uint32_t index; struct dt_node *dn; struct phb phb; + uint32_t status; struct pau_bar ntl_bar; struct pau_bar genid_bar; @@ -59,6 +60,7 @@ struct pau { uint64_t regs[2]; bool mmio_access; + uint32_t irq_base; struct lock lock; uint32_t links; From patchwork Thu Oct 14 15:56:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540988 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=mkoNsdnH; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYvg3LgJz9sNH for ; Fri, 15 Oct 2021 02:57:51 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYvg1rrvz3c4g for ; Fri, 15 Oct 2021 02:57:51 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=mkoNsdnH; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=mkoNsdnH; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYty03BGz2yg4 for ; Fri, 15 Oct 2021 02:57:13 +1100 (AEDT) Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EFb3Sp012315 for ; Thu, 14 Oct 2021 11:57:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=x8Kl66NKzHpRj+ClXzoEK0fsBV6jgNCRocRSPwaupqg=; b=mkoNsdnHYEKCIlk/0Bp3kkjMZldH1p4cdAP4kO+9UIT2nYLxTXNiI45jomSfL/Dg0fHX YzfVLGcuLEjWlu+B/GDp68F9JhIdfPsdmKX7A+ZGSLyaiShvTFq3cD/RV7RVmSx94IcT ytO62RiSlIZLJAdiEGcEDhk1WzBl36ulK9SW5jep6vQWAWykR1n/H1N9lqiZB4eCRTmV yGY0OWeXD6dddUHKgr5yTIwvMG3XdhdfU5Y9eVwQke7Ee0IuaRhfMm+lz+C1f0aSYEjA jhbWtaisgmKPeO3+R79j4JSyvytuWBGLIi/1DTxpDwSZcfNdKJ1PwR5t036zALrYo6nE cw== Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0b-001b2d01.pphosted.com with ESMTP id 3bnshjdr2x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:11 -0400 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFXmv6009937 for ; Thu, 14 Oct 2021 15:57:09 GMT Received: from b06avi18626390.portsmouth.uk.ibm.com (b06avi18626390.portsmouth.uk.ibm.com [9.149.26.192]) by ppma02fra.de.ibm.com with ESMTP id 3bk2qac035-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:09 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFpTpX61735412 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:51:29 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DCA2A11C04A for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BC8F711C04C for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:56:58 +0200 Message-Id: <20211014155704.46441-10-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: NhXoDY_5b4zcOc6LuO3hLsRXfO9cl6kP X-Proofpoint-GUID: NhXoDY_5b4zcOc6LuO3hLsRXfO9cl6kP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_09,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 priorityscore=1501 bulkscore=0 mlxlogscore=922 adultscore=0 clxscore=1015 phishscore=0 impostorscore=0 suspectscore=0 malwarescore=0 spamscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 09/15] pau: complete phb ops X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Add more PHB interfaces: - to control pci error type in case of freeze. - add the addresses of the registers needed by the OS to handle translation failures. - to detect the fence state of a specific brick - to configure BDF (Bus Device Function) and PE (Partitionable Endpoint) for context identification. Signed-off-by: Christophe Lombard Reviewed-by: Frederic Barrat --- hw/pau.c | 137 +++++++++++++++++++++++++++++++++++++++++++++ include/pau-regs.h | 9 +++ 2 files changed, 146 insertions(+) diff --git a/hw/pau.c b/hw/pau.c index eb317151..d632b2b0 100644 --- a/hw/pau.c +++ b/hw/pau.c @@ -598,6 +598,138 @@ PAU_OPENCAPI_PCI_CFG_WRITE(8, u8) PAU_OPENCAPI_PCI_CFG_WRITE(16, u16) PAU_OPENCAPI_PCI_CFG_WRITE(32, u32) +static int64_t pau_opencapi_eeh_freeze_status(struct phb *phb __unused, + uint64_t pe_num __unused, + uint8_t *freeze_state, + uint16_t *pci_error_type, + uint16_t *severity) +{ + *freeze_state = OPAL_EEH_STOPPED_NOT_FROZEN; + *pci_error_type = OPAL_EEH_NO_ERROR; + + if (severity) + *severity = OPAL_EEH_SEV_NO_ERROR; + + return OPAL_SUCCESS; +} + +static int64_t pau_opencapi_ioda_reset(struct phb __unused * phb, + bool __unused purge) +{ + /* Not relevant to OpenCAPI - we do this just to silence the error */ + return OPAL_SUCCESS; +} + +static int64_t pau_opencapi_next_error(struct phb *phb, + uint64_t *first_frozen_pe, + uint16_t *pci_error_type, + uint16_t *severity) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(phb); + struct pau *pau = dev->pau; + uint32_t pe_num; + uint64_t val; + + if (!first_frozen_pe || !pci_error_type || !severity) + return OPAL_PARAMETER; + + if (dev->status & PAU_DEV_STATUS_BROKEN) { + val = pau_read(pau, PAU_MISC_BDF2PE_CFG(dev->index)); + pe_num = GETFIELD(PAU_MISC_BDF2PE_CFG_PE, val); + + PAUDEVDBG(dev, "Reporting device as broken\n"); + PAUDEVDBG(dev, "Brick %d fenced! (pe_num: %08x\n", + pau_dev_index(dev, PAU_LINKS_OPENCAPI_PER_PAU), + pe_num); + *first_frozen_pe = pe_num; + *pci_error_type = OPAL_EEH_PHB_ERROR; + *severity = OPAL_EEH_SEV_PHB_DEAD; + } else { + *first_frozen_pe = -1; + *pci_error_type = OPAL_EEH_NO_ERROR; + *severity = OPAL_EEH_SEV_NO_ERROR; + } + return OPAL_SUCCESS; +} + +static uint32_t pau_opencapi_dev_interrupt_level(struct pau_dev *dev) +{ + /* Interrupt Levels + * 35: Translation failure for OCAPI link 0 + * 36: Translation failure for OCAPI link 1 + */ + const uint32_t level[2] = {35, 36}; + + return level[dev->index]; +} + +static int pau_opencapi_dt_add_interrupts(struct phb *phb, + struct pci_device *pd, + void *data __unused) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(phb); + struct pau *pau = dev->pau; + uint64_t dsisr, dar, tfc, handle; + uint32_t irq; + + irq = pau->irq_base + pau_opencapi_dev_interrupt_level(dev); + + /* When an address translation fail causes the PAU to send an + * interrupt, information is stored in three registers for use + * by the interrupt handler. The OS accesses them by mmio. + */ + dsisr = pau->regs[0] + PAU_OTL_MISC_PSL_DSISR_AN(dev->index); + dar = pau->regs[0] + PAU_OTL_MISC_PSL_DAR_AN(dev->index); + tfc = pau->regs[0] + PAU_OTL_MISC_PSL_TFC_AN(dev->index); + handle = pau->regs[0] + PAU_OTL_MISC_PSL_PEHANDLE_AN(dev->index); + dt_add_property_cells(pd->dn, "ibm,opal-xsl-irq", irq); + dt_add_property_cells(pd->dn, "ibm,opal-xsl-mmio", + hi32(dsisr), lo32(dsisr), + hi32(dar), lo32(dar), + hi32(tfc), lo32(tfc), + hi32(handle), lo32(handle)); + return 0; +} + +static void pau_opencapi_phb_final_fixup(struct phb *phb) +{ + pci_walk_dev(phb, NULL, pau_opencapi_dt_add_interrupts, NULL); +} + +static int64_t pau_opencapi_set_pe(struct phb *phb, + uint64_t pe_num, + uint64_t bdfn, + uint8_t bcompare, + uint8_t dcompare, + uint8_t fcompare, + uint8_t action) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(phb); + struct pau *pau = dev->pau; + uint64_t val; + + PAUDEVDBG(dev, "Set partitionable endpoint = %08llx, bdfn = %08llx\n", + pe_num, bdfn); + + if (action != OPAL_MAP_PE && action != OPAL_UNMAP_PE) + return OPAL_PARAMETER; + + if (pe_num >= PAU_MAX_PE_NUM) + return OPAL_PARAMETER; + + if (bcompare != OpalPciBusAll || + dcompare != OPAL_COMPARE_RID_DEVICE_NUMBER || + fcompare != OPAL_COMPARE_RID_FUNCTION_NUMBER) + return OPAL_UNSUPPORTED; + + val = PAU_MISC_BDF2PE_CFG_ENABLE; + val = SETFIELD(PAU_MISC_BDF2PE_CFG_PE, val, pe_num); + val = SETFIELD(PAU_MISC_BDF2PE_CFG_BDF, val, 0); + pau_write(pau, PAU_MISC_BDF2PE_CFG(dev->index), val); + + return OPAL_SUCCESS; +} + static const struct phb_ops pau_opencapi_ops = { .cfg_read8 = pau_opencapi_pcicfg_read8, .cfg_read16 = pau_opencapi_pcicfg_read16, @@ -605,6 +737,11 @@ static const struct phb_ops pau_opencapi_ops = { .cfg_write8 = pau_opencapi_pcicfg_write8, .cfg_write16 = pau_opencapi_pcicfg_write16, .cfg_write32 = pau_opencapi_pcicfg_write32, + .eeh_freeze_status = pau_opencapi_eeh_freeze_status, + .next_error = pau_opencapi_next_error, + .ioda_reset = pau_opencapi_ioda_reset, + .phb_final_fixup = pau_opencapi_phb_final_fixup, + .set_pe = pau_opencapi_set_pe, }; static void pau_opencapi_create_phb(struct pau_dev *dev) diff --git a/include/pau-regs.h b/include/pau-regs.h index d98f435b..19b0b7cd 100644 --- a/include/pau-regs.h +++ b/include/pau-regs.h @@ -33,6 +33,7 @@ #define PAU_BLOCK_CQ_CTL PAU_BLOCK(4, 4) #define PAU_BLOCK_CQ_DAT PAU_BLOCK(4, 5) #define PAU_BLOCK_OTL(brk) PAU_BLOCK(4, 0xC + (brk)) +#define PAU_BLOCK_OTL_PSL(brk) PAU_BLOCK(0, 0xC + (brk)) #define PAU_BLOCK_XSL PAU_BLOCK(4, 0xE) #define PAU_BLOCK_PAU_XTS PAU_BLOCK(7, 1) #define PAU_BLOCK_PAU_MISC PAU_BLOCK(7, 2) @@ -117,6 +118,10 @@ #define PAU_OTL_MISC_CFG_TX_TEMP2_RATE PPC_BITMASK(16, 19) #define PAU_OTL_MISC_CFG_TX_TEMP3_RATE PPC_BITMASK(20, 23) #define PAU_OTL_MISC_CFG_TX_CRET_FREQ PPC_BITMASK(32, 34) +#define PAU_OTL_MISC_PSL_DSISR_AN(brk) (PAU_BLOCK_OTL_PSL(brk) + 0x000) +#define PAU_OTL_MISC_PSL_DAR_AN(brk) (PAU_BLOCK_OTL_PSL(brk) + 0x008) +#define PAU_OTL_MISC_PSL_TFC_AN(brk) (PAU_BLOCK_OTL_PSL(brk) + 0x010) +#define PAU_OTL_MISC_PSL_PEHANDLE_AN(brk) (PAU_BLOCK_OTL_PSL(brk) + 0x018) /* XSL block registers */ #define PAU_XSL_WRAP_CFG (PAU_BLOCK_XSL + 0x100) @@ -149,6 +154,10 @@ #define PAU_MISC_INT_1_CONFIG (PAU_BLOCK_PAU_MISC + 0x068) #define PAU_MISC_INT_BAR (PAU_BLOCK_PAU_MISC + 0x098) #define PAU_MISC_INT_BAR_ADDR PPC_BITMASK(0, 39) +#define PAU_MISC_BDF2PE_CFG(n) (PAU_BLOCK_PAU_MISC + 0x100 + (n) * 8) +#define PAU_MISC_BDF2PE_CFG_ENABLE PPC_BIT(0) +#define PAU_MISC_BDF2PE_CFG_PE PPC_BITMASK(4, 7) +#define PAU_MISC_BDF2PE_CFG_BDF PPC_BITMASK(8, 23) #define PAU_MISC_INT_2_CONFIG (PAU_BLOCK_PAU_MISC + 0x408) #define PAU_MISC_INT_2_CONFIG_XFAULT_2_5(n) PPC_BIT(0 + (n)) #define PAU_MISC_INT_2_CONFIG_XFAULT_0_1(n) PPC_BIT(54 + (n)) From patchwork Thu Oct 14 15:56:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540991 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=HLue+pKy; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYw66zVzz9sNH for ; Fri, 15 Oct 2021 02:58:14 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYw65qG3z3c77 for ; Fri, 15 Oct 2021 02:58:14 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=HLue+pKy; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=HLue+pKy; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYty6phzz2ynX for ; Fri, 15 Oct 2021 02:57:14 +1100 (AEDT) Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EFjwr9028301 for ; Thu, 14 Oct 2021 11:57:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=B/kms9+EiLXDulXvwEFfHSJDtIoH1Qde7Tw/u/Yogck=; b=HLue+pKy7HkH/uOY0ch+khgHS56kdLsutBGHpQ7ueQUaneHZVDQd3Lf2AjYaHkfe0uMS HghhF40GNtY7EdP7hCS24BnudiebZH2scfHSARcI3mvt0wqB95cUS8xW/vUKP4eUCB0q H19MPtU+B/DDfdfeW5TLtu0/5jElYyFpdO1dBo7oMuwjefIgzj83PQ2wfAaEBi/HPnA/ STevJWby0Mvvqp5SvxWaw95LO9QJezqBz3JZNPjLXgOlxTNyrKe22hm3bLn1opg1LV5r 1TMsNZry3cMBw7l+SWcot0selgg2TN9EyZejFNSqeif4LFA7xIJ3xgDxy/56Gv86aomx 0Q== Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0b-001b2d01.pphosted.com with ESMTP id 3bnrnwkuby-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:11 -0400 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFVsr3005770 for ; Thu, 14 Oct 2021 15:57:10 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma04fra.de.ibm.com with ESMTP id 3bk2qav2cw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:09 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFv7rC33685828 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:57:07 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2682C11C066 for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EAF1811C05B for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:06 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:56:59 +0200 Message-Id: <20211014155704.46441-11-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: UUZ_kmExjuZV0ikvNsEdqp36he0_KApq X-Proofpoint-GUID: UUZ_kmExjuZV0ikvNsEdqp36he0_KApq X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_08,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 malwarescore=0 bulkscore=0 spamscore=0 mlxlogscore=999 lowpriorityscore=0 suspectscore=0 adultscore=0 priorityscore=1501 mlxscore=0 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 10/15] pau: hmi scom dump X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" This patch add a new function to dump PAU registers when a HMI has been raised and an OpenCAPI link has been hit by an error. For each register, the scom address and the register value are printed. The hmi.c has been redesigned in order to support the new PHB/PCIEX type (PAU OpenCapi). Now, the *npu* functions support NPU and PAU units of P8, P9 and P10 chips. Signed-off-by: Christophe Lombard --- core/hmi.c | 276 +++++++++++++++++++-------------------- hw/npu2-common.c | 27 +--- hw/pau.c | 44 +++++++ include/npu2-regs.h | 5 + include/npu2.h | 2 +- include/pau-regs.h | 24 ++++ include/pau.h | 2 + include/xscom-p10-regs.h | 3 + 8 files changed, 214 insertions(+), 169 deletions(-) diff --git a/core/hmi.c b/core/hmi.c index 9363cc5f..ce5abd7d 100644 --- a/core/hmi.c +++ b/core/hmi.c @@ -19,8 +19,10 @@ #include #include #include +#include #include #include +#include #include #include #include @@ -717,13 +719,7 @@ static void find_nx_checkstop_reason(int flat_chip_id, queue_hmi_event(hmi_evt, 0, out_flags); } -static bool phb_is_npu2(struct dt_node *dn) -{ - return (dt_node_is_compatible(dn, "ibm,power9-npu-pciex") || - dt_node_is_compatible(dn, "ibm,power9-npu-opencapi-pciex")); -} - -static void add_npu2_xstop_reason(uint32_t *xstop_reason, uint8_t reason) +static void add_npu_xstop_reason(uint32_t *xstop_reason, uint8_t reason) { int i, reason_count; uint8_t *ptr; @@ -739,8 +735,8 @@ static void add_npu2_xstop_reason(uint32_t *xstop_reason, uint8_t reason) } } -static void encode_npu2_xstop_reason(uint32_t *xstop_reason, - uint64_t fir, int fir_number) +static void encode_npu_xstop_reason(uint32_t *xstop_reason, + uint64_t fir, int fir_number) { int bit; uint8_t reason; @@ -758,114 +754,125 @@ static void encode_npu2_xstop_reason(uint32_t *xstop_reason, bit = ilog2(fir); reason = fir_number << 6; reason |= (63 - bit); // IBM numbering - add_npu2_xstop_reason(xstop_reason, reason); + add_npu_xstop_reason(xstop_reason, reason); fir ^= 1ULL << bit; } } -static void find_npu2_checkstop_reason(int flat_chip_id, - struct OpalHMIEvent *hmi_evt, - uint64_t *out_flags) +static bool npu_fir_errors(struct phb *phb, int flat_chip_id, + uint32_t *xstop_reason) { - struct phb *phb; - int i; - bool npu2_hmi_verbose = false, found = false; - uint64_t npu2_fir; - uint64_t npu2_fir_mask; - uint64_t npu2_fir_action0; - uint64_t npu2_fir_action1; - uint64_t npu2_fir_addr; - uint64_t npu2_fir_mask_addr; - uint64_t npu2_fir_action0_addr; - uint64_t npu2_fir_action1_addr; + uint64_t fir, fir_mask; + uint64_t fir_action0, fir_action1; + uint64_t fir_reg, fir_mask_reg; + uint64_t fir_action0_reg, fir_action1_reg; uint64_t fatal_errors; - uint32_t xstop_reason = 0; - int total_errors = 0; + uint64_t xscom_base; + bool fir_errors = false; + int fir_regs; const char *loc; - - /* NPU2 only */ - if (PVR_TYPE(mfspr(SPR_PVR)) != PVR_TYPE_P9) - return; - - /* Find the NPU on the chip associated with the HMI. */ - for_each_phb(phb) { - /* NOTE: if a chip ever has >1 NPU this will need adjusting */ - if (phb_is_npu2(phb->dt_node) && - (dt_get_chip_id(phb->dt_node) == flat_chip_id)) { - found = true; - break; + struct npu *npu; + struct npu2 *npu2 = NULL; + struct npu2_dev *dev; + struct pau *pau; + + fir_regs = (phb->phb_type == phb_type_pcie_v3) ? 1 : 3; + + for (uint32_t i = 0; i < fir_regs; i++) { + switch (phb->phb_type) { + case phb_type_pcie_v3: + fir_reg = NX_FIR; + fir_mask_reg = NX_FIR_MASK; + fir_action0_reg = NX_FIR_ACTION0; + fir_action1_reg = NX_FIR_ACTION1; + + npu = phb_to_npu(phb); + if (npu != NULL) + xscom_base = npu->at_xscom; + else + continue; + break; + case phb_type_npu_v2: + fir_reg = NPU2_FIR(i); + fir_mask_reg = NPU2_FIR_MASK(i); + fir_action0_reg = NPU2_FIR_ACTION0(i); + fir_action1_reg = NPU2_FIR_ACTION1(i); + npu2 = phb_to_npu2_nvlink(phb); + xscom_base = npu2->xscom_base; + break; + case phb_type_npu_v2_opencapi: + fir_reg = NPU2_FIR(i); + fir_mask_reg = NPU2_FIR_MASK(i); + fir_action0_reg = NPU2_FIR_ACTION0(i); + fir_action1_reg = NPU2_FIR_ACTION1(i); + dev = phb_to_npu2_dev_ocapi(phb); + npu2 = dev->npu; + xscom_base = npu2->xscom_base; + break; + case phb_type_pau_opencapi: + fir_reg = PAU_FIR(i); + fir_mask_reg = PAU_FIR_MASK(i); + fir_action0_reg = PAU_FIR_ACTION0(i); + fir_action1_reg = PAU_FIR_ACTION1(i); + pau = ((struct pau_dev *)(pau_phb_to_opencapi_dev(phb)))->pau; + xscom_base = pau->xscom_base; + break; + default: + continue; } - } - - /* If we didn't find a NPU on the chip, it's not our checkstop. */ - if (!found) - return; - npu2_fir_addr = NPU2_FIR_REGISTER_0; - npu2_fir_mask_addr = NPU2_FIR_REGISTER_0 + NPU2_FIR_MASK_OFFSET; - npu2_fir_action0_addr = NPU2_FIR_REGISTER_0 + NPU2_FIR_ACTION0_OFFSET; - npu2_fir_action1_addr = NPU2_FIR_REGISTER_0 + NPU2_FIR_ACTION1_OFFSET; - - for (i = 0; i < NPU2_TOTAL_FIR_REGISTERS; i++) { - /* Read all the registers necessary to find a checkstop condition. */ - if (xscom_read(flat_chip_id, npu2_fir_addr, &npu2_fir) || - xscom_read(flat_chip_id, npu2_fir_mask_addr, &npu2_fir_mask) || - xscom_read(flat_chip_id, npu2_fir_action0_addr, &npu2_fir_action0) || - xscom_read(flat_chip_id, npu2_fir_action1_addr, &npu2_fir_action1)) { - prerror("HMI: Couldn't read NPU FIR register%d with XSCOM\n", i); + if (xscom_read(flat_chip_id, xscom_base + fir_reg, &fir) || + xscom_read(flat_chip_id, xscom_base + fir_mask_reg, &fir_mask) || + xscom_read(flat_chip_id, xscom_base + fir_action0_reg, &fir_action0) || + xscom_read(flat_chip_id, xscom_base + fir_action1_reg, &fir_action1)) { + prerror("HMI: Couldn't read NPU/PAU FIR register%d with XSCOM\n", i); continue; } - fatal_errors = npu2_fir & ~npu2_fir_mask & npu2_fir_action0 & npu2_fir_action1; + fatal_errors = fir & ~fir_mask & fir_action0 & fir_action1; if (fatal_errors) { loc = chip_loc_code(flat_chip_id); if (!loc) loc = "Not Available"; - prlog(PR_ERR, "NPU: [Loc: %s] P:%d FIR#%d FIR 0x%016llx mask 0x%016llx\n", - loc, flat_chip_id, i, npu2_fir, npu2_fir_mask); - prlog(PR_ERR, "NPU: [Loc: %s] P:%d ACTION0 0x%016llx, ACTION1 0x%016llx\n", - loc, flat_chip_id, npu2_fir_action0, npu2_fir_action1); - total_errors++; - - encode_npu2_xstop_reason(&xstop_reason, fatal_errors, i); + prlog(PR_ERR, "NPU/PAU: [Loc: %s] P:%d FIR#%d " + "FIR 0x%016llx mask 0x%016llx\n", + loc, flat_chip_id, i, fir, fir_mask); + prlog(PR_ERR, "NPU/PAU: [Loc: %s] P:%d ACTION0 " + "0x%016llx, ACTION1 0x%016llx\n", + loc, flat_chip_id, fir_action0, fir_action1); + if (phb->phb_type != phb_type_pcie_v3) + encode_npu_xstop_reason(xstop_reason, + fatal_errors, + i); + fir_errors = true; } - - /* Can't do a fence yet, we are just logging fir information for now */ - npu2_fir_addr += NPU2_FIR_OFFSET; - npu2_fir_mask_addr += NPU2_FIR_OFFSET; - npu2_fir_action0_addr += NPU2_FIR_OFFSET; - npu2_fir_action1_addr += NPU2_FIR_OFFSET; - } - if (!total_errors) - return; - - npu2_hmi_verbose = nvram_query_eq_safe("npu2-hmi-verbose", "true"); - /* Force this for now until we sort out something better */ - npu2_hmi_verbose = true; + /* dump registers */ + if (fir_errors) { + switch (phb->phb_type) { + case phb_type_npu_v2: + case phb_type_npu_v2_opencapi: + npu2_dump_scoms(npu2, flat_chip_id); + break; + case phb_type_pau_opencapi: + pau_opencapi_dump_scoms(pau); + break; + default: + break; + } - if (npu2_hmi_verbose) { - npu2_dump_scoms(flat_chip_id); prlog(PR_ERR, " _________________________ \n"); - prlog(PR_ERR, "< It's Debug time! >\n"); + prlog(PR_ERR, "< It's Debug time! >\n"); prlog(PR_ERR, " ------------------------- \n"); - prlog(PR_ERR, " \\ ,__, \n"); - prlog(PR_ERR, " \\ (oo)____ \n"); - prlog(PR_ERR, " (__) )\\ \n"); + prlog(PR_ERR, " \\ ,__, \n"); + prlog(PR_ERR, " \\ (oo)____ \n"); + prlog(PR_ERR, " (__) )\\ \n"); prlog(PR_ERR, " ||--|| * \n"); } - /* Set up the HMI event */ - hmi_evt->severity = OpalHMI_SEV_WARNING; - hmi_evt->type = OpalHMI_ERROR_MALFUNC_ALERT; - hmi_evt->u.xstop_error.xstop_type = CHECKSTOP_TYPE_NPU; - hmi_evt->u.xstop_error.xstop_reason = cpu_to_be32(xstop_reason); - hmi_evt->u.xstop_error.u.chip_id = cpu_to_be32(flat_chip_id); - - /* Marking the event as recoverable so that we don't crash */ - queue_hmi_event(hmi_evt, 1, out_flags); + return fir_errors; } static void find_npu_checkstop_reason(int flat_chip_id, @@ -873,67 +880,47 @@ static void find_npu_checkstop_reason(int flat_chip_id, uint64_t *out_flags) { struct phb *phb; - struct npu *p = NULL; - - uint64_t npu_fir; - uint64_t npu_fir_mask; - uint64_t npu_fir_action0; - uint64_t npu_fir_action1; - uint64_t fatal_errors; - - /* Only check for NPU errors if the chip has a NPU */ - if (PVR_TYPE(mfspr(SPR_PVR)) != PVR_TYPE_P8NVL) - return find_npu2_checkstop_reason(flat_chip_id, hmi_evt, out_flags); - - /* Find the NPU on the chip associated with the HMI. */ - for_each_phb(phb) { - /* NOTE: if a chip ever has >1 NPU this will need adjusting */ - if (dt_node_is_compatible(phb->dt_node, "ibm,power8-npu-pciex") && - (dt_get_chip_id(phb->dt_node) == flat_chip_id)) { - p = phb_to_npu(phb); - break; - } - } + struct dt_node *dn; + uint32_t xstop_reason = 0; - /* If we didn't find a NPU on the chip, it's not our checkstop. */ - if (p == NULL) + /* Only check for NPU errors if the chip has a NPU/PAU */ + if ((PVR_TYPE(mfspr(SPR_PVR)) != PVR_TYPE_P8NVL) && + (PVR_TYPE(mfspr(SPR_PVR)) != PVR_TYPE_P9) && + (PVR_TYPE(mfspr(SPR_PVR)) != PVR_TYPE_P10)) return; - /* Read all the registers necessary to find a checkstop condition. */ - if (xscom_read(flat_chip_id, - p->at_xscom + NX_FIR, &npu_fir) || - xscom_read(flat_chip_id, - p->at_xscom + NX_FIR_MASK, &npu_fir_mask) || - xscom_read(flat_chip_id, - p->at_xscom + NX_FIR_ACTION0, &npu_fir_action0) || - xscom_read(flat_chip_id, - p->at_xscom + NX_FIR_ACTION1, &npu_fir_action1)) { - prerror("Couldn't read NPU registers with XSCOM\n"); - return; - } + /* Find the NPU/PAU on the chip associated with the HMI. */ + for_each_phb(phb) { + dn = phb->dt_node; - fatal_errors = npu_fir & ~npu_fir_mask & npu_fir_action0 & npu_fir_action1; + if (!(dt_node_is_compatible(dn, "ibm,power8-npu-pciex") || + dt_node_is_compatible(dn, "ibm,power9-npu-pciex") || + dt_node_is_compatible(dn, "ibm,power9-npu-opencapi-pciex") || + dt_node_is_compatible(dn, "ibm,power10-pau-opencapi-pciex"))) + continue; - /* If there's no errors, we don't need to do anything. */ - if (!fatal_errors) - return; + if (dt_get_chip_id(dn) != flat_chip_id) + continue; - prlog(PR_DEBUG, "NPU: FIR 0x%016llx mask 0x%016llx\n", - npu_fir, npu_fir_mask); - prlog(PR_DEBUG, "NPU: ACTION0 0x%016llx, ACTION1 0x%016llx\n", - npu_fir_action0, npu_fir_action1); + /* Read all the registers necessary to find a checkstop condition. */ + if (!npu_fir_errors(phb, flat_chip_id, &xstop_reason)) + continue; - /* Set the NPU to fenced since it can't recover. */ - npu_set_fence_state(p, true); + if (phb->phb_type == phb_type_pcie_v3) { + /* Set the NPU to fenced since it can't recover. */ + npu_set_fence_state(phb_to_npu(phb), true); + } - /* Set up the HMI event */ - hmi_evt->severity = OpalHMI_SEV_WARNING; - hmi_evt->type = OpalHMI_ERROR_MALFUNC_ALERT; - hmi_evt->u.xstop_error.xstop_type = CHECKSTOP_TYPE_NPU; - hmi_evt->u.xstop_error.u.chip_id = cpu_to_be32(flat_chip_id); + /* Set up the HMI event */ + hmi_evt->severity = OpalHMI_SEV_WARNING; + hmi_evt->type = OpalHMI_ERROR_MALFUNC_ALERT; + hmi_evt->u.xstop_error.xstop_type = CHECKSTOP_TYPE_NPU; + hmi_evt->u.xstop_error.xstop_reason = xstop_reason; + hmi_evt->u.xstop_error.u.chip_id = cpu_to_be32(flat_chip_id); - /* The HMI is "recoverable" because it shouldn't crash the system */ - queue_hmi_event(hmi_evt, 1, out_flags); + /* Marking the event as recoverable so that we don't crash */ + queue_hmi_event(hmi_evt, 1, out_flags); + } } static void decode_malfunction(struct OpalHMIEvent *hmi_evt, uint64_t *out_flags) @@ -962,7 +949,8 @@ static void decode_malfunction(struct OpalHMIEvent *hmi_evt, uint64_t *out_flags xscom_write(this_cpu()->chip_id, malf_alert_scom, ~PPC_BIT(i)); find_capp_checkstop_reason(i, hmi_evt, &flags); - find_nx_checkstop_reason(i, hmi_evt, &flags); + if (proc_gen != proc_gen_p10) + find_nx_checkstop_reason(i, hmi_evt, &flags); find_npu_checkstop_reason(i, hmi_evt, &flags); } } diff --git a/hw/npu2-common.c b/hw/npu2-common.c index 3bc9bcee..b3f500f4 100644 --- a/hw/npu2-common.c +++ b/hw/npu2-common.c @@ -296,31 +296,10 @@ static void show_all_regs(struct npu2 *npu, int brick_index) } } -void npu2_dump_scoms(int chip_id) +void npu2_dump_scoms(struct npu2 *npu, int chip_id) { - struct npu2 *npu; - struct phb *phb; - struct npu2_dev *dev; - - /* - * Look for the npu2 structure for that chip ID. We can access it - * through the array of phbs, looking for a nvlink or opencapi - * phb. We can have several entries, but they all point - * to the same npu2 structure - */ - for_each_phb(phb) { - npu = NULL; - if (phb->phb_type == phb_type_npu_v2) { - npu = phb_to_npu2_nvlink(phb); - } else if (phb->phb_type == phb_type_npu_v2_opencapi) { - dev = phb_to_npu2_dev_ocapi(phb); - npu = dev->npu; - } - if (npu && npu->chip_id == chip_id) { - show_all_regs(npu, -1 /* all bricks */); - break; - } - } + if (npu && npu->chip_id == chip_id) + show_all_regs(npu, -1 /* all bricks */); } static uint64_t npu2_ipi_attributes(struct irq_source *is __unused, uint32_t isn __unused) diff --git a/hw/pau.c b/hw/pau.c index d632b2b0..324d1afc 100644 --- a/hw/pau.c +++ b/hw/pau.c @@ -32,6 +32,50 @@ struct pau_dev *pau_next_dev(struct pau *pau, struct pau_dev *dev, return NULL; } +static void pau_opencapi_dump_scom_reg(struct pau *pau, uint64_t reg) +{ + PAUDBG(pau, "0x%llx = 0x%016llx\n", reg, pau_read(pau, reg)); +} + +void pau_opencapi_dump_scoms(struct pau *pau) +{ + struct pau_dev *dev; + uint64_t cq_sm; + + for (uint32_t i = 1; i < 4; i++) { + cq_sm = PAU_BLOCK_CQ_SM(i); + + pau_opencapi_dump_scom_reg(pau, cq_sm + PAU_REG_OFFSET(PAU_MCP_MISC_CERR_MESSAGE0)); + pau_opencapi_dump_scom_reg(pau, cq_sm + PAU_REG_OFFSET(PAU_MCP_MISC_CERR_MESSAGE1)); + pau_opencapi_dump_scom_reg(pau, cq_sm + PAU_REG_OFFSET(PAU_MCP_MISC_CERR_MESSAGE2)); + pau_opencapi_dump_scom_reg(pau, cq_sm + PAU_REG_OFFSET(PAU_MCP_MISC_CERR_MESSAGE3)); + pau_opencapi_dump_scom_reg(pau, cq_sm + PAU_REG_OFFSET(PAU_MCP_MISC_CERR_MESSAGE4)); + pau_opencapi_dump_scom_reg(pau, cq_sm + PAU_REG_OFFSET(PAU_MCP_MISC_CERR_MESSAGE5)); + pau_opencapi_dump_scom_reg(pau, cq_sm + PAU_REG_OFFSET(PAU_MCP_MISC_CERR_MESSAGE6)); + pau_opencapi_dump_scom_reg(pau, cq_sm + PAU_REG_OFFSET(PAU_MCP_MISC_CERR_MESSAGE7)); + pau_opencapi_dump_scom_reg(pau, cq_sm + PAU_REG_OFFSET(PAU_MCP_MISC_CERR_FIRST0)); + pau_opencapi_dump_scom_reg(pau, cq_sm + PAU_REG_OFFSET(PAU_MCP_MISC_CERR_FIRST1)); + pau_opencapi_dump_scom_reg(pau, cq_sm + PAU_REG_OFFSET(PAU_MCP_MISC_CERR_FIRST2)); + } + + pau_opencapi_dump_scom_reg(pau, PAU_CTL_MISC_CERR_MESSAGE0); + pau_opencapi_dump_scom_reg(pau, PAU_CTL_MISC_CERR_MESSAGE1); + pau_opencapi_dump_scom_reg(pau, PAU_CTL_MISC_CERR_MESSAGE2); + pau_opencapi_dump_scom_reg(pau, PAU_CTL_MISC_CERR_FIRST0); + pau_opencapi_dump_scom_reg(pau, PAU_CTL_MISC_CERR_FIRST1); + pau_opencapi_dump_scom_reg(pau, PAU_DAT_MISC_CERR_ECC_HOLD); + pau_opencapi_dump_scom_reg(pau, PAU_DAT_MISC_CERR_ECC_MASK); + pau_opencapi_dump_scom_reg(pau, PAU_DAT_MISC_CERR_ECC_FIRST); + + pau_for_each_opencapi_dev(dev, pau) { + pau_opencapi_dump_scom_reg(pau, PAU_OTL_MISC_ERR_RPT_HOLD0(dev->index)); + pau_opencapi_dump_scom_reg(pau, PAU_OTL_MISC_OTL_REM0(dev->index)); + pau_opencapi_dump_scom_reg(pau, PAU_OTL_MISC_ERROR_SIG_RXI(dev->index)); + pau_opencapi_dump_scom_reg(pau, PAU_OTL_MISC_ERROR_SIG_RXO(dev->index)); + pau_opencapi_dump_scom_reg(pau, PAU_OTL_MISC_ERR_RPT_HOLD1(dev->index)); + } +} + static void pau_dt_create_link(struct dt_node *pau, uint32_t pau_index, uint32_t dev_index) { diff --git a/include/npu2-regs.h b/include/npu2-regs.h index 22f58a6a..cb1d3956 100644 --- a/include/npu2-regs.h +++ b/include/npu2-regs.h @@ -610,6 +610,11 @@ void npu2_scom_write(uint64_t gcid, uint64_t scom_base, #define NPU2_TOTAL_FIR_REGISTERS 3 +#define NPU2_FIR(n) (0x2c00 + (n) * 0x40) +#define NPU2_FIR_MASK(n) (0x2c03 + (n) * 0x40) +#define NPU2_FIR_ACTION0(n) (0x2c06 + (n) * 0x40) +#define NPU2_FIR_ACTION1(n) (0x2c07 + (n) * 0x40) + /* * Can't use enums for 64 bit values, use #defines */ diff --git a/include/npu2.h b/include/npu2.h index 23b06b4b..a12bf98a 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -241,7 +241,7 @@ int64_t npu2_freeze_status(struct phb *phb __unused, uint8_t *freeze_state, uint16_t *pci_error_type __unused, uint16_t *severity __unused); -void npu2_dump_scoms(int chip_id); +void npu2_dump_scoms(struct npu2 *npu, int chip_id); int64_t npu2_init_context(struct phb *phb, uint64_t msr, uint64_t bdf); int64_t npu2_destroy_context(struct phb *phb, uint64_t bdf); diff --git a/include/pau-regs.h b/include/pau-regs.h index 19b0b7cd..b852a5b5 100644 --- a/include/pau-regs.h +++ b/include/pau-regs.h @@ -48,6 +48,17 @@ #define PAU_MCP_MISC_CFG0_MA_MCRESP_OPT_WRP PPC_BIT(9) #define PAU_MCP_MISC_CFG0_ENABLE_PBUS PPC_BIT(26) #define PAU_MCP_MISC_CFG0_OCAPI_MODE PPC_BITMASK(44, 48) +#define PAU_MCP_MISC_CERR_MESSAGE0 (PAU_BLOCK_CQ_SM(0) + 0x030) +#define PAU_MCP_MISC_CERR_MESSAGE1 (PAU_BLOCK_CQ_SM(0) + 0x038) +#define PAU_MCP_MISC_CERR_MESSAGE2 (PAU_BLOCK_CQ_SM(0) + 0x040) +#define PAU_MCP_MISC_CERR_MESSAGE3 (PAU_BLOCK_CQ_SM(0) + 0x048) +#define PAU_MCP_MISC_CERR_MESSAGE4 (PAU_BLOCK_CQ_SM(0) + 0x050) +#define PAU_MCP_MISC_CERR_MESSAGE5 (PAU_BLOCK_CQ_SM(0) + 0x058) +#define PAU_MCP_MISC_CERR_MESSAGE6 (PAU_BLOCK_CQ_SM(0) + 0x060) +#define PAU_MCP_MISC_CERR_MESSAGE7 (PAU_BLOCK_CQ_SM(0) + 0x068) +#define PAU_MCP_MISC_CERR_FIRST0 (PAU_BLOCK_CQ_SM(0) + 0x078) +#define PAU_MCP_MISC_CERR_FIRST1 (PAU_BLOCK_CQ_SM(0) + 0x080) +#define PAU_MCP_MISC_CERR_FIRST2 (PAU_BLOCK_CQ_SM(0) + 0x088) #define PAU_SNP_MISC_CFG0 (PAU_BLOCK_CQ_SM(0) + 0x180) #define PAU_SNP_MISC_CFG0_ENABLE_PBUS PPC_BIT(2) #define PAU_SNP_MISC_CFG0_OCAPI_MODE PPC_BITMASK(32, 36) @@ -79,6 +90,11 @@ #define PAU_CTL_MISC_MMIOPA_CONFIG(brk) (PAU_BLOCK_CQ_CTL + 0x098 + (brk) * 8) #define PAU_CTL_MISC_MMIOPA_CONFIG_BAR_ADDR PPC_BITMASK(1, 35) #define PAU_CTL_MISC_MMIOPA_CONFIG_BAR_SIZE PPC_BITMASK(39, 43) +#define PAU_CTL_MISC_CERR_MESSAGE0 (PAU_BLOCK_CQ_CTL + 0x0C0) +#define PAU_CTL_MISC_CERR_MESSAGE1 (PAU_BLOCK_CQ_CTL + 0x0C8) +#define PAU_CTL_MISC_CERR_MESSAGE2 (PAU_BLOCK_CQ_CTL + 0x0D0) +#define PAU_CTL_MISC_CERR_FIRST0 (PAU_BLOCK_CQ_CTL + 0x0D8) +#define PAU_CTL_MISC_CERR_FIRST1 (PAU_BLOCK_CQ_CTL + 0x0E0) #define PAU_CTL_MISC_FENCE_CTRL(brk) (PAU_BLOCK_CQ_CTL + 0x108 + (brk) * 8) #define PAU_CTL_MISC_FENCE_REQUEST PPC_BITMASK(0, 1) #define PAU_CTL_MISC_CFG_ADDR(brk) (PAU_BLOCK_CQ_CTL + 0x250 + (brk) * 8) @@ -93,6 +109,9 @@ /* CQ_DAT block registers */ #define PAU_DAT_MISC_CFG1 (PAU_BLOCK_CQ_DAT + 0x008) #define PAU_DAT_MISC_CFG1_OCAPI_MODE PPC_BITMASK(40, 44) +#define PAU_DAT_MISC_CERR_ECC_HOLD (PAU_BLOCK_CQ_DAT + 0x020) +#define PAU_DAT_MISC_CERR_ECC_MASK (PAU_BLOCK_CQ_DAT + 0x028) +#define PAU_DAT_MISC_CERR_ECC_FIRST (PAU_BLOCK_CQ_DAT + 0x030) /* OTL block registers */ #define PAU_OTL_MISC_CFG0(brk) (PAU_BLOCK_OTL(brk) + 0x000) @@ -102,6 +121,7 @@ #define PAU_OTL_MISC_CFG0_ENABLE_4_0 PPC_BIT(51) #define PAU_OTL_MISC_CFG0_XLATE_RELEASE PPC_BIT(62) #define PAU_OTL_MISC_CFG0_ENABLE_5_0 PPC_BIT(63) +#define PAU_OTL_MISC_ERR_RPT_HOLD0(brk) (PAU_BLOCK_OTL(brk) + 0x030) #define PAU_OTL_MISC_CFG_TLX_CREDITS(brk) (PAU_BLOCK_OTL(brk) + 0x050) #define PAU_OTL_MISC_CFG_TLX_CREDITS_VC0 PPC_BITMASK(0, 7) #define PAU_OTL_MISC_CFG_TLX_CREDITS_VC1 PPC_BITMASK(8, 15) @@ -118,6 +138,10 @@ #define PAU_OTL_MISC_CFG_TX_TEMP2_RATE PPC_BITMASK(16, 19) #define PAU_OTL_MISC_CFG_TX_TEMP3_RATE PPC_BITMASK(20, 23) #define PAU_OTL_MISC_CFG_TX_CRET_FREQ PPC_BITMASK(32, 34) +#define PAU_OTL_MISC_OTL_REM0(brk) (PAU_BLOCK_OTL(brk) + 0x068) +#define PAU_OTL_MISC_ERROR_SIG_RXI(brk) (PAU_BLOCK_OTL(brk) + 0x070) +#define PAU_OTL_MISC_ERROR_SIG_RXO(brk) (PAU_BLOCK_OTL(brk) + 0x078) +#define PAU_OTL_MISC_ERR_RPT_HOLD1(brk) (PAU_BLOCK_OTL(brk) + 0x0B0) #define PAU_OTL_MISC_PSL_DSISR_AN(brk) (PAU_BLOCK_OTL_PSL(brk) + 0x000) #define PAU_OTL_MISC_PSL_DAR_AN(brk) (PAU_BLOCK_OTL_PSL(brk) + 0x008) #define PAU_OTL_MISC_PSL_TFC_AN(brk) (PAU_BLOCK_OTL_PSL(brk) + 0x010) diff --git a/include/pau.h b/include/pau.h index b6fabe7f..547c1934 100644 --- a/include/pau.h +++ b/include/pau.h @@ -189,4 +189,6 @@ static inline uint64_t pau_read(struct pau *pau, uint64_t reg) return pau_scom_read(pau, reg, PAU_MISC_DA_LEN_8B); } +void pau_opencapi_dump_scoms(struct pau *pau); + #endif /* __PAU_H */ diff --git a/include/xscom-p10-regs.h b/include/xscom-p10-regs.h index 21ac21f8..5ca4703f 100644 --- a/include/xscom-p10-regs.h +++ b/include/xscom-p10-regs.h @@ -15,6 +15,9 @@ #define P10_NX_DMA_ENGINE_FIR 0x02011100 /* DMA & Engine FIR Data Register */ #define P10_NX_PBI_FIR 0x02011080 /* PowerBus Interface FIR Register */ +/* pMisc Receive Malfunction Alert Register */ +#define P10_MALFUNC_ALERT 0x00090022 + #define P10_EC_CORE_THREAD_STATE 0x412 /* XXX P10 is this right? */ #define P10_THREAD_STOPPED(t) PPC_BIT(56 + (t)) From patchwork Thu Oct 14 15:57:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540990 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=JcpWfoMj; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYvz0c3Kz9sNH for ; Fri, 15 Oct 2021 02:58:07 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYvy69C4z30JK for ; Fri, 15 Oct 2021 02:58:06 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=JcpWfoMj; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=JcpWfoMj; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYty5nM3z2yg4 for ; Fri, 15 Oct 2021 02:57:14 +1100 (AEDT) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EF1KcX030561 for ; Thu, 14 Oct 2021 11:57:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=NRb856b9DQauGHFOVPiHzPHTMjoqeg2C8bYjMh7DX0M=; b=JcpWfoMjodgX5RA1TwSCkHx1WiSh2V2W5EhSxjlGkXNwtGnFLWiY+mrpq9lFSErS4n23 fvp4b0o69DmbhZqgQRwMhI6u24msDf0Krof2PEaDC3wIK5rsPgeZvR8wvqa3R4r4/aNa 4Pt+M1/P5g9oKBoSHIjm4JTGuzxvC7n5vy0Eh29Mo3Z18Aaag9pfFZ3RYjl+08kTcDhL tqs1I5/zc9l++54HcRIMor9CAckPPv0NvZn3KxsJF+IQFtJ879E77X6WQTAPdInirY9H /3XeD5RahcBYGaKqkkcNowU6RC6ytnDo7+gHzabw8a+2yinnLBAsRaPvB5LONLFaR7rd 7A== Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0b-001b2d01.pphosted.com with ESMTP id 3bpgv4k37p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:11 -0400 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFXVfl029854 for ; Thu, 14 Oct 2021 15:57:10 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma03ams.nl.ibm.com with ESMTP id 3bk2qa6yst-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:09 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFv7EU57672178 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:57:07 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5BB9211C052 for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 359B711C069 for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:57:00 +0200 Message-Id: <20211014155704.46441-12-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: DXG2MkckDFUWnFrXhu36QaqLZ_pRrUPg X-Proofpoint-ORIG-GUID: DXG2MkckDFUWnFrXhu36QaqLZ_pRrUPg X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_08,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 phishscore=0 mlxscore=0 priorityscore=1501 mlxlogscore=999 impostorscore=0 malwarescore=0 lowpriorityscore=0 bulkscore=0 spamscore=0 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 11/15] pau: phy init X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Follow the Procedure IO_INIT_RESET_PON as described in the P10 OPHY workbook document to reset and initialize the PHY lanes. The memory mapped SRAM (64 bit aligned) has to be used to configure the PHY, which is reachable the linked registers: address and data. The different links can be configured at the same time, that implies using a global lock to avoid conflicts. Authored-by: Frederic Barrat Signed-off-by: Christophe Lombard --- hw/Makefile.inc | 2 +- hw/pau-hw-procedures.c | 310 +++++++++++++++++++++++++++++++++++++++++ include/pau.h | 11 ++ 3 files changed, 322 insertions(+), 1 deletion(-) create mode 100644 hw/pau-hw-procedures.c diff --git a/hw/Makefile.inc b/hw/Makefile.inc index 0b12f0be..9fcbb63b 100644 --- a/hw/Makefile.inc +++ b/hw/Makefile.inc @@ -8,7 +8,7 @@ HW_OBJS += dts.o lpc-rtc.o npu.o npu-hw-procedures.o xive.o phb4.o HW_OBJS += fake-nvram.o lpc-mbox.o npu2.o npu2-hw-procedures.o HW_OBJS += npu2-common.o npu2-opencapi.o phys-map.o sbe-p9.o capp.o HW_OBJS += occ-sensor.o vas.o sbe-p8.o dio-p9.o lpc-port80h.o cache-p9.o -HW_OBJS += npu-opal.o ocmb.o xive2.o pau.o +HW_OBJS += npu-opal.o ocmb.o xive2.o pau.o pau-hw-procedures.o HW=hw/built-in.a include $(SRC)/hw/fsp/Makefile.inc diff --git a/hw/pau-hw-procedures.c b/hw/pau-hw-procedures.c new file mode 100644 index 00000000..bd299b3f --- /dev/null +++ b/hw/pau-hw-procedures.c @@ -0,0 +1,310 @@ +// SPDX-License-Identifier: Apache-2.0 +/* + * Copyright 2020 IBM Corp. + */ +#include +#include + +#define PAU_PHY_INIT_TIMEOUT 8000 /* ms */ + +#define PAU_PHY_ADDR_REG 0x10012C0D +#define PAU_PHY_ADDR_CHIPLET PPC_BITMASK(32, 39) +#define PAU_PHY_ADDR_SRAM_ADDR PPC_BITMASK(15, 31) +#define PAU_PHY_DATA_REG 0x10012C0E +#define PAU_PHY_DATA_CHIPLET PPC_BITMASK(32, 39) + +#define PAU_MAX_PHY_LANE 18 + +/* + * We configure the PHY using the memory mapped SRAM, which is + * accessible through a pair of (addr, data) registers. The caveat is + * that accesses to the SRAM must be 64-bit aligned, yet the PHY + * registers are 16-bit, so special care is needed. + * + * A PAU chiplet may control up to 2 OP units = 4 links and each link + * has its own virtual PHB in skiboot. They can be initialized or + * reset concurrently so we need a lock when accessing the SRAM. + + * See section "5.2.5 PPE SRAM" of the workbook for the layout of the + * SRAM registers. Here is the subset of the table which is meaningful + * for us, since we're only touching a few registers: + * + * Address Bytes Linker Symbol Description + * FFFF_11B0 16 _fw_regs0_start fw_regs for thread 0 + * FFFF_11C0 16 _fw_regs1_start fw_regs for thread 1 + * + * FFFF_2800 1024 _mem_regs0_start mem_regs for thread 0 + * FFFF_2C00 1024 _mem_regs1_start mem_regs for thread 1 + * + * In each PAU, per-group registers are replicated for every OP (each + * OP units is being called a 'thread' in the workbook). + * Per-lane registers have an offset < 0x10 and are replicated for + * each lane. Their offset in their section is: + * 0byyyyyxxxx (y = 5-bit lane number, x = 4-bit per-lane register offset) + */ + +struct PPE_sram_section { + uint32_t offset; + uint32_t size; +}; + +static struct PPE_sram_section PPE_FIRMWARE = { 0x111B0, 0x10 }; +static struct PPE_sram_section PPE_MEMORY = { 0x12800, 0x400 }; + +struct PPE_sram_reg { + struct PPE_sram_section *section; + uint32_t offset; +}; + +/* PPE firmware */ +static struct PPE_sram_reg PAU_PHY_EXT_CMD_LANES_00_15 = { &PPE_FIRMWARE, 0x000 }; +static struct PPE_sram_reg PAU_PHY_EXT_CMD_LANES_16_31 = { &PPE_FIRMWARE, 0x001 }; +static struct PPE_sram_reg PAU_PHY_EXT_CMD_REQ = { &PPE_FIRMWARE, 0x002 }; +#define PAU_PHY_EXT_CMD_REQ_IO_RESET PPC_BIT16(1) +#define PAU_PHY_EXT_CMD_REQ_DCCAL PPC_BIT16(3) +#define PAU_PHY_EXT_CMD_REQ_TX_ZCAL PPC_BIT16(4) +#define PAU_PHY_EXT_CMD_REQ_TX_FFE PPC_BIT16(5) +#define PAU_PHY_EXT_CMD_REQ_POWER_ON PPC_BIT16(7) +static struct PPE_sram_reg PAU_PHY_EXT_CMD_DONE = { &PPE_FIRMWARE, 0x005 }; + +/* PPE memory */ +static struct PPE_sram_reg PAU_PHY_RX_PPE_CNTL1 = { &PPE_MEMORY, 0x000 }; +#define PAU_PHY_RX_ENABLE_AUTO_RECAL PPC_BIT16(1) + +enum pau_phy_status { + PAU_PROC_INPROGRESS, + PAU_PROC_COMPLETE, + PAU_PROC_NEXT, + PAU_PROC_FAILED +}; + +struct procedure { + const char *name; + uint32_t (*steps[])(struct pau_dev *); +}; + +#define DEFINE_PROCEDURE(NAME, STEPS...) \ + static struct procedure procedure_##NAME = { \ + .name = #NAME, \ + .steps = { STEPS } \ + } + +/* + * We could/should have one phy_sram_lock per PAU chiplet. Each PAU + * chiplet drives 2 OPT units. Since we don't have a PAU chiplet + * structure to host the lock and don't anticipate much contention, we + * go with a global lock for now + */ +static struct lock phy_sram_lock = LOCK_UNLOCKED; + +static int get_thread_id(uint32_t op_unit) +{ + int ppe_thread[8] = { 0, 1, 1, 0, 1, 0, 1, 0 }; + + /* static mapping between OP unit and PPE thread ID */ + if (op_unit >= sizeof(ppe_thread)) + return -1; + return ppe_thread[op_unit]; +} + +/* + * Compute the address in the memory mapped SRAM of a 16-bit PHY register + */ +static uint32_t pau_phy_sram_addr(struct pau_dev *dev, + struct PPE_sram_reg *reg, + int lane) +{ + uint32_t base, addr; + + base = reg->section->offset + + reg->section->size * get_thread_id(dev->op_unit); + addr = reg->offset; + if (lane >= 0) { + assert(reg->offset < 0x10); + addr += lane << 4; + } + addr <<= 1; // each register is 16-bit + return base + addr; +} + +static void pau_phy_set_access(struct pau_dev *dev, + struct PPE_sram_reg *reg, int lane, + uint64_t *data_addr, uint64_t *mask) +{ + struct pau *pau = dev->pau; + uint64_t scom_addr, sram_addr, addr, bit_start; + + scom_addr = SETFIELD(PAU_PHY_ADDR_CHIPLET, PAU_PHY_ADDR_REG, + pau->op_chiplet); + sram_addr = pau_phy_sram_addr(dev, reg, lane); + bit_start = 8 * (sram_addr & 7); + + addr = SETFIELD(PAU_PHY_ADDR_SRAM_ADDR, 0ull, sram_addr & 0xFFFFFFF8); + xscom_write(pau->chip_id, scom_addr, addr); + + *data_addr = SETFIELD(PAU_PHY_DATA_CHIPLET, PAU_PHY_DATA_REG, + pau->op_chiplet); + *mask = PPC_BITMASK(bit_start, bit_start + 15); +} + +static void pau_phy_write_lane(struct pau_dev *dev, + struct PPE_sram_reg *reg, int lane, + uint16_t val) +{ + struct pau *pau = dev->pau; + uint64_t data_addr, scom_val, mask; + + lock(&phy_sram_lock); + pau_phy_set_access(dev, reg, lane, &data_addr, &mask); + xscom_read(pau->chip_id, data_addr, &scom_val); + scom_val = SETFIELD(mask, scom_val, val); + xscom_write(pau->chip_id, data_addr, scom_val); + unlock(&phy_sram_lock); +} + +static uint16_t pau_phy_read_lane(struct pau_dev *dev, + struct PPE_sram_reg *reg, int lane) +{ + struct pau *pau = dev->pau; + uint64_t data_addr, scom_val, mask; + uint16_t res; + + lock(&phy_sram_lock); + pau_phy_set_access(dev, reg, lane, &data_addr, &mask); + xscom_read(pau->chip_id, data_addr, &scom_val); + res = GETFIELD(mask, scom_val); + unlock(&phy_sram_lock); + return res; +} + +static void pau_phy_write(struct pau_dev *dev, struct PPE_sram_reg *reg, + uint16_t val) +{ + pau_phy_write_lane(dev, reg, -1, val); +} + +static uint16_t pau_phy_read(struct pau_dev *dev, struct PPE_sram_reg *reg) +{ + return pau_phy_read_lane(dev, reg, -1); +} + +static uint16_t get_reset_request_val(void) +{ + return PAU_PHY_EXT_CMD_REQ_IO_RESET | + PAU_PHY_EXT_CMD_REQ_DCCAL | + PAU_PHY_EXT_CMD_REQ_TX_ZCAL | + PAU_PHY_EXT_CMD_REQ_TX_FFE | + PAU_PHY_EXT_CMD_REQ_POWER_ON; +} + +static uint32_t reset_start(struct pau_dev *dev) +{ + uint16_t val16; + + // Procedure IO_INIT_RESET_PON + + // Clear external command request / done registers + val16 = 0; + pau_phy_write(dev, &PAU_PHY_EXT_CMD_REQ, val16); + pau_phy_write(dev, &PAU_PHY_EXT_CMD_DONE, val16); + + // Write the external command lanes to target + val16 = dev->phy_lane_mask >> 16; + pau_phy_write(dev, &PAU_PHY_EXT_CMD_LANES_00_15, val16); + val16 = dev->phy_lane_mask & 0xFFFF; + pau_phy_write(dev, &PAU_PHY_EXT_CMD_LANES_16_31, val16); + + // Initialize PHY Lanes + val16 = get_reset_request_val(); + pau_phy_write(dev, &PAU_PHY_EXT_CMD_REQ, val16); + return PAU_PROC_NEXT; +} + +static uint32_t reset_check(struct pau_dev *dev) +{ + uint16_t val16, done; + + val16 = get_reset_request_val(); + done = pau_phy_read(dev, &PAU_PHY_EXT_CMD_DONE); + + if (val16 == done) + return PAU_PROC_NEXT; + else + return PAU_PROC_INPROGRESS; +} + +static uint32_t enable_recal(struct pau_dev *dev) +{ + uint32_t lane; + + // Enable auto-recalibration + for (lane = 0; lane <= PAU_MAX_PHY_LANE; lane++) + if (!(dev->phy_lane_mask & (1 << (31 - lane)))) + continue; + else + pau_phy_write_lane(dev, &PAU_PHY_RX_PPE_CNTL1, + lane, PAU_PHY_RX_ENABLE_AUTO_RECAL); + + return PAU_PROC_COMPLETE; +} + +DEFINE_PROCEDURE(phy_reset, reset_start, reset_check, enable_recal); + +static enum pau_phy_status run_steps(struct pau_dev *dev) +{ + struct procedure *p = &procedure_phy_reset; + struct phy_proc_state *procedure_state = &dev->pau->procedure_state; + enum pau_phy_status rc; + + do { + rc = p->steps[procedure_state->step](dev); + if (rc == PAU_PROC_NEXT) { + procedure_state->step++; + PAUDEVDBG(dev, "Running procedure %s step %d\n", + p->name, procedure_state->step); + } + } while (rc == PAU_PROC_NEXT); + return rc; +} + +static enum pau_phy_status run_procedure(struct pau_dev *dev) +{ + struct procedure *p = &procedure_phy_reset; + struct phy_proc_state *procedure_state = &dev->pau->procedure_state; + enum pau_phy_status rc; + + do { + rc = run_steps(dev); + if (rc == PAU_PROC_INPROGRESS) { + if (tb_compare(mftb(), procedure_state->timeout) == TB_AAFTERB) { + PAUDEVERR(dev, "Procedure %s timed out\n", p->name); + rc = PAU_PROC_FAILED; + } else { + time_wait_ms(1); + } + } + } while (rc == PAU_PROC_INPROGRESS); + return rc; +} + +int pau_dev_phy_reset(struct pau_dev *dev) +{ + struct procedure *p = &procedure_phy_reset; + struct phy_proc_state *procedure_state = &dev->pau->procedure_state; + enum pau_phy_status rc; + + lock(&procedure_state->lock); + procedure_state->step = 0; + procedure_state->timeout = mftb() + msecs_to_tb(PAU_PHY_INIT_TIMEOUT); + PAUDEVDBG(dev, "Running procedure %s step %d\n", + p->name, procedure_state->step); + rc = run_procedure(dev); + unlock(&procedure_state->lock); + + if (rc == PAU_PROC_COMPLETE) { + PAUDEVDBG(dev, "Procedure %s complete\n", p->name); + return OPAL_SUCCESS; + } + PAUDEVDBG(dev, "Procedure %s failed\n", p->name); + return OPAL_HARDWARE; +} diff --git a/include/pau.h b/include/pau.h index 547c1934..9fbccb87 100644 --- a/include/pau.h +++ b/include/pau.h @@ -28,6 +28,12 @@ struct pau_bar { uint64_t cfg; }; +struct phy_proc_state { + struct lock lock; /* protect any change to this structure */ + unsigned long timeout; + uint16_t step; +}; + struct pau_dev { enum pau_dev_type type; uint32_t index; @@ -54,6 +60,7 @@ struct pau { uint32_t index; struct dt_node *dt_node; uint32_t chip_id; + uint32_t op_chiplet; /* from pervasive: 0x10 -> 0x13 */ uint64_t xscom_base; /* Global MMIO window (all PAU regs) */ @@ -65,6 +72,7 @@ struct pau { uint32_t links; struct pau_dev devices[PAU_LINKS_OPENCAPI_PER_PAU]; + struct phy_proc_state procedure_state; }; #define PAUDBG(pau, fmt, a...) PAULOG(PR_DEBUG, pau, fmt, ##a) @@ -191,4 +199,7 @@ static inline uint64_t pau_read(struct pau *pau, uint64_t reg) void pau_opencapi_dump_scoms(struct pau *pau); +/* PHY */ +int pau_dev_phy_reset(struct pau_dev *dev); + #endif /* __PAU_H */ From patchwork Thu Oct 14 15:57:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540996 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=FI61GsF8; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYws1C26z9sNH for ; Fri, 15 Oct 2021 02:58:53 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYws0Lkbz3cBj for ; Fri, 15 Oct 2021 02:58:53 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=FI61GsF8; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=FI61GsF8; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYv12wMGz2ypn for ; Fri, 15 Oct 2021 02:57:17 +1100 (AEDT) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EFGv4v017855 for ; Thu, 14 Oct 2021 11:57:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=pt2auQcjoBn4edZLlejYPYmI6N6z9NE556upscAEf3Q=; b=FI61GsF8pukh4qoeVeavfplmIRZieAOeTVVjkl3xIKHM8Pr6p3S5FkZbrev5pC+asEMU yFaQd/2I3QBpdLKXo9USk8I6l9TkpZitieDCk7KMRLDQVYxd0Yo+bzPJzsQWfR11eGsD 6xZ+j51yNlWNxtfW+K5VofzipYhXB/LcpisoFMZECl8z0nRG4UnNlGIYP6re/+XXbYey dXcr1SiCE4IQttP2zNzDYo6uRAIt/aiqyBXXB3SE+YZnMwHUBCTT1Thg4OAPfjg+hOQU LzyVGaqL8d9qAvJQxIhgZuzMEOBUaIJdJvvM3/ymbWdAvXx2KjDuEQamN4ciVjUPeCtb 7A== Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0a-001b2d01.pphosted.com with ESMTP id 3bnrnduqb1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:12 -0400 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFVpYW005710 for ; Thu, 14 Oct 2021 15:57:10 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma04fra.de.ibm.com with ESMTP id 3bk2qav2cy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:10 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFv7LM25100780 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:57:07 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8FE8911C050 for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6A2C411C04C for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:57:01 +0200 Message-Id: <20211014155704.46441-13-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: BvUiF7PdC6Kq2XlujWrEfgUigVrh3uvj X-Proofpoint-ORIG-GUID: BvUiF7PdC6Kq2XlujWrEfgUigVrh3uvj X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_08,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 mlxlogscore=999 adultscore=0 spamscore=0 lowpriorityscore=0 bulkscore=0 mlxscore=0 suspectscore=0 phishscore=0 impostorscore=0 clxscore=1015 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 12/15] pau: link training X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Add elementary functions to handle a phb complete, fundamental and hot resets. For the time being, specific creset and hreset are not supported. A complete fundamental reset is based on the following steps, in this order: - Place all bricks into Fence state - Disable BARs - Reset ODL to Power-on Values - Set the i2c reset pin in output mode - Initialize PHY Lanes - Deassert ODL reset - Clear the the i2c reset pin - Unfence bricks - Enable BARs - Enable ODL training mode Link training is also set up. Signed-off-by: Christophe Lombard --- hw/pau.c | 529 +++++++++++++++++++++++++++++++++++++++ include/pau-regs.h | 5 + include/pau.h | 2 + include/xscom-p10-regs.h | 46 ++++ 4 files changed, 582 insertions(+) diff --git a/hw/pau.c b/hw/pau.c index 324d1afc..8425567c 100644 --- a/hw/pau.c +++ b/hw/pau.c @@ -14,6 +14,22 @@ #define PAU_MAX_PE_NUM 16 #define PAU_RESERVED_PE_NUM 15 +#define PAU_SLOT_NORMAL PCI_SLOT_STATE_NORMAL +#define PAU_SLOT_LINK PCI_SLOT_STATE_LINK +#define PAU_SLOT_LINK_START (PAU_SLOT_LINK + 1) +#define PAU_SLOT_LINK_WAIT (PAU_SLOT_LINK + 2) +#define PAU_SLOT_LINK_TRAINED (PAU_SLOT_LINK + 3) +#define PAU_SLOT_FRESET PCI_SLOT_STATE_FRESET +#define PAU_SLOT_FRESET_START (PAU_SLOT_FRESET + 1) +#define PAU_SLOT_FRESET_INIT (PAU_SLOT_FRESET + 2) +#define PAU_SLOT_FRESET_ASSERT_DELAY (PAU_SLOT_FRESET + 3) +#define PAU_SLOT_FRESET_DEASSERT_DELAY (PAU_SLOT_FRESET + 4) +#define PAU_SLOT_FRESET_INIT_DELAY (PAU_SLOT_FRESET + 5) + +#define PAU_LINK_TRAINING_RETRIES 2 +#define PAU_LINK_TRAINING_TIMEOUT 15000 /* ms */ +#define PAU_LINK_STATE_TRAINED 0x7 + struct pau_dev *pau_next_dev(struct pau *pau, struct pau_dev *dev, enum pau_dev_type type) { @@ -167,6 +183,7 @@ static void pau_dt_create_pau(struct dt_node *xscom, uint32_t pau_index) dt_add_property_cells(pau, "#address-cells", 1); dt_add_property_cells(pau, "reg", pau_base[pau_index], 0x2c); dt_add_property_string(pau, "compatible", "ibm,power10-pau"); + dt_add_property_cells(pau, "ibm,pau-chiplet", pau_base[pau_index] >> 24); dt_add_property_cells(pau, "ibm,pau-index", pau_index); links = PAU_LINKS_OPENCAPI_PER_PAU; @@ -201,12 +218,14 @@ static struct pau *pau_create(struct dt_node *dn) assert(pau); init_lock(&pau->lock); + init_lock(&pau->procedure_state.lock); pau->dt_node = dn; pau->index = dt_prop_get_u32(dn, "ibm,pau-index"); pau->xscom_base = dt_get_address(dn, 0, NULL); pau->chip_id = dt_get_chip_id(dn); + pau->op_chiplet = dt_prop_get_u32(dn, "ibm,pau-chiplet"); assert(get_chip(pau->chip_id)); pau->links = PAU_LINKS_OPENCAPI_PER_PAU; @@ -502,6 +521,452 @@ static void pau_opencapi_enable_bars(struct pau_dev *dev, bool enable) pau_write(pau, reg, val); } +static int64_t pau_opencapi_creset(struct pci_slot *slot) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(slot->phb); + + PAUDEVERR(dev, "creset not supported\n"); + return OPAL_UNSUPPORTED; +} + +static int64_t pau_opencapi_hreset(struct pci_slot *slot) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(slot->phb); + + PAUDEVERR(dev, "hreset not supported\n"); + return OPAL_UNSUPPORTED; +} + +static void pau_opencapi_assert_odl_reset(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint64_t reg, val; + + reg = P10_OB_ODL_CONFIG(dev->op_unit, dev->odl_index); + val = P10_OB_ODL_CONFIG_RESET; + val = SETFIELD(P10_OB_ODL_CONFIG_VERSION, val, 0b000100); // OCAPI 4 + val = SETFIELD(P10_OB_ODL_CONFIG_TRAIN_MODE, val, 0b0101); // ts2 + val = SETFIELD(P10_OB_ODL_CONFIG_SUPPORTED_MODES, val, 0b0010); + val |= P10_OB_ODL_CONFIG_X4_BACKOFF_ENABLE; + val = SETFIELD(P10_OB_ODL_CONFIG_PHY_CNTR_LIMIT, val, 0b1111); + val |= P10_OB_ODL_CONFIG_DEBUG_ENABLE; + val = SETFIELD(P10_OB_ODL_CONFIG_FWD_PROGRESS_TIMER, val, 0b0110); + xscom_write(pau->chip_id, reg, val); +} + +static void pau_opencapi_deassert_odl_reset(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint64_t reg, val; + + reg = P10_OB_ODL_CONFIG(dev->op_unit, dev->odl_index); + xscom_read(pau->chip_id, reg, &val); + val &= ~P10_OB_ODL_CONFIG_RESET; + xscom_write(pau->chip_id, reg, val); +} + +static void pau_opencapi_training_mode(struct pau_dev *dev, + uint8_t pattern) +{ + struct pau *pau = dev->pau; + uint64_t reg, val; + + reg = P10_OB_ODL_CONFIG(dev->op_unit, dev->odl_index); + xscom_read(pau->chip_id, reg, &val); + val = SETFIELD(P10_OB_ODL_CONFIG_TRAIN_MODE, val, pattern); + xscom_write(pau->chip_id, reg, val); +} + +static int64_t pau_opencapi_assert_adapter_reset(struct pau_dev *dev) +{ + int64_t rc = OPAL_PARAMETER; + + if (platform.ocapi->i2c_assert_reset) + rc = platform.ocapi->i2c_assert_reset(dev->i2c_bus_id); + + if (rc) + PAUDEVERR(dev, "Error writing I2C reset signal: %lld\n", rc); + return rc; +} + +static int64_t pau_opencapi_deassert_adapter_reset(struct pau_dev *dev) +{ + int64_t rc = OPAL_PARAMETER; + + if (platform.ocapi->i2c_deassert_reset) + rc = platform.ocapi->i2c_deassert_reset(dev->i2c_bus_id); + + if (rc) + PAUDEVERR(dev, "Error writing I2C reset signal: %lld\n", rc); + return rc; +} + +static void pau_opencapi_fence_brick(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + + PAUDEVDBG(dev, "Fencing brick\n"); + pau_opencapi_set_fence_control(dev, 0b11); + + /* Place all bricks into Fence state */ + pau_write(pau, PAU_MISC_FENCE_STATE, + PAU_MISC_FENCE_STATE_SET(pau_dev_index(dev, PAU_LINKS_OPENCAPI_PER_PAU))); +} + +static void pau_opencapi_unfence_brick(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + + PAUDEVDBG(dev, "Unfencing brick\n"); + pau_write(pau, PAU_MISC_FENCE_STATE, + PAU_MISC_FENCE_STATE_CLEAR(pau_dev_index(dev, PAU_LINKS_OPENCAPI_PER_PAU))); + + pau_opencapi_set_fence_control(dev, 0b10); + pau_opencapi_set_fence_control(dev, 0b00); +} + +static int64_t pau_opencapi_freset(struct pci_slot *slot) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(slot->phb); + uint8_t presence = 1; + int64_t rc = OPAL_SUCCESS; + + switch (slot->state) { + case PAU_SLOT_NORMAL: + case PAU_SLOT_FRESET_START: + PAUDEVDBG(dev, "FRESET: Starts\n"); + + if (slot->ops.get_presence_state) + slot->ops.get_presence_state(slot, &presence); + if (!presence) { + /* + * FIXME: if there's no card on the link, we + * should consider powering off the unused + * lanes to save energy + */ + PAUDEVINF(dev, "no card detected\n"); + return OPAL_SUCCESS; + } + slot->link_retries = PAU_LINK_TRAINING_RETRIES; + /* fall-through */ + case PAU_SLOT_FRESET_INIT: + pau_opencapi_fence_brick(dev); + pau_opencapi_enable_bars(dev, false); + pau_opencapi_assert_odl_reset(dev); + pau_opencapi_assert_adapter_reset(dev); + pci_slot_set_state(slot, PAU_SLOT_FRESET_ASSERT_DELAY); + /* assert for 5ms */ + return pci_slot_set_sm_timeout(slot, msecs_to_tb(5)); + + case PAU_SLOT_FRESET_ASSERT_DELAY: + rc = pau_dev_phy_reset(dev); + if (rc) { + PAUDEVERR(dev, "FRESET: PHY reset error\n"); + return OPAL_HARDWARE; + } + pau_opencapi_deassert_odl_reset(dev); + pau_opencapi_deassert_adapter_reset(dev); + pci_slot_set_state(slot, PAU_SLOT_FRESET_DEASSERT_DELAY); + /* give 250ms to device to be ready */ + return pci_slot_set_sm_timeout(slot, msecs_to_tb(250)); + + case PAU_SLOT_FRESET_DEASSERT_DELAY: + pau_opencapi_unfence_brick(dev); + pau_opencapi_enable_bars(dev, true); + pau_opencapi_training_mode(dev, 0b0001); /* send pattern A */ + pci_slot_set_state(slot, PAU_SLOT_FRESET_INIT_DELAY); + return pci_slot_set_sm_timeout(slot, msecs_to_tb(5)); + + case PAU_SLOT_FRESET_INIT_DELAY: + pau_opencapi_training_mode(dev, 0b1000); /* enable training */ + dev->train_start = mftb(); + dev->train_timeout = dev->train_start + + msecs_to_tb(PAU_LINK_TRAINING_TIMEOUT); + pci_slot_set_state(slot, PAU_SLOT_LINK_START); + return slot->ops.poll_link(slot); + + default: + PAUDEVERR(dev, "FRESET: unexpected slot state %08x\n", + slot->state); + } + pci_slot_set_state(slot, PAU_SLOT_NORMAL); + return OPAL_HARDWARE; +} + +static uint64_t pau_opencapi_get_odl_endpoint_info(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint64_t val; + + xscom_read(pau->chip_id, + P10_OB_ODL_DLX_INFO(dev->op_unit, dev->odl_index), + &val); + return val; +} + +static uint64_t pau_opencapi_get_odl_training_status(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint64_t val; + + xscom_read(pau->chip_id, + P10_OB_ODL_TRAIN_STAT(dev->op_unit, dev->odl_index), + &val); + return val; +} + +static uint64_t pau_opencapi_get_odl_status(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint64_t val; + + xscom_read(pau->chip_id, + P10_OB_ODL_STATUS(dev->op_unit, dev->odl_index), + &val); + return val; +} + +static uint64_t pau_opencapi_get_odl_link_speed_status(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint64_t val; + + xscom_read(pau->chip_id, + P10_OB_ODL_LINK_SPEED_STATUS(dev->op_unit, dev->odl_index), + &val); + return val; +} + +static enum OpalShpcLinkState pau_opencapi_get_link_width(uint64_t status) +{ + uint64_t tx_lanes, rx_lanes, state; + + state = GETFIELD(P10_OB_ODL_STATUS_TRAINING_STATE, status); + if (state != PAU_LINK_STATE_TRAINED) + return OPAL_SHPC_LINK_DOWN; + + rx_lanes = GETFIELD(P10_OB_ODL_STATUS_RX_TRAINED_LANES, status); + tx_lanes = GETFIELD(P10_OB_ODL_STATUS_TX_TRAINED_LANES, status); + if ((rx_lanes != 0xFF) || (tx_lanes != 0xFF)) + return OPAL_SHPC_LINK_UP_x4; + else + return OPAL_SHPC_LINK_UP_x8; +} + +static int64_t pau_opencapi_get_link_state(struct pci_slot *slot, + uint8_t *val) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(slot->phb); + uint64_t status; + + status = pau_opencapi_get_odl_status(dev); + *val = pau_opencapi_get_link_width(status); + + return OPAL_SUCCESS; + +} + +static int64_t pau_opencapi_get_power_state(struct pci_slot *slot, + uint8_t *val) +{ + *val = slot->power_state; + return OPAL_SUCCESS; +} + +static int64_t pau_opencapi_get_presence_state(struct pci_slot __unused * slot, + uint8_t *val) +{ + /* + * Presence detection for OpenCAPI is currently done at the start of + * PAU initialisation, and we only create slots if a device is present. + * As such we will never be asked to get the presence of a slot that's + * empty. + * + * This may change if we ever support hotplug down the track. + */ + *val = OPAL_PCI_SLOT_PRESENT; + return OPAL_SUCCESS; +} + +static void pau_opencapi_check_trained_link(struct pau_dev *dev, + uint64_t status) +{ + if (pau_opencapi_get_link_width(status) != OPAL_SHPC_LINK_UP_x8) { + PAUDEVERR(dev, "Link trained in degraded mode (%016llx)\n", + status); + PAUDEVDBG(dev, "Link endpoint info: %016llx\n", + pau_opencapi_get_odl_endpoint_info(dev)); + } +} + +static int64_t pau_opencapi_retry_state(struct pci_slot *slot, + uint64_t status) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(slot->phb); + + if (!slot->link_retries--) { + /** + * @fwts-label OCAPILinkTrainingFailed + * @fwts-advice The OpenCAPI link training procedure failed. + * This indicates a hardware or firmware bug. OpenCAPI + * functionality will not be available on this link. + */ + PAUDEVERR(dev, + "Link failed to train, final link status: %016llx\n", + status); + PAUDEVDBG(dev, "Final link training status: %016llx (Link Speed Status: %016llx)\n", + pau_opencapi_get_odl_training_status(dev), + pau_opencapi_get_odl_link_speed_status(dev)); + return OPAL_HARDWARE; + } + + PAUDEVERR(dev, "Link failed to train, retrying\n"); + PAUDEVERR(dev, "Link status: %016llx, training status: %016llx " + "(Link Speed Status: %016llx)\n", + status, + pau_opencapi_get_odl_training_status(dev), + pau_opencapi_get_odl_link_speed_status(dev)); + + pci_slot_set_state(slot, PAU_SLOT_FRESET_INIT); + return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); +} + +static void pau_opencapi_otl_tx_send_enable(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint64_t reg, val; + + /* Allows OTL TX to send out packets to AFU */ + PAUDEVDBG(dev, "OTL TX Send Enable\n"); + + reg = PAU_OTL_MISC_CFG_TX2(dev->index); + val = pau_read(pau, reg); + val |= PAU_OTL_MISC_CFG_TX2_SEND_EN; + pau_write(pau, reg, val); +} + +static void pau_opencapi_setup_perf_counters(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint64_t reg, val; + + PAUDEVDBG(dev, "Setup perf counter\n"); + + reg = P10_OB_ODL_PERF_MON_CONFIG(dev->op_unit); + xscom_read(pau->chip_id, reg, &val); + val = SETFIELD(P10_OB_ODL_PERF_MON_CONFIG_ENABLE, val, + P10_OB_ODL_PERF_MON_CONFIG_LINK0 >> dev->index); + val = SETFIELD(P10_OB_ODL_PERF_MON_CONFIG_SIZE, val, + P10_OB_ODL_PERF_MON_CONFIG_SIZE16); + xscom_write(pau->chip_id, reg, val); + PAUDEVDBG(dev, "perf counter config %llx = %llx\n", reg, val); + + reg = P10_OB_ODL_PERF_MON_SELECT(dev->op_unit); + xscom_read(pau->chip_id, reg, &val); + val = SETFIELD(P10_OB_ODL_PERF_MON_SELECT_COUNTER >> (dev->index * 16), + val, P10_OB_ODL_PERF_MON_SELECT_CRC_ODL); + val = SETFIELD(P10_OB_ODL_PERF_MON_SELECT_COUNTER >> ((dev->index * 16) + 8), + val, P10_OB_ODL_PERF_MON_SELECT_CRC_DLX); + xscom_write(pau->chip_id, reg, val); + PAUDEVDBG(dev, "perf counter select %llx = %llx\n", reg, val); +} + +static void pau_opencapi_check_perf_counters(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint64_t reg, val; + + reg = P10_OB_PERF_COUNTER0(dev->op_unit); + xscom_read(pau->chip_id, reg, &val); + + if (val) + PAUDEVERR(dev, "CRC error count perf_counter0..3=0%#llx\n", + val); +} + +static int64_t pau_opencapi_poll_link(struct pci_slot *slot) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(slot->phb); + uint64_t status; + + switch (slot->state) { + case PAU_SLOT_NORMAL: + case PAU_SLOT_LINK_START: + PAUDEVDBG(dev, "Start polling\n"); + pci_slot_set_state(slot, PAU_SLOT_LINK_WAIT); + /* fall-through */ + case PAU_SLOT_LINK_WAIT: + status = pau_opencapi_get_odl_status(dev); + if (GETFIELD(P10_OB_ODL_STATUS_TRAINING_STATE, status) == + PAU_LINK_STATE_TRAINED) { + PAUDEVINF(dev, "link trained in %ld ms (Link Speed Status: %016llx)\n", + tb_to_msecs(mftb() - dev->train_start), + pau_opencapi_get_odl_link_speed_status(dev)); + pau_opencapi_check_trained_link(dev, status); + + pci_slot_set_state(slot, PAU_SLOT_LINK_TRAINED); + return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); + } + if (tb_compare(mftb(), dev->train_timeout) == TB_AAFTERB) + return pau_opencapi_retry_state(slot, status); + + return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); + + case PAU_SLOT_LINK_TRAINED: + pau_opencapi_otl_tx_send_enable(dev); + pci_slot_set_state(slot, PAU_SLOT_NORMAL); + if (dev->status & PAU_DEV_STATUS_BROKEN) { + PAUDEVERR(dev, "Resetting a device which hit a " + "previous error. Device recovery " + "is not supported, so future behavior is undefined\n"); + dev->status &= ~PAU_DEV_STATUS_BROKEN; + } + pau_opencapi_check_perf_counters(dev); + dev->phb.scan_map = 1; + return OPAL_SUCCESS; + + default: + PAUDEVERR(dev, "unexpected slot state %08x\n", slot->state); + + } + pci_slot_set_state(slot, PAU_SLOT_NORMAL); + return OPAL_HARDWARE; +} + +static void pau_opencapi_prepare_link_change(struct pci_slot *slot __unused, + bool up __unused) +{ + /* + * PCI hotplug wants it defined, but we don't need to do anything + */ +} + +static int64_t pau_opencapi_set_power_state(struct pci_slot *slot, + uint8_t val) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(slot->phb); + + switch (val) { + case PCI_SLOT_POWER_OFF: + PAUDEVDBG(dev, "Fake power off\n"); + pau_opencapi_fence_brick(dev); + pau_opencapi_assert_adapter_reset(dev); + slot->power_state = PCI_SLOT_POWER_OFF; + return OPAL_SUCCESS; + + case PCI_SLOT_POWER_ON: + if (slot->power_state != PCI_SLOT_POWER_OFF) + return OPAL_SUCCESS; + PAUDEVDBG(dev, "Fake power on\n"); + slot->power_state = PCI_SLOT_POWER_ON; + slot->state = PAU_SLOT_NORMAL; + return OPAL_SUCCESS; + + default: + return OPAL_UNSUPPORTED; + } +} + static void pau_opencapi_create_phb_slot(struct pau_dev *dev) { struct pci_slot *slot; @@ -515,6 +980,21 @@ static void pau_opencapi_create_phb_slot(struct pau_dev *dev) */ PAUDEVERR(dev, "Cannot create PHB slot\n"); } + + /* Elementary functions */ + slot->ops.creset = pau_opencapi_creset; + slot->ops.hreset = pau_opencapi_hreset; + slot->ops.freset = pau_opencapi_freset; + slot->ops.get_link_state = pau_opencapi_get_link_state; + slot->ops.get_power_state = pau_opencapi_get_power_state; + slot->ops.get_presence_state = pau_opencapi_get_presence_state; + slot->ops.poll_link = pau_opencapi_poll_link; + slot->ops.prepare_link_change = pau_opencapi_prepare_link_change; + slot->ops.set_power_state = pau_opencapi_set_power_state; + + /* hotplug capability */ + slot->pluggable = 1; + } static int64_t pau_opencapi_pcicfg_check(struct pau_dev *dev, @@ -826,6 +1306,26 @@ static void pau_opencapi_dt_add_mmio_window(struct pau_dev *dev) hi32(mm_win[1]), lo32(mm_win[1])); } +static void pau_opencapi_dt_add_hotpluggable(struct pau_dev *dev) +{ + struct pci_slot *slot = dev->phb.slot; + struct dt_node *dn = dev->phb.dt_node; + char label[40]; + + /* + * Add a few definitions to the DT so that the linux PCI + * hotplug framework can find the slot and identify it as + * hot-pluggable. + * + * The "ibm,slot-label" property is used by linux as the slot name + */ + pci_slot_add_dt_properties(slot, dn); + + snprintf(label, sizeof(label), "OPENCAPI-%04x", + (int)PCI_SLOT_PHB_INDEX(slot->id)); + dt_add_property_string(dn, "ibm,slot-label", label); +} + static void pau_opencapi_dt_add_props(struct pau_dev *dev) { struct dt_node *dn = dev->phb.dt_node; @@ -855,6 +1355,7 @@ static void pau_opencapi_dt_add_props(struct pau_dev *dev) dt_add_property_cells(dn, "ibm,opal-reserved-pe", PAU_RESERVED_PE_NUM); pau_opencapi_dt_add_mmio_window(dev); + pau_opencapi_dt_add_hotpluggable(dev); } static void pau_opencapi_set_transport_mux_controls(struct pau_dev *dev) @@ -873,6 +1374,30 @@ static void pau_opencapi_set_transport_mux_controls(struct pau_dev *dev) pau_write(pau, reg, val); } +static void pau_opencapi_odl_config_phy(struct pau_dev *dev) +{ + struct pau *pau = dev->pau; + uint8_t typemap = 0; + uint64_t reg, val; + + PAUDEVDBG(dev, "Configure ODL\n"); + + /* ODL must be in reset when enabling. + * It stays in reset until the link is trained + */ + pau_opencapi_assert_odl_reset(dev); + + /* DLO (Open CAPI links) */ + typemap = 0x2 >> dev->odl_index; + + reg = P10_OB_ODL_PHY_CONFIG(dev->op_unit); + xscom_read(pau->chip_id, reg, &val); + typemap |= GETFIELD(P10_OB_ODL_PHY_CONFIG_LINK_SELECT, val); + val = SETFIELD(P10_OB_ODL_PHY_CONFIG_LINK_SELECT, val, typemap); + val = SETFIELD(P10_OB_ODL_PHY_CONFIG_DL_SELECT, val, 0b10); + xscom_write(pau->chip_id, reg, val); +} + static void pau_opencapi_enable_xsl_clocks(struct pau *pau) { uint64_t reg, val; @@ -1164,6 +1689,7 @@ static void pau_opencapi_init_hw(struct pau *pau) pau_for_each_opencapi_dev(dev, pau) { PAUDEVINF(dev, "Configuring link ...\n"); pau_opencapi_set_transport_mux_controls(dev); /* step 1 */ + pau_opencapi_odl_config_phy(dev); } pau_opencapi_enable_xsl_clocks(pau); /* step 2 */ pau_opencapi_enable_misc_clocks(pau); /* step 3 */ @@ -1231,6 +1757,9 @@ static void pau_opencapi_init_hw(struct pau *pau) /* done in pau_opencapi_setup_irqs() */ pau_opencapi_enable_interrupt_on_error(dev); + /* enable performance monitor */ + pau_opencapi_setup_perf_counters(dev); + /* Reset disabled. Place OTLs into Run State */ pau_opencapi_set_fence_control(dev, 0b00); diff --git a/include/pau-regs.h b/include/pau-regs.h index b852a5b5..7a5aaa5f 100644 --- a/include/pau-regs.h +++ b/include/pau-regs.h @@ -142,6 +142,8 @@ #define PAU_OTL_MISC_ERROR_SIG_RXI(brk) (PAU_BLOCK_OTL(brk) + 0x070) #define PAU_OTL_MISC_ERROR_SIG_RXO(brk) (PAU_BLOCK_OTL(brk) + 0x078) #define PAU_OTL_MISC_ERR_RPT_HOLD1(brk) (PAU_BLOCK_OTL(brk) + 0x0B0) +#define PAU_OTL_MISC_CFG_TX2(brk) (PAU_BLOCK_OTL(brk) + 0x0C0) +#define PAU_OTL_MISC_CFG_TX2_SEND_EN PPC_BIT(0) #define PAU_OTL_MISC_PSL_DSISR_AN(brk) (PAU_BLOCK_OTL_PSL(brk) + 0x000) #define PAU_OTL_MISC_PSL_DAR_AN(brk) (PAU_BLOCK_OTL_PSL(brk) + 0x008) #define PAU_OTL_MISC_PSL_TFC_AN(brk) (PAU_BLOCK_OTL_PSL(brk) + 0x010) @@ -178,6 +180,9 @@ #define PAU_MISC_INT_1_CONFIG (PAU_BLOCK_PAU_MISC + 0x068) #define PAU_MISC_INT_BAR (PAU_BLOCK_PAU_MISC + 0x098) #define PAU_MISC_INT_BAR_ADDR PPC_BITMASK(0, 39) +#define PAU_MISC_FENCE_STATE (PAU_BLOCK_PAU_MISC + 0x0B0) +#define PAU_MISC_FENCE_STATE_CLEAR(brk) PPC_BIT(0 + (brk)) +#define PAU_MISC_FENCE_STATE_SET(brk) PPC_BIT(12 + (brk)) #define PAU_MISC_BDF2PE_CFG(n) (PAU_BLOCK_PAU_MISC + 0x100 + (n) * 8) #define PAU_MISC_BDF2PE_CFG_ENABLE PPC_BIT(0) #define PAU_MISC_BDF2PE_CFG_PE PPC_BITMASK(4, 7) diff --git a/include/pau.h b/include/pau.h index 9fbccb87..c0a09401 100644 --- a/include/pau.h +++ b/include/pau.h @@ -40,6 +40,8 @@ struct pau_dev { struct dt_node *dn; struct phb phb; uint32_t status; + unsigned long train_start; + unsigned long train_timeout; struct pau_bar ntl_bar; struct pau_bar genid_bar; diff --git a/include/xscom-p10-regs.h b/include/xscom-p10-regs.h index 5ca4703f..51fec518 100644 --- a/include/xscom-p10-regs.h +++ b/include/xscom-p10-regs.h @@ -58,4 +58,50 @@ #define P10_ROOT_CONTROL_7 0x50017 +/* PB DLL Configuration Registers */ +#define P10_OB_ODL(ob) (0x18011000 + (ob) * 0x1000000) + +#define P10_OB_ODL_PHY_CONFIG(ob) (P10_OB_ODL(ob) + 0x0C) +#define P10_OB_ODL_PHY_CONFIG_LINK_SELECT PPC_BITMASK(56, 57) +#define P10_OB_ODL_PHY_CONFIG_DL_SELECT PPC_BITMASK(62, 63) + +#define P10_OB_ODL_PERF_MON_CONFIG(ob) (P10_OB_ODL(ob) + 0x1C) +#define P10_OB_ODL_PERF_MON_CONFIG_ENABLE PPC_BITMASK(0, 1) +#define P10_OB_ODL_PERF_MON_CONFIG_LINK0 0b10 +#define P10_OB_ODL_PERF_MON_CONFIG_LINK1 0b01 +#define P10_OB_ODL_PERF_MON_CONFIG_SIZE PPC_BITMASK(16, 23) +#define P10_OB_ODL_PERF_MON_CONFIG_SIZE16 0xFF + +#define P10_OB_ODL_PERF_MON_SELECT(ob) (P10_OB_ODL(ob) + 0x1D) +#define P10_OB_ODL_PERF_MON_SELECT_COUNTER PPC_BITMASK(0, 7) +#define P10_OB_ODL_PERF_MON_SELECT_CRC_ODL 0x44 +#define P10_OB_ODL_PERF_MON_SELECT_CRC_DLX 0x45 + +#define P10_OB_PERF_COUNTER0(ob) (P10_OB_ODL(ob) + 0x1E) +#define P10_OB_PERF_COUNTER0_LOW PPC_BITMASK(0, 31) +#define P10_OB_PERF_COUNTER0_HIGH PPC_BITMASK(32, 63) + +#define P10_OB_ODL_CONFIG(ob, brk) (P10_OB_ODL(ob) + 0x2A + brk) +#define P10_OB_ODL_CONFIG_RESET PPC_BIT(0) +#define P10_OB_ODL_CONFIG_VERSION PPC_BITMASK(2, 7) +#define P10_OB_ODL_CONFIG_TRAIN_MODE PPC_BITMASK(8, 11) +#define P10_OB_ODL_CONFIG_SUPPORTED_MODES PPC_BITMASK(12, 15) +#define P10_OB_ODL_CONFIG_X4_BACKOFF_ENABLE PPC_BIT(16) +#define P10_OB_ODL_CONFIG_PHY_CNTR_LIMIT PPC_BITMASK(20, 23) +#define P10_OB_ODL_CONFIG_DEBUG_ENABLE PPC_BIT(33) +#define P10_OB_ODL_CONFIG_FWD_PROGRESS_TIMER PPC_BITMASK(40, 43) + +#define P10_OB_ODL_STATUS(ob, brk) (P10_OB_ODL(ob) + 0x2C + brk) +#define P10_OB_ODL_STATUS_TRAINED_MODE PPC_BITMASK(0, 3) +#define P10_OB_ODL_STATUS_RX_TRAINED_LANES PPC_BITMASK(16, 23) +#define P10_OB_ODL_STATUS_TX_TRAINED_LANES PPC_BITMASK(24, 31) +#define P10_OB_ODL_STATUS_TRAINING_STATE PPC_BITMASK(49, 51) + +#define P10_OB_ODL_TRAIN_STAT(ob, brk) (P10_OB_ODL(ob) + 0x2E + brk) +#define P10_OB_ODL_TRAIN_STAT_PATTERN_B PPC_BITMASK(8, 15) + +#define P10_OB_ODL_DLX_INFO(ob, brk) (P10_OB_ODL(ob) + 0x32 + brk) + +#define P10_OB_ODL_LINK_SPEED_STATUS(ob, brk) (P10_OB_ODL(ob) + 0x34 + brk) + #endif /* __XSCOM_P10_REGS_H__ */ From patchwork Thu Oct 14 15:57:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540995 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=a5lSu6YZ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYwj69B1z9sNH for ; Fri, 15 Oct 2021 02:58:45 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYwj5KR9z3c9h for ; Fri, 15 Oct 2021 02:58:45 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=a5lSu6YZ; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0b-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=a5lSu6YZ; dkim-atps=neutral Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYv04d31z2ypn for ; Fri, 15 Oct 2021 02:57:16 +1100 (AEDT) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EFaERB011745 for ; Thu, 14 Oct 2021 11:57:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Bhv1B+vbYcpPfdYA7edV/stbmdF6xsyinfsM5hCz7DQ=; b=a5lSu6YZS6P/+dHzaP+HM4dO6sZoj5MaBhTuVcOmRxlqqLOu1LHiMIFLF1Yu3xGdWyZ2 qRIb/CAO6N3AZMP/PhLmx1bU5C3ILp3cldUQ4ajusBOg3lj4ECCSc0KujkhvS4RFzm82 +u6IVUOdKuvQN1rzxDE8ePgy3H4N3klB4YNFDzyQVFmfZ5FWWmE8pd2WgDM76PoVBUFd 2sSh3JlBg/3B//3kC29lV7WOK9oDG/ISTYkNP3vW5nl/2OEX1AtZTSipGx/BqtEqarDZ aRRmGx8YyuNC0O9eY/Nlg+L1NPpRgTstCaRUUoKAT/cVxoxMOKZCMxIG7NIKCw/fTaIv sQ== Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0a-001b2d01.pphosted.com with ESMTP id 3bnnvgy56h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:12 -0400 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFVpYX005710 for ; Thu, 14 Oct 2021 15:57:10 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma04fra.de.ibm.com with ESMTP id 3bk2qav2d2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:10 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFv7rJ57672182 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:57:08 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CFD9211C054 for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A8D0E11C06E for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:57:02 +0200 Message-Id: <20211014155704.46441-14-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: GfxITk72oet401_XlO3_Nz1hTAcmdXls X-Proofpoint-ORIG-GUID: GfxITk72oet401_XlO3_Nz1hTAcmdXls X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_09,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 mlxlogscore=999 bulkscore=0 malwarescore=0 suspectscore=0 phishscore=0 mlxscore=0 lowpriorityscore=0 priorityscore=1501 spamscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 13/15] pau: update current opal call functions X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Update the content of three current OPAL API calls to support PAU. - OPAL_NPU_SPA_SETUP The Shared Process Area (SPA) is a table containing one entry (a "Process Element") per memory context which can be accessed by the OpenCAPI device. - OPAL_NPU_SPA_CLEAR_CACHE The PAU keeps a cache of recently accessed memory contexts. When a Process Element is removed from the SPA, the cache for the link must be cleared. - OPAL_NPU_TL_SET The Transaction Layer specification defines several templates for messages to be exchanged on the link. During link setup, the host and device must negotiate what templates are supported on both sides and at what rates those messages can be sent. Signed-off-by: Christophe Lombard Reviewed-by: Frederic Barrat --- hw/npu-opal.c | 11 ++++ hw/pau.c | 154 +++++++++++++++++++++++++++++++++++++++++++++ include/pau-regs.h | 13 ++++ include/pau.h | 7 +++ 4 files changed, 185 insertions(+) diff --git a/hw/npu-opal.c b/hw/npu-opal.c index 73158b1c..cf136901 100644 --- a/hw/npu-opal.c +++ b/hw/npu-opal.c @@ -7,6 +7,7 @@ #include #include #include +#include #define TL_RATE_BUF_SIZE 32 @@ -184,6 +185,9 @@ static int64_t opal_npu_spa_setup(uint64_t phb_id, uint32_t bdfn, if (phb->phb_type == phb_type_npu_v2_opencapi) return npu2_opencapi_spa_setup(phb, bdfn, addr, PE_mask); + if (phb->phb_type == phb_type_pau_opencapi) + return pau_opencapi_spa_setup(phb, bdfn, addr, PE_mask); + return OPAL_PARAMETER; } opal_call(OPAL_NPU_SPA_SETUP, opal_npu_spa_setup, 4); @@ -202,6 +206,9 @@ static int64_t opal_npu_spa_clear_cache(uint64_t phb_id, uint32_t bdfn, if (phb->phb_type == phb_type_npu_v2_opencapi) return npu2_opencapi_spa_clear_cache(phb, bdfn, PE_handle); + if (phb->phb_type == phb_type_pau_opencapi) + return pau_opencapi_spa_clear_cache(phb, bdfn, PE_handle); + return OPAL_PARAMETER; } opal_call(OPAL_NPU_SPA_CLEAR_CACHE, opal_npu_spa_clear_cache, 3); @@ -222,6 +229,10 @@ static int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t bdfn, return npu2_opencapi_tl_set(phb, bdfn, capabilities, rate); + if (phb->phb_type == phb_type_pau_opencapi) + return pau_opencapi_tl_set(phb, bdfn, capabilities, + rate); + return OPAL_PARAMETER; } opal_call(OPAL_NPU_TL_SET, opal_npu_tl_set, 5); diff --git a/hw/pau.c b/hw/pau.c index 8425567c..83cd6fe9 100644 --- a/hw/pau.c +++ b/hw/pau.c @@ -14,6 +14,9 @@ #define PAU_MAX_PE_NUM 16 #define PAU_RESERVED_PE_NUM 15 +#define PAU_TL_MAX_TEMPLATE 63 +#define PAU_TL_RATE_BUF_SIZE 32 + #define PAU_SLOT_NORMAL PCI_SLOT_STATE_NORMAL #define PAU_SLOT_LINK PCI_SLOT_STATE_LINK #define PAU_SLOT_LINK_START (PAU_SLOT_LINK + 1) @@ -265,6 +268,157 @@ static void pau_device_detect_fixup(struct pau_dev *dev) dt_add_property_strings(dn, "ibm,pau-link-type", "unknown"); } +int64_t pau_opencapi_spa_setup(struct phb *phb, uint32_t __unused bdfn, + uint64_t addr, uint64_t PE_mask) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(phb); + struct pau *pau = dev->pau; + uint64_t reg, val; + int64_t rc; + + lock(&pau->lock); + + reg = PAU_XSL_OSL_SPAP_AN(dev->index); + val = pau_read(pau, reg); + if ((addr && (val & PAU_XSL_OSL_SPAP_AN_EN)) || + (!addr && !(val & PAU_XSL_OSL_SPAP_AN_EN))) { + rc = OPAL_BUSY; + goto out; + } + + /* SPA is disabled by passing a NULL address */ + val = addr; + if (addr) + val = addr | PAU_XSL_OSL_SPAP_AN_EN; + pau_write(pau, reg, val); + + /* + * set the PE mask that the OS uses for PASID -> PE handle + * conversion + */ + reg = PAU_OTL_MISC_CFG0(dev->index); + val = pau_read(pau, reg); + val = SETFIELD(PAU_OTL_MISC_CFG0_PE_MASK, val, PE_mask); + pau_write(pau, reg, val); + rc = OPAL_SUCCESS; +out: + unlock(&pau->lock); + return rc; +} + +int64_t pau_opencapi_spa_clear_cache(struct phb *phb, + uint32_t __unused bdfn, + uint64_t PE_handle) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(phb); + struct pau *pau = dev->pau; + uint64_t reg, val; + int64_t rc, retries = 5; + + lock(&pau->lock); + + reg = PAU_XSL_OSL_CCINV; + val = pau_read(pau, reg); + if (val & PAU_XSL_OSL_CCINV_PENDING) { + rc = OPAL_BUSY; + goto out; + } + + val = PAU_XSL_OSL_CCINV_REMOVE; + val |= SETFIELD(PAU_XSL_OSL_CCINV_PE_HANDLE, val, PE_handle); + if (dev->index) + val |= PAU_XSL_OSL_CCINV_BRICK; + pau_write(pau, reg, val); + + rc = OPAL_HARDWARE; + while (retries--) { + val = pau_read(pau, reg); + if (!(val & PAU_XSL_OSL_CCINV_PENDING)) { + rc = OPAL_SUCCESS; + break; + } + /* the bit expected to flip in less than 200us */ + time_wait_us(200); + } +out: + unlock(&pau->lock); + return rc; +} + +static int pau_opencapi_get_templ_rate(unsigned int templ, + char *rate_buf) +{ + int shift, idx, val; + + /* + * Each rate is encoded over 4 bits (0->15), with 15 being the + * slowest. The buffer is a succession of rates for all the + * templates. The first 4 bits are for template 63, followed + * by 4 bits for template 62, ... etc. So the rate for + * template 0 is at the very end of the buffer. + */ + idx = (PAU_TL_MAX_TEMPLATE - templ) / 2; + shift = 4 * (1 - ((PAU_TL_MAX_TEMPLATE - templ) % 2)); + val = rate_buf[idx] >> shift; + return val; +} + +static bool pau_opencapi_is_templ_supported(unsigned int templ, + long capabilities) +{ + return !!(capabilities & (1ull << templ)); +} + +int64_t pau_opencapi_tl_set(struct phb *phb, uint32_t __unused bdfn, + long capabilities, char *rate_buf) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(phb); + struct pau *pau; + uint64_t reg, val, templ_rate; + int i, rate_pos; + + if (!dev) + return OPAL_PARAMETER; + pau = dev->pau; + + /* The 'capabilities' argument defines what TL template the + * device can receive. OpenCAPI 5.0 defines 64 templates, so + * that's one bit per template. + * + * For each template, the device processing time may vary, so + * the device advertises at what rate a message of a given + * template can be sent. That's encoded in the 'rate' buffer. + * + * On P10, PAU only knows about TL templates 0 -> 3. + * Per the spec, template 0 must be supported. + */ + if (!pau_opencapi_is_templ_supported(0, capabilities)) + return OPAL_PARAMETER; + + reg = PAU_OTL_MISC_CFG_TX(dev->index); + val = pau_read(pau, reg); + val &= ~PAU_OTL_MISC_CFG_TX_TEMP1_EN; + val &= ~PAU_OTL_MISC_CFG_TX_TEMP2_EN; + val &= ~PAU_OTL_MISC_CFG_TX_TEMP3_EN; + + for (i = 0; i < 4; i++) { + /* Skip template 0 as it is implicitly enabled. + * Enable other template If supported by AFU + */ + if (i && pau_opencapi_is_templ_supported(i, capabilities)) + val |= PAU_OTL_MISC_CFG_TX_TEMP_EN(i); + /* The tx rate should still be set for template 0 */ + templ_rate = pau_opencapi_get_templ_rate(i, rate_buf); + rate_pos = 8 + i * 4; + val = SETFIELD(PAU_OTL_MISC_CFG_TX_TEMP_RATE(rate_pos, rate_pos + 3), + val, templ_rate); + } + pau_write(pau, reg, val); + PAUDEVDBG(dev, "OTL configuration register set to %llx\n", val); + + return OPAL_SUCCESS; +} + #define CQ_CTL_STATUS_TIMEOUT 10 /* milliseconds */ static int pau_opencapi_set_fence_control(struct pau_dev *dev, diff --git a/include/pau-regs.h b/include/pau-regs.h index 7a5aaa5f..57c2d723 100644 --- a/include/pau-regs.h +++ b/include/pau-regs.h @@ -118,6 +118,7 @@ #define PAU_OTL_MISC_CFG0_EN PPC_BIT(0) #define PAU_OTL_MISC_CFG0_BLOCK_PE_HANDLE PPC_BIT(1) #define PAU_OTL_MISC_CFG0_BRICKID PPC_BITMASK(2, 3) +#define PAU_OTL_MISC_CFG0_PE_MASK PPC_BITMASK(4, 7) #define PAU_OTL_MISC_CFG0_ENABLE_4_0 PPC_BIT(51) #define PAU_OTL_MISC_CFG0_XLATE_RELEASE PPC_BIT(62) #define PAU_OTL_MISC_CFG0_ENABLE_5_0 PPC_BIT(63) @@ -132,11 +133,16 @@ #define PAU_OTL_MISC_CFG_TLX_CREDITS_DCP2 PPC_BITMASK(48, 55) #define PAU_OTL_MISC_CFG_TLX_CREDITS_DCP3 PPC_BITMASK(56, 63) #define PAU_OTL_MISC_CFG_TX(brk) (PAU_BLOCK_OTL(brk) + 0x058) +#define PAU_OTL_MISC_CFG_TX_TEMP1_EN PPC_BIT(1) +#define PAU_OTL_MISC_CFG_TX_TEMP2_EN PPC_BIT(2) +#define PAU_OTL_MISC_CFG_TX_TEMP3_EN PPC_BIT(3) +#define PAU_OTL_MISC_CFG_TX_TEMP_EN(n) PPC_BIT(n) #define PAU_OTL_MISC_CFG_TX_DRDY_WAIT PPC_BITMASK(5, 7) #define PAU_OTL_MISC_CFG_TX_TEMP0_RATE PPC_BITMASK(8, 11) #define PAU_OTL_MISC_CFG_TX_TEMP1_RATE PPC_BITMASK(12, 15) #define PAU_OTL_MISC_CFG_TX_TEMP2_RATE PPC_BITMASK(16, 19) #define PAU_OTL_MISC_CFG_TX_TEMP3_RATE PPC_BITMASK(20, 23) +#define PAU_OTL_MISC_CFG_TX_TEMP_RATE(nib0, nib1) PPC_BITMASK(nib0, nib1) #define PAU_OTL_MISC_CFG_TX_CRET_FREQ PPC_BITMASK(32, 34) #define PAU_OTL_MISC_OTL_REM0(brk) (PAU_BLOCK_OTL(brk) + 0x068) #define PAU_OTL_MISC_ERROR_SIG_RXI(brk) (PAU_BLOCK_OTL(brk) + 0x070) @@ -150,11 +156,18 @@ #define PAU_OTL_MISC_PSL_PEHANDLE_AN(brk) (PAU_BLOCK_OTL_PSL(brk) + 0x018) /* XSL block registers */ +#define PAU_XSL_OSL_SPAP_AN(brk) (PAU_BLOCK_XSL + 0x000 + (brk) * 8) +#define PAU_XSL_OSL_SPAP_AN_EN PPC_BIT(63) #define PAU_XSL_WRAP_CFG (PAU_BLOCK_XSL + 0x100) #define PAU_XSL_WRAP_CFG_CLOCK_ENABLE PPC_BIT(0) #define PAU_XSL_OSL_XLATE_CFG(brk) (PAU_BLOCK_XSL + 0x040 + (brk) * 8) #define PAU_XSL_OSL_XLATE_CFG_AFU_DIAL PPC_BIT(0) #define PAU_XSL_OSL_XLATE_CFG_OPENCAPI3 PPC_BIT(32) +#define PAU_XSL_OSL_CCINV (PAU_BLOCK_XSL + 0x070) +#define PAU_XSL_OSL_CCINV_REMOVE PPC_BIT(15) +#define PAU_XSL_OSL_CCINV_PENDING PPC_BIT(16) +#define PAU_XSL_OSL_CCINV_BRICK PPC_BIT(47) +#define PAU_XSL_OSL_CCINV_PE_HANDLE PPC_BITMASK(48, 62) /* XTS block registers */ #define PAU_XTS_CFG (PAU_BLOCK_PAU_XTS + 0x020) diff --git a/include/pau.h b/include/pau.h index c0a09401..894007d0 100644 --- a/include/pau.h +++ b/include/pau.h @@ -200,6 +200,13 @@ static inline uint64_t pau_read(struct pau *pau, uint64_t reg) } void pau_opencapi_dump_scoms(struct pau *pau); +int64_t pau_opencapi_spa_setup(struct phb *phb, uint32_t __unused bdfn, + uint64_t addr, uint64_t PE_mask); +int64_t pau_opencapi_spa_clear_cache(struct phb *phb, + uint32_t __unused bdfn, + uint64_t PE_handle); +int64_t pau_opencapi_tl_set(struct phb *phb, uint32_t __unused bdfn, + long capabilities, char *rate_buf); /* PHY */ int pau_dev_phy_reset(struct pau_dev *dev); From patchwork Thu Oct 14 15:57:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540994 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=Ci0v2TJN; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYwZ0xfLz9sNH for ; Fri, 15 Oct 2021 02:58:38 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYwZ08DDz3086 for ; Fri, 15 Oct 2021 02:58:38 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=Ci0v2TJN; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=Ci0v2TJN; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYtz60sFz2ypn for ; Fri, 15 Oct 2021 02:57:15 +1100 (AEDT) Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EFUdSu002371 for ; Thu, 14 Oct 2021 11:57:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=pp1; bh=dQ/621VFUFV4Wy7MEN7RHN9igVINRimaVgVA9ni6v00=; b=Ci0v2TJNZh7ZlPVMPgLXGx3QeBZFpcfAVAh50+Vl0vE/KgOYqayDpWTBtveKykBys8Bi 1dBKni2uibiXFpIX3vS2cYTxewIjMPwUMvOyXM0QQlveti2XIzevP6wHlv9q158vhXJx QDE3UXRRuYjFXudPCjkZ0aTJ28EJXJ9Z4EVVAuWjCgWrcE3z4rD78Xz28XzHKyveRnHT YxIcK4iQgo+2xo/9IYF/mylZa1lZrf4tPav+D7+ViNKFVjUu8kvq6UUJMB0ZnrLWEPR2 lCuVDf8b/UCMW026JhWVS/9zNTmNcvXd4EolZGfX63MPyOsNcV0zBEUveVXj/EzU/Mld 1A== Received: from ppma06fra.de.ibm.com (48.49.7a9f.ip4.static.sl-reverse.com [159.122.73.72]) by mx0a-001b2d01.pphosted.com with ESMTP id 3bpq8kgjty-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:13 -0400 Received: from pps.filterd (ppma06fra.de.ibm.com [127.0.0.1]) by ppma06fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFXiYk000390 for ; Thu, 14 Oct 2021 15:57:10 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma06fra.de.ibm.com with ESMTP id 3bk2bk444w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:10 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFv8t56619806 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:57:08 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0005E11C04C for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DDFD311C04A for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:07 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:57:03 +0200 Message-Id: <20211014155704.46441-15-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: X2Gojf95dLIYYbUg8zpLRRJGB1mvIz6k X-Proofpoint-ORIG-GUID: X2Gojf95dLIYYbUg8zpLRRJGB1mvIz6k X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_09,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 priorityscore=1501 phishscore=0 clxscore=1015 adultscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 suspectscore=0 mlxscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 14/15] pau: mmio invalidates X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" The remaining translation mode: OpenCAPI 5.0 with TLBI/SLBI Snooping, is not used due to performance problems caused by the mismatch between the ERAT and Bloom Filter sizes. When the Address Translation Mode requires TLB and SLB Invalidate operations to be initiated using MMIO registers, a set of registers like the following is used: • XTS MMIO ATSD0 LPARID register • XTS MMIO ATSD0 AVA register • XTS MMIO ATSD0 launch register, write access initiates a shoot down • XTS MMIO ATSD0 status register Signed-off-by: Christophe Lombard Reviewed-by: Frederic Barrat --- hw/npu-opal.c | 3 +++ hw/pau.c | 36 ++++++++++++++++++++++++++++++++++++ include/pau-regs.h | 8 ++++++++ include/pau.h | 2 ++ 4 files changed, 49 insertions(+) diff --git a/hw/npu-opal.c b/hw/npu-opal.c index cf136901..2e455dc9 100644 --- a/hw/npu-opal.c +++ b/hw/npu-opal.c @@ -52,6 +52,9 @@ static int64_t opal_npu_map_lpar(uint64_t phb_id, uint64_t bdf, uint64_t lparid, if (phb->phb_type == phb_type_npu_v2) return npu2_map_lpar(phb, bdf, lparid, lpcr); + if (phb->phb_type == phb_type_pau_opencapi) + return pau_opencapi_map_atsd_lpar(phb, bdf, lparid, lpcr); + return OPAL_PARAMETER; } opal_call(OPAL_NPU_MAP_LPAR, opal_npu_map_lpar, 4); diff --git a/hw/pau.c b/hw/pau.c index 83cd6fe9..9f6a7092 100644 --- a/hw/pau.c +++ b/hw/pau.c @@ -268,6 +268,29 @@ static void pau_device_detect_fixup(struct pau_dev *dev) dt_add_property_strings(dn, "ibm,pau-link-type", "unknown"); } +int64_t pau_opencapi_map_atsd_lpar(struct phb *phb, uint64_t __unused bdf, + uint64_t lparid, uint64_t __unused lpcr) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(phb); + struct pau *pau = dev->pau; + uint64_t val; + + if (lparid >= PAU_XTS_ATSD_MAX) + return OPAL_PARAMETER; + + lock(&pau->lock); + + /* We need to allocate an ATSD per link */ + val = SETFIELD(PAU_XTS_ATSD_HYP_LPARID, 0ull, lparid); + if (!lparid) + val |= PAU_XTS_ATSD_HYP_MSR_HV; + + pau_write(pau, PAU_XTS_ATSD_HYP(lparid), val); + + unlock(&pau->lock); + return OPAL_SUCCESS; +} + int64_t pau_opencapi_spa_setup(struct phb *phb, uint32_t __unused bdfn, uint64_t addr, uint64_t PE_mask) { @@ -1442,6 +1465,18 @@ static void pau_opencapi_create_phb(struct pau_dev *dev) pau_opencapi_create_phb_slot(dev); } +static void pau_dt_add_mmio_atsd(struct pau_dev *dev) +{ + struct dt_node *dn = dev->phb.dt_node; + struct pau *pau = dev->pau; + uint64_t mmio_atsd[PAU_XTS_ATSD_MAX]; + + for (uint32_t i = 0; i < PAU_XTS_ATSD_MAX; i++) + mmio_atsd[i] = pau->regs[0] + PAU_XTS_ATSD_LAUNCH(i); + + dt_add_property(dn, "ibm,mmio-atsd", mmio_atsd, sizeof(mmio_atsd)); +} + static void pau_opencapi_dt_add_mmio_window(struct pau_dev *dev) { struct dt_node *dn = dev->phb.dt_node; @@ -1508,6 +1543,7 @@ static void pau_opencapi_dt_add_props(struct pau_dev *dev) dt_add_property_cells(dn, "ibm,opal-num-pes", PAU_MAX_PE_NUM); dt_add_property_cells(dn, "ibm,opal-reserved-pe", PAU_RESERVED_PE_NUM); + pau_dt_add_mmio_atsd(dev); pau_opencapi_dt_add_mmio_window(dev); pau_opencapi_dt_add_hotpluggable(dev); } diff --git a/include/pau-regs.h b/include/pau-regs.h index 57c2d723..da83ad44 100644 --- a/include/pau-regs.h +++ b/include/pau-regs.h @@ -37,6 +37,7 @@ #define PAU_BLOCK_XSL PAU_BLOCK(4, 0xE) #define PAU_BLOCK_PAU_XTS PAU_BLOCK(7, 1) #define PAU_BLOCK_PAU_MISC PAU_BLOCK(7, 2) +#define PAU_BLOCK_PAU_XTS_ATSD(n) PAU_BLOCK(8, (n)) /* * CQ_SM block registers @@ -176,6 +177,9 @@ #define PAU_XTS_CFG2_XSL2_ENA PPC_BIT(55) #define PAU_XTS_CFG3 (PAU_BLOCK_PAU_XTS + 0x068) #define PAU_XTS_CFG3_MMIOSD_OCAPI PPC_BIT(5) +#define PAU_XTS_ATSD_HYP(n) (PAU_BLOCK_PAU_XTS + 0x100 + (n) * 8) +#define PAU_XTS_ATSD_HYP_MSR_HV PPC_BIT(51) +#define PAU_XTS_ATSD_HYP_LPARID PPC_BITMASK(52, 63) /* MISC block registers */ #define PAU_MISC_OPTICAL_IO_CONFIG (PAU_BLOCK_PAU_MISC + 0x018) @@ -204,4 +208,8 @@ #define PAU_MISC_INT_2_CONFIG_XFAULT_2_5(n) PPC_BIT(0 + (n)) #define PAU_MISC_INT_2_CONFIG_XFAULT_0_1(n) PPC_BIT(54 + (n)) +/* PAU_XTS_ATSD block registers */ +#define PAU_XTS_ATSD_LAUNCH(n) (PAU_BLOCK_PAU_XTS_ATSD(n) + 0x000) +#define PAU_XTS_ATSD_MAX 16 + #endif /* __PAU_REGS_H */ diff --git a/include/pau.h b/include/pau.h index 894007d0..a70058f2 100644 --- a/include/pau.h +++ b/include/pau.h @@ -200,6 +200,8 @@ static inline uint64_t pau_read(struct pau *pau, uint64_t reg) } void pau_opencapi_dump_scoms(struct pau *pau); +int64_t pau_opencapi_map_atsd_lpar(struct phb *phb, uint64_t __unused bdf, + uint64_t lparid, uint64_t __unused lpcr); int64_t pau_opencapi_spa_setup(struct phb *phb, uint32_t __unused bdfn, uint64_t addr, uint64_t PE_mask); int64_t pau_opencapi_spa_clear_cache(struct phb *phb, From patchwork Thu Oct 14 15:57:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lombard X-Patchwork-Id: 1540992 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=NlwWwkQB; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HVYwH2Bprz9sNH for ; Fri, 15 Oct 2021 02:58:23 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HVYwG4NVZz3bYD for ; Fri, 15 Oct 2021 02:58:22 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=NlwWwkQB; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=clombard@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=NlwWwkQB; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HVYtz2dn6z2yPk for ; Fri, 15 Oct 2021 02:57:14 +1100 (AEDT) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19EFlFrM025131 for ; Thu, 14 Oct 2021 11:57:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=AaGxUxE4IR8oPylRMfVEupMm9SQZQnsNiZI+JoTiUXQ=; b=NlwWwkQBw8FnjHYVtF9HG9NfZQNkowPq1sfVhwscSdqgiwz5bKmGxm5JXtQd9iU158X/ /VyAcxp6rnWYpVFiFdrlzYItrScJFbroyUt+SbjTU8Oi6PTFFprXV5V2bo02343EK4fY gGqSsxFQLyj+kHBiT4GXWHUZO6njofbqeTCAX5Zah1u/r1WvgwpCUAVPnlEk6/lMObXM TKQqFVoRDD4N179QILHIjtf2ixDcBzeSNSl9Hg+Ik0qJ3SJmy9TdwivMOEL/xAMkWp54 sR4FoI+CamdCWBi+w4iDGp1CKe5YjiWsv7nd2ekx6I/8lLoWPTPu+8hmxXJyMOaRMV6m Ng== Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0b-001b2d01.pphosted.com with ESMTP id 3bnprkde4a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 11:57:12 -0400 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 19EFXW11029887 for ; Thu, 14 Oct 2021 15:57:10 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma03ams.nl.ibm.com with ESMTP id 3bk2qa6ysy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 14 Oct 2021 15:57:10 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 19EFv8rF7209618 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 14 Oct 2021 15:57:08 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3C58011C04C for ; Thu, 14 Oct 2021 15:57:08 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 18A3E11C04A for ; Thu, 14 Oct 2021 15:57:08 +0000 (GMT) Received: from li-ed209401-43e8-11cb-8043-c0c0b85d70f7.biot.fr.ibm.com (unknown [9.144.57.15]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP for ; Thu, 14 Oct 2021 15:57:08 +0000 (GMT) From: Christophe Lombard To: skiboot@lists.ozlabs.org Date: Thu, 14 Oct 2021 17:57:04 +0200 Message-Id: <20211014155704.46441-16-clombard@linux.vnet.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> References: <20211014155704.46441-1-clombard@linux.vnet.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: sal_gq29GHTofX4WQ4Nbxy23Ab7Z2WCI X-Proofpoint-ORIG-GUID: sal_gq29GHTofX4WQ4Nbxy23Ab7Z2WCI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_08,2021-10-14_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 lowpriorityscore=0 adultscore=0 impostorscore=0 bulkscore=0 mlxlogscore=999 malwarescore=0 priorityscore=1501 spamscore=0 clxscore=1015 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2110140093 Subject: [Skiboot] [PATCH V3 15/15] pau: Add support for OpenCAPI Persistent Memory devices. X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Lowest Point of Coherency (LPC) memory allows the host to access memory on an OpenCAPI device. When the P10 chip accesses memory addresses on the AFU, the Real Address on the PowerBus must hit a BAR in the PAU such as GPU-Memory BAR. The BAR defines the range of Real Addresses that represent AFU memory. The two existing OPAL calls, OPAL_NPU_MEM_ALLOC and OPAL_NPU_MEM_RELEASE are used to manage the AFU momory. Signed-off-by: Christophe Lombard --- hw/npu-opal.c | 35 +++++++++++++++++ hw/npu2-opencapi.c | 18 ++------- hw/pau.c | 93 ++++++++++++++++++++++++++++++++++++++++++++++ hw/phys-map.c | 7 +++- include/npu2.h | 3 ++ include/pau-regs.h | 8 ++++ include/pau.h | 4 ++ 7 files changed, 152 insertions(+), 16 deletions(-) diff --git a/hw/npu-opal.c b/hw/npu-opal.c index 2e455dc9..1b66857f 100644 --- a/hw/npu-opal.c +++ b/hw/npu-opal.c @@ -239,3 +239,38 @@ static int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t bdfn, return OPAL_PARAMETER; } opal_call(OPAL_NPU_TL_SET, opal_npu_tl_set, 5); + +static int64_t opal_npu_mem_alloc(uint64_t phb_id, uint32_t bdfn, + uint64_t size, uint64_t *bar) +{ + struct phb *phb = pci_get_phb(phb_id); + + if (!phb) + return OPAL_PARAMETER; + + if (phb->phb_type == phb_type_npu_v2_opencapi) + return npu2_opencapi_mem_alloc(phb, bdfn, size, bar); + + if (phb->phb_type == phb_type_pau_opencapi) + return pau_opencapi_mem_alloc(phb, bdfn, size, bar); + + return OPAL_PARAMETER; +} +opal_call(OPAL_NPU_MEM_ALLOC, opal_npu_mem_alloc, 4); + +static int64_t opal_npu_mem_release(uint64_t phb_id, uint32_t bdfn) +{ + struct phb *phb = pci_get_phb(phb_id);; + + if (!phb) + return OPAL_PARAMETER; + + if (phb->phb_type == phb_type_npu_v2_opencapi) + return npu2_opencapi_mem_release(phb, bdfn); + + if (phb->phb_type == phb_type_pau_opencapi) + return pau_opencapi_mem_release(phb, bdfn); + + return OPAL_PARAMETER; +} +opal_call(OPAL_NPU_MEM_RELEASE, opal_npu_mem_release, 2); diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 272f924b..8e7bcca9 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -2295,18 +2295,13 @@ out: return rc; } -static int64_t opal_npu_mem_alloc(uint64_t phb_id, uint32_t __unused bdfn, - uint64_t size, __be64 *__bar) +int64_t npu2_opencapi_mem_alloc(struct phb *phb, uint32_t __unused bdfn, + uint64_t size, uint64_t *__bar) { - struct phb *phb = pci_get_phb(phb_id); struct npu2_dev *dev; uint64_t bar; int64_t rc; - - if (!phb || phb->phb_type != phb_type_npu_v2_opencapi) - return OPAL_PARAMETER; - dev = phb_to_npu2_dev_ocapi(phb); if (!dev) return OPAL_PARAMETER; @@ -2320,21 +2315,14 @@ static int64_t opal_npu_mem_alloc(uint64_t phb_id, uint32_t __unused bdfn, return rc; } -opal_call(OPAL_NPU_MEM_ALLOC, opal_npu_mem_alloc, 4); -static int64_t opal_npu_mem_release(uint64_t phb_id, uint32_t __unused bdfn) +int64_t npu2_opencapi_mem_release(struct phb *phb, uint32_t __unused bdfn) { - struct phb *phb = pci_get_phb(phb_id); struct npu2_dev *dev; - - if (!phb || phb->phb_type != phb_type_npu_v2_opencapi) - return OPAL_PARAMETER; - dev = phb_to_npu2_dev_ocapi(phb); if (!dev) return OPAL_PARAMETER; return release_mem_bar(dev); } -opal_call(OPAL_NPU_MEM_RELEASE, opal_npu_mem_release, 2); diff --git a/hw/pau.c b/hw/pau.c index 9f6a7092..a34d6dab 100644 --- a/hw/pau.c +++ b/hw/pau.c @@ -442,6 +442,99 @@ int64_t pau_opencapi_tl_set(struct phb *phb, uint32_t __unused bdfn, return OPAL_SUCCESS; } +static int64_t pau_opencapi_afu_memory_bars(struct pau_dev *dev, + uint64_t size, + uint64_t *bar) +{ + struct pau *pau = dev->pau; + uint64_t addr, psize; + uint64_t reg, val; + + PAUDEVDBG(dev, "Setup AFU Memory BARs\n"); + + if (dev->memory_bar.enable) { + PAUDEVERR(dev, "AFU memory allocation failed - BAR already in use\n"); + return OPAL_RESOURCE; + } + + phys_map_get(pau->chip_id, OCAPI_MEM, + dev->index, + &addr, &psize); + + if (size > psize) { + PAUDEVERR(dev, "Invalid AFU memory BAR allocation size " + "requested: 0x%llx bytes (limit 0x%llx)\n", + size, psize); + return OPAL_PARAMETER; + } + + if (size < (1 << 30)) + size = 1 << 30; + + dev->memory_bar.enable = true; + dev->memory_bar.addr = addr; + dev->memory_bar.size = size; + + reg = PAU_GPU_MEM_BAR(dev->index); + val = PAU_GPU_MEM_BAR_ENABLE | + PAU_GPU_MEM_BAR_POISON; + val = SETFIELD(PAU_GPU_MEM_BAR_ADDR, val, addr >> 30); + if (!is_pow2(size)) + size = 1ull << (ilog2(size) + 1); + + size = (size >> 30) - 1; + val = SETFIELD(PAU_GPU_MEM_BAR_SIZE, val, size); + pau_write(pau, reg, val); + + reg = PAU_CTL_MISC_GPU_MEM_BAR(dev->index); + pau_write(pau, reg, val); + + reg = PAU_XSL_GPU_MEM_BAR(dev->index); + pau_write(pau, reg, val); + + *bar = addr; + return OPAL_SUCCESS; +} + +int64_t pau_opencapi_mem_alloc(struct phb *phb, uint32_t __unused bdfn, + uint64_t size, uint64_t *bar) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(phb); + int64_t rc; + + if (!dev) + return OPAL_PARAMETER; + + if (!opal_addr_valid(bar)) + return OPAL_PARAMETER; + + lock(&dev->pau->lock); + rc = pau_opencapi_afu_memory_bars(dev, size, bar); + + unlock(&dev->pau->lock); + return rc; +} + +int64_t pau_opencapi_mem_release(struct phb *phb, uint32_t __unused bdfn) +{ + struct pau_dev *dev = pau_phb_to_opencapi_dev(phb); + + if (!dev) + return OPAL_PARAMETER; + + lock(&dev->pau->lock); + pau_write(dev->pau, PAU_GPU_MEM_BAR(dev->index), 0ull); + pau_write(dev->pau, PAU_CTL_MISC_GPU_MEM_BAR(dev->index), 0ull); + pau_write(dev->pau, PAU_XSL_GPU_MEM_BAR(dev->index), 0ull); + + dev->memory_bar.enable = false; + dev->memory_bar.addr = 0ull; + dev->memory_bar.size = 0ull; + unlock(&dev->pau->lock); + + return OPAL_SUCCESS; +} + #define CQ_CTL_STATUS_TIMEOUT 10 /* milliseconds */ static int pau_opencapi_set_fence_control(struct pau_dev *dev, diff --git a/hw/phys-map.c b/hw/phys-map.c index 711d626e..cfce4ab5 100644 --- a/hw/phys-map.c +++ b/hw/phys-map.c @@ -30,7 +30,12 @@ static const struct phys_map_entry phys_map_table_p10[] = { /* System memory upto 4TB minus GPU memory */ { SYSTEM_MEM, 0, 0x0000000000000000ull, 0x0000034000000000ull }, - /* TODO: Figure out GPU memory */ + /* Configure OpenCapi memory as before with the old chips. + * Keep in mind that we would need to rework this by allocating + * a new topology id. + */ + { OCAPI_MEM, 0, 0x0002000000000000ull, 0x0000040000000000ull }, + { OCAPI_MEM, 1, 0x0002040000000000ull, 0x0000040000000000ull }, /* 0 TB offset @ MMIO 0x0006000000000000ull */ { PHB5_64BIT_MMIO, 0, 0x0006000000000000ull, 0x0000004000000000ull }, diff --git a/include/npu2.h b/include/npu2.h index a12bf98a..b302108b 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -277,5 +277,8 @@ int64_t npu2_opencapi_spa_clear_cache(struct phb *phb, uint32_t __unused bdfn, uint64_t PE_handle); int64_t npu2_opencapi_tl_set(struct phb *phb, uint32_t __unused bdfn, long capabilities, char *rate); +int64_t npu2_opencapi_mem_alloc(struct phb *phb, uint32_t __unused bdfn, + uint64_t size, uint64_t *bar); +int64_t npu2_opencapi_mem_release(struct phb *phb, uint32_t __unused bdfn); #endif /* __NPU2_H */ diff --git a/include/pau-regs.h b/include/pau-regs.h index da83ad44..45c36037 100644 --- a/include/pau-regs.h +++ b/include/pau-regs.h @@ -64,6 +64,12 @@ #define PAU_SNP_MISC_CFG0_ENABLE_PBUS PPC_BIT(2) #define PAU_SNP_MISC_CFG0_OCAPI_MODE PPC_BITMASK(32, 36) #define PAU_SNP_MISC_CFG0_OCAPI_C2 PPC_BITMASK(45, 49) +#define PAU_GPU_MEM_BAR(brk) (PAU_BLOCK_CQ_SM(0) + 0x190 + (brk) * 8) +#define PAU_GPU_MEM_BAR_ENABLE PPC_BIT(0) +#define PAU_GPU_MEM_BAR_ADDR_MASK PPC_BITMASK(1, 35) +#define PAU_GPU_MEM_BAR_ADDR PPC_BITMASK(1, 21) +#define PAU_GPU_MEM_BAR_SIZE PPC_BITMASK(22, 35) +#define PAU_GPU_MEM_BAR_POISON PPC_BIT(45) #define PAU_NTL_BAR(brk) (PAU_BLOCK_CQ_SM(0) + 0x1b8 + (brk) * 8) #define PAU_NTL_BAR_ENABLE PPC_BIT(0) #define PAU_NTL_BAR_ADDR PPC_BITMASK(3, 35) @@ -88,6 +94,7 @@ #define PAU_CTL_MISC_CFG2_OCAPI_MEM_OS_BIT PPC_BITMASK(25, 29) #define PAU_CTL_MISC_STATUS(brk) (PAU_BLOCK_CQ_CTL + 0x060 + (brk) * 8) #define PAU_CTL_MISC_STATUS_AM_FENCED(brk) (PPC_BITMASK(41, 42) << ((brk)*32)) +#define PAU_CTL_MISC_GPU_MEM_BAR(brk) (PAU_BLOCK_CQ_CTL + 0x070 + (brk) * 8) #define PAU_CTL_MISC_MMIOPA_CONFIG(brk) (PAU_BLOCK_CQ_CTL + 0x098 + (brk) * 8) #define PAU_CTL_MISC_MMIOPA_CONFIG_BAR_ADDR PPC_BITMASK(1, 35) #define PAU_CTL_MISC_MMIOPA_CONFIG_BAR_SIZE PPC_BITMASK(39, 43) @@ -159,6 +166,7 @@ /* XSL block registers */ #define PAU_XSL_OSL_SPAP_AN(brk) (PAU_BLOCK_XSL + 0x000 + (brk) * 8) #define PAU_XSL_OSL_SPAP_AN_EN PPC_BIT(63) +#define PAU_XSL_GPU_MEM_BAR(brk) (PAU_BLOCK_XSL + 0x0D0 + (brk) * 8) #define PAU_XSL_WRAP_CFG (PAU_BLOCK_XSL + 0x100) #define PAU_XSL_WRAP_CFG_CLOCK_ENABLE PPC_BIT(0) #define PAU_XSL_OSL_XLATE_CFG(brk) (PAU_BLOCK_XSL + 0x040 + (brk) * 8) diff --git a/include/pau.h b/include/pau.h index a70058f2..4a6087cb 100644 --- a/include/pau.h +++ b/include/pau.h @@ -45,6 +45,7 @@ struct pau_dev { struct pau_bar ntl_bar; struct pau_bar genid_bar; + struct pau_bar memory_bar; /* Associated I2C information */ uint8_t i2c_bus_id; @@ -209,6 +210,9 @@ int64_t pau_opencapi_spa_clear_cache(struct phb *phb, uint64_t PE_handle); int64_t pau_opencapi_tl_set(struct phb *phb, uint32_t __unused bdfn, long capabilities, char *rate_buf); +int64_t pau_opencapi_mem_alloc(struct phb *phb, uint32_t __unused bdfn, + uint64_t size, uint64_t *bar); +int64_t pau_opencapi_mem_release(struct phb *phb, uint32_t __unused bdfn); /* PHY */ int pau_dev_phy_reset(struct pau_dev *dev);