From patchwork Sun Sep 9 11:41:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vaibhav Jain X-Patchwork-Id: 967743 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 427Tn53zptz9s2P for ; Sun, 9 Sep 2018 21:42:33 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 427Tn52czJzF3LP for ; Sun, 9 Sep 2018 21:42:33 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=vaibhav@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 427Tmk4kZHzF3CK for ; Sun, 9 Sep 2018 21:42:14 +1000 (AEST) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w89BdVBQ061239 for ; Sun, 9 Sep 2018 07:42:12 -0400 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0a-001b2d01.pphosted.com with ESMTP id 2mcuet3d52-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 09 Sep 2018 07:42:12 -0400 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Sun, 9 Sep 2018 12:42:10 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Sun, 9 Sep 2018 12:42:07 +0100 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w89Bg6IG61079692 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Sun, 9 Sep 2018 11:42:06 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8A5735204F; Sun, 9 Sep 2018 14:41:56 +0100 (BST) Received: from vajain21.in.ibm.com.com (unknown [9.199.52.245]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id EDCC95204E; Sun, 9 Sep 2018 14:41:54 +0100 (BST) From: Vaibhav Jain To: Frederic Barrat , Andrew Donnellan , Christophe Lombard Date: Sun, 9 Sep 2018 17:11:38 +0530 X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180909114140.9836-1-vaibhav@linux.ibm.com> References: <20180909114140.9836-1-vaibhav@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18090911-0016-0000-0000-00000202B281 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18090911-0017-0000-0000-000032596A7D Message-Id: <20180909114140.9836-2-vaibhav@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-09-09_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1809090130 Subject: [Skiboot] [RFC 1/3] capp: Add ability to fetch CAPP attribute override values from nvram X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: skiboot@lists.ozlabs.org MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" This patch implements capp_get_override() that queries the nvram 'ibm,skiboot' partition for an attribute override 'attr' present and return its value as an uint64_t. In case the the attribute-override isn't present in nvram then the 'def_val' argument to the function is returned. The expected key-format for attribute override in nvram is: chip.capp<0|1>. chip-id : Chip-index of the CAPP. capp<0|1> : CAPP index on the chip. There are max 2 CAPPs in a chip. attr : Attribute name as string. For example to allocate 2 STQ engines to CAPP-1 on Chip-0 the nvram config should be 'chip00.capp1.stq-engines=2' . During CAPP initialization capp_get_override() will be called to fetch override values for various CAPP initialization attributes and the value returned from the function will be used. The function itself wont parses nor understands the attr being requested and just checks if an attribute override for the given CAPP exists and returns its value. Signed-off-by: Vaibhav Jain --- hw/capp.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ include/capp.h | 2 ++ 2 files changed, 52 insertions(+) diff --git a/hw/capp.c b/hw/capp.c index eeaa4ac4..177a6f15 100644 --- a/hw/capp.c +++ b/hw/capp.c @@ -19,6 +19,11 @@ #include #include #include +#include +#include +#include +#include +#include #define PHBERR(opal_id, chip_id, index, fmt, a...) \ prlog(PR_ERR, "PHB#%04x[%d:%d]: " fmt, \ @@ -240,3 +245,48 @@ int64_t capp_get_info(int chip_id, struct phb *phb, struct capp_info *info) return OPAL_PARAMETER; } + +uint64_t capp_get_override(struct phb *phb, const char *attr, uint64_t def_val) +{ + char key_buffer[33]; + const char *val; + uint64_t nval; + unsigned int chip_index, capp_index; + + switch (phb->phb_type) { + case phb_type_pcie_v4: + { + struct phb4 *p = phb_to_phb4(phb); + + chip_index = p->chip_id; + capp_index = PHB4_CAPP_REG_OFFSET(p) ? 1 : 0; + } + break; + case phb_type_pcie_v3: + { + struct phb3 *p = phb_to_phb3(phb); + + chip_index = p->chip_id; + capp_index = PHB3_CAPP_REG_OFFSET(p) ? 1 : 0; + } + break; + default: + return def_val; + } + + /* Construct the key based on chip and capp index */ + snprintf(key_buffer, sizeof(key_buffer) - 1, "chip%02x.capp%01x.%s", + chip_index, capp_index, attr); + key_buffer[sizeof(key_buffer) - 1] = 0; + + /* Query the nvram for any override */ + val = nvram_query(key_buffer); + if (val) { + nval = strtoul(val, NULL, 0); + prlog(PR_DEBUG, "CAPP Override: %s: 0x%016llX => 0x%016llX\n", + key_buffer, def_val, nval); + return nval; + } else { + return def_val; + } +} diff --git a/include/capp.h b/include/capp.h index 597401d5..14122b17 100644 --- a/include/capp.h +++ b/include/capp.h @@ -96,4 +96,6 @@ extern int64_t capp_load_ucode(unsigned int chip_id, uint32_t opal_id, extern int64_t capp_get_info(int chip_id, struct phb *phb, struct capp_info *info); +uint64_t capp_get_override(struct phb *phb, const char *attr, uint64_t def_val); + #endif /* __CAPP_H */ From patchwork Sun Sep 9 11:41:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vaibhav Jain X-Patchwork-Id: 967744 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 427TnL5vVhz9s2P for ; Sun, 9 Sep 2018 21:42:46 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 427TnL4dBzzF3Cs for ; Sun, 9 Sep 2018 21:42:46 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=vaibhav@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 427Tmn3QN4zF3D3 for ; Sun, 9 Sep 2018 21:42:17 +1000 (AEST) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w89BdMvd062378 for ; Sun, 9 Sep 2018 07:42:15 -0400 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0b-001b2d01.pphosted.com with ESMTP id 2mcuebk90g-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 09 Sep 2018 07:42:14 -0400 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Sun, 9 Sep 2018 12:42:13 +0100 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Sun, 9 Sep 2018 12:42:11 +0100 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w89Bg9PV16777324 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Sun, 9 Sep 2018 11:42:09 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D5C645204E; Sun, 9 Sep 2018 14:41:59 +0100 (BST) Received: from vajain21.in.ibm.com.com (unknown [9.199.52.245]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id E91EA5204F; Sun, 9 Sep 2018 14:41:57 +0100 (BST) From: Vaibhav Jain To: Frederic Barrat , Andrew Donnellan , Christophe Lombard Date: Sun, 9 Sep 2018 17:11:39 +0530 X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180909114140.9836-1-vaibhav@linux.ibm.com> References: <20180909114140.9836-1-vaibhav@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18090911-0028-0000-0000-000002F6ADAF X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18090911-0029-0000-0000-000023B03ED2 Message-Id: <20180909114140.9836-3-vaibhav@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-09-09_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=864 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1809090130 Subject: [Skiboot] [RFC 2/3] phb4/capp: Introduce 'stq-buffers' capp attribute override X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: skiboot@lists.ozlabs.org MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" CAPP uses PEC STQ buffers to communicate with the PSL/XSL on the CAPI card. Presently during CAPP init a static number of these STQ buffers are allocated to based on the link-width of the card which is x16 => 14 Buffers and x8 => 6 Buffers. However in certain circumstances it will be useful to be able to tune this value to improve card throughput. For example in case of Mellanox CX5 adapter (x8+ x8), we just assign 2 STQ Buffers to CAPP. Other cards may need different value for STQ Buffers to ensure optimal performance. Hence this patch introduces a capp attribute override named 'stq-buffers' that can be used to specify the number of CI Buffers to be reserved for CAPP traffic on the PHB. This value will then be used to configure CAPP 'Transport Control Register' as well as PEC's 'CAPP Control Register'. The valid values for this attribute are [1, 14]. Signed-off-by: Vaibhav Jain --- hw/phb4.c | 72 +++++++++++++++++++++++++++---------------------------- 1 file changed, 36 insertions(+), 36 deletions(-) diff --git a/hw/phb4.c b/hw/phb4.c index 80238ea6..4a9e9847 100644 --- a/hw/phb4.c +++ b/hw/phb4.c @@ -3839,7 +3839,7 @@ static void phb4_init_capp_regs(struct phb4 *p, uint32_t capp_eng) { uint64_t reg; uint32_t offset; - uint8_t link_width_x16 = 1; + uint8_t link_width_x16 = 1, stq_eng = 0; offset = PHB4_CAPP_REG_OFFSET(p); @@ -3887,6 +3887,17 @@ static void phb4_init_capp_regs(struct phb4 *p, uint32_t capp_eng) reg |= PPC_BIT(50); xscom_write(p->chip_id, SNOOP_CONTROL + offset, reg); + + /* Calculate the number of stq engines needed */ + if (capp_eng & CAPP_MAX_STQ_ENGINES) { + /* 14 CAPP msg engines or 6 based on link width */ + stq_eng = link_width_x16 ? 14 : 6; + } else if (capp_eng & CAPP_MIN_STQ_ENGINES) { + /* 2 CAPP msg engines */ + stq_eng = 2; + } + stq_eng = (capp_get_override(&p->phb, "stq-buffers", stq_eng) & 0xF); + /* Transport Control Register */ xscom_read(p->chip_id, TRANSPORT_CONTROL + offset, ®); if (p->index == CAPP0_PHB_INDEX) { @@ -3894,41 +3905,27 @@ static void phb4_init_capp_regs(struct phb4 *p, uint32_t capp_eng) reg |= PPC_BITMASK(10, 13); /* Send Packet Timer Value */ reg &= ~PPC_BITMASK(14, 17); /* Set Max LPC CI store buffer to zeros */ reg &= ~PPC_BITMASK(18, 21); /* Set Max tlbi divider */ - if (capp_eng & CAPP_MIN_STQ_ENGINES) { - /* 2 CAPP msg engines */ - reg |= PPC_BIT(58); - reg |= PPC_BIT(59); - reg |= PPC_BIT(60); - } - if (capp_eng & CAPP_MAX_STQ_ENGINES) { - /* 14 CAPP msg engines */ - reg |= PPC_BIT(60); - } - reg |= PPC_BIT(62); } if (p->index == CAPP1_PHB_INDEX) { reg |= PPC_BIT(4); /* Send Packet Timer Value */ - reg &= ~PPC_BIT(10); /* Set CI Store Buffer Threshold=5 */ - reg |= PPC_BIT(11); /* Set CI Store Buffer Threshold=5 */ - reg &= ~PPC_BIT(12); /* Set CI Store Buffer Threshold=5 */ - reg |= PPC_BIT(13); /* Set CI Store Buffer Threshold=5 */ + + reg &= ~PPC_BITMASK(10, 13);/* CI Store Buffer Threshold=5 */ + reg |= PPC_BIT(11) | PPC_BIT(13); + reg &= ~PPC_BITMASK(14, 17); /* Set Max LPC CI store buffer to zeros */ reg &= ~PPC_BITMASK(18, 21); /* Set Max tlbi divider */ - if (capp_eng & CAPP_MIN_STQ_ENGINES) { - /* 2 CAPP msg engines */ - reg |= PPC_BIT(59); - reg |= PPC_BIT(60); - - } else if (capp_eng & CAPP_MAX_STQ_ENGINES) { + } - if (link_width_x16) - /* 14 CAPP msg engines */ - reg |= PPC_BIT(60) | PPC_BIT(62); - else - /* 6 CAPP msg engines */ - reg |= PPC_BIT(60); - } + reg &= ~PPC_BITMASK(58, 62); + /* encode stq engines in the reg */ + if (stq_eng <= 6) { + reg |= ((7 - stq_eng + 1) & 0x7) << 2; + } else if (stq_eng <= 14) { + reg |= ((15 - stq_eng + 1) & 0xF) << 2; + /* Mark CI Store Buffers Avail */ + reg |= PPC_BIT(62); } + PHBINF(p, "CAPP: Using %d STQ Engines. stq=0x%016llX\n", stq_eng, reg); xscom_write(p->chip_id, TRANSPORT_CONTROL + offset, reg); /* The transport control register needs to be loaded in two @@ -4103,7 +4100,7 @@ static int64_t enable_capi_mode(struct phb4 *p, uint64_t pe_number, * 14 and will be assigned in the order of STQ 15 to 2. * - 0-47 (Read machines) are available for capp use. */ - stq_eng = 0x000E000000000000ULL; /* 14 CAPP msg engines */ + stq_eng = 14; /* 14 CAPP msg engines */ dma_eng = 0x0000FFFFFFFFFFFFULL; /* 48 CAPP Read machines */ } @@ -4117,7 +4114,7 @@ static int64_t enable_capi_mode(struct phb4 *p, uint64_t pe_number, * 14 and will be assigned in the order of STQ 15 to 2. * - 0-47 (Read machines) are available for capp use. */ - stq_eng = 0x000E000000000000ULL; + stq_eng = 14; dma_eng = 0x0000FFFFFFFFFFFFULL; } else { @@ -4126,20 +4123,23 @@ static int64_t enable_capi_mode(struct phb4 *p, uint64_t pe_number, * be 6 and will be assigned in the order of 7 to 2. * - 0-30 (Read machines) are available for capp use. */ - stq_eng = 0x0006000000000000ULL; + stq_eng = 6; + /* 30 Read machines for CAPP Minus 20-27 for DMA */ dma_eng = 0x0000FFFFF00E0000ULL; } } if (capp_eng & CAPP_MIN_STQ_ENGINES) - stq_eng = 0x0002000000000000ULL; /* 2 capp msg engines */ + stq_eng = 2; /* 2 capp msg engines */ - /* CAPP Control Register. Enable CAPP Mode */ - reg = 0x8000000000000000ULL; /* PEC works in CAPP Mode */ - reg |= stq_eng; if (capp_eng & CAPP_MAX_DMA_READ_ENGINES) dma_eng = 0x0000F00000000000ULL; /* 4 CAPP Read machines */ + + /* CAPP Control Register. Enable CAPP Mode */ + reg = 0x8000000000000000ULL; /* PEC works in CAPP Mode */ + reg |= (capp_get_override(&p->phb, "stq-buffers", stq_eng) & 0xF) + << (64 - 16); reg |= dma_eng; xscom_write(p->chip_id, p->pe_xscom + XPEC_NEST_CAPP_CNTL, reg); From patchwork Sun Sep 9 11:41:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vaibhav Jain X-Patchwork-Id: 967745 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 427Tnc1blNz9s3l for ; Sun, 9 Sep 2018 21:43:00 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 427Tnc0HyXzF3LR for ; Sun, 9 Sep 2018 21:43:00 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=vaibhav@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 427Tmq6WMmzF3CK for ; Sun, 9 Sep 2018 21:42:19 +1000 (AEST) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w89BdMgb008710 for ; Sun, 9 Sep 2018 07:42:17 -0400 Received: from e06smtp05.uk.ibm.com (e06smtp05.uk.ibm.com [195.75.94.101]) by mx0b-001b2d01.pphosted.com with ESMTP id 2mcuya1j2h-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 09 Sep 2018 07:42:17 -0400 Received: from localhost by e06smtp05.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Sun, 9 Sep 2018 12:42:15 +0100 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp05.uk.ibm.com (192.168.101.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Sun, 9 Sep 2018 12:42:14 +0100 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w89BgCIQ51249234 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Sun, 9 Sep 2018 11:42:12 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DF6025204E; Sun, 9 Sep 2018 14:42:02 +0100 (BST) Received: from vajain21.in.ibm.com.com (unknown [9.199.52.245]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id 294F05204F; Sun, 9 Sep 2018 14:42:01 +0100 (BST) From: Vaibhav Jain To: Frederic Barrat , Andrew Donnellan , Christophe Lombard Date: Sun, 9 Sep 2018 17:11:40 +0530 X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180909114140.9836-1-vaibhav@linux.ibm.com> References: <20180909114140.9836-1-vaibhav@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18090911-0020-0000-0000-000002C3A0A1 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18090911-0021-0000-0000-00002110E4D6 Message-Id: <20180909114140.9836-4-vaibhav@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-09-09_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=831 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1809090130 Subject: [Skiboot] [RFC 3/3] phb4/capp: Introduce 'dma-engines' capp attribute override X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: skiboot@lists.ozlabs.org MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" CAPP-APC uses PEC DMA read engines to communicate with the PSL communicate with the PSL/XSL on the CAPI card. Presently during CAPP init a static number of these DMA read engines are allocated to based on the link-width of the card which is x16 => 48 Engines and x8 => [0-19,28-29] Engines. However in certain circumstances it will be useful to be able to tune this value to improve card throughput. For example in case of Mellanox CX5 adapter (x8+ x8), we just assign 4 DMA Read Engines to CAPP leaving other engines for native DMA. Other cards (especially multi function fpga adapters) may need different value for DMA engines to ensure optimal performance. Hence this patch introduces a capp attribute override named 'dma-engines' that can be used to specify the number of DMA read engines reserved for CAPP on the PEC. This value will then be used to configure CAPP 'APC FSM Read Machine Mask Register' as well as PEC's 'CAPP Control Register'. The valid values for this attribute is a bit mask in range 0x000000000000 - 0xFFFFFFFFFFFF. Each bit in the bit mask indicates a allocation of a single DMA read engine to CAPP from pool 0-47. For example to reserve DMA read engines range [0-19,27-29] use the mask 0xFFFFF00E0000. Signed-off-by: Vaibhav Jain --- hw/phb4.c | 46 ++++++++++++++++++++++------------------------ 1 file changed, 22 insertions(+), 24 deletions(-) diff --git a/hw/phb4.c b/hw/phb4.c index 4a9e9847..afdf3a38 100644 --- a/hw/phb4.c +++ b/hw/phb4.c @@ -3952,33 +3952,30 @@ static void phb4_init_capp_regs(struct phb4 *p, uint32_t capp_eng) } /* capp owns PHB read buffers */ - if (p->index == CAPP0_PHB_INDEX) { + if (capp_eng & CAPP_MAX_DMA_READ_ENGINES) { + reg = 0xF00000000000; + + } else if (p->index == CAPP0_PHB_INDEX) { /* max PHB read buffers 0-47 */ - reg = 0xFFFFFFFFFFFF0000; - if (capp_eng & CAPP_MAX_DMA_READ_ENGINES) - reg = 0xF000000000000000; - xscom_write(p->chip_id, APC_FSM_READ_MASK + offset, reg); - xscom_write(p->chip_id, XPT_FSM_RMM + offset, reg); - } - if (p->index == CAPP1_PHB_INDEX) { + reg = 0xFFFFFFFFFFFF; - if (capp_eng & CAPP_MAX_DMA_READ_ENGINES) { - reg = 0xF000000000000000ULL; - } else if (link_width_x16) { - /* 0-47 (Read machines) are available for - * capp use - */ - reg = 0x0000FFFFFFFFFFFFULL; - } else { - /* Set 30 Read machines for CAPP Minus - * 20-27 for DMA - */ - reg = 0xFFFFF00E00000000ULL; - } - xscom_write(p->chip_id, APC_FSM_READ_MASK + offset, reg); - xscom_write(p->chip_id, XPT_FSM_RMM + offset, reg); + } else if (p->index == CAPP1_PHB_INDEX) { + /* + * Based on link width allocate 0-47 read machines (x16) + * or 30 read machine Minus 20-27 + */ + reg = link_width_x16 ? 0xFFFFFFFFFFFFULL : + 0xFFFFF00E0000ULL; } + reg = (capp_get_override(&p->phb, "dma-engines", reg) & + 0xFFFFFFFFFFFF) << 16; + PHBINF(p, "CAPP: Using %d Engines. dma=0x%016llX\n", + __builtin_popcountll(reg), reg); + xscom_write(p->chip_id, APC_FSM_READ_MASK + offset, reg); + xscom_write(p->chip_id, XPT_FSM_RMM + offset, reg); + + /* CAPP FIR Action 0 */ xscom_write(p->chip_id, CAPP_FIR_ACTION0 + offset, 0x0b1c000104060000); @@ -4140,7 +4137,8 @@ static int64_t enable_capi_mode(struct phb4 *p, uint64_t pe_number, reg = 0x8000000000000000ULL; /* PEC works in CAPP Mode */ reg |= (capp_get_override(&p->phb, "stq-buffers", stq_eng) & 0xF) << (64 - 16); - reg |= dma_eng; + reg |= (capp_get_override(&p->phb, "dma-engines", dma_eng) & + 0xFFFFFFFFFFFF); xscom_write(p->chip_id, p->pe_xscom + XPEC_NEST_CAPP_CNTL, reg); /* PEC2 has 3 ETU's + 16 pci lanes that can operate as x16,