[RFC,3/3] phb4/capp: Introduce 'dma-engines' capp attribute override

Message ID 20180909114140.9836-4-vaibhav@linux.ibm.com
State New
Headers show
Series
  • phb4/capp: Provide user tunable for STQ/DMA engine allocation
Related show

Checks

Context Check Description
snowpatch_ozlabs/apply_patch fail Failed to apply to any branch

Commit Message

Vaibhav Jain Sept. 9, 2018, 11:41 a.m.
CAPP-APC uses PEC DMA read engines to communicate with the PSL
communicate with the PSL/XSL on the CAPI card. Presently during CAPP
init a static number of these DMA read engines are allocated to based
on the link-width of the card which is x16 => 48 Engines and x8 =>
[0-19,28-29] Engines.

However in certain circumstances it will be useful to be able to tune
this value to improve card throughput. For example in case of Mellanox
CX5 adapter (x8+ x8), we just assign 4 DMA Read Engines to CAPP
leaving other engines for native DMA. Other cards (especially multi
function fpga adapters) may need different value for DMA engines to
ensure optimal performance.

Hence this patch introduces a capp attribute override named
'dma-engines' that can be used to specify the number of DMA read
engines reserved for CAPP on the PEC. This value will then be used to
configure CAPP 'APC FSM Read Machine Mask Register' as well as PEC's
'CAPP Control Register'.

The valid values for this attribute is a bit mask in range
0x000000000000 - 0xFFFFFFFFFFFF. Each bit in the bit mask indicates a
allocation of a single DMA read engine to CAPP from pool 0-47. For
example to reserve DMA read engines range [0-19,27-29] use the mask
0xFFFFF00E0000.

Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
 hw/phb4.c | 46 ++++++++++++++++++++++------------------------
 1 file changed, 22 insertions(+), 24 deletions(-)

Patch

diff --git a/hw/phb4.c b/hw/phb4.c
index 4a9e9847..afdf3a38 100644
--- a/hw/phb4.c
+++ b/hw/phb4.c
@@ -3952,33 +3952,30 @@  static void phb4_init_capp_regs(struct phb4 *p, uint32_t capp_eng)
 	}
 
 	/* capp owns PHB read buffers */
-	if (p->index == CAPP0_PHB_INDEX) {
+	if (capp_eng & CAPP_MAX_DMA_READ_ENGINES) {
+		reg = 0xF00000000000;
+
+	} else if (p->index == CAPP0_PHB_INDEX) {
 		/* max PHB read buffers 0-47 */
-		reg = 0xFFFFFFFFFFFF0000;
-		if (capp_eng & CAPP_MAX_DMA_READ_ENGINES)
-			reg = 0xF000000000000000;
-		xscom_write(p->chip_id, APC_FSM_READ_MASK + offset, reg);
-		xscom_write(p->chip_id, XPT_FSM_RMM + offset, reg);
-	}
-	if (p->index == CAPP1_PHB_INDEX) {
+		reg = 0xFFFFFFFFFFFF;
 
-		if (capp_eng & CAPP_MAX_DMA_READ_ENGINES) {
-			reg = 0xF000000000000000ULL;
-		} else if (link_width_x16) {
-			/* 0-47 (Read machines) are available for
-			 * capp use
-			 */
-			reg = 0x0000FFFFFFFFFFFFULL;
-		} else {
-			/* Set 30 Read machines for CAPP Minus
-			 * 20-27 for DMA
-			 */
-			reg = 0xFFFFF00E00000000ULL;
-		}
-		xscom_write(p->chip_id, APC_FSM_READ_MASK + offset, reg);
-		xscom_write(p->chip_id, XPT_FSM_RMM + offset, reg);
+	} else if (p->index == CAPP1_PHB_INDEX) {
+		/*
+		 * Based on link width allocate 0-47 read machines (x16)
+		 * or 30 read machine Minus 20-27
+		 */
+		reg = link_width_x16 ? 0xFFFFFFFFFFFFULL :
+			0xFFFFF00E0000ULL;
 	}
 
+	reg = (capp_get_override(&p->phb, "dma-engines", reg) &
+		     0xFFFFFFFFFFFF) << 16;
+	PHBINF(p, "CAPP: Using %d Engines. dma=0x%016llX\n",
+	       __builtin_popcountll(reg), reg);
+	xscom_write(p->chip_id, APC_FSM_READ_MASK + offset, reg);
+	xscom_write(p->chip_id, XPT_FSM_RMM + offset, reg);
+
+
 	/* CAPP FIR Action 0 */
 	xscom_write(p->chip_id, CAPP_FIR_ACTION0 + offset, 0x0b1c000104060000);
 
@@ -4140,7 +4137,8 @@  static int64_t enable_capi_mode(struct phb4 *p, uint64_t pe_number,
 	reg = 0x8000000000000000ULL; /* PEC works in CAPP Mode */
 	reg |= (capp_get_override(&p->phb, "stq-buffers", stq_eng) & 0xF)
 		<< (64 - 16);
-	reg |= dma_eng;
+	reg |= (capp_get_override(&p->phb, "dma-engines", dma_eng) &
+		       0xFFFFFFFFFFFF);
 	xscom_write(p->chip_id, p->pe_xscom + XPEC_NEST_CAPP_CNTL, reg);
 
 	/* PEC2 has 3 ETU's + 16 pci lanes that can operate as x16,