[V2,4/5] capi: Load capp microcode

Message ID 1490979160-14535-5-git-send-email-clombard@linux.vnet.ibm.com
State Superseded
Headers show

Commit Message

christophe lombard March 31, 2017, 4:52 p.m.
CAPP microcode flash download and CAPP upload for PHB4.
A new file 'capp.c' is created to receive common capp code for PHB3 and
PHB4.

Signed-off-by: Christophe Lombard <clombard@linux.vnet.ibm.com>
---
 core/init.c       |   2 +-
 hw/Makefile.inc   |   2 +-
 hw/capp.c         | 229 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 hw/fsp/fsp.c      |   2 +
 hw/phb3.c         | 205 ++++--------------------------------------------
 hw/phb4.c         |  28 ++++++-
 include/capp.h    |  18 ++++-
 include/skiboot.h |   3 +-
 8 files changed, 288 insertions(+), 201 deletions(-)
 create mode 100644 hw/capp.c

Comments

Stewart Smith April 3, 2017, 2:30 a.m. | #1
Christophe Lombard <clombard@linux.vnet.ibm.com> writes:
> CAPP microcode flash download and CAPP upload for PHB4.
> A new file 'capp.c' is created to receive common capp code for PHB3 and
> PHB4.
>
> Signed-off-by: Christophe Lombard <clombard@linux.vnet.ibm.com>

When are we likely to see P9 CAPP ucode hit github? Currently it's
impossible to test functionality of capi code on p9 as there's no
locatable capp ucode to load in.

(well... except ZZ I guess, as FSP will have it)
Frederic Barrat April 3, 2017, 5:05 p.m. | #2
Le 03/04/2017 à 04:30, Stewart Smith a écrit :
> Christophe Lombard <clombard@linux.vnet.ibm.com> writes:
>> CAPP microcode flash download and CAPP upload for PHB4.
>> A new file 'capp.c' is created to receive common capp code for PHB3 and
>> PHB4.
>>
>> Signed-off-by: Christophe Lombard <clombard@linux.vnet.ibm.com>
>
> When are we likely to see P9 CAPP ucode hit github? Currently it's
> impossible to test functionality of capi code on p9 as there's no
> locatable capp ucode to load in.
>
> (well... except ZZ I guess, as FSP will have it)


Hi Stewart,

We are still doing capi bringup on real-hardware, so it's a bit early 
for you testing anything.

I'd also like to have the capp ucode in op-build, since I'm already 
tired of setting it up manually. I'm checking with the capp folks if 
they have any problem with it.

Then we'll need to work a bit the capp-ucode build to find out if we are 
building for p8 or p9, so that we populate cappucode.bin with the 
correct content.

   Fred
Stewart Smith April 4, 2017, 1:09 a.m. | #3
Frederic Barrat <fbarrat@linux.vnet.ibm.com> writes:
> Le 03/04/2017 à 04:30, Stewart Smith a écrit :
>> Christophe Lombard <clombard@linux.vnet.ibm.com> writes:
>>> CAPP microcode flash download and CAPP upload for PHB4.
>>> A new file 'capp.c' is created to receive common capp code for PHB3 and
>>> PHB4.
>>>
>>> Signed-off-by: Christophe Lombard <clombard@linux.vnet.ibm.com>
>>
>> When are we likely to see P9 CAPP ucode hit github? Currently it's
>> impossible to test functionality of capi code on p9 as there's no
>> locatable capp ucode to load in.
>>
>> (well... except ZZ I guess, as FSP will have it)
>
>
> Hi Stewart,
>
> We are still doing capi bringup on real-hardware, so it's a bit early 
> for you testing anything.

Okay - is there anything urgent for merging this or can it wait until
closer to the end of April? I'm out for the next 2 weeks and it'd be
better if we could merge something that is working pretty well on real
hardware so that we don't disturb others.

> I'd also like to have the capp ucode in op-build, since I'm already 
> tired of setting it up manually. I'm checking with the capp folks if 
> they have any problem with it.

There may be enough room in the PNOR partition to include them all, we
seem to have a 144kb partition with only 40kb used.

> Then we'll need to work a bit the capp-ucode build to find out if we are 
> building for p8 or p9, so that we populate cappucode.bin with the 
> correct content.

The subpartition logic in there could probably just append it, and
that'll be the least effort to get going. Optimising PNOR space usage
can always come later :)
Frederic Barrat April 4, 2017, 10:19 a.m. | #4
Le 04/04/2017 à 03:09, Stewart Smith a écrit :
> Frederic Barrat <fbarrat@linux.vnet.ibm.com> writes:
>> Le 03/04/2017 à 04:30, Stewart Smith a écrit :
>>> Christophe Lombard <clombard@linux.vnet.ibm.com> writes:
>>>> CAPP microcode flash download and CAPP upload for PHB4.
>>>> A new file 'capp.c' is created to receive common capp code for PHB3 and
>>>> PHB4.
>>>>
>>>> Signed-off-by: Christophe Lombard <clombard@linux.vnet.ibm.com>
>>>
>>> When are we likely to see P9 CAPP ucode hit github? Currently it's
>>> impossible to test functionality of capi code on p9 as there's no
>>> locatable capp ucode to load in.
>>>
>>> (well... except ZZ I guess, as FSP will have it)
>>
>>
>> Hi Stewart,
>>
>> We are still doing capi bringup on real-hardware, so it's a bit early
>> for you testing anything.
>
> Okay - is there anything urgent for merging this or can it wait until
> closer to the end of April? I'm out for the next 2 weeks and it'd be
> better if we could merge something that is working pretty well on real
> hardware so that we don't disturb others.


I believe it's ok to wait till the end of the month.



>> I'd also like to have the capp ucode in op-build, since I'm already
>> tired of setting it up manually. I'm checking with the capp folks if
>> they have any problem with it.
>
> There may be enough room in the PNOR partition to include them all, we
> seem to have a 144kb partition with only 40kb used.
>
>> Then we'll need to work a bit the capp-ucode build to find out if we are
>> building for p8 or p9, so that we populate cappucode.bin with the
>> correct content.
>
> The subpartition logic in there could probably just append it, and
> that'll be the least effort to get going. Optimising PNOR space usage
> can always come later :)


Fine, I'll take the offer and not worry about splitting the p8 and p9 
ucode for the time being. With the dd1 ucod, the capp partition is at 
around 80kb, so we'd still have room for one more.
I'll send a request to pull the p9 ucode once a few details are sorted 
out with the capp team, hopefully soon.

   Fred
Stewart Smith April 7, 2017, 1:43 a.m. | #5
Frederic Barrat <fbarrat@linux.vnet.ibm.com> writes:
> Le 04/04/2017 à 03:09, Stewart Smith a écrit :
>> Frederic Barrat <fbarrat@linux.vnet.ibm.com> writes:
>>> Le 03/04/2017 à 04:30, Stewart Smith a écrit :
>>>> Christophe Lombard <clombard@linux.vnet.ibm.com> writes:
>>>>> CAPP microcode flash download and CAPP upload for PHB4.
>>>>> A new file 'capp.c' is created to receive common capp code for PHB3 and
>>>>> PHB4.
>>>>>
>>>>> Signed-off-by: Christophe Lombard <clombard@linux.vnet.ibm.com>
>>>>
>>>> When are we likely to see P9 CAPP ucode hit github? Currently it's
>>>> impossible to test functionality of capi code on p9 as there's no
>>>> locatable capp ucode to load in.
>>>>
>>>> (well... except ZZ I guess, as FSP will have it)
>>>
>>>
>>> Hi Stewart,
>>>
>>> We are still doing capi bringup on real-hardware, so it's a bit early
>>> for you testing anything.
>>
>> Okay - is there anything urgent for merging this or can it wait until
>> closer to the end of April? I'm out for the next 2 weeks and it'd be
>> better if we could merge something that is working pretty well on real
>> hardware so that we don't disturb others.
>
>
> I believe it's ok to wait till the end of the month.

I think I just heard Mikey sigh with relief from 700km away :)

Patch

diff --git a/core/init.c b/core/init.c
index 6b8137c..9144847 100644
--- a/core/init.c
+++ b/core/init.c
@@ -935,7 +935,7 @@  void __noreturn __nomcount main_cpu_entry(const void *fdt)
 	pci_nvram_init();
 
 	phb3_preload_vpd();
-	phb3_preload_capp_ucode();
+	preload_capp_ucode();
 	start_preload_kernel();
 
 	/* NX init */
diff --git a/hw/Makefile.inc b/hw/Makefile.inc
index b0a8b7c..44f0872 100644
--- a/hw/Makefile.inc
+++ b/hw/Makefile.inc
@@ -6,7 +6,7 @@  HW_OBJS += nx.o nx-rng.o nx-crypto.o nx-842.o
 HW_OBJS += p7ioc.o p7ioc-inits.o p7ioc-phb.o
 HW_OBJS += phb3.o sfc-ctrl.o fake-rtc.o bt.o p8-i2c.o prd.o
 HW_OBJS += dts.o lpc-rtc.o npu.o npu-hw-procedures.o xive.o phb4.o
-HW_OBJS += fake-nvram.o lpc-mbox.o npu2.o npu2-hw-procedures.o
+HW_OBJS += fake-nvram.o lpc-mbox.o npu2.o npu2-hw-procedures.o capp.o
 HW=hw/built-in.o
 
 # FIXME hack this for now
diff --git a/hw/capp.c b/hw/capp.c
new file mode 100644
index 0000000..f3b5850
--- /dev/null
+++ b/hw/capp.c
@@ -0,0 +1,229 @@ 
+/* Copyright 2013-2017 IBM Corp.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * 	http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ * implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include <skiboot.h>
+#include <io.h>
+#include <opal.h>
+#include <chip.h>
+#include <xscom.h>
+#include <capp.h>
+
+#define PHBERR(opal_id, chip_id, index, fmt, a...) \
+	       prlog(PR_ERR, "PHB#%04x[%d:%d]: " fmt, \
+		     opal_id, chip_id, \
+		     index,  ## a)
+
+static struct {
+	uint32_t			ec_level;
+	struct capp_lid_hdr		*lid;
+	size_t size;
+	int load_result;
+} capp_ucode_info = { 0, NULL, 0, false };
+
+#define CAPP_UCODE_MAX_SIZE 0x20000
+
+struct lock capi_lock = LOCK_UNLOCKED;
+
+bool capp_ucode_loaded(struct proc_chip *chip, unsigned int index)
+{
+	return (chip->capp_ucode_loaded & (1 << index));
+}
+
+int preload_capp_ucode(void)
+{
+	struct dt_node *p;
+	struct proc_chip *chip;
+	uint32_t index;
+	uint64_t rc;
+	int ret;
+
+	p = dt_find_compatible_node(dt_root, NULL, "ibm,power8-pbcq");
+
+	if (!p) {
+		p = dt_find_compatible_node(dt_root, NULL, "ibm,power9-pbcq");
+		if (!p) {
+			prlog(PR_INFO, "CAPI: WARNING: no compat thing found\n");
+			return OPAL_SUCCESS;
+		}
+	}
+
+	chip = get_chip(dt_get_chip_id(p));
+
+	rc = xscom_read_cfam_chipid(chip->id, &index);
+	if (rc) {
+		prerror("CAPP: Error reading cfam chip-id\n");
+		ret = OPAL_HARDWARE;
+		return ret;
+	}
+	/* Keep ChipID and Major/Minor EC.  Mask out the Location Code. */
+	index = index & 0xf0fff;
+
+	/* Assert that we're preloading */
+	assert(capp_ucode_info.lid == NULL);
+	capp_ucode_info.load_result = OPAL_EMPTY;
+
+	capp_ucode_info.ec_level = index;
+
+	/* Is the ucode preloaded like for BML? */
+	if (dt_has_node_property(p, "ibm,capp-ucode", NULL)) {
+		capp_ucode_info.lid = (struct capp_lid_hdr *)(u64)
+			dt_prop_get_u32(p, "ibm,capp-ucode");
+		ret = OPAL_SUCCESS;
+		goto end;
+	}
+	/* If we successfully download the ucode, we leave it around forever */
+	capp_ucode_info.size = CAPP_UCODE_MAX_SIZE;
+	capp_ucode_info.lid = malloc(CAPP_UCODE_MAX_SIZE);
+	if (!capp_ucode_info.lid) {
+		prerror("CAPP: Can't allocate space for ucode lid\n");
+		ret = OPAL_NO_MEM;
+		goto end;
+	}
+
+	prlog(PR_INFO, "CAPI: Preloading ucode %x\n", capp_ucode_info.ec_level);
+
+	ret = start_preload_resource(RESOURCE_ID_CAPP, index,
+				     capp_ucode_info.lid,
+				     &capp_ucode_info.size);
+
+	if (ret != OPAL_SUCCESS)
+		prerror("CAPI: Failed to preload resource %d\n", ret);
+
+end:
+	return ret;
+}
+
+static int64_t capp_lid_download(void)
+{
+	int64_t ret;
+
+	if (capp_ucode_info.load_result != OPAL_EMPTY)
+		return capp_ucode_info.load_result;
+
+	capp_ucode_info.load_result = wait_for_resource_loaded(
+		RESOURCE_ID_CAPP,
+		capp_ucode_info.ec_level);
+
+	if (capp_ucode_info.load_result != OPAL_SUCCESS) {
+		prerror("CAPP: Error loading ucode lid. index=%x\n",
+			capp_ucode_info.ec_level);
+		ret = OPAL_RESOURCE;
+		free(capp_ucode_info.lid);
+		capp_ucode_info.lid = NULL;
+		goto end;
+	}
+
+	ret = OPAL_SUCCESS;
+end:
+	return ret;
+}
+
+int64_t capp_load_ucode(unsigned int chip_id, uint32_t opal_id,
+			unsigned int index, u64 lid_eyecatcher, uint32_t reg_offset,
+			uint64_t apc_master_addr, uint64_t apc_master_write,
+			uint64_t snp_array_addr, uint64_t snp_array_write)
+{
+	struct proc_chip *chip = get_chip(chip_id);
+	struct capp_ucode_lid *ucode;
+	struct capp_ucode_data *data;
+	struct capp_lid_hdr *lid;
+	uint64_t rc, val, addr;
+	uint32_t chunk_count, offset;
+	int i;
+
+	if (capp_ucode_loaded(chip, index))
+		return OPAL_SUCCESS;
+
+	rc = capp_lid_download();
+	if (rc)
+		return rc;
+
+	prlog(PR_INFO, "CHIP%i: CAPP ucode lid loaded at %p\n",
+	      chip_id, capp_ucode_info.lid);
+
+	lid = capp_ucode_info.lid;
+	/*
+	 * If lid header is present (on FSP machines), it'll tell us where to
+	 * find the ucode.  Otherwise this is the ucode.
+	 */
+	ucode = (struct capp_ucode_lid *)lid;
+	if (be64_to_cpu(lid->eyecatcher) == lid_eyecatcher) {
+		if (be64_to_cpu(lid->version) != 0x1) {
+			PHBERR(opal_id, chip_id, index,
+			       "capi ucode lid header invalid\n");
+			return OPAL_HARDWARE;
+		}
+		ucode = (struct capp_ucode_lid *)
+			((char *)ucode + be64_to_cpu(lid->ucode_offset));
+	}
+
+	/* 'CAPPULID' in ASCII */
+	if ((be64_to_cpu(ucode->eyecatcher) != 0x43415050554C4944) ||
+	    (be64_to_cpu(ucode->version != 1))) {
+		PHBERR(opal_id, chip_id, index,
+		       "CAPP: ucode header invalid\n");
+		return OPAL_HARDWARE;
+	}
+
+	offset = 0;
+	while (offset < be64_to_cpu(ucode->data_size)) {
+		data = (struct capp_ucode_data *)
+			((char *)&ucode->data + offset);
+		chunk_count = be32_to_cpu(data->hdr.chunk_count);
+		offset += sizeof(struct capp_ucode_data_hdr) + chunk_count * 8;
+
+		/* 'CAPPUCOD' in ASCII */
+		if (be64_to_cpu(data->hdr.eyecatcher) != 0x4341505055434F44) {
+			PHBERR(opal_id, chip_id, index,
+			       "CAPP: ucode data header invalid:%i\n",
+			       offset);
+			return OPAL_HARDWARE;
+		}
+
+		switch (data->hdr.reg) {
+		case apc_master_cresp:
+			xscom_write(chip_id, apc_master_addr + reg_offset,
+				    0);
+			addr = apc_master_write;
+			break;
+		case apc_master_uop_table:
+			xscom_write(chip_id, apc_master_addr + reg_offset,
+				    0x180ULL << 52);
+			addr = apc_master_write;
+			break;
+		case snp_ttype:
+			xscom_write(chip_id, snp_array_addr + reg_offset,
+				    0x5000ULL << 48);
+			addr = snp_array_write;
+			break;
+		case snp_uop_table:
+			xscom_write(chip_id, snp_array_addr + reg_offset,
+				    0x4000ULL << 48);
+			addr = snp_array_write;
+			break;
+		default:
+			continue;
+		}
+
+		for (i = 0; i < chunk_count; i++) {
+			val = be64_to_cpu(data->data[i]);
+			xscom_write(chip_id, addr + reg_offset, val);
+		}
+	}
+
+	chip->capp_ucode_loaded |= (1 << index);
+
+	return OPAL_SUCCESS;
+}
diff --git a/hw/fsp/fsp.c b/hw/fsp/fsp.c
index a0c5a78..4b269de 100644
--- a/hw/fsp/fsp.c
+++ b/hw/fsp/fsp.c
@@ -2312,6 +2312,7 @@  int fsp_fetch_data_queue(uint8_t flags, uint16_t id, uint32_t sub_id,
 #define CAPP_IDX_MURANO_DD20 0x200ef
 #define CAPP_IDX_MURANO_DD21 0x201ef
 #define CAPP_IDX_NAPLES_DD10 0x100d3
+#define CAPP_IDX_NIMBUS_DD10 0x100d1
 
 static struct {
 	enum resource_id	id;
@@ -2325,6 +2326,7 @@  static struct {
 	{ RESOURCE_ID_CAPP,	CAPP_IDX_VENICE_DD10,	0x80a02003 },
 	{ RESOURCE_ID_CAPP,	CAPP_IDX_VENICE_DD20,	0x80a02004 },
 	{ RESOURCE_ID_CAPP,	CAPP_IDX_NAPLES_DD10,	0x80a02005 },
+	{ RESOURCE_ID_CAPP,	CAPP_IDX_NIMBUS_DD10,	0x80a02006 },
 };
 
 static void fsp_start_fetching_next_lid(void);
diff --git a/hw/phb3.c b/hw/phb3.c
index 271cd09..a8f5890 100644
--- a/hw/phb3.c
+++ b/hw/phb3.c
@@ -2438,139 +2438,21 @@  static int64_t phb3_freset(struct pci_slot *slot)
 	return OPAL_HARDWARE;
 }
 
-struct lock capi_lock = LOCK_UNLOCKED;
-static struct {
-	uint32_t			ec_level;
-	struct capp_lid_hdr		*lid;
-	size_t size;
-	int load_result;
-} capp_ucode_info = { 0, NULL, 0, false };
-
-#define CAPP_UCODE_MAX_SIZE 0x20000
-
-#define CAPP_UCODE_LOADED(chip, p) \
-	 ((chip)->capp_ucode_loaded & (1 << (p)->index))
-
-static int64_t capp_lid_download(void)
-{
-	int64_t ret;
-
-	if (capp_ucode_info.load_result != OPAL_EMPTY)
-		return capp_ucode_info.load_result;
-
-	capp_ucode_info.load_result = wait_for_resource_loaded(
-		RESOURCE_ID_CAPP,
-		capp_ucode_info.ec_level);
-
-	if (capp_ucode_info.load_result != OPAL_SUCCESS) {
-		prerror("CAPP: Error loading ucode lid. index=%x\n",
-			capp_ucode_info.ec_level);
-		ret = OPAL_RESOURCE;
-		free(capp_ucode_info.lid);
-		capp_ucode_info.lid = NULL;
-		goto end;
-	}
-
-	ret = OPAL_SUCCESS;
-end:
-	return ret;
-}
-
-static int64_t capp_load_ucode(struct phb3 *p)
+static int64_t load_capp_ucode(struct phb3 *p)
 {
-	struct proc_chip *chip = get_chip(p->chip_id);
-	struct capp_ucode_lid *ucode;
-	struct capp_ucode_data *data;
-	struct capp_lid_hdr *lid;
-	uint64_t rc, val, addr;
-	uint32_t chunk_count, offset, reg_offset;
-	int i;
-
-	if (CAPP_UCODE_LOADED(chip, p))
-		return OPAL_SUCCESS;
+	int64_t rc;
 
-	/* Return if PHB not attached to a CAPP unit */
 	if (p->index > PHB3_CAPP_MAX_PHB_INDEX(p))
 		return OPAL_HARDWARE;
 
-	rc = capp_lid_download();
-	if (rc)
-		return rc;
-
-	prlog(PR_INFO, "CHIP%i: CAPP ucode lid loaded at %p\n",
-	      p->chip_id, capp_ucode_info.lid);
-	lid = capp_ucode_info.lid;
-	/*
-	 * If lid header is present (on FSP machines), it'll tell us where to
-	 * find the ucode.  Otherwise this is the ucode.
-	 */
-	ucode = (struct capp_ucode_lid *)lid;
-	if (be64_to_cpu(lid->eyecatcher) == 0x434150504c494448) {
-		if (be64_to_cpu(lid->version) != 0x1) {
-			PHBERR(p, "capi ucode lid header invalid\n");
-			return OPAL_HARDWARE;
-		}
-		ucode = (struct capp_ucode_lid *)
-			((char *)ucode + be64_to_cpu(lid->ucode_offset));
-	}
-
-	if ((be64_to_cpu(ucode->eyecatcher) != 0x43415050554C4944) ||
-	    (ucode->version != 1)) {
-		PHBERR(p, "CAPP: ucode header invalid\n");
-		return OPAL_HARDWARE;
-	}
-
-	reg_offset = PHB3_CAPP_REG_OFFSET(p);
-	offset = 0;
-	while (offset < be64_to_cpu(ucode->data_size)) {
-		data = (struct capp_ucode_data *)
-			((char *)&ucode->data + offset);
-		chunk_count = be32_to_cpu(data->hdr.chunk_count);
-		offset += sizeof(struct capp_ucode_data_hdr) + chunk_count * 8;
-
-		if (be64_to_cpu(data->hdr.eyecatcher) != 0x4341505055434F44) {
-			PHBERR(p, "CAPP: ucode data header invalid:%i\n",
-			       offset);
-			return OPAL_HARDWARE;
-		}
-
-		switch (data->hdr.reg) {
-		case apc_master_cresp:
-			xscom_write(p->chip_id,
-				    CAPP_APC_MASTER_ARRAY_ADDR_REG + reg_offset,
-				    0);
-			addr = CAPP_APC_MASTER_ARRAY_WRITE_REG;
-			break;
-		case apc_master_uop_table:
-			xscom_write(p->chip_id,
-				    CAPP_APC_MASTER_ARRAY_ADDR_REG + reg_offset,
-				    0x180ULL << 52);
-			addr = CAPP_APC_MASTER_ARRAY_WRITE_REG;
-			break;
-		case snp_ttype:
-			xscom_write(p->chip_id,
-				    CAPP_SNP_ARRAY_ADDR_REG + reg_offset,
-				    0x5000ULL << 48);
-			addr = CAPP_SNP_ARRAY_WRITE_REG;
-			break;
-		case snp_uop_table:
-			xscom_write(p->chip_id,
-				    CAPP_SNP_ARRAY_ADDR_REG + reg_offset,
-				    0x4000ULL << 48);
-			addr = CAPP_SNP_ARRAY_WRITE_REG;
-			break;
-		default:
-			continue;
-		}
-
-		for (i = 0; i < chunk_count; i++) {
-			val = be64_to_cpu(data->data[i]);
-			xscom_write(p->chip_id, addr + reg_offset, val);
-		}
-	}
-
-	chip->capp_ucode_loaded |= (1 << p->index);
-	return OPAL_SUCCESS;
+	/* 0x434150504c494448 = 'CAPPLIDH' in ASCII */
+	rc = capp_load_ucode(p->chip_id, p->phb.opal_id, p->index,
+			0x434150504c494448, PHB3_CAPP_REG_OFFSET(p),
+			CAPP_APC_MASTER_ARRAY_ADDR_REG,
+			CAPP_APC_MASTER_ARRAY_WRITE_REG,
+			CAPP_SNP_ARRAY_ADDR_REG,
+			CAPP_SNP_ARRAY_WRITE_REG);
+	return rc;
 }
 
 static void do_capp_recovery_scoms(struct phb3 *p)
@@ -2583,7 +2465,7 @@  static void do_capp_recovery_scoms(struct phb3 *p)
 	offset = PHB3_CAPP_REG_OFFSET(p);
 	/* disable snoops */
 	xscom_write(p->chip_id, SNOOP_CAPI_CONFIG + offset, 0);
-	capp_load_ucode(p);
+	load_capp_ucode(p);
 	/* clear err rpt reg*/
 	xscom_write(p->chip_id, CAPP_ERR_RPT_CLR + offset, 0);
 	/* clear capp fir */
@@ -3743,7 +3625,7 @@  static int64_t phb3_set_capi_mode(struct phb *phb, uint64_t mode,
 	uint32_t offset;
 	u8 mask;
 
-	if (!CAPP_UCODE_LOADED(chip, p)) {
+	if (!capp_ucode_loaded(chip, p->index)) {
 		PHBERR(p, "CAPP: ucode not loaded\n");
 		return OPAL_RESOURCE;
 	}
@@ -4823,7 +4705,7 @@  static void phb3_create(struct dt_node *np)
 	phb3_init_hw(p, true);
 
 	/* Load capp microcode into capp unit */
-	capp_load_ucode(p);
+	load_capp_ucode(p);
 
 	opal_add_host_sync_notifier(phb3_host_sync_reset, p);
 
@@ -5020,67 +4902,6 @@  static void phb3_probe_pbcq(struct dt_node *pbcq)
 	add_chip_dev_associativity(np);
 }
 
-int phb3_preload_capp_ucode(void)
-{
-	struct dt_node *p;
-	struct proc_chip *chip;
-	uint32_t index;
-	uint64_t rc;
-	int ret;
-
-	p = dt_find_compatible_node(dt_root, NULL, "ibm,power8-pbcq");
-
-	if (!p) {
-		printf("CAPI: WARNING: no compat thing found\n");
-		return OPAL_SUCCESS;
-	}
-
-	chip = get_chip(dt_get_chip_id(p));
-
-	rc = xscom_read_cfam_chipid(chip->id, &index);
-	if (rc) {
-		prerror("CAPP: Error reading cfam chip-id\n");
-		ret = OPAL_HARDWARE;
-		return ret;
-	}
-	/* Keep ChipID and Major/Minor EC.  Mask out the Location Code. */
-	index = index & 0xf0fff;
-
-	/* Assert that we're preloading */
-	assert(capp_ucode_info.lid == NULL);
-	capp_ucode_info.load_result = OPAL_EMPTY;
-
-	capp_ucode_info.ec_level = index;
-
-	/* Is the ucode preloaded like for BML? */
-	if (dt_has_node_property(p, "ibm,capp-ucode", NULL)) {
-		capp_ucode_info.lid = (struct capp_lid_hdr *)(u64)
-			dt_prop_get_u32(p, "ibm,capp-ucode");
-		ret = OPAL_SUCCESS;
-		goto end;
-	}
-	/* If we successfully download the ucode, we leave it around forever */
-	capp_ucode_info.size = CAPP_UCODE_MAX_SIZE;
-	capp_ucode_info.lid = malloc(CAPP_UCODE_MAX_SIZE);
-	if (!capp_ucode_info.lid) {
-		prerror("CAPP: Can't allocate space for ucode lid\n");
-		ret = OPAL_NO_MEM;
-		goto end;
-	}
-
-	printf("CAPI: Preloading ucode %x\n", capp_ucode_info.ec_level);
-
-	ret = start_preload_resource(RESOURCE_ID_CAPP, index,
-				     capp_ucode_info.lid,
-				     &capp_ucode_info.size);
-
-	if (ret != OPAL_SUCCESS)
-		prerror("CAPI: Failed to preload resource %d\n", ret);
-
-end:
-	return ret;
-}
-
 void phb3_preload_vpd(void)
 {
 	const struct dt_property *prop;
diff --git a/hw/phb4.c b/hw/phb4.c
index e6f29bc..ab1785b 100644
--- a/hw/phb4.c
+++ b/hw/phb4.c
@@ -49,6 +49,7 @@ 
 #include <phb4.h>
 #include <phb4-regs.h>
 #include <phb4-capp.h>
+#include <capp.h>
 #include <fsp.h>
 #include <chip.h>
 #include <chiptod.h>
@@ -1963,7 +1964,22 @@  static int64_t phb4_freset(struct pci_slot *slot)
 	return OPAL_HARDWARE;
 }
 
-extern struct lock capi_lock;
+static int64_t load_capp_ucode(struct phb4 *p)
+{
+	int64_t rc;
+
+	if (p->index != CAPP0_PHB_INDEX && p->index != CAPP1_PHB_INDEX)
+		return OPAL_HARDWARE;
+
+	/* 0x4341505050534C4C = 'CAPPPSLL' in ASCII */
+	rc = capp_load_ucode(p->chip_id, p->phb.opal_id, p->index,
+			0x4341505050534C4C, PHB4_CAPP_REG_OFFSET(p),
+			CAPP_APC_MASTER_ARRAY_ADDR_REG,
+			CAPP_APC_MASTER_ARRAY_WRITE_REG,
+			CAPP_SNP_ARRAY_ADDR_REG,
+			CAPP_SNP_ARRAY_WRITE_REG);
+	return rc;
+}
 
 static void do_capp_recovery_scoms(struct phb4 *p)
 {
@@ -1975,7 +1991,7 @@  static void do_capp_recovery_scoms(struct phb4 *p)
 	offset = PHB4_CAPP_REG_OFFSET(p);
 	/* disable snoops */
 	xscom_write(p->chip_id, SNOOP_CAPI_CONFIG + offset, 0);
-
+	load_capp_ucode(p);
 	/* clear err rpt reg*/
 	xscom_write(p->chip_id, CAPP_ERR_RPT_CLR + offset, 0);
 	/* clear capp fir */
@@ -2707,6 +2723,11 @@  static int64_t phb4_set_capi_mode(struct phb *phb, uint64_t mode,
 	uint64_t reg;
 	uint32_t offset;
 
+	if (!capp_ucode_loaded(chip, p->index)) {
+		PHBERR(p, "CAPP: ucode not loaded\n");
+		return OPAL_RESOURCE;
+	}
+
 	lock(&capi_lock);
 	chip->capp_phb4_attached_mask |= 1 << p->index;
 	unlock(&capi_lock);
@@ -3684,6 +3705,9 @@  static void phb4_create(struct dt_node *np)
 	/* Get the HW up and running */
 	phb4_init_hw(p, true);
 
+	/* Load capp microcode into capp unit */
+	load_capp_ucode(p);
+
 	/* Register all interrupt sources with XIVE */
 	xive_register_hw_source(p->base_msi, p->num_irqs - 8, 16,
 				p->int_mmio, XIVE_SRC_SHIFT_BUG,
diff --git a/include/capp.h b/include/capp.h
index d0c28c9..c1aa8d2 100644
--- a/include/capp.h
+++ b/include/capp.h
@@ -17,8 +17,12 @@ 
 #ifndef __CAPP_H
 #define __CAPP_H
 
+/*
+ * eyecatcher PHB3:  'CAPPLIDH' in ASCII
+ * eyecatcher PHB4:  'CAPPPSLL' in ASCII
+ */
 struct capp_lid_hdr {
-	be64 eyecatcher;	/* 'CAPPLIDH' in ASCII */
+	be64 eyecatcher;
 	be64 version;
 	be64 lid_no;
 	be64 pad;
@@ -27,7 +31,7 @@  struct capp_lid_hdr {
 };
 
 struct capp_ucode_data_hdr {
-	be64 eyecatcher;  	/* 'CAPPUCOD' in ASCII */
+	be64 eyecatcher;	/* 'CAPPUCOD' in ASCII */
 	u8 version;
 	u8 reg;
 	u8 reserved[2];
@@ -47,7 +51,6 @@  struct capp_ucode_lid {
 	struct capp_ucode_data data; /* This repeats */
 };
 
-
 enum capp_reg {
 	apc_master_cresp		= 0x1,
 	apc_master_uop_table		= 0x2,
@@ -62,4 +65,13 @@  enum capp_reg {
 	apc_master_powerbus_ctrl	= 0xB
 };
 
+struct proc_chip;
+extern struct lock capi_lock;
+extern bool capp_ucode_loaded(struct proc_chip *chip, unsigned int index);
+
+extern int64_t capp_load_ucode(unsigned int chip_id, uint32_t opal_id,
+			       unsigned int index, u64 lid_eyecatcher, uint32_t reg_offset,
+			       uint64_t apc_master_addr, uint64_t apc_master_write,
+			       uint64_t snp_array_addr, uint64_t snp_array_write);
+
 #endif /* __CAPP_H */
diff --git a/include/skiboot.h b/include/skiboot.h
index 8bc767a..76064b2 100644
--- a/include/skiboot.h
+++ b/include/skiboot.h
@@ -212,9 +212,8 @@  extern void setup_reset_vector(void);
 extern void probe_p7ioc(void);
 extern void probe_phb3(void);
 extern void probe_phb4(void);
-extern int phb3_preload_capp_ucode(void);
 extern void phb3_preload_vpd(void);
-extern int phb4_preload_capp_ucode(void);
+extern int preload_capp_ucode(void);
 extern void phb4_preload_vpd(void);
 extern void probe_npu(void);
 extern void probe_npu2(void);