From patchwork Thu Feb 27 20:40:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Grimm X-Patchwork-Id: 1246090 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48T4N23zs2z9sQt for ; Fri, 28 Feb 2020 07:41:54 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 48T4My5SDvzDqpq for ; Fri, 28 Feb 2020 07:41:50 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=grimm@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 48T4Lf1l6xzDr68 for ; Fri, 28 Feb 2020 07:40:41 +1100 (AEDT) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 01RKUNsS101574 for ; Thu, 27 Feb 2020 15:40:39 -0500 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 2ydxree5w6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 27 Feb 2020 15:40:39 -0500 Received: from m0098417.ppops.net (m0098417.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 01RKVkdU105282 for ; Thu, 27 Feb 2020 15:40:38 -0500 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 2ydxree5vs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 27 Feb 2020 15:40:38 -0500 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id 01RKeFqu006659; Thu, 27 Feb 2020 20:40:38 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma02wdc.us.ibm.com with ESMTP id 2ydcmm038s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 27 Feb 2020 20:40:38 +0000 Received: from b01ledav003.gho.pok.ibm.com (b01ledav003.gho.pok.ibm.com [9.57.199.108]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 01RKebpF30081394 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 27 Feb 2020 20:40:37 GMT Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 609ADB205F; Thu, 27 Feb 2020 20:40:37 +0000 (GMT) Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E62A8B2065; Thu, 27 Feb 2020 20:40:36 +0000 (GMT) Received: from alain.ibm.com (unknown [9.80.218.175]) by b01ledav003.gho.pok.ibm.com (Postfix) with ESMTP; Thu, 27 Feb 2020 20:40:36 +0000 (GMT) From: Ryan Grimm To: oohall@gmail.com Date: Thu, 27 Feb 2020 15:40:09 -0500 Message-Id: <20200227204023.22125-3-grimm@linux.ibm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200227204023.22125-1-grimm@linux.ibm.com> References: <20200227204023.22125-1-grimm@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-02-27_06:2020-02-26, 2020-02-27 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 phishscore=0 malwarescore=0 clxscore=1015 mlxlogscore=999 impostorscore=0 adultscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 suspectscore=3 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2001150001 definitions=main-2002270140 Subject: [Skiboot] [RFC PATCH v5 02/16] Add functions to initialize and start an ultravisor X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: janani@us.ibm.com, suka@us.ibm.com, skiboot@lists.ozlabs.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Power 9 introduces a mode called ultravisor mode. init_uv looks for uv-src-address in the device tree and copies the image to the address specified in "reg". start_ultravisor is called in load_and_boot_kernel with the pointer to the system fdt. Every thread is sent to the ultravisor image and returns with UV mode off. A minimal ultravisor could disable UV and PEF, instructions in commit "skiboot.tcl: ultravisor support." [ maddy: Initial implementation] [Signed-off-by: Madhavan Srinivasan [ santosh: Initial implementation] Signed-off-by: Santosh Sivaraj Signed-off-by: Ryan Grimm ultravisor.c: use reserve --- asm/misc.S | 22 ++++++++++++ core/init.c | 6 ++++ hw/Makefile.inc | 2 +- hw/ultravisor.c | 83 ++++++++++++++++++++++++++++++++++++++++++++ include/processor.h | 8 +++++ include/ultravisor.h | 17 +++++++++ 6 files changed, 137 insertions(+), 1 deletion(-) create mode 100644 hw/ultravisor.c create mode 100644 include/ultravisor.h diff --git a/asm/misc.S b/asm/misc.S index 647f60b2..f9dea492 100644 --- a/asm/misc.S +++ b/asm/misc.S @@ -255,3 +255,25 @@ enter_p9_pm_state: mtspr SPR_PSSCR,%r3 PPC_INST_STOP b . + +/* + * start_uv register usage: + * + * r3 : UV base addr + * r4 : system fdt + */ +.global start_uv +start_uv: + mflr %r0 + std %r0,16(%r1) + sync + /* flush caches, etc */ + icbi 0,%r3 + sync + isync + mtctr %r3 + mr %r3,%r4 + bctrl /* branch to UV here */ + ld %r0,16(%r1) + mtlr %r0 + blr diff --git a/core/init.c b/core/init.c index 339462e5..f124f893 100644 --- a/core/init.c +++ b/core/init.c @@ -46,6 +46,7 @@ #include #include #include +#include enum proc_gen proc_gen; unsigned int pcie_max_link_speed; @@ -602,6 +603,8 @@ void __noreturn load_and_boot_kernel(bool is_reboot) abort(); } + start_ultravisor(fdt); + op_display(OP_LOG, OP_MOD_INIT, 0x000C); mem_dump_free(); @@ -1354,6 +1357,9 @@ void __noreturn __nomcount main_cpu_entry(const void *fdt) /* Add the list of interrupts going to OPAL */ add_opal_interrupts(); + /* Initialize the ultravisor */ + init_uv(); + /* Now release parts of memory nodes we haven't used ourselves... */ mem_region_release_unused(); diff --git a/hw/Makefile.inc b/hw/Makefile.inc index b708bdfe..9a4872ca 100644 --- a/hw/Makefile.inc +++ b/hw/Makefile.inc @@ -8,7 +8,7 @@ HW_OBJS += dts.o lpc-rtc.o npu.o npu-hw-procedures.o xive.o phb4.o HW_OBJS += fake-nvram.o lpc-mbox.o npu2.o npu2-hw-procedures.o HW_OBJS += npu2-common.o npu2-opencapi.o phys-map.o sbe-p9.o capp.o HW_OBJS += occ-sensor.o vas.o sbe-p8.o dio-p9.o lpc-port80h.o cache-p9.o -HW_OBJS += npu-opal.o npu3.o npu3-nvlink.o npu3-hw-procedures.o +HW_OBJS += npu-opal.o npu3.o npu3-nvlink.o npu3-hw-procedures.o ultravisor.o HW=hw/built-in.a include $(SRC)/hw/fsp/Makefile.inc diff --git a/hw/ultravisor.c b/hw/ultravisor.c new file mode 100644 index 00000000..7f263e3e --- /dev/null +++ b/hw/ultravisor.c @@ -0,0 +1,83 @@ +// SPDX-License-Identifier: Apache-2.0 +/* Copyright 2018-2019 IBM Corp. */ + +#include +#include +#include +#include +#include +#include +#include + +static struct dt_node *uv_fw_node; +static uint64_t uv_base_addr; + +static void cpu_start_ultravisor(void *fdt) +{ + prlog(PR_DEBUG, "UV: Starting on CPU 0x%04x\n", this_cpu()->pir); + start_uv(uv_base_addr, fdt); +} + +int start_ultravisor(void *fdt) +{ + struct cpu_thread *cpu; + struct cpu_job **jobs; + int i = 0; + + if (!uv_base_addr || !fdt) { + prlog(PR_DEBUG, "UV: Bad pointers, not starting\n"); + return OPAL_INTERNAL_ERROR; + } + + jobs = zalloc(sizeof(struct cpu_job *) * cpu_max_pir); + + prlog(PR_DEBUG, "UV: Starting @0x%016llx fdt %p\n", + uv_base_addr, fdt); + + for_each_available_cpu(cpu) { + if (cpu == this_cpu()) + continue; + jobs[i++] = cpu_queue_job(cpu, "start_ultravisor", + cpu_start_ultravisor, fdt); + } + + cpu_start_ultravisor(fdt); + + while (i > 0) + cpu_wait_job(jobs[--i], true); + + free(jobs); + + return OPAL_SUCCESS; +} + +void init_uv() +{ + uint64_t uv_dt_src, uv_fw_sz; + struct dt_node *reserved_mem; + + if (!is_msr_bit_set(MSR_S)) { + prlog(PR_DEBUG, "UV: S bit not set\n"); + return; + } + + uv_fw_node = dt_find_compatible_node(dt_root, NULL, "ibm,uv-firmware"); + if (!uv_fw_node) { + prerror("UV: No ibm,uv-firmware node found\n"); + return; + } + + reserved_mem = dt_find_by_path(dt_root, "/reserved-memory/ibm,uv-firmware"); + if (!reserved_mem) { + prerror("UV: No reserved memory for ibm,uv-firmware found\n"); + return; + } + + uv_dt_src = dt_get_address(reserved_mem, 0, &uv_fw_sz); + uv_base_addr = dt_get_address(uv_fw_node, 0, NULL); + + prlog(PR_INFO, "UV: Copying 0x%llx bytes to protected memory 0x%llx from 0x%llx\n", + uv_fw_sz, uv_base_addr, uv_dt_src); + + memcpy((void *)uv_base_addr, (void *)uv_dt_src, uv_fw_sz); +} diff --git a/include/processor.h b/include/processor.h index a0c2864a..f1a88d32 100644 --- a/include/processor.h +++ b/include/processor.h @@ -11,6 +11,7 @@ #define MSR_HV PPC_BIT(3) /* Hypervisor mode */ #define MSR_VEC PPC_BIT(38) /* VMX enable */ #define MSR_VSX PPC_BIT(40) /* VSX enable */ +#define MSR_S PPC_BIT(41) /* Secure mode enable */ #define MSR_EE PPC_BIT(48) /* External Int. Enable */ #define MSR_PR PPC_BIT(49) /* Problem state */ #define MSR_FP PPC_BIT(50) /* Floating Point Enable */ @@ -371,6 +372,13 @@ static inline void st_le32(uint32_t *addr, uint32_t val) asm volatile("stwbrx %0,0,%1" : : "r"(val), "r"(addr), "m"(*addr)); } +static inline bool is_msr_bit_set(uint64_t bit) +{ + if (mfmsr() & bit) + return true; + return false; +} + #endif /* __TEST__ */ #endif /* __ASSEMBLY__ */ diff --git a/include/ultravisor.h b/include/ultravisor.h new file mode 100644 index 00000000..44cf36bf --- /dev/null +++ b/include/ultravisor.h @@ -0,0 +1,17 @@ +// SPDX-License-Identifier: Apache-2.0 +/* Copyright 2018-2019 IBM Corp. */ + +#ifndef __ULTRAVISOR_H +#define __ULTRAVISOR_H + +#include +#include + +#define UV_LOAD_MAX_SIZE 0x200000 + +extern int start_uv(uint64_t entry, void *fdt); + +int start_ultravisor(void *fdt); +void init_uv(void); + +#endif /* __ULTRAVISOR_H */