From patchwork Mon Feb 25 10:09:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 1047644 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 447Hkl26Mqz9s1l for ; Mon, 25 Feb 2019 21:10:23 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=kaod.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 447Hkk5GrmzDqL9 for ; Mon, 25 Feb 2019 21:10:22 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=softfail (mailfrom) smtp.mailfrom=kaod.org (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=clg@kaod.org; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=kaod.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 447HkZ3lVrzDqD7 for ; Mon, 25 Feb 2019 21:10:12 +1100 (AEDT) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x1PA8Ogl023812 for ; Mon, 25 Feb 2019 05:10:09 -0500 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0a-001b2d01.pphosted.com with ESMTP id 2qve5jgp1a-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 25 Feb 2019 05:10:08 -0500 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 25 Feb 2019 10:10:06 -0000 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 25 Feb 2019 10:10:03 -0000 Received: from b06wcsmtp001.portsmouth.uk.ibm.com (b06wcsmtp001.portsmouth.uk.ibm.com [9.149.105.160]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x1PAA2LS30933202 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Feb 2019 10:10:02 GMT Received: from b06wcsmtp001.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 92532A4054; Mon, 25 Feb 2019 10:10:02 +0000 (GMT) Received: from b06wcsmtp001.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 760E3A405B; Mon, 25 Feb 2019 10:10:02 +0000 (GMT) Received: from smtp.lab.toulouse-stg.fr.ibm.com (unknown [9.101.4.1]) by b06wcsmtp001.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 25 Feb 2019 10:10:02 +0000 (GMT) Received: from zorba.kaod.org.com (sig-9-145-177-241.de.ibm.com [9.145.177.241]) by smtp.lab.toulouse-stg.fr.ibm.com (Postfix) with ESMTP id CB33B220246; Mon, 25 Feb 2019 11:10:01 +0100 (CET) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: Stewart Smith Date: Mon, 25 Feb 2019 11:09:51 +0100 X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19022510-0016-0000-0000-0000025A8A79 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19022510-0017-0000-0000-000032B4E67B Message-Id: <20190225100951.1877-1-clg@kaod.org> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-02-25_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=8 phishscore=0 bulkscore=0 spamscore=0 clxscore=1034 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902250076 Subject: [Skiboot] [PATCH v2] xive: Add calls to save/restore the queues and VPs HW state X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: skiboot@lists.ozlabs.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" To be able to support migration of guests using the XIVE native exploitation mode, (where the queue is effectively owned by the guest), KVM needs to be able to save and restore the HW-modified fields of the queue, such as the current queue producer pointer and generation bit, and to retrieve the modified thread context registers of the VP from the NVT structure : the VP interrupt pending bits. However, there is no need to set back the NVT structure on P9. P10 should be the same. Based on previous work from BenH. Signed-off-by: Cédric Le Goater --- include/opal-api.h | 7 +-- hw/xive.c | 130 +++++++++++++++++++++++++++++++++++++++++++++ doc/xive.rst | 51 ++++++++++++++++++ 3 files changed, 185 insertions(+), 3 deletions(-) diff --git a/include/opal-api.h b/include/opal-api.h index 5f397c8e6731..52a61a22be75 100644 --- a/include/opal-api.h +++ b/include/opal-api.h @@ -199,8 +199,8 @@ #define OPAL_XIVE_FREE_IRQ 140 #define OPAL_XIVE_SYNC 141 #define OPAL_XIVE_DUMP 142 -#define OPAL_XIVE_RESERVED3 143 -#define OPAL_XIVE_RESERVED4 144 +#define OPAL_XIVE_GET_QUEUE_STATE 143 /* Get END state */ +#define OPAL_XIVE_SET_QUEUE_STATE 144 /* Set END state */ #define OPAL_SIGNAL_SYSTEM_RESET 145 #define OPAL_NPU_INIT_CONTEXT 146 #define OPAL_NPU_DESTROY_CONTEXT 147 @@ -226,7 +226,8 @@ #define OPAL_NX_COPROC_INIT 167 #define OPAL_NPU_SET_RELAXED_ORDER 168 #define OPAL_NPU_GET_RELAXED_ORDER 169 -#define OPAL_LAST 169 +#define OPAL_XIVE_GET_VP_STATE 170 /* Get NVT state */ +#define OPAL_LAST 170 #define QUIESCE_HOLD 1 /* Spin all calls at entry */ #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */ diff --git a/hw/xive.c b/hw/xive.c index 05e5531cbfca..18eb6cc3188d 100644 --- a/hw/xive.c +++ b/hw/xive.c @@ -1397,11 +1397,21 @@ static int64_t xive_ivc_scrub(struct xive *x, uint64_t block, uint64_t idx) return __xive_cache_scrub(x, xive_cache_ivc, block, idx, false, false); } +static int64_t xive_vpc_scrub(struct xive *x, uint64_t block, uint64_t idx) +{ + return __xive_cache_scrub(x, xive_cache_vpc, block, idx, false, false); +} + static int64_t xive_vpc_scrub_clean(struct xive *x, uint64_t block, uint64_t idx) { return __xive_cache_scrub(x, xive_cache_vpc, block, idx, true, false); } +static int64_t xive_eqc_scrub(struct xive *x, uint64_t block, uint64_t idx) +{ + return __xive_cache_scrub(x, xive_cache_eqc, block, idx, false, false); +} + static int64_t __xive_cache_watch(struct xive *x, enum xive_cache_type ctype, uint64_t block, uint64_t idx, uint32_t start_dword, uint32_t dword_count, @@ -4177,6 +4187,85 @@ static int64_t opal_xive_set_queue_info(uint64_t vp, uint32_t prio, return rc; } +static int64_t opal_xive_get_queue_state(uint64_t vp, uint32_t prio, + uint32_t *out_qtoggle, + uint32_t *out_qindex) +{ + uint32_t blk, idx; + struct xive *x; + struct xive_eq *eq; + int64_t rc; + + if (xive_mode != XIVE_MODE_EXPL) + return OPAL_WRONG_STATE; + + if (!out_qtoggle || !out_qindex || + !xive_eq_for_target(vp, prio, &blk, &idx)) + return OPAL_PARAMETER; + + x = xive_from_vc_blk(blk); + if (!x) + return OPAL_PARAMETER; + + eq = xive_get_eq(x, idx); + if (!eq) + return OPAL_PARAMETER; + + /* Scrub the queue */ + lock(&x->lock); + rc = xive_eqc_scrub(x, blk, idx); + unlock(&x->lock); + if (rc) + return rc; + + /* We don't do disable queues */ + if (!(eq->w0 & EQ_W0_VALID)) + return OPAL_WRONG_STATE; + + *out_qtoggle = GETFIELD(EQ_W1_GENERATION, eq->w1); + *out_qindex = GETFIELD(EQ_W1_PAGE_OFF, eq->w1); + + return OPAL_SUCCESS; +} + +static int64_t opal_xive_set_queue_state(uint64_t vp, uint32_t prio, + uint32_t qtoggle, uint32_t qindex) +{ + uint32_t blk, idx; + struct xive *x; + struct xive_eq *eq, new_eq; + int64_t rc; + + if (xive_mode != XIVE_MODE_EXPL) + return OPAL_WRONG_STATE; + + if (!xive_eq_for_target(vp, prio, &blk, &idx)) + return OPAL_PARAMETER; + + x = xive_from_vc_blk(blk); + if (!x) + return OPAL_PARAMETER; + + eq = xive_get_eq(x, idx); + if (!eq) + return OPAL_PARAMETER; + + /* We don't do disable queues */ + if (!(eq->w0 & EQ_W0_VALID)) + return OPAL_WRONG_STATE; + + new_eq = *eq; + + new_eq.w1 = SETFIELD(EQ_W1_GENERATION, new_eq.w1, qtoggle); + new_eq.w1 = SETFIELD(EQ_W1_PAGE_OFF, new_eq.w1, qindex); + + lock(&x->lock); + rc = xive_eqc_cache_update(x, blk, idx, 0, 4, &new_eq, false, false); + unlock(&x->lock); + + return rc; +} + static int64_t opal_xive_donate_page(uint32_t chip_id, uint64_t addr) { struct proc_chip *c = get_chip(chip_id); @@ -4415,6 +4504,44 @@ bail: return rc; } +static int64_t opal_xive_get_vp_state(uint64_t vp_id, uint64_t *out_state) +{ + struct xive *x; + struct xive_vp *vp; + uint32_t blk, idx; + int64_t rc; + bool group; + + if (!out_state || !xive_decode_vp(vp_id, &blk, &idx, NULL, &group)) + return OPAL_PARAMETER; + if (group) + return OPAL_PARAMETER; + x = xive_from_pc_blk(blk); + if (!x) + return OPAL_PARAMETER; + vp = xive_get_vp(x, idx); + if (!vp) + return OPAL_PARAMETER; + + /* Scrub the vp */ + lock(&x->lock); + rc = xive_vpc_scrub(x, blk, idx); + unlock(&x->lock); + if (rc) + return rc; + + if (!(vp->w0 & VP_W0_VALID)) + return OPAL_WRONG_STATE; + + /* + * Return word4 and word5 which contain the saved HW thread + * context. The IPB register is all we care for now on P9. + */ + *out_state = (((uint64_t)vp->w4) << 32) | vp->w5; + + return OPAL_SUCCESS; +} + static void xive_cleanup_cpu_tima(struct cpu_thread *c) { struct xive_cpu_state *xs = c->xstate; @@ -5336,5 +5463,8 @@ void init_xive(void) opal_register(OPAL_XIVE_SET_VP_INFO, opal_xive_set_vp_info, 3); opal_register(OPAL_XIVE_SYNC, opal_xive_sync, 2); opal_register(OPAL_XIVE_DUMP, opal_xive_dump, 2); + opal_register(OPAL_XIVE_GET_QUEUE_STATE, opal_xive_get_queue_state, 4); + opal_register(OPAL_XIVE_SET_QUEUE_STATE, opal_xive_set_queue_state, 4); + opal_register(OPAL_XIVE_GET_VP_STATE, opal_xive_get_vp_state, 2); } diff --git a/doc/xive.rst b/doc/xive.rst index 0997c7224e11..42a60907f393 100644 --- a/doc/xive.rst +++ b/doc/xive.rst @@ -790,3 +790,54 @@ state information about the XIVE. - XIVE_DUMP_EMU: Dump the state of the XICS emulation for a thread "id" is the PIR value of the thread + +OPAL_XIVE_GET_QUEUE_STATE +^^^^^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + + int64_t opal_xive_get_queue_state(uint64_t vp, uint32_t prio, + uint32_t *out_qtoggle, + uint32_t *out_qindex); + +This call saves the queue toggle bit and index. This must be called on +an enabled queue. + +* vp, prio: The target queue + +* out_qtoggle: toggle bit of the queue + +* out_qindex: index of the queue + + +OPAL_XIVE_SET_QUEUE_STATE +^^^^^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + + int64_t opal_xive_set_queue_state(uint64_t vp, uint32_t prio, + uint32_t qtoggle, + uint32_t qindex); + +This call restores the queue toggle bit and index that was previously +saved by a call to opal_xive_get_queue_state(). This must be called on +an enabled queue. + +* vp, prio: The target queue + +* qtoggle: toggle bit of the queue + +* qindex: index of the queue + + +OPAL_XIVE_GET_VP_STATE +^^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: c + + int64_t opal_xive_get_vp_state(uint64_t vp_id, + uint64_t *out_state); + +This call saves the VP HW state in "out_state". The format matches the +XIVE NVT word 4 and word 5. This must be called on an enabled VP. + +* vp_id: The target VP + +* out_state: Location where the state is to be stored