From patchwork Fri Apr 24 06:47:42 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bharata B Rao X-Patchwork-Id: 464127 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 9512A1400A0 for ; Fri, 24 Apr 2015 16:56:06 +1000 (AEST) Received: from localhost ([::1]:43145 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YlXWu-0005St-Ko for incoming@patchwork.ozlabs.org; Fri, 24 Apr 2015 02:56:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53968) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YlXUm-0001My-Vf for qemu-devel@nongnu.org; Fri, 24 Apr 2015 02:53:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YlXUi-0005rB-Km for qemu-devel@nongnu.org; Fri, 24 Apr 2015 02:53:52 -0400 Received: from e23smtp02.au.ibm.com ([202.81.31.144]:60799) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YlXUi-0005qk-0A for qemu-devel@nongnu.org; Fri, 24 Apr 2015 02:53:48 -0400 Received: from /spool/local by e23smtp02.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 24 Apr 2015 16:53:46 +1000 Received: from d23dlp02.au.ibm.com (202.81.31.213) by e23smtp02.au.ibm.com (202.81.31.208) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 24 Apr 2015 16:53:44 +1000 Received: from d23relay09.au.ibm.com (d23relay09.au.ibm.com [9.185.63.181]) by d23dlp02.au.ibm.com (Postfix) with ESMTP id D0E2C2BB004D; Fri, 24 Apr 2015 16:53:43 +1000 (EST) Received: from d23av02.au.ibm.com (d23av02.au.ibm.com [9.190.235.138]) by d23relay09.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t3O6rZ7S46399574; Fri, 24 Apr 2015 16:53:43 +1000 Received: from d23av02.au.ibm.com (localhost [127.0.0.1]) by d23av02.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t3O6rA5t032008; Fri, 24 Apr 2015 16:53:10 +1000 Received: from bharata.in.ibm.com ([9.79.216.71]) by d23av02.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id t3O6m5Qw023890; Fri, 24 Apr 2015 16:52:59 +1000 From: Bharata B Rao To: qemu-devel@nongnu.org Date: Fri, 24 Apr 2015 12:17:42 +0530 Message-Id: <1429858066-12088-21-git-send-email-bharata@linux.vnet.ibm.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1429858066-12088-1-git-send-email-bharata@linux.vnet.ibm.com> References: <1429858066-12088-1-git-send-email-bharata@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15042406-0005-0000-0000-000001B0ED44 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 202.81.31.144 Cc: aik@ozlabs.ru, Bharata B Rao , mdroth@linux.vnet.ibm.com, agraf@suse.de, qemu-ppc@nongnu.org, tyreld@linux.vnet.ibm.com, nfont@linux.vnet.ibm.com, imammedo@redhat.com, afaerber@suse.de, david@gibson.dropbear.id.au Subject: [Qemu-devel] [RFC PATCH v3 20/24] spapr: CPU hot unplug support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Support hot removal of CPU for sPAPR guests by sending the hot unplug notification to the guest via EPOW interrupt. Release the vCPU object after CPU hot unplug so that vCPU fd can be parked and reused. Signed-off-by: Bharata B Rao --- hw/ppc/spapr.c | 101 +++++++++++++++++++++++++++++++++++++++++++- target-ppc/translate_init.c | 10 +++++ 2 files changed, 110 insertions(+), 1 deletion(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 9b0701c..910a50f 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -1481,6 +1481,12 @@ static void spapr_cpu_init(PowerPCCPU *cpu) qemu_register_reset(spapr_cpu_reset, cpu); } +static void spapr_cpu_destroy(PowerPCCPU *cpu) +{ + xics_cpu_destroy(spapr->icp, cpu); + qemu_unregister_reset(spapr_cpu_reset, cpu); +} + /* pSeries LPAR / sPAPR hardware init */ static void ppc_spapr_init(MachineState *machine) { @@ -1883,6 +1889,24 @@ static void *spapr_populate_hotplug_cpu_dt(DeviceState *dev, CPUState *cs, return fdt; } +static void spapr_cpu_release(DeviceState *dev, void *opaque) +{ + CPUState *cs; + int i; + int id = ppc_get_vcpu_dt_id(POWERPC_CPU(CPU(dev))); + + for (i = id; i < id + smp_threads; i++) { + CPU_FOREACH(cs) { + PowerPCCPU *cpu = POWERPC_CPU(cs); + + if (i == ppc_get_vcpu_dt_id(cpu)) { + spapr_cpu_destroy(cpu); + cpu_remove(cs); + } + } + } +} + static void spapr_cpu_plug(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { @@ -1970,6 +1994,59 @@ static void spapr_cpu_plug(HotplugHandler *hotplug_dev, DeviceState *dev, return; } +static int spapr_cpu_unplug(Object *obj, void *opaque) +{ + Error **errp = opaque; + DeviceState *dev = DEVICE(obj); + CPUState *cs = CPU(dev); + PowerPCCPU *cpu = POWERPC_CPU(cs); + int id = ppc_get_vcpu_dt_id(cpu); + int smt = kvmppc_smt_threads(); + sPAPRDRConnector *drc = + spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_CPU, id); + sPAPRDRConnectorClass *drck; + Error *local_err = NULL; + + /* + * SMT threads return from here, only main thread (core) will + * continue and signal hot unplug event to the guest. + */ + if ((id % smt) != 0) { + return 0; + } + g_assert(drc); + + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); + drck->detach(drc, dev, spapr_cpu_release, NULL, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return -1; + } + + /* + * In addition to hotplugged CPUs, send the hot-unplug notification + * interrupt to the guest for coldplugged CPUs started via -device + * option too. + */ + spapr_hotplug_req_remove_event(drc); + + return 0; +} + +static int spapr_cpu_core_unplug(Object *obj, void *opaque) +{ + Error **errp = opaque; + + object_child_foreach(obj, spapr_cpu_unplug, errp); + return 0; +} + +static void spapr_cpu_socket_unplug(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + object_child_foreach(OBJECT(dev), spapr_cpu_core_unplug, errp); +} + static void spapr_machine_device_plug(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { @@ -1984,16 +2061,36 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev, * Fail hotplug on machines where CPU DR isn't enabled. */ if (!spapr->dr_cpu_enabled && dev->hotplugged) { + /* + * FIXME: Ideally should fail hotplug here by doing an error_setg, + * but failing hotplug here doesn't work well with the vCPU object + * removal code. Hence silently refusing to add CPUs here. + */ + spapr_cpu_destroy(cpu); + cpu_remove(cs); return; } spapr_cpu_plug(hotplug_dev, dev, errp); } } +static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + if (object_dynamic_cast(OBJECT(dev), TYPE_CPU_SOCKET)) { + if (!spapr->dr_cpu_enabled) { + error_setg(errp, "CPU hot unplug not supported on this machine"); + return; + } + spapr_cpu_socket_unplug(hotplug_dev, dev, errp); + } +} + static HotplugHandler *spapr_get_hotpug_handler(MachineState *machine, DeviceState *dev) { - if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) { + if (object_dynamic_cast(OBJECT(dev), TYPE_CPU) || + object_dynamic_cast(OBJECT(dev), TYPE_CPU_SOCKET)) { return HOTPLUG_HANDLER(machine); } return NULL; @@ -2017,6 +2114,8 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) mc->has_dynamic_sysbus = true; mc->get_hotplug_handler = spapr_get_hotpug_handler; hc->plug = spapr_machine_device_plug; + hc->unplug = spapr_machine_device_unplug; + smc->dr_phb_enabled = false; smc->dr_cpu_enabled = false; smc->dr_lmb_enabled = false; diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c index fccee82..8e24235 100644 --- a/target-ppc/translate_init.c +++ b/target-ppc/translate_init.c @@ -30,6 +30,7 @@ #include "qemu/error-report.h" #include "qapi/visitor.h" #include "hw/qdev-properties.h" +#include "migration/vmstate.h" //#define PPC_DUMP_CPU //#define PPC_DEBUG_SPR @@ -9143,9 +9144,18 @@ static void ppc_cpu_unrealizefn(DeviceState *dev, Error **errp) { PowerPCCPU *cpu = POWERPC_CPU(dev); CPUPPCState *env = &cpu->env; + CPUClass *cc = CPU_GET_CLASS(dev); opc_handler_t **table; int i, j; + if (qdev_get_vmsd(dev) == NULL) { + vmstate_unregister(NULL, &vmstate_cpu_common, cpu); + } + + if (cc->vmsd != NULL) { + vmstate_unregister(NULL, cc->vmsd, cpu); + } + cpu_exec_exit(CPU(dev)); for (i = 0; i < PPC_CPU_OPCODES_LEN; i++) {