From patchwork Mon Oct 26 09:53:05 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bharata B Rao X-Patchwork-Id: 535799 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 8AEE614076B for ; Mon, 26 Oct 2015 20:58:33 +1100 (AEDT) Received: from localhost ([::1]:51581 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZqeXv-000352-E9 for incoming@patchwork.ozlabs.org; Mon, 26 Oct 2015 05:58:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53919) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZqeXU-0002Pe-Vy for qemu-devel@nongnu.org; Mon, 26 Oct 2015 05:58:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZqeXR-0004NI-Nw for qemu-devel@nongnu.org; Mon, 26 Oct 2015 05:58:04 -0400 Received: from e28smtp04.in.ibm.com ([122.248.162.4]:57874) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZqeXQ-0004Ma-U4 for qemu-devel@nongnu.org; Mon, 26 Oct 2015 05:58:01 -0400 Received: from /spool/local by e28smtp04.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 26 Oct 2015 15:27:55 +0530 Received: from d28dlp02.in.ibm.com (9.184.220.127) by e28smtp04.in.ibm.com (192.168.1.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 26 Oct 2015 15:27:52 +0530 X-Helo: d28dlp02.in.ibm.com X-MailFrom: bharata@linux.vnet.ibm.com X-RcptTo: qemu-ppc@nongnu.org Received: from d28relay03.in.ibm.com (d28relay03.in.ibm.com [9.184.220.60]) by d28dlp02.in.ibm.com (Postfix) with ESMTP id C86653940062; Mon, 26 Oct 2015 15:27:50 +0530 (IST) Received: from d28av04.in.ibm.com (d28av04.in.ibm.com [9.184.220.66]) by d28relay03.in.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t9Q9ukiH58523652; Mon, 26 Oct 2015 15:27:49 +0530 Received: from d28av04.in.ibm.com (localhost [127.0.0.1]) by d28av04.in.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t9Q9rGhY018998; Mon, 26 Oct 2015 15:23:16 +0530 Received: from bharata.in.ibm.com ([9.124.35.232]) by d28av04.in.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id t9Q9rCi9018523; Mon, 26 Oct 2015 15:23:12 +0530 From: Bharata B Rao To: qemu-devel@nongnu.org Date: Mon, 26 Oct 2015 15:23:05 +0530 Message-Id: <1445853185-22518-1-git-send-email-bharata@linux.vnet.ibm.com> X-Mailer: git-send-email 2.1.0 X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15102609-0013-0000-0000-00000817EC2B X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 122.248.162.4 Cc: Bharata B Rao , agraf@suse.de, mdroth@linux.vnet.ibm.com, qemu-ppc@nongnu.org, tyreld@linux.vnet.ibm.com, nfont@linux.vnet.ibm.com, imammedo@redhat.com, david@gibson.dropbear.id.au Subject: [Qemu-devel] [RFC PATCH v1] spapr: Memory hot-unplug support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Add support to hot remove pc-dimm memory devices. TODO: In response to memory hot removal operation on a DIMM device, guest kernel might refuse to offline a few LMBs that are part of that device. In such cases, we will have a DIMM device that has some LMBs online and some LMBs offline. To avoid this situation, drmgr could be enhanced to support a command line option that results in removal of all the LMBs or none. Signed-off-by: Bharata B Rao --- Changes in v1: - Got rid of the patch that introduced a field in PCDIMMDevice to track DIMM marked for removal since we can track that using within DRC object. - Removed the patch that added return value to rtas_set_indicator() since the required changes are already pushed by Michael Roth. v0: hw/ppc/spapr.c | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++++- hw/ppc/spapr_drc.c | 18 +++++++++++ 2 files changed, 107 insertions(+), 1 deletion(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index e1202ce..f5b1ac2 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2174,6 +2174,85 @@ out: error_propagate(errp, local_err); } +typedef struct sPAPRDIMMState { + uint32_t nr_lmbs; +} sPAPRDIMMState; + +static void spapr_lmb_release(DeviceState *dev, void *opaque) +{ + sPAPRDIMMState *ds = (sPAPRDIMMState *)opaque; + HotplugHandler *hotplug_ctrl = NULL; + Error *local_err = NULL; + + if (--ds->nr_lmbs) { + return; + } + + g_free(ds); + + /* + * Now that all the LMBs have been removed by the guest, call the + * pc-dimm unplug handler to cleanup up the pc-dimm device. + */ + hotplug_ctrl = qdev_get_hotplug_handler(dev); + hotplug_handler_unplug(hotplug_ctrl, dev, &local_err); +} + +static void spapr_del_lmbs(DeviceState *dev, uint64_t addr, uint64_t size, + Error **errp) +{ + sPAPRDRConnector *drc; + sPAPRDRConnectorClass *drck; + uint32_t nr_lmbs = size/SPAPR_MEMORY_BLOCK_SIZE; + Error *local_err = NULL; + int i; + sPAPRDIMMState *ds = g_malloc0(sizeof(sPAPRDIMMState)); + + ds->nr_lmbs = nr_lmbs; + for (i = 0; i < nr_lmbs; i++) { + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_LMB, + addr/SPAPR_MEMORY_BLOCK_SIZE); + g_assert(drc); + + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); + drck->detach(drc, dev, spapr_lmb_release, ds, &local_err); + addr += SPAPR_MEMORY_BLOCK_SIZE; + } + spapr_hotplug_req_remove_by_count(SPAPR_DR_CONNECTOR_TYPE_LMB, nr_lmbs); +} + +static void spapr_memory_unplug(HotplugHandler *hotplug_dev, DeviceState *dev, + Error **errp) +{ + sPAPRMachineState *ms = SPAPR_MACHINE(hotplug_dev); + PCDIMMDevice *dimm = PC_DIMM(dev); + PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm); + MemoryRegion *mr = ddc->get_memory_region(dimm); + + pc_dimm_memory_unplug(dev, &ms->hotplug_memory, mr); + object_unparent(OBJECT(dev)); +} + +static void spapr_memory_unplug_request(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + Error *local_err = NULL; + PCDIMMDevice *dimm = PC_DIMM(dev); + PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm); + MemoryRegion *mr = ddc->get_memory_region(dimm); + uint64_t size = memory_region_size(mr); + uint64_t addr; + + addr = object_property_get_int(OBJECT(dimm), PC_DIMM_ADDR_PROP, &local_err); + if (local_err) { + goto out; + } + + spapr_del_lmbs(dev, addr, size, &local_err); +out: + error_propagate(errp, local_err); +} + static void spapr_machine_device_plug(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { @@ -2221,7 +2300,15 @@ static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { - error_setg(errp, "Memory hot unplug not supported by sPAPR"); + spapr_memory_unplug(hotplug_dev, dev, errp); + } +} + +static void spapr_machine_device_unplug_request(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { + spapr_memory_unplug_request(hotplug_dev, dev, errp); } } @@ -2263,6 +2350,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) hc->plug = spapr_machine_device_plug; hc->unplug = spapr_machine_device_unplug; mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id; + hc->unplug_request = spapr_machine_device_unplug_request; smc->dr_lmb_enabled = false; fwc->get_dev_path = spapr_get_fw_dev_path; diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c index 5d6ea7c..59b6ea9 100644 --- a/hw/ppc/spapr_drc.c +++ b/hw/ppc/spapr_drc.c @@ -11,6 +11,7 @@ */ #include "hw/ppc/spapr_drc.h" +#include "hw/ppc/spapr.h" #include "qom/object.h" #include "hw/qdev.h" #include "qapi/visitor.h" @@ -77,6 +78,23 @@ static uint32_t set_isolation_state(sPAPRDRConnector *drc, } } + /* + * Fail any request to ISOLATE the LMB DRC if this LMB doesn't + * belong to a DIMM device that is marked for removal. + * + * Currently the guest userspace tool drmgr that drives the memory + * hotplug/unplug will just try to remove a set of 'removable' LMBs + * in response to a hot unplug request that is based on drc-count. + * If the LMB being removed doesn't belong to a DIMM device that is + * actually being unplugged, fail the isolation request here. + */ + if (drc->type == SPAPR_DR_CONNECTOR_TYPE_LMB) { + if ((state == SPAPR_DR_ISOLATION_STATE_ISOLATED) && + !drc->awaiting_release) { + return RTAS_OUT_HW_ERROR; + } + } + drc->isolation_state = state; if (drc->isolation_state == SPAPR_DR_ISOLATION_STATE_ISOLATED) {