From patchwork Wed Nov 14 09:41:39 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Glauber X-Patchwork-Id: 198852 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 1F15F2C008F for ; Wed, 14 Nov 2012 20:44:29 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932856Ab2KNJoE (ORCPT ); Wed, 14 Nov 2012 04:44:04 -0500 Received: from e06smtp11.uk.ibm.com ([195.75.94.107]:45813 "EHLO e06smtp11.uk.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932837Ab2KNJmd (ORCPT ); Wed, 14 Nov 2012 04:42:33 -0500 Received: from /spool/local by e06smtp11.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 14 Nov 2012 09:42:32 -0000 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp11.uk.ibm.com (192.168.101.141) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 14 Nov 2012 09:42:30 -0000 Received: from d06av02.portsmouth.uk.ibm.com (d06av02.portsmouth.uk.ibm.com [9.149.37.228]) by b06cxnps4074.portsmouth.uk.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id qAE9gMhg51052678; Wed, 14 Nov 2012 09:42:22 GMT Received: from d06av02.portsmouth.uk.ibm.com (loopback [127.0.0.1]) by d06av02.portsmouth.uk.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id qAE9gRdk028254; Wed, 14 Nov 2012 02:42:29 -0700 Received: from tuxmaker.boeblingen.de.ibm.com (tuxmaker.boeblingen.de.ibm.com [9.152.85.9]) by d06av02.portsmouth.uk.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id qAE9gOBU028037; Wed, 14 Nov 2012 02:42:26 -0700 Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 45614) id D4C7F122443C; Wed, 14 Nov 2012 10:42:25 +0100 (CET) From: Jan Glauber To: linux-pci@vger.kernel.org Cc: linux-kernel@vger.kernel.org, heiko.carstens@de.ibm.com, schwidefsky@de.ibm.com, Jan Glauber Subject: [RFC PATCH 07/10] s390/pci: PCI hotplug support via SCLP Date: Wed, 14 Nov 2012 10:41:39 +0100 Message-Id: <24140c0677f914bfc51b88259155887c327a7103.1352719878.git.jang@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.12.4 In-Reply-To: References: In-Reply-To: References: x-cbid: 12111409-5024-0000-0000-0000045AFA8B Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Add SCLP PCI configure/deconfigure and implement a PCI hotplug controller (s390_pci_hpc). The hotplug controller creates a slot for every PCI function in stand-by or configured state. The PCI functions are named after the PCI function ID (fid). By writing to the power attribute in /sys/bus/pci/slots//power the PCI function is moved to stand-by or configured state. If moved to the configured state the device is automatically scanned by the s390 PCI layer. Signed-off-by: Jan Glauber --- arch/s390/include/asm/pci.h | 11 ++ arch/s390/include/asm/sclp.h | 2 + arch/s390/pci/pci.c | 9 ++ drivers/pci/hotplug/Kconfig | 11 ++ drivers/pci/hotplug/Makefile | 1 + drivers/pci/hotplug/s390_pci_hpc.c | 252 +++++++++++++++++++++++++++++++++++++ drivers/s390/char/sclp.h | 3 +- drivers/s390/char/sclp_cmd.c | 64 +++++++++- 8 files changed, 351 insertions(+), 2 deletions(-) create mode 100644 drivers/pci/hotplug/s390_pci_hpc.c diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 3dc2cb6..739d9f1 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -95,6 +95,11 @@ struct zpci_dev { enum pci_bus_speed max_bus_speed; }; +struct pci_hp_callback_ops { + int (*create_slot) (struct zpci_dev *zdev); + void (*remove_slot) (struct zpci_dev *zdev); +}; + static inline bool zdev_enabled(struct zpci_dev *zdev) { return (zdev->fh & (1UL << 31)) ? true : false; @@ -140,4 +145,10 @@ bool zpci_fid_present(u32); int zpci_dma_init(void); void zpci_dma_exit(void); +/* Hotplug */ +extern struct mutex zpci_list_lock; +extern struct list_head zpci_list; +extern struct pci_hp_callback_ops hotplug_ops; +extern unsigned int pci_probe; + #endif diff --git a/arch/s390/include/asm/sclp.h b/arch/s390/include/asm/sclp.h index e62a555..8337886 100644 --- a/arch/s390/include/asm/sclp.h +++ b/arch/s390/include/asm/sclp.h @@ -55,5 +55,7 @@ int sclp_chp_read_info(struct sclp_chp_info *info); void sclp_get_ipl_info(struct sclp_ipl_info *info); bool sclp_has_linemode(void); bool sclp_has_vt220(void); +int sclp_pci_configure(u32 fid); +int sclp_pci_deconfigure(u32 fid); #endif /* _ASM_S390_SCLP_H */ diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index 5a2ef9e..c523594 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -47,7 +47,12 @@ /* list of all detected zpci devices */ LIST_HEAD(zpci_list); +EXPORT_SYMBOL_GPL(zpci_list); DEFINE_MUTEX(zpci_list_lock); +EXPORT_SYMBOL_GPL(zpci_list_lock); + +struct pci_hp_callback_ops hotplug_ops; +EXPORT_SYMBOL_GPL(hotplug_ops); static DECLARE_BITMAP(zpci_domain, ZPCI_NR_DEVICES); static DEFINE_SPINLOCK(zpci_domain_lock); @@ -935,6 +940,8 @@ int zpci_create_device(struct zpci_dev *zdev) mutex_lock(&zpci_list_lock); list_add_tail(&zdev->entry, &zpci_list); + if (hotplug_ops.create_slot) + hotplug_ops.create_slot(zdev); mutex_unlock(&zpci_list_lock); if (zdev->state == ZPCI_FN_STATE_STANDBY) @@ -948,6 +955,8 @@ int zpci_create_device(struct zpci_dev *zdev) out_start: mutex_lock(&zpci_list_lock); list_del(&zdev->entry); + if (hotplug_ops.remove_slot) + hotplug_ops.remove_slot(zdev); mutex_unlock(&zpci_list_lock); out_bus: zpci_free_domain(zdev); diff --git a/drivers/pci/hotplug/Kconfig b/drivers/pci/hotplug/Kconfig index b0e46de..0594c4a 100644 --- a/drivers/pci/hotplug/Kconfig +++ b/drivers/pci/hotplug/Kconfig @@ -151,4 +151,15 @@ config HOTPLUG_PCI_SGI When in doubt, say N. +config HOTPLUG_PCI_S390 + tristate "System z PCI Hotplug Support" + depends on S390 && 64BIT + help + Say Y here if you want to use the System z PCI Hotplug + driver for PCI devices. Without this driver it is not + possible to access stand-by PCI functions nor to deconfigure + PCI functions. + + When in doubt, say Y. + endif # HOTPLUG_PCI diff --git a/drivers/pci/hotplug/Makefile b/drivers/pci/hotplug/Makefile index c459cd4..47ec8c8 100644 --- a/drivers/pci/hotplug/Makefile +++ b/drivers/pci/hotplug/Makefile @@ -18,6 +18,7 @@ obj-$(CONFIG_HOTPLUG_PCI_RPA) += rpaphp.o obj-$(CONFIG_HOTPLUG_PCI_RPA_DLPAR) += rpadlpar_io.o obj-$(CONFIG_HOTPLUG_PCI_SGI) += sgi_hotplug.o obj-$(CONFIG_HOTPLUG_PCI_ACPI) += acpiphp.o +obj-$(CONFIG_HOTPLUG_PCI_S390) += s390_pci_hpc.o # acpiphp_ibm extends acpiphp, so should be linked afterwards. diff --git a/drivers/pci/hotplug/s390_pci_hpc.c b/drivers/pci/hotplug/s390_pci_hpc.c new file mode 100644 index 0000000..d4de895 --- /dev/null +++ b/drivers/pci/hotplug/s390_pci_hpc.c @@ -0,0 +1,252 @@ +/* + * PCI Hot Plug Controller Driver for System z + * + * Copyright 2012 IBM Corp. + * + * Author(s): + * Jan Glauber + */ + +#define COMPONENT "zPCI hpc" +#define pr_fmt(fmt) COMPONENT ": " fmt + +#include +#include +#include +#include +#include +#include +#include + +#define SLOT_NAME_SIZE 10 +static LIST_HEAD(s390_hotplug_slot_list); + +MODULE_AUTHOR("Jan Glauber private; + int rc; + + if (slot->zdev->state != ZPCI_FN_STATE_STANDBY) + return -EIO; + + rc = sclp_pci_configure(slot->zdev->fid); + if (!rc) { + slot->zdev->state = ZPCI_FN_STATE_CONFIGURED; + /* automatically scan the device after is was configured */ + zpci_enable_device(slot->zdev); + zpci_scan_device(slot->zdev); + } + return rc; +} + +static int disable_slot(struct hotplug_slot *hotplug_slot) +{ + struct slot *slot = hotplug_slot->private; + int rc; + + if (!zpci_fn_configured(slot->zdev->state)) + return -EIO; + + /* TODO: we rely on the user to unbind/remove the device, is that plausible + * or do we need to trigger that here? + */ + rc = sclp_pci_deconfigure(slot->zdev->fid); + if (!rc) { + /* Fixme: better call List-PCI to find the disabled FH + for the FID since the FH should be opaque... */ + slot->zdev->fh &= 0x7fffffff; + slot->zdev->state = ZPCI_FN_STATE_STANDBY; + } + return rc; +} + +static int get_power_status(struct hotplug_slot *hotplug_slot, u8 *value) +{ + struct slot *slot = hotplug_slot->private; + + switch (slot->zdev->state) { + case ZPCI_FN_STATE_STANDBY: + *value = 0; + break; + default: + *value = 1; + break; + } + return 0; +} + +static int get_adapter_status(struct hotplug_slot *hotplug_slot, u8 *value) +{ + /* if the slot exits it always contains a function */ + *value = 1; + return 0; +} + +static void release_slot(struct hotplug_slot *hotplug_slot) +{ + struct slot *slot = hotplug_slot->private; + + pr_debug("%s - physical_slot = %s\n", __func__, hotplug_slot_name(hotplug_slot)); + kfree(slot->hotplug_slot->info); + kfree(slot->hotplug_slot); + kfree(slot); +} + +static struct hotplug_slot_ops s390_hotplug_slot_ops = { + .enable_slot = enable_slot, + .disable_slot = disable_slot, + .get_power_status = get_power_status, + .get_adapter_status = get_adapter_status, +}; + +static int init_pci_slot(struct zpci_dev *zdev) +{ + struct hotplug_slot *hotplug_slot; + struct hotplug_slot_info *info; + char name[SLOT_NAME_SIZE]; + struct slot *slot; + int rc; + + if (!zdev) + return 0; + + slot = kzalloc(sizeof(*slot), GFP_KERNEL); + if (!slot) + goto error; + + hotplug_slot = kzalloc(sizeof(*hotplug_slot), GFP_KERNEL); + if (!hotplug_slot) + goto error_hp; + hotplug_slot->private = slot; + + slot->hotplug_slot = hotplug_slot; + slot->zdev = zdev; + + info = kzalloc(sizeof(*info), GFP_KERNEL); + if (!info) + goto error_info; + hotplug_slot->info = info; + + hotplug_slot->ops = &s390_hotplug_slot_ops; + hotplug_slot->release = &release_slot; + + get_power_status(hotplug_slot, &info->power_status); + get_adapter_status(hotplug_slot, &info->adapter_status); + + snprintf(name, SLOT_NAME_SIZE, "%08x", zdev->fid); + rc = pci_hp_register(slot->hotplug_slot, zdev->bus, + ZPCI_DEVFN, name); + if (rc) { + pr_err("pci_hp_register failed with error %d\n", rc); + goto error_reg; + } + list_add(&slot->slot_list, &s390_hotplug_slot_list); + return 0; + +error_reg: + kfree(info); +error_info: + kfree(hotplug_slot); +error_hp: + kfree(slot); +error: + return -ENOMEM; +} + +static int __init init_pci_slots(void) +{ + struct zpci_dev *zdev; + int device = 0; + + /* + * Create a structure for each slot, and register that slot + * with the pci_hotplug subsystem. + */ + mutex_lock(&zpci_list_lock); + list_for_each_entry(zdev, &zpci_list, entry) { + init_pci_slot(zdev); + device++; + } + + mutex_unlock(&zpci_list_lock); + return (device) ? 0 : -ENODEV; +} + +static void exit_pci_slot(struct zpci_dev *zdev) +{ + struct list_head *tmp, *n; + struct slot *slot; + + list_for_each_safe(tmp, n, &s390_hotplug_slot_list) { + slot = list_entry(tmp, struct slot, slot_list); + if (slot->zdev != zdev) + continue; + list_del(&slot->slot_list); + pci_hp_deregister(slot->hotplug_slot); + } +} + +static void __exit exit_pci_slots(void) +{ + struct list_head *tmp, *n; + struct slot *slot; + + /* + * Unregister all of our slots with the pci_hotplug subsystem. + * Memory will be freed in release_slot() callback after slot's + * lifespan is finished. + */ + list_for_each_safe(tmp, n, &s390_hotplug_slot_list) { + slot = list_entry(tmp, struct slot, slot_list); + list_del(&slot->slot_list); + pci_hp_deregister(slot->hotplug_slot); + } +} + +static int __init pci_hotplug_s390_init(void) +{ + /* + * Do specific initialization stuff for your driver here + * like initializing your controller hardware (if any) and + * determining the number of slots you have in the system + * right now. + */ + + if (!pci_probe) + return -EOPNOTSUPP; + + /* register callbacks for slot handling from arch code */ + mutex_lock(&zpci_list_lock); + hotplug_ops.create_slot = init_pci_slot; + hotplug_ops.remove_slot = exit_pci_slot; + mutex_unlock(&zpci_list_lock); + pr_info("registered hotplug slot callbacks\n"); + return init_pci_slots(); +} + +static void __exit pci_hotplug_s390_exit(void) +{ + exit_pci_slots(); +} + +module_init(pci_hotplug_s390_init); +module_exit(pci_hotplug_s390_exit); diff --git a/drivers/s390/char/sclp.h b/drivers/s390/char/sclp.h index d7e97ae..25bcd4c 100644 --- a/drivers/s390/char/sclp.h +++ b/drivers/s390/char/sclp.h @@ -1,5 +1,5 @@ /* - * Copyright IBM Corp. 1999, 2009 + * Copyright IBM Corp. 1999,2012 * * Author(s): Martin Peschke * Martin Schwidefsky @@ -103,6 +103,7 @@ extern u64 sclp_facilities; #define SCLP_HAS_CHP_RECONFIG (sclp_facilities & 0x2000000000000000ULL) #define SCLP_HAS_CPU_INFO (sclp_facilities & 0x0800000000000000ULL) #define SCLP_HAS_CPU_RECONFIG (sclp_facilities & 0x0400000000000000ULL) +#define SCLP_HAS_PCI_RECONFIG (sclp_facilities & 0x0000000040000000ULL) struct gds_subvector { diff --git a/drivers/s390/char/sclp_cmd.c b/drivers/s390/char/sclp_cmd.c index 71ea923c..f308bac 100644 --- a/drivers/s390/char/sclp_cmd.c +++ b/drivers/s390/char/sclp_cmd.c @@ -1,5 +1,5 @@ /* - * Copyright IBM Corp. 2007, 2009 + * Copyright IBM Corp. 2007,2012 * * Author(s): Heiko Carstens , * Peter Oberparleiter @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -702,6 +703,67 @@ __initcall(sclp_detect_standby_memory); #endif /* CONFIG_MEMORY_HOTPLUG */ /* + * PCI I/O adapter configuration related functions. + */ +#define SCLP_CMDW_CONFIGURE_PCI 0x001a0001 +#define SCLP_CMDW_DECONFIGURE_PCI 0x001b0001 + +#define SCLP_RECONFIG_PCI_ATPYE 2 + +struct pci_cfg_sccb { + struct sccb_header header; + u8 atype; /* adapter type */ + u8 reserved1; + u16 reserved2; + u32 aid; /* adapter identifier */ +} __packed; + +static int do_pci_configure(sclp_cmdw_t cmd, u32 fid) +{ + struct pci_cfg_sccb *sccb; + int rc; + + if (!SCLP_HAS_PCI_RECONFIG) + return -EOPNOTSUPP; + + sccb = (struct pci_cfg_sccb *) get_zeroed_page(GFP_KERNEL | GFP_DMA); + if (!sccb) + return -ENOMEM; + + sccb->header.length = PAGE_SIZE; + sccb->atype = SCLP_RECONFIG_PCI_ATPYE; + sccb->aid = fid; + rc = do_sync_request(cmd, sccb); + if (rc) + goto out; + switch (sccb->header.response_code) { + case 0x0020: + case 0x0120: + break; + default: + pr_warn("configure PCI I/O adapter failed: cmd=0x%08x response=0x%04x\n", + cmd, sccb->header.response_code); + rc = -EIO; + break; + } +out: + free_page((unsigned long) sccb); + return rc; +} + +int sclp_pci_configure(u32 fid) +{ + return do_pci_configure(SCLP_CMDW_CONFIGURE_PCI, fid); +} +EXPORT_SYMBOL(sclp_pci_configure); + +int sclp_pci_deconfigure(u32 fid) +{ + return do_pci_configure(SCLP_CMDW_DECONFIGURE_PCI, fid); +} +EXPORT_SYMBOL(sclp_pci_deconfigure); + +/* * Channel path configuration related functions. */