From patchwork Mon Jul 11 11:50:12 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Munsie X-Patchwork-Id: 646920 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3rp3fm4zqlz9sCg for ; Mon, 11 Jul 2016 22:04:16 +1000 (AEST) Received: from ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3rp3fm4BWVzDrct for ; Mon, 11 Jul 2016 22:04:16 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3rp3Nv2bFJzDr11 for ; Mon, 11 Jul 2016 21:52:15 +1000 (AEST) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.11/8.16.0.11) with SMTP id u6BBnIsM067992 for ; Mon, 11 Jul 2016 07:52:13 -0400 Received: from e23smtp01.au.ibm.com (e23smtp01.au.ibm.com [202.81.31.143]) by mx0a-001b2d01.pphosted.com with ESMTP id 2448qjcqtj-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 11 Jul 2016 07:52:13 -0400 Received: from localhost by e23smtp01.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 11 Jul 2016 21:52:11 +1000 Received: from d23dlp03.au.ibm.com (202.81.31.214) by e23smtp01.au.ibm.com (202.81.31.207) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 11 Jul 2016 21:52:09 +1000 X-IBM-Helo: d23dlp03.au.ibm.com X-IBM-MailFrom: imunsie@au1.ibm.com X-IBM-RcptTo: linuxppc-dev@lists.ozlabs.org Received: from d23relay08.au.ibm.com (d23relay08.au.ibm.com [9.185.71.33]) by d23dlp03.au.ibm.com (Postfix) with ESMTP id 2F9A53578053 for ; Mon, 11 Jul 2016 21:52:09 +1000 (EST) Received: from d23av01.au.ibm.com (d23av01.au.ibm.com [9.190.234.96]) by d23relay08.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id u6BBq9Up21561468 for ; Mon, 11 Jul 2016 21:52:09 +1000 Received: from d23av01.au.ibm.com (localhost [127.0.0.1]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id u6BBq887017774 for ; Mon, 11 Jul 2016 21:52:09 +1000 Received: from ozlabs.au.ibm.com (ozlabs.au.ibm.com [9.192.253.14]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id u6BBq8OQ017771; Mon, 11 Jul 2016 21:52:08 +1000 Received: from x230.au.ibm.com (unknown [9.192.198.207]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 824DDA01F2; Mon, 11 Jul 2016 21:51:36 +1000 (AEST) From: "Ian Munsie" To: Michael Ellerman , Michael Neuling , Frederic Barrat , Andrew Donnellan , linuxppc-dev@lists.ozlabs.org, Huy Nguyen Subject: [PATCH 05/15] cxl: Allow a default context to be associated with an external pci_dev Date: Mon, 11 Jul 2016 21:50:12 +1000 X-Mailer: git-send-email 2.8.1 In-Reply-To: <1468237822-26276-1-git-send-email-imunsie@au.ibm.com> References: <1468237822-26276-1-git-send-email-imunsie@au.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16071111-1617-0000-0000-000001391C7C X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16071111-1618-0000-0000-000046405E52 Message-Id: <1468237822-26276-6-git-send-email-imunsie@au.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-07-11_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1604210000 definitions=main-1607110122 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ian Munsie MIME-Version: 1.0 Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: Ian Munsie The cxl kernel API has a concept of a default context associated with each PCI device under the virtual PHB. The Mellanox CX4 will also use the cxl kernel API, but it does not use a virtual PHB - rather, the AFU appears as a physical function as a peer to the networking functions. In order to allow the kernel API to work with those networking functions, we will need to associate a default context with them as well. To this end, refactor the corresponding code to do this in vphb.c and export it so that it can be called from the PHB code. Signed-off-by: Ian Munsie Reviewed-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- drivers/misc/cxl/base.c | 35 +++++++++++++++++++++++++++++++++++ drivers/misc/cxl/cxl.h | 6 ++++++ drivers/misc/cxl/main.c | 2 ++ drivers/misc/cxl/vphb.c | 37 +++++++++++++++++++++++-------------- include/misc/cxl-base.h | 6 ++++++ 5 files changed, 72 insertions(+), 14 deletions(-) diff --git a/drivers/misc/cxl/base.c b/drivers/misc/cxl/base.c index d7dcf5b..1c3e737f 100644 --- a/drivers/misc/cxl/base.c +++ b/drivers/misc/cxl/base.c @@ -106,6 +106,41 @@ int cxl_update_properties(struct device_node *dn, } EXPORT_SYMBOL_GPL(cxl_update_properties); +/* + * API calls into the driver that may be called from the PHB code and must be + * built in. + */ +bool cxl_pci_associate_default_context(struct pci_dev *dev, struct cxl_afu *afu) +{ + bool ret; + struct cxl_calls *calls; + + calls = cxl_calls_get(); + if (!calls) + return false; + + ret = calls->cxl_pci_associate_default_context(dev, afu); + + cxl_calls_put(calls); + + return ret; +} +EXPORT_SYMBOL_GPL(cxl_pci_associate_default_context); + +void cxl_pci_disable_device(struct pci_dev *dev) +{ + struct cxl_calls *calls; + + calls = cxl_calls_get(); + if (!calls) + return; + + calls->cxl_pci_disable_device(dev); + + cxl_calls_put(calls); +} +EXPORT_SYMBOL_GPL(cxl_pci_disable_device); + static int __init cxl_base_init(void) { struct device_node *np = NULL; diff --git a/drivers/misc/cxl/cxl.h b/drivers/misc/cxl/cxl.h index d4aae6f..b81f476 100644 --- a/drivers/misc/cxl/cxl.h +++ b/drivers/misc/cxl/cxl.h @@ -719,9 +719,15 @@ static inline u64 cxl_p2n_read(struct cxl_afu *afu, cxl_p2n_reg_t reg) ssize_t cxl_pci_afu_read_err_buffer(struct cxl_afu *afu, char *buf, loff_t off, size_t count); +/* Internal functions wrapped in cxl_base to allow PHB to call them */ +bool _cxl_pci_associate_default_context(struct pci_dev *dev, struct cxl_afu *afu); +void _cxl_pci_disable_device(struct pci_dev *dev); struct cxl_calls { void (*cxl_slbia)(struct mm_struct *mm); + bool (*cxl_pci_associate_default_context)(struct pci_dev *dev, struct cxl_afu *afu); + void (*cxl_pci_disable_device)(struct pci_dev *dev); + struct module *owner; }; int register_cxl_calls(struct cxl_calls *calls); diff --git a/drivers/misc/cxl/main.c b/drivers/misc/cxl/main.c index ae68c32..4e5474b 100644 --- a/drivers/misc/cxl/main.c +++ b/drivers/misc/cxl/main.c @@ -110,6 +110,8 @@ static inline void cxl_slbia_core(struct mm_struct *mm) static struct cxl_calls cxl_calls = { .cxl_slbia = cxl_slbia_core, + .cxl_pci_associate_default_context = _cxl_pci_associate_default_context, + .cxl_pci_disable_device = _cxl_pci_disable_device, .owner = THIS_MODULE, }; diff --git a/drivers/misc/cxl/vphb.c b/drivers/misc/cxl/vphb.c index 012b6aa..c5b9c201 100644 --- a/drivers/misc/cxl/vphb.c +++ b/drivers/misc/cxl/vphb.c @@ -40,11 +40,28 @@ static void cxl_teardown_msi_irqs(struct pci_dev *pdev) */ } +bool _cxl_pci_associate_default_context(struct pci_dev *dev, struct cxl_afu *afu) +{ + struct cxl_context *ctx; + + /* + * Allocate a context to do cxl things too. This is used for interrupts + * in the peer model using a real phb, and if we eventually do DMA ops + * in the virtual phb, we'll need a default context to attach them to. + */ + ctx = cxl_dev_context_init(dev); + if (!ctx) + return false; + dev->dev.archdata.cxl_ctx = ctx; + + return (cxl_ops->afu_check_and_enable(afu) == 0); +} +/* exported via cxl_base */ + static bool cxl_pci_enable_device_hook(struct pci_dev *dev) { struct pci_controller *phb; struct cxl_afu *afu; - struct cxl_context *ctx; phb = pci_bus_to_host(dev->bus); afu = (struct cxl_afu *)phb->private_data; @@ -57,19 +74,10 @@ static bool cxl_pci_enable_device_hook(struct pci_dev *dev) set_dma_ops(&dev->dev, &dma_direct_ops); set_dma_offset(&dev->dev, PAGE_OFFSET); - /* - * Allocate a context to do cxl things too. If we eventually do real - * DMA ops, we'll need a default context to attach them to - */ - ctx = cxl_dev_context_init(dev); - if (!ctx) - return false; - dev->dev.archdata.cxl_ctx = ctx; - - return (cxl_ops->afu_check_and_enable(afu) == 0); + return _cxl_pci_associate_default_context(dev, afu); } -static void cxl_pci_disable_device(struct pci_dev *dev) +void _cxl_pci_disable_device(struct pci_dev *dev) { struct cxl_context *ctx = cxl_get_context(dev); @@ -82,6 +90,7 @@ static void cxl_pci_disable_device(struct pci_dev *dev) cxl_release_context(ctx); } } +/* exported via cxl_base */ static resource_size_t cxl_pci_window_alignment(struct pci_bus *bus, unsigned long type) @@ -197,8 +206,8 @@ static struct pci_controller_ops cxl_pci_controller_ops = { .probe_mode = cxl_pci_probe_mode, .enable_device_hook = cxl_pci_enable_device_hook, - .disable_device = cxl_pci_disable_device, - .release_device = cxl_pci_disable_device, + .disable_device = _cxl_pci_disable_device, + .release_device = _cxl_pci_disable_device, .window_alignment = cxl_pci_window_alignment, .reset_secondary_bus = cxl_pci_reset_secondary_bus, .setup_msi_irqs = cxl_setup_msi_irqs, diff --git a/include/misc/cxl-base.h b/include/misc/cxl-base.h index f53808f..bb7e629 100644 --- a/include/misc/cxl-base.h +++ b/include/misc/cxl-base.h @@ -10,6 +10,8 @@ #ifndef _MISC_CXL_BASE_H #define _MISC_CXL_BASE_H +#include + #ifdef CONFIG_CXL_BASE #define CXL_IRQ_RANGES 4 @@ -39,6 +41,8 @@ static inline void cxl_ctx_put(void) struct cxl_afu *cxl_afu_get(struct cxl_afu *afu); void cxl_afu_put(struct cxl_afu *afu); void cxl_slbia(struct mm_struct *mm); +bool cxl_pci_associate_default_context(struct pci_dev *dev, struct cxl_afu *afu); +void cxl_pci_disable_device(struct pci_dev *dev); #else /* CONFIG_CXL_BASE */ @@ -46,6 +50,8 @@ static inline bool cxl_ctx_in_use(void) { return false; } static inline struct cxl_afu *cxl_afu_get(struct cxl_afu *afu) { return NULL; } static inline void cxl_afu_put(struct cxl_afu *afu) {} static inline void cxl_slbia(struct mm_struct *mm) {} +static inline bool cxl_pci_associate_default_context(struct pci_dev *dev, struct cxl_afu *afu) { return false; } +static inline void cxl_pci_disable_device(struct pci_dev *dev) {} #endif /* CONFIG_CXL_BASE */