From patchwork Fri May 31 15:59:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Henrique Cerri X-Patchwork-Id: 1108423 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45Fpzh0N9Nz9s4V; Sat, 1 Jun 2019 01:59:28 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1hWjwC-00027T-5s; Fri, 31 May 2019 15:59:24 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.86_2) (envelope-from ) id 1hWjw4-00024m-FI for kernel-team@lists.ubuntu.com; Fri, 31 May 2019 15:59:16 +0000 Received: from mail-qk1-f199.google.com ([209.85.222.199]) by youngberry.canonical.com with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.76) (envelope-from ) id 1hWjw3-00013U-Qa for kernel-team@lists.ubuntu.com; Fri, 31 May 2019 15:59:16 +0000 Received: by mail-qk1-f199.google.com with SMTP id t196so8280619qke.0 for ; Fri, 31 May 2019 08:59:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YcBywWfkFbsob3L/FO09oCqrSK7qkpuWymtoFEyAbh4=; b=rFQ5X27HVRLVCcZDf9d7bGGAf7pyJ8MY+iwUuiWvN6EBcikTNbB17+/pMkMh/dzB6/ 4Q24q7djZc17J+D6TFxUKuV3q7SE2Yughoh7TR41d5BFntp7iJez2bDh3ueCI/zFqUQd gb2ZA217D0xNI7HM+oc0ZPQl4ylJ2CaoaSkWgMYnwZnzAM0fhtBVlUBoRENBh8nw+KNA UZllD7DECfJ9G88TvOQZNhUZA1QhLQ2o0aViksmf5ufVWPu4WZqMFEOCqk31KpzyQK8B sOwl8t8h9GqHg4PreeD3jTm/MrOSVG71hfI50l8RAr+Xs5m/lrmgakVzW+fDGinoeAyP HlhQ== X-Gm-Message-State: APjAAAViWIGuLjrfXLzVzWpmxjiifW6B8UTXdVir98KfpFvC6hcoyDSA uH0jUnbhh1F9+ZusnVSKxLg1MGHai3gmv8QMVVsX1/ba3DkY0dbAGoaDYAZX3/Snx5jwXXm/dg0 CXnWycwHGlVBrSadm80ORbndrHZW6PWKch7H1qahx X-Received: by 2002:a0c:fde3:: with SMTP id m3mr9490708qvu.205.1559318354043; Fri, 31 May 2019 08:59:14 -0700 (PDT) X-Google-Smtp-Source: APXvYqyQJrZbaR8rsgdYagNW8VfMHtfuGPfgmaazUDoy7Yis2tFTvRuKeI7hGoLv3ZhAu13J2IO1Bw== X-Received: by 2002:a0c:fde3:: with SMTP id m3mr9490571qvu.205.1559318352368; Fri, 31 May 2019 08:59:12 -0700 (PDT) Received: from gallifrey.lan ([2804:14c:4e3:4a76:91e0:c4e3:a795:446d]) by smtp.gmail.com with ESMTPSA id z20sm4227331qtz.34.2019.05.31.08.59.10 for (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Fri, 31 May 2019 08:59:11 -0700 (PDT) From: Marcelo Henrique Cerri To: kernel-team@lists.ubuntu.com Subject: [d/azure][PATCH 1/2] UBUNTU: SAUCE: linux-azure: Include Catapult FPGA PCI driver Date: Fri, 31 May 2019 12:59:04 -0300 Message-Id: <20190531155905.29036-2-marcelo.cerri@canonical.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190531155905.29036-1-marcelo.cerri@canonical.com> References: <20190531155905.29036-1-marcelo.cerri@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" BugLink: http://bugs.launchpad.net/bugs/1824879 Signed-off-by: Marcelo Henrique Cerri --- drivers/Kconfig | 2 + drivers/Makefile | 3 + drivers/catapult/Kconfig | 11 + drivers/catapult/Makefile | 27 + drivers/catapult/catapult-attributes.c | 216 +++++++ drivers/catapult/catapult-device.c | 408 ++++++++++++ drivers/catapult/catapult-device.h | 23 + drivers/catapult/catapult-drv.c | 860 +++++++++++++++++++++++++ drivers/catapult/catapult-drv.h | 145 +++++ drivers/catapult/catapult-ioctl.c | 480 ++++++++++++++ drivers/catapult/catapult-ioctl.h | 16 + drivers/catapult/catapult-register.c | 193 ++++++ drivers/catapult/catapult-register.h | 36 ++ drivers/catapult/catapult-shell.h | 215 +++++++ drivers/catapult/catapult.h | 138 ++++ 15 files changed, 2773 insertions(+) create mode 100644 drivers/catapult/Kconfig create mode 100644 drivers/catapult/Makefile create mode 100644 drivers/catapult/catapult-attributes.c create mode 100644 drivers/catapult/catapult-device.c create mode 100644 drivers/catapult/catapult-device.h create mode 100644 drivers/catapult/catapult-drv.c create mode 100644 drivers/catapult/catapult-drv.h create mode 100644 drivers/catapult/catapult-ioctl.c create mode 100644 drivers/catapult/catapult-ioctl.h create mode 100644 drivers/catapult/catapult-register.c create mode 100644 drivers/catapult/catapult-register.h create mode 100644 drivers/catapult/catapult-shell.h create mode 100644 drivers/catapult/catapult.h diff --git a/drivers/Kconfig b/drivers/Kconfig index 4f9f99057ff8..4ba286b601f0 100644 --- a/drivers/Kconfig +++ b/drivers/Kconfig @@ -228,4 +228,6 @@ source "drivers/siox/Kconfig" source "drivers/slimbus/Kconfig" +source "drivers/catapult/Kconfig" + endmenu diff --git a/drivers/Makefile b/drivers/Makefile index d11eb42f856c..12627fbb844e 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -187,3 +187,6 @@ obj-$(CONFIG_MULTIPLEXER) += mux/ obj-$(CONFIG_UNISYS_VISORBUS) += visorbus/ obj-$(CONFIG_SIOX) += siox/ obj-$(CONFIG_GNSS) += gnss/ + +# Catapult FPGA driver for linux-azure +obj-$(CONFIG_CATAPULT_PCI) += catapult/ diff --git a/drivers/catapult/Kconfig b/drivers/catapult/Kconfig new file mode 100644 index 000000000000..206bb82a3107 --- /dev/null +++ b/drivers/catapult/Kconfig @@ -0,0 +1,11 @@ +config CATAPULT_PCI + tristate "Catapult FPGA PCI Driver" + depends on PCI + help + Select this option to enable PCI driver for PCI-based + Field-Programmable Gate Array (FPGA) solutions which + implement the Catapult FPGA interface. This driver + provides interfaces for userspace applications to configure, + open and access the Catapult-based accelerators on the FPGA. + + To compile this as a module, choose M here. diff --git a/drivers/catapult/Makefile b/drivers/catapult/Makefile new file mode 100644 index 000000000000..d58a3aff8160 --- /dev/null +++ b/drivers/catapult/Makefile @@ -0,0 +1,27 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Makefile for the Catapult FPGA driver +# +obj-m += catapult.o +catapult-y := catapult-attributes.o \ + catapult-device.o \ + catapult-drv.o \ + catapult-ioctl.o \ + catapult-register.o + +ifeq "$(LIBMODULES)" "" + LIBMODULES=/lib/modules/$(shell uname -r) +endif + +ifeq "$(M)" "" + M=$(shell pwd) +endif + +ccflags-y +=-Wdeclaration-after-statement + +all: + make -C $(LIBMODULES)/build M=$(M) modules + +clean: + make -C $(LIBMODULES)/build M=$(M) clean + rm -f *.o.ur-safe diff --git a/drivers/catapult/catapult-attributes.c b/drivers/catapult/catapult-attributes.c new file mode 100644 index 000000000000..2fe605947e5e --- /dev/null +++ b/drivers/catapult/catapult-attributes.c @@ -0,0 +1,216 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2019 Microsoft, Inc. + * + * Authors: + * Jesse Benson + */ + +#include "catapult-drv.h" +#include "catapult-shell.h" + +/* structures and callback functions for formatting an attribute */ + +struct catapult_attribute_handler { + struct device_attribute attr; + int (*get_value)(struct catapult_device *idev, + struct catapult_attribute_handler *handler, + void *value_buffer); + const char *format_string; +}; + +static ssize_t catapult_show_attribute_uint32(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct catapult_device *idev = to_catapult_dev(dev); + struct catapult_attribute_handler *handler = NULL; + uint32_t data = 0; + int err = 0; + + handler = container_of(attr, struct catapult_attribute_handler, attr); + err = handler->get_value(idev, handler, &data); + if (err) + return err; + + return sprintf(buf, handler->format_string, data); +} + +static ssize_t catapult_show_attribute_uint64(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct catapult_device *idev = to_catapult_dev(dev); + struct catapult_attribute_handler *handler = NULL; + uint64_t data = 0; + int err = 0; + + handler = container_of(attr, struct catapult_attribute_handler, attr); + err = handler->get_value(idev, handler, &data); + if (err) + return err; + + return sprintf(buf, handler->format_string, data); +} + +static ssize_t catapult_show_attribute_string(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct catapult_device *idev = to_catapult_dev(dev); + struct catapult_attribute_handler *handler = NULL; + char *data = NULL; + int err = 0; + + handler = container_of(attr, struct catapult_attribute_handler, attr); + err = handler->get_value(idev, handler, &data); + if (err) + return err; + + return sprintf(buf, handler->format_string, data); +} + +/* + * Structures and handlers for converting fields in the device extension + * into read-only attributes. + */ + +struct catapult_attribute_field_handler { + struct catapult_attribute_handler base; + size_t field_offset; +}; + +static int catapult_get_field_uint32(struct catapult_device *idev, + struct catapult_attribute_handler *handler, + void *buffer) +{ + struct catapult_attribute_field_handler *h = + (struct catapult_attribute_field_handler *)handler; + uint32_t *value = (uint32_t *)buffer; + uint32_t *data = (uint32_t *)(((uintptr_t)idev) + h->field_offset); + *value = *data; + return 0; +} + +static int catapult_get_field_uint64(struct catapult_device *idev, + struct catapult_attribute_handler *handler, + void *buffer) +{ + struct catapult_attribute_field_handler *h = + (struct catapult_attribute_field_handler *)handler; + uint64_t *value = (uint64_t *)buffer; + uint64_t *data = (uint64_t *)(((uintptr_t)idev) + h->field_offset); + *value = *data; + return 0; +} + +static int catapult_get_field_string(struct catapult_device *idev, + struct catapult_attribute_handler *handler, + void *buffer) +{ + struct catapult_attribute_field_handler* h = + (struct catapult_attribute_field_handler *)handler; + char **value = (char **)buffer; + char **data = (char **)(((uintptr_t)idev) + h->field_offset); + *value = *data; + return 0; +} + +/* + * Structures and callbacks for attributes that read (or write) to + * shell registers directly. + */ + +struct catapult_attribute_register_handler { + struct catapult_attribute_handler base; + int interp_address; + int app_address; + uint32_t mask; + int right_shift; +}; + +static int +catapult_get_attribute_register(struct catapult_device *idev, + struct catapult_attribute_handler *handler, + void *buffer) +{ + struct catapult_attribute_register_handler *h = + (struct catapult_attribute_register_handler *)handler; + uint32_t *value = (uint32_t *)buffer; + uint32_t data = 0; + + data = catapult_low_level_read(idev->registers, + h->interp_address, + h->app_address); + + if (h->mask != 0) + data &= h->mask; + + data >>= h->right_shift; + + *value = data; + return 0; +} + +#define DECLARE_CATATTR(_name, _attr_type) static struct catapult_attribute_##_attr_type##_handler _name##_attr_handler + +#define CATDEV_ATTR_RO(_name, _type, _format, _get) \ +{ \ + .attr = __ATTR(_name, S_IRUGO, catapult_show_attribute_##_type, NULL), \ + .format_string = _format, \ + .get_value = _get, \ +} + +#define CATDEV_ATTR_FIELD_RO(_name, _type, _format, _field_name) \ + DECLARE_CATATTR(_name, field) = \ + { \ + .base = CATDEV_ATTR_RO(_name, _type, _format, catapult_get_field_##_type ), \ + .field_offset = offsetof(struct catapult_device, _field_name), \ + } + +#define CATDEV_ATTR_REGISTER_RO(_name, _format, _interp_addr, _app_addr, _mask, _shift) \ + DECLARE_CATATTR(_name, register) = \ + { \ + .base = CATDEV_ATTR_RO(_name, uint32, _format, catapult_get_attribute_register ), \ + .interp_address = _interp_addr, \ + .app_address = _app_addr, \ + .mask = _mask, \ + .right_shift = _shift, \ + } + +/* Bespoke attribute handler functions and attributes */ + +CATDEV_ATTR_FIELD_RO(chip_id, uint64, "%lld\n", chip_id ); +CATDEV_ATTR_FIELD_RO(endpoint_number, uint32, "%d\n", endpoint_number ); +CATDEV_ATTR_FIELD_RO(function_number, uint32, "%d\n", function_number ); +CATDEV_ATTR_FIELD_RO(function_type, string, "%s\n", function_type_name ); + +CATDEV_ATTR_REGISTER_RO(board_id, "%#08x\n", INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_BOARD_ID, 0, 0); +CATDEV_ATTR_REGISTER_RO(board_revision, "%#08x\n", INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_BOARD_REVISION, 0, 0); +CATDEV_ATTR_REGISTER_RO(shell_version, "%#08x\n", INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_SHELL_RELEASE_VERSION, 0, 0); +CATDEV_ATTR_REGISTER_RO(shell_id, "%#08x\n", INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_SHELL_ID, 0, 0); +CATDEV_ATTR_REGISTER_RO(role_version, "%#08x\n", INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_ROLE_VERSION, 0, 0); +CATDEV_ATTR_REGISTER_RO(role_id, "%#08x\n", INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_ROLE_ID, 0, 0); + +CATDEV_ATTR_REGISTER_RO(temperature, "%d C\n", INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_TEMPERATURE, 0x0000ff00, 8); + +#define INCLUDE_ATTRIBUTE(_name) &_name##_attr_handler.base.attr.attr + +static struct attribute *device_attrs[] = { + INCLUDE_ATTRIBUTE(shell_version), + INCLUDE_ATTRIBUTE(shell_id), + INCLUDE_ATTRIBUTE(role_version), + INCLUDE_ATTRIBUTE(role_id), + INCLUDE_ATTRIBUTE(board_id), + INCLUDE_ATTRIBUTE(board_revision), + INCLUDE_ATTRIBUTE(chip_id), + INCLUDE_ATTRIBUTE(endpoint_number), + INCLUDE_ATTRIBUTE(function_number), + INCLUDE_ATTRIBUTE(function_type), + INCLUDE_ATTRIBUTE(temperature), + NULL, +}; + +const struct attribute_group device_group = { + .attrs = device_attrs, +}; diff --git a/drivers/catapult/catapult-device.c b/drivers/catapult/catapult-device.c new file mode 100644 index 000000000000..c03517ad8854 --- /dev/null +++ b/drivers/catapult/catapult-device.c @@ -0,0 +1,408 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * device.c - device management routines + * + * Copyright (C) 2019 Microsoft, Inc. + * + * Authors: + * Jesse Benson + */ + +#include + +#include "catapult-device.h" +#include "catapult-drv.h" +#include "catapult-shell.h" + +/* Function type GUID to enum mapping table */ +struct catapult_function_type { + const guid_t *function_type_guid; + enum fpga_function_type function_type_enum; +}; + +static const struct catapult_function_type function_type_table[] = { + { &CATAPULT_GUID_LEGACY_FUNCTION, FPGA_FUNCTION_TYPE_LEGACY }, + { &CATAPULT_GUID_ROLE_FUNCTION, FPGA_FUNCTION_TYPE_ROLE }, + { &CATAPULT_GUID_MANAGEMENT_FUNCTION, FPGA_FUNCTION_TYPE_MANAGEMENT }, +}; + +static int catapult_read_dfh_register(struct catapult_device *idev, + uint32_t offset, + uint64_t *value) +{ + const uintptr_t base = (uintptr_t) idev->registers; + const size_t bar_length = idev->registers_cb; + + if (value == NULL) + return -EINVAL; + if (offset >= bar_length) + return -EINVAL; + if (offset % sizeof(uint64_t) != 0) + return -EINVAL; + + *value = catapult_register_read64((uint64_t *)(base + offset)); + return 0; +} + +static int catapult_write_dfh_register(struct catapult_device *idev, + uint32_t offset, + uint64_t value) +{ + const uintptr_t base = (uintptr_t) idev->registers; + const size_t bar_length = idev->registers_cb; + + if (offset >= bar_length) + return -EINVAL; + if (offset % sizeof(uint64_t) != 0) + return -EINVAL; + + catapult_register_write64((uint64_t *)(base + offset), value); + return 0; +} + +/** + * Cycle through the list of DFH (Device Feature Headers) to locate the feature + * specified in the function parameters. Returns offset from the BAR base + * address where the feature header can be found. + * + * @idev: A handle to the driver device file. + * @feature_guid: The feature GUID to find in the DFH. + */ +static uint32_t catapult_get_dfh_offset(struct catapult_device *idev, + const guid_t *feature_guid) +{ + union catapult_dfh_header dfh_header = { 0 }; + uint32_t offset = 0; + guid_t read_guid = { 0 }; + + /* + * Check to see if this image supports the DFH. If reading this + * register doesn't have 0x04 for it's afu_type, it doesn't support + * the DFH. + */ + if (idev->avoid_hip1_access == false) + catapult_read_dfh_register(idev, offset, + &dfh_header.as_ulonglong); + + while (dfh_header.afu_type > DFH_TYPE_NOT_SUPPORTED && + dfh_header.afu_type < DFH_TYPE_MAX && !dfh_header.afu_eol) { + /* Get the first feature header */ + offset += (uint32_t) dfh_header.afu_offset; + + catapult_read_dfh_register(idev, + offset, + &dfh_header.as_ulonglong); + catapult_read_dfh_register(idev, + offset + DFH_FEATURE_GUID_OFFSET_LOWER, + (uint64_t *) &(read_guid.b[0])); + catapult_read_dfh_register(idev, + offset + DFH_FEATURE_GUID_OFFSET_HIGHER, + (uint64_t *) &(read_guid.b[8])); + + /* Check to see if this is the feature we're interested in */ + if (guid_equal(&read_guid, feature_guid)) + return offset; + } + + return 0; +} + +/** + * Read the function type GUID from the DFH (Device Function Headers). + * + * @idev: A handle to the driver device file. + */ +int catapult_read_function_type(struct catapult_device *idev) +{ + union catapult_dfh_header dfh_header = { 0 }; + guid_t function_type_guid = { 0 }; + uint32_t i = 0; + int function_type_known = false; + + idev->function_type = FPGA_FUNCTION_TYPE_UNKNOWN; + + /* + * Check to see if this image supports the DFH. If reading this register + * doesn't have, 0x04 for it's afu_type, it doesn't support the DFH. + */ + if (idev->avoid_hip1_access == false) { + catapult_read_dfh_register(idev, 0, &dfh_header.as_ulonglong); + dev_info(idev->dev, "%s: reading dfh register returned %#llx\n", + __func__, dfh_header.as_ulonglong); + } + + if (dfh_header.afu_type > DFH_TYPE_NOT_SUPPORTED && + dfh_header.afu_type < DFH_TYPE_MAX) { + uint64_t tmp[2] = { 0 }; + + dev_info(idev->dev, "%s: dfh header type %x\n", + __func__, dfh_header.afu_type); + + idev->dfh_supported = true; + idev->function_type = FPGA_FUNCTION_TYPE_LEGACY; + + /* Let's query the function type from the DFH */ + catapult_read_dfh_register(idev, DFH_FEATURE_GUID_OFFSET_LOWER, + &tmp[0]); + catapult_read_dfh_register(idev, DFH_FEATURE_GUID_OFFSET_HIGHER, + &tmp[1]); + + dev_info(idev->dev, "%s: dfh function type guid %llx%016llx\n", + __func__, tmp[0], tmp[1]); + + memcpy(&function_type_guid, tmp, sizeof(guid_t)); + + for (i = 0; i < FPGA_FUNCTION_TYPE_MAX; i++) { + if (guid_equal(function_type_table[i].function_type_guid, &function_type_guid)) { + uint64_t *gtmp = (uint64_t*)function_type_table[i].function_type_guid; + dev_info(idev->dev, + "%s: dfh function type guid matches type %d (%016llx%016llx)\n", + __func__, + i, + gtmp[0], + gtmp[1]); + idev->function_type = function_type_table[i].function_type_enum; + break; + } + } + } else { + dev_info(idev->dev, + "%s: not a DFH function - function_type is legacy\n", + __func__); + idev->function_type = FPGA_FUNCTION_TYPE_LEGACY; + idev->dfh_supported = false; + } + + switch (idev->function_type) { + case FPGA_FUNCTION_TYPE_LEGACY: + idev->function_type_name = "legacy"; + function_type_known = true; + break; + + case FPGA_FUNCTION_TYPE_ROLE: + idev->function_type_name = "role"; + function_type_known = true; + break; + + case FPGA_FUNCTION_TYPE_MANAGEMENT: + idev->function_type_name = "management"; + function_type_known = true; + break; + + default: + idev->function_type_name = "unknown"; + break; + } + + if (function_type_known) { + dev_info(idev->dev, "%s: function_type_name set to %s\n", + __func__, idev->function_type_name); + } else { + dev_err(idev->dev, + "%s: function_type %d is unknown. Setting function_type_name to %s\n", + __func__, + idev->function_type, + idev->function_type_name); + } + + return 0; +} + +/** + * Ensure interrupts are enabled for the Catapult Role function. + * + * @idev: A handle to the driver device file. + */ +int catapult_enable_role_function(struct catapult_device *idev) +{ + uint32_t shell_ctrl_offset = 0; + union catapult_dma_control_register dma_ctrl_reg = { 0 }; + union catapult_role_control_register role_ctrl_reg = { 0 }; + + dev_info(idev->dev, "%s: switching to role function (if supported)\n", + __func__); + + if (!idev->dfh_supported) { + dev_info(idev->dev, + "%s: device does not support DFH - no action\n", + __func__); + return 0; + } + + /* Get the interrupt feature header offset */ + idev->interrupt_feature_offset = + catapult_get_dfh_offset(idev, &GUID_FPGA_INTERRUPT_FEATURE); + dev_info(idev->dev, "%s: interrupt_feature_offset = %#llx\n", + __func__, (uint64_t) idev->interrupt_feature_offset); + + /* Get the shell control feature header offset */ + shell_ctrl_offset = + catapult_get_dfh_offset(idev, &GUID_FPGA_SHELL_CONTROL_FEATURE); + if (shell_ctrl_offset == 0) { + /* This doesn't support the shell control feature */ + dev_info(idev->dev, "%s: shell control feature not supported\n", + __func__); + return 0; + } + + if (idev->function_type != FPGA_FUNCTION_TYPE_MANAGEMENT) { + dev_info(idev->dev, + "%s: function is type role or legacy, so cannot switch control\n", + __func__); + return 0; + } + + /* + * This is a management function. We can assume there will be a role + * function and we want to enable the role function. + */ + dev_info(idev->dev, + "%s: found management function - switching control to role\n", + __func__); + + /* + * Now let's assign the DMA engine to the Role function. + * The dma function select bit is a toggle. We must first + * check the previous value to see if we should set it. + */ + catapult_read_dfh_register(idev, + shell_ctrl_offset + DFH_FEATURE_DMA_CONTROL_REG_OFFSET, + &dma_ctrl_reg.as_ulonglong); + + if (dma_ctrl_reg.dma_function_select != DMA_FUNCTION_ROLE) { + dma_ctrl_reg.dma_function_select = DMA_FUNCTION_ROLE; + catapult_write_dfh_register(idev, + shell_ctrl_offset + DFH_FEATURE_DMA_CONTROL_REG_OFFSET, + dma_ctrl_reg.as_ulonglong); + } else { + dev_info(idev->dev, "%s: role was already selected\n", + __func__); + } + + /* + * Set the isolate role bit last. The role isolation bit is + * only settable and cannot be unset. + * + * We want to write back what's currently in the role_interrupt + * mask. If the mask is set to 1, that means that the role + * cannot generate interrupts and we want to flip the bit by + * writing to it. If it's set to 0, we want to keep it the same + * value since the role is generating interrupts. + */ + catapult_read_dfh_register(idev, + shell_ctrl_offset + DFH_FEATURE_ROLE_CONTROL_REG_OFFSET, + &role_ctrl_reg.as_ulonglong); + role_ctrl_reg.isolate_role = ROLE_ISOLATED; + catapult_write_dfh_register(idev, + shell_ctrl_offset + DFH_FEATURE_ROLE_CONTROL_REG_OFFSET, + role_ctrl_reg.as_ulonglong); + + /* + * We want to do a sanity check on the registers to ensure they + * are in the proper state. + */ + catapult_read_dfh_register(idev, + shell_ctrl_offset + DFH_FEATURE_ROLE_CONTROL_REG_OFFSET, + &role_ctrl_reg.as_ulonglong); + catapult_read_dfh_register(idev, + shell_ctrl_offset + DFH_FEATURE_DMA_CONTROL_REG_OFFSET, + &dma_ctrl_reg.as_ulonglong); + + if ((role_ctrl_reg.isolate_role != ROLE_ISOLATED) || + (role_ctrl_reg.role_interrupt_mask != ROLE_INTERRUPT_ENABLED) || + (dma_ctrl_reg.dma_function_select != DMA_FUNCTION_ROLE)) { + dev_err(idev->dev, + "%s: failed to isolate role or enable interrupt (%#x %#x %#x): %d\n", + __func__, + role_ctrl_reg.isolate_role, + role_ctrl_reg.role_interrupt_mask, + dma_ctrl_reg.dma_function_select, + -EPERM); + + return -EPERM; + } + + dev_info(idev->dev, "%s: control switched to role function\n", + __func__); + + return 0; +} + +/** + * Handles the Catapult DMA interrupt by signalling completion + * to the user-mode code. + * + * @irq: The interrupt request number. + * @dev_id: A handle to the driver device file. + */ +irqreturn_t catapult_interrupt_handler(int irq, void *dev_id) +{ + struct catapult_device *idev = dev_id; + uintptr_t bar0_registers = 0; + uintptr_t offset = 0; + uint32_t i = 0; + uint32_t read_val = 0; + union catapult_interrupt_status_register int_status_reg = { 0 }; + struct completion *event_obj = NULL; + + if (idev == NULL) + return IRQ_NONE; + + dev_dbg(idev->dev, "%s: enter\n", __func__); + + /* + * Is interrupt signaling enabled? If so, then signal the event and give + * the waiting thread a big priority boost so it can quickly respond to + * the interrupt. + * + * If the shell supports it, read the Interrupt Feature's Interrupt + * Status register to determine the type of interrupt that fired. + */ + if (idev->interrupt_feature_offset != 0) + catapult_read_dfh_register(idev, idev->interrupt_feature_offset + DFH_FEATURE_INTERRUPT_STATUS_REG_OFFSET, &int_status_reg.as_ulonglong); + + /* + * If this is a legacy shell (no Interrupt Feature in the DFH) or the + * Interrupt Status indicated a Slot DMA interrupt, handle it here. + */ + if (idev->interrupt_feature_offset == 0 || int_status_reg.slot_dma_interrupt) { + bar0_registers = (uintptr_t) idev->registers; + if (bar0_registers != 0) { + offset = catapult_register_offset(INTER_ADDR_INTERRUPT, 256); + read_val = catapult_register_read32((uint32_t *)(bar0_registers + offset)); + + if (read_val == 0xffffffff) { + dev_err(idev->dev, + "%s: interrupt status register is reading 0xffffffff - dropping interrupt\n", + __func__); + } else { + /* Look at bottom 2 bits to determine how many buffers the interrupt is for, can be 0 to 3 inclusive */ + uint32_t num_buffers = read_val & 3; + + for (i = 1; i <= num_buffers; i++) { + uint32_t which_buffer = (read_val >> (8 * i)) & 0xff; + + if (which_buffer >= idev->number_of_slots) { + dev_err(idev->dev, + "%s: interrupt reporting completion on invalid slot# (%d) - dropping interrupt\n", + __func__, + which_buffer); + continue; + } + + event_obj = &(idev->event_obj[which_buffer]); + + /* Verbose logging - this has significant effect on performance and disk usage */ + dev_dbg(idev->dev, + "%s: interrupt slot %d (%p) - signalling interrupt\n", + __func__, + which_buffer, + event_obj); + complete(event_obj); + } + } + } + } + + return IRQ_HANDLED; +} diff --git a/drivers/catapult/catapult-device.h b/drivers/catapult/catapult-device.h new file mode 100644 index 000000000000..090a30732b18 --- /dev/null +++ b/drivers/catapult/catapult-device.h @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * catapult-device.h - device management routines + * + * Copyright (C) 2019 Microsoft, Inc. + * + * Authors: + * Jesse Benson + */ + +#ifndef __CATAPULT_DEVICE_H +#define __CATAPULT_DEVICE_H + +#include + +struct catapult_device; + +irqreturn_t catapult_interrupt_handler(int irq, void *dev_id); + +int catapult_read_function_type(struct catapult_device *idev); +int catapult_enable_role_function(struct catapult_device *idev); + +#endif /* __CATAPULT_DEVICE_H */ diff --git a/drivers/catapult/catapult-drv.c b/drivers/catapult/catapult-drv.c new file mode 100644 index 000000000000..0d86119cd0dd --- /dev/null +++ b/drivers/catapult/catapult-drv.c @@ -0,0 +1,860 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * catapult-drv.c - catapult driver for PCI 2.3 devices + * + * Copyright (C) 2019 Microsoft, Inc. + * + * Authors: + * Jesse Benson + */ + +#include +#include + +#include "catapult-device.h" +#include "catapult-drv.h" +#include "catapult-ioctl.h" +#include "catapult-shell.h" + +static dev_t catapult_dev = { 0 }; +static int catapult_major = 0; +static struct cdev *catapult_cdev = NULL; +static struct class *catapult_class = NULL; + +/* Catapult module parameters */ +static uint32_t dma_slot_count = SLOT_COUNT; +static uint32_t dma_slot_bytes = BYTES_PER_SLOT; + +DEFINE_IDR(catapult_idr); +DEFINE_MUTEX(minor_lock); + +extern const struct attribute_group device_group; +static const struct attribute_group *device_groups[] = { + &device_group, + NULL, +}; + +/* Convert a device pointer to a catapult device pointer */ +struct catapult_device *to_catapult_dev(struct device *dev) +{ + return (struct catapult_device *) dev_get_drvdata(dev); +} + +static char *catapult_devnode(struct device *dev, umode_t *mode) +{ + if (mode) + *mode = 0666; + + return NULL; +} + +/* Setup the PCI interrupt request handler for the catapult device */ +static int catapult_request_irq(struct catapult_device *idev) +{ + int err = 0; + int irq = 0; + + dev_info(idev->dev, "%s: requesting IRQ for device\n", __func__); + + err = pci_alloc_irq_vectors(idev->pdev, 1, 1, PCI_IRQ_MSI); + if (err < 0) { + dev_err(idev->dev, "%s: error requesting irq vectors: %d\n", + __func__, err); + return err; + } else if (err == 0) { + dev_err(idev->dev, "%s: failed to allocate irq vectors\n", + __func__); + return -ENODEV; + } + + irq = pci_irq_vector(idev->pdev, 0); + + err = request_threaded_irq(irq, NULL, catapult_interrupt_handler, + IRQF_ONESHOT, "catapult", idev); + if (err == 0) { + dev_info(idev->dev, "%s: registered irq line - %d\n", + __func__, irq); + idev->irq = irq; + } else { + dev_err(idev->dev, "%s: error requesting threaded irq: %d\n", + __func__, err); + } + + return err; +} + +static int catapult_slot_map_init(struct catapult_device *idev) +{ + int size = BITS_TO_LONGS(idev->number_of_slots) * sizeof(unsigned long); + unsigned long *bitmap = NULL; + pid_t *pid_map = NULL; + + bitmap = kmalloc(size, GFP_KERNEL); + if (!bitmap) + return -ENOMEM; + + idev->slot_map = bitmap; + bitmap_zero(idev->slot_map, idev->number_of_slots); + + /* Process id map where the pids which acquire the slot are held */ + pid_map = kzalloc(sizeof(pid_t) * idev->number_of_slots, GFP_KERNEL); + if (!pid_map) + return -ENOMEM; + + idev->slot_map_pids = pid_map; + + /* Single mutex lock for concurrent access of the bitmap */ + mutex_init(&idev->lock); + + return 0; +} + +static void catapult_slot_map_remove(struct catapult_device *idev) +{ + mutex_destroy(&idev->lock); + + if (idev->slot_map_pids) { + kfree(idev->slot_map_pids); + idev->slot_map_pids = NULL; + } + + if (idev->slot_map) { + kfree(idev->slot_map); + idev->slot_map = NULL; + } +} + +static void catapult_slot_map_release(struct catapult_device *idev, pid_t pid) +{ + uint32_t slot_count = idev->number_of_slots; + int slot = 0; + + if (idev->slot_map == NULL) { + WARN_ON(idev->slot_map == NULL); + return; + } + + mutex_lock(&idev->lock); + while (true) { + slot = find_next_bit(idev->slot_map, slot_count, slot); + if (slot < 0 || slot >= slot_count) + break; + + if (idev->slot_map_pids[slot] == pid) { + dev_err(idev->dev, + "%s: process id %d did not release slot %d before close. Force releasing the slot\n", + __func__, pid, slot); + clear_bit(slot, idev->slot_map); + } else { + slot++; + } + } + mutex_unlock(&idev->lock); +} + +static void catapult_dma_remove(struct catapult_device *idev) +{ + uint32_t i = 0; + + for (i = 0; i < idev->number_of_slots; i++) { + if (idev->dma_input_kernel_addr[i]) { + dma_free_coherent(&idev->pdev->dev, + idev->bytes_per_slot, + idev->dma_input_kernel_addr[i], + idev->dma_input_dma_addr[i]); + idev->dma_input_kernel_addr[i] = NULL; + } + if (idev->dma_output_kernel_addr[i]) { + dma_free_coherent(&idev->pdev->dev, + idev->bytes_per_slot, + idev->dma_output_kernel_addr[i], + idev->dma_output_dma_addr[i]); + idev->dma_output_kernel_addr[i] = NULL; + } + } + + if (idev->dma_control_kernel_addr) { + dma_free_coherent(&idev->pdev->dev, + idev->dma_control_len, + idev->dma_control_kernel_addr, + idev->dma_control_dma_addr); + idev->dma_control_kernel_addr = NULL; + } + if (idev->dma_result_kernel_addr) { + dma_free_coherent(&idev->pdev->dev, + idev->dma_result_len, + idev->dma_result_kernel_addr, + idev->dma_result_dma_addr); + idev->dma_result_kernel_addr = NULL; + } + + catapult_slot_map_remove(idev); +} + +static int catapult_dma_init(struct catapult_device *idev) +{ + int err = 0; + uint32_t i = 0; + uintptr_t registers = (uintptr_t) idev->registers; + uint32_t read_val = 0; + + idev->number_of_slots = dma_slot_count; + idev->bytes_per_slot = dma_slot_bytes; + + idev->dma_input_len = idev->number_of_slots * idev->bytes_per_slot; + idev->dma_output_len = idev->number_of_slots * idev->bytes_per_slot; + idev->dma_control_len = idev->number_of_slots * FPGA_CONTROL_SIZE; + idev->dma_result_len = idev->number_of_slots * FPGA_RESULT_SIZE; + + for (i = 0; i < idev->number_of_slots; i++) { + init_completion(&(idev->event_obj[i])); + } + + for (i = 0; i < idev->number_of_slots; i++) { + idev->dma_input_kernel_addr[i] = + dma_alloc_coherent(&idev->pdev->dev, + idev->bytes_per_slot, + &idev->dma_input_dma_addr[i], + GFP_KERNEL); + if (idev->dma_input_kernel_addr[i] == NULL) { + err = -EFAULT; + goto exit; + } + + idev->dma_output_kernel_addr[i] = + dma_alloc_coherent(&idev->pdev->dev, + idev->bytes_per_slot, + &idev->dma_output_dma_addr[i], + GFP_KERNEL); + if (idev->dma_output_kernel_addr[i] == NULL) { + err = -EFAULT; + goto exit; + } + } + + idev->dma_control_kernel_addr = + dma_alloc_coherent(&idev->pdev->dev, + idev->dma_control_len, + &idev->dma_control_dma_addr, + GFP_KERNEL); + if (idev->dma_control_kernel_addr == NULL) { + err = -EFAULT; + goto exit; + } + + idev->dma_result_kernel_addr = + dma_alloc_coherent(&idev->pdev->dev, + idev->dma_result_len, + &idev->dma_result_dma_addr, + GFP_KERNEL); + if (idev->dma_result_kernel_addr == NULL) { + err = -EFAULT; + goto exit; + } + + err = catapult_slot_map_init(idev); + if (err != 0) { + dev_err(&idev->pdev->dev, + "%s: error initializing the slot map - %d\n", + __func__, err); + goto exit; + } + + /* Write slot-specific buffer addresses to FPGA registers */ + for (i = 0; i < idev->number_of_slots; i++) { + catapult_register_write64((uint64_t *)(registers + DMA_SLOT_INPUT_BASE_ADDRESS + i * 0x20), idev->dma_input_dma_addr[i]); + catapult_register_write64((uint64_t *)(registers + DMA_SLOT_OUTPUT_BASE_ADDRESS + i * 0x20), idev->dma_output_dma_addr[i]); + catapult_register_write64((uint64_t *)(registers + DMA_SLOT_CONTROL_RESULT_BASE_ADDRESS + i * 0x20), idev->dma_result_dma_addr + i * FPGA_RESULT_SIZE); + } + + /* Flush any remaining unserviced interrupt from last time */ + do { + read_val = catapult_low_level_read(idev->registers, + INTER_ADDR_INTERRUPT, 256); + } while (read_val & 3); + + /* Set max payload size for FPGA TX engine back to default 128 bytes */ + catapult_low_level_write(idev->registers, + INTER_ADDR_HACK_OVERRIDE_OUT_DATA_SIZE, 2, 0); + + /* Set the number of interrupts to coalesce */ + catapult_low_level_write(idev->registers, + INTER_ADDR_INTERRUPT, 257, 1); + +exit: + if (err != 0) + catapult_dma_remove(idev); + + return err; +} + +/* Enable the PCI device for the corresponding catapult device */ +static int catapult_enable_pci(struct catapult_device *idev) +{ + int err = 0; + + dev_info(idev->dev, "%s: entry\n", __func__); + + err = pcim_enable_device(idev->pdev); + if (err) { + dev_err(idev->dev, "%s: pci_enable_device failed: %d\n", + __func__, err); + return err; + } + + if (idev->pdev->irq && !pci_intx_mask_supported(idev->pdev)) { + err = -ENODEV; + dev_err(&idev->pdev->dev, + "%s: device does not support INTX mask: %d\n", + __func__, err); + return err; + } + + err = catapult_request_irq(idev); + if (err != 0) { + dev_err(&idev->pdev->dev, + "%s: error requesting interrupt handler - %d\n", + __func__, err); + return err; + } + + err = pcim_iomap_regions(idev->pdev, 0x1, "catapult"); + if (err != 0) { + dev_err(&idev->pdev->dev, + "%s: error requesting BAR 0 region - %d\n", + __func__, err); + return err; + } + + idev->registers_cb = pci_resource_len(idev->pdev, 0); + idev->registers_physical_address = pci_resource_start(idev->pdev, 0); + idev->registers = pcim_iomap_table(idev->pdev)[0]; + + err = catapult_dma_init(idev); + if (err != 0) { + dev_err(&idev->pdev->dev, + "%s: error initializing DMA state - %d\n", + __func__, err); + return err; + } + + dev_info(&idev->pdev->dev, "%s: exit\n", __func__); + return 0; +} + +static void catapult_get_endpoint_info(struct catapult_device *idev) +{ + union catapult_shell_identity_register shell_id = { 0 }; + + idev->chip_id = catapult_low_level_read(idev->registers, INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_CHIP_ID_HIGH); + idev->chip_id <<= 32; + idev->chip_id |= catapult_low_level_read(idev->registers, INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_CHIP_ID_LOW); + + idev->board_id = catapult_low_level_read(idev->registers, INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_BOARD_ID); + idev->board_revision = catapult_low_level_read(idev->registers, INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_BOARD_REVISION); + idev->shell_version = catapult_low_level_read(idev->registers, INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_SHELL_RELEASE_VERSION); + idev->shell_id = catapult_low_level_read(idev->registers, INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_SHELL_ID); + idev->role_version = catapult_low_level_read(idev->registers, INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_ROLE_VERSION); + idev->role_id = catapult_low_level_read(idev->registers, INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_ROLE_ID); + + shell_id.as_ulong = catapult_low_level_read(idev->registers, INTER_ADDR_GENERAL_PURPOSE_REG, GP_REGISTER_INDEX_SHELL_IDENTITY); + + idev->endpoint_number = shell_id.endpoint_number; + idev->function_number = (unsigned short) (idev->pdev->devfn & 0xffff); + + switch (idev->pdev->device) { + case CATAPULT_PCI_DEVICE_ID_LP_HIP1_MANAGEMENT: + case CATAPULT_PCI_DEVICE_ID_LP_HIP2_MANAGEMENT: + idev->function_type_name = "management"; + break; + + case CATAPULT_PCI_DEVICE_ID_LP_HIP1_ROLE: + case CATAPULT_PCI_DEVICE_ID_LP_HIP2_ROLE: + idev->function_type_name = "role"; + break; + + default: + idev->function_type_name = "unknown"; + break; + } + + dev_info(&idev->pdev->dev, + "%s: chip_id = %llu, board_id = %d, board_rev = %d, fn = %d\n", + __func__, idev->chip_id, idev->board_id, + idev->board_revision, idev->function_number); + + snprintf(idev->name, sizeof(idev->name), "%llu:%d:%d", + idev->chip_id, idev->endpoint_number, idev->function_number); +} + +static int catapult_get_minor(struct catapult_device *idev) +{ + int retval = -ENOMEM; + + mutex_lock(&minor_lock); + retval = idr_alloc(&catapult_idr, idev, 0, + CATAPULT_MAX_DEVICES, GFP_KERNEL); + if (retval >= 0) { + idev->minor = retval; + retval = 0; + } else if (retval == -ENOSPC) { + dev_err(idev->dev, "too many catapult devices\n"); + retval = -EINVAL; + } + mutex_unlock(&minor_lock); + return retval; +} + +static void catapult_free_minor(struct catapult_device *idev) +{ + mutex_lock(&minor_lock); + idr_remove(&catapult_idr, idev->minor); + mutex_unlock(&minor_lock); +} + +static void catapult_release_device(void *context) +{ + struct catapult_device *idev = context; + + if (idev->irq) + free_irq(idev->irq, idev); + pci_free_irq_vectors(idev->pdev); + catapult_free_minor(idev); + kvfree(idev); +} + +static int catapult_create_device(struct device *parent, + struct catapult_device **result) +{ + struct catapult_device *idev = NULL; + struct device *dev = NULL; + int err = 0; + + *result = NULL; + + idev = kzalloc(sizeof(*idev), GFP_KERNEL); + if (!idev) { + err = -ENOMEM; + dev_err(parent, "%s: error allocating catapult_device - %d\n", + __func__, err); + return err; + } + + err = catapult_get_minor(idev); + if (err != 0) + goto exit1; + + /* + * initialize the device. After this succeeds, all cleanup should + * be attached to the device as an action + */ + dev = device_create_with_groups(catapult_class, + parent, + MKDEV(MAJOR(catapult_dev), idev->minor), + idev, + device_groups, + "catapult%d", + idev->minor); + if (dev == NULL) { + err = -ENOMEM; + dev_err(parent, "%s: error registering chrdev - %d\n", + __func__, err); + goto exit2; + } + + dev_info(parent, "%s: dev = %p devinfo = %p (kobj = %p)\n", + __func__, dev, dev_get_drvdata(dev), &(dev->kobj)); + + /* add a cleanup action to the device to free the containing device */ + err = devm_add_action(dev, catapult_release_device, idev); + if (err != 0) { + dev_err(parent, + "%s: error adding release action to device = %d\n", + __func__, err); + goto exit3; + } + + idev->dev = dev; + *result = idev; + return 0; + +exit3: + device_destroy(catapult_class, MKDEV(MAJOR(catapult_dev), idev->minor)); + +exit2: + catapult_free_minor(idev); + +exit1: + kvfree(idev); + return err; +} + +/* + * Probe indicates that a PCI device with the matching device ID has been + * discovered. Create the catapult device, then enable the PCI interface + * examine the function and create the appropriate character device + */ +static int catapult_probe(struct pci_dev *pdev, const struct pci_device_id *id) +{ + struct catapult_device *idev = NULL; + int err = 0; + + dev_info(&pdev->dev, "%s: entry\n", __func__); + + /* + * Create the idev for the device. this allows tracking of other + * resources under devm. + */ + err = catapult_create_device(&pdev->dev, &idev); + if (err) { + dev_err(&pdev->dev, "%s: failing probe - %d\n", __func__, err); + return err; + } + + idev->pdev = pdev; + pci_set_drvdata(pdev, idev); + + err = catapult_enable_pci(idev); + if (err) { + dev_err(&pdev->dev, "%s: catapult_enable_pci failed: %d\n", + __func__, err); + goto error; + } + + /* Read the hardware information from the endpoint */ + catapult_get_endpoint_info(idev); + + err = catapult_read_function_type(idev); + if (err) { + dev_err(&pdev->dev, + "%s: catapult_read_function_type failed: %d\n", + __func__, err); + goto error; + } + + dev_info(&pdev->dev, "%s: catapult_read_function_type got type %x\n", + __func__, idev->function_type); + + err = catapult_enable_role_function(idev); + if (err) { + dev_err(&pdev->dev, + "%s: catapult_enable_role_function failed: %d\n", + __func__, err); + goto error; + } + + return 0; + +error: + device_destroy(catapult_class, MKDEV(MAJOR(catapult_dev), idev->minor)); + return err; +} + +static void catapult_remove(struct pci_dev *pdev) +{ + dev_t dev; + struct catapult_device *idev = pci_get_drvdata(pdev); + + if (idev != NULL) { + catapult_dma_remove(idev); + dev = MKDEV(MAJOR(catapult_dev), idev->minor); + device_destroy(catapult_class, dev); + } +} + +static int catapult_open(struct inode *inode, struct file *filep) +{ + struct catapult_device *idev = NULL; + struct catapult_file *ifile = NULL; + int err = 0; + + pr_info("%s: inode = %p, filep = %p\n", __func__, inode, filep); + pr_info(" device # = (%d,%d)\n", imajor(inode), iminor(inode)); + + mutex_lock(&minor_lock); + idev = idr_find(&catapult_idr, iminor(inode)); + mutex_unlock(&minor_lock); + + if (idev == NULL) + return -ENODEV; + + if (!try_module_get(THIS_MODULE)) + return -ENODEV; + + ifile = kzalloc(sizeof(*ifile), GFP_KERNEL); + if (ifile == NULL) { + err = -ENOMEM; + goto error_alloc_file; + } + + ifile->inode = inode; + ifile->file = filep; + ifile->idev = idev; + + filep->private_data = ifile; + + return 0; + +error_alloc_file: + module_put(THIS_MODULE); + + return err; +} + +static int catapult_release(struct inode *inode, struct file *filep) +{ + struct catapult_file *ifile = filep->private_data; + struct catapult_device *idev = NULL; + + if (ifile == NULL) { + pr_err("%s: ifile was null\n", __func__); + return 0; + } + + idev = ifile->idev; + catapult_slot_map_release(idev, task_tgid_nr(current)); + + filep->private_data = NULL; + + kfree(ifile); + + module_put(THIS_MODULE); + + return 0; +} + +static const struct vm_operations_struct catapult_vm_ops = { +#ifdef CONFIG_HAVE_IOREMAP_PROT + .access = generic_access_phys, +#endif +}; + +static int catapult_mmap_get_slot(struct catapult_device *idev, + unsigned long offset, + unsigned long size, + uint32_t *slot) +{ + int err = 0; + + *slot = offset / idev->bytes_per_slot; + + if (*slot >= idev->number_of_slots) + return -EINVAL; + if (size != idev->bytes_per_slot) + return -EINVAL; + + /* Verify the current process acquired the requested slot */ + err = mutex_lock_interruptible(&idev->lock); + if (err == 0) { + BUG_ON(idev->slot_map == NULL); + if (!test_bit(*slot, idev->slot_map) || + idev->slot_map_pids[*slot] != task_tgid_nr(current)) + err = -EACCES; + + mutex_unlock(&idev->lock); + } + + return err; +} + +static int catapult_mmap(struct file *filep, struct vm_area_struct *vma) +{ + struct catapult_file *ifile = filep->private_data; + struct catapult_device *idev = ifile->idev; + int err = 0; + unsigned long offset = vma->vm_pgoff << PAGE_SHIFT; + uint32_t slot = 0; + uint64_t physical_address = 0; + + dev_dbg(idev->dev, "%s: request to mmap offset %lu and size %lu\n", + __func__, offset, vma->vm_end - vma->vm_start); + + if (vma->vm_end < vma->vm_start) + return -EINVAL; + + if (offset == CATAPULT_FPGA_REGISTER_ADDRESS) { + /* memory map BAR registers as non-cached */ + physical_address = idev->registers_physical_address; + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); + } else if (offset == CATAPULT_FPGA_DMA_RESULT_ADDRESS) { + /* memory map the DMA result registers */ + physical_address = virt_to_phys(idev->dma_result_kernel_addr); + } else if (offset == CATAPULT_FPGA_DMA_CONTROL_ADDRESS) { + /* memory map the DMA control registers */ + physical_address = virt_to_phys(idev->dma_control_kernel_addr); + } else if ((offset & CATAPULT_FPGA_DMA_BASE_ADDRESS_MASK) == CATAPULT_FPGA_DMA_INPUT_BASE_ADDRESS) { + /* memory map an input DMA slot */ + if ((err = catapult_mmap_get_slot(idev, offset & ~CATAPULT_FPGA_DMA_BASE_ADDRESS_MASK, vma->vm_end - vma->vm_start, &slot)) != 0) + return err; + physical_address = virt_to_phys(idev->dma_input_kernel_addr[slot]); + } else if ((offset & CATAPULT_FPGA_DMA_BASE_ADDRESS_MASK) == CATAPULT_FPGA_DMA_OUTPUT_BASE_ADDRESS) { + /* memory map an output DMA slot */ + if ((err = catapult_mmap_get_slot(idev, offset & ~CATAPULT_FPGA_DMA_BASE_ADDRESS_MASK, vma->vm_end - vma->vm_start, &slot)) != 0) + return err; + physical_address = virt_to_phys(idev->dma_output_kernel_addr[slot]); + } else { + dev_err(idev->dev, "%s: invalid address offset - %lu\n", __func__, offset); + return -EINVAL; + } + + vma->vm_private_data = ifile; + vma->vm_ops = &catapult_vm_ops; + + err = remap_pfn_range(vma, + vma->vm_start, + physical_address >> PAGE_SHIFT, + vma->vm_end - vma->vm_start, + vma->vm_page_prot); + + if (err != 0) + dev_err(idev->dev, "%s: remap_pfn_range failed - %d\n", + __func__, err); + + return err; +} + +static const struct pci_device_id catapult_pci_id[] = { + { PCI_DEVICE(CATAPULT_PCI_VENDOR_ID, CATAPULT_PCI_DEVICE_ID_LP_HIP1_MANAGEMENT) }, + { PCI_DEVICE(CATAPULT_PCI_VENDOR_ID, CATAPULT_PCI_DEVICE_ID_LP_HIP2_MANAGEMENT) }, + { PCI_DEVICE(CATAPULT_PCI_VENDOR_ID, CATAPULT_PCI_DEVICE_ID_LP_HIP1_ROLE) }, + { PCI_DEVICE(CATAPULT_PCI_VENDOR_ID, CATAPULT_PCI_DEVICE_ID_LP_HIP2_ROLE) }, + { 0, }, +}; + +static struct pci_driver catapult_driver = { + .name = "catapult", + .id_table = catapult_pci_id, + .probe = catapult_probe, + .remove = catapult_remove, +}; + +static const struct file_operations catapult_fileops = { + .owner = THIS_MODULE, + .open = catapult_open, + .release = catapult_release, + .read = NULL, + .write = NULL, + .unlocked_ioctl = catapult_ioctl, + .mmap = catapult_mmap, + .poll = NULL, + .fasync = NULL, + .llseek = noop_llseek, +}; + +static void catapult_cleanup_module(void) +{ + dev_t dev; + + pr_info("%s: unloading %s (%s) v%s\n", __func__, + VER_PRODUCTNAME_STR, VER_INTERNALNAME_STR, PRODUCT_NUMBER_STR); + + if (catapult_driver.driver.name != NULL) + pci_unregister_driver(&catapult_driver); + + if (catapult_class != NULL) { + class_destroy(catapult_class); + catapult_class = NULL; + } + + if (catapult_dev != 0) { + cdev_del(catapult_cdev); + catapult_cdev = NULL; + } + + if (catapult_major != 0) { + pr_info("%s: unregistering major # %d\n", + __func__, catapult_major); + dev = MKDEV(catapult_major, 0); + unregister_chrdev_region(dev, CATAPULT_MAX_DEVICES); + } +} + +static int __init catapult_init_module(void) +{ + struct cdev *cdev = NULL; + int err = 0; + + pr_err("%s: loading %s (%s) v%s\n", __func__, + VER_PRODUCTNAME_STR, VER_INTERNALNAME_STR, PRODUCT_NUMBER_STR); + + /* Verify module parameters */ + if (dma_slot_count > SLOT_COUNT) { + pr_err("%s: dma_slot_count (%d) cannot exceed %d\n", + __func__, dma_slot_count, SLOT_COUNT); + err = -EINVAL; + goto exit; + } + + /* Allocate a range of character device major/minor numbers */ + err = alloc_chrdev_region(&catapult_dev, 0, CATAPULT_MAX_DEVICES, + "catapult"); + if (err) { + pr_err("%s: error allocating catapult_dev - %d\n", + __func__, err); + goto exit; + } + + pr_info("%s: catapult_dev = (%d,%d)\n", __func__, + MAJOR(catapult_dev), MINOR(catapult_dev)); + catapult_major = MAJOR(catapult_dev); + + /* Allocate a character device with the right set of minor numbers */ + cdev = cdev_alloc(); + if (cdev == NULL) { + err = -ENOMEM; + goto exit; + } + + cdev->owner = THIS_MODULE; + cdev->ops = &catapult_fileops; + kobject_set_name(&cdev->kobj, "catapult"); + + err = cdev_add(cdev, catapult_dev, CATAPULT_MAX_DEVICES); + if (err) { + kobject_put(&cdev->kobj); + goto exit; + } + + catapult_cdev = cdev; + + /* + * Allocate the catapult class object, to create our + * /sys/class/catapult directory. + */ + catapult_class = class_create(THIS_MODULE, "catapult"); + if (catapult_class == NULL) { + pr_err("%s: error creating /sys/class/catapult", __func__); + err = -ENOMEM; + goto exit; + } + + catapult_class->devnode = catapult_devnode; + + /* Register this driver as a PCI driver so that we can get probes */ + err = pci_register_driver(&catapult_driver); + if (err) { + pr_err("%s: error registering driver - %d\n", __func__, err); + goto exit; + } + + pr_info("%s: success\n", __func__); + +exit: + if (err) + catapult_cleanup_module(); + + return err; +} + +module_init(catapult_init_module); +module_exit(catapult_cleanup_module); + +module_param(dma_slot_count, uint, S_IRUSR); +MODULE_PARM_DESC(dma_slot_count, "The number of DMA slots to allocate"); +module_param(dma_slot_bytes, uint, S_IRUSR); +MODULE_PARM_DESC(dma_slot_bytes, "The size in bytes of each DMA buffer"); + +MODULE_VERSION(PRODUCT_NUMBER_STR); +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR("Microsoft Corporation"); +MODULE_DESCRIPTION(VER_PRODUCTNAME_STR); diff --git a/drivers/catapult/catapult-drv.h b/drivers/catapult/catapult-drv.h new file mode 100644 index 000000000000..4b867d69e013 --- /dev/null +++ b/drivers/catapult/catapult-drv.h @@ -0,0 +1,145 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2019 Microsoft, Inc. + * + * Authors: + * Jesse Benson + */ + +#ifndef __CATAPULT_DRV_H +#define __CATAPULT_DRV_H + +#include + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "catapult.h" +#include "catapult-register.h" + +#define CATAPULT_MAX_DEVICES (1u << MINORBITS) +#define SLOT_COUNT 0x40 +#define BYTES_PER_SLOT (1024 * 1024) + +#define VER_PRODUCTNAME_STR "Catapult FPGA driver" +#define VER_INTERNALNAME_STR "catapult.ko" +#define PRODUCT_NUMBER_STR "5.1.4.12" +#define PRODUCT_MAJOR_NUMBER 5 +#define PRODUCT_MINOR_NUMBER 1 +#define BUILD_MAJOR_NUMBER 4 +#define BUILD_MINOR_NUMBER 12 + +/* Data structures related to the FPGA Function Type */ + +/* Role Function GUID */ +/* 4067F10B-C65B-44A7-AD6E-60E489BF32C5 */ +static const guid_t CATAPULT_GUID_ROLE_FUNCTION = + GUID_INIT(0x4067F10B, 0xC65B, 0x44A7, + 0xAD, 0x6E, 0x60, 0xE4, 0x89, 0xBF, 0x32, 0xC5); + +/* Management Function GUID */ +/* DC32A288-935D-4BA7-99CF-B51FBED5CA7C */ +static const guid_t CATAPULT_GUID_MANAGEMENT_FUNCTION = + GUID_INIT(0xDC32A288, 0x935D, 0x4BA7, + 0x99, 0xCF, 0xB5, 0x1F, 0xBE, 0xD5, 0xCA, 0x7C); + +/* + * Management/Role Function GUID + * Used for single function HIPs in a multi-function aware shell + */ +/* 2F97325A-6A0B-4A0E-8286-C5376CFFF60E */ +static const guid_t CATAPULT_GUID_MANAGEMENT_ROLE_FUNCTION = + GUID_INIT(0x2F97325A, 0x6A0B, 0x4A0E, + 0x82, 0x86, 0xC5, 0x37, 0x6C, 0xFF, 0xF6, 0x0E); + +/* + * Legacy Function GUID + * The Function Type GUID won't be set for Legacy, single function images. + * To simplify the code, declare this as a GUID filled with zeros + */ +/* 00000000-0000-0000-0000-000000000000 */ +static const guid_t CATAPULT_GUID_LEGACY_FUNCTION = + GUID_INIT(0x00000000, 0x0000, 0x0000, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00); + +enum fpga_function_type { + FPGA_FUNCTION_TYPE_LEGACY = 0, + FPGA_FUNCTION_TYPE_ROLE = 1, + FPGA_FUNCTION_TYPE_MANAGEMENT = 2, + FPGA_FUNCTION_TYPE_MAX = 3, + FPGA_FUNCTION_TYPE_UNKNOWN = 0xFF, +}; + +struct catapult_device { + uint64_t chip_id; + uint32_t board_id; + uint32_t board_revision; + + volatile void __iomem *registers; + size_t registers_cb; + uint64_t registers_physical_address; + + char name[32]; + int minor; + + bool dfh_supported; + bool avoid_hip1_access; + + int endpoint_number; + int function_number; + enum fpga_function_type function_type; + const char *function_type_name; + + uint32_t shell_version; + uint32_t shell_id; + uint32_t role_id; + uint32_t role_version; + + /* Completion event to signal when an interrupt occurs (e.g. for DMA) */ + struct completion event_obj[SLOT_COUNT]; + struct mutex lock; + + uint32_t number_of_slots; + uint32_t bytes_per_slot; + + uint32_t dma_input_len; + void *dma_input_kernel_addr[SLOT_COUNT]; + dma_addr_t dma_input_dma_addr[SLOT_COUNT]; + uint32_t dma_output_len; + void *dma_output_kernel_addr[SLOT_COUNT]; + dma_addr_t dma_output_dma_addr[SLOT_COUNT]; + uint32_t dma_control_len; + void *dma_control_kernel_addr; + dma_addr_t dma_control_dma_addr; + uint32_t dma_result_len; + void *dma_result_kernel_addr; + dma_addr_t dma_result_dma_addr; + + uint32_t interrupt_feature_offset; + int irq; + + struct pci_dev *pdev; + struct device *dev; + + unsigned long *slot_map; + pid_t *slot_map_pids; +}; + +struct catapult_file { + struct inode *inode; + struct file *file; + struct catapult_device *idev; + uint32_t registered_interrupt; +}; + +struct catapult_device *to_catapult_dev(struct device *dev); + +#endif /* __CATAPULT_DRV_H */ diff --git a/drivers/catapult/catapult-ioctl.c b/drivers/catapult/catapult-ioctl.c new file mode 100644 index 000000000000..c1223851ed00 --- /dev/null +++ b/drivers/catapult/catapult-ioctl.c @@ -0,0 +1,480 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * catapult-ioctl.c - I/O request processing + * + * Copyright (C) 2019 Microsoft, Inc. + * + * Authors: + * Jesse Benson + */ + +#include + +#include "catapult.h" +#include "catapult-drv.h" +#include "catapult-ioctl.h" + +/* Invalid/unsupported control code. */ +static long catapult_unsupported_ioctl(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + dev_err(idev->dev, "%s: unknown I/O control code 0x%08x\n", __func__, cmd); + return -EINVAL; +} + +/* Get metadata about the Catapult registers. */ +static long catapult_get_register_info(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + struct catapult_register_info reg_info = { + .region_count = 1, + .region_size = { idev->registers_cb, 0 }, + }; + + if (copy_to_user(arg, ®_info, sizeof(reg_info))) + return -EFAULT; + + return 0; +} + +/* Disable signaling to user-mode when interrupts occur. */ +static long catapult_interrupt_disable(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + struct catapult_file *ifile = filep->private_data; + + ifile->registered_interrupt = 0; + + dev_info(idev->dev, "%s: interrupts disabled\n", __func__); + + return 0; +} + +/* Enable signaling to user-mode when interrupts occur. */ +static long catapult_interrupt_enable(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + struct catapult_file *ifile = filep->private_data; + + ifile->registered_interrupt = 1; + + dev_info(idev->dev, "%s: interrupts enabled\n", __func__); + + return 0; +} + +/* Get pointers to allocated buffer. */ +static long catapult_get_buffer_pointers(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + struct catapult_buffer_ptrs info = { + .input_size = idev->dma_input_len, + .input = NULL, /* user-mode has to mmap */ + .input_phys = virt_to_phys(idev->dma_input_kernel_addr[0]), + + .output_size = idev->dma_output_len, + .output = NULL, /* user-mode has to mmap */ + .output_phys = virt_to_phys(idev->dma_output_kernel_addr[0]), + + .result_size = idev->dma_result_len, + .result = NULL, /* user-mode has to mmap */ + .result_phys = virt_to_phys(idev->dma_result_kernel_addr), + + .control_size = idev->dma_control_len, + .control = NULL, /* user-mode has to mmap */ + .control_phys = virt_to_phys(idev->dma_control_kernel_addr), + }; + + if (copy_to_user(arg, &info, sizeof(info))) + return -EFAULT; + + return 0; +} + +/* Get the driver version. */ +static long catapult_get_driver_version(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + struct catapult_driver_version info = { + .product_major_version = PRODUCT_MAJOR_NUMBER, + .product_minor_version = PRODUCT_MINOR_NUMBER, + .build_major_version = BUILD_MAJOR_NUMBER, + .build_minor_version = BUILD_MINOR_NUMBER, + }; + + if (copy_to_user(arg, &info, sizeof(info))) + return -EFAULT; + + return 0; +} + +/* Acquire a free DMA slot. */ +static long catapult_acquire_slot(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + long status = 0; + struct catapult_slot_reservation reservation = { + .slot = 0, + .input_buffer = NULL, + .output_buffer = NULL, + .result_buffer = NULL, + .control_buffer = NULL, + }; + + status = mutex_lock_interruptible(&idev->lock); + if (status == 0) { + BUG_ON(idev->slot_map == NULL); + reservation.slot = + bitmap_find_next_zero_area(idev->slot_map, + idev->number_of_slots, + /*start:*/ 0, + /*nr:*/ 1, + /*align_mask:*/ 0); + if (reservation.slot >= 0 && + reservation.slot < idev->number_of_slots) { + set_bit(reservation.slot, idev->slot_map); + idev->slot_map_pids[reservation.slot] = + task_tgid_nr(current); + } else { + status = -ENOSPC; + } + mutex_unlock(&idev->lock); + } + + if (status != 0) { + dev_err(idev->dev, "%s: failed to acquire slot - %ld\n", + __func__, status); + return status; + } + + if (copy_to_user(arg, &reservation, sizeof(reservation))) + status = -EFAULT; + + return status; +} + +/* Release a previously acquired DMA slot. */ +static long catapult_release_slot(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + struct catapult_slot_reservation input; + long status = 0; + + if (copy_from_user(&input, arg, sizeof(input))) + return -EFAULT; + + if (input.slot < 0 || input.slot >= idev->number_of_slots) + return -EINVAL; + + mutex_lock(&idev->lock); + BUG_ON(idev->slot_map == NULL); + if (test_bit(input.slot, idev->slot_map) && + idev->slot_map_pids[input.slot] == task_tgid_nr(current)) { + clear_bit(input.slot, idev->slot_map); + idev->slot_map_pids[input.slot] = 0; + } else { + status = -EACCES; + } + mutex_unlock(&idev->lock); + + return status; +} + +/* Acquire a range of DMA slots. */ +static long catapult_acquire_slot_range(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + long status = 0; + uint32_t i = 0; + uint32_t start = 0; + uint32_t end = 0; + struct catapult_acquire_slot_range *info = NULL; + + info = kzalloc(sizeof(*info), GFP_KERNEL); + if (info == NULL) + return -ENOMEM; + + if (copy_from_user(info, arg, sizeof(*info))) { + status = -EFAULT; + goto exit; + } + + /* For now only contiguous ranges are supported */ + if (info->slot_range.range_type != CATAPULT_SLOT_RANGE_CONTIGUOUS) { + status = -EINVAL; + goto exit; + } + + start = info->slot_range.start; + end = info->slot_range.end; + + if (start >= idev->number_of_slots || end >= idev->number_of_slots || + start > end) { + status = -EINVAL; + goto exit; + } + + /* Acquire the DMA slots */ + status = mutex_lock_interruptible(&idev->lock); + if (status != 0) + goto exit; + + BUG_ON(idev->slot_map == NULL); + for (i = start; i <= end; i++) { + if (test_bit(i, idev->slot_map)) { + status = -EBUSY; + break; + } + } + + if (status == 0) { + for (i = start; i <= end; i++) { + set_bit(i, idev->slot_map); + idev->slot_map_pids[i] = task_tgid_nr(current); + } + } + mutex_unlock(&idev->lock); + + /* Fill starting from info->reservations[0] */ + for (i = start; i <= end; i++) { + info->reservations[i - start].slot = i; + info->reservations[i - start].input_buffer = NULL; + info->reservations[i - start].output_buffer = NULL; + info->reservations[i - start].result_buffer = NULL; + info->reservations[i - start].control_buffer = NULL; + } + + if (copy_to_user(arg, info, sizeof(*info))) + status = -EFAULT; + +exit: + if (info != NULL) + kvfree(info); + + return status; +} + +/* Release all DMA slots previously acquired by the requesting process. */ +static long catapult_release_slot_range(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + long status = 0; + uint32_t i = 0; + + mutex_lock(&idev->lock); + for (i = 0; i < idev->number_of_slots; i++) { + BUG_ON(idev->slot_map == NULL); + if (test_bit(i, idev->slot_map) && + idev->slot_map_pids[i] == task_tgid_nr(current)) { + clear_bit(i, idev->slot_map); + } + } + mutex_unlock(&idev->lock); + + return status; +} + +/* Ensure the slot event is ready for use by user space code. */ +static long catapult_get_slot_event(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + struct catapult_get_slot_event input; + + if (copy_from_user(&input, arg, sizeof(input))) + return -EFAULT; + + if (input.slot_index >= idev->number_of_slots) + return -EINVAL; + + return 0; +} + +/* Block until the slot event has completed. */ +static long catapult_wait_slot_event(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + struct catapult_wait_slot_event input; + struct completion *completion = NULL; + unsigned long timeout = 0; + long status = 0; + + if (copy_from_user(&input, arg, sizeof(input))) + return -EFAULT; + + if (input.slot_index >= idev->number_of_slots) + return -EINVAL; + + completion = &(idev->event_obj[input.slot_index]); + dev_dbg(idev->dev, "%s: waiting on slot %u (%p)\n", + __func__, input.slot_index, completion); + + if (input.wait) { + if (input.timeout == 0) { /* Infinite timeout */ + /* Returns 0 for success, <0 for failure */ + status = wait_for_completion_interruptible(completion); + } else { + timeout = msecs_to_jiffies(input.timeout); + + /* Returns >0 for success, 0 for timeout, + * <0 for failure */ + status = wait_for_completion_interruptible_timeout( + completion, timeout); + + /* Convert status codes above to our return values + * (0 for success, <0 for failure). */ + if (status == 0) { + status = -ETIMEDOUT; + } else if (status < 0) { + /* Use error status as is */ + } else { + status = 0; + } + } + } else { + if (try_wait_for_completion(completion)) + status = 0; + else + status = -EWOULDBLOCK; + } + + dev_dbg(idev->dev, "%s: waiting for slot %u completed with %ld\n", + __func__, input.slot_index, status); + return status; +} + +/* Get slot configuration for the given catapult device. */ +static long catapult_get_slot_config(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + struct catapult_slot_configuration cfg = { + .bytes_per_slot = idev->bytes_per_slot, + .number_of_slots = idev->number_of_slots, + }; + + if (copy_to_user(arg, &cfg, sizeof(cfg))) + return -EFAULT; + + return 0; +} + +/* Reset the slot event so it can be signaled again. */ +static long catapult_reset_slot_event(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + struct completion *completion = NULL; + struct catapult_reset_slot_event input; + + if (copy_from_user(&input, arg, sizeof(input))) + return -EFAULT; + + if (input.slot_index >= idev->number_of_slots) + return -EINVAL; + + completion = &(idev->event_obj[input.slot_index]); + reinit_completion(completion); + + return 0; +} + +/* Complete the slot event to signal any waiters. */ +static long catapult_complete_slot_event(struct catapult_device *idev, + struct file *filep, + unsigned int cmd, + void __user *arg) +{ + struct completion *completion = NULL; + struct catapult_complete_slot_event input; + + if (copy_from_user(&input, arg, sizeof(input))) + return -EFAULT; + + if (input.slot_index >= idev->number_of_slots) + return -EINVAL; + + completion = &(idev->event_obj[input.slot_index]); + complete(completion); + + return 0; +} + +long catapult_ioctl(struct file *filep, unsigned int cmd, unsigned long arg) +{ + struct catapult_file *ifile = filep->private_data; + struct catapult_device *idev = ifile->idev; + void __user *uarg = (void __user *)arg; + + switch (cmd) { + case CATAPULT_IOCTL_GET_REGISTER_INFO: + return catapult_get_register_info(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_INTERRUPT_DISABLE: + return catapult_interrupt_disable(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_INTERRUPT_ENABLE: + return catapult_interrupt_enable(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_GET_BUFFER_POINTERS: + return catapult_get_buffer_pointers(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_GET_DRIVER_VERSION: + return catapult_get_driver_version(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_ACQUIRE_SLOT: + return catapult_acquire_slot(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_RELEASE_SLOT: + return catapult_release_slot(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_ACQUIRE_SLOT_RANGE: + return catapult_acquire_slot_range(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_RELEASE_SLOT_RANGE: + return catapult_release_slot_range(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_GET_SLOT_EVENT: + return catapult_get_slot_event(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_WAIT_SLOT_EVENT: + return catapult_wait_slot_event(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_RESET_SLOT_EVENT: + return catapult_reset_slot_event(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_GET_SLOT_CONFIG: + return catapult_get_slot_config(idev, filep, cmd, uarg); + + case CATAPULT_IOCTL_COMPLETE_SLOT_EVENT: + return catapult_complete_slot_event(idev, filep, cmd, uarg); + + default: + return catapult_unsupported_ioctl(idev, filep, cmd, uarg); + } +} diff --git a/drivers/catapult/catapult-ioctl.h b/drivers/catapult/catapult-ioctl.h new file mode 100644 index 000000000000..7f8c7ebed443 --- /dev/null +++ b/drivers/catapult/catapult-ioctl.h @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * catapult-ioctl.h - I/O request processing + * + * Copyright (C) 2019 Microsoft, Inc. + * + * Authors: + * Jesse Benson + */ + +#ifndef __CATAPULT_IOCTL_H +#define __CATAPULT_IOCTL_H + +long catapult_ioctl(struct file *filep, unsigned int cmd, unsigned long arg); + +#endif /* __CATAPULT_IOCTL_H */ diff --git a/drivers/catapult/catapult-register.c b/drivers/catapult/catapult-register.c new file mode 100644 index 000000000000..e037982eb538 --- /dev/null +++ b/drivers/catapult/catapult-register.c @@ -0,0 +1,193 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2019 Microsoft, Inc. + * + * Authors: + * Jesse Benson + */ + +#include +#include +#include + +#include "catapult-register.h" +#include "catapult-shell.h" + +/** + * catapult_register_read32 - Read a 32-bit device register. + * @address: Address of the memory mapped register + * + * Read a 32-bit value from a device register. This routine uses a barrier + * intrinsic to prevent re-ordering across the call and forces reads and + * writes to memory to complete at the point of the invocation. + */ +uint32_t catapult_register_read32(volatile uint32_t *address) +{ + mb(); + return readl((volatile void __iomem *)address); +} + +/** + * catapult_register_write32 - Write a 32-bit device register. + * @address: Address of the memory mapped register + * @value: Value to write + * + * Write a 32-bit value to a device register. This routine uses a barrier + * intrinsic to prevent re-ordering across the call and forces reads and + * writes to memory to complete at the point of the invocation. + */ +void catapult_register_write32(volatile uint32_t *address, uint32_t value) +{ + writel(value, (volatile void __iomem *)address); + mb(); +} + +/** + * catapult_register_read64 - Read a 64-bit device register. + * @address: Address of the memory mapped register + * + * Read a 64-bit value from a device register. This routine uses a barrier + * intrinsic to prevent re-ordering across the call and forces reads and + * writes to memory to complete at the point of the invocation. + */ +uint64_t catapult_register_read64(volatile uint64_t *address) +{ + mb(); + return readq((volatile void __iomem *)address); +} + +/** + * catapult_register_write64 - Write a 64-bit device register. + * @address: Address of the memory mapped register + * @value: Value to write + * + * Write a 64-bit value to a device register. This routine uses a barrier + * intrinsic to prevent re-ordering across the call and forces reads and + * writes to memory to complete at the point of the invocation. + */ +void catapult_register_write64(volatile uint64_t *address, uint64_t value) +{ + writeq(value, (volatile void __iomem *)address); + mb(); +} + +static uint32_t catapult_low_level_read_legacy(volatile void __iomem *registers, + uint32_t interp_address, + uint32_t app_address) +{ + uintptr_t byte_address = catapult_register_offset(interp_address, app_address); + return catapult_register_read32((uint32_t *)(registers + byte_address)); +} + +static void catapult_low_level_write_legacy(volatile void __iomem *registers, + uint32_t interp_address, + uint32_t app_address, + uint32_t value) +{ + uintptr_t byte_address = catapult_register_offset(interp_address, app_address); + catapult_register_write32((uint32_t *)(registers + byte_address), value); +} + +static uint64_t catapult_low_level_read_64(volatile void __iomem *registers, + uint32_t interp_address, + uint32_t app_address) +{ + uintptr_t byte_address = catapult_register_offset(interp_address, app_address); + return catapult_register_read64((uint64_t *)(registers + byte_address)); +} + +static void catapult_low_level_write_64(volatile void __iomem *registers, + uint32_t interp_address, + uint32_t app_address, + uint64_t value) +{ + uintptr_t byte_address = catapult_register_offset(interp_address, app_address); + catapult_register_write64((uint64_t *)(registers + byte_address), value); +} + +uint32_t catapult_low_level_read(volatile void __iomem *registers, + uint32_t interp_address, + uint32_t app_address) +{ + uint32_t readData = 0; + + switch (interp_address & 0xf) { + case INTER_ADDR_FULL_STATUS_REG: + /* Instead of 64 addresses each 1 bit, now it is 1 address + * with 64 bits, unpack results in software */ + readData = (uint32_t) ((catapult_low_level_read_64(registers, INTER_ADDR_SOFT_REG, SOFT_REG_SLOT_DMA_BASE_ADDR + 62) >> app_address) & 1); + break; + + case INTER_ADDR_DONE_STATUS_REG: + readData = (uint32_t) ((catapult_low_level_read_64(registers, INTER_ADDR_SOFT_REG, SOFT_REG_SLOT_DMA_BASE_ADDR + 61) >> app_address) & 1); + break; + + case INTER_ADDR_PEND_STATUS_REG: + readData = (uint32_t) ((catapult_low_level_read_64(registers, INTER_ADDR_SOFT_REG, SOFT_REG_SLOT_DMA_BASE_ADDR + 60) >> app_address) & 1); + break; + + case INTER_ADDR_GENERAL_PURPOSE_REG: + readData = catapult_low_level_read_legacy(registers, interp_address, app_address); + break; + + case INTER_ADDR_ASMI_RSU: + readData = catapult_low_level_read_legacy(registers, interp_address, app_address); + break; + + case INTER_ADDR_HACK_OVERRIDE_OUT_DATA_SIZE: + if (app_address >= 2 && app_address <= 6) + readData = (uint32_t) catapult_low_level_read_64(registers, INTER_ADDR_SOFT_REG, SOFT_REG_SLOT_DMA_BASE_ADDR + 55 + (app_address - 2)); + else + readData = 0; + break; + + case INTER_ADDR_INTERRUPT: + if (app_address == 257) + readData = (uint32_t) catapult_low_level_read_64(registers, INTER_ADDR_SOFT_REG, SOFT_REG_SLOT_DMA_BASE_ADDR + 54); + else + readData = 0; + break; + + case INTER_ADDR_DMA_DESCRIPTORS_AND_RESERVED: + if (app_address <= 53) { + /* force legacy, even if we have soft reg capability, role may not have these registers */ + if (app_address == 4 || app_address == 5 || app_address == 6) + readData = catapult_low_level_read_legacy(registers, interp_address, app_address); + else /* 0-3, 7-53 mapping for the factory tester registers */ + readData = (uint32_t) catapult_low_level_read_64(registers, INTER_ADDR_SOFT_REG, SOFT_REG_SLOT_DMA_BASE_ADDR + app_address); + } else { + readData = 0; + } + break; + + default: + readData = 0; + break; + } + + return readData; +} + +void catapult_low_level_write(volatile void __iomem *registers, + uint32_t interp_address, + uint32_t app_address, + uint32_t value) +{ + uint64_t write_data = 0; + + switch (interp_address & 0xf) { + case INTER_ADDR_GENERAL_PURPOSE_REG: + catapult_low_level_write_legacy(registers, interp_address, app_address, value); + break; + + case INTER_ADDR_ASMI_RSU: + catapult_low_level_write_legacy(registers, interp_address, app_address, value); + break; + + default: + write_data = catapult_register_offset(interp_address, app_address); + write_data = (write_data << 32) | value; + catapult_low_level_write_64(registers, INTER_ADDR_SOFT_REG, SOFT_REG_SLOT_DMA_BASE_ADDR + 63, write_data); + break; + } +} diff --git a/drivers/catapult/catapult-register.h b/drivers/catapult/catapult-register.h new file mode 100644 index 000000000000..f5c6a7b12a19 --- /dev/null +++ b/drivers/catapult/catapult-register.h @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2019 Microsoft, Inc. + * + * Authors: + * Jesse Benson + */ + +#ifndef __CATAPULT_REGISTER_H +#define __CATAPULT_REGISTER_H + +#include +#include + +uint32_t catapult_register_read32(volatile uint32_t *address); +void catapult_register_write32(volatile uint32_t *address, uint32_t value); + +uint64_t catapult_register_read64(volatile uint64_t *address); +void catapult_register_write64(volatile uint64_t *address, uint64_t value); + +uint32_t catapult_low_level_read(volatile void __iomem *registers, + uint32_t interp_address, + uint32_t app_address); + +void catapult_low_level_write(volatile void __iomem *registers, + uint32_t interp_address, + uint32_t app_address, + uint32_t value); + +static inline uintptr_t catapult_register_offset(uint32_t interp_addr, + uint32_t register_number) +{ + return (register_number << 8) | (interp_addr << 4) | 4; +} + +#endif /* __CATAPULT_REGISTER_H */ diff --git a/drivers/catapult/catapult-shell.h b/drivers/catapult/catapult-shell.h new file mode 100644 index 000000000000..e3310569f861 --- /dev/null +++ b/drivers/catapult/catapult-shell.h @@ -0,0 +1,215 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2019 Microsoft, Inc. + * + * Authors: + * Jesse Benson + */ + +#ifndef __CATAPULT_SHELL_H +#define __CATAPULT_SHELL_H + +#define CATAPULT_PCI_VENDOR_ID 0x1414 +#define CATAPULT_PCI_DEVICE_ID_LP_HIP1_MANAGEMENT 0xB204 +#define CATAPULT_PCI_DEVICE_ID_LP_HIP2_MANAGEMENT 0xB205 +#define CATAPULT_PCI_DEVICE_ID_LP_HIP1_ROLE 0xB284 +#define CATAPULT_PCI_DEVICE_ID_LP_HIP2_ROLE 0xB285 + +#define INTER_ADDR_FULL_STATUS_REG 0 /* repurposed */ +#define INTER_ADDR_DONE_STATUS_REG 1 /* repurposed */ +#define INTER_ADDR_PEND_STATUS_REG 2 /* repurposed */ +#define INTER_ADDR_GENERAL_PURPOSE_REG 3 +#define INTER_ADDR_PROBE_IN_FPGA_BUFFER_0 4 +#define INTER_ADDR_PROBE_IN_FPGA_BUFFER_1 5 +#define INTER_ADDR_PROBE_OUT_FPGA_BUFFER_0 6 +#define INTER_ADDR_PROBE_OUT_FPGA_BUFFER_1 7 +#define INTER_ADDR_PROBE_RES_FPGA_BUFFER_0 8 /* repurposed */ +#define INTER_ADDR_PROBE_RES_FPGA_BUFFER_1 9 /* repurposed */ +#define INTER_ADDR_ASMI_RSU 10 +#define INTER_ADDR_AVALON 11 +#define INTER_ADDR_HACK_OVERRIDE_OUT_DATA_SIZE 12 +#define INTER_ADDR_ENABLE_DISABLE 13 +#define INTER_ADDR_INTERRUPT 14 +#define INTER_ADDR_DMA_DESCRIPTORS_AND_RESERVED 15 + +/* Repurposed interpretation address for 64-bit soft register interface */ +#define INTER_ADDR_SOFT_REG 8 +#define INTER_ADDR_SOFT_REG_CAPABILITY 9 +#define SOFT_REG_CAPABILITY_SIGNATURE 0x50F750F7 +#define SOFT_REG_SLOT_DMA_BASE_ADDR 0x7E00 +#define SOFT_REG_SLOT_DMA_MAGIC_ADDR (SOFT_REG_SLOT_DMA_BASE_ADDR + 63) +#define SOFT_REG_MAPPING_SLOT_DMA_MAGIC_VALUE 0x8926fc9c4e6256d9ULL +/* This magic value is defined in hardware in SoftRegs_Adapter.sv */ + +/* Repurposed interpretation address for multi-function images */ +#define INTER_ADDR_DFH_0 0 +#define INTER_ADDR_DFH_1 1 +#define INTER_ADDR_DFH_2 2 + +/* Definitions for Device Function Header */ +union catapult_dfh_header { + struct { + uint64_t afu_feature_id : 12; /* 11:0 */ + uint64_t afu_major : 4; /* 15:12 */ + uint64_t afu_offset : 24; /* 39:16 */ + uint64_t afu_eol : 1; /* 40 */ + uint64_t afu_rsvd0 : 7; /* 47:41 */ + uint64_t afu_minor : 4; /* 51:48 */ + uint64_t afu_rsvd1 : 8; /* 59:52 */ + uint64_t afu_type : 4; /* 63:60 =0x04 if DFH supported */ + }; + + uint64_t as_ulonglong; + uint32_t as_ulongs[2]; +}; + +enum catapult_dfh_type { + DFH_TYPE_NOT_SUPPORTED = 0, + DFH_TYPE_INTEL_AFU = 1, + DFH_TYPE_BASIC_BUILDING_BLOCK = 2, + DFH_TYPE_PRIVATE_FEATURE = 3, + DFH_TYPE_FIU = 4, + DFH_TYPE_MAX = 5, +}; + +#define DFH_FEATURE_GUID_OFFSET_LOWER 0x08 +#define DFH_FEATURE_GUID_OFFSET_HIGHER 0x10 + +/* Bit offsets for the afu_feature_id field in the DFH */ +#define DFH_FEATURE_ASMI_RSU_PRESENT_MASK 0x01 +#define DFH_FEATURE_SOFTSHELL_PRESENT_MASK 0x02 + +/* Definitions for shell control feature */ +static const guid_t GUID_FPGA_SHELL_CONTROL_FEATURE = + GUID_INIT(0x3ABD40CA, 0x48B5, 0x450D, + 0x94, 0x79, 0x1B, 0xD9, 0x70, 0x00, 0x7B, 0x8D); + +#define DFH_FEATURE_DMA_CONTROL_REG_OFFSET 0x18 +#define DFH_FEATURE_ROLE_CONTROL_REG_OFFSET 0x20 + +/* Registers for the shell control feature */ +union catapult_dma_control_register { + struct { + uint64_t dma_function_select : 1; + uint64_t reserved : 63; + }; + + uint64_t as_ulonglong; +}; + +#define DMA_FUNCTION_MANAGEMENT 0x0 +#define DMA_FUNCTION_ROLE 0x1 + +union catapult_role_control_register { + struct { + uint64_t role_interrupt_mask : 1; + uint64_t isolate_role : 1; + uint64_t reserved : 62; + }; + + uint64_t as_ulonglong; +}; + +#define ROLE_INTERRUPT_ENABLED 0x0 +#define ROLE_INTERRUPT_DISABLED 0x1 + +#define ROLE_NOT_ISOLATED 0x0 +#define ROLE_ISOLATED 0x1 + +/* Definitions for interrupt feature */ +static const guid_t GUID_FPGA_INTERRUPT_FEATURE = + GUID_INIT(0x73ACD711, 0x2CCF, 0x4305, + 0xA4, 0x1F, 0x3E, 0x0A, 0xD6, 0x76, 0xB2, 0x52); + +#define DFH_FEATURE_INTERRUPT_MASK_REG_OFFSET 0x18 +#define DFH_FEATURE_INTERRUPT_STATUS_REG_OFFSET 0x20 + +/* Registers for the interrupt feature */ +union catapult_interrupt_mask_register { + struct { + uint64_t slot_dma_interrupt : 1; + uint64_t reserved : 63; + }; + + uint64_t as_ulonglong; +}; + +union catapult_interrupt_status_register { + struct { + uint64_t slot_dma_interrupt : 1; + uint64_t reserved : 63; + }; + + uint64_t as_ulonglong; +}; + +/* Constants for general purpose (aka. shell) register addresses */ +#define GP_REGISTER_INDEX_BOARD_REVISION 56 +#define GP_REGISTER_INDEX_BOARD_ID 57 +#define GP_REGISTER_INDEX_SHELL_RELEASE_VERSION 58 +#define GP_REGISTER_INDEX_BUILD_INFO 59 +#define GP_REGISTER_INDEX_TFS_CHANGESET_NUMBER 60 +#define GP_REGISTER_INDEX_CHIP_ID_LOW 62 +#define GP_REGISTER_INDEX_CHIP_ID_HIGH 63 +#define GP_REGISTER_INDEX_SHELL_ID 64 +#define GP_REGISTER_INDEX_ROLE_VERSION 65 +#define GP_REGISTER_INDEX_SHELL_STATUS 68 +#define GP_REGISTER_INDEX_ROLE_STATUS 70 +#define GP_REGISTER_INDEX_TEMPERATURE 71 +#define GP_REGISTER_INDEX_SHELL_IDENTITY 91 +#define GP_REGISTER_INDEX_ROLE_ID 101 + +/* Format for the Shell Identity Register */ +union catapult_shell_identity_register { + struct { + uint32_t function_number : 16; + uint32_t endpoint_number : 4; + uint32_t reserved : 12; + }; + + uint32_t as_ulong; +}; + +/* Structure of the host-side, per-slot DMA control buffer */ +struct catapult_dma_control_buffer { + uint32_t reserved1; + uint32_t full_status; + uint32_t reserved2; + uint32_t done_status; + uint32_t reserved3[12]; +}; + +/* Structure of the host-side, per-slot DMA results buffer */ +struct catapult_dma_result_buffer { + uint32_t bytes_received; + uint32_t reserved[15]; +}; + +struct catapult_dma_iso_control_result_combined { + struct catapult_dma_control_buffer control_buffer; + struct catapult_dma_result_buffer result_buffer; +}; + +/* Constants specific to slot isolation capable shells */ +#define SOFT_REGISTER_SHIFT_OFFSET 3 +#define MSB_SHIFT_FPGA_NUM_SHELL_REG_ISO 18 +#define SOFT_REGISTER_BASE_ADDRESS 0x800000 +#define DMA_SLOT_INPUT_BASE_ADDRESS 0x901000 +#define DMA_SLOT_OUTPUT_BASE_ADDRESS 0x901008 +#define DMA_SLOT_CONTROL_RESULT_BASE_ADDRESS 0x901010 +#define DMA_SLOT_FULL_BASE_ADDRESS 0x980000 +#define DMA_SLOT_DONE_BASE_ADDRESS 0x980008 + +#define FPGA_CONTROL_SIZE sizeof(struct catapult_dma_control_buffer) +#define FPGA_RESULT_SIZE sizeof(struct catapult_dma_iso_control_result_combined) + +#define SHELL_ID_ABALONE 0xCA7A0ABA +#define SHELL_VERSION_ABALONE_ISOLATION_CAPABLE 0x00030000 +#define SHELL_ID_BEDROCK 0xBED70C +#define SHELL_VERSION_BEDROCK_ISOLATION_CAPABLE 0x00020000 +#define ROLE_VERSION_GOLDEN_10A 0xCA7A010A +#define ROLE_ID_GOLDEN_10A 0x601D + +#define SHELL_CHIP_ID_DISCONNECTED_VALUE 0xdeadbeefdeadbeef + +#endif /* __CATAPULT_SHELL_H */ diff --git a/drivers/catapult/catapult.h b/drivers/catapult/catapult.h new file mode 100644 index 000000000000..8ca0b57d8b46 --- /dev/null +++ b/drivers/catapult/catapult.h @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Header file for Catapult FPGA driver user API + * + * Copyright (C) 2019 Microsoft, Inc. + * + * Authors: + * Jesse Benson + */ + +#ifndef __CATAPULT_H +#define __CATAPULT_H + +/* + * The number of slots must be at least 2 otherwise it breaks the verilog syntax + * for some multiplexers in hardware conceptually the design will support 1 slot + * but there is no practical point given the FPGA is double buffered. The + * software ISR handshaking (32-bit PCIe reads) requires that slot numbers are + * representable on 8 bits, hence up to 256 can be used. + */ +#define MIN_FPGA_NUM_SLOTS 2 +#define MAX_FPGA_NUM_SLOTS 256 + +/* 64-bit base addresses to support mmap requests for BAR and DMA registers */ +#define CATAPULT_FPGA_REGISTER_ADDRESS 0x0000000000000000 +#define CATAPULT_FPGA_DMA_INPUT_BASE_ADDRESS 0x1000000000000000 +#define CATAPULT_FPGA_DMA_OUTPUT_BASE_ADDRESS 0x2000000000000000 +#define CATAPULT_FPGA_DMA_RESULT_ADDRESS 0x3000000000000000 +#define CATAPULT_FPGA_DMA_CONTROL_ADDRESS 0x4000000000000000 +#define CATAPULT_FPGA_DMA_BASE_ADDRESS_MASK 0xF000000000000000 + +#define CATAPULT_IOCTL_MAGIC 0xF0 /* Customer range is 32768 - 65535 */ + +struct catapult_register_info { + uint8_t region_count; + uint32_t region_size[6]; +}; + +struct catapult_get_slot_event { + uint32_t slot_index; +}; + +struct catapult_wait_slot_event { + uint32_t slot_index; + uint32_t timeout; /* timeout in milliseconds (or 0 for INFINITE) */ + bool wait; /* true: block until timeout + * false: test for completion and return immediately */ +}; + +struct catapult_reset_slot_event { + uint32_t slot_index; +}; + +struct catapult_complete_slot_event { + uint32_t slot_index; +}; + +struct catapult_buffer_ptrs { + uint32_t input_size; + void *input; + uint64_t input_phys; + uint32_t output_size; + void *output; + uint64_t output_phys; + uint32_t result_size; + void *result; + uint64_t result_phys; + uint32_t control_size; + void *control; + uint64_t control_phys; +}; + +/* + * The product major and minor versions are manually maintained by the + * developer, and should be considered an indicator of non-breaking (minor) + * or breaking (major) interface or behavioral changes. + */ +struct catapult_driver_version { + uint16_t product_major_version; + uint16_t product_minor_version; + uint16_t build_major_version; + uint16_t build_minor_version; +}; + +/* Used to describe the configured slot values of the driver. */ +struct catapult_slot_configuration { + uint32_t bytes_per_slot; + uint32_t number_of_slots; +}; + +/* Used to reserve a slot for exclusive use by the calling process. */ +struct catapult_slot_reservation { + uint32_t slot; + uint32_t *input_buffer; + uint32_t *output_buffer; + uint32_t *result_buffer; + uint32_t *control_buffer; +}; + +enum catapult_slot_range_type { + CATAPULT_SLOT_RANGE_INVALID = 0, + CATAPULT_SLOT_RANGE_CONTIGUOUS, + CATAPULT_SLOT_RANGE_DISCONTIGUOUS, +}; + +/* Used to reserve multiple slots for exclusive use by the calling process. */ +struct catapult_slot_range_reservation { + enum catapult_slot_range_type range_type; + uint32_t start; + uint32_t end; +}; + +struct catapult_acquire_slot_range { + struct catapult_slot_range_reservation slot_range; + struct catapult_slot_reservation reservations[MAX_FPGA_NUM_SLOTS]; +}; + +#define CATAPULT_IOCTL_GET_REGISTER_INFO _IOR (CATAPULT_IOCTL_MAGIC, 1, struct catapult_register_info) +#define CATAPULT_IOCTL_INTERRUPT_DISABLE _IO (CATAPULT_IOCTL_MAGIC, 2) +#define CATAPULT_IOCTL_INTERRUPT_ENABLE _IO (CATAPULT_IOCTL_MAGIC, 3) + +#define CATAPULT_IOCTL_GET_BUFFER_POINTERS _IOR (CATAPULT_IOCTL_MAGIC, 11, struct catapult_buffer_ptrs) + +#define CATAPULT_IOCTL_GET_DRIVER_VERSION _IOR (CATAPULT_IOCTL_MAGIC, 16, struct catapult_driver_version) +#define CATAPULT_IOCTL_GET_SLOT_CONFIG _IOR (CATAPULT_IOCTL_MAGIC, 17, struct catapult_slot_configuration) + +/* IOCTLs associated with process isolation */ +#define CATAPULT_IOCTL_ACQUIRE_SLOT _IOR (CATAPULT_IOCTL_MAGIC, 19, struct catapult_slot_reservation) +#define CATAPULT_IOCTL_RELEASE_SLOT _IOW (CATAPULT_IOCTL_MAGIC, 20, struct catapult_slot_reservation) +#define CATAPULT_IOCTL_ACQUIRE_SLOT_RANGE _IOWR(CATAPULT_IOCTL_MAGIC, 21, struct catapult_acquire_slot_range) +#define CATAPULT_IOCTL_RELEASE_SLOT_RANGE _IO (CATAPULT_IOCTL_MAGIC, 22) + +#define CATAPULT_IOCTL_GET_SLOT_EVENT _IOW (CATAPULT_IOCTL_MAGIC, 30, struct catapult_get_slot_event) +#define CATAPULT_IOCTL_WAIT_SLOT_EVENT _IOW (CATAPULT_IOCTL_MAGIC, 31, struct catapult_wait_slot_event) +#define CATAPULT_IOCTL_RESET_SLOT_EVENT _IOW (CATAPULT_IOCTL_MAGIC, 32, struct catapult_reset_slot_event) +#define CATAPULT_IOCTL_COMPLETE_SLOT_EVENT _IOW (CATAPULT_IOCTL_MAGIC, 33, struct catapult_complete_slot_event) + +#endif /* __CATAPULT_H */