diff mbox series

[v2,12/13] ocxl: Documentation

Message ID 20180123113148.6024-13-fbarrat@linux.vnet.ibm.com (mailing list archive)
State Accepted
Commit 00b96c0e3c54a1fff9e8b909f46a672e43ef9598
Headers show
Series New driver to support OpenCAPI devices on POWER9 | expand

Commit Message

Frederic Barrat Jan. 23, 2018, 11:31 a.m. UTC
ocxl.rst gives a quick, high-level view of opencapi.

Update ioctl-number.txt to reflect ioctl numbers being used by the
ocxl driver

Signed-off-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
---
 Documentation/ABI/testing/sysfs-class-ocxl |  35 +++++++
 Documentation/accelerators/ocxl.rst        | 160 +++++++++++++++++++++++++++++
 Documentation/ioctl/ioctl-number.txt       |   1 +
 3 files changed, 196 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-class-ocxl
 create mode 100644 Documentation/accelerators/ocxl.rst

Comments

gregkh@linuxfoundation.org Jan. 25, 2018, 1:17 p.m. UTC | #1
On Tue, Jan 23, 2018 at 12:31:47PM +0100, Frederic Barrat wrote:
> ocxl.rst gives a quick, high-level view of opencapi.
> 
> Update ioctl-number.txt to reflect ioctl numbers being used by the
> ocxl driver
> 
> Signed-off-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
> ---
>  Documentation/ABI/testing/sysfs-class-ocxl |  35 +++++++
>  Documentation/accelerators/ocxl.rst        | 160 +++++++++++++++++++++++++++++
>  Documentation/ioctl/ioctl-number.txt       |   1 +
>  3 files changed, 196 insertions(+)
>  create mode 100644 Documentation/ABI/testing/sysfs-class-ocxl
>  create mode 100644 Documentation/accelerators/ocxl.rst
> 
> diff --git a/Documentation/ABI/testing/sysfs-class-ocxl b/Documentation/ABI/testing/sysfs-class-ocxl
> new file mode 100644
> index 000000000000..ac11deb71235
> --- /dev/null
> +++ b/Documentation/ABI/testing/sysfs-class-ocxl
> @@ -0,0 +1,35 @@
> +What:           /sys/class/ocxl/<afu name>/afu_version
> +Date:           January 2018
> +Contact:        linuxppc-dev@lists.ozlabs.org
> +Description:    read only
> +                Version of the AFU, in the format <major>:<minor>
> +		Reflects what is read in the configuration space of the AFU

Odd mix of tabs and spaces in this file, please just use tabs.

thanks,

greg k-h
Frederic Barrat Jan. 26, 2018, 3:10 p.m. UTC | #2
Le 25/01/2018 à 14:17, Greg KH a écrit :
> On Tue, Jan 23, 2018 at 12:31:47PM +0100, Frederic Barrat wrote:
>> ocxl.rst gives a quick, high-level view of opencapi.
>>
>> Update ioctl-number.txt to reflect ioctl numbers being used by the
>> ocxl driver
>>
>> Signed-off-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
>> ---
>>   Documentation/ABI/testing/sysfs-class-ocxl |  35 +++++++
>>   Documentation/accelerators/ocxl.rst        | 160 +++++++++++++++++++++++++++++
>>   Documentation/ioctl/ioctl-number.txt       |   1 +
>>   3 files changed, 196 insertions(+)
>>   create mode 100644 Documentation/ABI/testing/sysfs-class-ocxl
>>   create mode 100644 Documentation/accelerators/ocxl.rst
>>
>> diff --git a/Documentation/ABI/testing/sysfs-class-ocxl b/Documentation/ABI/testing/sysfs-class-ocxl
>> new file mode 100644
>> index 000000000000..ac11deb71235
>> --- /dev/null
>> +++ b/Documentation/ABI/testing/sysfs-class-ocxl
>> @@ -0,0 +1,35 @@
>> +What:           /sys/class/ocxl/<afu name>/afu_version
>> +Date:           January 2018
>> +Contact:        linuxppc-dev@lists.ozlabs.org
>> +Description:    read only
>> +                Version of the AFU, in the format <major>:<minor>
>> +		Reflects what is read in the configuration space of the AFU
> 
> Odd mix of tabs and spaces in this file, please just use tabs.

Oops! Will fix.

   Fred


> thanks,
> 
> greg k-h
>
Michael Ellerman Jan. 27, 2018, 9:02 a.m. UTC | #3
Frederic Barrat <fbarrat@linux.vnet.ibm.com> writes:

> Le 25/01/2018 à 14:17, Greg KH a écrit :
>> On Tue, Jan 23, 2018 at 12:31:47PM +0100, Frederic Barrat wrote:
>>> ocxl.rst gives a quick, high-level view of opencapi.
>>>
>>> Update ioctl-number.txt to reflect ioctl numbers being used by the
>>> ocxl driver
>>>
>>> Signed-off-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
>>> ---
>>>   Documentation/ABI/testing/sysfs-class-ocxl |  35 +++++++
>>>   Documentation/accelerators/ocxl.rst        | 160 +++++++++++++++++++++++++++++
>>>   Documentation/ioctl/ioctl-number.txt       |   1 +
>>>   3 files changed, 196 insertions(+)
>>>   create mode 100644 Documentation/ABI/testing/sysfs-class-ocxl
>>>   create mode 100644 Documentation/accelerators/ocxl.rst
>>>
>>> diff --git a/Documentation/ABI/testing/sysfs-class-ocxl b/Documentation/ABI/testing/sysfs-class-ocxl
>>> new file mode 100644
>>> index 000000000000..ac11deb71235
>>> --- /dev/null
>>> +++ b/Documentation/ABI/testing/sysfs-class-ocxl
>>> @@ -0,0 +1,35 @@
>>> +What:           /sys/class/ocxl/<afu name>/afu_version
>>> +Date:           January 2018
>>> +Contact:        linuxppc-dev@lists.ozlabs.org
>>> +Description:    read only
>>> +                Version of the AFU, in the format <major>:<minor>
>>> +		Reflects what is read in the configuration space of the AFU
>> 
>> Odd mix of tabs and spaces in this file, please just use tabs.
>
> Oops! Will fix.

I've already applied it to my test tree, I'll fix it up.

cheers
diff mbox series

Patch

diff --git a/Documentation/ABI/testing/sysfs-class-ocxl b/Documentation/ABI/testing/sysfs-class-ocxl
new file mode 100644
index 000000000000..ac11deb71235
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-class-ocxl
@@ -0,0 +1,35 @@ 
+What:           /sys/class/ocxl/<afu name>/afu_version
+Date:           January 2018
+Contact:        linuxppc-dev@lists.ozlabs.org
+Description:    read only
+                Version of the AFU, in the format <major>:<minor>
+		Reflects what is read in the configuration space of the AFU
+
+What:           /sys/class/ocxl/<afu name>/contexts
+Date:           January 2018
+Contact:        linuxppc-dev@lists.ozlabs.org
+Description:    read only
+		Number of contexts for the AFU, in the format <n>/<max>
+		where:
+			n: number of currently active contexts, for debug
+			max: maximum number of contexts supported by the AFU
+
+What:           /sys/class/ocxl/<afu name>/pp_mmio_size
+Date:           January 2018
+Contact:        linuxppc-dev@lists.ozlabs.org
+Description:    read only
+		Size of the per-process mmio area, as defined in the
+		configuration space of the AFU
+
+What:           /sys/class/ocxl/<afu name>/global_mmio_size
+Date:           January 2018
+Contact:        linuxppc-dev@lists.ozlabs.org
+Description:    read only
+		Size of the global mmio area, as defined in the
+		configuration space of the AFU
+
+What:           /sys/class/ocxl/<afu name>/global_mmio_area
+Date:           January 2018
+Contact:        linuxppc-dev@lists.ozlabs.org
+Description:    read/write
+		Give access the global mmio area for the AFU
diff --git a/Documentation/accelerators/ocxl.rst b/Documentation/accelerators/ocxl.rst
new file mode 100644
index 000000000000..4f7af841d935
--- /dev/null
+++ b/Documentation/accelerators/ocxl.rst
@@ -0,0 +1,160 @@ 
+========================================================
+OpenCAPI (Open Coherent Accelerator Processor Interface)
+========================================================
+
+OpenCAPI is an interface between processors and accelerators. It aims
+at being low-latency and high-bandwidth. The specification is
+developed by the `OpenCAPI Consortium <http://opencapi.org/>`_.
+
+It allows an accelerator (which could be a FPGA, ASICs, ...) to access
+the host memory coherently, using virtual addresses. An OpenCAPI
+device can also host its own memory, that can be accessed from the
+host.
+
+OpenCAPI is known in linux as 'ocxl', as the open, processor-agnostic
+evolution of 'cxl' (the driver for the IBM CAPI interface for
+powerpc), which was named that way to avoid confusion with the ISDN
+CAPI subsystem.
+
+
+High-level view
+===============
+
+OpenCAPI defines a Data Link Layer (DL) and Transaction Layer (TL), to
+be implemented on top of a physical link. Any processor or device
+implementing the DL and TL can start sharing memory.
+
+::
+
+  +-----------+                         +-------------+
+  |           |                         |             |
+  |           |                         | Accelerated |
+  | Processor |                         |  Function   |
+  |           |  +--------+             |    Unit     |  +--------+
+  |           |--| Memory |             |    (AFU)    |--| Memory |
+  |           |  +--------+             |             |  +--------+
+  +-----------+                         +-------------+
+       |                                       |
+  +-----------+                         +-------------+
+  |    TL     |                         |    TLX      |
+  +-----------+                         +-------------+
+       |                                       |
+  +-----------+                         +-------------+
+  |    DL     |                         |    DLX      |
+  +-----------+                         +-------------+
+       |                                       |
+       |                   PHY                 |
+       +---------------------------------------+
+
+
+
+Device discovery
+================
+
+OpenCAPI relies on a PCI-like configuration space, implemented on the
+device. So the host can discover AFUs by querying the config space.
+
+OpenCAPI devices in Linux are treated like PCI devices (with a few
+caveats). The firmware is expected to abstract the hardware as if it
+was a PCI link. A lot of the existing PCI infrastructure is reused:
+devices are scanned and BARs are assigned during the standard PCI
+enumeration. Commands like 'lspci' can therefore be used to see what
+devices are available.
+
+The configuration space defines the AFU(s) that can be found on the
+physical adapter, such as its name, how many memory contexts it can
+work with, the size of its MMIO areas, ...
+
+
+
+MMIO
+====
+
+OpenCAPI defines two MMIO areas for each AFU:
+
+* the global MMIO area, with registers pertinent to the whole AFU.
+* a per-process MMIO area, which has a fixed size for each context.
+
+
+
+AFU interrupts
+==============
+
+OpenCAPI includes the possibility for an AFU to send an interrupt to a
+host process. It is done through a 'intrp_req' defined in the
+Transaction Layer, specifying a 64-bit object handle which defines the
+interrupt.
+
+The driver allows a process to allocate an interrupt and obtain its
+64-bit object handle, that can be passed to the AFU.
+
+
+
+char devices
+============
+
+The driver creates one char device per AFU found on the physical
+device. A physical device may have multiple functions and each
+function can have multiple AFUs. At the time of this writing though,
+it has only been tested with devices exporting only one AFU.
+
+Char devices can be found in /dev/ocxl/ and are named as:
+/dev/ocxl/<AFU name>.<location>.<index>
+
+where <AFU name> is a max 20-character long name, as found in the
+config space of the AFU.
+<location> is added by the driver and can help distinguish devices
+when a system has more than one instance of the same OpenCAPI device.
+<index> is also to help distinguish AFUs in the unlikely case where a
+device carries multiple copies of the same AFU.
+
+
+
+Sysfs class
+===========
+
+An ocxl class is added for the devices representing the AFUs. See
+/sys/class/ocxl. The layout is described in
+Documentation/ABI/testing/sysfs-class-ocxl
+
+
+
+User API
+========
+
+open
+----
+
+Based on the AFU definition found in the config space, an AFU may
+support working with more than one memory context, in which case the
+associated char device may be opened multiple times by different
+processes.
+
+
+ioctl
+-----
+
+OCXL_IOCTL_ATTACH:
+
+  Attach the memory context of the calling process to the AFU so that
+  the AFU can access its memory.
+
+OCXL_IOCTL_IRQ_ALLOC:
+
+  Allocate an AFU interrupt and return an identifier.
+
+OCXL_IOCTL_IRQ_FREE:
+
+  Free a previously allocated AFU interrupt.
+
+OCXL_IOCTL_IRQ_SET_FD:
+
+  Associate an event fd to an AFU interrupt so that the user process
+  can be notified when the AFU sends an interrupt.
+
+
+mmap
+----
+
+A process can mmap the per-process MMIO area for interactions with the
+AFU.
diff --git a/Documentation/ioctl/ioctl-number.txt b/Documentation/ioctl/ioctl-number.txt
index 3e3fdae5f3ed..6501389d55b9 100644
--- a/Documentation/ioctl/ioctl-number.txt
+++ b/Documentation/ioctl/ioctl-number.txt
@@ -326,6 +326,7 @@  Code  Seq#(hex)	Include File		Comments
 0xB5	00-0F	uapi/linux/rpmsg.h	<mailto:linux-remoteproc@vger.kernel.org>
 0xC0	00-0F	linux/usb/iowarrior.h
 0xCA	00-0F	uapi/misc/cxl.h
+0xCA	10-2F	uapi/misc/ocxl.h
 0xCA	80-BF	uapi/scsi/cxlflash_ioctl.h
 0xCB	00-1F	CBM serial IEC bus	in development:
 					<mailto:michael.klein@puffin.lb.shuttle.de>