| Message ID | cover.5e9d6544b53f48a8aa4a412a9bf7cf8d9ec7f891.1519192720.git-series.andrew.donnellan@au1.ibm.com |
|---|---|
| Headers | show |
| Series | Initial OpenCAPI 3.0 Support for P9 | expand |
I reviewed the whole series, again focusing on the diff with the previous version (v5), and it looks ok to me. As hinted in the patchset, witherspoon support is still a work in progress but that's understood, you need a special adapter for the opencapi connector, so it's not like anybody else is going to try. Fred Le 21/02/2018 à 06:59, Andrew Donnellan a écrit : > This series implements OpenCAPI support for P9. > > The series is divided as follows: > > - Patches 1-3: general refactoring > > Add various structs and fields we'll need later. > > - Patches 4-5: setting up the NPU > > At present, we're doing NPU configuration separately from the existing > NVLink code. At a later point, we intend to rework this to make it > possible to support both NVLink and OpenCAPI links on the same NPU. > > - Patches 6-7: training links and creating OpenCAPI PHBs > > Unlike NVLink, which presents a single PHB to represent the entire NPU, > we present a single PHB per device/link. This is necessary due to > limitations in MMIO window allocation. Unfortunately this makes the > structures we share with NVLink a little more complex, oh well. > > - Patch 8: OPAL API calls > > We define three new API calls for handling the Shared Process Area and > setting OpenCAPI TL template capabilities. > > - Patch 9: platform support > > See below. > > - Patch 10: device tree documentation > > Notable limitations: > > - We only support the zaius platform for now. We'll be adding ZZ after a bit > more testing booting ZZs with hostboot rather than BML. Witherspoon > support will come at a later point. > > - No mixing of OpenCAPI and NVLink devices on the same NPU. This will come in a > later series. The only platform this currently impacts is Witherspoon. > > - No support for link ganging - there are also no OpenCAPI devices that support > link ganging yet, so this will come when we get to it... > > - Link information is hardcoded per platform, and we don't have any form of > presence detection apart from failing to train the link. Eventually, this > will be detected via I2C once Hostboot adds the relevant link info to HDAT. > > - No LPC memory - this will come in a later patch once we've done a bit more > testing internally. > > - No link reset functionality - this will come in a later patch. > > This series has been tested on a Zaius. I've also tested it on a > GPU-equipped Witherspoon to ensure it doesn't break NVLink. > > Thanks to everyone who's helped us with this, especially to Alistair Popple for > his advice on NVLink, and to the OpenCAPI hardware teams in Austin and > Rochester who provided us with a lot of vital assistance. > > Comments welcome! > > --- > > v1->v2: > > - Address various comments from Fred > > v2->v3: > > - Address minor comments from Fred > - Silence ioda_reset warning in Linux > - Fix PHB device tree issues > - Mask FIRs to correctly handle Datalink Layer Stall and NoStall signals > (Fred) > - Add device tree binding information > - Reverse order of lane masks, per Alistair's patch at > https://patchwork.ozlabs.org/patch/865155/ > > v3->v4: > > - Address comments from Fred and Alistair > > v4->v5: > > - I accidentally a --- line in patch 5, whoops > > v5->v6: > > - Reduce link training retries > - Fix brick<->ODL mapping on OBUS3 (only relevant to witherspoon) > - Skip unnecessary phy training subprocedure that triggers FIR bit > - Add some comments > > Andrew Donnellan (9): > npu2: Split out common helper functions into separate file > npu2: Rework NPU data structures for OpenCAPI > platform: Add fields for OpenCAPI platform data > p9-adu: Implement update sequence for PowerBus Hotplug Mode Control > npu2-opencapi: Configure NPU for OpenCAPI > npu2-hw-procedures: Add support for OpenCAPI PHY link training > npu2-opencapi: Train OpenCAPI links and setup devices > platforms: Add OpenCAPI platform data and device tree nodes > doc/device-tree: Add OpenCAPI device tree bindings > > Frederic Barrat (1): > npu2-opencapi: Add OpenCAPI OPAL API calls > > core/init.c | 2 +- > core/pci.c | 3 +- > core/platform.c | 11 +- > doc/device-tree/opencapi.rst | 95 +- > doc/opal-api/opal-npu2-opencapi-159-160-161.rst | 126 +- > hw/Makefile.inc | 3 +- > hw/npu2-common.c | 114 +- > hw/npu2-hw-procedures.c | 114 +- > hw/npu2-opencapi.c | 1690 ++++++++++++++++- > hw/npu2.c | 244 +-- > hw/p9-adu.c | 184 ++- > include/npu2-regs.h | 158 +- > include/npu2.h | 88 +- > include/opal-api.h | 6 +- > include/p9-adu.h | 59 +- > include/pci.h | 1 +- > include/platform.h | 14 +- > include/skiboot.h | 1 +- > platforms/astbmc/zaius.c | 96 +- > platforms/ibm-fsp/zz.c | 11 +- > 20 files changed, 2818 insertions(+), 202 deletions(-) > create mode 100644 doc/device-tree/opencapi.rst > create mode 100644 doc/opal-api/opal-npu2-opencapi-159-160-161.rst > create mode 100644 hw/npu2-common.c > create mode 100644 hw/npu2-opencapi.c > create mode 100644 hw/p9-adu.c > create mode 100644 include/p9-adu.h > > base-commit: 4769f3932c0b93d9e5f929ac8589cef2e4a7d5d6 >
This series implements OpenCAPI support for P9. The series is divided as follows: - Patches 1-3: general refactoring Add various structs and fields we'll need later. - Patches 4-5: setting up the NPU At present, we're doing NPU configuration separately from the existing NVLink code. At a later point, we intend to rework this to make it possible to support both NVLink and OpenCAPI links on the same NPU. - Patches 6-7: training links and creating OpenCAPI PHBs Unlike NVLink, which presents a single PHB to represent the entire NPU, we present a single PHB per device/link. This is necessary due to limitations in MMIO window allocation. Unfortunately this makes the structures we share with NVLink a little more complex, oh well. - Patch 8: OPAL API calls We define three new API calls for handling the Shared Process Area and setting OpenCAPI TL template capabilities. - Patch 9: platform support See below. - Patch 10: device tree documentation Notable limitations: - We only support the zaius platform for now. We'll be adding ZZ after a bit more testing booting ZZs with hostboot rather than BML. Witherspoon support will come at a later point. - No mixing of OpenCAPI and NVLink devices on the same NPU. This will come in a later series. The only platform this currently impacts is Witherspoon. - No support for link ganging - there are also no OpenCAPI devices that support link ganging yet, so this will come when we get to it... - Link information is hardcoded per platform, and we don't have any form of presence detection apart from failing to train the link. Eventually, this will be detected via I2C once Hostboot adds the relevant link info to HDAT. - No LPC memory - this will come in a later patch once we've done a bit more testing internally. - No link reset functionality - this will come in a later patch. This series has been tested on a Zaius. I've also tested it on a GPU-equipped Witherspoon to ensure it doesn't break NVLink. Thanks to everyone who's helped us with this, especially to Alistair Popple for his advice on NVLink, and to the OpenCAPI hardware teams in Austin and Rochester who provided us with a lot of vital assistance. Comments welcome! --- v1->v2: - Address various comments from Fred v2->v3: - Address minor comments from Fred - Silence ioda_reset warning in Linux - Fix PHB device tree issues - Mask FIRs to correctly handle Datalink Layer Stall and NoStall signals (Fred) - Add device tree binding information - Reverse order of lane masks, per Alistair's patch at https://patchwork.ozlabs.org/patch/865155/ v3->v4: - Address comments from Fred and Alistair v4->v5: - I accidentally a --- line in patch 5, whoops v5->v6: - Reduce link training retries - Fix brick<->ODL mapping on OBUS3 (only relevant to witherspoon) - Skip unnecessary phy training subprocedure that triggers FIR bit - Add some comments Andrew Donnellan (9): npu2: Split out common helper functions into separate file npu2: Rework NPU data structures for OpenCAPI platform: Add fields for OpenCAPI platform data p9-adu: Implement update sequence for PowerBus Hotplug Mode Control npu2-opencapi: Configure NPU for OpenCAPI npu2-hw-procedures: Add support for OpenCAPI PHY link training npu2-opencapi: Train OpenCAPI links and setup devices platforms: Add OpenCAPI platform data and device tree nodes doc/device-tree: Add OpenCAPI device tree bindings Frederic Barrat (1): npu2-opencapi: Add OpenCAPI OPAL API calls core/init.c | 2 +- core/pci.c | 3 +- core/platform.c | 11 +- doc/device-tree/opencapi.rst | 95 +- doc/opal-api/opal-npu2-opencapi-159-160-161.rst | 126 +- hw/Makefile.inc | 3 +- hw/npu2-common.c | 114 +- hw/npu2-hw-procedures.c | 114 +- hw/npu2-opencapi.c | 1690 ++++++++++++++++- hw/npu2.c | 244 +-- hw/p9-adu.c | 184 ++- include/npu2-regs.h | 158 +- include/npu2.h | 88 +- include/opal-api.h | 6 +- include/p9-adu.h | 59 +- include/pci.h | 1 +- include/platform.h | 14 +- include/skiboot.h | 1 +- platforms/astbmc/zaius.c | 96 +- platforms/ibm-fsp/zz.c | 11 +- 20 files changed, 2818 insertions(+), 202 deletions(-) create mode 100644 doc/device-tree/opencapi.rst create mode 100644 doc/opal-api/opal-npu2-opencapi-159-160-161.rst create mode 100644 hw/npu2-common.c create mode 100644 hw/npu2-opencapi.c create mode 100644 hw/p9-adu.c create mode 100644 include/p9-adu.h base-commit: 4769f3932c0b93d9e5f929ac8589cef2e4a7d5d6