[{"id":3684375,"web_url":"http://patchwork.ozlabs.org/comment/3684375/","msgid":"","list_archive_url":null,"date":"2026-04-30T02:20:53","subject":"Re: [PATCH v8 RESEND] Introduce Cray ClusterStor E1000 NVMe slot LED\n driver","submitter":{"id":88614,"url":"http://patchwork.ozlabs.org/api/people/88614/","name":"Corey Minyard","email":"corey@minyard.net"},"content":"On Wed, Apr 29, 2026 at 04:22:55PM -0700, Tony Hutter wrote:\n> Add driver to control the NVMe slot LEDs on the Cray ClusterStor E1000.\n> The driver provides hotplug attention status callbacks for the 24 NVMe\n> slots on the E1000. This allows users to access the E1000's locate and\n> fault LEDs via the normal /sys/bus/pci/slots//attention sysfs\n> entries. This driver uses IPMI to communicate with the E1000 controller\n> to toggle the LEDs.\n> \n> Signed-off-by: Tony Hutter \n\nFor the IPMI portions:\nReviewed-by: Corey Minyard \n\nHave you tested removing and adding the IPMI interface while this is up?\nYou can do that with the hotmod interface on IPMI. I didn't see any\nissues, but it's always good to test.\n\n-corey\n\n> ---\n> Changes in v8:\n> - Remove unused variable in craye1k_get_attention_status().\n> \n> Changes in v7:\n> - Update sysfs-bus-pci text from feedback.\n> - Add DMI dependency to Kconfig.\n> - Refactor pciehp_core.c to remove CONFIG_HOTPLUG_PCI_PCIE_CRAY_E1000\n> code blocks.\n> - Move errno.h #include into correct alphabetical order.\n> - Add comment describing the reasoning for the debugfs counters.\n> - Move craye1k_init() call from pcie_hp_init() to init_slot().\n> - Make craye1k mutex global rather than in craye1k->lock. This enables\n> handling of craye1k_[get|set]_attention_status() calls before the craye1k\n> driver is initialized.\n> - Do driver cleanup on craye1k_smi_gone().\n> \n> Changes in v6:\n> - Change some dev_info_ratelimited() calls to dev_info().\n> - Don't call craye1k_init() if pcie_port_service_register() fails.\n> - Fix stray indent in #define CRAYE1K_POST_CMD_WAIT_MS\n> \n> Changes in v5:\n> - Removed unnecessary ipmi_smi.h #include.\n> - Added WARN_ON() to craye1k_do_message() to sanity check that craye1k->lock\n> is held.\n> - Added additional comments for when craye1k->lock should be held.\n> \n> Changes in v4:\n> - Fix typo in Kconfig: \"is it\" -> \"it is\"\n> - Rename some #defines to CRAYE1K_SUBCMD_*\n> - Use IS_ERR() check in craye1k_debugfs_init()\n> - Return -EIO instead of -EINVAL when LED value check fails\n> \n> Changes in v3:\n> - Add 'attention' values in Documentation/ABI/testing/sysfs-bus-pci.\n> - Remove ACPI_PCI_SLOT dependency.\n> - Cleanup craye1k_do_message() error checking.\n> - Skip unneeded memcpy() on failure in __craye1k_do_command().\n> - Merge craye1k_do_command_and_netfn() code into craye1k_do_command().\n> - Make craye1k_is_primary() return boolean.\n> - Return negative error code on failure in craye1k_set_primary().\n> \n> Changes in v2:\n> - Integrated E1000 code into the pciehp driver as an built-in\n> extention rather than as a standalone module.\n> - Moved debug tunables and counters to debugfs.\n> - Removed forward declarations.\n> - Kept the /sys/bus/pci/slots//attention interface rather\n> than using NPEM/_DSM or led_classdev as suggested. The \"attention\"\n> interface is more beneficial for our site, since it allows us to\n> control the NVMe slot LEDs agnostically across different enclosure\n> vendors and kernel versions using the same scripts. It is also\n> nice to use the same /sys/bus/pci/slots// sysfs directory for\n> both slot LED toggling (\"attention\") and slot power control\n> (\"power\").\n> ---\n> Documentation/ABI/testing/sysfs-bus-pci | 21 +\n> MAINTAINERS | 5 +\n> drivers/pci/hotplug/Kconfig | 10 +\n> drivers/pci/hotplug/Makefile | 3 +\n> drivers/pci/hotplug/pciehp.h | 20 +\n> drivers/pci/hotplug/pciehp_core.c | 20 +-\n> drivers/pci/hotplug/pciehp_craye1k.c | 687 ++++++++++++++++++++++++\n> 7 files changed, 765 insertions(+), 1 deletion(-)\n> create mode 100644 drivers/pci/hotplug/pciehp_craye1k.c\n> \n> diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci\n> index 92debe879ffb..8536d2ff30d1 100644\n> --- a/Documentation/ABI/testing/sysfs-bus-pci\n> +++ b/Documentation/ABI/testing/sysfs-bus-pci\n> @@ -231,6 +231,27 @@ Description:\n> \t\t - scXX contains the device subclass;\n> \t\t - iXX contains the device class programming interface.\n> \n> +What:\t\t/sys/bus/pci/slots/.../attention\n> +Date:\t\tFebruary 2025\n> +Contact:\tlinux-pci@vger.kernel.org\n> +Description:\n> +\t\tThe attention attribute is used to read or write the attention\n> +\t\tstatus for an enclosure slot. This is often used to set the\n> +\t\tslot LED value on a NVMe storage enclosure.\n> +\n> +\t\tCommon values:\n> +\t\t0 = OFF\n> +\t\t1 = ON\n> +\t\t2 = blink\n> +\n> +\t\tUsing the Cray ClusterStor E1000 extensions:\n> +\t\t0 = fault LED OFF, locate LED OFF\n> +\t\t1 = fault LED ON, locate LED OFF\n> +\t\t2 = fault LED OFF, locate LED ON\n> +\t\t3 = fault LED ON, locate LED ON\n> +\n> +\t\tOther values are no-op, OFF, or ON depending on the driver.\n> +\n> What:\t\t/sys/bus/pci/slots/.../module\n> Date:\t\tJune 2009\n> Contact:\tlinux-pci@vger.kernel.org\n> diff --git a/MAINTAINERS b/MAINTAINERS\n> index 9ac254f4ec41..861576d60a36 100644\n> --- a/MAINTAINERS\n> +++ b/MAINTAINERS\n> @@ -6543,6 +6543,11 @@ S:\tMaintained\n> F:\tDocumentation/filesystems/cramfs.rst\n> F:\tfs/cramfs/\n> \n> +CRAY CLUSTERSTOR E1000 NVME SLOT LED DRIVER EXTENSIONS\n> +M:\tTony Hutter \n> +S:\tMaintained\n> +F:\tdrivers/pci/hotplug/pciehp_craye1k.c\n> +\n> CRC LIBRARY\n> M:\tEric Biggers \n> R:\tArd Biesheuvel \n> diff --git a/drivers/pci/hotplug/Kconfig b/drivers/pci/hotplug/Kconfig\n> index 3207860b52e4..3cb84e5e13e9 100644\n> --- a/drivers/pci/hotplug/Kconfig\n> +++ b/drivers/pci/hotplug/Kconfig\n> @@ -183,4 +183,14 @@ config HOTPLUG_PCI_S390\n> \n> \t When in doubt, say Y.\n> \n> +config HOTPLUG_PCI_PCIE_CRAY_E1000\n> +\tbool \"PCIe Hotplug extensions for Cray ClusterStor E1000\"\n> +\tdepends on DMI && HOTPLUG_PCI_PCIE && IPMI_HANDLER=y\n> +\thelp\n> +\t Say Y here if you have a Cray ClusterStor E1000 and want to control\n> +\t your NVMe slot LEDs. Without this driver it is not possible\n> +\t to control the fault and locate LEDs on the E1000's 24 NVMe slots.\n> +\n> +\t When in doubt, say N.\n> +\n> endif # HOTPLUG_PCI\n> diff --git a/drivers/pci/hotplug/Makefile b/drivers/pci/hotplug/Makefile\n> index 40aaf31fe338..82a1f0592d0a 100644\n> --- a/drivers/pci/hotplug/Makefile\n> +++ b/drivers/pci/hotplug/Makefile\n> @@ -66,6 +66,9 @@ pciehp-objs\t\t:=\tpciehp_core.o\t\\\n> \t\t\t\tpciehp_ctrl.o\t\\\n> \t\t\t\tpciehp_pci.o\t\\\n> \t\t\t\tpciehp_hpc.o\n> +ifdef CONFIG_HOTPLUG_PCI_PCIE_CRAY_E1000\n> +pciehp-objs\t\t+=\tpciehp_craye1k.o\n> +endif\n> \n> shpchp-objs\t\t:=\tshpchp_core.o\t\\\n> \t\t\t\tshpchp_ctrl.o\t\\\n> diff --git a/drivers/pci/hotplug/pciehp.h b/drivers/pci/hotplug/pciehp.h\n> index debc79b0adfb..3a8173f3e159 100644\n> --- a/drivers/pci/hotplug/pciehp.h\n> +++ b/drivers/pci/hotplug/pciehp.h\n> @@ -199,6 +199,17 @@ int pciehp_get_raw_indicator_status(struct hotplug_slot *h_slot, u8 *status);\n> \n> int pciehp_slot_reset(struct pcie_device *dev);\n> \n> +#ifdef CONFIG_HOTPLUG_PCI_PCIE_CRAY_E1000\n> +int craye1k_init(void);\n> +bool is_craye1k_board(void);\n> +int craye1k_get_attention_status(struct hotplug_slot *hotplug_slot, u8 *status);\n> +int craye1k_set_attention_status(struct hotplug_slot *hotplug_slot, u8 status);\n> +#else\n> +#define craye1k_init() (0)\n> +#define craye1k_get_attention_status NULL\n> +#define craye1k_set_attention_status NULL\n> +#endif\n> +\n> static inline const char *slot_name(struct controller *ctrl)\n> {\n> \treturn hotplug_slot_name(&ctrl->hotplug_slot);\n> @@ -209,4 +220,13 @@ static inline struct controller *to_ctrl(struct hotplug_slot *hotplug_slot)\n> \treturn container_of(hotplug_slot, struct controller, hotplug_slot);\n> }\n> \n> +static inline bool is_craye1k_slot(struct controller *ctrl)\n> +{\n> +#ifdef CONFIG_HOTPLUG_PCI_PCIE_CRAY_E1000\n> +\treturn (PSN(ctrl) >= 1 && PSN(ctrl) <= 24 && is_craye1k_board());\n> +#else\n> +\treturn false;\n> +#endif\n> +}\n> +\n> #endif\t\t\t\t/* _PCIEHP_H */\n> diff --git a/drivers/pci/hotplug/pciehp_core.c b/drivers/pci/hotplug/pciehp_core.c\n> index f59baa912970..3e8e2b3069bf 100644\n> --- a/drivers/pci/hotplug/pciehp_core.c\n> +++ b/drivers/pci/hotplug/pciehp_core.c\n> @@ -72,6 +72,22 @@ static int init_slot(struct controller *ctrl)\n> \t} else if (ctrl->pcie->port->hotplug_user_indicators) {\n> \t\tops->get_attention_status = pciehp_get_raw_indicator_status;\n> \t\tops->set_attention_status = pciehp_set_raw_indicator_status;\n> +\t} else if (is_craye1k_slot(ctrl)) {\n> +\t\t/*\n> +\t\t * The Cray E1000 driver controls slots 1-24. Initialize the\n> +\t\t * Cray E1000 driver when slot 1 is seen.\n> +\t\t */\n> +\t\tif (PSN(ctrl) == 1) {\n> +\t\t\tretval = craye1k_init();\n> +\t\t\tif (retval) {\n> +\t\t\t\tctrl_err(ctrl,\n> +\t\t\t\t\t \"Error loading Cray E1000 extensions\");\n> +\t\t\t\tkfree(ops);\n> +\t\t\t\treturn retval;\n> +\t\t\t}\n> +\t\t}\n> +\t\tops->get_attention_status = craye1k_get_attention_status;\n> +\t\tops->set_attention_status = craye1k_set_attention_status;\n> \t}\n> \n> \t/* register this slot with the hotplug pci core */\n> @@ -376,8 +392,10 @@ int __init pcie_hp_init(void)\n> \n> \tretval = pcie_port_service_register(&hpdriver_portdrv);\n> \tpr_debug(\"pcie_port_service_register = %d\\n\", retval);\n> -\tif (retval)\n> +\tif (retval) {\n> \t\tpr_debug(\"Failure to register service\\n\");\n> +\t\treturn retval;\n> +\t}\n> \n> \treturn retval;\n> }\n> diff --git a/drivers/pci/hotplug/pciehp_craye1k.c b/drivers/pci/hotplug/pciehp_craye1k.c\n> new file mode 100644\n> index 000000000000..9c5bee81fdf8\n> --- /dev/null\n> +++ b/drivers/pci/hotplug/pciehp_craye1k.c\n> @@ -0,0 +1,687 @@\n> +// SPDX-License-Identifier: GPL-2.0\n> +/*\n> + * Copyright 2022-2024 Lawrence Livermore National Security, LLC\n> + */\n> +/*\n> + * Cray ClusterStor E1000 hotplug slot LED driver extensions\n> + *\n> + * This driver controls the NVMe slot LEDs on the Cray ClusterStore E1000.\n> + * It provides hotplug attention status callbacks for the 24 NVMe slots on\n> + * the E1000. This allows users to access the E1000's locate and fault\n> + * LEDs via the normal /sys/bus/pci/slots//attention sysfs entries.\n> + * This driver uses IPMI to communicate with the E1000 controller to toggle\n> + * the LEDs.\n> + *\n> + * This driver is based off of ibmpex.c\n> + */\n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \"pciehp.h\"\n> +\n> +/* Cray E1000 commands */\n> +#define CRAYE1K_CMD_NETFN 0x3c\n> +#define CRAYE1K_CMD_PRIMARY 0x33\n> +#define CRAYE1K_CMD_FAULT_LED 0x39\n> +#define CRAYE1K_CMD_LOCATE_LED 0x22\n> +\n> +/* Subcommands */\n> +#define CRAYE1K_SUBCMD_GET_LED\t\t0x0\n> +#define CRAYE1K_SUBCMD_SET_LED\t\t0x1\n> +#define CRAYE1K_SUBCMD_SET_PRIMARY\t0x1\n> +\n> +/*\n> + * Milliseconds to wait after get/set LED command. 200ms value found though\n> + * experimentation\n> + */\n> +#define CRAYE1K_POST_CMD_WAIT_MS\t200\n> +\n> +struct craye1k {\n> +\tstruct device *dev; /* BMC device */\n> +\tstruct mutex lock;\n> +\tstruct completion read_complete;\n> +\tstruct ipmi_addr address;\n> +\tstruct ipmi_user *user;\n> +\tint iface;\n> +\n> +\tlong tx_msg_id;\n> +\tstruct kernel_ipmi_msg tx_msg;\n> +\tunsigned char tx_msg_data[IPMI_MAX_MSG_LENGTH];\n> +\tunsigned char rx_msg_data[IPMI_MAX_MSG_LENGTH];\n> +\tunsigned long rx_msg_len;\n> +\tunsigned char rx_result;\t/* IPMI completion code */\n> +\n> +\t/* Parent dir for all our debugfs entries */\n> +\tstruct dentry *parent;\n> +\n> +\t/* debugfs stats */\n> +\tu64 check_primary;\n> +\tu64 check_primary_failed;\n> +\tu64 was_already_primary;\n> +\tu64 was_not_already_primary;\n> +\tu64 set_primary;\n> +\tu64 set_initial_primary_failed;\n> +\tu64 set_primary_failed;\n> +\tu64 set_led_locate_failed;\n> +\tu64 set_led_fault_failed;\n> +\tu64 set_led_readback_failed;\n> +\tu64 set_led_failed;\n> +\tu64 get_led_failed;\n> +\tu64 completion_timeout;\n> +\tu64 wrong_msgid;\n> +\tu64 request_failed;\n> +\n> +\t/* debugfs configuration options */\n> +\n> +\t/* Print info on spurious IPMI messages */\n> +\tbool print_errors;\n> +\n> +\t/* Retries for kernel IPMI layer */\n> +\tu32 ipmi_retries;\n> +\n> +\t/* Timeout in ms for IPMI (0 = use IPMI default_retry_ms) */\n> +\tu32 ipmi_timeout_ms;\n> +\n> +\t/* Timeout in ms to wait for E1000 message completion */\n> +\tu32 completion_timeout_ms;\n> +};\n> +\n> +/*\n> + * Make our craye1k a global so get/set_attention_status() can access it.\n> + * This is safe since there's only one node controller on the board, and so it's\n> + * impossible to instantiate more than one craye1k.\n> + */\n> +static struct craye1k *craye1k_global;\n> +static DEFINE_MUTEX(craye1k_lock);\n> +\n> +/*\n> + * The E1000 command timeout and retry values were found though experimentation\n> + * by looking at the error counters. Keep the counters around to troubleshoot\n> + * any issues with our current timeout/retry values.\n> + */\n> +static struct dentry *\n> +craye1k_debugfs_init(struct craye1k *craye1k)\n> +{\n> +\tumode_t mode = 0644;\n> +\tstruct dentry *parent = debugfs_create_dir(\"pciehp_craye1k\", NULL);\n> +\n> +\tif (IS_ERR(parent))\n> +\t\treturn NULL;\n> +\n> +\tdebugfs_create_x64(\"check_primary\", mode, parent,\n> +\t\t\t &craye1k->check_primary);\n> +\tdebugfs_create_x64(\"check_primary_failed\", mode, parent,\n> +\t\t\t &craye1k->check_primary_failed);\n> +\tdebugfs_create_x64(\"was_already_primary\", mode, parent,\n> +\t\t\t &craye1k->was_already_primary);\n> +\tdebugfs_create_x64(\"was_not_already_primary\", mode, parent,\n> +\t\t\t &craye1k->was_not_already_primary);\n> +\tdebugfs_create_x64(\"set_primary\", mode, parent,\n> +\t\t\t &craye1k->set_primary);\n> +\tdebugfs_create_x64(\"set_initial_primary_failed\", mode, parent,\n> +\t\t\t &craye1k->set_initial_primary_failed);\n> +\tdebugfs_create_x64(\"set_primary_failed\", mode, parent,\n> +\t\t\t &craye1k->set_primary_failed);\n> +\tdebugfs_create_x64(\"set_led_locate_failed\", mode, parent,\n> +\t\t\t &craye1k->set_led_locate_failed);\n> +\tdebugfs_create_x64(\"set_led_fault_failed\", mode, parent,\n> +\t\t\t &craye1k->set_led_fault_failed);\n> +\tdebugfs_create_x64(\"set_led_readback_failed\", mode, parent,\n> +\t\t\t &craye1k->set_led_readback_failed);\n> +\tdebugfs_create_x64(\"set_led_failed\", mode, parent,\n> +\t\t\t &craye1k->set_led_failed);\n> +\tdebugfs_create_x64(\"get_led_failed\", mode, parent,\n> +\t\t\t &craye1k->get_led_failed);\n> +\tdebugfs_create_x64(\"completion_timeout\", mode, parent,\n> +\t\t\t &craye1k->completion_timeout);\n> +\tdebugfs_create_x64(\"wrong_msgid\", mode, parent,\n> +\t\t\t &craye1k->wrong_msgid);\n> +\tdebugfs_create_x64(\"request_failed\", mode, parent,\n> +\t\t\t &craye1k->request_failed);\n> +\n> +\tdebugfs_create_x32(\"ipmi_retries\", mode, parent,\n> +\t\t\t &craye1k->ipmi_retries);\n> +\tdebugfs_create_x32(\"ipmi_timeout_ms\", mode, parent,\n> +\t\t\t &craye1k->ipmi_timeout_ms);\n> +\tdebugfs_create_x32(\"completion_timeout_ms\", mode, parent,\n> +\t\t\t &craye1k->completion_timeout_ms);\n> +\tdebugfs_create_bool(\"print_errors\", mode, parent,\n> +\t\t\t &craye1k->print_errors);\n> +\n> +\t/* Return parent dir dentry */\n> +\treturn parent;\n> +}\n> +\n> +/*\n> + * craye1k_msg_handler() - IPMI message response handler\n> + */\n> +static void craye1k_msg_handler(struct ipmi_recv_msg *msg, void *user_msg_data)\n> +{\n> +\tstruct craye1k *craye1k = user_msg_data;\n> +\n> +\tif (msg->msgid != craye1k->tx_msg_id) {\n> +\t\tcraye1k->wrong_msgid++;\n> +\t\tif (craye1k->print_errors) {\n> +\t\t\tdev_warn_ratelimited(craye1k->dev,\n> +\t\t\t\t\t \"rx msgid %ld != %ld\",\n> +\t\t\t\t\t msg->msgid, craye1k->tx_msg_id);\n> +\t\t}\n> +\t\tipmi_free_recv_msg(msg);\n> +\t\treturn;\n> +\t}\n> +\n> +\t/* Set rx_result to the IPMI completion code */\n> +\tif (msg->msg.data_len > 0)\n> +\t\tcraye1k->rx_result = msg->msg.data[0];\n> +\telse\n> +\t\tcraye1k->rx_result = IPMI_UNKNOWN_ERR_COMPLETION_CODE;\n> +\n> +\tif (msg->msg.data_len > 1) {\n> +\t\t/* Exclude completion code from data bytes */\n> +\t\tcraye1k->rx_msg_len = msg->msg.data_len - 1;\n> +\t\tmemcpy(craye1k->rx_msg_data, msg->msg.data + 1,\n> +\t\t craye1k->rx_msg_len);\n> +\t} else {\n> +\t\tcraye1k->rx_msg_len = 0;\n> +\t}\n> +\n> +\tipmi_free_recv_msg(msg);\n> +\n> +\tcomplete(&craye1k->read_complete);\n> +}\n> +\n> +static const struct ipmi_user_hndl craye1k_user_hndl = {\n> +\t.ipmi_recv_hndl = craye1k_msg_handler\n> +};\n> +\n> +static void craye1k_new_smi(int iface, struct device *dev)\n> +{\n> +\tint rc;\n> +\tstruct craye1k *craye1k;\n> +\n> +\tcraye1k = kzalloc(sizeof(*craye1k), GFP_KERNEL);\n> +\tif (!craye1k)\n> +\t\treturn;\n> +\n> +\tcraye1k->address.addr_type = IPMI_SYSTEM_INTERFACE_ADDR_TYPE;\n> +\tcraye1k->address.channel = IPMI_BMC_CHANNEL;\n> +\tcraye1k->iface = iface;\n> +\tcraye1k->dev = dev;\n> +\tcraye1k->tx_msg.data = craye1k->tx_msg_data;\n> +\tcraye1k->ipmi_retries = 4;\n> +\tcraye1k->ipmi_timeout_ms = 500;\n> +\tcraye1k->completion_timeout_ms = 300;\n> +\n> +\tinit_completion(&craye1k->read_complete);\n> +\n> +\tdev_set_drvdata(dev, craye1k);\n> +\n> +\trc = ipmi_create_user(craye1k->iface, &craye1k_user_hndl, craye1k,\n> +\t\t\t &craye1k->user);\n> +\tif (rc < 0) {\n> +\t\tdev_err(dev, \"Unable to register IPMI user, iface %d\\n\",\n> +\t\t\tcraye1k->iface);\n> +\t\tkfree(craye1k);\n> +\t\tdev_set_drvdata(dev, NULL);\n> +\t\treturn;\n> +\t}\n> +\n> +\tmutex_lock(&craye1k_lock);\n> +\n> +\t/* There's only one node controller so driver data should not be set */\n> +\tWARN_ON(craye1k_global);\n> +\n> +\tcraye1k_global = craye1k;\n> +\tcraye1k->parent = craye1k_debugfs_init(craye1k);\n> +\tmutex_unlock(&craye1k_lock);\n> +\tif (!craye1k->parent)\n> +\t\tdev_warn(dev, \"Cannot create debugfs\");\n> +\n> +\tdev_info(dev, \"Cray ClusterStor E1000 slot LEDs registered\");\n> +}\n> +\n> +static void craye1k_smi_gone(int iface)\n> +{\n> +\tpr_warn(\"craye1k: Got unexpected smi_gone, iface=%d\", iface);\n> +\n> +\tmutex_lock(&craye1k_lock);\n> +\tif (craye1k_global) {\n> +\t\tdebugfs_remove_recursive(craye1k_global->parent);\n> +\t\tkfree(craye1k_global);\n> +\t\tcraye1k_global = NULL;\n> +\t}\n> +\tmutex_unlock(&craye1k_lock);\n> +}\n> +\n> +static struct ipmi_smi_watcher craye1k_smi_watcher = {\n> +\t.owner = THIS_MODULE,\n> +\t.new_smi = craye1k_new_smi,\n> +\t.smi_gone = craye1k_smi_gone\n> +};\n> +\n> +/*\n> + * craye1k_send_message() - Send the message already setup in 'craye1k'\n> + *\n> + * Context: craye1k_lock is already held.\n> + * Return: 0 on success, non-zero on error.\n> + */\n> +static int craye1k_send_message(struct craye1k *craye1k)\n> +{\n> +\tint rc;\n> +\n> +\trc = ipmi_validate_addr(&craye1k->address, sizeof(craye1k->address));\n> +\tif (rc) {\n> +\t\tdev_err_ratelimited(craye1k->dev, \"ipmi_validate_addr() = %d\\n\",\n> +\t\t\t\t rc);\n> +\t\treturn rc;\n> +\t}\n> +\n> +\tcraye1k->tx_msg_id++;\n> +\n> +\trc = ipmi_request_settime(craye1k->user, &craye1k->address,\n> +\t\t\t\t craye1k->tx_msg_id, &craye1k->tx_msg, craye1k,\n> +\t\t\t\t 0, craye1k->ipmi_retries,\n> +\t\t\t\t craye1k->ipmi_timeout_ms);\n> +\n> +\tif (rc) {\n> +\t\tcraye1k->request_failed++;\n> +\t\treturn rc;\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +/*\n> + * craye1k_do_message() - Send the message in 'craye1k' and wait for a response\n> + *\n> + * Context: craye1k_lock is already held.\n> + * Return: 0 on success, non-zero on error.\n> + */\n> +static int craye1k_do_message(struct craye1k *craye1k)\n> +{\n> +\tint rc;\n> +\tstruct completion *read_complete = &craye1k->read_complete;\n> +\tunsigned long tout = msecs_to_jiffies(craye1k->completion_timeout_ms);\n> +\n> +\tWARN_ON(!mutex_is_locked(&craye1k_lock));\n> +\n> +\trc = craye1k_send_message(craye1k);\n> +\tif (rc)\n> +\t\treturn rc;\n> +\n> +\trc = wait_for_completion_killable_timeout(read_complete, tout);\n> +\tif (rc == 0) {\n> +\t\t/* timed out */\n> +\t\tcraye1k->completion_timeout++;\n> +\t\treturn -ETIME;\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +/*\n> + * __craye1k_do_command() - Do an IPMI command\n> + *\n> + * Send a command with optional data bytes, and read back response bytes.\n> + *\n> + * Context: craye1k_lock is already held.\n> + * Returns: 0 on success, non-zero on error.\n> + */\n> +static int __craye1k_do_command(struct craye1k *craye1k, u8 netfn, u8 cmd,\n> +\t\t\t\tu8 *send_data, u8 send_data_len, u8 *recv_data,\n> +\t\t\t\tu8 recv_data_len)\n> +{\n> +\tint rc;\n> +\n> +\tcraye1k->tx_msg.netfn = netfn;\n> +\tcraye1k->tx_msg.cmd = cmd;\n> +\n> +\tif (send_data) {\n> +\t\tmemcpy(&craye1k->tx_msg_data[0], send_data, send_data_len);\n> +\t\tcraye1k->tx_msg.data_len = send_data_len;\n> +\t} else {\n> +\t\tcraye1k->tx_msg_data[0] = 0;\n> +\t\tcraye1k->tx_msg.data_len = 0;\n> +\t}\n> +\n> +\trc = craye1k_do_message(craye1k);\n> +\tif (rc == 0)\n> +\t\tmemcpy(recv_data, craye1k->rx_msg_data, recv_data_len);\n> +\n> +\treturn rc;\n> +}\n> +\n> +/*\n> + * craye1k_do_command() - Do a Cray E1000 specific IPMI command.\n> + * @cmd: Cray E1000 specific command\n> + * @send_data: Data to send after the command\n> + * @send_data_len: Data length\n> + *\n> + * Context: craye1k_lock is already held.\n> + * Returns: the last byte from the response or 0 if response had no response\n> + * data bytes, else -1 on error.\n> + */\n> +static int craye1k_do_command(struct craye1k *craye1k, u8 cmd, u8 *send_data,\n> +\t\t\t u8 send_data_len)\n> +{\n> +\tint rc;\n> +\n> +\trc = __craye1k_do_command(craye1k, CRAYE1K_CMD_NETFN, cmd, send_data,\n> +\t\t\t\t send_data_len, NULL, 0);\n> +\tif (rc != 0) {\n> +\t\t/* Error attempting command */\n> +\t\treturn -1;\n> +\t}\n> +\n> +\tif (craye1k->tx_msg.data_len == 0)\n> +\t\treturn 0;\n> +\n> +\t/* Return last received byte value */\n> +\treturn craye1k->rx_msg_data[craye1k->rx_msg_len - 1];\n> +}\n> +\n> +/*\n> + * __craye1k_set_primary() - Tell the BMC we want to be the primary server\n> + *\n> + * An E1000 board has two physical servers on it. In order to set a slot\n> + * NVMe LED, this server needs to first tell the BMC that it's the primary\n> + * server.\n> + *\n> + * Context: craye1k_lock is already held.\n> + * Returns: 0 on success, non-zero on error.\n> + */\n> +static int __craye1k_set_primary(struct craye1k *craye1k)\n> +{\n> +\tu8 bytes[2] = {CRAYE1K_SUBCMD_SET_PRIMARY, 1};\t/* set primary to 1 */\n> +\n> +\tcraye1k->set_primary++;\n> +\treturn craye1k_do_command(craye1k, CRAYE1K_CMD_PRIMARY, bytes, 2);\n> +}\n> +\n> +/*\n> + * craye1k_is_primary() - Are we the primary server?\n> + *\n> + * Context: craye1k_lock is already held.\n> + * Returns: true if we are the primary server, false otherwise.\n> + */\n> +static bool craye1k_is_primary(struct craye1k *craye1k)\n> +{\n> +\tu8 byte = 0;\n> +\tint rc;\n> +\n> +\t/* Response byte is 0x1 on success */\n> +\trc = craye1k_do_command(craye1k, CRAYE1K_CMD_PRIMARY, &byte, 1);\n> +\tcraye1k->check_primary++;\n> +\tif (rc == 0x1)\n> +\t\treturn true; /* success */\n> +\n> +\tcraye1k->check_primary_failed++;\n> +\treturn false; /* We are not the primary server node */\n> +}\n> +\n> +/*\n> + * craye1k_set_primary() - Attempt to set ourselves as the primary server\n> + *\n> + * Context: craye1k_lock is already held.\n> + * Returns: 0 on success, -1 otherwise.\n> + */\n> +static int craye1k_set_primary(struct craye1k *craye1k)\n> +{\n> +\tint tries = 10;\n> +\n> +\tif (craye1k_is_primary(craye1k)) {\n> +\t\tcraye1k->was_already_primary++;\n> +\t\treturn 0;\n> +\t}\n> +\tcraye1k->was_not_already_primary++;\n> +\n> +\t/* delay found through experimentation */\n> +\tmsleep(300);\n> +\n> +\tif (__craye1k_set_primary(craye1k) != 0) {\n> +\t\tcraye1k->set_initial_primary_failed++;\n> +\t\treturn -1;\t/* error */\n> +\t}\n> +\n> +\t/*\n> +\t * It can take 2 to 3 seconds after setting primary for the controller\n> +\t * to report that it is the primary.\n> +\t */\n> +\twhile (tries--) {\n> +\t\tmsleep(500);\n> +\t\tif (craye1k_is_primary(craye1k))\n> +\t\t\tbreak;\n> +\t}\n> +\n> +\tif (tries == 0) {\n> +\t\tcraye1k->set_primary_failed++;\n> +\t\treturn -1;\t/* never reported that it's primary */\n> +\t}\n> +\n> +\t/* Wait for primary switch to finish */\n> +\tmsleep(1500);\n> +\n> +\treturn 0;\n> +}\n> +\n> +/*\n> + * craye1k_get_slot_led() - Get slot LED value\n> + * @slot: Slot number (1-24)\n> + * @is_locate_led: 0 = get fault LED value, 1 = get locate LED value\n> + *\n> + * Context: craye1k_lock is already held.\n> + * Returns: slot value on success, -1 on failure.\n> + */\n> +static int craye1k_get_slot_led(struct craye1k *craye1k, unsigned char slot,\n> +\t\t\t\tbool is_locate_led)\n> +{\n> +\tu8 bytes[2];\n> +\tu8 cmd;\n> +\n> +\tbytes[0] = CRAYE1K_SUBCMD_GET_LED;\n> +\tbytes[1] = slot;\n> +\n> +\tcmd = is_locate_led ? CRAYE1K_CMD_LOCATE_LED : CRAYE1K_CMD_FAULT_LED;\n> +\n> +\treturn craye1k_do_command(craye1k, cmd, bytes, 2);\n> +}\n> +\n> +/*\n> + * craye1k_set_slot_led() - Attempt to set the locate/fault LED to a value\n> + * @slot: Slot number (1-24)\n> + * @is_locate_led: 0 = use fault LED, 1 = use locate LED\n> + * @value: Value to set (0 or 1)\n> + *\n> + * Check the LED value after calling this function to ensure it has been set\n> + * properly.\n> + *\n> + * Context: craye1k_lock is already held.\n> + * Returns: 0 on success, non-zero on failure.\n> + */\n> +static int craye1k_set_slot_led(struct craye1k *craye1k, unsigned char slot,\n> +\t\t\t\tunsigned char is_locate_led,\n> +\t\t\t\tunsigned char value)\n> +{\n> +\tu8 bytes[3];\n> +\tu8 cmd;\n> +\n> +\tbytes[0] = CRAYE1K_SUBCMD_SET_LED;\n> +\tbytes[1] = slot;\n> +\tbytes[2] = value;\n> +\n> +\tcmd = is_locate_led ? CRAYE1K_CMD_LOCATE_LED : CRAYE1K_CMD_FAULT_LED;\n> +\n> +\treturn craye1k_do_command(craye1k, cmd, bytes, 3);\n> +}\n> +\n> +/*\n> + * __craye1k_get_attention_status() - Get LED value\n> + *\n> + * Context: craye1k_lock is already held.\n> + * Returns: 0 on success, -EIO on failure.\n> + */\n> +static int __craye1k_get_attention_status(struct hotplug_slot *hotplug_slot,\n> +\t\t\t\t\t u8 *status, bool set_primary)\n> +{\n> +\tunsigned char slot;\n> +\tint locate, fault;\n> +\tstruct craye1k *craye1k;\n> +\n> +\tcraye1k = craye1k_global;\n> +\tslot = PSN(to_ctrl(hotplug_slot));\n> +\n> +\tif (set_primary) {\n> +\t\tif (craye1k_set_primary(craye1k) != 0) {\n> +\t\t\tcraye1k->get_led_failed++;\n> +\t\t\treturn -EIO;\n> +\t\t}\n> +\t}\n> +\n> +\tlocate = craye1k_get_slot_led(craye1k, slot, true);\n> +\tif (locate == -1) {\n> +\t\tcraye1k->get_led_failed++;\n> +\t\treturn -EIO;\n> +\t}\n> +\tmsleep(CRAYE1K_POST_CMD_WAIT_MS);\n> +\n> +\tfault = craye1k_get_slot_led(craye1k, slot, false);\n> +\tif (fault == -1) {\n> +\t\tcraye1k->get_led_failed++;\n> +\t\treturn -EIO;\n> +\t}\n> +\tmsleep(CRAYE1K_POST_CMD_WAIT_MS);\n> +\n> +\t*status = locate << 1 | fault;\n> +\n> +\treturn 0;\n> +}\n> +\n> +int craye1k_get_attention_status(struct hotplug_slot *hotplug_slot,\n> +\t\t\t\t u8 *status)\n> +{\n> +\tint rc;\n> +\n> +\tif (mutex_lock_interruptible(&craye1k_lock) != 0)\n> +\t\treturn -EINTR;\n> +\n> +\tif (!craye1k_global) {\n> +\t\t/* Driver isn't initialized yet */\n> +\t\tmutex_unlock(&craye1k_lock);\n> +\t\treturn -EOPNOTSUPP;\n> +\t}\n> +\n> +\trc = __craye1k_get_attention_status(hotplug_slot, status, true);\n> +\n> +\tmutex_unlock(&craye1k_lock);\n> +\treturn rc;\n> +}\n> +\n> +int craye1k_set_attention_status(struct hotplug_slot *hotplug_slot,\n> +\t\t\t\t u8 status)\n> +{\n> +\tunsigned char slot;\n> +\tint tries = 4;\n> +\tint rc;\n> +\tu8 new_status;\n> +\tstruct craye1k *craye1k;\n> +\tbool locate, fault;\n> +\n> +\tif (mutex_lock_interruptible(&craye1k_lock) != 0)\n> +\t\treturn -EINTR;\n> +\n> +\tif (!craye1k_global) {\n> +\t\t/* Driver isn't initialized yet */\n> +\t\tmutex_unlock(&craye1k_lock);\n> +\t\treturn -EOPNOTSUPP;\n> +\t}\n> +\n> +\tcraye1k = craye1k_global;\n> +\n> +\tslot = PSN(to_ctrl(hotplug_slot));\n> +\n> +\t/* Retry to ensure all LEDs are set */\n> +\twhile (tries--) {\n> +\t\t/*\n> +\t\t * The node must first set itself to be the primary node before\n> +\t\t * setting the slot LEDs (each board has two nodes, or\n> +\t\t * \"servers\" as they're called by the manufacturer). This can\n> +\t\t * lead to contention if both nodes are trying to set the LEDs\n> +\t\t * at the same time.\n> +\t\t */\n> +\t\trc = craye1k_set_primary(craye1k);\n> +\t\tif (rc != 0) {\n> +\t\t\t/* Could not set as primary node. Just retry again. */\n> +\t\t\tcontinue;\n> +\t\t}\n> +\n> +\t\t/* Write value twice to increase success rate */\n> +\t\tlocate = (status & 0x2) >> 1;\n> +\t\tcraye1k_set_slot_led(craye1k, slot, 1, locate);\n> +\t\tif (craye1k_set_slot_led(craye1k, slot, 1, locate) != 0) {\n> +\t\t\tcraye1k->set_led_locate_failed++;\n> +\t\t\tcontinue;\t/* fail, retry */\n> +\t\t}\n> +\n> +\t\tmsleep(CRAYE1K_POST_CMD_WAIT_MS);\n> +\n> +\t\tfault = status & 0x1;\n> +\t\tcraye1k_set_slot_led(craye1k, slot, 0, fault);\n> +\t\tif (craye1k_set_slot_led(craye1k, slot, 0, fault) != 0) {\n> +\t\t\tcraye1k->set_led_fault_failed++;\n> +\t\t\tcontinue;\t/* fail, retry */\n> +\t\t}\n> +\n> +\t\tmsleep(CRAYE1K_POST_CMD_WAIT_MS);\n> +\n> +\t\trc = __craye1k_get_attention_status(hotplug_slot, &new_status,\n> +\t\t\t\t\t\t false);\n> +\n> +\t\tmsleep(CRAYE1K_POST_CMD_WAIT_MS);\n> +\n> +\t\tif (rc == 0 && new_status == status)\n> +\t\t\tbreak;\t/* success */\n> +\n> +\t\tcraye1k->set_led_readback_failed++;\n> +\n> +\t\t/*\n> +\t\t * At this point we weren't successful in setting the LED and\n> +\t\t * need to try again.\n> +\t\t *\n> +\t\t * Do a random back-off to reduce contention with other server\n> +\t\t * node in the unlikely case that both server nodes are trying to\n> +\t\t * trying to set a LED at the same time.\n> +\t\t *\n> +\t\t * The 500ms minimum in the back-off reduced the chance of this\n> +\t\t * whole retry loop failing from 1 in 700 to none in 10000.\n> +\t\t */\n> +\t\tmsleep(500 + (get_random_long() % 500));\n> +\t}\n> +\tmutex_unlock(&craye1k_lock);\n> +\tif (tries == 0) {\n> +\t\tcraye1k->set_led_failed++;\n> +\t\treturn -EIO;\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +bool is_craye1k_board(void)\n> +{\n> +\treturn dmi_match(DMI_PRODUCT_NAME, \"VSSEP1EC\");\n> +}\n> +\n> +int craye1k_init(void)\n> +{\n> +\treturn ipmi_smi_watcher_register(&craye1k_smi_watcher);\n> +}\n> +\n> +MODULE_LICENSE(\"GPL\");\n> +MODULE_AUTHOR(\"Tony Hutter \");\n> +MODULE_DESCRIPTION(\"Cray E1000 NVMe Slot LED driver\");\n> -- \n> 2.43.7\n> \n>","headers":{"Return-Path":"\n ","X-Original-To":["incoming@patchwork.ozlabs.org","linux-pci@vger.kernel.org"],"Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=minyard.net header.i=@minyard.net header.a=rsa-sha256\n header.s=google header.b=c5MPME73;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org\n (client-ip=2600:3c04:e001:36c::12fc:5321; helo=tor.lore.kernel.org;\n envelope-from=linux-pci+bounces-53438-incoming=patchwork.ozlabs.org@vger.kernel.org;\n receiver=patchwork.ozlabs.org)","smtp.subspace.kernel.org;\n\tdkim=pass (2048-bit key) header.d=minyard.net header.i=@minyard.net\n header.b=\"c5MPME73\"","smtp.subspace.kernel.org;\n arc=none smtp.client-ip=209.85.167.177","smtp.subspace.kernel.org;\n dmarc=pass (p=none dis=none) header.from=minyard.net","smtp.subspace.kernel.org;\n spf=pass smtp.mailfrom=minyard.net"],"Received":["from tor.lore.kernel.org (tor.lore.kernel.org\n [IPv6:2600:3c04:e001:36c::12fc:5321])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4g5dFc5y4Tz1yHZ\n\tfor ; Thu, 30 Apr 2026 12:21:08 +1000 (AEST)","from smtp.subspace.kernel.org (conduit.subspace.kernel.org\n [100.90.174.1])\n\tby tor.lore.kernel.org (Postfix) with ESMTP id 91022301947D\n\tfor ; Thu, 30 Apr 2026 02:21:05 +0000 (UTC)","from localhost.localdomain (localhost.localdomain [127.0.0.1])\n\tby smtp.subspace.kernel.org (Postfix) with ESMTP id 0A28825FA29;\n\tThu, 30 Apr 2026 02:21:03 +0000 (UTC)","from mail-oi1-f177.google.com (mail-oi1-f177.google.com\n [209.85.167.177])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))\n\t(No client certificate requested)\n\tby smtp.subspace.kernel.org (Postfix) with ESMTPS id 6306219E839\n\tfor ; Thu, 30 Apr 2026 02:21:00 +0000 (UTC)","by mail-oi1-f177.google.com with SMTP id\n 5614622812f47-479ef2b7979so353157b6e.3\n for ; Wed, 29 Apr 2026 19:21:00 -0700 (PDT)","from mail.minyard.net ([2001:470:b8f6:1b:3d3:3600:9626:fd66])\n by smtp.gmail.com with ESMTPSA id\n 586e51a60fabf-434372ab2besm469989fac.15.2026.04.29.19.20.57\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Wed, 29 Apr 2026 19:20:58 -0700 (PDT)"],"ARC-Seal":"i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;\n\tt=1777515662; cv=none;\n b=W0/c6xy2T9XRRvSgm5N5BTDOm9Uf2BHf/v6nLYbF4F4RywhtmvDaCwV+spEyMUNouQjKf30j8Sm15BLOzkGP9pR24SV5oS3TfRmX9Oo5UALJSQUrfRuEO4wm1Kl1JncRmhS3WAsDJA05ec6eB479nJ5WbVunGafCTRdcnES4y4A=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=subspace.kernel.org;\n\ts=arc-20240116; t=1777515662; c=relaxed/simple;\n\tbh=nJ38WGHH/9ptSsr/XrRg/ran+ApGpGt5q28iSqm2Olk=;\n\th=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version:\n\t Content-Type:Content-Disposition:In-Reply-To;\n b=kjB9bWfvpW9Cm29Hp17EPyUa/M8aBcUcl0xfJmQnFWur7+Ej5M/ayi7PiElQcT/i/biUlQFrTuSmGi3V7mjut73T+QGkug3ytk3OWvR7duiFAqyAWkAAD50atLtsBams4WXCHj5fT0AFOvj3GPyH8kR4WefODBwnaWDc3i+i9B8=","ARC-Authentication-Results":"i=1; smtp.subspace.kernel.org;\n dmarc=pass (p=none dis=none) header.from=minyard.net;\n spf=pass smtp.mailfrom=minyard.net;\n dkim=pass (2048-bit key) header.d=minyard.net header.i=@minyard.net\n header.b=c5MPME73; arc=none smtp.client-ip=209.85.167.177","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=minyard.net; s=google; t=1777515659; x=1778120459;\n darn=vger.kernel.org;\n h=in-reply-to:content-disposition:mime-version:references:reply-to\n :message-id:subject:cc:to:from:date:from:to:cc:subject:date\n :message-id:reply-to;\n bh=k31n70UW3aN+dr+eCON/dcK1cC8U/hNuFVuJYM3gpDs=;\n b=c5MPME73qB9cKLZULiiOZ1br86VACTxTZerBBSfxHdIflpN0Dwrp16MoQSWrXGcjgv\n cYxXWoLRYwsXG/BKSGdekmKBuaEwS9VHaIUiPG+JQwCJWkb/cAFi3TPm8F1sjUrtGZRL\n HemJA3YYTqjpN+bU39tF3zkPzoa8e/ocbmP+yu3RpZgWyX/CpsuY+EGMLMi9B4/SvqRN\n Qc4LN/rp3M4UWxhF7lEEFwicy5hLChfD1aB8FRDnlC5qX+inXbIKCsmOBTLTfMKD797a\n Q8nHKqH3MvOIWRhtEqA7yjuwD5zDv7avAn+2jS6ySfhRs/pKplrdtEBH5ehn1v8QVCJa\n 38cA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1777515659; x=1778120459;\n h=in-reply-to:content-disposition:mime-version:references:reply-to\n :message-id:subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from\n :to:cc:subject:date:message-id:reply-to;\n bh=k31n70UW3aN+dr+eCON/dcK1cC8U/hNuFVuJYM3gpDs=;\n b=Wo3iYlodzZF2TbF1gz/WILBW6+E755S4ia0g0BzGPSRJ2BFEoaRBJJW0IbeRuCoj2Q\n G9M5cyQHffLFCMLU4BdCY2Zj0ObC5TkU2nMLEcvq5HgqjsXSzvhVa93JetLgHT57hZzn\n xF3PJGc7KMGJAr/zla+ddbe64cmn2epf5GbNTXB1IZz5c0qiA5hTiij3tjmLh3ynknqL\n wfftgyr+6AVgcBTuxiM2J/BfOdg25BX1COrDDWXhFs8bduB9NwSBxK7uUYtLKFR7mWyi\n Ua05/MmuWp0iv/P/KWvkfHvH29ZF6+2xRY7vWsLWwb35hZjv3t+0ywZ/VYwOvF8fANl+\n +Mng==","X-Forwarded-Encrypted":"i=1;\n AFNElJ8NXT9K5aCZK3hUlGKi06pw5Drw+ny7RbX3vHo2GtkvtDRWcwrbojNjuVnSzT6VGUvprVcs0HEswAg=@vger.kernel.org","X-Gm-Message-State":"AOJu0YzpoUt5MCP9MUhbwpNUAJueEvvyE54lKmdK5Uayn1UIfwT/Yr/q\n\tRIEy2XwqkNI6sIbO9oWYdIGdPGqjCPdcXdZOMv539yxPjiYC3kl1AuWGmpH0J6ekOk2dcBDDZ7R\n\t2wSRo","X-Gm-Gg":"AeBDieudZfpnpxUjzZ2il2Svq4RTyCaG2Tkb07T5TeX+PWKSfUKIir3vml1aCxwE4g0\n\txpViUV39JkF6Up+lm6dyPOhvDptQ6zZ7KgJd3GNTlPbQ10IFT+W8C5W1nSwOuQNRvs9W016KFtu\n\tMjUA0x/oroX7w1eGvINPiX9Wrc3//WpNAKnQFbYMImQwL8HqnzfN0TIBRntq6vZALPl+/tXbqPb\n\tQqmurbGo5UFu6Xn1iU0sVuHjpvfgES1SpMepxZztk8Fcekuuy0B6d+k3MkYxI6BaSAARu9GKzvk\n\tw1GZQH5S4QIeBgz5IGzlC0uvoq6vk1R6zh5UnjESaVrsPoWmAToEWGG+UNDZLtZ9lNNenYfTI36\n\tmnIYm+fpVi70/l3mMIIFN2Ul8hzpKQDLlK2l7FXLjGAlyZxe+8j+B+ACW34f730CZcwV2Vns98R\n\t8YllSnWgDWH9J32Rej5PfcWJ0tgV/AVcu9Ov7x5ehBD0HvChaPWQR4yp7F2ZSxOsBYifhXeB2UE\n\t+Ed3ZGPlP0vN5oGqbkLkXc+","X-Received":"by 2002:a05:6808:144c:b0:479:d779:353e with SMTP id\n 5614622812f47-47c5fd3d0b2mr623098b6e.24.1777515659036;\n Wed, 29 Apr 2026 19:20:59 -0700 (PDT)","Date":"Wed, 29 Apr 2026 21:20:53 -0500","From":"Corey Minyard ","To":"Tony Hutter ","Cc":"Lukas Wunner , Bjorn Helgaas ,\n\talok.a.tiwari@oracle.com, mariusz.tkaczyk@linux.intel.com,\n\tminyard@acm.org, linux-pci@vger.kernel.org,\n\topenipmi-developer@lists.sourceforge.net,\n\tLinux Kernel Mailing List ","Subject":"Re: [PATCH v8 RESEND] Introduce Cray ClusterStor E1000 NVMe slot LED\n driver","Message-ID":"","Reply-To":"corey@minyard.net","References":"<768409a2-0593-49bd-9065-e8b93c60d4ce@llnl.gov>","Precedence":"bulk","X-Mailing-List":"linux-pci@vger.kernel.org","List-Id":"","List-Subscribe":"","List-Unsubscribe":"","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<768409a2-0593-49bd-9065-e8b93c60d4ce@llnl.gov>"}}]