From patchwork Wed Nov 25 22:15:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Reed X-Patchwork-Id: 1406298 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4ChFc06cblz9sRK; Thu, 26 Nov 2020 09:16:48 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1ki35f-0005qx-Tb; Wed, 25 Nov 2020 22:16:43 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ki35C-0005UB-U1 for kernel-team@lists.ubuntu.com; Wed, 25 Nov 2020 22:16:14 +0000 Received: from 104-54-219-103.lightspeed.austtx.sbcglobal.net ([104.54.219.103] helo=canonical.com) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1ki35A-00021m-BL for kernel-team@lists.ubuntu.com; Wed, 25 Nov 2020 22:16:12 +0000 From: Michael Reed To: kernel-team@lists.ubuntu.com Subject: [PATCH 12/14] scsi: mpt3sas: Handle vSES vphy object during HBA reset Date: Wed, 25 Nov 2020 16:15:46 -0600 Message-Id: <20201125221548.24127-13-michael.reed@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201125221548.24127-2-michael.reed@canonical.com> References: <20201125221548.24127-2-michael.reed@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Sreekanth Reddy During HBA reset the Port ID of vSES device may change. As a result, it is necessary to refresh virtual_phy objects after reset. Each Port's vphy_list table needs to be updated after updating the HBA port table. The algorithm is as follows: - Loop over each port entry from HBA port table * Loop over each virtual phy entry from port's vphys_list table - Mark virtual phy entry as dirty by setting dirty bit in virtual phy entry's flags field - Read SASIOUnitPage0 page - Loop over each HBA Phy's Phy data from SASIOUnitPage0 * If phy's remote attached device is not SES device then continue with processing next HBA Phy's Phy data; * Read SASPhyPage0 data for this Phy number and determine whether current phy is a virtual phy or not. If it is not a virtual phy then continue with next Phy data; * Get the current phy's remote attached vSES device's SAS Address; * Loop over each port entry from HBA port table - If Port's vphys_mask field is zero then continue with next Port entry, - Loop over each virtual phy entry from Port's vphy_list table - If the current phy's remote SAS Address is different from virtual phy entry's SAS Address then continue with next virtual phy entry, - Set bit corresponding to current phy number in virtual phy entry's phy_mask field, - Get the HBA port table's Port entry corresponding to Phy data's 'Port' value, * If there is no Port entry corresponding to Phy data's 'Port' value in HBA port table then create a new port entry and add it to HBA port table. - If this retrieved Port entry is the same as the current Port entry then don't do anything, just clear the dirty bit from virtual phy entry's flag field and continue with processing next HBA Phy's Phy data. - If this retrieved Port entry is different from the current Port entry then move the current virtual phy entry from current Port's vphys_list to retrieved Port entry's vphys_list. * Clear current phy bit in current Port entry's vphys_mask and set the current phy bit in the retrieved Port entry's vphys_mask field. * Clear the dirty bit from virtual phy entry's flag field and continue with next HBA Phy's Phy data. - Delete the 'virtual phy' entries and HBA's 'Port table' entries which are still marked as 'dirty'. Link: https://lore.kernel.org/r/20201027130847.9962-13-sreekanth.reddy@broadcom.com Signed-off-by: Sreekanth Reddy Signed-off-by: Martin K. Petersen (cherry picked from commit ffa381d6373b10e83dbdac425fb72affc64084f3) Signed-off-by: Michael Reed BugLink: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1899802 --- drivers/scsi/mpt3sas/mpt3sas_scsih.c | 233 +++++++++++++++++++++++++++ 1 file changed, 233 insertions(+) diff --git a/drivers/scsi/mpt3sas/mpt3sas_scsih.c b/drivers/scsi/mpt3sas/mpt3sas_scsih.c index 0cceb0676236..31209834bce6 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_scsih.c +++ b/drivers/scsi/mpt3sas/mpt3sas_scsih.c @@ -5648,6 +5648,204 @@ _scsih_io_done(struct MPT3SAS_ADAPTER *ioc, u16 smid, u8 msix_index, u32 reply) return 0; } +/** + * _scsih_update_vphys_after_reset - update the Port's + * vphys_list after reset + * @ioc: per adapter object + * + * Returns nothing. + */ +static void +_scsih_update_vphys_after_reset(struct MPT3SAS_ADAPTER *ioc) +{ + u16 sz, ioc_status; + int i; + Mpi2ConfigReply_t mpi_reply; + Mpi2SasIOUnitPage0_t *sas_iounit_pg0 = NULL; + u16 attached_handle; + u64 attached_sas_addr; + u8 found = 0, port_id; + Mpi2SasPhyPage0_t phy_pg0; + struct hba_port *port, *port_next, *mport; + struct virtual_phy *vphy, *vphy_next; + struct _sas_device *sas_device; + + /* + * Mark all the vphys objects as dirty. + */ + list_for_each_entry_safe(port, port_next, + &ioc->port_table_list, list) { + if (!port->vphys_mask) + continue; + list_for_each_entry_safe(vphy, vphy_next, + &port->vphys_list, list) { + vphy->flags |= MPT_VPHY_FLAG_DIRTY_PHY; + } + } + + /* + * Read SASIOUnitPage0 to get each HBA Phy's data. + */ + sz = offsetof(Mpi2SasIOUnitPage0_t, PhyData) + + (ioc->sas_hba.num_phys * sizeof(Mpi2SasIOUnit0PhyData_t)); + sas_iounit_pg0 = kzalloc(sz, GFP_KERNEL); + if (!sas_iounit_pg0) { + ioc_err(ioc, "failure at %s:%d/%s()!\n", + __FILE__, __LINE__, __func__); + return; + } + if ((mpt3sas_config_get_sas_iounit_pg0(ioc, &mpi_reply, + sas_iounit_pg0, sz)) != 0) + goto out; + ioc_status = le16_to_cpu(mpi_reply.IOCStatus) & MPI2_IOCSTATUS_MASK; + if (ioc_status != MPI2_IOCSTATUS_SUCCESS) + goto out; + /* + * Loop over each HBA Phy. + */ + for (i = 0; i < ioc->sas_hba.num_phys; i++) { + /* + * Check whether Phy's Negotiation Link Rate is > 1.5G or not. + */ + if ((sas_iounit_pg0->PhyData[i].NegotiatedLinkRate >> 4) < + MPI2_SAS_NEG_LINK_RATE_1_5) + continue; + /* + * Check whether Phy is connected to SEP device or not, + * if it is SEP device then read the Phy's SASPHYPage0 data to + * determine whether Phy is a virtual Phy or not. if it is + * virtual phy then it is conformed that the attached remote + * device is a HBA's vSES device. + */ + if (!(le32_to_cpu( + sas_iounit_pg0->PhyData[i].ControllerPhyDeviceInfo) & + MPI2_SAS_DEVICE_INFO_SEP)) + continue; + + if ((mpt3sas_config_get_phy_pg0(ioc, &mpi_reply, &phy_pg0, + i))) { + ioc_err(ioc, "failure at %s:%d/%s()!\n", + __FILE__, __LINE__, __func__); + continue; + } + + if (!(le32_to_cpu(phy_pg0.PhyInfo) & + MPI2_SAS_PHYINFO_VIRTUAL_PHY)) + continue; + /* + * Get the vSES device's SAS Address. + */ + attached_handle = le16_to_cpu( + sas_iounit_pg0->PhyData[i].AttachedDevHandle); + if (_scsih_get_sas_address(ioc, attached_handle, + &attached_sas_addr) != 0) { + ioc_err(ioc, "failure at %s:%d/%s()!\n", + __FILE__, __LINE__, __func__); + continue; + } + + found = 0; + port = port_next = NULL; + /* + * Loop over each virtual_phy object from + * each port's vphys_list. + */ + list_for_each_entry_safe(port, + port_next, &ioc->port_table_list, list) { + if (!port->vphys_mask) + continue; + list_for_each_entry_safe(vphy, vphy_next, + &port->vphys_list, list) { + /* + * Continue with next virtual_phy object + * if the object is not marked as dirty. + */ + if (!(vphy->flags & MPT_VPHY_FLAG_DIRTY_PHY)) + continue; + + /* + * Continue with next virtual_phy object + * if the object's SAS Address is not equals + * to current Phy's vSES device SAS Address. + */ + if (vphy->sas_address != attached_sas_addr) + continue; + /* + * Enable current Phy number bit in object's + * phy_mask field. + */ + if (!(vphy->phy_mask & (1 << i))) + vphy->phy_mask = (1 << i); + /* + * Get hba_port object from hba_port table + * corresponding to current phy's Port ID. + * if there is no hba_port object corresponding + * to Phy's Port ID then create a new hba_port + * object & add to hba_port table. + */ + port_id = sas_iounit_pg0->PhyData[i].Port; + mport = mpt3sas_get_port_by_id(ioc, port_id, 1); + if (!mport) { + mport = kzalloc( + sizeof(struct hba_port), GFP_KERNEL); + if (!mport) + break; + mport->port_id = port_id; + ioc_info(ioc, + "%s: hba_port entry: %p, port: %d is added to hba_port list\n", + __func__, mport, mport->port_id); + list_add_tail(&mport->list, + &ioc->port_table_list); + } + /* + * If mport & port pointers are not pointing to + * same hba_port object then it means that vSES + * device's Port ID got changed after reset and + * hence move current virtual_phy object from + * port's vphys_list to mport's vphys_list. + */ + if (port != mport) { + if (!mport->vphys_mask) + INIT_LIST_HEAD( + &mport->vphys_list); + mport->vphys_mask |= (1 << i); + port->vphys_mask &= ~(1 << i); + list_move(&vphy->list, + &mport->vphys_list); + sas_device = mpt3sas_get_sdev_by_addr( + ioc, attached_sas_addr, port); + if (sas_device) + sas_device->port = mport; + } + /* + * Earlier while updating the hba_port table, + * it is determined that there is no other + * direct attached device with mport's Port ID, + * Hence mport was marked as dirty. Only vSES + * device has this Port ID, so unmark the mport + * as dirt. + */ + if (mport->flags & HBA_PORT_FLAG_DIRTY_PORT) { + mport->sas_address = 0; + mport->phy_mask = 0; + mport->flags &= + ~HBA_PORT_FLAG_DIRTY_PORT; + } + /* + * Unmark current virtual_phy object as dirty. + */ + vphy->flags &= ~MPT_VPHY_FLAG_DIRTY_PHY; + found = 1; + break; + } + if (found) + break; + } + } +out: + kfree(sas_iounit_pg0); +} + /** * _scsih_get_port_table_after_reset - Construct temporary port table * @ioc: per adapter object @@ -5868,6 +6066,39 @@ _scsih_add_or_del_phys_from_existing_port(struct MPT3SAS_ADAPTER *ioc, } } +/** + * _scsih_del_dirty_vphy - delete virtual_phy objects marked as dirty. + * @ioc: per adapter object + * + * Returns nothing. + */ +static void +_scsih_del_dirty_vphy(struct MPT3SAS_ADAPTER *ioc) +{ + struct hba_port *port, *port_next; + struct virtual_phy *vphy, *vphy_next; + + list_for_each_entry_safe(port, port_next, + &ioc->port_table_list, list) { + if (!port->vphys_mask) + continue; + list_for_each_entry_safe(vphy, vphy_next, + &port->vphys_list, list) { + if (vphy->flags & MPT_VPHY_FLAG_DIRTY_PHY) { + drsprintk(ioc, ioc_info(ioc, + "Deleting vphy %p entry from port id: %d\t, Phy_mask 0x%08x\n", + vphy, port->port_id, + vphy->phy_mask)); + port->vphys_mask &= ~vphy->phy_mask; + list_del(&vphy->list); + kfree(vphy); + } + } + if (!port->vphys_mask && !port->sas_address) + port->flags |= HBA_PORT_FLAG_DIRTY_PORT; + } +} + /** * _scsih_del_dirty_port_entries - delete dirty port entries from port list * after host reset @@ -10048,6 +10279,7 @@ mpt3sas_scsih_reset_done_handler(struct MPT3SAS_ADAPTER *ioc) if ((!ioc->is_driver_loading) && !(disable_discovery > 0 && !ioc->sas_hba.num_phys)) { _scsih_sas_port_refresh(ioc); + _scsih_update_vphys_after_reset(ioc); _scsih_prep_device_scan(ioc); _scsih_create_enclosure_list_after_reset(ioc); _scsih_search_responding_sas_devices(ioc); @@ -10093,6 +10325,7 @@ _mpt3sas_fw_work(struct MPT3SAS_ADAPTER *ioc, struct fw_event_work *fw_event) ssleep(1); } _scsih_remove_unresponding_devices(ioc); + _scsih_del_dirty_vphy(ioc); _scsih_del_dirty_port_entries(ioc); _scsih_scan_for_devices_after_reset(ioc); _scsih_set_nvme_max_shutdown_latency(ioc);