From patchwork Mon Sep 9 12:31:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159702 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46Rnf15r6dz9sCJ for ; Mon, 9 Sep 2019 22:34:01 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46Rnf11hqLzDqNC for ; Mon, 9 Sep 2019 22:34:01 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbk48LHzDqNC for ; Mon, 9 Sep 2019 22:32:01 +1000 (AEST) Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CNfvA100653 for ; Mon, 9 Sep 2019 08:31:59 -0400 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0b-001b2d01.pphosted.com with ESMTP id 2uwm9fqdtw-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:31:58 -0400 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:57 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:55 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVrIZ51446014 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:53 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 81C57AE068; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4E233AE058; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:39 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0028-0000-0000-00000399BF36 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0029-0000-0000-0000245C1E86 Message-Id: <20190909123151.21944-5-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 04/16] core/pci: Train link of PHB slots when hotplugging X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" The link of PHB slots must be trained after powering on. This can be done by calling the fundamental reset callback of the slot. We could force a reset for all the slots and have a common path in set_power_state(). But this patch only resets the PHB slot. Some slot implementations do a power cycle during fundamental reset, so calling a reset after powering on would repeat that operation. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- core/pci-opal.c | 130 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 108 insertions(+), 22 deletions(-) diff --git a/core/pci-opal.c b/core/pci-opal.c index 69bd65c6..6c070030 100644 --- a/core/pci-opal.c +++ b/core/pci-opal.c @@ -678,11 +678,111 @@ static void remove_slot_devices(struct pci_slot *slot) pci_remove_bus(phb, &pd->children); } +static void link_training_timer(struct timer *t, void *data, + uint64_t now __unused) +{ + struct pci_slot *slot = data; + struct phb *phb = slot->phb; + uint8_t link; + int64_t rc = 0; + + phb_lock(phb); + + rc = slot->ops.run_sm(slot); + if (rc < 0) + goto out; + if (rc > 0) { + schedule_timer(t, rc); + phb_unlock(phb); + return; + } + + if (slot->ops.get_link_state(slot, &link) != OPAL_SUCCESS) + link = 0; + if (!link) { + rc = OPAL_HARDWARE; + goto out; + } + + rescan_slot_devices(slot); +out: + opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, + slot->async_token, get_slot_phandle(slot), + slot->power_state, rc <= 0 ? rc : OPAL_BUSY); + phb_unlock(phb); +} + +static void link_up_timer(struct timer *t, void *data, uint64_t now __unused) +{ + struct pci_slot *slot = data; + struct phb *phb = slot->phb; + uint8_t link; + int64_t rc = OPAL_SUCCESS; + + phb_lock(phb); + if (slot->ops.get_link_state(slot, &link) != OPAL_SUCCESS) + link = 0; + if (!link) { + if (slot->retries-- == 0) { + rc = OPAL_BUSY; + goto out; + } else { + schedule_timer(t, msecs_to_tb(10)); + phb_unlock(phb); + return; + } + } + + rescan_slot_devices(slot); +out: + opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, + slot->async_token, get_slot_phandle(slot), + slot->power_state, rc <= 0 ? rc : OPAL_BUSY); + phb_unlock(phb); +} + +static bool training_needed(struct pci_slot *slot) +{ + struct phb *phb = slot->phb; + struct pci_device *pd = slot->pd; + + if ((pd && !pd->parent) || /* "real" PHB */ + (!pd && phb->phb_type == phb_type_npu_v2_opencapi)) + return true; + return false; +} + +static void wait_for_link_up_and_rescan(struct pci_slot *slot) +{ + int64_t rc; + + /* + * Links for PHB slots need to be retrained by triggering a + * fundamental reset. Other slots also need to be tested for + * readiness + */ + if (training_needed(slot)) { + rc = slot->ops.freset(slot); + if (rc < 0) { + opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, + slot->async_token, + get_slot_phandle(slot), + slot->power_state, + rc <= 0 ? rc : OPAL_BUSY); + return; + } + init_timer(&slot->timer, link_training_timer, slot); + } else { + init_timer(&slot->timer, link_up_timer, slot); + rc = 1; + } + schedule_timer(&slot->timer, rc); +} + static void set_power_timer(struct timer *t __unused, void *data, uint64_t now __unused) { struct pci_slot *slot = data; - uint8_t link; struct phb *phb = slot->phb; phb_lock(phb); @@ -699,9 +799,9 @@ static void set_power_timer(struct timer *t __unused, void *data, break; case PCI_SLOT_STATE_SPOWER_DONE: + pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); if (slot->power_state == OPAL_PCI_SLOT_POWER_OFF) { remove_slot_devices(slot); - pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, slot->async_token, get_slot_phandle(slot), OPAL_PCI_SLOT_POWER_OFF, OPAL_SUCCESS); @@ -709,23 +809,7 @@ static void set_power_timer(struct timer *t __unused, void *data, } /* Power on */ - if (slot->ops.get_link_state(slot, &link) != OPAL_SUCCESS) - link = 0; - if (link) { - rescan_slot_devices(slot); - pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); - opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, - slot->async_token, get_slot_phandle(slot), - OPAL_PCI_SLOT_POWER_ON, OPAL_SUCCESS); - } else if (slot->retries-- == 0) { - pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); - opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, - slot->async_token, get_slot_phandle(slot), - OPAL_PCI_SLOT_POWER_ON, OPAL_BUSY); - } else { - schedule_timer(&slot->timer, msecs_to_tb(10)); - } - + wait_for_link_up_and_rescan(slot); break; default: prlog(PR_ERR, "PCI SLOT %016llx: Unexpected state 0x%08x\n", @@ -808,10 +892,12 @@ static int64_t opal_pci_set_power_state(uint64_t async_token, init_timer(&slot->timer, set_power_timer, slot); schedule_timer(&slot->timer, msecs_to_tb(10)); } else if (rc == OPAL_SUCCESS) { - if (*state == OPAL_PCI_SLOT_POWER_OFF) + if (*state == OPAL_PCI_SLOT_POWER_OFF) { remove_slot_devices(slot); - else - rescan_slot_devices(slot); + } else { + wait_for_link_up_and_rescan(slot); + rc = OPAL_ASYNC_COMPLETION; + } } phb_unlock(phb);