From patchwork Tue Apr 18 08:53:32 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vasant Hegde X-Patchwork-Id: 751739 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3w6f8R0cw5z9s2s for ; Tue, 18 Apr 2017 18:54:47 +1000 (AEST) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3w6f8Q6WFYzDqHv for ; Tue, 18 Apr 2017 18:54:46 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3w6f7q0FmVzDqBP for ; Tue, 18 Apr 2017 18:54:14 +1000 (AEST) Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v3I8rvnw121500 for ; Tue, 18 Apr 2017 04:54:00 -0400 Received: from e28smtp04.in.ibm.com (e28smtp04.in.ibm.com [125.16.236.4]) by mx0a-001b2d01.pphosted.com with ESMTP id 29w4j9xg85-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 18 Apr 2017 04:53:59 -0400 Received: from localhost by e28smtp04.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 18 Apr 2017 14:23:41 +0530 Received: from d28relay09.in.ibm.com (9.184.220.160) by e28smtp04.in.ibm.com (192.168.1.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 18 Apr 2017 14:23:39 +0530 Received: from d28av03.in.ibm.com (d28av03.in.ibm.com [9.184.220.65]) by d28relay09.in.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v3I8rc1T21102680 for ; Tue, 18 Apr 2017 14:23:38 +0530 Received: from d28av03.in.ibm.com (localhost [127.0.0.1]) by d28av03.in.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id v3I8ralK000599 for ; Tue, 18 Apr 2017 14:23:38 +0530 Received: from hegdevasant.in.ibm.com ([9.109.223.130]) by d28av03.in.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id v3I8rY0a000529; Tue, 18 Apr 2017 14:23:36 +0530 From: Vasant Hegde To: skiboot@lists.ozlabs.org Date: Tue, 18 Apr 2017 14:23:32 +0530 X-Mailer: git-send-email 2.9.3 In-Reply-To: <20170418085332.9558-1-hegdevasant@linux.vnet.ibm.com> References: <20170418085332.9558-1-hegdevasant@linux.vnet.ibm.com> X-TM-AS-MML: disable x-cbid: 17041808-0012-0000-0000-000003E44315 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17041808-0013-0000-0000-00001B6AF47F Message-Id: <20170418085332.9558-6-hegdevasant@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-04-18_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1704180074 Subject: [Skiboot] [PATCH v4 5/5] SBE: Add timer support X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" SBE on P9 provides one shot programmable timer facility. We can use this to implement OPAL timers and hence limit the relieance on the Linux heartbeat (Similar to HW timer facility provided by SLW on P8). Design: - We will continue to run Linux heartbeat. - Each chip has SBE. Presently we are always scheduling timer on SBE on master chip. - Timer MBOX has two options (start/stop timer). We cannot modify inflight timer message. Hence we reschedule timer if new timer value is less than scheduled timer value. - SBE expects timeout value in milliseconds. We track timeout value in TB. Hence we convert tb to milliseconds before sending request to SBE. - We are not requesting ack/response from SBE for timer message. This will reduce unnecessary interrupts (as SBE send interrupt for ack/response). - Disabling SBE timer We expect SBE to send timer expiry interrupt whenever timer expires. We wait for few more millisecond (presently SBE_TIMER_DEFAULT_MS) before disabling timer. In future we can consider below alternative approaches: - Presently SBE timer disable is permanent (until we reboot system). SBE sends "I'm back" interrupt after reset. We can consider restarting timer after SBE reset. - Reset SBE and start timer again. - Each chip has SBE. On multi chip system we can try to schedule timer on different chip. Signed-off-by: Vasant Hegde CC: Benjamin Herrenschmidt --- core/interrupts.c | 3 +- core/timer.c | 16 ++++-- hw/sbe-p9.c | 151 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ include/sbe-p9.h | 6 +++ 4 files changed, 172 insertions(+), 4 deletions(-) diff --git a/core/interrupts.c b/core/interrupts.c index ce1df95..7986296 100644 --- a/core/interrupts.c +++ b/core/interrupts.c @@ -26,6 +26,7 @@ #include #include #include +#include /* ICP registers */ #define ICP_XIRR 0x4 /* 32-bit access */ @@ -458,7 +459,7 @@ static int64_t opal_handle_interrupt(uint32_t isn, __be64 *outstanding_event_mas is->ops->interrupt(is, isn); /* Check timers if SBE timer isn't working */ - if (!sbe_p8_timer_ok()) + if (!sbe_p8_timer_ok() && !sbe_timer_ok()) check_timers(true); /* Update output events */ diff --git a/core/timer.c b/core/timer.c index 2b4c921..f406f81 100644 --- a/core/timer.c +++ b/core/timer.c @@ -5,6 +5,7 @@ #include #include #include +#include #ifdef __TEST__ #define this_cpu() ((void *)-1) @@ -106,8 +107,12 @@ static void __schedule_timer_at(struct timer *t, uint64_t when) bail: /* Pick up the next timer and upddate the SBE HW timer */ lt = list_top(&timer_list, struct timer, link); - if (lt) - sbe_p8_update_timer_expiry(lt->target); + if (lt) { + if (proc_gen < proc_gen_p9) + sbe_p8_update_timer_expiry(lt->target); + else + sbe_update_timer_expiry(lt->target); + } } void schedule_timer_at(struct timer *t, uint64_t when) @@ -164,7 +169,10 @@ static void __check_poll_timers(uint64_t now) * arbitrarily 1us. */ if (t->running) { - sbe_p8_update_timer_expiry(now + usecs_to_tb(1)); + if (proc_gen < proc_gen_p9) + sbe_p8_update_timer_expiry(now + usecs_to_tb(1)); + else + sbe_update_timer_expiry(now + usecs_to_tb(1)); break; } @@ -262,6 +270,8 @@ void late_init_timers(void) */ if (platform.heartbeat_time) { heartbeat = platform.heartbeat_time(); + } else if (sbe_timer_ok()) { + heartbeat = HEARTBEAT_DEFAULT_MS * 10; } else if (sbe_p8_timer_ok() || fsp_present()) { heartbeat = HEARTBEAT_DEFAULT_MS * 10; } diff --git a/hw/sbe-p9.c b/hw/sbe-p9.c index 88474ac..359ebfb 100644 --- a/hw/sbe-p9.c +++ b/hw/sbe-p9.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include @@ -48,6 +49,25 @@ struct sbe { /* Default SBE chip ID */ static u32 sbe_default_chip_id = -1; +/* Is SBE timer running? */ +static bool sbe_has_timer = false; +static bool sbe_timer_in_progress = false; + +/* Inflight and next timer in TB */ +static uint64_t sbe_last_gen_stamp; +static uint64_t sbe_timer_target; + +/* Default timeout value */ +#define SBE_TIMER_DEFAULT_MS 10 +static uint64_t sbe_timer_inc_tb; + +/* Timer control message */ +static struct sbe_msg *timer_ctrl_msg; + +/* Forward declaration */ +static void sbe_timer_start(void); + + #define SBE_STATUS_PRI_SHIFT 0x30 #define SBE_STATUS_SEC_SHIFT 0x20 #define SBE_FFDC_PRESENT PPC_BIT(1) @@ -471,6 +491,13 @@ static void sbe_handle_response(struct sbe *sbe) sbe_msg_complete(sbe, msg); } +static inline void sbe_check_timer_expiry(void) +{ + check_timers(true); + sbe_timer_in_progress = false; + sbe_timer_start(); +} + void sbe_interrupt(uint32_t chip_id) { int rc; @@ -506,6 +533,8 @@ void sbe_interrupt(uint32_t chip_id) sbe_poke_queue(sbe); } unlock(&sbe->lock); + + sbe_check_timer_expiry(); goto clr_interrupt; } @@ -517,6 +546,10 @@ void sbe_interrupt(uint32_t chip_id) unlock(&sbe->lock); } + /* Timer expired */ + if (data & SBE_HOST_TIMER_EXPIRY) + sbe_check_timer_expiry(); + clr_interrupt: /* Clears all the bits */ rc = xscom_write(chip_id, PSU_HOST_DOORBELL_REG_AND, @@ -569,6 +602,121 @@ static void sbe_timeout_poll(void *user_data __unused) sbe_poke_queue(sbe); unlock(&sbe->lock); } + + /* + * Check if the timer is working. If at least SBE_TIMER_DEFAULT_MS + * milliseconds elapsed since last scheduled timer expiry. + */ + if (sbe_has_timer) { + if (tb_compare(now, sbe_last_gen_stamp + sbe_timer_inc_tb) + != TB_AAFTERB) + return; + + /* + * In some cases there will be a delay in calling OPAL interrupt + * handler routine (opal_handle_interrupt). In such cases its + * possible that SBE has responded, but OPAL didn't act on that. + * Hence check for SBE response before disabling timer. + */ + sbe_interrupt(sbe_default_chip_id); + + if (tb_compare(now, sbe_last_gen_stamp + sbe_timer_inc_tb) + != TB_AAFTERB) + return; + + prlog(PR_ERR, "Timer stuck, falling back to OPAL pollers.\n"); + prlog(PR_ERR, "You will likely have slower I2C and may have " + "experienced increased jitter.\n"); + sbe_has_timer = false; + } +} + +static void sbe_timer_start(void) +{ + int rc; + u32 ms = SBE_TIMER_DEFAULT_MS; + u64 tb_cnt, now = mftb(); + + if (!sbe_has_timer) + return; + + if (sbe_timer_in_progress) { + if (sbe_timer_target < now || + sbe_timer_target > sbe_last_gen_stamp) + return; + + /* Update timer control message */ + timer_ctrl_msg->reg[0] &= 0xffff; + timer_ctrl_msg->reg[0] |= ((u64)CONTROL_TIMER_STOP << 32); + + rc = sbe_sync_msg(sbe_default_chip_id, timer_ctrl_msg, false); + if (rc != OPAL_SUCCESS) { + /* + * Lets hope SBE will respond whenever original + * timer expires. + */ + prlog(PR_ERR, "Failed to stop timer [chip id = %x]\n", + sbe_default_chip_id); + return; + } + } + + if (now < sbe_timer_target && + sbe_timer_target < now + sbe_timer_inc_tb) { + /* Calculate how many ms from now, rounded up */ + tb_cnt = sbe_timer_target - now + msecs_to_tb(1) - 1; + ms = tb_to_msecs(tb_cnt); + } + + /* Update timer control message */ + timer_ctrl_msg->reg[0] &= 0xffff; + timer_ctrl_msg->reg[0] |= ((u64)CONTROL_TIMER_START << 32); + timer_ctrl_msg->reg[1] = ms; + + rc = sbe_sync_msg(sbe_default_chip_id, timer_ctrl_msg, false); + if (rc != OPAL_SUCCESS) { + prlog(PR_ERR, "Failed to start timer [chip id = %x]\n", + sbe_default_chip_id); + return; + } + + /* Update last scheduled timer value */ + sbe_last_gen_stamp = now + msecs_to_tb(ms); + + sbe_timer_in_progress = true; +} + +void sbe_update_timer_expiry(uint64_t new_target) +{ + if (!sbe_has_timer || new_target == sbe_timer_target) + return; + + sbe_timer_target = new_target; + sbe_timer_start(); +} + +/* Initialize SBE timer */ +static void sbe_timer_init(void) +{ + /* Do not request ack/response for timer message */ + timer_ctrl_msg = sbe_mkmsg(SBE_CMD_CONTROL_TIMER, + CONTROL_TIMER_START, 0, 0, 0); + assert(timer_ctrl_msg); + + sbe_has_timer = true; + sbe_timer_target = mftb(); + sbe_last_gen_stamp = ~0ull; + sbe_timer_inc_tb = msecs_to_tb(SBE_TIMER_DEFAULT_MS); + + prlog(PR_INFO, "Timer facility on chip %x, default resolution %dms\n", + sbe_default_chip_id, SBE_TIMER_DEFAULT_MS); + + sbe_timer_start(); +} + +bool sbe_timer_ok(void) +{ + return sbe_has_timer; } void sbe_init(void) @@ -605,6 +753,9 @@ void sbe_init(void) return; } + /* Initiate SBE timer */ + sbe_timer_init(); + /* Initiate SBE timeout poller */ opal_add_poller(sbe_timeout_poll, NULL); } diff --git a/include/sbe-p9.h b/include/sbe-p9.h index 69da8eb..3017003 100644 --- a/include/sbe-p9.h +++ b/include/sbe-p9.h @@ -225,4 +225,10 @@ extern void sbe_init(void); /* SBE interrupt */ extern void sbe_interrupt(uint32_t chip_id); +/* Is SBE timer available ? */ +extern bool sbe_timer_ok(void); + +/* Update SBE timer expiry */ +extern void sbe_update_timer_expiry(uint64_t new_target); + #endif /* __SBE_P9_H */