From patchwork Mon Apr 16 11:48:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vasant Hegde X-Patchwork-Id: 898603 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40Pmz10hjPz9s08 for ; Mon, 16 Apr 2018 21:55:09 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40Pmz06SKDzF1qc for ; Mon, 16 Apr 2018 21:55:08 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=hegdevasant@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40Pmr23JjdzF1s3 for ; Mon, 16 Apr 2018 21:49:06 +1000 (AEST) Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w3GBiFef102197 for ; Mon, 16 Apr 2018 07:49:03 -0400 Received: from e06smtp11.uk.ibm.com (e06smtp11.uk.ibm.com [195.75.94.107]) by mx0b-001b2d01.pphosted.com with ESMTP id 2hcswhuw2f-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Mon, 16 Apr 2018 07:49:02 -0400 Received: from localhost by e06smtp11.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 16 Apr 2018 12:49:01 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp11.uk.ibm.com (192.168.101.141) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 16 Apr 2018 12:48:59 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w3GBmwJv58261722; Mon, 16 Apr 2018 11:48:58 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 022BFAE05A; Mon, 16 Apr 2018 12:38:50 +0100 (BST) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8D07DAE058; Mon, 16 Apr 2018 12:38:48 +0100 (BST) Received: from hegdevasant.in.ibm.com (unknown [9.109.198.237]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 16 Apr 2018 12:38:48 +0100 (BST) From: Vasant Hegde To: skiboot@lists.ozlabs.org Date: Mon, 16 Apr 2018 17:18:47 +0530 X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180416114848.19618-1-hegdevasant@linux.vnet.ibm.com> References: <20180416114848.19618-1-hegdevasant@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18041611-0040-0000-0000-0000044E1032 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18041611-0041-0000-0000-000020F2516F Message-Id: <20180416114848.19618-2-hegdevasant@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-04-16_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804160112 Subject: [Skiboot] [PATCH v7 1/2] Move P8 timer code to separate file X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Lets move P8 timer support code from slw.c to sbe-p8.c (as suggested by Ben). There is a difference between timer support in P8 and P9. Hence I think it makes sense to name it as sbe-p8.c. Note that this is pure code movement and renaming functions/variables. No functionality changes. Suggested-by: Benjamin Herrenschmidt Signed-off-by: Vasant Hegde --- core/interrupts.c | 5 +- core/test/run-timer.c | 2 +- core/timer.c | 9 ++- hw/Makefile.inc | 2 +- hw/sbe-p8.c | 203 ++++++++++++++++++++++++++++++++++++++++++++++++++ hw/slw.c | 185 +-------------------------------------------- include/sbe-p8.h | 28 +++++++ include/skiboot.h | 6 -- 8 files changed, 243 insertions(+), 197 deletions(-) create mode 100644 hw/sbe-p8.c create mode 100644 include/sbe-p8.h diff --git a/core/interrupts.c b/core/interrupts.c index 50ff77aa5..4452511f0 100644 --- a/core/interrupts.c +++ b/core/interrupts.c @@ -25,6 +25,7 @@ #include #include #include +#include /* ICP registers */ #define ICP_XIRR 0x4 /* 32-bit access */ @@ -487,8 +488,8 @@ static int64_t opal_handle_interrupt(uint32_t isn, __be64 *outstanding_event_mas /* Run it */ is->ops->interrupt(is, isn); - /* Check timers if SLW timer isn't working */ - if (!slw_timer_ok()) + /* Check timers if SBE timer isn't working */ + if (!p8_sbe_timer_ok()) check_timers(true); /* Update output events */ diff --git a/core/test/run-timer.c b/core/test/run-timer.c index e45cf6398..986af28d8 100644 --- a/core/test/run-timer.c +++ b/core/test/run-timer.c @@ -47,7 +47,7 @@ static void expiry(struct timer *t, void *data, uint64_t now) count--; } -void slw_update_timer_expiry(uint64_t new_target) +void p8_sbe_update_timer_expiry(uint64_t new_target) { (void)new_target; /* FIXME: do intersting SLW timer sim */ diff --git a/core/timer.c b/core/timer.c index b10a56464..21f62a492 100644 --- a/core/timer.c +++ b/core/timer.c @@ -4,6 +4,7 @@ #include #include #include +#include #ifdef __TEST__ #define this_cpu() ((void *)-1) @@ -109,7 +110,7 @@ static void __schedule_timer_at(struct timer *t, uint64_t when) /* Pick up the next timer and upddate the SBE HW timer */ lt = list_top(&timer_list, struct timer, link); if (lt) - slw_update_timer_expiry(lt->target); + p8_sbe_update_timer_expiry(lt->target); } void schedule_timer_at(struct timer *t, uint64_t when) @@ -166,7 +167,7 @@ static void __check_poll_timers(uint64_t now) * arbitrarily 1us. */ if (t->running) { - slw_update_timer_expiry(now + usecs_to_tb(1)); + p8_sbe_update_timer_expiry(now + usecs_to_tb(1)); break; } @@ -257,7 +258,7 @@ void late_init_timers(void) * * If a platform quirk exists, use that, else use the default. * - * If we have an SLW timer facility, we run this 10 times slower, + * If we have an SBE timer facility, we run this 10 times slower, * we could possibly completely get rid of it. * * We use a value in milliseconds, we don't want this to ever be @@ -265,7 +266,7 @@ void late_init_timers(void) */ if (platform.heartbeat_time) { heartbeat = platform.heartbeat_time(); - } else if (slw_timer_ok() || fsp_present()) { + } else if (p8_sbe_timer_ok() || fsp_present()) { heartbeat = HEARTBEAT_DEFAULT_MS * 10; } diff --git a/hw/Makefile.inc b/hw/Makefile.inc index 15bf5cf5d..4dec98670 100644 --- a/hw/Makefile.inc +++ b/hw/Makefile.inc @@ -9,7 +9,7 @@ HW_OBJS += dts.o lpc-rtc.o npu.o npu-hw-procedures.o xive.o phb4.o HW_OBJS += fake-nvram.o lpc-mbox.o npu2.o npu2-hw-procedures.o HW_OBJS += npu2-common.o phys-map.o sbe-p9.o capp.o occ-sensor.o vas.o HW_OBJS += npu2-common.o npu2-opencapi.o phys-map.o sbe-p9.o capp.o occ-sensor.o -HW_OBJS += vas.o +HW_OBJS += vas.o sbe-p8.o HW=hw/built-in.a # FIXME hack this for now diff --git a/hw/sbe-p8.c b/hw/sbe-p8.c new file mode 100644 index 000000000..58049c715 --- /dev/null +++ b/hw/sbe-p8.c @@ -0,0 +1,203 @@ +/* Copyright 2013-2018 IBM Corp. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + * implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include +#include +#include +#include + +/* SLW timer related stuff */ +static bool sbe_has_timer; +static uint64_t sbe_timer_inc; +static uint64_t sbe_timer_target; +static uint32_t sbe_timer_chip; +static uint64_t sbe_last_gen; +static uint64_t sbe_last_gen_stamp; + +static void p8_sbe_dump_timer_ffdc(void) +{ + uint64_t i, val; + int64_t rc; + + static const uint32_t dump_regs[] = { + 0xe0000, 0xe0001, 0xe0002, 0xe0003, + 0xe0004, 0xe0005, 0xe0006, 0xe0007, + 0xe0008, 0xe0009, 0xe000a, 0xe000b, + 0xe000c, 0xe000d, 0xe000e, 0xe000f, + 0xe0010, 0xe0011, 0xe0012, 0xe0013, + 0xe0014, 0xe0015, 0xe0016, 0xe0017, + 0xe0018, 0xe0019, + 0x5001c, + 0x50038, 0x50039, 0x5003a, 0x5003b + }; + + /** + * @fwts-label SLWRegisterDump + * @fwts-advice An error condition occurred in sleep/winkle + * engines timer state machine. Dumping debug information to + * root-cause. OPAL/skiboot may be stuck on some operation that + * requires SLW timer state machine (e.g. core powersaving) + */ + prlog(PR_DEBUG, "SLW: Register state:\n"); + + for (i = 0; i < ARRAY_SIZE(dump_regs); i++) { + uint32_t reg = dump_regs[i]; + rc = xscom_read(sbe_timer_chip, reg, &val); + if (rc) { + prlog(PR_DEBUG, "SLW: XSCOM error %lld reading" + " reg 0x%x\n", rc, reg); + break; + } + prlog(PR_DEBUG, "SLW: %5x = %016llx\n", reg, val); + } +} + +/* This is called with the timer lock held, so there is no + * issue with re-entrancy or concurrence + */ +void p8_sbe_update_timer_expiry(uint64_t new_target) +{ + uint64_t count, gen, gen2, req, now = mftb(); + int64_t rc; + + if (!sbe_has_timer || new_target == sbe_timer_target) + return; + + sbe_timer_target = new_target; + + /* Calculate how many increments from now, rounded up */ + if (now < new_target) + count = (new_target - now + sbe_timer_inc - 1) / sbe_timer_inc; + else + count = 1; + + /* Max counter is 24-bit */ + if (count > 0xffffff) + count = 0xffffff; + /* Fabricate update request */ + req = (1ull << 63) | (count << 32); + + prlog(PR_TRACE, "SLW: TMR expiry: 0x%llx, req: %016llx\n", count, req); + + do { + /* Grab generation and spin if odd */ + _xscom_lock(); + for (;;) { + rc = _xscom_read(sbe_timer_chip, 0xE0006, &gen, false); + if (rc) { + prerror("SLW: Error %lld reading tmr gen " + " count\n", rc); + _xscom_unlock(); + return; + } + if (!(gen & 1)) + break; + if (tb_compare(now + msecs_to_tb(1), mftb()) == TB_ABEFOREB) { + /** + * @fwts-label SLWTimerStuck + * @fwts-advice The SLeep/Winkle Engine (SLW) + * failed to increment the generation number + * within our timeout period (it *should* have + * done so within ~10us, not >1ms. OPAL uses + * the SLW timer to schedule some operations, + * but can fall back to the (much less frequent + * OPAL poller, which although does not affect + * functionality, runs *much* less frequently. + * This could have the effect of slow I2C + * operations (for example). It may also mean + * that you *had* an increase in jitter, due + * to slow interactions with SLW. + * This error may also occur if the machine + * is connected to via soft FSI. + */ + prerror("SLW: timer stuck, falling back to OPAL pollers. You will likely have slower I2C and may have experienced increased jitter.\n"); + prlog(PR_DEBUG, "SLW: Stuck with odd generation !\n"); + _xscom_unlock(); + sbe_has_timer = false; + p8_sbe_dump_timer_ffdc(); + return; + } + } + + rc = _xscom_write(sbe_timer_chip, 0x5003A, req, false); + if (rc) { + prerror("SLW: Error %lld writing tmr request\n", rc); + _xscom_unlock(); + return; + } + + /* Re-check gen count */ + rc = _xscom_read(sbe_timer_chip, 0xE0006, &gen2, false); + if (rc) { + prerror("SLW: Error %lld re-reading tmr gen " + " count\n", rc); + _xscom_unlock(); + return; + } + _xscom_unlock(); + } while(gen != gen2); + + /* Check if the timer is working. If at least 1ms has elapsed + * since the last call to this function, check that the gen + * count has changed + */ + if (tb_compare(sbe_last_gen_stamp + msecs_to_tb(1), now) + == TB_ABEFOREB) { + if (sbe_last_gen == gen) { + prlog(PR_ERR, + "SLW: Timer appears to not be running !\n"); + sbe_has_timer = false; + p8_sbe_dump_timer_ffdc(); + } + sbe_last_gen = gen; + sbe_last_gen_stamp = mftb(); + } + + prlog(PR_TRACE, "SLW: gen: %llx\n", gen); +} + +bool p8_sbe_timer_ok(void) +{ + return sbe_has_timer; +} + +void p8_sbe_init_timer(void) +{ + struct dt_node *np; + int64_t rc; + uint32_t tick_us; + + np = dt_find_compatible_node(dt_root, NULL, "ibm,power8-sbe-timer"); + if (!np) + return; + + sbe_timer_chip = dt_get_chip_id(np); + tick_us = dt_prop_get_u32(np, "tick-time-us"); + sbe_timer_inc = usecs_to_tb(tick_us); + sbe_timer_target = ~0ull; + + rc = xscom_read(sbe_timer_chip, 0xE0006, &sbe_last_gen); + if (rc) { + prerror("SLW: Error %lld reading tmr gen count\n", rc); + return; + } + sbe_last_gen_stamp = mftb(); + + prlog(PR_INFO, "SLW: Timer facility on chip %d, resolution %dus\n", + sbe_timer_chip, tick_us); + sbe_has_timer = true; +} diff --git a/hw/slw.c b/hw/slw.c index 3f9abaad2..2dd58c191 100644 --- a/hw/slw.c +++ b/hw/slw.c @@ -30,6 +30,7 @@ #include #include #include +#include #include #include @@ -44,14 +45,6 @@ static bool slw_current_le = false; enum wakeup_engine_states wakeup_engine_state = WAKEUP_ENGINE_NOT_PRESENT; bool has_deep_states = false; -/* SLW timer related stuff */ -static bool slw_has_timer; -static uint64_t slw_timer_inc; -static uint64_t slw_timer_target; -static uint32_t slw_timer_chip; -static uint64_t slw_last_gen; -static uint64_t slw_last_gen_stamp; - DEFINE_LOG_ENTRY(OPAL_RC_SLW_INIT, OPAL_PLATFORM_ERR_EVT, OPAL_SLW, OPAL_PLATFORM_FIRMWARE, OPAL_PREDICTIVE_ERR_GENERAL, OPAL_NA); @@ -1574,180 +1567,6 @@ int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val) opal_call(OPAL_SLW_SET_REG, opal_slw_set_reg, 3); -static void slw_dump_timer_ffdc(void) -{ - uint64_t i, val; - int64_t rc; - - static const uint32_t dump_regs[] = { - 0xe0000, 0xe0001, 0xe0002, 0xe0003, - 0xe0004, 0xe0005, 0xe0006, 0xe0007, - 0xe0008, 0xe0009, 0xe000a, 0xe000b, - 0xe000c, 0xe000d, 0xe000e, 0xe000f, - 0xe0010, 0xe0011, 0xe0012, 0xe0013, - 0xe0014, 0xe0015, 0xe0016, 0xe0017, - 0xe0018, 0xe0019, - 0x5001c, - 0x50038, 0x50039, 0x5003a, 0x5003b - }; - - /** - * @fwts-label SLWRegisterDump - * @fwts-advice An error condition occurred in sleep/winkle - * engines timer state machine. Dumping debug information to - * root-cause. OPAL/skiboot may be stuck on some operation that - * requires SLW timer state machine (e.g. core powersaving) - */ - prlog(PR_DEBUG, "SLW: Register state:\n"); - - for (i = 0; i < ARRAY_SIZE(dump_regs); i++) { - uint32_t reg = dump_regs[i]; - rc = xscom_read(slw_timer_chip, reg, &val); - if (rc) { - prlog(PR_DEBUG, "SLW: XSCOM error %lld reading" - " reg 0x%x\n", rc, reg); - break; - } - prlog(PR_DEBUG, "SLW: %5x = %016llx\n", reg, val); - } -} - -/* This is called with the timer lock held, so there is no - * issue with re-entrancy or concurrence - */ -void slw_update_timer_expiry(uint64_t new_target) -{ - uint64_t count, gen, gen2, req, now = mftb(); - int64_t rc; - - if (!slw_has_timer || new_target == slw_timer_target) - return; - - slw_timer_target = new_target; - - /* Calculate how many increments from now, rounded up */ - if (now < new_target) - count = (new_target - now + slw_timer_inc - 1) / slw_timer_inc; - else - count = 1; - - /* Max counter is 24-bit */ - if (count > 0xffffff) - count = 0xffffff; - /* Fabricate update request */ - req = (1ull << 63) | (count << 32); - - prlog(PR_TRACE, "SLW: TMR expiry: 0x%llx, req: %016llx\n", count, req); - - do { - /* Grab generation and spin if odd */ - _xscom_lock(); - for (;;) { - rc = _xscom_read(slw_timer_chip, 0xE0006, &gen, false); - if (rc) { - prerror("SLW: Error %lld reading tmr gen " - " count\n", rc); - _xscom_unlock(); - return; - } - if (!(gen & 1)) - break; - if (tb_compare(now + msecs_to_tb(1), mftb()) == TB_ABEFOREB) { - /** - * @fwts-label SLWTimerStuck - * @fwts-advice The SLeep/Winkle Engine (SLW) - * failed to increment the generation number - * within our timeout period (it *should* have - * done so within ~10us, not >1ms. OPAL uses - * the SLW timer to schedule some operations, - * but can fall back to the (much less frequent - * OPAL poller, which although does not affect - * functionality, runs *much* less frequently. - * This could have the effect of slow I2C - * operations (for example). It may also mean - * that you *had* an increase in jitter, due - * to slow interactions with SLW. - * This error may also occur if the machine - * is connected to via soft FSI. - */ - prerror("SLW: timer stuck, falling back to OPAL pollers. You will likely have slower I2C and may have experienced increased jitter.\n"); - prlog(PR_DEBUG, "SLW: Stuck with odd generation !\n"); - _xscom_unlock(); - slw_has_timer = false; - slw_dump_timer_ffdc(); - return; - } - } - - rc = _xscom_write(slw_timer_chip, 0x5003A, req, false); - if (rc) { - prerror("SLW: Error %lld writing tmr request\n", rc); - _xscom_unlock(); - return; - } - - /* Re-check gen count */ - rc = _xscom_read(slw_timer_chip, 0xE0006, &gen2, false); - if (rc) { - prerror("SLW: Error %lld re-reading tmr gen " - " count\n", rc); - _xscom_unlock(); - return; - } - _xscom_unlock(); - } while(gen != gen2); - - /* Check if the timer is working. If at least 1ms has elapsed - * since the last call to this function, check that the gen - * count has changed - */ - if (tb_compare(slw_last_gen_stamp + msecs_to_tb(1), now) - == TB_ABEFOREB) { - if (slw_last_gen == gen) { - prlog(PR_ERR, - "SLW: Timer appears to not be running !\n"); - slw_has_timer = false; - slw_dump_timer_ffdc(); - } - slw_last_gen = gen; - slw_last_gen_stamp = mftb(); - } - - prlog(PR_TRACE, "SLW: gen: %llx\n", gen); -} - -bool slw_timer_ok(void) -{ - return slw_has_timer; -} - -static void slw_init_timer(void) -{ - struct dt_node *np; - int64_t rc; - uint32_t tick_us; - - np = dt_find_compatible_node(dt_root, NULL, "ibm,power8-sbe-timer"); - if (!np) - return; - - slw_timer_chip = dt_get_chip_id(np); - tick_us = dt_prop_get_u32(np, "tick-time-us"); - slw_timer_inc = usecs_to_tb(tick_us); - slw_timer_target = ~0ull; - - rc = xscom_read(slw_timer_chip, 0xE0006, &slw_last_gen); - if (rc) { - prerror("SLW: Error %lld reading tmr gen count\n", rc); - return; - } - slw_last_gen_stamp = mftb(); - - prlog(PR_INFO, "SLW: Timer facility on chip %d, resolution %dus\n", - slw_timer_chip, tick_us); - slw_has_timer = true; -} - void slw_init(void) { struct proc_chip *chip; @@ -1764,7 +1583,7 @@ void slw_init(void) if (wakeup_engine_state == WAKEUP_ENGINE_PRESENT) slw_late_init_p8(chip); } - slw_init_timer(); + p8_sbe_init_timer(); } else if (proc_gen == proc_gen_p9) { for_each_chip(chip) { slw_init_chip_p9(chip); diff --git a/include/sbe-p8.h b/include/sbe-p8.h new file mode 100644 index 000000000..785c2cd8c --- /dev/null +++ b/include/sbe-p8.h @@ -0,0 +1,28 @@ +/* Copyright 2013-2018 IBM Corp. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + * implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef __SBE_P8_H +#define __SBE_P8_H + +/* P8 SBE update timer function */ +extern void p8_sbe_update_timer_expiry(uint64_t new_target); + +/* Is SBE timer available ? */ +extern bool p8_sbe_timer_ok(void); + +/* Initialize SBE timer */ +extern void p8_sbe_init_timer(void); + +#endif /* __SBE_P8_H */ diff --git a/include/skiboot.h b/include/skiboot.h index 1180a1714..2ff9cf764 100644 --- a/include/skiboot.h +++ b/include/skiboot.h @@ -323,12 +323,6 @@ extern void xive_late_init(void); /* SLW reinit function for switching core settings */ extern int64_t slw_reinit(uint64_t flags); -/* SLW update timer function */ -extern void slw_update_timer_expiry(uint64_t new_target); - -/* Is SLW timer available ? */ -extern bool slw_timer_ok(void); - /* Patch SPR in SLW image */ extern int64_t opal_slw_set_reg(uint64_t cpu_pir, uint64_t sprn, uint64_t val); From patchwork Mon Apr 16 11:48:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vasant Hegde X-Patchwork-Id: 898604 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40Pmzb3DWGz9s08 for ; Mon, 16 Apr 2018 21:55:39 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40Pmzb212zzF201 for ; Mon, 16 Apr 2018 21:55:39 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=hegdevasant@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40Pmr45VByzF1Rv for ; Mon, 16 Apr 2018 21:49:08 +1000 (AEST) Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w3GBm6lS001205 for ; Mon, 16 Apr 2018 07:49:05 -0400 Received: from e06smtp15.uk.ibm.com (e06smtp15.uk.ibm.com [195.75.94.111]) by mx0b-001b2d01.pphosted.com with ESMTP id 2hcs5upae7-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Mon, 16 Apr 2018 07:49:05 -0400 Received: from localhost by e06smtp15.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 16 Apr 2018 12:49:03 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp15.uk.ibm.com (192.168.101.145) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 16 Apr 2018 12:49:02 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w3GBn1YL11338106; Mon, 16 Apr 2018 11:49:01 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BE36CAE04D; Mon, 16 Apr 2018 12:38:52 +0100 (BST) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 72097AE056; Mon, 16 Apr 2018 12:38:51 +0100 (BST) Received: from hegdevasant.in.ibm.com (unknown [9.109.198.237]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 16 Apr 2018 12:38:51 +0100 (BST) From: Vasant Hegde To: skiboot@lists.ozlabs.org Date: Mon, 16 Apr 2018 17:18:48 +0530 X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180416114848.19618-1-hegdevasant@linux.vnet.ibm.com> References: <20180416114848.19618-1-hegdevasant@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18041611-0020-0000-0000-000004126D37 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18041611-0021-0000-0000-000042A6A73B Message-Id: <20180416114848.19618-3-hegdevasant@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-04-16_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804160112 Subject: [Skiboot] [PATCH v7 2/2] SBE: Add timer support X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" SBE on P9 provides one shot programmable timer facility. We can use this to implement OPAL timers and hence limit the reliance on the Linux heartbeat (similar to HW timer facility provided by SLW on P8). Design: - We will continue to run Linux heartbeat. - Each chip has SBE. This patch always schedules timer on SBE on master chip. - Start timer option starts new timer or modifies an active timer for the specified timeout. - SBE expects timeout value in microseconds. We track timeout value in TB. Hence we convert tb to microseconds before sending request to SBE. - We are requesting ack from SBE for timer message. It gaurantees that SBE has scheduled timer. - Disabling SBE timer We expect SBE to send timer expiry interrupt whenever timer expires. We wait for 10 more ms before disabling timer. In future we can consider below alternative approaches: - Presently SBE timer disable is permanent (until we reboot system). SBE sends "I'm back" interrupt after reset. We can consider restarting timer after SBE reset. - Reset SBE and start timer again. - Each chip has SBE. On multi chip system we can try to schedule timer on different chip. Signed-off-by: Vasant Hegde CC: Benjamin Herrenschmidt --- core/interrupts.c | 3 +- core/test/run-timer.c | 8 ++ core/timer.c | 17 ++++- hw/sbe-p9.c | 205 ++++++++++++++++++++++++++++++++++++++++++++++++++ include/sbe-p9.h | 6 ++ 5 files changed, 235 insertions(+), 4 deletions(-) diff --git a/core/interrupts.c b/core/interrupts.c index 4452511f0..5d7a68cd5 100644 --- a/core/interrupts.c +++ b/core/interrupts.c @@ -26,6 +26,7 @@ #include #include #include +#include /* ICP registers */ #define ICP_XIRR 0x4 /* 32-bit access */ @@ -489,7 +490,7 @@ static int64_t opal_handle_interrupt(uint32_t isn, __be64 *outstanding_event_mas is->ops->interrupt(is, isn); /* Check timers if SBE timer isn't working */ - if (!p8_sbe_timer_ok()) + if (!p8_sbe_timer_ok() && !p9_sbe_timer_ok()) check_timers(true); /* Update output events */ diff --git a/core/test/run-timer.c b/core/test/run-timer.c index 986af28d8..159e007ac 100644 --- a/core/test/run-timer.c +++ b/core/test/run-timer.c @@ -4,12 +4,15 @@ #define __TEST__ #include +#include #define mftb() (stamp) #define sync() #define smt_lowest() #define smt_medium() +enum proc_gen proc_gen = proc_gen_p9; + static uint64_t stamp, last; struct lock; static inline void lock_caller(struct lock *l, const char *caller) @@ -53,6 +56,11 @@ void p8_sbe_update_timer_expiry(uint64_t new_target) /* FIXME: do intersting SLW timer sim */ } +void p9_sbe_update_timer_expiry(uint64_t new_target) +{ + (void)new_target; +} + int main(void) { unsigned int i; diff --git a/core/timer.c b/core/timer.c index 21f62a492..8eefb74be 100644 --- a/core/timer.c +++ b/core/timer.c @@ -5,6 +5,7 @@ #include #include #include +#include #ifdef __TEST__ #define this_cpu() ((void *)-1) @@ -109,8 +110,12 @@ static void __schedule_timer_at(struct timer *t, uint64_t when) bail: /* Pick up the next timer and upddate the SBE HW timer */ lt = list_top(&timer_list, struct timer, link); - if (lt) - p8_sbe_update_timer_expiry(lt->target); + if (lt) { + if (proc_gen < proc_gen_p9) + p8_sbe_update_timer_expiry(lt->target); + else + p9_sbe_update_timer_expiry(lt->target); + } } void schedule_timer_at(struct timer *t, uint64_t when) @@ -167,7 +172,11 @@ static void __check_poll_timers(uint64_t now) * arbitrarily 1us. */ if (t->running) { - p8_sbe_update_timer_expiry(now + usecs_to_tb(1)); + if (proc_gen < proc_gen_p9) + p8_sbe_update_timer_expiry(now + usecs_to_tb(1)); + else + p9_sbe_update_timer_expiry(now + usecs_to_tb(1)); + break; } @@ -266,6 +275,8 @@ void late_init_timers(void) */ if (platform.heartbeat_time) { heartbeat = platform.heartbeat_time(); + } else if (p9_sbe_timer_ok()) { + heartbeat = HEARTBEAT_DEFAULT_MS * 10; } else if (p8_sbe_timer_ok() || fsp_present()) { heartbeat = HEARTBEAT_DEFAULT_MS * 10; } diff --git a/hw/sbe-p9.c b/hw/sbe-p9.c index 9dc487ba2..a5b0706ef 100644 --- a/hw/sbe-p9.c +++ b/hw/sbe-p9.c @@ -53,6 +53,7 @@ #include #include #include +#include #include #include @@ -80,11 +81,36 @@ struct p9_sbe { /* Default SBE chip ID */ static int sbe_default_chip_id = -1; +/* Is SBE timer running? */ +static bool sbe_has_timer = false; +static bool sbe_timer_in_progress = false; + +/* Inflight and next timer in TB */ +static uint64_t sbe_last_gen_stamp; +static uint64_t sbe_timer_target; + +static bool has_new_target = false; +static bool got_timer_interrupt = false; + +/* Timer lock */ +struct lock sbe_timer_lock; + +/* + * Minimum timeout value for P9 is 500 microseconds. After that + * SBE timer can handle granularity of 1 microsecond. + */ +#define SBE_TIMER_DEFAULT_US 500 +static uint64_t sbe_timer_def_tb; + +/* Timer control message */ +static struct p9_sbe_msg *timer_ctrl_msg; + #define SBE_STATUS_PRI_SHIFT 0x30 #define SBE_STATUS_SEC_SHIFT 0x20 /* Forward declaration */ static void p9_sbe_timeout_poll_one(struct p9_sbe *sbe); +static void p9_sbe_timer_schedule(void); /* bit 0-15 : Primary status code */ static inline u16 p9_sbe_get_primary_rc(struct p9_sbe_msg *resp) @@ -562,6 +588,14 @@ static void __p9_sbe_interrupt(struct p9_sbe *sbe) p9_sbe_msg_complete(sbe, msg); } + /* Timer expired */ + if (data & SBE_HOST_TIMER_EXPIRY) { + if (sbe->chip_id == sbe_default_chip_id) { + sbe_timer_in_progress = false; + got_timer_interrupt = true; + } + } + next_msg: p9_sbe_process_queue(sbe); } @@ -578,6 +612,15 @@ void p9_sbe_interrupt(uint32_t chip_id) sbe = chip->sbe; lock(&sbe->lock); __p9_sbe_interrupt(sbe); + + if (got_timer_interrupt) { + got_timer_interrupt = false; + /* Drop lock and call timers */ + unlock(&sbe->lock); + check_timers(true); + return; + } + unlock(&sbe->lock); } @@ -634,6 +677,55 @@ out: unlock(&sbe->lock); } +/* + * Check if the timer is working. If at least 10ms elapsed since + * last scheduled timer expiry. + */ +static void p9_sbe_timer_poll(struct p9_sbe *sbe) +{ + if (!sbe_has_timer || !sbe_timer_in_progress) + return; + + lock(&sbe->lock); + if (tb_compare(mftb(), sbe_last_gen_stamp + msecs_to_tb(10)) + != TB_AAFTERB) + goto out; + + /* + * In some cases there will be a delay in calling OPAL interrupt + * handler routine (opal_handle_interrupt). In such cases its + * possible that SBE has responded, but OPAL didn't act on that. + * Hence check for SBE response before disabling timer. + */ + __p9_sbe_interrupt(sbe); + if (got_timer_interrupt) + goto call_timer; + + if (!sbe_timer_in_progress) + goto out; + + if (tb_compare(mftb(), sbe_last_gen_stamp + msecs_to_tb(10)) + != TB_AAFTERB) + goto out; + + prlog(PR_ERR, "Timer stuck, falling back to OPAL pollers.\n"); + prlog(PR_ERR, "You will likely have slower I2C and may have " + "experienced increased jitter.\n"); + p9_sbe_reg_dump(sbe_default_chip_id); + sbe_has_timer = false; + sbe_timer_in_progress = false; + +out: + unlock(&sbe->lock); + return; + +call_timer: + got_timer_interrupt = false; + /* Drop lock and call timers */ + unlock(&sbe->lock); + check_timers(true); +} + static void p9_sbe_timeout_poll(void *user_data __unused) { struct p9_sbe *sbe; @@ -644,9 +736,119 @@ static void p9_sbe_timeout_poll(void *user_data __unused) continue; sbe = chip->sbe; p9_sbe_timeout_poll_one(sbe); + if (sbe->chip_id == sbe_default_chip_id) + p9_sbe_timer_poll(sbe); } } +static void p9_sbe_timer_resp(struct p9_sbe_msg *msg) +{ + if (msg->state != sbe_msg_done) { + prlog(PR_DEBUG, "Failed to schedule timer [chip id %x]\n", + sbe_default_chip_id); + + if (!has_new_target) + return; + } else { + /* Update last scheduled timer value */ + sbe_last_gen_stamp = mftb() + + usecs_to_tb(timer_ctrl_msg->reg[1]); + sbe_timer_in_progress = true; + } + + lock(&sbe_timer_lock); + if (has_new_target) { + has_new_target = false; + p9_sbe_timer_schedule(); + } + unlock(&sbe_timer_lock); +} + +static void p9_sbe_timer_schedule(void) +{ + int rc; + u32 tick_us = SBE_TIMER_DEFAULT_US; + u64 tb_cnt, now = mftb(); + + if (sbe_timer_in_progress) { + /* Remaining time of inflight timer <= sbe_timer_def_tb */ + if ((sbe_last_gen_stamp - now) <= sbe_timer_def_tb) + return; + } + + if (now < sbe_timer_target) { + /* Calculate how many microseconds from now, rounded up */ + if ((sbe_timer_target - now) > sbe_timer_def_tb) { + tb_cnt = sbe_timer_target - now + usecs_to_tb(1) - 1; + tick_us = tb_to_usecs(tb_cnt); + } + } + + /* Clear sequence number. p9_sbe_queue_msg will add new sequene ID */ + timer_ctrl_msg->reg[0] &= ~(PPC_BITMASK(32, 47)); + /* Update timeout value */ + timer_ctrl_msg->reg[1] = tick_us; + rc = p9_sbe_queue_msg(sbe_default_chip_id, timer_ctrl_msg, + p9_sbe_timer_resp); + if (rc != OPAL_SUCCESS) { + prlog(PR_ERR, "Failed to start timer [chip id = %x]\n", + sbe_default_chip_id); + return; + } +} + +/* + * This is called with the timer lock held, so there is no + * issue with re-entrancy or concurrence + */ +void p9_sbe_update_timer_expiry(uint64_t new_target) +{ + u64 now = mftb(); + + if (!sbe_has_timer || new_target == sbe_timer_target) + return; + + lock(&sbe_timer_lock); + /* Timer message is in flight. Record new timer and schedule later */ + if (p9_sbe_msg_busy(timer_ctrl_msg) || has_new_target) { + if (new_target < now) + goto out; + + if (new_target > sbe_timer_target) + goto out; + + sbe_timer_target = new_target; + has_new_target = true; + goto out; + } + + sbe_timer_target = new_target; + p9_sbe_timer_schedule(); + +out: + unlock(&sbe_timer_lock); +} + +/* Initialize SBE timer */ +static void p9_sbe_timer_init(void) +{ + timer_ctrl_msg = p9_sbe_mkmsg(SBE_CMD_CONTROL_TIMER, + CONTROL_TIMER_START, 0, 0, 0); + assert(timer_ctrl_msg); + + init_lock(&sbe_timer_lock); + sbe_has_timer = true; + sbe_timer_target = mftb(); + sbe_last_gen_stamp = ~0ull; + sbe_timer_def_tb = usecs_to_tb(SBE_TIMER_DEFAULT_US); + prlog(PR_INFO, "Timer facility on chip %x\n", sbe_default_chip_id); +} + +bool p9_sbe_timer_ok(void) +{ + return sbe_has_timer; +} + void p9_sbe_init(void) { struct dt_node *xn; @@ -680,6 +882,9 @@ void p9_sbe_init(void) return; } + /* Initiate SBE timer */ + p9_sbe_timer_init(); + /* Initiate SBE timeout poller */ opal_add_poller(p9_sbe_timeout_poll, NULL); } diff --git a/include/sbe-p9.h b/include/sbe-p9.h index 329afff80..4b839d8ba 100644 --- a/include/sbe-p9.h +++ b/include/sbe-p9.h @@ -231,4 +231,10 @@ extern void p9_sbe_init(void); /* SBE interrupt */ extern void p9_sbe_interrupt(uint32_t chip_id); +/* Is SBE timer available ? */ +extern bool p9_sbe_timer_ok(void); + +/* Update SBE timer expiry */ +extern void p9_sbe_update_timer_expiry(uint64_t new_target); + #endif /* __SBE_P9_H */