From patchwork Tue May 8 06:58:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 910036 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40g9MN5vHhz9rvt for ; Tue, 8 May 2018 16:59:12 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ITOKHege"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40g9MN3sh2zF1dk for ; Tue, 8 May 2018 16:59:12 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ITOKHege"; dkim-atps=neutral X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:400e:c01::244; helo=mail-pl0-x244.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ITOKHege"; dkim-atps=neutral Received: from mail-pl0-x244.google.com (mail-pl0-x244.google.com [IPv6:2607:f8b0:400e:c01::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40g9MH1jhWzF1KS for ; Tue, 8 May 2018 16:59:06 +1000 (AEST) Received: by mail-pl0-x244.google.com with SMTP id e6-v6so1654711plt.11 for ; Mon, 07 May 2018 23:59:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=zLm59vGFioQC90z1sYfwlO50JbaXVUlqcVPZTxjo7Vs=; b=ITOKHegelMpIUggqdgxQ0RW1b6C8bG8a4xHNrWmL+9oJzv1FXz+GDFsgHA8xMvIB3Y KvFs865DP8yTFBtIOSzt4Zy3ShXsD43BxXTOdM4SMQC8vxV+YGDEkPdd8iPVYShNojda /mEkIOyCJDEcX338bN3uF1uQZlIU8dgDj9MFFAw1inTbQ5aDwcyUraR2BkjoFhiB8lKW SATG4rDwCy7Zukzc9cnBj5j3mzhEXUjpPZCOP/C1jK/f0MTh+IDU6LN+SB5BuXhqa5dp N0WqApXLN3bFc8naPSiNtHnDvjDb3xuti0Xys1Lu5jqAA6qGiPwzXra819DbZdNiWNvl At0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=zLm59vGFioQC90z1sYfwlO50JbaXVUlqcVPZTxjo7Vs=; b=GX1111oOdroqWnxLKIK3Kuedobubxsx6JSAXPcrEUt2a1mp9cc3KsA2OQeqcbotYhb KdPYiy3Q+fTLfEnbCxn2fsgp56Wz/jDhvvZGkaNrlsHE9qM3QRTNvl6xEwvozrJURkAd GrY+TYSwp+BSAXxCKBVPNVKOX4/d9miyPA1TSXCWecJRc/sJGIrwJ0BOwE5FxIJSVZjV HKAvmTLSuzHfEerNStZxSMT2fXGh71a7omPgXKVgYFuztDTR8FuinAFpqZk7FfH556aZ 33TRO8LmvGPjci5BrB1LVOWTVf7jMHcqkMXiPs6qNH5UYXfQywUvozeSzTXje4aALXPK 5GFw== X-Gm-Message-State: ALQs6tDND3Uas+/cyjbZkZ5E7YDmGvpkGKnu4y2olshN6vZlGu/kZ61s sPBdCO+Uuygfho/Y8VgND/QXCg== X-Google-Smtp-Source: AB8JxZqxewjAKMumQPSPNmY30oeCoo5QnvNBNymiJj5p16ID0lPKAwxX/PeSvI/ZbQS07ABN8JvpMg== X-Received: by 2002:a17:902:b10f:: with SMTP id q15-v6mr32006149plr.142.1525762744408; Mon, 07 May 2018 23:59:04 -0700 (PDT) Received: from roar.au.ibm.com (59-102-70-78.tpgi.com.au. [59.102.70.78]) by smtp.gmail.com with ESMTPSA id e5sm48642587pfk.28.2018.05.07.23.59.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 May 2018 23:59:03 -0700 (PDT) From: Nicholas Piggin To: skiboot@lists.ozlabs.org Date: Tue, 8 May 2018 16:58:53 +1000 Message-Id: <20180508065854.23064-1-npiggin@gmail.com> X-Mailer: git-send-email 2.17.0 Subject: [Skiboot] [PATCH] core/console: fix deadlock when printing with console lock held X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrew Donnellan MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Some debugging options will print while the console lock is held, which is why the console lock is taken as a recursive lock. However console_write calls __flush_console, which will drop and re-take the lock non-recursively in some cases. Just set con_need_flush and return from __flush_console if we are holding the console lock already. This stack usage message (taken with this patch applied) could lead to a deadlock without this: CPU 0000 lowest stack mark 11768 bytes left pc=300cb808 token=0 CPU 0000 Backtrace: S: 0000000031c03370 R: 00000000300cb808 .list_check_node+0x1c S: 0000000031c03410 R: 00000000300cb910 .list_check+0x38 S: 0000000031c034b0 R: 00000000300190ac .try_lock_caller+0xb8 S: 0000000031c03540 R: 00000000300192e0 .lock_caller+0x80 S: 0000000031c03600 R: 0000000030012c70 .__flush_console+0x134 S: 0000000031c036d0 R: 00000000300130cc .console_write+0x68 S: 0000000031c03780 R: 00000000300347bc .vprlog+0xc8 S: 0000000031c03970 R: 0000000030034844 ._prlog+0x50 S: 0000000031c03a00 R: 00000000300364a4 .log_simple_error+0x74 S: 0000000031c03b90 R: 000000003004ab48 .occ_pstates_init+0x184 S: 0000000031c03d50 R: 000000003001480c .load_and_boot_kernel+0x38c S: 0000000031c03e30 R: 000000003001571c .main_cpu_entry+0x62c S: 0000000031c03f00 R: 0000000030002700 boot_entry+0x1c0 Reported-by: Andrew Donnellan Signed-off-by: Nicholas Piggin --- core/console.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/core/console.c b/core/console.c index b9129c9f..e8256758 100644 --- a/core/console.c +++ b/core/console.c @@ -100,7 +100,7 @@ void clear_console(void) * Optionally can skip flushing to drivers, leaving messages * just in memory console. */ -static bool __flush_console(bool flush_to_drivers) +static bool __flush_console(bool flush_to_drivers, bool need_unlock) { struct cpu_thread *cpu = this_cpu(); size_t req, len = 0; @@ -114,8 +114,12 @@ static bool __flush_console(bool flush_to_drivers) * Console flushing is suspended on this CPU, typically because * some critical locks are held that would potentially cause a * flush to deadlock + * + * Also if it recursed on con_lock (need_unlock is false). This + * can happen due to debug code firing (e.g., list or stack + * debugging). */ - if (cpu->con_suspend) { + if (cpu->con_suspend || !need_unlock) { cpu->con_need_flush = true; return false; } @@ -176,7 +180,7 @@ bool flush_console(void) bool ret; lock(&con_lock); - ret = __flush_console(true); + ret = __flush_console(true, true); unlock(&con_lock); return ret; @@ -250,7 +254,7 @@ ssize_t console_write(bool flush_to_drivers, const void *buf, size_t count) write_char(c); } - __flush_console(flush_to_drivers); + __flush_console(flush_to_drivers, need_unlock); if (need_unlock) unlock(&con_lock);