From patchwork Wed Aug 7 12:46:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Righi X-Patchwork-Id: 1143480 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 463WTf5Gfxz9sNk; Wed, 7 Aug 2019 22:46:30 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1hvLKi-0001Hq-Uu; Wed, 07 Aug 2019 12:46:24 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.86_2) (envelope-from ) id 1hvLKg-0001GR-LR for kernel-team@lists.ubuntu.com; Wed, 07 Aug 2019 12:46:22 +0000 Received: from mail-wm1-f71.google.com ([209.85.128.71]) by youngberry.canonical.com with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.76) (envelope-from ) id 1hvLKg-0007Ge-5R for kernel-team@lists.ubuntu.com; Wed, 07 Aug 2019 12:46:22 +0000 Received: by mail-wm1-f71.google.com with SMTP id t76so20927269wmt.9 for ; Wed, 07 Aug 2019 05:46:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=hXn+hZ+3/zQA30GZ5nZy5X9Ww77afrL+YGSBaPTLyCI=; b=RwLb7wWRdC8Ub8PcnzZU8vXkClI4QIJJkk9waNb5ahlsXMlM6bLDvmVv5vdLh9+7HN +d3cfMV85HOWluArF7SftjhGETwyaqXD3mjQ5Hdo/hS8e9oGEbHm+gHmEsFMLhWnoJh9 IZXyRUQMqU2vdroF8/mQblnLaP+zcRZ8qrunnw3dvWO74WO67MJPl//2TEC3ynxfyvXa hwkqUY3iK83BovFST0EWrtYO03HHX5bUbuiKxYmCbG0c/b4SdgR3CHS0I9yMw4QaDuZE /jUJOPb+mfCw0+C8/CVnS947/T/2ktVaJ46o2qWY/5oeY811sHB9NSQ8KRFClFWgmFNT 05gQ== X-Gm-Message-State: APjAAAU7H5UWh2Pz8xVNufeBYv+qW9CrgqCLGA2GU5Zo3GfoP0P4aIuY DuCAbjPG9bGfm0B8ZHjTWe8AFVbBwjJL3swb23B1OWJMRj63gqhNWbRwSUnP4x7C32XtKf6qb6l SaNeqV/sBYmC+i/ZJ8EpkhpBj/FQFbotujB6VF3cdeQ== X-Received: by 2002:adf:eb0f:: with SMTP id s15mr10697493wrn.324.1565181981542; Wed, 07 Aug 2019 05:46:21 -0700 (PDT) X-Google-Smtp-Source: APXvYqyTv3Z8kjYngR1W/B1Ex3UnDTzCj+J9OyAQZQ0KVL2cD4cNul3BJTYET7peWIp5xRqMtno1cw== X-Received: by 2002:adf:eb0f:: with SMTP id s15mr10697461wrn.324.1565181981262; Wed, 07 Aug 2019 05:46:21 -0700 (PDT) Received: from xps-13.homenet.telecomitalia.it (host21-131-dynamic.46-79-r.retail.telecomitalia.it. [79.46.131.21]) by smtp.gmail.com with ESMTPSA id p7sm64020091wrs.6.2019.08.07.05.46.20 for (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 07 Aug 2019 05:46:20 -0700 (PDT) From: Andrea Righi To: kernel-team@lists.ubuntu.com Subject: [SRU][D][PATCH v2 0/3] bcache: fix hung task timeout in bch_bucket_alloc() Date: Wed, 7 Aug 2019 14:46:14 +0200 Message-Id: <20190807124617.30224-1-andrea.righi@canonical.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" BugLink: https://bugs.launchpad.net/bugs/1784665 [Impact] bcache_allocator can call the following: bch_allocator_thread() -> bch_prio_write() -> bch_bucket_alloc() -> wait on &ca->set->bucket_wait But the wake up event on bucket_wait is supposed to come from bch_allocator_thread() itself causing a deadlock. [Test Case] This is a simple script that can easily trigger the deadlock condition: https://launchpadlibrarian.net/381282009/bcache-basic-repro.sh A better test case has been also provided in LP: #1796292: https://bugs.launchpad.net/curtin/+bug/1796292/+attachment/5280353/+files/curtin-nvme.sh [Fix] Fix by making the call to bch_prio_write() non-blocking, so that bch_allocator_thread() never waits on itself. Moreover, make sure to wake up the garbage collector thread when bch_prio_write() is failing to allocate buckets to increase the chance of freeing up more buckets. In addition to that it would be safer to also import other upstream bcache fixes (all clean cherry picks): eb8cbb6df38f6e5124a3d5f1f8a3dbf519537c60 bcache: improve bcache_reboot() 9951379b0ca88c95876ad9778b9099e19a95d566 bcache: never writeback a discard operation [Regression Potential] The upstream fixes are all clean cherry picks from stable (most of them are small cleanups), so regression potential is minimal. The only special patch is "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator" that is addressing the main deadlock bug (that seems to be a mainline bug - not fixed yet). We should spend more time trying to reproduce this deadlock with a mainline kernel and post the patch to the LKML for review / feedback. However, considering that this patch seems to fix/prevent the specific deadlock problem reported in this bug (tested on the affected platform), it should be considered safe to apply it as it is for now, to prevent potential hung task timeout conditions. Changes in v2: - fix potential buckets leak in "UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator" ---------------------------------------------------------------- Andrea Righi (1): UBUNTU: SAUCE: bcache: fix deadlock in bcache_allocator Coly Li (1): bcache: improve bcache_reboot() Daniel Axtens (1): bcache: never writeback a discard operation drivers/md/bcache/alloc.c | 5 ++++- drivers/md/bcache/bcache.h | 2 +- drivers/md/bcache/super.c | 39 +++++++++++++++++++++++++++++++-------- drivers/md/bcache/writeback.h | 3 +++ 4 files changed, 39 insertions(+), 10 deletions(-) Acked-by: Stefan Bader Acked-by: Kleber Sacilotto de Souza