From patchwork Sat Apr 21 21:08:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Seth Forshee X-Patchwork-Id: 902550 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) by ozlabs.org (Postfix) with ESMTP id 40T56K3sxcz9s19; Sun, 22 Apr 2018 07:12:57 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1f9zoS-0001dc-Ug; Sat, 21 Apr 2018 21:12:52 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.86_2) (envelope-from ) id 1f9zoQ-0001dO-U6 for kernel-team@lists.ubuntu.com; Sat, 21 Apr 2018 21:12:50 +0000 Received: from mail-io0-f198.google.com ([209.85.223.198]) by youngberry.canonical.com with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.76) (envelope-from ) id 1f9zoQ-0001Go-Jn for kernel-team@lists.ubuntu.com; Sat, 21 Apr 2018 21:12:50 +0000 Received: by mail-io0-f198.google.com with SMTP id a193-v6so4827551ioa.23 for ; Sat, 21 Apr 2018 14:12:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=TTaX2BygxGQt8+rWIKHVQTKmsM6EjuJFcoWi88W5G+M=; b=SCd4nVhdp0uLkA81O1xxvgVp8oZlCfycONjIuGHpjgSIZvDDRBbOqfBs49wY8tIQ2l 321yCA8YCPT3nQ+ToRO1TT2FoEuMlGPTPRH5C/eRs9HUaHa12h/a3MPUYaAs3RKNncCT ON29cF1xqBooBihB/av5k+TkzX+juoEQPQkgRkQvCQinGCaJml1Q/yhrwQhk4w32NlgD 4XCACslkDzM8fSn+wUMDSd2yFh05uVOVHZQWejQma+yfDdx+GjsGKUUMvNVPRmreYMat UKKTxKy5pWFBZva2rqlWBK3De1KRo/p5Zv6tRYuyK4PgEvf9vaJXSqaCnu9maXUrNZvz adbQ== X-Gm-Message-State: ALQs6tCqh1pZZQ1/Q1gXH6qxRidTndT45mV/ZhCmkXRqh6hbRdLPVKG3 0IipvuZC4jzwAuJVyf4567WO7YlSLiVKA3b5S/XWR4sELtcU8ocL/A9Dh5wl3UIKhhVqVXr0Bc/ DdKIyLMpRsnANDW4n89FPLiF/agAVOFoMgXdUrNhMsw== X-Received: by 2002:a6b:b4cc:: with SMTP id d195-v6mr16365941iof.77.1524345169339; Sat, 21 Apr 2018 14:12:49 -0700 (PDT) X-Google-Smtp-Source: AIpwx49HP36mnPw0mZc+eMK8h+e1CvTx+pggK+v6xAqag/zFp+Zd5tkWE5kDeqdI0ozLgOox5CPyIA== X-Received: by 2002:a6b:b4cc:: with SMTP id d195-v6mr16365932iof.77.1524345169083; Sat, 21 Apr 2018 14:12:49 -0700 (PDT) Received: from localhost ([136.63.232.127]) by smtp.gmail.com with ESMTPSA id 124-v6sm2326467itw.25.2018.04.21.14.12.48 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 21 Apr 2018 14:12:48 -0700 (PDT) From: Seth Forshee To: kernel-team@lists.ubuntu.com Subject: [SRU][Bionic][PATCH 1/2] Revert "blk-mq: simplify queue mapping & schedule with each possisble CPU" Date: Sat, 21 Apr 2018 16:08:25 -0500 Message-Id: <20180421210826.27042-2-seth.forshee@canonical.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180421210826.27042-1-seth.forshee@canonical.com> References: <20180421210826.27042-1-seth.forshee@canonical.com> X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" BugLink: http://bugs.launchpad.net/bugs/1765232 This reverts commit 9403a13fd07ef6a5fff7a69860f419d16641bb60. This patch was applied as part of the fix for LP #1759723 and is resulting in IO hangs due to some drivers selecting reply queues which do not have any online CPU mapped. Signed-off-by: Seth Forshee --- block/blk-mq.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 50ef7dc4c41e..1a80d8c4f3ec 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -443,7 +443,7 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q, blk_queue_exit(q); return ERR_PTR(-EXDEV); } - cpu = cpumask_first_and(alloc_data.hctx->cpumask, cpu_online_mask); + cpu = cpumask_first(alloc_data.hctx->cpumask); alloc_data.ctx = __blk_mq_get_ctx(q, cpu); rq = blk_mq_get_request(q, NULL, op, &alloc_data); @@ -1263,10 +1263,9 @@ static int blk_mq_hctx_next_cpu(struct blk_mq_hw_ctx *hctx) if (--hctx->next_cpu_batch <= 0) { int next_cpu; - next_cpu = cpumask_next_and(hctx->next_cpu, hctx->cpumask, - cpu_online_mask); + next_cpu = cpumask_next(hctx->next_cpu, hctx->cpumask); if (next_cpu >= nr_cpu_ids) - next_cpu = cpumask_first_and(hctx->cpumask,cpu_online_mask); + next_cpu = cpumask_first(hctx->cpumask); hctx->next_cpu = next_cpu; hctx->next_cpu_batch = BLK_MQ_CPU_WORK_BATCH; @@ -2138,11 +2137,16 @@ static void blk_mq_init_cpu_queues(struct request_queue *q, INIT_LIST_HEAD(&__ctx->rq_list); __ctx->queue = q; + /* If the cpu isn't present, the cpu is mapped to first hctx */ + if (!cpu_present(i)) + continue; + + hctx = blk_mq_map_queue(q, i); + /* * Set local node, IFF we have more than one hw queue. If * not, we remain on the home node of the device */ - hctx = blk_mq_map_queue(q, i); if (nr_hw_queues > 1 && hctx->numa_node == NUMA_NO_NODE) hctx->numa_node = local_memory_node(cpu_to_node(i)); } @@ -2199,7 +2203,7 @@ static void blk_mq_map_swqueue(struct request_queue *q) * * If the cpu isn't present, the cpu is mapped to first hctx. */ - for_each_possible_cpu(i) { + for_each_present_cpu(i) { hctx_idx = q->mq_map[i]; /* unmapped hw queue can be remapped after CPU topo changed */ if (!set->tags[hctx_idx] && @@ -2253,8 +2257,7 @@ static void blk_mq_map_swqueue(struct request_queue *q) /* * Initialize batch roundrobin counts */ - hctx->next_cpu = cpumask_first_and(hctx->cpumask, - cpu_online_mask); + hctx->next_cpu = cpumask_first(hctx->cpumask); hctx->next_cpu_batch = BLK_MQ_CPU_WORK_BATCH; } } From patchwork Sat Apr 21 21:08:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Seth Forshee X-Patchwork-Id: 902551 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) by ozlabs.org (Postfix) with ESMTP id 40T58t59w7z9s19; Sun, 22 Apr 2018 07:15:10 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1f9zqa-0001sl-Pc; Sat, 21 Apr 2018 21:15:04 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.86_2) (envelope-from ) id 1f9zqY-0001sF-AY for kernel-team@lists.ubuntu.com; Sat, 21 Apr 2018 21:15:02 +0000 Received: from mail-io0-f198.google.com ([209.85.223.198]) by youngberry.canonical.com with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.76) (envelope-from ) id 1f9zqX-0001MJ-Sm for kernel-team@lists.ubuntu.com; Sat, 21 Apr 2018 21:15:01 +0000 Received: by mail-io0-f198.google.com with SMTP id o17-v6so10549228iob.12 for ; Sat, 21 Apr 2018 14:15:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=d2AdOKkItmylJqAz1/fSmLVWcPRdFRgtsKq9GsiJRqs=; b=Hi2zocDY51jblnN3rchMukt17nIlFOJPoPLweU0iucl8lIAMc0aA7P/9ah7Wlf116T ArbbcCCiJo5h3hjSbThan3G8pwovU/vTFbQRpQ/eQrlsxFYbZcMZC5i0+2b0VUK0fONp J3Cqpfp58S4pBnCOr0wNrS6n6L0ammZIiYbUDVyXT9KTATFQihHRFCzbhtl6B/rCUD4M GyHCSL9oc+lO/M1cPCCbsW8JGzBWT9TQpqbF+aDF4f+RF/WGIkpzx/GZgj3E2qFDTXxy AaM4pNGH2NzwaYYd/hvQYXiMFhId0jToRplPvITdw0vIpmO2egEH8D9dGT75IFNYqIAY vTbA== X-Gm-Message-State: ALQs6tBlpjtxcG7v040w5FbPai4yjvmyjd8YCYhtGPsC4JKcYYhyZ0fj BegMDIDyVqQfiSESbnjJAIcTvLSTZkFzyl/G7oabadP71geDzvRAOUORPDHKGrb9DcPxfr/8yWf rKlGzvdh1Y2WJvesoLlkSEumIdaiD07NQw2KUHA+jmQ== X-Received: by 2002:a24:c107:: with SMTP id e7-v6mr2602310itg.40.1524345300599; Sat, 21 Apr 2018 14:15:00 -0700 (PDT) X-Google-Smtp-Source: AB8JxZp9qpZA+RrX7N4rOoCsCi/8dm8iKhvKpfXeF3A9MdlogXT4JkOCcSbkj2IeJHBJM6k8xrLvgw== X-Received: by 2002:a24:c107:: with SMTP id e7-v6mr2602290itg.40.1524345300302; Sat, 21 Apr 2018 14:15:00 -0700 (PDT) Received: from localhost ([136.63.232.127]) by smtp.gmail.com with ESMTPSA id w133-v6sm2561918itc.1.2018.04.21.14.14.59 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 21 Apr 2018 14:14:59 -0700 (PDT) From: Seth Forshee To: kernel-team@lists.ubuntu.com Subject: [SRU][Bionic][PATCH 2/2] Revert "genirq/affinity: assign vectors to all possible CPUs" Date: Sat, 21 Apr 2018 16:08:26 -0500 Message-Id: <20180421210826.27042-3-seth.forshee@canonical.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180421210826.27042-1-seth.forshee@canonical.com> References: <20180421210826.27042-1-seth.forshee@canonical.com> X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" BugLink: http://bugs.launchpad.net/bugs/1765232 This reverts commit f0aff9ccc834422db981a2eb505ede27f589ad50. This patch was applied as part of the fix for LP #1759723 and is resulting in IO hangs due to some drivers selecting reply queues which do not have any online CPU mapped. Signed-off-by: Seth Forshee --- kernel/irq/affinity.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/kernel/irq/affinity.c b/kernel/irq/affinity.c index a37a3b4b6342..e12d35108225 100644 --- a/kernel/irq/affinity.c +++ b/kernel/irq/affinity.c @@ -39,7 +39,7 @@ static void irq_spread_init_one(struct cpumask *irqmsk, struct cpumask *nmsk, } } -static cpumask_var_t *alloc_node_to_possible_cpumask(void) +static cpumask_var_t *alloc_node_to_present_cpumask(void) { cpumask_var_t *masks; int node; @@ -62,7 +62,7 @@ static cpumask_var_t *alloc_node_to_possible_cpumask(void) return NULL; } -static void free_node_to_possible_cpumask(cpumask_var_t *masks) +static void free_node_to_present_cpumask(cpumask_var_t *masks) { int node; @@ -71,22 +71,22 @@ static void free_node_to_possible_cpumask(cpumask_var_t *masks) kfree(masks); } -static void build_node_to_possible_cpumask(cpumask_var_t *masks) +static void build_node_to_present_cpumask(cpumask_var_t *masks) { int cpu; - for_each_possible_cpu(cpu) + for_each_present_cpu(cpu) cpumask_set_cpu(cpu, masks[cpu_to_node(cpu)]); } -static int get_nodes_in_cpumask(cpumask_var_t *node_to_possible_cpumask, +static int get_nodes_in_cpumask(cpumask_var_t *node_to_present_cpumask, const struct cpumask *mask, nodemask_t *nodemsk) { int n, nodes = 0; /* Calculate the number of nodes in the supplied affinity mask */ for_each_node(n) { - if (cpumask_intersects(mask, node_to_possible_cpumask[n])) { + if (cpumask_intersects(mask, node_to_present_cpumask[n])) { node_set(n, *nodemsk); nodes++; } @@ -109,7 +109,7 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd) int last_affv = affv + affd->pre_vectors; nodemask_t nodemsk = NODE_MASK_NONE; struct cpumask *masks; - cpumask_var_t nmsk, *node_to_possible_cpumask; + cpumask_var_t nmsk, *node_to_present_cpumask; /* * If there aren't any vectors left after applying the pre/post @@ -125,8 +125,8 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd) if (!masks) goto out; - node_to_possible_cpumask = alloc_node_to_possible_cpumask(); - if (!node_to_possible_cpumask) + node_to_present_cpumask = alloc_node_to_present_cpumask(); + if (!node_to_present_cpumask) goto out; /* Fill out vectors at the beginning that don't need affinity */ @@ -135,8 +135,8 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd) /* Stabilize the cpumasks */ get_online_cpus(); - build_node_to_possible_cpumask(node_to_possible_cpumask); - nodes = get_nodes_in_cpumask(node_to_possible_cpumask, cpu_possible_mask, + build_node_to_present_cpumask(node_to_present_cpumask); + nodes = get_nodes_in_cpumask(node_to_present_cpumask, cpu_present_mask, &nodemsk); /* @@ -146,7 +146,7 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd) if (affv <= nodes) { for_each_node_mask(n, nodemsk) { cpumask_copy(masks + curvec, - node_to_possible_cpumask[n]); + node_to_present_cpumask[n]); if (++curvec == last_affv) break; } @@ -160,7 +160,7 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd) vecs_per_node = (affv - (curvec - affd->pre_vectors)) / nodes; /* Get the cpus on this node which are in the mask */ - cpumask_and(nmsk, cpu_possible_mask, node_to_possible_cpumask[n]); + cpumask_and(nmsk, cpu_present_mask, node_to_present_cpumask[n]); /* Calculate the number of cpus per vector */ ncpus = cpumask_weight(nmsk); @@ -192,7 +192,7 @@ irq_create_affinity_masks(int nvecs, const struct irq_affinity *affd) /* Fill out vectors at the end that don't need affinity */ for (; curvec < nvecs; curvec++) cpumask_copy(masks + curvec, irq_default_affinity); - free_node_to_possible_cpumask(node_to_possible_cpumask); + free_node_to_present_cpumask(node_to_present_cpumask); out: free_cpumask_var(nmsk); return masks; @@ -214,7 +214,7 @@ int irq_calc_affinity_vectors(int minvec, int maxvec, const struct irq_affinity return 0; get_online_cpus(); - ret = min_t(int, cpumask_weight(cpu_possible_mask), vecs) + resv; + ret = min_t(int, cpumask_weight(cpu_present_mask), vecs) + resv; put_online_cpus(); return ret; }