From patchwork Mon Sep 18 18:28:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Bringmann X-Patchwork-Id: 815075 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xwvp45jLlz9s7m for ; Tue, 19 Sep 2017 04:35:40 +1000 (AEST) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3xwvp33lcxzDqNh for ; Tue, 19 Sep 2017 04:35:39 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=mwb@linux.vnet.ibm.com; receiver=) Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3xwvf14GcpzDqXs for ; Tue, 19 Sep 2017 04:28:41 +1000 (AEST) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v8IIOHbi063535 for ; Mon, 18 Sep 2017 14:28:39 -0400 Received: from e14.ny.us.ibm.com (e14.ny.us.ibm.com [129.33.205.204]) by mx0a-001b2d01.pphosted.com with ESMTP id 2d2hacfq2w-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 18 Sep 2017 14:28:39 -0400 Received: from localhost by e14.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 18 Sep 2017 14:28:38 -0400 Received: from b01cxnp22033.gho.pok.ibm.com (9.57.198.23) by e14.ny.us.ibm.com (146.89.104.201) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 18 Sep 2017 14:28:35 -0400 Received: from b01ledav005.gho.pok.ibm.com (b01ledav005.gho.pok.ibm.com [9.57.199.110]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v8IISYkV41484422; Mon, 18 Sep 2017 18:28:34 GMT Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7C24EAE043; Mon, 18 Sep 2017 14:29:05 -0400 (EDT) Received: from oc1554177480.ibm.com (unknown [9.53.92.236]) by b01ledav005.gho.pok.ibm.com (Postfix) with ESMTP id F418EAE034; Mon, 18 Sep 2017 14:29:04 -0400 (EDT) To: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org From: Michael Bringmann Subject: [PATCH 1/2] powerpc/nodes: Ensure enough nodes avail for operations Organization: IBM Linux Technology Center In-Reply-To: <49626c85-d466-3f99-bde8-f459e278f6ee@linux.vnet.ibm.com> Date: Mon, 18 Sep 2017 13:28:33 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 Content-Language: en-US X-TM-AS-GCONF: 00 x-cbid: 17091818-0052-0000-0000-000002635A2B X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007757; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000230; SDB=6.00918904; UDB=6.00461633; IPR=6.00699131; BA=6.00005595; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00017198; XFM=3.00000015; UTC=2017-09-18 18:28:36 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17091818-0053-0000-0000-0000520C54B4 Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-09-18_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1709180260 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Bringmann , Nathan Fontenot , John Allen Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" powerpc/nodes: On systems like PowerPC which allow 'hot-add' of CPU or memory resources, it may occur that the new resources are to be inserted into nodes that were not used for these resources at bootup. In the kernel, any node that is used must be defined and initialized at boot. This patch extracts the value of the lowest domain level (number of allocable resources) from the "rtas" device tree property "ibm,current-associativity-domains" or the device tree property "ibm,max-associativity-domains" to use as the maximum number of nodes to setup as possibly available in the system. This new setting will override the instruction, nodes_and(node_possible_map, node_possible_map, node_online_map); presently seen in the function arch/powerpc/mm/numa.c:initmem_init(). If the property is not present at boot, no operation will be performed to define or enable additional nodes. Signed-off-by: Michael Bringmann --- arch/powerpc/mm/numa.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c index ec098b3..b385cd0 100644 --- a/arch/powerpc/mm/numa.c +++ b/arch/powerpc/mm/numa.c @@ -892,6 +892,51 @@ static void __init setup_node_data(int nid, u64 start_pfn, u64 end_pfn) NODE_DATA(nid)->node_spanned_pages = spanned_pages; } +static void __init node_associativity_setup(void) +{ + struct device_node *rtas; + + rtas = of_find_node_by_path("/rtas"); + if (rtas) { + const __be32 *prop; + u32 len, entries, numnodes, i; + + prop = of_get_property(rtas, + "ibm,current-associativity-domains", &len); + if (!prop || len < sizeof(unsigned int)) { + prop = of_get_property(rtas, + "ibm,max-associativity-domains", &len); + goto endit; + } + + entries = of_read_number(prop++, 1); + + if (len < (entries * sizeof(unsigned int))) + goto endit; + + if ((0 <= min_common_depth) && (min_common_depth <= (entries-1))) + entries = min_common_depth; + else + entries -= 1; + + numnodes = of_read_number(&prop[entries], 1); + + printk(KERN_INFO "numa: Nodes = %d (mcd = %d)\n", numnodes, + min_common_depth); + + for (i = 0; i < numnodes; i++) { + if (!node_possible(i)) { + setup_node_data(i, 0, 0); + node_set(i, node_possible_map); + } + } + } + +endit: + if (rtas) + of_node_put(rtas); +} + void __init initmem_init(void) { int nid, cpu; @@ -911,6 +956,8 @@ void __init initmem_init(void) */ nodes_and(node_possible_map, node_possible_map, node_online_map); + node_associativity_setup(); + for_each_online_node(nid) { unsigned long start_pfn, end_pfn; From patchwork Mon Sep 18 18:28:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Bringmann X-Patchwork-Id: 815077 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xwvtX49Rdz9s7p for ; Tue, 19 Sep 2017 04:39:32 +1000 (AEST) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3xwvtX1bt4zDqGq for ; Tue, 19 Sep 2017 04:39:32 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=mwb@linux.vnet.ibm.com; receiver=) Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3xwvf972K9zDqGs for ; Tue, 19 Sep 2017 04:28:49 +1000 (AEST) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v8IIOQTF085090 for ; Mon, 18 Sep 2017 14:28:47 -0400 Received: from e11.ny.us.ibm.com (e11.ny.us.ibm.com [129.33.205.201]) by mx0a-001b2d01.pphosted.com with ESMTP id 2d2ea04hm1-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 18 Sep 2017 14:28:46 -0400 Received: from localhost by e11.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 18 Sep 2017 14:28:46 -0400 Received: from b01cxnp22034.gho.pok.ibm.com (9.57.198.24) by e11.ny.us.ibm.com (146.89.104.198) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 18 Sep 2017 14:28:43 -0400 Received: from b01ledav005.gho.pok.ibm.com (b01ledav005.gho.pok.ibm.com [9.57.199.110]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v8IIShbm52297742; Mon, 18 Sep 2017 18:28:43 GMT Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F1FB2AE04B; Mon, 18 Sep 2017 14:29:13 -0400 (EDT) Received: from oc1554177480.ibm.com (unknown [9.53.92.236]) by b01ledav005.gho.pok.ibm.com (Postfix) with ESMTP id 75782AE034; Mon, 18 Sep 2017 14:29:13 -0400 (EDT) To: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org From: Michael Bringmann Subject: [PATCH 2/2] powerpc/hotplug: Ensure nodes initialized for hotplug Organization: IBM Linux Technology Center In-Reply-To: <49626c85-d466-3f99-bde8-f459e278f6ee@linux.vnet.ibm.com> Date: Mon, 18 Sep 2017 13:28:41 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 Content-Language: en-US X-TM-AS-GCONF: 00 x-cbid: 17091818-2213-0000-0000-0000021D33C4 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007757; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000230; SDB=6.00918904; UDB=6.00461633; IPR=6.00699131; BA=6.00005595; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00017198; XFM=3.00000015; UTC=2017-09-18 18:28:44 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17091818-2214-0000-0000-000057963B11 Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-09-18_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1709180260 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Bringmann , Nathan Fontenot , John Allen Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" powerpc/hotplug: On systems like PowerPC which allow 'hot-add' of CPU, it may occur that the new resources are to be inserted into nodes that were not used for memory resources at bootup. Many different configurations of PowerPC resources may need to be supported depending upon the environment. This patch fixes some problems encountered at runtime with configurations that support memory-less nodes, but which allow CPUs to be added at and after boot. Signed-off-by: Michael Bringmann --- arch/powerpc/mm/numa.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c index b385cd0..e811dd1 100644 --- a/arch/powerpc/mm/numa.c +++ b/arch/powerpc/mm/numa.c @@ -551,7 +551,7 @@ static int numa_setup_cpu(unsigned long lcpu) nid = of_node_to_nid_single(cpu); out_present: - if (nid < 0 || !node_online(nid)) + if (nid < 0 || !node_possible(nid)) nid = first_online_node; map_cpu_to_node(lcpu, nid); @@ -1325,6 +1325,17 @@ static long vphn_get_associativity(unsigned long cpu, return rc; } +static int verify_node_preparation(int nid) +{ + if ((NODE_DATA(nid) == NULL) || + (NODE_DATA(nid)->node_spanned_pages == 0)) { + if (try_online_node(nid)) + return first_online_node; + } + + return nid; +} + /* * Update the CPU maps and sysfs entries for a single CPU when its NUMA * characteristics change. This function doesn't perform any locking and is @@ -1433,9 +1444,11 @@ int numa_update_cpu_topology(bool cpus_locked) /* Use associativity from first thread for all siblings */ vphn_get_associativity(cpu, associativity); new_nid = associativity_to_nid(associativity); - if (new_nid < 0 || !node_online(new_nid)) + if (new_nid < 0 || !node_possible(new_nid)) new_nid = first_online_node; + new_nid = verify_node_preparation(new_nid); + if (new_nid == numa_cpu_lookup_table[cpu]) { cpumask_andnot(&cpu_associativity_changes_mask, &cpu_associativity_changes_mask,