From patchwork Mon Oct 23 13:46:12 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Bringmann X-Patchwork-Id: 829305 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yLHp43w6fz9tXQ for ; Tue, 24 Oct 2017 00:49:48 +1100 (AEDT) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3yLHp4353czDrFM for ; Tue, 24 Oct 2017 00:49:48 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=mwb@linux.vnet.ibm.com; receiver=) Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3yLHk43Vb5zDqkm for ; Tue, 24 Oct 2017 00:46:20 +1100 (AEDT) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v9NDi7oQ044669 for ; Mon, 23 Oct 2017 09:46:18 -0400 Received: from e33.co.us.ibm.com (e33.co.us.ibm.com [32.97.110.151]) by mx0b-001b2d01.pphosted.com with ESMTP id 2dsfjuphys-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 23 Oct 2017 09:46:17 -0400 Received: from localhost by e33.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 23 Oct 2017 07:46:17 -0600 Received: from b03cxnp08025.gho.boulder.ibm.com (9.17.130.17) by e33.co.us.ibm.com (192.168.1.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 23 Oct 2017 07:46:15 -0600 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp08025.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v9NDkEPU65863874; Mon, 23 Oct 2017 06:46:14 -0700 Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C1E576E03A; Mon, 23 Oct 2017 07:46:14 -0600 (MDT) Received: from oc1554177480.ibm.com (unknown [9.80.192.130]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP id 7EC126E035; Mon, 23 Oct 2017 07:46:13 -0600 (MDT) To: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org In-Reply-To: <79edcd04-4e59-461c-4480-bad51ba21196@linux.vnet.ibm.com> From: Michael Bringmann Subject: [PATCH V3 2/2] pseries/initnodes: Ensure nodes initialized for hotplug Organization: IBM Linux Technology Center Date: Mon, 23 Oct 2017 08:46:12 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 Content-Language: en-US X-TM-AS-GCONF: 00 x-cbid: 17102313-0008-0000-0000-000008C0352B X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007939; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000239; SDB=6.00935307; UDB=6.00471211; IPR=6.00715573; BA=6.00005656; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00017670; XFM=3.00000015; UTC=2017-10-23 13:46:16 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17102313-0009-0000-0000-000044792F71 Message-Id: <8d171641-b2ed-77f4-fc3f-2b5e4a56c4e8@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-10-23_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1710230197 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Falcon , John Allen , Michael Bringmann , Tyrel Datwyler , Nathan Fontenot Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" To: linuxppc-dev@lists.ozlabs.org To: linux-kernel@vger.kernel.org Cc: Michael Ellerman Cc: Michael Bringmann Cc: John Allen Cc: Nathan Fontenot Subject: [PATCH V3 2/2] pseries/initnodes: Ensure nodes initialized for hotplug pseries/nodes: On pseries systems which allow 'hot-add' of CPU, it may occur that the new resources are to be inserted into nodes that were not used for memory resources at bootup. Many different configurations of PowerPC resources may need to be supported depending upon the environment. This patch fixes some problems encountered at runtime with configurations that support memory-less nodes, or that hot-add resources during system execution after boot. Signed-off-by: Michael Bringmann --- arch/powerpc/mm/numa.c | 35 +++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c index f885ab7..37f697dc1 100644 --- a/arch/powerpc/mm/numa.c +++ b/arch/powerpc/mm/numa.c @@ -551,7 +551,7 @@ static int numa_setup_cpu(unsigned long lcpu) nid = of_node_to_nid_single(cpu); out_present: - if (nid < 0 || !node_online(nid)) + if (nid < 0 || !node_possible(nid)) nid = first_online_node; map_cpu_to_node(lcpu, nid); @@ -1311,6 +1311,35 @@ static long vphn_get_associativity(unsigned long cpu, return rc; } +static int verify_node_preparation(int nid) +{ + /* + * Need to allocate/initialize NODE_DATA from a node with + * memory (see memblock_alloc_try_nid). Code executed after + * boot (like local_memory_node) often does not know enough + * to recover fully for memoryless nodes. + */ + if (NODE_DATA(nid) == NULL) { + int ret; + printk(KERN_INFO "%s:%d node %d (%d)\n", __FUNCTION__, __LINE__, nid, first_online_node); + setup_node_data(nid, 0, 0); + printk(KERN_INFO "%s:%d node %d (%d)\n", __FUNCTION__, __LINE__, nid, first_online_node); + ret = try_online_node(nid); + printk(KERN_INFO "%s:%d node %d (%d) %d %p\n", __FUNCTION__, __LINE__, nid, first_online_node, ret, NODE_DATA(nid)); + + /* + * Node structures successfully initialized, but + * we still need the node to be offline. + */ + node_set_offline(nid); + } + + if (NODE_DATA(nid)->node_spanned_pages == 0) + return first_online_node; + + return nid; +} + /* * Update the CPU maps and sysfs entries for a single CPU when its NUMA * characteristics change. This function doesn't perform any locking and is @@ -1419,9 +1448,11 @@ int numa_update_cpu_topology(bool cpus_locked) /* Use associativity from first thread for all siblings */ vphn_get_associativity(cpu, associativity); new_nid = associativity_to_nid(associativity); - if (new_nid < 0 || !node_online(new_nid)) + if (new_nid < 0 || !node_possible(new_nid)) new_nid = first_online_node; + new_nid = verify_node_preparation(new_nid); + if (new_nid == numa_cpu_lookup_table[cpu]) { cpumask_andnot(&cpu_associativity_changes_mask, &cpu_associativity_changes_mask,