From patchwork Fri Jul 6 09:05:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautham R Shenoy X-Patchwork-Id: 940345 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41MTQV2pkzz9s1B for ; Fri, 6 Jul 2018 19:07:46 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 41MTQV10tvzDqrT for ; Fri, 6 Jul 2018 19:07:46 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=ego@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 41MTNT6nh3zDqFG for ; Fri, 6 Jul 2018 19:06:01 +1000 (AEST) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w6693xS7023897 for ; Fri, 6 Jul 2018 05:05:59 -0400 Received: from e12.ny.us.ibm.com (e12.ny.us.ibm.com [129.33.205.202]) by mx0a-001b2d01.pphosted.com with ESMTP id 2k221xfxwq-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 06 Jul 2018 05:05:58 -0400 Received: from localhost by e12.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 6 Jul 2018 05:05:58 -0400 Received: from b01cxnp22035.gho.pok.ibm.com (9.57.198.25) by e12.ny.us.ibm.com (146.89.104.199) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Fri, 6 Jul 2018 05:05:56 -0400 Received: from b01ledav006.gho.pok.ibm.com (b01ledav006.gho.pok.ibm.com [9.57.199.111]) by b01cxnp22035.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w6695tVJ8979120 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 6 Jul 2018 09:05:55 GMT Received: from b01ledav006.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id ACACDAC05B; Fri, 6 Jul 2018 05:06:56 -0400 (EDT) Received: from b01ledav006.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6C7B1AC059; Fri, 6 Jul 2018 05:06:56 -0400 (EDT) Received: from sofia.ibm.com (unknown [9.124.35.39]) by b01ledav006.gho.pok.ibm.com (Postfix) with ESMTP; Fri, 6 Jul 2018 05:06:56 -0400 (EDT) Received: by sofia.ibm.com (Postfix, from userid 1000) id 792C42E2F47; Fri, 6 Jul 2018 14:35:52 +0530 (IST) From: "Gautham R. Shenoy" To: Michael Ellerman , Benjamin Herrenschmidt , Michael Neuling , Vaidyanathan Srinivasan , Akshay Adiga , Shilpasri G Bhat , "Oliver O'Halloran" , Nicholas Piggin , Murilo Opsfelder Araujo Subject: [PATCH v3 0/2] powerpc: Detection and scheduler optimization for POWER9 bigcore Date: Fri, 6 Jul 2018 14:35:47 +0530 X-Mailer: git-send-email 1.8.3.1 X-TM-AS-GCONF: 00 x-cbid: 18070609-0060-0000-0000-00000288560B X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00009318; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000266; SDB=6.01057307; UDB=6.00542452; IPR=6.00835227; MB=3.00022022; MTD=3.00000008; XFM=3.00000015; UTC=2018-07-06 09:05:57 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18070609-0061-0000-0000-000045B348B0 Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-07-06_02:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1807060101 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Gautham R. Shenoy" , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: "Gautham R. Shenoy" Hi, This is the third iteration of the patchset to add support for big-core on POWER9. The previous versions can be found here: v2: https://lkml.org/lkml/2018/7/3/401 v1: https://lkml.org/lkml/2018/5/11/245 Changes : v2 --> v3 - Set sane values in the tg->property, tg->nr_groups inside parse_thread_groups before returning due to an error. - Define a helper function to determine whether a CPU device node is a big-core or not. - Updated the comments around the functions to describe the arguments passed to them. These changes were suggested by Murilo Opsfelder Araujo. v1 --> v2 - Added comments explaining the "ibm,thread-groups" device tree property. - Uses cleaner device-tree parsing functions to parse the u32 arrays. - Adds a sysfs file listing the small-core siblings for every CPU. - Enables the scheduler optimization by setting the CPU_FTR_ASYM_SMT bit in the cur_cpu_spec->cpu_features on detecting the presence of interleaved big-core. - Handles the corner case where there is only a single thread-group or when there is a single thread in a thread-group. Description: ~~~~~~~~~~~~~~~~~~~~ A pair of IBM POWER9 SMT4 cores can be fused together to form a big-core with 8 SMT threads. This can be discovered via the "ibm,thread-groups" CPU property in the device tree which will indicate which group of threads that share the L1 cache, translation cache and instruction data flow. If there are multiple such group of threads, then the core is a big-core. Furthermore, the thread-ids of such a big-core is obtained by interleaving the thread-ids of the component SMT4 cores. Eg: Threads in the pair of component SMT4 cores of an interleaved big-core are numbered {0,2,4,6} and {1,3,5,7} respectively. On such a big-core, when multiple tasks are scheduled to run on the big-core, we get the best performance when the tasks are spread across the pair of SMT4 cores. The Linux scheduler supports a flag called "SD_ASYM_PACKING" which when set in the SMT sched-domain, biases the load-balancing of the tasks on the smaller numbered threads in the core. On an big-core whose threads are interleavings of the threads of the small cores, enabling SD_ASYM_PACKING in the SMT sched-domain automatically results in spreading the tasks uniformly across the associated pair of SMT4 cores, thereby yielding better performance. This patchset contains two patches which on detecting the presence of interleaved big-cores will enable the the CPU_FTR_ASYM_SMT bit in the cur_cpu_spec->cpu_feature. Patch 1: adds support to detect the presence of big-cores and reports the small-core siblings of each CPU X via the sysfs file "/sys/devices/system/cpu/cpuX/big_core_siblings". Patch 2: checks if the thread-ids of the component small-cores are interleaved, in which case we enable the the CPU_FTR_ASYM_SMT bit in the cur_cpu_spec->cpu_features which results in the SD_ASYM_PACKING flag being set at the SMT level sched-domain. Results: ~~~~~~~~~~~~~~~~~ Experimental results for ebizzy with 2 threads, bound to a single big-core show a marked improvement with this patchset over the 4.18-rc3 vanilla kernel. The result of 100 such runs for 4.18-rc3 kernel and the 4.18-rc3 + big-core-patches are as follows 4.18-rc3: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ records/s : # samples : Histogram ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [0 - 1000000] : 0 : # [1000000 - 2000000] : 6 : ## [2000000 - 3000000] : 8 : ## [3000000 - 4000000] : 15 : #### [4000000 - 5000000] : 0 : # [5000000 - 6000000] : 71 : ############### 4.18-rc3 + big-core-patches ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ records/s : # samples : Histogram ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ [0 - 1000000] : 0 : # [1000000 - 2000000] : 0 : # [2000000 - 3000000] : 9 : ## [3000000 - 4000000] : 0 : # [4000000 - 5000000] : 0 : # [5000000 - 6000000] : 91 : ################### Gautham R. Shenoy (2): powerpc: Detect the presence of big-cores via "ibm,thread-groups" powerpc: Enable CPU_FTR_ASYM_SMT for interleaved big-cores Documentation/ABI/testing/sysfs-devices-system-cpu | 8 + arch/powerpc/include/asm/cputhreads.h | 22 ++ arch/powerpc/kernel/setup-common.c | 221 ++++++++++++++++++++- arch/powerpc/kernel/sysfs.c | 35 ++++ 4 files changed, 285 insertions(+), 1 deletion(-)