From patchwork Sun Aug 25 07:15:51 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Hao X-Patchwork-Id: 269688 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from ozlabs.org (localhost [IPv6:::1]) by ozlabs.org (Postfix) with ESMTP id E00BD2C0743 for ; Sun, 25 Aug 2013 17:18:52 +1000 (EST) Received: from mail.windriver.com (mail.windriver.com [147.11.1.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.windriver.com", Issuer "Intel External Basic Issuing CA 3A" (not verified)) by ozlabs.org (Postfix) with ESMTPS id 5B13B2C0193 for ; Sun, 25 Aug 2013 17:17:06 +1000 (EST) Received: from ALA-HCA.corp.ad.wrs.com (ala-hca.corp.ad.wrs.com [147.11.189.40]) by mail.windriver.com (8.14.5/8.14.3) with ESMTP id r7P7H32l014403 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sun, 25 Aug 2013 00:17:03 -0700 (PDT) Received: from pek-khao-d1.corp.ad.wrs.com (128.224.162.196) by ALA-HCA.corp.ad.wrs.com (147.11.189.50) with Microsoft SMTP Server id 14.2.347.0; Sun, 25 Aug 2013 00:17:02 -0700 From: Kevin Hao To: Benjamin Herrenschmidt Subject: [PATCH 4/5] powerpc: use the jump label for cpu_has_feature Date: Sun, 25 Aug 2013 15:15:51 +0800 Message-ID: <1377414952-15995-5-git-send-email-haokexin@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1377414952-15995-1-git-send-email-haokexin@gmail.com> References: <1377414952-15995-1-git-send-email-haokexin@gmail.com> MIME-Version: 1.0 Cc: linuxppc X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.16rc2 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" The cpu features are fixed once the probe of cpu features are done. And the function cpu_has_feature() does be used in some hot path. The checking of the cpu features for each time of invoking of cpu_has_feature() seems suboptimal. This tries to reduce this overhead of this check by using jump label. But we can only use the jump label for this check only after the execution of jump_label_init(), so we introduce another jump label to still do the feature check by default before all the cpu feature jump labels are initialized. Signed-off-by: Kevin Hao --- arch/powerpc/include/asm/cpufeatures.h | 28 ++++++++++++++++++++++++++++ arch/powerpc/kernel/cputable.c | 23 +++++++++++++++++++++++ 2 files changed, 51 insertions(+) diff --git a/arch/powerpc/include/asm/cpufeatures.h b/arch/powerpc/include/asm/cpufeatures.h index 37650db..598ac91 100644 --- a/arch/powerpc/include/asm/cpufeatures.h +++ b/arch/powerpc/include/asm/cpufeatures.h @@ -2,7 +2,34 @@ #define __ASM_POWERPC_CPUFEATURES_H #include +#ifdef CONFIG_JUMP_LABEL +#include +#include +#ifdef __powerpc64__ +#define MAX_CPU_FEATURES 64 +#else +#define MAX_CPU_FEATURES 32 +#endif +extern struct static_key cpu_feat_keys[MAX_CPU_FEATURES]; +extern struct static_key cpu_feat_keys_enabled; + +static inline int cpu_has_feature(unsigned long feature) +{ + if (CPU_FTRS_ALWAYS & feature) + return 1; + + if (!(CPU_FTRS_POSSIBLE | feature)) + return 0; + + if (static_key_false(&cpu_feat_keys_enabled)) { + int i = __builtin_ctzl(feature); + + return static_key_false(&cpu_feat_keys[i]); + } else + return !!(cur_cpu_spec->cpu_features & feature); +} +#else static inline int cpu_has_feature(unsigned long feature) { return (CPU_FTRS_ALWAYS & feature) || @@ -10,5 +37,6 @@ static inline int cpu_has_feature(unsigned long feature) & cur_cpu_spec->cpu_features & feature); } +#endif #endif /* __ASM_POWERPC_CPUFEATURE_H */ diff --git a/arch/powerpc/kernel/cputable.c b/arch/powerpc/kernel/cputable.c index 22973a7..2014ab7 100644 --- a/arch/powerpc/kernel/cputable.c +++ b/arch/powerpc/kernel/cputable.c @@ -21,6 +21,7 @@ #include /* for PTRRELOC on ARCH=ppc */ #include #include +#include struct cpu_spec* cur_cpu_spec = NULL; EXPORT_SYMBOL(cur_cpu_spec); @@ -2258,3 +2259,25 @@ struct cpu_spec * __init identify_cpu(unsigned long offset, unsigned int pvr) return NULL; } + +#ifdef CONFIG_JUMP_LABEL +struct static_key cpu_feat_keys[MAX_CPU_FEATURES]; +struct static_key cpu_feat_keys_enabled; + +static __init int cpu_feat_keys_init(void) +{ + int i; + + for (i = 0; i < MAX_CPU_FEATURES; i++) { + unsigned long f = 1 << i; + + if (cur_cpu_spec->cpu_features & f) + static_key_slow_inc(&cpu_feat_keys[i]); + } + + static_key_slow_inc(&cpu_feat_keys_enabled); + + return 0; +} +early_initcall(cpu_feat_keys_init); +#endif