From patchwork Fri Jan 4 02:58:38 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srivatsa Vaddagiri X-Patchwork-Id: 209364 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from ozlabs.org (localhost [IPv6:::1]) by ozlabs.org (Postfix) with ESMTP id 484E32C041A for ; Fri, 4 Jan 2013 15:36:34 +1100 (EST) Received: from wolverine01.qualcomm.com (wolverine01.qualcomm.com [199.106.114.254]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "wolverine01.qualcomm.com", Issuer "VeriSign Class 3 Secure Server CA" (not verified)) by ozlabs.org (Postfix) with ESMTPS id A27362C008F for ; Fri, 4 Jan 2013 14:08:29 +1100 (EST) X-IronPort-AV: E=Sophos;i="4.84,406,1355126400"; d="scan'208";a="17669315" Received: from pdmz-ns-snip_115_219.qualcomm.com (HELO mostmsg01.qualcomm.com) ([199.106.115.219]) by wolverine01.qualcomm.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 03 Jan 2013 18:58:46 -0800 Received: from svaddagi-linux.qualcomm.com (pdmz-ns-snip_218_1.qualcomm.com [192.168.218.1]) by mostmsg01.qualcomm.com (Postfix) with ESMTPA id 8CDF910004BE; Thu, 3 Jan 2013 18:58:46 -0800 (PST) From: Srivatsa Vaddagiri To: Russell King , "Paul E. McKenney" , Stephen Boyd , linux-arm-kernel@lists.infradead.org, Mike Frysinger , uclinux-dist-devel@blackfin.uclinux.org, Ralf Baechle , linux-mips@linux-mips.org, Benjamin Herrenschmidt , linuxppc-dev@lists.ozlabs.org, Martin Schwidefsky , linux-s390@vger.kernel.org, Paul Mundt , linux-sh@vger.kernel.org, "David S. Miller" , sparclinux@vger.kernel.org, Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , x86@kernel.org, mhocko@suse.cz, srivatsa.bhat@linux.vnet.ibm.com Subject: [PATCH 1/2] cpuhotplug/nohz: Remove offline cpus from nohz-idle state Date: Thu, 3 Jan 2013 18:58:38 -0800 Message-Id: <1357268318-7993-1-git-send-email-vatsa@codeaurora.org> X-Mailer: git-send-email 1.7.8.3 X-Mailman-Approved-At: Fri, 04 Jan 2013 15:35:38 +1100 Cc: linux-arm-msm@vger.kernel.org, Srivatsa Vaddagiri , linux-kernel@vger.kernel.org X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Modify idle loop of arm, mips, s390, sh and x86 architectures to exit from nohz state before dying upon hot-remove. This change is needed to avoid userspace tools like top command from seeing a rollback in total idle time over some sampling periods. Additionaly, modify idle loop on all architectures supporting cpu hotplug to have idle thread of a dying cpu die immediately after scheduler returns control to it. There is no point in wasting time via calls to *_enter()/*_exit() before noticing the need to die and dying. Additional ARM specific change: Revert commit ff081e05 ("ARM: 7457/1: smp: Fix suspicious RCU originating from cpu_die()"), which added a RCU_NONIDLE() wrapper around call to complete(). That wrapper is no longer needed as cpu_die() is now called outside of a rcu_idle_enter()/exit() section. I also think that the wait_for_completion() based wait in ARM's __cpu_die() can be replaced with a busy-loop based one, as the wait there in general should be terminated within few cycles. Cc: Russell King Cc: Paul E. McKenney Cc: Stephen Boyd Cc: linux-arm-kernel@lists.infradead.org Cc: Mike Frysinger Cc: uclinux-dist-devel@blackfin.uclinux.org Cc: Ralf Baechle Cc: linux-mips@linux-mips.org Cc: Benjamin Herrenschmidt Cc: linuxppc-dev@lists.ozlabs.org Cc: Martin Schwidefsky Cc: linux-s390@vger.kernel.org Cc: Paul Mundt Cc: linux-sh@vger.kernel.org Cc: "David S. Miller" Cc: sparclinux@vger.kernel.org Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: x86@kernel.org Cc: mhocko@suse.cz Cc: srivatsa.bhat@linux.vnet.ibm.com Signed-off-by: Srivatsa Vaddagiri --- arch/arm/kernel/process.c | 9 ++++----- arch/arm/kernel/smp.c | 2 +- arch/blackfin/kernel/process.c | 8 ++++---- arch/mips/kernel/process.c | 6 +++--- arch/powerpc/kernel/idle.c | 2 +- arch/s390/kernel/process.c | 4 ++-- arch/sh/kernel/idle.c | 5 ++--- arch/sparc/kernel/process_64.c | 3 ++- arch/x86/kernel/process.c | 5 ++--- 9 files changed, 21 insertions(+), 23 deletions(-) diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c index c6dec5f..254099b 100644 --- a/arch/arm/kernel/process.c +++ b/arch/arm/kernel/process.c @@ -191,11 +191,6 @@ void cpu_idle(void) rcu_idle_enter(); ledtrig_cpu(CPU_LED_IDLE_START); while (!need_resched()) { -#ifdef CONFIG_HOTPLUG_CPU - if (cpu_is_offline(smp_processor_id())) - cpu_die(); -#endif - /* * We need to disable interrupts here * to ensure we don't miss a wakeup call. @@ -224,6 +219,10 @@ void cpu_idle(void) rcu_idle_exit(); tick_nohz_idle_exit(); schedule_preempt_disabled(); +#ifdef CONFIG_HOTPLUG_CPU + if (cpu_is_offline(smp_processor_id())) + cpu_die(); +#endif } } diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index 84f4cbf..a8e3b8a 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -251,7 +251,7 @@ void __ref cpu_die(void) mb(); /* Tell __cpu_die() that this CPU is now safe to dispose of */ - RCU_NONIDLE(complete(&cpu_died)); + complete(&cpu_died); /* * actual CPU shutdown procedure is at least platform (if not diff --git a/arch/blackfin/kernel/process.c b/arch/blackfin/kernel/process.c index 3e16ad9..2bee1af 100644 --- a/arch/blackfin/kernel/process.c +++ b/arch/blackfin/kernel/process.c @@ -83,10 +83,6 @@ void cpu_idle(void) while (1) { void (*idle)(void) = pm_idle; -#ifdef CONFIG_HOTPLUG_CPU - if (cpu_is_offline(smp_processor_id())) - cpu_die(); -#endif if (!idle) idle = default_idle; tick_nohz_idle_enter(); @@ -98,6 +94,10 @@ void cpu_idle(void) preempt_enable_no_resched(); schedule(); preempt_disable(); +#ifdef CONFIG_HOTPLUG_CPU + if (cpu_is_offline(smp_processor_id())) + cpu_die(); +#endif } } diff --git a/arch/mips/kernel/process.c b/arch/mips/kernel/process.c index a11c6f9..41102a0 100644 --- a/arch/mips/kernel/process.c +++ b/arch/mips/kernel/process.c @@ -71,13 +71,13 @@ void __noreturn cpu_idle(void) start_critical_timings(); } } + rcu_idle_exit(); + tick_nohz_idle_exit(); + schedule_preempt_disabled(); #ifdef CONFIG_HOTPLUG_CPU if (!cpu_online(cpu) && !cpu_isset(cpu, cpu_callin_map)) play_dead(); #endif - rcu_idle_exit(); - tick_nohz_idle_exit(); - schedule_preempt_disabled(); } } diff --git a/arch/powerpc/kernel/idle.c b/arch/powerpc/kernel/idle.c index ea78761..39ad029 100644 --- a/arch/powerpc/kernel/idle.c +++ b/arch/powerpc/kernel/idle.c @@ -102,11 +102,11 @@ void cpu_idle(void) ppc64_runlatch_on(); rcu_idle_exit(); tick_nohz_idle_exit(); + schedule_preempt_disabled(); if (cpu_should_die()) { sched_preempt_enable_no_resched(); cpu_die(); } - schedule_preempt_disabled(); } } diff --git a/arch/s390/kernel/process.c b/arch/s390/kernel/process.c index 536d645..5290556 100644 --- a/arch/s390/kernel/process.c +++ b/arch/s390/kernel/process.c @@ -66,8 +66,6 @@ unsigned long thread_saved_pc(struct task_struct *tsk) */ static void default_idle(void) { - if (cpu_is_offline(smp_processor_id())) - cpu_die(); local_irq_disable(); if (need_resched()) { local_irq_enable(); @@ -95,6 +93,8 @@ void cpu_idle(void) if (test_thread_flag(TIF_MCCK_PENDING)) s390_handle_mcck(); schedule_preempt_disabled(); + if (cpu_is_offline(smp_processor_id())) + cpu_die(); } } diff --git a/arch/sh/kernel/idle.c b/arch/sh/kernel/idle.c index 0c91016..f8bc2f0 100644 --- a/arch/sh/kernel/idle.c +++ b/arch/sh/kernel/idle.c @@ -96,9 +96,6 @@ void cpu_idle(void) check_pgt_cache(); rmb(); - if (cpu_is_offline(cpu)) - play_dead(); - local_irq_disable(); /* Don't trace irqs off for idle */ stop_critical_timings(); @@ -115,6 +112,8 @@ void cpu_idle(void) rcu_idle_exit(); tick_nohz_idle_exit(); schedule_preempt_disabled(); + if (cpu_is_offline(cpu)) + play_dead(); } } diff --git a/arch/sparc/kernel/process_64.c b/arch/sparc/kernel/process_64.c index cdb80b2..01589e7 100644 --- a/arch/sparc/kernel/process_64.c +++ b/arch/sparc/kernel/process_64.c @@ -105,13 +105,14 @@ void cpu_idle(void) rcu_idle_exit(); tick_nohz_idle_exit(); + schedule_preempt_disabled(); + #ifdef CONFIG_HOTPLUG_CPU if (cpu_is_offline(cpu)) { sched_preempt_enable_no_resched(); cpu_play_dead(); } #endif - schedule_preempt_disabled(); } } diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c index 2ed787f..3d5f142 100644 --- a/arch/x86/kernel/process.c +++ b/arch/x86/kernel/process.c @@ -331,9 +331,6 @@ void cpu_idle(void) while (!need_resched()) { rmb(); - if (cpu_is_offline(smp_processor_id())) - play_dead(); - /* * Idle routines should keep interrupts disabled * from here on, until they go to idle. @@ -366,6 +363,8 @@ void cpu_idle(void) preempt_enable_no_resched(); schedule(); preempt_disable(); + if (cpu_is_offline(smp_processor_id())) + play_dead(); } }