From patchwork Fri Jan 17 15:18:29 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Christoph Lameter (Ampere)" X-Patchwork-Id: 312120 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id BF58F2C0079 for ; Sat, 18 Jan 2014 02:30:23 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752726AbaAQP3H (ORCPT ); Fri, 17 Jan 2014 10:29:07 -0500 Received: from qmta07.emeryville.ca.mail.comcast.net ([76.96.30.64]:57472 "EHLO qmta07.emeryville.ca.mail.comcast.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753335AbaAQP1Q (ORCPT ); Fri, 17 Jan 2014 10:27:16 -0500 Received: from omta02.emeryville.ca.mail.comcast.net ([76.96.30.19]) by qmta07.emeryville.ca.mail.comcast.net with comcast id F3061n0090QkzPwA73TG82; Fri, 17 Jan 2014 15:27:16 +0000 Received: from gentwo.org ([98.213.233.247]) by omta02.emeryville.ca.mail.comcast.net with comcast id F3TD1n00Q5Lw0ES8N3TEaV; Fri, 17 Jan 2014 15:27:15 +0000 Received: by gentwo.org (Postfix, from userid 1001) id 366782F0EE; Fri, 17 Jan 2014 09:18:36 -0600 (CST) Message-Id: <20140117151836.140608046@linux.com> Date: Fri, 17 Jan 2014 09:18:29 -0600 From: Christoph Lameter To: Tejun Heo Cc: akpm@linuxfoundation.org, rostedt@goodmis.org, linux-kernel@vger.kernel.org, Ingo Molnar , Peter Zijlstra , Thomas Gleixner , netdev@vger.kernel.org, davem@davemloft.net, edumazet@google.com Subject: [PATCH 17/41] net: Replace __this_cpu_inc in route.c with raw_cpu_inc References: <20140117151812.770437629@linux.com> Content-Disposition: inline; filename=preempt_rt_cache_stat DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20121106; t=1389972436; bh=Bv757SBXA+O7s8YWMqHiuju2q4f7PChFOqhynTaCcRk=; h=Received:Received:Received:Message-Id:Date:From:To:Subject: Content-Type; b=CgNHRWfP2/j6X3xk8rst7nYuvnaCytUrNKiHxPMqwssxIqYNRjiHz/C2CsLx7Q0en lDoEOOI3BnOScNEt7iVx6bXCUN/aXyQ3QMH4qn65HKWQZNoyVKaBPC16Kd7uYW6NFy izMU/GXriv4y+tLo/iN4rGPnYWZ6pCZi/BPoSy434OrQLk/63OukMOKCQwp3R/pBmx fThRDS2YweINe0wvhfyuGcYZJg9miJbURB67vYRJvHbUfbr4iIVyNeEMfpUmCrPNqU 6yq2IEj/2U+d20aD32pV4dD5FMUdzePmBR/J5sr3df/5DvtN68Drfdu0bGa2cNstb2 0Zcj15Z8sASnA== Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org [Patch depends on another patch in this series that introduces raw_cpu_ops] The RT_CACHE_STAT_INC macro triggers the new preemption checks for __this_cpu ops. I do not see any other synchronization that would allow the use of a __this_cpu operation here however in commit dbd2915ce87e811165da0717f8e159276ebb803e Andrew justifies the use of raw_smp_processor_id() here because "we do not care" about races. In the past we agreed that the price of disabling interrupts here to get consistent counters would be too high. These counters may be inaccurate due to race conditions. The use of __this_cpu op improves the situation already from what commit dbd2915ce87e811165da0717f8e159276ebb803e did since the single instruction emitted on x86 does not allow the race to occur anymore. However, non x86 platforms could still experience a race here. Trace: [ 1277.189084] __this_cpu_add operation in preemptible [00000000] code: avahi-daemon/1193 [ 1277.189085] caller is __this_cpu_preempt_check+0x38/0x60 [ 1277.189086] CPU: 1 PID: 1193 Comm: avahi-daemon Tainted: GF 3.12.0-rc4+ #187 [ 1277.189087] Hardware name: FUJITSU CELSIUS W530 Power/D3227-A1, BIOS V4.6.5.4 R1.10.0 for D3227-A1x 09/16/2013 [ 1277.189088] 0000000000000001 ffff8807ef78fa00 ffffffff816d5a57 ffff8807ef78ffd8 [ 1277.189089] ffff8807ef78fa30 ffffffff8137359c ffff8807ef78fba0 ffff88079f822b40 [ 1277.189091] 0000000020000000 ffff8807ee32c800 ffff8807ef78fa70 ffffffff813735f8 [ 1277.189093] Call Trace: [ 1277.189094] [] dump_stack+0x4e/0x82 [ 1277.189096] [] check_preemption_disabled+0xec/0x110 [ 1277.189097] [] __this_cpu_preempt_check+0x38/0x60 [ 1277.189098] [] __ip_route_output_key+0x575/0x8c0 [ 1277.189100] [] ip_route_output_flow+0x27/0x70 [ 1277.189101] [] ? ip_copy_metadata+0x1a0/0x1a0 [ 1277.189102] [] udp_sendmsg+0x825/0xa20 [ 1277.189104] [] ? do_sys_poll+0x449/0x5d0 [ 1277.189105] [] inet_sendmsg+0x85/0xc0 [ 1277.189106] [] sock_sendmsg+0x9c/0xd0 [ 1277.189108] [] ? __this_cpu_preempt_check+0x38/0x60 [ 1277.189109] [] ? move_addr_to_kernel+0x40/0xa0 [ 1277.189111] [] ___sys_sendmsg+0x37c/0x390 [ 1277.189112] [] ? string.isra.3+0x3a/0xd0 [ 1277.189113] [] ? string.isra.3+0x3a/0xd0 [ 1277.189115] [] ? vsnprintf+0x364/0x650 [ 1277.189116] [] ? snprintf+0x39/0x40 [ 1277.189118] [] ? __this_cpu_preempt_check+0x38/0x60 [ 1277.189119] [] __sys_sendmsg+0x49/0x90 [ 1277.189121] [] SyS_sendmsg+0x12/0x20 [ 1277.189122] [] tracesys+0xe1/0xe6 Cc: netdev@vger.kernel.org Cc: davem@davemloft.net Cc: edumazet@google.com Acked-by: Ingo Molnar Signed-off-by: Christoph Lameter Acked-by: David S. Miller --- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Index: linux/net/ipv4/route.c =================================================================== --- linux.orig/net/ipv4/route.c 2013-12-02 16:07:51.964573250 -0600 +++ linux/net/ipv4/route.c 2013-12-02 16:07:51.954573526 -0600 @@ -197,7 +197,7 @@ const __u8 ip_tos2prio[16] = { EXPORT_SYMBOL(ip_tos2prio); static DEFINE_PER_CPU(struct rt_cache_stat, rt_cache_stat); -#define RT_CACHE_STAT_INC(field) __this_cpu_inc(rt_cache_stat.field) +#define RT_CACHE_STAT_INC(field) raw_cpu_inc(rt_cache_stat.field) #ifdef CONFIG_PROC_FS static void *rt_cache_seq_start(struct seq_file *seq, loff_t *pos)