From patchwork Mon Nov 5 09:23:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taehee Yoo X-Patchwork-Id: 992961 X-Patchwork-Delegate: pablo@netfilter.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netfilter-devel-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="F44CffVV"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 42pS086JBrz9sCV for ; Mon, 5 Nov 2018 20:23:20 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726706AbeKESmG (ORCPT ); Mon, 5 Nov 2018 13:42:06 -0500 Received: from mail-pf1-f196.google.com ([209.85.210.196]:34540 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726086AbeKESmG (ORCPT ); Mon, 5 Nov 2018 13:42:06 -0500 Received: by mail-pf1-f196.google.com with SMTP id y18-v6so1744519pfn.1 for ; Mon, 05 Nov 2018 01:23:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=dwQpDluU+eVRCqqVka/QjiiyXYADK+6lov+EskjrOdw=; b=F44CffVVJ78POPhyhspzoqswND4XfcJN6xN/t8Gf31y9bfLuwMMT5WV9YGClM3UIOo JDUvyyBpqJeY9zoGgs/syKjZ2ZXZuBnbLHkZOdk9Dd8WzFesLccygkuw4BMM9pxwz5nG MkAgELwyXdtwzwYsW9H3YJaZsQF6K9gTrhdaXFnbB+JnfxeCo0Ujx06gEI8UNnWeaeDg dhwIMlYh5BnFR99MpKYjQ2xYzb/fGvxcesYbbjIGYX9HeEuH0WnxTRD3nzJa+1lA9iaf 24Wk5/21Io6HxdJlCrcesk3B2+9eZYpZUHMAbS+u5OmQqzyRwv8lWD81EQ0mOpLk14eT Gruw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=dwQpDluU+eVRCqqVka/QjiiyXYADK+6lov+EskjrOdw=; b=i1qHQQLUwpqJGPHi34ET2emq22sZNX6Ma1tJMIbzquLSAKLmO+k7cWJuoVXbgOTWNw WmWs4pYuQMkb9ElIgpenvO4PQBnavnKYumX+WhI7/9F7XQaAmDNaHDfQitWBpEkbkqV2 vP+BuMQru3pwxQ1xl0RXPitEp8mfrHu9HTX29Of6EPYz3GHtHhiobdpuUxMKRAlS12pU juDuv5NzNcF/Iu01eBVLCtWuU/1JpgSsSt38Qk90/W/YUEGHo0jUnDgv8Mw2extXkX/v clhp2hwHTtGV9ffDd4TCvUu1dlXGubGx8kUYuJK8hubfXKBHSffkxkBB3Z8ysIQRy2uT 6chw== X-Gm-Message-State: AGRZ1gK8H/6cJs9JZLWrBsUJd9aoM/HQK6W5m5HLCa3bw68orOYpFaMQ aYHj5VZFvSsyPwKL8/GNwXpNBefA X-Google-Smtp-Source: AJdET5erCIrdDC3aIFgYXloPGLQb9GI07EuDSps2N1Wtc3HO/ZaLcJZwmCoTcdAbENX+LgYAfqw5Ig== X-Received: by 2002:a63:b218:: with SMTP id x24mr18729542pge.223.1541409799478; Mon, 05 Nov 2018 01:23:19 -0800 (PST) Received: from ap-To-be-filled-by-O-E-M.8.8.8.8 ([125.130.197.10]) by smtp.gmail.com with ESMTPSA id o83-v6sm31981368pfa.90.2018.11.05.01.23.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Nov 2018 01:23:18 -0800 (PST) From: Taehee Yoo To: pablo@netfilter.org, netfilter-devel@vger.kernel.org Cc: ap420073@gmail.com Subject: [PATCH nf v3 3/4] netfilter: ipt_CLUSTERIP: fix sleep-in-atomic bug in clusterip_config_entry_put() Date: Mon, 5 Nov 2018 18:23:13 +0900 Message-Id: <20181105092313.25284-1-ap420073@gmail.com> X-Mailer: git-send-email 2.17.1 Sender: netfilter-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netfilter-devel@vger.kernel.org A proc_remove() can sleep. so that it can't be inside of spin_lock. Hence proc_remove() is moved to outside of spin_lock. and it also adds mutex to sync create and remove of proc entry(config->pde). test commands: SHELL#1 %while :; do iptables -A INPUT -p udp -i enp2s0 -d 192.168.1.100 \ --dport 9000 -j CLUSTERIP --new --hashmode sourceip \ --clustermac 01:00:5e:00:00:21 --total-nodes 3 --local-node 3; \ iptables -F; done SHELL#2 %while :; do echo +1 > /proc/net/ipt_CLUSTERIP/192.168.1.100; \ echo -1 > /proc/net/ipt_CLUSTERIP/192.168.1.100; done [ 2949.569864] BUG: sleeping function called from invalid context at kernel/sched/completion.c:99 [ 2949.579944] in_atomic(): 1, irqs_disabled(): 0, pid: 5472, name: iptables [ 2949.587920] 1 lock held by iptables/5472: [ 2949.592711] #0: 000000008f0ebcf2 (&(&cn->lock)->rlock){+...}, at: refcount_dec_and_lock+0x24/0x50 [ 2949.603307] CPU: 1 PID: 5472 Comm: iptables Tainted: G W 4.19.0-rc5+ #16 [ 2949.604212] Hardware name: To be filled by O.E.M. To be filled by O.E.M./Aptio CRB, BIOS 5.6.5 07/08/2015 [ 2949.604212] Call Trace: [ 2949.604212] dump_stack+0xc9/0x16b [ 2949.604212] ? show_regs_print_info+0x5/0x5 [ 2949.604212] ___might_sleep+0x2eb/0x420 [ 2949.604212] ? set_rq_offline.part.87+0x140/0x140 [ 2949.604212] ? _rcu_barrier_trace+0x400/0x400 [ 2949.604212] wait_for_completion+0x94/0x710 [ 2949.604212] ? wait_for_completion_interruptible+0x780/0x780 [ 2949.604212] ? __kernel_text_address+0xe/0x30 [ 2949.604212] ? __lockdep_init_map+0x10e/0x5c0 [ 2949.604212] ? __lockdep_init_map+0x10e/0x5c0 [ 2949.604212] ? __init_waitqueue_head+0x86/0x130 [ 2949.604212] ? init_wait_entry+0x1a0/0x1a0 [ 2949.604212] proc_entry_rundown+0x208/0x270 [ 2949.604212] ? proc_reg_get_unmapped_area+0x370/0x370 [ 2949.604212] ? __lock_acquire+0x4500/0x4500 [ 2949.604212] ? complete+0x18/0x70 [ 2949.604212] remove_proc_subtree+0x143/0x2a0 [ 2949.708655] ? remove_proc_entry+0x390/0x390 [ 2949.708655] clusterip_tg_destroy+0x27a/0x630 [ipt_CLUSTERIP] [ ... ] v3: add Fourth patch. v2: - use spin_lock_bh() instead of spin_lock() (Pablo Neira Ayuso) - add missing dev_mc_add() and dev_mc_del(). - add Third patch. v1: Initial patch Fixes: b3e456fce9f5 ("netfilter: ipt_CLUSTERIP: fix a race condition of proc file creation") Signed-off-by: Taehee Yoo --- net/ipv4/netfilter/ipt_CLUSTERIP.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/net/ipv4/netfilter/ipt_CLUSTERIP.c b/net/ipv4/netfilter/ipt_CLUSTERIP.c index e734a57cd9f1..7fd399751c2e 100644 --- a/net/ipv4/netfilter/ipt_CLUSTERIP.c +++ b/net/ipv4/netfilter/ipt_CLUSTERIP.c @@ -56,7 +56,7 @@ struct clusterip_config { #endif enum clusterip_hashmode hash_mode; /* which hashing mode */ u_int32_t hash_initval; /* hash initialization */ - struct rcu_head rcu; + struct rcu_head rcu; /* for call_rcu_bh */ struct net *net; /* netns for pernet list */ char ifname[IFNAMSIZ]; /* device ifname */ }; @@ -72,6 +72,8 @@ struct clusterip_net { #ifdef CONFIG_PROC_FS struct proc_dir_entry *procdir; + /* mutex protects the config->pde*/ + struct mutex mutex; #endif }; @@ -118,17 +120,18 @@ clusterip_config_entry_put(struct clusterip_config *c) local_bh_disable(); if (refcount_dec_and_lock(&c->entries, &cn->lock)) { + list_del_rcu(&c->list); + spin_unlock(&cn->lock); + local_bh_enable(); /* In case anyone still accesses the file, the open/close * functions are also incrementing the refcount on their own, * so it's safe to remove the entry even if it's in use. */ #ifdef CONFIG_PROC_FS + mutex_lock(&cn->mutex); if (cn->procdir) proc_remove(c->pde); + mutex_unlock(&cn->mutex); #endif - list_del_rcu(&c->list); - spin_unlock(&cn->lock); - local_bh_enable(); - return; } local_bh_enable(); @@ -278,9 +281,11 @@ clusterip_config_init(struct net *net, const struct ipt_clusterip_tgt_info *i, /* create proc dir entry */ sprintf(buffer, "%pI4", &ip); + mutex_lock(&cn->mutex); c->pde = proc_create_data(buffer, 0600, cn->procdir, &clusterip_proc_fops, c); + mutex_unlock(&cn->mutex); if (!c->pde) { err = -ENOMEM; goto err; @@ -832,6 +837,7 @@ static int clusterip_net_init(struct net *net) pr_err("Unable to proc dir entry\n"); return -ENOMEM; } + mutex_init(&cn->mutex); #endif /* CONFIG_PROC_FS */ return 0; @@ -840,9 +846,12 @@ static int clusterip_net_init(struct net *net) static void clusterip_net_exit(struct net *net) { struct clusterip_net *cn = clusterip_pernet(net); + #ifdef CONFIG_PROC_FS + mutex_lock(&cn->mutex); proc_remove(cn->procdir); cn->procdir = NULL; + mutex_unlock(&cn->mutex); #endif nf_unregister_net_hook(net, &cip_arp_ops); }