mbox series

[nf,0/2] netfilter: ipt_CLUSTERIP: fix bugs in ipt_CLUSTERIP

Message ID 20181005164216.9452-1-ap420073@gmail.com
Headers show
Series netfilter: ipt_CLUSTERIP: fix bugs in ipt_CLUSTERIP | expand

Message

Taehee Yoo Oct. 5, 2018, 4:42 p.m. UTC
This patchset fixes bug in ipt_CLUSTERIP when network namespace is destroyed.
First patch fixes deadlock.
Second patch removes wrong warning message.

test commands:
   test commands:
   %ip netns add vm1
   %ip netns exec vm1 bash
   %ip link set lo up
   %iptables -A INPUT -p tcp -i lo -d 192.168.0.5 --dport 80 \
	 -j CLUSTERIP --new --hashmode sourceip \
	 --clustermac 01:00:5e:00:00:20 --total-nodes 2 --local-node 1
   %exit
   %ip netns del vm1

splat looks like:
[  341.184508] WARNING: CPU: 1 PID: 87 at net/ipv4/netfilter/ipt_CLUSTERIP.c:840 clusterip_net_exit+0x319/0x380 [ipt_CLUSTERIP]
[  341.184850] Modules linked in: ipt_CLUSTERIP nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_tcpudp iptable_filter bpfilter ip_tables x_tables
[  341.184850] CPU: 1 PID: 87 Comm: kworker/u4:2 Not tainted 4.19.0-rc5+ #16
[  341.184850] Hardware name: To be filled by O.E.M. To be filled by O.E.M./Aptio CRB, BIOS 5.6.5 07/08/2015
[  341.227509] Workqueue: netns cleanup_net
[  341.227509] RIP: 0010:clusterip_net_exit+0x319/0x380 [ipt_CLUSTERIP]
[  341.227509] Code: 0f 85 7f fe ff ff 48 c7 c2 80 64 2c c0 be a8 02 00 00 48 c7 c7 a0 63 2c c0 c6 05 18 6e 00 00 01 e8 bc 38 ff f5 e9 5b fe ff ff <0f> 0b e9 33 ff ff ff e8 4b 90 50 f6 e9 2d fe ff ff 48 89 df e8 de
[  341.227509] RSP: 0018:ffff88011086f408 EFLAGS: 00010202
[  341.227509] RAX: dffffc0000000000 RBX: 1ffff1002210de85 RCX: 0000000000000000
[  341.227509] RDX: 1ffff1002210de85 RSI: ffff880110813be8 RDI: ffffed002210de58
[  341.227509] RBP: ffff88011086f4d0 R08: 0000000000000000 R09: 0000000000000000
[  341.227509] R10: 0000000000000000 R11: 0000000000000000 R12: 1ffff1002210de81
[  341.227509] R13: ffff880110625a48 R14: ffff880114cec8c8 R15: 0000000000000014
[  341.227509] FS:  0000000000000000(0000) GS:ffff880116600000(0000) knlGS:0000000000000000
[  341.227509] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  341.227509] CR2: 00007f11fd38e000 CR3: 000000013ca16000 CR4: 00000000001006e0
[  341.227509] Call Trace:
[  341.227509]  ? __clusterip_config_find+0x460/0x460 [ipt_CLUSTERIP]
[  341.227509]  ? default_device_exit+0x1ca/0x270
[  341.227509]  ? remove_proc_entry+0x1cd/0x390
[  341.227509]  ? dev_change_net_namespace+0xd00/0xd00
[  341.227509]  ? __init_waitqueue_head+0x130/0x130
[  341.227509]  ops_exit_list.isra.10+0x94/0x140
[  341.227509]  cleanup_net+0x45b/0x900
[  341.227509]  ? net_drop_ns+0x110/0x110
[  341.227509]  ? swapgs_restore_regs_and_return_to_usermode+0x3c/0x80
[  341.227509]  ? save_trace+0x300/0x300
[  341.227509]  ? lock_acquire+0x1e3/0x470
[  341.227509]  ? _raw_spin_unlock_irq+0x29/0x40
[  341.227509]  ? _raw_spin_unlock_irq+0x29/0x40
[  341.227509]  ? __lock_acquire+0x4500/0x4500
[  341.227509]  ? __lock_is_held+0xb4/0x140
[  341.227509]  process_one_work+0xc13/0x1de0
[  341.227509]  ? pwq_dec_nr_in_flight+0x3c0/0x3c0
[  341.227509]  ? set_load_weight+0x270/0x270
[  341.227509]  ? __switch_to_asm+0x24/0x60
[  341.227509]  ? __switch_to_asm+0x30/0x60
[  341.227509]  ? __switch_to_asm+0x24/0x60
[  341.227509]  ? __switch_to_asm+0x24/0x60
[  341.227509]  ? __switch_to_asm+0x30/0x60
[  341.227509]  ? __switch_to_asm+0x24/0x60
[  341.227509]  ? __switch_to_asm+0x24/0x60
[  341.227509]  ? __switch_to_asm+0x30/0x60
[  341.227509]  ? __switch_to_asm+0x24/0x60
[  341.227509]  ? __switch_to_asm+0x24/0x60
[  341.227509]  ? __switch_to_asm+0x30/0x60
[  341.227509]  ? __switch_to_asm+0x24/0x60
[  341.227509]  ? __schedule+0x695/0x1e40
[  341.227509]  ? trace_hardirqs_on_thunk+0x1a/0x1c
[  341.227509]  ? __sched_text_start+0x8/0x8
[  341.227509]  ? cyc2ns_read_end+0x10/0x10
[  341.227509]  ? save_trace+0x300/0x300
[  341.227509]  ? sched_clock_local+0xd4/0x140
[  341.227509]  ? find_held_lock+0x39/0x1c0
[  341.227509]  ? worker_thread+0x353/0x1120
[  341.227509]  ? lock_contended+0xdb0/0xdb0
[  341.227509]  ? __lock_acquire+0x4500/0x4500
[  341.227509]  ? do_raw_spin_trylock+0x101/0x1a0
[  341.227509]  ? do_raw_spin_lock+0x1f0/0x1f0
[  341.227509]  worker_thread+0x15d/0x1120
[  341.227509]  ? process_one_work+0x1de0/0x1de0
[  341.227509]  ? cyc2ns_read_end+0x10/0x10
[  341.227509]  ? save_trace+0x300/0x300
[  341.227509]  ? cyc2ns_read_end+0x10/0x10
[  341.227509]  ? kasan_kmalloc+0xa0/0xd0
[  341.227509]  ? sched_clock_local+0xd4/0x140
[  341.227509]  ? find_held_lock+0x39/0x1c0
[  341.227509]  ? check_flags.part.36+0x450/0x450
[  341.227509]  ? _raw_spin_unlock_irqrestore+0x32/0x50
[  341.227509]  ? __kthread_parkme+0x44/0x180
[  341.227509]  ? __bpf_trace_preemptirq_template+0x10/0x10
[  341.227509]  ? __kthread_parkme+0xb6/0x180
[  341.227509]  ? process_one_work+0x1de0/0x1de0
[  341.227509]  kthread+0x322/0x3e0
[  341.227509]  ? kthread_create_worker_on_cpu+0xc0/0xc0
[  341.227509]  ret_from_fork+0x3a/0x50
[  341.227509] irq event stamp: 61102
[  341.227509] hardirqs last  enabled at (61101): [<ffffffffb63408da>] __call_rcu.constprop.72+0x40a/0x1010
[  341.227509] hardirqs last disabled at (61102): [<ffffffffb6004979>] trace_hardirqs_off_thunk+0x1a/0x1c
[  341.227509] softirqs last  enabled at (61062): [<ffffffffb82006db>] __do_softirq+0x6db/0xa11
[  341.227509] softirqs last disabled at (61045): [<ffffffffb61801ae>] irq_exit+0x19e/0x1d0
[  341.227509] ---[ end trace f9d42961412cbe53 ]---

[  399.083884] INFO: task kworker/u4:2:87 blocked for more than 30 seconds.
[  399.091797]       Tainted: G        W         4.19.0-rc5+ #16
[  399.098707] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  399.107640] kworker/u4:2    D17096    87      2 0x80000000
[  399.113957] Workqueue: netns cleanup_net
[  399.118474] Call Trace:
[  399.121347]  ? __schedule+0x68d/0x1e40
[  399.125688]  ? __sched_text_start+0x8/0x8
[  399.130301]  ? dump_stack+0x13a/0x16b
[  399.134524]  ? trace_hardirqs_off+0x6b/0x220
[  399.139438]  ? save_trace+0x300/0x300
[  399.143669]  ? ret_from_fork+0x3a/0x50
[  399.148001]  ? dump_stack+0x13a/0x16b
[  399.152237]  ? find_held_lock+0x13f/0x1c0
[  399.156868]  ? debug_locks_off+0x11/0x80
[  399.161395]  schedule+0xe7/0x380
[  399.165133]  ? __schedule+0x1e40/0x1e40
[  399.169559]  ? _raw_spin_unlock_irq+0x29/0x40
[  399.174659]  ? trace_hardirqs_on+0x93/0x210
[  399.179470]  ? __bpf_trace_preemptirq_template+0x10/0x10
[  399.185646]  __rwsem_down_write_failed_common+0x786/0x1540
[  399.191932]  ? rwsem_spin_on_owner+0x600/0x600
[  399.197057]  ? debug_show_all_locks+0x500/0x500
[  399.202257]  ? debug_show_all_locks+0x500/0x500
[  399.207455]  ? trace_hardirqs_on_thunk+0x1a/0x1c
[  399.212754]  ? __lock_is_held+0xb4/0x140
[  399.217275]  ? ftrace_regs_caller_op_ptr+0x6b/0x6b
[  399.222765]  ? ftrace_caller_op_ptr+0xe/0xe
[  399.227628]  ? iptable_filter_net_exit+0x42/0x80 [iptable_filter]
[  399.234635]  ? stop_critical_timings+0x420/0x420
[  399.239933]  ? rcu_read_lock_sched_held+0x114/0x130
[  399.245528]  ? print_irqtrace_events+0x270/0x270
[  399.250836]  ? retint_kernel+0x10/0x10
[  399.255155]  ? trace_hardirqs_on_thunk+0x1a/0x1c
[  399.260449]  ? retint_kernel+0x10/0x10
[  399.264765]  ? trace_hardirqs_on_caller+0x9a/0x220
[  399.270262]  ? cyc2ns_read_end+0x10/0x10
[  399.274790]  ? save_trace+0x300/0x300
[  399.279030]  ? sched_clock_local+0xd4/0x140
[  399.283861]  ? sched_clock_cpu+0x126/0x170
[  399.288624]  ? find_held_lock+0x39/0x1c0
[  399.293141]  ? unregister_netdevice_notifier+0x8c/0x460
[  399.299130]  ? lock_acquire+0x1e3/0x470
[  399.303627]  ? __lock_acquire+0x4500/0x4500
[  399.308445]  ? call_rwsem_down_write_failed+0x13/0x20
[  399.314228]  call_rwsem_down_write_failed+0x13/0x20
[  399.319825]  ? cleanup_net+0x119/0x900
[  399.324152]  down_write+0xd3/0x160
[  399.328086]  ? down_read+0x1e0/0x1e0
[  399.332197]  ? sched_clock_cpu+0x126/0x170
[  399.336906]  ? find_held_lock+0x39/0x1c0
[  399.341437]  unregister_netdevice_notifier+0x8c/0x460
[  399.347235]  ? register_netdevice_notifier+0x790/0x790
[  399.353118]  ? __local_bh_enable_ip+0xe9/0x1b0
[  399.358216]  ? __local_bh_enable_ip+0xe9/0x1b0
[  399.363322]  ? clusterip_tg_destroy+0x372/0x650 [ipt_CLUSTERIP]
[  399.370081]  ? trace_hardirqs_on+0x93/0x210
[  399.374884]  ? __bpf_trace_preemptirq_template+0x10/0x10
[  399.380981]  ? clusterip_tg_destroy+0x372/0x650 [ipt_CLUSTERIP]
[  399.387759]  clusterip_tg_destroy+0x3ad/0x650 [ipt_CLUSTERIP]
[  399.394344]  ? clusterip_net_init+0x3d0/0x3d0 [ipt_CLUSTERIP]
[  399.400924]  ? cleanup_match+0x17d/0x200 [ip_tables]
[  399.406681]  ? xt_unregister_table+0x215/0x300 [x_tables]
[  399.412860]  ? kfree+0xe2/0x2a0
[  399.416510]  cleanup_entry+0x1d5/0x2f0 [ip_tables]
[  399.422015]  ? cleanup_match+0x200/0x200 [ip_tables]
[  399.427726]  __ipt_unregister_table+0x9b/0x1a0 [ip_tables]
[  399.434014]  iptable_filter_net_exit+0x43/0x80 [iptable_filter]
[  399.440788]  ops_exit_list.isra.10+0x94/0x140
[  399.445813]  cleanup_net+0x45b/0x900
[  399.449955]  ? net_drop_ns+0x110/0x110
[  399.454281]  ? swapgs_restore_regs_and_return_to_usermode+0x3c/0x80
[  399.461453]  ? save_trace+0x300/0x300
[  399.465676]  ? lock_acquire+0x1e3/0x470
[  399.470105]  ? _raw_spin_unlock_irq+0x29/0x40
[  399.475102]  ? _raw_spin_unlock_irq+0x29/0x40
[  399.480109]  ? __lock_acquire+0x4500/0x4500
[  399.484930]  ? __lock_is_held+0xb4/0x140
[  399.489468]  process_one_work+0xc13/0x1de0
[  399.494206]  ? pwq_dec_nr_in_flight+0x3c0/0x3c0
[  399.499411]  ? set_load_weight+0x270/0x270
[  399.504112]  ? __switch_to_asm+0x24/0x60
[  399.508674]  ? __switch_to_asm+0x30/0x60
[  399.513183]  ? __switch_to_asm+0x24/0x60
[  399.517696]  ? __switch_to_asm+0x24/0x60
[  399.522201]  ? __switch_to_asm+0x30/0x60
[  399.526711]  ? __switch_to_asm+0x24/0x60
[  399.531228]  ? __switch_to_asm+0x24/0x60
[  399.535739]  ? __switch_to_asm+0x30/0x60
[  399.540246]  ? __switch_to_asm+0x24/0x60
[  399.544761]  ? __switch_to_asm+0x24/0x60
[  399.549263]  ? __switch_to_asm+0x30/0x60
[  399.553775]  ? __switch_to_asm+0x24/0x60
[  399.558298]  ? __schedule+0x695/0x1e40
[  399.562652]  ? trace_hardirqs_on_thunk+0x1a/0x1c
[  399.567963]  ? __sched_text_start+0x8/0x8
[  399.572624]  ? cyc2ns_read_end+0x10/0x10
[  399.577146]  ? save_trace+0x300/0x300
[  399.581379]  ? sched_clock_local+0xd4/0x140
[  399.586198]  ? find_held_lock+0x39/0x1c0
[  399.590730]  ? worker_thread+0x353/0x1120
[  399.595355]  ? lock_contended+0xdb0/0xdb0
[  399.599974]  ? __lock_acquire+0x4500/0x4500
[  399.604792]  ? do_raw_spin_trylock+0x101/0x1a0
[  399.609894]  ? do_raw_spin_lock+0x1f0/0x1f0
[  399.614728]  worker_thread+0x15d/0x1120
[  399.619177]  ? process_one_work+0x1de0/0x1de0
[  399.624187]  ? cyc2ns_read_end+0x10/0x10
[  399.628699]  ? save_trace+0x300/0x300
[  399.632922]  ? cyc2ns_read_end+0x10/0x10
[  399.637434]  ? kasan_kmalloc+0xa0/0xd0
[  399.641766]  ? sched_clock_local+0xd4/0x140
[  399.646638]  ? find_held_lock+0x39/0x1c0
[  399.651177]  ? check_flags.part.36+0x450/0x450
[  399.656284]  ? _raw_spin_unlock_irqrestore+0x32/0x50
[  399.661968]  ? __kthread_parkme+0x44/0x180
[  399.666688]  ? __bpf_trace_preemptirq_template+0x10/0x10
[  399.672787]  ? __kthread_parkme+0xb6/0x180
[  399.677504]  ? process_one_work+0x1de0/0x1de0
[  399.682501]  kthread+0x322/0x3e0
[  399.686238]  ? kthread_create_worker_on_cpu+0xc0/0xc0
[  399.692027]  ret_from_fork+0x3a/0x50
[  399.696210] INFO: lockdep is turned off.


Taehee Yoo (2):
  netfilter: ipt_CLUSTERIP: fix deadlock in netns exit routine
  netfilter: ipt_CLUSTERIP: remove wrong WARN_ON_ONCE in netns
    exit routine

 net/ipv4/netfilter/ipt_CLUSTERIP.c | 72 +++++++++++++++++-------------
 1 file changed, 40 insertions(+), 32 deletions(-)