Patchwork ppoll() stuck on POLLIN while TCP peer is sending

login
register
mail settings
Submitter Eric Dumazet
Date Jan. 9, 2013, 2:54 a.m.
Message ID <1357700082.27446.11.camel@edumazet-glaptop>
Download mbox | patch
Permalink /patch/210607/
State RFC
Delegated to: David Miller
Headers show

Comments

Eric Dumazet - Jan. 9, 2013, 2:54 a.m.
On Tue, 2013-01-08 at 18:32 -0800, Eric Dumazet wrote:

> 
> Hmm, it seems sk_filter() can return -ENOMEM because skb has the
> pfmemalloc() set.

> 
> One TCP socket keeps retransmitting an SKB via loopback, and TCP stack 
> drops the packet again and again.

sock_init_data() sets sk->sk_allocation to GFP_KERNEL

Shouldnt it use (GFP_KERNEL | __GFP_NOMEMALLOC) instead ?





--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Wong - Jan. 9, 2013, 3:55 a.m.
Eric Dumazet <erdnetdev@gmail.com> wrote:
> On Tue, 2013-01-08 at 18:32 -0800, Eric Dumazet wrote:
> > Hmm, it seems sk_filter() can return -ENOMEM because skb has the
> > pfmemalloc() set.
> 
> > 
> > One TCP socket keeps retransmitting an SKB via loopback, and TCP stack 
> > drops the packet again and again.
> 
> sock_init_data() sets sk->sk_allocation to GFP_KERNEL
> 
> Shouldnt it use (GFP_KERNEL | __GFP_NOMEMALLOC) instead ?

Thanks, things are running good after ~35 minutes so far.
Will report back if things break (hopefully I don't run out
of laptop battery power :x).

I'm now getting allocation failure warnings (which I don't believe
happened before, and should be expected, I think...)

kworker/1:1: page allocation failure: order:0, mode:0x20
Pid: 236, comm: kworker/1:1 Not tainted 3.8.0-rc2w5+ #76
Call Trace:
 <IRQ>  [<ffffffff810a2411>] warn_alloc_failed+0xe1/0x130
 [<ffffffff810a5779>] __alloc_pages_nodemask+0x5e9/0x840
 [<ffffffff8133df8d>] ? ip_rcv+0x24d/0x340
 [<ffffffff811f35b3>] ? sg_init_table+0x23/0x50
 [<ffffffffa002162a>] get_a_page.isra.25+0x3a/0x40 [virtio_net]
 [<ffffffffa0022258>] try_fill_recv+0x318/0x4a0 [virtio_net]
 [<ffffffffa00227bd>] virtnet_poll+0x3dd/0x610 [virtio_net]
 [<ffffffff8131767d>] net_rx_action+0x9d/0x1a0
 [<ffffffff8104284a>] __do_softirq+0xba/0x170
 [<ffffffff813b199c>] call_softirq+0x1c/0x30
 <EOI>  [<ffffffff8100c61d>] do_softirq+0x6d/0xa0
 [<ffffffff81042424>] local_bh_enable+0x94/0xa0
 [<ffffffff813aed45>] __cond_resched_softirq+0x35/0x50
 [<ffffffff81305e7c>] release_sock+0x9c/0x150
 [<ffffffff8134b90e>] tcp_sendmsg+0x11e/0xd80
 [<ffffffff81370cee>] inet_sendmsg+0x5e/0xa0
 [<ffffffff81300a77>] sock_sendmsg+0x87/0xa0
 [<ffffffff810a48c9>] ? __free_memcg_kmem_pages+0x9/0x10
 [<ffffffff81067819>] ? select_task_rq_fair+0x699/0x6b0
 [<ffffffff81300acb>] kernel_sendmsg+0x3b/0x50
 [<ffffffffa0052dc9>] xs_send_kvec+0x89/0xa0 [sunrpc]
 [<ffffffffa00534bf>] xs_sendpages+0x5f/0x1e0 [sunrpc]
 [<ffffffff81047d63>] ? lock_timer_base.isra.32+0x33/0x60
 [<ffffffffa00548e7>] xs_tcp_send_request+0x57/0x110 [sunrpc]
 [<ffffffffa0051c0d>] xprt_transmit+0x6d/0x260 [sunrpc]
 [<ffffffffa004f108>] call_transmit+0x1a8/0x240 [sunrpc]
 [<ffffffffa0056316>] __rpc_execute+0x56/0x250 [sunrpc]
 [<ffffffffa0056535>] rpc_async_schedule+0x25/0x40 [sunrpc]
 [<ffffffff810515cc>] process_one_work+0x12c/0x480
 [<ffffffffa0056510>] ? __rpc_execute+0x250/0x250 [sunrpc]
 [<ffffffff810538ad>] worker_thread+0x15d/0x460
 [<ffffffff81053750>] ? flush_delayed_work+0x60/0x60
 [<ffffffff8105865b>] kthread+0xbb/0xc0
 [<ffffffff810585a0>] ? kthread_create_on_node+0x120/0x120
 [<ffffffff813b063c>] ret_from_fork+0x7c/0xb0
 [<ffffffff810585a0>] ? kthread_create_on_node+0x120/0x120
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:   0
CPU    1: hi:  186, btch:  31 usd:   0
active_anon:3620 inactive_anon:3624 isolated_anon:0
 active_file:4290 inactive_file:101218 isolated_file:0
 unevictable:0 dirty:2306 writeback:0 unstable:0
 free:1711 slab_reclaimable:1529 slab_unreclaimable:5796
 mapped:2325 shmem:66 pagetables:759 bounce:0
 free_cma:0
DMA free:2012kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:0kB active_file:4kB inactive_file:13624kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15644kB managed:15900kB mlocked:0kB dirty:244kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:16kB slab_unreclaimable:80kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 488 488 488
DMA32 free:4832kB min:2784kB low:3480kB high:4176kB active_anon:14480kB inactive_anon:14496kB active_file:17156kB inactive_file:391248kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:499960kB managed:491256kB mlocked:0kB dirty:8980kB writeback:0kB mapped:9300kB shmem:264kB slab_reclaimable:6100kB slab_unreclaimable:23104kB kernel_stack:1336kB pagetables:3036kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 2*4kB (U) 1*8kB (R) 1*16kB (U) 0*32kB 1*64kB (R) 1*128kB (R) 1*256kB (R) 1*512kB (R) 1*1024kB (R) 0*2048kB 0*4096kB = 2016kB
DMA32: 207*4kB (UEM) 116*8kB (UEM) 32*16kB (UM) 58*32kB (UM) 13*64kB (UM) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4956kB
108890 total pagecache pages
3302 pages in swap cache
Swap cache stats: add 4086, delete 784, find 494/535
Free swap  = 378980kB
Total swap = 392188kB
131054 pages RAM
3820 pages reserved
541743 pages shared
117221 pages non-shared
cat: page allocation failure: order:0, mode:0x20
Pid: 23684, comm: cat Not tainted 3.8.0-rc2w5+ #76
Call Trace:
 <IRQ>  [<ffffffff810a2411>] warn_alloc_failed+0xe1/0x130
 [<ffffffff810a5779>] __alloc_pages_nodemask+0x5e9/0x840
 [<ffffffff8133df8d>] ? ip_rcv+0x24d/0x340
 [<ffffffff811f35b3>] ? sg_init_table+0x23/0x50
 [<ffffffffa002162a>] get_a_page.isra.25+0x3a/0x40 [virtio_net]
 [<ffffffffa0022258>] try_fill_recv+0x318/0x4a0 [virtio_net]
 [<ffffffffa00227bd>] virtnet_poll+0x3dd/0x610 [virtio_net]
 [<ffffffff8131767d>] net_rx_action+0x9d/0x1a0
 [<ffffffff8104284a>] __do_softirq+0xba/0x170
 [<ffffffff813b199c>] call_softirq+0x1c/0x30
 [<ffffffff8100c61d>] do_softirq+0x6d/0xa0
 [<ffffffff81042a75>] irq_exit+0xa5/0xb0
 [<ffffffff8100c25e>] do_IRQ+0x5e/0xd0
 [<ffffffff813afe2d>] common_interrupt+0x6d/0x6d
 <EOI>  [<ffffffff813af82c>] ? _raw_spin_unlock_irqrestore+0xc/0x20
 [<ffffffff810a91b6>] pagevec_lru_move_fn+0xb6/0xe0
 [<ffffffff810a8780>] ? compound_unlock_irqrestore+0x20/0x20
 [<ffffffffa00acd30>] ? nfs_read_completion+0x190/0x190 [nfs]
 [<ffffffff810a91f7>] __pagevec_lru_add+0x17/0x20
 [<ffffffff810a95c8>] __lru_cache_add+0x68/0x90
 [<ffffffff8109e869>] add_to_page_cache_lru+0x29/0x40
 [<ffffffff810a80cc>] read_cache_pages+0x6c/0x100
 [<ffffffffa00ad4dc>] nfs_readpages+0xcc/0x160 [nfs]
 [<ffffffff810a7f57>] __do_page_cache_readahead+0x1c7/0x280
 [<ffffffff810a827c>] ra_submit+0x1c/0x20
 [<ffffffff810a83ad>] ondemand_readahead+0x12d/0x250
 [<ffffffff8109f37d>] ? __generic_file_aio_write+0x1bd/0x3c0
 [<ffffffff810a8550>] page_cache_async_readahead+0x80/0xa0
 [<ffffffff8109e0b8>] ? find_get_page+0x28/0xd0
 [<ffffffff8109fb73>] generic_file_aio_read+0x503/0x6c0
 [<ffffffffa00a4231>] nfs_file_read+0x91/0xb0 [nfs]
 [<ffffffff810e4477>] do_sync_read+0xa7/0xe0
 [<ffffffff810e4b50>] vfs_read+0xa0/0x160
 [<ffffffff810e4c5d>] sys_read+0x4d/0x90
 [<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  86
CPU    1: hi:  186, btch:  31 usd: 167
active_anon:2376 inactive_anon:2431 isolated_anon:0
 active_file:3712 inactive_file:103686 isolated_file:17
 unevictable:0 dirty:646 writeback:0 unstable:0
 free:807 slab_reclaimable:1485 slab_unreclaimable:5873
 mapped:2343 shmem:66 pagetables:791 bounce:0
 free_cma:0
DMA free:2032kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:0kB active_file:24kB inactive_file:12916kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15644kB managed:15900kB mlocked:0kB dirty:0kB writeback:0kB mapped:12kB shmem:0kB slab_reclaimable:16kB slab_unreclaimable:124kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 488 488 488
DMA32 free:1196kB min:2784kB low:3480kB high:4176kB active_anon:9504kB inactive_anon:9724kB active_file:14824kB inactive_file:401828kB unevictable:0kB isolated(anon):0kB isolated(file):68kB present:499960kB managed:491256kB mlocked:0kB dirty:2588kB writeback:0kB mapped:9360kB shmem:264kB slab_reclaimable:5924kB slab_unreclaimable:23368kB kernel_stack:1336kB pagetables:3164kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 11*4kB (MR) 1*8kB (R) 0*16kB 2*32kB (R) 4*64kB (R) 3*128kB (R) 1*256kB (R) 0*512kB 1*1024kB (R) 0*2048kB 0*4096kB = 2036kB
DMA32: 40*4kB (UEM) 28*8kB (UM) 22*16kB (UEM) 5*32kB (UM) 5*64kB (UM) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1216kB
108199 total pagecache pages
714 pages in swap cache
Swap cache stats: add 4829, delete 4115, find 583/626
Free swap  = 376280kB
Total swap = 392188kB
131054 pages RAM
3820 pages reserved
542491 pages shared
117305 pages non-shared
cat: page allocation failure: order:0, mode:0x20
Pid: 24171, comm: cat Not tainted 3.8.0-rc2w5+ #76
Call Trace:
 <IRQ>  [<ffffffff810a2411>] warn_alloc_failed+0xe1/0x130
 [<ffffffff8137871e>] ? fib_table_lookup+0x26e/0x2d0
 [<ffffffff810a5779>] __alloc_pages_nodemask+0x5e9/0x840
 [<ffffffff8130a6c0>] __netdev_alloc_frag+0xa0/0x150
 [<ffffffff8130d9d2>] __netdev_alloc_skb+0x82/0xe0
 [<ffffffffa00225b7>] virtnet_poll+0x1d7/0x610 [virtio_net]
 [<ffffffff8131767d>] net_rx_action+0x9d/0x1a0
 [<ffffffff8104284a>] __do_softirq+0xba/0x170
 [<ffffffff813b199c>] call_softirq+0x1c/0x30
 [<ffffffff8100c61d>] do_softirq+0x6d/0xa0
 [<ffffffff81042a75>] irq_exit+0xa5/0xb0
 [<ffffffff8100c25e>] do_IRQ+0x5e/0xd0
 [<ffffffff813afe2d>] common_interrupt+0x6d/0x6d
 <EOI>  [<ffffffff813aeeb5>] ? io_schedule+0xa5/0xd0
 [<ffffffff811ef000>] ? copy_user_generic_string+0x30/0x40
 [<ffffffff8109db32>] ? __lock_page_killable+0x62/0x70
 [<ffffffff8109da85>] ? file_read_actor+0x135/0x180
 [<ffffffff8109f950>] generic_file_aio_read+0x2e0/0x6c0
 [<ffffffffa00a4231>] nfs_file_read+0x91/0xb0 [nfs]
 [<ffffffff810e4477>] do_sync_read+0xa7/0xe0
 [<ffffffff810e4b50>] vfs_read+0xa0/0x160
 [<ffffffff810e4c5d>] sys_read+0x4d/0x90
 [<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  50
CPU    1: hi:  186, btch:  31 usd:  30
active_anon:2367 inactive_anon:2431 isolated_anon:0
 active_file:3719 inactive_file:103732 isolated_file:0
 unevictable:0 dirty:1302 writeback:0 unstable:0
 free:754 slab_reclaimable:1589 slab_unreclaimable:5896
 mapped:2343 shmem:66 pagetables:781 bounce:0
 free_cma:0
DMA free:1980kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:0kB active_file:24kB inactive_file:9704kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15644kB managed:15900kB mlocked:0kB dirty:84kB writeback:0kB mapped:12kB shmem:0kB slab_reclaimable:16kB slab_unreclaimable:180kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 488 488 488
DMA32 free:1036kB min:2784kB low:3480kB high:4176kB active_anon:9468kB inactive_anon:9724kB active_file:14852kB inactive_file:405224kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:499960kB managed:491256kB mlocked:0kB dirty:5124kB writeback:0kB mapped:9360kB shmem:264kB slab_reclaimable:6340kB slab_unreclaimable:23404kB kernel_stack:1368kB pagetables:3124kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 34*4kB (UR) 23*8kB (UMR) 40*16kB (UMR) 2*32kB (UM) 1*64kB (R) 1*128kB (R) 1*256kB (R) 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 1984kB
DMA32: 1*4kB (U) 33*8kB (UEM) 8*16kB (UM) 4*32kB (UM) 8*64kB (UM) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1036kB
108257 total pagecache pages
712 pages in swap cache
Swap cache stats: add 4829, delete 4117, find 585/628
Free swap  = 376288kB
Total swap = 392188kB
131054 pages RAM
3820 pages reserved
280574 pages shared
116800 pages non-shared
Eric Wong - Jan. 9, 2013, 8:42 a.m.
Eric Wong <normalperson@yhbt.net> wrote:
> Eric Dumazet <erdnetdev@gmail.com> wrote:
> > On Tue, 2013-01-08 at 18:32 -0800, Eric Dumazet wrote:
> > > Hmm, it seems sk_filter() can return -ENOMEM because skb has the
> > > pfmemalloc() set.
> > 
> > > 
> > > One TCP socket keeps retransmitting an SKB via loopback, and TCP stack 
> > > drops the packet again and again.
> > 
> > sock_init_data() sets sk->sk_allocation to GFP_KERNEL
> > 
> > Shouldnt it use (GFP_KERNEL | __GFP_NOMEMALLOC) instead ?
> 
> Thanks, things are running good after ~35 minutes so far.
> Will report back if things break (hopefully I don't run out
> of laptop battery power :x).

Oops, I had to restart my test :x.  However, I was able to reproduce the
issue very quickly again with your patch.  I've double-checked I'm
booting into the correct kernel, but I do have more load on this
laptop host now, so maybe that made it happen more quickly...
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Eric Wong - Jan. 9, 2013, 8:51 a.m.
Eric Wong <normalperson@yhbt.net> wrote:
> Oops, I had to restart my test :x.  However, I was able to reproduce the
> issue very quickly again with your patch.  I've double-checked I'm
> booting into the correct kernel, but I do have more load on this
> laptop host now, so maybe that made it happen more quickly...

Oops, I forgot to include the debugging output.
(Is this information still useful to you guys?)

2724 process stuck!
===> /proc/vmstat <===
nr_free_pages 2401
nr_inactive_anon 3242
nr_active_anon 3044
nr_inactive_file 103091
nr_active_file 4305
nr_unevictable 0
nr_mlock 0
nr_anon_pages 6204
nr_mapped 2332
nr_file_pages 107533
nr_dirty 144
nr_writeback 0
nr_slab_reclaimable 1440
nr_slab_unreclaimable 5202
nr_page_table_pages 773
nr_kernel_stack 167
nr_unstable 0
nr_bounce 0
nr_vmscan_write 0
nr_vmscan_immediate_reclaim 0
nr_writeback_temp 0
nr_isolated_anon 0
nr_isolated_file 0
nr_shmem 115
nr_dirtied 340718
nr_written 4979
nr_anon_transparent_hugepages 0
nr_free_cma 0
nr_dirty_threshold 22904
nr_dirty_background_threshold 11452
pgpgin 43068
pgpgout 20484
pswpin 0
pswpout 0
pgalloc_dma 57018
pgalloc_dma32 9428296
pgalloc_normal 0
pgalloc_movable 0
pgfree 9488417
pgactivate 5151
pgdeactivate 3251
pgfault 751069
pgmajfault 272
pgrefill_dma 115
pgrefill_dma32 3136
pgrefill_normal 0
pgrefill_movable 0
pgsteal_kswapd_dma 2865
pgsteal_kswapd_dma32 209744
pgsteal_kswapd_normal 0
pgsteal_kswapd_movable 0
pgsteal_direct_dma 568
pgsteal_direct_dma32 31692
pgsteal_direct_normal 0
pgsteal_direct_movable 0
pgscan_kswapd_dma 2865
pgscan_kswapd_dma32 210678
pgscan_kswapd_normal 0
pgscan_kswapd_movable 0
pgscan_direct_dma 568
pgscan_direct_dma32 31760
pgscan_direct_normal 0
pgscan_direct_movable 0
pgscan_direct_throttle 0
pginodesteal 0
slabs_scanned 0
kswapd_inodesteal 0
kswapd_low_wmark_hit_quickly 666
kswapd_high_wmark_hit_quickly 2
kswapd_skip_congestion_wait 0
pageoutrun 3135
allocstall 566
pgrotated 2
pgmigrate_success 348
pgmigrate_fail 0
compact_migrate_scanned 335538
compact_free_scanned 144705
compact_isolated 11328
compact_stall 451
compact_fail 279
compact_success 172
unevictable_pgs_culled 1064
unevictable_pgs_scanned 0
unevictable_pgs_rescued 1632
unevictable_pgs_mlocked 1632
unevictable_pgs_munlocked 1632
unevictable_pgs_cleared 0
unevictable_pgs_stranded 0
thp_fault_alloc 0
thp_fault_fallback 0
thp_collapse_alloc 0
thp_collapse_alloc_failed 0
thp_split 0
thp_zero_page_alloc 0
thp_zero_page_alloc_failed 0
===> 2724[2724]/stack <===
[<ffffffff81077300>] futex_wait_queue_me+0xc0/0xf0
[<ffffffff81077a9d>] futex_wait+0x17d/0x280
[<ffffffff8107988c>] do_futex+0x11c/0xae0
[<ffffffff8107a2d8>] sys_futex+0x88/0x180
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2724[2725]/stack <===
[<ffffffff810f5904>] poll_schedule_timeout+0x44/0x60
[<ffffffff810f6d54>] do_sys_poll+0x374/0x4b0
[<ffffffff810f718e>] sys_ppoll+0x19e/0x1b0
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2724[2726]/stack <===
[<ffffffff810f5904>] poll_schedule_timeout+0x44/0x60
[<ffffffff810f6d54>] do_sys_poll+0x374/0x4b0
[<ffffffff810f718e>] sys_ppoll+0x19e/0x1b0
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2724[2727]/stack <===
[<ffffffff81310078>] sk_stream_wait_memory+0x1b8/0x250
[<ffffffff8134be87>] tcp_sendmsg+0x697/0xd80
[<ffffffff81370cee>] inet_sendmsg+0x5e/0xa0
[<ffffffff81300a77>] sock_sendmsg+0x87/0xa0
[<ffffffff81303a59>] sys_sendto+0x119/0x160
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2724[2728]/stack <===
[<ffffffff810400b8>] do_wait+0x1f8/0x220
[<ffffffff81040ea0>] sys_wait4+0x70/0xf0
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2773[2773]/stack <===
[<ffffffff81077300>] futex_wait_queue_me+0xc0/0xf0
[<ffffffff81077a9d>] futex_wait+0x17d/0x280
[<ffffffff8107988c>] do_futex+0x11c/0xae0
[<ffffffff8107a2d8>] sys_futex+0x88/0x180
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2773[2774]/stack <===
[<ffffffff810f5904>] poll_schedule_timeout+0x44/0x60
[<ffffffff810f6d54>] do_sys_poll+0x374/0x4b0
[<ffffffff810f718e>] sys_ppoll+0x19e/0x1b0
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2773[2775]/stack <===
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2773[2776]/stack <===
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2773[2777]/stack <===
[<ffffffff8105d02c>] hrtimer_nanosleep+0x9c/0x150
[<ffffffff8105d13e>] sys_nanosleep+0x5e/0x80
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
SysRq : Show Memory
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:  72
CPU    1: hi:  186, btch:  31 usd: 100
active_anon:3130 inactive_anon:3283 isolated_anon:0
 active_file:4305 inactive_file:101390 isolated_file:0
 unevictable:0 dirty:103 writeback:0 unstable:0
 free:3675 slab_reclaimable:1453 slab_unreclaimable:5186
 mapped:2332 shmem:115 pagetables:754 bounce:0
 free_cma:0
DMA free:2116kB min:84kB low:104kB high:124kB active_anon:0kB inactive_anon:16kB active_file:0kB inactive_file:13692kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15644kB managed:15900kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:4kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 488 488 488
DMA32 free:12584kB min:2784kB low:3480kB high:4176kB active_anon:12520kB inactive_anon:13116kB active_file:17220kB inactive_file:391868kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:499960kB managed:491256kB mlocked:0kB dirty:420kB writeback:0kB mapped:9328kB shmem:460kB slab_reclaimable:5812kB slab_unreclaimable:20740kB kernel_stack:1336kB pagetables:3016kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 4*4kB (UMR) 1*8kB (M) 1*16kB (M) 3*32kB (MR) 1*64kB (R) 1*128kB (R) 1*256kB (R) 1*512kB (R) 1*1024kB (R) 0*2048kB 0*4096kB = 2120kB
DMA32: 411*4kB (UEM) 359*8kB (UEM) 207*16kB (UM) 84*32kB (UM) 25*64kB (UM) 4*128kB (M) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 12628kB
105835 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 392188kB
Total swap = 392188kB
131054 pages RAM
3820 pages reserved
276721 pages shared
117464 pages non-shared

2773 process stuck!
===> /proc/vmstat <===
nr_free_pages 1579
nr_inactive_anon 3302
nr_active_anon 3078
nr_inactive_file 103991
nr_active_file 4357
nr_unevictable 0
nr_mlock 0
nr_anon_pages 6260
nr_mapped 2319
nr_file_pages 108478
nr_dirty 648
nr_writeback 0
nr_slab_reclaimable 1603
nr_slab_unreclaimable 5380
nr_page_table_pages 748
nr_kernel_stack 171
nr_unstable 0
nr_bounce 0
nr_vmscan_write 0
nr_vmscan_immediate_reclaim 0
nr_writeback_temp 0
nr_isolated_anon 0
nr_isolated_file 0
nr_shmem 115
nr_dirtied 841467
nr_written 15931
nr_anon_transparent_hugepages 0
nr_free_cma 0
nr_dirty_threshold 22949
nr_dirty_background_threshold 11474
pgpgin 43832
pgpgout 64464
pswpin 0
pswpout 0
pgalloc_dma 105241
pgalloc_dma32 12655633
pgalloc_normal 0
pgalloc_movable 0
pgfree 12763390
pgactivate 5358
pgdeactivate 3607
pgfault 1011343
pgmajfault 302
pgrefill_dma 407
pgrefill_dma32 3200
pgrefill_normal 0
pgrefill_movable 0
pgsteal_kswapd_dma 10785
pgsteal_kswapd_dma32 612431
pgsteal_kswapd_normal 0
pgsteal_kswapd_movable 0
pgsteal_direct_dma 2159
pgsteal_direct_dma32 120797
pgsteal_direct_normal 0
pgsteal_direct_movable 0
pgscan_kswapd_dma 10785
pgscan_kswapd_dma32 613376
pgscan_kswapd_normal 0
pgscan_kswapd_movable 0
pgscan_direct_dma 2159
pgscan_direct_dma32 120866
pgscan_direct_normal 0
pgscan_direct_movable 0
pgscan_direct_throttle 0
pginodesteal 0
slabs_scanned 3072
kswapd_inodesteal 0
kswapd_low_wmark_hit_quickly 1810
kswapd_high_wmark_hit_quickly 13
kswapd_skip_congestion_wait 0
pageoutrun 9178
allocstall 2157
pgrotated 2
pgmigrate_success 509
pgmigrate_fail 0
compact_migrate_scanned 818935
compact_free_scanned 214217
compact_isolated 27006
compact_stall 1014
compact_fail 674
compact_success 340
unevictable_pgs_culled 1064
unevictable_pgs_scanned 0
unevictable_pgs_rescued 1632
unevictable_pgs_mlocked 1632
unevictable_pgs_munlocked 1632
unevictable_pgs_cleared 0
unevictable_pgs_stranded 0
thp_fault_alloc 0
thp_fault_fallback 0
thp_collapse_alloc 0
thp_collapse_alloc_failed 0
thp_split 0
thp_zero_page_alloc 0
thp_zero_page_alloc_failed 0
===> 2724[2724]/stack <===
[<ffffffff81077300>] futex_wait_queue_me+0xc0/0xf0
[<ffffffff81077a9d>] futex_wait+0x17d/0x280
[<ffffffff8107988c>] do_futex+0x11c/0xae0
[<ffffffff8107a2d8>] sys_futex+0x88/0x180
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2724[2725]/stack <===
[<ffffffff810f5904>] poll_schedule_timeout+0x44/0x60
[<ffffffff810f6d54>] do_sys_poll+0x374/0x4b0
[<ffffffff810f718e>] sys_ppoll+0x19e/0x1b0
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2724[2726]/stack <===
[<ffffffff810f5904>] poll_schedule_timeout+0x44/0x60
[<ffffffff810f6d54>] do_sys_poll+0x374/0x4b0
[<ffffffff810f718e>] sys_ppoll+0x19e/0x1b0
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2724[2727]/stack <===
[<ffffffff81310078>] sk_stream_wait_memory+0x1b8/0x250
[<ffffffff8134be87>] tcp_sendmsg+0x697/0xd80
[<ffffffff81370cee>] inet_sendmsg+0x5e/0xa0
[<ffffffff81300a77>] sock_sendmsg+0x87/0xa0
[<ffffffff81303a59>] sys_sendto+0x119/0x160
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2724[2728]/stack <===
[<ffffffff8105d02c>] hrtimer_nanosleep+0x9c/0x150
[<ffffffff8105d13e>] sys_nanosleep+0x5e/0x80
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2773[2773]/stack <===
[<ffffffff81077300>] futex_wait_queue_me+0xc0/0xf0
[<ffffffff81077a9d>] futex_wait+0x17d/0x280
[<ffffffff8107988c>] do_futex+0x11c/0xae0
[<ffffffff8107a2d8>] sys_futex+0x88/0x180
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2773[2774]/stack <===
[<ffffffff810f5904>] poll_schedule_timeout+0x44/0x60
[<ffffffff810f6d54>] do_sys_poll+0x374/0x4b0
[<ffffffff810f718e>] sys_ppoll+0x19e/0x1b0
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2773[2775]/stack <===
[<ffffffff810f5904>] poll_schedule_timeout+0x44/0x60
[<ffffffff810f6d54>] do_sys_poll+0x374/0x4b0
[<ffffffff810f718e>] sys_ppoll+0x19e/0x1b0
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2773[2776]/stack <===
[<ffffffff81310078>] sk_stream_wait_memory+0x1b8/0x250
[<ffffffff8134be87>] tcp_sendmsg+0x697/0xd80
[<ffffffff81370cee>] inet_sendmsg+0x5e/0xa0
[<ffffffff81300a77>] sock_sendmsg+0x87/0xa0
[<ffffffff81303a59>] sys_sendto+0x119/0x160
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
===> 2773[2777]/stack <===
[<ffffffff810400b8>] do_wait+0x1f8/0x220
[<ffffffff81040ea0>] sys_wait4+0x70/0xf0
[<ffffffff813b06e9>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
<redundant "SysRq : Show Memory" from previous process omitted>
SysRq : Show Memory
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd: 164
CPU    1: hi:  186, btch:  31 usd: 117
active_anon:3016 inactive_anon:3281 isolated_anon:0
 active_file:4357 inactive_file:104163 isolated_file:0
 unevictable:0 dirty:142 writeback:0 unstable:0
 free:1582 slab_reclaimable:1598 slab_unreclaimable:5380
 mapped:2316 shmem:115 pagetables:773 bounce:0
 free_cma:0
DMA free:2332kB min:84kB low:104kB high:124kB active_anon:8kB inactive_anon:8kB active_file:0kB inactive_file:13476kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15644kB managed:15900kB mlocked:0kB dirty:12kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:12kB kernel_stack:0kB pagetables:8kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 488 488 488
DMA32 free:3996kB min:2784kB low:3480kB high:4176kB active_anon:12056kB inactive_anon:13116kB active_file:17428kB inactive_file:403176kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:499960kB managed:491256kB mlocked:0kB dirty:556kB writeback:0kB mapped:9264kB shmem:460kB slab_reclaimable:6392kB slab_unreclaimable:21508kB kernel_stack:1360kB pagetables:3084kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 19*4kB (UER) 20*8kB (U) 7*16kB (U) 2*32kB (R) 0*64kB 3*128kB (R) 0*256kB 1*512kB (R) 1*1024kB (R) 0*2048kB 0*4096kB = 2332kB
DMA32: 151*4kB (UEM) 210*8kB (UE) 108*16kB (U) 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4012kB
108629 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 392188kB
Total swap = 392188kB
131054 pages RAM
3820 pages reserved
275952 pages shared
119896 pages non-shared
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/net/core/sock.c b/net/core/sock.c
index bc131d4..76c4b39 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -286,6 +286,7 @@  void sk_set_memalloc(struct sock *sk)
 {
 	sock_set_flag(sk, SOCK_MEMALLOC);
 	sk->sk_allocation |= __GFP_MEMALLOC;
+	sk->sk_allocation &= ~__GFP_NOMEMALLOC;
 	static_key_slow_inc(&memalloc_socks);
 }
 EXPORT_SYMBOL_GPL(sk_set_memalloc);
@@ -294,6 +295,7 @@  void sk_clear_memalloc(struct sock *sk)
 {
 	sock_reset_flag(sk, SOCK_MEMALLOC);
 	sk->sk_allocation &= ~__GFP_MEMALLOC;
+	sk->sk_allocation |= __GFP_NOMEMALLOC;
 	static_key_slow_dec(&memalloc_socks);
 
 	/*
@@ -2230,7 +2232,7 @@  void sock_init_data(struct socket *sock, struct sock *sk)
 
 	init_timer(&sk->sk_timer);
 
-	sk->sk_allocation	=	GFP_KERNEL;
+	sk->sk_allocation	=	GFP_KERNEL | __GFP_NOMEMALLOC;
 	sk->sk_rcvbuf		=	sysctl_rmem_default;
 	sk->sk_sndbuf		=	sysctl_wmem_default;
 	sk->sk_state		=	TCP_CLOSE;