| Message ID | CAATkVExrpHZHDFFdfR2ab7SeFSBLyAm0BLoVqLDGHF1ZPwEBZA@mail.gmail.com |
|---|---|
| State | RFC, archived |
| Delegated to: | David Miller |
| Headers | show |
On Fri, 2014-01-03 at 17:47 -0500, Debabrata Banerjee wrote: > >> On Thu, 2014-01-02 at 16:56 -0800, Eric Dumazet wrote: > >> > >> Hmm... it looks like I missed __GFP_NORETRY > >> > >> > >> > >> diff --git a/net/core/sock.c b/net/core/sock.c > >> index 5393b4b719d7..5f42a4d70cb2 100644 > >> --- a/net/core/sock.c > >> +++ b/net/core/sock.c > >> @@ -1872,7 +1872,7 @@ bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t prio) > >> gfp_t gfp = prio; > >> > >> if (order) > >> - gfp |= __GFP_COMP | __GFP_NOWARN; > >> + gfp |= __GFP_COMP | __GFP_NOWARN | __GFP_NORETRY; > >> pfrag->page = alloc_pages(gfp, order); > >> if (likely(pfrag->page)) { > >> pfrag->offset = 0; > >> > >> > >> > > There is another patch needed (looks like good stable fixes): > diff --git a/net/core/skbuff.c b/net/core/skbuff.c > index 06e72d3..d42d48c 100644 > --- a/net/core/skbuff.c > +++ b/net/core/skbuff.c > @@ -378,7 +378,7 @@ refill: > gfp_t gfp = gfp_mask; > > if (order) > - gfp |= __GFP_COMP | __GFP_NOWARN; > + gfp |= __GFP_COMP | __GFP_NOWARN | > __GFP_NORETRY; > nc->frag.page = alloc_pages(gfp, order); > if (likely(nc->frag.page)) > break; > This is in GFP_ATOMIC cases, I dont think it can ever start compaction. > This reduces the really pathological compact/reclaim behavior but > doesn't fix it. Actually it still really quite bad because the whole > thing loops until it gets to order-0 so it's effectively trying the > allocation 4 times anyway. I typically see non-zero order allocations > very rarely without these two pieces of code. I hotpatched a running > system to get results from this quickly. Even setting the max order to > order-1 I still see bad behavior. If anything this behavior should be > conditional until this is ironed out. > > Performance data: http://pastebin.ubuntu.com/6687527/ It seems that you shoot the messenger : If memory is fragmented, then one order-1 allocation is going to start compaction. It can be a simple fork(). If your workload never fork(), then yes, you never needed compaction. It doesn't really matter to say that which memory allocation triggered compaction, which is a normal step in mm layer. If you believe its badly done, you should ask to mm guys to fix/improve it, not netdev... We are not trying to optimize the kernel behavior for hosts in deep memory pressure. Using order-3 pages in TCP stack improves performance for 99% of the hosts, there might be something wrong on your side ? -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Jan 3, 2014 at 5:54 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote: > > This is in GFP_ATOMIC cases, I dont think it can ever start compaction. I think that's right I probably finally got it back to normal behavior with order-0 allocations. > > It seems that you shoot the messenger : If memory is fragmented, then > one order-1 allocation is going to start compaction. > > It can be a simple fork(). > > If your workload never fork(), then yes, you never needed compaction. > Sure but the rate of network packets in and out and subsequent allocations would be more equivalent to a fork bomb than normal forking. I understand mm should work more sanely in this scenario but at the same time we see a bad regression with this code, I see we're not alone. > > We are not trying to optimize the kernel behavior for hosts in deep > memory pressure. We're leaving about half for the kernel so I wouldn't call it "deep". Any server application that is using page cache and mlocked memory will run into similar issues. > It doesn't really matter to say that which memory allocation triggered > compaction, which is a normal step in mm layer. > > If you believe its badly done, you should ask to mm guys to fix/improve > it, not netdev... > > Using order-3 pages in TCP stack improves performance for 99% of the > hosts, there might be something wrong on your side ? > Having lots of memory mlocked is bad right now yes, but not necessarily an uncommon scenario. We're handing mm an almost intractable problem. I see compaction of mlocked pages has been discussed a few times over there, but no patch has actually made it in. -Debabrata -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 06e72d3..d42d48c 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -378,7 +378,7 @@ refill: gfp_t gfp = gfp_mask; if (order) - gfp |= __GFP_COMP | __GFP_NOWARN; + gfp |= __GFP_COMP | __GFP_NOWARN | __GFP_NORETRY; nc->frag.page = alloc_pages(gfp, order); if (likely(nc->frag.page))