From patchwork Mon Mar 28 00:57:26 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Blanchard X-Patchwork-Id: 88527 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 92D1D1007D1 for ; Mon, 28 Mar 2011 11:57:36 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752660Ab1C1A5c (ORCPT ); Sun, 27 Mar 2011 20:57:32 -0400 Received: from ozlabs.org ([203.10.76.45]:54818 "EHLO ozlabs.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752543Ab1C1A5b (ORCPT ); Sun, 27 Mar 2011 20:57:31 -0400 Received: from kryten (ppp121-44-112-144.lns20.syd6.internode.on.net [121.44.112.144]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPSA id 498FF1007D1; Mon, 28 Mar 2011 11:57:30 +1100 (EST) Date: Mon, 28 Mar 2011 11:57:26 +1100 From: Anton Blanchard To: davem@davemloft.net, eric.dumazet@gmail.com, herbert@gondor.apana.org.au Cc: netdev@vger.kernel.org Subject: [PATCH] net: Always allocate at least 16 skb frags regardless of page size Message-ID: <20110328115726.4cca214d@kryten> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.22.0; i486-pc-linux-gnu) Mime-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org When analysing performance of the cxgb3 on a ppc64 box I noticed that we weren't doing much GRO merging. It turns out we are limited by the number of SKB frags: #define MAX_SKB_FRAGS (65536/PAGE_SIZE + 2) With a 4kB page size we have 18 frags, but with a 64kB page size we only have 3 frags. I ran a single stream TCP bandwidth test to compare the performance of --- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html different values of MAX_SKB_FRAGS on the receiver: MAX_SKB_FRAGS Mbps 3 7080 8 7931 (+12%) 16 8335 (+17%) 32 8349 (+17%) Performance continues to increase up to 16 frags then levels off so the patch below puts a lower bound of 16 on MAX_SKB_FRAGS. Signed-off-by: Anton Blanchard --- Index: powerpc.git/include/linux/skbuff.h =================================================================== --- powerpc.git.orig/include/linux/skbuff.h 2011-03-28 09:41:25.392124844 +1100 +++ powerpc.git/include/linux/skbuff.h 2011-03-28 10:18:58.253050000 +1100 @@ -122,8 +122,14 @@ struct sk_buff_head { struct sk_buff; -/* To allow 64K frame to be packed as single skb without frag_list */ +/* To allow 64K frame to be packed as single skb without frag_list. Since + * GRO uses frags we allocate at least 16 regardless of page size. + */ +#if (65536/PAGE_SIZE + 2) < 16 +#define MAX_SKB_FRAGS 16 +#else #define MAX_SKB_FRAGS (65536/PAGE_SIZE + 2) +#endif typedef struct skb_frag_struct skb_frag_t;