From patchwork Fri Jun 26 00:03:55 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ramu Ramamurthy X-Patchwork-Id: 488586 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 107A4140273 for ; Fri, 26 Jun 2015 10:04:19 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751641AbbFZAED (ORCPT ); Thu, 25 Jun 2015 20:04:03 -0400 Received: from e37.co.us.ibm.com ([32.97.110.158]:36132 "EHLO e37.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751426AbbFZAD7 (ORCPT ); Thu, 25 Jun 2015 20:03:59 -0400 Received: from /spool/local by e37.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 25 Jun 2015 18:03:59 -0600 Received: from d03dlp02.boulder.ibm.com (9.17.202.178) by e37.co.us.ibm.com (192.168.1.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 25 Jun 2015 18:03:57 -0600 X-Helo: d03dlp02.boulder.ibm.com X-MailFrom: sramamur@linux.vnet.ibm.com X-RcptTo: netdev@vger.kernel.org Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by d03dlp02.boulder.ibm.com (Postfix) with ESMTP id D9E5C3E4003B for ; Thu, 25 Jun 2015 18:03:56 -0600 (MDT) Received: from d03av03.boulder.ibm.com (d03av03.boulder.ibm.com [9.17.195.169]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t5Q03Hkd42926304 for ; Thu, 25 Jun 2015 17:03:17 -0700 Received: from d03av03.boulder.ibm.com (localhost [127.0.0.1]) by d03av03.boulder.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t5Q03tSW014032 for ; Thu, 25 Jun 2015 18:03:56 -0600 Received: from ltcweb.rtp.raleigh.ibm.com (ltcweb.rtp.raleigh.ibm.com [9.37.210.204]) by d03av03.boulder.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id t5Q03tbR014010; Thu, 25 Jun 2015 18:03:55 -0600 Received: from ltc.linux.ibm.com (localhost.localdomain [127.0.0.1]) by ltcweb.rtp.raleigh.ibm.com (Postfix) with ESMTP id 1C98DC0034; Thu, 25 Jun 2015 20:03:55 -0400 (EDT) MIME-Version: 1.0 Date: Thu, 25 Jun 2015 17:03:55 -0700 From: Ramu Ramamurthy To: "David S. Miller" , Tom Herbert , Jiri Benc , James Morris Cc: netdev@vger.kernel.org, pradeeps@linux.vnet.ibm.com, jkidambi@us.ibm.com Subject: [PATCH] - vxlan: gro not effective for intel 82599 Message-ID: <5981772fe36e64f8fec5997a4c7aa08f@imap.linux.ibm.com> X-Sender: sramamur@linux.vnet.ibm.com User-Agent: Roundcube Webmail/1.0.1 X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15062600-0025-0000-0000-00000FA0788D Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Problem: ------- GRO is enabled on the interfaces in the following test, but GRO does not take effect for vxlan-encapsulated tcp streams. The root cause of why GRO does not take effect is described below. VM nic (mtu 1450)---bridge---vxlan----10Gb nic (intel 82599ES)-----| VM nic (mtu 1450)---bridge---vxlan----10Gb nic (intel 82599ES)-----| Because gro is not effective, the throughput for vxlan-encapsulated tcp-stream is around 3 Gbps. With the proposed patch, gro takes effect for vxlan-encapsulated tcp streams, and performance in the same test is around 8.6 Gbps. Root Cause: ---------- At entry to udp4_gro_receive(), the gro parameters are set as follows: skb->ip_summed == 0 (CHECKSUM_NONE) NAPI_GRO_CB(skb)->csum_cnt == 0 NAPI_GRO_CB(skb)->csum_valid == 0 UDH header checksum is 0. static struct sk_buff **udp4_gro_receive(struct sk_buff **head, struct sk_buff *skb) { if (skb_gro_checksum_validate_zero_check(skb, IPPROTO_UDP, uh->check, inet_gro_compute_pseudo)) >>> This calls __skb_incr_checksum_unnecessary which sets >>> skb->ip_summed to CHECKSUM_UNNECESSARY >>> goto flush; else if (uh->check) skb_gro_checksum_try_convert(skb, IPPROTO_UDP, uh->check, inet_gro_compute_pseudo); skip: NAPI_GRO_CB(skb)->is_ipv6 = 0; return udp_gro_receive(head, skb, uh); } struct sk_buff **udp_gro_receive(struct sk_buff **head, struct sk_buff *skb, struct udphdr *uh) { struct udp_offload_priv *uo_priv; struct sk_buff *p, **pp = NULL; struct udphdr *uh2; unsigned int off = skb_gro_offset(skb); int flush = 1; if (NAPI_GRO_CB(skb)->udp_mark || (skb->ip_summed != CHECKSUM_PARTIAL && NAPI_GRO_CB(skb)->csum_cnt == 0 && !NAPI_GRO_CB(skb)->csum_valid)) goto out; >>> >>> vxlan GRO gets skipped due to the above condition because here,: >>> skb->ip_summed == CHECKSUM_UNNECESSARY >>> NAPI_GRO_CB(skb)->csum_cnt == 0 >>> NAPI_GRO_CB(skb)->csum_valid == 0 There is no reason for skipping vxlan gro in the above combination of conditions, because, tcp4_gro_receive() validates the inner tcp checksum anyway ! Patch: ------ Signed-off-by: Ramu Ramamurthy --- net/ipv4/udp_offload.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) goto out; diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index f938616..17fc12b 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -301,6 +301,7 @@ struct sk_buff **udp_gro_receive(struct sk_buff **head, struct sk_buff *skb, if (NAPI_GRO_CB(skb)->udp_mark || (skb->ip_summed != CHECKSUM_PARTIAL && + skb->ip_summed != CHECKSUM_UNNECESSARY && NAPI_GRO_CB(skb)->csum_cnt == 0 && !NAPI_GRO_CB(skb)->csum_valid))