From patchwork Thu Apr 4 12:57:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?UmFmYcWCIE1pxYJlY2tp?= X-Patchwork-Id: 1077267 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="cp7Eiiaz"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 44Zjf0084Cz9sPc for ; Thu, 4 Apr 2019 23:57:28 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729037AbfDDM5Z (ORCPT ); Thu, 4 Apr 2019 08:57:25 -0400 Received: from mail-ed1-f66.google.com ([209.85.208.66]:35254 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727108AbfDDM5Z (ORCPT ); Thu, 4 Apr 2019 08:57:25 -0400 Received: by mail-ed1-f66.google.com with SMTP id s39so2128832edb.2 for ; Thu, 04 Apr 2019 05:57:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:cc:from:subject:message-id:date:user-agent:mime-version :content-language; bh=MkMDAmE2Q+cFj1jwRKbeOY9BxWRQ1yXCGX0eRensI0Q=; b=cp7EiiazQcPA3LQRYuB03r3GTA1IUNqNQ/L2tCk3TUdcEfQU7rzM8ggoxlSBvXDidH tYCaK2Cc1M/caRqgbmScZ1xzUkRhJlOnDPGQaRM1FNbwbuoY/qa9G8bvYkJjpTezqYG3 l9/8nYXZXTAAWr+DQD0CGPRbzamSJagDI6f1zWsmt8L2kIYiACx8C9aFxIE7jlqcE2kU +KWNhPORtghwFOwYZ/wcRsO9FpXzcnDvFB4ej/obQD95/fQmMLHzJcIt6Tp2+s4CwIae a9wpYFOJYvLflvJyMggfqfdtRsnPeCzvs5UGYs6Kxp4mwFiDScIyBRjpmAXFqmdlonss 1UtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:cc:from:subject:message-id:date:user-agent :mime-version:content-language; bh=MkMDAmE2Q+cFj1jwRKbeOY9BxWRQ1yXCGX0eRensI0Q=; b=V2TFhRzk70FUysXq3kvY7o5j/F2hOsqSe+vPwWTY+Qs1/Zey595ByS2B25jblAprbu ZKIEZES5s1q08SnKNBSzffl6zchZGQYtghQAAX+4/AmRxQYl69QR4ptOLQ2C5G3zj+3M xR104vhrDh0ICIA5BW7DolsOlBXZlbrmmSftIVi29Ckezdtug3FnUyt2NDx7ZNqLjmZG osv6FYS0nfu7AiOcZwXdvC7sWVVwgEgSO0jeWLf97+SJjJ9CNfia79WRIkQ9SaKNh8Dt vQzHhwAUxBKVf6jbGhCUTLtgQO/PoEz50T9XJu4orvh85Yk0capxlHKfFwTsn36gI+cV HGJA== X-Gm-Message-State: APjAAAXPYAS7SwilT9sJxFaxam0aUaOUx/r98lii/bkVQ9Yzm8y1HqS6 SMfWC8Qoj0KJM7i5OwhVLcw= X-Google-Smtp-Source: APXvYqwPjCcQ1+m12eKdn0kVY1CEJ9yIDSCP18qKcCW1bKXZGQlX02oGMp5hyyi0y1JCcIehRDSdAg== X-Received: by 2002:a17:906:c2d0:: with SMTP id ch16mr3538759ejb.197.1554382638314; Thu, 04 Apr 2019 05:57:18 -0700 (PDT) Received: from elitebook.lan (ip-194-187-74-233.konfederacka.maverick.com.pl. [194.187.74.233]) by smtp.googlemail.com with ESMTPSA id z51sm5629359edb.9.2019.04.04.05.57.16 (version=TLS1_3 cipher=AEAD-AES128-GCM-SHA256 bits=128/128); Thu, 04 Apr 2019 05:57:17 -0700 (PDT) To: netdev@vger.kernel.org, "David S. Miller" , Toshiaki Makita Cc: Stefano Brivio , Sabrina Dubroca , David Ahern , Felix Fietkau , Jo-Philipp Wich , Koen Vandeputte From: =?utf-8?b?UmFmYcWCIE1pxYJlY2tp?= Subject: NAT performance regression caused by vlan GRO support Message-ID: <73223229-6bc0-2647-6952-975961811866@gmail.com> Date: Thu, 4 Apr 2019 14:57:15 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.2 MIME-Version: 1.0 Content-Language: en-US Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Hello, I'd like to report a regression that goes back to the 2015. I know it's damn late, but the good thing is, the regression is still easy to reproduce, verify & revert. Long story short, starting with the commit 66e5133f19e9 ("vlan: Add GRO support for non hardware accelerated vlan") - which first hit kernel 4.2 - NAT performance of my router dropped by 30% - 40%. My hardware is BCM47094 SoC (dual core ARM) with integrated network controller and external BCM53012 switch. Relevant setup: * SoC network controller is wired to the hardware switch * Switch passes 802.1q frames with VID 1 to four LAN ports * Switch passes 802.1q frames with VID 2 to WAN port * Linux does NAT for LAN (eth0.1) to WAN (eth0.2) * Linux uses pfifo and "echo 2 > rps_cpus" * Ryzen 5 PRO 2500U (x86_64) laptop connected to a LAN port * Intel i7-2670QM laptop connected to a WAN port * Speed of LAN to WAN measured using iperf & TCP over 10 minutes 1) 5.1.0-rc3 [ 6] 0.0-600.0 sec 39.9 GBytes 572 Mbits/sec 2) 5.1.0-rc3 + rtcache patch [ 6] 0.0-600.0 sec 40.0 GBytes 572 Mbits/sec 3) 5.1.0-rc3 + disable GRO support [ 6] 0.0-300.4 sec 27.5 GBytes 786 Mbits/sec 4) 5.1.0-rc3 + rtcache patch + disable GRO support [ 6] 0.0-600.0 sec 65.6 GBytes 939 Mbits/sec 5) 4.1.15 + rtcache patch 934 Mb/s 6) 4.3.4 + rtcache patch 565 Mb/s As you can see I can achieve a big performance gain by disabling/reverting a GRO support. Getting up to 65% faster NAT makes a huge difference and ideally I'd like to get that with upstream Linux code. Could someone help me and check the reported commit/code, please? Is there any other info I can provide or anything I can test for you? --- a/net/8021q/vlan_core.c +++ b/net/8021q/vlan_core.c @@ -545,6 +545,8 @@ static int __init vlan_offload_init(void) { unsigned int i; + return -ENOTSUPP; + for (i = 0; i < ARRAY_SIZE(vlan_packet_offloads); i++) dev_add_offload(&vlan_packet_offloads[i]);