From patchwork Thu Jan 11 15:04:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koen Vandeputte X-Patchwork-Id: 859170 X-Patchwork-Delegate: blogic@openwrt.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.infradead.org (client-ip=65.50.211.133; helo=bombadil.infradead.org; envelope-from=lede-dev-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="YGNVyNmi"; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ncentric.com header.i=@ncentric.com header.b="ihYa5Q0I"; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zHThR5GdNz9s8J for ; Fri, 12 Jan 2018 02:05:27 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Subject:References: In-Reply-To:Message-Id:Date:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=TqL+WSfuNgydEtUoCg2no7uqkmt9qEJeGqLyBBGGX+o=; b=YGNVyNmi2tbXnn Mn38i5+UQjbaSwezBE0HfT7MUO21Yk5rlIUKPcPSC40mfuLXaZjZpwwZ6fT3IJR5fEuM/Q3I3Vof6 ulaXyRwyZ55iGjPf2wNhNr23geSMLsEoTOlYC6311mKSKVsabEyj/pPSIp0dWmzcv0IK//Raa3NTN NoygPdN/qCcJO6OhCc8NCIPQ9Kf/0L2MPzYlGD4oq7Uwn99oySZwMHJywJV6r9UI0UiU5XydXuunJ eRoEbNzvWCPIj6mID38ZfHhkFKdZe8fjWbX2d459MGA30pibU3h0TloCJ+zmFetxxOUo50b9GPswf Dq/OT/QsNp6HyDj9phuA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1eZeQ1-000390-Ux; Thu, 11 Jan 2018 15:05:25 +0000 Received: from mail-wm0-x244.google.com ([2a00:1450:400c:c09::244]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eZePl-0001so-E7 for lede-dev@lists.infradead.org; Thu, 11 Jan 2018 15:05:12 +0000 Received: by mail-wm0-x244.google.com with SMTP id 143so6091143wma.5 for ; Thu, 11 Jan 2018 07:05:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ncentric.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IlnoPgT3jrTlELr2JTN4zbPYlkDo9u5D9xFPAPpTgIA=; b=ihYa5Q0Ij1XjUYhYz+d/ZY42NyWMx/Da6TihSTkVu+Mmi4HUdNcZEWjgaLBQOQf99S uRSEfLifOfcviZ2bEpmqEntBE69bwpDRXpG41AXUI6iQCaYXgkqXbZtex+KI5aEaZGZU ami+CSbUdtsII3zv2eiU01e5txd6PEy0+QctE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IlnoPgT3jrTlELr2JTN4zbPYlkDo9u5D9xFPAPpTgIA=; b=Nz+lo0xXkaJLpbn0E/O8tqqM6AvYFowcTKK/C2mJ5LmpGPdCYdBhdgMorsUqLX4eM8 9tM1foJaBMek+ZKmpu10rUBIq7Qg3Nr7Pi8iewIPbMODNNzMlTamzC3HQsoyVCmiN1Vm MdzYDiQhFQ0/ouGO9Xo/9bpDE71lYME7ZH1pw6AijmTRi4+1phvXNjSGjIM2tnUpxABB 4UH7Trnu5jebERiP05/iBxvJukF7335fT2PBhZu1z92jZ42nY1QSA8dNG9f8inoCiGhF 0Bj7MNdHe7gSBLVA2NO2yy+TTm5zGfXADtCvERQj3se8cTzIYTHQzeun/Os2aoIOJ+UI 6LNw== X-Gm-Message-State: AKwxytcstrian3ys56xpjBx9cmc5TlD3+BUeYB7nsxrkqyRIPZoLCBPn 45fykI9IKmY2kxhrCWPPblLLbHoG23k= X-Google-Smtp-Source: ACJfBosiqKQuXRmhP9xqpYTAQrOICEW9HTSr9FrMztFISUAz7XfSJNvcwe+Ns9S/Z3HApi1MfdZp4A== X-Received: by 10.80.149.216 with SMTP id x24mr3770822eda.74.1515683101395; Thu, 11 Jan 2018 07:05:01 -0800 (PST) Received: from localhost.localdomain (d515300d8.static.telenet.be. [81.83.0.216]) by smtp.googlemail.com with ESMTPSA id x28sm13398188edd.0.2018.01.11.07.05.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 11 Jan 2018 07:05:00 -0800 (PST) From: Koen Vandeputte To: lede-dev@lists.infradead.org Date: Thu, 11 Jan 2018 16:04:37 +0100 Message-Id: <1515683081-31607-6-git-send-email-koen.vandeputte@ncentric.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1515683081-31607-1-git-send-email-koen.vandeputte@ncentric.com> References: <1515683081-31607-1-git-send-email-koen.vandeputte@ncentric.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180111_070509_639546_812EB81C X-CRM114-Status: GOOD ( 16.73 ) X-Spam-Score: -2.0 (--) X-Spam-Report: SpamAssassin version 3.4.1 on bombadil.infradead.org summary: Content analysis details: (-2.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [2a00:1450:400c:c09:0:0:0:244 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid Subject: [LEDE-DEV] [PATCH 5/9] cns3xxx: fix ethernet IRQ imbalance in 4.14 X-BeenThere: lede-dev@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: tharvey@gateworks.com, Koen Vandeputte , nbd@nbd.name MIME-Version: 1.0 Sender: "Lede-dev" Errors-To: lede-dev-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org Kernel 4.14 contains more warnings regarding NAPI usage, which showed following warning: [ 4016.420000] ------------[ cut here ]------------ [ 4016.430000] WARNING: CPU: 1 PID: 14 at kernel/irq/manage.c:525 __enable_irq+0x3c/0x78 [ 4016.440000] Unbalanced enable for IRQ 51 [ 4016.440000] Modules linked in: ath9k ath9k_common pppoe ppp_async ath9k_hw ath pppox ppp_generic nf_conntrack_ipv6 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG slhc nf_reject_ipv4 nf_nat_redirs [ 4016.500000] CPU: 1 PID: 14 Comm: ksoftirqd/1 Tainted: G W 4.14.11 #0 [ 4016.500000] Hardware name: Gateworks Corporation Laguna Platform [ 4016.510000] Backtrace: [ 4016.510000] [] (dump_backtrace) from [] (show_stack+0x18/0x1c) [ 4016.520000] r7:c059df4b r6:00000000 r5:60000093 r4:00000000 [ 4016.530000] [] (show_stack) from [] (dump_stack+0x94/0xb4) [ 4016.530000] [] (dump_stack) from [] (__warn+0xc8/0xf8) [ 4016.540000] r7:c059df4b r6:c015144c r5:00000000 r4:cf863e08 [ 4016.550000] [] (__warn) from [] (warn_slowpath_fmt+0x40/0x48) [ 4016.550000] r9:cf863eb0 r8:00000036 r7:00000000 r6:c0636360 r5:00000033 r4:cf80a500 [ 4016.560000] [] (warn_slowpath_fmt) from [] (__enable_irq+0x3c/0x78) [ 4016.570000] r3:00000033 r2:c059e0c5 [ 4016.570000] [] (__enable_irq) from [] (enable_irq+0x64/0x7c) [ 4016.580000] [] (enable_irq) from [] (eth_poll+0x28c/0x558) [ 4016.590000] r5:d08926c0 r4:cf894488 [ 4016.590000] [] (eth_poll) from [] (net_rx_action+0xfc/0x2e8) [ 4016.600000] r10:cf863eb8 r9:cf863eb0 r8:00000001 r7:0f7b5000 r6:c0636360 r5:cfdeb360 [ 4016.610000] r4:cf894488 [ 4016.610000] [] (net_rx_action) from [] (__do_softirq+0xe0/0x228) [ 4016.620000] r10:00000100 r9:c070204c r8:c0702040 r7:00000003 r6:00000008 r5:40000003 [ 4016.620000] r4:cf862000 [ 4016.630000] [] (__do_softirq) from [] (run_ksoftirqd+0x34/0x58) [ 4016.630000] r10:c0133964 r9:00000000 r8:00000001 r7:00000000 r6:c070b44c r5:cf862000 [ 4016.640000] r4:cf8037e0 [ 4016.640000] [] (run_ksoftirqd) from [] (smpboot_thread_fn+0x178/0x190) [ 4016.650000] [] (smpboot_thread_fn) from [] (kthread+0x11c/0x138) [ 4016.660000] r9:cf8037e0 r8:cf802cdc r7:cf839df8 r6:cf803800 r5:00000000 r4:cf802cc0 [ 4016.670000] [] (kthread) from [] (ret_from_fork+0x14/0x3c) [ 4016.680000] r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c0130880 [ 4016.680000] r4:cf803800 r3:cf862000 [ 4016.690000] ---[ end trace 51114df08f429115 ]--- This is triggered by calling eth_schedule_poll(sw) after the IRQ has been re-enabled. Rework the network code to only enable IRQ's again if NAPI agrees it's safe to do so. Also only re-enable IRQ *after* cleaning up the RX ring and re-enabling DMA, which otherwise resulted in ugly warnings regarding dirty page fragments. These popped up nearly immediately when building the kernel with O2 iso Os. --> Note that this change fixes stability issues, at the cost of ~8% throughput performance. While at it, also change the iface name being used in warning prints, making it more obvious: "switch%d" --> "cns3xxx_eth" The changes have been tested on 4 boards, each moving ~30TB of data Signed-off-by: Koen Vandeputte --- .../140-fix-ethernet-irq-imbalance.patch | 77 ++++++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 target/linux/cns3xxx/patches-4.14/140-fix-ethernet-irq-imbalance.patch diff --git a/target/linux/cns3xxx/patches-4.14/140-fix-ethernet-irq-imbalance.patch b/target/linux/cns3xxx/patches-4.14/140-fix-ethernet-irq-imbalance.patch new file mode 100644 index 000000000000..3538586f6ff8 --- /dev/null +++ b/target/linux/cns3xxx/patches-4.14/140-fix-ethernet-irq-imbalance.patch @@ -0,0 +1,77 @@ +--- a/drivers/net/ethernet/cavium/cns3xxx_eth.c ++++ b/drivers/net/ethernet/cavium/cns3xxx_eth.c +@@ -712,26 +712,20 @@ static int eth_poll(struct napi_struct * + } + + rx_ring->cur_index = i; +- if (!received) { +- napi_complete(napi); +- enable_irq(sw->rx_irq); +- budget = 0; +- +- /* If 1 or more frames came in during IRQ enable, re-schedule */ +- if (rx_ring->desc[i].cown) +- eth_schedule_poll(sw); +- } +- +- spin_lock_bh(&tx_lock); +- eth_complete_tx(sw); +- spin_unlock_bh(&tx_lock); + + cns3xxx_alloc_rx_buf(sw, received); +- + wmb(); + enable_rx_dma(sw); + +- return budget; ++ if (received < budget && napi_complete_done(napi, received)) { ++ enable_irq(sw->rx_irq); ++ } ++ ++ spin_lock_bh(&tx_lock); ++ eth_complete_tx(sw); ++ spin_unlock_bh(&tx_lock); ++ ++ return received; + } + + static void eth_set_desc(struct sw *sw, struct _tx_ring *tx_ring, int index, +@@ -856,18 +850,6 @@ static void cns3xxx_get_drvinfo(struct n + strcpy(info->bus_info, "internal"); + } + +-static int cns3xxx_get_settings(struct net_device *dev, struct ethtool_cmd *cmd) +-{ +- struct port *port = netdev_priv(dev); +- return phy_ethtool_gset(port->phydev, cmd); +-} +- +-static int cns3xxx_set_settings(struct net_device *dev, struct ethtool_cmd *cmd) +-{ +- struct port *port = netdev_priv(dev); +- return phy_ethtool_sset(port->phydev, cmd); +-} +- + static int cns3xxx_nway_reset(struct net_device *dev) + { + struct port *port = netdev_priv(dev); +@@ -876,8 +858,8 @@ static int cns3xxx_nway_reset(struct net + + static struct ethtool_ops cns3xxx_ethtool_ops = { + .get_drvinfo = cns3xxx_get_drvinfo, +- .get_settings = cns3xxx_get_settings, +- .set_settings = cns3xxx_set_settings, ++ .get_link_ksettings = phy_ethtool_get_link_ksettings, ++ .set_link_ksettings = phy_ethtool_set_link_ksettings, + .nway_reset = cns3xxx_nway_reset, + .get_link = ethtool_op_get_link, + }; +@@ -1177,7 +1159,7 @@ static int eth_init_one(struct platform_ + goto err_remove_mdio; + } + +- strcpy(napi_dev->name, "switch%d"); ++ strcpy(napi_dev->name, "cns3xxx_eth"); + napi_dev->features = NETIF_F_IP_CSUM | NETIF_F_SG | NETIF_F_FRAGLIST; + + SET_NETDEV_DEV(napi_dev, &pdev->dev);