From patchwork Wed Apr 25 04:17:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 903944 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=netronome.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="qB4n0HcH"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40W6Nj0ZVzz9ryk for ; Wed, 25 Apr 2018 14:17:25 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751201AbeDYERW (ORCPT ); Wed, 25 Apr 2018 00:17:22 -0400 Received: from mail-pg0-f65.google.com ([74.125.83.65]:35167 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750941AbeDYERQ (ORCPT ); Wed, 25 Apr 2018 00:17:16 -0400 Received: by mail-pg0-f65.google.com with SMTP id j11so12400382pgf.2 for ; Tue, 24 Apr 2018 21:17:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=O5YNCqLPWhEtsLdKYu1IsBwVv2T9HaehNSYRG/rqQ5A=; b=qB4n0HcHWUvrdRj7GtyN7px4HACaQicqHklsPzS63GtOAVgN9mHopXyRsbgrK0Bsis L+BplYeMaW8WS3z655/cwcRyjv8UaT5Ai9Ee1Ibd6FMCJeUHaR9j8iWWCtIphJ++69us 8Rbj6TDNswDvQQ2HeH1Y4pCxCTMwsjeIlq8yKZl72q9BwBW7Ihn08i1F40VSNU4fCdNw RPFH4spjEkPrsCc4kwUTJrfntCtztWFUFGU9G6G8sAHGssNJsmPQZPIduR2V1o7lATUU FJqGkFBoWVtd9yDOSUwyfPNym2t141G87xDToFaI3VKrxYQSj6HqY20RlxOwvgD+aMyB kO4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=O5YNCqLPWhEtsLdKYu1IsBwVv2T9HaehNSYRG/rqQ5A=; b=r/xk54lYFILmi93CVKkqSaksI5KqZwIDpSH1rfzbJ3i2f3uA3Y68354DjMfC33nHnQ jf6kK1u6CEkJw7DwXWKsiY4Kj3VILW1dkr5wgk2qOcyV2PrULn2vkE5kw1TBbgqPIy4Z KD5Q7VZ75ZSmji9VbAjTiDWvoU/eGzrVv+yz0ZH/C7SnCyamgPddgjqiASoslFonwxkm ioInn4jPvg0+oRnt1rptSxNelqqDiXPSpqEsDUYFsUdmoMkAg4p90cIx1fIdQkvymAIN 2+JwuQ5DcOhHWS54LP6/Sa1Ut644p1P79XsKzhZXzTMhelIkhn6lxBuSIUx+8omx82tl Gwmw== X-Gm-Message-State: ALQs6tDH+O/Fhcrnwj+qVAFTjImgP9guGfazYuRg6+MSZIMDZ6hp1NNI Tw9WYp9XkulAy7h9+QDmpn84Up9q X-Google-Smtp-Source: AIpwx4/+3HWsy9J6agZIFyZLsdAA7n+vSrpGXLcsJeDA/jxCY4ZU2Q6vyBKfaYFxfHYRhgRjQ+X2uA== X-Received: by 10.99.123.85 with SMTP id k21mr22921231pgn.364.1524629835789; Tue, 24 Apr 2018 21:17:15 -0700 (PDT) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id m185sm30622153pfm.118.2018.04.24.21.17.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Apr 2018 21:17:15 -0700 (PDT) From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, oss-drivers@netronome.com, Jakub Kicinski Subject: [PATCH net-next 1/4] nfp: reset local locks on init Date: Tue, 24 Apr 2018 21:17:01 -0700 Message-Id: <20180425041704.26882-2-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180425041704.26882-1-jakub.kicinski@netronome.com> References: <20180425041704.26882-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org NFP locks record the owner when held, for PCIe devices the owner ID will be the PCIe link number. When driver loads it should scan known locks and if they indicate that they are held by local endpoint but the driver doesn't hold them - release them. Locks can be left taken for instance when kernel gets kexec-ed or after a crash. Management FW tries to clean up stale locks too, but it currently depends on PCIe link going down which doesn't always happen. Signed-off-by: Jakub Kicinski Reviewed-by: Dirk van der Merwe --- drivers/net/ethernet/netronome/nfp/nfp_main.c | 5 ++ drivers/net/ethernet/netronome/nfp/nfpcore/nfp.h | 2 + .../net/ethernet/netronome/nfp/nfpcore/nfp_cpp.h | 2 + .../net/ethernet/netronome/nfp/nfpcore/nfp_mutex.c | 45 +++++++++++++++++ .../ethernet/netronome/nfp/nfpcore/nfp_resource.c | 59 ++++++++++++++++++++++ 5 files changed, 113 insertions(+) diff --git a/drivers/net/ethernet/netronome/nfp/nfp_main.c b/drivers/net/ethernet/netronome/nfp/nfp_main.c index 6a3e3231e111..c8aef9508109 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_main.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_main.c @@ -489,6 +489,10 @@ static int nfp_pci_probe(struct pci_dev *pdev, goto err_disable_msix; } + err = nfp_resource_table_init(pf->cpp); + if (err) + goto err_cpp_free; + pf->hwinfo = nfp_hwinfo_read(pf->cpp); dev_info(&pdev->dev, "Assembly: %s%s%s-%s CPLD: %s\n", @@ -551,6 +555,7 @@ static int nfp_pci_probe(struct pci_dev *pdev, vfree(pf->dumpspec); err_hwinfo_free: kfree(pf->hwinfo); +err_cpp_free: nfp_cpp_free(pf->cpp); err_disable_msix: destroy_workqueue(pf->wq); diff --git a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp.h b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp.h index ced62d112aa2..f44d0a857314 100644 --- a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp.h +++ b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp.h @@ -94,6 +94,8 @@ int nfp_nsp_read_sensors(struct nfp_nsp *state, unsigned int sensor_mask, /* MAC Statistics Accumulator */ #define NFP_RESOURCE_MAC_STATISTICS "mac.stat" +int nfp_resource_table_init(struct nfp_cpp *cpp); + struct nfp_resource * nfp_resource_acquire(struct nfp_cpp *cpp, const char *name); diff --git a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_cpp.h b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_cpp.h index c8f2c064cce3..4e19add1c539 100644 --- a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_cpp.h +++ b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_cpp.h @@ -295,6 +295,8 @@ void nfp_cpp_mutex_free(struct nfp_cpp_mutex *mutex); int nfp_cpp_mutex_lock(struct nfp_cpp_mutex *mutex); int nfp_cpp_mutex_unlock(struct nfp_cpp_mutex *mutex); int nfp_cpp_mutex_trylock(struct nfp_cpp_mutex *mutex); +int nfp_cpp_mutex_reclaim(struct nfp_cpp *cpp, int target, + unsigned long long address); /** * nfp_cppcore_pcie_unit() - Get PCI Unit of a CPP handle diff --git a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_mutex.c b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_mutex.c index cb28ac03e4ca..c88bf673cb76 100644 --- a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_mutex.c +++ b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_mutex.c @@ -59,6 +59,11 @@ static u32 nfp_mutex_unlocked(u16 interface) return (u32)interface << 16 | 0x0000; } +static u32 nfp_mutex_owner(u32 val) +{ + return val >> 16; +} + static bool nfp_mutex_is_locked(u32 val) { return (val & 0xffff) == 0x000f; @@ -351,3 +356,43 @@ int nfp_cpp_mutex_trylock(struct nfp_cpp_mutex *mutex) return nfp_mutex_is_locked(tmp) ? -EBUSY : -EINVAL; } + +/** + * nfp_cpp_mutex_reclaim() - Unlock mutex if held by local endpoint + * @cpp: NFP CPP handle + * @target: NFP CPP target ID (ie NFP_CPP_TARGET_CLS or NFP_CPP_TARGET_MU) + * @address: Offset into the address space of the NFP CPP target ID + * + * Release lock if held by local system. Extreme care is advised, call only + * when no local lock users can exist. + * + * Return: 0 if the lock was OK, 1 if locked by us, -errno on invalid mutex + */ +int nfp_cpp_mutex_reclaim(struct nfp_cpp *cpp, int target, + unsigned long long address) +{ + const u32 mur = NFP_CPP_ID(target, 3, 0); /* atomic_read */ + const u32 muw = NFP_CPP_ID(target, 4, 0); /* atomic_write */ + u16 interface = nfp_cpp_interface(cpp); + int err; + u32 tmp; + + err = nfp_cpp_mutex_validate(interface, &target, address); + if (err) + return err; + + /* Check lock */ + err = nfp_cpp_readl(cpp, mur, address, &tmp); + if (err < 0) + return err; + + if (nfp_mutex_is_unlocked(tmp) || nfp_mutex_owner(tmp) != interface) + return 0; + + /* Bust the lock */ + err = nfp_cpp_writel(cpp, muw, address, nfp_mutex_unlocked(interface)); + if (err < 0) + return err; + + return 1; +} diff --git a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_resource.c b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_resource.c index 7e14725055c7..2dd89dba9311 100644 --- a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_resource.c +++ b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp_resource.c @@ -338,3 +338,62 @@ u64 nfp_resource_size(struct nfp_resource *res) { return res->size; } + +/** + * nfp_resource_table_init() - Run initial checks on the resource table + * @cpp: NFP CPP handle + * + * Start-of-day init procedure for resource table. Must be called before + * any local resource table users may exist. + * + * Return: 0 on success, -errno on failure + */ +int nfp_resource_table_init(struct nfp_cpp *cpp) +{ + struct nfp_cpp_mutex *dev_mutex; + int i, err; + + err = nfp_cpp_mutex_reclaim(cpp, NFP_RESOURCE_TBL_TARGET, + NFP_RESOURCE_TBL_BASE); + if (err < 0) { + nfp_err(cpp, "Error: failed to reclaim resource table mutex\n"); + return err; + } + if (err) + nfp_warn(cpp, "Warning: busted main resource table mutex\n"); + + dev_mutex = nfp_cpp_mutex_alloc(cpp, NFP_RESOURCE_TBL_TARGET, + NFP_RESOURCE_TBL_BASE, + NFP_RESOURCE_TBL_KEY); + if (!dev_mutex) + return -ENOMEM; + + if (nfp_cpp_mutex_lock(dev_mutex)) { + nfp_err(cpp, "Error: failed to claim resource table mutex\n"); + nfp_cpp_mutex_free(dev_mutex); + return -EINVAL; + } + + /* Resource 0 is the dev_mutex, start from 1 */ + for (i = 1; i < NFP_RESOURCE_TBL_ENTRIES; i++) { + u64 addr = NFP_RESOURCE_TBL_BASE + + sizeof(struct nfp_resource_entry) * i; + + err = nfp_cpp_mutex_reclaim(cpp, NFP_RESOURCE_TBL_TARGET, addr); + if (err < 0) { + nfp_err(cpp, + "Error: failed to reclaim resource %d mutex\n", + i); + goto err_unlock; + } + if (err) + nfp_warn(cpp, "Warning: busted resource %d mutex\n", i); + } + + err = 0; +err_unlock: + nfp_cpp_mutex_unlock(dev_mutex); + nfp_cpp_mutex_free(dev_mutex); + + return err; +} From patchwork Wed Apr 25 04:17:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 903946 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=netronome.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="0JBHTnXQ"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40W6Nr52SQz9ryk for ; Wed, 25 Apr 2018 14:17:32 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751244AbeDYERa (ORCPT ); Wed, 25 Apr 2018 00:17:30 -0400 Received: from mail-pf0-f194.google.com ([209.85.192.194]:46230 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750807AbeDYERR (ORCPT ); Wed, 25 Apr 2018 00:17:17 -0400 Received: by mail-pf0-f194.google.com with SMTP id h69so14074743pfe.13 for ; Tue, 24 Apr 2018 21:17:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=aAwVgYHp9r4uMz9Bwh/X2PlY5xHOI7QWbwjhyMSAPk8=; b=0JBHTnXQrh4bv8ICpHnUWg9WASHak1HzYrO30+p7wwcI1rQQzJO+oJlNgKxA8wM4GM Dn6NRj/pj5YN78dXP+skCnWiOW5LB45Yt5RxSHdXR9gIZbCc264McrnY523h76B9TCLv /+Olh9Q4RCWptfhMlyZ5s1fYzzE0d7IWaAEP/0jfQadTzBWHgHXx38YHNVrMFHXS9nyd uUbZaHIYKGv+UyHZZFKVMcRwyOktLTFW/xetUy9IjHbqXGOu1hUeTpU5jk5B5AJr6lrL KeJASbn8NFHbVoXi3f47AMR0z7vkoVMehnscrXx5oTBW7muyl9zOBRJNshmSot2pn+6t 5KQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=aAwVgYHp9r4uMz9Bwh/X2PlY5xHOI7QWbwjhyMSAPk8=; b=dNlsECShNtn13FppoTKcbVua3grGOE9gI36TX9IazX42M7A16dP0rAM9DWKFLuPv7L Aox2ER1qsxxoRly6V4B+e1L/Amk8nrQHG3XqphQWr1jtdI/ZXfMZWcacHcWgwKbm5Keg zCG2ypQ/5ruCeTnAyCJgvc6EGTap7syWg4kUfq3fSFqaWCN8iqqgl6/MG2/AWyP077hg 6mhiaFucpAukJlttGTOEAHJsDvlQ12TlU0vRGtb8sSw/0ZRV9HKNFN0DLCZMLuP8OEgM /0fvpjvONv6ebcI2ls2JGpwDtJ39n3lV0OWFVfsOJzCXt17+DH2S3aP767mrp7cQYuFN gxxg== X-Gm-Message-State: ALQs6tCvJfYiRC3nYVhRf8aReybFh8ypr/pijEffx+lt8T9RmUD5Yw58 VEblfsc2wglt/d1ng0xABlja2A== X-Google-Smtp-Source: AIpwx4+sNV03gpdbNUi7t7h4pnVQUufBYTwucRlUaKHsh5e6Bk4X45uv4FK3VZCt9c0S5I/1ex3T+w== X-Received: by 2002:a17:902:3f83:: with SMTP id a3-v6mr27738955pld.279.1524629836677; Tue, 24 Apr 2018 21:17:16 -0700 (PDT) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id m185sm30622153pfm.118.2018.04.24.21.17.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Apr 2018 21:17:16 -0700 (PDT) From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, oss-drivers@netronome.com, Jakub Kicinski Subject: [PATCH net-next 2/4] nfp: print PCIe link bandwidth on probe Date: Tue, 24 Apr 2018 21:17:02 -0700 Message-Id: <20180425041704.26882-3-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180425041704.26882-1-jakub.kicinski@netronome.com> References: <20180425041704.26882-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org To aid debugging of performance issues caused by limited PCIe bandwidth print the PCIe link information on probe. Signed-off-by: Jakub Kicinski Reviewed-by: Dirk van der Merwe --- drivers/net/ethernet/netronome/nfp/nfpcore/nfp6000_pcie.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp6000_pcie.c b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp6000_pcie.c index cd678323bacb..a0e336bd1d85 100644 --- a/drivers/net/ethernet/netronome/nfp/nfpcore/nfp6000_pcie.c +++ b/drivers/net/ethernet/netronome/nfp/nfpcore/nfp6000_pcie.c @@ -1330,6 +1330,7 @@ struct nfp_cpp *nfp_cpp_from_nfp6000_pcie(struct pci_dev *pdev) /* Finished with card initialization. */ dev_info(&pdev->dev, "Netronome Flow Processor NFP4000/NFP6000 PCIe Card Probe\n"); + pcie_print_link_status(pdev); nfp = kzalloc(sizeof(*nfp), GFP_KERNEL); if (!nfp) { From patchwork Wed Apr 25 04:17:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 903945 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=netronome.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="CHwSxOPu"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40W6Nn5bqGz9s0n for ; Wed, 25 Apr 2018 14:17:29 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751229AbeDYER1 (ORCPT ); Wed, 25 Apr 2018 00:17:27 -0400 Received: from mail-pf0-f193.google.com ([209.85.192.193]:46230 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750969AbeDYERS (ORCPT ); Wed, 25 Apr 2018 00:17:18 -0400 Received: by mail-pf0-f193.google.com with SMTP id h69so14074765pfe.13 for ; Tue, 24 Apr 2018 21:17:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BjgGruqFy9honHENkEZs4xQTwoWCY9cJ8EaVM5BNrT0=; b=CHwSxOPu3qg7V1zTQH/kabI5WX/lAMAX0ezAThmG17CYJkxFdKyjMimWm43GWTmFqG AjLT8IWSU4u0KgCMNqMP8OI7EaXJ0dYD6W1/itoczWaPVevrEyu9OffY6SsvCTERekT6 LVgktlNCapvuj+WYAD5V5CdWqe5iBB9q3FOJ1nSLp+GHwzdMUlLHW6pUaz0R7vh+tQ/G jMSkmybrSnyvkfmQNoPr4nB92kVC+kTLKxQ9V1JztqrVjUq/+lwi1MRX01dcx+WapO7e eer+nMq5MVwe2ey34GdNAmknNJ+VoCGBRuc25tJ+1Y10euBjY7/xynDRqGczxYkeLrfH yBcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BjgGruqFy9honHENkEZs4xQTwoWCY9cJ8EaVM5BNrT0=; b=DyfaO6vkLhNr4R16idTe+le/mhR3wZW53pd8+Dwd/9Kd9UiCen8XysGTDidcL0lVF7 IikKLkl1Omk/NB+Pvuk6WRxd6yzNtkik3IkJPJHSoXHnXHIkRnraxNSTVl9bmcij5hSg Q7QhwFwJaAZvoHAhYUM3bgSQ3ObMfyf/2JG/DQjPxzeeDesXRJw2+CbEwNVy4xFY3oIa SV4M4sSu+Ro130NXiY3oxjUPuASfXtU1iqQ2di3CmcxZxNB0QGMowPpY1eNeVmTeSX+t nQ+ouI8NIn9SIq6auIeUKi5E34IrjrwdfLFV8epy1Y4ZOywFcw3An6cTGGZvBp6YdJdP jEyA== X-Gm-Message-State: ALQs6tDAy7mmuI1mm5IApmUiI9zaYlfrWpU0BvGI57PurGmj5YVpwtU+ NAspKHBX8JGr4EOUO/7PP8R1aw== X-Google-Smtp-Source: AIpwx4+PRT75WXY+Fz9PRt6/ahVtwFzUgCzSzcMtXelii7xqFkgr2/PmTbG4tYAPvOGEyHxY+ZWJag== X-Received: by 10.99.127.88 with SMTP id p24mr22144718pgn.290.1524629837515; Tue, 24 Apr 2018 21:17:17 -0700 (PDT) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id m185sm30622153pfm.118.2018.04.24.21.17.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Apr 2018 21:17:16 -0700 (PDT) From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, oss-drivers@netronome.com, John Hurley Subject: [PATCH net-next 3/4] nfp: flower: support offloading multiple rules with same cookie Date: Tue, 24 Apr 2018 21:17:03 -0700 Message-Id: <20180425041704.26882-4-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180425041704.26882-1-jakub.kicinski@netronome.com> References: <20180425041704.26882-1-jakub.kicinski@netronome.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: John Hurley When multiple netdevs are attached to a tc offload block and register for callbacks, a rule added to the block will be propogated to all netdevs. Previously these were detected as duplicates (based on cookie) and rejected. Modify the rule nfp lookup function to optionally include an ingress netdev and a host context along with the cookie value when searching for a rule. When a new rule is passed to the driver, the netdev the rule is to be attached to is considered when searching for dublicates. When a stats update is received from HW, the host context is used alongside the cookie to map to the correct host rule. Signed-off-by: John Hurley Reviewed-by: Jakub Kicinski --- drivers/net/ethernet/netronome/nfp/flower/main.h | 8 +++++-- .../net/ethernet/netronome/nfp/flower/metadata.c | 20 +++++++++------- .../net/ethernet/netronome/nfp/flower/offload.c | 27 ++++++++++++++++------ 3 files changed, 38 insertions(+), 17 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/flower/main.h b/drivers/net/ethernet/netronome/nfp/flower/main.h index c67e1b54c614..9e6804bc9b40 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/main.h +++ b/drivers/net/ethernet/netronome/nfp/flower/main.h @@ -47,6 +47,7 @@ struct net_device; struct nfp_app; +#define NFP_FL_STATS_CTX_DONT_CARE cpu_to_be32(0xffffffff) #define NFP_FL_STATS_ENTRY_RS BIT(20) #define NFP_FL_STATS_ELEM_RS 4 #define NFP_FL_REPEATED_HASH_MAX BIT(17) @@ -189,6 +190,7 @@ struct nfp_fl_payload { spinlock_t lock; /* lock stats */ struct nfp_fl_stats stats; __be32 nfp_tun_ipv4_addr; + struct net_device *ingress_dev; char *unmasked_data; char *mask_data; char *action_data; @@ -216,12 +218,14 @@ int nfp_flower_compile_action(struct tc_cls_flower_offload *flow, struct nfp_fl_payload *nfp_flow); int nfp_compile_flow_metadata(struct nfp_app *app, struct tc_cls_flower_offload *flow, - struct nfp_fl_payload *nfp_flow); + struct nfp_fl_payload *nfp_flow, + struct net_device *netdev); int nfp_modify_flow_metadata(struct nfp_app *app, struct nfp_fl_payload *nfp_flow); struct nfp_fl_payload * -nfp_flower_search_fl_table(struct nfp_app *app, unsigned long tc_flower_cookie); +nfp_flower_search_fl_table(struct nfp_app *app, unsigned long tc_flower_cookie, + struct net_device *netdev, __be32 host_ctx); struct nfp_fl_payload * nfp_flower_remove_fl_table(struct nfp_app *app, unsigned long tc_flower_cookie); diff --git a/drivers/net/ethernet/netronome/nfp/flower/metadata.c b/drivers/net/ethernet/netronome/nfp/flower/metadata.c index db977cf8e933..21668aa435e8 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/metadata.c +++ b/drivers/net/ethernet/netronome/nfp/flower/metadata.c @@ -99,14 +99,18 @@ static int nfp_get_stats_entry(struct nfp_app *app, u32 *stats_context_id) /* Must be called with either RTNL or rcu_read_lock */ struct nfp_fl_payload * -nfp_flower_search_fl_table(struct nfp_app *app, unsigned long tc_flower_cookie) +nfp_flower_search_fl_table(struct nfp_app *app, unsigned long tc_flower_cookie, + struct net_device *netdev, __be32 host_ctx) { struct nfp_flower_priv *priv = app->priv; struct nfp_fl_payload *flower_entry; hash_for_each_possible_rcu(priv->flow_table, flower_entry, link, tc_flower_cookie) - if (flower_entry->tc_flower_cookie == tc_flower_cookie) + if (flower_entry->tc_flower_cookie == tc_flower_cookie && + (!netdev || flower_entry->ingress_dev == netdev) && + (host_ctx == NFP_FL_STATS_CTX_DONT_CARE || + flower_entry->meta.host_ctx_id == host_ctx)) return flower_entry; return NULL; @@ -121,13 +125,11 @@ nfp_flower_update_stats(struct nfp_app *app, struct nfp_fl_stats_frame *stats) flower_cookie = be64_to_cpu(stats->stats_cookie); rcu_read_lock(); - nfp_flow = nfp_flower_search_fl_table(app, flower_cookie); + nfp_flow = nfp_flower_search_fl_table(app, flower_cookie, NULL, + stats->stats_con_id); if (!nfp_flow) goto exit_rcu_unlock; - if (nfp_flow->meta.host_ctx_id != stats->stats_con_id) - goto exit_rcu_unlock; - spin_lock(&nfp_flow->lock); nfp_flow->stats.pkts += be32_to_cpu(stats->pkt_count); nfp_flow->stats.bytes += be64_to_cpu(stats->byte_count); @@ -317,7 +319,8 @@ nfp_check_mask_remove(struct nfp_app *app, char *mask_data, u32 mask_len, int nfp_compile_flow_metadata(struct nfp_app *app, struct tc_cls_flower_offload *flow, - struct nfp_fl_payload *nfp_flow) + struct nfp_fl_payload *nfp_flow, + struct net_device *netdev) { struct nfp_flower_priv *priv = app->priv; struct nfp_fl_payload *check_entry; @@ -348,7 +351,8 @@ int nfp_compile_flow_metadata(struct nfp_app *app, nfp_flow->stats.bytes = 0; nfp_flow->stats.used = jiffies; - check_entry = nfp_flower_search_fl_table(app, flow->cookie); + check_entry = nfp_flower_search_fl_table(app, flow->cookie, netdev, + NFP_FL_STATS_CTX_DONT_CARE); if (check_entry) { if (nfp_release_stats_entry(app, stats_cxt)) return -EINVAL; diff --git a/drivers/net/ethernet/netronome/nfp/flower/offload.c b/drivers/net/ethernet/netronome/nfp/flower/offload.c index 114d2ab02a38..bdc82e11a31e 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/offload.c +++ b/drivers/net/ethernet/netronome/nfp/flower/offload.c @@ -419,6 +419,8 @@ nfp_flower_add_offload(struct nfp_app *app, struct net_device *netdev, goto err_free_key_ls; } + flow_pay->ingress_dev = egress ? NULL : netdev; + err = nfp_flower_compile_flow_match(flow, key_layer, netdev, flow_pay, tun_type); if (err) @@ -428,7 +430,8 @@ nfp_flower_add_offload(struct nfp_app *app, struct net_device *netdev, if (err) goto err_destroy_flow; - err = nfp_compile_flow_metadata(app, flow, flow_pay); + err = nfp_compile_flow_metadata(app, flow, flow_pay, + flow_pay->ingress_dev); if (err) goto err_destroy_flow; @@ -462,6 +465,7 @@ nfp_flower_add_offload(struct nfp_app *app, struct net_device *netdev, * @app: Pointer to the APP handle * @netdev: netdev structure. * @flow: TC flower classifier offload structure + * @egress: Netdev is the egress dev. * * Removes a flow from the repeated hash structure and clears the * action payload. @@ -470,13 +474,16 @@ nfp_flower_add_offload(struct nfp_app *app, struct net_device *netdev, */ static int nfp_flower_del_offload(struct nfp_app *app, struct net_device *netdev, - struct tc_cls_flower_offload *flow) + struct tc_cls_flower_offload *flow, bool egress) { struct nfp_port *port = nfp_port_from_netdev(netdev); struct nfp_fl_payload *nfp_flow; + struct net_device *ingr_dev; int err; - nfp_flow = nfp_flower_search_fl_table(app, flow->cookie); + ingr_dev = egress ? NULL : netdev; + nfp_flow = nfp_flower_search_fl_table(app, flow->cookie, ingr_dev, + NFP_FL_STATS_CTX_DONT_CARE); if (!nfp_flow) return -ENOENT; @@ -505,7 +512,9 @@ nfp_flower_del_offload(struct nfp_app *app, struct net_device *netdev, /** * nfp_flower_get_stats() - Populates flow stats obtained from hardware. * @app: Pointer to the APP handle + * @netdev: Netdev structure. * @flow: TC flower classifier offload structure + * @egress: Netdev is the egress dev. * * Populates a flow statistics structure which which corresponds to a * specific flow. @@ -513,11 +522,15 @@ nfp_flower_del_offload(struct nfp_app *app, struct net_device *netdev, * Return: negative value on error, 0 if stats populated successfully. */ static int -nfp_flower_get_stats(struct nfp_app *app, struct tc_cls_flower_offload *flow) +nfp_flower_get_stats(struct nfp_app *app, struct net_device *netdev, + struct tc_cls_flower_offload *flow, bool egress) { struct nfp_fl_payload *nfp_flow; + struct net_device *ingr_dev; - nfp_flow = nfp_flower_search_fl_table(app, flow->cookie); + ingr_dev = egress ? NULL : netdev; + nfp_flow = nfp_flower_search_fl_table(app, flow->cookie, ingr_dev, + NFP_FL_STATS_CTX_DONT_CARE); if (!nfp_flow) return -EINVAL; @@ -543,9 +556,9 @@ nfp_flower_repr_offload(struct nfp_app *app, struct net_device *netdev, case TC_CLSFLOWER_REPLACE: return nfp_flower_add_offload(app, netdev, flower, egress); case TC_CLSFLOWER_DESTROY: - return nfp_flower_del_offload(app, netdev, flower); + return nfp_flower_del_offload(app, netdev, flower, egress); case TC_CLSFLOWER_STATS: - return nfp_flower_get_stats(app, flower); + return nfp_flower_get_stats(app, netdev, flower, egress); } return -EOPNOTSUPP; From patchwork Wed Apr 25 04:17:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 903947 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=netronome.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="OOXGNMqW"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40W6Ns5ckYz9s0n for ; Wed, 25 Apr 2018 14:17:33 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751259AbeDYERc (ORCPT ); Wed, 25 Apr 2018 00:17:32 -0400 Received: from mail-pg0-f65.google.com ([74.125.83.65]:32986 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751031AbeDYERT (ORCPT ); Wed, 25 Apr 2018 00:17:19 -0400 Received: by mail-pg0-f65.google.com with SMTP id i194so12405957pgd.0 for ; Tue, 24 Apr 2018 21:17:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=E3HIsPncqBcI6VgaD1F9zdiAhPEK/rFb1WFnc0ZUXLg=; b=OOXGNMqWnBlYfXNzNPAGJJ2/A5fi25eX10RdytVNG1WNz9nnr4ZE3BZDVeF3XhwSCA p2TL3AaBKdZ3exraTpeNTEnSI4tO54BEisMDmcKGeaGN/1tBM38s/qDJwfOp4MgRjZD8 1taGlcPPfcjQ7QOOS7L/AXbDnxdlDTmznRbVy3Ra0jCIi6CouEeAiOv+SSaKambW8izB VAMzgarlhsaKcdF5wPqzmWa9zj0uij8CO45i5de9Ih5F5u+jb2NXebXZpJbvpgt+BAhI 6grtOZm6NwArPXrZEZ8CRT1FqIYi+iLcKnWpmdKs7NBuwcyxKTZSRUOyD5jLbNTzevEx B0qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=E3HIsPncqBcI6VgaD1F9zdiAhPEK/rFb1WFnc0ZUXLg=; b=I+CtKmUQvSqbHUm5BLqoJDmHQpsKPMIh8mXb4LqSuH3BLlvnf4sTvXjnnNoZQ8poWW SCubZWvAC/IlwGLUN7IfstozmZpNJsG6Ut49IVUE4wV1WeEUZs2s1Zb/YkcLEoJDD0wT Vc2JYYzJvOVVYqhHV834qvtRIRn4k+/VFV2C1FIHs8qDQPYk/002CDInFjac4ijbS829 Inxs47T1lCieowcxVIOsTnJ9bpUREq20U7IhXFVLMm6/QMUM99jaQdV8bd0hXePSwWds LHSJtT75zn0ktWLRgo7PYP5HBq11GmrNFM79prrVWpx1hZjm/eCThU5a4Q/oWKMLPUUI HVGA== X-Gm-Message-State: ALQs6tC7CHBrp1pBYsicOSaEJEm6whRWlUAEGSaswMEOdo45dHQzGamT VNhP9mX7EzYV2SfDzrcpVBk8fQ== X-Google-Smtp-Source: AB8JxZo9GGQILhAn+SqJJTImK+e1G91//PByiLt+F6PvggS0o8tC8UxOHzLTkovEe99mQWhdIodRdg== X-Received: by 10.101.75.2 with SMTP id r2mr3505753pgq.82.1524629838589; Tue, 24 Apr 2018 21:17:18 -0700 (PDT) Received: from jkicinski-Precision-T1700.netronome.com ([75.53.12.129]) by smtp.gmail.com with ESMTPSA id m185sm30622153pfm.118.2018.04.24.21.17.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Apr 2018 21:17:17 -0700 (PDT) From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, oss-drivers@netronome.com, John Hurley Subject: [PATCH net-next 4/4] nfp: flower: ignore duplicate cb requests for same rule Date: Tue, 24 Apr 2018 21:17:04 -0700 Message-Id: <20180425041704.26882-5-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180425041704.26882-1-jakub.kicinski@netronome.com> References: <20180425041704.26882-1-jakub.kicinski@netronome.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: John Hurley If a flower rule has a repr both as ingress and egress port then 2 callbacks may be generated for the same rule request. Add an indicator to each flow as to whether or not it was added from an ingress registered cb. If so then ignore add/del/stat requests to it from an egress cb. Signed-off-by: John Hurley Reviewed-by: Jakub Kicinski --- drivers/net/ethernet/netronome/nfp/flower/main.h | 1 + .../net/ethernet/netronome/nfp/flower/offload.c | 23 +++++++++++++++++++--- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/flower/main.h b/drivers/net/ethernet/netronome/nfp/flower/main.h index 9e6804bc9b40..733ff53cc601 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/main.h +++ b/drivers/net/ethernet/netronome/nfp/flower/main.h @@ -194,6 +194,7 @@ struct nfp_fl_payload { char *unmasked_data; char *mask_data; char *action_data; + bool ingress_offload; }; struct nfp_fl_stats_frame { diff --git a/drivers/net/ethernet/netronome/nfp/flower/offload.c b/drivers/net/ethernet/netronome/nfp/flower/offload.c index bdc82e11a31e..70ec9d821b91 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/offload.c +++ b/drivers/net/ethernet/netronome/nfp/flower/offload.c @@ -345,7 +345,7 @@ nfp_flower_calculate_key_layers(struct nfp_app *app, } static struct nfp_fl_payload * -nfp_flower_allocate_new(struct nfp_fl_key_ls *key_layer) +nfp_flower_allocate_new(struct nfp_fl_key_ls *key_layer, bool egress) { struct nfp_fl_payload *flow_pay; @@ -371,6 +371,8 @@ nfp_flower_allocate_new(struct nfp_fl_key_ls *key_layer) flow_pay->meta.flags = 0; spin_lock_init(&flow_pay->lock); + flow_pay->ingress_offload = !egress; + return flow_pay; err_free_mask: @@ -402,8 +404,20 @@ nfp_flower_add_offload(struct nfp_app *app, struct net_device *netdev, struct nfp_flower_priv *priv = app->priv; struct nfp_fl_payload *flow_pay; struct nfp_fl_key_ls *key_layer; + struct net_device *ingr_dev; int err; + ingr_dev = egress ? NULL : netdev; + flow_pay = nfp_flower_search_fl_table(app, flow->cookie, ingr_dev, + NFP_FL_STATS_CTX_DONT_CARE); + if (flow_pay) { + /* Ignore as duplicate if it has been added by different cb. */ + if (flow_pay->ingress_offload && egress) + return 0; + else + return -EOPNOTSUPP; + } + key_layer = kmalloc(sizeof(*key_layer), GFP_KERNEL); if (!key_layer) return -ENOMEM; @@ -413,7 +427,7 @@ nfp_flower_add_offload(struct nfp_app *app, struct net_device *netdev, if (err) goto err_free_key_ls; - flow_pay = nfp_flower_allocate_new(key_layer); + flow_pay = nfp_flower_allocate_new(key_layer, egress); if (!flow_pay) { err = -ENOMEM; goto err_free_key_ls; @@ -485,7 +499,7 @@ nfp_flower_del_offload(struct nfp_app *app, struct net_device *netdev, nfp_flow = nfp_flower_search_fl_table(app, flow->cookie, ingr_dev, NFP_FL_STATS_CTX_DONT_CARE); if (!nfp_flow) - return -ENOENT; + return egress ? 0 : -ENOENT; err = nfp_modify_flow_metadata(app, nfp_flow); if (err) @@ -534,6 +548,9 @@ nfp_flower_get_stats(struct nfp_app *app, struct net_device *netdev, if (!nfp_flow) return -EINVAL; + if (nfp_flow->ingress_offload && egress) + return 0; + spin_lock_bh(&nfp_flow->lock); tcf_exts_stats_update(flow->exts, nfp_flow->stats.bytes, nfp_flow->stats.pkts, nfp_flow->stats.used);