From patchwork Fri Apr 12 03:27:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Kicinski X-Patchwork-Id: 1084400 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=netronome.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="zPxrffL3"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 44gNdY0t8Jz9s6w for ; Fri, 12 Apr 2019 13:28:17 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726777AbfDLD2P (ORCPT ); Thu, 11 Apr 2019 23:28:15 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:35097 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726709AbfDLD2O (ORCPT ); Thu, 11 Apr 2019 23:28:14 -0400 Received: by mail-qk1-f195.google.com with SMTP id a71so4857926qkg.2 for ; Thu, 11 Apr 2019 20:28:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MESZlZzZbiIgV2okI5Xx9qcXsHrIjDhgBEzFomWH768=; b=zPxrffL3qteh9Yi2CRYwG+s4Y9QFF0KTBZJp8C2shOY58RABfMVgJzk8l8dP79a+mW bbpF1aD57TEhFRVpUyeL7/sMdfJIkhwiX1PqL1xwr0KzPuh2OdrWm6KvIK3Q7xgkAbFP 0wQJ9zEcOk315RLVYk7YYbihZSY/gqKoI9532gCyzs5UtF5MkWu7/YuSX/DLWt0jyBbF G3P2JHFdzQ1nQ2yrj4E+05oDoNqIypjjUy4QVUDFVgCnEymtxxmVeeDp7aO7+mo45mA7 oa6xn6CkPLvxfL9iBmAQZItOjH/OfBoGBo0/oRiLngga3Lbb0zbB1txh4uRcDtpPVL7c LCqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MESZlZzZbiIgV2okI5Xx9qcXsHrIjDhgBEzFomWH768=; b=J/O2OhaA/YhGrv9dEekPe7cxF8Z/HwyOhVAui3b35f72tptKwXtKehnNBLBDVqdxVI s9CnMRiSVuf9Kb0fk9R11Jj3jiam4dd+TkkJdWxnZhZqEa3IkwCB/aRAeScpCL40fSKh RG9rs0wS5trovEtms9oFlaG94T5Ix7+yKSu3g4oFaH3gzymLpggKFVX43yLL7vxYSpS3 +xuw0uU5VMIJa/QCjD1A92JdEs9/ncMPORTikOsYWTP7qsJqMck/S5k27ET1m0q9k6Ue fVh4aurr9t2Mgb+lYFKIw2sFUxaUddDXSjb9YzbQ7UPksNYZGHNjRK45iER9phT8MTTR rUxQ== X-Gm-Message-State: APjAAAWXRAZonc1ElwUm207bifVLolHjmS/aXsLDMouRhT1e9OZ4flj2 s0TF3TdHdKer0iKqWERfZZe5tQ== X-Google-Smtp-Source: APXvYqxaftsEAjoRvLJykvFDnOPewpsGmOF94iuH4s+LiDRtCuaqZXnEdaZw48ZYHWYO1PHE0pMfAg== X-Received: by 2002:a37:4896:: with SMTP id v144mr41684232qka.194.1555039691910; Thu, 11 Apr 2019 20:28:11 -0700 (PDT) Received: from jkicinski-Precision-T1700.netronome.com ([66.60.152.14]) by smtp.gmail.com with ESMTPSA id y55sm26592801qtk.13.2019.04.11.20.28.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Apr 2019 20:28:11 -0700 (PDT) From: Jakub Kicinski To: davem@davemloft.net Cc: netdev@vger.kernel.org, oss-drivers@netronome.com, alexei.starovoitov@gmail.com, Jakub Kicinski , Dirk van der Merwe Subject: [PATCH net-next 2/4] nfp: add a mutex lock for the vNIC ctrl BAR Date: Thu, 11 Apr 2019 20:27:05 -0700 Message-Id: <20190412032707.4522-3-jakub.kicinski@netronome.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190412032707.4522-1-jakub.kicinski@netronome.com> References: <20190412032707.4522-1-jakub.kicinski@netronome.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Soon we will try to write to the vNIC mailbox without RTNL held. Add a new mutex to protect access to specific parts of the PCI control BAR. Move the mailbox size checking to the mailbox lock() helper, where it can be more effective (happen prior to potential overwrite of other data). Signed-off-by: Jakub Kicinski Reviewed-by: Dirk van der Merwe --- drivers/net/ethernet/netronome/nfp/abm/ctrl.c | 8 ++- drivers/net/ethernet/netronome/nfp/nfp_net.h | 23 +++++- .../ethernet/netronome/nfp/nfp_net_common.c | 72 +++++++++++++++---- .../net/ethernet/netronome/nfp/nfp_net_ctrl.h | 7 -- 4 files changed, 87 insertions(+), 23 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/abm/ctrl.c b/drivers/net/ethernet/netronome/nfp/abm/ctrl.c index 9584f03f3efa..69e84ff7f2e5 100644 --- a/drivers/net/ethernet/netronome/nfp/abm/ctrl.c +++ b/drivers/net/ethernet/netronome/nfp/abm/ctrl.c @@ -261,10 +261,15 @@ int nfp_abm_ctrl_qm_disable(struct nfp_abm *abm) int nfp_abm_ctrl_prio_map_update(struct nfp_abm_link *alink, u32 *packed) { + const u32 cmd = NFP_NET_CFG_MBOX_CMD_PCI_DSCP_PRIOMAP_SET; struct nfp_net *nn = alink->vnic; unsigned int i; int err; + err = nfp_net_mbox_lock(nn, alink->abm->prio_map_len); + if (err) + return err; + /* Write data_len and wipe reserved */ nn_writeq(nn, nn->tlv_caps.mbox_off + NFP_NET_ABM_MBOX_DATALEN, alink->abm->prio_map_len); @@ -273,8 +278,7 @@ int nfp_abm_ctrl_prio_map_update(struct nfp_abm_link *alink, u32 *packed) nn_writel(nn, nn->tlv_caps.mbox_off + NFP_NET_ABM_MBOX_DATA + i, packed[i / sizeof(u32)]); - err = nfp_net_reconfig_mbox(nn, - NFP_NET_CFG_MBOX_CMD_PCI_DSCP_PRIOMAP_SET); + err = nfp_net_mbox_reconfig_and_unlock(nn, cmd); if (err) nfp_err(alink->abm->app->cpp, "setting DSCP -> VQ map failed with error %d\n", err); diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net.h b/drivers/net/ethernet/netronome/nfp/nfp_net.h index be37c2d6151c..df9aff2684ed 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_net.h @@ -539,12 +539,17 @@ struct nfp_net_dp { * @shared_handler: Handler for shared interrupts * @shared_name: Name for shared interrupt * @me_freq_mhz: ME clock_freq (MHz) - * @reconfig_lock: Protects HW reconfiguration request regs/machinery + * @reconfig_lock: Protects @reconfig_posted, @reconfig_timer_active, + * @reconfig_sync_present and HW reconfiguration request + * regs/machinery from async requests (sync must take + * @bar_lock) * @reconfig_posted: Pending reconfig bits coming from async sources * @reconfig_timer_active: Timer for reading reconfiguration results is pending * @reconfig_sync_present: Some thread is performing synchronous reconfig * @reconfig_timer: Timer for async reading of reconfig results * @reconfig_in_progress_update: Update FW is processing now (debug only) + * @bar_lock: vNIC config BAR access lock, protects: update, + * mailbox area * @link_up: Is the link up? * @link_status_lock: Protects @link_* and ensures atomicity with BAR reading * @rx_coalesce_usecs: RX interrupt moderation usecs delay parameter @@ -615,6 +620,8 @@ struct nfp_net { struct timer_list reconfig_timer; u32 reconfig_in_progress_update; + struct mutex bar_lock; + u32 rx_coalesce_usecs; u32 rx_coalesce_max_frames; u32 tx_coalesce_usecs; @@ -839,6 +846,16 @@ static inline void nfp_ctrl_unlock(struct nfp_net *nn) spin_unlock_bh(&nn->r_vecs[0].lock); } +static inline void nn_ctrl_bar_lock(struct nfp_net *nn) +{ + mutex_lock(&nn->bar_lock); +} + +static inline void nn_ctrl_bar_unlock(struct nfp_net *nn) +{ + mutex_unlock(&nn->bar_lock); +} + /* Globals */ extern const char nfp_driver_version[]; @@ -871,7 +888,9 @@ unsigned int nfp_net_rss_key_sz(struct nfp_net *nn); void nfp_net_rss_write_itbl(struct nfp_net *nn); void nfp_net_rss_write_key(struct nfp_net *nn); void nfp_net_coalesce_write_cfg(struct nfp_net *nn); -int nfp_net_reconfig_mbox(struct nfp_net *nn, u32 mbox_cmd); +int nfp_net_mbox_lock(struct nfp_net *nn, unsigned int data_size); +int nfp_net_mbox_reconfig(struct nfp_net *nn, u32 mbox_cmd); +int nfp_net_mbox_reconfig_and_unlock(struct nfp_net *nn, u32 mbox_cmd); unsigned int nfp_net_irqs_alloc(struct pci_dev *pdev, struct msix_entry *irq_entries, diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index 60f2da438990..ab84a6c1ce66 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -260,7 +261,7 @@ static void nfp_net_reconfig_wait_posted(struct nfp_net *nn) } /** - * nfp_net_reconfig() - Reconfigure the firmware + * __nfp_net_reconfig() - Reconfigure the firmware * @nn: NFP Net device to reconfigure * @update: The value for the update field in the BAR config * @@ -270,10 +271,12 @@ static void nfp_net_reconfig_wait_posted(struct nfp_net *nn) * * Return: Negative errno on error, 0 on success */ -int nfp_net_reconfig(struct nfp_net *nn, u32 update) +static int __nfp_net_reconfig(struct nfp_net *nn, u32 update) { int ret; + lockdep_assert_held(&nn->bar_lock); + nfp_net_reconfig_sync_enter(nn); nfp_net_reconfig_start(nn, update); @@ -291,8 +294,31 @@ int nfp_net_reconfig(struct nfp_net *nn, u32 update) return ret; } +int nfp_net_reconfig(struct nfp_net *nn, u32 update) +{ + int ret; + + nn_ctrl_bar_lock(nn); + ret = __nfp_net_reconfig(nn, update); + nn_ctrl_bar_unlock(nn); + + return ret; +} + +int nfp_net_mbox_lock(struct nfp_net *nn, unsigned int data_size) +{ + if (nn->tlv_caps.mbox_len < NFP_NET_CFG_MBOX_SIMPLE_VAL + data_size) { + nn_err(nn, "mailbox too small for %u of data (%u)\n", + data_size, nn->tlv_caps.mbox_len); + return -EIO; + } + + nn_ctrl_bar_lock(nn); + return 0; +} + /** - * nfp_net_reconfig_mbox() - Reconfigure the firmware via the mailbox + * nfp_net_mbox_reconfig() - Reconfigure the firmware via the mailbox * @nn: NFP Net device to reconfigure * @mbox_cmd: The value for the mailbox command * @@ -300,19 +326,15 @@ int nfp_net_reconfig(struct nfp_net *nn, u32 update) * * Return: Negative errno on error, 0 on success */ -int nfp_net_reconfig_mbox(struct nfp_net *nn, u32 mbox_cmd) +int nfp_net_mbox_reconfig(struct nfp_net *nn, u32 mbox_cmd) { u32 mbox = nn->tlv_caps.mbox_off; int ret; - if (!nfp_net_has_mbox(&nn->tlv_caps)) { - nn_err(nn, "no mailbox present, command: %u\n", mbox_cmd); - return -EIO; - } - + lockdep_assert_held(&nn->bar_lock); nn_writeq(nn, mbox + NFP_NET_CFG_MBOX_SIMPLE_CMD, mbox_cmd); - ret = nfp_net_reconfig(nn, NFP_NET_CFG_UPDATE_MBOX); + ret = __nfp_net_reconfig(nn, NFP_NET_CFG_UPDATE_MBOX); if (ret) { nn_err(nn, "Mailbox update error\n"); return ret; @@ -321,6 +343,15 @@ int nfp_net_reconfig_mbox(struct nfp_net *nn, u32 mbox_cmd) return -nn_readl(nn, mbox + NFP_NET_CFG_MBOX_SIMPLE_RET); } +int nfp_net_mbox_reconfig_and_unlock(struct nfp_net *nn, u32 mbox_cmd) +{ + int ret; + + ret = nfp_net_mbox_reconfig(nn, mbox_cmd); + nn_ctrl_bar_unlock(nn); + return ret; +} + /* Interrupt configuration and handling */ @@ -3128,7 +3159,9 @@ static int nfp_net_change_mtu(struct net_device *netdev, int new_mtu) static int nfp_net_vlan_rx_add_vid(struct net_device *netdev, __be16 proto, u16 vid) { + const u32 cmd = NFP_NET_CFG_MBOX_CMD_CTAG_FILTER_ADD; struct nfp_net *nn = netdev_priv(netdev); + int err; /* Priority tagged packets with vlan id 0 are processed by the * NFP as untagged packets @@ -3136,17 +3169,23 @@ nfp_net_vlan_rx_add_vid(struct net_device *netdev, __be16 proto, u16 vid) if (!vid) return 0; + err = nfp_net_mbox_lock(nn, NFP_NET_CFG_VLAN_FILTER_SZ); + if (err) + return err; + nn_writew(nn, nn->tlv_caps.mbox_off + NFP_NET_CFG_VLAN_FILTER_VID, vid); nn_writew(nn, nn->tlv_caps.mbox_off + NFP_NET_CFG_VLAN_FILTER_PROTO, ETH_P_8021Q); - return nfp_net_reconfig_mbox(nn, NFP_NET_CFG_MBOX_CMD_CTAG_FILTER_ADD); + return nfp_net_mbox_reconfig_and_unlock(nn, cmd); } static int nfp_net_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto, u16 vid) { + const u32 cmd = NFP_NET_CFG_MBOX_CMD_CTAG_FILTER_KILL; struct nfp_net *nn = netdev_priv(netdev); + int err; /* Priority tagged packets with vlan id 0 are processed by the * NFP as untagged packets @@ -3154,11 +3193,15 @@ nfp_net_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto, u16 vid) if (!vid) return 0; + err = nfp_net_mbox_lock(nn, NFP_NET_CFG_VLAN_FILTER_SZ); + if (err) + return err; + nn_writew(nn, nn->tlv_caps.mbox_off + NFP_NET_CFG_VLAN_FILTER_VID, vid); nn_writew(nn, nn->tlv_caps.mbox_off + NFP_NET_CFG_VLAN_FILTER_PROTO, ETH_P_8021Q); - return nfp_net_reconfig_mbox(nn, NFP_NET_CFG_MBOX_CMD_CTAG_FILTER_KILL); + return nfp_net_mbox_reconfig_and_unlock(nn, cmd); } static void nfp_net_stat64(struct net_device *netdev, @@ -3650,6 +3693,8 @@ nfp_net_alloc(struct pci_dev *pdev, void __iomem *ctrl_bar, bool needs_netdev, nn->dp.txd_cnt = NFP_NET_TX_DESCS_DEFAULT; nn->dp.rxd_cnt = NFP_NET_RX_DESCS_DEFAULT; + mutex_init(&nn->bar_lock); + spin_lock_init(&nn->reconfig_lock); spin_lock_init(&nn->link_status_lock); @@ -3677,6 +3722,9 @@ nfp_net_alloc(struct pci_dev *pdev, void __iomem *ctrl_bar, bool needs_netdev, void nfp_net_free(struct nfp_net *nn) { WARN_ON(timer_pending(&nn->reconfig_timer) || nn->reconfig_posted); + + mutex_destroy(&nn->bar_lock); + if (nn->dp.netdev) free_netdev(nn->dp.netdev); else diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_ctrl.h b/drivers/net/ethernet/netronome/nfp/nfp_net_ctrl.h index f5d564bbb55a..25919e338071 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_ctrl.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_ctrl.h @@ -389,7 +389,6 @@ #define NFP_NET_CFG_MBOX_SIMPLE_CMD 0x0 #define NFP_NET_CFG_MBOX_SIMPLE_RET 0x4 #define NFP_NET_CFG_MBOX_SIMPLE_VAL 0x8 -#define NFP_NET_CFG_MBOX_SIMPLE_LEN 12 #define NFP_NET_CFG_MBOX_CMD_CTAG_FILTER_ADD 1 #define NFP_NET_CFG_MBOX_CMD_CTAG_FILTER_KILL 2 @@ -495,10 +494,4 @@ struct nfp_net_tlv_caps { int nfp_net_tlv_caps_parse(struct device *dev, u8 __iomem *ctrl_mem, struct nfp_net_tlv_caps *caps); - -static inline bool nfp_net_has_mbox(struct nfp_net_tlv_caps *caps) -{ - return caps->mbox_len >= NFP_NET_CFG_MBOX_SIMPLE_LEN; -} - #endif /* _NFP_NET_CTRL_H_ */