From patchwork Tue Nov 14 16:18:52 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Pirko X-Patchwork-Id: 837925 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=resnulli-us.20150623.gappssmtp.com header.i=@resnulli-us.20150623.gappssmtp.com header.b="zyMvfBtE"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3ybt5G6LQ2z9sPr for ; Wed, 15 Nov 2017 03:20:01 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755804AbdKNQT6 (ORCPT ); Tue, 14 Nov 2017 11:19:58 -0500 Received: from mail-wm0-f66.google.com ([74.125.82.66]:55446 "EHLO mail-wm0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754120AbdKNQTF (ORCPT ); Tue, 14 Nov 2017 11:19:05 -0500 Received: by mail-wm0-f66.google.com with SMTP id 9so17574878wme.4 for ; Tue, 14 Nov 2017 08:19:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=resnulli-us.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pzEfDYwUKoKoBnZqAtj0ZqzaCq4m4xaKT06/nq5xXQk=; b=zyMvfBtE4K4S5SPaC1BgU40RqJyZNMV9GiuPv0Wx3P8XcPXH9ZUYDH48TQFpwnC3Eu 4XbtufhD/lzLquXwK6PYl3k9LjEV7qWghMz7A1efnhuglKvovwKUEbFA8+Ij6+eryScV 25nwqVHitKLXefG2FcDF5g798mfacqwtYLP7sOaXMJDYLOLnNz5LCNQ3TfjREthVJZWG RTF9q/V0zsNE7LXNLMpq1lmh6pWmnucL+URVPEex/pj5sX5ZkminwHw+0PrJrLBVXqI1 90jUFETKfGu6tTaaCJFGwsp2mdtnI+uYhQyQ2Bza5C9BGYhmW74bEiqYaLcPfUdVqeyz 3zsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pzEfDYwUKoKoBnZqAtj0ZqzaCq4m4xaKT06/nq5xXQk=; b=iybKXADVp+quMxOJuzmUJe/VSTZCpsqpHwhl592ocub6x9bomvuYSwDjL3axATHy4K Y/jFHvqm4I+dGYzqkpyLsbxIgxdGprrvGMsT7oYGhqFWYAGhRmUZ/52ZWLYHYKV7lPQG bUbBSsLdq5BlnlwpvdQ9jJtNxKNWZlLXIFUYL2R/lHeiF8Gj0o2LwBR8rvqQuoKnxaS4 JRMxxDOH18IVjTTxRJ6rclHnSgGDkGMhvfPFyzxQTAkNLSmKSWZSY7Dp5iFVPDz8gvIt WfdRXL61BeAQjOMLN6RvR4SgHfgqjJfTWYAgaJWweZMxixHPiLd9A8sG8ZDWQGp9QjNT UQ+g== X-Gm-Message-State: AJaThX4reG8TOLJigbvTKV9pieEznLPjvIIpzAbllGuNBoQayxfruDFp hzCWvO9m2f0OUt81lGAm0ctRCEkX X-Google-Smtp-Source: AGs4zMY/IRm0mp9AEcQ29TBiPvqi52vhKhgtCkYTrhtiJu1DkcXmviCfQtfVg2rNwqCj4qqOZnCK3Q== X-Received: by 10.28.87.17 with SMTP id l17mr9401784wmb.158.1510676343252; Tue, 14 Nov 2017 08:19:03 -0800 (PST) Received: from localhost (jirka.pirko.cz. [84.16.102.26]) by smtp.gmail.com with ESMTPSA id o20sm18998309wro.6.2017.11.14.08.19.02 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 Nov 2017 08:19:02 -0800 (PST) From: Jiri Pirko To: netdev@vger.kernel.org Cc: davem@davemloft.net, mlxsw@mellanox.com, andrew@lunn.ch, vivien.didelot@savoirfairelinux.com, f.fainelli@gmail.com, michael.chan@broadcom.com, ganeshgr@chelsio.com, saeedm@mellanox.com, matanb@mellanox.com, leonro@mellanox.com, idosch@mellanox.com, jakub.kicinski@netronome.com, ast@kernel.org, daniel@iogearbox.net, simon.horman@netronome.com, pieter.jansenvanvuuren@netronome.com, john.hurley@netronome.com, alexander.h.duyck@intel.com, linville@tuxdriver.com, gospo@broadcom.com, steven.lin1@broadcom.com, yuvalm@mellanox.com, ogerlitz@mellanox.com, dsa@cumulusnetworks.com, roopa@cumulusnetworks.com Subject: [patch net-next RFC v2 11/11] mlxsw: core: Add support for reload Date: Tue, 14 Nov 2017 17:18:52 +0100 Message-Id: <20171114161852.6633-12-jiri@resnulli.us> X-Mailer: git-send-email 2.9.5 In-Reply-To: <20171114161852.6633-1-jiri@resnulli.us> References: <20171114161852.6633-1-jiri@resnulli.us> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Arkadi Sharshevsky Add support for hot reload. First, all the driver/core resources are released but the PCI and devlink instances, then reset is performed through the PCI interface. Finally the driver performs initialization. In case of reload failure the driver is left in a partially initialized state. Special care is taken during the driver removal in order to properly handle this state. Signed-off-by: Arkadi Sharshevsky Signed-off-by: Jiri Pirko --- drivers/net/ethernet/mellanox/mlxsw/core.c | 64 +++++++++++++++++++++++------- drivers/net/ethernet/mellanox/mlxsw/core.h | 5 ++- drivers/net/ethernet/mellanox/mlxsw/i2c.c | 5 ++- drivers/net/ethernet/mellanox/mlxsw/pci.c | 5 ++- 4 files changed, 59 insertions(+), 20 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c index 9fe25b1..4b33919 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core.c +++ b/drivers/net/ethernet/mellanox/mlxsw/core.c @@ -113,6 +113,7 @@ struct mlxsw_core { struct mlxsw_thermal *thermal; struct mlxsw_core_port *ports; unsigned int max_ports; + bool reload_fail; unsigned long driver_priv[0]; /* driver_priv has to be always the last item */ }; @@ -962,7 +963,28 @@ mlxsw_devlink_sb_occ_tc_port_bind_get(struct devlink_port *devlink_port, pool_type, p_cur, p_max); } +static int mlxsw_devlink_core_bus_device_reload(struct devlink *devlink) +{ + struct mlxsw_core *mlxsw_core = devlink_priv(devlink); + const struct mlxsw_bus *mlxsw_bus = mlxsw_core->bus; + int err; + + if (!mlxsw_bus->reset) + return -EOPNOTSUPP; + + mlxsw_core_bus_device_unregister(mlxsw_core, true); + mlxsw_bus->reset(mlxsw_core->bus_priv); + err = mlxsw_core_bus_device_register(mlxsw_core->bus_info, + mlxsw_core->bus, + mlxsw_core->bus_priv, true, + devlink); + if (err) + mlxsw_core->reload_fail = true; + return err; +} + static const struct devlink_ops mlxsw_devlink_ops = { + .reload = mlxsw_devlink_core_bus_device_reload, .port_type_set = mlxsw_devlink_port_type_set, .port_split = mlxsw_devlink_port_split, .port_unsplit = mlxsw_devlink_port_unsplit, @@ -980,23 +1002,26 @@ static const struct devlink_ops mlxsw_devlink_ops = { int mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, const struct mlxsw_bus *mlxsw_bus, - void *bus_priv) + void *bus_priv, bool reload, + struct devlink *devlink) { const char *device_kind = mlxsw_bus_info->device_kind; struct mlxsw_core *mlxsw_core; struct mlxsw_driver *mlxsw_driver; - struct devlink *devlink; size_t alloc_size; int err; mlxsw_driver = mlxsw_core_driver_get(device_kind); if (!mlxsw_driver) return -EINVAL; - alloc_size = sizeof(*mlxsw_core) + mlxsw_driver->priv_size; - devlink = devlink_alloc(&mlxsw_devlink_ops, alloc_size); - if (!devlink) { - err = -ENOMEM; - goto err_devlink_alloc; + + if (!reload) { + alloc_size = sizeof(*mlxsw_core) + mlxsw_driver->priv_size; + devlink = devlink_alloc(&mlxsw_devlink_ops, alloc_size); + if (!devlink) { + err = -ENOMEM; + goto err_devlink_alloc; + } } mlxsw_core = devlink_priv(devlink); @@ -1012,7 +1037,7 @@ int mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, if (err) goto err_bus_init; - if (mlxsw_driver->resources_register) { + if (mlxsw_driver->resources_register && !reload) { err = mlxsw_driver->resources_register(mlxsw_core); if (err) goto err_register_resources; @@ -1038,9 +1063,11 @@ int mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, if (err) goto err_emad_init; - err = devlink_register(devlink, mlxsw_bus_info->dev); - if (err) - goto err_devlink_register; + if (!reload) { + err = devlink_register(devlink, mlxsw_bus_info->dev); + if (err) + goto err_devlink_register; + } err = mlxsw_hwmon_init(mlxsw_core, mlxsw_bus_info, &mlxsw_core->hwmon); if (err) @@ -1082,20 +1109,29 @@ int mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, } EXPORT_SYMBOL(mlxsw_core_bus_device_register); -void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core) +void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core, + bool reload) { const char *device_kind = mlxsw_core->bus_info->device_kind; struct devlink *devlink = priv_to_devlink(mlxsw_core); + if (mlxsw_core->reload_fail) + goto out; + if (mlxsw_core->driver->fini) mlxsw_core->driver->fini(mlxsw_core); mlxsw_thermal_fini(mlxsw_core->thermal); - devlink_unregister(devlink); + if (!reload) + devlink_unregister(devlink); mlxsw_emad_fini(mlxsw_core); kfree(mlxsw_core->lag.mapping); mlxsw_ports_fini(mlxsw_core); - devlink_resources_unregister(devlink, NULL); + if (!reload) + devlink_resources_unregister(devlink, NULL); mlxsw_core->bus->fini(mlxsw_core->bus_priv); + if (reload) + return; +out: devlink_free(devlink); mlxsw_core_driver_put(device_kind); } diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.h b/drivers/net/ethernet/mellanox/mlxsw/core.h index e44061d..5ddafd7 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core.h +++ b/drivers/net/ethernet/mellanox/mlxsw/core.h @@ -66,8 +66,9 @@ void mlxsw_core_driver_unregister(struct mlxsw_driver *mlxsw_driver); int mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info, const struct mlxsw_bus *mlxsw_bus, - void *bus_priv); -void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core); + void *bus_priv, bool reload, + struct devlink *devlink); +void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core, bool reload); struct mlxsw_tx_info { u8 local_port; diff --git a/drivers/net/ethernet/mellanox/mlxsw/i2c.c b/drivers/net/ethernet/mellanox/mlxsw/i2c.c index c0dcfa0..25f9915 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/i2c.c +++ b/drivers/net/ethernet/mellanox/mlxsw/i2c.c @@ -539,7 +539,8 @@ static int mlxsw_i2c_probe(struct i2c_client *client, mlxsw_i2c->dev = &client->dev; err = mlxsw_core_bus_device_register(&mlxsw_i2c->bus_info, - &mlxsw_i2c_bus, mlxsw_i2c); + &mlxsw_i2c_bus, mlxsw_i2c, false, + NULL); if (err) { dev_err(&client->dev, "Fail to register core bus\n"); return err; @@ -557,7 +558,7 @@ static int mlxsw_i2c_remove(struct i2c_client *client) { struct mlxsw_i2c *mlxsw_i2c = i2c_get_clientdata(client); - mlxsw_core_bus_device_unregister(mlxsw_i2c->core); + mlxsw_core_bus_device_unregister(mlxsw_i2c->core, false); mutex_destroy(&mlxsw_i2c->cmd.lock); return 0; diff --git a/drivers/net/ethernet/mellanox/mlxsw/pci.c b/drivers/net/ethernet/mellanox/mlxsw/pci.c index da08f08..bb96620 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/pci.c +++ b/drivers/net/ethernet/mellanox/mlxsw/pci.c @@ -1737,7 +1737,8 @@ static int mlxsw_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) mlxsw_pci->id = id; err = mlxsw_core_bus_device_register(&mlxsw_pci->bus_info, - &mlxsw_pci_bus, mlxsw_pci); + &mlxsw_pci_bus, mlxsw_pci, false, + NULL); if (err) { dev_err(&pdev->dev, "cannot register bus device\n"); goto err_bus_device_register; @@ -1765,7 +1766,7 @@ static void mlxsw_pci_remove(struct pci_dev *pdev) { struct mlxsw_pci *mlxsw_pci = pci_get_drvdata(pdev); - mlxsw_core_bus_device_unregister(mlxsw_pci->core); + mlxsw_core_bus_device_unregister(mlxsw_pci->core, false); mlxsw_pci_free_irq_vectors(mlxsw_pci); iounmap(mlxsw_pci->hw_addr); pci_release_regions(mlxsw_pci->pdev);