| Message ID | 20200221175415.390884-1-idosch@idosch.org |
|---|---|
| Headers | show |
| Series | mlxsw: Remove RTNL from route insertion path | expand |
From: Ido Schimmel <idosch@idosch.org> Date: Fri, 21 Feb 2020 19:54:03 +0200 > From: Ido Schimmel <idosch@mellanox.com> > > This patch set removes RTNL from the route insertion path in mlxsw in > order to reduce the control plane latency: the time it takes to push > routes from user space to the kernel and mlxsw. > > Up until now mlxsw did not have a lock to protect its shared router data > structures and instead relied on RTNL. While this was simple and worked, > it resulted in large control plane latencies as RTNL was heavily > contended - by both the task receiving the netlink messages from user > space and the mlxsw workqueue that programs the routes to the device. > > By removing RTNL and introducing a new router mutex, this patch set > reduces the control plane latency by ~80%. A single mutex is added as > inside mlxsw there is not a lot of concurrency. In addition, a more > fine-grained locking scheme is much more error-prone. > > Patches #1-#6 are preparations. They add needed locking in NVE and > multicast routing code instead of relying on RTNL > Patch #7 introduces the new mutex > Patches #8-#12 gradually take the lock in various entry points into the > routing code > Patch #13 removes RTNL in places where it is no longer required Series applied, thanks.
From: Ido Schimmel <idosch@mellanox.com> This patch set removes RTNL from the route insertion path in mlxsw in order to reduce the control plane latency: the time it takes to push routes from user space to the kernel and mlxsw. Up until now mlxsw did not have a lock to protect its shared router data structures and instead relied on RTNL. While this was simple and worked, it resulted in large control plane latencies as RTNL was heavily contended - by both the task receiving the netlink messages from user space and the mlxsw workqueue that programs the routes to the device. By removing RTNL and introducing a new router mutex, this patch set reduces the control plane latency by ~80%. A single mutex is added as inside mlxsw there is not a lot of concurrency. In addition, a more fine-grained locking scheme is much more error-prone. Patches #1-#6 are preparations. They add needed locking in NVE and multicast routing code instead of relying on RTNL Patch #7 introduces the new mutex Patches #8-#12 gradually take the lock in various entry points into the routing code Patch #13 removes RTNL in places where it is no longer required Ido Schimmel (12): mlxsw: spectrum_mr: Publish multicast route after writing it to the device mlxsw: spectrum_mr: Protect multicast table list with a lock mlxsw: spectrum_mr: Protect multicast route list with a lock mlxsw: spectrum_router: Expose router struct to internal users mlxsw: spectrum_router: Store NVE decapsulation configuration in router mlxsw: spectrum_router: Introduce router lock mlxsw: spectrum_router: Take router lock from inside routing code mlxsw: spectrum_dpipe: Take router lock from dpipe code mlxsw: spectrum_router: Take router lock from netdev listener mlxsw: spectrum_router: Take router lock from inetaddr listeners mlxsw: spectrum_router: Take router lock from exported helpers mlxsw: spectrum: Remove RTNL where possible .../net/ethernet/mellanox/mlxsw/spectrum.h | 5 +- .../ethernet/mellanox/mlxsw/spectrum_dpipe.c | 35 +-- .../net/ethernet/mellanox/mlxsw/spectrum_mr.c | 52 +++- .../ethernet/mellanox/mlxsw/spectrum_nve.c | 21 -- .../ethernet/mellanox/mlxsw/spectrum_router.c | 267 +++++++++++------- .../ethernet/mellanox/mlxsw/spectrum_router.h | 43 +++ 6 files changed, 267 insertions(+), 156 deletions(-)