mbox series

[rdma-next,v1,00/12] DEVX asynchronous events

Message ID 20190618171540.11729-1-leon@kernel.org
Headers show
Series DEVX asynchronous events | expand

Message

Leon Romanovsky June 18, 2019, 5:15 p.m. UTC
From: Leon Romanovsky <leonro@mellanox.com>

Changelog:
 v0 -> v1:
 * Fix the unbind / hot unplug flows to work properly.
 * Fix Ref count handling on the eventfd mode in some flow.
 * Rebased to latest rdma-next

Thanks

------------------------------------------------------------------------------------
From Yishai:

This series enables RDMA applications that use the DEVX interface to
subscribe and read device asynchronous events.

The solution is designed to allow extension of events in the future
without need to perform any changes in the driver code.

To enable that few changes had been done in mlx5_core, it includes:
 * Reading device event capabilities that are user related
   (affiliated and un-affiliated) and set the matching mask upon
   creating the matching EQ.
 * Enable DEVX/mlx5_ib to register for ANY event instead of the option to
   get some hard-coded ones.
 * Enable DEVX/mlx5_ib to get the device raw data for CQ completion events.
 * Enhance mlx5_core_create/destroy CQ to enable DEVX using them so that CQ
   events will be reported as well.

In mlx5_ib layer the below changes were done:
 * A new DEVX API was introduced to allocate an event channel by using
   the uverbs FD object type.
 * Implement the FD channel operations to enable read/poo/close over it.
 * A new DEVX API was introduced to subscribe for specific events over an
   event channel.
 * Manage an internal data structure  over XA(s) to subscribe/dispatch events
   over the different event channels.
 * Use from DEVX the mlx5_core APIs to create/destroy a CQ to be able to
   get its relevant events.

Yishai

Yishai Hadas (12):
  net/mlx5: Fix mlx5_core_destroy_cq() error flow
  net/mlx5: Use event mask based on device capabilities
  net/mlx5: Expose the API to register for ANY event
  net/mlx5: mlx5_core_create_cq() enhancements
  net/mlx5: Report a CQ error event only when a handler was set
  net/mlx5: Report EQE data upon CQ completion
  net/mlx5: Expose device definitions for object events
  IB/mlx5: Introduce MLX5_IB_OBJECT_DEVX_ASYNC_EVENT_FD
  IB/mlx5: Register DEVX with mlx5_core to get async events
  IB/mlx5: Enable subscription for device events over DEVX
  IB/mlx5: Implement DEVX dispatching event
  IB/mlx5: Add DEVX support for CQ events

 drivers/infiniband/hw/mlx5/cq.c               |    5 +-
 drivers/infiniband/hw/mlx5/devx.c             | 1082 ++++++++++++++++-
 drivers/infiniband/hw/mlx5/main.c             |   10 +-
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |   12 +
 drivers/infiniband/hw/mlx5/odp.c              |    3 +-
 drivers/infiniband/hw/mlx5/qp.c               |    2 +-
 drivers/net/ethernet/mellanox/mlx5/core/cq.c  |   21 +-
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |    2 +-
 .../net/ethernet/mellanox/mlx5/core/en_main.c |    3 +-
 .../net/ethernet/mellanox/mlx5/core/en_txrx.c |    2 +-
 drivers/net/ethernet/mellanox/mlx5/core/eq.c  |   68 +-
 .../ethernet/mellanox/mlx5/core/fpga/conn.c   |    6 +-
 drivers/net/ethernet/mellanox/mlx5/core/fw.c  |    6 +
 .../net/ethernet/mellanox/mlx5/core/lib/eq.h  |    5 +-
 include/linux/mlx5/cq.h                       |    6 +-
 include/linux/mlx5/device.h                   |    6 +-
 include/linux/mlx5/driver.h                   |    2 +
 include/linux/mlx5/eq.h                       |    4 +-
 include/linux/mlx5/mlx5_ifc.h                 |   34 +-
 include/uapi/rdma/mlx5_user_ioctl_cmds.h      |   19 +
 include/uapi/rdma/mlx5_user_ioctl_verbs.h     |    9 +
 21 files changed, 1237 insertions(+), 70 deletions(-)

--
2.20.1

Comments

Saeed Mahameed June 18, 2019, 6:51 p.m. UTC | #1
On Tue, 2019-06-18 at 20:15 +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@mellanox.com>
> 
> Changelog:
>  v0 -> v1:

Normally 1st submission is V1 and 2nd is V2.
so this should have been v1->v2.

For mlx5-next patches:

Acked-by: Saeed Mahameed <saeedm@mellanox.com>


>  * Fix the unbind / hot unplug flows to work properly.
>  * Fix Ref count handling on the eventfd mode in some flow.
>  * Rebased to latest rdma-next
> 
> Thanks
> 
> -------------------------------------------------------------------
> -----------------
> From Yishai:
> 
> This series enables RDMA applications that use the DEVX interface to
> subscribe and read device asynchronous events.
> 
> The solution is designed to allow extension of events in the future
> without need to perform any changes in the driver code.
> 
> To enable that few changes had been done in mlx5_core, it includes:
>  * Reading device event capabilities that are user related
>    (affiliated and un-affiliated) and set the matching mask upon
>    creating the matching EQ.
>  * Enable DEVX/mlx5_ib to register for ANY event instead of the
> option to
>    get some hard-coded ones.
>  * Enable DEVX/mlx5_ib to get the device raw data for CQ completion
> events.
>  * Enhance mlx5_core_create/destroy CQ to enable DEVX using them so
> that CQ
>    events will be reported as well.
> 
> In mlx5_ib layer the below changes were done:
>  * A new DEVX API was introduced to allocate an event channel by
> using
>    the uverbs FD object type.
>  * Implement the FD channel operations to enable read/poo/close over
> it.
>  * A new DEVX API was introduced to subscribe for specific events
> over an
>    event channel.
>  * Manage an internal data structure  over XA(s) to
> subscribe/dispatch events
>    over the different event channels.
>  * Use from DEVX the mlx5_core APIs to create/destroy a CQ to be able
> to
>    get its relevant events.
> 
> Yishai
> 
> Yishai Hadas (12):
>   net/mlx5: Fix mlx5_core_destroy_cq() error flow
>   net/mlx5: Use event mask based on device capabilities
>   net/mlx5: Expose the API to register for ANY event
>   net/mlx5: mlx5_core_create_cq() enhancements
>   net/mlx5: Report a CQ error event only when a handler was set
>   net/mlx5: Report EQE data upon CQ completion
>   net/mlx5: Expose device definitions for object events
>   IB/mlx5: Introduce MLX5_IB_OBJECT_DEVX_ASYNC_EVENT_FD
>   IB/mlx5: Register DEVX with mlx5_core to get async events
>   IB/mlx5: Enable subscription for device events over DEVX
>   IB/mlx5: Implement DEVX dispatching event
>   IB/mlx5: Add DEVX support for CQ events
> 
>  drivers/infiniband/hw/mlx5/cq.c               |    5 +-
>  drivers/infiniband/hw/mlx5/devx.c             | 1082
> ++++++++++++++++-
>  drivers/infiniband/hw/mlx5/main.c             |   10 +-
>  drivers/infiniband/hw/mlx5/mlx5_ib.h          |   12 +
>  drivers/infiniband/hw/mlx5/odp.c              |    3 +-
>  drivers/infiniband/hw/mlx5/qp.c               |    2 +-
>  drivers/net/ethernet/mellanox/mlx5/core/cq.c  |   21 +-
>  drivers/net/ethernet/mellanox/mlx5/core/en.h  |    2 +-
>  .../net/ethernet/mellanox/mlx5/core/en_main.c |    3 +-
>  .../net/ethernet/mellanox/mlx5/core/en_txrx.c |    2 +-
>  drivers/net/ethernet/mellanox/mlx5/core/eq.c  |   68 +-
>  .../ethernet/mellanox/mlx5/core/fpga/conn.c   |    6 +-
>  drivers/net/ethernet/mellanox/mlx5/core/fw.c  |    6 +
>  .../net/ethernet/mellanox/mlx5/core/lib/eq.h  |    5 +-
>  include/linux/mlx5/cq.h                       |    6 +-
>  include/linux/mlx5/device.h                   |    6 +-
>  include/linux/mlx5/driver.h                   |    2 +
>  include/linux/mlx5/eq.h                       |    4 +-
>  include/linux/mlx5/mlx5_ifc.h                 |   34 +-
>  include/uapi/rdma/mlx5_user_ioctl_cmds.h      |   19 +
>  include/uapi/rdma/mlx5_user_ioctl_verbs.h     |    9 +
>  21 files changed, 1237 insertions(+), 70 deletions(-)
> 
> --
> 2.20.1
>
Leon Romanovsky June 19, 2019, 4:45 a.m. UTC | #2
On Tue, Jun 18, 2019 at 06:51:45PM +0000, Saeed Mahameed wrote:
> On Tue, 2019-06-18 at 20:15 +0300, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@mellanox.com>
> >
> > Changelog:
> >  v0 -> v1:
>
> Normally 1st submission is V1 and 2nd is V2.
> so this should have been v1->v2.

"Normally" depends on the language you are using. In C, everything
starts from 0, including version of patches :).

>
> For mlx5-next patches:
>
> Acked-by: Saeed Mahameed <saeedm@mellanox.com>

Thanks
Saeed Mahameed June 24, 2019, 9:57 p.m. UTC | #3
On Wed, 2019-06-19 at 07:45 +0300, Leon Romanovsky wrote:
> On Tue, Jun 18, 2019 at 06:51:45PM +0000, Saeed Mahameed wrote:
> > On Tue, 2019-06-18 at 20:15 +0300, Leon Romanovsky wrote:
> > > From: Leon Romanovsky <leonro@mellanox.com>
> > > 
> > > Changelog:
> > >  v0 -> v1:
> > 
> > Normally 1st submission is V1 and 2nd is V2.
> > so this should have been v1->v2.
> 
> "Normally" depends on the language you are using. In C, everything
> starts from 0, including version of patches :).
> 

You are wrong:
quoting: https://kernelnewbies.org/PatchTipsAndTricks

"For example, if you're sending the second revision of a patch, you
should use [PATCH v2]."

now don't tell me that second revision is actually 3rd revision or 1st
is 2nd :).. 

> > For mlx5-next patches:
> > 
> > Acked-by: Saeed Mahameed <saeedm@mellanox.com>
> 
> Thanks
Leon Romanovsky June 30, 2019, 8:53 a.m. UTC | #4
On Mon, Jun 24, 2019 at 09:57:05PM +0000, Saeed Mahameed wrote:
> On Wed, 2019-06-19 at 07:45 +0300, Leon Romanovsky wrote:
> > On Tue, Jun 18, 2019 at 06:51:45PM +0000, Saeed Mahameed wrote:
> > > On Tue, 2019-06-18 at 20:15 +0300, Leon Romanovsky wrote:
> > > > From: Leon Romanovsky <leonro@mellanox.com>
> > > >
> > > > Changelog:
> > > >  v0 -> v1:
> > >
> > > Normally 1st submission is V1 and 2nd is V2.
> > > so this should have been v1->v2.
> >
> > "Normally" depends on the language you are using. In C, everything
> > starts from 0, including version of patches :).
> >
>
> You are wrong:
> quoting: https://kernelnewbies.org/PatchTipsAndTricks
>
> "For example, if you're sending the second revision of a patch, you
> should use [PATCH v2]."
>
> now don't tell me that second revision is actually 3rd revision or 1st
> is 2nd :)..

:)

If you don't mind, I will stick to common sense (v0, v1, v2 ...)
and official kernel documentation, which mentions existence of v1.

https://elixir.bootlin.com/linux/latest/source/Documentation/process/submitting-patches.rst#L682

>
> > > For mlx5-next patches:
> > >
> > > Acked-by: Saeed Mahameed <saeedm@mellanox.com>
> >
> > Thanks