| Message ID | 1520240651-11308-1-git-send-email-Michal.Kalderon@cavium.com |
|---|---|
| State | Superseded, archived |
| Delegated to: | David Miller |
| Headers | show |
| Series | [net] qed: Free RoCE ILT Memory on rmmod qedr | expand |
On Mon, Mar 05, 2018 at 11:04:11AM +0200, Michal Kalderon wrote: > Rdma requires ILT Memory to be allocated for it's QPs. > Each ILT entry points to a page used by several Rdma QPs. > To avoid allocating all the memory in advance, the rdma > implementation dynamically allocates memory as more QPs are > added, however it does not dynamically free the memory. > The memory should have been freed on rmmod qedr, but isn't. > This patch adds the memory freeing on rmmod qedr (currently > it will be freed with qed is removed). > > An outcome of this bug, is that if qedr is unloaded and loaded > without unloaded qed, there will be no more RoCE traffic. > > The reason these are related, is that the logic of detecting the > first QP ever opened is by asking whether ILT memory for RoCE has > been allocated. > > In addition, this patch modifies freeing of the Task context to > always use the PROTOCOLID_ROCE and not the protocol passed, > this is because task context for iWARP and ROCE both use the > ROCE protocol id, as opposed to the connection context. > > Fixes: dbb799c39717e7b7 Your fixes line was truncated. Thanks
> From: Leon Romanovsky [mailto:leon@kernel.org] > Sent: Monday, March 05, 2018 11:41 AM > To: Kalderon, Michal <Michal.Kalderon@cavium.com> > Cc: davem@davemloft.net; netdev@vger.kernel.org; dledford@redhat.com; > jgg@mellanox.com; linux-rdma@vger.kernel.org; Elior, Ariel > <Ariel.Elior@cavium.com> > Subject: Re: [PATCH net] qed: Free RoCE ILT Memory on rmmod qedr > > On Mon, Mar 05, 2018 at 11:04:11AM +0200, Michal Kalderon wrote: > > Rdma requires ILT Memory to be allocated for it's QPs. > > Each ILT entry points to a page used by several Rdma QPs. > > To avoid allocating all the memory in advance, the rdma > > implementation dynamically allocates memory as more QPs are > > added, however it does not dynamically free the memory. > > The memory should have been freed on rmmod qedr, but isn't. > > This patch adds the memory freeing on rmmod qedr (currently > > it will be freed with qed is removed). > > > > An outcome of this bug, is that if qedr is unloaded and loaded > > without unloaded qed, there will be no more RoCE traffic. > > > > The reason these are related, is that the logic of detecting the > > first QP ever opened is by asking whether ILT memory for RoCE has > > been allocated. > > > > In addition, this patch modifies freeing of the Task context to > > always use the PROTOCOLID_ROCE and not the protocol passed, > > this is because task context for iWARP and ROCE both use the > > ROCE protocol id, as opposed to the connection context. > > > > Fixes: dbb799c39717e7b7 > > Your fixes line was truncated. > > Thanks Thanks for noticing, sending V2
diff --git a/drivers/net/ethernet/qlogic/qed/qed_cxt.c b/drivers/net/ethernet/qlogic/qed/qed_cxt.c index 6f546e8..b6f55bc 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_cxt.c +++ b/drivers/net/ethernet/qlogic/qed/qed_cxt.c @@ -2480,7 +2480,10 @@ int qed_cxt_free_proto_ilt(struct qed_hwfn *p_hwfn, enum protocol_type proto) if (rc) return rc; - /* Free Task CXT */ + /* Free Task CXT ( Intentionally RoCE as task-id is shared between + * RoCE and iWARP + */ + proto = PROTOCOLID_ROCE; rc = qed_cxt_free_ilt_range(p_hwfn, QED_ELEM_TASK, 0, qed_cxt_get_proto_tid_count(p_hwfn, proto)); if (rc) diff --git a/drivers/net/ethernet/qlogic/qed/qed_rdma.c b/drivers/net/ethernet/qlogic/qed/qed_rdma.c index 5d040b8..f3ee653 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_rdma.c +++ b/drivers/net/ethernet/qlogic/qed/qed_rdma.c @@ -380,6 +380,7 @@ static void qed_rdma_free(struct qed_hwfn *p_hwfn) qed_rdma_free_reserved_lkey(p_hwfn); qed_rdma_resc_free(p_hwfn); + qed_cxt_free_proto_ilt(p_hwfn, p_hwfn->p_rdma_info->proto); } static void qed_rdma_get_guid(struct qed_hwfn *p_hwfn, u8 *guid)