| Message ID | 1525618499-1560-1-git-send-email-lidongchen@tencent.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/2] migration: update index field when delete or qsort RDMALocalBlock | expand |
* Lidong Chen (jemmy858585@gmail.com) wrote: > rdma_delete_block function deletes RDMALocalBlock base on index field, > but not update the index field. So when next time invoke rdma_delete_block, > it will not work correctly. > > If start and cancel migration repeatedly, some RDMALocalBlock not invoke > ibv_dereg_mr to decrease kernel mm_struct vmpin. When vmpin is large than > max locked memory limitation, ibv_reg_mr will failed, and migration can not > start successfully again. > > Signed-off-by: Lidong Chen <lidongchen@tencent.com> > --- > migration/rdma.c | 7 +++++++ > 1 file changed, 7 insertions(+) > > diff --git a/migration/rdma.c b/migration/rdma.c > index ed9cfb1..0dd4033 100644 > --- a/migration/rdma.c > +++ b/migration/rdma.c > @@ -713,6 +713,9 @@ static int rdma_delete_block(RDMAContext *rdma, RDMALocalBlock *block) > memcpy(local->block + block->index, old + (block->index + 1), > sizeof(RDMALocalBlock) * > (local->nb_blocks - (block->index + 1))); > + for (x = block->index; x < local->nb_blocks - 1; x++) { > + local->block[x].index--; > + } Yes; is that equivalent to local->blocks[x].index = x; ? > } > } else { > assert(block == local->block); > @@ -3398,6 +3401,10 @@ static int qemu_rdma_registration_handle(QEMUFile *f, void *opaque) > qsort(rdma->local_ram_blocks.block, > rdma->local_ram_blocks.nb_blocks, > sizeof(RDMALocalBlock), dest_ram_sort_func); > + for (i = 0; i < local->nb_blocks; i++) { > + local->block[i].index = i; > + } > + Which is basically the way that one does it; OK, it's a while since I looked at this but I think it fixes my 3 year old 03fcab38617 patch, so Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > if (rdma->pin_all) { > ret = qemu_rdma_reg_whole_ram_blocks(rdma); > if (ret) { > -- > 1.8.3.1 > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
On Wed, May 9, 2018 at 1:19 AM, Dr. David Alan Gilbert <dgilbert@redhat.com> wrote: > * Lidong Chen (jemmy858585@gmail.com) wrote: >> rdma_delete_block function deletes RDMALocalBlock base on index field, >> but not update the index field. So when next time invoke rdma_delete_block, >> it will not work correctly. >> >> If start and cancel migration repeatedly, some RDMALocalBlock not invoke >> ibv_dereg_mr to decrease kernel mm_struct vmpin. When vmpin is large than >> max locked memory limitation, ibv_reg_mr will failed, and migration can not >> start successfully again. >> >> Signed-off-by: Lidong Chen <lidongchen@tencent.com> >> --- >> migration/rdma.c | 7 +++++++ >> 1 file changed, 7 insertions(+) >> >> diff --git a/migration/rdma.c b/migration/rdma.c >> index ed9cfb1..0dd4033 100644 >> --- a/migration/rdma.c >> +++ b/migration/rdma.c >> @@ -713,6 +713,9 @@ static int rdma_delete_block(RDMAContext *rdma, RDMALocalBlock *block) >> memcpy(local->block + block->index, old + (block->index + 1), >> sizeof(RDMALocalBlock) * >> (local->nb_blocks - (block->index + 1))); >> + for (x = block->index; x < local->nb_blocks - 1; x++) { >> + local->block[x].index--; >> + } > > Yes; is that equivalent to local->blocks[x].index = x; ? yes, it's equivalent. > >> } >> } else { >> assert(block == local->block); >> @@ -3398,6 +3401,10 @@ static int qemu_rdma_registration_handle(QEMUFile *f, void *opaque) >> qsort(rdma->local_ram_blocks.block, >> rdma->local_ram_blocks.nb_blocks, >> sizeof(RDMALocalBlock), dest_ram_sort_func); >> + for (i = 0; i < local->nb_blocks; i++) { >> + local->block[i].index = i; >> + } >> + > > Which is basically the way that one does it; > > OK, it's a while since I looked at this but I think it fixes my 3 year > old 03fcab38617 patch, so > > > > Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > >> if (rdma->pin_all) { >> ret = qemu_rdma_reg_whole_ram_blocks(rdma); >> if (ret) { >> -- >> 1.8.3.1 >> > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
diff --git a/migration/rdma.c b/migration/rdma.c index ed9cfb1..0dd4033 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -713,6 +713,9 @@ static int rdma_delete_block(RDMAContext *rdma, RDMALocalBlock *block) memcpy(local->block + block->index, old + (block->index + 1), sizeof(RDMALocalBlock) * (local->nb_blocks - (block->index + 1))); + for (x = block->index; x < local->nb_blocks - 1; x++) { + local->block[x].index--; + } } } else { assert(block == local->block); @@ -3398,6 +3401,10 @@ static int qemu_rdma_registration_handle(QEMUFile *f, void *opaque) qsort(rdma->local_ram_blocks.block, rdma->local_ram_blocks.nb_blocks, sizeof(RDMALocalBlock), dest_ram_sort_func); + for (i = 0; i < local->nb_blocks; i++) { + local->block[i].index = i; + } + if (rdma->pin_all) { ret = qemu_rdma_reg_whole_ram_blocks(rdma); if (ret) {
rdma_delete_block function deletes RDMALocalBlock base on index field, but not update the index field. So when next time invoke rdma_delete_block, it will not work correctly. If start and cancel migration repeatedly, some RDMALocalBlock not invoke ibv_dereg_mr to decrease kernel mm_struct vmpin. When vmpin is large than max locked memory limitation, ibv_reg_mr will failed, and migration can not start successfully again. Signed-off-by: Lidong Chen <lidongchen@tencent.com> --- migration/rdma.c | 7 +++++++ 1 file changed, 7 insertions(+)