[v5,03/10] migration: avoid concurrent invoke channel_close by different threads
diff mbox series

Message ID 1528212489-19137-4-git-send-email-lidongchen@tencent.com
State New
Headers show
Series
  • Enable postcopy RDMA live migration
Related show

Commit Message

858585 jemmy June 5, 2018, 3:28 p.m. UTC
The channel_close maybe invoked by different threads. For example, source
qemu invokes qemu_fclose in main thread, migration thread and return path
thread. Destination qemu invokes qemu_fclose in main thread, listen thread
and COLO incoming thread.

Signed-off-by: Lidong Chen <lidongchen@tencent.com>
---
 migration/migration.c | 2 ++
 migration/migration.h | 7 +++++++
 migration/qemu-file.c | 6 ++++--
 3 files changed, 13 insertions(+), 2 deletions(-)

Comments

Daniel P. Berrangé June 13, 2018, 5:02 p.m. UTC | #1
On Tue, Jun 05, 2018 at 11:28:02PM +0800, Lidong Chen wrote:
> The channel_close maybe invoked by different threads. For example, source
> qemu invokes qemu_fclose in main thread, migration thread and return path
> thread. Destination qemu invokes qemu_fclose in main thread, listen thread
> and COLO incoming thread.

The only alternative approach would be to try to get all close calls
happening in the same thread, by having the other threads just set
some flag the primary thread watches.  This likely quite alot of
work to achieve though, so I agree just doing mutex locking is a
prudent solution.

> 
> Signed-off-by: Lidong Chen <lidongchen@tencent.com>
> ---
>  migration/migration.c | 2 ++
>  migration/migration.h | 7 +++++++
>  migration/qemu-file.c | 6 ++++--
>  3 files changed, 13 insertions(+), 2 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


Regards,
Daniel

Patch
diff mbox series

diff --git a/migration/migration.c b/migration/migration.c
index 1e99ec9..1d0aaec 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -3075,6 +3075,7 @@  static void migration_instance_finalize(Object *obj)
 
     qemu_mutex_destroy(&ms->error_mutex);
     qemu_mutex_destroy(&ms->qemu_file_lock);
+    qemu_mutex_destroy(&ms->qemu_file_close_lock);
     g_free(params->tls_hostname);
     g_free(params->tls_creds);
     qemu_sem_destroy(&ms->pause_sem);
@@ -3115,6 +3116,7 @@  static void migration_instance_init(Object *obj)
     qemu_sem_init(&ms->postcopy_pause_rp_sem, 0);
     qemu_sem_init(&ms->rp_state.rp_sem, 0);
     qemu_mutex_init(&ms->qemu_file_lock);
+    qemu_mutex_init(&ms->qemu_file_close_lock);
 }
 
 /*
diff --git a/migration/migration.h b/migration/migration.h
index 5af57d6..7a6025a 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -121,6 +121,13 @@  struct MigrationState
      */
     QemuMutex qemu_file_lock;
 
+    /*
+     * The to_src_file and from_dst_file point to one QIOChannelRDMA,
+     * And qemu_fclose maybe invoked by different threads. use this lock
+     * to avoid concurrent invoke channel_close by different threads.
+     */
+    QemuMutex qemu_file_close_lock;
+
     /* bytes already send at the beggining of current interation */
     uint64_t iteration_initial_bytes;
     /* time at the start of current iteration */
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 977b9ae..74c48e0 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -323,12 +323,14 @@  void qemu_update_position(QEMUFile *f, size_t size)
  */
 int qemu_fclose(QEMUFile *f)
 {
-    int ret;
+    int ret, ret2;
     qemu_fflush(f);
     ret = qemu_file_get_error(f);
 
     if (f->ops->close) {
-        int ret2 = f->ops->close(f->opaque);
+        qemu_mutex_lock(&migrate_get_current()->qemu_file_close_lock);
+        ret2 = f->ops->close(f->opaque);
+        qemu_mutex_unlock(&migrate_get_current()->qemu_file_close_lock);
         if (ret >= 0) {
             ret = ret2;
         }