From patchwork Thu Feb 13 09:37:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: fengzhimin X-Patchwork-Id: 1237370 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=huawei.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48JBKn5QhWz9sPK for ; Thu, 13 Feb 2020 20:39:08 +1100 (AEDT) Received: from localhost ([::1]:49474 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j2Axe-0006jn-AA for incoming@patchwork.ozlabs.org; Thu, 13 Feb 2020 04:39:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45953) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j2Ax1-0006hU-7x for qemu-devel@nongnu.org; Thu, 13 Feb 2020 04:38:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j2Ax0-00052w-9B for qemu-devel@nongnu.org; Thu, 13 Feb 2020 04:38:27 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:2772 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j2Awz-00050e-QQ for qemu-devel@nongnu.org; Thu, 13 Feb 2020 04:38:26 -0500 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id AB4DFBC2EA1D7D187D71; Thu, 13 Feb 2020 17:38:20 +0800 (CST) Received: from huawei.com (10.173.220.198) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.439.0; Thu, 13 Feb 2020 17:38:13 +0800 From: Zhimin Feng To: , , , Subject: [PATCH RFC 10/14] migration/rdma: Wait for all multifd to complete registration Date: Thu, 13 Feb 2020 17:37:51 +0800 Message-ID: <20200213093755.370-11-fengzhimin1@huawei.com> X-Mailer: git-send-email 2.24.0.windows.2 In-Reply-To: <20200213093755.370-1-fengzhimin1@huawei.com> References: <20200213093755.370-1-fengzhimin1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.173.220.198] X-CFilter-Loop: Reflected X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 45.249.212.191 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jemmy858585@gmail.com, Zhimin Feng , qemu-devel@nongnu.org, zhang.zhanghailiang@huawei.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Zhimin Feng --- migration/multifd.c | 6 ++++++ migration/rdma.c | 17 +++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/migration/multifd.c b/migration/multifd.c index 4ae25fc88f..c986d4c247 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -389,6 +389,7 @@ static void multifd_send_terminate_threads(Error *err) qemu_mutex_lock(&p->mutex); p->quit = true; if (migrate_use_rdma()) { + qemu_sem_post(&p->sem); qemu_sem_post(&p->sem_sync); } else { qemu_sem_post(&p->sem); @@ -502,6 +503,11 @@ static void *multifd_rdma_send_thread(void *opaque) if (qemu_rdma_registration(p->rdma) < 0) { goto out; } + /* + * Inform the main RDMA thread to run when multifd + * RDMA thread have completed registration. + */ + qemu_sem_post(&p->sem); while (true) { qemu_sem_wait(&p->sem_sync); diff --git a/migration/rdma.c b/migration/rdma.c index 5de3a29712..4c48e9832c 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -3733,6 +3733,23 @@ static int qemu_rdma_registration_stop(QEMUFile *f, void *opaque, rdma->dest_blocks[i].remote_host_addr; local->block[i].remote_rkey = rdma->dest_blocks[i].remote_rkey; } + + /* Wait for all multifd channels to complete registration */ + if (migrate_use_multifd()) { + int i; + int thread_count = migrate_multifd_channels(); + MultiFDSendParams *multifd_send_param = NULL; + for (i = 0; i < thread_count; i++) { + ret = get_multifd_send_param(i, &multifd_send_param); + if (ret) { + ERROR(errp, "rdma: error" + "getting multifd_send_param(%d)", i); + return ret; + } + + qemu_sem_wait(&multifd_send_param->sem); + } + } } trace_qemu_rdma_registration_stop(flags);