From patchwork Tue Sep 19 16:48:00 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Perevalov X-Patchwork-Id: 815700 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xxTyJ2DMtz9sMN for ; Wed, 20 Sep 2017 03:14:48 +1000 (AEST) Received: from localhost ([::1]:44211 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1duM6g-0007I2-8w for incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 13:14:46 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42497) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1duLhG-0001Ds-FC for qemu-devel@nongnu.org; Tue, 19 Sep 2017 12:48:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1duLhD-0006QB-L8 for qemu-devel@nongnu.org; Tue, 19 Sep 2017 12:48:30 -0400 Received: from mailout2.w1.samsung.com ([210.118.77.12]:46476) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1duLhD-0006PF-BT for qemu-devel@nongnu.org; Tue, 19 Sep 2017 12:48:27 -0400 Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20170919164825euoutp0227ebde261a43cdc8881214fbb3da047e~l0Tv1Q-p82806628066euoutp022; Tue, 19 Sep 2017 16:48:25 +0000 (GMT) Received: from eusmges5.samsung.com (unknown [203.254.199.245]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20170919164825eucas1p17b36f8a15970098cab98a7d030e63f9f~l0TvFfD1I2430324303eucas1p1A; Tue, 19 Sep 2017 16:48:25 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges5.samsung.com (EUCPMTA) with SMTP id FE.9B.12743.85A41C95; Tue, 19 Sep 2017 17:48:24 +0100 (BST) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20170919164824eucas1p23607e53e3ea38f8be4e885bb960e803f~l0TuXb3Pc2115421154eucas1p2l; Tue, 19 Sep 2017 16:48:24 +0000 (GMT) X-AuditID: cbfec7f5-f79d06d0000031c7-bf-59c14a582a73 Received: from eusync1.samsung.com ( [203.254.199.211]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id A5.6C.18832.85A41C95; Tue, 19 Sep 2017 17:48:24 +0100 (BST) Received: from aperevalov-ubuntu.rnd.samsung.ru ([106.109.129.199]) by eusync1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0OWJ003OHDC8XU40@eusync1.samsung.com>; Tue, 19 Sep 2017 17:48:24 +0100 (BST) From: Alexey Perevalov To: qemu-devel@nongnu.org Date: Tue, 19 Sep 2017 19:48:00 +0300 Message-id: <1505839684-10046-7-git-send-email-a.perevalov@samsung.com> X-Mailer: git-send-email 1.9.1 In-reply-to: <1505839684-10046-1-git-send-email-a.perevalov@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrOIsWRmVeSWpSXmKPExsWy7djP87oRXgcjDZ5PNrSYe/c8i0Xvtnvs FhPfrme1mPb5NrvFlfaf7BZb9n9jtzjeu4PF4s6WPiYHDo8n1zYzebzfd5XNo2/LKsYA5igu m5TUnMyy1CJ9uwSujDXX+1kLmjUqvs2Ra2BsVuhi5OSQEDCROPnrGyuELSZx4d56ti5GLg4h gaWMEpun/WCEcD4zSuzo384G03F21wpmiMQyRom9018xgiSEBLqZJFa9jeli5OBgEzCQ2HfP FiQsIiAp8bvrNFg9s8AWRok7916zgtQIC8RJ9OxlB6lhEVCVeLL5A9gVvALuEnvf72SE2CUn cfLYZLA4p4CHxK3ja9kh4j1sEms22IKMkRBwkWh/YggRFpZ4dXwLVImMRGfHQSaQtRIC7YwS 3Ts7WSGcCYwSZ6b/haqylzh18yoTiM0swCcxadt0ZoihvBIdbUIQJR4S1zo/QN3jKLH+zAdo oMxmlOh7tZd9AqP0AkaGVYwiqaXFuempxaZ6xYm5xaV56XrJ+bmbGIHRefrf8a87GJceszrE KMDBqMTDu8LmYKQQa2JZcWXuIUYJDmYlEd7XTkAh3pTEyqrUovz4otKc1OJDjNIcLErivLZR bZFCAumJJanZqakFqUUwWSYOTqkGRsk/3T0/FIsYdRvLdtpvKTX8NmXR/bcnFfzFg2Wm8mjf 4S/oldXadt7//5rpDmGJrzduPHHp7NrwNXttdRoPZi8V2Jr6zO3Q26eNjLMaV7xmOHsrrCBq q/0a7oSaTfN0z5irMCz2+17HNmnx7SNPS85sjJme5DrjPYt1kN+hvYIf7vH7/5miZKnEUpyR aKjFXFScCAC4M6lWygIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFuplluLIzCtJLcpLzFFi42I5/e/4Zd0Ir4ORBn8nCFjMvXuexaJ32z12 i4lv17NaTPt8m93iSvtPdost+7+xWxzv3cFicWdLH5MDh8eTa5uZPN7vu8rm0bdlFWMAcxSX TUpqTmZZapG+XQJXxprr/awFzRoV3+bINTA2K3QxcnJICJhInN21ghnCFpO4cG89WxcjF4eQ wBJGifkPnzNCOL1MEo+OrmfvYuTgYBMwkNh3zxakQURAUuJ312lmkBpmgS2MEhuPr2QEqREW iJPo2csOUsMioCrxZPMHVhCbV8BdYu/7nYwQy+QkTh6bDBbnFPCQuHV8LVi9EFDNjbXXmSYw 8i5gZFjFKJJaWpybnltsqFecmFtcmpeul5yfu4kRGEjbjv3cvIPx0sbgQ4wCHIxKPLwrbA5G CrEmlhVX5h5ilOBgVhLhfe0EFOJNSaysSi3Kjy8qzUktPsQozcGiJM7bu2d1pJBAemJJanZq akFqEUyWiYNTqoFRWdvBIDj5maKNB8O9D9PUqy0iMk7eeW3Y/z79Yv0mq4gtv7w6zxpNeMe2 VeOwQWrajFfeCyucXV+5HA7c+zvfZRNLodXt75JSK6RWnRfd/32jgNt8SS7ne26/VnvsO/ds 7mtRt69mwsIcapcnhzI9V30cdUrJ9bTmFH8P76Uv9bz+KK08dDRZiaU4I9FQi7moOBEAnSV6 WiACAAA= X-CMS-MailID: 20170919164824eucas1p23607e53e3ea38f8be4e885bb960e803f X-Msg-Generator: CA X-Sender-IP: 182.198.249.179 X-Local-Sender: =?utf-8?q?Alexey_Perevalov=1BSRR-Virtualization_Lab=1B?= =?utf-8?b?7IK87ISx7KCE7J6QG1NlbmlvciBFbmdpbmVlcg==?= X-Global-Sender: =?utf-8?q?Alexey_Perevalov=1BSRR-Virtualization_Lab=1BSa?= =?utf-8?q?msung_Electronics=1BSenior_Engineer?= X-Sender-Code: =?utf-8?q?C10=1BCISHQ=1BC10GD01GD010154?= CMS-TYPE: 201P X-CMS-RootMailID: 20170919164824eucas1p23607e53e3ea38f8be4e885bb960e803f X-RootMTR: 20170919164824eucas1p23607e53e3ea38f8be4e885bb960e803f References: <1505839684-10046-1-git-send-email-a.perevalov@samsung.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 210.118.77.12 Subject: [Qemu-devel] [PATCH v10 06/10] migration: add postcopy blocktime ctx into MigrationIncomingState X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: heetae82.ahn@samsung.com, quintela@redhat.com, Alexey Perevalov , peterx@redhat.com, dgilbert@redhat.com, i.maximets@samsung.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch adds request to kernel space for UFFD_FEATURE_THREAD_ID, in case when this feature is provided by kernel. PostcopyBlocktimeContext is incapsulated inside postcopy-ram.c, due to it's postcopy only feature. Also it defines PostcopyBlocktimeContext's instance live time. Information from PostcopyBlocktimeContext instance will be provided much after postcopy migration end, instance of PostcopyBlocktimeContext will live till QEMU exit, but part of it (vcpu_addr, page_fault_vcpu_time) used only during calculation, will be released when postcopy ended or failed. To enable postcopy blocktime calculation on destination, need to request proper capabiltiy (Patch for documentation will be at the tail of the patch set). As an example following command enable that capability, assume QEMU was started with -chardev socket,id=charmonitor,path=/var/lib/migrate-vm-monitor.sock option to control it [root@host]#printf "{\"execute\" : \"qmp_capabilities\"}\r\n \ {\"execute\": \"migrate-set-capabilities\" , \"arguments\": { \"capabilities\": [ { \"capability\": \"postcopy-blocktime\", \"state\": true } ] } }" | nc -U /var/lib/migrate-vm-monitor.sock Or just with HMP (qemu) migrate_set_capability postcopy-blocktime on Signed-off-by: Alexey Perevalov --- migration/migration.h | 8 ++++++ migration/postcopy-ram.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 73 insertions(+) diff --git a/migration/migration.h b/migration/migration.h index 56bf33c..770466b 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -21,6 +21,8 @@ #include "qemu/coroutine_int.h" #include "hw/qdev.h" +struct PostcopyBlocktimeContext; + /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; @@ -58,6 +60,12 @@ struct MigrationIncomingState { /* The coroutine we should enter (back) after failover */ Coroutine *migration_incoming_co; QemuSemaphore colo_incoming_sem; + + /* + * PostcopyBlocktimeContext to keep information for postcopy + * live migration, to calculate vCPU block time + * */ + struct PostcopyBlocktimeContext *blocktime_ctx; }; MigrationIncomingState *migration_incoming_get_current(void); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index bec6c2c..cc78981 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -61,6 +61,58 @@ struct PostcopyDiscardState { #include #include +typedef struct PostcopyBlocktimeContext { + /* time when page fault initiated per vCPU */ + int64_t *page_fault_vcpu_time; + /* page address per vCPU */ + uint64_t *vcpu_addr; + int64_t total_blocktime; + /* blocktime per vCPU */ + int64_t *vcpu_blocktime; + /* point in time when last page fault was initiated */ + int64_t last_begin; + /* number of vCPU are suspended */ + int smp_cpus_down; + + /* + * Handler for exit event, necessary for + * releasing whole blocktime_ctx + */ + Notifier exit_notifier; + /* + * Handler for postcopy event, necessary for + * releasing unnecessary part of blocktime_ctx + */ + Notifier postcopy_notifier; +} PostcopyBlocktimeContext; + +static void destroy_blocktime_context(struct PostcopyBlocktimeContext *ctx) +{ + g_free(ctx->page_fault_vcpu_time); + g_free(ctx->vcpu_addr); + g_free(ctx->vcpu_blocktime); + g_free(ctx); +} + +static void migration_exit_cb(Notifier *n, void *data) +{ + PostcopyBlocktimeContext *ctx = container_of(n, PostcopyBlocktimeContext, + exit_notifier); + destroy_blocktime_context(ctx); +} + +static struct PostcopyBlocktimeContext *blocktime_context_new(void) +{ + PostcopyBlocktimeContext *ctx = g_new0(PostcopyBlocktimeContext, 1); + ctx->page_fault_vcpu_time = g_new0(int64_t, smp_cpus); + ctx->vcpu_addr = g_new0(uint64_t, smp_cpus); + ctx->vcpu_blocktime = g_new0(int64_t, smp_cpus); + + ctx->exit_notifier.notify = migration_exit_cb; + qemu_add_exit_notifier(&ctx->exit_notifier); + add_migration_state_change_notifier(&ctx->postcopy_notifier); + return ctx; +} /** * receive_ufd_features: check userfault fd features, to request only supported @@ -153,6 +205,19 @@ static bool ufd_check_and_apply(int ufd, MigrationIncomingState *mis) } } +#ifdef UFFD_FEATURE_THREAD_ID + if (migrate_postcopy_blocktime() && mis && + UFFD_FEATURE_THREAD_ID & supported_features) { + /* kernel supports that feature */ + /* don't create blocktime_context if it exists */ + if (!mis->blocktime_ctx) { + mis->blocktime_ctx = blocktime_context_new(); + } + + asked_features |= UFFD_FEATURE_THREAD_ID; + } +#endif + /* * request features, even if asked_features is 0, due to * kernel expects UFFD_API before UFFDIO_REGISTER, per