From patchwork Thu Oct 5 11:16:06 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Perevalov X-Patchwork-Id: 821721 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3y79Hj5hzBz9t2h for ; Thu, 5 Oct 2017 22:18:25 +1100 (AEDT) Received: from localhost ([::1]:39115 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e04AZ-0004vb-ON for incoming@patchwork.ozlabs.org; Thu, 05 Oct 2017 07:18:23 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36294) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e048j-0003wZ-9d for qemu-devel@nongnu.org; Thu, 05 Oct 2017 07:16:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e048g-0000Gp-HM for qemu-devel@nongnu.org; Thu, 05 Oct 2017 07:16:29 -0400 Received: from mailout1.w1.samsung.com ([210.118.77.11]:42759) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e048g-0000GA-7u for qemu-devel@nongnu.org; Thu, 05 Oct 2017 07:16:26 -0400 Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20171005111624euoutp0129333d31a4b0c2443f21c845f260a5ea~qqGbQMHpx1673016730euoutp01K; Thu, 5 Oct 2017 11:16:24 +0000 (GMT) Received: from eusmges4.samsung.com (unknown [203.254.199.244]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20171005111624eucas1p183c1c469ba243b595413d0e3883718c4~qqGao-2if1494214942eucas1p1A; Thu, 5 Oct 2017 11:16:24 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges4.samsung.com (EUCPMTA) with SMTP id 96.19.12944.78416D95; Thu, 5 Oct 2017 12:16:23 +0100 (BST) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20171005111623eucas1p272597c60842087cac3ade92b88212eff~qqGZ_OvGs2880128801eucas1p2a; Thu, 5 Oct 2017 11:16:23 +0000 (GMT) X-AuditID: cbfec7f4-f79ab6d000003290-b3-59d61487e12d Received: from eusync4.samsung.com ( [203.254.199.214]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 8E.A0.18832.78416D95; Thu, 5 Oct 2017 12:16:23 +0100 (BST) Received: from aperevalov-ubuntu.rnd.samsung.ru ([106.109.129.199]) by eusync4.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0OXC00M4YKN1XNA0@eusync4.samsung.com>; Thu, 05 Oct 2017 12:16:23 +0100 (BST) From: Alexey Perevalov To: qemu-devel@nongnu.org Date: Thu, 05 Oct 2017 14:16:06 +0300 Message-id: <1507202170-22619-3-git-send-email-a.perevalov@samsung.com> X-Mailer: git-send-email 2.7.4 In-reply-to: <1507202170-22619-1-git-send-email-a.perevalov@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrKIsWRmVeSWpSXmKPExsWy7djP87rtItciDaYesrCYe/c8i0Xvtnvs FtM+32a3uNL+k91iy/5v7BbHe3ewWNzZ0sfkwO7x5NpmJo/3+66yefRtWcUYwBzFZZOSmpNZ llqkb5fAlXFx6nWWgtlqFUdfLmFuYDwn18XIySEhYCKx+uAHdghbTOLCvfVsXYxcHEICSxkl tl5vgnI+M0rMnvCdCaZj64GzrCC2kMAyRokf7c4QRd1MEku37wEq4uBgEzCQ2HfPFqRGREBS 4nfXaWaQGmaBhYwSDy72gK0TFoiRaHjxihHEZhFQlbh8qw/M5hVwl/gyqYcRYpmcxM1zncwg MzkFPCQ2HZAEmSMh0MgmsWnjNzaIGheJE9+7oeqFJV4d3wL1joxEZ8dBJoiGdkaJ7p2drBDO BEaJM9P/QlXZS5y6eRXsNWYBPolJ26aDbZMQ4JXoaBOCKPGQ+H0b5iBHicbjzYwQH89mlFi9 sYdlAqP0AkaGVYwiqaXFuempxSZ6xYm5xaV56XrJ+bmbGIGRefrf8S87GBcfszrEKMDBqMTD y/DgSqQQa2JZcWXuIUYJDmYlEV6l/1cjhXhTEiurUovy44tKc1KLDzFKc7AoifPaRrVFCgmk J5akZqemFqQWwWSZODilGhitV985P/uohEL+n8NTp9hJqyovr8tOYtjfvLjnG/9xPf8Lu15E Jm9RqnrgfabIskF94q+DQVv+Lrbjidrpt8y1WLph9tHjS4KlHzu926S09snqhetDgiYqZNbJ 3VqVa/vSMmz5rNcP/V2E+qw3ONy8UxTzevOLMzNu39wvfcDQ584bqT0N+ZJPlFiKMxINtZiL ihMBaAUQoMgCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupkluLIzCtJLcpLzFFi42I5/e/4Nd12kWuRBhtnSVvMvXuexaJ32z12 i2mfb7NbXGn/yW6xZf83dovjvTtYLO5s6WNyYPd4cm0zk8f7fVfZPPq2rGIMYI7isklJzcks Sy3St0vgyrg49TpLwWy1iqMvlzA3MJ6T62Lk5JAQMJHYeuAsK4QtJnHh3nq2LkYuDiGBJYwS dzZcYoVwepkk1jyez9zFyMHBJmAgse+eLUiDiICkxO+u08wgNcwCCxkl5kx9wgiSEBaIkWh4 8QrMZhFQlbh8qw/M5hVwl/gyqYcRYpucxM1znWAzOQU8JDYdkAQJCwGVnOw6yzaBkXcBI8Mq RpHU0uLc9NxiQ73ixNzi0rx0veT83E2MwADaduzn5h2MlzYGH2IU4GBU4uH12Ho1Uog1say4 MvcQowQHs5IIr9J/oBBvSmJlVWpRfnxRaU5q8SFGaQ4WJXHe3j2rI4UE0hNLUrNTUwtSi2Cy TBycUg2M/Z9ucB7hWXjHd8KmK+k/g2SY/bz+1iz7ulsycdJP03uOwbsY9jX9/e9Z+3jBUf6g +5NtpBkOfVpXsOZKnDyb35xJnGftj4hJrj7RUaF/kmndSZtJP6u5E26XbmZz5nxs5F/BF/H2 cH/LjKr29aemPJ60f/MdpztVtrGb513TC2jaZrbvjOSuGUosxRmJhlrMRcWJAME0hm0cAgAA X-CMS-MailID: 20171005111623eucas1p272597c60842087cac3ade92b88212eff X-Msg-Generator: CA X-Sender-IP: 182.198.249.179 X-Local-Sender: =?utf-8?q?Alexey_Perevalov=1BSRR-Virtualization_Lab=1B?= =?utf-8?b?7IK87ISx7KCE7J6QG1NlbmlvciBFbmdpbmVlcg==?= X-Global-Sender: =?utf-8?q?Alexey_Perevalov=1BSRR-Virtualization_Lab=1BSa?= =?utf-8?q?msung_Electronics=1BSenior_Engineer?= X-Sender-Code: =?utf-8?q?C10=1BCISHQ=1BC10GD01GD010154?= CMS-TYPE: 201P X-CMS-RootMailID: 20171005111623eucas1p272597c60842087cac3ade92b88212eff X-RootMTR: 20171005111623eucas1p272597c60842087cac3ade92b88212eff References: <1507202170-22619-1-git-send-email-a.perevalov@samsung.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 210.118.77.11 Subject: [Qemu-devel] [PATCH v11 2/6] migration: add postcopy blocktime ctx into MigrationIncomingState X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: heetae82.ahn@samsung.com, quintela@redhat.com, Alexey Perevalov , peterx@redhat.com, dgilbert@redhat.com, i.maximets@samsung.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch adds request to kernel space for UFFD_FEATURE_THREAD_ID, in case when this feature is provided by kernel. PostcopyBlocktimeContext is incapsulated inside postcopy-ram.c, due to it's postcopy only feature. Also it defines PostcopyBlocktimeContext's instance live time. Information from PostcopyBlocktimeContext instance will be provided much after postcopy migration end, instance of PostcopyBlocktimeContext will live till QEMU exit, but part of it (vcpu_addr, page_fault_vcpu_time) used only during calculation, will be released when postcopy ended or failed. To enable postcopy blocktime calculation on destination, need to request proper capabiltiy (Patch for documentation will be at the tail of the patch set). As an example following command enable that capability, assume QEMU was started with -chardev socket,id=charmonitor,path=/var/lib/migrate-vm-monitor.sock option to control it [root@host]#printf "{\"execute\" : \"qmp_capabilities\"}\r\n \ {\"execute\": \"migrate-set-capabilities\" , \"arguments\": { \"capabilities\": [ { \"capability\": \"postcopy-blocktime\", \"state\": true } ] } }" | nc -U /var/lib/migrate-vm-monitor.sock Or just with HMP (qemu) migrate_set_capability postcopy-blocktime on Signed-off-by: Alexey Perevalov Reviewed-by: Dr. David Alan Gilbert --- migration/migration.h | 8 +++++++ migration/postcopy-ram.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 67 insertions(+) diff --git a/migration/migration.h b/migration/migration.h index c12ceba..2bae992 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -22,6 +22,8 @@ #include "hw/qdev.h" #include "io/channel.h" +struct PostcopyBlocktimeContext; + /* State for the incoming migration */ struct MigrationIncomingState { QEMUFile *from_src_file; @@ -59,6 +61,12 @@ struct MigrationIncomingState { /* The coroutine we should enter (back) after failover */ Coroutine *migration_incoming_co; QemuSemaphore colo_incoming_sem; + + /* + * PostcopyBlocktimeContext to keep information for postcopy + * live migration, to calculate vCPU block time + * */ + struct PostcopyBlocktimeContext *blocktime_ctx; }; MigrationIncomingState *migration_incoming_get_current(void); diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index bec6c2c..c18ec5a 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -61,6 +61,52 @@ struct PostcopyDiscardState { #include #include +typedef struct PostcopyBlocktimeContext { + /* time when page fault initiated per vCPU */ + int64_t *page_fault_vcpu_time; + /* page address per vCPU */ + uint64_t *vcpu_addr; + int64_t total_blocktime; + /* blocktime per vCPU */ + int64_t *vcpu_blocktime; + /* point in time when last page fault was initiated */ + int64_t last_begin; + /* number of vCPU are suspended */ + int smp_cpus_down; + + /* + * Handler for exit event, necessary for + * releasing whole blocktime_ctx + */ + Notifier exit_notifier; +} PostcopyBlocktimeContext; + +static void destroy_blocktime_context(struct PostcopyBlocktimeContext *ctx) +{ + g_free(ctx->page_fault_vcpu_time); + g_free(ctx->vcpu_addr); + g_free(ctx->vcpu_blocktime); + g_free(ctx); +} + +static void migration_exit_cb(Notifier *n, void *data) +{ + PostcopyBlocktimeContext *ctx = container_of(n, PostcopyBlocktimeContext, + exit_notifier); + destroy_blocktime_context(ctx); +} + +static struct PostcopyBlocktimeContext *blocktime_context_new(void) +{ + PostcopyBlocktimeContext *ctx = g_new0(PostcopyBlocktimeContext, 1); + ctx->page_fault_vcpu_time = g_new0(int64_t, smp_cpus); + ctx->vcpu_addr = g_new0(uint64_t, smp_cpus); + ctx->vcpu_blocktime = g_new0(int64_t, smp_cpus); + + ctx->exit_notifier.notify = migration_exit_cb; + qemu_add_exit_notifier(&ctx->exit_notifier); + return ctx; +} /** * receive_ufd_features: check userfault fd features, to request only supported @@ -153,6 +199,19 @@ static bool ufd_check_and_apply(int ufd, MigrationIncomingState *mis) } } +#ifdef UFFD_FEATURE_THREAD_ID + if (migrate_postcopy_blocktime() && mis && + UFFD_FEATURE_THREAD_ID & supported_features) { + /* kernel supports that feature */ + /* don't create blocktime_context if it exists */ + if (!mis->blocktime_ctx) { + mis->blocktime_ctx = blocktime_context_new(); + } + + asked_features |= UFFD_FEATURE_THREAD_ID; + } +#endif + /* * request features, even if asked_features is 0, due to * kernel expects UFFD_API before UFFDIO_REGISTER, per