From patchwork Sat Aug 15 02:22:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zheng Chuan X-Patchwork-Id: 1345325 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=huawei.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4BTSwg2xDcz9sRK for ; Sun, 16 Aug 2020 04:08:35 +1000 (AEST) Received: from localhost ([::1]:47962 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k70bZ-00010B-6t for incoming@patchwork.ozlabs.org; Sat, 15 Aug 2020 14:08:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51880) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k70VU-0001KZ-N0 for qemu-devel@nongnu.org; Sat, 15 Aug 2020 14:02:17 -0400 Received: from szxga07-in.huawei.com ([45.249.212.35]:39950 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k70VS-0007F7-OW for qemu-devel@nongnu.org; Sat, 15 Aug 2020 14:02:16 -0400 Received: from DGGEMS401-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id E3AC91247FBDD6645ADD; Sat, 15 Aug 2020 10:11:11 +0800 (CST) Received: from huawei.com (10.175.101.6) by DGGEMS401-HUB.china.huawei.com (10.3.19.201) with Microsoft SMTP Server id 14.3.487.0; Sat, 15 Aug 2020 10:11:04 +0800 From: Chuan Zheng To: , , Subject: [PATCH v2 00/10] *** A Method for evaluating dirty page rate *** Date: Sat, 15 Aug 2020 10:22:50 +0800 Message-ID: <1597458180-16945-1-git-send-email-zhengchuan@huawei.com> X-Mailer: git-send-email 1.8.3.1 MIME-Version: 1.0 X-Originating-IP: [10.175.101.6] X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.35; envelope-from=zhengchuan@huawei.com; helo=huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/15 14:02:09 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhang.zhanghailiang@huawei.com, linyilu@huawei.com, qemu-devel@nongnu.org, alex.chen@huawei.com, ann.zhuangyanying@huawei.com, fangying1@huawei.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" v1 -> v2: use g_rand_new() to generate rand_buf move RAMBLOCK_FOREACH_MIGRATABLE into migration/ram.h add skip_sample_ramblock to filter sampled ramblock fix multi-numa vm coredump when query dirtyrate rename qapi interface and rename some structures and functions succeed to compile by appling each patch add test for migrating vm Sometimes it is neccessary to evaluate dirty page rate before migration. Users could decide whether to proceed migration based on the evaluation in case of vm performance loss due to heavy workload. Unlikey simulating dirtylog sync which could do harm on runnning vm, we provide a sample-hash method to compare hash results for samping page. In this way, it would have hardly no impact on vm performance. Evaluate the dirtypage rate both on running and migration vm. The VM specifications for migration are as follows: - VM use 4-K page; - the number of VCPU is 32; - the total memory is 32Gigabit; - use 'mempress' tool to pressurize VM(mempress 4096 1024); - migration bandwidth is 1GB/s ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | | running | migrating | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | no mempress | 4MB/s | 8MB/s (migrated success) | ------------------------------------------------------------------------------------ | mempress 4096 1024 | 1188MB/s | 536MB/s ~ 1044MB/s (cpu throttle triggered) | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | mempress 4096 4096 | 4152MB/s | 608MB/s ~ 4125MB/s (migation failed) | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Test dirtyrate by qmp command like this: 1. virsh qemu-monitor-command [vmname] '{"execute":"calc-dirty-rate", "arguments": {"calc-time": [sleep-time]}}'; 2. sleep specific time which is a bit larger than sleep-time 3. virsh qemu-monitor-command [vmname] '{"execute":"query-dirty-rate"}' Further test dirtyrate by libvirt api like this: virsh getdirtyrate [vmname] [sleep-time] Zheng Chuan (10): migration/dirtyrate: Add get_dirtyrate_thread() function migration/dirtyrate: Add RamlockDirtyInfo to store sampled page info migration/dirtyrate: Add dirtyrate statistics series functions migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h migration/dirtyrate: Record hash results for each sampled page migration/dirtyrate: Compare page hash results for recorded sampled page migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE migration/dirtyrate: Implement get_sample_page_period() and block_sample_page_period() migration/dirtyrate: Implement calculate_dirtyrate() function migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function migration/Makefile.objs | 1 + migration/dirtyrate.c | 448 ++++++++++++++++++++++++++++++++++++++++++++++++ migration/dirtyrate.h | 86 ++++++++++ migration/ram.c | 11 +- migration/ram.h | 10 ++ qapi/migration.json | 42 +++++ 6 files changed, 588 insertions(+), 10 deletions(-) create mode 100644 migration/dirtyrate.c create mode 100644 migration/dirtyrate.h