From patchwork Tue Feb 6 20:30:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Kagan X-Patchwork-Id: 870115 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="DVpVeuQt"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zbcdr5yjSz9s7F for ; Wed, 7 Feb 2018 08:14:08 +1100 (AEDT) Received: from localhost ([::1]:34754 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ejAZ4-0000mm-RJ for incoming@patchwork.ozlabs.org; Tue, 06 Feb 2018 16:14:06 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44711) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ej9tp-0000lU-Or for qemu-devel@nongnu.org; Tue, 06 Feb 2018 15:31:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ej9tm-00036g-CF for qemu-devel@nongnu.org; Tue, 06 Feb 2018 15:31:29 -0500 Received: from mail-eopbgr00101.outbound.protection.outlook.com ([40.107.0.101]:58891 helo=EUR02-AM5-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ej9tl-000360-O4 for qemu-devel@nongnu.org; Tue, 06 Feb 2018 15:31:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=JR44dtPGv2W9cH0yaUFlGIAkMdNSPuI2qFALn+zLGiE=; b=DVpVeuQtJjR2pXNTDXHdjS8mr63/LcQFwCuqpYj1oQpsX2fW09L2xUpsvI2+bBCtcwEECLkIoiLgaXEYpvsOR9jchQ3+MrB0JjlNdOHNFK6IMkEH19Gx9rydDq4m9rr+J0oCN8oTlOiFexs+EVPeHTptMjSQ1sM7ri9MLBKKXVI= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=rkagan@virtuozzo.com; Received: from rkaganb.sw.ru (195.214.232.6) by VI1PR0801MB1983.eurprd08.prod.outlook.com (2603:10a6:800:8a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.464.11; Tue, 6 Feb 2018 20:31:22 +0000 From: Roman Kagan To: qemu-devel@nongnu.org Date: Tue, 6 Feb 2018 23:30:24 +0300 Message-Id: <20180206203048.11096-11-rkagan@virtuozzo.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180206203048.11096-1-rkagan@virtuozzo.com> References: <20180206203048.11096-1-rkagan@virtuozzo.com> MIME-Version: 1.0 X-Originating-IP: [195.214.232.6] X-ClientProxiedBy: HE1PR0301CA0008.eurprd03.prod.outlook.com (2603:10a6:3:76::18) To VI1PR0801MB1983.eurprd08.prod.outlook.com (2603:10a6:800:8a::16) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e8d12459-1f77-4213-9943-08d56da096b4 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(5600026)(4604075)(2017052603307)(7153060)(7193020); SRVR:VI1PR0801MB1983; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0801MB1983; 3:GdF8gurS+s9OomPNY3NvY3ckzlqa6MJXUqWKUmMktoCxgDapi3zJD7Zer/Xc6TNGsHj00d3zBkRApV44xafB2osAoV8WWZ5ZvDzOWwYXVvblAEh14fXTRwLFBRjUX3VRteG4/fpy36ndwv13oVl1j/ujDYdiwsWXORmELuUEMxnND5gNDdfURMHDYW6CtFuvehMOg8IUgrCvsWWxRTBUcqS7tq+Hqfe2Ps7pW6Yo6NmTTkqhJ9XxDJgMe4IlPyx/; 25:Tju9+Dbzz91rj+GlVilE8XTkuUyLgDwABF6EXlvro2yRoXCbSO5UQ9891nQrEk4zeb/ZrRzf3mLk0kvjv6xv9dqYY8mXUJLsFJzoRNzdNIWBGymhxNimuFQCxjbch0uSQPqGv4qJZqtbzIibkaf/LUmxL3udIsH1TlYZvNXyPAy5o0cGYwTXKatdmx1JDp/naqOXqh/ck5aiCSkFKdGzl843Qf3r4ZwtnUS+slshnUkBKN9JJIPHDHIAxhQvXHiHSqtWtG+89FRevBQE9Q6w7i+ZsRMdLsowN7FzWlJFyDfsEcx5e1hJtS4K3bm9wZJ55bkBrrGSjCP7eN173EPOvg==; 31:K+yprhtaOYAjOjfGsubSkK+2QLIEq98d2kYovlCxpgOON3gTHpt1v5+lbcvGI0mLlQDXLZPSddsHrVkBZ4dTtSqM55buaFfWos+6nZNuB+ZwLUG5+aJ4V0Z5b87URwM8z7Cw3NVEqD9u7CEA72ZFQ7Y65gUOj/eCL3CqYIeb1E7XbOoE6imocg5TRcng5Tzh7GjYWi8e6M67tgSgNbxJTV6Aij5XFGrgQ+3g/8C+ay0= X-MS-TrafficTypeDiagnostic: VI1PR0801MB1983: X-Microsoft-Exchange-Diagnostics: 1; VI1PR0801MB1983; 20:OMTIlo4O7Ic8CmbKs209TY08MfNHjZuaEFuzKkd/kU0HAnmOHE3PWZIxuNMyYNvZ8+iJ80l11z5OPW2I7FIDJJjEL3SFw2KBOS7CqWeGdKUDNBF6daJGvYQ53igSCuZnSrleCfibUBsYrrIMfwZ3ioIEg0c7bS5aubNE3G9PWjYDr9EUwuJGVXf8Acy6uH9fUSNQB/ImcEnL1YukL5n2SpdxlV4hY2u7rIGd7O2SIWX6vXeU+2s3Aj+CzATGiHqL/apZ9HYZnZCNShYFuvgtrphxrWWEVWZm5Wjv7oj6VSaUBYH1eab02/kaWtAUTJLj8dpMN65/mJJwiQ7LahNy3QiuyeQ/m8Gtk+CF+ttXTzBpIeyo2BUlr18BnKjyI474DY/DcWWm9wk52MzmuLn6IgZdDlyC3vMBhasGf5nxn/0=; 4:0nvJO25DBVB07V9hzLJ0a4lpTOJoqCPwEsU3hcO+SfPcXoAgddSPfHy9jyzomzrJslZxsyRk+xpyk5mLftCa8wewgjCyCPxIPuQb/xbX6OZKkoHmI7kSXRXkn4xYwvxiT9rucdsH790Vfr7DC4DyIEJ8CS4sW4CFUWY18ASusLXpw+YVWiMqfwCUx+RGO0ruy3gmBBF0iKHvqCoPO8kuX35+J2/5yFzC4aMICLOKAWSY/oY5msTkSViTb++4oVvMSw6eCN6GrtzAg4m6iQVAJA== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040501)(2401047)(5005006)(8121501046)(3002001)(93006095)(93001095)(3231101)(2400082)(944501161)(10201501046)(6041288)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123564045)(20161123562045)(20161123560045)(6072148)(201708071742011); SRVR:VI1PR0801MB1983; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0801MB1983; X-Forefront-PRVS: 0575F81B58 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(396003)(366004)(376002)(39380400002)(39850400004)(346002)(189003)(199004)(16586007)(53936002)(305945005)(54906003)(7736002)(3846002)(316002)(105586002)(2361001)(106356001)(2351001)(6116002)(1076002)(50226002)(8936002)(81156014)(5660300001)(8676002)(81166006)(7416002)(2950100002)(6916009)(76176011)(86362001)(68736007)(66066001)(16526019)(47776003)(50466002)(4326008)(97736004)(26005)(6512007)(51416003)(2906002)(52116002)(386003)(6506007)(59450400001)(48376002)(55236004)(36756003)(53416004)(69596002)(6486002)(478600001)(186003)(25786009)(217873001); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR0801MB1983; H:rkaganb.sw.ru; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; VI1PR0801MB1983; 23:DPNR6DOnzZuc5L4bl6RX2JRVjiVaXI1ft1S0tAT?= +7eJQljnmkhT3cJxRUkG4MzjNv/LuQzGWXVA113lduxhjmQS40uiipwxWWntmxqaald+s9/GV6IaCyv2uuYpz4hffSjq7tPIXUUlI+q3WsoRyl1pCpdsaRDekMGR1ZlcjLYIumzBHL/5NV89d/M0RF1UxLeSwY0DFrsHKj5vdhkbollIN/uKCFLNa5BJtkEQkoIlgWCBGrYhPWUXIfFp9KqC1egTpRgHw3efM4bZo29SyeJpurpyM7WaPIVWvQcRyXGO40wHIGXiOKEZUro5TW9bH8dz47vpyC6VNrEFkCj7Ip592WnNyCneczXWAg0lZ102TTc6NFvKXQ7S959BBkavwDimFODXjdgTrH5Xh7q1mwREYIQNEy8OtVtbKlvEVOOB0Gcw6rxaSTIyTpwDuATorfQItOcTwYp5MPNrzoFtM//eP3ulDgytX9Q+k8Y+Iwd7VBV1QuPeFDGu/qIWOcmFYKvgGdDrNqO1o0Cv6aVcai4q0gOM+ho46roGOI5npdtfobKPz1nGlMIIGuJp8DEtfUrYh52MKwUQXIypuIDi7lleySNLBa1TztffXRPqHARH0SrrLLZ3b5/NK69kysXi6JrJzCaHQl5R6WvXS+xLFF96PB6RJNwHejfcIdFIk3bAtxguyY4odx52+gGELHT7OjZAQzF6Tu7bNqS/B08/trvjvL83bhVng8WJNy5yMZFy/uXlQ5aRd8LbQcC592TdFq1yIvFCawzL6a5wUfFRCyDgWhdO0kzoX+Q+Eb5rF/pGwRray8ThH6BRdSYo6HwHChPQfDqmsnCR4qVAGsXXv8k+8vezoH7nmlAEH7joElygyHm6FMSF3gv7l0XmMm+BvxJKwM+bH7vtESbxFOCDjJn+9fiWbzGkkdBF4eqgEgHIEIh9VE05+EPyTjff/il4erJH4WDcRz57PrNAJNvfPrQu7OCZXyIkGXB/hGBvlLr4uYt8h8Ba9bVUF44RQaOszLNCvRioUQOkM1ymFjNax6OEOmYaHWP6PymjMlUMxUPiyHFeRQeLS2IQowvqgxAYNokyt/LdbYZGjYgc9EnRqa+lxCQnKgidwjXA59//L/twLWrKbBuxwEBYVkZSy4RBN/H+QeJAgCuTUXwybX1eZL3nd7tFHFKCyqWD0MuUKvy6DW84norlhIQnXvGJ5P+9YeaZIMfSdOvYjmQiLzrRig49i3dYIvPU9SgHe89vxkfLwZBzjh+ZBir+pi08Z/KGOkK1kE8wZhEh85heUjIZ+uQ== X-Microsoft-Exchange-Diagnostics: 1; VI1PR0801MB1983; 6:iV52Refe1xKZQeS5cgMCD6TuJYCzdMjtnusjLZeDshkIx2sncQrjjCh1D4gofrctgNAqiXRguWPPntV11oHucKJbHB0cf3PuDaOYr0GnJwppdUoDlRnqXPMMG6N5Ios+i0jE7hZHdZ32eo0f+Waxdf0JMl1waFFvbUsFmk/ctJSn647znK7XZUdq18O1HTg67c4jChaT7d5BktlwBPdD2W45+/FXt1ti0hBxfMl2D8R1bxwjdaVt4/3ZHCojhsfY0OSMzlfq3uBkH3dE3AipvF4Ph9FqDyKHzsqQ4YVmMzjwtP3yahb0HTAYmbnIwoS8Eb7rMRrfoJkusFwFbQOstNUHgZHkzsUfkcG9DlNveN0=; 5:gWtKGPY+KtL+VYIIsXUpokvUg+IpnQ1ZT++t2O/vH3jUDAdeGdE4M+qSrN/ECgu4RSwtXL5jHrkx0I7atLJIMsVMEG++VOWwRegaG5Xgna4scJr7AS32v4BQ8xi1wfvOOINYLzC7jtkcU3SK7WJRJ+MorVCtqkT0m5zEsTlm2Wo=; 24:CdqudPH+3S6rXbTYIedo8LZbzoUliz6u5CYRrEmm4YWksGms+eUlYem8s74NVc00l8w1wd1rjGxrcSW/31iLleAyVgBByy8a1jQFZ2K1HuM=; 7:BsEJmnUbIaBwToENrswXKaxaOMjdoMVQnCQCHd1oQ9UHZpc/iAIJkMd1w3NDvtujZcTJ4GtCXso6WHrhD9N1purBAuJ6NVQDyL6S3VL9aSWpJxHa1ptG59+Y6+0W7DzHV//nKTILSBW2aIKyyCpmnV1V6ee7cdy1bShrRB8cG5NivpsDM4WjjZLA++BK0igr/8owk1+QvdZ7HZoA7KJD1z6PwrFxsDyJfQ2B+bdnZWLoWvNwNGmX9/F3Dyewi0n9 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; VI1PR0801MB1983; 20:WD/elgLJUKcGIvsnABlT0jU9h7Brxr7NhsubNshf2NZnBPdV7/pf4lKq5oQIyODG3VFqlKGqODanyiRUDJnunYDbYn9k8ay7pRHLzFzY/ewWlnDFRgYKjqxgKBTYKv3YxBheMhwE0BN5HX0uLC1SRvEZEWNBsCFzD1xLci53c10= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2018 20:31:22.1062 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e8d12459-1f77-4213-9943-08d56da096b4 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1983 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 40.107.0.101 Subject: [Qemu-devel] [RFC PATCH 10/34] hyperv: make overlay pages for SynIC X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ben Warren , Konrad Rzeszutek Wilk , Krish Sadhukhan , "Marcos E. Matsunaga" , Jan Dakinevich , Vadim Rozenfeld , "Denis V. Lunev" , si-wei liu , Paolo Bonzini , Vitaly Kuznetsov , Cathy Avery Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Per Hyper-V spec, SynIC message and event flag pages are to be implemented as so called overlay pages. That is, they are owned by the hypervisor and, when mapped into the guest physical address space, overlay the guest physical pages such that 1) the overlaid guest page becomes invisible to the guest CPUs until the overlay page is turned off 2) the contents of the overlay page is preserved when it's turned off and back on, even at a different address; it's only zeroed at vcpu reset This particular nature of SynIC message and event flag pages is ignored in the current code, and guest physical pages are used directly instead. This (mostly) works because the actual guests seem not to depend on the features listed above. This patch implements those pages as the spec mandates. Since the extra RAM regions, which introduce migration incompatibility, are only added when in_kvm_only == false, no extra compat logic is necessary. Signed-off-by: Roman Kagan --- target/i386/hyperv.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 64 insertions(+), 6 deletions(-) diff --git a/target/i386/hyperv.c b/target/i386/hyperv.c index 933bfe5bcb..514cd27216 100644 --- a/target/i386/hyperv.c +++ b/target/i386/hyperv.c @@ -16,6 +16,9 @@ #include "qapi/error.h" #include "qemu/error-report.h" #include "hw/qdev-properties.h" +#include "exec/address-spaces.h" +#include "sysemu/cpus.h" +#include "migration/vmstate.h" #include "hyperv.h" #include "hyperv-proto.h" @@ -29,6 +32,10 @@ typedef struct SynICState { bool enabled; hwaddr msg_page_addr; hwaddr evt_page_addr; + MemoryRegion msg_page_mr; + MemoryRegion evt_page_mr; + struct hyperv_message_page *msg_page; + struct hyperv_event_flags_page *evt_page; } SynICState; #define TYPE_SYNIC "hyperv-synic" @@ -68,6 +75,17 @@ static void synic_update_msg_page_addr(SynICState *synic) uint64_t msr = synic->cpu->env.msr_hv_synic_msg_page; hwaddr new_addr = (msr & HV_SIMP_ENABLE) ? (msr & TARGET_PAGE_MASK) : 0; + if (new_addr == synic->msg_page_addr) { + return; + } + + if (synic->msg_page_addr) { + memory_region_del_subregion(get_system_memory(), &synic->msg_page_mr); + } + if (new_addr) { + memory_region_add_subregion(get_system_memory(), new_addr, + &synic->msg_page_mr); + } synic->msg_page_addr = new_addr; } @@ -76,6 +94,17 @@ static void synic_update_evt_page_addr(SynICState *synic) uint64_t msr = synic->cpu->env.msr_hv_synic_evt_page; hwaddr new_addr = (msr & HV_SIEFP_ENABLE) ? (msr & TARGET_PAGE_MASK) : 0; + if (new_addr == synic->evt_page_addr) { + return; + } + + if (synic->evt_page_addr) { + memory_region_del_subregion(get_system_memory(), &synic->evt_page_mr); + } + if (new_addr) { + memory_region_add_subregion(get_system_memory(), new_addr, + &synic->evt_page_mr); + } synic->evt_page_addr = new_addr; } @@ -90,6 +119,15 @@ static void synic_update(SynICState *synic) synic_update_evt_page_addr(synic); } + +static void async_synic_update(CPUState *cs, run_on_cpu_data data) +{ + SynICState *synic = data.host_ptr; + qemu_mutex_lock_iothread(); + synic_update(synic); + qemu_mutex_unlock_iothread(); +} + int kvm_hv_handle_exit(X86CPU *cpu, struct kvm_hyperv_exit *exit) { CPUX86State *env = &cpu->env; @@ -100,11 +138,6 @@ int kvm_hv_handle_exit(X86CPU *cpu, struct kvm_hyperv_exit *exit) return -1; } - /* - * For now just track changes in SynIC control and msg/evt pages msr's. - * When SynIC messaging/events processing will be added in future - * here we will do messages queues flushing and pages remapping. - */ switch (exit->u.synic.msr) { case HV_X64_MSR_SCONTROL: env->msr_hv_synic_control = exit->u.synic.control; @@ -118,7 +151,13 @@ int kvm_hv_handle_exit(X86CPU *cpu, struct kvm_hyperv_exit *exit) default: return -1; } - synic_update(get_synic(cpu)); + /* + * this will run in this cpu thread before it returns to KVM, but in a + * safe environment (i.e. when all cpus are quiescent) -- this is + * necessary because we're changing memory hierarchy + */ + async_safe_run_on_cpu(CPU(cpu), async_synic_update, + RUN_ON_CPU_HOST_PTR(get_synic(cpu))); return 0; case KVM_EXIT_HYPERV_HCALL: { uint16_t code; @@ -258,12 +297,29 @@ static void synic_realize(DeviceState *dev, Error **errp) { Object *obj = OBJECT(dev); SynICState *synic = SYNIC(dev); + char *msgp_name, *evtp_name; + uint32_t vp_index; if (synic->in_kvm_only) { return; } synic->cpu = X86_CPU(obj->parent); + + /* memory region names have to be globally unique */ + vp_index = hyperv_vp_index(synic->cpu); + msgp_name = g_strdup_printf("synic-%u-msg-page", vp_index); + evtp_name = g_strdup_printf("synic-%u-evt-page", vp_index); + + memory_region_init_ram(&synic->msg_page_mr, obj, msgp_name, + sizeof(*synic->msg_page), &error_abort); + memory_region_init_ram(&synic->evt_page_mr, obj, evtp_name, + sizeof(*synic->evt_page), &error_abort); + synic->msg_page = memory_region_get_ram_ptr(&synic->msg_page_mr); + synic->evt_page = memory_region_get_ram_ptr(&synic->evt_page_mr); + + g_free(msgp_name); + g_free(evtp_name); } static void synic_reset(DeviceState *dev) @@ -274,6 +330,8 @@ static void synic_reset(DeviceState *dev) return; } + memset(synic->msg_page, 0, sizeof(*synic->msg_page)); + memset(synic->evt_page, 0, sizeof(*synic->evt_page)); synic_update(synic); }