From patchwork Tue Feb 6 20:30:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Kagan X-Patchwork-Id: 870077 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="Zi8zDe62"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zbbmZ1Rd1z9s8J for ; Wed, 7 Feb 2018 07:34:54 +1100 (AEDT) Received: from localhost ([::1]:59027 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ej9x6-0004Ab-7v for incoming@patchwork.ozlabs.org; Tue, 06 Feb 2018 15:34:52 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44545) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ej9tY-0000Q5-De for qemu-devel@nongnu.org; Tue, 06 Feb 2018 15:31:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ej9tV-0002xj-FY for qemu-devel@nongnu.org; Tue, 06 Feb 2018 15:31:12 -0500 Received: from mail-eopbgr00138.outbound.protection.outlook.com ([40.107.0.138]:58112 helo=EUR02-AM5-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ej9tV-0002u4-8P for qemu-devel@nongnu.org; Tue, 06 Feb 2018 15:31:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=9E2NpJyY2+IsT/41PofkY+zSYZNnWyZKsGRkqMm66v4=; b=Zi8zDe62wPASg0C92fuWzW9SnAkj21X8WKxju/BrlxdxPyA7GyYX1AfyDadOJFzP3RDXNoczXcBEuWyZ1pnuOEh+VEkcPszeSkRSFK0QSNScW7Jj8CngA0QIyQCDXSUEqLWBc2zhSO0eDs85/7DNQkpXZuHt8RXKStaXXC+RS10= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=rkagan@virtuozzo.com; Received: from rkaganb.sw.ru (195.214.232.6) by VI1PR0801MB1983.eurprd08.prod.outlook.com (2603:10a6:800:8a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.464.11; Tue, 6 Feb 2018 20:31:04 +0000 From: Roman Kagan To: qemu-devel@nongnu.org Date: Tue, 6 Feb 2018 23:30:15 +0300 Message-Id: <20180206203048.11096-2-rkagan@virtuozzo.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180206203048.11096-1-rkagan@virtuozzo.com> References: <20180206203048.11096-1-rkagan@virtuozzo.com> MIME-Version: 1.0 X-Originating-IP: [195.214.232.6] X-ClientProxiedBy: HE1PR0301CA0008.eurprd03.prod.outlook.com (2603:10a6:3:76::18) To VI1PR0801MB1983.eurprd08.prod.outlook.com (2603:10a6:800:8a::16) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7419507c-c43b-4678-8024-08d56da08c60 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(5600026)(4604075)(2017052603307)(7153060)(7193020); SRVR:VI1PR0801MB1983; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0801MB1983; 3:9hvmJsMNFhWIWFTTqtGV+aDbpTH4KPePtEljs01b4hN2LxM7VNevT20FhTWOj5G5D3t8mN2vuq2LupiebNTG+QFmztlCjvDDjs6CBeNrZl2cQUlC3LMriju5wgegh/PHkmtn/xIWpMGUo7TND4YlzveCHmnkgHgkZ5OdoJ8qZ2XDmYm3TJiaHnf6CXQKnMcCc1HBmYPJCm6NruTaALrBzxfm3Z/tWg9bArgQGWZ/htee83CfJhf4q/KhrSiSSF6M; 25:w5jlOqkzvBNW+3en5KaIFnR+whyY+RXX5GX1WT3yPlxC/k5VYTNM3qnnoNXHcXfts/ZjrvrcvwXgHAISEkSTFhgPxdN57FojcWJ2j5mGdZDUuKNCPsHTKR6JeYszxfPYq3R43ayLyo5ihyQiumgB91pPEmppoCjE5NB5/Dg6505KXMEW1DFeN2r1i2TxTKiVtKFUAcltDh7s6Mrx0hCnnsYd6HMOq7kAIgcsUjLqRp7S6qYLYFQMf+oz9RcDEXTq+nmK075GAS5+MnXJjNvHca6pbjxKXDgOxqOAllPWaXmFVoFTavVzH8/DdPx4DGiCUajeSXxG2uP2siOIRBjBOA==; 31:bMx8JzvFXx8vtuAqUsdNHCy8kh2GUES7i4v88oIo5mlApFsfjE11GhB7nupyLDRpHO/o/kLjxcKvMSV/yb7CHK/qxapS0gaIvTdTYrlILtipByVVDgUMxs0Z9JmQQ3Sqx8o7zh46XWXIaefImiZLcF+iKBw/ku9phoL0mOBHmaZ/+GjF2vpmVgQSBMwdekC9TN4PJIIowUvwXalMxENp8l/WHvxm3ee6X8UA5uVvIXA= X-MS-TrafficTypeDiagnostic: VI1PR0801MB1983: X-Microsoft-Exchange-Diagnostics: 1; VI1PR0801MB1983; 20:RoqokXIfi1PE7wqhBUUSltDonEiiVQAl/pILrgB6KAiXe4Db142yPrz3/oycfEQFssG4PAySdL2PaJEWERjBR7jCeId1m8OVOUY6Gg77cbDv0eFc8CLWiN6On5SPIgGmScd5tOWeUBUvqZFRXWjEbkcnIvZCmmQbvPulW7wr6f2ILQQrPHfakPl8Kd6o4rg0YI5uaJejRxg3js3MDRr61VtNnQdsIHVMwIcsqwnsTXpTxo5cdkJ4bNYNyc8qQrqHbqk6M1Ai/MUrGYQh62hX0+axViJAynNxALBdLGVD2LZbc+MQF/JObP+MN1lue+Cv+9UmJJe4kkq+vb/qlR8ffm4EOgbhxG1Vbx+Wk5b4LGmkd2DUpcBwK6zcJQT9ASZcn7i2KiUdpFWM2oTEZNpPEqU8CQI9Wou2DHsFY2NzpqA=; 4:TNVILboUwoJeqAanRvA7gTx+rgKITG4wL71eQioM6WSAX5FMdCZ+5Yu6coLEiGESSqXRgbYb7DRKVll7FBrI1RnrSHhvGoA5SNaxstKzHELWpd10Z63zR7AxyxqHmw/cAkBtbyocjdW++Xr53W7W+qH8RWAcJKhMgre7b08mG2SVzcQnrR0GwJ93pfrsRqwoVXQ1pgUL63uBCUszsskv7mWstLzJXxM0SaR1enhJJDewOzM30mgXQxtGPhIy3QaYZXfeYZOZxUOnniPkpQQF5w== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040501)(2401047)(5005006)(8121501046)(3002001)(93006095)(93001095)(3231101)(2400082)(944501161)(10201501046)(6041288)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123564045)(20161123562045)(20161123560045)(6072148)(201708071742011); SRVR:VI1PR0801MB1983; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0801MB1983; X-Forefront-PRVS: 0575F81B58 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(396003)(366004)(376002)(39380400002)(39850400004)(346002)(189003)(199004)(16586007)(53936002)(305945005)(54906003)(7736002)(3846002)(316002)(105586002)(2361001)(106356001)(2351001)(6116002)(1076002)(50226002)(8936002)(81156014)(5660300001)(8676002)(81166006)(7416002)(2950100002)(6666003)(6916009)(76176011)(86362001)(68736007)(66066001)(16526019)(47776003)(50466002)(4326008)(97736004)(26005)(6512007)(51416003)(2906002)(52116002)(386003)(6506007)(59450400001)(48376002)(55236004)(36756003)(53416004)(69596002)(6486002)(478600001)(186003)(25786009)(217873001); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR0801MB1983; H:rkaganb.sw.ru; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: virtuozzo.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; VI1PR0801MB1983; 23:XZwgMCxnuGqhR5oiVc9CpXShMR4/xMIBKqL4fWY?= v1c6e42hKBDD0QSTNwatq41nqoZOKqXbM0HYrNrVJKnG4pRzbqgBmDlSBkUbWW0GRCT/AxFaHbWEURd1rMLduQWoG2aqZj4aMs8thmAxUeQV5VZr3rUZQW7dWhIsZqY6B0JI+HGT3sJQkf/6dbjVIWt+OoRhxrRSTpsrRjJi3ZXbM1SaiGX3GRLTjoh1ylKmGLrgl39XlPR98lVh5YOL7P6mfEchUkrocnimFzfokhmUgvKE7SnYioULTp09S3ks/AwuujPtm4om61CsldV8q+FSVqejKVFij1svhoWJgV01aK5VvGy77wT4+ajs4hAfW8BYyYNsxrkTRAcaT8yZ/H4YNj4pROLqE1UMe4HEPV3vsna5lugRN1PHagViaPLT/QVzgG5wSz3B9B8BBy7X2dUkseng68pgiUgUlqXKdiUcCZS3n3h7uIgXn6uczKZoFz/IMcUwcZ+Aprj63213FhEiCRRuW5c8ZX2NRP/JLPYkyYIkp+Syk0wGg3Y3UFThf9wCquWbgOoHk4szIYzM+EkC0+oQPBWkqJ0lmhvm/yizpm5XU7GVEwyAOtFbc9sv0GG528jL/4f+ORCI3ui9a6t9B4acBnLAzSWwQumbrprJrL9zSI3iHTvh7QAERKjfrdZTGSuN6SYeTWO+jabA+hlqJCfyDTwCUBG1PRyQgbPbeGj0udAt9R0ppDUNcOmR1qJNkIOZ/OXkGYFTTWWA3oW0DXbq+TCxvxUIfeNSUdAKZd9S4g4rrHMzY5YREEzHfci19U23jPSc6MiSQZpC+x7d5Dhdl82XMqXYnzcYB3XNy7rD8HErnZTQhg6n/X+BsbhRlKLbPNoz79TqDbDmrppZ5hACvKP8+30wA1Vi8vyOnzX367TtrVWtG1DJ1QGvEYje3/z0Ou6gLaHmeVZlCDlRqYoU7QFYcVxrSChEtFzviNCtMS14qlGVYPON/D2HMAkSj4y+9HpElfgyxeUD2qVg0BBlf1tLWtRdTWG8zT12izGzr1t+C4pnmp6xY9d4HTK41BaAWgLyAdxQ1hfsJec63S+eHbCZpG1IWob5XCnHxOJKswnIBrQfCaRTCOyRqXZiUf/EQ02FjG0c3+w2XXO7PpFwfmyeps3EH4jtPlPm4yRaiMMmdSdLtIR8eAfaOdbYqj2RnL4P+P/ysjQtE20CflveGtpM3CEa2euH7oQRhY1KQgS6uxk9XGky5UoJ8VXeCG4f/BMS5i60v2bCd7KvhGckNWxYc2bLidN/FvbZRAFCb57oLktvpXwTKlQYJbfM= X-Microsoft-Exchange-Diagnostics: 1; VI1PR0801MB1983; 6:vaTa4woykgjyOeyThpZdsAvzqBdfUWVIt0w/9EvMQw+xTeDVa+06TADdKT4ur9emeb3dQ+XL3dM3GqcQ7YOV/SyMN8LokP+EwUqv2N99IIPvmlb+Po1nVKd1vyaMIZrVcCSueWUuOoVpPLJYqope5/DMjcxvmFfEFfUI7myMdJRNJsSzdl59WzKD/zfRGewnlEJkzgXuZrMmF4Omu/qsepf/tqkIAW8ZW7uIeA+tstnSdCqzn1euY/fx1AxvZj2GgUK2lOdd0ddmav1VbihBW9z4nVq3ykJ6Nm/YqLVoJUe/ERZSgOIrA7xzwITJnqeg+FzYV56SkWDjcDoNh49vLVVCYIAcXeVVIW3hRxVy8lA=; 5:nSYjeGHVHNmCa+Varp8CXoKpFxJkyydmDPGgixojcKBf06qgpQX2n35aOkXiHjtUNuvORN/u0eTQ2lRpyTOMySj9l3IDcZI3rUJ+OLSMKfZgv6cvQhVp+UOZEmFz5YWWccWRhLG+JOp2FqTs9rdEiLlakuMZjXeUeT7GdduqEw0=; 24:iNAIGCAnukfqMrPFP7f29OX4HTqH6lLQKC7JJuqyYKPgzjRC/9AXR9M+Snj8WTwrSN3XSxJkKcM0msW/1wFgxhuX64x6ahU30aR1NH56T3I=; 7:rM8Df1P9F4vHtVwxy3yDcGP3fPzW4mLZylppit2gR1CVi7vMZQUrKzyC9UClUcHzqrmTu6meK7ttdah23blMYZ8L5t8+cMWqmxR1yFykN9myOYADfH0iUd4Bk3VP8V66C4+Ax3sY+eUrd+2GUf7NtllFeNMtPpjMqkXaMGrJXRNmWUL0moL+36jZ+BPYXfab/4I/46vacCR0NSErdB+q57FSeqpVHW0nzHV1J+tO65quRI1fJNoA3MRlVQCgUGoM SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; VI1PR0801MB1983; 20:hzBUZGIV3KYvMhxz96rqKxpua8D+mrfNWE4YGHNKxA6cu6obmGxxvgedkmOWhZ+Rux8ZCgTzjDPXhSMt7vkED5+bmh9HVzb+o0/Zn7Ryn5ACqRM1XJEdhkCTx/ef14050C7vpT3I7Ju9Cz24S0cN87BFMAdDZPveIf0B/r0BdUQ= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2018 20:31:04.7778 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7419507c-c43b-4678-8024-08d56da08c60 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1983 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 40.107.0.138 Subject: [Qemu-devel] [RFC PATCH 01/34] hyperv: ensure VP index equal to QEMU cpu_index X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ben Warren , Konrad Rzeszutek Wilk , Krish Sadhukhan , "Marcos E. Matsunaga" , Jan Dakinevich , Vadim Rozenfeld , "Denis V. Lunev" , si-wei liu , Paolo Bonzini , Vitaly Kuznetsov , Cathy Avery Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Hyper-V identifies vCPUs by Virtual Processor (VP) index which can be queried by the guest via HV_X64_MSR_VP_INDEX msr. It is defined by the spec as a sequential number which can't exceed the maximum number of vCPUs per VM. It has to be owned by QEMU in order to preserve it across migration. However, the initial implementation in KVM didn't allow to set this msr, and KVM used its own notion of VP index. Fortunately, the way vCPUs are created in QEMU/KVM makes it likely that the KVM value is equal to QEMU cpu_index. So choose cpu_index as the value for vp_index, and push that to KVM on kernels that support setting the msr. On older ones that don't, query the kernel value and assert that it's in sync with QEMU. Besides, since handling errors from vCPU init at hotplug time is impossible, disable vCPU hotplug. This patch also introduces accessor functions to wrap the mapping between a vCPU and its vp_index. Besides, a few variables are renamed to avoid confusion of vp_index with vcpu_id (== apic_id). Signed-off-by: Roman Kagan --- target/i386/hyperv.h | 5 ++++- target/i386/hyperv.c | 16 +++++++++++++--- target/i386/kvm.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 68 insertions(+), 4 deletions(-) diff --git a/target/i386/hyperv.h b/target/i386/hyperv.h index 0c3b562018..82f4757975 100644 --- a/target/i386/hyperv.h +++ b/target/i386/hyperv.h @@ -32,11 +32,14 @@ struct HvSintRoute { int kvm_hv_handle_exit(X86CPU *cpu, struct kvm_hyperv_exit *exit); -HvSintRoute *kvm_hv_sint_route_create(uint32_t vcpu_id, uint32_t sint, +HvSintRoute *kvm_hv_sint_route_create(uint32_t vp_index, uint32_t sint, HvSintAckClb sint_ack_clb); void kvm_hv_sint_route_destroy(HvSintRoute *sint_route); int kvm_hv_sint_route_set_sint(HvSintRoute *sint_route); +uint32_t hyperv_vp_index(X86CPU *cpu); +X86CPU *hyperv_find_vcpu(uint32_t vp_index); + #endif diff --git a/target/i386/hyperv.c b/target/i386/hyperv.c index a050c9d2d1..c9a4bb539d 100644 --- a/target/i386/hyperv.c +++ b/target/i386/hyperv.c @@ -16,6 +16,16 @@ #include "hyperv.h" #include "hyperv-proto.h" +uint32_t hyperv_vp_index(X86CPU *cpu) +{ + return CPU(cpu)->cpu_index; +} + +X86CPU *hyperv_find_vcpu(uint32_t vp_index) +{ + return X86_CPU(qemu_get_cpu(vp_index)); +} + int kvm_hv_handle_exit(X86CPU *cpu, struct kvm_hyperv_exit *exit) { CPUX86State *env = &cpu->env; @@ -72,7 +82,7 @@ static void kvm_hv_sint_ack_handler(EventNotifier *notifier) } } -HvSintRoute *kvm_hv_sint_route_create(uint32_t vcpu_id, uint32_t sint, +HvSintRoute *kvm_hv_sint_route_create(uint32_t vp_index, uint32_t sint, HvSintAckClb sint_ack_clb) { HvSintRoute *sint_route; @@ -92,7 +102,7 @@ HvSintRoute *kvm_hv_sint_route_create(uint32_t vcpu_id, uint32_t sint, event_notifier_set_handler(&sint_route->sint_ack_notifier, kvm_hv_sint_ack_handler); - gsi = kvm_irqchip_add_hv_sint_route(kvm_state, vcpu_id, sint); + gsi = kvm_irqchip_add_hv_sint_route(kvm_state, vp_index, sint); if (gsi < 0) { goto err_gsi; } @@ -105,7 +115,7 @@ HvSintRoute *kvm_hv_sint_route_create(uint32_t vcpu_id, uint32_t sint, } sint_route->gsi = gsi; sint_route->sint_ack_clb = sint_ack_clb; - sint_route->vcpu_id = vcpu_id; + sint_route->vcpu_id = vp_index; sint_route->sint = sint; return sint_route; diff --git a/target/i386/kvm.c b/target/i386/kvm.c index ad4b159b28..dfce60e5cf 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -86,6 +86,7 @@ static bool has_msr_hv_hypercall; static bool has_msr_hv_crash; static bool has_msr_hv_reset; static bool has_msr_hv_vpindex; +static bool is_hv_vpindex_settable; static bool has_msr_hv_runtime; static bool has_msr_hv_synic; static bool has_msr_hv_stimer; @@ -682,6 +683,43 @@ static int hyperv_handle_properties(CPUState *cs) return 0; } +static int hyperv_init_vcpu(X86CPU *cpu) +{ + if (cpu->hyperv_vpindex && !is_hv_vpindex_settable) { + /* + * the kernel doesn't support setting vp_index; assert that its value + * is in sync + */ + int ret; + struct { + struct kvm_msrs info; + struct kvm_msr_entry entries[1]; + } msr_data = { + .info.nmsrs = 1, + .entries[0].index = HV_X64_MSR_VP_INDEX, + }; + + /* + * handling errors from vcpu init at hotplug time is impossible, so + * disallow cpu hotplug + */ + MACHINE_GET_CLASS(current_machine)->hot_add_cpu = NULL; + + ret = kvm_vcpu_ioctl(CPU(cpu), KVM_GET_MSRS, &msr_data); + if (ret < 0) { + return ret; + } + assert(ret == 1); + + if (msr_data.entries[0].data != hyperv_vp_index(cpu)) { + fprintf(stderr, "kernel's vp_index != QEMU's vp_index\n"); + return -ENXIO; + } + } + + return 0; +} + static Error *invtsc_mig_blocker; #define KVM_MAX_CPUID_ENTRIES 100 @@ -1038,6 +1076,13 @@ int kvm_arch_init_vcpu(CPUState *cs) has_msr_tsc_aux = false; } + if (hyperv_enabled(cpu)) { + r = hyperv_init_vcpu(cpu); + if (r) { + goto fail; + } + } + return 0; fail: @@ -1220,6 +1265,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s) has_pit_state2 = kvm_check_extension(s, KVM_CAP_PIT_STATE2); #endif + is_hv_vpindex_settable = kvm_check_extension(s, KVM_CAP_HYPERV_VP_INDEX); + ret = kvm_get_supported_msrs(s); if (ret < 0) { return ret; @@ -1731,6 +1778,10 @@ static int kvm_put_msrs(X86CPU *cpu, int level) if (has_msr_hv_runtime) { kvm_msr_entry_add(cpu, HV_X64_MSR_VP_RUNTIME, env->msr_hv_runtime); } + if (cpu->hyperv_vpindex && has_msr_hv_vpindex && + is_hv_vpindex_settable) { + kvm_msr_entry_add(cpu, HV_X64_MSR_VP_INDEX, hyperv_vp_index(cpu)); + } if (cpu->hyperv_synic) { int j;