From patchwork Sat Aug 26 00:57:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krister Johansen X-Patchwork-Id: 1826323 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=templeofstupid.com header.i=@templeofstupid.com header.a=rsa-sha256 header.s=dreamhost header.b=oCjx4HrH; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RXdhw3t7kz1yg9 for ; Sat, 26 Aug 2023 10:57:52 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qZhca-0001EF-9I; Sat, 26 Aug 2023 00:57:48 +0000 Received: from caracal.birch.relay.mailchannels.net ([23.83.209.30]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qZhcX-0001DE-GE for kernel-team@lists.ubuntu.com; Sat, 26 Aug 2023 00:57:45 +0000 X-Sender-Id: dreamhost|x-authsender|kjlx@templeofstupid.com Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id C49B7C1051 for ; Sat, 26 Aug 2023 00:57:43 +0000 (UTC) Received: from pdx1-sub0-mail-a310.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 65261C115D for ; Sat, 26 Aug 2023 00:57:43 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1693011463; a=rsa-sha256; cv=none; b=QAcT5KqdSiRTj1w/dJcGE5ElEAhOD6/HZiGGzpcdDC1R39u6DwZKrDf4rFClyQBe4R3XSq Iv6yJnqHG89DX9gC0crNgZSCd8qRd349DFjudLR0k9JzQBAZghsuFOU+bTGqXhyxr1/rCH RKy4K+C4lh1Z0injjzcbsc84TybuSRmiTBinK/hHSDDOZHlCDdyQvvyRxALwY6/OmJ1pK/ JMTfboS9Y4IdqiTkcU7qVhqr5S7Oh5dJdkwvHgABk66iu75FsVpVizqXhJDsG3Dg3fkpBo uGaeVYi/Tbqh399k3aRDHi4EW07UQUK899yj25SYsXIhXXMlXLEnI1qlWAgz3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1693011463; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Z1oy/tLUqjJcxUA5GBOpXpl8KOAkwe3f0R79Gl3p4hM=; b=n7uVhDJ/lPOexOdBFvjkSlPS31xdlHMUViE/Bei8vXfMIeFTG6pxgwr+9dMG0yWXHWHDbn fziJGG0UjyZqAkCD8nRL/7boJTie/CBP36U5s+SxzNnNoUJIbE+97lIHKTYA9BfFaY6Z8o uK/XRzA8yFl4QMQ+Q32gXpsoIhK6P95h3fYXXxtAvNT0Kps0VWSRl1Wu323oBI5xmgWDnc q7GFQnWY3nAnOl64j4ZhUuaQn6D9DFaeAnd+QgHJu2BmPmv+DL3pBauv9RuMmWCcg0MapZ cQyVEBkS479IoteGgowbYCPJ16G969g1gyp8i0w+BQLT9JpLXs+sAstdE81pIA== ARC-Authentication-Results: i=1; rspamd-6fd95854bb-t5qs5; auth=pass smtp.auth=dreamhost smtp.mailfrom=kjlx@templeofstupid.com X-Sender-Id: dreamhost|x-authsender|kjlx@templeofstupid.com X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|kjlx@templeofstupid.com X-MailChannels-Auth-Id: dreamhost X-Trouble-Thoughtful: 541fff960d3275a8_1693011463633_1525973934 X-MC-Loop-Signature: 1693011463633:1735535754 X-MC-Ingress-Time: 1693011463632 Received: from pdx1-sub0-mail-a310.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.120.143.185 (trex/6.9.1); Sat, 26 Aug 2023 00:57:43 +0000 Received: from kmjvbox (c-73-231-176-24.hsd1.ca.comcast.net [73.231.176.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: kjlx@templeofstupid.com) by pdx1-sub0-mail-a310.dreamhost.com (Postfix) with ESMTPSA id 4RXdhl0yLRz2G for ; Fri, 25 Aug 2023 17:57:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=templeofstupid.com; s=dreamhost; t=1693011463; bh=Z1oy/tLUqjJcxUA5GBOpXpl8KOAkwe3f0R79Gl3p4hM=; h=Date:From:To:Cc:Subject:Content-Type; b=oCjx4HrHHvUUBLrsZ8XD6hvxfqQ+DgZ3xoOtKaF4XDVg1FMsAQqZVnESOvMHdtNLt WJ4MtOwS+9Ie1cnZuCa6oE7d73VIOzr8uGTfQrRzFYMqQ9w97LmHXGanGWsVgCp71+ cG4sSjN1VkaElq+UfDvGAI1xaeO8MtaQCqAlJrezNtv9l9jmKGW3VHoKwR1DW6waXs gXtcxfo+T52nlX2uBgzL0RNJgzcCcIsM4WRj4NLcEABeK5OmM7hB2nqYJXVuXpmi/j qLWOiUxkH3wA44d/Vv0s4JZ3iBktVVkvug3E6dsLoXdXH3xaamUi5CWfo2kEq8odkO 3tebHBtwCf/zw== Received: from johansen (uid 1000) (envelope-from kjlx@templeofstupid.com) id e009d by kmjvbox (DragonFly Mail Agent v0.12); Fri, 25 Aug 2023 17:57:30 -0700 Date: Fri, 25 Aug 2023 17:57:30 -0700 From: Krister Johansen To: kernel-team@lists.ubuntu.com Subject: [SRU][J/L][PATCH 1/1] x86/xen/time: prefer tsc as clocksource when it is invariant Message-ID: <470bb1be5bde7540d219748a77b0124ee8a0451a.1693009521.git.ubuntu@ip-172-31-11-177.us-west-2.compute.internal> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: David Reaver Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" BugLink: http://bugs.launchpad.net/bugs/2033122 Kvm elects to use tsc instead of kvm-clock when it can detect that the TSC is invariant. (As of commit 7539b174aef4 ("x86: kvmguest: use TSC clocksource if invariant TSC is exposed")). Notable cloud vendors[1] and performance engineers[2] recommend that Xen users preferentially select tsc over xen-clocksource due the performance penalty incurred by the latter. These articles are persuasive and tailored to specific use cases. In order to understand the tradeoffs around this choice more fully, this author had to reference the documented[3] complexities around the Xen configuration, as well as the kernel's clocksource selection algorithm. Many users may not attempt this to correctly configure the right clock source in their guest. The approach taken in the kvm-clock module spares users this confusion, where possible. Both the Intel SDM[4] and the Xen tsc documentation explain that marking a tsc as invariant means that it should be considered stable by the OS and is elibile to be used as a wall clock source. In order to obtain better out-of-the-box performance, and reduce the need for user tuning, follow kvm's approach and decrease the xen clock rating so that tsc is preferable, if it is invariant, stable, and the tsc will never be emulated. [1] https://aws.amazon.com/premiumsupport/knowledge-center/manage-ec2-linux-clock-source/ [2] https://www.brendangregg.com/blog/2021-09-26/the-speed-of-time.html [3] https://xenbits.xen.org/docs/unstable/man/xen-tscmode.7.html [4] Intel 64 and IA-32 Architectures Sofware Developer's Manual Volume 3b: System Programming Guide, Part 2, Section 17.17.1, Invariant TSC Signed-off-by: Krister Johansen Code-reviewed-by: David Reaver Reviewed-by: Juergen Gross Link: https://lore.kernel.org/r/20221216162118.GB2633@templeofstupid.com Signed-off-by: Juergen Gross (cherry picked from commit 99a7bcafbd0d04555074554573019096a8c10450) Signed-off-by: Krister Johansen Acked-by: Thadeu Lima de Souza Cascardo --- arch/x86/xen/time.c | 38 +++++++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index cc11dd2e2f48..062120dd8b98 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -474,15 +474,51 @@ static void xen_setup_vsyscall_time_info(void) xen_clocksource.vdso_clock_mode = VDSO_CLOCKMODE_PVCLOCK; } +/* + * Check if it is possible to safely use the tsc as a clocksource. This is + * only true if the hypervisor notifies the guest that its tsc is invariant, + * the tsc is stable, and the tsc instruction will never be emulated. + */ +static int __init xen_tsc_safe_clocksource(void) +{ + u32 eax, ebx, ecx, edx; + + if (!(boot_cpu_has(X86_FEATURE_CONSTANT_TSC))) + return 0; + + if (!(boot_cpu_has(X86_FEATURE_NONSTOP_TSC))) + return 0; + + if (check_tsc_unstable()) + return 0; + + /* Leaf 4, sub-leaf 0 (0x40000x03) */ + cpuid_count(xen_cpuid_base() + 3, 0, &eax, &ebx, &ecx, &edx); + + /* tsc_mode = no_emulate (2) */ + if (ebx != 2) + return 0; + + return 1; +} + static void __init xen_time_init(void) { struct pvclock_vcpu_time_info *pvti; int cpu = smp_processor_id(); struct timespec64 tp; - /* As Dom0 is never moved, no penalty on using TSC there */ + /* + * As Dom0 is never moved, no penalty on using TSC there. + * + * If it is possible for the guest to determine that the tsc is a safe + * clocksource, then set xen_clocksource rating below that of the tsc + * so that the system prefers tsc instead. + */ if (xen_initial_domain()) xen_clocksource.rating = 275; + else if (xen_tsc_safe_clocksource()) + xen_clocksource.rating = 299; clocksource_register_hz(&xen_clocksource, NSEC_PER_SEC);