From patchwork Fri Jul 23 18:14:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Guilherme G. Piccoli" X-Patchwork-Id: 1509319 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=UfBGH7CD; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GWctV3gp9z9snk; Sat, 24 Jul 2021 04:15:14 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1m6zhQ-0006Yz-Uo; Fri, 23 Jul 2021 18:15:04 +0000 Received: from smtp-relay-canonical-1.internal ([10.131.114.174] helo=smtp-relay-canonical-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1m6zhO-0006Yi-Bg for kernel-team@lists.ubuntu.com; Fri, 23 Jul 2021 18:15:02 +0000 Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPS id 0D16D3F228 for ; Fri, 23 Jul 2021 18:15:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1627064102; bh=AstWegVE8QQJnQxkNv8G44HpJbII9DSMAccYXnxlfmY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UfBGH7CDBSkM9tNEVfPjdb9SvXhYBzDgVFZz9axVDghPDTJC5OJvOJEN2B39fXjzB GEM1QJ+VnXx9fX8vnWiCbQmcwEvPrbkn8wEbQDYALcEmVO0CI0h4ND229CU9neUNW5 93JBazAyqTXeE81yjgqbZ4cvGUgQiDg28VLpip8N/hRlUcJxUe2XbXs7t32uh0t+mJ mOQqiDqy5BSOAeluFAva1nbt+F5otVZIjYWdSvfkM+KhdBRUVyyln85psQKcT0tftC A0KdvvhDJj0j9qBaXhbc2BjwNe4AFNKDIqzkUsLGNRBzi1lSYm56l9SnyO25Y0zHkA OgxaRDzlBtJ1Q== Received: by mail-pj1-f71.google.com with SMTP id z15-20020a17090a468fb0290175f5313b8dso2561431pjf.8 for ; Fri, 23 Jul 2021 11:15:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AstWegVE8QQJnQxkNv8G44HpJbII9DSMAccYXnxlfmY=; b=Kja9Y4OS2wIufBILLZ6icsahG5PFUbLVKF0HttIpyC9SbTW0ZHN46/+cBUOOmxalxw Qjn3zUhgtqd6Qv6Xzy5c1yo0kW/nTGxi3Q/1siiA3TqCaJs3OV7kYJGBE5X8857JaUPX ex7c+39jR1wA0LU8e9cnG45vytQvfYspJK1ydzrhydWXuxdltaSqM7CKk+SEoBdHKpnh i0TCFkzmW4z5wQf0q0xilVZG5heshYSo2sI+RDRyXqm/THGnAuyFERflyDRKSE5FV5v+ PJZvUVHAvz9n4nPJMCcuX6/Eavz6rYKQKyyNThGbpmLSbX1k+KBWSZ2G2TuPy7K+rVDc 0OtA== X-Gm-Message-State: AOAM532xvdNnbWaeuNaXVDlCylY5dbnyZ+zysQiclBnoq8yFSKKQr4q/ k0ySXKe0YFHkdqKNJjAMt/BGnT/eN68TCzwiyebiBH5lgDrIGlTXD3QPETYarUgdJLNLhs8o0HJ D41XXhW5keeJqNWu3/tPBlbrQhRlPA5Dhyqdnt2pI6w== X-Received: by 2002:a05:6a00:1889:b029:332:13d6:a6eb with SMTP id x9-20020a056a001889b029033213d6a6ebmr5540296pfh.25.1627064100554; Fri, 23 Jul 2021 11:15:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJywz7f3+LPxNHwoYxuUA110UzX0n/A+AvkZn64wj8UQSVx32iufX81XXqJA/AGZ2kW4FscgpA== X-Received: by 2002:a05:6a00:1889:b029:332:13d6:a6eb with SMTP id x9-20020a056a001889b029033213d6a6ebmr5540275pfh.25.1627064100207; Fri, 23 Jul 2021 11:15:00 -0700 (PDT) Received: from localhost ([187.183.41.59]) by smtp.gmail.com with ESMTPSA id x40sm35403905pfu.176.2021.07.23.11.14.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Jul 2021 11:14:59 -0700 (PDT) From: "Guilherme G. Piccoli" To: kernel-team@lists.ubuntu.com Subject: [bionic:linux-azure][PATCH 1/1] Drivers: hv: vmbus: Implement Direct Mode for stimer0 Date: Fri, 23 Jul 2021 15:14:53 -0300 Message-Id: <20210723181453.18987-2-gpiccoli@canonical.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210723181453.18987-1-gpiccoli@canonical.com> References: <20210723181453.18987-1-gpiccoli@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel@gpiccoli.net Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Michael Kelley BugLink: https://bugs.launchpad.net/bugs/1882623 The 2016 version of Hyper-V offers the option to operate the guest VM per-vcpu stimer's in Direct Mode, which means the timer interupts on its own vector rather than queueing a VMbus message. Direct Mode reduces timer processing overhead in both the hypervisor and the guest, and avoids having timer interrupts pollute the VMbus interrupt stream for the synthetic NIC and storage. This patch enables Direct Mode by default on stimer0 when running on a version of Hyper-V that supports it. In prep for coming support of Hyper-V on ARM64, the arch independent portion of the code contains calls to routines that will be populated on ARM64 but are not needed and do nothing on x86. Signed-off-by: Michael Kelley Signed-off-by: K. Y. Srinivasan Signed-off-by: Greg Kroah-Hartman (backported from commit 248e742a396e7f00b283f1c56e14b1bef6e3ec56) [gpiccoli: context adjustments, specially due to the lack of the following commits in our kernel: - commit 51d4e5daa328 ("x86/irq: Count Hyper-V reenlightenment interrupts") - commit 5a4858032217 ("x86/hyper-v: move hyperv.h out of uapi") Note that the hyperv.h uapi file changed to hyperv-tlfs.h not in uapi.] Signed-off-by: Guilherme G. Piccoli Acked-by: Krzysztof Kozlowski --- arch/x86/entry/entry_32.S | 3 ++ arch/x86/entry/entry_64.S | 3 ++ arch/x86/include/asm/hardirq.h | 3 ++ arch/x86/include/asm/hyperv-tlfs.h | 3 ++ arch/x86/include/asm/irq_vectors.h | 3 +- arch/x86/include/asm/mshyperv.h | 13 +++++++ arch/x86/kernel/cpu/mshyperv.c | 40 ++++++++++++++++++++ arch/x86/kernel/irq.c | 7 ++++ drivers/hv/hv.c | 59 ++++++++++++++++++++++++++++-- drivers/hv/hyperv_vmbus.h | 4 +- 10 files changed, 133 insertions(+), 5 deletions(-) diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S index 4fe66b264268..6272460d5218 100644 --- a/arch/x86/entry/entry_32.S +++ b/arch/x86/entry/entry_32.S @@ -1333,6 +1333,9 @@ BUILD_INTERRUPT3(hyperv_callback_vector, HYPERVISOR_CALLBACK_VECTOR, BUILD_INTERRUPT3(hyperv_reenlightenment_vector, HYPERV_REENLIGHTENMENT_VECTOR, hyperv_reenlightenment_intr) +BUILD_INTERRUPT3(hv_stimer0_callback_vector, HYPERV_STIMER0_VECTOR, + hv_stimer0_vector_handler) + #endif /* CONFIG_HYPERV */ ENTRY(page_fault) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index f82461190afa..47b009e8bf4e 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -1118,6 +1118,9 @@ apicinterrupt3 HYPERVISOR_CALLBACK_VECTOR \ apicinterrupt3 HYPERV_REENLIGHTENMENT_VECTOR \ hyperv_reenlightenment_vector hyperv_reenlightenment_intr + +apicinterrupt3 HYPERV_STIMER0_VECTOR \ + hv_stimer0_callback_vector hv_stimer0_vector_handler #endif /* CONFIG_HYPERV */ idtentry debug do_debug has_error_code=0 paranoid=1 shift_ist=DEBUG_STACK diff --git a/arch/x86/include/asm/hardirq.h b/arch/x86/include/asm/hardirq.h index 486c843273c4..232520467ee3 100644 --- a/arch/x86/include/asm/hardirq.h +++ b/arch/x86/include/asm/hardirq.h @@ -40,6 +40,9 @@ typedef struct { #if IS_ENABLED(CONFIG_HYPERV) || defined(CONFIG_XEN) unsigned int irq_hv_callback_count; #endif +#if IS_ENABLED(CONFIG_HYPERV) + unsigned int hyperv_stimer0_count; +#endif } ____cacheline_aligned irq_cpustat_t; DECLARE_PER_CPU_SHARED_ALIGNED(irq_cpustat_t, irq_stat); diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h index 032903ee4164..1e2313d2a3e1 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -86,6 +86,9 @@ /* Crash MSR available */ #define HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE (1 << 10) +/* stimer Direct Mode is available */ +#define HV_X64_STIMER_DIRECT_MODE_AVAILABLE (1 << 19) + /* * Feature identification: EBX indicates which flags were specified at * partition creation. The format is the same as the partition creation diff --git a/arch/x86/include/asm/irq_vectors.h b/arch/x86/include/asm/irq_vectors.h index e71c1120426b..404c5fdff859 100644 --- a/arch/x86/include/asm/irq_vectors.h +++ b/arch/x86/include/asm/irq_vectors.h @@ -106,9 +106,10 @@ #if IS_ENABLED(CONFIG_HYPERV) #define HYPERV_REENLIGHTENMENT_VECTOR 0xee +#define HYPERV_STIMER0_VECTOR 0xed #endif -#define LOCAL_TIMER_VECTOR 0xed +#define LOCAL_TIMER_VECTOR 0xec #define NR_VECTORS 256 diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index 1fbee89cd9ac..4e1c5c431c5c 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -106,6 +106,19 @@ void hv_remove_kexec_handler(void); void hv_setup_crash_handler(void (*handler)(struct pt_regs *regs)); void hv_remove_crash_handler(void); +/* + * Routines for stimer0 Direct Mode handling. + * On x86/x64, there are no percpu actions to take. + */ +void hv_stimer0_vector_handler(struct pt_regs *regs); +void hv_stimer0_callback_vector(void); +int hv_setup_stimer0_irq(int *irq, int *vector, void (*handler)(void)); +void hv_remove_stimer0_irq(int irq); + +static inline void hv_enable_stimer0_percpu_irq(int irq) {} +static inline void hv_disable_stimer0_percpu_irq(int irq) {} + + #if IS_ENABLED(CONFIG_HYPERV) extern struct clocksource *hyperv_cs; extern void *hv_hypercall_pg; diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 4b3e38923884..89d86ff64b6f 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -38,6 +38,7 @@ EXPORT_SYMBOL_GPL(ms_hyperv); #if IS_ENABLED(CONFIG_HYPERV) static void (*vmbus_handler)(void); +static void (*hv_stimer0_handler)(void); static void (*hv_kexec_handler)(void); static void (*hv_crash_handler)(struct pt_regs *regs); @@ -70,6 +71,41 @@ void hv_remove_vmbus_irq(void) EXPORT_SYMBOL_GPL(hv_setup_vmbus_irq); EXPORT_SYMBOL_GPL(hv_remove_vmbus_irq); +/* + * Routines to do per-architecture handling of stimer0 + * interrupts when in Direct Mode + */ + +__visible void __irq_entry hv_stimer0_vector_handler(struct pt_regs *regs) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + + entering_irq(); + inc_irq_stat(hyperv_stimer0_count); + if (hv_stimer0_handler) + hv_stimer0_handler(); + ack_APIC_irq(); + + exiting_irq(); + set_irq_regs(old_regs); +} + +int hv_setup_stimer0_irq(int *irq, int *vector, void (*handler)(void)) +{ + *vector = HYPERV_STIMER0_VECTOR; + *irq = 0; /* Unused on x86/x64 */ + hv_stimer0_handler = handler; + return 0; +} +EXPORT_SYMBOL_GPL(hv_setup_stimer0_irq); + +void hv_remove_stimer0_irq(int irq) +{ + /* We have no way to deallocate the interrupt gate */ + hv_stimer0_handler = NULL; +} +EXPORT_SYMBOL_GPL(hv_remove_stimer0_irq); + void hv_setup_kexec_handler(void (*handler)(void)) { hv_kexec_handler = handler; @@ -284,6 +320,10 @@ static void __init ms_hyperv_init_platform(void) alloc_intr_gate(HYPERV_REENLIGHTENMENT_VECTOR, hyperv_reenlightenment_vector); + /* Setup the IDT for stimer0 */ + if (ms_hyperv.misc_features & HV_X64_STIMER_DIRECT_MODE_AVAILABLE) + alloc_intr_gate(HYPERV_STIMER0_VECTOR, + hv_stimer0_callback_vector); #endif } diff --git a/arch/x86/kernel/irq.c b/arch/x86/kernel/irq.c index e8312828d2c2..81592f0dddcc 100644 --- a/arch/x86/kernel/irq.c +++ b/arch/x86/kernel/irq.c @@ -142,6 +142,13 @@ int arch_show_interrupts(struct seq_file *p, int prec) irq_stats(j)->irq_hv_callback_count); seq_puts(p, " Hypervisor callback interrupts\n"); } + if (test_bit(HYPERV_STIMER0_VECTOR, system_vectors)) { + seq_printf(p, "%*s: ", prec, "HVS"); + for_each_online_cpu(j) + seq_printf(p, "%10u ", + irq_stats(j)->hyperv_stimer0_count); + seq_puts(p, " Hyper-V stimer0 interrupts\n"); + } #endif seq_printf(p, "%*s: %10u\n", prec, "ERR", atomic_read(&irq_err_count)); #if defined(CONFIG_X86_IO_APIC) diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c index 953e10999539..9a9b2e4fb8a5 100644 --- a/drivers/hv/hv.c +++ b/drivers/hv/hv.c @@ -27,7 +27,7 @@ #include #include #include -#include +#include #include #include #include "hyperv_vmbus.h" @@ -37,6 +37,17 @@ struct hv_context hv_context = { .synic_initialized = false, }; +/* + * If false, we're using the old mechanism for stimer0 interrupts + * where it sends a VMbus message when it expires. The old + * mechanism is used when running on older versions of Hyper-V + * that don't support Direct Mode. While Hyper-V provides + * four stimer's per CPU, Linux uses only stimer0. + */ +static bool direct_mode_enabled; +static int stimer0_irq; +static int stimer0_vector; + #define HV_TIMER_FREQUENCY (10 * 1000 * 1000) /* 100ns period */ #define HV_MAX_MAX_DELTA_TICKS 0xffffffff #define HV_MIN_DELTA_TICKS 1 @@ -52,6 +63,8 @@ int hv_init(void) if (!hv_context.cpu_context) return -ENOMEM; + direct_mode_enabled = ms_hyperv.misc_features & + HV_X64_STIMER_DIRECT_MODE_AVAILABLE; return 0; } @@ -90,6 +103,21 @@ int hv_post_message(union hv_connection_id connection_id, return status & 0xFFFF; } +/* + * ISR for when stimer0 is operating in Direct Mode. Direct Mode + * does not use VMbus or any VMbus messages, so process here and not + * in the VMbus driver code. + */ + +static void hv_stimer0_isr(void) +{ + struct hv_per_cpu_context *hv_cpu; + + hv_cpu = this_cpu_ptr(hv_context.cpu_context); + hv_cpu->clk_evt->event_handler(hv_cpu->clk_evt); + add_interrupt_randomness(stimer0_vector, 0); +} + static int hv_ce_set_next_event(unsigned long delta, struct clock_event_device *evt) { @@ -107,6 +135,8 @@ static int hv_ce_shutdown(struct clock_event_device *evt) { hv_init_timer(HV_X64_MSR_STIMER0_COUNT, 0); hv_init_timer_config(HV_X64_MSR_STIMER0_CONFIG, 0); + if (direct_mode_enabled) + hv_disable_stimer0_percpu_irq(stimer0_irq); return 0; } @@ -115,11 +145,26 @@ static int hv_ce_set_oneshot(struct clock_event_device *evt) { union hv_timer_config timer_cfg; + timer_cfg.as_uint64 = 0; timer_cfg.enable = 1; timer_cfg.auto_enable = 1; - timer_cfg.sintx = VMBUS_MESSAGE_SINT; + if (direct_mode_enabled) { + /* + * When it expires, the timer will directly interrupt + * on the specified hardware vector/IRQ. + */ + timer_cfg.direct_mode = 1; + timer_cfg.apic_vector = stimer0_vector; + hv_enable_stimer0_percpu_irq(stimer0_irq); + } else { + /* + * When it expires, the timer will generate a VMbus message, + * to be handled by the normal VMbus interrupt handler. + */ + timer_cfg.direct_mode = 0; + timer_cfg.sintx = VMBUS_MESSAGE_SINT; + } hv_init_timer_config(HV_X64_MSR_STIMER0_CONFIG, timer_cfg.as_uint64); - return 0; } @@ -199,6 +244,11 @@ int hv_synic_alloc(void) INIT_LIST_HEAD(&hv_cpu->chan_list); } + if (direct_mode_enabled && + hv_setup_stimer0_irq(&stimer0_irq, &stimer0_vector, + hv_stimer0_isr)) + goto err; + return 0; err: /* @@ -302,6 +352,9 @@ void hv_synic_clockevents_cleanup(void) if (!(ms_hyperv.features & HV_X64_MSR_SYNTIMER_AVAILABLE)) return; + if (direct_mode_enabled) + hv_remove_stimer0_irq(stimer0_irq); + for_each_present_cpu(cpu) { struct hv_per_cpu_context *hv_cpu = per_cpu_ptr(hv_context.cpu_context, cpu); diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h index c3ea539587db..5038e51acdf0 100644 --- a/drivers/hv/hyperv_vmbus.h +++ b/drivers/hv/hyperv_vmbus.h @@ -58,7 +58,9 @@ union hv_timer_config { u64 periodic:1; u64 lazy:1; u64 auto_enable:1; - u64 reserved_z0:12; + u64 apic_vector:8; + u64 direct_mode:1; + u64 reserved_z0:3; u64 sintx:4; u64 reserved_z1:44; };