From patchwork Mon Jul 7 12:27:21 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 367541 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 36452140086 for ; Mon, 7 Jul 2014 22:39:47 +1000 (EST) Received: from localhost ([::1]:49898 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X48Cv-0004St-Fq for incoming@patchwork.ozlabs.org; Mon, 07 Jul 2014 08:39:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51642) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X484i-0007H2-62 for qemu-devel@nongnu.org; Mon, 07 Jul 2014 08:31:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1X484c-00063b-7S for qemu-devel@nongnu.org; Mon, 07 Jul 2014 08:31:16 -0400 Received: from mail-wi0-f181.google.com ([209.85.212.181]:64396) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X484b-00063F-SV for qemu-devel@nongnu.org; Mon, 07 Jul 2014 08:31:10 -0400 Received: by mail-wi0-f181.google.com with SMTP id n3so6775072wiv.8 for ; Mon, 07 Jul 2014 05:31:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9yInX6SNON+AeroRlrpGCTQxS4qmBuB5xvlB9KtD26k=; b=Wba4ITjVA55hvyT9m4dJ0pMAwz+ddKxk43Ur1iA8013C2AtXXylCsOjHUGFHgHpw6N u9M679sHet3ZkTzrFEc1UF8fiwuccufxqhJIGEvkhM682iiisVpxxseVwFbFZnRTButw Qx31jyWtHbQw1HDN5RUvgOld5a7SoclJ9WVjd7fLxFIGU9M+PXuiOlmgWY6kk2wUpWAm iADIft9JGksTmyKiUCOAoJCXrNeijGlRak3ecEsDYLgn9Cm+x9Df5gxWoExjB1q1vMnG kwavlgRyuS9wqsxOY6bXLUb48Ao+lMASGDL+J1u6ShkJaGFW/TsC+8g/F0ZuNGdbIBrY 85dg== X-Gm-Message-State: ALoCoQnECacFraslg4VVBDrN+5bR/XEXH1ZR7lfj6qZZ3WSykfM/9OQtFjmAapqNolPwjoR9PTZ3 X-Received: by 10.180.14.33 with SMTP id m1mr52160328wic.50.1404736269053; Mon, 07 Jul 2014 05:31:09 -0700 (PDT) Received: from midway01-04-00.lavalab ([88.98.47.97]) by mx.google.com with ESMTPSA id ec8sm95805182wic.10.2014.07.07.05.31.08 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Jul 2014 05:31:08 -0700 (PDT) From: Eric Auger To: eric.auger@st.com, christoffer.dall@linaro.org, qemu-devel@nongnu.org, kim.phillips@freescale.com, a.rigo@virtualopensystems.com Date: Mon, 7 Jul 2014 13:27:21 +0100 Message-Id: <1404736043-22900-12-git-send-email-eric.auger@linaro.org> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1404736043-22900-1-git-send-email-eric.auger@linaro.org> References: <1404736043-22900-1-git-send-email-eric.auger@linaro.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.85.212.181 Cc: peter.maydell@linaro.org, eric.auger@linaro.org, patches@linaro.org, will.deacon@arm.com, agraf@suse.de, stuart.yoder@freescale.com, Bharat.Bhushan@freescale.com, alex.williamson@redhat.com, a.motakis@virtualopensystems.com, kvmarm@lists.cs.columbia.edu Subject: [Qemu-devel] [RFC v4 11/13] hw/vfio/platform: Add irqfd support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This patch aims at optimizing IRQ handling using irqfd framework. Instead of handling the eventfds on user-side they are handled on kernel side using - the KVM irqfd framework, - the VFIO driver virqfd framework. the virtual IRQ completion is trapped at interrupt controller instead of on guest 1st access to any region after IRQ hit. This removes the need for fast/slow path swap. Overall this brings significant performance improvements. It depends on host kernel KVM irqfd/GSI routing capability. Signed-off-by: Alvise Rigo Signed-off-by: Eric Auger --- v3 -> v4: [Alvise Rigo] Use of VFIO Platform driver v6 unmask/virqfd feature and removal of resamplefd handler. Physical IRQ unmasking is now done in VFIO driver. v3: [Eric Auger] initial support with resamplefd handled on QEMU side since the unmask was not supported on VFIO platform driver v5. --- hw/vfio/platform.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) diff --git a/hw/vfio/platform.c b/hw/vfio/platform.c index a5fc22b..fb0f7c9 100644 --- a/hw/vfio/platform.c +++ b/hw/vfio/platform.c @@ -381,6 +381,101 @@ static int vfio_populate_interrupts(VFIODevice *vbasedev) return 0; } +static void vfio_enable_intp_kvm(VFIOINTp *intp) +{ +#ifdef CONFIG_KVM + struct kvm_irqfd irqfd = { + .fd = event_notifier_get_fd(&intp->interrupt), + .gsi = intp->virtualID, + .flags = KVM_IRQFD_FLAG_RESAMPLE, + }; + + struct vfio_irq_set *irq_set; + int ret, argsz; + int32_t *pfd; + VFIODevice *vbasedev = &intp->vdev->vbasedev; + + if (!kvm_irqfds_enabled() || + !kvm_check_extension(kvm_state, KVM_CAP_IRQFD_RESAMPLE)) { + return; + } + + /* Get to a known interrupt state */ + qemu_set_fd_handler(irqfd.fd, NULL, NULL, NULL); + vfio_mask_irqindex(vbasedev, intp->pin); + intp->state = VFIO_IRQ_INACTIVE; + qemu_set_irq(intp->qemuirq, 0); + + /* Get an eventfd for resample/unmask */ + if (event_notifier_init(&intp->unmask, 0)) { + error_report("vfio: Error: event_notifier_init failed eoi"); + goto fail; + } + + /* KVM triggers it, VFIO listens for it */ + irqfd.resamplefd = event_notifier_get_fd(&intp->unmask); + + if (kvm_vm_ioctl(kvm_state, KVM_IRQFD, &irqfd)) { + error_report("vfio: Error: Failed to setup resample irqfd: %m"); + goto fail_irqfd; + } + + argsz = sizeof(*irq_set) + sizeof(*pfd); + + irq_set = g_malloc0(argsz); + irq_set->argsz = argsz; + irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_UNMASK; + irq_set->index = intp->pin; + irq_set->start = 0; + irq_set->count = 1; + pfd = (int32_t *)&irq_set->data; + + *pfd = irqfd.resamplefd; + + ret = ioctl(vbasedev->fd, VFIO_DEVICE_SET_IRQS, irq_set); + g_free(irq_set); + if (ret) { + error_report("vfio: Error: Failed to setup INTx unmask fd: %m"); + goto fail_vfio; + } + + /* Let'em rip */ + vfio_unmask_irqindex(vbasedev, intp->pin); + + intp->kvm_accel = true; + + DPRINTF("%s irqfd pin=%d to virtID = %d fd=%d, resamplefd=%d)\n", + __func__, intp->pin, intp->virtualID, + irqfd.fd, irqfd.resamplefd); + + return; + +fail_vfio: + irqfd.flags = KVM_IRQFD_FLAG_DEASSIGN; + kvm_vm_ioctl(kvm_state, KVM_IRQFD, &irqfd); +fail_irqfd: + event_notifier_cleanup(&intp->unmask); +fail: + qemu_set_fd_handler(irqfd.fd, vfio_intp_interrupt, NULL, intp); + vfio_unmask_irqindex(vbasedev, intp->pin); +#endif +} + +void vfio_setup_irqfd(SysBusDevice *s, int index, int virq) +{ + VFIOPlatformDevice *vdev = container_of(s, VFIOPlatformDevice, sbdev); + VFIOINTp *intp; + + QLIST_FOREACH(intp, &vdev->intp_list, next) { + if (intp->pin == index) { + intp->virtualID = virq; + DPRINTF("enable irqfd for irq index %d (virtual IRQ %d)\n", + index, virq); + vfio_enable_intp_kvm(intp); + } + } +} + static VFIODeviceOps vfio_platform_ops = { .vfio_compute_needs_reset = vfio_platform_compute_needs_reset, .vfio_hot_reset_multi = vfio_platform_hot_reset_multi,