From patchwork Mon Dec 5 09:08:42 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 129251 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [140.186.70.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id A03BC1007D6 for ; Mon, 5 Dec 2011 20:07:24 +1100 (EST) Received: from localhost ([::1]:42544 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RXUW1-000467-AM for incoming@patchwork.ozlabs.org; Mon, 05 Dec 2011 04:07:13 -0500 Received: from eggs.gnu.org ([140.186.70.92]:54729) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RXUVu-00045t-52 for qemu-devel@nongnu.org; Mon, 05 Dec 2011 04:07:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RXUVs-0002Nf-N6 for qemu-devel@nongnu.org; Mon, 05 Dec 2011 04:07:06 -0500 Received: from mx1.redhat.com ([209.132.183.28]:23870) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RXUVs-0002NM-Ea for qemu-devel@nongnu.org; Mon, 05 Dec 2011 04:07:04 -0500 Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id pB59716b027339 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Mon, 5 Dec 2011 04:07:01 -0500 Received: from redhat.com (reserved-201-250.tlv.redhat.com [10.35.201.250] (may be forged)) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with SMTP id pB596v2J032651; Mon, 5 Dec 2011 04:06:58 -0500 Date: Mon, 5 Dec 2011 11:08:42 +0200 From: "Michael S. Tsirkin" To: Cam Macdonell Message-ID: <20111205090841.GA10526@redhat.com> References: <27ef57671106d9214757df8a1d4cc28287dae07c.1321893802.git.mst@redhat.com> <20111204102036.GB15464@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 3) X-Received-From: 209.132.183.28 Cc: Blue Swirl , Jan Kiszka , Anthony Liguori , qemu-devel@nongnu.org, Alexander Graf Subject: Re: [Qemu-devel] [PATCH for v1.0 1/3] msix: track function masked in pci device state X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org On Sun, Dec 04, 2011 at 04:47:17PM -0700, Cam Macdonell wrote: > On Sun, Dec 4, 2011 at 3:20 AM, Michael S. Tsirkin wrote: > > On Fri, Dec 02, 2011 at 04:34:21PM -0700, Cam Macdonell wrote: > >> Based on a git bisect, this patch breaks msi-x interrupt delivery in > >> the ivshmem device. > > > > I think the following should fix it. Compiled-only - > > could you pls check? If yes let's apply to the stable branch. > > Thanks for the patch Michael. > > It addresses the need for msix_write_config() to be called, but the > addition of the msix_reset() is causing a reset of the vectors after > they've been initialized in pci_ivshmem_init(). So, interrupts still > aren't delivered with this patch applied as it is. > > In particular, a reset occurs after pci_ivshmem_init runs, so the > msix_entry_used array is reset to 0s, which causes the interrupt > delivery to fail. > > If I comment out the msix_reset(), then interrupts are delivered. > Would the reset be caused by a bug in the guest driver? or do I need > to reconfigure the msix after reset? I'm unclear as to the proper > behaviour after a reset. > > Thanks, > Cam > I didn't anticipate this, virtio only configures vectors when msi-x is enabled. For 1.0 it's safest to do the same and just re-enable after reset. So we'll end up with something like the following - does it help? Side note: exit(1) is not the best way to handle errors in the init function. I think you should add error_report, then goto err on failures to init, cleanup what you have set up meanwhile and return an error code. diff --git a/hw/ivshmem.c b/hw/ivshmem.c index 242fbea..c58f4d3 100644 --- a/hw/ivshmem.c +++ b/hw/ivshmem.c @@ -500,11 +500,29 @@ static void ivshmem_read(void *opaque, const uint8_t * buf, int flags) return; } +/* Select the MSI-X vectors used by device. + * ivshmem maps events to vectors statically, so + * we just enable all vectors on init and after reset. */ +static void ivshmem_use_msix(IVShmemState * s) +{ + int i; + + if (!msix_present(&s->dev)) { + return; + } + + for (i = 0; i < s->vectors; i++) { + msix_vector_use(&s->dev, i); + } +} + static void ivshmem_reset(DeviceState *d) { IVShmemState *s = DO_UPCAST(IVShmemState, dev.qdev, d); s->intrstatus = 0; + msix_reset(&s->dev); + ivshmem_use_msix(s); return; } @@ -535,12 +553,8 @@ static uint64_t ivshmem_get_size(IVShmemState * s) { return value; } -static void ivshmem_setup_msi(IVShmemState * s) { - - int i; - - /* allocate the MSI-X vectors */ - +static void ivshmem_setup_msi(IVShmemState * s) +{ memory_region_init(&s->msix_bar, "ivshmem-msix", 4096); if (!msix_init(&s->dev, s->vectors, &s->msix_bar, 1, 0)) { pci_register_bar(&s->dev, 1, PCI_BASE_ADDRESS_SPACE_MEMORY, @@ -551,13 +565,10 @@ static void ivshmem_setup_msi(IVShmemState * s) { exit(1); } - /* 'activate' the vectors */ - for (i = 0; i < s->vectors; i++) { - msix_vector_use(&s->dev, i); - } - /* allocate Qemu char devices for receiving interrupts */ s->eventfd_table = g_malloc0(s->vectors * sizeof(EventfdEntry)); + + ivshmem_use_msix(s); } static void ivshmem_save(QEMUFile* f, void *opaque) @@ -610,6 +621,13 @@ static int ivshmem_load(QEMUFile* f, void *opaque, int version_id) return 0; } +static void ivshmem_write_config(PCIDevice *pci_dev, uint32_t address, + uint32_t val, int len) +{ + pci_default_write_config(pci_dev, address, val, len); + msix_write_config(pci_dev, address, val, len); +} + static int pci_ivshmem_init(PCIDevice *dev) { IVShmemState *s = DO_UPCAST(IVShmemState, dev, dev); @@ -734,6 +752,8 @@ static int pci_ivshmem_init(PCIDevice *dev) } + s->dev.config_write = ivshmem_write_config; + return 0; }