From patchwork Tue Aug 21 17:05:43 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Roth X-Patchwork-Id: 179132 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 1F5C22C00A6 for ; Wed, 22 Aug 2012 04:28:18 +1000 (EST) Received: from localhost ([::1]:43453 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1T3rvQ-0000c0-Ew for incoming@patchwork.ozlabs.org; Tue, 21 Aug 2012 13:07:32 -0400 Received: from eggs.gnu.org ([208.118.235.92]:45536) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1T3ruZ-0007So-5W for qemu-devel@nongnu.org; Tue, 21 Aug 2012 13:06:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1T3ruT-0002jm-AK for qemu-devel@nongnu.org; Tue, 21 Aug 2012 13:06:38 -0400 Received: from mail-yx0-f173.google.com ([209.85.213.173]:46780) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1T3ruT-0002gw-51 for qemu-devel@nongnu.org; Tue, 21 Aug 2012 13:06:33 -0400 Received: by mail-yx0-f173.google.com with SMTP id m4so32713yen.4 for ; Tue, 21 Aug 2012 10:06:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=UonKmjXDcwtV8S3L6nk0Cmjoejw/OUEwLOfca+yEza8=; b=HK+T35EWnqdwsGjuiPHJcWEmL4fNb9nCrVD89h+wBThGXuf3yjhV3YH96YPu6j/QIU k0epiykB2r8dp71GpQa1rP79Aoj6KSBduQDZimd5Jc6Ixm4nXXKLV36PJ+vBgygF6jsP dSmgxFxvyJ6AuTg85DEf+BdNK2Td5bwIgPh3oT8z5MwTKRVqGI10//mxEixkOX+lRZPP VCQNBMK5/mlZI5xpP0CM6N/HLA6n61hF6P/DiAgrzZQSaww+r76c48hiM33El1OkuSgH cGr4QUdAF7cLcKPXO51UVEWVzIagAlPZ2qarKEQgzz+9AOv/540xPbnbb1kLV2T7Lr+v gGMw== Received: by 10.50.157.196 with SMTP id wo4mr14487780igb.22.1345568792411; Tue, 21 Aug 2012 10:06:32 -0700 (PDT) Received: from loki.morrigu.org (cpe-72-179-62-111.austin.res.rr.com. [72.179.62.111]) by mx.google.com with ESMTPS id xm2sm3717964igb.3.2012.08.21.10.06.31 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 21 Aug 2012 10:06:31 -0700 (PDT) From: Michael Roth To: qemu-devel@nongnu.org Date: Tue, 21 Aug 2012 12:05:43 -0500 Message-Id: <1345568757-14365-10-git-send-email-mdroth@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1345568757-14365-1-git-send-email-mdroth@linux.vnet.ibm.com> References: <1345568757-14365-1-git-send-email-mdroth@linux.vnet.ibm.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 209.85.213.173 Cc: aliguori@us.ibm.com Subject: [Qemu-devel] [PATCH 09/23] ehci: don't flush cache on doorbell rings. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Gerd Hoffmann Commit 4be23939ab0d7019c7e59a37485b416fbbf0f073 makes ehci instantly zap any unlinked queue heads when the guest rings the doorbell. While hacking up uas support this turned out to be a problem. The linux kernel can unlink and instantly relink the very same queue head, thereby killing any async packets in flight. That alone isn't an issue yet, the packet will canceled and resubmitted and everything is fine. We'll run into trouble though in case the async packet is completed already, so we can't cancel it any more. The transaction is simply lost then. usb_ehci_qh_ptrs q (nil) - QH @ 39c4f000: next 39c4f122 qtds 00000000,00000001,39c50000 usb_ehci_qh_fields QH @ 39c4f000 - rl 0, mplen 0, eps 0, ep 0, dev 0 usb_ehci_qh_ptrs q 0x7f95feba90a0 - QH @ 39c4f000: next 39c4f122 qtds 00000000,00000001,39c50000 usb_ehci_qh_fields QH @ 39c4f000 - rl 0, mplen 0, eps 0, ep 0, dev 0 usb_ehci_qh_ptrs q 0x7f95fe515210 - QH @ 39c4f120: next 39c4f0c2 qtds 29dbce40,29dbc4e0,00000009 usb_ehci_qh_fields QH @ 39c4f120 - rl 4, mplen 512, eps 2, ep 1, dev 2 usb_ehci_packet_action q 0x7f95fe515210 p 0x7f95fdec32a0: alloc usb_packet_state_change bus 0, port 2, ep 1, packet 0x7f95fdec32e0, state undef -> setup usb_ehci_packet_action q 0x7f95fe515210 p 0x7f95fdec32a0: process usb_uas_command dev 2, tag 0x2, lun 0, lun64 00000000-00000000 scsi_req_parsed target 0 lun 0 tag 2 command 42 dir 2 length 16384 scsi_req_parsed_lba target 0 lun 0 tag 2 command 42 lba 5933312 scsi_req_alloc target 0 lun 0 tag 2 scsi_req_continue target 0 lun 0 tag 2 scsi_req_data target 0 lun 0 tag 2 len 16384 usb_uas_scsi_data dev 2, tag 0x2, bytes 16384 usb_uas_write_ready dev 2, tag 0x2 usb_packet_state_change bus 0, port 2, ep 1, packet 0x7f95fdec32e0, state setup -> complete usb_ehci_packet_action q 0x7f95fe515210 p 0x7f95fdec32a0: free usb_ehci_qh_ptrs q 0x7f95fdec3210 - QH @ 39c4f0c0: next 39c4f002 qtds 29dbce40,00000001,00000009 usb_ehci_qh_fields QH @ 39c4f0c0 - rl 4, mplen 512, eps 2, ep 2, dev 2 usb_ehci_queue_action q 0x7f95fe5152a0: free usb_packet_state_change bus 0, port 2, ep 2, packet 0x7f95feba9170, state async -> complete ^^^ async packets completes. usb_ehci_packet_action q 0x7f95fdec3210 p 0x7f95feba9130: wakeup usb_ehci_qh_ptrs q (nil) - QH @ 39c4f000: next 39c4f122 qtds 00000000,00000001,39c50000 usb_ehci_qh_fields QH @ 39c4f000 - rl 0, mplen 0, eps 0, ep 0, dev 0 usb_ehci_qh_ptrs q 0x7f95feba90a0 - QH @ 39c4f000: next 39c4f122 qtds 00000000,00000001,39c50000 usb_ehci_qh_fields QH @ 39c4f000 - rl 0, mplen 0, eps 0, ep 0, dev 0 usb_ehci_qh_ptrs q 0x7f95fe515210 - QH @ 39c4f120: next 39c4f002 qtds 29dbc4e0,29dbc8a0,00000009 usb_ehci_qh_fields QH @ 39c4f120 - rl 4, mplen 512, eps 2, ep 1, dev 2 usb_ehci_queue_action q 0x7f95fdec3210: free usb_ehci_packet_action q 0x7f95fdec3210 p 0x7f95feba9130: free ^^^ endpoint #2 queue head removed from schedule, doorbell makes ehci zap the queue, the (completed) usb packet is freed too and gets lost. usb_ehci_qh_ptrs q (nil) - QH @ 39c4f000: next 39c4f0c2 qtds 00000000,00000001,39c50000 usb_ehci_qh_fields QH @ 39c4f000 - rl 0, mplen 0, eps 0, ep 0, dev 0 usb_ehci_qh_ptrs q 0x7f95feba90a0 - QH @ 39c4f000: next 39c4f0c2 qtds 00000000,00000001,39c50000 usb_ehci_qh_fields QH @ 39c4f000 - rl 0, mplen 0, eps 0, ep 0, dev 0 usb_ehci_queue_action q 0x7f9600dff570: alloc usb_ehci_qh_ptrs q 0x7f9600dff570 - QH @ 39c4f0c0: next 39c4f122 qtds 29dbce40,00000001,00000009 usb_ehci_qh_fields QH @ 39c4f0c0 - rl 4, mplen 512, eps 2, ep 2, dev 2 usb_ehci_packet_action q 0x7f9600dff570 p 0x7f95feba9130: alloc usb_packet_state_change bus 0, port 2, ep 2, packet 0x7f95feba9170, state undef -> setup usb_ehci_packet_action q 0x7f9600dff570 p 0x7f95feba9130: process usb_packet_state_change bus 0, port 2, ep 2, packet 0x7f95feba9170, state setup -> async usb_ehci_packet_action q 0x7f9600dff570 p 0x7f95feba9130: async ^^^ linux kernel relinked the queue head, ehci creates a new usb packet, but we should have delivered the completed one instead. usb_ehci_qh_ptrs q 0x7f95fe515210 - QH @ 39c4f120: next 39c4f002 qtds 29dbc4e0,29dbc8a0,00000009 usb_ehci_qh_fields QH @ 39c4f120 - rl 4, mplen 512, eps 2, ep 1, dev 2 So instead of instantly zapping the queue we'll set a flag that the queue needs revalidation in case we'll see it again in the schedule. ehci then checks that the queue head fields addressing / describing the endpoint and the qtd pointer match the cached content before reusing it. Cc: Hans de Goede Signed-off-by: Gerd Hoffmann (cherry picked from commit 9bc3a3a216e2689bfcdd36c3e079333bbdbf3ba0) Conflicts: hw/usb/hcd-ehci.c Signed-off-by: Michael Roth --- hw/usb/hcd-ehci.c | 34 ++++++++++++++++++++++++++++------ 1 file changed, 28 insertions(+), 6 deletions(-) diff --git a/hw/usb/hcd-ehci.c b/hw/usb/hcd-ehci.c index 5855c5d..ce8b405 100644 --- a/hw/usb/hcd-ehci.c +++ b/hw/usb/hcd-ehci.c @@ -348,6 +348,7 @@ struct EHCIQueue { QTAILQ_ENTRY(EHCIQueue) next; uint32_t seen; uint64_t ts; + int revalidate; /* cached data from guest - needs to be flushed * when guest removes an entry (doorbell, handshake sequence) @@ -695,7 +696,18 @@ static EHCIQueue *ehci_find_queue_by_qh(EHCIState *ehci, uint32_t addr, return NULL; } -static void ehci_queues_rip_unused(EHCIState *ehci, int async, int flush) +static void ehci_queues_tag_unused_async(EHCIState *ehci) +{ + EHCIQueue *q; + + QTAILQ_FOREACH(q, &ehci->aqueues, next) { + if (!q->seen) { + q->revalidate = 1; + } + } +} + +static void ehci_queues_rip_unused(EHCIState *ehci, int async) { EHCIQueueHead *head = async ? &ehci->aqueues : &ehci->pqueues; EHCIQueue *q, *tmp; @@ -706,7 +718,7 @@ static void ehci_queues_rip_unused(EHCIState *ehci, int async, int flush) q->ts = ehci->last_run_ns; continue; } - if (!flush && ehci->last_run_ns < q->ts + 250000000) { + if (ehci->last_run_ns < q->ts + 250000000) { /* allow 0.25 sec idle */ continue; } @@ -1521,7 +1533,7 @@ static int ehci_state_waitlisthead(EHCIState *ehci, int async) ehci_set_usbsts(ehci, USBSTS_REC); } - ehci_queues_rip_unused(ehci, async, 0); + ehci_queues_rip_unused(ehci, async); /* Find the head of the list (4.9.1.1) */ for(i = 0; i < MAX_QH; i++) { @@ -1605,6 +1617,7 @@ static EHCIQueue *ehci_state_fetchqh(EHCIState *ehci, int async) { uint32_t entry; EHCIQueue *q; + EHCIqh qh; entry = ehci_get_fetch_addr(ehci, async); q = ehci_find_queue_by_qh(ehci, entry, async); @@ -1622,7 +1635,16 @@ static EHCIQueue *ehci_state_fetchqh(EHCIState *ehci, int async) } get_dwords(ehci, NLPTR_GET(q->qhaddr), - (uint32_t *) &q->qh, sizeof(EHCIqh) >> 2); + (uint32_t *) &qh, sizeof(EHCIqh) >> 2); + if (q->revalidate && (q->qh.epchar != qh.epchar || + q->qh.epcap != qh.epcap || + q->qh.current_qtd != qh.current_qtd)) { + ehci_free_queue(q, async); + q = ehci_alloc_queue(ehci, async); + q->seen++; + } + q->qh = qh; + q->revalidate = 0; ehci_trace_qh(q, NLPTR_GET(q->qhaddr), &q->qh); if (q->async == EHCI_ASYNC_INFLIGHT) { @@ -2047,7 +2069,7 @@ static void ehci_advance_async_state(EHCIState *ehci) */ if (ehci->usbcmd & USBCMD_IAAD) { /* Remove all unseen qhs from the async qhs queue */ - ehci_queues_rip_unused(ehci, async, 1); + ehci_queues_tag_unused_async(ehci); DPRINTF("ASYNC: doorbell request acknowledged\n"); ehci->usbcmd &= ~USBCMD_IAAD; ehci_set_interrupt(ehci, USBSTS_IAA); @@ -2102,7 +2124,7 @@ static void ehci_advance_periodic_state(EHCIState *ehci) ehci_set_fetch_addr(ehci, async,entry); ehci_set_state(ehci, async, EST_FETCHENTRY); ehci_advance_state(ehci, async); - ehci_queues_rip_unused(ehci, async, 0); + ehci_queues_rip_unused(ehci, async); break; default: