[{"id":1754973,"web_url":"http://patchwork.ozlabs.org/comment/1754973/","msgid":"<2131ff31-09f8-6955-502d-a3fce031c31e@free.fr>","list_archive_url":null,"date":"2017-08-23T09:18:36","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":66150,"url":"http://patchwork.ozlabs.org/api/people/66150/","name":"Mason","email":"slash.tmp@free.fr"},"content":"On 23/08/2017 09:51, Mathias Nyman wrote:\n\n> On 23.08.2017 09:07, Felipe Balbi wrote:\n>\n>> Mason writes:\n>>\n>>> Any idea what could have changed between 4.9 and 4.13 ?\n>>\n>> Quite a bit:\n>>\n>> $ git rev-list --no-merges --count v4.13-rc6 ^v4.9 -- drivers/usb/host/xhci drivers/usb/core/\n>> 58\n> \n> very likely cause is the more aggressive detection of pci removed xhci hosts\n> \n> See commit d9f11ba9f107aa335091ab8d7ba5eea714e46e8b\n> xhci: Rework how we handle unresponsive or hoptlug removed hosts\n> \n> It checks if a xhci register reads returns 0xffffffff and assumes xhci\n> died in that case.\n> \n> Could you add something like the below to check which what is killing the host?\n> Or a BUG()/WARN() in xhci_hc_died() to get a backtrace of who called it.\n> \n> diff --git a/drivers/usb/host/xhci-ring.c b/drivers/usb/host/xhci-ring.c\n> index 51cd4b8..ade2ad6 100644\n> --- a/drivers/usb/host/xhci-ring.c\n> +++ b/drivers/usb/host/xhci-ring.c\n> @@ -922,7 +922,8 @@ void xhci_hc_died(struct xhci_hcd *xhci)\n> if (xhci->xhc_state & XHCI_STATE_DYING)\n> return;\n> \n> - xhci_err(xhci, \"xHCI host controller not responding, assume dead\\n\");\n> + xhci_err(xhci, \"xHC not responding in %pf, assume controller is dead\\n\",\n> + __builtin_return_address(0));\n> xhci->xhc_state |= XHCI_STATE_DYING;\n> \n> xhci_cleanup_command_queue(xhci);\n\nI'll try some coarse bisection to narrow it down.\n\n$ git describe --contains d9f11ba9f107aa335091ab8d7ba5eea714e46e8b\nv4.12-rc1~97^2~39\n\nI'll check 4.11 first.\n\nI wanted to mention that the XHCI setup on 4.9 and 4.13 print\nslightly different things (at the beginning).\n\nOn 4.9\n[ 1.240322] xhci_hcd 0000:01:00.0: xHCI Host Controller\n[ 1.245617] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 1\n[ 1.258691] xhci_hcd 0000:01:00.0: hcc params 0x014051cf hci version 0x100 quirks 0x00000010\n[ 1.268090] hub 1-0:1.0: USB hub found\n[ 1.271905] hub 1-0:1.0: 4 ports detected\n[ 1.276372] xhci_hcd 0000:01:00.0: xHCI Host Controller\n[ 1.281645] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 2\n[ 1.289173] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.\n[ 1.297775] hub 2-0:1.0: USB hub found\n[ 1.301577] hub 2-0:1.0: 4 ports detected\n[ 1.306194] usbcore: registered new interface driver usb-storage\n\nOn 4.13\n[ 1.222471] pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22\n[ 1.229156] xhci_hcd 0000:01:00.0: Resetting\n[ 2.268836] xhci_hcd 0000:01:00.0: xHCI Host Controller\n[ 2.274126] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 1\n[ 2.287222] xhci_hcd 0000:01:00.0: hcc params 0x014051cf hci version 0x100 quirks 0x00000010\n[ 2.296653] hub 1-0:1.0: USB hub found\n[ 2.300478] hub 1-0:1.0: 4 ports detected\n[ 2.304962] xhci_hcd 0000:01:00.0: xHCI Host Controller\n[ 2.310246] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 2\n[ 2.317776] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.\n[ 2.326419] hub 2-0:1.0: USB hub found\n[ 2.330229] hub 2-0:1.0: 4 ports detected\n[ 2.334869] usbcore: registered new interface driver usb-storage\n\nFWIW, \"of_irq_parse_pci: failed with rc=-22\"\nseems to come from:\n\n[ 1.257411] [] (of_irq_parse_pci) from [] (of_irq_parse_and_map_pci+0x10/0x2c)\n[ 1.266420] [] (of_irq_parse_and_map_pci) from [] (pci_assign_irq+0x78/0xb0)\n[ 1.275254] [] (pci_assign_irq) from [] (pci_device_probe+0x18/0x128)\n[ 1.283476] [] (pci_device_probe) from [] (driver_probe_device+0x244/0x2c8)\n\nThe error logging was added by f1aa54840657f\nNo, that just turned one specific error into a warning.\nNeed to dig a bit more.\n\nRegards.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xchgl2BP3z9s9Y\n\tfor ;\n\tWed, 23 Aug 2017 19:18:59 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753572AbdHWJS6 (ORCPT );\n\tWed, 23 Aug 2017 05:18:58 -0400","from smtp5-g21.free.fr ([212.27.42.5]:27338 \"EHLO\n\tsmtp5-g21.free.fr\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1753445AbdHWJS4 (ORCPT );\n\tWed, 23 Aug 2017 05:18:56 -0400","from [172.27.0.114] (unknown [92.154.11.170])\n\t(Authenticated sender: slash.tmp)\n\tby smtp5-g21.free.fr (Postfix) with ESMTPSA id 4DB135FFD8;\n\tWed, 23 Aug 2017 11:18:37 +0200 (CEST)"],"Subject":"Re: Possible regression between 4.9 and 4.13","To":"Mathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ","Cc":"Bjorn Helgaas ,\n\tAlan Stern ,\n\tGreg Kroah-Hartman ","References":"<4dee5523-2d76-e731-6e81-f3027e88827f@free.fr>\n\t<87a82qbyv5.fsf@linux.intel.com> <599D3410.9050504@intel.com>","From":"Mason ","Message-ID":"<2131ff31-09f8-6955-502d-a3fce031c31e@free.fr>","Date":"Wed, 23 Aug 2017 11:18:36 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 SeaMonkey/2.49.1","MIME-Version":"1.0","In-Reply-To":"<599D3410.9050504@intel.com>","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1754984,"web_url":"http://patchwork.ozlabs.org/comment/1754984/","msgid":"<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>","list_archive_url":null,"date":"2017-08-23T09:31:29","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":66150,"url":"http://patchwork.ozlabs.org/api/people/66150/","name":"Mason","email":"slash.tmp@free.fr"},"content":"On 23/08/2017 09:51, Mathias Nyman wrote:\n\n> very likely cause is the more aggressive detection of pci removed xhci hosts\n> \n> See commit d9f11ba9f107aa335091ab8d7ba5eea714e46e8b\n> xhci: Rework how we handle unresponsive or hoptlug removed hosts\n> \n> It checks if a xhci register reads returns 0xffffffff and assumes xhci\n> died in that case.\n> \n> Could you add something like the below to check which what is killing the host?\n> Or a BUG()/WARN() in xhci_hc_died() to get a backtrace of who called it.\n\n[ 46.525247] usb 2-2: new SuperSpeed USB device number 2 using xhci_hcd\n[ 46.565496] usb-storage 2-2:1.0: USB Mass Storage device detected\n[ 46.571934] scsi host0: usb-storage 2-2:1.0\n[ 47.601227] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 3.0 PQ: 0 ANSI: 6\n[ 47.611340] sd 0:0:0:0: [sda] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)\n[ 47.621624] sd 0:0:0:0: [sda] Write Protect is off\n[ 47.627131] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA\n[ 47.639637] sda: sda1\n[ 47.648091] sd 0:0:0:0: [sda] Attached SCSI removable disk\n[ 58.100306] xhci_hcd 0000:01:00.0: xHCI host controller not responding, assume dead\n[ 58.108021] CPU: 0 PID: 939 Comm: kworker/0:2 Tainted: G C 4.13.0-rc6 #11\n[ 58.115976] Hardware name: Sigma Tango DT\n[ 58.120016] Workqueue: usb_hub_wq hub_event\n[ 58.124241] [] (unwind_backtrace) from [] (show_stack+0x10/0x14)\n[ 58.132033] [] (show_stack) from [] (dump_stack+0x84/0x98)\n[ 58.139302] [] (dump_stack) from [] (xhci_hc_died.part.9+0x50/0x23c)\n[ 58.147438] [] (xhci_hc_died.part.9) from [] (xhci_hub_control+0xf3c/0x175c)\n[ 58.156273] [] (xhci_hub_control) from [] (usb_hcd_submit_urb+0x264/0x814)\n[ 58.164932] [] (usb_hcd_submit_urb) from [] (usb_start_wait_urb+0x4c/0xbc)\n[ 58.173591] [] (usb_start_wait_urb) from [] (usb_control_msg+0xa0/0xcc)\n[ 58.181985] [] (usb_control_msg) from [] (usb_clear_port_feature+0x44/0x4c)\n[ 58.190730] [] (usb_clear_port_feature) from [] (hub_port_reset+0x228/0x51c)\n[ 58.199561] [] (hub_port_reset) from [] (hub_event+0x87c/0x108c)\n[ 58.207349] [] (hub_event) from [] (process_one_work+0x1d8/0x3f0)\n[ 58.215220] [] (process_one_work) from [] (worker_thread+0x38/0x554)\n[ 58.223354] [] (worker_thread) from [] (kthread+0x108/0x138)\n[ 58.230789] [] (kthread) from [] (ret_from_fork+0x14/0x3c)\n[ 58.238056] xhci_hcd 0000:01:00.0: HC died; cleaning up\n[ 58.243391] usb 2-2: USB disconnect, device number 2","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xchyc4lMKz9s8V\n\tfor ;\n\tWed, 23 Aug 2017 19:31:52 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753630AbdHWJbu (ORCPT );\n\tWed, 23 Aug 2017 05:31:50 -0400","from smtp5-g21.free.fr ([212.27.42.5]:34736 \"EHLO\n\tsmtp5-g21.free.fr\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1753593AbdHWJbt (ORCPT );\n\tWed, 23 Aug 2017 05:31:49 -0400","from [172.27.0.114] (unknown [92.154.11.170])\n\t(Authenticated sender: slash.tmp)\n\tby smtp5-g21.free.fr (Postfix) with ESMTPSA id 21CD45FEF1;\n\tWed, 23 Aug 2017 11:31:30 +0200 (CEST)"],"Subject":"Re: Possible regression between 4.9 and 4.13","To":"Mathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ","Cc":"Bjorn Helgaas ,\n\tAlan Stern ,\n\tGreg Kroah-Hartman ","References":"<4dee5523-2d76-e731-6e81-f3027e88827f@free.fr>\n\t<87a82qbyv5.fsf@linux.intel.com> <599D3410.9050504@intel.com>","From":"Mason ","Message-ID":"<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>","Date":"Wed, 23 Aug 2017 11:31:29 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 SeaMonkey/2.49.1","MIME-Version":"1.0","In-Reply-To":"<599D3410.9050504@intel.com>","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1755040,"web_url":"http://patchwork.ozlabs.org/comment/1755040/","msgid":"<48fcf483-3bab-8ab3-4bfa-e81ca8ee8353@free.fr>","list_archive_url":null,"date":"2017-08-23T10:19:49","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":66150,"url":"http://patchwork.ozlabs.org/api/people/66150/","name":"Mason","email":"slash.tmp@free.fr"},"content":"On 23/08/2017 09:51, Mathias Nyman wrote:\n\n> very likely cause is the more aggressive detection of pci removed xhci hosts\n> \n> See commit d9f11ba9f107aa335091ab8d7ba5eea714e46e8b\n> xhci: Rework how we handle unresponsive or hoptlug removed hosts\n> \n> It checks if a xhci register reads returns 0xffffffff and assumes xhci\n> died in that case.\n\nI've just tested 4.11.12 + a few local patches to back-port\nPCIe host bridge support.\n\nIt \"works\" as well as 4.9\n(i.e. modulo the \"AER: Uncorrected (Non-Fatal) error received\")\n\n[ 0.508533] pcie_tango 50000000.pcie: simultaneous PCI config and MMIO accesses may cause data corruption\n[ 0.519622] OF: PCI: host bridge /soc/pcie@2e000 ranges:\n[ 0.519645] OF: PCI: MEM 0x50400000..0x53ffffff -> 0x00400000\n[ 0.519725] pcie_tango 50000000.pcie: ECAM at [mem 0x50000000-0x503fffff] for [bus 00-03]\n[ 0.519872] pcie_tango 50000000.pcie: PCI host bridge to bus 0000:00\n[ 0.519886] pci_bus 0000:00: root bus resource [bus 00-03]\n[ 0.519898] pci_bus 0000:00: root bus resource [mem 0x50400000-0x53ffffff] (bus address [0x00400000-0x03ffffff])\n[ 0.520201] PCI: bus0: Fast back to back transfers disabled\n[ 0.520213] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring\n[ 0.520922] PCI: bus1: Fast back to back transfers disabled\n[ 0.520964] pci 0000:00:00.0: of_irq_parse_pci: failed with rc=-22\n[ 0.520993] pci 0000:00:00.0: BAR 8: assigned [mem 0x50400000-0x504fffff]\n[ 0.521004] pci 0000:01:00.0: BAR 0: assigned [mem 0x50400000-0x50401fff 64bit]\n[ 0.521025] pci 0000:00:00.0: PCI bridge to [bus 01]\n[ 0.521033] pci 0000:00:00.0: bridge window [mem 0x50400000-0x504fffff]\n[ 0.521085] pcieport 0000:00:00.0: enabling device (0140 -> 0142)\n[ 0.521282] pcieport 0000:00:00.0: Signaling PME with IRQ 30\n[ 0.521402] pcieport 0000:00:00.0: AER enabled with IRQ 30\n[ 0.521526] pci 0000:01:00.0: enabling device (0140 -> 0142)\n...\n[ 1.239706] xhci_hcd 0000:01:00.0: xHCI Host Controller\n[ 1.244998] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 1\n[ 1.258048] xhci_hcd 0000:01:00.0: hcc params 0x014051cf hci version 0x100 quirks 0x00000010\n[ 1.267467] hub 1-0:1.0: USB hub found\n[ 1.271287] hub 1-0:1.0: 4 ports detected\n[ 1.275761] xhci_hcd 0000:01:00.0: xHCI Host Controller\n[ 1.281048] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 2\n[ 1.288578] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.\n[ 1.297234] hub 2-0:1.0: USB hub found\n[ 1.301042] hub 2-0:1.0: 4 ports detected\n[ 1.305681] usbcore: registered new interface driver usb-storage\n\n\nPLUG #1\n[ 26.104607] usb 2-2: new SuperSpeed USB device number 2 using xhci_hcd\n[ 26.143799] usb-storage 2-2:1.0: USB Mass Storage device detected\n[ 26.150253] scsi host0: usb-storage 2-2:1.0\n[ 27.177298] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 3.0 PQ: 0 ANSI: 6\n[ 27.187586] sd 0:0:0:0: [sda] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)\n[ 27.199000] sd 0:0:0:0: [sda] Write Protect is off\n[ 27.204186] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA\n[ 27.216322] sda: sda1\n[ 27.220584] sd 0:0:0:0: [sda] Attached SCSI removable disk\n[ 27.252046] random: fast init done\n\nUNPLUG #1\n[ 37.334040] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 37.342135] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 37.353970] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 37.362589] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 37.369485] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 38.066538] xhci_hcd 0000:01:00.0: Cannot set link state.\n[ 38.072039] usb usb2-port2: cannot disable (err = -32)\n[ 38.077348] usb 2-2: USB disconnect, device number 2\n[ 38.082711] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 38.094279] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 38.108006] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 38.116878] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 38.123954] pcieport 0000:00:00.0: AER: Device recovery failed\n\nPLUG #2\n[ 55.097922] usb 2-2: new SuperSpeed USB device number 3 using xhci_hcd\n[ 55.137590] usb-storage 2-2:1.0: USB Mass Storage device detected\n[ 55.144016] scsi host0: usb-storage 2-2:1.0\n[ 56.163907] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 3.0 PQ: 0 ANSI: 6\n[ 56.174851] sd 0:0:0:0: [sda] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)\n[ 56.184218] sd 0:0:0:0: [sda] Write Protect is off\n[ 56.190162] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA\n[ 56.202117] sda: sda1\n[ 56.207112] sd 0:0:0:0: [sda] Attached SCSI removable disk\n\nUNPLUG #2\n[ 63.228310] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 63.236403] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 63.248220] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 63.256653] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 63.263523] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 63.959768] xhci_hcd 0000:01:00.0: Cannot set link state.\n[ 63.965227] usb usb2-port2: cannot disable (err = -32)\n[ 63.970409] usb 2-2: USB disconnect, device number 3\n[ 63.975664] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 63.987356] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 64.000021] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 64.008655] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 64.015553] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 64.021449] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 64.029580] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 64.041410] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 64.049818] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 64.056658] pcieport 0000:00:00.0: AER: Device recovery failed\n\n\nBjorn,\n\nWhat do you make of the AER logs?\nWhat can I do to debug this issue?\n\nRegards.\n\n\n\nFWIW, verbose lspci output below.\n\n# lspci -vv\n00:00.0 PCI bridge: Sigma Designs, Inc. Device 0024 (rev 01) (prog-if 00 [Normal decode])\n Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+\n Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR+ (64-bit, non-prefetchable)\n Bus: primary=00, secondary=01, subordinate=01, sec-latency=0\n I/O behind bridge: 00000000-00000fff\n Memory behind bridge: 00400000-004fffff\n Prefetchable memory behind bridge: 00000000-000fffff\n Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- Reset- FastB2B-\n PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-\n Capabilities: [50] MSI: Enable+ Count=1/4 Maskable- 64bit+\n Address: 00000000a002e07c Data: 0000\n Capabilities: [78] Power Management version 3\n Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)\n Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=3 PME-\n Capabilities: [80] Express (v2) Root Port (Slot-), MSI 03\n DevCap: MaxPayload 256 bytes, PhantFunc 0\n ExtTag- RBE+\n DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+\n RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+\n MaxPayload 128 bytes, MaxReadReq 512 bytes\n DevSta: CorrErr+ UncorrErr+ FatalErr- UnsuppReq- AuxPwr- TransPend+\n LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <2us, L1 <4us\n ClockPM- Surprise- LLActRep- BwNot+ ASPMOptComp-\n LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- CommClk-\n ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-\n LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-\n RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-\n RootCap: CRSVisible-\n RootSta: PME ReqID 0000, PMEStatus- PMEPending-\n DevCap2: Completion Timeout: Range B, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd-\n DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-\n LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-\n Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-\n Compliance De-emphasis: -6dB\n LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-\n EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-\n Capabilities: [100 v1] Virtual Channel\n Caps: LPEVC=0 RefClk=100ns PATEntryBits=1\n Arb: Fixed- WRR32- WRR64- WRR128-\n Ctrl: ArbSelect=Fixed\n Status: InProgress-\n VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-\n Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-\n Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff\n Status: NegoPending- InProgress-\n Capabilities: [800 v1] Advanced Error Reporting\n UESta: DLP- SDES- TLP- FCP- CmpltTO+ CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-\n UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-\n UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-\n CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+\n CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+\n AERCap: First Error Pointer: 0e, GenCap- CGenEn- ChkCap- ChkEn-\n Kernel driver in use: pcieport\n\n01:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03) (prog-if 30 [XHCI])\n Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+\n Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- ","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xck2T5684z9s78\n\tfor ;\n\tWed, 23 Aug 2017 20:20:17 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753656AbdHWKUP (ORCPT );\n\tWed, 23 Aug 2017 06:20:15 -0400","from smtp5-g21.free.fr ([212.27.42.5]:37123 \"EHLO\n\tsmtp5-g21.free.fr\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1753635AbdHWKUO (ORCPT );\n\tWed, 23 Aug 2017 06:20:14 -0400","from [172.27.0.114] (unknown [92.154.11.170])\n\t(Authenticated sender: slash.tmp)\n\tby smtp5-g21.free.fr (Postfix) with ESMTPSA id 6ABC15FF33;\n\tWed, 23 Aug 2017 12:19:49 +0200 (CEST)"],"Subject":"Re: Possible regression between 4.9 and 4.13","To":"Mathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ","Cc":"Bjorn Helgaas ,\n\tAlan Stern ,\n\tGreg Kroah-Hartman ,\n\tJon Derrick ,\n\tKeith Busch ","References":"<4dee5523-2d76-e731-6e81-f3027e88827f@free.fr>\n\t<87a82qbyv5.fsf@linux.intel.com> <599D3410.9050504@intel.com>","From":"Mason ","Message-ID":"<48fcf483-3bab-8ab3-4bfa-e81ca8ee8353@free.fr>","Date":"Wed, 23 Aug 2017 12:19:49 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 SeaMonkey/2.49.1","MIME-Version":"1.0","In-Reply-To":"<599D3410.9050504@intel.com>","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1755091,"web_url":"http://patchwork.ozlabs.org/comment/1755091/","msgid":"<599D62EA.7050100@linux.intel.com>","list_archive_url":null,"date":"2017-08-23T11:11:38","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":63937,"url":"http://patchwork.ozlabs.org/api/people/63937/","name":"Mathias Nyman","email":"mathias.nyman@linux.intel.com"},"content":"On 23.08.2017 12:31, Mason wrote:\n> On 23/08/2017 09:51, Mathias Nyman wrote:\n>\n>> very likely cause is the more aggressive detection of pci removed xhci hosts\n>>\n>> See commit d9f11ba9f107aa335091ab8d7ba5eea714e46e8b\n>> xhci: Rework how we handle unresponsive or hoptlug removed hosts\n>>\n>> It checks if a xhci register reads returns 0xffffffff and assumes xhci\n>> died in that case.\n>>\n>> Could you add something like the below to check which what is killing the host?\n>> Or a BUG()/WARN() in xhci_hc_died() to get a backtrace of who called it.\n>\n> [ 46.525247] usb 2-2: new SuperSpeed USB device number 2 using xhci_hcd\n> [ 46.565496] usb-storage 2-2:1.0: USB Mass Storage device detected\n> [ 46.571934] scsi host0: usb-storage 2-2:1.0\n> [ 47.601227] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 3.0 PQ: 0 ANSI: 6\n> [ 47.611340] sd 0:0:0:0: [sda] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)\n> [ 47.621624] sd 0:0:0:0: [sda] Write Protect is off\n> [ 47.627131] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA\n> [ 47.639637] sda: sda1\n> [ 47.648091] sd 0:0:0:0: [sda] Attached SCSI removable disk\n> [ 58.100306] xhci_hcd 0000:01:00.0: xHCI host controller not responding, assume dead\n> [ 58.108021] CPU: 0 PID: 939 Comm: kworker/0:2 Tainted: G C 4.13.0-rc6 #11\n> [ 58.115976] Hardware name: Sigma Tango DT\n> [ 58.120016] Workqueue: usb_hub_wq hub_event\n> [ 58.124241] [] (unwind_backtrace) from [] (show_stack+0x10/0x14)\n> [ 58.132033] [] (show_stack) from [] (dump_stack+0x84/0x98)\n> [ 58.139302] [] (dump_stack) from [] (xhci_hc_died.part.9+0x50/0x23c)\n> [ 58.147438] [] (xhci_hc_died.part.9) from [] (xhci_hub_control+0xf3c/0x175c)\n> [ 58.156273] [] (xhci_hub_control) from [] (usb_hcd_submit_urb+0x264/0x814)\n> [ 58.164932] [] (usb_hcd_submit_urb) from [] (usb_start_wait_urb+0x4c/0xbc)\n> [ 58.173591] [] (usb_start_wait_urb) from [] (usb_control_msg+0xa0/0xcc)\n> [ 58.181985] [] (usb_control_msg) from [] (usb_clear_port_feature+0x44/0x4c)\n> [ 58.190730] [] (usb_clear_port_feature) from [] (hub_port_reset+0x228/0x51c)\n> [ 58.199561] [] (hub_port_reset) from [] (hub_event+0x87c/0x108c)\n> [ 58.207349] [] (hub_event) from [] (process_one_work+0x1d8/0x3f0)\n> [ 58.215220] [] (process_one_work) from [] (worker_thread+0x38/0x554)\n> [ 58.223354] [] (worker_thread) from [] (kthread+0x108/0x138)\n> [ 58.230789] [] (kthread) from [] (ret_from_fork+0x14/0x3c)\n> [ 58.238056] xhci_hcd 0000:01:00.0: HC died; cleaning up\n> [ 58.243391] usb 2-2: USB disconnect, device number 2\n> --\n\nxhci driver reads 0xffffffff from a mmio mapped xhci portsc register and bails out in:\nxhci-hub.c:\n temp = readl(port_array[wIndex]);\n if (temp == ~(u32)0) {\n xhci_hc_died(xhci);\n\t\t\tretval = -ENODEV;\n\t break;\n\t\t}\n\nIn this case we read the register when hub thread asks to clear port feature.\n\nwhy portsc returns 0xffffffff is a nother quiestion, could the hub thread be running while xhci controller is (in D3)?\nWas xhci runtime suspended?\nThere were some pcieport errors in another log you showed, maybe PCI devices are not properly recovered\nand the registers return 0xffffffff?\n\n-Mathias","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xcl5r2GfWz9s8J\n\tfor ;\n\tWed, 23 Aug 2017 21:08:16 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753834AbdHWLIO (ORCPT );\n\tWed, 23 Aug 2017 07:08:14 -0400","from mga02.intel.com ([134.134.136.20]:29111 \"EHLO mga02.intel.com\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1753691AbdHWLIO (ORCPT );\n\tWed, 23 Aug 2017 07:08:14 -0400","from fmsmga002.fm.intel.com ([10.253.24.26])\n\tby orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t23 Aug 2017 04:08:12 -0700","from mattu-haswell.fi.intel.com (HELO [10.237.72.164])\n\t([10.237.72.164])\n\tby fmsmga002.fm.intel.com with ESMTP; 23 Aug 2017 04:08:10 -0700"],"X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos;i=\"5.41,416,1498546800\"; d=\"scan'208\";a=\"1209365783\"","Subject":"Re: Possible regression between 4.9 and 4.13","To":"Mason , Mathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ","References":"<4dee5523-2d76-e731-6e81-f3027e88827f@free.fr>\n\t<87a82qbyv5.fsf@linux.intel.com> <599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>","Cc":"Bjorn Helgaas ,\n\tAlan Stern ,\n\tGreg Kroah-Hartman ","From":"Mathias Nyman ","Message-ID":"<599D62EA.7050100@linux.intel.com>","Date":"Wed, 23 Aug 2017 14:11:38 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101\n\tThunderbird/38.8.0","MIME-Version":"1.0","In-Reply-To":"<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1755117,"web_url":"http://patchwork.ozlabs.org/comment/1755117/","msgid":"","list_archive_url":null,"date":"2017-08-23T11:54:40","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":66150,"url":"http://patchwork.ozlabs.org/api/people/66150/","name":"Mason","email":"slash.tmp@free.fr"},"content":"On 23/08/2017 13:11, Mathias Nyman wrote:\n\n> On 23.08.2017 12:31, Mason wrote:\n> \n>> [ 46.525247] usb 2-2: new SuperSpeed USB device number 2 using xhci_hcd\n>> [ 46.565496] usb-storage 2-2:1.0: USB Mass Storage device detected\n>> [ 46.571934] scsi host0: usb-storage 2-2:1.0\n>> [ 47.601227] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 3.0 PQ: 0 ANSI: 6\n>> [ 47.611340] sd 0:0:0:0: [sda] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)\n>> [ 47.621624] sd 0:0:0:0: [sda] Write Protect is off\n>> [ 47.627131] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA\n>> [ 47.639637] sda: sda1\n>> [ 47.648091] sd 0:0:0:0: [sda] Attached SCSI removable disk\n>> [ 58.100306] xhci_hcd 0000:01:00.0: xHCI host controller not responding, assume dead\n>> [ 58.108021] CPU: 0 PID: 939 Comm: kworker/0:2 Tainted: G C 4.13.0-rc6 #11\n>> [ 58.115976] Hardware name: Sigma Tango DT\n>> [ 58.120016] Workqueue: usb_hub_wq hub_event\n>> [ 58.124241] [] (unwind_backtrace) from [] (show_stack+0x10/0x14)\n>> [ 58.132033] [] (show_stack) from [] (dump_stack+0x84/0x98)\n>> [ 58.139302] [] (dump_stack) from [] (xhci_hc_died.part.9+0x50/0x23c)\n>> [ 58.147438] [] (xhci_hc_died.part.9) from [] (xhci_hub_control+0xf3c/0x175c)\n>> [ 58.156273] [] (xhci_hub_control) from [] (usb_hcd_submit_urb+0x264/0x814)\n>> [ 58.164932] [] (usb_hcd_submit_urb) from [] (usb_start_wait_urb+0x4c/0xbc)\n>> [ 58.173591] [] (usb_start_wait_urb) from [] (usb_control_msg+0xa0/0xcc)\n>> [ 58.181985] [] (usb_control_msg) from [] (usb_clear_port_feature+0x44/0x4c)\n>> [ 58.190730] [] (usb_clear_port_feature) from [] (hub_port_reset+0x228/0x51c)\n>> [ 58.199561] [] (hub_port_reset) from [] (hub_event+0x87c/0x108c)\n>> [ 58.207349] [] (hub_event) from [] (process_one_work+0x1d8/0x3f0)\n>> [ 58.215220] [] (process_one_work) from [] (worker_thread+0x38/0x554)\n>> [ 58.223354] [] (worker_thread) from [] (kthread+0x108/0x138)\n>> [ 58.230789] [] (kthread) from [] (ret_from_fork+0x14/0x3c)\n>> [ 58.238056] xhci_hcd 0000:01:00.0: HC died; cleaning up\n>> [ 58.243391] usb 2-2: USB disconnect, device number 2\n> \n> xhci driver reads 0xffffffff from a mmio mapped xhci portsc register and bails out in:\n> xhci-hub.c:\n> temp = readl(port_array[wIndex]);\n> if (temp == ~(u32)0) {\n> xhci_hc_died(xhci);\n> \t\t\tretval = -ENODEV;\n> \t break;\n> \t\t}\n> \n> In this case we read the register when hub thread asks to clear port feature.\n> \n> why portsc returns 0xffffffff is a another question, could the hub thread be running while xhci controller is (in D3)?\n> Was xhci runtime suspended?\n\nHow do I tell?\nShould I disable SUSPEND support and all kinds of power management?\n\n> There were some pcieport errors in another log you showed, maybe PCI devices are not properly recovered\n> and the registers return 0xffffffff?\n\nFWIW, I just compiled v4.12-rc1 and I do get the broken behavior.\n\nv4.11.12 = OK\nv4.12-rc1 = KO\n\nPLUG\n[ 17.226953] usb 2-2: new SuperSpeed USB device number 2 using xhci_hcd\n[ 17.267195] usb-storage 2-2:1.0: USB Mass Storage device detected\n[ 17.273612] scsi host0: usb-storage 2-2:1.0\n[ 18.296369] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 3.0 PQ: 0 ANSI: 6\n[ 18.307772] sd 0:0:0:0: [sda] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)\n[ 18.316991] sd 0:0:0:0: [sda] Write Protect is off\n[ 18.322588] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA\n[ 18.334828] sda: sda1\n[ 18.339507] sd 0:0:0:0: [sda] Attached SCSI removable disk\n[ 18.366202] random: fast init done\n\nUNPLUG\n[ 21.314111] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 21.322219] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 21.334039] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 21.342453] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 21.349306] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 22.055471] xhci_hcd 0000:01:00.0: xHCI host controller not responding, assume dead\n[ 22.063187] xhci_hcd 0000:01:00.0: HC died; cleaning up\n[ 22.068523] usb 2-2: USB disconnect, device number 2\n[ 22.073774] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 22.085369] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 22.098823] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 22.107245] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 22.114130] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 22.120026] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 22.128096] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 22.139916] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 22.148320] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 22.155162] pcieport 0000:00:00.0: AER: Device recovery failed\n\n\nThe defconfig I used for testing:\n\n# CONFIG_SWAP is not set\nCONFIG_SYSVIPC=y\nCONFIG_NO_HZ_IDLE=y\nCONFIG_HIGH_RES_TIMERS=y\n# CONFIG_COMPAT_BRK is not set\nCONFIG_SLAB=y\nCONFIG_MODULES=y\nCONFIG_MODULE_UNLOAD=y\nCONFIG_MODVERSIONS=y\nCONFIG_ARCH_TANGO=y\n# CONFIG_ARM_ERRATA_643719 is not set\nCONFIG_PCI=y\nCONFIG_PCIEPORTBUS=y\nCONFIG_PCI_MSI=y\nCONFIG_PCIE_TANGO_SMP8759=y\nCONFIG_SMP=y\nCONFIG_PREEMPT=y\nCONFIG_HZ_300=y\nCONFIG_AEABI=y\nCONFIG_HIGHMEM=y\n# CONFIG_ATAGS is not set\nCONFIG_ARM_APPENDED_DTB=y\nCONFIG_ARM_ATAG_DTB_COMPAT=y\nCONFIG_CPU_FREQ=y\nCONFIG_CPU_FREQ_GOV_ONDEMAND=y\nCONFIG_CPUFREQ_DT=y\nCONFIG_VFP=y\nCONFIG_NEON=y\nCONFIG_NET=y\nCONFIG_PACKET=y\nCONFIG_UNIX=y\nCONFIG_INET=y\nCONFIG_IP_MULTICAST=y\nCONFIG_IP_PNP=y\nCONFIG_IP_PNP_DHCP=y\n# CONFIG_INET_XFRM_MODE_TRANSPORT is not set\n# CONFIG_INET_XFRM_MODE_TUNNEL is not set\n# CONFIG_INET_XFRM_MODE_BEET is not set\n# CONFIG_IPV6 is not set\nCONFIG_UEVENT_HELPER_PATH=\"/sbin/hotplug\"\nCONFIG_DEVTMPFS=y\nCONFIG_DEVTMPFS_MOUNT=y\nCONFIG_BLK_DEV_LOOP=y\nCONFIG_SCSI=y\nCONFIG_BLK_DEV_SD=y\nCONFIG_NETDEVICES=y\nCONFIG_NET_VENDOR_AURORA=y\nCONFIG_AURORA_NB8800=y\nCONFIG_AT803X_PHY=y\n# CONFIG_WLAN is not set\n# CONFIG_INPUT_KEYBOARD is not set\n# CONFIG_INPUT_MOUSE is not set\n# CONFIG_SERIO is not set\nCONFIG_SERIAL_8250=y\n# CONFIG_SERIAL_8250_DEPRECATED_OPTIONS is not set\nCONFIG_SERIAL_8250_CONSOLE=y\nCONFIG_SERIAL_8250_RT288X=y\nCONFIG_SERIAL_OF_PLATFORM=y\n# CONFIG_HW_RANDOM is not set\nCONFIG_I2C=y\nCONFIG_I2C_XLR=y\nCONFIG_GPIOLIB=y\nCONFIG_THERMAL=y\nCONFIG_CPU_THERMAL=y\nCONFIG_TANGO_THERMAL=y\nCONFIG_WATCHDOG=y\nCONFIG_TANGOX_WATCHDOG=y\nCONFIG_FB=y\n# CONFIG_HID is not set\n# CONFIG_USB_HID is not set\nCONFIG_USB=y\nCONFIG_USB_XHCI_HCD=y\nCONFIG_USB_STORAGE=y\nCONFIG_EXT4_FS=y\nCONFIG_FUSE_FS=m\nCONFIG_VFAT_FS=m\nCONFIG_TMPFS=y\nCONFIG_NFS_FS=y\n# CONFIG_NFS_V2 is not set\nCONFIG_ROOT_NFS=y\nCONFIG_NLS_CODEPAGE_437=m\nCONFIG_NLS_ISO8859_1=m\nCONFIG_NLS_UTF8=m\nCONFIG_PRINTK_TIME=y\n# CONFIG_CRYPTO_ECHAINIV is not set","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xcm7p2L57z9s9Y\n\tfor ;\n\tWed, 23 Aug 2017 21:55:02 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753829AbdHWLzA (ORCPT );\n\tWed, 23 Aug 2017 07:55:00 -0400","from smtp5-g21.free.fr ([212.27.42.5]:55213 \"EHLO\n\tsmtp5-g21.free.fr\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1753768AbdHWLy7 (ORCPT );\n\tWed, 23 Aug 2017 07:54:59 -0400","from [172.27.0.114] (unknown [92.154.11.170])\n\t(Authenticated sender: slash.tmp)\n\tby smtp5-g21.free.fr (Postfix) with ESMTPSA id BFDB55FFA3;\n\tWed, 23 Aug 2017 13:54:40 +0200 (CEST)"],"Subject":"Re: Possible regression between 4.9 and 4.13","To":"Mathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ","Cc":"Bjorn Helgaas ,\n\tAlan Stern ,\n\tGreg Kroah-Hartman ","References":"<4dee5523-2d76-e731-6e81-f3027e88827f@free.fr>\n\t<87a82qbyv5.fsf@linux.intel.com> <599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>","From":"Mason ","Message-ID":"","Date":"Wed, 23 Aug 2017 13:54:40 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 SeaMonkey/2.49.1","MIME-Version":"1.0","In-Reply-To":"<599D62EA.7050100@linux.intel.com>","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1755153,"web_url":"http://patchwork.ozlabs.org/comment/1755153/","msgid":"<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>","list_archive_url":null,"date":"2017-08-23T12:41:01","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":66150,"url":"http://patchwork.ozlabs.org/api/people/66150/","name":"Mason","email":"slash.tmp@free.fr"},"content":"On 23/08/2017 13:54, Mason wrote:\n\n> On 23/08/2017 13:11, Mathias Nyman wrote:\n> \n>> In this case we read the register when hub thread asks to clear port feature.\n>>\n>> why portsc returns 0xffffffff is a another question, could the hub thread be running while xhci controller is (in D3)?\n>> Was xhci runtime suspended?\n> \n> How do I tell?\n> Should I disable SUSPEND support and all kinds of power management?\n\nI compiled a minimal kernel, with lots of irrelevant drivers and\nframeworks left out, including power management. I still get the\n\"xHCI host controller not responding, assume dead\" issue.\n\nPLUG\n[ 59.803499] usb 2-2: new SuperSpeed USB device number 2 using xhci_hcd\n[ 59.836902] usb 2-2: New USB device found, idVendor=0951, idProduct=1666\n[ 59.843653] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3\n[ 59.850900] usb 2-2: Product: DataTraveler 3.0\n[ 59.855417] usb 2-2: Manufacturer: Kingston\n[ 59.859661] usb 2-2: SerialNumber: 002618887865F0C0F8646BFA\n[ 59.868249] usb-storage 2-2:1.0: USB Mass Storage device detected\n[ 59.874691] scsi host0: usb-storage 2-2:1.0\n[ 60.882801] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 3.0 PQ: 0 ANSI: 6\n[ 60.891640] sd 0:0:0:0: [sda] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)\n[ 60.899662] sd 0:0:0:0: [sda] Write Protect is off\n[ 60.904763] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA\n[ 60.916154] sda: sda1\n[ 60.919798] sd 0:0:0:0: [sda] Attached SCSI removable disk\n\nUNPLUG\n[ 70.545087] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 70.553169] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 70.565084] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 70.573528] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 70.580402] pcieport 0000:00:00.0: AER: Device recovery failed\n\n[ 71.275253] xhci_hcd 0000:01:00.0: xHCI host controller not responding, assume dead\n[ 71.282956] xhci_hcd 0000:01:00.0: HC died; cleaning up\n[ 71.288304] usb 2-2: USB disconnect, device number 2\n\n[ 71.293445] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 71.301851] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 71.313785] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 71.322240] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 71.329115] pcieport 0000:00:00.0: AER: Device recovery failed\n\n[ 71.335042] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 71.343137] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 71.354984] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 71.363443] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 71.370289] pcieport 0000:00:00.0: AER: Device recovery failed\n\n\ndefconfig for reference\n\n# CONFIG_SWAP is not set\nCONFIG_SYSVIPC=y\nCONFIG_NO_HZ_IDLE=y\nCONFIG_HIGH_RES_TIMERS=y\n# CONFIG_COMPAT_BRK is not set\nCONFIG_SLAB=y\nCONFIG_MODULES=y\nCONFIG_MODULE_UNLOAD=y\nCONFIG_MODVERSIONS=y\nCONFIG_ARCH_TANGO=y\n# CONFIG_ARM_ERRATA_643719 is not set\nCONFIG_PCI=y\nCONFIG_PCIEPORTBUS=y\nCONFIG_PCI_MSI=y\nCONFIG_PCIE_TANGO_SMP8759=y\nCONFIG_SMP=y\nCONFIG_PREEMPT=y\nCONFIG_HZ_300=y\nCONFIG_AEABI=y\nCONFIG_HIGHMEM=y\n# CONFIG_ATAGS is not set\nCONFIG_ARM_APPENDED_DTB=y\nCONFIG_ARM_ATAG_DTB_COMPAT=y\nCONFIG_VFP=y\nCONFIG_NEON=y\n# CONFIG_SUSPEND is not set\nCONFIG_UEVENT_HELPER_PATH=\"/sbin/hotplug\"\nCONFIG_DEVTMPFS=y\nCONFIG_DEVTMPFS_MOUNT=y\nCONFIG_BLK_DEV_LOOP=y\nCONFIG_SCSI=y\nCONFIG_BLK_DEV_SD=y\n# CONFIG_INPUT_KEYBOARD is not set\n# CONFIG_INPUT_MOUSE is not set\n# CONFIG_SERIO is not set\nCONFIG_SERIAL_8250=y\n# CONFIG_SERIAL_8250_DEPRECATED_OPTIONS is not set\nCONFIG_SERIAL_8250_CONSOLE=y\nCONFIG_SERIAL_8250_RT288X=y\nCONFIG_SERIAL_OF_PLATFORM=y\n# CONFIG_HW_RANDOM is not set\n# CONFIG_HWMON is not set\n# CONFIG_HID is not set\n# CONFIG_USB_HID is not set\nCONFIG_USB=y\nCONFIG_USB_ANNOUNCE_NEW_DEVICES=y\nCONFIG_USB_XHCI_HCD=y\nCONFIG_USB_STORAGE=y\nCONFIG_VFAT_FS=m\nCONFIG_TMPFS=y\nCONFIG_NLS_CODEPAGE_437=m\nCONFIG_NLS_ISO8859_1=m\nCONFIG_NLS_UTF8=m\nCONFIG_PRINTK_TIME=y","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xcn9H2hnSz9s8V\n\tfor ;\n\tWed, 23 Aug 2017 22:41:23 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1753926AbdHWMlV (ORCPT );\n\tWed, 23 Aug 2017 08:41:21 -0400","from smtp5-g21.free.fr ([212.27.42.5]:36155 \"EHLO\n\tsmtp5-g21.free.fr\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1753921AbdHWMlV (ORCPT );\n\tWed, 23 Aug 2017 08:41:21 -0400","from [172.27.0.114] (unknown [92.154.11.170])\n\t(Authenticated sender: slash.tmp)\n\tby smtp5-g21.free.fr (Postfix) with ESMTPSA id 0F1D75FE84;\n\tWed, 23 Aug 2017 14:41:02 +0200 (CEST)"],"Subject":"Re: Possible regression between 4.9 and 4.13","From":"Mason ","To":"Mathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ","Cc":"Bjorn Helgaas ,\n\tAlan Stern ,\n\tGreg Kroah-Hartman ","References":"<4dee5523-2d76-e731-6e81-f3027e88827f@free.fr>\n\t<87a82qbyv5.fsf@linux.intel.com> <599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t","Message-ID":"<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>","Date":"Wed, 23 Aug 2017 14:41:01 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 SeaMonkey/2.49.1","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1755314,"web_url":"http://patchwork.ozlabs.org/comment/1755314/","msgid":"<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>","list_archive_url":null,"date":"2017-08-23T14:30:03","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":66150,"url":"http://patchwork.ozlabs.org/api/people/66150/","name":"Mason","email":"slash.tmp@free.fr"},"content":"On 23/08/2017 14:41, Mason wrote:\n\n> I compiled a minimal kernel, with lots of irrelevant drivers and\n> frameworks left out, including power management. I still get the\n> \"xHCI host controller not responding, assume dead\" issue.\n\nThe problem seems to have a timing-related aspect.\n\nI added a bunch of logs (to a slow serial console) and the HC was\nnot killed. I was able to plug the Flash drive a second time.\n(I am logging config space reads and writes.)\n\n[ 1.098314] READ: bus=1 devfn=0 where=84 size=2 val=0x8\n[ 1.103779] READ: bus=1 devfn=0 where=4 size=2 val=0x142\n[ 1.109315] READ: bus=1 devfn=0 where=61 size=1 val=0x1\n[ 1.114746] READ: bus=1 devfn=0 where=4 size=2 val=0x142\n[ 1.120311] READ: bus=1 devfn=0 where=4 size=2 val=0x142\n[ 1.125841] WRITE: bus=1 devfn=0 where=4 size=2 val=0x146\n\nNB: I added msleep(2500) in usb_add_hcd()\n\n[ 3.681867] xhci_hcd 0000:01:00.0: xHCI Host Controller\n[ 3.687154] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 1\n[ 3.694656] READ: bus=1 devfn=0 where=96 size=1 val=0x30\n[ 3.705736] xhci_hcd 0000:01:00.0: hcc params 0x014051cf hci version 0x100 quirks 0x00000010\n[ 3.714233] READ: bus=1 devfn=0 where=12 size=1 val=0x10\n[ 3.719752] READ: bus=1 devfn=0 where=4 size=2 val=0x146\n[ 3.725269] WRITE: bus=1 devfn=0 where=4 size=2 val=0x156\n[ 3.730794] READ: bus=1 devfn=0 where=146 size=2 val=0x7\n[ 3.736314] READ: bus=1 devfn=0 where=146 size=2 val=0x7\n[ 3.741835] WRITE: bus=1 devfn=0 where=146 size=2 val=0x7\n[ 3.747354] READ: bus=1 devfn=0 where=146 size=2 val=0x7\n[ 3.752871] READ: bus=1 devfn=0 where=148 size=4 val=0x1000\n[ 3.758775] READ: bus=1 devfn=0 where=146 size=2 val=0x7\n[ 3.764297] WRITE: bus=1 devfn=0 where=146 size=2 val=0xc007\n[ 3.770108] READ: bus=1 devfn=0 where=4 size=2 val=0x146\n[ 3.775626] WRITE: bus=1 devfn=0 where=4 size=2 val=0x546\n[ 3.781146] READ: bus=1 devfn=0 where=146 size=2 val=0xc007\n[ 3.786925] WRITE: bus=1 devfn=0 where=146 size=2 val=0x8007\n[ 3.792919] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002\n[ 3.799756] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1\n[ 3.807021] usb usb1: Product: xHCI Host Controller\n[ 3.811933] usb usb1: Manufacturer: Linux 4.12.0-rc1 xhci-hcd\n[ 3.817713] usb usb1: SerialNumber: 0000:01:00.0\n[ 3.822773] hub 1-0:1.0: USB hub found\n[ 3.826598] hub 1-0:1.0: 4 ports detected\n\nNB: I added msleep(2500) in usb_add_hcd()\n\n[ 6.455246] xhci_hcd 0000:01:00.0: xHCI Host Controller\n[ 6.460520] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 2\n[ 6.468028] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.\n[ 6.476236] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003\n[ 6.483068] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1\n[ 6.490334] usb usb2: Product: xHCI Host Controller\n[ 6.495240] usb usb2: Manufacturer: Linux 4.12.0-rc1 xhci-hcd\n[ 6.501020] usb usb2: SerialNumber: 0000:01:00.0\n[ 6.505994] hub 2-0:1.0: USB hub found\n[ 6.509806] hub 2-0:1.0: 4 ports detected\n[ 6.514215] usbcore: registered new interface driver usb-storage\n[ 6.520313] Registering SWP/SWPB emulation handler\n[ 6.525541] READ: bus=0 devfn=0 where=132 size=4 val=0x8001\n[ 6.531334] READ: bus=0 devfn=0 where=6 size=2 val=0x4010\n[ 6.536955] READ: bus=0 devfn=0 where=52 size=1 val=0x50\n[ 6.542484] READ: bus=0 devfn=0 where=80 size=2 val=0x7805\n[ 6.548180] READ: bus=0 devfn=0 where=120 size=2 val=0x8001\n[ 6.553969] READ: bus=0 devfn=0 where=128 size=2 val=0x10\n[ 6.559584] READ: bus=0 devfn=0 where=124 size=2 val=0x6008\n[ 6.565387] READ: bus=1 devfn=0 where=164 size=4 val=0x8fc0\n[ 6.571167] READ: bus=1 devfn=0 where=6 size=2 val=0x10\n[ 6.576609] READ: bus=1 devfn=0 where=52 size=1 val=0x50\n[ 6.582129] READ: bus=1 devfn=0 where=80 size=2 val=0x7001\n[ 6.587821] READ: bus=1 devfn=0 where=112 size=2 val=0x9005\n[ 6.593601] READ: bus=1 devfn=0 where=144 size=2 val=0xa011\n[ 6.599381] READ: bus=1 devfn=0 where=160 size=2 val=0x10\n[ 6.604985] READ: bus=1 devfn=0 where=84 size=2 val=0x8\n[ 6.623665] Freeing unused kernel memory: 9216K\n\n\nPLUG #1\n[ 66.783559] usb 2-2: new SuperSpeed USB device number 2 using xhci_hcd\n[ 66.816910] usb 2-2: New USB device found, idVendor=0951, idProduct=1666\n[ 66.823661] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3\n[ 66.830909] usb 2-2: Product: DataTraveler 3.0\n[ 66.835417] usb 2-2: Manufacturer: Kingston\n[ 66.839660] usb 2-2: SerialNumber: 002618887865F0C0F8646BFA\n[ 66.848131] usb-storage 2-2:1.0: USB Mass Storage device detected\n[ 66.854584] scsi host0: usb-storage 2-2:1.0\n[ 67.869446] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 3.0 PQ: 0 ANSI: 6\n[ 67.878270] sd 0:0:0:0: [sda] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)\n[ 67.886248] sd 0:0:0:0: [sda] Write Protect is off\n[ 67.891347] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA\n[ 67.902708] sda: sda1\n[ 67.906372] sd 0:0:0:0: [sda] Attached SCSI removable disk\n\n\nUNPLUG #1\n[ 71.697358] READ: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 71.703572] READ: bus=0 devfn=0 where=2100 size=4 val=0x0\n[ 71.709170] WRITE: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 71.715569] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 71.723632] READ: bus=0 devfn=0 where=136 size=2 val=0x281f\n[ 71.729470] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 71.735373] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 71.741013] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 71.746914] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 71.752552] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 71.758194] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 71.770008] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 71.778494] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 71.785358] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 71.791259] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 71.796897] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 71.802524] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 72.451908] READ: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 72.458120] READ: bus=0 devfn=0 where=2100 size=4 val=0x0\n[ 72.463717] WRITE: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 72.470012] READ: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 72.476221] READ: bus=0 devfn=0 where=2100 size=4 val=0x0\n[ 72.481819] WRITE: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 72.488109] READ: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 72.494319] READ: bus=0 devfn=0 where=2100 size=4 val=0x0\n[ 72.499916] WRITE: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 72.506205] READ: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 72.512415] READ: bus=0 devfn=0 where=2100 size=4 val=0x0\n[ 72.518011] WRITE: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 72.524263] xhci_hcd 0000:01:00.0: Cannot set link state.\n[ 72.529711] usb usb2-port2: cannot disable (err = -32)\n[ 72.534883] usb 2-2: USB disconnect, device number 2\n[ 72.540042] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 72.548365] READ: bus=0 devfn=0 where=136 size=2 val=0x281f\n[ 72.554264] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 72.560157] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 72.565778] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 72.571654] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 72.577273] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 72.582891] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 72.594705] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 72.603122] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 72.609955] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 72.615833] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 72.621441] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 72.627061] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 72.632931] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 72.640984] READ: bus=0 devfn=0 where=136 size=2 val=0x281f\n[ 72.646769] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 72.652636] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 72.658245] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 72.664114] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 72.669722] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 72.675330] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 72.687142] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 72.695545] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 72.702376] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 72.708244] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 72.713856] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 72.719473] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 72.725342] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 72.733394] READ: bus=0 devfn=0 where=136 size=2 val=0x281f\n[ 72.739178] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 72.745044] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 72.750653] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 72.756520] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 72.762128] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 72.767734] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 72.779548] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 72.787950] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 72.794781] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 72.800649] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 72.806258] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 72.811873] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 72.817741] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 72.825793] READ: bus=0 devfn=0 where=136 size=2 val=0x281f\n[ 72.831574] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 72.837442] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 72.843054] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 72.848922] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 72.854529] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 72.860137] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 72.871951] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 72.880353] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 72.887184] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 72.893051] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 72.898660] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 72.904273] pcieport 0000:00:00.0: AER: Device recovery failed\n\n\nPLUG #2\n[ 165.860193] usb 2-2: new SuperSpeed USB device number 3 using xhci_hcd\n[ 165.893583] usb 2-2: New USB device found, idVendor=0951, idProduct=1666\n[ 165.900333] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3\n[ 165.907515] usb 2-2: Product: DataTraveler 3.0\n[ 165.911989] usb 2-2: Manufacturer: Kingston\n[ 165.916198] usb 2-2: SerialNumber: 002618887865F0C0F8646BFA\n[ 165.924547] usb-storage 2-2:1.0: USB Mass Storage device detected\n[ 165.930970] scsi host0: usb-storage 2-2:1.0\n[ 166.962705] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 3.0 PQ: 0 ANSI: 6\n[ 166.971494] sd 0:0:0:0: [sda] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)\n[ 166.979556] sd 0:0:0:0: [sda] Write Protect is off\n[ 166.984591] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA\n[ 166.995847] random: fast init done\n[ 166.999430] sda: sda1\n[ 167.003039] sd 0:0:0:0: [sda] Attached SCSI removable disk\n\n\nUNPLUG #2\n[ 171.918834] READ: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 171.925046] READ: bus=0 devfn=0 where=2100 size=4 val=0x0\n[ 171.930645] WRITE: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 171.936941] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 171.945000] READ: bus=0 devfn=0 where=136 size=2 val=0x281f\n[ 171.950784] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 171.956656] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 171.962263] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 171.968134] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 171.973741] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 171.979354] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 171.991164] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 171.999597] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 172.006429] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 172.012300] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 172.017908] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 172.023529] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 172.675221] READ: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 172.681432] READ: bus=0 devfn=0 where=2100 size=4 val=0x0\n[ 172.687030] WRITE: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 172.693325] READ: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 172.699536] READ: bus=0 devfn=0 where=2100 size=4 val=0x0\n[ 172.705133] WRITE: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 172.711424] READ: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 172.717633] READ: bus=0 devfn=0 where=2100 size=4 val=0x0\n[ 172.723230] WRITE: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 172.729517] READ: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 172.735726] READ: bus=0 devfn=0 where=2100 size=4 val=0x0\n[ 172.741322] WRITE: bus=0 devfn=0 where=2096 size=4 val=0x10000024\n[ 172.747574] xhci_hcd 0000:01:00.0: Cannot set link state.\n[ 172.753021] usb usb2-port2: cannot disable (err = -32)\n[ 172.758193] usb 2-2: USB disconnect, device number 3\n[ 172.763340] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 172.771627] READ: bus=0 devfn=0 where=136 size=2 val=0x281f\n[ 172.777515] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 172.783408] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 172.789030] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 172.794907] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 172.800526] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 172.806146] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 172.817960] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 172.826375] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 172.833208] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 172.839078] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 172.844685] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 172.850305] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 172.856183] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 172.864236] READ: bus=0 devfn=0 where=136 size=2 val=0x281f\n[ 172.870020] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 172.875889] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 172.881497] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 172.887365] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 172.892974] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 172.898582] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 172.910393] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 172.918796] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 172.925627] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 172.931494] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 172.937107] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 172.942724] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 172.948593] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 172.956644] READ: bus=0 devfn=0 where=136 size=2 val=0x281f\n[ 172.962428] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 172.968295] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 172.973903] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 172.979771] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 172.985379] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 172.990985] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 173.002799] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 173.011202] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 173.018033] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 173.023901] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 173.029510] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 173.035123] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 173.040990] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 173.049042] READ: bus=0 devfn=0 where=136 size=2 val=0x281f\n[ 173.054825] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 173.060693] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 173.066305] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 173.072173] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 173.077780] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 173.083388] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 173.095202] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 173.103605] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 173.110435] READ: bus=0 devfn=0 where=2052 size=4 val=0x4000\n[ 173.116303] READ: bus=0 devfn=0 where=2056 size=4 val=0x0\n[ 173.121911] READ: bus=0 devfn=0 where=2072 size=4 val=0xe\n[ 173.127524] pcieport 0000:00:00.0: AER: Device recovery failed\n\n\n\n\nNOTE BENE: these issues do not occur at all with a USB2 Flash drive.\n\n[ 2093.564771] usb 1-2: new high-speed USB device number 2 using xhci_hcd\n[ 2093.790646] usb 1-2: New USB device found, idVendor=058f, idProduct=6387\n[ 2093.797397] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3\n[ 2093.804583] usb 1-2: Product: Mass Storage\n[ 2093.808707] usb 1-2: Manufacturer: Generic\n[ 2093.812829] usb 1-2: SerialNumber: 31A69E70\n[ 2093.819244] usb-storage 1-2:1.0: USB Mass Storage device detected\n[ 2093.825624] scsi host0: usb-storage 1-2:1.0\n[ 2094.856918] scsi 0:0:0:0: Direct-Access Generic Flash Disk 8.07 PQ: 0 ANSI: 2\n[ 2094.866196] sd 0:0:0:0: [sda] 4106240 512-byte logical blocks: (2.10 GB/1.96 GiB)\n[ 2094.874232] sd 0:0:0:0: [sda] Write Protect is off\n[ 2094.879350] sd 0:0:0:0: [sda] No Caching mode page found\n[ 2094.884816] sd 0:0:0:0: [sda] Assuming drive cache: write through\n[ 2094.909111] sda: sda1\n[ 2094.912935] sd 0:0:0:0: [sda] Attached SCSI removable disk\n\n[ 2100.516396] usb 1-2: USB disconnect, device number 2\n\n\nRegards.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xcqb557gFz9s8V\n\tfor ;\n\tThu, 24 Aug 2017 00:30:25 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S932089AbdHWOaX (ORCPT );\n\tWed, 23 Aug 2017 10:30:23 -0400","from smtp5-g21.free.fr ([212.27.42.5]:7218 \"EHLO smtp5-g21.free.fr\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S932080AbdHWOaX (ORCPT );\n\tWed, 23 Aug 2017 10:30:23 -0400","from [172.27.0.114] (unknown [92.154.11.170])\n\t(Authenticated sender: slash.tmp)\n\tby smtp5-g21.free.fr (Postfix) with ESMTPSA id 0E3F35FFD0;\n\tWed, 23 Aug 2017 16:30:04 +0200 (CEST)"],"Subject":"Re: Possible regression between 4.9 and 4.13","From":"Mason ","To":"Mathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ","Cc":"Bjorn Helgaas ,\n\tAlan Stern ,\n\tGreg Kroah-Hartman ","References":"<4dee5523-2d76-e731-6e81-f3027e88827f@free.fr>\n\t<87a82qbyv5.fsf@linux.intel.com> <599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>","Message-ID":"<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>","Date":"Wed, 23 Aug 2017 16:30:03 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 SeaMonkey/2.49.1","MIME-Version":"1.0","In-Reply-To":"<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1758438,"web_url":"http://patchwork.ozlabs.org/comment/1758438/","msgid":"<59A3D6BF.7010400@linux.intel.com>","list_archive_url":null,"date":"2017-08-28T08:39:27","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":63937,"url":"http://patchwork.ozlabs.org/api/people/63937/","name":"Mathias Nyman","email":"mathias.nyman@linux.intel.com"},"content":"On 23.08.2017 17:30, Mason wrote:\n> On 23/08/2017 14:41, Mason wrote:\n>\n>> I compiled a minimal kernel, with lots of irrelevant drivers and\n>> frameworks left out, including power management. I still get the\n>> \"xHCI host controller not responding, assume dead\" issue.\n>\n> The problem seems to have a timing-related aspect.\n>\n> I added a bunch of logs (to a slow serial console) and the HC was\n> not killed. I was able to plug the Flash drive a second time.\n> (I am logging config space reads and writes.)\n\n\nCould you take a log with the following added debug, without\nyour extra delays, It should show a bit more about the state\nof the controller when we read 0xffffffff\n\n\ndiff --git a/drivers/usb/host/xhci-hub.c b/drivers/usb/host/xhci-hub.c\nindex 4bc6f42..a124c3d 100644\n--- a/drivers/usb/host/xhci-hub.c\n+++ b/drivers/usb/host/xhci-hub.c\n@@ -23,6 +23,7 @@\n \n #include \n #include \n+#include \n \n #include \"xhci.h\"\n #include \"xhci-trace.h\"\n@@ -1280,7 +1281,11 @@ int xhci_hub_control(struct usb_hcd *hcd, u16 typeReq, u16 wValue,\n wIndex--;\n temp = readl(port_array[wIndex]);\n if (temp == ~(u32)0) {\n- xhci_hc_died(xhci);\n+ struct pci_dev *pdev = to_pci_dev(hcd->self.controller);\n+ xhci_err(xhci, \"ClearPortFeat port%d @%p=%x, hcd->state:0x%x hcd->flags:0x%x, pci_state 0x%x\\n\",\n+ wIndex, port_array[wIndex], temp, hcd->state, hcd->flags, pdev->current_state);\n+\n+ WARN_ON(1);\n retval = -ENODEV;\n break;\n }\n\nThanks\n-Mathias","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xglVM68Ttz9sNr\n\tfor ;\n\tMon, 28 Aug 2017 18:36:27 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1752463AbdH1IgW (ORCPT );\n\tMon, 28 Aug 2017 04:36:22 -0400","from mga02.intel.com ([134.134.136.20]:52521 \"EHLO mga02.intel.com\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1752238AbdH1IgS (ORCPT );\n\tMon, 28 Aug 2017 04:36:18 -0400","from orsmga002.jf.intel.com ([10.7.209.21])\n\tby orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t28 Aug 2017 01:35:59 -0700","from mattu-haswell.fi.intel.com (HELO [10.237.72.164])\n\t([10.237.72.164])\n\tby orsmga002.jf.intel.com with ESMTP; 28 Aug 2017 01:35:57 -0700"],"X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos;i=\"5.41,440,1498546800\"; d=\"scan'208\";a=\"128902652\"","Subject":"Re: Possible regression between 4.9 and 4.13","To":"Mason , Mathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ","References":"<4dee5523-2d76-e731-6e81-f3027e88827f@free.fr>\n\t<87a82qbyv5.fsf@linux.intel.com> <599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>","Cc":"Bjorn Helgaas ,\n\tAlan Stern ,\n\tGreg Kroah-Hartman ","From":"Mathias Nyman ","Message-ID":"<59A3D6BF.7010400@linux.intel.com>","Date":"Mon, 28 Aug 2017 11:39:27 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101\n\tThunderbird/38.8.0","MIME-Version":"1.0","In-Reply-To":"<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1758615,"web_url":"http://patchwork.ozlabs.org/comment/1758615/","msgid":"<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>","list_archive_url":null,"date":"2017-08-28T14:40:39","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":66150,"url":"http://patchwork.ozlabs.org/api/people/66150/","name":"Mason","email":"slash.tmp@free.fr"},"content":"On 28/08/2017 10:39, Mathias Nyman wrote:\n\n> Could you take a log with the following added debug, without\n> your extra delays, It should show a bit more about the state\n> of the controller when we read 0xffffffff\n\nI applied the following patch on top of v4.12-rc1\n\ndiff --git a/drivers/usb/host/xhci-hub.c b/drivers/usb/host/xhci-hub.c\nindex 5e3e9d4c6956..c7ea7d4c801f 100644\n--- a/drivers/usb/host/xhci-hub.c\n+++ b/drivers/usb/host/xhci-hub.c\n@@ -23,6 +23,7 @@\n \n #include \n #include \n+#include \n \n #include \"xhci.h\"\n #include \"xhci-trace.h\"\n@@ -1268,7 +1269,10 @@ int xhci_hub_control(struct usb_hcd *hcd, u16 typeReq, u16 wValue,\n \t\twIndex--;\n \t\ttemp = readl(port_array[wIndex]);\n \t\tif (temp == ~(u32)0) {\n-\t\t\txhci_hc_died(xhci);\n+\t\t\tstruct pci_dev *pdev = to_pci_dev(hcd->self.controller);\n+\t\t\txhci_err(xhci, \"ClearPortFeat port%d @%p=%x, hcd->state:0x%x hcd->flags:0x%x, pci_state 0x%x\\n\",\n+\t\t\t\t\twIndex, port_array[wIndex], temp, hcd->state, hcd->flags, pdev->current_state);\n+\t\t\tWARN_ON(1);\n \t\t\tretval = -ENODEV;\n \t\t\tbreak;\n \t\t}\n\n\nAnd here are logs I get when I plug/unplug my USB3 device.\n\n[ 14.970148] usb 2-2: new SuperSpeed USB device number 2 using xhci_hcd\n[ 15.003487] usb 2-2: New USB device found, idVendor=0951, idProduct=1666\n[ 15.010237] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3\n[ 15.017483] usb 2-2: Product: DataTraveler 3.0\n[ 15.021990] usb 2-2: Manufacturer: Kingston\n[ 15.026234] usb 2-2: SerialNumber: 002618887865F0C0F8646BFA\n[ 15.034830] usb-storage 2-2:1.0: USB Mass Storage device detected\n[ 15.041269] scsi host0: usb-storage 2-2:1.0\n[ 16.056140] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 3.0 PQ: 0 ANSI: 6\n[ 16.064979] sd 0:0:0:0: [sda] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)\n[ 16.072978] sd 0:0:0:0: [sda] Write Protect is off\n[ 16.078076] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA\n[ 16.089417] sda: sda1\n[ 16.093050] sd 0:0:0:0: [sda] Attached SCSI removable disk\n\n\n[ 22.152078] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 22.160157] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 22.172051] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 22.180493] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 22.187368] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 22.885269] xhci_hcd 0000:01:00.0: ClearPortFeat port1 @e0852430=ffffffff, hcd->state:0x1 hcd->flags:0x1a5, pci_state 0x0\n[ 22.896284] ------------[ cut here ]------------\n[ 22.900938] WARNING: CPU: 0 PID: 127 at drivers/usb/host/xhci-hub.c:1275 xhci_hub_control+0x10f4/0x1778\n[ 22.910377] Modules linked in:\n[ 22.913447] CPU: 0 PID: 127 Comm: kworker/0:1 Tainted: G C 4.12.0-rc1 #4\n[ 22.921314] Hardware name: Sigma Tango DT\n[ 22.925342] Workqueue: usb_hub_wq hub_event\n[ 22.929564] [] (unwind_backtrace) from [] (show_stack+0x10/0x14)\n[ 22.937353] [] (show_stack) from [] (dump_stack+0x84/0x98)\n[ 22.944617] [] (dump_stack) from [] (__warn+0xe8/0x100)\n[ 22.951616] [] (__warn) from [] (warn_slowpath_null+0x20/0x28)\n[ 22.959227] [] (warn_slowpath_null) from [] (xhci_hub_control+0x10f4/0x1778)\n[ 22.968062] [] (xhci_hub_control) from [] (usb_hcd_submit_urb+0x264/0x810)\n[ 22.976719] [] (usb_hcd_submit_urb) from [] (usb_submit_urb+0x2b0/0x4b4)\n[ 22.985201] [] (usb_submit_urb) from [] (usb_start_wait_urb+0x4c/0xbc)\n[ 22.993509] [] (usb_start_wait_urb) from [] (usb_control_msg+0xa0/0xcc)\n[ 23.001904] [] (usb_control_msg) from [] (usb_clear_port_feature+0x44/0x4c)\n[ 23.010648] [] (usb_clear_port_feature) from [] (hub_port_reset+0x228/0x51c)\n[ 23.019479] [] (hub_port_reset) from [] (hub_event+0x1f4/0xe64)\n[ 23.027177] [] (hub_event) from [] (process_one_work+0x1d4/0x3ec)\n[ 23.035049] [] (process_one_work) from [] (worker_thread+0x38/0x554)\n[ 23.043185] [] (worker_thread) from [] (kthread+0x108/0x138)\n[ 23.050620] [] (kthread) from [] (ret_from_fork+0x14/0x3c)\n[ 23.057877] ---[ end trace 5e4494cf1f6e3761 ]---\n[ 23.062691] xhci_hcd 0000:01:00.0: ClearPortFeat port1 @e0852430=ffffffff, hcd->state:0x1 hcd->flags:0x1a5, pci_state 0x0\n[ 23.073707] ------------[ cut here ]------------\n[ 23.078349] WARNING: CPU: 0 PID: 127 at drivers/usb/host/xhci-hub.c:1275 xhci_hub_control+0x10f4/0x1778\n[ 23.087787] Modules linked in:\n[ 23.090854] CPU: 0 PID: 127 Comm: kworker/0:1 Tainted: G WC 4.12.0-rc1 #4\n[ 23.098720] Hardware name: Sigma Tango DT\n[ 23.102745] Workqueue: usb_hub_wq hub_event\n[ 23.106953] [] (unwind_backtrace) from [] (show_stack+0x10/0x14)\n[ 23.114737] [] (show_stack) from [] (dump_stack+0x84/0x98)\n[ 23.121998] [] (dump_stack) from [] (__warn+0xe8/0x100)\n[ 23.128996] [] (__warn) from [] (warn_slowpath_null+0x20/0x28)\n[ 23.136606] [] (warn_slowpath_null) from [] (xhci_hub_control+0x10f4/0x1778)\n[ 23.145439] [] (xhci_hub_control) from [] (usb_hcd_submit_urb+0x264/0x810)\n[ 23.154095] [] (usb_hcd_submit_urb) from [] (usb_submit_urb+0x2b0/0x4b4)\n[ 23.162577] [] (usb_submit_urb) from [] (usb_start_wait_urb+0x4c/0xbc)\n[ 23.170884] [] (usb_start_wait_urb) from [] (usb_control_msg+0xa0/0xcc)\n[ 23.179278] [] (usb_control_msg) from [] (usb_clear_port_feature+0x44/0x4c)\n[ 23.188021] [] (usb_clear_port_feature) from [] (hub_port_reset+0x248/0x51c)\n[ 23.196851] [] (hub_port_reset) from [] (hub_event+0x1f4/0xe64)\n[ 23.204547] [] (hub_event) from [] (process_one_work+0x1d4/0x3ec)\n[ 23.212418] [] (process_one_work) from [] (worker_thread+0x38/0x554)\n[ 23.220551] [] (worker_thread) from [] (kthread+0x108/0x138)\n[ 23.227986] [] (kthread) from [] (ret_from_fork+0x14/0x3c)\n[ 23.235242] ---[ end trace 5e4494cf1f6e3762 ]---\n[ 23.239953] xhci_hcd 0000:01:00.0: Cannot set link state.\n[ 23.245403] usb usb2-port2: cannot disable (err = -32)\n[ 23.250575] usb 2-2: USB disconnect, device number 2\n[ 23.255724] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 23.264048] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 23.275985] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 23.284417] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 23.291256] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 23.297144] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 23.305218] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 23.317047] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 23.325467] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 23.332309] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 23.338188] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 23.346273] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 23.358093] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 23.366518] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 23.373357] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 23.379229] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 23.387287] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 23.399101] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 23.407504] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 23.414344] pcieport 0000:00:00.0: AER: Device recovery failed\n[ 23.434143] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n[ 23.442263] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n[ 23.454100] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n[ 23.462542] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n[ 23.469427] pcieport 0000:00:00.0: AER: Device recovery failed\n\n\nRegards.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xgvb423fpz9s7p\n\tfor ;\n\tTue, 29 Aug 2017 00:41:04 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751236AbdH1OlB (ORCPT );\n\tMon, 28 Aug 2017 10:41:01 -0400","from smtp5-g21.free.fr ([212.27.42.5]:1857 \"EHLO smtp5-g21.free.fr\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1751170AbdH1Ok7 (ORCPT );\n\tMon, 28 Aug 2017 10:40:59 -0400","from [172.27.0.114] (unknown [92.154.11.170])\n\t(Authenticated sender: slash.tmp)\n\tby smtp5-g21.free.fr (Postfix) with ESMTPSA id 1D8975FF67;\n\tMon, 28 Aug 2017 16:40:40 +0200 (CEST)"],"Subject":"Re: Possible regression between 4.9 and 4.13","To":"Mathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ","Cc":"Bjorn Helgaas ,\n\tAlan Stern ,\n\tGreg Kroah-Hartman ","References":"<4dee5523-2d76-e731-6e81-f3027e88827f@free.fr>\n\t<87a82qbyv5.fsf@linux.intel.com> <599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>","From":"Mason ","Message-ID":"<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>","Date":"Mon, 28 Aug 2017 16:40:39 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 SeaMonkey/2.49.1","MIME-Version":"1.0","In-Reply-To":"<59A3D6BF.7010400@linux.intel.com>","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759334,"web_url":"http://patchwork.ozlabs.org/comment/1759334/","msgid":"<59A56C15.2000403@linux.intel.com>","list_archive_url":null,"date":"2017-08-29T13:28:53","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":63937,"url":"http://patchwork.ozlabs.org/api/people/63937/","name":"Mathias Nyman","email":"mathias.nyman@linux.intel.com"},"content":"On 28.08.2017 17:40, Mason wrote:\n> On 28/08/2017 10:39, Mathias Nyman wrote:\n>\n>> Could you take a log with the following added debug, without\n>> your extra delays, It should show a bit more about the state\n>> of the controller when we read 0xffffffff\n>\n> I applied the following patch on top of v4.12-rc1\n>\n> diff --git a/drivers/usb/host/xhci-hub.c b/drivers/usb/host/xhci-hub.c\n> index 5e3e9d4c6956..c7ea7d4c801f 100644\n> --- a/drivers/usb/host/xhci-hub.c\n> +++ b/drivers/usb/host/xhci-hub.c\n> @@ -23,6 +23,7 @@\n>\n> #include \n> #include \n> +#include \n>\n> #include \"xhci.h\"\n> #include \"xhci-trace.h\"\n> @@ -1268,7 +1269,10 @@ int xhci_hub_control(struct usb_hcd *hcd, u16 typeReq, u16 wValue,\n> \t\twIndex--;\n> \t\ttemp = readl(port_array[wIndex]);\n> \t\tif (temp == ~(u32)0) {\n> -\t\t\txhci_hc_died(xhci);\n> +\t\t\tstruct pci_dev *pdev = to_pci_dev(hcd->self.controller);\n> +\t\t\txhci_err(xhci, \"ClearPortFeat port%d @%p=%x, hcd->state:0x%x hcd->flags:0x%x, pci_state 0x%x\\n\",\n> +\t\t\t\t\twIndex, port_array[wIndex], temp, hcd->state, hcd->flags, pdev->current_state);\n> +\t\t\tWARN_ON(1);\n> \t\t\tretval = -ENODEV;\n> \t\t\tbreak;\n> \t\t}\n>\n>\n> And here are logs I get when I plug/unplug my USB3 device.\n>\n> [ 14.970148] usb 2-2: new SuperSpeed USB device number 2 using xhci_hcd\n> [ 15.003487] usb 2-2: New USB device found, idVendor=0951, idProduct=1666\n> [ 15.010237] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3\n> [ 15.017483] usb 2-2: Product: DataTraveler 3.0\n> [ 15.021990] usb 2-2: Manufacturer: Kingston\n> [ 15.026234] usb 2-2: SerialNumber: 002618887865F0C0F8646BFA\n> [ 15.034830] usb-storage 2-2:1.0: USB Mass Storage device detected\n> [ 15.041269] scsi host0: usb-storage 2-2:1.0\n> [ 16.056140] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 3.0 PQ: 0 ANSI: 6\n> [ 16.064979] sd 0:0:0:0: [sda] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)\n> [ 16.072978] sd 0:0:0:0: [sda] Write Protect is off\n> [ 16.078076] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA\n> [ 16.089417] sda: sda1\n> [ 16.093050] sd 0:0:0:0: [sda] Attached SCSI removable disk\n>\n>\n> [ 22.152078] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error received: id=0000\n> [ 22.160157] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, id=0000(Requester ID)\n> [ 22.172051] pcieport 0000:00:00.0: device [1105:0024] error status/mask=00004000/00000000\n> [ 22.180493] pcieport 0000:00:00.0: [14] Completion Timeout (First)\n> [ 22.187368] pcieport 0000:00:00.0: AER: Device recovery failed\n> [ 22.885269] xhci_hcd 0000:01:00.0: ClearPortFeat port1 @e0852430=ffffffff, hcd->state:0x1 hcd->flags:0x1a5, pci_state 0x0\n\nState is HC_STATE_RUNNING,\n\nFlags bits 0,2,5,7,8 set:\n#define HCD_FLAG_HW_ACCESSIBLE 0 /* at full power */\n#define HCD_FLAG_POLL_RH 2 /* poll for rh status? */\n#define HCD_FLAG_RH_RUNNING 5 /* root hub is running? */\n#define HCD_FLAG_INTF_AUTHORIZED 7 /* authorize interfaces? */\n#define HCD_FLAG_DEV_AUTHORIZED 8 /* authorize devices? */\n\nAnd pci state seems to be D0 (according to driver, pdev->current_state)\n\nI can't see anything wrong from xhci/usb point of view.\nI'd focus more on the PCI errors in the logs as the cause for reading 0xffffffff from xhci mmio.\n\nThen again it might be a bit too drastic to kill xhci just because we read 0xffffffff once\nfrom a mmio xhci register. Maybe we should return an error a couple times before actually\ntearing down xhci.\n\nThis tight check was originally done to detect pci hotplug removed hosts as soon as possible.\n\n-Mathias","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xhTsN2cp3z9t2v\n\tfor ;\n\tTue, 29 Aug 2017 23:25:28 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1752693AbdH2NZ1 (ORCPT );\n\tTue, 29 Aug 2017 09:25:27 -0400","from mga11.intel.com ([192.55.52.93]:17072 \"EHLO mga11.intel.com\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1752648AbdH2NZ0 (ORCPT );\n\tTue, 29 Aug 2017 09:25:26 -0400","from fmsmga003.fm.intel.com ([10.253.24.29])\n\tby fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t29 Aug 2017 06:25:23 -0700","from mattu-haswell.fi.intel.com (HELO [10.237.72.164])\n\t([10.237.72.164])\n\tby FMSMGA003.fm.intel.com with ESMTP; 29 Aug 2017 06:25:21 -0700"],"X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos;i=\"5.41,444,1498546800\"; d=\"scan'208\";a=\"895168136\"","Subject":"Re: Possible regression between 4.9 and 4.13","To":"Mason , Felipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ","References":"<4dee5523-2d76-e731-6e81-f3027e88827f@free.fr>\n\t<87a82qbyv5.fsf@linux.intel.com> <599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>","Cc":"Bjorn Helgaas ,\n\tAlan Stern ,\n\tGreg Kroah-Hartman ","From":"Mathias Nyman ","Message-ID":"<59A56C15.2000403@linux.intel.com>","Date":"Tue, 29 Aug 2017 16:28:53 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101\n\tThunderbird/38.8.0","MIME-Version":"1.0","In-Reply-To":"<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759345,"web_url":"http://patchwork.ozlabs.org/comment/1759345/","msgid":"<20170829133852.GA13355@wunner.de>","list_archive_url":null,"date":"2017-08-29T13:38:52","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":68499,"url":"http://patchwork.ozlabs.org/api/people/68499/","name":"Lukas Wunner","email":"lukas@wunner.de"},"content":"On Tue, Aug 29, 2017 at 04:28:53PM +0300, Mathias Nyman wrote:\n> Then again it might be a bit too drastic to kill xhci just because\n> we read 0xffffffff once from a mmio xhci register. Maybe we should\n> return an error a couple times before actually tearing down xhci.\n> \n> This tight check was originally done to detect pci hotplug removed\n> hosts as soon as possible.\n\nJust make pci_dev_is_disconnected() public to detect PCI hot removal.\nWe *know* when the device was hot removed, so I think there's no need\nto guess that based on reading \"all ones\" from mmio (which may happen\nfor entirely legitimate reasons unrelated to hot removal).\n\nThanks,\n\nLukas","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xhV900g7Bz9sRV\n\tfor ;\n\tTue, 29 Aug 2017 23:39:00 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751935AbdH2Ni6 (ORCPT );\n\tTue, 29 Aug 2017 09:38:58 -0400","from mailout3.hostsharing.net ([176.9.242.54]:47873 \"EHLO\n\tmailout3.hostsharing.net\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1753505AbdH2Ni5 (ORCPT\n\t); Tue, 29 Aug 2017 09:38:57 -0400","from h08.hostsharing.net (h08.hostsharing.net [83.223.95.28])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby mailout3.hostsharing.net (Postfix) with ESMTPS id 220B6101F0C72;\n\tTue, 29 Aug 2017 15:38:53 +0200 (CEST)","by h08.hostsharing.net (Postfix, from userid 100393)\n\tid C03442D046; Tue, 29 Aug 2017 15:38:52 +0200 (CEST)"],"Date":"Tue, 29 Aug 2017 15:38:52 +0200","From":"Lukas Wunner ","To":"Mathias Nyman ","Cc":"Mason , Felipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ,\n\tBjorn Helgaas ,\n\tAlan Stern ,\n\tGreg Kroah-Hartman ","Subject":"Re: Possible regression between 4.9 and 4.13","Message-ID":"<20170829133852.GA13355@wunner.de>","References":"<87a82qbyv5.fsf@linux.intel.com> <599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<59A56C15.2000403@linux.intel.com>","User-Agent":"Mutt/1.5.23 (2014-03-12)","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759407,"web_url":"http://patchwork.ozlabs.org/comment/1759407/","msgid":"<20170829144725.GB22532@kroah.com>","list_archive_url":null,"date":"2017-08-29T14:47:25","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":11800,"url":"http://patchwork.ozlabs.org/api/people/11800/","name":"Greg Kroah-Hartman","email":"gregkh@linuxfoundation.org"},"content":"On Tue, Aug 29, 2017 at 03:38:52PM +0200, Lukas Wunner wrote:\n> On Tue, Aug 29, 2017 at 04:28:53PM +0300, Mathias Nyman wrote:\n> > Then again it might be a bit too drastic to kill xhci just because\n> > we read 0xffffffff once from a mmio xhci register. Maybe we should\n> > return an error a couple times before actually tearing down xhci.\n> > \n> > This tight check was originally done to detect pci hotplug removed\n> > hosts as soon as possible.\n> \n> Just make pci_dev_is_disconnected() public to detect PCI hot removal.\n> We *know* when the device was hot removed, so I think there's no need\n> to guess that based on reading \"all ones\" from mmio (which may happen\n> for entirely legitimate reasons unrelated to hot removal).\n\nNo, you don't always \"know\" when a device is removed, don't rely on it,\nnot all platforms support that.\n\nOne more reason why I hate that function and I'm glad it's not exported\nfor others to think it somehow actually works for their system...\n\nReading all ff shows the device is removed, that's all the PCI spec\nguarantees. What other legitimate reason could that happen for?\n\nthanks,\n\ngreg k-h","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xhWgt2lltz9sR9\n\tfor ;\n\tWed, 30 Aug 2017 00:47:22 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751377AbdH2OrU (ORCPT );\n\tTue, 29 Aug 2017 10:47:20 -0400","from mail.linuxfoundation.org ([140.211.169.12]:35668 \"EHLO\n\tmail.linuxfoundation.org\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751297AbdH2OrT (ORCPT\n\t); Tue, 29 Aug 2017 10:47:19 -0400","from localhost (LFbn-1-12253-150.w90-92.abo.wanadoo.fr\n\t[90.92.67.150])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPSA id E05689B9;\n\tTue, 29 Aug 2017 14:47:18 +0000 (UTC)"],"Date":"Tue, 29 Aug 2017 16:47:25 +0200","From":"Greg Kroah-Hartman ","To":"Lukas Wunner ","Cc":"Mathias Nyman , Mason ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ,\n\tBjorn Helgaas ,\n\tAlan Stern ","Subject":"Re: Possible regression between 4.9 and 4.13","Message-ID":"<20170829144725.GB22532@kroah.com>","References":"<599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com>\n\t<20170829133852.GA13355@wunner.de>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20170829133852.GA13355@wunner.de>","User-Agent":"Mutt/1.8.3 (2017-05-23)","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759461,"web_url":"http://patchwork.ozlabs.org/comment/1759461/","msgid":"<20170829153456.GA13712@wunner.de>","list_archive_url":null,"date":"2017-08-29T15:34:56","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":68499,"url":"http://patchwork.ozlabs.org/api/people/68499/","name":"Lukas Wunner","email":"lukas@wunner.de"},"content":"On Tue, Aug 29, 2017 at 04:47:25PM +0200, Greg Kroah-Hartman wrote:\n> On Tue, Aug 29, 2017 at 03:38:52PM +0200, Lukas Wunner wrote:\n> > On Tue, Aug 29, 2017 at 04:28:53PM +0300, Mathias Nyman wrote:\n> > > Then again it might be a bit too drastic to kill xhci just because\n> > > we read 0xffffffff once from a mmio xhci register. Maybe we should\n> > > return an error a couple times before actually tearing down xhci.\n> > > \n> > > This tight check was originally done to detect pci hotplug removed\n> > > hosts as soon as possible.\n> > \n> > Just make pci_dev_is_disconnected() public to detect PCI hot removal.\n> > We *know* when the device was hot removed, so I think there's no need\n> > to guess that based on reading \"all ones\" from mmio (which may happen\n> > for entirely legitimate reasons unrelated to hot removal).\n> \n> No, you don't always \"know\" when a device is removed, don't rely on it,\n> not all platforms support that.\n\nPlease explain, which platforms don't support that? They wouldn't be\ncompliant with the spec it seems.\n\nPCIe r3.1, section 6.7.3:\n\n\t\"A Downstream Port with hot-plug capabilities supports the\n\t following hot-plug events:\n\n\t Presence Detect Changed\n\n\t A Downstream Port with hot-plug capabilities monitors the slot\n\t it controls for the slot events listed above. [...]\n\t If enabled through the associated enable field, slot events\n\t must generate a software notification.\"\n\nAnd pciehp sets the flag on all downstream devices that they're removed\nonce the software notification has been received and processed.\n\n\n> Reading all ff shows the device is removed, that's all the PCI spec\n> guarantees. What other legitimate reason could that happen for?\n\nIs 0xffffffff not a valid value to be stored in and read from mmio space?\n\nBest regards,\n\nLukas","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xhXkv0M7Kz9t33\n\tfor ;\n\tWed, 30 Aug 2017 01:35:03 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751443AbdH2PfB (ORCPT );\n\tTue, 29 Aug 2017 11:35:01 -0400","from mailout1.hostsharing.net ([83.223.95.204]:37847 \"EHLO\n\tmailout1.hostsharing.net\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751280AbdH2PfA (ORCPT\n\t); Tue, 29 Aug 2017 11:35:00 -0400","from h08.hostsharing.net (h08.hostsharing.net [83.223.95.28])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby mailout1.hostsharing.net (Postfix) with ESMTPS id 071B1101621D1;\n\tTue, 29 Aug 2017 17:34:59 +0200 (CEST)","from localhost (5-38-90-81.adsl.cmo.de [81.90.38.5])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits))\n\t(No client certificate requested)\n\tby h08.hostsharing.net (Postfix) with ESMTPSA id AB419603E121;\n\tTue, 29 Aug 2017 17:34:57 +0200 (CEST)"],"Date":"Tue, 29 Aug 2017 17:34:56 +0200","From":"Lukas Wunner ","To":"Greg Kroah-Hartman ","Cc":"Mathias Nyman , Mason ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ,\n\tBjorn Helgaas ,\n\tAlan Stern ","Subject":"Re: Possible regression between 4.9 and 4.13","Message-ID":"<20170829153456.GA13712@wunner.de>","References":"<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com>\n\t<20170829133852.GA13355@wunner.de>\n\t<20170829144725.GB22532@kroah.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20170829144725.GB22532@kroah.com>","User-Agent":"Mutt/1.6.1 (2016-04-27)","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759486,"web_url":"http://patchwork.ozlabs.org/comment/1759486/","msgid":"<20170829155138.GA32369@kroah.com>","list_archive_url":null,"date":"2017-08-29T15:51:38","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":11800,"url":"http://patchwork.ozlabs.org/api/people/11800/","name":"Greg Kroah-Hartman","email":"gregkh@linuxfoundation.org"},"content":"On Tue, Aug 29, 2017 at 05:34:56PM +0200, Lukas Wunner wrote:\n> On Tue, Aug 29, 2017 at 04:47:25PM +0200, Greg Kroah-Hartman wrote:\n> > On Tue, Aug 29, 2017 at 03:38:52PM +0200, Lukas Wunner wrote:\n> > > On Tue, Aug 29, 2017 at 04:28:53PM +0300, Mathias Nyman wrote:\n> > > > Then again it might be a bit too drastic to kill xhci just because\n> > > > we read 0xffffffff once from a mmio xhci register. Maybe we should\n> > > > return an error a couple times before actually tearing down xhci.\n> > > > \n> > > > This tight check was originally done to detect pci hotplug removed\n> > > > hosts as soon as possible.\n> > > \n> > > Just make pci_dev_is_disconnected() public to detect PCI hot removal.\n> > > We *know* when the device was hot removed, so I think there's no need\n> > > to guess that based on reading \"all ones\" from mmio (which may happen\n> > > for entirely legitimate reasons unrelated to hot removal).\n> > \n> > No, you don't always \"know\" when a device is removed, don't rely on it,\n> > not all platforms support that.\n> \n> Please explain, which platforms don't support that? They wouldn't be\n> compliant with the spec it seems.\n> \n> PCIe r3.1, section 6.7.3:\n> \n> \t\"A Downstream Port with hot-plug capabilities supports the\n> \t following hot-plug events:\n> \n> \t Presence Detect Changed\n> \n> \t A Downstream Port with hot-plug capabilities monitors the slot\n> \t it controls for the slot events listed above. [...]\n> \t If enabled through the associated enable field, slot events\n> \t must generate a software notification.\"\n> \n> And pciehp sets the flag on all downstream devices that they're removed\n> once the software notification has been received and processed.\n\nWhat about all of the non-pciehp platforms? :)\n\nAlso, there is always a race between when that notification has been\nsent and processed on the PCIe channel and the reading of all 1s on the\nPCI bus by the driver.\n\nFor fun, try disconnecting a USB device from one of the more modern\nlaptops with a USB 3.1 connection on it. The bios treats those as a pci\nhotpluggable xhci controller on the PCI bus. When the device is\ndisconnected, the BIOS rips out the PCI device as well, but all that\ntime, the xhci driver is thinking the device is still present as it\ntakes a while for the BIOS to do all of the needed housekeeping. It's a\nreally long time for everything to shut down and to help prevent the\ndriver from going crazy, it has to detect ffff reads as \"disconnection\nhappened\".\n\nAll PCI drivers have had to do this for decades now, it's nothing new\nhere, PCIe just gave us a chance to be notified that the device really\nis gone now, PCI hotplug has always been out-of-band like this.\n\n> > Reading all ff shows the device is removed, that's all the PCI spec\n> > guarantees. What other legitimate reason could that happen for?\n> \n> Is 0xffffffff not a valid value to be stored in and read from mmio space?\n\nFor a specific register, doubtful, which is why the code errors out,\nright? If it is a valid value, then it shouldn't be exiting, and move\non to the next read.\n\nI don't understand what we are arguing about here anymore...\n\nthanks,\n\ngreg k-h","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xhY601CY0z9t2v\n\tfor ;\n\tWed, 30 Aug 2017 01:51:36 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1752810AbdH2Pve (ORCPT );\n\tTue, 29 Aug 2017 11:51:34 -0400","from mail.linuxfoundation.org ([140.211.169.12]:45188 \"EHLO\n\tmail.linuxfoundation.org\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751867AbdH2Pvd (ORCPT\n\t); Tue, 29 Aug 2017 11:51:33 -0400","from localhost (LFbn-1-12253-150.w90-92.abo.wanadoo.fr\n\t[90.92.67.150])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPSA id 5527BA81;\n\tTue, 29 Aug 2017 15:51:32 +0000 (UTC)"],"Date":"Tue, 29 Aug 2017 17:51:38 +0200","From":"Greg Kroah-Hartman ","To":"Lukas Wunner ","Cc":"Mathias Nyman , Mason ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ,\n\tBjorn Helgaas ,\n\tAlan Stern ","Subject":"Re: Possible regression between 4.9 and 4.13","Message-ID":"<20170829155138.GA32369@kroah.com>","References":"<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com>\n\t<20170829133852.GA13355@wunner.de>\n\t<20170829144725.GB22532@kroah.com>\n\t<20170829153456.GA13712@wunner.de>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20170829153456.GA13712@wunner.de>","User-Agent":"Mutt/1.8.3 (2017-05-23)","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759773,"web_url":"http://patchwork.ozlabs.org/comment/1759773/","msgid":"<20170829235310.GA20214@wunner.de>","list_archive_url":null,"date":"2017-08-29T23:53:10","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":68499,"url":"http://patchwork.ozlabs.org/api/people/68499/","name":"Lukas Wunner","email":"lukas@wunner.de"},"content":"On Tue, Aug 29, 2017 at 04:28:53PM +0300, Mathias Nyman wrote:\n> This tight check was originally done to detect pci hotplug removed\n> hosts as soon as possible.\n\nIn Mason's case, the parent of the XHCI controller isn't a hotplug port,\nsee this lspci output:\n\nhttps://www.spinics.net/lists/linux-usb/msg160010.html\n\nPlease check is_hotplug_bridge in the parent's struct pci_dev before\nassuming that the XHCI controller was unplugged.\n\nThanks,\n\nLukas","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xhlnk6wlhz9sMN\n\tfor ;\n\tWed, 30 Aug 2017 09:53:14 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751425AbdH2XxN (ORCPT );\n\tTue, 29 Aug 2017 19:53:13 -0400","from mailout1.hostsharing.net ([83.223.95.204]:38059 \"EHLO\n\tmailout1.hostsharing.net\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751318AbdH2XxM (ORCPT\n\t); Tue, 29 Aug 2017 19:53:12 -0400","from h08.hostsharing.net (h08.hostsharing.net [83.223.95.28])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby mailout1.hostsharing.net (Postfix) with ESMTPS id D9E6E1018D6AA;\n\tWed, 30 Aug 2017 01:53:10 +0200 (CEST)","by h08.hostsharing.net (Postfix, from userid 100393)\n\tid C67F25905B; Wed, 30 Aug 2017 01:53:10 +0200 (CEST)"],"Date":"Wed, 30 Aug 2017 01:53:10 +0200","From":"Lukas Wunner ","To":"Mathias Nyman ","Cc":"Mason , Felipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ,\n\tBjorn Helgaas ,\n\tAlan Stern ,\n\tGreg Kroah-Hartman ","Subject":"Re: Possible regression between 4.9 and 4.13","Message-ID":"<20170829235310.GA20214@wunner.de>","References":"<87a82qbyv5.fsf@linux.intel.com> <599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<59A56C15.2000403@linux.intel.com>","User-Agent":"Mutt/1.5.23 (2014-03-12)","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759892,"web_url":"http://patchwork.ozlabs.org/comment/1759892/","msgid":"<20170830060237.GA2782@kroah.com>","list_archive_url":null,"date":"2017-08-30T06:02:37","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":11800,"url":"http://patchwork.ozlabs.org/api/people/11800/","name":"Greg Kroah-Hartman","email":"gregkh@linuxfoundation.org"},"content":"On Wed, Aug 30, 2017 at 01:53:10AM +0200, Lukas Wunner wrote:\n> On Tue, Aug 29, 2017 at 04:28:53PM +0300, Mathias Nyman wrote:\n> > This tight check was originally done to detect pci hotplug removed\n> > hosts as soon as possible.\n> \n> In Mason's case, the parent of the XHCI controller isn't a hotplug port,\n> see this lspci output:\n> \n> https://www.spinics.net/lists/linux-usb/msg160010.html\n> \n> Please check is_hotplug_bridge in the parent's struct pci_dev before\n> assuming that the XHCI controller was unplugged.\n\nHow can you guarantee that this is set on some systems? Will it be set\non cardbus devices? What about on a \"normal\" system where I can just go\nand yank out a PCI card at will?\n\nI don't think this is a valid thing to check, and again, why are we\narguing this point? It's been this way since the 1990's, this isn't a\nnew thing...\n\nTo get back to the original issue here, the hardware seems to have died,\nthe driver stops talking to it, and all is good. The \"regression\" here\nis that we now properly can determine that the hardware is crap.\n\nSo, how do you think we should proceed, delay a bit longer before saying\nthe device is gone? How long is \"long enough\"? How many bus errors are\nwe allowed to tolerate (hint, the PCI spec says none...)\n\nMaybe someone wants to get to the root problem here, why is the hardware\nsuddenly reporting all 1s?\n\nthanks,\n\ngreg k-h","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xhw0V4xhRz9t0M\n\tfor ;\n\tWed, 30 Aug 2017 16:03:06 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1750844AbdH3GDE (ORCPT );\n\tWed, 30 Aug 2017 02:03:04 -0400","from mail.linuxfoundation.org ([140.211.169.12]:37404 \"EHLO\n\tmail.linuxfoundation.org\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1750835AbdH3GDE (ORCPT\n\t); Wed, 30 Aug 2017 02:03:04 -0400","from localhost (LFbn-1-12253-150.w90-92.abo.wanadoo.fr\n\t[90.92.67.150])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPSA id 90EF69F8;\n\tWed, 30 Aug 2017 06:02:32 +0000 (UTC)"],"Date":"Wed, 30 Aug 2017 08:02:37 +0200","From":"Greg Kroah-Hartman ","To":"Lukas Wunner ","Cc":"Mathias Nyman , Mason ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ,\n\tBjorn Helgaas ,\n\tAlan Stern ","Subject":"Re: Possible regression between 4.9 and 4.13","Message-ID":"<20170830060237.GA2782@kroah.com>","References":"<599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com>\n\t<20170829235310.GA20214@wunner.de>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20170829235310.GA20214@wunner.de>","User-Agent":"Mutt/1.8.3 (2017-05-23)","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759909,"web_url":"http://patchwork.ozlabs.org/comment/1759909/","msgid":"<20170830063623.GA16076@wunner.de>","list_archive_url":null,"date":"2017-08-30T06:36:23","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":68499,"url":"http://patchwork.ozlabs.org/api/people/68499/","name":"Lukas Wunner","email":"lukas@wunner.de"},"content":"On Tue, Aug 29, 2017 at 05:51:38PM +0200, Greg Kroah-Hartman wrote:\n> On Tue, Aug 29, 2017 at 05:34:56PM +0200, Lukas Wunner wrote:\n> > On Tue, Aug 29, 2017 at 04:47:25PM +0200, Greg Kroah-Hartman wrote:\n> > > On Tue, Aug 29, 2017 at 03:38:52PM +0200, Lukas Wunner wrote:\n> > Is 0xffffffff not a valid value to be stored in and read from mmio space?\n> \n> For a specific register, doubtful\n\nWell, \"doubtful\" means you don't know for sure.\n\nIt's fine to check for \"all ones\" as a heuristic if that's not a valid\nvalue for the register read, however a hotplug notification is a\n*definitive* indication the hardware is gone.\n\nI you seem to prefer forgoing a *definitive* indication for a mere\nheuristic, that doesn't make sense from my point of view.\n\n\n> It's a really long time for everything to shut down and to help\n> prevent the driver from going crazy, [...]\n> Also, there is always a race between when that notification has been\n> sent and processed on the PCIe channel and the reading of all 1s on the\n> PCI bus by the driver.\n\nYes I know that. In practice the interrupt signaling hot removal\ncomes fast enough to prevent drivers from \"going crazy\", as I've\nmentioned in this patch:\n\nhttps://patchwork.kernel.org/patch/9405255/\n\n\n> For fun, try disconnecting a USB device from one of the more modern\n> laptops with a USB 3.1 connection on it. The bios treats those as a pci\n> hotpluggable xhci controller on the PCI bus. When the device is\n> disconnected, the BIOS rips out the PCI device as well, but all that\n> time, the xhci driver is thinking the device is still present as it\n> takes a while for the BIOS to do all of the needed housekeeping.\n\nYes, that is the case with Thunderbolt 3 controllers on non-Macs:\nThe XHCI controller appears below downstream bridge 2 of the Thunderbolt\ncontroller's PCIe switch. Once the last device is removed, the PCIe\nswitch and all devices below it disappear and the controller is powered\ndown. The controller is thus only visible if powered up. On Mac this\nis all native instead: Native pciehp, native tunnel setup, native PM.\n\n\n> > > > Just make pci_dev_is_disconnected() public to detect PCI hot removal.\n> > > > We *know* when the device was hot removed, so I think there's no need\n> > > > to guess that based on reading \"all ones\" from mmio (which may happen\n> > > > for entirely legitimate reasons unrelated to hot removal).\n> > > \n> > > No, you don't always \"know\" when a device is removed, don't rely on it,\n> > > not all platforms support that.\n> > \n> > Please explain, which platforms don't support that? They wouldn't be\n> > compliant with the spec it seems.\n> \n> What about all of the non-pciehp platforms? :)\n\nFair enough, those should be extended to set PCI_DEV_DISCONNECTED as well.\n\nThanks,\n\nLukas","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xhwkz5C9rz9sN7\n\tfor ;\n\tWed, 30 Aug 2017 16:36:27 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1750761AbdH3Gg0 (ORCPT );\n\tWed, 30 Aug 2017 02:36:26 -0400","from mailout1.hostsharing.net ([83.223.95.204]:53201 \"EHLO\n\tmailout1.hostsharing.net\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1750756AbdH3GgZ (ORCPT\n\t); Wed, 30 Aug 2017 02:36:25 -0400","from h08.hostsharing.net (h08.hostsharing.net [83.223.95.28])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256\n\tbits)) (No client certificate requested)\n\tby mailout1.hostsharing.net (Postfix) with ESMTPS id 8DAA11018AEFB;\n\tWed, 30 Aug 2017 08:36:23 +0200 (CEST)","by h08.hostsharing.net (Postfix, from userid 100393)\n\tid 727DC4AAE2; Wed, 30 Aug 2017 08:36:23 +0200 (CEST)"],"Date":"Wed, 30 Aug 2017 08:36:23 +0200","From":"Lukas Wunner ","To":"Greg Kroah-Hartman ","Cc":"Mathias Nyman , Mason ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ,\n\tBjorn Helgaas ,\n\tAlan Stern ","Subject":"Re: Possible regression between 4.9 and 4.13","Message-ID":"<20170830063623.GA16076@wunner.de>","References":"\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com>\n\t<20170829133852.GA13355@wunner.de>\n\t<20170829144725.GB22532@kroah.com>\n\t<20170829153456.GA13712@wunner.de>\n\t<20170829155138.GA32369@kroah.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20170829155138.GA32369@kroah.com>","User-Agent":"Mutt/1.5.23 (2014-03-12)","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759914,"web_url":"http://patchwork.ozlabs.org/comment/1759914/","msgid":"<20170830064537.GA8856@kroah.com>","list_archive_url":null,"date":"2017-08-30T06:45:37","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":11800,"url":"http://patchwork.ozlabs.org/api/people/11800/","name":"Greg Kroah-Hartman","email":"gregkh@linuxfoundation.org"},"content":"On Wed, Aug 30, 2017 at 08:36:23AM +0200, Lukas Wunner wrote:\n> On Tue, Aug 29, 2017 at 05:51:38PM +0200, Greg Kroah-Hartman wrote:\n> > On Tue, Aug 29, 2017 at 05:34:56PM +0200, Lukas Wunner wrote:\n> > > On Tue, Aug 29, 2017 at 04:47:25PM +0200, Greg Kroah-Hartman wrote:\n> > > > On Tue, Aug 29, 2017 at 03:38:52PM +0200, Lukas Wunner wrote:\n> > > Is 0xffffffff not a valid value to be stored in and read from mmio space?\n> > \n> > For a specific register, doubtful\n> \n> Well, \"doubtful\" means you don't know for sure.\n> \n> It's fine to check for \"all ones\" as a heuristic if that's not a valid\n> value for the register read, however a hotplug notification is a\n> *definitive* indication the hardware is gone.\n> \n> I you seem to prefer forgoing a *definitive* indication for a mere\n> heuristic, that doesn't make sense from my point of view.\n\nI still don't know what you are arguing about here. The _driver_ knows\nif a specific read allows all ones as a valid return value. If it\nisn't, then the driver knows the device is now gone. It's that simple,\ndon't do that type of check if all ones is a valid read.\n\nAnd that's not what is happening here anyway, so again, what is this\ndiscussion about?\n\nUnless there's something specific we can do here for the xhci driver, I\nthink this thread is dead until someone determines what is going wrong\nwith the hardware the original reporter posted about.\n\ngreg k-h","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xhwxW1C56z9s3T\n\tfor ;\n\tWed, 30 Aug 2017 16:45:35 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1750860AbdH3Gpd (ORCPT );\n\tWed, 30 Aug 2017 02:45:33 -0400","from mail.linuxfoundation.org ([140.211.169.12]:38552 \"EHLO\n\tmail.linuxfoundation.org\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1750851AbdH3Gpc (ORCPT\n\t); Wed, 30 Aug 2017 02:45:32 -0400","from localhost (LFbn-1-12253-150.w90-92.abo.wanadoo.fr\n\t[90.92.67.150])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPSA id D9D1FAAC;\n\tWed, 30 Aug 2017 06:45:31 +0000 (UTC)"],"Date":"Wed, 30 Aug 2017 08:45:37 +0200","From":"Greg Kroah-Hartman ","To":"Lukas Wunner ","Cc":"Mathias Nyman , Mason ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ,\n\tBjorn Helgaas ,\n\tAlan Stern ","Subject":"Re: Possible regression between 4.9 and 4.13","Message-ID":"<20170830064537.GA8856@kroah.com>","References":"<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com>\n\t<20170829133852.GA13355@wunner.de>\n\t<20170829144725.GB22532@kroah.com>\n\t<20170829153456.GA13712@wunner.de>\n\t<20170829155138.GA32369@kroah.com>\n\t<20170830063623.GA16076@wunner.de>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20170830063623.GA16076@wunner.de>","User-Agent":"Mutt/1.8.3 (2017-05-23)","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759965,"web_url":"http://patchwork.ozlabs.org/comment/1759965/","msgid":"<678490ce-9381-e63e-7a12-33d3eff7f894@free.fr>","list_archive_url":null,"date":"2017-08-30T08:55:37","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":66150,"url":"http://patchwork.ozlabs.org/api/people/66150/","name":"Mason","email":"slash.tmp@free.fr"},"content":"On 30/08/2017 08:02, Greg Kroah-Hartman wrote:\n\n> To get back to the original issue here, the hardware seems to have died,\n> the driver stops talking to it, and all is good. The \"regression\" here\n> is that we now properly can determine that the hardware is crap.\n\nBefore 4.12, when I unplugged my USB3 Flash drive, Linux would\ndetect a few \"Uncorrected Non-Fatal errors\" via AER, but it was\nstill possible to plug the drive back in.\n\nSince 4.12, once I unplug the drive, the whole USB3 card is marked\nas dead (all 4 ports), and I can no longer plug anything in (not even\nthe USB2 drive that didn't have any issues, IIRC).\n\nIt seems a bit premature to \"mark as dead\" something that remains\nfunctional, doesn't it?\n\nDisclaimer, there are many variables in this setup, and I've only\ntested a small fraction of the problem space: only one system,\nonly one USB3 board, only one USB3 Flash drive.\n\n> So, how do you think we should proceed, delay a bit longer before saying\n> the device is gone? How long is \"long enough\"? How many bus errors are\n> we allowed to tolerate (hint, the PCI spec says none...)\n> \n> Maybe someone wants to get to the root problem here, why is the hardware\n> suddenly reporting all 1s?\n\nI'm afraid I won't be able to make any progress on this front,\nunless I can get my hands on a PCIe packet analyzer.\n\nRegards.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xhzqm5YN9z9t2Q\n\tfor ;\n\tWed, 30 Aug 2017 18:55:48 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751404AbdH3Izp (ORCPT );\n\tWed, 30 Aug 2017 04:55:45 -0400","from smtp2-g21.free.fr ([212.27.42.2]:34927 \"EHLO\n\tsmtp2-g21.free.fr\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1751382AbdH3Izm (ORCPT );\n\tWed, 30 Aug 2017 04:55:42 -0400","from [192.168.0.66] (unknown [88.191.210.51])\n\tby smtp2-g21.free.fr (Postfix) with ESMTP id 0DC402002C4;\n\tWed, 30 Aug 2017 10:55:40 +0200 (CEST)"],"Subject":"Re: Possible regression between 4.9 and 4.13","To":"Greg Kroah-Hartman ,\n\tLukas Wunner ","Cc":"Mathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ,\n\tBjorn Helgaas ,\n\tAlan Stern ","References":"<599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com> <20170829235310.GA20214@wunner.de>\n\t<20170830060237.GA2782@kroah.com>","From":"Mason ","Message-ID":"<678490ce-9381-e63e-7a12-33d3eff7f894@free.fr>","Date":"Wed, 30 Aug 2017 10:55:37 +0200","User-Agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 SeaMonkey/2.49.1","MIME-Version":"1.0","In-Reply-To":"<20170830060237.GA2782@kroah.com>","Content-Type":"text/plain; charset=ISO-8859-15","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759970,"web_url":"http://patchwork.ozlabs.org/comment/1759970/","msgid":"<20170830090633.GA1208@kroah.com>","list_archive_url":null,"date":"2017-08-30T09:06:33","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":11800,"url":"http://patchwork.ozlabs.org/api/people/11800/","name":"Greg Kroah-Hartman","email":"gregkh@linuxfoundation.org"},"content":"On Wed, Aug 30, 2017 at 10:55:37AM +0200, Mason wrote:\n> On 30/08/2017 08:02, Greg Kroah-Hartman wrote:\n> \n> > To get back to the original issue here, the hardware seems to have died,\n> > the driver stops talking to it, and all is good. The \"regression\" here\n> > is that we now properly can determine that the hardware is crap.\n> \n> Before 4.12, when I unplugged my USB3 Flash drive, Linux would\n> detect a few \"Uncorrected Non-Fatal errors\" via AER, but it was\n> still possible to plug the drive back in.\n> \n> Since 4.12, once I unplug the drive, the whole USB3 card is marked\n> as dead (all 4 ports), and I can no longer plug anything in (not even\n> the USB2 drive that didn't have any issues, IIRC).\n> \n> It seems a bit premature to \"mark as dead\" something that remains\n> functional, doesn't it?\n\nI agree, but if the device sends all ones, it's a good indication it is\nreally dead, right? Or something is wrong with it.\n\n> Disclaimer, there are many variables in this setup, and I've only\n> tested a small fraction of the problem space: only one system,\n> only one USB3 board, only one USB3 Flash drive.\n\nDid you ever happen to narrow this down to a single git commit using\n'git bisect'? I can't remember what happened in the beginning of this\nthread...\n\n> > So, how do you think we should proceed, delay a bit longer before saying\n> > the device is gone? How long is \"long enough\"? How many bus errors are\n> > we allowed to tolerate (hint, the PCI spec says none...)\n> > \n> > Maybe someone wants to get to the root problem here, why is the hardware\n> > suddenly reporting all 1s?\n> \n> I'm afraid I won't be able to make any progress on this front,\n> unless I can get my hands on a PCIe packet analyzer.\n\nOdds of that happening are pretty rare, right? I've never even seen one\nof those...\n\nthanks,\n\ngreg k-h","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xj0475xfnz9t2R\n\tfor ;\n\tWed, 30 Aug 2017 19:06:31 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751271AbdH3JGa (ORCPT );\n\tWed, 30 Aug 2017 05:06:30 -0400","from mail.linuxfoundation.org ([140.211.169.12]:47342 \"EHLO\n\tmail.linuxfoundation.org\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751295AbdH3JG3 (ORCPT\n\t); Wed, 30 Aug 2017 05:06:29 -0400","from localhost (LFbn-1-12253-150.w90-92.abo.wanadoo.fr\n\t[90.92.67.150])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPSA id AC847516;\n\tWed, 30 Aug 2017 09:06:28 +0000 (UTC)"],"Date":"Wed, 30 Aug 2017 11:06:33 +0200","From":"Greg Kroah-Hartman ","To":"Mason ","Cc":"Lukas Wunner ,\n\tMathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ,\n\tBjorn Helgaas ,\n\tAlan Stern ","Subject":"Re: Possible regression between 4.9 and 4.13","Message-ID":"<20170830090633.GA1208@kroah.com>","References":"<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com>\n\t<20170829235310.GA20214@wunner.de>\n\t<20170830060237.GA2782@kroah.com>\n\t<678490ce-9381-e63e-7a12-33d3eff7f894@free.fr>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<678490ce-9381-e63e-7a12-33d3eff7f894@free.fr>","User-Agent":"Mutt/1.8.3 (2017-05-23)","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759971,"web_url":"http://patchwork.ozlabs.org/comment/1759971/","msgid":"","list_archive_url":null,"date":"2017-08-30T09:07:59","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":26857,"url":"http://patchwork.ozlabs.org/api/people/26857/","name":"Ard Biesheuvel","email":"ard.biesheuvel@linaro.org"},"content":"On 30 August 2017 at 09:55, Mason wrote:\n> On 30/08/2017 08:02, Greg Kroah-Hartman wrote:\n>\n>> To get back to the original issue here, the hardware seems to have died,\n>> the driver stops talking to it, and all is good. The \"regression\" here\n>> is that we now properly can determine that the hardware is crap.\n>\n> Before 4.12, when I unplugged my USB3 Flash drive, Linux would\n> detect a few \"Uncorrected Non-Fatal errors\" via AER, but it was\n> still possible to plug the drive back in.\n>\n> Since 4.12, once I unplug the drive, the whole USB3 card is marked\n> as dead (all 4 ports), and I can no longer plug anything in (not even\n> the USB2 drive that didn't have any issues, IIRC).\n>\n> It seems a bit premature to \"mark as dead\" something that remains\n> functional, doesn't it?\n>\n> Disclaimer, there are many variables in this setup, and I've only\n> tested a small fraction of the problem space: only one system,\n> only one USB3 board, only one USB3 Flash drive.\n>\n\nPlease don't forget to mention that this is quirky hardware that\ndepends on BROKEN because it multiplexes MMIO and config space\naccesses in the same memory window without any locking whatsoever\n(which would be difficult to do in the first place because we don't\nuse accessors for MMIO in the kernel).\n\nSo how likely is it that you are attempting to read from the xhci BAR\nwindow while a config space access is in progress? Any way to\ninstrument this in your driver?\n\n>> So, how do you think we should proceed, delay a bit longer before saying\n>> the device is gone? How long is \"long enough\"? How many bus errors are\n>> we allowed to tolerate (hint, the PCI spec says none...)\n>>\n>> Maybe someone wants to get to the root problem here, why is the hardware\n>> suddenly reporting all 1s?\n>\n> I'm afraid I won't be able to make any progress on this front,\n> unless I can get my hands on a PCIe packet analyzer.\n>\n> Regards.\n>\n> _______________________________________________\n> linux-arm-kernel mailing list\n> linux-arm-kernel@lists.infradead.org\n> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","ozlabs.org; dkim=pass (1024-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"XoFfqbid\"; dkim-atps=neutral"],"Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xj05v28Mpz9t2R\n\tfor ;\n\tWed, 30 Aug 2017 19:08:03 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751298AbdH3JIB (ORCPT );\n\tWed, 30 Aug 2017 05:08:01 -0400","from mail-it0-f53.google.com ([209.85.214.53]:35989 \"EHLO\n\tmail-it0-f53.google.com\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751321AbdH3JIA (ORCPT\n\t); Wed, 30 Aug 2017 05:08:00 -0400","by mail-it0-f53.google.com with SMTP id o132so3698299itc.1\n\tfor ; Wed, 30 Aug 2017 02:08:00 -0700 (PDT)","by 10.107.162.1 with HTTP; Wed, 30 Aug 2017 02:07:59 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=mime-version:in-reply-to:references:from:date:message-id:subject:to\n\t:cc; bh=Da7eHp9VyrH1WPbECVIJ04Ag8GrMV4xpvTqnRY59OH4=;\n\tb=XoFfqbid4FIYCiB5olA+BpXSxijkIWeltk0bK7zhQbFRK0XQhMlVEhq4ARSaPj6yhR\n\tHqb7nuIMCLWVHTUDlZsA2hg+O4mppYljvuLf0WCftuLpXJ9B+Pv7npuTR2GzSiRiHyJ+\n\tkMQg4uT2pXFSo9n8ObNQwGgKYp6+IoVa19c5E=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:in-reply-to:references:from:date\n\t:message-id:subject:to:cc;\n\tbh=Da7eHp9VyrH1WPbECVIJ04Ag8GrMV4xpvTqnRY59OH4=;\n\tb=L95IlpTsT4VjBnrNjjH/MjVk4EXZx3AVn4+NdceehQ5j61BUP9Gp4T+5CpP80LOQQL\n\tNl8aW2XUwPg998AVDG5vqf27VtV1AgJ3Fs/Y4JFo/1p6toZj6/C38I3lS374hazars90\n\t0aDZfdJHPcxN8yg12sxhetliw5NhknhrbZvAf09nWxujUFHM/BMCVZJSZuI0i3y2T5cP\n\tVcgmpm7is2kOSE1cVDuqHX/NkPHE+j+u8sfDeQyyUST5KeKO939wQSh0CYlc6ULYda4h\n\ttmizUeSDV09hOG5dNMU85auKb6EpVbKVAmjXlbwQUr2svZjDq0cT+w/W1MoCu6pteteU\n\tC9YA==","X-Gm-Message-State":"AHYfb5h7wpoWysCKoHJeMG1Vy3dhNAVJgfP1rSFvvNmrjRV1DKYUTnVU\n\tPA2xL/rP7QCWqMp+obVjdoiHJEVzcqIn","X-Received":"by 10.36.201.70 with SMTP id h67mr757530itg.162.1504084080203;\n\tWed, 30 Aug 2017 02:08:00 -0700 (PDT)","MIME-Version":"1.0","In-Reply-To":"<678490ce-9381-e63e-7a12-33d3eff7f894@free.fr>","References":"<599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com> <20170829235310.GA20214@wunner.de>\n\t<20170830060237.GA2782@kroah.com>\n\t<678490ce-9381-e63e-7a12-33d3eff7f894@free.fr>","From":"Ard Biesheuvel ","Date":"Wed, 30 Aug 2017 10:07:59 +0100","Message-ID":"","Subject":"Re: Possible regression between 4.9 and 4.13","To":"Mason ","Cc":"Greg Kroah-Hartman ,\n\tLukas Wunner ,\n\tMathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tBjorn Helgaas ,\n\tAlan Stern ,\n\tLinux ARM ","Content-Type":"text/plain; charset=\"UTF-8\"","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759989,"web_url":"http://patchwork.ozlabs.org/comment/1759989/","msgid":"<20170830092201.GB16835@kroah.com>","list_archive_url":null,"date":"2017-08-30T09:22:01","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":11800,"url":"http://patchwork.ozlabs.org/api/people/11800/","name":"Greg Kroah-Hartman","email":"gregkh@linuxfoundation.org"},"content":"On Wed, Aug 30, 2017 at 10:07:59AM +0100, Ard Biesheuvel wrote:\n> On 30 August 2017 at 09:55, Mason wrote:\n> > On 30/08/2017 08:02, Greg Kroah-Hartman wrote:\n> >\n> >> To get back to the original issue here, the hardware seems to have died,\n> >> the driver stops talking to it, and all is good. The \"regression\" here\n> >> is that we now properly can determine that the hardware is crap.\n> >\n> > Before 4.12, when I unplugged my USB3 Flash drive, Linux would\n> > detect a few \"Uncorrected Non-Fatal errors\" via AER, but it was\n> > still possible to plug the drive back in.\n> >\n> > Since 4.12, once I unplug the drive, the whole USB3 card is marked\n> > as dead (all 4 ports), and I can no longer plug anything in (not even\n> > the USB2 drive that didn't have any issues, IIRC).\n> >\n> > It seems a bit premature to \"mark as dead\" something that remains\n> > functional, doesn't it?\n> >\n> > Disclaimer, there are many variables in this setup, and I've only\n> > tested a small fraction of the problem space: only one system,\n> > only one USB3 board, only one USB3 Flash drive.\n> >\n> \n> Please don't forget to mention that this is quirky hardware that\n> depends on BROKEN because it multiplexes MMIO and config space\n> accesses in the same memory window without any locking whatsoever\n> (which would be difficult to do in the first place because we don't\n> use accessors for MMIO in the kernel).\n> \n> So how likely is it that you are attempting to read from the xhci BAR\n> window while a config space access is in progress? Any way to\n> instrument this in your driver?\n\nSeriously? Ok, that's crap hardware, sorry, I don't feel bad at all\nhere. You are going to have worse problems than just a single USB\ncontroller issue if that's your hardware design, go kick some hardware\nengineers for me please.\n\ngood luck, you are on your own :(\n\ngreg k-h","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xj0Pz4pnCz9t2R\n\tfor ;\n\tWed, 30 Aug 2017 19:21:59 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751377AbdH3JV5 (ORCPT );\n\tWed, 30 Aug 2017 05:21:57 -0400","from mail.linuxfoundation.org ([140.211.169.12]:47988 \"EHLO\n\tmail.linuxfoundation.org\" rhost-flags-OK-OK-OK-OK) by vger.kernel.org\n\twith ESMTP id S1751295AbdH3JV5 (ORCPT\n\t); Wed, 30 Aug 2017 05:21:57 -0400","from localhost (LFbn-1-12253-150.w90-92.abo.wanadoo.fr\n\t[90.92.67.150])\n\tby mail.linuxfoundation.org (Postfix) with ESMTPSA id 84D11901;\n\tWed, 30 Aug 2017 09:21:56 +0000 (UTC)"],"Date":"Wed, 30 Aug 2017 11:22:01 +0200","From":"Greg Kroah-Hartman ","To":"Ard Biesheuvel ","Cc":"Mason , Lukas Wunner ,\n\tMathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tBjorn Helgaas ,\n\tAlan Stern ,\n\tLinux ARM ","Subject":"Re: Possible regression between 4.9 and 4.13","Message-ID":"<20170830092201.GB16835@kroah.com>","References":"\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com>\n\t<20170829235310.GA20214@wunner.de>\n\t<20170830060237.GA2782@kroah.com>\n\t<678490ce-9381-e63e-7a12-33d3eff7f894@free.fr>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.8.3 (2017-05-23)","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1759996,"web_url":"http://patchwork.ozlabs.org/comment/1759996/","msgid":"","list_archive_url":null,"date":"2017-08-30T09:37:41","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":66150,"url":"http://patchwork.ozlabs.org/api/people/66150/","name":"Mason","email":"slash.tmp@free.fr"},"content":"On 30/08/2017 11:07, Ard Biesheuvel wrote:\n\n> Please don't forget to mention that this is quirky hardware that\n> depends on BROKEN because it multiplexes MMIO and config space\n> accesses in the same memory window without any locking whatsoever\n> (which would be difficult to do in the first place because we don't\n> use accessors for MMIO in the kernel).\n\nYou're right, it was in the back of my mind, but I didn't state\nit explicitly for the benefit of linux-usb readers.\n\n> So how likely is it that you are attempting to read from the xhci BAR\n> window while a config space access is in progress? Any way to\n> instrument this in your driver?\n\nI logged config space accesses here:\n\nhttps://www.spinics.net/lists/arm-kernel/msg602832.html\n\nIIRC, the config space accesses are generated by the AER ISR.\nSo disabling the AER driver should guarantee that no config space\naccesses are occurring when the drive is unplugged.\n\nRegards.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xj0mF1FfRz9sN7\n\tfor ;\n\tWed, 30 Aug 2017 19:37:49 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751300AbdH3Jhr (ORCPT );\n\tWed, 30 Aug 2017 05:37:47 -0400","from smtp2-g21.free.fr ([212.27.42.2]:46982 \"EHLO\n\tsmtp2-g21.free.fr\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1750835AbdH3Jhq (ORCPT );\n\tWed, 30 Aug 2017 05:37:46 -0400","from [192.168.0.66] (unknown [88.191.210.51])\n\tby smtp2-g21.free.fr (Postfix) with ESMTP id 369E5200291;\n\tWed, 30 Aug 2017 11:37:43 +0200 (CEST)"],"Subject":"Re: Possible regression between 4.9 and 4.13","To":"Ard Biesheuvel ","Cc":"Greg Kroah-Hartman ,\n\tLukas Wunner ,\n\tMathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tBjorn Helgaas ,\n\tAlan Stern ,\n\tLinux ARM ","References":"<599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com> <20170829235310.GA20214@wunner.de>\n\t<20170830060237.GA2782@kroah.com>\n\t<678490ce-9381-e63e-7a12-33d3eff7f894@free.fr>\n\t","From":"Mason ","Message-ID":"","Date":"Wed, 30 Aug 2017 11:37:41 +0200","User-Agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 SeaMonkey/2.49.1","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=ISO-8859-15","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1760768,"web_url":"http://patchwork.ozlabs.org/comment/1760768/","msgid":"<18588e5d-4f29-d259-329e-533a21ce10ad@free.fr>","list_archive_url":null,"date":"2017-08-31T09:17:28","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":66150,"url":"http://patchwork.ozlabs.org/api/people/66150/","name":"Mason","email":"slash.tmp@free.fr"},"content":"On 30/08/2017 11:37, Mason wrote:\n\n> On 30/08/2017 11:07, Ard Biesheuvel wrote:\n> \n>> Please don't forget to mention that this is quirky hardware that\n>> depends on BROKEN because it multiplexes MMIO and config space\n>> accesses in the same memory window without any locking whatsoever\n>> (which would be difficult to do in the first place because we don't\n>> use accessors for MMIO in the kernel).\n> \n> You're right, it was in the back of my mind, but I didn't state\n> it explicitly for the benefit of linux-usb readers.\n> \n>> So how likely is it that you are attempting to read from the xhci\n>> BAR window while a config space access is in progress? Any way to\n>> instrument this in your driver?\n> \n> I logged config space accesses here:\n> \n> https://www.spinics.net/lists/arm-kernel/msg602832.html\n> \n> IIRC, the config space accesses are generated by the AER ISR.\n> So disabling the AER driver should guarantee that no config space\n> accesses are occurring when the drive is unplugged.\n\nI checked, and I *did* remember correctly.\n\nDisabling the AER driver results in 0 config space access occurring\nwhen the USB3 drive is unplugged. This confirms that the controller's\nbroken design (muxing config and mem space) is not responsible for\nthe glitches occurring on unplug events.\n\nFurthermore, I confirm that once the controller has been deemed \"dead\",\neven USB2 drives are no longer detected, and all USB port on the PCIe\nboard are disabled.\n\nRegards.\n\n\nFor reads/writes in config space, I have:\n\n\tif (do_debug) {\n\t\tprintk(\"\\t READ: bus=%d devfn=%u where=%d size=%d val=0x%x\\n\",\n\t\t\tbus->number, devfn, where, size, *val);\n\t\tdump_stack();\n\t}\n\n\tif (do_debug) {\n\t\tprintk(\"\\tWRITE: bus=%d devfn=%u where=%d size=%d val=0x%x\\n\",\n\t\t\tbus->number, devfn, where, size, val);\n\t\tdump_stack();\n\t}\n\nDuring setup I do get, e.g.\n\n[ 7.621417] READ: bus=1 devfn=0 where=84 size=2 val=0x8\n[ 7.626840] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G C 4.12.0-rc1 #2\n[ 7.634358] Hardware name: Sigma Tango DT\n[ 7.638387] [] (unwind_backtrace) from [] (show_stack+0x10/0x14)\n[ 7.646171] [] (show_stack) from [] (dump_stack+0x84/0x98)\n[ 7.653429] [] (dump_stack) from [] (smp8759_config_read+0xa0/0xa4)\n[ 7.661474] [] (smp8759_config_read) from [] (pci_bus_read_config_word+0x6c/0x94)\n[ 7.670742] [] (pci_bus_read_config_word) from [] (pci_read_config_word+0x24/0x38)\n[ 7.680097] [] (pci_read_config_word) from [] (__pci_dev_reset+0x11c/0x2fc)\n[ 7.688841] [] (__pci_dev_reset) from [] (pci_probe_reset_function+0xc/0x10)\n[ 7.697673] [] (pci_probe_reset_function) from [] (pci_create_sysfs_dev_files+0x2a8/0x374)\n[ 7.707728] [] (pci_create_sysfs_dev_files) from [] (pci_sysfs_init+0x34/0x54)\n[ 7.716734] [] (pci_sysfs_init) from [] (do_one_initcall+0x44/0x168)\n[ 7.724867] [] (do_one_initcall) from [] (kernel_init_freeable+0x15c/0x1e8)\n[ 7.733611] [] (kernel_init_freeable) from [] (kernel_init+0x8/0x108)\n[ 7.741831] [] (kernel_init) from [] (ret_from_fork+0x14/0x3c)\n\n\nOn plug/unplug events, there are no config space accesses:\n\n[ 88.006750] usb 2-2: new SuperSpeed USB device number 2 using xhci_hcd\n[ 88.040179] usb 2-2: New USB device found, idVendor=0951, idProduct=1666\n[ 88.046930] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3\n[ 88.054177] usb 2-2: Product: DataTraveler 3.0\n[ 88.058684] usb 2-2: Manufacturer: Kingston\n[ 88.062927] usb 2-2: SerialNumber: 002618887865F0C0F8646BFA\n[ 88.071523] usb-storage 2-2:1.0: USB Mass Storage device detected\n[ 88.081334] scsi host0: usb-storage 2-2:1.0\n[ 89.096074] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 3.0 PQ: 0 ANSI: 6\n[ 89.104828] sd 0:0:0:0: [sda] 15109516 512-byte logical blocks: (7.74 GB/7.20 GiB)\n[ 89.112996] sd 0:0:0:0: [sda] Write Protect is off\n[ 89.118060] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA\n[ 89.129463] sda: sda1\n[ 89.133104] sd 0:0:0:0: [sda] Attached SCSI removable disk\n\n[ 103.375210] xhci_hcd 0000:01:00.0: xHCI host controller not responding, assume dead\n[ 103.382917] xhci_hcd 0000:01:00.0: HC died; cleaning up\n[ 103.388281] usb 2-2: USB disconnect, device number 2","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xjcGr38zxz9sRW\n\tfor ;\n\tThu, 31 Aug 2017 19:17:56 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1750883AbdHaJRy (ORCPT );\n\tThu, 31 Aug 2017 05:17:54 -0400","from smtp5-g21.free.fr ([212.27.42.5]:45937 \"EHLO\n\tsmtp5-g21.free.fr\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1750791AbdHaJRx (ORCPT );\n\tThu, 31 Aug 2017 05:17:53 -0400","from [172.27.0.114] (unknown [92.154.11.170])\n\t(Authenticated sender: slash.tmp)\n\tby smtp5-g21.free.fr (Postfix) with ESMTPSA id F17315FFEB;\n\tThu, 31 Aug 2017 11:17:28 +0200 (CEST)"],"Subject":"Re: Possible regression between 4.9 and 4.13","From":"Mason ","To":"Ard Biesheuvel ,\n\tGreg Kroah-Hartman ","Cc":"Lukas Wunner ,\n\tMathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tBjorn Helgaas ,\n\tAlan Stern ,\n\tLinux ARM ","References":"<599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com> <20170829235310.GA20214@wunner.de>\n\t<20170830060237.GA2782@kroah.com>\n\t<678490ce-9381-e63e-7a12-33d3eff7f894@free.fr>\n\t\n\t","Message-ID":"<18588e5d-4f29-d259-329e-533a21ce10ad@free.fr>","Date":"Thu, 31 Aug 2017 11:17:28 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 SeaMonkey/2.49.1","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1760788,"web_url":"http://patchwork.ozlabs.org/comment/1760788/","msgid":"<1e3bde27-5597-41ed-11d1-0450b17f2344@free.fr>","list_archive_url":null,"date":"2017-08-31T09:39:39","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":66150,"url":"http://patchwork.ozlabs.org/api/people/66150/","name":"Mason","email":"slash.tmp@free.fr"},"content":"On 30/08/2017 11:06, Greg Kroah-Hartman wrote:\n\n> On Wed, Aug 30, 2017 at 10:55:37AM +0200, Mason wrote:\n>\n>> On 30/08/2017 08:02, Greg Kroah-Hartman wrote:\n>>\n>>> To get back to the original issue here, the hardware seems to have died,\n>>> the driver stops talking to it, and all is good. The \"regression\" here\n>>> is that we now properly can determine that the hardware is crap.\n>>\n>> Before 4.12, when I unplugged my USB3 Flash drive, Linux would\n>> detect a few \"Uncorrected Non-Fatal errors\" via AER, but it was\n>> still possible to plug the drive back in.\n>>\n>> Since 4.12, once I unplug the drive, the whole USB3 card is marked\n>> as dead (all 4 ports), and I can no longer plug anything in (not even\n>> the USB2 drive that didn't have any issues, IIRC).\n>>\n>> It seems a bit premature to \"mark as dead\" something that remains\n>> functional, doesn't it?\n> \n> I agree, but if the device sends all ones, it's a good indication it is\n> really dead, right? Or something is wrong with it.\n\nI wouldn't call it dead if I can plug the drive back in, and have\nit working... But I agree that something fishy is happening...\n\n>> Disclaimer, there are many variables in this setup, and I've only\n>> tested a small fraction of the problem space: only one system,\n>> only one USB3 board, only one USB3 Flash drive.\n> \n> Did you ever happen to narrow this down to a single git commit using\n> 'git bisect'? I can't remember what happened in the beginning of this\n> thread...\n\nMathias pointed out d9f11ba9f107aa335091ab8d7ba5eea714e46e8b\n\n>>> So, how do you think we should proceed, delay a bit longer before saying\n>>> the device is gone? How long is \"long enough\"? How many bus errors are\n>>> we allowed to tolerate (hint, the PCI spec says none...)\n>>>\n>>> Maybe someone wants to get to the root problem here, why is the hardware\n>>> suddenly reporting all 1s?\n>>\n>> I'm afraid I won't be able to make any progress on this front,\n>> unless I can get my hands on a PCIe packet analyzer.\n> \n> Odds of that happening are pretty rare, right? I've never even seen one\n> of those...\n\nI had a \"Summit T24 Analyzer\" on my desk a few months ago, but I was getting\nstrange results, and the knowledgeable people in my company were not available\nat the time.\n\nhttp://teledynelecroy.com/protocolanalyzer/protocoloverview.aspx?seriesid=445\n\nRegards.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xjcmM0Y4Jz9s83\n\tfor ;\n\tThu, 31 Aug 2017 19:40:03 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1750882AbdHaJkB (ORCPT );\n\tThu, 31 Aug 2017 05:40:01 -0400","from smtp5-g21.free.fr ([212.27.42.5]:51068 \"EHLO\n\tsmtp5-g21.free.fr\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1750816AbdHaJkA (ORCPT );\n\tThu, 31 Aug 2017 05:40:00 -0400","from [172.27.0.114] (unknown [92.154.11.170])\n\t(Authenticated sender: slash.tmp)\n\tby smtp5-g21.free.fr (Postfix) with ESMTPSA id 9C1085FF7E;\n\tThu, 31 Aug 2017 11:39:39 +0200 (CEST)"],"Subject":"Re: Possible regression between 4.9 and 4.13","To":"Greg Kroah-Hartman ","Cc":"Lukas Wunner ,\n\tMathias Nyman ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ,\n\tBjorn Helgaas ,\n\tAlan Stern ","References":"<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com> <20170829235310.GA20214@wunner.de>\n\t<20170830060237.GA2782@kroah.com>\n\t<678490ce-9381-e63e-7a12-33d3eff7f894@free.fr>\n\t<20170830090633.GA1208@kroah.com>","From":"Mason ","Message-ID":"<1e3bde27-5597-41ed-11d1-0450b17f2344@free.fr>","Date":"Thu, 31 Aug 2017 11:39:39 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tFirefox/52.0 SeaMonkey/2.49.1","MIME-Version":"1.0","In-Reply-To":"<20170830090633.GA1208@kroah.com>","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1760844,"web_url":"http://patchwork.ozlabs.org/comment/1760844/","msgid":"<59A7F52E.7000301@linux.intel.com>","list_archive_url":null,"date":"2017-08-31T11:38:22","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":63937,"url":"http://patchwork.ozlabs.org/api/people/63937/","name":"Mathias Nyman","email":"mathias.nyman@linux.intel.com"},"content":"On 31.08.2017 12:17, Mason wrote:\n> On 30/08/2017 11:37, Mason wrote:\n>\n>> On 30/08/2017 11:07, Ard Biesheuvel wrote:\n>>\n>>> Please don't forget to mention that this is quirky hardware that\n>>> depends on BROKEN because it multiplexes MMIO and config space\n>>> accesses in the same memory window without any locking whatsoever\n>>> (which would be difficult to do in the first place because we don't\n>>> use accessors for MMIO in the kernel).\n>>\n>> You're right, it was in the back of my mind, but I didn't state\n>> it explicitly for the benefit of linux-usb readers.\n>>\n>>> So how likely is it that you are attempting to read from the xhci\n>>> BAR window while a config space access is in progress? Any way to\n>>> instrument this in your driver?\n>>\n>> I logged config space accesses here:\n>>\n>> https://www.spinics.net/lists/arm-kernel/msg602832.html\n>>\n>> IIRC, the config space accesses are generated by the AER ISR.\n>> So disabling the AER driver should guarantee that no config space\n>> accesses are occurring when the drive is unplugged.\n>\n> I checked, and I *did* remember correctly.\n>\n> Disabling the AER driver results in 0 config space access occurring\n> when the USB3 drive is unplugged. This confirms that the controller's\n> broken design (muxing config and mem space) is not responsible for\n> the glitches occurring on unplug events.\n>\n> Furthermore, I confirm that once the controller has been deemed \"dead\",\n> even USB2 drives are no longer detected, and all USB port on the PCIe\n> board are disabled.\n\nxhci handles both USB3 and USB2, If there is only a xhci in use then all\nusb ports will be disabled.\nMany systems have both ehci and xhci, where ehci handles USB2 side.\nI'm guessing yours only have the xhci.\n\n-Mathias","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xjgJt43gJz9sQl\n\tfor ;\n\tThu, 31 Aug 2017 21:34:54 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1750918AbdHaLew (ORCPT );\n\tThu, 31 Aug 2017 07:34:52 -0400","from mga03.intel.com ([134.134.136.65]:48819 \"EHLO mga03.intel.com\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1750893AbdHaLew (ORCPT );\n\tThu, 31 Aug 2017 07:34:52 -0400","from fmsmga003.fm.intel.com ([10.253.24.29])\n\tby orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t31 Aug 2017 04:34:51 -0700","from mattu-haswell.fi.intel.com (HELO [10.237.72.164])\n\t([10.237.72.164])\n\tby FMSMGA003.fm.intel.com with ESMTP; 31 Aug 2017 04:34:48 -0700"],"X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos;i=\"5.41,453,1498546800\"; d=\"scan'208\";a=\"895857822\"","Subject":"Re: Possible regression between 4.9 and 4.13","To":"Mason , Ard Biesheuvel ,\n\tGreg Kroah-Hartman ","References":"<599D3410.9050504@intel.com>\n\t<251c41c0-a4fd-8aae-88e0-5d5928ce45cf@free.fr>\n\t<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com> <20170829235310.GA20214@wunner.de>\n\t<20170830060237.GA2782@kroah.com>\n\t<678490ce-9381-e63e-7a12-33d3eff7f894@free.fr>\n\t\n\t\n\t<18588e5d-4f29-d259-329e-533a21ce10ad@free.fr>","Cc":"Lukas Wunner ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tBjorn Helgaas ,\n\tAlan Stern ,\n\tLinux ARM ","From":"Mathias Nyman ","Message-ID":"<59A7F52E.7000301@linux.intel.com>","Date":"Thu, 31 Aug 2017 14:38:22 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101\n\tThunderbird/38.8.0","MIME-Version":"1.0","In-Reply-To":"<18588e5d-4f29-d259-329e-533a21ce10ad@free.fr>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}},{"id":1760854,"web_url":"http://patchwork.ozlabs.org/comment/1760854/","msgid":"<59A7F5B1.7060501@linux.intel.com>","list_archive_url":null,"date":"2017-08-31T11:40:33","subject":"Re: Possible regression between 4.9 and 4.13","submitter":{"id":63937,"url":"http://patchwork.ozlabs.org/api/people/63937/","name":"Mathias Nyman","email":"mathias.nyman@linux.intel.com"},"content":"On 31.08.2017 12:39, Mason wrote:\n> On 30/08/2017 11:06, Greg Kroah-Hartman wrote:\n>\n>> On Wed, Aug 30, 2017 at 10:55:37AM +0200, Mason wrote:\n>>\n>>> On 30/08/2017 08:02, Greg Kroah-Hartman wrote:\n>>>\n>>>> To get back to the original issue here, the hardware seems to have died,\n>>>> the driver stops talking to it, and all is good. The \"regression\" here\n>>>> is that we now properly can determine that the hardware is crap.\n>>>\n>>> Before 4.12, when I unplugged my USB3 Flash drive, Linux would\n>>> detect a few \"Uncorrected Non-Fatal errors\" via AER, but it was\n>>> still possible to plug the drive back in.\n>>>\n>>> Since 4.12, once I unplug the drive, the whole USB3 card is marked\n>>> as dead (all 4 ports), and I can no longer plug anything in (not even\n>>> the USB2 drive that didn't have any issues, IIRC).\n>>>\n>>> It seems a bit premature to \"mark as dead\" something that remains\n>>> functional, doesn't it?\n>>\n>> I agree, but if the device sends all ones, it's a good indication it is\n>> really dead, right? Or something is wrong with it.\n>\n> I wouldn't call it dead if I can plug the drive back in, and have\n> it working... But I agree that something fishy is happening...\n>\n>>> Disclaimer, there are many variables in this setup, and I've only\n>>> tested a small fraction of the problem space: only one system,\n>>> only one USB3 board, only one USB3 Flash drive.\n>>\n>> Did you ever happen to narrow this down to a single git commit using\n>> 'git bisect'? I can't remember what happened in the beginning of this\n>> thread...\n>\n> Mathias pointed out d9f11ba9f107aa335091ab8d7ba5eea714e46e8b\n>\n\nThat patch only changes how xhci reacts to reading 0xffffffff.\nwe used to just returned -ENODEV, but after patch we assume\nhardware is broken or removed.\n\n-Mathias","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=none (mailfrom) smtp.mailfrom=vger.kernel.org\n\t(client-ip=209.132.180.67; helo=vger.kernel.org;\n\tenvelope-from=linux-pci-owner@vger.kernel.org;\n\treceiver=)","Received":["from vger.kernel.org (vger.kernel.org [209.132.180.67])\n\tby ozlabs.org (Postfix) with ESMTP id 3xjgMT3XzWz9t2M\n\tfor ;\n\tThu, 31 Aug 2017 21:37:09 +1000 (AEST)","(majordomo@vger.kernel.org) by vger.kernel.org via listexpand\n\tid S1751724AbdHaLhG (ORCPT );\n\tThu, 31 Aug 2017 07:37:06 -0400","from mga14.intel.com ([192.55.52.115]:28800 \"EHLO mga14.intel.com\"\n\trhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP\n\tid S1751359AbdHaLhD (ORCPT );\n\tThu, 31 Aug 2017 07:37:03 -0400","from orsmga003.jf.intel.com ([10.7.209.27])\n\tby fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;\n\t31 Aug 2017 04:37:03 -0700","from mattu-haswell.fi.intel.com (HELO [10.237.72.164])\n\t([10.237.72.164])\n\tby orsmga003.jf.intel.com with ESMTP; 31 Aug 2017 04:37:00 -0700"],"X-ExtLoop1":"1","X-IronPort-AV":"E=Sophos;i=\"5.41,453,1498546800\"; d=\"scan'208\";a=\"1009613058\"","Subject":"Re: Possible regression between 4.9 and 4.13","To":"Mason ,\n\tGreg Kroah-Hartman ","References":"<599D62EA.7050100@linux.intel.com>\n\t\n\t<8ac92197-907a-282b-2165-f50d1b09bd55@free.fr>\n\t<61d34811-f17c-6faf-252f-c4c81feb9e89@free.fr>\n\t<59A3D6BF.7010400@linux.intel.com>\n\t<0b089b17-00fc-5a7c-baa3-e6141029b191@free.fr>\n\t<59A56C15.2000403@linux.intel.com> <20170829235310.GA20214@wunner.de>\n\t<20170830060237.GA2782@kroah.com>\n\t<678490ce-9381-e63e-7a12-33d3eff7f894@free.fr>\n\t<20170830090633.GA1208@kroah.com>\n\t<1e3bde27-5597-41ed-11d1-0450b17f2344@free.fr>","Cc":"Lukas Wunner ,\n\tFelipe Balbi ,\n\tlinux-pci ,\n\tlinux-usb ,\n\tLinux ARM ,\n\tBjorn Helgaas ,\n\tAlan Stern ","From":"Mathias Nyman ","Message-ID":"<59A7F5B1.7060501@linux.intel.com>","Date":"Thu, 31 Aug 2017 14:40:33 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101\n\tThunderbird/38.8.0","MIME-Version":"1.0","In-Reply-To":"<1e3bde27-5597-41ed-11d1-0450b17f2344@free.fr>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Transfer-Encoding":"7bit","Sender":"linux-pci-owner@vger.kernel.org","Precedence":"bulk","List-ID":"","X-Mailing-List":"linux-pci@vger.kernel.org"}}]