| Message ID | 20201016130726.1378666-1-jckuo@nvidia.com |
|---|---|
| Headers | show |
| Series | Tegra XHCI controller ELPG support | expand |
On Fri, Oct 16, 2020 at 09:07:10PM +0800, JC Kuo wrote: > Tegra XHCI controler can be placed in ELPG (Engine Level PowerGated) > state for power saving when all of the connected USB devices are in > suspended state. This patch series includes clk, phy and pmc changes > that are required for properly place controller in ELPG and bring > controller out of ELPG. > > JC Kuo (16): > clk: tegra: Add PLLE HW power sequencer control > clk: tegra: Don't enable PLLE HW sequencer at init > phy: tegra: xusb: Move usb3 port init for Tegra210 > phy: tegra: xusb: tegra210: Do not reset UPHY PLL > phy: tegra: xusb: Rearrange UPHY init on Tegra210 > phy: tegra: xusb: Add Tegra210 lane_iddq operation > phy: tegra: xusb: Add sleepwalk and suspend/resume > soc/tegra: pmc: Provide USB sleepwalk register map > arm64: tegra210: XUSB PADCTL add "nvidia,pmc" prop > dt-bindings: phy: tegra-xusb: Add nvidia,pmc prop > phy: tegra: xusb: Add wake/sleepwalk for Tegra210 > phy: tegra: xusb: Tegra210 host mode VBUS control > phy: tegra: xusb: Add wake/sleepwalk for Tegra186 > arm64: tegra210/tegra186/tegra194: XUSB PADCTL irq > usb: host: xhci-tegra: Unlink power domain devices > xhci: tegra: Enable ELPG for runtime/system PM > > .../phy/nvidia,tegra124-xusb-padctl.txt | 1 + > arch/arm64/boot/dts/nvidia/tegra186.dtsi | 1 + > arch/arm64/boot/dts/nvidia/tegra194.dtsi | 1 + > arch/arm64/boot/dts/nvidia/tegra210.dtsi | 2 + > drivers/clk/tegra/clk-pll.c | 12 - > drivers/clk/tegra/clk-tegra210.c | 53 +- > drivers/phy/tegra/xusb-tegra186.c | 558 ++++- > drivers/phy/tegra/xusb-tegra210.c | 1889 +++++++++++++---- > drivers/phy/tegra/xusb.c | 92 +- > drivers/phy/tegra/xusb.h | 22 +- > drivers/soc/tegra/pmc.c | 94 + > drivers/usb/host/xhci-tegra.c | 610 ++++-- > include/linux/clk/tegra.h | 4 +- > include/linux/phy/tegra/xusb.h | 10 +- > 14 files changed, 2785 insertions(+), 564 deletions(-) I've been testing this, but I keep seeing the following oops on suspend on a Jetson TX1: [ 153.451108] tegra-xusb-padctl phy-usb2.0: > tegra_xusb_padctl_suspend_noirq(dev=ffff000080917000) [ 153.460353] tegra-xusb-padctl phy-usb2.0: driver: ffff8000114453e0 (tegra_xusb_padctl_driver) [ 153.469245] tegra-xusb-padctl phy-usb2.0: padctl: ffff0000829f6480 [ 153.475772] tegra-xusb-padctl phy-usb2.0: soc: ef7bdd7fffffffff (0xef7bdd7fffffffff) [ 153.484061] Unable to handle kernel paging request at virtual address 007bdd800000004f [ 153.492132] Mem abort info: [ 153.495083] ESR = 0x96000004 [ 153.498308] EC = 0x25: DABT (current EL), IL = 32 bits [ 153.503771] SET = 0, FnV = 0 [ 153.506979] EA = 0, S1PTW = 0 [ 153.510260] Data abort info: [ 153.513200] ISV = 0, ISS = 0x00000004 [ 153.517181] CM = 0, WnR = 0 [ 153.520302] [007bdd800000004f] address between user and kernel address ranges [ 153.527600] Internal error: Oops: 96000004 [#1] PREEMPT SMP [ 153.533231] Modules linked in: nouveau panel_simple tegra_video(C) tegra_drm drm_ttm_helper videobuf2_dma_contig ttm videobuf2_memops cec videobuf2_v4l2 videobuf2_common drm_kms_helper v4l2_fwnode videodev drm mc snd_hda_codec_hdmi cdc_ether usbnet snd_hda_tegra r8152 crct10dif_ce snd_hda_codec snd_hda_core tegra_xudc host1x lp855x_bl at24 ip_tables x_tables ipv6 [ 153.566417] CPU: 0 PID: 300 Comm: systemd-sleep Tainted: G C 5.10.0-rc3-next-20201113-00019-g5c064d5372b0-dirty #624 [ 153.578283] Hardware name: NVIDIA Jetson TX1 Developer Kit (DT) [ 153.584281] pstate: 40000005 (nZcv daif -PAN -UAO -TCO BTYPE=--) [ 153.590381] pc : tegra_xusb_padctl_suspend_noirq+0x88/0x100 [ 153.596016] lr : tegra_xusb_padctl_suspend_noirq+0x80/0x100 [ 153.601632] sp : ffff8000120dbb60 [ 153.604999] x29: ffff8000120dbb60 x28: ffff000080a1df00 [ 153.610430] x27: 0000000000000002 x26: ffff8000106f8540 [ 153.615858] x25: ffff8000113ac4a4 x24: ffff80001148c198 [ 153.621277] x23: ffff800010c4538c x22: 0000000000000002 [ 153.626692] x21: ffff800010ccde80 x20: ffff0000829f6480 [ 153.632107] x19: ffff000080917000 x18: 0000000000000030 [ 153.637521] x17: 0000000000000000 x16: 0000000000000000 [ 153.642933] x15: ffff000080a1e380 x14: 74636461702d6273 [ 153.648346] x13: ffff8000113ad058 x12: 0000000000000f39 [ 153.653759] x11: 0000000000000513 x10: ffff800011405058 [ 153.659176] x9 : 00000000fffff000 x8 : ffff8000113ad058 [ 153.664590] x7 : ffff800011405058 x6 : 0000000000000000 [ 153.670002] x5 : 0000000000000000 x4 : ffff0000fe908bc0 [ 153.675414] x3 : ffff0000fe910228 x2 : 162ef67e0581e700 [ 153.680826] x1 : 162ef67e0581e700 x0 : ef7bdd7fffffffff [ 153.686241] Call trace: [ 153.688769] tegra_xusb_padctl_suspend_noirq+0x88/0x100 [ 153.694077] __device_suspend_noirq+0x68/0x1cc [ 153.698594] dpm_noirq_suspend_devices+0x10c/0x1d0 [ 153.703456] dpm_suspend_noirq+0x28/0xa0 [ 153.707461] suspend_devices_and_enter+0x234/0x4bc [ 153.712314] pm_suspend+0x1e4/0x270 [ 153.715868] state_store+0x8c/0x110 [ 153.719440] kobj_attr_store+0x1c/0x30 [ 153.723259] sysfs_kf_write+0x4c/0x7c [ 153.726981] kernfs_fop_write+0x124/0x240 [ 153.731065] vfs_write+0xe4/0x204 [ 153.734449] ksys_write+0x6c/0x100 [ 153.737925] __arm64_sys_write+0x20/0x30 [ 153.741931] el0_svc_common.constprop.0+0x78/0x1a0 [ 153.746789] do_el0_svc+0x24/0x90 [ 153.750181] el0_sync_handler+0x254/0x260 [ 153.754251] el0_sync+0x174/0x180 [ 153.757663] Code: aa0303e2 94000f64 f9405680 b40000e0 (f9402803) [ 153.763826] ---[ end trace 81543a3394cb409d ]--- Note that I've added a bit of debug information there to show what's going on. See how tegra_xusb_padctl_suspend_noirq() is being called for the phy-usb2.0 device? That's one of the PHYs that's being created for the USB2 lanes. Sometimes I do see that padctl->soc ends up being NULL for that device and in that case the function just aborts early and then tegra_xusb_padctl_suspend_noirq() will get called again for the padctl device and succeed. I can't explain what's happening here. tegra_xusb_padctl_driver never binds to the phy-usb2.0 device, so I don't understand how it could end up suspending the device with that set of dev_pm_ops. Perhaps this is some weird type of corruption somewhere? Thierry
On 11/14/20 12:44 AM, Thierry Reding wrote: > On Fri, Oct 16, 2020 at 09:07:10PM +0800, JC Kuo wrote: >> Tegra XHCI controler can be placed in ELPG (Engine Level PowerGated) >> state for power saving when all of the connected USB devices are in >> suspended state. This patch series includes clk, phy and pmc changes >> that are required for properly place controller in ELPG and bring >> controller out of ELPG. >> >> JC Kuo (16): >> clk: tegra: Add PLLE HW power sequencer control >> clk: tegra: Don't enable PLLE HW sequencer at init >> phy: tegra: xusb: Move usb3 port init for Tegra210 >> phy: tegra: xusb: tegra210: Do not reset UPHY PLL >> phy: tegra: xusb: Rearrange UPHY init on Tegra210 >> phy: tegra: xusb: Add Tegra210 lane_iddq operation >> phy: tegra: xusb: Add sleepwalk and suspend/resume >> soc/tegra: pmc: Provide USB sleepwalk register map >> arm64: tegra210: XUSB PADCTL add "nvidia,pmc" prop >> dt-bindings: phy: tegra-xusb: Add nvidia,pmc prop >> phy: tegra: xusb: Add wake/sleepwalk for Tegra210 >> phy: tegra: xusb: Tegra210 host mode VBUS control >> phy: tegra: xusb: Add wake/sleepwalk for Tegra186 >> arm64: tegra210/tegra186/tegra194: XUSB PADCTL irq >> usb: host: xhci-tegra: Unlink power domain devices >> xhci: tegra: Enable ELPG for runtime/system PM >> >> .../phy/nvidia,tegra124-xusb-padctl.txt | 1 + >> arch/arm64/boot/dts/nvidia/tegra186.dtsi | 1 + >> arch/arm64/boot/dts/nvidia/tegra194.dtsi | 1 + >> arch/arm64/boot/dts/nvidia/tegra210.dtsi | 2 + >> drivers/clk/tegra/clk-pll.c | 12 - >> drivers/clk/tegra/clk-tegra210.c | 53 +- >> drivers/phy/tegra/xusb-tegra186.c | 558 ++++- >> drivers/phy/tegra/xusb-tegra210.c | 1889 +++++++++++++---- >> drivers/phy/tegra/xusb.c | 92 +- >> drivers/phy/tegra/xusb.h | 22 +- >> drivers/soc/tegra/pmc.c | 94 + >> drivers/usb/host/xhci-tegra.c | 610 ++++-- >> include/linux/clk/tegra.h | 4 +- >> include/linux/phy/tegra/xusb.h | 10 +- >> 14 files changed, 2785 insertions(+), 564 deletions(-) > > I've been testing this, but I keep seeing the following oops on suspend > on a Jetson TX1: > > [ 153.451108] tegra-xusb-padctl phy-usb2.0: > tegra_xusb_padctl_suspend_noirq(dev=ffff000080917000) > [ 153.460353] tegra-xusb-padctl phy-usb2.0: driver: ffff8000114453e0 (tegra_xusb_padctl_driver) > [ 153.469245] tegra-xusb-padctl phy-usb2.0: padctl: ffff0000829f6480 > [ 153.475772] tegra-xusb-padctl phy-usb2.0: soc: ef7bdd7fffffffff (0xef7bdd7fffffffff) > [ 153.484061] Unable to handle kernel paging request at virtual address 007bdd800000004f > [ 153.492132] Mem abort info: > [ 153.495083] ESR = 0x96000004 > [ 153.498308] EC = 0x25: DABT (current EL), IL = 32 bits > [ 153.503771] SET = 0, FnV = 0 > [ 153.506979] EA = 0, S1PTW = 0 > [ 153.510260] Data abort info: > [ 153.513200] ISV = 0, ISS = 0x00000004 > [ 153.517181] CM = 0, WnR = 0 > [ 153.520302] [007bdd800000004f] address between user and kernel address ranges > [ 153.527600] Internal error: Oops: 96000004 [#1] PREEMPT SMP > [ 153.533231] Modules linked in: nouveau panel_simple tegra_video(C) tegra_drm drm_ttm_helper videobuf2_dma_contig ttm videobuf2_memops cec videobuf2_v4l2 videobuf2_common drm_kms_helper v4l2_fwnode videodev drm mc snd_hda_codec_hdmi cdc_ether usbnet snd_hda_tegra r8152 crct10dif_ce snd_hda_codec snd_hda_core tegra_xudc host1x lp855x_bl at24 ip_tables x_tables ipv6 > [ 153.566417] CPU: 0 PID: 300 Comm: systemd-sleep Tainted: G C 5.10.0-rc3-next-20201113-00019-g5c064d5372b0-dirty #624 > [ 153.578283] Hardware name: NVIDIA Jetson TX1 Developer Kit (DT) > [ 153.584281] pstate: 40000005 (nZcv daif -PAN -UAO -TCO BTYPE=--) > [ 153.590381] pc : tegra_xusb_padctl_suspend_noirq+0x88/0x100 > [ 153.596016] lr : tegra_xusb_padctl_suspend_noirq+0x80/0x100 > [ 153.601632] sp : ffff8000120dbb60 > [ 153.604999] x29: ffff8000120dbb60 x28: ffff000080a1df00 > [ 153.610430] x27: 0000000000000002 x26: ffff8000106f8540 > [ 153.615858] x25: ffff8000113ac4a4 x24: ffff80001148c198 > [ 153.621277] x23: ffff800010c4538c x22: 0000000000000002 > [ 153.626692] x21: ffff800010ccde80 x20: ffff0000829f6480 > [ 153.632107] x19: ffff000080917000 x18: 0000000000000030 > [ 153.637521] x17: 0000000000000000 x16: 0000000000000000 > [ 153.642933] x15: ffff000080a1e380 x14: 74636461702d6273 > [ 153.648346] x13: ffff8000113ad058 x12: 0000000000000f39 > [ 153.653759] x11: 0000000000000513 x10: ffff800011405058 > [ 153.659176] x9 : 00000000fffff000 x8 : ffff8000113ad058 > [ 153.664590] x7 : ffff800011405058 x6 : 0000000000000000 > [ 153.670002] x5 : 0000000000000000 x4 : ffff0000fe908bc0 > [ 153.675414] x3 : ffff0000fe910228 x2 : 162ef67e0581e700 > [ 153.680826] x1 : 162ef67e0581e700 x0 : ef7bdd7fffffffff > [ 153.686241] Call trace: > [ 153.688769] tegra_xusb_padctl_suspend_noirq+0x88/0x100 > [ 153.694077] __device_suspend_noirq+0x68/0x1cc > [ 153.698594] dpm_noirq_suspend_devices+0x10c/0x1d0 > [ 153.703456] dpm_suspend_noirq+0x28/0xa0 > [ 153.707461] suspend_devices_and_enter+0x234/0x4bc > [ 153.712314] pm_suspend+0x1e4/0x270 > [ 153.715868] state_store+0x8c/0x110 > [ 153.719440] kobj_attr_store+0x1c/0x30 > [ 153.723259] sysfs_kf_write+0x4c/0x7c > [ 153.726981] kernfs_fop_write+0x124/0x240 > [ 153.731065] vfs_write+0xe4/0x204 > [ 153.734449] ksys_write+0x6c/0x100 > [ 153.737925] __arm64_sys_write+0x20/0x30 > [ 153.741931] el0_svc_common.constprop.0+0x78/0x1a0 > [ 153.746789] do_el0_svc+0x24/0x90 > [ 153.750181] el0_sync_handler+0x254/0x260 > [ 153.754251] el0_sync+0x174/0x180 > [ 153.757663] Code: aa0303e2 94000f64 f9405680 b40000e0 (f9402803) > [ 153.763826] ---[ end trace 81543a3394cb409d ]--- > > Note that I've added a bit of debug information there to show what's > going on. See how tegra_xusb_padctl_suspend_noirq() is being called for > the phy-usb2.0 device? That's one of the PHYs that's being created for > the USB2 lanes. Sometimes I do see that padctl->soc ends up being NULL > for that device and in that case the function just aborts early and then > tegra_xusb_padctl_suspend_noirq() will get called again for the padctl > device and succeed. > > I can't explain what's happening here. tegra_xusb_padctl_driver never > binds to the phy-usb2.0 device, so I don't understand how it could end > up suspending the device with that set of dev_pm_ops. Perhaps this is > some weird type of corruption somewhere? > > Thierry > Hi Thierry, I figured out that phy-usb2.0 device is bound to tegra-xusb-padctl driver by those two lines in tegra_xusb_setup_usb_role_switch() port->usb_phy.dev = &lane->pad->lanes[port->index]->dev; port->usb_phy.dev->driver = port->padctl->dev->driver; Instead of assigning lane (phy) device to 'struct usb_phy', I think it should be the port device (port->dev) since the 'usb-role-switch' capability belongs to port device. port->usb_phy.dev = &port->dev; I will submit a patch and ask for your review. Thanks, JC