mbox series

[SRU,0/1,J/OEM-5.17/OEM-6.0/K/U] Fix ath11k deadlock on WCN6855

Message ID 20221028050706.48086-1-aaron.ma@canonical.com
Headers show
Series Fix ath11k deadlock on WCN6855 | expand

Message

Aaron Ma Oct. 28, 2022, 5:07 a.m. UTC 
BugLink: https://bugs.launchpad.net/bugs/1995041

[Impact]
ath11k is in deadlock when stress reboot or suspend on WCN6855.
sometimes kernel hang.

Aug 19 01:29:28 Thor-P3-AMD-2 kernel: Call Trace:
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: <TASK>
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __schedule+0x240/0x5a0
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? resched_curr+0x52/0xc0
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: schedule+0x55/0xd0
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: schedule_timeout+0x115/0x150
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? raw_spin_rq_unlock+0x10/0x30
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? try_to_wake_up+0x211/0x600
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? ath11k_ce_send+0x17a/0x2e0
[ath11k]
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: wait_for_completion+0x8b/0xf0
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __flush_work.isra.0+0x171/0x270
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ?
flush_workqueue_prep_pwqs+0x140/0x140
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __cancel_work_timer+0x11b/0x1a0
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ?
ath11k_mac_config_mon_status_default+0xcc/0x170 [ath11k]
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: cancel_work_sync+0x10/0x20
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ath11k_mac_op_stop+0x9f/0x1e0
[ath11k]
Aug 19 01:29:28 Thor-P3-AMD-2 kernel: drv_stop+0x45/0x110 [mac80211]

[Fix]
Fix this by switching from using regulatory_set_wiphy_regd_sync() to
regulatory_set_wiphy_regd(). Now cfg80211 will schedule another
workqueue which handles the locking on it's own.
So the ath11k workqueue can simply exit
without taking any locks, avoiding the deadlock.

[Test]
Verified on hardware, stress reboot and suspend 30 times OK.

[Where problems could occur]
It may break ath11k wifi driver.
Because of the schedule, SRU it as SAUCE from maintainer's tree.

Wen Gong (1):
  UBUNTU: SAUCE: wifi: ath11k: avoid deadlock during regulatory update
    in ath11k_regd_update()

 drivers/net/wireless/ath/ath11k/reg.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

Comments

Tim Gardner Oct. 28, 2022, 7:02 p.m. UTC | #1 
On 10/27/22 23:07, Aaron Ma wrote:
> BugLink: https://bugs.launchpad.net/bugs/1995041
> 
> [Impact]
> ath11k is in deadlock when stress reboot or suspend on WCN6855.
> sometimes kernel hang.
> 
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: Call Trace:
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: <TASK>
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __schedule+0x240/0x5a0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? resched_curr+0x52/0xc0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: schedule+0x55/0xd0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: schedule_timeout+0x115/0x150
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? raw_spin_rq_unlock+0x10/0x30
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? try_to_wake_up+0x211/0x600
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? ath11k_ce_send+0x17a/0x2e0
> [ath11k]
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: wait_for_completion+0x8b/0xf0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __flush_work.isra.0+0x171/0x270
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ?
> flush_workqueue_prep_pwqs+0x140/0x140
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __cancel_work_timer+0x11b/0x1a0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ?
> ath11k_mac_config_mon_status_default+0xcc/0x170 [ath11k]
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: cancel_work_sync+0x10/0x20
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ath11k_mac_op_stop+0x9f/0x1e0
> [ath11k]
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: drv_stop+0x45/0x110 [mac80211]
> 
> [Fix]
> Fix this by switching from using regulatory_set_wiphy_regd_sync() to
> regulatory_set_wiphy_regd(). Now cfg80211 will schedule another
> workqueue which handles the locking on it's own.
> So the ath11k workqueue can simply exit
> without taking any locks, avoiding the deadlock.
> 
> [Test]
> Verified on hardware, stress reboot and suspend 30 times OK.
> 
> [Where problems could occur]
> It may break ath11k wifi driver.
> Because of the schedule, SRU it as SAUCE from maintainer's tree.
> 
> Wen Gong (1):
>    UBUNTU: SAUCE: wifi: ath11k: avoid deadlock during regulatory update
>      in ath11k_regd_update()
> 
>   drivers/net/wireless/ath/ath11k/reg.c | 6 +-----
>   1 file changed, 1 insertion(+), 5 deletions(-)
> 
Acked-by: Tim Gardner <tim.gardner@canonical.com>
Hui Wang Nov. 8, 2022, 1:37 a.m. UTC | #2 
The patch is in the linux-next now and CCed to stable@vger.kernel.org

Acked-by: Hui Wang <hui.wang@canonical.com>

On 10/28/22 13:07, Aaron Ma wrote:
> BugLink: https://bugs.launchpad.net/bugs/1995041
>
> [Impact]
> ath11k is in deadlock when stress reboot or suspend on WCN6855.
> sometimes kernel hang.
>
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: Call Trace:
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: <TASK>
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __schedule+0x240/0x5a0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? resched_curr+0x52/0xc0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: schedule+0x55/0xd0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: schedule_timeout+0x115/0x150
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? raw_spin_rq_unlock+0x10/0x30
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? try_to_wake_up+0x211/0x600
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? ath11k_ce_send+0x17a/0x2e0
> [ath11k]
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: wait_for_completion+0x8b/0xf0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __flush_work.isra.0+0x171/0x270
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ?
> flush_workqueue_prep_pwqs+0x140/0x140
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __cancel_work_timer+0x11b/0x1a0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ?
> ath11k_mac_config_mon_status_default+0xcc/0x170 [ath11k]
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: cancel_work_sync+0x10/0x20
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ath11k_mac_op_stop+0x9f/0x1e0
> [ath11k]
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: drv_stop+0x45/0x110 [mac80211]
>
> [Fix]
> Fix this by switching from using regulatory_set_wiphy_regd_sync() to
> regulatory_set_wiphy_regd(). Now cfg80211 will schedule another
> workqueue which handles the locking on it's own.
> So the ath11k workqueue can simply exit
> without taking any locks, avoiding the deadlock.
>
> [Test]
> Verified on hardware, stress reboot and suspend 30 times OK.
>
> [Where problems could occur]
> It may break ath11k wifi driver.
> Because of the schedule, SRU it as SAUCE from maintainer's tree.
>
> Wen Gong (1):
>    UBUNTU: SAUCE: wifi: ath11k: avoid deadlock during regulatory update
>      in ath11k_regd_update()
>
>   drivers/net/wireless/ath/ath11k/reg.c | 6 +-----
>   1 file changed, 1 insertion(+), 5 deletions(-)
>
Timo Aaltonen Nov. 10, 2022, 4:18 p.m. UTC | #3 
Aaron Ma kirjoitti 28.10.2022 klo 8.07:
> BugLink: https://bugs.launchpad.net/bugs/1995041
> 
> [Impact]
> ath11k is in deadlock when stress reboot or suspend on WCN6855.
> sometimes kernel hang.
> 
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: Call Trace:
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: <TASK>
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __schedule+0x240/0x5a0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? resched_curr+0x52/0xc0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: schedule+0x55/0xd0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: schedule_timeout+0x115/0x150
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? raw_spin_rq_unlock+0x10/0x30
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? try_to_wake_up+0x211/0x600
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? ath11k_ce_send+0x17a/0x2e0
> [ath11k]
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: wait_for_completion+0x8b/0xf0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __flush_work.isra.0+0x171/0x270
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ?
> flush_workqueue_prep_pwqs+0x140/0x140
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __cancel_work_timer+0x11b/0x1a0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ?
> ath11k_mac_config_mon_status_default+0xcc/0x170 [ath11k]
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: cancel_work_sync+0x10/0x20
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ath11k_mac_op_stop+0x9f/0x1e0
> [ath11k]
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: drv_stop+0x45/0x110 [mac80211]
> 
> [Fix]
> Fix this by switching from using regulatory_set_wiphy_regd_sync() to
> regulatory_set_wiphy_regd(). Now cfg80211 will schedule another
> workqueue which handles the locking on it's own.
> So the ath11k workqueue can simply exit
> without taking any locks, avoiding the deadlock.
> 
> [Test]
> Verified on hardware, stress reboot and suspend 30 times OK.
> 
> [Where problems could occur]
> It may break ath11k wifi driver.
> Because of the schedule, SRU it as SAUCE from maintainer's tree.
> 
> Wen Gong (1):
>    UBUNTU: SAUCE: wifi: ath11k: avoid deadlock during regulatory update
>      in ath11k_regd_update()
> 
>   drivers/net/wireless/ath/ath11k/reg.c | 6 +-----
>   1 file changed, 1 insertion(+), 5 deletions(-)
> 

applied to oem-kernels, thanks
Stefan Bader Nov. 11, 2022, 11:27 a.m. UTC | #4 
On 28.10.22 07:07, Aaron Ma wrote:
> BugLink: https://bugs.launchpad.net/bugs/1995041
> 
> [Impact]
> ath11k is in deadlock when stress reboot or suspend on WCN6855.
> sometimes kernel hang.
> 
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: Call Trace:
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: <TASK>
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __schedule+0x240/0x5a0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? resched_curr+0x52/0xc0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: schedule+0x55/0xd0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: schedule_timeout+0x115/0x150
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? raw_spin_rq_unlock+0x10/0x30
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? try_to_wake_up+0x211/0x600
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ? ath11k_ce_send+0x17a/0x2e0
> [ath11k]
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: wait_for_completion+0x8b/0xf0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __flush_work.isra.0+0x171/0x270
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ?
> flush_workqueue_prep_pwqs+0x140/0x140
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: __cancel_work_timer+0x11b/0x1a0
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ?
> ath11k_mac_config_mon_status_default+0xcc/0x170 [ath11k]
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: cancel_work_sync+0x10/0x20
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: ath11k_mac_op_stop+0x9f/0x1e0
> [ath11k]
> Aug 19 01:29:28 Thor-P3-AMD-2 kernel: drv_stop+0x45/0x110 [mac80211]
> 
> [Fix]
> Fix this by switching from using regulatory_set_wiphy_regd_sync() to
> regulatory_set_wiphy_regd(). Now cfg80211 will schedule another
> workqueue which handles the locking on it's own.
> So the ath11k workqueue can simply exit
> without taking any locks, avoiding the deadlock.
> 
> [Test]
> Verified on hardware, stress reboot and suspend 30 times OK.
> 
> [Where problems could occur]
> It may break ath11k wifi driver.
> Because of the schedule, SRU it as SAUCE from maintainer's tree.
> 
> Wen Gong (1):
>    UBUNTU: SAUCE: wifi: ath11k: avoid deadlock during regulatory update
>      in ath11k_regd_update()
> 
>   drivers/net/wireless/ath/ath11k/reg.c | 6 +-----
>   1 file changed, 1 insertion(+), 5 deletions(-)
> 

Applied to kinetic,jammy:linux/master-next from upstream commit 
f45cb6b29cd36514e13f7519770873d8c0457008. Thanks.

-Stefan