A311D, wifi issues, "firmware failed to leave lps state", disconnects

I recently started seeing many wifi disconnects, every 60 sec to 15 minutes. I of course assumed it was a wifi issue. Got a little more formal, comparing to another laptop on my bench, same distance to AP etc and it experienced zero wifi dropouts.

Via dmesg I found:

[  +3.167162] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +2.077631] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[  +2.714754] panfrost_gem_shrinker_scan: 11 callbacks suppressed
[  +0.000022] Purging 569344 bytes
[  +0.000635] Purging 565248 bytes
[  +3.211144] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +2.037847] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[  +0.008615] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +1.634856] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[  +0.289722] wlan0: disconnect from AP f0:72:ea:2a:61:5a for new auth to f0:72:ea:2a:61:5d
[  +0.639130] wlan0: authenticate with f0:72:ea:2a:61:5d (local address=c4:3c:b0:ad:cd:f1)
[  +0.383308] wlan0: send auth to f0:72:ea:2a:61:5d (try 1/3)
[  +0.017098] wlan0: authenticated
[  +0.002597] wlan0: associate with f0:72:ea:2a:61:5d (try 1/3)
[  +0.004380] wlan0: RX ReassocResp from f0:72:ea:2a:61:5d (capab=0x1011 status=0 aid=6)
[  +0.010392] wlan0: associated
[  +0.045861] wlan0: Limiting TX power to 30 (30 - 0) dBm as advertised by f0:72:ea:2a:61:5d
[Jun 1 10:49] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +2.047876] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[ +29.867939] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +2.091759] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[ +14.927270] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +2.064036] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[Jun 1 10:50] rtw_8822cs mmc2:0001:1: failed to send h2c command

Lots of instances of the failed to send h2c command/failed to leave lps state error message with rtw_8822cs driver.

Then a few minutes later this in dmesg…

[Jun 1 10:55] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +2.358867] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[Jun 1 10:56] meson8b-dwmac ff3f0000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[  +0.001564] meson8b-dwmac ff3f0000.ethernet end0: __stmmac_open: Cannot attach to PHY (error: -19)
[  +0.008459] meson8b-dwmac ff3f0000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[  +0.001633] meson8b-dwmac ff3f0000.ethernet end0: __stmmac_open: Cannot attach to PHY (error: -19)
[  +0.011391] meson8b-dwmac ff3f0000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[  +0.002031] meson8b-dwmac ff3f0000.ethernet end0: __stmmac_open: Cannot attach to PHY (error: -19)
[  +0.011665] meson8b-dwmac ff3f0000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[  +0.002655] meson8b-dwmac ff3f0000.ethernet end0: __stmmac_open: Cannot attach to PHY (error: -19)
[ +34.770467] ------------[ cut here ]------------
[  +0.000012] firmware failed to ack driver for entering Deep Power mode
[  +0.000032] WARNING: CPU: 5 PID: 5358 at drivers/net/wireless/realtek/rtw88/ps.c:105 rtw_power_mode_change+0x138/0x190 [rtw88_core]
[  +0.000025] Modules linked in: snd_seq_dummy snd_hrtimer rfcomm snd_seq aes_ce_ccm dwmac_meson8b stmmac_platform stmmac mdio_mux_meson_g12a mdio_mux of_mdio uhid algif_hash algif_skcipher af_alg qrtr bnep rtw88_8822cs rtw88_8822c rtw88_sdio rtw88_core binfmt_misc mac80211 hci_uart libarc4 btqca btrtl cfg80211 btintel btbcm bluetooth ecdh_generic snd_usb_audio snd_usbmidi_lib ecc snd_hwdep snd_rawmidi reform2_lpc(OE) snd_seq_device snd_soc_meson_axg_sound_card cdc_acm snd_soc_wm8960 rfkill snd_soc_meson_card_utils onboard_usb_hub snd_soc_meson_g12a_tohdmitx snd_soc_meson_axg_tdmin meson_gxbb_wdt meson_saradc snd_soc_meson_axg_frddr meson_rng industrialio snd_soc_meson_axg_tdmout leds_gpio nvmem_meson_efuse cpufreq_dt snd_soc_meson_axg_tdm_interface snd_soc_meson_axg_tdm_formatter sg evdev loop dm_mod efi_pstore configfs dax nfnetlink ip_tables x_tables autofs4 ext4 mbcache jbd2 crc32c_generic mali_dp cdns_mhdp_imx cdns_mhdp_drmcore snd_soc_meson_codec_glue mxsfb mux_mmio imx_dcss nwl_dsi mux_core pwm_imx27 ses
[  +0.000112]  enclosure sd_mod scsi_transport_sas uas usb_storage scsi_mod scsi_common hid_generic usbhid hid snd_soc_hdmi_codec xhci_plat_hcd xhci_hcd dwc2 dwc3 dw_hdmi_i2s_audio udc_core usbcore ulpi meson_gxl smsc nvme crc16 meson_vdec(C) videobuf2_dma_contig nvme_core snd_soc_meson_axg_toddr videobuf2_memops snd_soc_meson_axg_fifo v4l2_mem2mem t10_pi governor_simpleondemand videobuf2_v4l2 snd_soc_core crc64_rocksoft_generic polyval_ce videodev polyval_generic snd_compress panfrost crc64_rocksoft videobuf2_common crc_t10dif snd_pcm_dmaengine ghash_ce crct10dif_generic gpu_sched meson_dw_mipi_dsi gf128mul spi_gpio ti_sn65dsi86 rtc_pcf8523 sha2_ce drm_shmem_helper dw_mipi_dsi rtc_meson_vrtc mc dwc3_meson_g12a spi_bitbang sha256_arm64 snd_pcm meson_dw_hdmi sha1_ce pcs_xpcs crct10dif_ce clk_pwm phylink usb_common snd_timer crc64 snd crct10dif_common meson_drm ao_cec_g12a soundcore dw_hdmi reset_meson_audio_arb panel_edp axg_audio fixed_phy sclk_div cec clk_phase drm_dp_aux_bus fwnode_mdio meson_canvas rc_core libphy
[  +0.000116]  gpio_regulator pwm_regulator drm_display_helper fixed aes_neon_bs aes_neon_blk aes_ce_blk aes_ce_cipher [last unloaded: of_mdio]
[  +0.000017] CPU: 5 PID: 5358 Comm: kworker/u12:1 Tainted: G        WC OE      6.8.11-mnt-reform-arm64 #1  Debian 6.8.11-1+reform20240529T181037Z
[  +0.000005] Hardware name: MNT Reform 2 with BPI-CM4 Module (DT)
[  +0.000004] Workqueue: phy0 rtw_watch_dog_work [rtw88_core]
[  +0.000013] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  +0.000004] pc : rtw_power_mode_change+0x138/0x190 [rtw88_core]
[  +0.000011] lr : rtw_power_mode_change+0x138/0x190 [rtw88_core]
[  +0.000010] sp : ffff800088adbcc0
[  +0.000002] x29: ffff800088adbcc0 x28: ffff000003061840 x27: ffff000000011098
[  +0.000006] x26: ffff000004d48900 x25: ffff0000030618c0 x24: 0000000000000000
[  +0.000005] x23: 0000000000000001 x22: 0000000000068dbc x21: 0000000000000000
[  +0.000005] x20: 0000000000000000 x19: ffff000004d49fa0 x18: 0000000000000000
[  +0.000006] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[  +0.000005] x14: 000000000000000c x13: 65646f6d20726577 x12: ffff800081c30528
[  +0.000005] x11: 0000000000000001 x10: 0000000000000001 x9 : ffff800080134940
[  +0.000005] x8 : ffff800081bd8358 x7 : c0000000ffffefff x6 : 0000000000000001
[  +0.000006] x5 : ffff0000e47bcd08 x4 : 0000000000000000 x3 : 0000000000000027
[  +0.000005] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff00000469b780
[  +0.000006] Call trace:
[  +0.000002]  rtw_power_mode_change+0x138/0x190 [rtw88_core]
[  +0.000011]  rtw_sdio_deep_ps+0x90/0xd0 [rtw88_sdio]
[  +0.000008]  rtw_enter_lps+0xf4/0x138 [rtw88_core]
[  +0.000010]  rtw_watch_dog_work+0x2a8/0x2c0 [rtw88_core]
[  +0.000009]  process_one_work+0x170/0x3a8
[  +0.000007]  worker_thread+0x23c/0x460
[  +0.000004]  kthread+0xe8/0xf8
[  +0.000005]  ret_from_fork+0x10/0x20
[  +0.000005] ---[ end trace 0000000000000000 ]---

Any suggestions on how to address this?

This may be helpful:

2 Likes

Before the (any) fix, the workaround, in sway or wayfire, was to simply click the wifi AP icon, which disconnected/reconnected fairly rapidly. This brought the wifi connection back each time.

Disabling low-power mode solves the problem (an unfortunate fix?). I’d previously found workarounds discussed here 215496 – dmesg flood: realtek wi-fi: rtw_8822ce firmware failed to leave lps state which points to this firmware failed to leave lps state · Issue #61 · lwfinger/rtw88 · GitHub. There’s additional fixes for rtw88 driver as module or compiled but the args imply it’s just another way to disable low-power I didn’t bother exploring further.

pff’s solution is mentioned, and works here.

File: /etc/NetworkManager/conf.d/default-wifi-powersave-on.conf

[connection] 
wifi.powersave = 2

At least, this ends frequent “firmware failed to leave lps”, will take a while to know if it fixes dropouts but I’m reasonably confident.

Thanks so much!!

tom

PS: This also worked, here on my bench. I qualify that because the bug manifests under circumstances that don’t superficially make sense, eg weak RX signal:

  • A one-second ping to a known host.

Before I added the recommended config file, I did this and the net connection stayed up for six hours without a single lost ping and the streaming music did not fail.

This implies some timeout time is miscalculated; eg it’s trying to change power modes while the chip is in use in some way and driver or hardware gets confused. If so, the one-second (/usr/bin/ping default) is sufficiently often to preempt the timeout.

1 Like

OMG I literally redid my antenna layout of my Reform thinking it was due to a radio dead zone in my house (my desktop has lousy reception there fwiw) and it was driving me a little crazy. I’ll continue to test the powersave solution on my A311D Reform.

3 Likes