A311D, wifi issues, "firmware failed to leave lps state", disconnects

MNT Reform
1

I recently started seeing many wifi disconnects, every 60 sec to 15 minutes. I of course assumed it was a wifi issue. Got a little more formal, comparing to another laptop on my bench, same distance to AP etc and it experienced zero wifi dropouts.

Via dmesg I found:

[  +3.167162] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +2.077631] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[  +2.714754] panfrost_gem_shrinker_scan: 11 callbacks suppressed
[  +0.000022] Purging 569344 bytes
[  +0.000635] Purging 565248 bytes
[  +3.211144] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +2.037847] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[  +0.008615] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +1.634856] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[  +0.289722] wlan0: disconnect from AP f0:72:ea:2a:61:5a for new auth to f0:72:ea:2a:61:5d
[  +0.639130] wlan0: authenticate with f0:72:ea:2a:61:5d (local address=c4:3c:b0:ad:cd:f1)
[  +0.383308] wlan0: send auth to f0:72:ea:2a:61:5d (try 1/3)
[  +0.017098] wlan0: authenticated
[  +0.002597] wlan0: associate with f0:72:ea:2a:61:5d (try 1/3)
[  +0.004380] wlan0: RX ReassocResp from f0:72:ea:2a:61:5d (capab=0x1011 status=0 aid=6)
[  +0.010392] wlan0: associated
[  +0.045861] wlan0: Limiting TX power to 30 (30 - 0) dBm as advertised by f0:72:ea:2a:61:5d
[Jun 1 10:49] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +2.047876] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[ +29.867939] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +2.091759] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[ +14.927270] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +2.064036] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[Jun 1 10:50] rtw_8822cs mmc2:0001:1: failed to send h2c command

Lots of instances of the failed to send h2c command/failed to leave lps state error message with rtw_8822cs driver.

Then a few minutes later this in dmesg…

[Jun 1 10:55] rtw_8822cs mmc2:0001:1: failed to send h2c command
[  +2.358867] rtw_8822cs mmc2:0001:1: firmware failed to leave lps state
[Jun 1 10:56] meson8b-dwmac ff3f0000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[  +0.001564] meson8b-dwmac ff3f0000.ethernet end0: __stmmac_open: Cannot attach to PHY (error: -19)
[  +0.008459] meson8b-dwmac ff3f0000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[  +0.001633] meson8b-dwmac ff3f0000.ethernet end0: __stmmac_open: Cannot attach to PHY (error: -19)
[  +0.011391] meson8b-dwmac ff3f0000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[  +0.002031] meson8b-dwmac ff3f0000.ethernet end0: __stmmac_open: Cannot attach to PHY (error: -19)
[  +0.011665] meson8b-dwmac ff3f0000.ethernet end0: Register MEM_TYPE_PAGE_POOL RxQ-0
[  +0.002655] meson8b-dwmac ff3f0000.ethernet end0: __stmmac_open: Cannot attach to PHY (error: -19)
[ +34.770467] ------------[ cut here ]------------
[  +0.000012] firmware failed to ack driver for entering Deep Power mode
[  +0.000032] WARNING: CPU: 5 PID: 5358 at drivers/net/wireless/realtek/rtw88/ps.c:105 rtw_power_mode_change+0x138/0x190 [rtw88_core]
[  +0.000025] Modules linked in: snd_seq_dummy snd_hrtimer rfcomm snd_seq aes_ce_ccm dwmac_meson8b stmmac_platform stmmac mdio_mux_meson_g12a mdio_mux of_mdio uhid algif_hash algif_skcipher af_alg qrtr bnep rtw88_8822cs rtw88_8822c rtw88_sdio rtw88_core binfmt_misc mac80211 hci_uart libarc4 btqca btrtl cfg80211 btintel btbcm bluetooth ecdh_generic snd_usb_audio snd_usbmidi_lib ecc snd_hwdep snd_rawmidi reform2_lpc(OE) snd_seq_device snd_soc_meson_axg_sound_card cdc_acm snd_soc_wm8960 rfkill snd_soc_meson_card_utils onboard_usb_hub snd_soc_meson_g12a_tohdmitx snd_soc_meson_axg_tdmin meson_gxbb_wdt meson_saradc snd_soc_meson_axg_frddr meson_rng industrialio snd_soc_meson_axg_tdmout leds_gpio nvmem_meson_efuse cpufreq_dt snd_soc_meson_axg_tdm_interface snd_soc_meson_axg_tdm_formatter sg evdev loop dm_mod efi_pstore configfs dax nfnetlink ip_tables x_tables autofs4 ext4 mbcache jbd2 crc32c_generic mali_dp cdns_mhdp_imx cdns_mhdp_drmcore snd_soc_meson_codec_glue mxsfb mux_mmio imx_dcss nwl_dsi mux_core pwm_imx27 ses
[  +0.000112]  enclosure sd_mod scsi_transport_sas uas usb_storage scsi_mod scsi_common hid_generic usbhid hid snd_soc_hdmi_codec xhci_plat_hcd xhci_hcd dwc2 dwc3 dw_hdmi_i2s_audio udc_core usbcore ulpi meson_gxl smsc nvme crc16 meson_vdec(C) videobuf2_dma_contig nvme_core snd_soc_meson_axg_toddr videobuf2_memops snd_soc_meson_axg_fifo v4l2_mem2mem t10_pi governor_simpleondemand videobuf2_v4l2 snd_soc_core crc64_rocksoft_generic polyval_ce videodev polyval_generic snd_compress panfrost crc64_rocksoft videobuf2_common crc_t10dif snd_pcm_dmaengine ghash_ce crct10dif_generic gpu_sched meson_dw_mipi_dsi gf128mul spi_gpio ti_sn65dsi86 rtc_pcf8523 sha2_ce drm_shmem_helper dw_mipi_dsi rtc_meson_vrtc mc dwc3_meson_g12a spi_bitbang sha256_arm64 snd_pcm meson_dw_hdmi sha1_ce pcs_xpcs crct10dif_ce clk_pwm phylink usb_common snd_timer crc64 snd crct10dif_common meson_drm ao_cec_g12a soundcore dw_hdmi reset_meson_audio_arb panel_edp axg_audio fixed_phy sclk_div cec clk_phase drm_dp_aux_bus fwnode_mdio meson_canvas rc_core libphy
[  +0.000116]  gpio_regulator pwm_regulator drm_display_helper fixed aes_neon_bs aes_neon_blk aes_ce_blk aes_ce_cipher [last unloaded: of_mdio]
[  +0.000017] CPU: 5 PID: 5358 Comm: kworker/u12:1 Tainted: G        WC OE      6.8.11-mnt-reform-arm64 #1  Debian 6.8.11-1+reform20240529T181037Z
[  +0.000005] Hardware name: MNT Reform 2 with BPI-CM4 Module (DT)
[  +0.000004] Workqueue: phy0 rtw_watch_dog_work [rtw88_core]
[  +0.000013] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  +0.000004] pc : rtw_power_mode_change+0x138/0x190 [rtw88_core]
[  +0.000011] lr : rtw_power_mode_change+0x138/0x190 [rtw88_core]
[  +0.000010] sp : ffff800088adbcc0
[  +0.000002] x29: ffff800088adbcc0 x28: ffff000003061840 x27: ffff000000011098
[  +0.000006] x26: ffff000004d48900 x25: ffff0000030618c0 x24: 0000000000000000
[  +0.000005] x23: 0000000000000001 x22: 0000000000068dbc x21: 0000000000000000
[  +0.000005] x20: 0000000000000000 x19: ffff000004d49fa0 x18: 0000000000000000
[  +0.000006] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[  +0.000005] x14: 000000000000000c x13: 65646f6d20726577 x12: ffff800081c30528
[  +0.000005] x11: 0000000000000001 x10: 0000000000000001 x9 : ffff800080134940
[  +0.000005] x8 : ffff800081bd8358 x7 : c0000000ffffefff x6 : 0000000000000001
[  +0.000006] x5 : ffff0000e47bcd08 x4 : 0000000000000000 x3 : 0000000000000027
[  +0.000005] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff00000469b780
[  +0.000006] Call trace:
[  +0.000002]  rtw_power_mode_change+0x138/0x190 [rtw88_core]
[  +0.000011]  rtw_sdio_deep_ps+0x90/0xd0 [rtw88_sdio]
[  +0.000008]  rtw_enter_lps+0xf4/0x138 [rtw88_core]
[  +0.000010]  rtw_watch_dog_work+0x2a8/0x2c0 [rtw88_core]
[  +0.000009]  process_one_work+0x170/0x3a8
[  +0.000007]  worker_thread+0x23c/0x460
[  +0.000004]  kthread+0xe8/0xf8
[  +0.000005]  ret_from_fork+0x10/0x20
[  +0.000005] ---[ end trace 0000000000000000 ]---

Any suggestions on how to address this?

2

This may be helpful:

2 Likes
3

Before the (any) fix, the workaround, in sway or wayfire, was to simply click the wifi AP icon, which disconnected/reconnected fairly rapidly. This brought the wifi connection back each time.

Disabling low-power mode solves the problem (an unfortunate fix?). I’d previously found workarounds discussed here 215496 – dmesg flood: realtek wi-fi: rtw_8822ce firmware failed to leave lps state which points to this firmware failed to leave lps state · Issue #61 · lwfinger/rtw88 · GitHub. There’s additional fixes for rtw88 driver as module or compiled but the args imply it’s just another way to disable low-power I didn’t bother exploring further.

pff’s solution is mentioned, and works here.

File: /etc/NetworkManager/conf.d/default-wifi-powersave-on.conf

[connection] 
wifi.powersave = 2

At least, this ends frequent “firmware failed to leave lps”, will take a while to know if it fixes dropouts but I’m reasonably confident.

Thanks so much!!

tom

PS: This also worked, here on my bench. I qualify that because the bug manifests under circumstances that don’t superficially make sense, eg weak RX signal:

  • A one-second ping to a known host.

Before I added the recommended config file, I did this and the net connection stayed up for six hours without a single lost ping and the streaming music did not fail.

This implies some timeout time is miscalculated; eg it’s trying to change power modes while the chip is in use in some way and driver or hardware gets confused. If so, the one-second (/usr/bin/ping default) is sufficiently often to preempt the timeout.

1 Like
4

OMG I literally redid my antenna layout of my Reform thinking it was due to a radio dead zone in my house (my desktop has lousy reception there fwiw) and it was driving me a little crazy. I’ll continue to test the powersave solution on my A311D Reform.

3 Likes
5

I’m currently dogfooding RCM4 with A311D in my Pocket Reform and wanted to see if something can be done about the glitch that the WiFi SDIO module sometimes fails to load during boot. Indeed this brings it back up (as root):

echo ffe03000.mmc > /sys/class/mmc_host/mmc2/device/driver/unbind
echo ffe03000.mmc > /sys/bus/platform/drivers/meson-gx-mmc/bind

We can probably add this as a workaround in reform-hw-setup. Also we should ship the powersave disable config mentioned in this thread.