Restarts without warning

Looks like mine got it. It’s differently ordered: “aes pmull sha1 sha2” but it’s there.

1 Like

I think this is affecting me as well. I have sent a support ticket, and in the meantime am running that logger @minute

Jackpot! Been running reform-crashland in the background for a while and now I got a reboot. Eyeing the tail of the log it looks like it’s the battery suddenly at 0% problem others have. Have sent it by email, and made that bug report mentioned earlier in the thread.

3 Likes

Thanks for the logs, @aaa! Today I’m looking more closely at logs. I started with @holo_memory’s logs that I have since a longer time now, and I can already see 2 things:

  • There are still reform2_lpc ← SPI → sysctl transfer issues sometimes, where cells (voltages) read 0.0 0.0 or 0.100 0.0, like in @aaa’s log
  • On @holo_memory’s system, the charger seems to stop supplying power at regular intervals. This can be filtered from the logs like that:
for f in *.log; do echo $f; grep batstat $f | grep -v "V 0"; done

We’ll get lines like these:

20251023180311.log
batstat: 8.399V -1.111A 100% [status=0] [API=2]
batstat: 8.362V -1.149A 100% [status=0] [API=2]
batstat: 8.399V -1.118A 100% [status=0] [API=2]
batstat: 8.399V -1.24A 100% [status=0] [API=2]
batstat: 8.399V -1.80A 100% [status=0] [API=2]
batstat: 8.375V -1.225A 100% [status=0] [API=2]
batstat: 8.399V -1.55A 100% [status=0] [API=2]
batstat: 8.362V -1.111A 100% [status=0] [API=2]
batstat: 8.387V -1.24A 100% [status=0] [API=2]
batstat: 8.487V -1.143A 100% [status=0] [API=2]
batstat: 8.362V -1.24A 100% [status=0] [API=2]
batstat: 8.399V -1.149A 100% [status=0] [API=2]
batstat: 8.375V -1.11A 100% [status=0] [API=2]
batstat: 8.437V -1.36A 100% [status=0] [API=2]
batstat: 8.399V -1.100A 100% [status=0] [API=2]
batstat: 8.312V -1.6A 100% [status=0] [API=2]
batstat: 8.336V -1.42A 100% [status=0] [API=2]
batstat: 8.437V -1.49A 100% [status=0] [API=2]

This shows that the assumption “on AC” is not always what it seems. There are 100s of times where the AC adapter (USB-C power supply) doesn’t actually deliver power, so if there’s a big load spike in that moment as well, I can imagine it causing the brownout as well.

I should have added a timestamp on every log line, now it will take a bit of extra processing to figure out if there are regular time intervals involved.

What I can already say is that Charger Board V2 does not magically fix all brownout situations, simply because in a lower voltage situation, the 2 battery cells (rated 4Ah at nominal discharge rate 0.5C = 2A = 14.4W @ 7.2V) can’t deliver the needed voltage at large current spikes–which can happen at sudden high GPU+CPU bursts.

I’ve done some experiments with this recently and was able to watch a movie at 0% battery when limiting the GPU freq to 300 MHz and turning off the high performance CPU cores. So far I think limiting/controlling the GPU frequency is key, as the GPU seems to be causing large current spikes when clocked to 1GHz, which the default “simple_ondemand” GPU governor can do. Yesterday I wrote a tool called reform-power-daemon that can run as a service and limit the GPU frequency when no AC power is supplied. You can follow the development here: Draft: reform-power-daemon (!149) · Merge requests · Reform / MNT Reform Tools · GitLab

8 Likes

We’ve now released reform-power-daemon via reform-tools 1.82, available via apt. I’ve posted some more details here, testers wanted!

4 Likes

Hi! Thank you for this!

I tested running sudo journalctl -u reform-power-daemon -f and removing/replacing the charger, both with a power brick only and my dock which charges the device, but I don’t get any output at all in either scenario.

I’m on the 1.82 version of reform-tools.

Does reform-power-daemon need me to manually undo the changes that tuned-adm made or the gpu locking step? I’ve uninstalled tuned-* but didn’t do anymore changes than that.

I fear I’m missing something. Let me know what I might try next to debug!

@minute Reform tools fixes the random restarts for me while the machine is botted up, thank you!

However, if the battery drops below a certain percentage (call it 80%, but it probably closer to 85%), I see the same rebooting issue while powering on the device, so I to plug it in in order to power it on.

It’s the same issues I talked about in this issue: Continuing battery and voltage issues

Is reform-power-daemon started? Can you try the following:

sudo systemctl enable reform-power-daemon
sudo systemctl start reform-power-daemon
sudo systemctl status reform-power-daemon

And paste the output of the last command here please.

Also, it might error out if you don’t have reform2-lpc-dkms installed (the kernel module for the system controller/battery system).

1 Like

Just to confirm: it sounds like the batteries / the charger module v1 + battery combo is not strong enough to power through the boot, but once the system is booted and reform-power-daemon is running, you can unplug the power supply and it’s stable, or no?

Edit: do you already have a ticket open requesting charger module v2? It should be interesting to send you one and to see if it alleviates the boot issue, but it could also be that the cells are too weakened, and in this case (other than replacing with fresh cells) we need to come up with a way to limit the CPU+GPU freq early in boot (i.e. change the default governors, that should hopefully not be too hard in kernel configuration).

What you described is what is happening, yes.

Edit: do you already have a ticket open requesting charger module v2?

I purchased one through the site yes. Do you want the order number to expedite shipping me one?

1 Like

Just a random data point – I’d had random restarts a couple times over the past couple weeks on my pocket, but since I upgraded to firmware 20251118 and the reform-power-daemoninstalled and has been running (I checked), it’s been rock solid. YMMV.

2 Likes

Just started testing this, with the v2 charger board, and so far it seems to have fixed the restarts! Will report later as I continue to test.

1 Like

It was not running, the commands you provided worked and now it runs consistently. I should have checked this!

Here’s the output of sudo systemctl status reform-power-daemon:

● reform-power-daemon.service - MNT (Pocket) Reform Power Daemon Service (currently for RCORE/RK3588 only)
     Loaded: loaded (/usr/lib/systemd/system/reform-power-daemon.service; enabled; preset: enabled)
     Active: active (running) since Thu 2025-11-27 15:33:33 AST; 6min ago
 Invocation: 4d8cdc702fb64d00a3e9fbfbfe967505
   Main PID: 957 (reform-power-da)
      Tasks: 2 (limit: 37708)
     Memory: 5.1M (peak: 16.8M)
        CPU: 2.934s
     CGroup: /system.slice/reform-power-daemon.service
             ├─ 957 /bin/bash /usr/libexec/reform-tools/reform-power-daemon
             └─6581 sleep 1

Nov 27 15:33:33 pea systemd[1]: Started reform-power-daemon.service - MNT (Pocket) Reform Power Daemon Service (currently for RCORE/RK3588 only).
Nov 27 15:33:33 pea reform-power-daemon[985]: grep: /proc/device-tree/model: binary file matches
Nov 27 15:33:33 pea reform-power-daemon[988]: reform2_lpc            20480  0
Nov 27 15:33:33 pea reform-power-daemon[957]: System Controller firmware version: "20251118"
Nov 27 15:33:33 pea reform-power-daemon[957]: Keyboard firmware version: "20251118"
Nov 27 15:33:33 pea reform-power-daemon[957]: /usr/libexec/reform-tools/reform-power-daemon: line 31: ((: "20251118" < 20251030 : arithmetic syntax error: operand expected (error token is ""20251118" < 20251030 ")
Nov 27 15:33:33 pea reform-power-daemon[957]: /usr/libexec/reform-tools/reform-power-daemon: line 35: ((: "20251118" < 20251118 : arithmetic syntax error: operand expected (error token is ""20251118" < 20251118 ")
Nov 27 15:33:34 pea reform-power-daemon[957]: Battery status changed to Charging.
Nov 27 15:33:34 pea reform-power-daemon[957]: Charging

When I unplug and replug the power source now I get:

~►sudo journalctl -u reform-power-daemon -f                                                                                                                                                                                                                                                                       0.134s 15:39
Nov 27 15:33:33 pea systemd[1]: Started reform-power-daemon.service - MNT (Pocket) Reform Power Daemon Service (currently for RCORE/RK3588 only).
Nov 27 15:33:33 pea reform-power-daemon[985]: grep: /proc/device-tree/model: binary file matches
Nov 27 15:33:33 pea reform-power-daemon[988]: reform2_lpc            20480  0
Nov 27 15:33:33 pea reform-power-daemon[957]: System Controller firmware version: "20251118"
Nov 27 15:33:33 pea reform-power-daemon[957]: Keyboard firmware version: "20251118"
Nov 27 15:33:33 pea reform-power-daemon[957]: /usr/libexec/reform-tools/reform-power-daemon: line 31: ((: "20251118" < 20251030 : arithmetic syntax error: operand expected (error token is ""20251118" < 20251030 ")
Nov 27 15:33:33 pea reform-power-daemon[957]: /usr/libexec/reform-tools/reform-power-daemon: line 35: ((: "20251118" < 20251118 : arithmetic syntax error: operand expected (error token is ""20251118" < 20251118 ")
Nov 27 15:33:34 pea reform-power-daemon[957]: Battery status changed to Charging.
Nov 27 15:33:34 pea reform-power-daemon[957]: Charging
Nov 27 15:40:47 pea reform-power-daemon[957]: Battery status changed to Discharging.
Nov 27 15:40:47 pea reform-power-daemon[957]: Discharging
Nov 27 15:42:59 pea reform-power-daemon[957]: Battery status changed to Charging.
Nov 27 15:42:59 pea reform-power-daemon[957]: Charging

Thank you!

1 Like

The fact that you get this error message suggests that you do not have the latest version of reform-tools installed. Version 1.82-1 of reform-tools has the bug that the number contained double quotes leading to the output you see above and it had the bug that the service was not started automatically which is also a problem you have seen. Both of these problems were fixed in reform-tools 1.82-2, at least they should’ve been. Are you sure that you are running the latest version? I uploaded version 1.82-2 a week ago.

1 Like

ah, I think I just missed your patch last time I tried. I didn’t realize it meant the service wouldn’t start :person_facepalming: I’m all up to date now and on 1.82-2, thank you!

This shows that the assumption “on AC” is not always what it seems. There are 100s of times where the AC adapter (USB-C power supply) doesn’t actually deliver power, so if there’s a big load spike in that moment as well, I can imagine it causing the brownout as well.

@minute do you happen to have sysctl logs for this? Curious if there is a PD handshake issue.

Just wanna mention that since i installed the new board and reform-power-daemon, my brownout problem is gone :3

Another positive feature of the power daemon is, that the rk stays considerably cooler (an not accidentally burn your finger at the back XD) without feeling much of performance loss

6 Likes

I’ve started running into this problem over the past few days. I have a video, but I need to find a place to put it.

I have a CrowdSupply device with the RK3588 upgrade and firmware 120240730 (I held off on updating the firmware after the problems people seemed to have this Fall.

When I log in after boot, the cursor shows up on a black screen. The cursor is frozen, the screen turns gray, and the cursor gradually fades out, at which point the device reboots back to the login screen.

This seems to happen on battery power when charging is below 50%. Based on what I’ve read in other threads about reboots and brownouts, I watched the battery voltages while this happens and noticed that they jump between 3.0 and 3.7. If it successfully logs in, the voltage seems to settle at about 3.3-3.4. I have not had this happen when the device is plugged in.

Kinda seems more like a software than a hardware issue, but I do have the updated charging board ordered. These are also the original batteries, and I’ve let them go completely flat multiple times over the course of their lifetime. I didn’t notice any swelling when I was recently in the device.

Are these the “brownouts” you guys have been noticing?

Yes, probably! Are you up-to-date enough that your system is running reform-power-daemon? (Try running systemctl status reform-power-daemon; if it doesn’t say “Active: active (running)”, or says it doesn’t know what it is, you aren’t.) If you aren’t, you should update your system (or at least reform-tools), as it has made a big difference in stability for me.

1 Like

I just ran apt update (I was indeed behind: 1063 packages).

I didn’t see reform-power-daemon in it. Just to be sure, I tried the status command and an install command and was told it couldn’t be found. Is it a separate download?

I also got a message saying “Notice: Missing Signed-By in th sources.list(5) entry for ‘https://mntre.com/reform-debian-repo’,” so I don’t know whether that’s significant.