Hello again, everyone!
I’m not sure if it’s directly related, but after a system upgrade yesterday and restarting the computer, I encountered the following problem:
As far as I understand, it doesn’t detect the MMC device and drops me in a console. The issue is that I can’t input any commands through the keyboard. I tried connecting an external keyboard through a USB-A → USB-C adapter, but had no luck either.
You are running Debian unstable on your machines. The apt-listbugs package is installed and will warn you before you upgrade your system about any release critical bugs in the package versions that you are about to install. Did you see those warnings before you upgraded?
I uploaded a fixed version of the kmod package into the mntre.com repository about 12 hours ago. But if you upgraded yesterday, then indeed your system is now unbootable because an initramfs was created which is missing virtually all kernel modules.
Fixing this involves booting a system from sd-card and then upgrading your real system to the fixed package versions using that.
Could you elaborate these steps? I have got it to boot from SDcard, but I am not quite familiar about what commands to issue to upgrade system on NVMe from SDcard.
Yes, but I need to know more about your system first. How did you set it up to boot from NVMe? Did you use the reform-setup-encrypted-nvme tool? I assume your /boot is on eMMC then?
Yes, you are right. I did see those warnings and I chose the wrong option when it asked me. I’m sorry, newbie here.
I have the same doubts as @shigeru. I managed to boot with the SD card and did a bit of research. If I’m correct, it seems that the system partitions need to be mounted. When I run lsblk, I can see the SD card partitions with their mounting points and also those of the eMMC: mmcblk2p1 (with the same size as the one on the SD card with the /boot mount point), mmcblk2p2 (116.1G). Although it doesn’t appear listed with the lsblk command, when I try to autocomplete the mount /dev/mmcblk2 command, it also shows me mmcblk2rpm, which I don’t know if it’s another partition that needs to be mounted.
Thank you once again for your help @josch. If there’s any kind of documentation we can check out to learn how to do this, it would be very helpful.
No worries, at least with this bug, there was no data lost. If you do not want to have to worry about upgrading your packages to versions which might break your system, there is also the option to use Debian stable with packages from reform.debian.net.
Unfortunately I do not think that there is any. Every time one of these bugs happens, a different approach is needed to get the original state back. Or, as in this case, the exact instructions depend on your setup. For example, this is what you would do (as root) if you ran reform-setup-encrypted-nvme and have your /boot on eMMC:
This is of course untested, so please be careful! If you have questions about any of these steps, please do not hesitate to ask. As usual when running Debian unstable: please do regular backups!
Yes, the last upgrade I have had the prompt, I should be more careful in the future.
critical bugs of kmod (32+20240611-1 → 33+20240816-1) <Outstanding>
b1 - #1079022 - kmod: symbol lookup error: /usr/lib/dracut/dracut-install: undefined symbol: kmod_module_get_weakdeps, version LIBKMOD_5
Summary:
kmod(1 bug)
Are you sure you want to install/upgrade the above packages? [Y/n/?/...]
For my NVMe, I have ran reform-migrate for setup without any encryption. I did not know about reform-setup-encrypted-nvme tool at that time. The tool was not mentioned in the handbook. I found the tool when I ssh into Pocket Reform the first time a bit later.
The last time I ran lsblk I had this result, which I was wondering…
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
mmcblk0 179:0 0 119.3G 0 disk
├─mmcblk0p1 179:1 0 488M 0 part /boot
│ /boot
│ /boot
└─mmcblk0p2 179:2 0 4G 0 part
nvme0n1 259:0 0 1.8T 0 disk
└─nvme0n1p1 259:1 0 1.8T 0 part /
mmcblk2 179:256 0 116.6G 0 disk
├─mmcblk2p1 179:257 0 488M 0 part
└─mmcblk2p2 179:258 0 116.1G 0 part
mmcblk2boot0 179:512 0 4M 1 disk
mmcblk2boot1 179:768 0 4M 1 disk
This means /boot is on SDcard, right? I would like it on eMMC, obviously.
Hope these helps you providing me with instruction for recovery.
Great! Against all odds, it seems that everything has worked succesfully. In my case, I don’t have NVMe installed, so I simplified the batch of commands to the following:
$ mount /dev/mmcblk2p2 /mnt
$ mount /dev/mmcblk2p1 /mnt/boot
$ mount -o bind /dev /mnt/dev
$ mount -t sysfs sys /mnt/sys
$ mount -t proc proc /mnt/proc
$ chroot /mnt apt update
$ chroot /mnt apt upgrade
// I've also used this command, but it's not clear to me if it's necessary. I believe the upgrade itself does it automatically.
chroot /mnt update-initramfs -u
// With this umount command I had some problems as /mnt/boot seemed to be busy
$ umount --recursive /mnt
sudo reboot
If you have a working /boot partition on your sd-card, you could copy its contents to the first partition on eMMC, adjust your /etc/fstab, ensure that the correct partition is now mounted in /boot and then run update-initramfs -u.
As you now lost your /boot partition and its contents, things became a bit more tricky. Restoring those contents means you have to re-install your linux image package. You can still try copying things but you need to regenerate your initramfs after you edited your /etc/fstab or otherwise it will not mount the desired rootfs.
It is looking that I better completely rebuild the system from scratch with newly flashed SDcard, I have. I wanted to have encrypted NVMe storage, anyway.
I was only able to boot from a SD that had the reform.debian.net image, so I had to make a wired connection to my router using a USB to Ethernet adaptor so I could run the apt update and upgrade steps.
I also had to boot twice, but it works. Thanks everyone!
If you do not want to have to worry about upgrading your packages to versions which might break your system, there is also the option to use Debian stable with packages from