MNT Reform System Image V3 Beta

@bnys there were no major improvements in recent months, just integrating of the workaround and adjustments that were already known. But as far as I know, no major breakthrough was achieved in the past months and suspend is still not 100% reliable.

If you want to see what is done different on the reform when it suspends, then you have to look at the reform-tools package. I moved everything to there so if somebody finds something that improves suspend and isn’t a kernel fix then it goes there. Essentially it’s this systemd unit: debian/reform-tools.reform-sleep.service · main · Reform / MNT Reform Tools · GitLab calling this script sbin/reform-standby · main · Reform / MNT Reform Tools · GitLab

2 Likes

@bnys @josch thank you for the information folks. After banging my head against the wall for some time I realised that I had disabled reform-sleep service at some point so that explains the bizarre behaviour.

After enabling it I still have issues with wifi. Namely, my wifi card doesn’t show up in lspci output and the wifi interface is absent from ifconfig.

This time there were no issues with the deriver, in fact dmesg says this:

[ 3184.420370] Qualcomm Atheros AR8035 30be0000.ethernet-1:04: attached PHY driver (mii_bus:phy_addr=30be0000.ethernet-1:04, irq=POLL)

Simple things like restarting networking and reloading atheros drivers didn’t help :frowning:

I Had a bit of a carry-on with my new reform; I ran an apt-update after setup (habit) and it failed, and then it wouldn’t boot. Of course I didn’t have a backed up System Image, and I found this thread, but I don’t have a working V3 SD image; I am using V2 as I write this.

V3 just doesn’t boot; I read the suggestion from up-thread to DD an updated U-boot to the SD image, but this link is now dead too. I would like to try that, does somebody have a working link to the U-boot copy?

Latest apt upgrade bricked mine as well… :no_mouth:

I had already ran apt before seeing your comment and unfortunately mine is also bricked now :sweat_smile:

Edi: Bricked was too strong of a term, i can just swap to my other SD card and boot fine.

You were better prepared then me. I should have made a copy of the SD card. I’ve been running trouble free for 2 days now. I’d like to try the latest image, but I also don’t mind waiting for the official release.

I chatted briefly in IRC to @josch who helped regarding my difficulty with V3, and how I’d tried to buiid it with mkimage.sh without success (I was using the incorrect branch) and actually this knowledge worked for me. I ran the script with the correct branch and I got a working image. I’m now typing this on V3, so thanks for the help.

1 Like

So my understanding is that from the pipelines:

The latest build #588 is bad, possibly for the same reason that apt upgrade goes bad?

The previous build #571 is OK, but the artifacts are no longer available.

I’ve uploaded what I think is #571 from my downloads folder, in case anyone is desperate for a v3 build.
https://numbatlogic.com/Sekrit/artifacts.zip

I don’t understand. I just downloaded the rescue image from #588 and it booted just fine. What problem are you experiencing with it?

My experience was going from build #xxx (probably 571) → #588 using apt upgrade caused an issue after reboot.

Keyboard lights turn on, however the screen does not and it is unclear if anything more is happening at this point.

Had been using apt upgrade successfully on a daily basis for a couple weeks before this.

You cannot go from one CI build number to another via “apt upgrade”. The reform-system-image CI artifacts are snapshots of what is published on Index of /reform-debian-repo/ and since the CI pipeline is only run weekly, there may be package uploads that never make it into a CI build of reform-system-image.

If a kernel upgrade broke your system, you could probably revive it by booting your reform from a SD-card, connecting to the internet and then running:

mount /dev/mmcblk0p2 /mnt # assuming your system is on eMMC
mount /dev/mmcblk0p1 /mnt/boot # assuming you boot from eMMC
mount -o bind /dev /mnt/dev
mount -t proc proc /mnt/proc
mount -t sysfs sysfs /mnt/sys
cat /etc/resolv.conf > /mnt/etc/resolv.conf
chroot /mnt apt update
chroot /mnt apt upgrade
umount /mnt/sys /mnt/proc /mnt/dev /mnt/boot /mnt
1 Like

Thank you for the clarification and information!

I can confirm that the pre-built image for #763 does not boot from SD.

The kernal is kaput :broken_heart:

Ah, I now see what you guys mean. Yes, the latest image as published by the reform-system-image CI pipeline does something that it should not: it boots from eMMC instead of the SD-Card and if you don’t have a sysimage-v3 on your eMMC, then your system doesn’t boot. This is why I didn’t see any problems a few messages earlier because my system booted just fine but I didn’t notice that what booted was my eMMC system and not the system I flashed on the SD-Card.

Since an image that I build locally boots from SD-Card just fine, maybe @minute can quickly re-trigger the reform-system-image pipeline for the sysimage-v3 branch? If that build works then all good and if not, then there is a more fundamental problem in how images are created on the CI pipeline.

EDIT: I now know why the image produced by the latest pipeline run doesn’t boot: it’s missing a boot.scr on the first partition. I don’t yet know why this is, especially because an image created locally contains a boot.scr just fine. But this explains why the current image boots from eMMC: u-boot on the latest CI image will first try reading boot.scr from SD-Card and cannot find it. It tries eMMC next and will boot that if the first partition on eMMC contains boot.scr. If your eMMC doesn’t contain a sysimage-v3 then it will just not boot.

Maybe just re-running the pipeline will fix the problem. There was a recent upload of flash-kernel (which is responsible for creating boot.scr) to Debian which integrated MNT Reform support. And maybe the image built at a time when the Debian mirror used by the CI didn’t have a flash-kernel version that was new enough yet.

2 Likes

New Image builds:

2 Likes

Thanks @minute and @josch this community is fantastic.

I have good news and bad news.

The bad news: the latest reform-system-image build still isn’t bootable and there is no ETA on when it will be.

The good news: I now understand why this happens.

Essentially this happened because of some recent changes in the flash-kernel package. I already filed a merge request with the fix for this particular problem: Draft: Ignore the existance of /sys/firmware/efi if we are inside a chroot (!33) · Merge requests · Debian Installer / flash-kernel · GitLab Essentially, reform-system-image will create a bootable image on systems without /sys/firmware/efi (like my own intel laptop) and will create unbootable images on systems with it (like the system running the CI).

Until this is fixed in flash-kernel I added a patch to the flash-kernel package we build to temporarily revert the change: patches/flash-kernel: revert skipping flash-kernel on EFI systems (6da705dc) · Commits · Reform / reform-debian-packages · GitLab

Unfortunately, this also doesn’t completely fix the problem yet, as the reform-debian-packages pipeline doesn’t succeed. Originally the problem was, that our old blender wasn’t supporting OpenImageIO 2.3.4 but I also fixed that: Patch blender to support OpenImageIO 2.3.4 (f7bb0c4d) · Commits · Reform / reform-debian-packages · GitLab

The new problem is, that a new version of dcmtk was uploaded to Debian which causes unrelated breakage. This has to be fixed first but could take a bit longer because we also have to fix all its reverse dependencies. If you want you can keep track of the progress in the bug I filed here: https://bugs.debian.org/1010536

4 Likes

Appreciate the continued support and thank you for the thorough transparency as well @josch

Thanks everyone for working on this and keeping us updated!

Good to know it wasn’t just me doing something wrong.