Operating System on NVMe (without SD Card)

I followed the instructions and they worked but it uses the old version of the kernel from the eMMC. I was able to fix this by flashing a new recovery system.

dd if=reform-rescue-system.img of=/dev/mmcblk0 bs=1024

Then reboot into recovery and set boot medium again.

echo nvme > /reform-boot-medium

Now it boots without SD card to NVMe with the latest kernel.

1 Like

Hi all,

I have a much more basic question than some of the other issues in this thread. What should be the partition scheme for the ssd, and what is the “correct” way to run the reform-migrate script?

I tried what I thought was a rather basic partition scheme of:

  • boot (250mb)
  • root (60GB)
  • home (rest)
  • swap (24gb)

I think I mistakenly assumed the reform-migrate script wanted to use the first partition to install some boot stuff, but the program bugged out. Now I can’t even get the system to boot from the SD card (kernel panic).

Obviously I’m still a little confused about the whole boot process on this system.

Boot partition lives on the SD or eMMC, the SSD does not need one. I created a Master Boot Record partition map with fdisk and a single partition. I formatted it as ext4 and then ran reform-migrate /dev/nvme0n1p1

If your SD card was corrupted, you can make a new one with the latest image reform2-imx8mq · Artifacts · build (#426) · Jobs · Reform / reform-system-image · GitLab

dd if=reform-system.img of=/dev/sdcard bs=1024
2 Likes

Thanks, @8088mph.

Happy to report that I am now booting from the SSD! From my perspective, this is a huge speed and responsiveness improvement – ballpark 3x speedup all around, maybe more.

One question: I partitioned my drive such that there is a swap partition at the end, but I’ve noticed that when running free the system reports that there is 0 available swap space. I suppose this is some deeper-level linux thing I just don’t know, but is there some way to automatically read the swap space on boot and configure it correctly?

You need to do more than just create the partition.
First you need to set it up using mkswap /dev/whatever where whatever is your nvme partition such as nvme0p2
Then you can manually enable it with swapon /dev/whatever and it should show up. To make it permanent you can then add a line to your /etc/fstab

/dev/whatever none swap defaults 0 0

https://wiki.archlinux.org/title/swap#Swap_partition

1 Like

You might want to use UUID instead of /dev/whatever in case you ever change your partitions.

Is there any reason to use a swap partition over a swap file for an SSD?

True, it was just easier to show it with a device name rather than UUID, on the reform it’s likely not an issue either way as you can’t attach another nvme ssd if that’s what you’re running from.

A reason for a swap partition over a swap file is speed, though I’m currently using a file myself as I forgot to make the partition. With the partition it doesn’t have to go through the filesystem layer to access data, it’s doing direct disk accesses. With a file it has to go through the translation and possible fragmentation of the swap file, which may not be contiguous.

If you resize your primary partition and create a new partition before the swap, could the device name change from nvme0p2 to nvme0p3?

According to this thread, the kernel will bypass the filesystem when accessing a swap file. LKML: Andrew Morton: Re: Swap partition vs swap file

Interesting, I did not know that. Might end up using them more often then over partitions. Benefit of a partition though would be that it can be used with more than one install on the same system. I think ideally the swap file would be continuous to simplify that mapping.

Nothing can stop you mounting a partition from another system, if only to swapon the file. The real benefit (maybe the only) is suspend-to-disk.

I’d prefer not to mount my default the root of my other OSes just to get the swap file. But yeah, suspend to disk is kinda a handy feature. Surprised it can’t do it to a swap file though, the early kernel already should have the filesystem driver to load root.

I don’t know if hibernation (suspend to disk) works on the SoC the Reform is using, but hibernation is one reason you might want to use a partition over a file.

It also allows you to encrypt your swap with a different password as your main.

Does anyone know if hibernation is working?

In my experiments hibernation did not resume correctly, but I also did not spend much time with it. Hibernation with encrypted disks is kind of problematic.

2 Likes

Hibernation is kinda problematic not only with encrypted disks :slight_smile:

Indeed. With suspend working on the Reform, for the most part, I don’t really see a need for hibernation.

One question I still have is the point of switching to the eMMC for booting.

I mean I get that it allows you to remove the SD card to boot, but let’s say you don’t do that: after you boot, the card can be removed, and you can use another if you want. I mean you don’t get any kind of performance boost or anything by switching to the eMMC, right?

What I am really asking is help in understanding the bootloader. The SD card is needed just until the bootlander hands off to Debian right? After that Debian isn’t looking back to the bootloader for anything, correct?

So after booting you could totally remove the SD card if you wanted, or swap it for something else, for example? Right?

Or will Debian look for something on the SD during use and this is why you can’t really remove it?

Yes, it is only needed by the bootloader and you can safely remove it after boot. The issue is that the i.MX8 has several boot modes (external SD Card, internal eMMC and serial Download) which are set with dip switches here (under the heat sink on the board).

And to make it convenient it is set to boot from SD Card by default.

Raspberry Pi is similar in that you can run everything from a USB drive but you still need the SD card to boot. I boot from eMMC because I don’t want to leave the SD card sticking out of the Reform. Maybe one of the shorter microSD card adapters they make to fit flush inside of MacBooks would fit the Reform and make it more convenient to always leave the SD card inserted.

No you don’t: both the Raspberry Pi 3 and Raspberry Pi 4 families support direct boot from USB or Ethernet PXE without a microSD card, once properly configured.

1 Like

Cool! I didn’t know that. The first models would only boot from SD card.