Operating System on NVMe (without SD Card)

Interesting, I did not know that. Might end up using them more often then over partitions. Benefit of a partition though would be that it can be used with more than one install on the same system. I think ideally the swap file would be continuous to simplify that mapping.

Nothing can stop you mounting a partition from another system, if only to swapon the file. The real benefit (maybe the only) is suspend-to-disk.

Iā€™d prefer not to mount my default the root of my other OSes just to get the swap file. But yeah, suspend to disk is kinda a handy feature. Surprised it canā€™t do it to a swap file though, the early kernel already should have the filesystem driver to load root.

I donā€™t know if hibernation (suspend to disk) works on the SoC the Reform is using, but hibernation is one reason you might want to use a partition over a file.

It also allows you to encrypt your swap with a different password as your main.

Does anyone know if hibernation is working?

In my experiments hibernation did not resume correctly, but I also did not spend much time with it. Hibernation with encrypted disks is kind of problematic.

2 Likes

Hibernation is kinda problematic not only with encrypted disks :slight_smile:

Indeed. With suspend working on the Reform, for the most part, I donā€™t really see a need for hibernation.

One question I still have is the point of switching to the eMMC for booting.

I mean I get that it allows you to remove the SD card to boot, but letā€™s say you donā€™t do that: after you boot, the card can be removed, and you can use another if you want. I mean you donā€™t get any kind of performance boost or anything by switching to the eMMC, right?

What I am really asking is help in understanding the bootloader. The SD card is needed just until the bootlander hands off to Debian right? After that Debian isnā€™t looking back to the bootloader for anything, correct?

So after booting you could totally remove the SD card if you wanted, or swap it for something else, for example? Right?

Or will Debian look for something on the SD during use and this is why you canā€™t really remove it?

Yes, it is only needed by the bootloader and you can safely remove it after boot. The issue is that the i.MX8 has several boot modes (external SD Card, internal eMMC and serial Download) which are set with dip switches here (under the heat sink on the board).

And to make it convenient it is set to boot from SD Card by default.

Raspberry Pi is similar in that you can run everything from a USB drive but you still need the SD card to boot. I boot from eMMC because I donā€™t want to leave the SD card sticking out of the Reform. Maybe one of the shorter microSD card adapters they make to fit flush inside of MacBooks would fit the Reform and make it more convenient to always leave the SD card inserted.

No you donā€™t: both the Raspberry Pi 3 and Raspberry Pi 4 families support direct boot from USB or Ethernet PXE without a microSD card, once properly configured.

1 Like

Cool! I didnā€™t know that. The first models would only boot from SD card.

TL;DR - HOW CAN I BOOT DIRECTLY FROM USB with NO SDCARD QUESTION:

Hello from sunny Tucson, Arizona, US. Iā€™m still working on one script to go from SDcard to FDE (Luks) and Iā€™ve run into a problem.

Iā€™ve Google searched and donā€™t see the answerā€¦ or perhaps I see the answer and donā€™t like it.

I want to boot my Reform2 from USB without an SDcard or NVMe device present. I am currently unable to figure out how to do that, and the documentation on the boot process is sparse, as in ā€˜the script does Xā€™ but not which script, or who called it, or where it all got started.

Iā€™ve done the Google search. Iā€™ve looked at /boot/dtbs and the .scr files. I havenā€™t cloned the whole archive because it seems to me that things like ā€˜u-bootā€™ are separate from the ā€˜reformā€™ stuff. I did download the ā€˜newā€™ u-boot that is supposed to be able to boot from everything, but without the SD card and NVMe it wonā€™t boot.

I have loads of SD cards, and a couple of NVMe ones. This is about solving a general long-term need, not anything immediate. If I canā€™t get it to workā€¦ Iā€™ll try different hardware.

Unrelated to the question, other things on my mind when hardware is being discussedā€¦

Hardware Project #1: Iā€™m not messing with the NVMe until I get my SPST DIP switch and wire an external switch so I only have to open the case, pop the batteries out, and undo the heatsink ONCE. Still Iā€™d rather SOLVE the USB issue before going to NVME. (Yes I know I could use NVME and have u-boot on that go to USB just as Iā€™m doing now with SDcard, but Iā€™d rather rely on NOTHING other than that USB stickā€¦)

Hardware project #2: Add a DPST switch to allow disconnecting the batteries without opening the case.

Hardware Project #3: The USB ports are 2.0 and upside down. It would be nice to add a USB3.1Gen2 aka USB3.2. Iā€™m not knowledgeable enough about this virtualized chrooted systemā€™s compatibility with that goal tho, so for now thatā€™s just a dream. Josch has a great post at Adding LTE Modem and other USB devices - #6 by svp about adding a USB hub in the case, and he makes reference to the Nanohub which now offers 4 ports for $20 USD and fits in almost any space.

Have you read this?

it looks like the only options for loading u-boot are from an SD card or from EMMC. So youā€™ll have to flash EMMC, remove the heatsink, and flip the DIP switch to be able to boot without an SD card, independent of any other configuration you do.

I had to remove and replace the heatsink multiple times while getting my Reform to boot from EMMC. I suggest getting a reusable thermal pad and using that instead of thermal paste to make future removals less painful. I havenā€™t noticed any difference in temperature since switching to pads.

Yes, and I was hoping that the fact that USB is an option in ā€˜reform-initā€™ and ā€˜reform-boot-configā€™ meant it could be done without chainloading AFTER either SD/eMMC. Iā€™m not entirely sure why the functionality of the boot process is as limited as an rPi(1) and I had no end of issues with SDcard deaths with the Pis. I would even be happy with a USB ā†’ SDcard adapter but the only ones Iā€™ve found thus far are the other way around (SDcard to USB).

As for the tape, glue, etc., it still requires opening the case for every switch change. Iā€™d much rather do this in hardware and be able to do the bat disco and the switch change with two flips of a DIP switch.

Thank you for replying. I think youā€™re confirming (again) what Iā€™d hoped was not the case.

Best regards,

E

The boot process is defined by the SoM being used. The NXP i.MX8MQ boots from eMMC or SD card. This seems to be fairly common for ARM SoMs, from what I can tell.

Iā€™m booting from eMMC into an NVMe, myself. So long as youā€™re not writing frequently to flash memory, you shouldnā€™t have to worry about it dying on you. E.g., most of the SD cards that have died on me in other ARM-based single-board computers have been because I was running the whole filesystem off the card without turning off logging. In my present setup, the eMMC only gets read from, except for when I upgrade the kernel. I donā€™t expect itā€™ll be dying on me anytime soon!

1 Like

Thatā€™s super-helpful! Thank you. I wondered why my Pis kept having corrupt SDs, and YES all of them had the entire fs on the SD. It didnā€™t occur to me at the time to have a secondary device to switch to.

I am thinking instead of using my 500GB NVMes, to get a smaller one whose only purpose is switching to USB. Then I can ignore the SD, do what I want with USB (the ultimate goal is multiboot and multiple OS options), and the NVMe is of little relevance except for kernel/initramfs (why oh why did initrd get renamed to initramfs?)

Anyway, I havenā€™t done hardware projects in a while, and wiring three switches should be fun. I havenā€™t done software coding at a low level in a while so messing with the DPT is fun. Seriously MNT and the Reform(2) are a lot of fun.

My DPST switches arrive tomorrow. Outside-case switch to disco the batts. SPST switches arrive this weekend. Outside-case switch to switch from SD to NVMe. (Wire in parallel, setting inside to off, so outside switch controlsā€¦ OR wire in serial, setting inside to on, so outside switch controls. The goal here beingā€¦ no need to open the case or pop the batteries or mess with the heatsink for future testing.

Thank you for your help! If I get the switches working right there will be a write-up with pics so others can do it. I understand the concept is ā€œonce you set it to NVMe you wonā€™t want to look backā€ but for testing things itā€™s crucial to do so, and opening the case, removing the heatsink, too much work! :slight_smile:

Best regards,

Ehud Gavron
Tucson, Arizona, US

2 Likes

Iā€™d love to see pictures of how you do this, Iā€™ve also been thinking about adding a switch to disconnect the batteries.

Of course, pictures will be forthcoming once I get to it. I opened it up and quickly figured out why there are two sets of wires totalling 9 connectors. The schematic diagram confirms that all 8 + terminals as well as 1 - terminal ā€˜groundā€™ are sent up to the motherboard.

I think the easy button there is to put a switch on the - or ā€˜groundā€™ wire. Iā€™ll take another look at the schematics because I donā€™t easily see with my eye on the board how the - from the one pack (only 4 wires go out) is joined with the - from the other pack (5 wires).

Onto the SD/NVMe switch, the Operatorā€™s Handbook says a dab of thermal paste sits between the SOC and the heatsinkā€¦ and mine is too old to be useful. Iā€™ll pick up some thermal paste and then look at that one.

Iā€™m trying to minimize how many times I open/close the case, so if I can do these two projects and put in the NVMe at the same time that would be great. Iā€™ve found a place for the switchesā€¦ between the batteries coming right out of the caseā€¦ little micro (or mini) toggle switches.

More to follow as I get the paste and figure out where to easily disco the batts.

E

Please be very careful if attempting to wire switch(es) to disconnect the batteries, and check the schematics (for both the battery boards, and the mainboard) extremely carefully.

The wires coming from the battery boards are very definitely NOT positive from each battery, and a common negative/ground. Rather, the batteries are wired in series, and the wires coming from the battery boards are from ā€œbetweenā€ the batteries. The mainboard also has balancing resistors between the batteries, and I believe that these resistors can potentially be fried if removing batteries (or isolating them with switches) while the battery board is still connected to the motherboard.

I think there is a thread somewhere on these forums (around 6-9 months ago?) where someone has had exactly this problem, while trying something similar (isolating battery switch(es)). I think that thread even had further discussion including a design for doing the isolation properly with MOSFETs etc, and possibly also a design for a second version of the battery boards which includes this additional circuitry?

Anyway, I highly recommend that you find and read that thread, and very carefully study all the schematics, before attempting this work.

1 Like