How do I create a bootable image?

I have never seen this error message before. I do not know what to make of it. If you run the process as the normal user and not as root, the script will not use mke2fs but genext2fs which might give you no error or a different error.

Did you build uboot yourself or did you use the latest built uboot flash.bin from the CI? If you build it yourself, make sure to build the latest tag and not just HEAD.

I agree. I have found that writing the documentation before writing the code, that is, thinking about how you want to use the software before writing it, makes for better software. It’s hard to not fall into that trap when just writing code for fun though. :slightly_smiling_face:

The documents were written when the system still booted with a custom init script and there was no initramfs and the kernel was built without modules. These days, the boot process looks very different as we are trying to reduce the custom scripts in favor of using the same tools that everybody else already uses. This means that instead of compiling our own kernel binary from kernel.org git, we patch the official Debian kernel a bit and install that as a normal Debian package. This has multiple advantages:

  • allow upgrading the kernel via ‘apt upgrade’
  • system is closer to a ‘real Debian’ so that no special knowledge is needed
  • integration with flash-kernel and update-initramfs
  • allows building kernel modules like zfs-dkms via linux-kbuild & linux-headers
  • allows features like supermin+guestfs or anbox
  • support for all hardware that Debian supports via modules
  • minimize the diff so that reform support can be added to the official packaging

But it also has disadvantages, namely that the boot process is not the one documented in the handbook anymore but closer to what a normal Debian does. So somebody needs to update the documentation in the reform handbook. That will not be me because this is just a hobby project for me and I dislike writing documentation. :smiley:

Please consider this commentary, not whining or complaining. It’s an FOSS project and I respect everyone’s time and commitment (whatever it is).

  1. Once again thank you for your insightful help. In the future I’ll just put a thumbs up emoji (which in some countries is not a good thing…)

  2. I was running as me for the 2nd run of building sysimage-v3. A log is below.

  3. It did use genext2fs and it did take 7 HOURS 5 MINUTES during which time mk2efs was eating up CPU cycles and the md5sum on ‘reform-system.img’ changed.

  4. I did build the entire sysimage-v3 and put it on the SD card. It booted fine to a root shell… no magic MNT init stuff, no colorful logo, no ‘gnome’ command, no iwconfig… so sneakernet until I unbox the MoGo and set it up to provide a wired LAN port. In any case, this was too ‘primitive’ to use as is, so I went back to the previously downloaded sysimage.

  5. It is interesting to note, referencing your comments about the standard debian repositories and user experience… that on previous flashes of this image to the sd… I could just ‘apt install timeshift’. This time there was a conflict with bind-libs after apt checked bug reports. I tried upgrading 600+ packages (‘apt -y upgrade’) and yet it still had that conflict. Manually resolved by forcing that upgrade. Same for the new kernel. Like I said, I’m not thankless, and yes I know what “unstable” means :wink: just reporting data for future use.

  6. So right now it’s running fine SD/SD. Today’s remaining steps:

Convert to SD–>USB config (same comment)
Convert back to SD–>SD (has not worked, but for me if there’s no regression option, something is not right)
Convert back to SD–> USB (depends on 4)
Convert back to SD–>SD (second test, depends on 4, requires 5)
Convert to SD–>encrypted USB
Convert back to SD–>SD (third test, requires 7, no deps)

  1. Yes, spare time. I’m relegating my other projects to priority-3 and moving this up to priority-2. I see the opportunities here and am hopeful to help.

LOG BELOW!!

Ehud Gavron
Tucson, Arizona, US
FAA Commercial Helicopter Pilot
former VMS kernel hacker. That’s like 20 years ahead of the COBOL hackers :wink:

Command showing as top in ‘top-d 1’ - mkefsk. Last log output before 7+ hr delay: mk2efs
+ genext2fs --block-size 1024 --size-in-blocks 9216000 --bytes-per-inode 16384 --tarball - root.img
copying from tar archive

gavron@reform:/usr/local/src/reform-system-image/reform2-imx8mq$ ls -alh
total 21G
drwxr-xr-x 2 gavron gavron 4.0K Oct 10 20:06 .
drwxr-xr-x 5 gavron gavron 4.0K Oct 10 00:29 …
-rwxr-xr-x 1 gavron gavron 149 Oct 10 00:29 cleanup.sh
-rwxr-xr-x 1 gavron gavron 582 Oct 10 00:29 emulate.sh
-rw-r–r-- 1 gavron gavron 183 Oct 10 00:29 flash-image.sh
-rw-r–r-- 1 gavron gavron 273 Oct 10 00:29 .gitignore
-rw-r–r-- 1 gavron gavron 1.3K Oct 10 12:33 mk.err1
-rwxr-xr-x 1 gavron gavron 5.3K Oct 10 00:29 mkimage.sh
-rwxr-xr-x 1 gavron gavron 46 Oct 10 00:29 mkreform-tools.sh
-rwxr-xr-x 1 gavron gavron 2.5K Oct 10 00:29 mkuserland3.sh
-rwxr-xr-x 1 gavron gavron 4.3K Oct 10 00:29 mkuserland.sh
-rwxr-xr-x 1 gavron gavron 146 Oct 10 00:29 mount-img.sh
-rw-r–r-- 1 gavron gavron 2.5G Oct 10 13:01 reform-rescue-system.img
-rw-r–r-- 1 gavron gavron 9.3G Oct 10 20:06 reform-system.img <— time continuted to update while size stayed same. md5sum changed regularly.
-rw-r–r-- 1 gavron gavron 6.6G Oct 10 13:22 target-userland-full.tar
-rw-r–r-- 1 gavron gavron 1.9G Oct 10 12:43 target-userland.tar

sudo dd if=reform-system.img of=/dev/mmcblk0 status=progress

When I run the new reform-boot-config script I get an error:

check if rootfs is already mounted somewhere that is not /

MOUNTROOT=“$(lsblk --noheadings --output=MOUNTROOT “/dev/$ROOTPART”)”

  • lsblk --noheadings --output=MOUNTROOT /dev/sda1
    lsblk: unknown column: MOUNTROOT

I suspect you meant MOUNTPOINT, but that raises a problem I have with the script as a whole. Variable names are confusingly similar and sometimes interchanged. BOOTPART and ROOTPART look similar as well. Here’s another snippet:

OLDBOOTPART=“$(LIBMOUNT_FSTAB=”$MOUNTROOT/etc/fstab" findmnt --fstab --noheadings --evaluate --mountpoint /boot --output SOURCE)"
if [ -z “$OLDBOOTPART” ]; then
echo “cannot find /boot device referenced by /etc/fstab in rootfs at /dev/$ROOTPART” >&2
exit 1
fi

I’m going to go out on a limb and suggest that the condition doesn’t equate to the error message, but I don’t know because honestly BOOTPREF, BOOTPART, OLDBOOTPART, ROOTPART, etc. are just swimming in my head as I stare at this code. After thinking about trying to figure what’s going on here I thought I’d kick it up to you for your thoughts, as you’ve spent a lot of time on it.

Minor patch below.

Ehud
P.S. I hope I’m not offending in using PRE instead of CODE tags. The former requires a lot less ‘fixup’ in post :slight_smile:

# diff -u 20221010_reform-boot-config.orig 20221010_reform-boot-config
--- 20221010_reform-boot-config.orig	2022-10-12 11:46:08.088940957 -0700
+++ 20221010_reform-boot-config	2022-10-12 12:48:17.870261940 -0700
@@ -16,38 +16,49 @@
 # that must fit the correct kernel version. The choice of rootfs stored inside
 # the initramfs is derived from the settings of /etc/fstab in the rootfs.
 
+# Constants:
+EXIT_FAILURE=1
+EXIT_SUCCESS=0
+UID_ROOT=0
+
 set -eu
 
-if [ "$(id -u)" -ne 0 ]
-  then echo "reform-boot-config has to be run as root / using sudo."
+if [ "${UID_ROOT}" -ne "$(id -u)" ]
+  then 
+	  echo "$0 has to be run as root / or by using sudo."
   exit
 fi
 
-echo "This script selects your preferred boot medium. It writes your choice to the file /etc/fstab"
-echo
+echo "This script selects your preferred boot medium. It writes your choice to the file /etc/fstab \n"
 
 usage() {
-	echo "Usage: " >&2
-	echo "  reform-boot-config [--emmc] sd    # rootfs on SD card (/dev/mmcblk1p2, default)." >&2
-	echo "  reform-boot-config [--emmc] nvme  # rootfs on NVMe SSD (/dev/nvme0n1p1)." >&2
-	echo "  reform-boot-config [--emmc] usb   # rootfs on USB storage device (/dev/sda1)." >&2
-	echo "  reform-boot-config [--emmc] emmc  # rootfs on eMMC (/dev/mmcblk0p2)." >&2
-	echo "" >&2
-	echo "      --emmc Record boot preference on eMMC instead of SD card." >&2
-	echo "             This is only useful with SoM dip switch turned off." >&2
-	echo "" >&2
-	echo "Choosing sd, nvme, usb or emmc will set the root partition to" >&2
-	echo "/dev/mmcblk1p2, /dev/nvme0n1p1, /dev/sda1 or /dev/mmcblk0p2," >&2
-	echo "respectively.i You can choose another root partition by passing" >&2
-	echo "the absolute device path starting with /dev/ explicitly." >&2
-	echo "For example, to boot a rootfs on an LVM volume, run:" >&2
-	echo "" >&2
-	echo "    reform-boot-config /dev/reformvg/root" >&2
+	cat >&2 <<EOD
+	Usage:
+	  reform-boot-config [--emmc] {sd,nvme,usb,emmc}
+
+		--emmc Record boot preference on eMMC instead of SD card. 
+			This is used once the SoM dip switch is turned off. 
+
+	  Examples:
+		reform-boot-config [--emmc] sd    # rootfs on SD card (/dev/mmcblk1p2, default). 
+		reform-boot-config [--emmc] nvme  # rootfs on NVMe SSD (/dev/nvme0n1p1). 
+		reform-boot-config [--emmc] usb   # rootfs on USB storage device (/dev/sda1). 
+		reform-boot-config [--emmc] emmc  # rootfs on eMMC (/dev/mmcblk0p2). 
+	 
+	Choosing sd, nvme, usb or emmc will set the root partition to 
+	/dev/mmcblk1p2, /dev/nvme0n1p1, /dev/sda1 or /dev/mmcblk0p2, 
+	respectively.  A different root partition can be specified by passing 
+	the absolute device path starting with /dev/ explicitly. 
+
+	For example, to boot a rootfs on an LVM volume, run: 
+	 
+	    reform-boot-config /dev/reformvg/root 
+EOD
 }
 
 if [ $# -ne 1 ] && [ $# -ne 2 ]; then
 	usage
-	exit 1
+	exit $EXIT_FAILURE
 fi
 
 BOOTPREF="$1"

TL;DR This project started with a goal. It’s FOSS, so my goal was to get a script to go from normal SD boot to normal SD root to eventually running on a full LUKS-encrypted USB stick. That is now working. Details are below.
E
Josch: your gitlab reform-config-boot is not working for me. Before I touch your code I ask if you want to look at it or if you want to. Please see previous post.

Status report, and it’s good. My script (above) which got munged by MarkDown, works. It creates a USB (possibly NVMe) system with two partitions, one for ‘/’ and one for ‘swap’.

The provided script ‘reform-boot-config /dev/reformvg/root’ will do the right stuff to make this work. The commands inside will be integrated into the script as I document them in step-by-step detail.

Sometimes step-by-step in FOSS help sites means “I copied step-by-step from someone-else” and most of the time it says “copy and paste this.” My goal is to explain what we wish to accomplish and why we do so, and how some commands will get us there. See above for novice-explanations of physical volumes, volume groups, logical volumes, and mountpoints. It’s not magic and we can all do it, but knowing what we’re doing makes it meaningful.

SD/SD - works
→ USB works on MNT script. I’m hoping Josch can jump in and fix his script out of respect. Otherwise I’ll roll my own.
→ back to SD/SD works on MNT script.
→ Create encrypted system with no work using script above works. Having MNT script set it for boot works.

So here’s what remains to be done:

  1. Add the code from reform-config-boot to allow the new boot to automatically use the encrypted USB stick. In other words, a new user with an SD card can plug in a stick and have it work.

  2. Rework the scripts so that people can go from SD to USB to NVMe easily. Ideally one should be able to switch from SD to USB to NVMe to USB to SD seamlesslly. See #3.

  3. Hardware wire an SPST switch and mount in the case so the heatsink stays where it is and the switch below it is irrelevant and the user can select SD/NVMe from outside the case without opening it.

  4. While doing that, put a switch to disable the batteries without taking all 8 cells out.

Night.

E

2 Likes