Testers wanted for new tool reform-rescue-shell (maybe interesting for users with the gdm issue)

Hi,

while the issue with gdm/mozjs/gcj is fixed in unstable, it might be tricky to apply the fix without being able to login. Since we still don’t have display support in u-boot, we cannot just boot Debian into single user mode to fix problems like this. You can switch to another tty but when I tried that, the tty froze after a short time (maybe because gdm tries to restart?). If you don’t have a serial uart adapter and you didn’t set up ssh access, then upgrading your system to a fixed version might have to involve flashing a new system image to an SD-card, booting that and then unlocking luks, enabling lvm logical volume, mounting rootfs, boot, sys, proc, bind-mount dev, copying resolv.conf, then you chroot and when you are done you undo everything. Maybe this is becoming enough for a new script. Here is a draft:

#!/bin/sh
# SPDX-License-Identifier: GPL-3.0+
# Copyright 2026 Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>

set -eu

usage() {
  echo "Run this command from a rescue system on SD-card." >&2
  echo "With no positional arguments, execute a bash shell inside the system" >&2
  echo "on eMMC/NVMe." >&2
  echo "With positional arguments, instead of an interactive shell, run the" >&2
  echo "command and its arguments passed as positional arguments to this tool" >&2
  echo "inside the system on eMMC/NVMe" >&2
  echo "This tool takes care of unlocking full disk encryption set up by" >&2
  echo "reform-setup-encrypted-disk and mounts /boot, /sys, /proc and /dev" >&2
  echo "It also copies /etc/resolv.conf from the current system into the" >&2
  echo "chroot, so if you have set up network connection on the outside," >&2
  echo "then the system on eMMC/NVMe will also have network access" >&2
  echo >&2
  echo "Usage: $0 [--help] [CMD...]" >&2
  echo >&2
  echo "Options:" >&2
  echo "  -h, --help       Display this help and exit." >&2
  echo >&2
  echo "Examples:" >&2
  echo >&2
  echo "Get a shell inside the system you have on NVMe or eMMC:" >&2
  echo >&2
  echo "  $0" >&2
  echo >&2
  echo "Run reform-check inside the system on NVMe or eMMC" >&2
  echo >&2
  echo "  $0 reform-check" >&2
  echo >&2
}

while getopts :h-: OPTCHAR; do
  case "$OPTCHAR" in
    h)
      usage
      exit 0
      ;;
    -)
      case "$OPTARG" in
        help)
          usage
          exit 0
          ;;
        *)
          echo "E: unrecognized option: --$OPTARG" >&2
          exit 1
          ;;
      esac
      ;;
    :)
      echo "E: missing argument for -$OPTARG" >&2
      exit 1
      ;;
    '?')
      echo "E: unrecognized option -$OPTARG" >&2
      exit 1
      ;;
    *)
      echo "E: error parsing options" >&2
      exit 1
      ;;
  esac
done
shift "$((OPTIND - 1))"

# by default, if no positional arguments were passed, run bash inside the chroot
if [ "$#" -eq 0 ]; then
  set -- /bin/bash
fi

if [ "$(id -u)" -ne 0 ]; then
  echo "reform-setup-encrypted-disk has to be run as root / using sudo."
  exit
fi

command -v "cryptsetup" >/dev/null 2>&1 || {
  echo >&2 'Please install "cryptsetup" using: apt install cryptsetup'
  exit 1
}
command -v "vgchange" >/dev/null 2>&1 || {
  echo >&2 'Please install "lvm2" using: apt install lvm2'
  exit 1
}

# shellcheck source=/dev/null
if [ -e "./machines/$(cat /proc/device-tree/model).conf" ]; then
  . "./machines/$(cat /proc/device-tree/model).conf"
elif [ -e "/usr/share/reform-tools/machines/$(cat /proc/device-tree/model).conf" ]; then
  . "/usr/share/reform-tools/machines/$(cat /proc/device-tree/model).conf"
else
  echo "E: unable to find config for $(cat /proc/device-tree/model)" >&2
  exit 1
fi

# We need to wrap findmnt output in realpath because if the SD-card was mounted
# via its label, then we need to resolve /dev/disk/by-label to the device name
case "$(realpath "$(findmnt --noheadings --evaluate --mountpoint / --output SOURCE)")" in
  "/dev/${DEV_SD}p"*) : ;;
  *)
    echo "E: This script is meant to be run from a rescue system from SD-card" >&2
    exit 1
    ;;
esac

# eMMC device is being used (case 1): mount points are known, show them. Includes swap.
if [ -n "$(lsblk --noheadings --output=MOUNTPOINT "/dev/${DEV_MMC}")" ]; then
  echo "E: eMMC has the following mounted volumes, unmount them before running this tool" >&2
  lsblk --noheadings --output=MOUNTPOINT "/dev/${DEV_MMC}" | xargs --no-run-if-empty -I '{}' echo "E:   {}" >&2
  exit 1
fi

# eMMC device is being used (case 2): there are not file systems directly mounted on the block device
# but it is opened by consumers like device-mapper, raid or luks, to name some examples. In this situation
# it is not trivial to locate the consumer.
# reform-boot-config and reform-emmc-bootstrap do the same thing (could share code?)
get_exclusive_write_lock() {
  ret=0
  python3 - "$1" <<EOF || ret=$?
import errno, os, sys

try:
    os.open(sys.argv[1], os.O_WRONLY | os.O_EXCL)
except OSError as e:
    if e.errno == errno.EBUSY:
        sys.exit(1)
    raise
EOF
  return $ret
}

if ! get_exclusive_write_lock "/dev/${DEV_MMC}"; then
  echo "E: device /dev/${DEV_MMC} (eMMC) is still in use" >&2
  exit 1
fi

mmc_num_parts=0
mmc_disk_label=$(parted --script --json "/dev/${DEV_MMC}" print 2>/dev/null | jq --raw-output '.disk.label')
case $mmc_disk_label in
  msdos | gpt)
    mmc_num_parts=$(parted --script --json "/dev/${DEV_MMC}" print 2>/dev/null | jq --raw-output '.disk.partitions | length')
    ;;
  unknown)
    # could be luks
    if cryptsetup isLuks "/dev/${DEV_MMC}"; then
      mmc_num_parts=luks
    fi
    ;;
  *) ;;
esac

ssd_num_parts=0
if [ -b "/dev/${DEV_SSD}" ]; then
  # NVMe device is being used (case 1): mount points are known, show them. Includes swap.
  if [ -n "$(lsblk --noheadings --output=MOUNTPOINT "/dev/${DEV_SSD}")" ]; then
    echo "E: NVMe has the following mounted volumes, unmount them before running this tool" >&2
    lsblk --noheadings --output=MOUNTPOINT "/dev/${DEV_SSD}" | xargs --no-run-if-empty -I '{}' echo "E:   {}" >&2
    exit 1
  fi

  if ! get_exclusive_write_lock "/dev/${DEV_SSD}"; then
    echo "E: device /dev/${DEV_SSD} (SSD) is still in use" >&2
    exit 1
  fi

  ssd_disk_label=$(parted --script --json "/dev/${DEV_SSD}" print 2>/dev/null | jq --raw-output '.disk.label')
  case $ssd_disk_label in
    msdos | gpt)
      ssd_num_parts=$(parted --script --json "/dev/${DEV_SSD}" print 2>/dev/null | jq --raw-output '.disk.partitions | length')
      ;;
    unknown)
      # could be luks
      if cryptsetup isLuks "/dev/${DEV_SSD}"; then
        ssd_num_parts=luks
      fi
      ;;
    *) ;;
  esac
fi

trap cleanup EXIT INT TERM
MOUNTROOT="$(mktemp --tmpdir --directory reform-emmc-bootstrap.XXXXXXXXXX)"

cleanup() {
  if [ -e "$MOUNTROOT/etc/resolv.conf.reform-rescue-shell.bak" ] || [ -L "$MOUNTROOT/etc/resolv.conf.reform-rescue-shell.bak" ]; then
    rm "$MOUNTROOT/etc/resolv.conf"
    mv "$MOUNTROOT/etc/resolv.conf.reform-rescue-shell.bak" "$MOUNTROOT/etc/resolv.conf"
  fi
  if mountpoint --quiet "$MOUNTROOT"; then
    umount --recursive "$MOUNTROOT"
  fi
  rmdir "$MOUNTROOT"
  if [ -e /dev/reformvg ]; then
    vgchange -an reformvg
  fi
  if [ -e /dev/mapper/reform_crypt ]; then
    cryptsetup luksClose reform_crypt
  fi
}

main() {
  rootdev="$1"
  bootdev="$2"
  shift 2
  mount "$rootdev" "$MOUNTROOT"
  for dir in etc boot dev sys proc; do
    if [ ! -d "$MOUNTROOT/$dir" ]; then
      echo "E: The directory '$dir' does not exist in the filesystem on $rootdev" >&2
      exit 1
    fi
  done

  mount -o bind /dev "$MOUNTROOT/dev/"
  mount -t sysfs sys "$MOUNTROOT/sys/"
  mount -t proc proc "$MOUNTROOT/proc/"
  mount "$bootdev" "$MOUNTROOT/boot/"

  if [ ! -d "$MOUNTROOT/boot/extlinux" ] && [ ! -e "$MOUNTROOT/boot/boot.scr" ]; then
    echo "E: Neither extlinux directory nor boot.scr exist in filesystem on $bootdev" >&2
    echo "I: If this is intentional, touch boot.scr on that partition." >&2
    exit 1
  fi

  if [ -e "$MOUNTROOT/etc/resolv.conf" ] || [ -L "$MOUNTROOT/etc/resolv.conf" ]; then
    mv "$MOUNTROOT/etc/resolv.conf" "$MOUNTROOT/etc/resolv.conf.reform-rescue-shell.bak"
  fi
  cp --dereference /etc/resolv.conf "$MOUNTROOT/etc/resolv.conf"

  chroot "$MOUNTROOT" "$@"

  if [ -e "$MOUNTROOT/etc/resolv.conf.reform-rescue-shell.bak" ] || [ -L "$MOUNTROOT/etc/resolv.conf.reform-rescue-shell.bak" ]; then
    rm "$MOUNTROOT/etc/resolv.conf"
    mv "$MOUNTROOT/etc/resolv.conf.reform-rescue-shell.bak" "$MOUNTROOT/etc/resolv.conf"
  fi

  umount --recursive "$MOUNTROOT"
}

case "$mmc_num_parts:$ssd_num_parts" in
  "2:luks") echo "I: Assuming /boot on eMMC and root filesystem on encrypted SSD." >&2 ;;
  "2:0") echo "I: Assuming both /boot and root filesystem on eMMC." >&2 ;;
  *)
    echo "E: your configuration $mmc_num_parts:$ssd_num_parts is currently unsupported, please file a ticket with details of your setup to get it supported" >&2
    exit 1
    ;;
esac

printf "Does that sound right? [y/N] " >&2
read -r response
if [ "$response" != "y" ]; then
  echo "Exiting."
  exit 1
fi

case "$mmc_num_parts:$ssd_num_parts" in
  "2:luks")
    cryptsetup luksOpen "/dev/${DEV_SSD}" reform_crypt
    vgchange -ay reformvg

    main /dev/reformvg/root "/dev/${DEV_MMC}p1" "$@"

    vgchange -an reformvg
    cryptsetup luksClose reform_crypt
    ;;
  "2:0")
    main "/dev/${DEV_MMC}p2" "/dev/${DEV_MMC}p1"
    ;;
esac

rmdir "$MOUNTROOT"

trap - EXIT INT TERM

It currently only supports the setup created by reform-setup-encrypted-disk but I can add more upon request. If you experience problems, run with sh -x and give me the output.

I’m looking forward to your feedback!

6 Likes

Hi,

i never booted the reform from an sdcard and don’t have any experience with u-boot, but i wanna try this before i have an real problem and don’t know how to do it :smiley:

If i understand your post correctly i flash the sd-card with the pocket-image and also place your script inside it? And then insert the sd-card and it should boot automatically from the card?

Or are there some extra steps i’m missing?
I’ve done with this with arch and an x86 may times, but the pocket is kinda special in that way, so excuse me for my maybe dumb questions ^^

1 Like

You flash the system image to an sd-card. There are multiple ways to get the script into it. You could also for example download the script from the internet after you have booted that system image from sd-card. You could also put the script on a usb-stick and copy it from there after you have booted.

The thing that makes the Pocket or all modules except imx8mq special is that you do not have early display support. So until Linux boots from sd-card and initializes your display you are unfortunately “blind” unless you also connect a usb uart adapter to look at the serial output.

1 Like

The thing that makes the Pocket or all modules except imx8mq special is that you do not have early display support.

yeah, thats probably why i’m currerntly more insecure than with my other linux devices. BUT, since i have an uart adapter now, i can also test this :3

@josch your script works! its awesome, no more manual mounting luks encrypted devices ^^

i also checked the script with the argument ‘reform-check’ which also worked fine

two remarks thou:

  1. it will provably help some users to display that they are now in the chroot environment. maybe this can be done to edit $PS1 to sth like ‘chrooted-mnt’ or sth else, i’m not good with defining names :smiley:

  2. after reboot to my normal disk,my network connection was broken, cause your script overwrites /etc/resolve.conf. i saw this in the logs and recreated the link: /etc/resolvconf/update.d/libc: Warning: /etc/resolv.conf is not a symbolic link to /run/resolvconf/resolv.conf . Maybe we could fix this somehow or hind it to the user somewhere.

In case someone asks how i put the script on the image.
I dumped the image to the sd-card normally and mounted it afterwards to /mnt. now you can simly place the script within sth like /mnt/home/reform-rescue.sh

It did? I only added a version that can do this just now.

Apologies. I had this fixed locally but didn’t see any urgency of copying it into this post because I thought that whatever network management tool is used will probably restore /etc/resolv.conf once it connects to a network to whatever it needs. Now I learn that it is indeed necessary to do the dance of moving it out of the way and then back afterwards.

Hrm… if we are moving away and restoring /etc/resolv.conf maybe the same can be done with /etc/hostname?

It did yes. i downloaded the script shortly after 6pm..

Isn’t that a bit of an overkill? :smiley:
is that even active without an reboot? afaik even with hostnamectl i have at least to log off and on again, to see the effect.

Do you mean that you manually edited the script and replaced /bin/bash with reform-check?

no, i downloaded the script, and it was already in there.. i didn’t edit anything
here are the file stats. so i downloaded it around 17:48
00:47:50] rick@mnt-rick /mnt/home (0)

.rw-r--r-- 7.3k root 21 Jan 17:48 reform_resc.sh
[00:47:50] rick@mnt-rick /mnt/home (0) 
> md5sum  reform_resc.sh 
e313fd156872658e49782ddfe9ff823f  reform_resc.sh

And your reform-check was already there:

> cat reform_resc.sh | grep reform-check
	echo "Run reform-check inside the system on NVMe or eMMC" >&2
	echo "  $0 reform-check" >&2
1 Like

@josch edit:
forget that, i think its too late for today.. my sway config is in ~/.config/sway

I attempted to run the script and received the message:

It ran perfectly. (After i ran reform-emmc-bootstrap to fix my drive layout)

Thank you

1 Like

Potentially. Do you happen to know how I change PS1 without changing the bashrc?

urgh you got a point.. i thought just passing it to bash will be enough, but nope.. bash overrides it every time from .bashrc..

I’m going to try to get back to this today. IS this one still the latest, or has it been updated? Also, is there possibility of data loss on the NVMe? Do I need to back up first? (Even if the system is unbootable, I can pull it out and get the data, but it’s a bit of an inconvenience.)

I’m going to try to get back to this today. IS this one still the latest, or has it been updated? Also, is there possibility of data loss on the NVMe? Do I need to back up first? (Even if the system is unbootable, I can pull it out and get the data, but it’s a bit of an inconvenience.)

Edit: I looked it over and ran it, and am doing a quick backup via rsync now. Will look into resolving the problems after that.

It was the latest version when you asked. With the help of @johl I improved the script further and updated my original post.

There shouldn’t but as is usual with computers, there always is.

What should I say? Of course the scripts I write shouldn’t make you loose data but the risk is always there.

Thanks for the reply. Your script worked great, I did a basic backup with rsync, and got (by replacing the two packages suggested in one of the threads) the system booting normally, and subsequently fully patched, and working! Thanks for the script, and all the help!

1 Like

Hi! Just wanted to chime in and say thank you for this script; it worked great and saved me from being stuck with the buggy upgrade (blank screen problem in [Fixed] Attention when upgrading: only black screen with cursor instead of gdm - #5 by josch) as intended.

2 Likes