Attention u-boot-menu users: run u-boot-update after kernel upgrade

Hi,

I just upgraded my system that was using u-boot-menu and it failed to boot afterwards. The reason for that was that after installing the new kernel version, this is how /boot/extlinux/extlinux.conf looked like:

## /boot/extlinux/extlinux.conf
##
## IMPORTANT WARNING
##
## The configuration of this file is generated automatically.
## Do not edit this file manually, use: u-boot-update

default l0
menu title U-Boot menu
prompt 1
timeout 1


label l0
	menu label Debian GNU/Linux 13 (trixie) 6.14.6-mnt-reform-arm64
	linux /vmlinuz-6.14.6-mnt-reform-arm64
	initrd /initrd.img-6.14.6-mnt-reform-arm64
	
	
	append   ro no_console_suspend cryptomgr.notests ${bootargs} console=tty1

label l0r
	menu label Debian GNU/Linux 13 (trixie) 6.14.6-mnt-reform-arm64 (rescue target)
	linux /vmlinuz-6.14.6-mnt-reform-arm64
	initrd /initrd.img-6.14.6-mnt-reform-arm64
	
	append   ro no_console_suspend cryptomgr.notests ${bootargs} console=tty1 single
	

That cannot work as it is missing any references to which device tree to load. So, if you are booting using u-boot-menu, after you upgrade your kernel, have a look at your /boot/extlinux/extlinux.conf and check if it’s broken like one can see above. If it is, you have two options:

  • run sudo u-boot-update
  • edit /boot/extlinux/extlinux.conf manually and add fdtdir /dtbs/ entries to the first label label l0 (the trailing slash is relevant)

I’m working on fixing this but it may take a bit because this is very tricky to reproduce, sorry…

A fixed /boot/extlinux/extlinux.conf might look like this:

## /boot/extlinux/extlinux.conf
##
## IMPORTANT WARNING
##
## The configuration of this file is generated automatically.
## Do not edit this file manually, use: u-boot-update

default l0
menu title U-Boot menu
prompt 1
timeout 1


label l0
	menu label Debian GNU/Linux 13 (trixie) 6.14.6-mnt-reform-arm64
	linux /vmlinuz-6.14.6-mnt-reform-arm64
	initrd /initrd.img-6.14.6-mnt-reform-arm64
	fdt /dtb-6.14.6-mnt-reform-arm64
	
	append   ro no_console_suspend cryptomgr.notests ${bootargs} console=tty1

label l0r
	menu label Debian GNU/Linux 13 (trixie) 6.14.6-mnt-reform-arm64 (rescue target)
	linux /vmlinuz-6.14.6-mnt-reform-arm64
	initrd /initrd.img-6.14.6-mnt-reform-arm64
	fdt /dtb-6.14.6-mnt-reform-arm64
	append   ro no_console_suspend cryptomgr.notests ${bootargs} console=tty1 single
	

@AbortRetryFail I now have a theory how this happened. Could you confirm that this happened for you not on your existing installation but for a fresh installation from a reform-system-any image you downloaded?

@vagrantc you also expressed in IRC that you have seen u-boot-menu problems. Is your problem maybe equal to the one I describe above?

It was my existing installation after apt dist-upgrade, not a fresh install. I’d have to write a new SD card to try a clean image.

No need, I can reproduce it on a clean image and I can fix it.

If the problem happened for you on an existing installation, then your problem is a different one.

I guess? Your OP description matches my experience exactly. The fdtdir is missing from extlinux.conf.

Well, after running sudo sh -x u-boot-update a whole lot of times, here’s how I fixed my existing Reform (imx8) install. If just running u-boot-update doesn’t put the fdt lines in extlinux.conf, try this:

echo 'U_BOOT_FDT_DIR="/dtbs/"' | sudo tee /etc/u-boot-menu/conf.d/u-boot-menu-sucks.conf

Then re-run u-boot-update. I hope this helps.

Wait, so you are saying that you are on a setup where running u-boot-update reliably does not put the fdt entry in?

Instead of working around this issue by using U_BOOT_FDT_DIR, could we debug this problem and fix it for good before the Trixie release so that others do not have to work around it?

If you remove /boot/extlinux/extlinux.conf and /etc/u-boot-menu/conf.d/u-boot-menu-sucks.conf and then run u-boot-update, is the resulting /boot/extlinux/extlinux.conf still missing its fdt entries? If yes, could you run u-boot-update like this and copypaste the output please?

sudo sh -x /usr/sbin/u-boot-update

Yep. Reliably broken.

I don’t know what’s more reliable, depending on u-boot-update’s defaults to know where to find the dtbs or just telling it, since we already tell it which one to use in reform.conf, why not give it the correct directory path as well?

Perfect! Then we have a reliable reproducer to fix this issue. :slight_smile:

Because this would mean that we are hiding a bug by working around it with a custom config. Why not instead fix the bug such that also users without a custom workaround have a functioning setup? :slight_smile:

Lets have a look at your output (thank you for that!). These lines in your log:

+ [ -e /freescale/imx8mq-mnt-reform2-hdmi.dtb ]
+ [ -e /boot6.14.6-mnt-reform-arm64//freescale/imx8mq-mnt-reform2-hdmi.dtb ]
+ [ -d /boot6.14.6-mnt-reform-arm64/ ]
+ [ -f /boot//freescale/imx8mq-mnt-reform2-hdmi.dtb ]
+ _FDT=

Correspond to these lines in u-boot-update:

	if [ -e "${U_BOOT_FDT}" ] && [ -n "${U_BOOT_FDT}" ] && [ "/" = $(echo "${U_BOOT_FDT}" | head -c1) ]
	then
		_FDT="fdt ${U_BOOT_FDT}"
	elif [ -e "${_BOOT_PATH}${U_BOOT_FDT_DIR}${_VERSION}/${U_BOOT_FDT}" ] && [ -n "${U_BOOT_FDT}" ]
	then
		_FDT="fdt ${U_BOOT_FDT_DIR}${_VERSION}/${U_BOOT_FDT}"
	elif [ -d "${_BOOT_PATH}${U_BOOT_FDT_DIR}${_VERSION}/" ]
	then
		_FDT="fdtdir ${U_BOOT_FDT_DIR}${_VERSION}/"
	elif [ -f "${_BOOT_PATH}/${U_BOOT_FDT:-dtb-${_VERSION}}" ]
	then
		_FDT="fdt /${U_BOOT_FDT:-dtb-${_VERSION}}"
	else
		_FDT=""
	fi

The problem is that there are slashes missing and then a bad path gets constructed which of course does not exist: /boot6.14.6-mnt-reform-arm64//freescale/imx8mq-mnt-reform2-hdmi.dtb

@vagrantc I already wrote about the missing slashes in those lines in my initial mail to this bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1091979 Can we not just add slashes there? Even if you have too many slashes, the path would still be valid. But with no slashes you get something invalid. This is also why (as noted in that bug) you have to write:

U_BOOT_FDT_DIR="/dtbs/"

The trailing slash is mandatory because the shell script concatenates path components without a slash. Can we change that?

1 Like