Background: I recently purchased a Reform DIY Kit from Crowd Supply. I forgot to add a Molex antenna to the order, but the Wi-Fi card worked with a limited range. I started a system upgrade running off of wall power because the included batteries were duds, but the system hung and glitched out - most definitely from the shaky wall voltage. Weeks later a fresh set of batteries arrived with an antenna and I figured it best to grab an up-to-date system image from the GitLab. Everything goes smoothly until It’s time to connect to Wi-Fi. I tried both the official image and the Debian community image, both GNOME and sway, and with, then without, then with the antenna again, even reversing which antenna I plugged into what hole.
Problem: My issue is that I can see a complete list of the networks I expect, but when attempting to connect to mine the Wi-Fi icon spins for a bit before prompting me for my password again, then spins some more, and then eventually the system tells me it failed to connect. I have tried this on two separate Wi-Fi networks, multiple times, while also double-checking my password spelling before hitting enter pretty much every time.
How can I fix this? Any ideas as to why it would work before but not after?
(I wanted to see if anything was different on OpenBSD as well but it seemed to hang before getting to an installer - a separate issue altogether, though I think it’s because I’m on an older version of u-boot, and I can’t reform-flash-uboot because I can’t connect to the internet (no spare Ethernet cables either unfortunately))
What is your signal strength?
Can you show us a photo of how you connected your antennas?
Did you try writing your wifi passphrase elsewhere and then copypaste it into the wifi passphrase prompt? (to make sure that you have no typos)
Did you try removing the password from your wifi to see if it works then?
Can you take your Reform elsewhere and try if you can connect to a network different from yours?
Signal strength shows as good, positive there’s no typos, I use T-Mobile Home Internet and there doesn’t appear to be a way to disable the password from the app they force you to use. I’ve tried it elsewhere at a friend’s house and still no dice.
Here’s the photo, I reinserted the card itself and made sure both antennas clicked in, still nothing:
I would like to point out that I am not an expert with wifi on linux. I have not many ideas left of where to look for clues, maybe others have an idea?
Maybe it helps to post the last lines of these commands after you fail to connect to your wifi:
sudo dmesg -T
sudo journalctl -xeu NetworkManager.service
Maybe they contain helpful error messages related to your case?
I have seen similar behavior (wifi appears fine until you try to actually join, and it spins…) in other situations (not reform) and the fix both times was to turn off the power saving feature of the wifi driver.
No idea if this will help, but you can try creating a file /etc/NetworkManager/conf.d/wifi-powersave-off.conf
As you have a DIY kit from CS you’re on an i.MX8MQ processor (right?), and I recall that this symptom can happen if PCI MSI is not disabled in the kernel command line. @josch is that still disabled in the uboot/boot args? (pci=nomsi)
@geo could you send us the output of running sudo reform-check which will answer a bunch of questions in one go, like: are you using imx8mq? what kernel and reform-tools version are you on? do you have pci=nomsi in your cmdline?
Indeed the reform-check output tells you that for some reason your system is not using pci=nomsi. Maybe your system is using an ancient u-boot version? Do you boot it with u-boot from eMMC or from sd-card?
You can use the first two images in this article to locate the DIP switch on your imx8mq: document boot options (#2) · Issues · Reform / reform-handbook · GitLab If it’s set to OFF, then your imx8mq will load u-boot from eMMC which might be ancient. You said that you loaded an up-to-date system image on an sd-card. If you flip the switch to ON, then it will use the latest u-boot from sd-card instead. And that should include options like pci=nomsi.
If you don’t want to flip the switch, then you can do the following to insert custom boot options. Create a file called /etc/flash-kernel/ubootenv.d/01pcinomsi and fill it with the following line:
If you don’t want to type much, you can also start with just listing pci=nomsi for a start and see if that worked. You enable the change you did by regenerating /boot/boot.scr running the following command:
sudo flash-kernel
Theoretically, the default boot.scr should guard against this. There is an if test "${board}" block I put above to make sure that what you experience does not happen. I would find it very interesting to get a hold of the u-boot version flashed to your eMMC and prevent the next person from running into this issue. If you like, after you have your system working with wifi, send me boot0.img created by the following command:
sudo dd if=/dev/mmcblk0boot0 of=boot0.img
Then there is another funny thing in your output. It says that the /boot/dtb-6.15.4-mnt-reform-arm64 symlink points to the dtb of imx8mp instead of imx8mq. Did you change that manually? Or could something be broken with the imx8mq system image? Which one exactly did you download?
I imagined u-boot was the culprit, the date next to the MNT logo on startup was sometime in 2023.
The module discrepancy was my bad, I moved back from the Community Image to the Official the night I asked this question and misread the filename.
No luck with editing the files or flipping the switch, when I flipped it the system wouldn’t boot at all, just a black screen. The fix ended up being a simple
reform-flash-uboot --offline
If only I tried it before assuming it only worked with internet! (might be worth adding a note into the handbook?) Thanks for the pointers everyone, sorry for wasting you all’s time. I’ve sent the file you requested to your email @josch
You absolutely did not waste my time! Thank you for sticking around to try and get to the bottom of it and thank you so much for sending me the contents of your boot0 partition!
I investigated it and indeed your reform came with the following u-boot version: U-Boot SPL 2018.07 MNT Reform 2023-01-25 (Feb 08 2023 - 20:00:28 +0100) That is rather old-ish (current is 2024-07-19). One big issue of 2023-01-25 is, that indeed it does not have pci=nomsi in its bootargs.
When you say “Community Image”, do you mean the ones I offer on reform.debian.net? If yes, do you think it would’ve helped you in finding what the problem is if the download page would’ve said that you need to have recent u-boot installed for this image to work? I fear it might not have because how should the user know which version is “recent-enough”? I am surprised the image booted at all with your version of u-boot!
When debugging problems it would be nice to learn what happens on the official images. The images I offer on reform.debian.net are not the official ones and I try to be explicit about it with a big yellow admonition with an exclamation mark at the top of the front page.
I’m not trying to blame you. I’m trying to find out what I can or should change such that the next person does not run into the same issues that you had to go through. Do you have an idea what you think might’ve helped?
@minute I am aware that unfortunately reform.debian.net is the first hit on many search engines for “reform debian system image” I tried to make it clear that the images are not the official ones on the front page and direct them to the official images but maybe I should make this warning more prominent? What do you think? I’m also thinking about @selfawaresoup’s HDMI issue which only happened with the Debian Trixie kernel and went away with the official MNT images. I can of course only provide the content on reform.debian.net on a best-effort basis and I only have my free-time to work on it and it’s not helpful if these images/packages actually make the experience with the Reform worse…
I don’t think this is really true. Overall, I am way happier with my Reform thanks to your Trixie images. My system has been way more reliable since.
My HDMI issue is unfortunate but can hopefully be resolved now that we know it’s not a hardware fault.
When you say “Community Image”, do you mean the ones I offer on reform.debian.net?
Yep.
Do you think it would’ve helped you in finding what the problem is if the download page would’ve said that you need to have recent u-boot installed for this image to work?
No, as the confusion stemmed from the problem only manifesting after I flashed the newest Official Image to my SD card, leading me to believe that it was an OS-level problem, not a u-boot problem. The Community Image booted exactly like the Official one did, but with the same Wi-Fi issue.
When debugging problems it would be nice to learn what happens on the official images. The images I offer on reform.debian.net are not the official ones and I try to be explicit about it with a big yellow admonition with an exclamation mark at the top of the front page.
I’m well aware, just to clear up any confusion about what image was used when and how, here’s a complete chronicle of the issue:
Reform arrives with MNT Official Image from way back in 2023, the batteries are drained beyond recharge. I power up the system using wall power and connect to Wi-Fi perfectly fine, even without the antenna. I start an apt update && upgrade and, being that the system image is over two years old, am stuck waiting awhile. The wall power then must have dipped momentarily mid-update, causing a system hang, with graphical glitches everywhere I try to click. So I unplug the system. When I attempt to power it back on I hear an electric sputtering/buzzing noise and turn it back off again quickly (I’d be interested if @minute knew what caused this and if it’s a concern, as it happened again when the system was running low on battery power recently. The system seemed to be booting normally both times.) I’m discouraged from trying to run the system off the wall again, and order new batteries.
My replacement batteries and antenna arrive and, assuming that the botched update will be more trouble than it’s worth to sort out, I opt to not even bother trying to boot the old image and grab a fresh MNT Official Image off of the GitLab. It boots fine, but my Wi-Fi trouble begins. I search for basic troubleshooting ideas and see a Linux Mint Forum thread that mentions switching NetworkManager out for iwd, though, installing software is rendered difficult without internet (I briefly considered trying to load iwd onto a USB drive using another computer, but I realized that would probably be a pain and opted to try more straightforward options first, before forgetting the idea entirely in the process.) I reinstall the image and select sway instead of GNOME, wondering if it uses a different Wi-Fi backend. Nothing.
I double check the Wi-Fi card to no avail, then begin trying other OSes, starting with the Debian Community Image. I figure “If the Wi-Fi worked before the new image, maybe a weird package is breaking something, so this should make it work.” Unfortunately, that was not the case. I repeat the process of trying GNOME then sway, and move on to the next OS.
I flash OpenBSD to the SD card. “Surely a totally different OS will fix this!” I am then met with a system that will not even boot. I find the only OpenBSD-related thread posted to this forum from way back in 2021 and read through. I learn that older versions of u-boot can’t boot OpenBSD. “Darn, I should update this u-boot thing once I get my Wi-Fi working!” I think to myself, oblivious.
I’m left with two non-UNIX derived options - 9front and SculptOS. I had a vague notion of 9front prior to buying my Reform from meandering through a webring and landing on cat-v.org awhile back. I’m led down a rabbit hole (…) of the history of Plan 9. It is interesting, but once I was spat out into rio after the boot process I realized I’m in over my head and should just back up and ask for help at this point. (Also, I wasn’t keen on going any further with 9front because I was bummed I couldn’t use acme effectively without a trackball module.)
As a last ditch half-effort I try the Debian Community Image one more time, to no avail.
I hastily flash the (wrong) Official Image back to the SD card before starting this thread, being that it is the most studied and bespoke OS for the Reform.
I flash the proper Official Image, Wi-Fi is fixed.
The Community Image did not hinder the coming of a solution, if anything, it helped to rule out the problem being OS-level :^)
I’m trying to find out what I can or should change such that the next person does not run into the same issues that you had to go through. Do you have an idea what you think might’ve helped?
I don’t believe it should fall on OS maintainers to mention this bug in older versions of u-boot. The first thing I did when I learned it could be u-boot was to look “u-boot” and “uboot” up in the handbook. The linked result is how I learned about reform-flash-uboot. Being that the explanation for this tool is only This updates the bootloader (rarely needed), it could be beneficial to add a note in the next handbook revision (if one is planned) to the effect of *A known Wi-Fi issue on i.MX8MQ modules running older versions of u-boot can be fixed by connecting to the internet via Ethernet and running this command OR flashing an up-to-date system image to an SD card and running this command with the '--offline' option. (Although this just explains the fix, not the mechanism behind the issue, I don’t know why neither changing the bootargs nor flipping the switch failed to fix anything…)
Anyways, this was a long reply. Let me know if it answered your questions effectively, and thanks again for your help.
Yes, investigating in that direction makes sense, I think. I mean who suspects a missing kernel parameter at boot to be responsible for this!
Wow, you went all out! But yes, I think you strategy to try a vanilla system to make sure that it’s not somehow your weird config is a good idea.
Wow!
I’m also at a loss here.
Thanks a lot for your detailed write-up. And thank you for not giving up and sticking around. I hope this was not too frustrating for you! I’m glad we got to the bottom of this and that your system is running fine now!
Please feel encouraged to post issues you have to this forum also, if you like, before having gone done all sorts of rabbit holes. That’s what this forum is for.
