U-Boot and internal screen?

[johkra]

A recent commit seems to indicate that HDMI output from U-Boot is now supported.

Is it feasible to get output on the internal screen? What would be required?

---

I was working on getting ArchLinux ARM running (Spoiler: Unless you have a kernel with I suspect at least the PCIe clock patch, USB and Wifi won’t work; the internal panel doesn’t work on ALARM 12.3 either.) and the U-Boot experience isn’t great.

The serial connector is on the bottom, but the only way to turn on the SOM I found is by using the keyboard (Circle, 1) on the other side. Restarting the SOM is possible with a button on the back.

Saving the U-Boot environment doesn’t seem to work (I saw comments around MMC/SD for env storage and crashes when trying to save on SD), so I had to copy commands during testing and then recompile U-Boot and add it to the SD to hardcode working commands.

I’m not sure on the preferred way to handle testing (typing commands on the internal keyboard without copy/paste would be tedious and error prone as well), but for general boot I’m considering changing U-Boot to read a script file from SD/MMC, writing U-Boot to MMC and changing the boot order on the SOM to set boot from MMC as default as hinted at in Advanced Chapter of the handbook.

[minute]

I would love to get this to work, I simply didn’t have the time/capacity to do it. Enabling the HDMI solves half of this, basically, because this gets the DCSS engine up and running that can feed the MIPI-DSI output as well (if one mux register bit is flipped). I think two IP blocks and one external chip need to be poked, plus the PWM output for the backlight:

1. nwl-dsi (internal to i.MX8MQ). Driver: nwl-dsi.c - drivers/gpu/drm/bridge/nwl-dsi.c - Linux source code (v5.13-rc6) - Bootlin
2. mixel MIPI DPHY (internal to i.MX8MQ). Driver: phy-fsl-imx8-mipi-dphy.c - drivers/phy/freescale/phy-fsl-imx8-mipi-dphy.c - Linux source code (v5.13-rc6) - Bootlin (pretty small)
3. TI sn56dsi86 MIPI->eDP bridge (external, poked via I2C). Driver: ti-sn65dsi86.c - drivers/gpu/drm/bridge/ti-sn65dsi86.c - Linux source code (v5.13-rc6) - Bootlin

The bridge chip also has a “color bars” test register to get some output on the display without a valid input. This could be a good first step.

[johkra]

Thanks for the pointers, this is helpful!

I poked around a bit using the helpful i2c U-Boot command and also tried to use the gpio command for BACKLIGHT_EN (GPIO1_IO10 per schematic). I haven’t been able to change its value so far. I could change the pin type to output by modifying the code and recompiling U-Boot, but actually changing the value using the gpio command didn’t work. I think I need to look at the device tree inside U-Boot instead of the code.

I²C poking worked fine, but I need to properly follow the power up sequence to make it show anything. For this, I need to figure out the right parameters. They should all be in the Linux device tree, but I hit my limit for looking at weird numbers for today.

In case someone else wants to poke around on the U-Boot command line:

=> i2c bus
Bus 0:  i2c@30a20000
Bus 1:  i2c@30a30000
Bus 2:  i2c@30a40000
Bus 3:  i2c@30a50000

# Device tree shows sn65dsi86@2c attached to i2c@30a50000, i.e. bus 3
=> i2c dev 3
Setting bus to 3
# Device tree shows address 0x2c, let's confirm:
=> i2c probe
Valid chip addresses: 2C

=> i2c md 0x2c 0x00 8
0000: 36 38 49 53 44 20 20 20    68ISD
# (Reversed) magic string from data sheet checks out
=> i2c md 0x2c 0x08 1
0008: 02    .
# So does the version

# Set COLOR_BAR_EN, this doesn't show anything yet as output requires following power up sequence.
=> i2c mw 0x2c 0x3c 0x10
=> i2c md 0x2c 0x3c 1
003c: 10   

[johkra]

To follow up:

I’m still trying to get this to work, but my lack of knowledge on how U-Boot or Device Trees work give me little to show for so far.

First: Boundary has a newer version of their U-Boot fork at GitHub - boundarydevices/u-boot-imx6 at boundary-v2020.10.

That one fixes the conflict between video output (legacy video driver) and using the device tree driver model (needed for pwm-backlight) from the older version and includes a driver for the SN65DSI83 which might make creating a driver for the SN65DSI86 easier.

I’m still trying to get the backlight to work (dm tree shows the backlight entry I created, but I don’t know how to force it to be probed/enabled) and hope to make progress their over the next days.

[johkra]

The U-Boot documentation in the doc directory in the source tree was helpful in understanding things better.

I can control the backlight now. I also discovered that reading the GPIO state doesn’t seem to work (dm_gpio_get_value just after dm_gpio_set_value returns 0). I can use the U-Boot gpio command to enable/disable the backlight even though it will always report the pin as off.

My current patch for the boundary U-Boot fork (the define for GP_SN65DSI86_ENABLE doesn’t do anything):

$ git --no-pager diff
diff --git a/arch/arm/dts/imx8mq-nitrogen8m_som.dts b/arch/arm/dts/imx8mq-nitrogen8m_som.dts
index 103a1037..7fec04f4 100644
--- a/arch/arm/dts/imx8mq-nitrogen8m_som.dts
+++ b/arch/arm/dts/imx8mq-nitrogen8m_som.dts
@@ -64,10 +64,24 @@
 		>;
 	};

+        pinctrl_backlight: backlightgrp {
+                fsl,pins = <
+#define GP_BACKLIGHT	<&gpio1 10 GPIO_ACTIVE_HIGH>
+                        MX8MQ_IOMUXC_GPIO1_IO10_GPIO1_IO10         0x19
+                >;
+        };
+
+	pinctrl_sn65dsi86_en: sn65dsi86_engrp {
+		fsl,pins = <
+#define GP_SN65DSI86_ENABLE	<&gpio3 20 GPIO_ACTIVE_HIGH>
+			MX8MQ_IOMUXC_SAI5_RXC_GPIO3_IO20		0x16
+		>;
+	};
+
 	pinctrl_hog: hoggrp {
 		fsl,pins = <
 			/* J1 connector, odd */
-			MX8MQ_IOMUXC_GPIO1_IO10_GPIO1_IO10		0x19	/* Pin 105 */
+			/*MX8MQ_IOMUXC_GPIO1_IO10_GPIO1_IO10		0x19*/	/* Pin 105 */
 			MX8MQ_IOMUXC_GPIO1_IO05_GPIO1_IO5		0x19	/* Pin 143 */
 			MX8MQ_IOMUXC_GPIO1_IO03_GPIO1_IO3		0x19	/* Pin 145 */
 			MX8MQ_IOMUXC_NAND_CE3_B_GPIO3_IO4		0x19	/* Pin 149 */
@@ -584,6 +598,14 @@
 		status = "disabled";
 	};

+	backlight: backlight {
+		compatible = "pwm-backlight";
+		pwms = <&pwm2 0 10000>;
+		enable-gpios = GP_BACKLIGHT;
+		brightness-levels = <0 32 64 128 160 200 255>;
+		default-brightness-level = <6>;
+	};
+
 	bt-rfkill {
 		compatible = "net,rfkill-gpio";
 		pinctrl-names = "default";
diff --git a/board/boundary/nitrogen8m_som/nitrogen8m_som.c b/board/boundary/nitrogen8m_som/nitrogen8m_som.c
index 34ae32c6..fd73d78a 100644
--- a/board/boundary/nitrogen8m_som/nitrogen8m_som.c
+++ b/board/boundary/nitrogen8m_som/nitrogen8m_som.c
@@ -23,6 +23,7 @@
 #include <asm/arch/clock.h>
 #include <asm/mach-imx/video.h>
 #include <linux/delay.h>
+#include <backlight.h>
 #include <video_fb.h>
 #include <spl.h>
 #include <power/pmic.h>
@@ -174,6 +175,8 @@ static const struct display_info_t displays[] = {

 int board_init(void)
 {
+	struct udevice *backlight;
+	struct gpio_desc desc;
 #ifndef CONFIG_DM_VIDEO
 	gpio_request(GP_I2C1D_SN65DSI83_EN, "sn65dsi83_enable");
 	gpio_request(GP_LTK08_MIPI_EN, "lkt08_mipi_en");
@@ -189,6 +192,69 @@ int board_init(void)
 #endif
 	board_usb_reset(0, USB_INIT_DEVICE);

+	if (uclass_get_device_by_name(UCLASS_PANEL_BACKLIGHT, "backlight", &backlight) == 0) {
+		backlight_enable(backlight);
+	}
+	if (dm_gpio_lookup_name("GPIO3_20", &desc) == 0) {
+		if (dm_gpio_request(&desc, "sn65dsi86_enable") == 0) {
+			dm_gpio_set_dir_flags(&desc, GPIOD_IS_OUT);
+			dm_gpio_set_value(&desc, 1);
+		}
+	}
 	return 0;
 }

Based on my reading of the data sheet, the following steps are required to get Color bars to work:

1. Set Enable pin high.
2. Configure DSI Clock (DACP/N) frequency.
   Needed for DP PLL.
   Need to enable DSI clock on IMX?
3. Enable relevant interrupt sources.
   Probably not needed for testing.
4. Disable Hot Plug Detection (HPD)
   TODO: Read register 0x5c bit 4 to check if monitor is detected as plugged in, then this is not needed.
5. Read EDID using I2C_ADDR_CLAIMx registers if clock stretching is supported.
6. Read eDP Panel DisplayPort Configuration Data (DPCD).
7. Program appropriate number of data lanes and data rate.
   There are two physical lanes for eDP.
8. Enable DisplayPort PLL and confirm PLL is locked.
9. Train DP link.
   Use Semi-Auto link training?
10. Set Video Registers.
    Enable Color bar.
11. Enable Video Stream.

Checking HPD should be easy. I’ll see if I can read EDID/DPCD information without DSI clock / from within Linux, then I have to see how to enable the DSI clock.

[minute]

Cool! You can skip the interrupts, and the hotplug business because the display is always connected, and you can also skip EDID and just use the timings:

https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg2393231.html

Points 7-11 are correct AFAIK.

[johkra]

After some hackery to get clock input running, I managed to talk to the display and read EDID:

=> i2c dev 3
Setting bus to 3
=> i2c probe
Valid chip addresses: 2C
# Enable I2C passthrough for address 0x50, i.e. EDID
=> i2c mw 0x2c 0x60 0xa1
=> i2c probe
Valid chip addresses: 2C 50
=> i2c md 0x50 0x00 0x80
0000: 00 ff ff ff ff ff ff 00 0d ae 39 12 00 00 00 00    ..........9.....
0010: 06 19 01 04 a5 1c 10 78 02 5d 45 96 58 52 98 27    .......x.]E.XR.'
0020: 26 50 54 00 00 00 01 01 01 01 01 01 01 01 01 01    &PT.............
0030: 01 01 01 01 01 01 36 36 80 a0 70 38 20 40 2e 1e    ......66..p8 @..
0040: 24 00 14 9b 10 00 00 1a 00 00 00 fe 00 4e 31 32    $............N12
0050: 35 48 43 45 2d 47 4e 31 0a 20 00 00 00 fe 00 43    5HCE-GN1. .....C
0060: 4d 4e 0a 20 20 20 20 20 20 20 20 20 00 00 00 fe    MN.         ....
0070: 00 4e 31 32 35 48 43 45 2d 47 4e 31 0a 20 00 91    .N125HCE-GN1. ..

I2C passthrough didn’t work previously and I’m not quite sure what change exactly made it work. I spent far too much time this evening dumping DPHY related registers while consulting the reference manual for their meaning only to try I2C passthrough again and suddenly having it work. This makes it an excellent stopping point for today before I try to get eDP working.

The clock input hackery should work out as setting mipi_dsi and dphy as status=“okay” in the device tree, setting dsi->lanes = max_data_lanes in nwl_init and something like the following in board_init in board/boundary/nitrogen8m_som/nitrogen8m_som.c:

        int ret;

        struct udevice *nwl_dsi;

        struct mipi_dsi_device device;
        struct display_timing timings;

        if (uclass_get_device(UCLASS_DSI_HOST, 0, &nwl_dsi) == 0) {
                device = (struct mipi_dsi_device) {
                        .lanes = 4,
                        .format = MIPI_DSI_FMT_RGB888,
                        .mode_flags = MIPI_DSI_MODE_VIDEO,
                        .hsmult = 1,
                };
                timings = (struct display_timing) {
                        .pixelclock = { 162000000, 162000000, 162000000 },
                        .hactive = { 1920, 1920, 1920 },
                        .hfront_porch = { 40, 40, 40 },
                        .hback_porch = { 40, 40, 40 },
                        .hsync_len = { 80, 80, 80 },
                        .vactive = { 1080, 1080, 1080 },
                        .vfront_porch = { 4, 4, 4 },
                        .vback_porch = { 4, 4, 4 },
                        .vsync_len = { 24, 24, 24},
                };
                dsi_host_init(nwl_dsi, &device, &timings, 4 /* lanes */, NU
LL /* mipi_dsi_phy_ops */);
                ret = dsi_host_enable(nwl_dsi);
                if (ret) {
                        printf("### Failed to enable host: %d", ret);
                }
        }

[minute]

Very good progress! I didn’t realize nwl-dsi was in (vendor?) u-boot, or did you port it from Linux?

[johkra]

It looks like nwl-dsi was added to vendor (boundary) U-Boot in March: video: imx: nwl-drv, nwl-dsi: initial addition · boundarydevices/u-boot-imx6@74ce775 · GitHub

I used a web based tool to decode the EDID. The interesting part:

Detailed Timing Descriptor
Pixel Clock: 138.78MHz
Horizontal Active: 1920
Horizontal Blanking: 160
Vertical Active: 1080
Vertical Blanking: 32
Horizontal Sync Offset: 46
Horizontal Sync Pulse: 30
Vertical Sync Offset: 2
Vertical Sync Pulse: 4
Horizontal Display Size: 276
Vertical Display Size: 155
Horizontal Border: 0
Vertical Border: 0
Interlaced: false
Stereo Mode: 0
Sync Type: 3

Let’s also read DPCD:

=> i2c dev 3
Setting bus to 3
# Set read length to 16 bytes (maximum length)
=> i2c mw 0x2c 0x77 0x10
# Start read at default address
=> i2c mw 0x2c 0x78 0x91
=> i2c md 0x2c 0x79 0x10
0079: 11 0a 82 41 00 00 01 80 02 00 00 00 0f 0b 00 00    ...A............

Decoding via table 2-52 of DP-1.1 spec:

0x00 0x11 Rev 1.1
0x01 0x0a Max link rate 0x0a = 2.7 Gbps per lane
0x02 0x82 Enhanced frame cap supported, 2 lanes
0x03 0x41 0.5% down spread, Does not require AUX CH handshake
0x04 0x00 Zero receiver ports ?
0x05 0x00 No downstream ports
0x06 0x01 Supports 8b/10b coding
0x07 0x80 OUI supported
0x08 0x02 Local EDID present
0x09 0x00 32 bytes per lane buffer size

# Set read start address to 0x100 and read 16 bytes
=> i2c mw 0x2c 0x75 0x01
=> i2c mw 0x2c 0x78 0x91
=> i2c md 0x2c 0x79 0x10
0079: 0a 82 00 00 00 00 00 00 00 08 00 00 00 00 00 00    ................

Decoding (only interesting fields):

0x100 0x0a 2.7 Gbps per lane for main link
0x101 0x82 Enhanced frame symbol sequences enabled, 2 lanes

# Set read start address to 0x200 and read 16 bytes
=> i2c mw 0x2c 0x75 0x02
=> i2c mw 0x2c 0x78 0x91
=> i2c md 0x2c 0x79 0x10
0079: 01 00 33 00 80 00 00 00 00 00 00 00 00 00 00 00    ..3.............

Decoding:

0x200 0x01 One sink device
0x202 0x33 Lane0-1 CR, Channel EQ done
0x204 0x80 Link status updated since last read
0x205 0x00 Links not synchronised

There are a lot more values, mostly related to the intricacies of training.

[johkra]

I got colour bars. The stripes patterns don’t work, but the other patterns do.

# Set DSI clock to 486 MHz
=> i2c mw 0x2c 0x0a 0x06
# Single Channel, 4 DSI lanes
=> i2c mw 0x2c 0x10 0x26
# Enhanced framing and ASSR
=> i2c mw 0x2c 0x5a 0x05
# 2 DP lanes w/o SSC
=> i2c mw 0x2c 0x93 0x20
# 2.7 Gbps DP data rate
=> i2c mw 0x2c 0x94 0x80
# Enable PLL and confirm PLL is locked
=> i2c mw 0x2c 0x0d 0x01
=> i2c md 0x2c 0x0a 1
000a: 87    .

# Enable ASSR on display
=> i2c mw 0x2c 0x64 0x01
=> i2c mw 0x2c 0x75 0x01
=> i2c mw 0x2c 0x76 0x0a
=> i2c mw 0x2c 0x77 0x01
=> i2c mw 0x2c 0x78 0x81
# Train link and confirm link is trained
=> i2c mw 0x2c 0x96 0x0a
=> i2c md 0x2c 0x96
0096: 01    .

The following timing values are likely wrong, I need to correct them using the calculations in section 9.2.1.2.2 in the data sheet and the EDID values above.

# Line length 1920
=> i2c mw 0x2c 0x20 0x80
=> i2c mw 0x2c 0x21 0x07
# Vertical display size 1080
=> i2c mw 0x2c 0x24 0x38
=> i2c mw 0x2c 0x25 0x04
# HSync pulse width 40
=> i2c mw 0x2c 0x2c 0x28
# VSync pulse width 4
=> i2c mw 0x2c 0x30 0x04
# Horizonal back porch 40
=> i2c mw 0x2c 0x35 0x28
# Vertical back porch 4
=> i2c mw 0x2c 0x36 0x04
# Horizonal front porch 40
=> i2c mw 0x2c 0x38 0x28
# Vertical front porch 4
=> i2c mw 0x2c 0x3a 0x04

# Enable color bar
=> i2c mw 0x2c 0x3c 0x10
# Enable video stream, ASSR, enhanced framing
=> i2c mw 0x2c 0x5a 0x0d

Section 9.2.1.2.5 Example Script in the data sheet was extremely helpful in getting to this milestone, the above is an adaption of it. (After my own attempts didn’t get me a picture…)

[minute]

Great work! Not far from the goal now.

[johkra]

After some struggles with corrupted display output due to off-by-one errors (and being without access to my Reform for a few weeks), I now have U-Boot booting into colour bars with EDID provided timing values.

The screen visibly flickers when showing Gray Scale patterns, but is fine when showing colour patterns. The N125HCE-GN1 data sheet version 3.0 mentions that the PWM control frequency should be between 190 and 2000 Hz. I currently have the following in the device tree: pwms = <&pwm2 0 10000>;. I suspect the resulting frequency is not in the recommended range.

The data sheet has the following suggested formula for PWM control frequency:

PWM control frequency f PWM should be in the range
( N + 0.33) * f <= fPWM <= ( N + 0.66) * f
N : Integer ( N >= 3)
f : Frame rate

They also mention water fall effects may be seen below 1000 Hz, so e.g. 1950 Hz (N=32 for f=60) might be worth a try.

Actually getting video output is likely easier by writing a driver compared to configuring everything by hand. I had started doing so before getting distracted by the corrupted output.

With the code from boundary the bridge has to present itself as a panel. Linux permits chaining another bridge and implemented the driver as such, so the device tree can’t be shared with Linux. That sounds OK to me.

[minute]

Are you sure the display is flickering because of the backlight, or is it because of the refresh rate of the display mode?

Also, you should compare notes with bluerise who already has all the driver infrastructure in place, but unfortunately he got only color bars or black screen, no framebuffer to be seen yet: Commits · bluerise/u-boot · GitHub

[johkra]

Oh, exciting! I wasn’t aware of bluerise’s work. If I get things to work earlier than bluerise, I’ll contribute to the repo, otherwise I’m looking forward to see their progress.

Thinking about this, I’m certain it’s not the backlight…because it was configured to run at 100% duty cycle. When it’s always on, the frequency shouldn’t matter.

Dumping parameters in pwm-imx.c, the following parameters are used when setting the backlight to 50% (for 100% duty_cycles is 249):

imx_pwm_set_config: Config parameters: per_rate: 25000000, period_ns: 10000, duty_ns: 5000, period_cycles: 248, duty_cycles: 125, prescale: 1

While the parameter in the device tree is documented as the period in nanoseconds (with 10’000 ns that’s 10µs, i.e. 100’000 Hz), it seems it’s actually using 250 Hz (the code subtracts 2 from the period_c value). Edit: 250 cycles with a clock rate of 25000000 (per_rate) is 100kHz.

I’m pretty confident I’m using the 1920x1080@60 mode from EDID. But I also confess to not fully understanding how this actually works and what parameters would be different for e.g. a 30 Hz mode (longer vsync?).

I also don’t perceive any flickering today. I don’t think I changed anything since yesterday. Ah well, I’m always happy to not have a problem.

[minute]

From my experience, for reasons I don’t fully (technically) understand, the displays can retain a flicker from a period of faulty/out of spec input for several minutes, even if the signal is OK again.