NVMe - bottleneck investigation

MNT Reform
1

Continuing the discussion from Confirmed Working NVME Drives to avoid going further off topic.

I did a new run of gnome disks and had htop running to see if any of the cores got pegged. But none of the cores went over 25% usage, so doesn’t look like the CPU would be the bottleneck.

I could push the XPG sx8200 pro somewhat close to its max performance in my other machine (~90% read and ~80% write) when using gnome disks. So I don’t think it’s an issue with the program.

However my tests does not rule out the three suggestions you mentioned, i.e. the controller, a limitation of the SoC (at least my test gives a hint that it isn’t the CPU) or the kernel.

I’m not entirely sure how to test this though.

I did find a video by Jeff Geerling that shows it is possible to get around 395MB/s read and 361MB/s write on a Raspberry Pi CM4 with a TOFU board. Which, if I’m hearing it correctly, also uses a PCIe 2.0 interface. Not sure if that information is useful in our case but at least it shows you can get higher speed on a PCIe 2.0 interface.

2

Yeah the Pi CM4 also has a PCIe 2.0 1x link, which has a theoretical max bandwidth of 500MB/s. It is an A72 vs A53, but if the CPU isn’t being pegged I have to wonder where it lies. I do remember some of Jeff Geerling’s on trying to get the most Ethernet performance over it was running into interrupt issues, where the CPU was basically spending all its time handling interrupts for the packets. I’ll have to re-watch that to check how he realized that and see if that’s applicable.