Continuing the discussion from Confirmed Working NVME Drives to avoid going further off topic.
I did a new run of gnome disks and had
htop running to see if any of the cores got pegged. But none of the cores went over 25% usage, so doesn’t look like the CPU would be the bottleneck.
I could push the XPG sx8200 pro somewhat close to its max performance in my other machine (~90% read and ~80% write) when using gnome disks. So I don’t think it’s an issue with the program.
However my tests does not rule out the three suggestions you mentioned, i.e. the controller, a limitation of the SoC (at least my test gives a hint that it isn’t the CPU) or the kernel.
I’m not entirely sure how to test this though.
I did find a video by Jeff Geerling that shows it is possible to get around 395MB/s read and 361MB/s write on a Raspberry Pi CM4 with a TOFU board. Which, if I’m hearing it correctly, also uses a PCIe 2.0 interface. Not sure if that information is useful in our case but at least it shows you can get higher speed on a PCIe 2.0 interface.