I know the MNT Pocket reform has an M.2 slot.
How feasible would this be to add an AI accelerator chip like M5 stacks to it:
Or perhaps the Hailo-8 M.2 AI Accelerator Module: https://www.newegg.com/p/3C6-005Y-01AC5
2 Likes
The Hailo looks more compatible as it doesn’t have a giant heatsink. The RK3588 also has a 6 TOPS NPU already, have you looked into it? I haven’t tested any of these as I’m quite upset about people using LLM coding agents causing much trouble for free and open source software in general, and the cloud AI datacenter hype ruining general purpose computing prices/supplies for many people, not to speak about the environmental impact and the web being filled with garbage generated content. That said, perhaps you have a very different application in mind, and you have of course the freedom to run any software/hardware you want in your MNT device.
8 Likes
There is no way these accelerators could run a LLM with any reasonable power. But what they can do is accelerate models like BERT and kNN algorithms. Vector word embeddings are really cool and I think you should check them out. My personal goal is to have the equivalent of Google circa 2010 (with all of wikipedia, wikidata, etc) on a single device.
I promise: no chat bots 
6 Likes