Hi, did anyone tried Tokenspeed? It’s more focused on agentic workflows but it’s very new.
I’ll try it soon on Jetson Thor, but I’m knee deep on other works right now. I’ve seen a post on linkedin of someone using it on GB10.
They provide a Docker image which apparently is not reproducible based on the GitHub repo. The repo’s Dockerfile is based on their image.
Then you customize and apparently finish the build inside that image, before starting a server.
I can’t figure out if they have an ARM64 version or not, but if not, we’d be out of luck. And even if they did, I really dislike this method (it’s not open if we cannot reproduce the Docker image).