Hi there, i’ve been trying very hard to install various LLMs with openclaw. It has been a total nightmare total chaos total complete ultra utter chaos hahahaha.
So its been 15 days of “trying to conceive” something from this complete mashup potato salad GB10 LLMs fantasy. I bought this machine to have local LLMs running my company. The orchestration, the openclaw and paperlcip stuff its not that hard. but trying to figure out witch Paperclip over OPENCLAW over vLLM over Dokers, over LLM flavors over Linux and make this work together in this moment is kind of almost not doable. Its plagued with interruptions and strange behavior. Its really the vLLM and the LLM flavour with all this paramenters to try find some goofy stability.
I’ve tried all this :
Qwen 3.5 35B A3B NVFP8 / FP8 Runs fast (max of 97tok/s), but its crowded with strage errors like cache overflow, error loading model, etc. I run it on AVAROK, and EUGR vLLM dokers, but its all bad, no stable options.
Qwen 3.5 80B A12B FP8/NVFP4 Run on EUGR vLLM Doker. NOTHING GOOD, dint manage to run it properlly.
Qwen 3.5 27B FP8 . Never manager to get it actually good performance 7Tok/s max.
Qwen 3.5 122B … never mind i didnt work.
The list can go on, but…
I need advise or a PROPER LINK to download a secure and stable option so i can make this setup work properly. Im no engineer, nor programmer, im just a regular guy trying to make this setup work. I have downloaded everything up to date, but without Ai help which is really not that good, i must say i need some really useful guidance!!! please community help me to make this work!!!
The most stable tool available for inference, recipes for the best stable models, and a well-organized readme file. I’m not a programmer either. Good luck. You also have this https://sparkrun.dev very complete one, it has many more features, very good documentation, and we have https://spark-arena.com for gladiator models.
I didn’t touch OpenClaw yet and I probably will never touch that security nightmare, but:
NVIDIA made a guide of it. Avoid Ollama[1]. LM Studio is a good tool for beginners or people that aren’t that deep into Linux / bash / cli tasks.
If you already had your hands on eugr’s community build and that didn’t shock you then that is the way to go.
Qwen 3.5 80B A12B - there is no 80B version of Qwen3.5 - but Qwen3 Next and Qwen3 Coder Next in 80B. They run with other tools quite well (VS Code CoPilot, Claude Code, opencode for example).
Qwen3.5 122B works quite good on a single spark using the Intel/Qwen3.5-122B-A10B-int4-AutoRound quant.
NVFP4 is still not “ready”, but the community did make great steps to improve performance. I recommend AWQ quants (if the models don’t fit into 128 GB) or int4-AutoRound quants. If the model fits in FP8 - you also could use FP8 quants. The speed doesn’t need to hide behind the 4 bit quants (AWQ, NVFP4, int4-AutRound).
If the Qwens do not work for you, you could try NVIDIAs Nemotrons. In the playbook they refer to Nemotron-3-Nano-30B-A3B - it is optimized for agentic workloads.
If you have problems to run a specific model with vLLM (pure/eugr/whatever flavor) you should report the errors and/or problems you run into when trying to launch.
I’m sure that the community in here will help you to get running.
But be warned! Even if some marketing people tried to tell “its super easy to do supercomputing” with a Spark… it’s a developer box. Still bleeding edge at some spots, but quite capable and thanks to people like eugr, raphael.amorim and dbsci (to name a few) it get’s easier to use every day… ;-)
[1] Ollama is just a modified llama.cpp which is not as uptodate as the pure llama.cpp.
Ah. And may be there will be a better alternative for OpenClaw announced today at GTC - or they announce a partnership… the author of OpenClaw has already been spotted at GTC.
That could make things easier or even better.
Or you have a look at alternatives like:
…might avoid nervous breakdowns. ;-) Or create at least different kinds of breakdowns.
yeah i have been trying my ass of with various configurations, and when i finally did have something that seem fairly satisfactory, something always comes along from the past, should it be; drivers, versions, incompatibility with tensors, its always something going on. I find myself having to do this, its frustrating, although i can see why some guys here are so specific with data from the LLMS, and its “setups”.
About the NEMOCLAW, it sound really grate news, since it is exactly why this machine was acquired for me at least. Installing nemotron models… i cant get the NIM key anywhere, and even though i have registered my self with nvidia i cant get them? . I will try those links ! thanks **vedcsolution and cosinus.
Thanks for the fast replays! i post about my results!!! and try the nemoclaw when released!
Nemoclaw is the OpenClaw plugin for NVIDIA OpenShell. It runs OpenClaw inside a sandboxed environment with NVIDIA inference, such as Nemotron 3 Super 120B through build.nvidia.com or local vLLM. The sandbox enforces strict network policies and operator-controlled egress approval.
Stack: NemoClaw v0.1.0 (ARM64, required custom gosu patch to build) → OpenClaw 2026.3.11 → vLLM 0.17.2rc1 (eugr spark-vllm-docker) → Nemotron 3 Super 120B NVFP4 (Marlin backend, SM12.1 workaround), port 8000
Symptoms:
- Simple “say hello in 5 words” takes 23-60 seconds on a fresh session
systemPromptChars=26471 injected on EVERY request
vLLM side is fast (193-290 tok/s prefill, prefix cache hit rate 79-81%)
Bottleneck is entirely inside OpenClaw’s agent loop, not the model
Root cause appears to be the workspace file injection issue (#9157) — 26k chars of system context being sent every single turn regardless of whether it changed.
Workarounds tried:
Clearing sessions (helped slightly)
Reducing contextWindow in openclaw.json (blocked by Landlock read-only filesystem in NemoClaw sandbox)
bootstrapMaxChars reduction (cannot write config due to sandbox policy)
The NemoClaw sandbox makes it nearly impossible to tune OpenClaw’s context settings since /sandbox/.openclaw is read-only by Landlock policy.
For DGX Spark users specifically: the ARM64 gosu build issue in NemoClaw v0.1.0 requires a Dockerfile patch (gosu-arm64 + correct sha256) before the sandbox will build. Happy to share the patch script.
Is there a supported way to reduce workspace injection inside a NemoClaw sandbox, or pass bootstrapMaxChars via environment variable before sandbox creation?
I didn’t read all replays, but are you running LLMs on Spark? or what’s your hardware.
Also, try n8n. I personally found it most robust to build workflow (there is chat too) than making claw’s work as I want. Also my personal experience evaluating claws agents with locally running models was horrible as well… so I think Claude Sonnet/Opus do the heavy lifting for claw projects.
I had the same experience with my setup. My solution was the following, and I probably will stick to it:
Get VSCode (or something similar) on your Windows machine or whatever you are using, activate the Claude extension within. Let Claude establish a ssh connection to your spark. Now you have to decide if you want to give Claude sudo access. I temporarily do and change the password afterwards.
From that moment on, let Claude access your Spark, point Claude to the http link which describes what you try to achieve ( Nvidia workbooks, github repositories, whatever) and tell him to install and to debug it. The rest is hitting the spacebar to accept. Claude debugs everything, runs benchmarks, checks with findings of other spark users, fine-tunes and debugs again.. In case you get stuck let him do the research or point him to this forum to find the solution. (Hint: Always start Claude in planning modus)
It’s that horrifying simple …….
The best thing is. After each session tell Claude to write down his findings, his solutions, just everything and he writes it to a memory.md, that gets stored in his folder on the spark. Next time he remembers everything he did. Just don’t forget to tell him before you close the session. Sonnet 4.6 Usually does the job in 99% of all cases.
A full OpenClaw setup with subagents, comfyUI, ollama, vllm costs roughly 30 USD in Antropic tokens