Getting MiniMax-M2.7 Running on Two DGX Sparks: What Finally Worked (and What Didn’t)
A field report from the Leathery Tendons lab — running a frontier agentic MoE across two NVIDIA DGX Spark (GB10) boxes, the wedges that nearly killed it, and the exact recipe that fixed it.
The goal
I have two DGX Sparks (GB10 Grace-Blackwell, 128 GB unified memory each, joined by a direct ConnectX-7 / QSFP56 200 Gb link running RoCE). The point of two of them was simple: run a bigger, better open-weight model than I could fit on one box, and use it as the brain for my self-hosted Hermes agent (“Agent 13”).
The target: cyankiwi/MiniMax-M2.7-AWQ-4bit — a ~229B-parameter Mixture-of-Experts model at 4-bit, with a native tool-call parser and a huge context window. It’s the model Techno Tim reports running on his two ASUS Ascent GX10s at ~40 tok/s, and it’s a genuinely strong agentic/ops model. The plan was tensor-parallel across both Sparks (TP=2).
It took several failed attempts and a hard lesson before it worked. Here’s the honest version.
What didn’t work — the wedges
Every early attempt died the same way: the model would start loading, then both Sparks would lock up — SSH itself would time out at banner exchange, the endpoint never opened, and the only recovery was a physical power-cycle of both boxes. Classic host saturation during model load.
The natural assumption — and the one that cost the most time — was “it’s a memory problem; lower --gpu-memory-utilization.” We dropped it to 0.6, then planned 0.5. It kept wedging. That assumption was wrong, and here’s the tell that should have caught it sooner:
Techno Tim runs the same model at --gpu-memory-utilization 0.75 and 192K context and it works. We were wedging at 0.6 and 8K context. Higher memory pressure, more context — and his works while ours died. That proves the wedge was never about the memory fraction. It was about deviating from the known-good recipe.
The misses (documented honestly, because they’re instructive)
The automated agent driving the earlier attempts (“Codex”) got impressively far — it formed the Ray cluster, resolved the model path, got world_size=2 with both ranks joined — but it deviated from the proven recipe in ways that each, on their own, can wedge a GB10:
- It used a self-built vLLM image instead of the proven one. A custom
vllm-gb10:v0.24.0build rather than the image the working recipe was validated against. - It used
--no-ray(PyTorch-distributed) instead of the Ray backend the recipe specifies. - The big one: it almost certainly never set
NCCL_NET_GDR_LEVEL=0. GPUDirect RDMA is unsafe on the GB10 unified-memory SoC — the GPU and CPU share the same DRAM, there’s no PCIe peer path for GDR, and if NCCL attempts it, it can hard-lock the SoC during load. That is exactly the wedge symptom. This single environment variable is the difference between a clean load and two frozen machines. - It chased the memory fraction (0.6 → 0.5) as the fix, when 0.5 is actually too low for this model — at ~61 GB of weights per node (TP=2), a 0.5 budget leaves no room for KV cache and fails the other way.
- Reboot persistence gaps bit us too: the NAS mount (
/mnt/ai) didn’t auto-remount after a Spark reboot, and the agent’s egress proxy (tinyproxy) wasn’t enabled on boot — both silent failures that look like model bugs but aren’t.
None of this is a knock on building your own tooling — it’s just that on brand-new hardware (GB10 / sm_121a), the gap between “almost right” and “right” is a frozen machine.
What worked — Techno Tim’s recipe, verbatim
The fix was to stop improvising and run Techno Tim’s recipe exactly as published, changing only the model source (to a local copy on my NAS, to skip a 122 GB re-download). The pieces that mattered:
- His image:
ghcr.io/timothystewart6/vllm-gb10:latest— a vLLM build pinned and compiled natively for GB10 / sm_121a, tracking upstream. (Built on eugr’s originalspark-vllm-dockerproject, which deserves the foundational credit — that’s what made the two-node path possible in the first place.) - Ray backend + a
sleep 60in the head command. The head starts Ray, waits 60 seconds for the worker to join, then launches vLLM. Skip the wait and vLLM can silently fall back to single-GPU mode and OOM. NCCL_NET_GDR_LEVEL=0(withNCCL_IB_DISABLE=0so CPU-side RoCE stays on). This is the anti-wedge flag. The whole NCCL block — pinning the HCA, GID index 3, Ring algorithm — matters, but this is the one that stops the SoC hard-lock.--gpu-memory-utilization 0.75(the floor for a 61 GB/node model, not an aggressive setting),--load-format fastsafetensors,--kv-cache-dtype fp8_e4m3.privileged: true,ipc: host,--ulimit memlock=-1,/dev/infiniband— privileged is required on GB10 to hit full throughput.
One extra gotcha worth flagging for anyone cloning the repo: the compose file’s command: is a folded YAML block scalar, and on my system the extra-indented flags got parsed with newlines instead of spaces, so bash saw each flag as its own command and the container crash-looped. Rewriting command: as a single line fixed it instantly.
The result
Clean launch. Ray cluster formed, the worker joined the TP=2 placement group (real cross-node tensor parallelism, not a single-node fallback), weights loaded over fastsafetensors, the endpoint came up, and memory stayed healthy — no wedge.
Then the test that actually matters for an agent brain — structured tool calls:
PLAIN CHAT -> "OK"
call 1: OK args={"city": "Paris"}
call 2: OK args={"city": "Paris"}
call 3: OK args={"city": "Paris"}
call 4: OK args={"city": "Paris"}
call 5: OK args={"city": "Paris"}
=== TOOL-CALL SCORE: 5/5 ===
MiniMax-M2.7 serving across two DGX Sparks, tensor-parallel, with a perfect tool-call score. The thing that had frozen both machines repeatedly came up clean once we ran the recipe as written.
The lessons, distilled
- On GB10, “almost right” is a frozen machine. Use the known-good image and recipe before you tune.
NCCL_NET_GDR_LEVEL=0is mandatory on the unified-memory SoC — GPUDirect RDMA will hard-lock it.- Don’t chase the memory fraction. A wedge at lower utilization than a known-good config means your problem is elsewhere. (And for a big model, too-low utilization fails for lack of KV room.)
--no-rayvs Ray, thesleep 60worker-join wait, single-line commands — small things, each capable of sinking the run.- Verify reboot persistence (NAS auto-mount, proxy enabled) before blaming the model.
Credit
- eugr —
spark-vllm-docker, the original community project that made dual-Spark vLLM real. - Techno Tim — the published, working
vllm-gb10image and two-node MiniMax recipe. This worked because he did the hard part and wrote it down. The right move, when you’re on bleeding-edge hardware, is to run the proven recipe verbatim — and then experiment.
— Leathery Tendons