Wanted to share a win for the Spark community: we’ve got **MiniMax-M3-AWQ-INT4 serving tensor-parallel across all four DGX Sparks** over a RoCE fabric — 262K context, EAGLE3 speculative decode, fp8 KV.
**First and most important — full credit to @CosmicRaisins.** This stands entirely on top of their MiniMax-M3-AWQ recipe (model choice, the 4 mods, the vLLM serve flags — all theirs). We didn’t reinvent anything; we just did the GB10 deployment legwork on top of their work. Concurrency-tuning lineage also to **Keys (@u1tra_instinct)**, whose approach has shaped all our Spark deployments.
**What we added = the GB10 deployment manual.** Building Cosmic’s recipe naively on these Sparks hit 5 layered issues, in order:
- `layer_types` validation — transformers 5.9 rejects M3’s custom `minimax_m3_sparse` (patch ALLOWED_LAYER_TYPES)
- RoCE/IB not in the container → NCCL “invalid usage” (pass `/dev/infiniband`, memlock, IPC_LOCK + fabric NCCL env)
- **The big one: CUDA toolkit mismatch.** The stock vLLM wheel is built with CUDA 13.2, but the GB10 driver only supports 13.0 → `cudaErrorUnsupportedPtxVersion` in the Marlin/MoE weight-load. Fix = rebuild the wheel against the 13.0 toolkit.
- nccl 2.30.4 (2.28.9 wedges multi-node Ray on first inference)
- GMU tuning — 0.85 loads but OOMs at runtime under EAGLE3 + long prefill; 0.80 is stable
**Results:** ~33.7 tok/s decode single-stream, ~79 tok/s aggregate at 4 concurrent, ~965K-token KV pool. On a 69-task agent/tool-calling eval it scored ★★★★★ (63/69 pass, 92.5 deployability, thinking-on).
Full recipe, the 5 fixes, scripts + benchmarks here:
Thanks again @CosmicRaisins — hope the GB10 notes help others reproduce it. 🙏