# Successfully serving MiniMax-M3-NVFP4 on 4x DGX Spark with vLLM **URL:** \ **Category:** DGX Spark / GB10\ **Tags:** llama\ **Created:** [June 20, 2026, 4:00am UTC](https://forums.developer.nvidia.com/t/successfully-serving-minimax-m3-nvfp4-on-4x-dgx-spark-with-vllm/373927 "2026-06-20T04:00:48Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![OllieJW](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/olliejw/32/532670_2.png) [@OllieJW](https://forums.developer.nvidia.com/u/OllieJW)\ **Post date:** [June 20, 2026, 4:00am UTC](https://forums.developer.nvidia.com/t/successfully-serving-minimax-m3-nvfp4-on-4x-dgx-spark-with-vllm/373927/1 "2026-06-20T04:00:49Z")
The model card for [lukealonso/MiniMax-M3-NVFP4](https://huggingface.co/lukealonso/MiniMax-M3-NVFP4) notes that a vLLM container for SM120 is forthcoming and that the model may be difficult to run “due to the specialized kernels required.” After spending time waiting for upstream instructions I worked out a working configuration for 4x DGX Spark (GB10, sm\_121) and wanted to share what worked so that other folks with DGX Spark clusters can serve the model without starting from scratch. @eugr’s [spark-vllm-docker](https://github.com/eugr/spark-vllm-docker) was used for this process. ## Requirements 1. A custom container image. The model needs: 2. Multi-node launch must be `--no-ray` (PyTorch distributed), not Ray. With Ray the chthonic build does not initialize the MiniMax M3 parsers cleanly. The launcher is `launch-cluster.sh --no-ray --master-port 29501`. 3. The vLLM serve flags that worked for me on 4 nodes: 4. NCCL/RoCE environment. On DGX Spark with a CX7 fabric, the values that mattered were: 5. The `--no-ray` flag in step 2 is what makes the b12x runtime metadata check pass. If you skip it you get a startup error related to `B12XPagedAttentionScratchCaps. __init__ ` missing `copy_runtime_metadata`. ## Startup Checks You should see all of the following in the head node logs: ```auto Detected ModelOpt NVFP4 checkpoint Using B12X MiniMax M3 MSA attention Using 'B12X' NvFp4 MoE backend FORCED_NCCL_VERSION 23007 vLLM is using nccl==2.30.7 NCCL version 2.30.7+cuda13.2 Application startup complete ``` If `FORCED_NCCL_VERSION` is not `23007`, the `LD_PRELOAD`/symlink chain did not take effect and multi-node collectives will hang. ## Notes - Cold startup is long… roughly 10 minutes for shard loading plus several more minutes for profiling, KV cache allocation, B12X MSA warmup, and CUDA graph capture (FULL mode is the slowest to capture). Multimodal warmup adds another 2 to 3 minutes. Total cold start on the head node was about 15 to 18 minutes on my setup. - Multimodal… send images as data URIs in `image_url.url` to avoid the model server fetching external URLs. Multimodal warmup completes successfully with `--skip-mm-profiling` on the chthonic build. - Performance snapshot from my llama-benchy run ( [LLM - Spark Arena Benchmark](https://spark-arena.com/benchmark/sub1781926492932:) ): - Generation tokens/sec, mean plus or minus std, concurrency 1: roughly 9 to 10 t/s across depths - Concurrency 2: 14 to 18 t/s for depths up to 32768 (higher depths skipped because KV budget exceeded) - Concurrency 5: roughly 26 t/s at depth 0 - Prompt processing (`pp2048`) ranges from about 5000 t/s at depth 0 down to about 600 t/s at depth 262144 ## DGX Spark Cluster ![image](https://global.discourse-cdn.com/nvidia/original/4X/f/7/c/f7ce4caeda6220161a60fca0cee87f356ec8f2ec.jpeg) ---
**Author:** ![p33zy](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@p33zy](https://forums.developer.nvidia.com/u/p33zy)\ **Post date:** [June 20, 2026, 8:03am UTC](https://forums.developer.nvidia.com/t/successfully-serving-minimax-m3-nvfp4-on-4x-dgx-spark-with-vllm/373927/2 "2026-06-20T08:03:21Z")
Thanks for sharing this, appreciated to have the entire process documented rather than rely on other people’s docker images. Do you have any thoughts on why performance is much worse than the spark arena version? Is it just because more layers have been preserved in bf16? You could also try lukealonso/MiniMax-M3-MXFP8-NVFP4 which converts the preserved bf16 layers to FP8. Waiting for my 4th spark to arrive, otherwise would give it a go myself. ---
**Author:** ![Teason2026](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Teason2026](https://forums.developer.nvidia.com/u/Teason2026)\ **Post date:** [June 20, 2026, 2:29pm UTC](https://forums.developer.nvidia.com/t/successfully-serving-minimax-m3-nvfp4-on-4x-dgx-spark-with-vllm/373927/3 "2026-06-20T14:29:13Z")
that what I’m afraid of, minimax m2.7 was more than fine on TP=2 gb10, m3 is larger and even on TP=4 gb10 PP/TG numbers does not look awesome. Maybe some optimization will come, but I think we are stuck on 9-10TPS TG without drafter. ---
**Author:** ![OllieJW](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/olliejw/32/532670_2.png) [@OllieJW](https://forums.developer.nvidia.com/u/OllieJW)\ **Post date:** [June 20, 2026, 3:14pm UTC](https://forums.developer.nvidia.com/t/successfully-serving-minimax-m3-nvfp4-on-4x-dgx-spark-with-vllm/373927/4 "2026-06-20T15:14:40Z")
`sparkarena/Minimax-M3-v0-NVFP4` is likely faster because it combines broader quantization coverage with a serving stack (sglang) that is better optimized for the DGX Spark NVFP4 workload. More BF16-preserved layers in `lukealonso/MiniMax-M3-NVFP4` are a big contributor, but the benchmark results show the runtime and decode path are also major contributors. ---
**Author:** ![OllieJW](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/olliejw/32/532670_2.png) [@OllieJW](https://forums.developer.nvidia.com/u/OllieJW)\ **Post date:** [June 20, 2026, 3:17pm UTC](https://forums.developer.nvidia.com/t/successfully-serving-minimax-m3-nvfp4-on-4x-dgx-spark-with-vllm/373927/5 "2026-06-20T15:17:46Z")
I suspect more optimizations will come along not to mention the NVFP4 quant that will likely come from Nvidia. This model will be my daily driver moving forward and I will continue to find ways to improve throughput. ---
**Author:** ![renek](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@renek](https://forums.developer.nvidia.com/u/renek)\ **Post date:** [June 21, 2026, 9:19pm UTC](https://forums.developer.nvidia.com/t/successfully-serving-minimax-m3-nvfp4-on-4x-dgx-spark-with-vllm/373927/7 "2026-06-21T21:19:40Z")
How’s quality so far for you ? In agentic it was worse than mimo v2.5 for me, currently optimize mimo but only for 2 Nodes ---
**Author:** ![truxnor](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@truxnor](https://forums.developer.nvidia.com/u/truxnor)\ **Post date:** [June 21, 2026, 9:53pm UTC](https://forums.developer.nvidia.com/t/successfully-serving-minimax-m3-nvfp4-on-4x-dgx-spark-with-vllm/373927/8 "2026-06-21T21:53:21Z")
I can honestly say this is pretty good for my uses. I’m running the recipe from sparkarena, I have played around with a vllm recipe, but Ive not had enough time this past week to spend too much time with that. I’m using it for DFIR, so having a bigger context is really useful, this is a good test for what I am building. Initially I thought the reasoning was a problem, but now with some decent amount of time using the model I actually like that the fact the reasoning helps me understand why it came to that conclusion. I can push back in the instances it was wrong and get it to do change its opinion or to go back and look for more evidence. The only coding I use it for currently, is to create smallish scripts when it needs to create something to look at the log files. ---
**Author:** ![Teason2026](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Teason2026](https://forums.developer.nvidia.com/u/Teason2026)\ **Post date:** [June 22, 2026, 6:43am UTC](https://forums.developer.nvidia.com/t/successfully-serving-minimax-m3-nvfp4-on-4x-dgx-spark-with-vllm/373927/9 "2026-06-22T06:43:26Z")
I have 4 gb10s and I use it in 2 pairs or 1 pair and 2 single mode. Minimax m2.5-2.7 was niche for that use case and delivered fine long assistant loops with some code researching/adjusting (yet main coder is qwen3.6-27b at the moment, served in TP=1 or TP=2 depending on plans for the day). At quad node, I go for glm-4.7-nvfp4 max context, or glm-5, 5.1, 5.2 min context (or compressed KV cache, K Q8-\> V Q4). MiniMax M3 is not good enough to get this quad node niche (at least vs GLM), you will get better results by combining minimax TP=2 + qwen TP=1/2 or minimax TP=2 + ds4f TP=2 (minimax workhorse, deepseek needle seeker/reviewer). That of course unless you use tasks from claude/codex fusion with local models. ---
**Author:** ![OllieJW](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/olliejw/32/532670_2.png) [@OllieJW](https://forums.developer.nvidia.com/u/OllieJW)\ **Post date:** [June 22, 2026, 9:43pm UTC](https://forums.developer.nvidia.com/t/successfully-serving-minimax-m3-nvfp4-on-4x-dgx-spark-with-vllm/373927/10 "2026-06-22T21:43:51Z")
Great question - I am updating my system prompts to be more inline with MiniMax-M3 prompting. So far, my anecdotal results have been positive. My primary use case involves VL capabilities, so MiniMax-M3 meets input text and image requirements unlike many of the other usual suspects that can fit in 512GB. ---
**Author:** ![OllieJW](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/olliejw/32/532670_2.png) [@OllieJW](https://forums.developer.nvidia.com/u/OllieJW)\ **Post date:** [June 22, 2026, 9:47pm UTC](https://forums.developer.nvidia.com/t/successfully-serving-minimax-m3-nvfp4-on-4x-dgx-spark-with-vllm/373927/11 "2026-06-22T21:47:49Z")
This is a good way to look at the reasoning transparency. I use it as an early indicator of instruction following.