Anyone have any luck running MiniMaxAi/MiniMax-M2 on a cluster DGX Spark?

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

I’ve been trying all weekend to get the latest version of VLLM and MiniMaxAi/MiniMax-M2 on a cluster of Sparks. I’m currently stuck. Wondering if anyone has been able to run MiniMaxAi/MiniMax-M2 on a DGX Spark cluster?

Some more info:
VLLM Image; llm/vllm-openai:nightly-17eb25e3271f8f22a0d8920a8115158495827cba

Ray status output:

======== Autoscaler status: 2025-12-08 00:00:19.931554 ========

Node status

---------------------------------------------------------------

Active:

1 node_e8f39ef4412378d799b591343a1053b769dd7f874db5461be91a34b3

1 node_56cfb88f56b8784f2c5c7a30292f11e1ae48a1a2ad2d289067bc0bde

1 node_2c2d2ee76539602bf33c848a18362ed4f8fdb82bd27370dd6456bf1a

1 node_f5e06b111023ecde2feaeb1855b13c10e9c72c03a4862b34c1710111

Pending:

(no pending nodes)

Recent failures:

(no failures)

Resources

---------------------------------------------------------------

Total Usage:

0.0/80.0 CPU

0.0/4.0 GPU

0B/439.05GiB memory

0B/38.91GiB object_store_memory

VLLM serve command:

root@spark-0001:/vllm-workspace# SAFETENSORS_FAST_GPU=1 vllm serve \

    MiniMaxAI/MiniMax-M2 --trust-remote-code \

    --tensor-parallel-size 4 \

    --enable-auto-tool-choice --tool-call-parser minimax_m2 \

    --reasoning-parser minimax_m2_append_think \

    --distributed-executor-backend ray \

    --host 0.0.0.0 \

    --port 8000

Where it’s stuck at:


....snip....
(EngineCore_DP0 pid=466) (RayWorkerWrapper pid=195, ip=10.0.0.3) INFO 12-08 00:01:13 [layer.py:379] Enabled separate cuda stream for MoE shared_experts [repeated 3x across cluster]
(EngineCore_DP0 pid=466) (RayWorkerWrapper pid=195, ip=10.0.0.3) INFO 12-08 00:01:13 [fp8.py:206] Using Triton backend for FP8 MoE [repeated 3x across cluster]
(EngineCore_DP0 pid=466) (RayWorkerWrapper pid=195, ip=10.0.0.2) INFO 12-08 00:03:35 [gpu_model_runner.py:3602] Model loading took 53.7422 GiB memory and 149.992080 seconds
(EngineCore_DP0 pid=466) (RayWorkerWrapper pid=195, ip=10.0.0.1 INFO 12-08 00:03:38 [default_loader.py:308] Loading weights took 142.18 seconds
2

Yes, but you will need a lower quant, FP8 model would not fit on two sparks with CUDA graphs and any reasonable context.

I’ve posted the instructions in other threads, but all you need to do is to use my Docker build - it compiles vllm from main branch with cuda13 pytorch and proper architecture selection: GitHub - eugr/spark-vllm-docker: Docker configuration for running VLLM on dual DGX Sparks

Nightly vllm builds are still based on Cuda 12.9 and do not support Spark well.

Then you can run the AWQ quant like this (it will give you ~40 t/s):

vllm serve QuantTrio/MiniMax-M2-AWQ \
  --port 8888 \
  --host 0.0.0.0 \ 
  --gpu-memory-utilization 0.7 \
  -tp 2 --distributed-executor-backend ray \
  --max-model-len 128000 \
  --load-format fastsafetensors \
  --enable-auto-tool-choice \
  --tool-call-parser minimax_m2 \
  --reasoning-parser minimax_m2_append_think
3

Hi @eugr ,

Thanks for the links and the reply. Just want to clarify this is on 4 networked Spark nodes via 100GbE hub.

4

Well, you should be able to launch FP8 using my docker then.

6

Something is odd.

When serving MiniMaxAI/MiniMax-M2 on 4x Sparks using vLLM’s OpenAI-compatible API, the model produces corrupted output (infinite repetition inside blocks, token spam, failure to follow instructions), even for trivial prompts. This also occurs when calling the model directly (curl.)

Have you experienced this @eugr on the Sparks with VLLM before?

7

Yes, there was a bug in VLLM introduced a few days ago. I’ve reported it, and it’s been fixed now - please rebuild vllm from main.

8

Actually, there were two separate bugs - the fix that fixed broken BF16 and FP8 versions, broke AWQ quants. The second fix fixed AWQ ones.

This is the second one: [Bug]: QuantTrio/MiniMax-M2-AWQ produces garbage in 12/10/2025 build · Issue #30445 · vllm-project/vllm · GitHub

9

Awesome! Thanks for confirming. This would explain the issue since VLLM defaults to FP8 when loading MiniMaxAI/MiniMax-M2 (full model).

Will go it at it tomorrow morning.

10

BTW, if you are using my Docker build, you can try the improved version: GitHub - eugr/spark-vllm-docker at multistage

I’m currently testing it, and will merge into main if I don’t find any issues. So far so good. Lots of improvements, main being reduced image size and much faster container rebuilds.