I haven’t read the details on the model yet, but we’ve already tested cyankiwi/MiniMax-M2.1-AWQ-4bit and QuantTrio/MiniMax-M2-AWQ on vLLM. If vLLM already has support it should be straightforward.
Given their previous track record, it will be a few weeks until they release open weights on HF and another few days until suitable quants pop up. The model architecture seems to be the same as MiniMax M2 and M2.1, so should work out of the box.
Releasing in a few more hours, but yes, few days for quants we can use.
Just released! Might it be possible to run the 230 GB version on two sparks?
The model is 230GB, which would mean that the model would take up entire memory from 2 Sparks. Better to have 3 Sparks.
vLLM will have day 0 support for 2.5
Perfect!
cyankiwi! We need MiniMax-M2.5-AWQ-4bit :-)
Maybe we could quantize it ourselves specifically for DGX Spark?
If we join forces as a community, we could explore producing a MiniMax-M2.5 AWQ 4-bit (or even NVFP4 / MXFP4) variant optimized for Spark’s memory and bandwidth constraints.
I think it would be a useful exercise if the nvidia team could help us convert this to a proper nvFP4 format that we could run directly. This model is the reason why people buy two of these devices.
Until NVFP4 is properly supported on Spark, AWQ quants perform better.
Ran MiniMaxAI/MiniMax-M2.5 (no quant) on 4x Spark Cluster using image: scitrera/dgx-spark-vllm:0.16.0-t5. Haven’t really explored/verified optimization yet – just wanted to start trying it out.
Short Contexts: uvx llama-benchy --base-url "http://10.24.11.13:8000/v1" --pp 512 2048 8192 --tg 32 128 --runs 5 --model "MiniMaxAI/MiniMax-M2.5"
| model | test | t/s | peak t/s | peak t/s (req) | ttfr (ms) | est_ppt (ms) | e2e_ttft (ms) |
|:-----------------------|-------:|------------------:|-------------:|-----------------:|------------------:|------------------:|------------------:|
| MiniMaxAI/MiniMax-M2.5 | pp512 | 1037.42 ± 154.71 | | | 509.70 ± 96.28 | 507.68 ± 96.28 | 509.75 ± 96.29 |
| MiniMaxAI/MiniMax-M2.5 | tg32 | 30.83 ± 2.49 | 32.24 ± 1.59 | 32.24 ± 1.59 | | | |
| MiniMaxAI/MiniMax-M2.5 | pp512 | 1158.23 ± 15.50 | | | 444.16 ± 5.89 | 442.13 ± 5.89 | 444.20 ± 5.90 |
| MiniMaxAI/MiniMax-M2.5 | tg128 | 33.01 ± 0.59 | 34.40 ± 0.80 | 34.40 ± 0.80 | | | |
| MiniMaxAI/MiniMax-M2.5 | pp2048 | 2186.74 ± 1404.98 | | | 3029.38 ± 3882.54 | 3027.35 ± 3882.54 | 3029.43 ± 3882.54 |
| MiniMaxAI/MiniMax-M2.5 | tg32 | 29.95 ± 3.81 | 30.88 ± 3.93 | 30.88 ± 3.93 | | | |
| MiniMaxAI/MiniMax-M2.5 | pp2048 | 3376.65 ± 26.55 | | | 608.58 ± 4.79 | 606.56 ± 4.79 | 608.62 ± 4.79 |
| MiniMaxAI/MiniMax-M2.5 | tg128 | 32.80 ± 0.55 | 34.00 ± 0.00 | 34.00 ± 0.00 | | | |
| MiniMaxAI/MiniMax-M2.5 | pp8192 | 2843.38 ± 856.86 | | | 3455.29 ± 1898.81 | 3453.26 ± 1898.81 | 3455.34 ± 1898.81 |
| MiniMaxAI/MiniMax-M2.5 | tg32 | 31.00 ± 0.28 | 31.74 ± 0.61 | 31.74 ± 0.61 | | | |
| MiniMaxAI/MiniMax-M2.5 | pp8192 | 3235.93 ± 381.45 | | | 2571.71 ± 326.21 | 2569.68 ± 326.21 | 2571.76 ± 326.22 |
| MiniMaxAI/MiniMax-M2.5 | tg128 | 29.93 ± 2.79 | 32.00 ± 0.63 | 32.00 ± 0.63 | | | |
llama-benchy (0.3.0)
date: 2026-02-13 11:46:55 | latency mode: api
Which vLLM Docker image works with MiniMaxs?
any image that works with M2.1, so anything built in the last couple of months.
works with these flags:
exec vllm serve
mratsim/Minimax-M2.5-BF16-INT4-AWQ
–served-model-name MiniMax-M2.5
–port 8000
–host 0.0.0.0
–gpu-memory-utilization 0.85
-tp 2
–distributed-executor-backend ray
–max-model-len 131072
–enable-auto-tool-choice
–tool-call-parser minimax_m2
–reasoning-parser minimax_m2_append_think
–trust-remote-code
haven’t benched or anything - around 25tks is what i’m getting.
note: fp8 is cooking
Ever tried vllm with int4 autoround quant on dgx? 96 tok/s on a3b on single dgx, zero context, marlin kernel. Minimax has 10b, so single dgx peak at 32 tok/s, 4x dgx at ~100 tok/s?
Maybe that runs on the spark
minimax-m2.5-Q3_K_L.gguf Q3_K_L ~110 GB Sweet spot for 128GB Macs. Runs natively in RAM with high t/s ( 28 ON MAC M3 MAX ).
The question is: is it still useful in 3 bit?
I made some tests in 64 core ryzen with 256gb glm 5.0 2bit reachinf 12 tok/s (8 channels ddr5). Large models seem to work properly even in low quant. just proof.
minimax will be fast x20 active/total and 3bits. If you have a drafter (eagle3) you may reach … 94/10x3x4/3 = ~ 37.6 tok/s zero context (proportional calculation)
Unsloth should have their dynamic Q3_K_XL quant that works on a single Spark.
