MiniMax-2.5 on DGX Spark (thanks to Unsloth https://unsloth.ai/docs/models/minimax-2.5)

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

snakegameminimax
snakegameminimax1468×1626 217 KB

Minimax1
Minimax11920×1071 175 KB

This is a speedrun, so it’s not super optimized, but it works!

Strategy: Start with a small Llama model to make sure your environment is solid, then move to the heavyweight: MiniMax-2.5.

Credits: All credit to the unsloth.ai team for the optimized model quants: MiniMax-2.5: How to Run Guide | Unsloth Documentation

OK , First Before downloading 100GB+, start small. This verifies your Blackwell GB10 drivers, CUDA paths, and the hf CLI. Build time is only 2–3 minutes with Llama 3.2-1B Instruct.

Paste this in your Spark terminal:

#Bash
cat << ‘EOF’ > spark_test_small.sh && chmod +x spark_test_small.sh
#!/bin/bash

DGX Spark Small Model Smoke Test - Llama-3.2-1B (Standalone CLI Version)

1. Install the standalone ‘hf’ binary

if ! command -v hf &> /dev/null; then
echo “Installing Hugging Face standalone CLI…”
curl -LsSf https://hf.co/cli/install.sh | bash
export PATH=“$HOME/.local/bin:$PATH”
fi

2. Download model

mkdir -p ~/models/small-tests
echo “Downloading model…”
hf download unsloth/Llama-3.2-1B-Instruct-GGUF
–local-dir ~/models/small-tests
–include “Q8_0.gguf”

3. Verification Check

if [ ! -f ~/models/small-tests/Llama-3.2-1B-Instruct-Q8_0.gguf ]; then
echo “Model file not found. Note: Llama 3.2 is ‘gated’.”
echo “Run ‘hf login’ first with your Hugging Face token.”
exit 1
fi

4. Run the Blackwell-optimized test

echo “Loading Llama-3.2-1B on Blackwell GB10…”
~/llama.cpp/build/bin/llama-cli
-m ~/models/small-tests/Llama-3.2-1B-Instruct-Q8_0.gguf
-ngl 999
–flash-attn on
-p “Write a short haiku about a fast computer named Spark.”
EOF

./spark_test_small.sh

Step 2: yolo TIME (MiniMax-2.5 Deployment)
MiniMax-2.5 is a 230B parameter beast. The UD-Q3_K_XL quant is 101GB, split into 4 shards.
The Main Event: (Make sure to add your HF token if its asks !)

#Bash
cat << ‘EOF’ > spark_minimax.sh && chmod +x spark_minimax.sh
#!/bin/bash

DGX Spark - MiniMax-2.5 (101GB Deployment)

1. Path and Token Setup

export PATH=“$HOME/.local/bin:$PATH”
export HF_HUB_ENABLE_HF_TRANSFER=1

ENTER YOUR TOKEN ( Hugging Face – The AI community building the future. )

HF_TOKEN=“your_token_here”

2. Authenticate

if [ “$HF_TOKEN” != “your_token_here” ]; then
hf login --token “$HF_TOKEN”
fi

3. Download the Model

mkdir -p ~/models/MiniMax-2.5-GGUF
hf download unsloth/MiniMax-M2.5-GGUF
–local-dir ~/models/MiniMax-2.5-GGUF
–include “UD-Q3_K_XL”

4. Clear memory (Ensure Ollama is fully stopped , i was running it in parallel for somethign else )

sudo systemctl stop ollama.service 2>/dev/null

5. Run with Blackwell GB10 Optimizations

Points to the first of 4 shards; llama.cpp links the rest automatically.

~/llama.cpp/build/bin/llama-cli
-m ~/models/MiniMax-2.5-GGUF/UD-Q3_K_XL/MiniMax-M2.5-UD-Q3_K_XL-00001-of-00004.gguf
–no-mmap
–jinja
-ngl 999
–flash-attn on
-t 20
-c 16384
-p “Explain the Mixture of Experts (MoE) architecture like I’m five.”
EOF

./spark_minimax.sh

2

Took a while to download and it max’s out the spark but i still get 20+ tokens/seconds and its so good !..

sparkmaxdout
sparkmaxdout1078×860 38.3 KB

waitingfordownload

3

I was able to replicate your results with llama-cli. However, when I attempted to hook this up to my claude coder or other OpenAI API compatible UI, I got very poor throughput with the llama-server. I am new to working with DGX Spark and these tools, but were you able to get it working well as a server?

4

cool :)..the biggest difference for me was setting parallel slots to 1 ..basically np1

~/llama.cpp/build/bin/llama-server
-m ~/models/MiniMax-2.5-GGUF/UD-Q3_K_XL/MiniMax-M2.5-UD-Q3_K_XL-00001-of-00004.gguf
–no-mmap
–jinja
-ngl 999
–flash-attn on
–port 8080
-c 32768
-np 1

5

You don’t need to bother with the hf cli. llama-server can download directly.

llama-server -hf unsloth/MiniMax-M2.5-GGUF:UD-Q3_K_XL

You also don’t need to specify –no-mmap, –jinja,-ngl 999, –flash-attn or -t. These are all defaults now.

Nor context. –fit is on by default and will max out the context for you. Just an FYI.

There are SOME flags that will turn –fit and –fa off. I think –context and -ts will disable fit. FA is disabled with -ctv,-ctk

6

Published the benchmarks for Minimax M2.5 4-bit yesterday:

Spark Arena - LLM Leaderboard

7

I wonder whether -fit could be problematic with Spark, since it reacts critically to overload. But I don’t know when -fit calculates the available memory? At the beginning or continuously?

8

As a general rule, you seem to get better results with vLLM, especially when using concurrency where it smashes most other methods. Currently AWQ works best but NVFP4 will dominate once the software ecosystem catches up.

I tend to use this playbook from the community: GitHub - NVIDIA/dgx-spark-playbooks: Collection of step-by-step playbooks for setting up AI/ML workloads on NVIDIA DGX Spark devices with Blackwell architecture. .

For coding, qwen-next-coder-awq works like a dream with 256k context window. This is my script:

# Recipe: Qwen3-Coder-Next-AWQ-4bit
# bullpoint's AWQ 4-bit quantized Qwen3-Coder-Next model
# Fits on a single DGX Spark GPU with AWQ quantization

recipe_version: "1"
name: Qwen3-Coder-Next-AWQ
description: vLLM serving bullpoint/Qwen3-Coder-Next-AWQ-4bit on a single node

# HuggingFace model to download (optional, for --download-model)
model: bullpoint/Qwen3-Coder-Next-AWQ-4bit

# AWQ 4-bit fits on a single GPU
solo_only: true

# Container image to use
container: vllm-node

# Mod required to fix slowness and crash (same as FP8 variant)
mods:
  - mods/fix-qwen3-coder-next

# Default settings (can be overridden via CLI)
defaults:
  port: 8002
  host: 0.0.0.0
  tensor_parallel: 1
  gpu_memory_utilization: 0.70
  max_model_len: 262144
  max_num_seqs: 16
  max_num_batched_tokens: 131072
  served_model_name: qwen3-coder-next

# Environment variables
env:
  VLLM_FLASHINFER_FORCE_TENSOR_CORES: 1
  VLLM_FLASHINFER_MOE_BACKEND: "throughput"

# The vLLM serve command template
command: |
  vllm serve bullpoint/Qwen3-Coder-Next-AWQ-4bit \
      --served-model-name {served_model_name} \
      --load-format fastsafetensors \
      --attention-backend flashinfer \
      --port {port} \
      --host {host} \
      --max-model-len {max_model_len} \
      --block-size 128 \
      --max-num-seqs {max_num_seqs} \
      --max-num-batched-tokens {max_num_batched_tokens} \
      --gpu-memory-utilization {gpu_memory_utilization} \
      --kv-cache-dtype auto \
      --enable-prefix-caching \
      --trust-remote-code \
      --enable-auto-tool-choice \
      --tool-call-parser qwen3_coder \
      -tp {tensor_parallel}

Some people prefer the FP8 but it overloads with high context for me.

9

When I ran this model the benchmark rebooted the spark (OOM):

llama-benchy --base-url http://:8002/v1 bullpoint/Qwen3-Coder-Next-AWQ-4bit
–depth 0 4096 8192 16384 32768 65535 100000
–pp 2048
–tg 128
–enable-prefix-caching
–concurrency 1 2 5 10
–save-result tests.csv

10

Hmm makes sense - there’s already been so much progress. It’s pretty amazing that it’s this “simple” to run such a complex model. I just read the docs on NVFP4 as I’d seen others talking about it, as you say lets hope makes good support makes its way into the mainline vLLM build sometime soon. Thanks for the playbook link - worth it’s weight in gold!

11

Hm… thats odd. I tended to get reboots with the fp8 above a mid range context, but AWQ has been my daily driver for the last week or two. Does it happen every time?

12

TPS for minimax 2.5 q3 ??

13

I only used it via ollama (the above more was of an “in general” opposed to that model), but it struggled with long context so went back to Qwen-next-coder-awq until my second spark arrives next week.
I avoid using anything below 4-bit or anything that struggles below 200k context currently, but i think some peeps who run that repo have posted their Minimax benchs somewhere on here.
I suspect there’ll be some amazing new quant methods coming out over the next 6 months so will drop below 4-bit then.