PSA: State of FP4/NVFP4 Support for DGX Spark in VLLM

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
205

Actually, it does now, starting from yesterday build :)

Not fully tested, and it’s hard to measure MTP performance as it’s not very consistent. In llama-benchy, it consistently underperforms compared to non-MTP, but I’m seeing slight boost in some real-world usage, although it is also variable and doesn’t work in all situations. When I have time, I will come up with a good MTP benchmark, but until then at least you can try for yourself.

Recipe for Nemotron 3 Super:

recipes/nemotron-3-super-nvfp4-flashinfer.yaml

# Recipe: Nemotron-3-Super-NVFP4
# Uses VLLM_CUTLASS for NVFP4
recipe_version: "1"
name: Nemotron-3-Super-NVFP4-CUTLASS-Optimized
description: vLLM serving Nemotron-3-Super-120B using CUTLASS kernels

model: nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4
container: vllm-node
cluster_only: false
solo_only: false

# mods:
#   - mods/nemotron-super

env:
  VLLM_FLASHINFER_ALLREDUCE_BACKEND: trtllm
  VLLM_ALLOW_LONG_MAX_MODEL_LEN: 1

container: vllm-node
defaults:
  port: 8000
  host: 0.0.0.0
  tensor_parallel: 2
  gpu_memory_utilization: 0.7
  max_model_len: 262144
  max_num_seqs: 4
  max_num_batched_tokens: 8600 # needs to be > 8400

command: |
  vllm serve nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 \
  --kv-cache-dtype fp8 \
  --trust-remote-code \
  --gpu-memory-utilization {gpu_memory_utilization} \
  --max-model-len {max_model_len} \
  --max-num-seqs {max_num_seqs} \
  --host {host} \
  --port {port} \
  --enable-auto-tool-choice \
  --load-format fastsafetensors \
  --tool-call-parser qwen3_coder \
  --reasoning-parser nemotron_v3 \
  --mamba_ssm_cache_dtype float32 \
  --enable-prefix-caching --mamba-cache-mode align \
  --max-num-batched-tokens {max_num_batched_tokens} \
  --tensor-parallel-size {tensor_parallel} \
  --distributed-executor-backend ray

Then you can run it like:

./run-recipe.sh --solo  recipes/nemotron-3-super-nvfp4-flashinfer.yaml --max-model-len 131072 --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'