vLLM on GB10: gpt-oss-120b MXFP4 slower than SGLang/llama.cpp... what’s missing?

[raphael.amorim]

NVFP4/FP4 isn’t being “properly utilized” on DGX Spark (GB10 / sm121) in current vLLM builds, so NVFP4 quants can be slower than AWQ 4-bit on the same workload. FP4 kernels / NVFP4 paths are better optimized for sm120 (RTX 50xx / RTX Pro 6000) than for Spark’s sm121. So, the summary is: installs got way smoother (cu130 wheels + better Docker tooling + cluster scripts), but NVFP4 performance on Spark still isn’t quite there yet.

PSA: State of FP4/NVFP4 Support for DGX Spark in VLLM

We have so many posts about VLLM now, so I decided to make a new one regarding FP4 quants. As of today, FP4 is not properly utilized in current VLLM builds on our hardware, so you lose a lot of performance picking NVFP4 quants compared to AWQ 4-bit ones. Here is a comparison between Qwen3-VL-235-A22B in NVFP4 quantization and AWQ 4-bit on my dual DGX Spark cluster using Friday version of VLLM main branch (my Docker build). I retested one model with today version, and the performance was the sam…

Run VLLM in Spark

I’ve added a convenience script to start cluster on all nodes with autodiscovery option to the repo. Nothing too fancy, wanted to keep it barebones, but useful enough. I’ll merge the pull requests after we resolve some questions there.