Getting vLLM running on the DGX Spark hasn’t been easy since launch. The Spark shipped in October 2025, ahead of mature library support from most vendors, inference engines, and even NVIDIA itself — so a lot of us have spent the months since troubleshooting.
I put together two blog posts documenting the timeline I’ve worked through since December, in case they save someone else the hours:
-
Field notes: vLLM on the DGX Spark, the Hard Way — the operational story: what broke, which images actually work today, the FP4 checkpoint-format rule, unified-memory OOM math, and measured tok/s.
-
Technical report: A Complete Technical Guide — the kernel-level why: sm_121 vs sm_100, CUTLASS/FlashInfer/Marlin, NVFP4/MXFP4/FP8 wiring, and how vLLM’s backend oracle picks a path.
A lot of this meant reading vLLM source to understand how it implements the inference optimizations that have become mainstream. Which is funny, because digging into the internals of LLMs and inference engines is exactly why I bought the Spark — so in a sense the early, immature pipeline gave me precisely what I wanted.
Issues have become much less common since May '26, so even if you’re not hitting errors, I hope these help you understand how your DGX Spark and vLLM work together underneath.
To be clear, none of this is meant to be flippant — I’m forever grateful to the open-source community, and I think both NVIDIA and vLLM are doing amazing work. I’m no expert, so if you spot a technical mistake in either post, please reach out. I’d love to learn from it.