NVIDIA/Qwen3.8-Flash-Next-NVFP4 + DFlash2 on a Single DGX Spark ( ~57 tok/s )

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

Hi everyone!

I’ve been working on a vLLM recipe to run NVIDIA’s Qwen3.8-Flash-Next-NVFP4 on a single DGX Spark (128 GB), with added support for DFlash2 speculative decoding.

I’d love for other DGX Spark / GB10 owners to give it a try and share their benchmarks!

The problem

NVIDIA provides two FP4 checkpoints:

The official DGX Spark recipe uses FP4-Local, since the original NVFP4 checkpoint is too large to fit comfortably in 128 GB while leaving enough memory for the KV cache.

The solution

GitHub: GitHub - roozich/Qwen3.8-Flash-Spark: vLLm recipe for running Qwen3.8-Flash-Next on a single DGX Spark or RTX Spark family + DFlash2 support · GitHub

The recipe uses NVMe-backed memory mapping to offload the large PLE tables, allowing the NVFP4 model to run on a single Spark with usable KV cache capacity.

I also patched vLLM to support DFlash2 speculative decoding using the incoai DFlash2 drafter.

The repo includes an optional optimized hybrid layout, prefix caching, GB10 fixes, and different performance profiles.

Performance so far

Configuration Single-stream decode
Default MTP ~45.2 tok/s
DFlash2 ~57.0 tok/s

That’s around 26% faster decoding with DFlash2 in our tests.

How to try it

Default MTP:

./flash doctor
./flash setup
./flash serve

DFlash2:

./flash doctor dflash
./flash setup dflash
./flash serve dflash

Looking for community benchmarks!

If you have a DGX Spark, ASUS Ascent GX10, or similar GB10 hardware, I’d really appreciate your feedback!

I’m especially curious about:

  • Can you reproduce or beat 57 tok/s?
  • How are prefill speed, TTFT, and long-context performance?
  • How does it compare with NVIDIA’s official FP4-Local recipe?

Feel free to share your results, setup details, bugs, or optimization ideas. Even slower results are useful!

Repo: GitHub - roozich/Qwen3.8-Flash-Spark: vLLm recipe for running Qwen3.8-Flash-Next on a single DGX Spark or RTX Spark family + DFlash2 support · GitHub

Thanks, and happy benchmarking!

2

it is using RadixArk isn’t it? Is there a possibility to use Qwen3.8-Flash-Next-FP4-Local?
when it comes to quality I have best results with it

3

Thanks for this. I didn’t know about the offical recipe page for some reason. Sparkun give me fits every now and then.

4

I’ve been working on a vLLM recipe to run NVIDIA’s Qwen3.8-Flash-Next-NVFP4 on a single DGX Spark (128 GB), with added support for DFlash2 speculative decoding.

Careful, you may want to rephrase to make clear that you forked blazux’s work and added a DFlash2 drafter model.

It’s also not clear what your benchmarks are based on. You tested an old version of blazux’s YARN configuration against your DFlash2? What was the workload?

I’m trying not to be overly distrustful, but this is your first and only post on this forum and you’re trying to get people to run your code right off the bat.

Nvidia providing two checkpoints is not the problem. In fact, the *-Local model is the younger of the two and it exists to open up more headroom for KV. It doesn’t change anything about serving PLE from disk.

5

Yes, it supports the RadixArk checkpoint with MODE=hybrid-mtp. Please refer to the configuration documentation for more details.

6

For your concern please check PR #62 and also Credits section.

This is an independent fork to focus more on DFlash benchmarking for different checkpoints for both latency and throughput, improve the documentation, and add support for the RTX Spark family.

Your feedback and contributions are always welcome. Thanks!

7

the fp4-local support will come soon with new mode with eugr/spark-vllm-b12x:nightly image to be able bench easily