Hi all, If you’ve been running FP8 models on vLLM, you’ve probably seen these warnings during startup: WARNING Using default W8A8 Block FP8 kernel config. Performance might be sub-optimal! WARNING Using default MoE config. Performance might be sub-optimal! vLLM’s Triton kernels for MoE expert dis…

Witchcraft!!! 🧙♂️ This community is insane… so many talents. I am deeply impressed! Thank you! Looking forward to test it. 🚀

I haven’t looked into the underpinnings of sparkrun enough to know whether that’s supported here by way of the existing docker recipe support.. Any guidance on what the workflow would be there?

It looks like sparkrun offers a very similar functionality – I haven’t tested it but this may get you similar optimization/the same for MoE, however it doesn’t seem to offer the FP8 tuning counterpart that vLLM-Tune brings to the table.

Is this something everybody should have to do, or is there a way to have these better defaults just included upstream so everyone gets them by default? (I don’t understand the issue well enough to know if everyone is producing the same file here, or if they are device/model specific in some way)

Technically these could be provided via spark-vllm-docker or even vLLM itself. Given vLLM cannot support a whole catalogue of configurations for various TP (and potential EP or PP) combinations, chances are that people have to accept living with the default values or will have to rely on benchmarkin…

Well… Just discover this today and here’s my before/after of my Qwen3.6-35B-A3B-FP8 with DFlash before and after: [image] After vLLM Tune: [image] It’s like way more consistent across different scenarios. I like it!

Glad it works for you, too! The process takes a little long but the results are worth it.

Great project, please tell me if you have any plans for configuring other models for tp=2, for example Qwen3.6-27b-fp8?

The whole idea is that people can run it to create their own configs – you won’t have to rely on me. I may create a few more configs over time but can’t guarantee to cover all models – I only have two sparks and can’t use them while they generate the configuration. I am super happy to accept PRs wi…

Does the tp=x make a difference for the final result? Basically, can i use the Qwen3.6 that was done for TP=2 on TP=1, or do they actually differ?

Introducing vLLM-Tune — Kernel tuning CLI for vLLM on DGX Spark

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10 Projects

azampatti April 29, 2026, 5:14am 9

Well… Just discover this today and here’s my before/after of my Qwen3.6-35B-A3B-FP8 with DFlash before and after:

After vLLM Tune:

It’s like way more consistent across different scenarios. I like it!

DDTree plus diffusion drafting (DFlash) to optimize GB10

Topic		Replies	Views
Qwen3.5-122B-A10B on single Spark: up to 51 tok/s (v2.1 — patches + quick-start + benchmark) DGX Spark / GB10 cuda , performance , docker , performance-tuning , llm	434	31949	June 24, 2026
New bleeding-edge vLLM Docker Image: avarok/vllm-nvfp4-gb10-sm120 DGX Spark / GB10 Projects	32	3858	December 17, 2025
50%+ Improvement on spark?! DGX Spark / GB10 cuda , deepseek	25	3092	March 24, 2026
Can someone please just help me set the DGX Spark up for optimal LLM use? DGX Spark / GB10 llama	11	2914	June 20, 2026
Moving from Mac to NVIDIA: bought powerful hardware, but drowning in configs DGX Spark / GB10 llama , nemotron	37	3726	February 25, 2026
Running nvidia/Qwen3.8-Flash-Next-NVFP4 on a single DGX Spark DGX Spark / GB10	14	2970	September 19, 2026
vLLM on GB10: gpt-oss-120b MXFP4 slower than SGLang/llama.cpp... what’s missing? DGX Spark / GB10	143	10444	February 24, 2026
Install and Use vLLM for Inference on two Sparks does not work DGX Spark / GB10	159	7797	December 9, 2025
DGX Spark performance DGX Spark / GB10	49	8523	February 13, 2026
Benchmark Report: Qwen3.6-35B-A3B-NVFP4 on NVIDIA DGX Spark, Jetson Thor, Blackwell 6000 Pro DGX Spark / GB10 Projects	12	6568	August 21, 2026

Introducing vLLM-Tune — Kernel tuning CLI for vLLM on DGX Spark

Related topics