As baristankut and eugr shown in their posts, and after a lot of benchmarking and digging into the architecture, here’s what i like to summarize (llm formated, no molt). FP16 → FP8: scales. Half the bytes, tensor cores handle it natively via the Ampere MMA path, real-world speedup ~1.3–1.7x. Solid. …

[image] We unlocked NVFP4 on the DGX Spark: 20% faster than AWQ! DGX Spark / GB10 After months of iteration and research, we have finally filled in the gap of NVFP4 not being leveraged. The results speak for themselves. We are working with Nvidia to include thi…

no script failure. podman setup not complete. i’ll add a README

Qwen/Qwen3-Coder-Next is on the hot top next DFLASH Drafters list! So according to the latest performance gains, 100 token/s are possible. only with sglang at the moment

bench.py test results with qwen3 coder next … [ vllm-marlin-sm12x/whgthomas/RESULTS.md at main · flash7777/vllm-marlin-sm12x · GitHub ] Context-Matrix (tok/s) Context short (20) medium (150) long (400) 0 36.3 69.4 70.3 1K 9.5 66.9 69.0 4K 7.6 65.3 67.5 8K 6.2 64.4 66.1 16K 3.6 60.4 6…

I previously ran into an error while building with Podman, but running the command: sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml should fix the issue.

Thanks for this - will try in your setup next. The atomicAdd is noticeable with opencode - where I am regularly seeing 96% cache hits

Done. You can launch like this: ./launch-cluster.sh -t vllm-node-20260224 \ --apply-mod mods/fix-qwen3-next-autoround \ -e VLLM_MARLIN_USE_ATOMIC_ADD=1 \ --solo \ exec vllm serve Intel/Qwen3-Coder-Next-int4-AutoRound \ --max-model-len 262144 \ --gpu-memory-utilization 0.7 \ --port 8888 --host 0.0.0…

I am currently quantising Qwen 3.5 → int4 AutoRound auto-round \ --model "Qwen/Qwen3.5-35B-A3B" \ --device "cuda" \ --scheme "W4A16" \ --output_dir "./autoround" \ --format "auto_gptq,auto_round" to see if I can get the new slightly smaller model running for inference with my w…

I did the git pull and ./build-and-copy.sh in preparation – blink and you miss it – is there anything else I need to do or was that it? Thanks again - super cool having this

FP4 on DGX Spark — Why It Doesn't Scale Like You'd Expect

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

eugr February 25, 2026, 2:23am 181

If you build from main vllm now, this model will be broken - see this post for details.
I’m about to push a mod as a temporary fix.

Topic		Replies	Views
We unlocked NVFP4 on the DGX Spark: 20% faster than AWQ! DGX Spark / GB10	144	10545	March 14, 2026
PSA: State of FP4/NVFP4 Support for DGX Spark in VLLM DGX Spark / GB10	234	15103	May 15, 2026
Comprehensive Qwen3.8-27B Study on DGX Sparks: Quantization, Speculative Decoding, and TP/DP Scaling DGX Spark / GB10 Projects	10	2360	August 26, 2026
Help: Running NVFP4 model on 2x DGX Spark with vLLM + Ray (multi-node) DGX Spark / GB10 mistral-large	18	2956	December 25, 2025
I am EXTREMely disappointed with the current state of DGX Spark DGX Spark / GB10	91	26752	June 24, 2026
Two multi-node DGX Spark wins: RoCE 2× inference throughput + Qwen3.5-397B-A17B-NVFP4 serving (with SM121 CUTLASS patch) DGX Spark / GB10 Projects	5	1437	July 21, 2026
Two-Spark cluster with vLLM using tensor-parallel-size 2 causes one node to drop while the other's GPU goes 100% forever DGX Spark / GB10	36	2419	February 13, 2026
Qwen3.8-27B on DGX Spark using vllm: NVFP4 vs FP8 performance DGX Spark / GB10 llm	8	4055	August 24, 2026
Why 273 GB/s? Less Is More, Until It Isn’t DGX Spark / GB10	67	3346	March 27, 2026
Marlin Fix: NVFP4 Actually Works on SM121 (DGX Spark) DGX Spark / GB10 Projects jetson , nemotron	15	3419	April 12, 2026

FP4 on DGX Spark — Why It Doesn't Scale Like You'd Expect

Related topics