As baristankut and eugr shown in their posts, and after a lot of benchmarking and digging into the architecture, here’s what i like to summarize (llm formated, no molt). FP16 → FP8: scales. Half the bytes, tensor cores handle it natively via the Ampere MMA path, real-world speedup ~1.3–1.7x. Solid. …

[image] We unlocked NVFP4 on the DGX Spark: 20% faster than AWQ! DGX Spark / GB10 After months of iteration and research, we have finally filled in the gap of NVFP4 not being leveraged. The results speak for themselves. We are working with Nvidia to include thi…

We don’t know when this will be resolved or how we can revert to the previous firmware.

Have you tried joining the DGX using PP to another cluster, Nvidia video card, load balancing by actual FLOPs?

all of my test worked with socket and roce. there was a vllm head on dgx and a vllm worker on pgx. i havent mixed rtx and dgx/pgx because they are not in the same network. rtx is located in the datacenter. “load balancing” doesnt fit the term. its about load distribution, as i summarized some post…

ever tried tuning with NCCL_ALGO=Tree (statt Ring) — bei 2 Nodes potentiell besser NCCL_BUFFSIZE — größere Buffer für weniger Roundtrips NCCL_NTHREADS — mehr Threads pro Collective

I haven’t tried it, but I’m thinking of using a log to detect bottlenecks. I’m really doing my best to learn from you all.

Before tuning NCCL env vars, I’d run nccl-tests (all_reduce_perf) outside of inference first — gives you a clean baseline so you know if the issue is network config or vLLM overhead. See NCCL for Two Sparks | DGX Spark for NCCL tests. I also have nccl tests prebuilt into all of my images.

Unlocked NVFP4. Ok. (Math) accuracy, drafters at models accuracy and throughput too? Three in one? … It was not the era of “Use 4-bit AWQ over NVFP4.” By the way, Marlin + INT4 are simply faster and show more accuracy.

A new way beside FlashInfer JIT, more native, has the chance to be faster than FlashInfer’s multi-arch implementation. But does it compete with Marlin + INT4 + AutoRound? The work is impressive even because no one plans to go this deep. Finally its a new quantization method: #if defined(__CUDA_ARC…

Qwen3-Coder-30B NVFP4 with CUTLASS 4.2.1 and Avaroks Patches: Context short medium long ctx=0 39.1 67.5 68.0 ctx=512 28.2 66.6 67.4 ctx=2K 19.8 65.4 66.1 ctx=8K 6.1 60.5 61.9 ctx=16K 3.6 55.9 57.2 it is faster than Flashinfer CUTLASS and delivers better accuracy does not compet…

Eagle3 on vllm native CUTLASS: Qwen3-Coder-30B NVFP4 : +EAGLE3 NST=1: Context short medium long ctx=0 33.3 56.8 73.9 ctx=512 21.6 54.8 71.4 ctx=2K 15.9 53.3 70.8 ctx=8K 6.0 46.7 48.4 ctx=16K 3.6 44.6 45.9 Math: 78% Vergleich ctx=0 long (Hauptmetrik): Engine MoE-Kernel V…

FP4 on DGX Spark — Why It Doesn't Scale Like You'd Expect

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

cosinus February 20, 2026, 7:21pm 89

Something to test for the weekend as it seems.

Topic		Replies	Views
We unlocked NVFP4 on the DGX Spark: 20% faster than AWQ! DGX Spark / GB10	144	11346	March 14, 2026
PSA: State of FP4/NVFP4 Support for DGX Spark in VLLM DGX Spark / GB10	234	16030	May 15, 2026
Comprehensive Qwen3.8-27B Study on DGX Sparks: Quantization, Speculative Decoding, and TP/DP Scaling DGX Spark / GB10 Projects	10	5694	August 26, 2026
Help: Running NVFP4 model on 2x DGX Spark with vLLM + Ray (multi-node) DGX Spark / GB10 mistral-large	18	3119	December 25, 2025
I am EXTREMely disappointed with the current state of DGX Spark DGX Spark / GB10	91	30044	June 24, 2026
Two multi-node DGX Spark wins: RoCE 2× inference throughput + Qwen3.5-397B-A17B-NVFP4 serving (with SM121 CUTLASS patch) DGX Spark / GB10 Projects	5	1570	July 21, 2026
Two-Spark cluster with vLLM using tensor-parallel-size 2 causes one node to drop while the other's GPU goes 100% forever DGX Spark / GB10	36	2681	February 13, 2026
Qwen3.8-27B on DGX Spark using vllm: NVFP4 vs FP8 performance DGX Spark / GB10 llm	8	5301	August 24, 2026
Let's optimize nvidia/GLM-5.3-Flash-NVFP4 for 2× DGX Spark DGX Spark / GB10	181	9076	October 2, 2026
Why 273 GB/s? Less Is More, Until It Isn’t DGX Spark / GB10	67	3833	March 27, 2026

FP4 on DGX Spark — Why It Doesn't Scale Like You'd Expect

Related topics