Performance test: unsloth/gemma-4-26B-A4B-it-NVFP4 vs nvidia/Gemma-4-26B-A4B-NVFP4

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10 Projects
1

I have been evaluating the new Gemma 4 26B A4B NVFP4 models with vllm and wanted to share some real-world serving results comparing the RTX Blackwell 6000 Pro against the DGX Spark platform.

The tests were performed using both:

  • unsloth/gemma-4-26B-A4B-it-NVFP4
  • nvidia/Gemma-4-26B-A4B-NVFP4

with identical vllm serving parameters and benchmark settings.

vllm server:

vllm serve nvidia/Gemma-4-26B-A4B-NVFP4 \
  --host 0.0.0.0 --port 8000 \
  --kv-cache-dtype fp8 \
  --max-model-len 65536 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 4096 \
  --gpu-memory-utilization 0.85 \
  --reasoning-parser gemma4 \
  --enable-prefix-caching --trust-remote-code


vllm bench serve \
  --model nvidia/Gemma-4-26B-A4B-NVFP4 \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 1000 \
  --request-rate 100 \
  --num-prompts 100 \
  --ignore-eos

unsloth/gemma-4-26B-A4B-it-NVFP4

#Blackwell 6000 Pro 

============ Serving Benchmark Result ============
Successful requests:                     100
Failed requests:                         0
Request rate configured (RPS):           100.00
Benchmark duration (s):                  94.45
Total input tokens:                      100000
Total generated tokens:                  100000
Request throughput (req/s):              1.06
Output token throughput (tok/s):         1058.79
Peak output token throughput (tok/s):    1176.00
Peak concurrent requests:                100.00
Total token throughput (tok/s):          2117.59
---------------Time to First Token----------------
Mean TTFT (ms):                          42477.29
Median TTFT (ms):                        44040.83
P99 TTFT (ms):                           87065.66
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          7.08
Median TPOT (ms):                        7.13
P99 TPOT (ms):                           7.26
---------------Inter-token Latency----------------
Mean ITL (ms):                           7.08
Median ITL (ms):                         7.03
P99 ITL (ms):                            9.35
==================================================


# DGX Spark



============ Serving Benchmark Result ============
Successful requests:                     100
Failed requests:                         0
Request rate configured (RPS):           100.00
Benchmark duration (s):                  625.89
Total input tokens:                      100000
Total generated tokens:                  100000
Request throughput (req/s):              0.16
Output token throughput (tok/s):         159.77
Peak output token throughput (tok/s):    224.00
Peak concurrent requests:                100.00
Total token throughput (tok/s):          319.54
---------------Time to First Token----------------
Mean TTFT (ms):                          267128.25
Median TTFT (ms):                        269571.93
P99 TTFT (ms):                           582529.76
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          47.14
Median TPOT (ms):                        49.97
P99 TPOT (ms):                           53.48
---------------Inter-token Latency----------------
Mean ITL (ms):                           47.14
Median ITL (ms):                         49.39
P99 ITL (ms):                            54.84
==================================================

nvidia/Gemma-4-26B-A4B-NVFP4

#Blackwell 6000 Pro 

============ Serving Benchmark Result ============
Successful requests:                     100
Failed requests:                         0
Request rate configured (RPS):           100.00
Benchmark duration (s):                  110.88
Total input tokens:                      100000
Total generated tokens:                  100000
Request throughput (req/s):              0.90
Output token throughput (tok/s):         901.84
Peak output token throughput (tok/s):    992.00
Peak concurrent requests:                100.00
Total token throughput (tok/s):          1803.69
---------------Time to First Token----------------
Mean TTFT (ms):                          49993.02
Median TTFT (ms):                        52030.02
P99 TTFT (ms):                           102256.80
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          8.30
Median TPOT (ms):                        8.31
P99 TPOT (ms):                           8.56
---------------Inter-token Latency----------------
Mean ITL (ms):                           8.30
Median ITL (ms):                         8.22
P99 ITL (ms):                            10.59
==================================================


# DGX Spark

============ Serving Benchmark Result ============
Successful requests:                     100
Failed requests:                         0
Request rate configured (RPS):           100.00
Benchmark duration (s):                  779.98
Total input tokens:                      100000
Total generated tokens:                  100000
Request throughput (req/s):              0.13
Output token throughput (tok/s):         128.21
Peak output token throughput (tok/s):    184.00
Peak concurrent requests:                100.00
Total token throughput (tok/s):          256.42
---------------Time to First Token----------------
Mean TTFT (ms):                          339880.92
Median TTFT (ms):                        351526.18
P99 TTFT (ms):                           724149.62
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          58.67
Median TPOT (ms):                        60.60
P99 TPOT (ms):                           64.45
---------------Inter-token Latency----------------
Mean ITL (ms):                           58.67
Median ITL (ms):                         59.88
P99 ITL (ms):                            65.06
==================================================

The Unsloth variant achieved approximately:

  • 17.4% higher output throughput
  • 17.4% higher total throughput
  • 15% lower TPOT
  • 15% faster TTFT

RTX Blackwell 6000 Pro delivered roughly 6-7X higher throughput than DGX Spark.

3 Likes