I think this approach worth a try: Gemma 4 26B-A4B MoE running at 45-60 tok/s on DGX Spark — here's how Spent half the night on getting google/gemma-4-26B-A4B-it running fast on a single NVIDIA DGX Spark (128GB unified memory, GB10 Blackwell). Some things I learned that might save other…

1 54.01 t/s 54.06 t/s 43.23 t/s 50.43 t/s 2 69.13 t/s 62.47 t/s 51.53 t/s 61.04 t/s 4 138.75 t/s 141.14 t/s 140.10 t/s 140.00 t/s

Can confirm, get around 50 tok/s pp

The community vllm container by @eugr works fine for this model too (there is a recipe): recipes%2Fgemma4-26b-a4b.yaml main # Recipe: Gemma4-26B-A4B # Gemma4-26B-A4B model in online FP8 quantization recipe_version: "1" name: Gemma4-26B-A4B description: vLLM serving Gemma4-26B-…

Someone post this: Gemma 4 26B-A4B MoE running at 45-60 tok/s on DGX Spark

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

lewald_jens April 4, 2026, 10:34am 4

Jup runs here with 45- 60

Topic		Replies	Views
46tok/s with RedHatAI/gemma-4-26B-A4B-it-NVFP4 DGX Spark / GB10 llama	18	1681	May 6, 2026
Gemma 4 Day-1 Inference on NVIDIA DGX Spark — Preliminary Benchmarks DGX Spark / GB10 llama , agentic-ai	17	9924	April 7, 2026
[Guide] Uncensored Gemma-4-26B at 45 tok/s on DGX Spark — Actually Feels Great to Use! DGX Spark / GB10 Projects openclaw	9	5092	April 20, 2026
Google Gemma 4 - It will work on DGX Spark? DGX Spark / GB10 agentic-ai	22	2959	April 5, 2026
Does anyone have Gemma 4 31B running on Spark DGX? DGX Spark / GB10	8	3508	April 9, 2026
Gemma 4 31B on DGX Spark: Runtime FP8 Benchmarks — Single & Dual Node (TP=2) DGX Spark / GB10 llama , agentic-ai	0	3235	April 7, 2026
Slow inference with 31b model Gemma 4? Optimizations? DGX Spark / GB10	21	6087	June 11, 2026
Gemma 4 -- here we go again DGX Spark / GB10	11	3574	April 15, 2026
Gemma 4 Models - which vLLM version? Any PRs spotted? DGX Spark / GB10 nim , llama	177	13684	April 16, 2026
Gemma4 Benchmarks on double DGX Sparks Ray Cluster and single DGX DGX Spark / GB10 Projects	2	881	April 6, 2026

Someone post this: Gemma 4 26B-A4B MoE running at 45-60 tok/s on DGX Spark

Related topics