Gemma 4 just released. Obviously Google has “collaborated” with NVIDIA on the release. Google’s announcements boasts all kinds of zero day support on multiple backends. NVIDIA is handy with the press release RTX to Spark: Gemma 4 Accelerated for Agentic AI | NVIDIA Blog . – and if you read down t…

It’s just up. :-D Missing the announcement for vLLM, yet. But the community zero day support squad has already started its work: [image] Gemma 4 Models - which vLLM version? Any PRs spotted? DGX Spark / GB10 This is from Luciano on the DeepMind DevRel team: f…

Nvidia has an official NVFP4 for the 31B and suggests serving with vLLM nvidia/Gemma-4-31B-IT-NVFP4

We have just updated our vLLM and llama.cpp playbooks to use Gemma4 on DGX Spark. Check them out here: VLLM , Llama.cpp

That’s a pleasant surprise. Thank you.

Hello everyone, I’m new to running AI models locally. I just got my hands on nvdia DGX spark. I run gemma 4 31b using Ollama. It works but is sluggish when I use it as model provider for my Hermes agent. Are there better (more performant) ways to run Gemma 4 on DGX Spark. Or are there model varian…

I use this for my Hermes agent(on a separate PC): GitHub - albond/DGX_Spark_Qwen3.5-122B-A10B-AR-INT4: Qwen3.5-122B-A10B on DGX Spark: 28.3 → 51 tok/s (+80%) · GitHub You can get 50 tok/s

Dense models are the Spark’s kryptonite. The architecture trades off memory capacity for bandwidth. Gemma 4 26b-a4b will run much faster, as will GPT-OSS 120b, Qwen 3.5 122b, etc. Basically, MoE models are much better suited for the design of the Spark.

Been using the exact same also. It is a very goot combo. Hermes + Qwen3.5-122B Int4 on the DGX work wonders.

Thanks a lot for the tip! I’ll definitely give this a shot. What tool does it use to run the model? Is it Ollama too? From the README.md I see it uses docker. Does this imply Ollama? Does it have openai-like API built in so that I can just point Hermes to it or does it require additional setup?

Gemma 4 -- here we go again

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

phrogzy April 2, 2026, 11:38pm 6

Some numbers for Ollama and Gemma4 (added Qwen3.5 to have a value to compare with):

qwen3.5:35b-a3b-q4_K_M, response token/s: 59.68

qwen3.5:122b-a10b-q4_K_M, response token/s: 24

gemma4:26b-a4b-it-q4_K_M, response token/s: 58.67

gemma4:31b-it-q4_K_M, response token/s: 10.34

Topic		Replies	Views
Google Gemma 4 - It will work on DGX Spark? DGX Spark / GB10 agentic-ai	22	2959	April 5, 2026
How to run Gemma-4-NVFP4 in vLLM Docker? DGX Spark / GB10	11	6312	April 12, 2026
Does anyone have Gemma 4 31B running on Spark DGX? DGX Spark / GB10	8	3508	April 9, 2026
"vLLM + Gemma 4 on NVIDIA DGX Spark GB10" - has anyone testing this implementation? DGX Spark / GB10	1	665	April 29, 2026
Gemma 4 Models - which vLLM version? Any PRs spotted? DGX Spark / GB10 nim , llama	177	13684	April 16, 2026
Gemma 4 Day-1 Inference on NVIDIA DGX Spark — Preliminary Benchmarks DGX Spark / GB10 llama , agentic-ai	17	9924	April 7, 2026
Someone post this: Gemma 4 26B-A4B MoE running at 45-60 tok/s on DGX Spark DGX Spark / GB10	4	3074	April 5, 2026
Gemma 4 31B on DGX Spark: Runtime FP8 Benchmarks — Single & Dual Node (TP=2) DGX Spark / GB10 llama , agentic-ai	0	3235	April 7, 2026
Newb alert! Qwen 3.5/3.6 Gemma 4 26B / 35B downloading and speed! Help! DGX Spark / GB10	2	1549	May 5, 2026
Slow inference with 31b model Gemma 4? Optimizations? DGX Spark / GB10	21	6087	June 11, 2026

Gemma 4 -- here we go again

Related topics