I’m new hear and new to getting AI running locally on a NVIDIA Spark. I’m trying to get Gemma-4 31B to run. I’ve tried a couple versions and keep running into errors. Does anyone have this working correctly and can give me the commands to do it? I have nvidia/Gemma-4-31B-IT-NVFP4 downloaded. This i…

@aniculescu , Thanks for the reply. My process to get it running was round-about. I started with an AI telling me how to do it and that did not work well. I finally did get to the VLLM playbook and got it working. I am hoping for better than 6.7 tokens per second of throughput. That is not usable. I…

Here’s the command line I’m using: scott@spark:~$ docker run -it --rm --name gemma4-31b –gpus all -p 8000:8000 –ipc=host -v /home/scott/.cache/huggingface:/root/.cache/huggingface -e HF_TOKEN=$HF_TOKEN vllm/vllm-openai:gemma4-cu130 nvidia/Gemma-4-31B-IT-NVFP4 –max-model-len 65536 –gpu-memo…

Our VLLM playbook can help with running on Spark: vLLM for Inference | DGX Spark However, that command looks right. What errors are you seeing?

Does it have to be 31B? 26B-A4B is going to be a lot faster on DGX Spark due to its MoE architecture. For that model the community vLLM image that @eugr manages has a recipe which works well: [image] GitHub - eugr/spark-vllm-docker: Docker configuration for running VLLM on dual DGX..…

@haidij Thanks for the reply. I’ll try that next. I need something with serious coding capabilities, very close to the Pro models out there and I’m skeptical. But it’s next on the roster. Scott

Welcome to the community. You can also use sparkrun to automate running recipes from @eugr ’s spark-vllm-docker . It helps you get setup, apply best practices, and then focus on doing stuff instead of how to make it work. @eugr , @raphael.amorim , and myself are also the maintainers of Spark Arena whe…

I installed LM Studio on the Spark today and got the Q4_K_M model running at about 10-11 tok/sec [image] lmstudio-community/gemma-4-31B-it-GGUF · Hugging Face We’re on a journey to advance and democratize artificial intelligence through open source and open science. …

Does anyone have Gemma 4 31B running on Spark DGX?

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

scott264 April 5, 2026, 10:20pm 2

I got this thing working to the tune of 7 tokens per second which is about twice what it was doing. Is that all we expect to get out of Gemma-4 on a Spark?

Topic		Replies	Views
Google Gemma 4 - It will work on DGX Spark? DGX Spark / GB10 agentic-ai	22	3121	April 5, 2026
Gemma 4 -- here we go again DGX Spark / GB10	11	3763	April 15, 2026
Someone post this: Gemma 4 26B-A4B MoE running at 45-60 tok/s on DGX Spark DGX Spark / GB10	4	3270	April 5, 2026
Slow inference with 31b model Gemma 4? Optimizations? DGX Spark / GB10	21	6710	June 11, 2026
Gemma 4 Models - which vLLM version? Any PRs spotted? DGX Spark / GB10 nim , llama	177	15472	April 16, 2026
How to run Gemma-4-NVFP4 in vLLM Docker? DGX Spark / GB10	11	6643	April 12, 2026
Gemma 4 31B on DGX Spark: Runtime FP8 Benchmarks — Single & Dual Node (TP=2) DGX Spark / GB10 llama , agentic-ai	0	3534	April 7, 2026
Gemma 4 Day-1 Inference on NVIDIA DGX Spark — Preliminary Benchmarks DGX Spark / GB10 llama , agentic-ai	17	10835	April 7, 2026
[Guide] Uncensored Gemma-4-26B at 45 tok/s on DGX Spark — Actually Feels Great to Use! DGX Spark / GB10 Projects openclaw	9	5646	April 20, 2026
"vLLM + Gemma 4 on NVIDIA DGX Spark GB10" - has anyone testing this implementation? DGX Spark / GB10	1	704	April 29, 2026

Does anyone have Gemma 4 31B running on Spark DGX?

Related topics