This is my first time using Ascent GX10. On Hugging Face, there is already an NVFP4 model for Gemma-4 (nvidia/Gemma-4-31B-IT-NVFP4). How can I deploy it on Docker using vLLM? Thank you everyone for listening.

Yeah, I was traveling and a few things broke in the build pipeline. They have been fixed, and I’ll look into gemma 4 tool parser patch issue.

i am playing with latest https://github.com/johnnynunez/vllm/commits/main/ eugr repo seems to have something something for nemotron nvfp4 that might be useful to play with else wait for offical support there. Nothing wrong going down the rabbit hole with claude and that link johnny repo either for…

If you haven’t already, I would take a look at the messages in this topic [image] Gemma 4 Models - which vLLM version? Any PRs spotted? DGX Spark / GB10 Maybe it’s just to early to test, but the official announcement mentions vLLM also in its list of inference…

Our community Docker should be able to run that NVFP4 model just fine, however since it’s a dense model, I’d recommend to stick to 26B one instead, and we have a recipe for it: GitHub - eugr/spark-vllm-docker: Docker configuration for running VLLM on dual DGX Sparks · GitHub Having said that, I’m d…

So, you can run with the most recent build from the community Docker. Don’t forget to pull and rebuild the TF5 image (git pull && ./build-and-copy.sh -t vllm-node-tf5 --tf5). ./launch-cluster.sh --solo -t vllm-node-tf5 exec vllm serve nvidia/Gemma-4-31B-IT-NVFP4 \ --max-model-len auto \ --gpu-m…

Intel Autoround quant is only marginally better on a single Spark (and worse pp): model test t/s peak t/s ttfr (ms) est_ppt (ms) e2e_ttft (ms) Intel/gemma-4-31B-it-int4-AutoRound pp2048 822.54 ± 16.25 2497.11 ± 49.35 2491.64 ± 49.35 2497.32 ± 49.37 Intel/gemma-4-31B-it-int4-AutoRound tg3…

I vibe coded a simple Chainlit chat + image + video UI in Pinokio to test Gemma-4, I figured everyone post stats I want to show you how it runs, the first section of the video is regular speed the second pass i sped it up because as you can tell its not the fastest, I think I have room to twea…

2.5 t/s seems very slow, i get 12t/s on mine with the same model.

I created the identical app using Gemma-4-26B-A4B-IT and it’s much faster, though someone posted on reddit that they were getting 45-60 tok/s. but even with 10-12 tok/s running it on an application is not bad, if I can get it to 20 tok/s this is usable in applications: Used this as a guide, nee…

Or you can use our community Docker at GitHub - eugr/spark-vllm-docker: Docker configuration for running VLLM on dual DGX Sparks · GitHub And then run this recipe: ./run-recipe.sh gemma4-26b-a4b --solo It will run the same model at ~40 t/s on a single Spark.

Hi, @eugr , thank you for the great job! But it seems like latest build has some flaws. Just pulled and rebuild the TF5 image (git pull && ./build-and-copy.sh -t vllm-node-tf5 --tf5). ~/spark-vllm-docker$ ./run-recipe.sh gemma4-26b-a4b --solo --setup Recipe: Gemma4-26B-A4B vLLM serving Gemma4-26…

How to run Gemma-4-NVFP4 in vLLM Docker?

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

blainesworld April 3, 2026, 11:12pm 4

If you haven’t already, I would take a look at the messages in this topic

Regarding NVFP4, while it’s supported, we still don’t have great performance with it. Lots of work to do. FP8 or INT4 will likely perform better until the issues are sorted

Topic		Replies	Views
Gemma 4 Models - which vLLM version? Any PRs spotted? DGX Spark / GB10 nim , llama	178	9949	April 30, 2026
Google Gemma 4 - It will work on DGX Spark? DGX Spark / GB10 agentic-ai	23	2372	April 19, 2026
"vLLM + Gemma 4 on NVIDIA DGX Spark GB10" - has anyone testing this implementation? DGX Spark / GB10	1	476	April 29, 2026
Gemma 4 -- here we go again DGX Spark / GB10	11	2908	April 15, 2026
Gemma 4 31B on DGX Spark: Runtime FP8 Benchmarks — Single & Dual Node (TP=2) DGX Spark / GB10 llama , agentic-ai	0	1919	April 7, 2026
Docker container image for recent vLLM release that enables GGUF loading Docker and NVIDIA Docker	5	1009	April 14, 2026
Does anyone have Gemma 4 31B running on Spark DGX? DGX Spark / GB10	9	2430	April 23, 2026
GLM-4.7-Flash-NVFP4 was just released, but for Transformers 5.0 + vLLM 0.14...? DGX Spark / GB10	90	4371	February 27, 2026
Gemma 4 MTP DGX Spark / GB10	3	1471	May 5, 2026
46tok/s with RedHatAI/gemma-4-26B-A4B-it-NVFP4 DGX Spark / GB10 llama	18	1303	May 6, 2026

How to run Gemma-4-NVFP4 in vLLM Docker?

Related topics