Hi, I know Qwen is the most chatted and loved model but for my specific use case Gemma4 performs better.
- eugr/spark-vllm-docker after 1/2 hours start looping (timeout/fastapi hang).
- vllm/vllm-openai:gemma4-cu130 runs for days/weeks.
Model: google/gemma-4-26B-A4B-it (from huggingface).
eugr/sprk-vllm-docker is run via ./run-recipe.sh gemma4-26b-a4b --solo
vllm/vllm-openai:gemma4-cu130 is run via
docker pull vllm/vllm-openai:gemma4-cu130
docker run
-it -d –rm –gpus all
-p 8000:8000
-e HUGGING_FACE_HUB_TOKEN
-v ~/.cache/huggingface:/root/.cache/huggingface
vllm/vllm-openai:gemma4-cu130 “${HF_MODEL_HANDLE}”
–host 0.0.0.0
–port 8000
As you can see, each are started with very standard paramethers.
Asking to the nerdiest people here… Where do you will search for inside my machine files/configuration to spot the issue?
eugr/spark-vllm-docker is faster but not reliable.
My first idea is to try to rebuild it using an old vllm release but spark-vllm-docker uses precopiled wheels that are download from github and suspect that will make me get unrielable test.
I need a push!