Multiple Docker Build to Compare vLLM version

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

I have a fairly old and working vLLM image with MTP for Intel Autoround working. The vLLM image was built from a now 2 month old eugr’s spark-vllm-docker. The vLLM version is 0.19

I want to experience what the latest vLLM does for spark. However since I cannot pin an older release of eugr’s git repo, are there ways I can keep my current working vLLM built? Like cloning to a separate location on the SSD and build from there?

2

Try to tag you already build image so to be able to refer to it.

Can you recognize it when issuing:

docker image ls

3

Yeah I saved my old image and pulled latest eugr’s vLLM. The results for the same Qwen3.5 122B auto-round is terrible:

Old vLLM 0.19 results

--- Run 1 ---

Testing Q&A... 256 tokens in 6.902465764s = 37.0 tok/s

Testing Code... 512 tokens in 12.405148885s = 41.2 tok/s

Testing JSON... 220 tokens in 5.217208787s = 42.1 tok/s

--- Run 2 ---

Testing Q&A... 256 tokens in 7.114789215s = 35.9 tok/s

Testing Code... 512 tokens in 12.496776833s = 40.9 tok/s

Testing JSON... 252 tokens in 6.109955064s = 41.2 tok/s

New vLLM 0.23 results:

--- Run 1 ---

Testing Q&A... 256 tokens in 7.861930100s = 32.5 tok/s

Testing Code... 512 tokens in 13.841316755s = 36.9 tok/s

Testing JSON... 227 tokens in 6.133877232s = 37.0 tok/s

--- Run 2 ---

Testing Q&A... 256 tokens in 7.893144882s = 32.4 tok/s

Testing Code... 508 tokens in 13.842084409s = 36.6 tok/s

Testing JSON... 220 tokens in 5.834997968s = 37.7 tok/s

The old vLLM uses about 104GB unified RAM after fully loaded. With exact same settings, the new vLLM use about 120GB! So performance regression and memory footprint regression. Terrible on both!