Run VLLM in Spark Install uv curl -LsSf https://astral.sh/uv/install.sh | sh Create environment sudo apt install python3-dev python3.12-dev uv venv .vllm --python 3.12 source .vllm/bin/activate Install Pytorch & Install flashinfer and triton uv pip install torch torchvision torchaudio --inde…

I just ran the updated instructions in top-level post as of 1/16 and it seems something has changed, likely a small bump in some PyTorch dependencies which the specific vLLM wheel now tries to update. Specifically, uv installs torch etc properly from the cu130 URL but then when the specific vLLM wh…

Hi, sorry, I accidentally deleted the previous message. Anyway, for convenience, I built and pushed the image generated by @eugr ’s magnificent work to DockerHub. The tag is: isnob46/dgx-vllm-cu13:014 updated to the vllm 0.14.0 release that came out yesterday. I also have a question: has anyone ma…

Helpful information, thanks for sharing. It can help us avoid mistakes while setting up vLLM.

Can I concurrently run both oss-20B and oss-120B? I’m currently utilizing TensorRT for simultaneous execution, but I’m experiencing significant performance degradation.

You would have to launch two instances of vllm on different ports. Would that work for you?

You can run two separate instances of vLLM on different ports, like @PrinceHal suggested, just make sure you set --gpu-memory-utilization accordingly, so both take less than 0.9 of total VRAM (e.g. 0.2+0.7). If you want to access both on a single port, or launch/unload on demand, you can try llama-…

Hey, can you please update with the instruction to run on vllm with spark GB10(in virtual env)? currently I am hitting this error after when sending the curl request: (EngineCore pid=419501) out, softmax_lse = torch.ops._vllm_fa2_C.varlen_fwd( (EngineCore pid=419501) ^…

At this point, I’d suggest to just use our community Docker: GitHub - eugr/spark-vllm-docker: Docker configuration for running VLLM on dual DGX Sparks · GitHub It is battle tested, has a CI pipeline that generates nightly wheels and runs the resulting image through a series of regression tests. We …

Run VLLM in Spark

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10

christopher_owen January 31, 2026, 9:59am 146

Not sure if it would be helpful for you, but you could give this a shot for improving performance:

https://forums.developer.nvidia.com/t/vllm-on-gb10-gpt-oss-120b-mxfp4-slower-than-sglang-llama-cpp-what-s-missing/

Topic		Replies	Views
vLLM container out of date for new models DGX Spark / GB10	9	2004	October 31, 2025
vLLM containers DGX Spark / GB10	45	2636	July 22, 2026
Install and Use vLLM for Inference on two Sparks does not work DGX Spark / GB10	159	5878	December 9, 2025
vLLM on GB10: gpt-oss-120b MXFP4 slower than SGLang/llama.cpp... what’s missing? DGX Spark / GB10	143	8047	February 24, 2026
I'd like to learn how to use the latest vLLM on DGX Spark DGX Spark / GB10 cuda	9	2484	November 29, 2025
GLM-4.7-Flash-NVFP4 was just released, but for Transformers 5.0 + vLLM 0.14...? DGX Spark / GB10	89	4824	February 13, 2026
New pre-built vLLM Docker Images for NVIDIA DGX Spark DGX Spark / GB10	73	9692	March 27, 2026
VLLM -- the $150M train wreck? DGX Spark / GB10 llama	24	1591	February 27, 2026
HOW-TO: Run Qwen3-Coder-Next on Spark DGX Spark / GB10 llama	92	10952	March 24, 2026
Running Mistral Small 4 119B NVFP4 on NVIDIA DGX Spark (GB10) DGX Spark / GB10 deepseek	66	5546	July 22, 2026

Run VLLM in Spark

Related topics