Failed to run tp or pp on two-nodes ray cluster using docker vllm:25.11

[terryzhv83]

two DGX spark connected by qsfp56 cable,

docker pull from nvidia, vllm 25.11

failed to run vllm serve on either tp=2 or pp=2, logs attached.

failed-on-vllm-25.11.log (43.6 KB)

[raphael.amorim]

It looks like nccl problem. I would just ignore the NVIDIA instructions on this for now. They’re not functional. Please go through these threads and save yourself some time. Lots of discussions already happened and lots of other insights you don’t need to find out by yourself:

Install and Use vLLM for Inference on two Sparks does not work

I noticed the same thing. librdmacm1 is installed on the Spark but isn’t visible in the container. I ran the install inside the container but of course it would be vaporized when the ray-head is rebuilt. So it probably should be added to the docker file.

vLLM on dual sparks

Look into this thread. There’s a ton of information and insights and code about using vLLM and Ray on stacked sparks, including some improvements on loading times:

This repo has a simple, but functional setup for a vLLM cluster with Ray support:

GitHub - eugr/spark-vllm-docker: Docker configuration for running VLLM on dual DGX...