Greetings to all, I’m trying to run various LLM inference engines (vLLM, TensorRT-LLM) on two RTX 5090 GPUs, specifically using tensor parallelism with a size of 2. Inference on a single GPU works fine, but I’m having trouble with two GPUs. Could someone confirm whether peer-to-peer (P2P) communic…

Fallback mechanism of NCCL, anyways issue was fixed with the release of the new version of NCCL. See details here: NCCL P2P issue using two RTX 5090 · Issue #1637 · NVIDIA/nccl · GitHub Thank you!

My setup: PyTorch version: 2.7.0a0+ecf3bae40a.nv25.02 Is debug build: False CUDA used to build PyTorch: 12.8 ROCM used to build PyTorch: N/A OS: Ubuntu 24.04.1 LTS (x86_64) GCC version: (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0 Clang version: 18.1.3 (1ubuntu1) CMake version: version 3.31.6 Libc versio…

As far as I know, NVIDIA removed p2p capability for consumer RTX 40xx gpus, and it was / is possible to use a modified driver for p2p. I would assume that it is also disabled for RTX 50xx.

You did see the following, correct? [image] shahizat: NCCL_P2P_DISABLE=1

@njuffa Yeap, that didn’t help. But it did allow the inference to start on a different machine with other GPUs.

Other information GDS release version: 1.13.1.3 libcufile version: 2.12 Platform: x86_64 ============ ENVIRONMENT: ============ ===================== DRIVER CONFIGURATION: ===================== NVMe P2PDMA : Unsupported NVMe : Unsupported NVMeOF : Unsuppo…

[image] shahizat: Could someone confirm whether peer-to-peer (P2P) communication is supported by the RTX 5090? You may find this post useful.

Thanks for reaching out to us . RTX 50-series are Geforce GPU series , and p2p is not supported for GPU<->GPU P2P. For GPU lists that can support p2p , there are mainly 2 series , Quadro RTX and Data Center GPUs . Quadro RTX - desktop class systems, which would need an actively cooled GPU […

Hi @Yuki_Ni , thanks for your reply, but how can we use the shared RAM to make a communcation between RTX 50-series GPUs instead of P2P? [image] [image]

You may wish to keep an eye on this project, based on the open gpu kernel module driver, which enables P2P between RTX 40 series cards. 50 series not currently.

[image] shahizat: how can we use the shared RAM what do you mean by “the shared RAM” ?

P2P issue using two RTX 5090 GPUs

Accelerated Computing CUDA CUDA Programming and Performance

rs277 March 11, 2025, 5:47pm 7

You may find this post useful.

Topic		Replies	Views
Problems about RTX 3090 p2p supports GPU-Accelerated Libraries	0	263	July 5, 2025
Standard nVidia CUDA tests fail with dual RTX 4090 Linux box Linux	54	23019	April 29, 2024
simpleP2P failing for A5000 Ubuntu server 20.04 TensorRT cuda	1	816	October 22, 2021
Parallelization on RTX 4060 Ti cards TensorRT	4	482	June 26, 2024
How to enable P2P access? CUDA Setup and Installation cuda	3	5550	February 6, 2023
Direct GPU <-> GPU communication does not seem to work properly CUDA Programming and Performance	8	1925	February 22, 2024
RuntimeError: [TensorRT-LLM][ERROR] CUDA runtime error in error: peer access is not supported between these two devices TensorRT	1	1453	February 29, 2024
Using multiple RTX 2080 Ti cards in parallel not possible? CUDA Programming and Performance	7	4768	May 13, 2019
P2P on GTX 1050 Ti CUDA Developer Tools	0	402	March 12, 2021
One GPU NOT capable of Peer-to-Peer (P2P) CUDA Programming and Performance	22	5724	November 27, 2018

P2P issue using two RTX 5090 GPUs

Related topics