Problem with "Simple Peer-to-Peer Transfers with Multi-GPU" I got an exception when I run th

Hi,
I’ve tryed to run the example program simpleP2P.exe in the SDK 4.0. I’m using Windows 7 professional (64bit), and I’ve 2 tesla C2050 on my workstation (HP Z800).
Finally I got this error message:

[simpleP2P.exe] starting…
Checking for multiple GPUs…
CUDA-capable device count: 3

GPU0 = “Tesla C2050 / C2070” IS capable of Peer-to-Peer (P2P)
GPU1 = " Quadro FX 380" IS capable of Peer-to-Peer (P2P)
GPU2 = “Tesla C2050 / C2070” IS capable of Peer-to-Peer (P2P)

Checking GPU(s) for support of peer to peer memory access…

Peer access from Tesla C2050 / C2070 (GPU0) -> Tesla C2050 / C2070 (GPU2) : No
Peer access from Tesla C2050 / C2070 (GPU2) -> Tesla C2050 / C2070 (GPU0) : No
Two or more Tesla(s) with class GPUs are required for C:\Users\cambim1a\AppData\Local\NVIDIA Corporation\NVIDIA GPU Computing SDK 4.0\C\bin\win64\Release\simpleP2P.exe to run.
Support for UVA requires a Tesla with SM 2.0 capabilities.
Peer to Peer access is not available between GPU0 <-> GPU2, waiving test.
PASSED

I’ve activated the TCC driver by nvidia-smi.exe. The following is the result of nvidia-smi.exe -q:

==============NVSMI LOG==============

Timestamp : Mon Nov 21 14:05:59 2011

Driver Version : 285.79

Attached GPUs : 3

GPU 0000:0F:00.0
Product Name : Tesla C2050 / C2070
Display Mode : Disabled
Persistence Mode : N/A
Driver Model
Current : TCC
Pending : TCC
Serial Number : 0321411034554
GPU UUID : GPU-388b61ea55a0383a-9a972739-f0505eb6-3aed3f49-de0ffcdad5757dcde653590b
VBIOS Version : 70.00.2B.00.0E
Inforom Version
OEM Object : 1.0
ECC Object : 1.0
Power Management Object : N/A
PCI
Bus : 0x0F
Device : 0x00
Domain : 0x0000
Device Id : 0x06D110DE
Bus Id : 0000:0F:00.0
Sub System Id : 0x0771103C
GPU Link Info
PCIe Generation
Max : 2
Current : 1
Link Width
Max : 16x
Current : 16x
Fan Speed : 30 %
Performance State : P12
Memory Usage
Total : 2687 MB
Used : 5 MB
Free : 2682 MB
Compute Mode : Default
Utilization
Gpu : 0 %
Memory : 0 %
Ecc Mode
Current : Enabled
Pending : Enabled
ECC Errors
Volatile
Single Bit
Device Memory : 0
Register File : 0
L1 Cache : 0
L2 Cache : 0
Total : 0
Double Bit
Device Memory : 0
Register File : 0
L1 Cache : 0
L2 Cache : 0
Total : 0
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Total : 0
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Total : 0
Temperature
Gpu : 61 C
Power Readings
Power Management : N/A
Power Draw : N/A
Power Limit : N/A
Clocks
Graphics : 50 MHz
SM : 101 MHz
Memory : 135 MHz
Max Clocks
Graphics : 573 MHz
SM : 1147 MHz
Memory : 1500 MHz
Compute Processes : None

GPU 0000:42:00.0
Product Name : Tesla C2050 / C2070
Display Mode : Disabled
Persistence Mode : N/A
Driver Model
Current : TCC
Pending : TCC
Serial Number : 0321411034306
GPU UUID : GPU-63d460b3809c51a4-900a4f79-6f4da1de-4286f315-5a696308a4843f580320fc93
VBIOS Version : 70.00.2B.00.0E
Inforom Version
OEM Object : 1.0
ECC Object : 1.0
Power Management Object : N/A
PCI
Bus : 0x42
Device : 0x00
Domain : 0x0000
Device Id : 0x06D110DE
Bus Id : 0000:42:00.0
Sub System Id : 0x0771103C
GPU Link Info
PCIe Generation
Max : 2
Current : 1
Link Width
Max : 16x
Current : 16x
Fan Speed : 30 %
Performance State : P12
Memory Usage
Total : 2687 MB
Used : 5 MB
Free : 2682 MB
Compute Mode : Default
Utilization
Gpu : 0 %
Memory : 0 %
Ecc Mode
Current : Enabled
Pending : Enabled
ECC Errors
Volatile
Single Bit
Device Memory : 0
Register File : 0
L1 Cache : 0
L2 Cache : 0
Total : 0
Double Bit
Device Memory : 0
Register File : 0
L1 Cache : 0
L2 Cache : 0
Total : 0
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Total : 0
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Total : 0
Temperature
Gpu : 62 C
Power Readings
Power Management : N/A
Power Draw : N/A
Power Limit : N/A
Clocks
Graphics : 50 MHz
SM : 101 MHz
Memory : 135 MHz
Max Clocks
Graphics : 573 MHz
SM : 1147 MHz
Memory : 1500 MHz
Compute Processes : None

GPU 0000:60:00.0
Product Name : Quadro FX 380
Display Mode : N/A
Persistence Mode : N/A
Driver Model
Current : WDDM
Pending : WDDM
Serial Number : N/A
GPU UUID : N/A
VBIOS Version : 62.94.6E.00.25
Inforom Version
OEM Object : N/A
ECC Object : N/A
Power Management Object : N/A
PCI
Bus : 0x60
Device : 0x00
Domain : 0x0000
Device Id : 0x065810DE
Bus Id : 0000:60:00.0
Sub System Id : 0x063B10DE
GPU Link Info
PCIe Generation
Max : 2
Current : 2
Link Width
Max : 16x
Current : 8x
Fan Speed : 40 %
Performance State : N/A
Memory Usage
Total : 232 MB
Used : 231 MB
Free : 0 MB
Compute Mode : Default
Utilization
Gpu : N/A
Memory : N/A
Ecc Mode
Current : N/A
Pending : N/A
ECC Errors
Volatile
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Total : N/A
Aggregate
Single Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Total : N/A
Double Bit
Device Memory : N/A
Register File : N/A
L1 Cache : N/A
L2 Cache : N/A
Total : N/A
Temperature
Gpu : 52 C
Power Readings
Power Management : N/A
Power Draw : N/A
Power Limit : N/A
Clocks
Graphics : N/A
SM : N/A
Memory : N/A
Max Clocks
Graphics : N/A
SM : N/A
Memory : N/A
Compute Processes : Not Supported

Can you help me?

Thanks a lot,
Mirko

Hi!

I had a similar problem an finally figured out that the GPU cards are connected to different PCI roots (Which might the case especially if you happen to have more than 2 PCI-e slots connected to your motherboard).

“P2P works only for GPUs under the same PCI-e root. In Linux, you could see the PCI-e tree with “lspci -t”, I am not familiar with Windows but there should be something equivalent.”

even though in my case the driver actually signals that P2P is possible, but using it crashes the program.

I could not identify the command to use for Windows to find this out however, but the documentation of the motherboard shows it.

(see [topic=‘213223’]Peer-To-Peer Access with cudaPitchedPtr[/topic])

Rolf