Tesla S1070 4GUPs

Accelerated Computing CUDA CUDA Programming and Performance
#1

Hi all,

Our group just installed a Tesla S1070 on a Linux-64 bit operation system. I run the example “deviceQuery” in SDK package and found there are only 2 GPUs detected.

Here is the message:

Device 0: “Tesla T10 Processor”
Major revision number: 1
Minor revision number: 3
Total amount of global memory: 4294705152 bytes
Number of multiprocessors: 30
Number of cores: 240
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 16384 bytes
Total number of registers available per block: 16384
Warp size: 32
Maximum number of threads per block: 512
Maximum sizes of each dimension of a block: 512 x 512 x 64
Maximum sizes of each dimension of a grid: 65535 x 65535 x 1
Maximum memory pitch: 262144 bytes
Texture alignment: 256 bytes
Clock rate: 1.30 GHz
Concurrent copy and execution: Yes

Device 1: “Tesla T10 Processor”
Major revision number: 1
Minor revision number: 3
Total amount of global memory: 4294705152 bytes
Number of multiprocessors: 30
Number of cores: 240
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 16384 bytes
Total number of registers available per block: 16384
Warp size: 32
Maximum number of threads per block: 512
Maximum sizes of each dimension of a block: 512 x 512 x 64
Maximum sizes of each dimension of a grid: 65535 x 65535 x 1
Maximum memory pitch: 262144 bytes
Texture alignment: 256 bytes
Clock rate: 1.30 GHz
Concurrent copy and execution: Yes

Test PASSED

Press ENTER to exit…

I thought Tesla S1070 had 4 GPUs, why I only saw 2? Is there something wrong with my hardware connections? Thanks for any advise.

Xuejun

#2

Are all 4 physically attached? Do they appear in lspci output?

#3

Here is lspci message regarding GPU parts:

84:00.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
85:00.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
85:01.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
85:02.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
85:03.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
8a:00.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
8b:00.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
8b:01.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
8b:02.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
8b:03.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
8e:00.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
8f:00.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
8f:01.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
8f:02.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
8f:03.0 PCI bridge: nVidia Corporation Tesla S870 (rev a3)
90:00.0 3D controller: nVidia Corporation Unknown device 05e7 (rev a1)
92:00.0 3D controller: nVidia Corporation Unknown device 05e7 (rev a1)

If we say 4 GPUs are physically connected well, why I only see 2 in the “deviceQurey” run?

Other questions: Does a CUDA code (or a kernel) using 4GPUs at the same time on Tesla S1070, or they just use 1GPU or 2GPUs? Do I need to specify GPU(s) in a CUDA code if I want use more than 1?

#4

I only see two S1070 GPUs in the output you provided. Do you also have several S870s attached?

Please generate and attach an nvidia-bug-report.log

#5

Hi

Thanks for your quick reply.

I believe that we don’t have S870s. We have only one S1070 connected to a cpu server through 2 pcie cables.

Also how can I generate nvidia-bug-report.log? Thanks for your help!

Best,

Xuejun

#6

Hi,

Finally we see 4 GPUs. It is our mistake that we forgot to reboot the system after we plug in another PCIe cable.

Thanks.

Best,

Xuejun