Extremely low bandwidth

Hello,

When executing the oclBandwidthTest program from the SDK (Ubuntu 10.04 64bit), I get extremely low bandwidth on both HostToDevice and DeviceToHost.

./oclBandwidthTest Starting...

Running on...

Device GeForce 8600 GT

 Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			181.1

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			197.0

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			10333.4

Anyone has any ideas as to why this happens?

–Liad Weinberger.

What does the driver tell you about PCI-E link width? Look into /var/log/Xorg.0.log, there should be a line like

(II) Aug 31 14:50:15 NVIDIA(0): Detected PCI Express Link width: 16X

I have a hunch that you are running on 1X Gen1 PCI-E or something nearly that low.

What does the driver tell you about PCI-E link width? Look into /var/log/Xorg.0.log, there should be a line like

(II) Aug 31 14:50:15 NVIDIA(0): Detected PCI Express Link width: 16X

I have a hunch that you are running on 1X Gen1 PCI-E or something nearly that low.

Hi,

Oddly enough, you’re right, though I’m not sure how that happened. From what I recall the last time I checked, the bandwidth was fine (with an older driver). I don’t recall doing anything that might have caused this. The board specs say it has 2 PCI-E x16 slots. The card is in one of them. I’ll try moving it to the other one.

Hi,

Oddly enough, you’re right, though I’m not sure how that happened. From what I recall the last time I checked, the bandwidth was fine (with an older driver). I don’t recall doing anything that might have caused this. The board specs say it has 2 PCI-E x16 slots. The card is in one of them. I’ll try moving it to the other one.

Mmmm… That actually made a difference. Now it states it found it at 16x. Running the bandwidth test gives better results, but still not what I expected:

./oclBandwidthTest Starting...

Running on...

Device GeForce 8600 GT

 Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			1071.5

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			1011.2

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			10426.5

Can anyone confirm on a similar configuration? From what I know PCI-E1.0 should be closer to 4GB/s shouldn’t it?

Mmmm… That actually made a difference. Now it states it found it at 16x. Running the bandwidth test gives better results, but still not what I expected:

./oclBandwidthTest Starting...

Running on...

Device GeForce 8600 GT

 Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			1071.5

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			1011.2

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			10426.5

Can anyone confirm on a similar configuration? From what I know PCI-E1.0 should be closer to 4GB/s shouldn’t it?

I’m getting over 2 GB/s with PCI-E 1.0 16x and a GT 240. Pinned memory improves this further to a bit over 3 GB/s.

I’m getting over 2 GB/s with PCI-E 1.0 16x and a GT 240. Pinned memory improves this further to a bit over 3 GB/s.

Hi,

I encountered similar bandwidth problem under Ubuntu 10.04 - but neither in host->device nor device->host region but device->device which is amazing.

Here is what I got under Win7:

C:\NVIDIA GPU Computing SDK\OpenCL\bin\Win64\Release\oclBandwidthTest.exe Starting...

Running on...

GeForce GTX 260

Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			2006.8

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			1678.1

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			87479.1

PASSED

Press <Enter> to Quit...

For CUDA it is similar:

C:\NVIDIA GPU Computing SDK\C\bin\win64\Release>bandwidthTest.exe

[bandwidthTest]

bandwidthTest.exe Starting...

Running on...

Device 0: GeForce GTX 260

 Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 2050.5

Device to Host Bandwidth, 1 Device(s), Paged memory

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 1612.2

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 90653.2

[bandwidthTest] - Test results:

PASSED

Press <Enter> to Quit...

And now Linux:

josef@josef:~/NVIDIA_GPU_Computing_SDK/OpenCL/bin/linux/release$ ./oclBandwidthTest 

./oclBandwidthTest Starting...

Running on...

GeForce GTX 260

Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 1224.5

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 1487.1

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 5030.1

PASSED

Press <Enter> to Quit...

How it could be so extremely low :teehee: ?

I checked GPU clocking and it seems the same as in Windows.


problem solved - it had to be caused by wrong driver - I’ve reinstalled 256.40 with current Linux kernel and got good results (even better than Windows) :rolleyes:

./oclBandwidthTest Starting...

Running on...

GeForce GTX 260

Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 2321.5

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 1865.4

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 91117.4

PASSED

Press <Enter> to Quit...

Hi,

I encountered similar bandwidth problem under Ubuntu 10.04 - but neither in host->device nor device->host region but device->device which is amazing.

Here is what I got under Win7:

C:\NVIDIA GPU Computing SDK\OpenCL\bin\Win64\Release\oclBandwidthTest.exe Starting...

Running on...

GeForce GTX 260

Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			2006.8

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			1678.1

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			87479.1

PASSED

Press <Enter> to Quit...

For CUDA it is similar:

C:\NVIDIA GPU Computing SDK\C\bin\win64\Release>bandwidthTest.exe

[bandwidthTest]

bandwidthTest.exe Starting...

Running on...

Device 0: GeForce GTX 260

 Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 2050.5

Device to Host Bandwidth, 1 Device(s), Paged memory

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 1612.2

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 90653.2

[bandwidthTest] - Test results:

PASSED

Press <Enter> to Quit...

And now Linux:

josef@josef:~/NVIDIA_GPU_Computing_SDK/OpenCL/bin/linux/release$ ./oclBandwidthTest 

./oclBandwidthTest Starting...

Running on...

GeForce GTX 260

Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 1224.5

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 1487.1

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 5030.1

PASSED

Press <Enter> to Quit...

How it could be so extremely low :teehee: ?

I checked GPU clocking and it seems the same as in Windows.


problem solved - it had to be caused by wrong driver - I’ve reinstalled 256.40 with current Linux kernel and got good results (even better than Windows) :rolleyes:

./oclBandwidthTest Starting...

Running on...

GeForce GTX 260

Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 2321.5

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 1865.4

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 91117.4

PASSED

Press <Enter> to Quit...