Extremely low bandwidth

weliad · September 2, 2010, 8:08pm

Hello,

When executing the oclBandwidthTest program from the SDK (Ubuntu 10.04 64bit), I get extremely low bandwidth on both HostToDevice and DeviceToHost.

./oclBandwidthTest Starting...

Running on...

Device GeForce 8600 GT

 Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			181.1

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			197.0

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			10333.4

Anyone has any ideas as to why this happens?

–Liad Weinberger.

daemonized · September 3, 2010, 1:08am

What does the driver tell you about PCI-E link width? Look into /var/log/Xorg.0.log, there should be a line like

(II) Aug 31 14:50:15 NVIDIA(0): Detected PCI Express Link width: 16X

I have a hunch that you are running on 1X Gen1 PCI-E or something nearly that low.

daemonized · September 3, 2010, 1:08am

What does the driver tell you about PCI-E link width? Look into /var/log/Xorg.0.log, there should be a line like

(II) Aug 31 14:50:15 NVIDIA(0): Detected PCI Express Link width: 16X

I have a hunch that you are running on 1X Gen1 PCI-E or something nearly that low.

weliad · September 3, 2010, 9:55am

Hi,

Oddly enough, you’re right, though I’m not sure how that happened. From what I recall the last time I checked, the bandwidth was fine (with an older driver). I don’t recall doing anything that might have caused this. The board specs say it has 2 PCI-E x16 slots. The card is in one of them. I’ll try moving it to the other one.

weliad · September 3, 2010, 9:55am

Hi,

Oddly enough, you’re right, though I’m not sure how that happened. From what I recall the last time I checked, the bandwidth was fine (with an older driver). I don’t recall doing anything that might have caused this. The board specs say it has 2 PCI-E x16 slots. The card is in one of them. I’ll try moving it to the other one.

weliad · September 3, 2010, 10:09am

Mmmm… That actually made a difference. Now it states it found it at 16x. Running the bandwidth test gives better results, but still not what I expected:

./oclBandwidthTest Starting...

Running on...

Device GeForce 8600 GT

 Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			1071.5

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			1011.2

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			10426.5

Can anyone confirm on a similar configuration? From what I know PCI-E1.0 should be closer to 4GB/s shouldn’t it?

weliad · September 3, 2010, 10:09am

Mmmm… That actually made a difference. Now it states it found it at 16x. Running the bandwidth test gives better results, but still not what I expected:

./oclBandwidthTest Starting...

Running on...

Device GeForce 8600 GT

 Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			1071.5

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			1011.2

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			10426.5

Can anyone confirm on a similar configuration? From what I know PCI-E1.0 should be closer to 4GB/s shouldn’t it?

daemonized · September 3, 2010, 1:34pm

I’m getting over 2 GB/s with PCI-E 1.0 16x and a GT 240. Pinned memory improves this further to a bit over 3 GB/s.

daemonized · September 3, 2010, 1:34pm

I’m getting over 2 GB/s with PCI-E 1.0 16x and a GT 240. Pinned memory improves this further to a bit over 3 GB/s.

BlackBishop · September 4, 2010, 2:31pm

Hi,

I encountered similar bandwidth problem under Ubuntu 10.04 - but neither in host->device nor device->host region but device->device which is amazing.

Here is what I got under Win7:

C:\NVIDIA GPU Computing SDK\OpenCL\bin\Win64\Release\oclBandwidthTest.exe Starting...

Running on...

GeForce GTX 260

Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			2006.8

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			1678.1

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)	Bandwidth(MB/s)

   33554432			87479.1

PASSED

Press <Enter> to Quit...

For CUDA it is similar:

C:\NVIDIA GPU Computing SDK\C\bin\win64\Release>bandwidthTest.exe

[bandwidthTest]

bandwidthTest.exe Starting...

Running on...

Device 0: GeForce GTX 260

 Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 2050.5

Device to Host Bandwidth, 1 Device(s), Paged memory

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 1612.2

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 90653.2

[bandwidthTest] - Test results:

PASSED

Press <Enter> to Quit...

And now Linux:

josef@josef:~/NVIDIA_GPU_Computing_SDK/OpenCL/bin/linux/release$ ./oclBandwidthTest 

./oclBandwidthTest Starting...

Running on...

GeForce GTX 260

Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 1224.5

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 1487.1

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 5030.1

PASSED

Press <Enter> to Quit...

How it could be so extremely low External Image ?

I checked GPU clocking and it seems the same as in Windows.

problem solved - it had to be caused by wrong driver - I’ve reinstalled 256.40 with current Linux kernel and got good results (even better than Windows) :rolleyes:

./oclBandwidthTest Starting...

Running on...

GeForce GTX 260

Quick Mode

Host to Device Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 2321.5

Device to Host Bandwidth, 1 Device(s), Paged memory, direct access

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 1865.4

Device to Device Bandwidth, 1 Device(s)

   Transfer Size (Bytes)		Bandwidth(MB/s)

   33554432					 91117.4

PASSED

Press <Enter> to Quit...

BlackBishop · September 4, 2010, 2:31pm

Hi,