GTX Titan Win7 x64 gets PCIe 2.0 speed :(

Accelerated Computing CUDA CUDA Setup and Installation
1

Hi,
I’m testing a GTX Titan under Win7 x64 and it seems it doesn’t run at PCIe 3.0 speed.

I run it on a PCI-E 3.0 interface and my PB supports it (gigabyte Z77x UD5H), GPU-z tells me its PCI-E 3.0x16@x8 1.1. So I guess it could be some powersaving issue?

Output of my bandwidthTest:

[CUDA Bandwidth Test] - Starting...
Running on...

 Device 0: GeForce GTX TITAN
 Quick Mode

 Host to Device Bandwidth, 1 Device(s)
 PINNED Memory Transfers
   Transfer Size (Bytes)        Bandwidth(MB/s)
   33554432                     6010.5

 Device to Host Bandwidth, 1 Device(s)
 PINNED Memory Transfers
   Transfer Size (Bytes)        Bandwidth(MB/s)
   33554432                     6221.4

 Device to Device Bandwidth, 1 Device(s)
 PINNED Memory Transfers
   Transfer Size (Bytes)        Bandwidth(MB/s)
   33554432                     220604.5

Press any key to continue . . .
2

more info: If I let opened the TechPowerUp GPU-Z and I run the bandwidth test I see the PCI-E 3.0 x16@8 1.1 changes to PCI-E 3.0 x16 @8 3.0

What should I do in order to make this GPU running full bandwidth speed?

3

Looking at your motherboard on NewEgg, (interesting, it has an MSATA slot) it does have a PCI-E 3.0 x16 slot, which matches with GPU-z telling you it’s running @ PCI-E 3.0x16@x8 1.1… so it technically is running at PCI-E 3.0… just not at the full bandwidth. Your board also has 2 other PCI-E slots, but none that run at the full bandwidth (x8, and x4).

Those bandwidth results you posted are pretty much equivalent to the full PCI-E 2.0 x16 bandwidth. I would try a few things to resolve the problem:

  1. If you are running multi-GPU’s that would be the reason why you’re getting x8 speeds. See:
    [url]http://www.anandtech.com/show/6108/gigabyte-gaz77xud5h-review-functionality-meets-competitive-pricing[/url] specifically: “The rest of the PCIe are an x16 (x8 in multi-GPU), x1, x1, x8 (x4 in tri-GPU), PCI, x4.” It doesn’t look to be the case based on your bandwidthTest results though, unless you cropped them.

  2. If you placed the card in the incorrect slot (non-x16 capable slot), you would be seeing the x8 behavior, and in fact I am betting this is the issue. You should place the card on the x16 slot. Check your motherboard manual, I believe it is the first one closest to the processor/memory banks, but don’t quote me on that as I haven’t looked, just what I can guess from Newegg’s description.

  3. If (2) still doesn’t give you full speeds and you have the card in the right slot, try a registry hack and/or patch to see if it enables full PCI-E 3.0 x16 speeds. See:
    [url]http://www.techpowerup.com/downloads/2148/NVIDIA_GeForce_Kepler_PCIe_3.0_mode-enabling_patch_for_Sandy_Bridge-E_systems.html[/url] and/or [url]http://www.evga.com/forums/tm.aspx?m=1607292&mpage=1&print=true[/url]. On Win8, the first solution didn’t work for me, and I had to do the 2nd one, doing the manual registry edit.

4

Thanks for the fast reply,
I’m running single GPU (besides the display, I use the HD4000 as display). And this GPU is placed in the PCIEX16 slot, according to the manual (the closest to the CPU and RAM, yes).

I will try the 3) but I see it refers to Sandy_Bridge_E systems and I they are based on X79 (mine is Z77). I’ll have a look on the second link of point 3)

5

No luck … still I see PCI-E @ x8 :(

btw my dirver version is 314.22

6

I don’t think I have any other valid points to add, short of looking at your BIOS options and/or seeing if using the GTX Titan as the primary display gives you the 3.0 x16 speeds… On my motherboard’s BIOS (MSI X79A-GD45 8D) there is an explicit option to turn on PCI-E 3.0 support, and also to tweak PCI-E latency timings… the latter probably isn’t the issue though.

My other idea is to update your motherboard BIOS to the latest version available if you haven’t already and see if that gives you the full speed.

7

I have updated BIOS firmware and check all the PCIe options (I have just seen PCIe Gen3 option / Gen2 / Gen1 and Auto, I’ve set Gen3 and nothing changes)

I guess my only out is to ask gigabyte about this issue … but I have heard they aren’t too good at customer support :/

I might check installing a linux distro and see what happens

8

My last option was to suggest to get support from your motherboard manufacturer, indeed. I suspect you’ll see the same in Linux also, but it’s worth a try. If so, follow my advice here to force PCI-E 3.0 under Linux drivers: [url]https://devtalk.nvidia.com/default/topic/533200/linux/gtx-titan-drivers-for-linux-32-64-bit-release-/post/3753244/#3753244[/url]

9

In adition to this, I’m running another GTX Titan at home, plugged into a slighty different motherboard, a Z77x D3H and here it DOES work as intended … so I guess something is wrong with the pcie switch in my Z77x UD5h.

I have already sent an email to gigabyte technical service…