GTX690 Issue

Developer Tools Nsight Visual Studio Edition
1

Have just installed nsight build 3.2.0.13289 on a new rampage iv extreme with with an i7 3960X and an Asus GTX690. The GTX690 is the only graphics card in the system. When I run any Cuda app (eg clock) in Cuda Debugging, I receive lots of screen flashing then an error is generated:

Cuda grid launch failed: CUcontext 55787904 CUmodule 58689536 Function: _Z14timedReduction PKfPfPI

I am wondering if I need a separate graphics card, given I am debugging on the one that is being used for display purposes.

Looking for ideas as to what it may be.

cheers

Paul

2

Further Information:

CUDA context created : 02f119c0
CUDA module loaded: 033e8800 C:/NVIDIA/CUDA/CUDASamples/0_Simple/clock/clock.cu
CUDA grid launch failed: CUcontext: 49355200 CUmodule: 54429696 Function: _Z14timedReductionPKfPfPl

NVIDIA System Information report created on: 11/11/2013 14:15:57
System name: SPEEDY

[Display]
Operating System: Windows 7 Ultimate, 64-bit (Service Pack 1)
DirectX version: 11.0
GPU processor: GeForce GTX 690 (GPU 1 of 2)
Driver version: 331.58
Direct3D API version: 11
Direct3D feature level: 11_0
CUDA Cores: 1536
Core clock: 915 MHz
Memory data rate: 6008 MHz
Memory interface: 256-bit
Memory bandwidth: 192.26 GB/s
Total available graphics memory: 4096 MB
Dedicated video memory: 2048 MB GDDR5
System video memory: 0 MB
Shared system memory: 2048 MB
Video BIOS version: 80.04.1E.00.17
IRQ: 40
Bus: PCI Express x16 Gen3
Device Id: 10DE 1188 84061043
Part Number: 2000 0000
GPU processor: GeForce GTX 690 (GPU 2 of 2)
Driver version: 331.58
Direct3D API version: 11
Direct3D feature level: 11_0
CUDA Cores: 1536
Core clock: 915 MHz
Memory data rate: 6008 MHz
Memory interface: 256-bit
Memory bandwidth: 192.26 GB/s
Total available graphics memory: 4096 MB
Dedicated video memory: 2048 MB GDDR5
System video memory: 0 MB
Shared system memory: 2048 MB
Video BIOS version: 80.04.1E.00.18
IRQ: 떀Y
Bus: PCI Express x16 Gen3
Device Id: 10DE 1188 84061043
Part Number: 2000 0000

[Components]

easyDaemonAPIU64.DLL 8.3.23.0 NVIDIA GeForce Experience
WLMerger.exe 8.3.23.0 NVIDIA GeForce Experience
daemonu.exe 8.3.23.0 NVIDIA GeForce Experience
ComUpdatus.exe 8.3.23.0 NVIDIA GeForce Experience
NvUpdtr.dll 8.3.23.0 NVIDIA GeForce Experience
NvUpdt.dll 8.3.23.0 NVIDIA GeForce Experience
nvui.dll 8.17.13.3158 NVIDIA User Experience Driver Component
nvxdsync.exe 8.17.13.3158 NVIDIA User Experience Driver Component
nvxdplcy.dll 8.17.13.3158 NVIDIA User Experience Driver Component
nvxdbat.dll 8.17.13.3158 NVIDIA User Experience Driver Component
nvxdapix.dll 8.17.13.3158 NVIDIA User Experience Driver Component
NVCPL.DLL 8.17.13.3158 NVIDIA User Experience Driver Component
nvCplUI.exe 7.5.750.0 NVIDIA Control Panel
nvWSS.dll 6.14.13.3158 NVIDIA Workstation Server
nvViTvS.dll 6.14.13.3158 NVIDIA Video Server
NVMCTRAY.DLL 8.17.13.3158 NVIDIA Media Center Library
nvDispS.dll 6.14.13.3158 NVIDIA Display Server
PhysX 09.13.0725 NVIDIA PhysX
NVCUDA.DLL 8.17.13.3158 NVIDIA CUDA 6.0.1 driver
nvGameS.dll 6.14.13.3158 NVIDIA 3D Settings Server

3

Hi Paul,

Nsight supports debugging/analysis on a machine that has 1 GPU and is driving the display. In your (lucky! :-) ) situation, you have 2 GPUs actually, so assuming you are using only 1 of the GPUs to drive your display(s), then you can have the other be “headless” (meaning no displays attached to it), and get better performance while debugging.

A number of questions and things to try & test if you don’t mind:
Does the application run OK by itself (non-Nsight)? Does it happen with other samples? Which version of the CUDA samples and CUDA toolkit?

Are you able to force the application to run on the “headless” GPU? What happens?

You can look at the chapter on How to turn change the TDR setting (requires a machine reboot), to see if it helps.

If none of of that works, and if you are able to, can you try downgrading to the 327.23 driver:

and see if that helps you?

Thanks

4

Hi There,

I tried the TDR suggestion for multi-GPU devices of setting the timeout to off. Ended up having to do a hard reboot…

Not sure how to force an application onto a specific GPU without using cudaSetDevice. Is there a configuration method I can use? Also for folks that have a single GPU card, are they able to use the same GPU for debug and active screen display, or do they have to use either a separate card or remote debug?

Will make some mods to the sample apps and see if cudaSetDevice works anyway.

cheers

Paul

5

Have tried using alternative heads with no success. The behaviour is that the monitor freezes for the period of the TDR timeout then flashes rapidly for about three seconds and the board fan runs rapidly. Then I get the following messages…

External Media

6

Ok latest update. Command line parameters set in the project properties are not used when executing under NSight. ASs I was using this approach to set the device, I was still executing on Device 0. Have verified that this is the problem with a modified separate sample.

Thanks for your help. Cheers Paul