Configuration problem -- cudaGetDeviceCount fails Cuda / Linux configuration problem

Accelerated Computing CUDA CUDA Programming and Performance
1

I seem to have some sort of Cuda configuration problem. Yesterday, the examples in the SDK worked perfectly, and today, after I logged into our Linux box again, they crash with an error like this:

deviceQuery
deviceQuery Starting…

CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount FAILED CUDA Driver and Runtime version may be mismatched.

FAILED

I must have some environmental variable that isn’t set properly, or maybe a problem in nvcc.profile…? If you can suggest where to look, that would be appreciated. I am running

x86_64
Red Hat Enterprise Linux Client release 5.4 (Tikanga)

Nvidia driver version 256.40

The Cuda toolkit I downloaded was cudatoolkit_3.1_linux_64_rhel5.4.run

The SDK I downloaded was gpucomputingsdk_3.1_linux.run

2

I think we figured it out. The Linux installation guide, in the section “Verify the Installation”, says that on multiuser systems, access to NVIDIA devices must be enabled for remote users. So we added RW privileges to all users for every /dev/nv* device:

crwxrw-rw- 1 root root 195, 0 Sep 2 09:51 /dev/nvidia0
crwxrw-rw- 1 root root 195, 1 Sep 2 09:51 /dev/nvidia1
crwxrw-rw- 1 root root 195, 2 Sep 2 09:51 /dev/nvidia2
crwxrw-rw- 1 root root 195, 3 Sep 2 09:51 /dev/nvidia3
crwxrw-rw- 1 root root 195, 255 Sep 2 09:51 /dev/nvidiactl
crwxrw-rw- 1 root root 10, 144 Sep 1 10:40 /dev/nvram

The SDK examples are now working from a remote terminal.

3

I think we figured it out. The Linux installation guide, in the section “Verify the Installation”, says that on multiuser systems, access to NVIDIA devices must be enabled for remote users. So we added RW privileges to all users for every /dev/nv* device:

crwxrw-rw- 1 root root 195, 0 Sep 2 09:51 /dev/nvidia0
crwxrw-rw- 1 root root 195, 1 Sep 2 09:51 /dev/nvidia1
crwxrw-rw- 1 root root 195, 2 Sep 2 09:51 /dev/nvidia2
crwxrw-rw- 1 root root 195, 3 Sep 2 09:51 /dev/nvidia3
crwxrw-rw- 1 root root 195, 255 Sep 2 09:51 /dev/nvidiactl
crwxrw-rw- 1 root root 10, 144 Sep 1 10:40 /dev/nvram

The SDK examples are now working from a remote terminal.