There is no device supporting CUDA

[philhughes]

I have a machine with Red Hat Enterprise Linux 4 installed. The machine has 3 8800 GTX in it. I installed the drivers, toolkit and SDK and compiled all the sample code.

When I try and run something in debug or release (meaning, on the GPU), I get this error.

NVIDIA: could not open the device file /dev/nvidia1 (Input/output error).

There is no device supporting CUDA.

In case this is informative:

ls /dev/nvidia*

/dev/nvidia0  /dev/nvidia1  /dev/nvidia2  /dev/nvidiactl

What should I do to fix this?

[netllama]

Specifically which drivers did you install?
Did you start X first?

[philhughes]

Specifically which drivers did you install?

Did you start X first?

[snapback]256975[/snapback]

I installed the ones off the website. The file is called NVIDIA-Linux-x86-100.14.11.pkg1.run. I am guessing that means I installed 100.14.11

X is running.

[mfatica]

What is the output of “ls -l /dev/nv*”?

[philhughes]

What is the output of “ls -l /dev/nv*”?

[snapback]256992[/snapback]

[cudauser@cuda3-linux ~]$ ls -l /dev/nv*

crw------- 1 cudauser root 195, 0 Sep 26 07:39 /dev/nvidia0

crw------- 1 cudauser root 195, 1 Sep 26 07:39 /dev/nvidia1

crw------- 1 cudauser root 195, 2 Sep 26 07:39 /dev/nvidia2

crw------- 1 cudauser root 195, 255 Sep 26 07:39 /dev/nvidiactl

[mfatica]

Permissions are ok.
If you try to run deviceQuery, what is the output?
Do you have some old Cuda installation on your system?
Check with ldd that you are pointing to the new runtime.

[philhughes]

Permissions are ok.

If you try to run deviceQuery, what is the output?

Do you have some old Cuda installation on your system?

Check with ldd that you are pointing to the new runtime.

[snapback]256999[/snapback]

The install is brand new. No previous versions of CUDA.

Can you elaborate on deviceQuery and ldd?

Is deviceQuery a CUDA function?

[philhughes]

Here is the output from ldd in the /usr/local/cuda/lib directory. Not sure if this is what you meant.

[cudauser@cuda3-linux lib]$ ldd *.1.0

libcublasemu.so.1.0:

libtlshook.so (0x00819000)

libpthread.so.0 => /lib/tls/libpthread.so.0 (0x004dc000)

libcuda.so (0x004ee000)

libcudart.so (0x00917000)

libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00a6f000)

libm.so.6 => /lib/tls/libm.so.6 (0x00a0d000)

libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x003b6000)

libc.so.6 => /lib/tls/libc.so.6 (0x00cd9000)

/lib/ld-linux.so.2 (0x007c8000)

libdl.so.2 => /lib/libdl.so.2 (0x003c0000)

libz.so.1 => /usr/lib/libz.so.1 (0x006f6000)

libcublas.so.1.0:

libtlshook.so (0x00658000)

libpthread.so.0 => /lib/tls/libpthread.so.0 (0x004aa000)

libcuda.so (0x00111000)

libcudart.so (0x002a0000)

libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x002cc000)

libm.so.6 => /lib/tls/libm.so.6 (0x00394000)

libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00da7000)

libc.so.6 => /lib/tls/libc.so.6 (0x004bc000)

/lib/ld-linux.so.2 (0x00c48000)

libdl.so.2 => /lib/libdl.so.2 (0x003b7000)

libz.so.1 => /usr/lib/libz.so.1 (0x003bb000)

libcudart.so.1.0:

libcuda.so (0x00368000)

libtlshook.so (0x00e10000)

libdl.so.2 => /lib/libdl.so.2 (0x0034d000)

libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00111000)

libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x008a9000)

libm.so.6 => /lib/tls/libm.so.6 (0x00123000)

libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00146000)

libc.so.6 => /lib/tls/libc.so.6 (0x00b65000)

libz.so.1 => /usr/lib/libz.so.1 (0x00150000)

/lib/ld-linux.so.2 (0x00617000)

libcuda.so.1.0:

libdl.so.2 => /lib/libdl.so.2 (0x0038d000)

libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00534000)

libz.so.1 => /usr/lib/libz.so.1 (0x004da000)

libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x008e4000)

libm.so.6 => /lib/tls/libm.so.6 (0x00831000)

libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x003cc000)

libc.so.6 => /lib/tls/libc.so.6 (0x00d7d000)

/lib/ld-linux.so.2 (0x003e4000)

libcufftemu.so.1.0:

libtlshook.so (0x003f2000)

libdl.so.2 => /lib/libdl.so.2 (0x004ae000)

libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00111000)

libcuda.so (0x00123000)

libcudart.so (0x006f1000)

libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00ce6000)

libm.so.6 => /lib/tls/libm.so.6 (0x005c8000)

libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x002b2000)

libc.so.6 => /lib/tls/libc.so.6 (0x002bc000)

/lib/ld-linux.so.2 (0x00b4a000)

libz.so.1 => /usr/lib/libz.so.1 (0x003f4000)

libcufft.so.1.0:

libtlshook.so (0x00dfe000)

libdl.so.2 => /lib/libdl.so.2 (0x00f8a000)

libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00b81000)

libcuda.so (0x00111000)

libcudart.so (0x002a0000)

libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x004ab000)

libm.so.6 => /lib/tls/libm.so.6 (0x006eb000)

libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00b51000)

libc.so.6 => /lib/tls/libc.so.6 (0x002cc000)

/lib/ld-linux.so.2 (0x00e9d000)

libz.so.1 => /usr/lib/libz.so.1 (0x003f8000)

libtlshook.so.1.0:

libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00c1b000)

libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00e70000)

libm.so.6 => /lib/tls/libm.so.6 (0x0017e000)

libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00875000)

libc.so.6 => /lib/tls/libc.so.6 (0x006f2000)

/lib/ld-linux.so.2 (0x00d17000)

[philhughes]

Sorry, I “found” deviceQuery. Here is the output from release, emurelease, emudebug:

[cudauser@cuda3-linux release]$ ./deviceQuery

NVIDIA: could not open the device file /dev/nvidia1 (Input/output error).

There is no device supporting CUDA.

Device 0: “Device Emulation (CPU)”

Major revision number:                        -1

Minor revision number:                        -1

Total amount of global memory:                2147483647 bytes

Total amount of constant memory:              65536 bytes

Total amount of shared memory per block:      16384 bytes

Total number of registers available per block: 8192

Warp size:                                    32

Maximum number of threads per block:          512

Maximum sizes of each dimension of a block:    512 x 512 x 64

Maximum sizes of each dimension of a grid:    65535 x 65535 x 1

Maximum memory pitch:                          262144 bytes

Texture alignment:                            256 bytes

Clock rate:                                    1350000 kilohertz

Test PASSED

BUT, if I run it in debug mode:

[cudauser@cuda3-linux debug]$ ./deviceQuery

NVIDIA: could not open the device file /dev/nvidia1 (Input/output error).

Cuda error in file ‘deviceQuery.cu’ in line 53 : initialization error.

[philhughes]

Problem solved!

Looking in /var/log/nvidia-installer.log, there was something about things failing because “out of vmalloc space”. With this hint, I found a thread

http://forums.nvidia.com/index.php?showtop…8975&hl=vmalloc

where someone has solved this problem.

You need to edit your grub to make it like the one below. For me, that meant adding the uppermem, vmalloc and pci parts. Now I have three GPUs.

title Red Hat Desktop (2.6.9-42.ELsmp) (Dual GPU SLI CUDA)

root (hd0,0)

uppermem 524288

kernel /vmlinuz-2.6.9-42.ELsmp ro root=LABEL=/1 rhgb quiet vmalloc=256MB

pci=nommconf

initrd /initrd-2.6.9-42.ELsmp.img

Thanks everyone for your help and quick responses.

[matuno]

Hi,

I’m running openSUSE 10.3 (AMD64) and GeForce 8800 GT. I managed to compile some examples in the SDK after having performed an installation according to http://developer.download.nvidia.com/compu…notes_linux.txt, along with some previously posted instructions (concerning gcc-4.2->4.1 and glut).

When I execute for example matrixMul I get “There is no device supporting CUDA”. Make and execution of deviceQuery yields:

There is no device supporting CUDA.

Device 0: “Device Emulation (CPU)”

Major revision number:                        -1

Minor revision number:                        -1

Total amount of global memory:                -1 bytes

Total amount of constant memory:              65536 bytes

Total amount of shared memory per block:      16384 bytes

Total number of registers available per block: 8192

Warp size:                                    32

Maximum number of threads per block:          512

Maximum sizes of each dimension of a block:    512 x 512 x 64

Maximum sizes of each dimension of a grid:    65535 x 65535 x 1

Maximum memory pitch:                          262144 bytes

Texture alignment:                            256 bytes

Clock rate:                                    1350000 kilohertz

Test PASSED

Press ENTER to exit…

There is one difference in the output compared to the previous post in this thread:

Total amount of global memory:                -1 bytes

That doesn’t sound to good … but I do not know how to fix it. Can someone help me solve this? See also the attached nvidia-bug-report.log (extension changed to .txt due to upload restrictions).
nvidia_bug_report.txt (121 KB)

[matuno]

Hi,

I’m running openSUSE 10.3 (AMD64) and GeForce 8800 GT. I managed to compile some examples in the SDK after having performed an installation according to http://developer.download.nvidia.com/compu…notes_linux.txt, along with some previously posted instructions (concerning gcc-4.2->4.1 and glut).

When I execute for example matrixMul I get “There is no device supporting CUDA”. Make and execution of deviceQuery yields:

There is one difference in the output compared to the previous post in this thread:

That doesn’t sound to good … but I do not know how to fix it. Can someone help me solve this? See also the attached nvidia-bug-report.log (extension changed to .txt due to upload restrictions).

[snapback]367465[/snapback]

I decided to try Fedora 7 instead, and it worked fine. A possible explanation to my suse troubles could be that I did not kill X before installing the nvidia driver.