Dear CUDA fans,
I have a machine with 3 PCI-e x16 slots and 3 GTX 260s occupying them. I installed the 64-bit version of Ubuntu 9.04 and succeeded in getting the CUDA driver (version 2.3), toolkit, and SDK installed, but only when one GPU is plugged in. If I plug the second or third card in, X windows will not start (using the Nvidia driver), and if I try running deviceQuery at the command line, it doesn’t detect any CUDA-enabled cards – not even the first one. The devices /dev/nvidia0 and /dev/nvidiactl (that were there when one card worked) don’t exist anymore.
I saw elsewhere on the web that people using 32-bit versions of Linux had problems adding multiple cards, and theirs could be solved by passing some parameters to the kernel at boot time:
kernel /boot/vmlinuz-2.6.(blah blah blah) vmalloc=256MB pci=nommconf
initrd /boot/initrd-2.6.(blah blah)
I tried this (adding uppermem 524288 as a separate line, and vmalloc=256MB pci=nommconf at the end of the kernel line), but it had no effect. One card works alone, but no cards work if multiple ones are installed. The solution above may be irrelevant for 64-bit versions, since one poster commented it is intended to overcome the 32-bit limit of 4 GB addressable RAM.
I’m not sure how to debug this any further. Has anyone else encountered this and fixed it?