I spent the whole week trying to setup a Ubuntu 10.04 64-bit machine for Xen virtualization (Xen 4.1) for GPGPU tests on several virtual machines (using Xen’s GPU passthrough capability). My problem is that I just can’t get the nvidia development driver to work on the xen kernel (works perfectly fine with the standard kernel).
For compiling the kernel, I followed the instructions from http://www.zeroaccess.org/2011/04/xen-4-1-on-ubuntu-10-04-64bit/ (but built the kernel the debian way (make-kpkg) to get a nice .deb package). It boots fine, I modified grub to set the kernel options, etc. All the Xen setup seems to be working as it should.
I can’t get the NVIDIA driver working on the host (Dom0) though. I used the driver with CUDA 3.2, and also the 4.0 RC2 driver (for a GTX 590 card). When the system tries to start X11, the screen turns blank and the system gets very slow. Booting the system into text mode works fine. I can load the nvidia driver manually (modprobe nvidia), and create the device nodes in /dev using mknode (so I have the /dev/nvidia0, /dev/nvidia1, /dev/nvidiactl - with major number 195, and minors 0, 1, 255, respectively). When I try to build anything using OpenCL, it just reports that no platforms have been found. With CUDA, I get the error: “cudaSafeCall() Runtime API error : invalid device ordinal.” Both work completely fine when I boot the system into a standard kernel (linux-image-generic, default with ubuntu 10.04).
The X11 log just says that it failed to load the NVIDIA module. Syslog gives messages like “NVRM: RmInitAdapter failed!” .
I tried various suggestions for installing the the driver found on the web (e.g., http://wiki.xensource.com/xenwiki/NvidiaGPU?highlight=(nvidia) ) but with no success.
Did any of you get NVIDIA and Xen work together?? How? Any help is appreciated!