Trying CUDA in a Xen DomU

Hi all,

First post, already a call for help :)

I’m trying to use a gtx 580 with Xen 4 VGA Passthrough, the MB is an Asus P7P55 LX with a i7 870, VT-D is enabled in bios and reported as active by Xen.

I’m using Debian Squeeze as Dom0 and DomU OS.

Dom0 is configured with iommu=1 and iommu=verbose parameters and xen-pciback.hide=(01:00.0)(01:00.1) (but I still see them with lspci in dom0, is that normal?)

I configured DomU .cfg VGA passthrough with

gfx_passthru=1

pci=['01:00.0']

and can see the devices as assignable

root@cloud:~/nvidia# xm pci-list-assignable-devices

0000:01:00.0

0000:01:00.1

(0000:01:00.1 refers to the hdmi audio port)

and then assigned to the DomU

root@cloud:~/nvidia# xm pci-list squeeze1

domain bus  slot func

0x0000 0x01 0x00 0x0

Everything seems fine, I can see the card in the DomU

root@squeeze1:~# lspci

00:00.0 VGA compatible controller: nVidia Corporation Device 1080 (rev a1)

the module is loaded and the devices exists

root@squeeze1:~# lsmod

Module                  Size  Used by

nvidia              10442361  0 

snd_pcm                60551  0 

snd_timer              15582  1 snd_pcm

snd                    46446  2 snd_pcm,snd_timer

soundcore               4598  1 snd

snd_page_alloc          6249  1 snd_pcm

i2c_core               15712  1 nvidia

evdev                   7352  0 

pcspkr                  1699  0 

ext3                  106502  1 

jbd                    37085  1 ext3

mbcache                 5050  1 ext3

xen_netfront           16257  0 

xen_blkfront            9435  2
root@squeeze1:~# ls -lh /dev/nvidia*

crw-rw-rw- 1 root root 195,   0 Mar  3 16:26 /dev/nvidia0

crw-rw-rw- 1 root root 195, 255 Mar  3 16:26 /dev/nvidiactl

but deviceQuery fails :(

./NVIDIA_GPU_Computing_SDK/C/bin/linux/release/deviceQuery Starting...

CUDA Device Query (Runtime API) version (CUDART static linking)

[ 3276.036642] NVRM: failed to register with the ACPI subsystem!

[ 3276.047716] NVRM: RmInitAdapter failed! (0x23:0x2f:646)

[ 3276.047737] NVRM: rm_init_adapter(0) failed

[ 3276.047839] NVRM: failed to unregister from the ACPI subsystem!

cudaGetDeviceCount FAILED CUDA Driver and Runtime version may be mismatched.

I’m using cuda toolkit 3.2.16, sdk_3.2.16 and I tried both dev driver 260.19.26 and 270.29 (is there a difference between “dev” and normal driver?)

The kernel in Dom0 is the 2.6.32-5-xen-amd64 as packaged in Debian, Xen package version is 4.0.1-2.

Any idea?
DomU-dmesg.txt (14.7 KB)
Dom0-xm_dmesg.txt (6.31 KB)

I’d like to do the same with an i5 CPU (Z68 chipset) with an nVidia geforce 560 TI card.

Have you had any luck with the GFX 580?

  • John