2 64 GB VMs sharing same GRID K2 GPU

Hi,

I have a Dell R720 equiped with 2 GRID K2 boards. When I try to start 2 virtual machines with 64GB memory and sharing same GPU, the GPU of second VM doesn’t work properly. Device manager shows an exclamation on K260Q GPU and nvidia-smi shows only 1 vGPU process. Anybody is using this setup? 2 VMs with 64GB RAM and sharing the same GPU? Thank you.

What is the OS and exact configuration of your two VMs (number of VCPUs, memory,e tc.)? Are Xentools properly installed? I take it you are running each with a K260Q instance for each, but are they coming from the same K2 or different K2 boards? Does nvidia-smi.exe show the allocations properly?
Are you running the latest R720 firmware and the latest GRID drivers? Above all, have you ever had this work on the system before? You could try removing one K2 and see if you can at least get the two VMs to share the one as a first step.

Hi Tobias,

I’m using Windows 7 x64, 8 vCPUs, 64GB RAM for each VM. Xen Tools installed. They are using the same K2 GPU. nvidia-smi doesn’t show allocations properly. If I change one VM to 56GB, both VMs work. Yes, latest firmware and drivers. We have the same issue with a Cisco server. If 2 VMs share a GPU with more than 56GB of memory, the GPU only works for the VM started first.

Odd, as Windows 7 64-bit should support up to 128 GB of RAM. Do you have bad RAM perhaps in your server?
How much RAM is in your server? You could also try swapping DIMMs around to see if that makes a difference or running a comprehensive memory test on your server. I’d also make sure your firmware/BIOS is fully up-to-date.

No no, there is no problem with memory. We have a Cisco server with same issue. We have 256GB on this server. Citrix/Nvidia said today that it is a know problem. They said it will be fixed on next driver release.

Ah, good to know it’s not your setup! :-)