Where does GPU virtualization occur?

Virtual GPU Forums More vGPU Forums NVIDIA Virtual GPU Technology
1

Where does GPU virtualization occur? Does it occur within the GRID cards and then the vGPUs are presented to the hypervisor and then onto the guests? Or, is the virtualization and scheduling of the GPU actually happen with the GRID manager software installed on the hypervisor? Is SR-IOV used?

I have scoured the internet for this answer and have found nothing that is official and definitive.

Thanks!!

R

2

doesn’t your other post answer this question?

3

There are two drivers involved in the process. One driver resides in the hypervisor to manage GPU state, and one driver resides in the VM itself, to manage graphics API calls made to the graphics card, on the PCIe bus. Memory is statically divided, per the hypervisor/VM configuration. See here http://blogs.citrix.com/2013/10/25/vgpu-a-developer-to-developer-deep-dive-with-xenserver-engineering/ for more information.

4

Thanks nvEric,

I did some further investigation and here’s a paraphrasing of what I was told.

The GRID cards essentially present a full GPU to the hypervisor. The GRID Manager software, installed in Dom0, is what carves up the GPUs and presents the vGPU profiles. The NVIDIA driver in the guest OS is the same as a regular NVIDIA driver that would be installed on a physical workstation. Basically, the virtualization of the GPU occurs in the hypervisor/Dom0 and not in the GRID cards or in the guest OS.

Assuming that the above is correct, I have a few other questions. The scheduling of the VMs’ access to the GPU cores is done within the hypervisor, correct? If so, is the hypervisor itself doing the scheduling, or does the GRID manager do it, or both in-tandem?

Is SR-IOV the method used to present a single GPU as multiple vGPUs, or is that a proprietary technology from NVIDIA?

Thanks all!

Richard

5

Whoever told you that was misguided.

The technology is proprietary to NVIDIA and the virtualisation occurs at the hardware level.

Scheduling is handled by the scheduler in the GPU, at the GPU hardware level.

vGPU manager manages which VM talks to which GPU, and what GPU’s are currently running which profile to determine where other vGPU sessions can be placed.

Essentially, when a VM boots vGPU manager determines where to place the vGPU based on the profile, placement policies and available resources and then provides the details of which channels the driver should use to post work to the GPU. Once that’s done, vGPU manager gets out of the way and the comms are direct to the GPU.

vGPU manager is then simply monitoring whether the VM is still running. Once the VM is shut down vGPU manager can now allocate another VM to that GPU when it boots.

Have a look at some of the recordings from GTC last year, there’s a couple that cover this.

https://gridforums.nvidia.com/default/topic/11/?comment=28

It’s also in my partner training which will be coming to GPU Genius next year.

6

Thanks for the explanation, Jason. I have listened to most all of the GTC sessions. I think the deepest dive into how GPU virtualization works is Andy Currid’s session, which was really good. However, it did not address my questions specifically.

I would love to hear some even deeper dives at GTC this year.

Thanks again!

Richard

7

Hi Jason,

I have a question about your explanation above regarding where GPU virtualization occurs. Here’s the line, "Scheduling is handled by the scheduler in the GPU, at the GPU hardware level."

When you say, "in the gpu," do you mean within the GRID card? Or, do you mean that the GPU chip actually has some scheduling technology baked in? If it is the former, I am assuming there is another chip outside of the GPUs that is doing the scheduling of the cores.

Thanks again!

Richard

8

Scheduling is handled in the GPU itself.

9

Got it. Very interesting. Thanks Jason!

Richard