Wondering how difficult would it be to run the Nvidia containers in a VM on a host that has multiple GPU cards ( RTX 2080 etc ).
What would be the configuration ?
What would the performance penalty , if any ?
There would definitely be a performance penalty. The VM has to virtualize the CPU, RAM, GPU interface, while the container just virtualizes the OS - disk reading/writing. Once everything is dropped into the GPU it should have the same performance, but the pipeline in and out will slower in the VM and running in native speed in the container. Here’s a good, quick intro to the differences and advantages of each: Docker Containers vs. VMs: A Look at the Pros and Cons
However, putting a container in a VM is never going to be a win – with putting one virtualization inside the other, you will be getting both performance penalties added together at best; at worst, the VM’s virtualizations could interfere with the container’s virtualizations and multiply the performance penalties.
I am having the issue with everything working great on my slow but steady Linux server and struggling putting together the CUDA drivers with Tensorflow-GPU on my gaming windows machine. I think for your case and mine, dual-booting into Linux will solve our problems. I’m here looking for one last guide to Windows Tensorflow-GPU to try, and then I’m going to install a dual-booting Ubuntu. If that’s not palatable for you, look into Windows Docker, a container solution. Unless you really need the hardware isolation of VM’s, the container solution is closer to the metal and usually has a lower penalty. However, dual-booting into Linux has zero penalty.
Good luck!
So, do the cloud provider’s (e.g. AWS, GCP) GPU instances suffer from performance penalty too ? They are running in VMs.