I’m running a personal project where I aim to run multiple algorithms that utilize GPU such as detection and tracking. I want to run both of these algorithms on a single GPU with near bare-metal performance. For this purpose I was planning on buying the L40 GPU since it has a vGPU framework support, but then I came across another framework called Multi-Instance GPU and it only supports Blackwell and Hopper generation GPUs. So I got confused which of these frameworks is more suitable for this high performance algorithm parallelization task? (Also keep in mind that the number of algorithms I want to implement may increase)
Well, I try to give a more generic answer: vGPU virtualizes the GPU in timesharing its GFX and compute resources, and that needs a license and a special driver in hypervisor as well as in guest OS. vGPU allows for a lot of flexibility in how to assign fractions of the GPU to users, from full GPU to single user up to 12/16/24 fragments, one per user…
Compared to MIG, vGPU might have less predictable, sometime longer latency, if the user/jobs needs to wait for his time to use all GPU resources is to come again…
MIG has a fraction of all GPU resources fix assigned to a user/tenant/job, but is much less flexible in how to change the fraction size per user/job. MIG is only avail on the high end datacenter GPUs, can fragment the GPU between a single or up to 7 users/jobs/instances, but to change assignment, all jobs need to be stopped, and GPU needs to be reconfigured and like re-set…
So basically with MIG you trade lower, more predictability of resources/response/result for easier and more flexible managebility via vGPU…
Hope this helps as some guidance…