Is GPU memory oversubscription allowed in MIG?

Accelerated Computing CUDA CUDA Programming and Performance
1

Whether MIG instances can share memory resources with GPU memory oversubscription?
Is there any way for MIG instances to share the same physical memory?

Best
Max

2

Yes, MIG instances can use managed memory, including oversubscription. I don’t think this constitutes “sharing”, however. I’m not sure what “sharing” means in the context of oversubscription.

I don’t really understand the second question, but I think the answer is probably no. MIG instances don’t share the same physical memory on the device side except via limited IPC possibilities, nor do they share the same physical memory on the host side except via Host-based IPC.

Remember that multiple MIG instances cannot be exposed/enumerated to or used by a single process.

The MIG User Guide may be of interest.

1 Like
3

Thanks.

4

Hi Robert,
I get another question that
For example, there is one 2-GPCs instance, and I will try to split it into two 1-GPC instance.
Is there any way I could keep the model on instance memory and still split it? Then one instance could access the model later.

Best
Tianyu

5

No. Redefining the MIG configuration will wipe out any contexts that were previously on those MIG slices/instances.