Dynamically change GPU instances using MIG?

Accelerated Computing CUDA CUDA Programming and Performance
1

Hello all,

I’m new to using MIG functionality. I wanted to know if there is support for dynamically changing mig profiles on a GPU. For example, on an A30, lets say that it is currently in MIG config 5 from the table. GPU slice #0 is running a workload. Is it possible to switch to configuration 4 without having to halt the running process on slice #0?

Thanks.

a30_configs
a30_configs777×298 11 KB

2

I’m not sure you can change this via config change. But you should be able to create and delete individual slices that are not running workloads. So you should be able to achieve something similar (to the example you indicate) via individual slices. The MIG user guide is here. This may also be of interest, in a k8s setting.

3

Would that be possible to do with the NVML library? I know it has functions to create and destroy GPU instances, but I have not tested the exact scenario I mentioned above yet. Thanks!

4

I don’t know much about MIG reconfiguration using NVML, but there is a separate forum for NVML questions. If I were trying this, I would first try it using functionality available in nvidia-smi, to convince myself of the proper path. I don’t have a recipe for you at the moment.

1 Like