cudaGetDeviceCount always return 1 on multiple MIG CIs env

Accelerated Computing CUDA CUDA Programming and Performance
1

Hello,
I find cudaGetDeviceCount and cuDeviceGetCount always return 1 in MIG environment，even though there are multiple MIG CIs configured. I’m confused isn’t this returning the number of CIS available？

ENV： cuda 11.5 ， PCIE-A100-40G have spilited into 7 x1g.5gb CIs

2

This is expected behavior. From here:

  1. CUDA can only enumerate a single compute instance

If you want to use multiple MIG instances, you must do it across separate processes. The linked document gives examples of using CUDA_VISIBLE_DEVICES to “assign” a MIG instance to each process.

1 Like