cudaGetDeviceCount always return 1 on multiple MIG CIs env

I find cudaGetDeviceCount and cuDeviceGetCount always return 1 in MIG environment,even though there are multiple MIG CIs configured. I’m confused isn’t this returning the number of CIS available?

ENV: cuda 11.5 , PCIE-A100-40G have spilited into 7 x1g.5gb CIs

This is expected behavior. From here:

  1. CUDA can only enumerate a single compute instance

If you want to use multiple MIG instances, you must do it across separate processes. The linked document gives examples of using CUDA_VISIBLE_DEVICES to “assign” a MIG instance to each process.

1 Like