H100 mig issue

I am having a problem in creating GPU instances on H100 after enabling “MIG”.

  1. Succesfully enabled mig with “sudo nvidia-smi mig 1”,
  2. However, passing “sudo nvidia-smi mig -cgi 19,1g.10gb -C” gives an error message “Unable to create a GPU instance on GPU 1 using profile 19: In use by another client
    Failed to create GPU instances: In use by another client”.
    Please help!
    Thank you.

I had the same issue. You should check the processes running on the NVIDIA GPUs by using the following command.

sudo fuser -v /dev/nvidia*

Usually, before being able to create GPU instances, I have to at least kill one process that’s running on the specific GPU.