Nvida Container Toolkit: Failed to initialize NVML: Unknown Error

I solved it, with the help of https://bobcares.com/blog/docker-failed-to-initialize-nvml-unknown-error/ !

I used just the second part of “Method 1”:

  1. nvidia-container configuration
    In the file
/etc/nvidia-container-runtime/config.toml

set the parameter

no-cgroups = false

After that restart docker and run test container:

sudo systemctl restart docker
sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

With that, the container loaded and worked immediately. I have no idea why.

6 Likes