Docker not detecting Nvidia GPU

[sunnykotwal]

Hey having problems getting docker to recognize my GPU.

I am getting this error:
docker run -it --rm --gpus all --ipc=host --network host -v $(pwd):/workspace nvcr.io/nvidia/nemo:24.01.framework
docker: Error response from daemon: could not select device driver “” with capabilities: [[gpu]].

I am trying to get this container working for this older NeMo tutorial: GenerativeAIExamples/finetuning/NeMo/slm/slm_pretraining_sft.ipynb at main · NVIDIA/GenerativeAIExamples · GitHub

I think the problem is in the runtime with nvidia not showing up in the output from this line:
docker info | grep Runtimes
Runtimes: io.containerd.runc.v2 runc

I am using Ubuntu 24.04
Docker version 27.5.0, build a187fa5
Cuda compliation tools: version: release 12.0, V12.0.140

I have the Nvidia tool kit loaded as shown on:
https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

Added a screen shot of nvidia-smi from the terminal in case it helps.
Please let me know if any more info is needed to help locate issue.
Apologies if this is in wrong area of forums.

Thanks,
Sunny K

[sunnykotwal]

Finally got this to work using the steps here: Can't start containers with GPU access on Linux Mint - Docker Desktop - Docker Community Forums

[sophwats]

Pleased you were able to get this resolve. Thanks for sharing the solution!