Nvidia-smi not available inside docker

Graphics / Linux Linux
1

I’m setting up an Akash provider node on Ubuntu 24.04 with an NVIDIA RTX 3090 GPU. On the host machine, everything works fine:
nvidia-smi, Returns the expected GPU information (driver: 575.64.03, CUDA version: 12.9).

However, inside Docker containers (using --gpus all and --runtime=nvidia),

Container Behavior:

  1. When I run:

bash

CopyEdit

docker run --rm --gpus all --runtime=nvidia nvidia/cuda:12.9.0-base-ubuntu20.04 nvidia-smi

I get:

bash

CopyEdit

/bin/bash: nvidia-smi: command not found

When I run CUDA workload tests like:

bash

CopyEdit

docker run --gpus all --env NVIDIA_DISABLE_REQUIRE=1 nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark

It works fine — GPU is utilized correctly.

What I’ve Verified:

  • NVIDIA drivers (575.64.03) are correctly installed on the host.
  • nvidia-smi works on the host without any issue.
  • Docker is using nvidia-container-toolkit and --gpus all runtime.
  • I can run CUDA programs in containers, but nvidia-smi is missing inside containers.
  • I also tried official images like nvidia/cuopt, nvidia/cuda, and nvcr.io/nvidia/k8s/cuda-sample.

❓ What I Need Help With:

  • Is it required for nvidia-smi to work inside containers for Akash GPU workloads to function properly?
  • Is there a known workaround to get nvidia-smi working inside containers (do I need to mount host binaries like /usr/bin/nvidia-smi into the container)?
  • Can I safely ignore this if CUDA programs are running fine?
2

Install the nvidia-utils-xxx package corresponding to your driver. On nvcr.io/nvidia/cuda:latestthe newest option is:

Package: nvidia-utils-570
Version: 570.133.07-0ubuntu0.20.04.1
Priority: optional
Section: restricted/libs
Source: nvidia-graphics-drivers-570
Origin: Ubuntu
Maintainer: Ubuntu Core Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 1640 kB
Provides: nvidia-smi, nvidia-utils
Depends: libc6 (>= 2.7), libnvidia-compute-570 (>= 570.133.07)
Suggests: nvidia-driver-570
Conflicts: nvidia-smi, nvidia-utils
Replaces: nvidia-smi, nvidia-utils
Homepage: http://www.nvidia.com
Download-Size: 521 kB
APT-Manual-Installed: yes
APT-Sources: http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages
Description: NVIDIA driver support binaries

root@9b16faa7df03:~# command -v nvidia-smi
/usr/bin/nvidia-smi