Nvidia TAO cuda version error

• Hardware (T4/V100/Xavier/Nano/etc)
T4
• Network Type
Detectnet_v2

When I run below TAO command it throws cuda error
tao detectnet_v2 --help

2021-09-06 13:34:20,048 [INFO] root: Registry: [‘nvcr.io’]
2021-09-06 13:34:20,149 [WARNING] tlt.components.docker_handler.docker_handler:
Docker will run the commands as root. If you would like to retain your
local host permissions, please add the “user”:“UID:GID” in the
DockerOptions portion of the “/home/ubuntu/.tao_mounts.json” file. You can obtain your
users UID and GID by using the “id -u” and “id -g” commands on the
terminal.
Docker instantiation failed with error: 500 Server Error: Internal Server Error ("OCI runtime create failed: container_linux.go:367: starting container process caused: process_linux.go:495: container init caused: Running hook #0:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: requirement error: unsatisfied condition: cuda>=11.1, please update your driver to a newer version, or use an earlier cuda container: unknown")

Thanks!

What is the driver of your T4?
$ nvidia-smi

Hi Morgan,

±----------------------------------------------------------------------------+
| NVIDIA-SMI 450.119.03 Driver Version: 450.119.03 CUDA Version: 11.0 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla T4 On | 00000000:00:1E.0 Off | 0 |
| N/A 38C P8 10W / 70W | 0MiB / 15109MiB | 0% Default |
| | | N/A |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
±----------------------------------------------------------------------------+

Thanks

Please update to 455 or above.
See TAO Toolkit Quick Start Guide — TAO Toolkit 3.22.05 documentation

1 Like

Thanks Morgan. It works after upgrading to 455 as you said.