Error while running TLT Docker

Hi, I am new on TLT, I am getting error while running

sudo docker run --runtime=nvidia -it -v /home/ubuntu/Downloads/Transfer_learning_toolkit/docker/tlt-experiments:/workspace/tlt-experiments nvcr.io/nvidia/tlt-streamanalytics:v1.0.1_py2 /bin/bash
docker: Error response from daemon: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v1.linux/moby/44e242ca1a85153a0c0c4e6932d13e9adabf37e7d20ed3c784b9f2f4a07f007e/log.json: no such file or directory): exec: "nvidia-container-runtime": executable file not found in $PATH: : unknown.
ERRO[0000] error waiting for container: context canceled

please help me out.
thanks.

Hi pritam,
Can you please run below command and paste the result?
$ sudo docker run --runtime=nvidia -it nvcr.io/nvidia/tlt-streamanalytics:v1.0.1_py2 nvidia-smi

Reference:
https://github.com/NVIDIA/nvidia-docker/issues/388
https://stackoverflow.com/questions/53171188/why-is-my-nvidia-docker-not-working-error-response-from-daemon?r=SearchResults

Hi morganh,
I have run this command but i am getting same issue.

ubuntu@ubuntu-B365M-D3H:~/Downloads/TLT_Experiments$ sudo docker run --runtime=nvidia -it nvcr.io/nvidia/tlt-streamanalytics:v1.0.1_py2 nvidia-smi
docker: Error response from daemon: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v1.linux/moby/7b0bb1604671c3f6fd80dcf3eb9031d654e519483f6746e9ffe1d77124a03d3a/log.json: no such file or directory): exec: "nvidia-container-runtime": executable file not found in $PATH: : unknown.
ERRO[0000] error waiting for container: context canceled 

ubuntu@ubuntu-B365M-D3H:~/Downloads/TLT_Experiments$ sudo docker logout nvcr.io
Removing login credentials for nvcr.io

ubuntu@ubuntu-B365M-D3H:~/Downloads/TLT_Experiments$ sudo docker login nvcr.io
Username: $oauthtoken
Password: 
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

ubuntu@ubuntu-B365M-D3H:~/Downloads/TLT_Experiments$ sudo docker pull nvcr.io/nvidia/tlt-streamanalytics:v1.0.1_py2
v1.0.1_py2: Pulling from nvidia/tlt-streamanalytics
Digest: sha256:cce954d342d1f3e2a0bb3a74e48c92b4089d24a1a1d5a8de0ef0d28316403163
Status: Image is up to date for nvcr.io/nvidia/tlt-streamanalytics:v1.0.1_py2
nvcr.io/nvidia/tlt-streamanalytics:v1.0.1_py2

ubuntu@ubuntu-B365M-D3H:~/Downloads/TLT_Experiments$ sudo docker run --runtime=nvidia -it -v /home/ubuntu/Downloads/TLT_Experiments/Head_Detection/tlt-experiments:/workspace/tlt-experiments -p 8888:8888 nvcr.io/nvidia/tlt
Unable to find image 'nvcr.io/nvidia/tlt:latest' locally
docker: Error response from daemon: pull access denied for nvcr.io/nvidia/tlt, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.

ubuntu@ubuntu-B365M-D3H:~/Downloads/TLT_Experiments$ sudo docker run --runtime=nvidia -it nvcr.io/nvidia/tlt-streamanalytics:v1.0.1_py2 nvidia-smi
docker: Error response from daemon: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v1.linux/moby/f13dbd1b04d41564dac28d962448faa3ce48b28f3c46f547c359bd9b8e08e3bd/log.json: no such file or directory): exec: "nvidia-container-runtime": executable file not found in $PATH: : unknown.
ERRO[0000] error waiting for container: context canceled

This is what i have done please suggest what should I do next.
It was working pretty fine and i have done my training on TLT but I have downgrade the version of tensorRT and cuda after that DS3 is working fine but TLT is giving error like this.

Can you refer to https://github.com/NVIDIA/nvidia-docker/issues/388 to do some triage?

Thanks Morganh issue resolved.

Good to know. I’m closing this topic. Thanks.