Docker run deepstream error:OCI runtime create failed: container_linux.go:345 in Jetson NX

sudo docker run -it --rm --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/deepstream-peopledetection:r32.4.2 deepstream-test5-app -c deepstream-5.0/samples/configs/deepstream-app/sourceX_1080p_dec_infer-resnet_tracker_tiled_display_int8_hq_dla_nx.txt

and I got this error as follow:

docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:430: container init caused \"process_linux.go:413: running prestart hook 0 caused \\\"error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: driver error: failed to process request\\\\n\\\"\"": unknown.

I check my docker info ,and got this:

 docker version
Client:
 Version:           19.03.6
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        369ce74a3c
 Built:             Fri Feb 28 23:47:53 2020
 OS/Arch:           linux/arm64
 Experimental:      false

Server:
 Engine:
  Version:          19.03.6
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       369ce74a3c
  Built:            Wed Feb 19 01:06:16 2020
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.2.6-0ubuntu1~18.04.2
  GitCommit:        
 runc:
  Version:          spec: 1.0.1-dev
  GitCommit:        
 docker-init:
  Version:          0.18.0
  GitCommit:        

docker info
Client:
 Debug Mode: false

Server:
 Containers: 2
  Running: 2
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 19.03.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: nvidia runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 
 runc version: 
 init version: 
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.9.140-tegra
 Operating System: Ubuntu 18.04.4 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 7.589GiB
 Name: incar-desktop
 ID: SRWJ:CMUN:N2J2:5Z6C:ZNQI:75IW:EWL4:B73Z:4SSC:Y3R4:XVE5:GPCX
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

but ,I got error when docker nvidia-container-cli

sudo nvidia-container-cli -k -d /dev/tty info

-- WARNING, the following logs are for debugging purposes only --

I0528 00:00:19.799962 17533 nvc.c:281] initializing library context (version=1.1.1, build=e5d6156aba457559979597c8e3d22c5d8d0622db)
I0528 00:00:19.800159 17533 nvc.c:255] using root /
I0528 00:00:19.800223 17533 nvc.c:256] using ldcache /etc/ld.so.cache
I0528 00:00:19.800260 17533 nvc.c:257] using unprivileged user 65534:65534
W0528 00:00:19.800735 17533 nvc.c:171] failed to detect NVIDIA devices
I0528 00:00:19.801205 17534 nvc.c:191] loading kernel module nvidia
E0528 00:00:19.801896 17534 nvc.c:193] could not load kernel module nvidia
I0528 00:00:19.801937 17534 nvc.c:203] loading kernel module nvidia_uvm
E0528 00:00:19.802571 17534 nvc.c:205] could not load kernel module nvidia_uvm
I0528 00:00:19.802600 17534 nvc.c:211] loading kernel module nvidia_modeset
E0528 00:00:19.803045 17534 nvc.c:213] could not load kernel module nvidia_modeset
I0528 00:00:19.803727 17535 driver.c:101] starting driver service
E0528 00:00:19.804331 17535 driver.c:161] could not start driver service: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory
I0528 00:00:19.804644 17533 driver.c:196] driver service terminated successfully
nvidia-container-cli: initialization error: driver error: failed to process request

why I cannot found nvidia driver? and how can I fix it ?

**• Hardware Platform :Jetson NX **
• Deepstream 5
• JetPack Version 4.4DP
• TensorRT 10.2
• NVIDIA GPU Driver Version (valid for GPU only)

Hi,

Suppose your nvidia-docker is installed via sdkmanager directly.
Is that correct?

Thanks.

how can I use sdkmanager in jetson nx ?

Hi,

Sorry for the non-clear statement .
Suppose you have run sdkmanager on the host to flash and install all the package for XavierNX.
Is the nvidia-docker is pre-installed from SDKmanager rather than manually install from apt?

The reason for this is that you will need to use the package from SDKmanager to make sure the compatibility.

Thanks.

Hi,
I’m seeing a similar issue. I can’t run any of the images and failing with the same error.
"Suppose you have run sdkmanager on the host to flash and install all the package for XavierNX.
" —> Yes.

This is my info:

:~/docker$ head -1 /etc/nv_tegra_release

R32 (release), REVISION: 2.1, GCID: 16294929, BOARD: t186ref, EABI: aarch64, DATE: Tue Aug 13 04:45:36 UTC 2019

:~/docker$ sudo docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
nvcr.io/nvidia/l4t-base r32.4.2 a36d66956611 8 weeks ago 1.04GB
arm64v8/hello-world latest a29f45ccde2a 5 months ago 9.14kB
nvidia/cuda latest 9e47e9dfcb9a 6 months ago 2.83GB
nvcr.io/nvidia/l4t-base r32.3.1 aaaa63e7b12d 6 months ago 1.03GB
nvcr.io/nvidia/cuda 10.0-runtime 6515014535d6 6 months ago 903MB
nvidia/cuda 10.0-base 841d44dd4b3c 6 months ago 110MB
nvcr.io/nvidia/l4t-base r32.2.1 593e936de911 9 months ago 1.06GB
nvcr.io/nvidia/l4t-base r32.2 e3c56b78e93f 10 months ago 1.06GB
nvcr.io/nvidia/l4t-base r32.1 9f4d8a642cfb 11 months ago 993MB

sudo nvidia-docker run -it --rm --runtime nvidia nvcr.io/nvidia/cuda:10.0-runtime
docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused “process_linux.go:449: container init caused "process_linux.go:432: running prestart hook 1 caused \"error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: driver error: failed to process request\\n\""”: unknown.

sudo docker run -it --rm --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/l4t-base:r32.2.1
docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused “process_linux.go:449: container init caused "process_linux.go:432: running prestart hook 0 caused \"error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: driver error: failed to process request\\n\""”: unknown.

Any idea?

Hi,

It looks like your docker is running rel32.2.1 OS but XavierNX is using rel32.4.2.
So please update your image to ‘nvcr.io/nvidia/l4t-base:r32.4.2’ to ensure the OS compatibility.
NVIDIA L4T Base | NVIDIA NGC

Thanks.

1 Like