Good morning eveybody.
Me and my supervisor are working on a project and we are trying to run Tensorflow inside a Docker image on our Jetson TX2 devevelopment board. We are running the L4T 32.4.4. The output of the jetsonInfo.py
script from jetsonhacks is the following:
vwm@sva-ms:~/$ python jetsonInfo.py
NVIDIA Jetson TX2
L4T 32.4.4 [ JetPack UNKNOWN ]
Ubuntu 18.04.5 LTS
Kernel Version: 4.9.140-tegra
CUDA 10.2.89
CUDA Architecture: 6.2
OpenCV version: 4.1.1
OpenCV Cuda: NO
CUDNN: 8.0.0.180
TensorRT: 7.1.3.0
Vision Works: 1.6.0.501
VPI: 0.4.4
Weirdly enough, I don’t get the Jetpack version. A run with apt-get gives me this:
vwm@sva-ms:~/drive$ sudo apt show nvidia-jetpack
Package: nvidia-jetpack
Version: 4.4.1-b50
Priority: standard
Section: metapackages
Maintainer: NVIDIA Corporation
Installed-Size: 199 kB
Depends: nvidia-cuda (= 4.4.1-b50), nvidia-opencv (= 4.4.1-b50), nvidia-cudnn8 (= 4.4.1-b50), nvidia-tensorrt (= 4.4.1-b50), nvidia-visionworks (= 4.4.1-b50), nvidia-container (= 4.4.1-b50), nvidia-vpi (= 4.4.1-b50), nvidia-l4t-jetson-multimedia-api (>> 32.4-0), nvidia-l4t-jetson-multimedia-api (<< 32.5-0)
Homepage: http://developer.nvidia.com/jetson
Download-Size: 29,4 kB
APT-Sources: https://repo.download.nvidia.com/jetson/t186 r32.4/main arm64 Packages
Description: NVIDIA Jetpack Meta Package
N: There are 2 additional records. Please use the '-a' switch to see them
Based on this, we pulled the image nvcr.io/nvidia/l4t-tensorflow:r32.4.4-tf2.3-py3
with the command:
sudo docker pull nvcr.io/nvidia/l4t-tensorflow:r32.4.4-tf2.3-py3
And we tried to run it with the command:
sudo docker run -it --rm --runtime nvidia --network host nvcr.io/nvidia/l4t-tensorflow:r32.4.4-tf2.3-py3
Only to get the following error:
vwm@sva-ms:~$ sudo docker run -it --rm --runtime nvidia --network host nvcr.io/nvidia/l4t-tensorflow:r32.4.4-tf2.3-py3
docker: Error response from daemon: OCI runtime create failed: container_linux.go:370:
starting container process caused: process_linux.go:459: container init caused:
Running hook #0:: error running hook: exit status 1, stdout: , stderr:
nvidia-container-cli: initialization error: driver error: failed to process request: unknown.
We also tried the L4T base image nvcr.io/nvidia/l4t-base:r32.4.4
only to get the exact same result:
vwm@sva-ms:~/drive$ sudo docker run -it --rm --runtime nvidia --network host nvcr.io/nvidia/l4t-base:r32.4.4
docker: Error response from daemon: OCI runtime create failed: container_linux.go:370:
starting container process caused: process_linux.go:459: container init caused:
Running hook #0:: error running hook: exit status 1, stdout: , stderr:
nvidia-container-cli: initialization error: driver error: failed to process request: unknown.
At this point, I’m not sure what is going on. I suspect something is wrong with the JetPack install, from the jetsonInfo.py
script output. But I don’t know what is wrong and how to proceed.
The outputs from docker info
:
vwm@sva-ms:~/drive$ sudo docker info
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 2
Server Version: 20.10.3
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux nvidia
Default Runtime: runc
Init Binary: docker-init
containerd version: 269548fa27e0089a8b8278fc4fc781d7f65a939b
runc version: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.140-tegra
Operating System: Ubuntu 18.04.5 LTS
OSType: linux
Architecture: aarch64
CPUs: 4
Total Memory: 7.676GiB
Name: sva-ms
ID: 5F4R:RSXN:BDZP:O2HN:JA7H:2Q3M:F3EP:7EAO:LRIN:WYK6:GSUP:2RDH
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No blkio weight support
WARNING: No blkio weight_device support
We also tried to update the Jetpack packages with this and it also didn’t work:
vwm@sva-ms:~/drive$ sudo apt install nvidia-jetpack
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
nvidia-jetpack : Depends: nvidia-container (= 4.4.1-b50) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
We are in the second day trying to make this thing work and nothing. I just keep receiving that Docker error driver error
and etc.
What is it not working?
Thanks in advance for any help.