Problem to build a docker container and use the GPU on JETSON AGX ORIGIN

david.mataix4054 · August 17, 2023, 7:55am

When i launch the command:

 sudo docker run -it --runtime nvidia --network host nvcr.io/nvidia/l4t-pytorch:r34.1.1-pth1.11-py3

Appear the following error:

sudo docker run -it --runtime nvidia --network host nvcr.io/nvidia/l4t-pytorch:r34.1.1-pth1.11-py3
docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: src: /etc/vulkan/icd.d/nvidia_icd.json, src_lnk: /usr/lib/aarch64-linux-gnu/tegra/nvidia_icd.json, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/etc/vulkan/icd.d/nvidia_icd.json, dst_lnk: /usr/lib/aarch64-linux-gnu/tegra/nvidia_icd.json
src: /usr/lib/aarch64-linux-gnu/libcuda.so, src_lnk: tegra/libcuda.so, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/libcuda.so, dst_lnk: tegra/libcuda.so
src: /usr/lib/aarch64-linux-gnu/libnvcucompat.so, src_lnk: tegra/libnvcucompat.so, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/libnvcucompat.so, dst_lnk: tegra/libnvcucompat.so
src: /usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so, src_lnk: tegra/libnvidia-nvvm.so, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so, dst_lnk: tegra/libnvidia-nvvm.so
src: /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.999999, src_lnk: tegra/libnvv4l2.so, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.999999, dst_lnk: tegra/libnvv4l2.so
src: /usr/lib/aarch64-linux-gnu/libv4lconvert.so.0.0.999999, src_lnk: tegra/libnvv4lconvert.so, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/libv4lconvert.so.0.0.999999, dst_lnk: tegra/libnvv4lconvert.so
src: /usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvargus.so, src_lnk: ../../../tegra/libv4l2_nvargus.so, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvargus.so, dst_lnk: ../../../tegra/libv4l2_nvargus.so
src: /usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvcuvidvideocodec.so, src_lnk: ../../../tegra/libv4l2_nvcuvidvideocodec.so, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvcuvidvideocodec.so, dst_lnk: ../../../tegra/libv4l2_nvcuvidvideocodec.so
src: /usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvvideocodec.so, src_lnk: ../../../tegra/libv4l2_nvvideocodec.so, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvvideocodec.so, dst_lnk: ../../../tegra/libv4l2_nvvideocodec.so
src: /usr/lib/aarch64-linux-gnu/libvulkan.so.1.3.203, src_lnk: tegra/libvulkan.so.1.3.203, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/libvulkan.so.1.3.203, dst_lnk: tegra/libvulkan.so.1.3.203
src: /usr/lib/aarch64-linux-gnu/tegra/libcuda.so, src_lnk: libcuda.so.1.1, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libcuda.so, dst_lnk: libcuda.so.1.1
src: /usr/lib/aarch64-linux-gnu/tegra/libgstnvdsseimeta.so, src_lnk: libgstnvdsseimeta.so.1.0.0, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libgstnvdsseimeta.so, dst_lnk: libgstnvdsseimeta.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvbufsurface.so, src_lnk: libnvbufsurface.so.1.0.0, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libnvbufsurface.so, dst_lnk: libnvbufsurface.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvbufsurftransform.so, src_lnk: libnvbufsurftransform.so.1.0.0, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libnvbufsurftransform.so, dst_lnk: libnvbufsurftransform.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so, src_lnk: libnvbuf_utils.so.1.0.0, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so, dst_lnk: libnvbuf_utils.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvdsbufferpool.so, src_lnk: libnvdsbufferpool.so.1.0.0, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libnvdsbufferpool.so, dst_lnk: libnvdsbufferpool.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvidia-nvvm.so, src_lnk: libnvidia-nvvm.so.4, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libnvidia-nvvm.so, dst_lnk: libnvidia-nvvm.so.4
src: /usr/lib/aarch64-linux-gnu/tegra/libnvidia-nvvm.so.4, src_lnk: libnvidia-nvvm.so.4.0.0, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libnvidia-nvvm.so.4, dst_lnk: libnvidia-nvvm.so.4.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so, src_lnk: libnvid_mapper.so.1.0.0, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so, dst_lnk: libnvid_mapper.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvscibuf.so, src_lnk: libnvscibuf.so.1, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libnvscibuf.so, dst_lnk: libnvscibuf.so.1
src: /usr/lib/aarch64-linux-gnu/tegra/libnvscicommon.so, src_lnk: libnvscicommon.so.1, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libnvscicommon.so, dst_lnk: libnvscicommon.so.1
src: /usr/lib/aarch64-linux-gnu/tegra/libnvscistream.so, src_lnk: libnvscistream.so.1, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libnvscistream.so, dst_lnk: libnvscistream.so.1
src: /usr/lib/aarch64-linux-gnu/tegra/libnvscisync.so, src_lnk: libnvscisync.so.1, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/lib/aarch64-linux-gnu/tegra/libnvscisync.so, dst_lnk: libnvscisync.so.1
src: /usr/share/glvnd/egl_vendor.d/10_nvidia.json, src_lnk: ../../../lib/aarch64-linux-gnu/tegra-egl/nvidia.json, dst: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/share/glvnd/egl_vendor.d/10_nvidia.json, dst_lnk: ../../../lib/aarch64-linux-gnu/tegra-egl/nvidia.json
, stderr: nvidia-container-cli: mount error: stat failed: /var/lib/docker/overlay2/8190cabf21d73bbfb7ca56dacd037479fa105fb88515b9c8c128d53b69844a8a/merged/usr/local/cuda-11.4/compat/libcuda.so.1: no such file or directory: unknown.
ERRO[0000] error waiting for container: context canceled

I am trying to build a docker container in a Jetson AGX Origin, but I could not do it, I think that I am having problems with the CUDA version installed on the Jetson.
When I do nvcc --version appear CUDA 11.4, but when I do jetson_release -v appear CUDA 12.1.
I do not know if I could change the CUDA version in the jetson_release, or any idea of what is the problem?.
Other information of the installation:
JetPack version: 5.0.1DP
L4T: 34.1.1
cuDNN: 8.3.2.49
TensorRT: 5.0.1
Thanks!

kayccc · August 29, 2023, 8:42am

Sorry for the late response, have you managed to get issue resolved or still need the support? Thanks

AastaLLL · August 30, 2023, 3:00am

Hi,

r34 (JetPack 5.0.1 DP) is a DP release.
Would you mind upgrading your environment into a GA release (ex. 5.0.2 or the latest 5.1.2?) and trying it again?

Thanks.

system · September 26, 2023, 5:43am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Error running privileged L4T containers with the NVidia Runtime on JetPack 5.0 Developer Preview Jetson Xavier NX jetpack , docker	9	4405	June 14, 2022
Cannot run docker containers, incorrect nvidia-container-toolkit dependencies? Jetson AGX Orin docker , containers , jetson	4	130	February 12, 2025
Failed to run docker container in privileged mode on Jetson AGX Orin Jetson AGX Orin ros , docker	3	1123	July 8, 2022
Docker build + sudden repository change Jetson AGX Orin nvbugs , docker	22	1514	May 31, 2023
Cannot start base nvidia's docker images on Jetpack 4.6.2 Jetson AGX Xavier docker	4	830	June 13, 2022
NVIDIA docker issue with privileged Jetson AGX Orin docker	5	2154	April 29, 2022
Creating Containers Using nvidia-docker with AGX Xavier Jetson AGX Xavier docker	8	2745	October 18, 2021
Dependencies missing when building VPI with docker build Jetson AGX Orin vpi	3	105	August 13, 2024
Unable to create a docker to run CUDA on JETSON AGX ORIN for opencv Jetson AGX Orin cuda , docker	3	310	August 1, 2024
Containerization using docker of deepstream python script DeepStream SDK deepstream	14	57	December 18, 2024

Problem to build a docker container and use the GPU on JETSON AGX ORIGIN

Related topics