Hello,
I’ve just grabbed the JetPack 5.0 Developer Preview for Xavier NX (T194), and tried running a couple Docker containers on it.
I’ve managed to run the following Docker containers without any trouble. Alpine:
$ docker run -ti --rm alpine true
$ docker run -ti --rm --runtime nvidia alpine true
$ docker run -ti --rm --privileged alpine true
$ docker run -ti --rm --privileged --runtime nvidia alpine true
The super freshly just-dropped r34.1 release of L4T:
$ docker run -ti --rm nvcr.io/nvidia/l4t-base:r34.1 true
$ docker run -ti --rm --privileged nvcr.io/nvidia/l4t-base:r34.1 true
$ docker run -ti --rm --runtime nvidia nvcr.io/nvidia/l4t-base:r34.1 true
All of the above 7 variants work fine. However, combining the privileged flag with the NVidia Docker runtime, on this specific image, causes a lengthy wall of errors:
$ docker run -ti --rm --privileged --runtime nvidia nvcr.io/nvidia/l4t-base:r34.1 true
docker: Error response from daemon: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #1:: error running hook: exit status 1, stdout: src: /etc/vulkan/icd.d/nvidia_icd.json, src_lnk: /usr/lib/aarch64-linux-gnu/tegra/nvidia_icd.json, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/etc/vulkan/icd.d/nvidia_icd.json, dst_lnk: /usr/lib/aarch64-linux-gnu/tegra/nvidia_icd.json
src: /usr/lib/aarch64-linux-gnu/libcuda.so, src_lnk: tegra/libcuda.so, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/libcuda.so, dst_lnk: tegra/libcuda.so
src: /usr/lib/aarch64-linux-gnu/libnvcucompat.so, src_lnk: tegra/libnvcucompat.so, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/libnvcucompat.so, dst_lnk: tegra/libnvcucompat.so
src: /usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so, src_lnk: tegra/libnvidia-nvvm.so, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so, dst_lnk: tegra/libnvidia-nvvm.so
src: /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.999999, src_lnk: tegra/libnvv4l2.so, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.999999, dst_lnk: tegra/libnvv4l2.so
src: /usr/lib/aarch64-linux-gnu/libv4lconvert.so.0.0.999999, src_lnk: tegra/libnvv4lconvert.so, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/libv4lconvert.so.0.0.999999, dst_lnk: tegra/libnvv4lconvert.so
src: /usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvargus.so, src_lnk: ../../../tegra/libv4l2_nvargus.so, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvargus.so, dst_lnk: ../../../tegra/libv4l2_nvargus.so
src: /usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvcuvidvideocodec.so, src_lnk: ../../../tegra/libv4l2_nvcuvidvideocodec.so, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvcuvidvideocodec.so, dst_lnk: ../../../tegra/libv4l2_nvcuvidvideocodec.so
src: /usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvvideocodec.so, src_lnk: ../../../tegra/libv4l2_nvvideocodec.so, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvvideocodec.so, dst_lnk: ../../../tegra/libv4l2_nvvideocodec.so
src: /usr/lib/aarch64-linux-gnu/libvulkan.so.1.3.203, src_lnk: tegra/libvulkan.so.1.3.203, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/libvulkan.so.1.3.203, dst_lnk: tegra/libvulkan.so.1.3.203
src: /usr/lib/aarch64-linux-gnu/tegra/libcuda.so, src_lnk: libcuda.so.1.1, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libcuda.so, dst_lnk: libcuda.so.1.1
src: /usr/lib/aarch64-linux-gnu/tegra/libgstnvdsseimeta.so, src_lnk: libgstnvdsseimeta.so.1.0.0, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libgstnvdsseimeta.so, dst_lnk: libgstnvdsseimeta.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvbufsurface.so, src_lnk: libnvbufsurface.so.1.0.0, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libnvbufsurface.so, dst_lnk: libnvbufsurface.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvbufsurftransform.so, src_lnk: libnvbufsurftransform.so.1.0.0, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libnvbufsurftransform.so, dst_lnk: libnvbufsurftransform.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so, src_lnk: libnvbuf_utils.so.1.0.0, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so, dst_lnk: libnvbuf_utils.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvdsbufferpool.so, src_lnk: libnvdsbufferpool.so.1.0.0, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libnvdsbufferpool.so, dst_lnk: libnvdsbufferpool.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvidia-nvvm.so, src_lnk: libnvidia-nvvm.so.4, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libnvidia-nvvm.so, dst_lnk: libnvidia-nvvm.so.4
src: /usr/lib/aarch64-linux-gnu/tegra/libnvidia-nvvm.so.4, src_lnk: libnvidia-nvvm.so.4.0.0, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libnvidia-nvvm.so.4, dst_lnk: libnvidia-nvvm.so.4.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so, src_lnk: libnvid_mapper.so.1.0.0, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so, dst_lnk: libnvid_mapper.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvscibuf.so, src_lnk: libnvscibuf.so.1, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libnvscibuf.so, dst_lnk: libnvscibuf.so.1
src: /usr/lib/aarch64-linux-gnu/tegra/libnvscicommon.so, src_lnk: libnvscicommon.so.1, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libnvscicommon.so, dst_lnk: libnvscicommon.so.1
src: /usr/lib/aarch64-linux-gnu/tegra/libnvscistream.so, src_lnk: libnvscistream.so.1, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libnvscistream.so, dst_lnk: libnvscistream.so.1
src: /usr/lib/aarch64-linux-gnu/tegra/libnvscisync.so, src_lnk: libnvscisync.so.1, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/lib/aarch64-linux-gnu/tegra/libnvscisync.so, dst_lnk: libnvscisync.so.1
src: /usr/share/glvnd/egl_vendor.d/10_nvidia.json, src_lnk: ../../../lib/aarch64-linux-gnu/tegra-egl/nvidia.json, dst: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/usr/share/glvnd/egl_vendor.d/10_nvidia.json, dst_lnk: ../../../lib/aarch64-linux-gnu/tegra-egl/nvidia.json
, stderr: nvidia-container-cli: mount error: file creation failed: /var/lib/docker/overlay2/7fa5fde86f94e78b85f15f95e051bded2eb32e2e192a6dc776a8b275ac16e245/merged/dev/nvhost-as-gpu: invalid argument: unknown.
I did remove the following packages from the host OS to get a slimmed down, headless environment:
sudo apt-get remove --autoremove -qy \
lxde 'lxde-*' lxsession unity 'unity-*' 'gnome-*' xorg 'xserver-*' 'fonts-*' \
lightdm gdm3 'update-*' 'vlc-*' 'rhythmbox-*' 'libreoffice-*' orca 'cuda-*' \
'chromium-*' 'python2.*' 'python3.*' 'gstreamer1.0-*' 'xdg-*' 'x11-*' \
'printer-*' ghostscript
An identical test on the stable release of JetPack, with the same L4T base image on the corresponding tag (r32.6.1), with the same flags, on the same hardware, has been working fine.
I can re-flash the SD card and try again if removing the above packages is the suspected root cause, although I would love to keep the base system slimmed down.