Trouble trying to install torch in Docker container on JP6.0-dp

System info:
Device: Jetson Orin NX 16GB
Jetpack: 6.0-dp (freshly flashed, clean system)
CUDA: 12.2
torch: (I tried using nv.24.02 as well)
MarkupSafe: MarkupSafe-2.1.5-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Additional info:

  1. Using Hatch I am able to get "python3 -c “import torch; print(torch.cuda.is_available())” → True. (Hatch is basically just virtualenv in this case)
  2. The regular cpu arm64 version of torch installs fine, but torch.cuda.is_available() is False… of course.
  3. The output is from tmux with funky formatting. I tried to fix it a bit, but it may still be funky.

jet@ubuntu:~$ sudo docker run -it --runtime nvidia bash → success
root@hostname:/# nvidia-smi → success
root@hostname:/# apt update && apt install -y python3-pip libopenblas-dev libopenmpi3 → success
root@hostname:/# pip3 install -i torch → success
root@hostname:/# python3 -c “import torch; print(torch.cuda.is_available())” → failure
Traceback (most recent call last):
File “/usr/local/lib/python3.10/dist-packages/torch/”, line 175, in _load_global_deps ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
File “/usr/lib/python3.10/ctypes/”, line 374, in init
self._handle = _dlopen(self._name, mode)
OSError: cannot open shared object file: No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “”, line 1, in
File “/usr/local/lib/python3.10/dist-packages/torch/”, line 235, in _load_global_deps()
File “/usr/local/lib/python3.10/dist-packages/torch/”, line 196, in _load_global_deps_preload_cuda_deps(lib_folder, lib_name)
File “/usr/local/lib/python3.10/dist-packages/torch/”, line 161, in _preload_cuda_deps
raise ValueError(f"{lib_name} not found in the system path {sys.path}")
ValueError:*[0-9] not found in the system path [‘’, ‘/usr/lib/’
, ‘/usr/lib/python3.10’, ‘/usr/lib/python3.10/lib-dynload’, ‘/usr/local/lib/python3.10/dis
t-packages’, ‘/usr/lib/python3/dist-packages’, ‘/usr/lib/python3.10/dist-packages’]

Questions (Really I’m just looking for general advice on how to proceed from here):

  1. This dl error appears to be from a cuda version mismatch. Is this a mismatch between the version of cuda pytorch was compiled for vs what’s on my JP6.0-dp system? JP6.0-dp uses CUDA12.2.
  2. Is installing pytorch manually from l4t-base even feasible or is l4t-pytorch for jp6.0 really the only option? Is that coming out soon?
  3. Should we give up on Docker and port to a system level deployment? This is possible, but a lot of effort and technical debt :(


1. l4t-base doesn’t have CUDA preinstalled. It only contains the basic OS.
Please try l4t-cuda instead.

2. You can try l4t-ml.

3. You don’t need to. Please try the above container first.


1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.