Error importing torch inside a L4T container in Jetson Nano

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

My Jetpack version is L4T 32.6.1 [JetPack 4.6] and I’m using the image ’ l4t-pytorch:r32.6.1-pth1.9-py3. The problem is that when I’m trying to import torch this error appears:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/torch/__init__.py", line 195, in <module>
    _load_global_deps()
  File "/usr/local/lib/python3.6/dist-packages/torch/__init__.py", line 148, in _load_global_deps
    ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
  File "/usr/lib/python3.6/ctypes/__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libcurand.so.10: cannot open shared object file: No such file or directory

In order to give you more infomation:

* Ubuntu 18.04.6 LTS
* Kernel version: 4.9.253-tegra
* CUDA 10.2.300
* CUDA Architecture: 5.3
* OpenCV version: 4.1.1.
* OpenCV Cuda: NO
* CUDNN:8.2.1.32
* TensorRT: 8.2.1.8
* Docker version: 24.0.2

Thank you in advanced.

3

Hi,

JetPack 4.6 use r32.7.
Could you try if the r32.7-based image works? Ex. r32.7.1-pth1.10-py3

Thanks.

4

Hi,

I have tried with the r32.7 based images and I’m getting the same error.

docker run -it --runtime nvidia --network host -v /path  nvcr.io/nvidia/l4t-pytorch:r32.7.1-pth1.10-py3

My docker version is 24.0.2, could be something related with this?

Also, I was checking the libraries installed with cuda in this path ’ /usr/local/cuda/lib64’ and I have the following ones in reference to the error:

libcurand.so
libcurand.so.10
libcurand.so.10.1.2.300

Thanks.

5

Hi @gsolsan, on JetPack 4 the CUDA libraries get mounted into the containers dynamically by the NVIDIA runtime - do you have these CSV files on your system under /etc/nvidia-container-runtime/host-files-for-container.d ?

ls -ll /etc/nvidia-container-runtime/host-files-for-container.d/
total 32
-rw-r--r-- 1 root root    26 May 23  2021 cuda.csv
-rw-r--r-- 1 root root  4250 Jul 13  2021 cudnn.csv
-rw-r--r-- 1 root root 12240 Jun  9  2023 l4t.csv
-rw-r--r-- 1 root root  1590 Oct  7  2022 tensorrt.csv
-rw-r--r-- 1 root root   325 Aug 11  2020 visionworks.csv

These files are what specify the libraries to the NVIDIA runtime that it should mount. They should have already been pre-installed on your SD card image from the nvidia-container-csv-* packages.

Checking my docker version on a Jetson running L4T R32.7, it is showing docker version 20.10.7. My guess is that your docker packages got upgraded at some point, and the dynamic mounting mechanism I mentioned above is no longer working for that configuration. I would recommend either downgrading docker, or reflashing your SD card (and docker should be pre-installed with NVIDIA runtime on the fresh SD card image)

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.