OSError: libcurand.so.10: cannot open shared object file: No such file or directory when running from L4t image

Hi, I’m using Avermedia EA713-AAMN powered by Jetson AGX Xavier.
Avermedia there is the firmware EA713_R1.1.4.4.5.1 with JetPack 4.5.1 ( L4T 32.5.1 )
I have checked the version by
cat /etc/nv_tegra_release

R32 (release), REVISION: 5.1, GCID: 26202423, BOARD: t186ref, EABI: aarch64, DATE: Fri Feb 19 16:50:29 UTC 2021

`
I checked nvidia jetpack version:

sudo apt-cache show nvidia-jetpack

It’s installed. The output is:

Package: nvidia-jetpack
Version: 4.5.1-b17
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-cuda (= 4.5.1-b17), nvidia-opencv (= 4.5.1-b17), nvidia-cudnn8 (= 4.5.1-b17), nvidia-tensorrt (= 4.5.1-b17), nvidia-visionworks (= 4.5.1-b17), nvidia-container (= 4.5.1-b17), nvidia-vpi (= 4.5.1-b17), nvidia-l4t-jetson-multimedia-api (>> 32.5-0), nvidia-l4t-jetson-multimedia-api (<< 32.6-0)
Homepage: Autonomous Machines | NVIDIA Developer
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.5.1-b17_arm64.deb
Size: 29372
SHA256: 378f7588e15c35692eb1bed6f336be74f4f396d88fad45af67c68e22b63be04b
SHA1: e41f26a3d8326e9952915eee12fa37e17de3245f
MD5sum: 31b2bd9d0f214f74acaeb3d8e4279e9d
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8

Package: nvidia-jetpack
Version: 4.5-b129
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-cuda (= 4.5-b129), nvidia-opencv (= 4.5-b129), nvidia-cudnn8 (= 4.5-b129), nvidia-tensorrt (= 4.5-b129), nvidia-visionworks (= 4.5-b129), nvidia-container (= 4.5-b129), nvidia-vpi (= 4.5-b129), nvidia-l4t-jetson-multimedia-api (>> 32.5-0), nvidia-l4t-jetson-multimedia-api (<< 32.6-0)
Homepage: Autonomous Machines | NVIDIA Developer
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.5-b129_arm64.deb
Size: 29360
SHA256: 002646e6d81d13526ade23d7c45180014f3cd9e9f5fb0f8896b77dff85d6b9fe
SHA1: cb17547b902b2793e0df86d561809ecdbf7e401f
MD5sum: 06962c42e462f643455d6194d1a2d641
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8

I had not cuda installed so I ran:
sudo apt-get update && sudo apt-get install cuda-toolkit-10-2
after that check cuda version:
cat /usr/local/cuda/version.txt
The output:
CUDA Version 10.2.89
I try to use the following L4t image nvcr.io/nvidia/l4t-pytorch:r32.5.0-pth1.6-py3

When I run
import torch
I have the follow error:
import torch File "/usr/local/lib/python3.6/dist-packages/torch/__init__.py", line 188, in <module> _load_global_deps() File "/usr/local/lib/python3.6/dist-packages/torch/__init__.py", line 141, in _load_global_deps ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL) File "/usr/lib/python3.6/ctypes/__init__.py", line 348, in __init__ self._handle = _dlopen(self._name, mode) OSError: libcurand.so.10: cannot open shared object file: No such file or directory

Hi,

Could you check if there is a libcurand.so.10 library located at /usr/local/cuda-10.2/lib64/?
If yes, please try to add the information into environment variable.

$ export LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib64:$LD_LIBRARY_PATH

Thanks.

Hi. I’ve double checked the library libcurand.so.10 , and it is presented in /usr/local/cuda-10.2/lib64/.
After that exported variable, but the issue remained the same.
After that I’ve tried not to build my image from nvcr.io/nvidia/l4t-pytorch:r32.5.0-pth1.6-py3 but simple to run in container:
docker run -it --rm --net=host --privileged \ --runtime nvidia -e DISPLAY=$DISPLAY \ -v /tmp/.X11-unix/:/tmp/.X11-unix \ --device /dev/video0:/dev/video0 \ <image_id>

After that I ran in container:
python3
and
import torch
and got the same error:
`Python 3.6.9 (default, Oct 8 2020, 12:12:24)
[GCC 8.4.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import torch
Traceback (most recent call last):
File “”, line 1, in
File “/usr/local/lib/python3.6/dist-packages/torch/init.py”, line 188, in
_load_global_deps()
File “/usr/local/lib/python3.6/dist-packages/torch/init.py”, line 141, in _load_global_deps
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
File “/usr/lib/python3.6/ctypes/init.py”, line 348, in init
self._handle = _dlopen(self._name, mode)
OSError: libcurand.so.10: cannot open shared object file: No such file or directory`

My version of jetpack is JetPack 4.5.1 ( L4T 32.5.1 ) but the image nvcr.io/nvidia/l4t-pytorch:r32.5.0-pth1.6-py3 is for
L4T 32.5.0.

Could it be the issue?

Actually I found the issue:
The were are missing csv file cuda.csv in /etc/nvidia-container-runtime/host-files-for-container.d/
I ran in terminal:
sudo apt install nvidia-container-csv-cuda
After that I get the next error:
OSError: /usr/lib/aarch64-linux-gnu/libcudnn.so.8: file too short
So it was missing the next file cudnn.csv in /etc/nvidia-container-runtime/host-files-for-container.d/
To fix:
sudo apt install nvidia-container-csv-cudnn

After that all works fine

3 Likes

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.