ROS2+NVIDIA ERROR:Could not load library dlopen error: libnvll.so: cannot open shared object file file or directory,

Hello,

I am using NVDIA jetson xavier for my robotic application.
I am currently working on OpenVSLAM using ros2 using docker.
But when i run OpenVSLAM application in docker container i get issue as below(tried with ros2 foxy as well as galactic). Is there any dependency with NVIDA version ?
Error:
|```
what(): Could not load library dlopen error: libnvll.so: cannot open shared object file: No such file or directory, at /tmp/binarydeb/ros-galactic-rcutils-4.0.2/src/shared_library.c:99


My Jetson runs on ubuntu 18.04
NVDIA docker version is as below:

NVIDIA Docker: 2.0.3
Client:
Version: 19.03.6
API version: 1.40
Go version: go1.12.17
Git commit: 369ce74a3c
Built: Fri Dec 18 12:25:49 2020
OS/Arch: linux/arm64
Experimental: false

Server:
Engine:
Version: 19.03.6
API version: 1.40 (minimum version 1.12)
Go version: go1.12.17
Git commit: 369ce74a3c
Built: Thu Dec 10 13:23:49 2020
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.3.3-0ubuntu1~18.04.4
GitCommit:
runc:
Version: spec: 1.0.1-dev
GitCommit:
docker-init:
Version: 0.18.0
GitCommit:


My docker file contains below lines specific to NVDIA:

ENV NVIDIA_VISIBLE_DEVICES ${NVIDIA_VISIBLE_DEVICES:-all}
ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES:+$NVIDIA_DRIVER_CAPABILITIES,}graphics


I am using Pangolin viewer for display.
For additional dependency error you can refer to below url.
https://openvslam-community.readthedocs.io/en/latest/ros2_package.html

Could you please help me to understand and resolve the issue.

Thanks and Regards,
Udaykiran Patnaik.

Hi @uday.patnaik, are you starting the container with --runtime nvidia? That flag is needed to use the JetPack libraries in container.

Hello @dusty_nv ,

Thank you for your reply.
Yes, My docker command is as below:

docker run -it --rm --device /dev/video0 -e DISPLAY=$DISPLAY --runtime=nvidia --net=host -v /tmp/.X11-unix/:/tmp/.X11-unix:ro openvslam-ros-g

i get above error when i try to run image-tools ros2 package inside container. Command is as below inside the container:
ros2 run image_tools cam2image -t camera

Thanks and Regards,
Udaykiran Patnaik.
Dockerfile (5.4 KB)

I have attached my docker file for your reference.

Thanks and Regards,
Udaykiran Patnaik.

So, I am confused to use –runtime=nvidia or not in my docker run command in my jetson agx xavier (ubuntu 18.04).

Summary of error:
If i use –runtime=nvidia in docker run i can not run "ros2 run image_tools cam2image" ,
but if don’t use –runtime=nvidia then i can run "ros2 run image_tools cam2image" but after that when i run
ros2 run openvslam_ros run_slam -v ./orb_vocab.fbow -c ../tmp/openvslam/example/aist/config/camera.yaml
i get below error.

libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
X11 Error: BadMatch (invalid parameter attributes)
X11 Error: BadValue (integer parameter out of range for operation)
terminate called after throwing an instance of 'std::runtime_error'
  what():  Pangolin X11: Failed to create an OpenGL context

output of nvcc --version outside docker is :
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_21:14:42_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89

I do not know if running docker container resulted in any mismatch and do not know how to check. I do not to cross verify the cuda version inside docker container.
As per below thread my CUDA version should match with LD_LIBRARY_PATH inside docker.

But when i run echo $LD_LIBRARY_PATH inside my container i get output like this, which does not say about cuda version.
/opt/ros/galactic/opt/yaml_cpp_vendor/lib:/opt/ros/galactic/lib/aarch64-linux-gnu:/opt/ros/galactic/lib:/lib:

Thanks and Regards,
Udaykiran Patnaik.

You should be using --runtime=nvidia in order to use the GPU, and your container needs to be derived from l4t-base (or some other container that derives from l4t-base)

FROM ros:galactic As a

In your dockerfile, you aren’t using a container that was built against l4t-base, which may be why GPU is not working for you. Instead, try FROM dustynv/ros:galactic-ros-base-l4t-r32.5.0. I build these containers from https://github.com/dusty-nv/jetson-containers

Note that if you are using a different version of JetPack-L4T, you should use a different tag of the container that matches your L4T version (except L4T R32.5.0/R32.5.1 share the same r32.5.0 container tag)

@dusty_nv
Thank you so much for your help as always.

I modified my docker file, but unfortunately due to space issue in my jetson docker build stopped at first line itself.
FROM dustynv/ros:galactic-ros-base-l4t-r32.4.4

My jetson has only 4gb free space and above line itself took around 3.5gb.

I need to speak to my manager and see how to solve this.
I will update you soon.

Thanks and Regards,
Udaykiran Patnaik.

On Xavier NX devkit, you can use bigger SD card or mount an NVME drive. Then you can change the docker data-root to a folder on the larger drive to store your containers.

@dusty_nv
Thank you so much for your suggestion.
I will share above info with my manager.

Thanks and Regards,
Udaykiran Patnaik.