ROS2+NVIDIA ERROR:Could not load library dlopen error: libnvll.so: cannot open shared object file file or directory,

Robotics & Edge Computing Jetson Systems Jetson Xavier NX
1

Hello,

I am using NVDIA jetson xavier for my robotic application.
I am currently working on OpenVSLAM using ros2 using docker.
But when i run OpenVSLAM application in docker container i get issue as below(tried with ros2 foxy as well as galactic). Is there any dependency with NVIDA version ?
Error:
|```
what(): Could not load library dlopen error: libnvll.so: cannot open shared object file: No such file or directory, at /tmp/binarydeb/ros-galactic-rcutils-4.0.2/src/shared_library.c:99


My Jetson runs on ubuntu 18.04
NVDIA docker version is as below:

NVIDIA Docker: 2.0.3
Client:
Version: 19.03.6
API version: 1.40
Go version: go1.12.17
Git commit: 369ce74a3c
Built: Fri Dec 18 12:25:49 2020
OS/Arch: linux/arm64
Experimental: false

Server:
Engine:
Version: 19.03.6
API version: 1.40 (minimum version 1.12)
Go version: go1.12.17
Git commit: 369ce74a3c
Built: Thu Dec 10 13:23:49 2020
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.3.3-0ubuntu1~18.04.4
GitCommit:
runc:
Version: spec: 1.0.1-dev
GitCommit:
docker-init:
Version: 0.18.0
GitCommit:


My docker file contains below lines specific to NVDIA:

ENV NVIDIA_VISIBLE_DEVICES ${NVIDIA_VISIBLE_DEVICES:-all}
ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES:+$NVIDIA_DRIVER_CAPABILITIES,}graphics


I am using Pangolin viewer for display.
For additional dependency error you can refer to below url.
https://openvslam-community.readthedocs.io/en/latest/ros2_package.html

Could you please help me to understand and resolve the issue.

Thanks and Regards,
Udaykiran Patnaik.
3

Hi @uday.patnaik, are you starting the container with --runtime nvidia? That flag is needed to use the JetPack libraries in container.

4

Hello @dusty_nv ,

Thank you for your reply.
Yes, My docker command is as below:

docker run -it --rm --device /dev/video0 -e DISPLAY=$DISPLAY --runtime=nvidia --net=host -v /tmp/.X11-unix/:/tmp/.X11-unix:ro openvslam-ros-g

i get above error when i try to run image-tools ros2 package inside container. Command is as below inside the container:
ros2 run image_tools cam2image -t camera

Thanks and Regards,
Udaykiran Patnaik.
Dockerfile (5.4 KB)

I have attached my docker file for your reference.

Thanks and Regards,
Udaykiran Patnaik.

5

So, I am confused to use –runtime=nvidia or not in my docker run command in my jetson agx xavier (ubuntu 18.04).

Summary of error:
If i use –runtime=nvidia in docker run i can not run "ros2 run image_tools cam2image" ,
but if don’t use –runtime=nvidia then i can run "ros2 run image_tools cam2image" but after that when i run
ros2 run openvslam_ros run_slam -v ./orb_vocab.fbow -c ../tmp/openvslam/example/aist/config/camera.yaml
i get below error.

libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
X11 Error: BadMatch (invalid parameter attributes)
X11 Error: BadValue (integer parameter out of range for operation)
terminate called after throwing an instance of 'std::runtime_error'
  what():  Pangolin X11: Failed to create an OpenGL context

output of nvcc --version outside docker is :
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_21:14:42_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89

I do not know if running docker container resulted in any mismatch and do not know how to check. I do not to cross verify the cuda version inside docker container.
As per below thread my CUDA version should match with LD_LIBRARY_PATH inside docker.

But when i run echo $LD_LIBRARY_PATH inside my container i get output like this, which does not say about cuda version.
/opt/ros/galactic/opt/yaml_cpp_vendor/lib:/opt/ros/galactic/lib/aarch64-linux-gnu:/opt/ros/galactic/lib:/lib:

Thanks and Regards,
Udaykiran Patnaik.

6

You should be using --runtime=nvidia in order to use the GPU, and your container needs to be derived from l4t-base (or some other container that derives from l4t-base)

FROM ros:galactic As a

In your dockerfile, you aren’t using a container that was built against l4t-base, which may be why GPU is not working for you. Instead, try FROM dustynv/ros:galactic-ros-base-l4t-r32.5.0. I build these containers from https://github.com/dusty-nv/jetson-containers

Note that if you are using a different version of JetPack-L4T, you should use a different tag of the container that matches your L4T version (except L4T R32.5.0/R32.5.1 share the same r32.5.0 container tag)

7

@dusty_nv
Thank you so much for your help as always.

I modified my docker file, but unfortunately due to space issue in my jetson docker build stopped at first line itself.
FROM dustynv/ros:galactic-ros-base-l4t-r32.4.4

My jetson has only 4gb free space and above line itself took around 3.5gb.

I need to speak to my manager and see how to solve this.
I will update you soon.

Thanks and Regards,
Udaykiran Patnaik.

8

On Xavier NX devkit, you can use bigger SD card or mount an NVME drive. Then you can change the docker data-root to a folder on the larger drive to store your containers.

9

@dusty_nv
Thank you so much for your suggestion.
I will share above info with my manager.

Thanks and Regards,
Udaykiran Patnaik.