No /usr/lib/aarch64-linux-gnu/tegra/libGL.so.0 with Jetpack 4.4.1

elvis.dowson · November 14, 2018, 12:41pm

Hi,

I need the correct aarch64-linux-gnu/tegra/libGL.so, but for some reason JetPack 4.4.1 does not come with that particular file.

Is there any way to get that particular file?

I’m getting the following errors at the moment on the Jetson AGX with JetPack 4.4.1 when building a custom ROS RViz plugin project:

/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmGetDevices2'
/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmCloseOnce'
/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmOpenOnce'
/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmFreeDevice'
/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmGetDeviceNameFromFd2'
/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmGetDevice2'
/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmFreeDevices'
collect2: error: ld returned 1 exit status

dusty_nv · November 14, 2018, 3:29pm

Hi elvis.dowson, can you check if this post addresses your issue?

[url]https://devtalk.nvidia.com/default/topic/1042381/jetson-agx-xavier/problem-with-gazebo-/post/5295390/#5295390[/url]

When ROS is installed, it may redirect symbolic links that need corrected.

elvis.dowson · November 14, 2018, 4:41pm

Hi Dusty,

If I check where libdrm.so.2 points to , it points to the tegra/libdrm.so.2

Isn’t that correct?

$ ls -l /usr/lib/aarch64-linux-gnu/libdrm*
# showing only relevant libdrm* files here
lrwxrwxrwx 1 root root     14 Nov 13 20:14 /usr/lib/aarch64-linux-gnu/libdrm.so.2 -> libdrm_nvdc.so

lrwxrwxrwx 1 root root     17 Nov 13 19:01 /usr/lib/aarch64-linux-gnu/libdrm_nvdc.so -> tegra/libdrm.so.2

I ran the following commands as suggested in the link you provided:

sudo rm /usr/lib/aarch64-linux-gnu/libdrm.so.2
sudo -H ln -s /usr/lib/aarch64-linux-gnu/libdrm.so /usr/lib/aarch64-linux-gnu/libdrm.so.2

so that now I get

lrwxrwxrwx 1 root root     15 Jun  7 23:30 /usr/lib/aarch64-linux-gnu/libdrm.so -> libdrm.so.2.4.0
lrwxrwxrwx 1 root root     36 Nov 14 20:32 /usr/lib/aarch64-linux-gnu/libdrm.so.2 -> /usr/lib/aarch64-linux-gnu/libdrm.so
-rw-r--r-- 1 root root  63856 Jun  7 23:31 /usr/lib/aarch64-linux-gnu/libdrm.so.2.4.0

but still gives me the same build error:

/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmGetDevices2'
/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmCloseOnce'
/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmOpenOnce'
/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmFreeDevice'
/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmGetDeviceNameFromFd2'
/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmGetDevice2'
/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/libGL.so: undefined reference to `drmFreeDevices'

I also did an additional check to see which package provides libGL.so and it turns out to be libgl1-mesa-dev:arm64

dpkg -S /usr/lib/aarch64-linux-gnu/libGL.so /usr/local/lib/aarch64-linux-gnu/libGL.so.1 /usr/local/lib/aarch64-linux-gnu/libGL.so.1.0.0

libgl1-mesa-dev:arm64: /usr/lib/aarch64-linux-gnu/libGL.so
dpkg-query: no path found matching pattern /usr/local/lib/aarch64-linux-gnu/libGL.so.1
dpkg-query: no path found matching pattern /usr/local/lib/aarch64-linux-gnu/libGL.so.1.0.0

Shouldn’t libGL.so be provided by the JetPack 4.4.1 Tegra driver?

The libGL.so file exists in JetPack 3.3.

When I looked at the 3.3 tegra driver sources, it had support only for TX2 graphics chip 18x. The JetPack 4.4.1 driver sources is the only one at present, that has driver support for Xavier AGX 19x chip, but it is currently missing libGL.so

dusty_nv · November 14, 2018, 5:07pm

On Xavier, the libGL exists at /usr/lib/aarch64-linux-gnu/libGL.so

It may that during the ROS install it is getting overwritten by another package.

elvis.dowson · November 14, 2018, 5:20pm

I took a look the host Ubuntu-18.04 rootfs on my Jetson AGX and I can see that libGL.so is provided by the libglvnd-dev package

elvis@jetson-agx:/usr/lib$ dpkg -S /usr/lib/aarch64-linux-gnu/libGL.so.1
libgl1:arm64: /usr/lib/aarch64-linux-gnu/libGL.so.1
elvis@jetson-agx:/usr/lib$ dpkg -S /usr/lib/aarch64-linux-gnu/libGL.so
libglvnd-dev:arm64: /usr/lib/aarch64-linux-gnu/libGL.so

Since the docker image that I’m using is based on Ubuntu-16.04, I had to manually build and install libglvnd to /usr/local/lib/aarch64-linux-gnu.

The LD_LIBRARY_PATH order was set so that the locally installed libraries would come up first:

/usr/local/cuda/lib64:/usr/local/lib/aarch64-linux-gnu:/usr/local/lib:/usr/lib/aarch64-linux-gnu/tegra:/usr/lib/aarch64-linux-gnu/tegra-egl:/usr/lib/aarch64-linux-gnu:/opt/ros/kinetic/lib:/opt/ros/kinetic/lib/aarch64-linux-gnu

Perhaps I can try symlinking to the libglvnd library that I had installed manually.

elvis.dowson · November 14, 2018, 5:43pm

So the solution was to symlink to the version of libGL.so provided by libglvnd https://github.com/NVIDIA/libglvnd.

Now this solution is specific to the following combination:

Ubuntu-16.04 docker image
JetPack-4.4.1 drivers into the docker image
libglvnd v1.1.0 manually built from sources and installed to /usr/local/lib/aarch64-linux-gnu

You can find the Ubuntu-16.04 OpenGL base docker image here: https://devtalk.nvidia.com/default/topic/1044051/jetson-agx-xavier/docker-gpu-acceleration-on-jetson-agx-for-ubuntu-16-04-image-using-jetpack-4-4-1/

At some point during the ROS installation process, a version of libGL.so gets installed into /usr/lib/aarch64-linux-gnu by the libgl1-mesa-dev:arm64 package.

The solution was therefore to add the following lines to any ROS docker image towards the end, to fix the symlink to point to the manually compiled libglvnd, for Ubuntu-16.04. (You don’t need to do this step for Ubuntu-18.04.)

# fix symlink to libGL.so to point to the one built by libglvnd
RUN rm /usr/lib/aarch64-linux-gnu/libGL.so \
    && ln -s /usr/local/lib/aarch64-linux-gnu/libGL.so /usr/lib/aarch64-linux-gnu/libGL.so

I left the symlinnk to the /usr/lib/aarch64-linux-gnutegra/libdrm.so file untouched. It wasn’t necessary to touch that in this particular instance.

The ROS project builds fine now.